Compare commits

...

51 Commits

Author SHA1 Message Date
Brian Gan 0398c6718e color: Replace anonymous sentinel with named class
Replace the bare `object()` sentinel used for `_CHECK_CONSOLE` with an
instance of a dedicated `_CheckConsoleSentinel` class. This gives the
sentinel a meaningful repr and type, making it easier to identify in
debugging output and type checks compared to an opaque `<object>`.

Change-Id: I916521d47ba13207e29a2412e00f3576aff8afff
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/605021
Commit-Queue: Brian Gan <brgan@google.com>
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-07-07 14:55:56 -07:00
Gavin Mak 3bb4871c44 rebase: Resolve revisionExpr to tracking branch for --onto-manifest
When running `repo rebase -m` (`--onto-manifest`), the command uses the
raw `revisionExpr` from the manifest (e.g. `main`) directly as the
`--onto` target. This can fail or behave incorrectly if it should
reference the local tracking branch (e.g. `refs/remotes/<remote>/main`).

Resolve `project.revisionExpr` to its local tracking branch using
`project.GetRemote().ToLocal()`. Fall back to using the raw
`revisionExpr` value if the resolution fails (raising a `GitError`).

Bug: 532028666
Change-Id: I4c1bca1374a5842688be227f6aa2afffcdad5397
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/604941
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-07-07 13:40:52 -07:00
Brian Gan 35bbf701d0 color: Treat "true" and "yes" as "auto", not "always"
Per the git documentation for color.ui [1], setting color.ui to "true"
(or "yes") should behave identically to "auto", enabling color only
when output is written to a terminal or an active pager. Previously,
repo was equating "true" and "yes" with "always", which caused color
escape codes to be emitted unconditionally, even when output was piped
or redirected.

Replace the duplicated string-matching logic in SetDefaultColoring and
Coloring.__init__ with a single CONFIG_TO_COLOR_SETTING dict that maps
all git color config values to their behavior. This makes the mapping
easy to verify against the git docs and impossible to get out of sync
between the two call sites.

Added tests for SetDefaultColoring and Coloring.__init__ covering
all color mode values (auto, true, yes, always, never, no, false),
case insensitivity, TTY vs pipe behavior, active pager detection,
and unrecognised input.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-colorui

Bug: 295841573
Change-Id: I8a04b9c7e4154de37ed7518c010233039e0afdc9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602981
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-07-01 15:59:45 -07:00
Gavin Mak 881af15cdc agents: run review agents manually
The review agents are broken right now, so don't run them automatically.
They should still be runnable on demand.

Change-Id: I22e5ff46f917a49d0c267a0b1e8c63ae2165347d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/603461
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
2026-07-01 15:45:23 -07:00
Brian Gan fbc9c79192 tests: Fix test compatibility with Python 3.14 forkserver
Python 3.14 changed the default multiprocessing start method on Linux
from "fork" to "forkserver". The test_forall_all_projects_called_once
test uses mock.patch.object on Project.GetRevisionId, but class-level
mock patches do not survive into forkserver worker processes because
they start from a clean Python interpreter rather than inheriting the
parent's memory.

Replace the mock with setting revisionId directly on each Project
instance so GetRevisionId() short-circuits without touching git.
This works with any multiprocessing start method since the string
attribute is part of the Project objects stored in _parallel_context,
which is properly serialized to workers via initargs.

Bug: 425319437
Change-Id: Icd3bbd010921d7652bb2425fad85974df9198367
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602941
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-30 15:35:57 -07:00
Gavin Mak ead4b2d7aa sync: Implement fetchcmd for standard Git layouts
Introduce support for `repo.fetchcmd` configuration, allowing users to
specify a custom command to fetch project data instead of
`_RemoteFetch`.

When `repo.fetchcmd` is specified, `repo` will execute it instead of
`_RemoteFetch` during the network half of sync. Note that
`repo.fetchcmd` requires `repo.uselocalgitdirs` to be enabled.

The command is executed in a subshell with project-context environment
variables, including the new `REPO_TREV` (target revision resolved to a
commit hash) and `REPO_PROJECT_FETCH_URL`.

After execution, `repo` verifies that the target commit is available and
that the tracking ref and FETCH_HEAD are correctly updated.

Tested with:
```
> ~/git-repo/repo init -u https://android-review.googlesource.com/platform/manifest \
    --repo-url file:///usr/local/google/home/gavinmak/git-repo \
    --groups developers \
    --no-repo-verify \
    --use-local-gitdirs
...

> git config --file .repo/manifests.git/config repo.fetchcmd 'mkdir -p $REPO_PATH && cd $REPO_PATH && if [ ! -d .git ]; then git init && git remote add aosp $REPO_PROJECT_FETCH_URL; fi && git fetch aosp $REPO_TREV && git reset --hard $REPO_TREV && mkdir -p .git/refs/remotes/aosp && echo $REPO_TREV > .git/refs/remotes/aosp/main && echo $REPO_TREV > .git/FETCH_HEAD'

> ~/git-repo/repo sync -j32
warning: repo is not tracking a remote branch, so it will not receive updates; run `repo init --repo-rev=stable` to fix.
You are currently enrolled in Git submodules experiment (go/android-submodules-quickstart).  Use --no-use-superproject to override.

Syncing: 100% (4/4), done in 1m15.686s
Finalizing sync state...
repo sync has finished successfully.
```

Bug: 513329573
Change-Id: I754d3f3c78e86fdeee1a72115297a75b571bc497
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583883
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-30 13:20:15 -07:00
Brian Gan e7cac4bca6 status: Show ahead/behind info for local branches
When viewing `repo status`, it is difficult to distinguish between branches that have active unpushed changes and stale branches that are fully synced. Previously, developers had to run commands like `repo forall -c "git status"` to see their ahead/behind counts.

This change updates Project.PrintWorkTreeStatus to automatically calculate and display the number of commits a branch is ahead and/or behind its upstream tracking branch. We use `git rev-list --left-right --count` to fetch this information natively and efficiently.

If the branch is completely synced with upstream, no extra text is shown.

Added tests for ahead-only, behind-only, diverged, no-tracking, and fully-synced branch states.

Bug: 319412954
Change-Id: I23879b2d472c7a7e11d01b565428a84b1b4f09c1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602423
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-06-30 11:13:14 -07:00
Gavin Mak 91986011b0 info: Parallelize project data gathering for JSON output
https://gerrit-review.googlesource.com/c/git-repo/+/581921 parallelized
`repo info` for text output. This commit does the same for JSON output
format.

Benchmarked `repo info --format=json` on an Android workspace with ~3k
projects (N=3):
- Before (sequential): 1m 30s average
- After (parallelized): 46s average (~2x speedup)

Verified that the JSON output is identical before and after.

Bug: 526685287
Change-Id: If573223aba584f8b932f87d29e34ed565c5c930a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601861
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-30 09:19:35 -07:00
Rahul Yadav a27dbcdb7b git_trace2_event_log: Fix index out of range on empty config values
In git_trace2_event_log_base.py's GetDataEventName method, it parses
value to identify if it represents a JSON list. When a config key has an
empty string value, GetDataEventName evaluates value[0], which raises
IndexError: string index out of range.

This change fixes the crash by checking if the value is a string and using
startswith/endswith to check for JSON lists instead of direct indexing.

Test: PYTHONPATH=. pytest tests/test_git_trace2_event_log.py

Bug: 512518342
TAG=agy
CONV=ff5d70d7-e5b3-42b3-8f16-23b9e3070754

Change-Id: Ic40a8c6a22df57d0e97f268f6e1bc8a14a5024a4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602201
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
2026-06-30 04:44:38 -07:00
Gavin Mak 547dc9985c repo: Normalize GNUPGHOME path for MSYS GPG on Windows
Convert GNUPGHOME to a POSIX path using cygpath on Windows
so MSYS GPG can read it correctly.

Bug: 510840000
Change-Id: I7d25564dd6521f6e887ff45548f42984e709ef5e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601121
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-29 11:06:23 -07:00
Gavin Mak 88a7e88e54 info: Report actual checked-out HEAD revision
When `repo info` runs, the reported "Current revision" is resolved using
the manifest's target branch tracking ref (e.g. refs/remotes/goog/main).

Introduce Project.GetHeadRevisionId(), which gets the checked-out HEAD
commit in the worktree, and use it in `repo info` with a fallback to the
old behavior if the project is not checked out.

Bug: 526685287
Change-Id: I72280ce27daa210cada27d722a94e365644f06e0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/599481
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-29 10:50:02 -07:00
Gavin Mak c21a41c7cc git_config: Support SHA-256 object IDs
Update ID_RE to match both 40-char (SHA-1) and 64-char (SHA-256) IDs as
part of Git 3.0 support.

Bug: 483758905
Change-Id: Ie5d9a2a5df4c6e7adda7f1d89f1bfb7724846057
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/600341
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-26 15:40:55 -07:00
Brian Gan 6586efe79a man: disable line wrapping
When help2man runs the repo script to generate man pages, the argparse
module relies on the COLUMNS environment variable to wrap help text.
This wrapping can cause URLs and descriptions to be awkwardly broken
across lines. Setting COLUMNS="10000" prevents argparse from wrapping
the output, keeping URLs intact in the generated man pages.

Additionally, this fixes a broken git documentation URL fragment for
--partial-clone in the repo script.

Bug: 295374161
Change-Id: I0c79f37fbfe2bebe71ff90585f2e5e1f88ea33cb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601482
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-25 16:07:53 -07:00
Sainath Varanasi 3af9e2f146 project: fix sync of shallow projects sharing objdir
Repo sync fails when the following conditions are met:

* There are several checkouts of the same project
  in different paths.
* The checkouts are using git hashes as revisions
  (not branches).
* There is a clone-depth set on these projects.
* sync-c="true" is set in the manifest.
* The revision specified in the manifest
  has moved forward since the first repo init.

The sync fails because only the first gitdir gets the "shallow" file,
and subsequent dirs can't be synced.

Do not optimize away the fetch when the conditions above happen.

Simplified the boolean check in Sync_NetworkHalf and _RemoteFetch
using has_shallow, renamed loop variable to avoid shadowing.

Test: create a manifest matching conditions above, repo init,
forward the hash, and repo sync.
Test: added tests in test_project.py
Bug: 505072873
Originally-by: Elvira Khabirova <elvira.khabirova@volvocars.com>

Change-Id: I37c533c382e34fc5ddab489c5593b9e5d3875be2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601441
Tested-by: Sainath Varanasi <varanasisai@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Sainath Varanasi <varanasisai@google.com>
2026-06-25 15:28:46 -07:00
Gavin Mak d32b70275c agents: Add CRAG-generated review agents and skills
With cl/930783235, this change sets up two AI review agents that run
automatically on new changes.

Bug: 522929179
Change-Id: I315a7ec327dd30af842ec890818cf697756fd55c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/596721
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-24 13:31:45 -07:00
Nasser Grainawi 7f58543703 gitignore: Add AI agent files
Allow contributors to use local AI agent files without polluting the
repository with brand-specific files.

Further AI agent improvements will be done roughly following the same
framework being used for Chromium [1].

[1] https://chromium.googlesource.com/chromium/src/+/HEAD/agents

Change-Id: I791f318b4f9cad1705f94e05522089809097cc96
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/573601
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-06-22 16:19:30 -07:00
Rahul Yadav 39c0b60900 sync: Support pluggable remote helpers for smart sync manifest server.
Introduce support for pluggable remote helpers (declared via the
optional 'helper' attribute in <manifest-server>) to dynamically resolve
proxy addresses. Route the XML-RPC manifest server connection through
the resolved proxy.

Bug: b/517477903
Change-Id: I3b6b8ea2640bb077521df4b4a9e8a34a8c6ecdad
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591642
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-18 09:48:49 -07:00
Gavin Mak cd307a6089 project: Add REPO_PROJECT_FETCH_URL environment variable
Add REPO_PROJECT_FETCH_URL to Project.GetEnvVars(), which resolves to
the remote fetch URL of the project. This is useful for exposing the
URL to custom fetch commands or other external scripts.

Bug: 513329573
Change-Id: Ic2b0a83493934d16bb1152366ee4e1a2c35ea2dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/596121
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-16 10:30:47 -07:00
Josef Malmström 4b462634e0 sync: do not init sibling submodules in parallel
Initializing a submodule requires locking <parent>/.git/config.
As the implementation was currently doing this in parallel for
sibling submodules, it lead to a race condition causing
intermittent errors like:

error: could not lock config file .git/config: File exists

This commit enforces that sibling submodules are initialized
sequentially, eliminating the race condition.

Change-Id: I5ffb3de90276ba43e262d0e279a3d34324220b63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591241
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-16 00:21:27 -07:00
Gavin Mak d9d86fb595 repo: Bump launcher version to 2.65
https://gerrit-review.git.corp.google.com/c/git-repo/+/569001 didn't
update the repo launcher script version for the newly added
`--use-local-gitdirs` flag.

Bug: 513329573
Bug: 508146070
Change-Id: I07ad939a40466ed50e439c815608797689fd505f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/587381
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-06-11 10:39:45 -07:00
Gavin Mak f7a24df00c project: handle corrupted projects in DeleteWorktree
Catch GitError from IsDirty() during DeleteWorktree so corrupted
projects can be cleanly wiped instead of crashing.

Bug: 515415221
Change-Id: Ic03ae77c30a722f9aa06f2220747251e0aea4ab7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591001
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-05 09:16:22 -07:00
Gavin Mak e0bd39c691 project: Extract project envvar generation to GetEnvVars
Move project environment variable setup from subcmds/forall.py to a
reusable Project.GetEnvVars() helper method.

Bug: 513329573
Change-Id: I3b4b113aa5a086e5fa5eaf4461c7ce517d928610
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583881
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-27 12:58:02 -07:00
Xin Li c883613e31 sync: Skip copyfile/linkfile for unavailable projects
Commit 5534f16 made UpdateCopyLinkfileList to retry _CopyAndLinkFiles
for all projects in the manifest to handle directory-to-symlink
transistions. However, for partial sync's (e.g. repo sync <project>),
projects not included in the sync may not have their local directories
populated, attempts to perform the operation on unavailable projects
would result in dangling symlinks, or errors like:

error: Cannot copy file <source> to <destination>

Fix this by adding a check inside _CopyAndLinkFiles in project.py to
so these operations happen only when the project's worktree exists
and is a directory.

TAG=agy
CONV=90f5fca7-2199-4017-8f62-a010b0ab1dbf

Change-Id: I9ae1e8d62d8fd129cb4535e574c38339c87af441
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/587501
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
2026-05-27 10:36:32 -07:00
Gavin Mak 384c059f9e tests: Deduplicate test setup in test_project.py
Bug: 513329573
Change-Id: Id729c590a69d2e81d4bce2942c65bd0b3ce3a11a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583882
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Dan Willemsen <dwillemsen@google.com>
2026-05-26 10:29:06 -07:00
Gavin Mak b8531133de init: Add --use-local-gitdirs for standard Git layouts
Introduce --use-local-gitdirs to bypass repo's symlink-based layouts in
favor of standard local .git directories.

Bug: 513329573
Bug: 508146070
Change-Id: I53d1602e61be0b86964529bcbea3dc801471f9c9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569001
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Dan Willemsen <dwillemsen@google.com>
2026-05-26 10:26:25 -07:00
Gavin Mak 2d54384a5e sync: Add --superproject-rev flag to sync to specific revision
Allow syncing the outer manifest to a state defined by a specific
superproject revision. It updates the superproject, reads the manifest
commit from .supermanifest, and checks out the outer manifest project
to that commit.

Submanifests are then processed normally, allowing them to be updated
to the revisions specified in the new outer manifest state.

Bug: 416589884
Change-Id: I304c37a2b8794f9b74cb7e5e209a8a93762bdb52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/576321
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-20 12:28:55 -07:00
Josef Malmström 1b4e7a04be Fix submodules not synced for repeated repo
If I check out e.g. multiple branches of the same repo in one
manifest, only the submodules of one checkout are synced.

Fix this by adding the relative path  of the checkout as a key
to the mapping of derived projects, preventing overwrite.

The fix was originally proposed here:
https://issues.gerritcodereview.com/issues/40013218

Bug: 40013218
Change-Id: Ia86518ccf2c0af7bd7e4daf8d703a55b7e10ba52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581062
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
2026-05-20 06:05:57 -07:00
gurusai-voleti a94c9e2b52 Automated: Migrate gerrit/git-repo from gsutil to gcloud storage
Bug: 486536908
Change-Id: I248b093e189a3784b8959e837a5d66857f1ce0fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/577201
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Guru sai rama subbarao Voleti (xWF) <gvoleti@google.com>
Commit-Queue: Guru sai rama subbarao Voleti (xWF) <gvoleti@google.com>
2026-05-14 21:31:49 -07:00
Gavin Mak 51021fb209 abandon/start/info: Make them respect smart sync override
Call TryOverrideManifestWithSmartSync in start, abandon, and info
commands.

This ensures they pick up the pinned revisions from the smart sync
override manifest if it exists, rather than falling back to ToT from
the default manifest.

Bug: 279204331
Change-Id: I637f054a77773805daf0bf9cf5a712d82a5959b4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582503
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-14 13:19:55 -07:00
Gavin Mak d453273f06 command: Move smart sync override logic to Command base class
Deduplicate the logic for loading `smart_sync_override.xml` by
moving it from `forall.py` to the `Command` base class. This allows
other commands to reuse it to respect smart tags.

Bug: 279204331
Change-Id: I6f2f03995266c2a68c3225cacb92b2f580a89178
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582502
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-14 13:19:26 -07:00
Mike Frysinger 0129ac1c46 docs: drop period in headers
We don't do this in most places, so standardize the outliers.

Change-Id: I80c1ef16c5fa7355dcb7797bc8e56852b22c24d5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583641
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-14 09:03:56 -07:00
Mike Frysinger bf8c59f4a0 run_tests: help2man: update to latest release
Change-Id: I30e4a6452ac5504b782fe545a59ca1f58ac70385
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583621
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-05-14 09:03:52 -07:00
Josef Malmström 003f0407fc Fix missing None check in Remote.Save
This code was causing an exception in cases where `pushUrl` was set
but `projectname` was not.

This can happen when `pushUrl` is set for the manifest repo which does
not have a `projectname`. For example:

repo init --manifest-url ssh://url.to/my/manifest

cd .repo/manifests
git config remote.origin.pushurl ssh://url.to/my/manifest

repo init --manifest-url ssh://url.to/my/manifest --repo-rev main

The last `repo init` invocation causes an error.

Change-Id: Ibb68c8446880cfbac22feee595d1fd1b678c7ade
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579162
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-13 00:00:27 -07:00
Josef Malmström 1db50e49fb Add support for self referencing submodules
When working with relative submodule paths, The "./" needs special
handling similar to "../".

See information on:
https://git-scm.com/docs/git-submodule
Which currently states:
"<repository> is the URL of the new submodule’s origin repository.
This may be either an absolute URL, or (if it begins with ./ or ../),
the location relative to the superproject’s default remote
repository (Please note that to specify a repository foo.git which is
located right next to a superproject bar.git, you’ll have to use
../foo.git instead of ./foo.git - as one might expect when following
the rules for relative URLs - because the evaluation of relative URLs
in Git is identical to that of relative directories)."

The implementation also was not handling file/directory names
starting with "." or "..". Explicitly look for "./" and "../"
instead.

Change-Id: I8ae68d61fb0cbb1624183b175236e98a36e4afdb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579182
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
2026-05-13 00:00:11 -07:00
Mike Frysinger c97a306fe7 release: update-manpages: revert color filtering
We pull a pinned help2man from cipd now, so we should get stable
behavior between developers.  That means the color filtering should
not be necessary anymore, and we can drop the logic & unittest.

Change-Id: Ib53e1ce7f8d610d7f624c9a019c79dc5f438ac0d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582402
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-12 11:50:38 -07:00
Carlos Fernandez 5534f164d6 linkfile: Handle directory-to-symlink transitions safely
When a manifest changes from individual linkfiles inside a directory
(e.g. dest=".llms/rules", dest=".llms/skills") to a single linkfile
for the whole directory (e.g. dest=".llms", src="dot-llms"), two
things need to happen:

1. __linkIt must replace a real directory with a symlink.  Use
   os.rmdir() instead of platform_utils.remove() for real directories.
   rmdir only removes empty directories, so user-created content is
   never deleted.

2. UpdateCopyLinkfileList must handle the cleanup correctly:
   - Use os.rmdir() for directories (safe for non-empty)
   - Remove empty parent directories after cleaning old dests
   - Retry _CopyAndLinkFiles for all projects, since in interleaved
     sync mode _CopyAndLinkFiles runs before cleanup and may have
     failed because the directory was not yet empty

Change-Id: I0437b80beab98bce064cea81c11c47d699be91aa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569243
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
2026-05-12 11:10:43 -07:00
Mike Frysinger 67e52a120b run_tests: leverage cipd when available for help2man
This tool isn't installed on CI bot images so we've been skipping it,
but this is causing people to not run tests locally, and ignore errors.
Use cipd to pull the tool in when available.

Then revert a recent man change that the tool rejects.

Change-Id: I1030d0070fd5a624656eba7434ae6ec99b2e3f2d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582401
Reviewed-by: Greg Edelston <gredelston@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-05-12 10:45:13 -07:00
Gavin Mak d5037230e9 sync: Re-raise KeyboardInterrupt in main process
When running sync -j1, worker functions run directly in the main
process. Swallowing KeyboardInterrupt causes the loop to continue to the
next project instead of aborting.

Re-raise KeyboardInterrupt if running in the MainProcess, while
maintaining the suppression of stack traces in worker processes.

Bug: 468170157
Change-Id: I156d66bc209a265f7fa25eea0eb88737d1b51a34
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581342
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-05-12 10:41:47 -07:00
Mike Frysinger 7cc99b24c1 git_config: fix error message command output
str() inside an f-string is redundant.  Actually join the array together
as a string to show a proper error message, and include the full command
line, not the extra options passed in.

Before:
error.GitError: git config ('--null', '--list'): fatal: unable to read config file 'xxx': No such file or directory

After:
error.GitError: git config --system --includes --null --list: fatal: unable to read config file 'xxx': No such file or directory

Change-Id: I8983389aa2e0de7808991e73e636b77810f04c4b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582341
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-12 10:04:15 -07:00
Greg Edelston 12ad396f67 info: Parallelize repo info to improve performance
For a large checkout like chromiumos or android, `repo info` takes a
really long time! On my machine it took ~6 minutes. On a randomly
selected ChromiumOS cq-orchestrator build it took 4.1 minutes:
https://ci.chromium.org/b/8682060180498819729. This adds up to a lot of
wasted runtime for both humans and bots.

The problem is that `repo info` was single-threaded, which causes poor
performance when the checkout has 1000+ projects. We already have a
pattern for parallelization; let's use it.

BUG=None
TEST=Manually run, ensure no diff

Change-Id: I6b82b9495eb2a0e602a142dd3a16f09217871e1b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581921
Tested-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-12 09:59:07 -07:00
Gavin Mak a6bc1b7cf0 sync: Exclude stateless sync pruned projects from bloat check
If a project has been pruned for stateless sync, it has already
undergone aggressive garbage collection. There should be no bloat
to check for.

Change-Id: I9233a4611e05b7a0b7c097827d6f408144f7380d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581841
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-11 19:36:59 -07:00
Gavin Mak a98e422113 completion: document installation and usage in README
Add a "Shell Completion" section to README.md to document how to install
and use `completion.bash` and `completion.zsh`.

Change-Id: Ibf6c81043af6c24d45ea2dc6a6caa26d98ab7374
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581261
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-08 17:41:44 -07:00
Gavin Mak 5d8585012f sync: Suggest "git repack -a -d" in bloat warning
The warning suggests running "repo sync --auto-gc" which runs "git gc
--auto". Git may decide not to repack if its internal thresholds are not
met, leaving the warning active even after running the suggested
command.

"git repack -a -d" forces all reachable objects into a single pack and
deletes redundant packs, reducing the pack count to 1. This guarantees
that the warning (which triggers when pack count exceeds 10) goes away.

Bug: 505755299
Change-Id: I10163ef8efb7f3b7c5055378ad95051974d11b88
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580821
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-05-08 14:12:22 -07:00
Kamal Sacranie e8deeabf2a Add zsh completion
This PR adds `completion.zsh` to allow for relatively intelligent
completion of `repo` command-line arguments for all commands of the
`repo` CLI.

The `completion.zsh` can be added to your zsh completions by copying
(and renaming) the completion script to a directory that you have added
to your zsh completion path.

```
cp completion.zsh $ZSH_COMPLETION_DIRECTORY/_repo
```

You must name the file `_repo` for it to register as completion for
`repo`.

You can add a directory to your zsh completion by updating your `fpath`
variable before calling `compinit` as follows:

```
fpath=( /path/to/zsh-completions/dir $fpath )
```

Future work: Generate this file using the subcommand classes and python
reflection.

Change-Id: I6a4e785c1efcf9076bd693976ac03578836b691b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579561
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Kamal Sacranie <sacranie@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-08 07:46:00 -07:00
Gavin Mak 11428ae984 forall: Document REPO_UPSTREAM and REPO_DEST_BRANCH envvars
Change-Id: I74365295152f8828587c6b4ed93029efc6000881
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580761
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-07 20:27:08 -07:00
Carlos Fernandez 27d2232eb3 info: add --format and --include-summary/--include-projects options
Add --format={text,json} to produce machine-readable output, and
boolean options to control which sections are displayed:
  --include-summary / --no-include-summary (default: on)
  --include-projects / --no-include-projects (default: on)

The JSON output respects the include flags, so callers can request
only the fields they need (e.g. `repo info --format=json
--no-include-projects` for manifest metadata only).

Change-Id: I9641bc4023b630d9c61c5170eb86e5f3b787236f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569203
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
2026-05-07 19:23:24 -07:00
Josef Malmström e3eadd3728 sync: Fix force_checkout propagation
The force_checkout parameter was not propagated in all calls to
Checkout in Sync_LocalHalf.

Without this, repo sync --force-checkout can still fail for projects
currently on a local branch with no upstream/tracking configuration,
because the detach-to-manifest checkout was executed without -f,
leaving local modifications or untracked files able to block sync.

Change-Id: I58551388e2f906c4db96e220707a369057a71c24
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579181
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-06 08:53:27 -07:00
Gavin Mak 12cfc6036a git_superproject: Remove redundant _branch variable
Both variables were initialized to the same value and never modified.

Bug: 416589884
Change-Id: Iaa1ffffb4543d4cd9391ac679d9234fe01678861
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/576921
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-04-28 10:49:54 -07:00
Nasser Grainawi 134eeb024b tests: Add tests for repo status output
Build out status subcommand unit coverage using a minimal fake repo
checkout wired through XmlManifest.

The new tests verify:
- clean status output prints the expected project header
- modified tracked files appear with the expected status marker
- `-o` output includes the orphan section and orphan entries
- branch names shown in status reflect a started non-default branch

Change-Id: Ia7c22593d0bbdc4aed81faeb168b846f3e4016ab
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558501
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-23 17:52:09 -07:00
Nasser Grainawi 03fb18109f color: type SetDefaultColoring and drop bool states
Also add a test fixture to always disable coloring so that we get
reproducible test behavior. The few tests that want to check color
behavior (e.g. test_color.py) can still reset/change color values as
needed.

Change-Id: I1808139a63e0862c29bf81d7097e885ca8561040
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/573621
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
2026-04-22 17:05:55 -07:00
Ram Peri 5af71ce907 Add timing keyword argument for hooks
Measure the duration of the sync operation in the Execute method of the
Sync command and pass it to post-sync hooks as a standard keyword
argument (`sync_duration_seconds`).

Updates based on code review:
- Update _API_ARGS in hooks.py to allow sync_duration_seconds for post-sync hooks.
- Do not cast sync_duration_seconds to int for better granularity.
- Update docs/repo-hooks.md to document sync_duration_seconds.
- Add unit test for argument validation in test_hooks.py.

Test: Ran run_tests using venv python, all 554 tests passed.
Bug: TBD
Change-Id: Ie29e002a5d283460d993ad96c224dbf4b6d7985c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/575021
Tested-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-21 17:10:07 -07:00
74 changed files with 11742 additions and 608 deletions
+6
View File
@@ -6,8 +6,14 @@ __pycache__
/dist
.repopickle_*
/repoc
/.cipd_bin
/.tox
/.venv
# PyCharm related
/.idea/
# AI tool related.
/AGENTS.md
/CLAUDE.md
/GEMINI.md
+2 -2
View File
@@ -24,7 +24,7 @@ However there are some differences, so please review and familiarize
yourself with the following relevant bits.
## Make separate commits for logically separate changes.
## Make separate commits for logically separate changes
Unless your patch is really trivial, you should not be sending out a patch that
was generated between your working tree and your commit head.
@@ -118,7 +118,7 @@ your patch. It is virtually impossible to remove a patch once it
has been applied and pushed out.
## Sending your patches.
## Sending your patches
Do not email your patches to anyone.
+37
View File
@@ -49,6 +49,43 @@ $ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/.bin/repo
$ chmod a+rx ~/.bin/repo
```
## Shell Completion
Repo includes completion scripts for Bash and Zsh.
### Bash
To enable completion in Bash, source `completion.bash` in your `~/.bashrc`:
```sh
source /path/to/git-repo/completion.bash
```
### Zsh
To enable completion in Zsh, you can either:
1. Copy or symlink `completion.zsh` to a file named `_repo` in a directory in your `$fpath`:
```sh
mkdir -p ~/.zsh/completion
# You can copy the file:
cp /path/to/git-repo/completion.zsh ~/.zsh/completion/_repo
# Or symlink it:
ln -s /path/to/git-repo/completion.zsh ~/.zsh/completion/_repo
```
Then add that directory to your `fpath` in `~/.zshrc` before `compinit`:
```zsh
fpath=(~/.zsh/completion $fpath)
autoload -Uz compinit
compinit
```
2. Or source the file directly and call `compdef` in your `~/.zshrc`:
```zsh
source /path/to/git-repo/completion.zsh
compdef _repo repo
```
[new-bug]: https://issues.gerritcodereview.com/issues/new?component=1370071
[issue tracker]: https://issues.gerritcodereview.com/issues?q=is:open%20componentid:1370071
+42
View File
@@ -0,0 +1,42 @@
# git-repo AI Review Agents (WIP)
**Note:** This project is a work in progress and is subject to change.
This directory contains configurations and skills for AI review agents that
automatically analyze changes in the `git-repo` codebase.
These agents help maintain code quality, enforce style guidelines, and catch
common pitfalls before code is merged.
## Directory Structure
* [`agent_configs.txtpb`](agent_configs.txtpb): Defines the active AI
agents, their configurations, and which skills they are equipped with.
* [`skills/`](skills/): Contains the "skills" (rules, guidelines, and traps)
used by the agents.
* [`code_review_workflow/`](skills/code_review_workflow/SKILL.md):
Guidelines for code review processes, commit messages, and testing.
* [`core_internals/`](skills/core_internals/SKILL.md): Technical
guidelines for `git-repo` core logic (sync, manifest, git integration,
etc.).
## How It Works
The agents defined in `agent_configs.txtpb` are configured to run
automatically on new changes. They analyze the diffs against the rules defined
in their respective skills and provide feedback in the code review interface
(e.g., Gerrit).
## Contributing
To improve the agent's review quality or add new rules:
1. **Update existing skills**: Modify the `SKILL.md` files under `skills/` to
add new rules, "What" explanations, "Why" rationales, and "Traps"
(Don't/Do code examples).
2. **Add new skills**:
* Create a new directory under `skills/`.
* Add a `SKILL.md` following the established format (see existing skills
for reference).
* Register the new skill in `agent_configs.txtpb` by adding it to the
`skills` field of an agent configuration.
+26
View File
@@ -0,0 +1,26 @@
# proto-file: google/corp/android/engprod/codereviewagentconfiguration/v1/agent.proto
# proto-message: HostAgents
# Code Review Workflow Agent
configs {
id: "code-review-workflow"
display_name: "Code Review Workflow"
description: "Analyzes git-repo Gerrit submission labeling, commit metadata, Python linting, and testing strategy."
skills: "code_review_workflow"
include_filters {
project: "git-repo"
}
automatic: false
}
# Core Internals Agent
configs {
id: "core-internals"
display_name: "Core Internals"
description: "Analyzes git-repo synchronization, multiprocessing, manifest parsing, git integration, worktree layouts, and CLI commands."
skills: "core_internals"
include_filters {
project: "git-repo"
}
automatic: false
}
+968
View File
@@ -0,0 +1,968 @@
---
name: code-review-workflow
description: Provides guidance and best practices on Gerrit submission labeling, CI builder execution, Python code formatting/linting, commit metadata standardization, and testing strategy in git-repo.
---
# Code Review Workflow Engineering Guide
## Executive Summary
Welcome to the authoritative engineering guide for the Code Review Workflow.
This living repository exists to capture critical folk knowledge, prevent the
recurrence of historical failure modes, and enforce strict architectural and
procedural boundaries across our integration pipeline. By standardizing these
protocols, we ensure high development velocity while maintaining rock-solid
codebase stability and traceability.
This guide covers the complete lifecycle of a change list (CL) from local
development to automated submission. It defines the strict Gerrit labeling
mechanisms required to trigger the Commit-Queue, mandates comprehensive CI
builder environment checks, and enforces centralized Python static analysis.
Furthermore, it outlines uncompromising standards for atomic commit metadata and
pragmatic testing state isolation to guarantee that every integration is fully
bisectable and verifiable.
For incoming engineers, adherence to these mandates eliminates the friction of
stalled pipelines, unreviewable monolithic changes, and silent CI regressions.
Treat this guide as your primary roadmap for navigating the repository's strict
submission requirements, enabling seamless transitions from peer approval to
successfully integrated code.
## Summary
| Chapter Theme / Title | Scope & Objective |
| :------------------------------- | :---------------------------------------- |
| **Gerrit Submission and Labeling | Dictates strict access controls, review |
: Workflow** : enforcement protocols, and Gerrit :
: : labeling mechanisms required to advance :
: : changes through the CI pipeline, ensuring :
: : seamless transitions to automated :
: : integration via the Commit-Queue. :
| **CI Builder Environment and | Defines guidelines for ensuring build |
: Execution Integrity** : script resilience against missing :
: : dependencies and managing process :
: : execution contexts within LUCI and local :
: : testing environments to prevent silent :
: : builder failures. :
| **Python Code Formatting and | Governs the automated enforcement of |
: Linting** : Python style guidelines, mandating strict :
: : PEP-8 compliance, import sorting, and :
: : consistent string quoting to ensure :
: : codebase uniformity and prevent CI :
: : regressions. :
| **Commit Metadata and History | Establishes the structural composition |
: Standardization** : and metadata formatting of change lists :
: : (CLs) to ensure precise issue tracker :
: : integration, reliable CI/CD parsing, and :
: : an atomic, bisectable repository history. :
| **Testing Strategy and State | Outlines test implementation boundaries, |
: Isolation** : emphasizing pragmatic mocking limits to :
: : prevent false positives and detailing :
: : acceptable workflows for deferred test :
: : coverage while maintaining verification :
: : integrity. :
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
## Chapter: Gerrit Submission and Labeling Workflow
**Context:** This domain dictates the strict access controls, review enforcement
protocols, and specific Gerrit labeling mechanisms required to advance changes
through the CI pipeline. Adherence ensures seamless transitions from peer
approval to automated integration via the Commit-Queue.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / |
: : : : Trap :
| :-------- | :------------------------------ | :------- | :----------------- |
| **T1-01** | Explicit Labeling for Gerrit | High | Leaving a change |
: : Automated Submission : : idle after :
: : : : addressing :
: : : : comments or :
: : : : receiving a :
: : : : reviewer's LGTM, :
: : : : expecting the :
: : : : reviewer to merge :
: : : : it. :
| **T1-02** | Automated Submission via | Medium | Requesting a |
: : Commit-Queue (CQ) : : manual push or :
: : : : direct submit from :
: : : : repository :
: : : : maintainers after :
: : : : receiving code :
: : : : review approval. :
| **T1-03** | Gerrit Trusted Contributor | Medium | Relying on a |
: : Review Enforcement Verification : : standard +2 vote :
: : : : from a non-trusted :
: : : : contributor to :
: : : : fulfill strict :
: : : : Review-Enforcement :
: : : : requirements. :
| **T1-04** | Mandatory Gerrit Labels for | High | Acknowledging an |
: : Automated Submission : : approval but :
: : : : failing to apply :
: : : : the appropriate :
: : : : Gerrit labels to :
: : : : initiate the merge :
: : : : pipeline. :
| **T1-05** | Gerrit Automated Submission | Medium | Leaving an |
: : Triggers : : approved patchset :
: : : : idle and waiting :
: : : : for maintainers to :
: : : : manually merge it. :
| **T1-06** | Active Reviewer Rerouting for | Medium | Waiting weeks or |
: : Stalled Changes : : months for an :
: : : : inactive or OOO :
: : : : reviewer to :
: : : : respond to a :
: : : : patchset update. :
--------------------------------------------------------------------------------
### Rules
#### T1-01: Explicit Labeling for Gerrit Automated Submission
> **Rule:** Always apply `Verified+1` and `Commit-Queue+2` explicitly to trigger
> the final submission phase. Never assume a code approval automatically
> initiates the pipeline.
>
> **What:** Changes are not merged automatically upon receiving approval;
> contributors must explicitly set the `Verified+1` and `Commit-Queue+2` labels
> to trigger the final submission phase.
>
> **Applies To:** Gerrit review UI and change submission pipeline as defined in
> `CONTRIBUTING.md`.
>
> **Why:** Contributors often mistakenly assume an LGTM implies an immediate
> merge, leading to stalled changes. The project relies on explicitly triggering
> the Commit-Queue to finalize CI checks and perform the merge. Failing to
> adhere to this typically results in **Stalled Submission Pipeline**.
**Trap 1: Leaving a change idle after addressing comments or receiving a
reviewer's LGTM, expecting the reviewer to merge it.**
**Don't:**
* Waiting indefinitely after reviewer posts 'LGTM'.
**Do:**
* Vote `Verified+1` and `Commit-Queue+2` manually to submit the change to the
automated queue.
**Exceptions:** Contributors lacking trusted permissions must ping a repository
maintainer to apply the final `Commit-Queue+2` vote.
--------------------------------------------------------------------------------
#### T1-02: Automated Submission via Commit-Queue (CQ)
> **Rule:** Must utilize the Gerrit Commit-Queue (CQ) labeling system to merge
> code. Maintainers must never perform direct manual submissions.
>
> **What:** Merging code must be triggered via the Gerrit Commit-Queue (CQ)
> labeling system rather than relying on direct manual submission by
> maintainers.
>
> **Applies To:** Gerrit code review UI and CI/CD submission workflow.
>
> **Why:** Contributors would request maintainers to directly merge patches once
> approved, bypassing the automated commit-queue pipeline, which guarantees that
> final integration tests pass before pushing to the target branch. Failing to
> adhere to this typically results in **Bypassed CI / Direct Submit**.
**Trap 1: Requesting a manual push or direct submit from repository maintainers
after receiving code review approval.**
**Don't:**
* Leaving a comment: "I believe everything is ready for integrating this. So
if either of you can submit it, it would be appreciated."
**Do:**
* Applying the `Commit-Queue+2` (CQ+2) label in Gerrit, which delegates
testing and the final merge operation to the automated bot.
--------------------------------------------------------------------------------
#### T1-03: Gerrit Trusted Contributor Review Enforcement Verification
> **Rule:** Verify review enforcement requirements are satisfied by contributors
> within the explicitly configured trusted group. Never cast misleading +2 votes
> if you lack valid trusted group privileges.
>
> **What:** Gerrit submission requirements may mandate specific approval levels
> (e.g., two trusted contributors). Votes from users with +2 access who are not
> in the designated 'trusted' group do not satisfy the 'Review-Enforcement'
> submit requirement.
>
> **Applies To:** Gerrit repository administration and code review voting
> workflows.
>
> **Why:** Non-trusted contributors with +2 rights were casting +2 votes on
> changes. These votes did not fulfill the 'Two trusted contributors'
> Review-Enforcement requirement, leading to stalled submissions and confusion
> regarding why the UI showed a +2 but blocked submission. Failing to adhere to
> this typically results in **Blocked Submission / Silent Requirement Failure**.
**Trap 1: Relying on a standard +2 vote from a non-trusted contributor to
fulfill strict Review-Enforcement requirements.**
**Don't:**
* Leaving a +2 vote on a change as a non-trusted contributor, creating the
false appearance that the Review-Enforcement requirement has been partially
or fully met.
**Do:**
* Verifying the reviewer is in the explicitly configured trusted group for the
repository. If not, the reviewer should manually downgrade their invalid +2
vote to a +1 to clearly indicate that their vote does not count toward the
enforcement threshold.
**Exceptions:** Repositories where specific non-employee groups have been
explicitly added to the trusted administrators list.
--------------------------------------------------------------------------------
#### T1-04: Mandatory Gerrit Labels for Automated Submission
> **Rule:** Always apply `Verified+1` and `Commit-Queue+2` labels to initiate
> the CI merge process. Never leave an approved CL in a technically unlabeled
> state.
>
> **What:** A code change must receive explicit `Verified+1` and
> `Commit-Queue+2` labels by the author or reviewer to trigger the automated CI
> merge process.
>
> **Applies To:** Gerrit workflow / Merge execution phase.
>
> **Why:** Historically, leaving a Change List (CL) in an approved but unlabeled
> state causes the integration pipeline to stall indefinitely, requiring manual
> intervention or reviewer pinging to trigger the CI queue. Failing to adhere to
> this typically results in **Merge Pipeline Stall**.
**Trap 1: Acknowledging an approval but failing to apply the appropriate Gerrit
labels to initiate the merge pipeline.**
**Don't:**
* Leaving the CL in an approved state and waiting for auto-submission without
applying the `Verified+1` or `Commit-Queue+2` labels.
**Do:**
* Explicitly applying `Verified+1` (and `Commit-Queue+2` if ready) once
reviewers have approved the logic, to instruct the automation to merge the
code.
--------------------------------------------------------------------------------
#### T1-05: Gerrit Automated Submission Triggers
> **Rule:** Must actively signal patch readiness to Gerrit systems using proper
> label thresholds. Avoid leaving patchsets idle assuming upstream maintainer
> action.
>
> **What:** A patchset requires specific label thresholds ('Verified+1' and
> 'Commit-Queue+2') to trigger automated submission in the Gerrit workflow.
>
> **Applies To:** Gerrit review UI and automated CI/CD submission process for
> the git-repo codebase.
>
> **Why:** Contributors frequently asked how to integrate changes after
> receiving an approval, leading to stalled patches because the automated
> pipeline was not explicitly triggered. Failing to adhere to this typically
> results in **Stalled Patch Integration**.
**Trap 1: Leaving an approved patchset idle and waiting for maintainers to
manually merge it.**
**Don't:**
* Waiting indefinitely after receiving an 'LGTM' without setting workflow
labels.
**Do:**
* The patch author manually sets the 'Verified' flag (if locally tested) and
applies the 'Commit-Queue+2' vote to signal readiness for automated merge.
--------------------------------------------------------------------------------
#### T1-06: Active Reviewer Rerouting for Stalled Changes
> **Rule:** Actively reroute reviews stalled by unresponsive or out-of-office
> (OOO) primary reviewers. Must explicitly tag alternate maintainers and
> document the absence to prevent lifecycle stalls.
>
> **What:** If the primary reviewer is out-of-office (OOO) or unresponsive for
> an extended period, contributors must actively CC and reroute the review to
> another active maintainer.
>
> **Applies To:** Gerrit review cycle and reviewer assignment process.
>
> **Why:** Patchsets have historically stalled for over a month due to reviewers
> taking extended leave without actively delegating their review queues. Failing
> to adhere to this typically results in **Indefinite Review Stalls**.
**Trap 1: Waiting weeks or months for an inactive or OOO reviewer to respond to
a patchset update.**
**Don't:**
* Leaving a review assigned strictly to an unresponsive reviewer without
notifying other maintainers or attempting to escalate.
**Do:**
* Tag a new reviewer with 'PTAL' (Please Take A Look) in the thread,
explicitly noting the original reviewer's absence, and confirm alignment
with the original author.
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Upstream:** T4 | Python Code Formatting and Linting - *Proper formatting
and static analysis are enforced before changes become eligible for final
Gerrit review and automated integration.*
* **Upstream:** T5 | Commit Metadata and History Standardization - *Accurate
commit messaging and isolated history must be validated by reviewers prior
to receiving approval labels.*
* **Downstream:** T3 | CI Builder Environment and Execution Integrity -
*Triggering the Commit-Queue directly invokes downstream LUCI environments
to guarantee execution integrity prior to branch merge.*
## Chapter: CI Builder Environment and Execution Integrity
**Context:** This section defines strict guidelines for ensuring the resilience
of build scripts against missing dependencies and managing process execution
contexts within LUCI and local testing environments. Adherence guarantees robust
verification across diverse operating systems and CI pipelines while preventing
silent builder failures.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / Trap |
| :-------- | :------------------------ | :------- | :------------------------ |
| **T3-01** | Verification Against | High | Running a standard local |
: : Breaking Change Build : : `make` without testing :
: : Configurations : : strict configurations or :
: : : : breaking-change flags. :
| **T3-02** | Windows Developer Mode | Medium | Attempting to run full |
: : Requirements for Tool : : local verification on a :
: : Verification : : standard Windows user :
: : : : account. :
| **T3-03** | Graceful Degradation for | Medium | Assuming all local |
: : Missing Builder Utilities : : developer utilities exist :
: : : : in the strict CI builder :
: : : : environment and :
: : : : unconditionally executing :
: : : : them. :
| **T3-04** | Contextual Diagnostic | High | Observing a generic CI |
: : Logging for LUCI CI : : failure without isolating :
: : Failures : : the specific process :
: : : : execution context or :
: : : : dependency resolution :
: : : : step. :
--------------------------------------------------------------------------------
### Rules
#### T3-01: Verification Against Breaking Change Build Configurations
> **Rule:** Always explicitly test core build structure modifications with
> breaking changes enabled to ensure forward compatibility.
>
> **What:** When modifying core build structures, the build must be tested
> explicitly with breaking changes enabled to ensure forward compatibility and
> correct regeneration of generated files.
>
> **Applies To:** Local build environments and Makefile targets.
>
> **Why:** Changes might succeed in a standard default build but fail when
> breaking change toggles are activated, hiding underlying dependency or
> regeneration issues. Failing to adhere to this typically results in **Build
> Breakage / Stale Artifacts**.
**Trap 1: Running a standard local `make` without testing strict configurations
or breaking-change flags.**
**Don't:**
```bash
make -j
```
**Do:**
```bash
make -j WITH_BREAKING_CHANGES=1
```
--------------------------------------------------------------------------------
#### T3-02: Windows Developer Mode Requirements for Tool Verification
> **Rule:** Must execute local tool verification on Windows (gWindows) using an
> Administrator account to enable Developer Mode.
>
> **What:** Local verification of git-repo tooling on Windows (gWindows)
> explicitly requires the host environment to be running with Administrator
> privileges to enable Developer Mode.
>
> **Applies To:** Windows (gWindows) test environments verifying file system
> operations.
>
> **Why:** Without Developer Mode enabled (which necessitates Admin rights),
> features relying on advanced OS-level file system operations (like symlinks)
> cannot execute, permanently blocking full local test suite execution on
> standard accounts. Failing to adhere to this typically results in
> **Verification Blocked / OS Permission Error**.
**Trap 1: Attempting to run full local verification on a standard Windows user
account.**
**Don't:**
* Executing the test suite from a non-elevated command prompt on Windows
without Developer Mode.
**Do:**
* Elevate to an Administrator account to enable Developer Mode before
executing the test suite on gWindows.
--------------------------------------------------------------------------------
#### T3-03: Graceful Degradation for Missing Builder Utilities
> **Rule:** Always implement auto-skip logic for optional utilities in build
> scripts rather than hard-failing when unavailable on the CI builder.
>
> **What:** Build scripts and test suites must implement auto-skip logic for
> optional, environment-specific utilities rather than hard-failing when the
> utility is unavailable on the CI builder.
>
> **Applies To:** CI Builder environment scripts and test suites, specifically
> testing external CLI utilities (e.g., `help2man`).
>
> **Why:** When a required utility was not pre-installed on the CI builder
> image, the build hard-failed. Adding auto-skip logic allows the CI pipeline to
> remain unblocked while still providing local testing benefits for developers
> who have the tool installed. Failing to adhere to this typically results in
> **Build Failure / Blocked CI**.
**Trap 1: Assuming all local developer utilities exist in the strict CI builder
environment and unconditionally executing them.**
**Don't:**
```python
# BAD: Hard failure if utility is missing
subprocess.run(["help2man", "repo"], check=True)
```
**Do:**
```python
# GOOD: Auto-skip test if utility is missing in the environment
if not shutil.which("help2man"):
self.skipTest("help2man not installed")
subprocess.run(["help2man", "repo"], check=True)
```
**Exceptions:** Core dependencies required for fundamental build steps cannot be
skipped and must be installed on the bot image.
--------------------------------------------------------------------------------
#### T3-04: Contextual Diagnostic Logging for LUCI CI Failures
> **Rule:** Must investigate CI builder failures by extracting and analyzing
> full execution context logs to isolate environmental roadblocks.
>
> **What:** CI builder failures must be investigated using full execution
> context logs (e.g., LUCI context, vpython3 resolution, and retcode outputs) to
> isolate environmental roadblocks.
>
> **Applies To:** LUCI builder execution environment, vpython3 resolution, and
> CI pipeline debugging.
>
> **Why:** CI commands failed with `retcode 1` due to external factors like
> specific URLs being flagged as suspect by internal security tools, breaking
> the build environment. Failing to adhere to this typically results in **Silent
> Builder Failure**.
**Trap 1: Observing a generic CI failure without isolating the specific process
execution context or dependency resolution step.**
**Don't:**
* Restarting the CI pipeline blindly when a job fails with a generic retcode,
ignoring potential external network or security blockers.
**Do:**
* Extract the step-by-step LUCI context log, verify path resolution (e.g.,
CIPD packages), and explicitly document external blockers like security
flags in the review.
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Upstream:** T6 | Testing Strategy and State Isolation - *Test
implementation dictates how missing builder utilities are mocked or
gracefully skipped during execution.*
* **Downstream:** T1 | Gerrit Submission and Labeling Workflow - *Automated
Verified+1 labels rely entirely on the stable, unblocked execution of CI
builder pipelines.*
## Chapter: Python Code Formatting and Linting
**Context:** This domain governs the automated enforcement of Python style
guidelines, mandating strict PEP-8 compliance, import sorting, and consistent
string quoting. All Python modifications must pass centralized static analysis
pipelines before integration to ensure codebase uniformity and prevent CI
regressions.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / Trap |
| :-------- | :----------------------- | :------- | :------------------------ |
| **T4-01** | Automated Flake8 | Medium | Relying purely on manual |
: : Post-Submit Verification : : code review or sporadic :
: : : : local linting without a :
: : : : continuous integration :
: : : : check. :
| **T4-02** | Mandatory Python | High | Using single quotes for |
: : Formatting and Import : : strings and appending new :
: : Sorting : : imports to the bottom of :
: : : : the import block without :
: : : : alphabetical or :
: : : : categorical sorting. :
| **T4-03** | Strict Python Import | High | Mixing local application |
: : Ordering : : imports with standard :
: : : : library imports, causing :
: : : : linting tools to fail the :
: : : : CQ job. :
--------------------------------------------------------------------------------
### Rules
#### T4-01: Automated Flake8 Post-Submit Verification
> **Rule:** Always configure and maintain centralized CI workflows to
> automatically run static analysis and validate Python code styling
> post-submit.
>
> **What:** Static analysis and Python linting must be automated via a
> centralized CI pipeline (e.g., Flake8 post-submit workflows) to enforce
> consistent style and prevent basic errors.
>
> **Applies To:** All Python files in the git-repo codebase; specifically
> validated via `.github/workflows/flake8-postsubmit.yml`.
>
> **Why:** Relying strictly on manual code review to catch styling and linting
> violations is error-prone. Automation ensures a baseline of code quality on
> every code push without consuming human review cycles. Failing to adhere to
> this typically results in **Linting Regression / Style Violation**.
**Trap 1: Relying purely on manual code review or sporadic local linting without
a continuous integration check.**
**Don't:**
* Committing Python code without an active CI linting workflow configuration.
**Do:**
* Maintain `.github/workflows/flake8-postsubmit.yml` to automatically run
flake8 on target branches.
--------------------------------------------------------------------------------
#### T4-02: Mandatory Python Formatting and Import Sorting
> **Rule:** Must format Python code to enforce double-quoted strings and
> alphabetically sorted import blocks to satisfy automated formatting checks.
>
> **What:** Python code modifications must pass automated style and linting
> checks ('Verify git-repo CL'), which strictly enforce string quote conventions
> (preferring double quotes), import block sorting, and PEP-8 style formatting.
>
> **Applies To:** All Python source files modified in the git-repo codebase.
>
> **Why:** Developers submitting patches with single-quoted strings or unsorted
> imports triggered automated CI failures in the `Verify git-repo CL` job,
> completely blocking code submission until formatting tools were executed
> locally. Failing to adhere to this typically results in **CI Pipeline
> Failure**.
**Trap 1: Using single quotes for strings and appending new imports to the
bottom of the import block without alphabetical or categorical sorting.**
**Don't:**
```python
import sys
import os
msg = 'This is an error'
```
**Do:**
```python
import os
import sys
msg = "This is an error"
```
--------------------------------------------------------------------------------
#### T4-03: Strict Python Import Ordering
> **Rule:** Always segment and order Python imports strictly according to
> project standards (standard library, third-party, local) to prevent CQ
> pipeline failures.
>
> **What:** Python module imports must adhere strictly to the project's
> formatting rules (e.g., standard library, third-party, local module ordering)
> to pass automated Commit-Queue (CQ) checks.
>
> **Applies To:** Python source files.
>
> **Why:** Non-standard import blocks cause the automated CI/CQ linting pipeline
> to fail, completely blocking submission even if the core functional logic of
> the patch is flawless. Failing to adhere to this typically results in **CI
> Linting Failure**.
**Trap 1: Mixing local application imports with standard library imports,
causing linting tools to fail the CQ job.**
**Don't:**
```python
import sys
import my_local_module
import os
```
**Do:**
```python
import os
import sys
import my_local_module
```
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Upstream:** T3 | CI Builder Environment and Execution Integrity -
*Reliable CI builder environments must be available to execute the static
analysis and Python formatting verifications.*
* **Downstream:** T1 | Gerrit Submission and Labeling Workflow - *Formatting
and linting rules must be fully satisfied before automated mechanisms like
the Commit-Queue (CQ+2) will merge code into the repository.*
## Chapter: Commit Metadata and History Standardization
**Context:** This domain governs the structural composition and metadata
formatting of change lists (CLs) within the git-repo codebase. Strict adherence
ensures precise issue tracker integration, reliable CI/CD parsing, and atomic,
bisectable repository history.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / Trap |
| :-------- | :------------------------ | :------- | :------------------------ |
| **T5-01** | Strict Commit Message Bug | Medium | Providing free-text |
: : Tag Formatting : : descriptions, arbitrary :
: : : : prefixes, or non-standard :
: : : : bug references in the :
: : : : commit block. :
| **T5-02** | Atomic and Bisectable | High | Waiting for an entire |
: : Change Integration : : feature stack of multiple :
: : : : interdependent CLs to be :
: : : : approved before merging :
: : : : the base commits. :
| **T5-03** | Explicit Bug Tracker | Medium | Submitting a fix or |
: : Linking for Context : : revert without :
: : Restoration : : referencing the :
: : : : corresponding bug tracker :
: : : : issue detailing the :
: : : : specific regression or :
: : : : stack trace. :
| **T5-04** | Atomic Change List | Medium | Submitting a single large |
: : Decomposition : : CL that touches multiple :
: : : : isolated components or :
: : : : implements several :
: : : : distinct features :
: : : : simultaneously. :
--------------------------------------------------------------------------------
### Rules
#### T5-01: Strict Commit Message Bug Tag Formatting
> **Rule:** Must use the exact `Bug: <number>` syntax in commit messages to
> properly link issue trackers.
>
> **What:** Commit messages must link directly to issue trackers using the
> explicit 'Bug: <number>' syntax to allow reliable parsing by CI/CD and history
> tracking systems.
>
> **Applies To:** Commit messages across all git-repo changes.
>
> **Why:** Improperly formatted bug tags fail to link with the external issue
> tracker, severing historical context and breaking automated post-submit
> tracking workflows. Failing to adhere to this typically results in **Broken
> Traceability / Pre-submit Failure**.
**Trap 1: Providing free-text descriptions, arbitrary prefixes, or non-standard
bug references in the commit block.**
**Don't:**
```text
Fixes bug 486536908
Closes issue 486536908
```
**Do:**
```text
Bug: 486536908
```
--------------------------------------------------------------------------------
#### T5-02: Atomic and Bisectable Change Integration
> **Rule:** Always submit code incrementally as isolated, functional units
> rather than hoarding monolithic stacks.
>
> **What:** Code changes must be submitted incrementally as isolated, functional
> units rather than waiting to merge a massive interdependent stack all at once.
>
> **Applies To:** Git commit history, PR structuring, and stack-based code
> integration.
>
> **Why:** Contributors accustomed to integrating full monolithic stacks at once
> held off on landing initial, stable changes. This practice hinders the ability
> to isolate regressions via `git bisect` and prevents foundational code from
> "baking" in production. Failing to adhere to this typically results in
> **Bisection Breakage / Monolithic Rollbacks**.
**Trap 1: Waiting for an entire feature stack of multiple interdependent CLs to
be approved before merging the base commits.**
**Don't:**
* Holding all changes in a stack locally or in code review until the final
feature patch is approved, then landing 10+ patches simultaneously.
**Do:**
* Landing initial, independent CLs one-by-one as soon as they are approved.
Ensuring each commit is independently usable and does not break the build.
--------------------------------------------------------------------------------
#### T5-03: Explicit Bug Tracker Linking for Context Restoration
> **Rule:** Must include a direct URL to the relevant bug tracker issue
> documenting the failure traceback when submitting a regression fix or revert.
>
> **What:** When submitting a change (especially a revert or bug fix) addressing
> a specific runtime regression, the commit metadata or patchset-level comments
> must include a direct link to the bug tracker issue documenting the failure
> traceback.
>
> **Applies To:** Commit messages and patchset documentation during code
> reviews, particularly for reverts.
>
> **Why:** A previous commit caused a runtime regression (e.g., an
> AttributeError related to a missing object attribute). Without linking the
> specific issue containing the traceback, reviewers lacked the necessary
> context to justify restoring the previous codebase state. Failing to adhere to
> this typically results in **Undocumented Regression / Context Loss**.
**Trap 1: Submitting a fix or revert without referencing the corresponding bug
tracker issue detailing the specific regression or stack trace.**
**Don't:**
* Reverting a change with a vague description like "Fixing previous breakage"
or "Reverting due to pipeline failure" without providing the traceback
source.
**Do:**
* Linking the specific issue tracker URL containing the exact failure mode.
Example: "for more context, see
https://g-issues.gerritcodereview.com/issues/[ISSUE_ID]#comment4"
--------------------------------------------------------------------------------
#### T5-04: Atomic Change List Decomposition
> **Rule:** Never submit large, monolithic change lists; always decompose them
> into logically independent patchsets.
>
> **What:** Large, monolithic change lists (CLs) must be broken down into
> smaller, logically independent patchsets to ensure accurate review and
> historical bisectability.
>
> **Applies To:** Version control history and code review scoping.
>
> **Why:** Massive CLs heavily increase reviewer cognitive load, making thorough
> reviews impossible and complicating future `git bisect` operations when
> tracking down the origin of a regression. Failing to adhere to this typically
> results in **Unreviewable Monolithic Change**.
**Trap 1: Submitting a single large CL that touches multiple isolated components
or implements several distinct features simultaneously.**
**Don't:**
* A single CL containing sweeping refactoring, new feature implementation, and
unrelated bug fixes.
**Do:**
* Breaking the monolithic change into smaller, logically dependent or
independent CLs where each addresses one specific piece of the feature or
refactor.
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Downstream:** T1 | Gerrit Submission and Labeling Workflow - *Gerrit and
CI pipelines strictly rely on standardized commit metadata to link tracking
issues and depend on atomic patchsets to execute automated review and
verification correctly.*
## Chapter: Testing Strategy and State Isolation
**Context:** This chapter governs test implementation boundaries, emphasizing
pragmatic mocking limits to prevent false positives and detailing acceptable
workflows for deferred test coverage. Strict adherence ensures robust state
isolation and maintains development velocity without compromising verification
integrity.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / Trap |
| :-------- | :----------------------- | :------- | :------------------------- |
| **T6-01** | Pragmatic Mocking | Medium | Mocking the entire core |
: : Boundaries in Unit Tests : : state or framework :
: : : : dependencies just to force :
: : : : a unit test for a highly :
: : : : integrated function. :
| **T6-02** | Deferred Test | Medium | Submitting functional code |
: : Implementation via : : without matching test :
: : Follow-up : : coverage and stalling the :
: : : : merge while complex tests :
: : : : are written. :
--------------------------------------------------------------------------------
### Rules
#### T6-01: Pragmatic Mocking Boundaries in Unit Tests
> **Rule:** Always restrict unit tests to isolated methods and avoid aggressive
> mocking of core functionality to prevent brittle, false-positive verification.
>
> **What:** Do not aggressively mock core functionality in unit tests; restrict
> unit tests to isolated methods to avoid creating brittle tests based on false
> assumptions when an integration framework is unavailable.
>
> **Applies To:** Test suite implementation (Unit vs. Integration testing
> boundaries).
>
> **Why:** Over-mocking complex systems in unit tests leads to scenarios where
> tests pass but the core integration fails in production because the unit test
> mocks assumed incorrect behavior about the underlying environment. Failing to
> adhere to this typically results in **False Positive Test Passage**.
**Trap 1: Mocking the entire core state or framework dependencies just to force
a unit test for a highly integrated function.**
**Don't:**
* Mocking file systems, external processes, and global state heavily to test a
core workflow orchestrator in a unit test suite.
**Do:**
* Limiting unit tests strictly to isolated utility methods (e.g., adding
promisor files) and explicitly documenting testing gaps that require
integration test frameworks.
**Exceptions:** Isolated helper methods or purely functional data
transformations should be fully unit tested with appropriate mocked inputs.
--------------------------------------------------------------------------------
#### T6-02: Deferred Test Implementation via Follow-up
> **Rule:** Never stall critical feature merges indefinitely for test
> implementation if maintainers authorize formalized, immediate follow-up test
> coverage.
>
> **What:** New logic requires automated tests; however, reviewers may permit
> test coverage to be implemented in a subsequent follow-up CL to maintain
> development velocity.
>
> **Applies To:** Feature development, regression testing, and code review
> criteria.
>
> **Why:** Reviewers identified a lack of test coverage for new functionality
> but opted not to block the immediate patchset, instead formalizing the test
> requirement as a near-term follow-up task. Failing to adhere to this typically
> results in **Missing Test Coverage**.
**Trap 1: Submitting functional code without matching test coverage and stalling
the merge while complex tests are written.**
**Don't:**
* Blocking a necessary feature indefinitely due to missing unit tests when a
follow-up CL is viable and acceptable to maintainers.
**Do:**
* Approve the feature with an explicit, documented 'TODO' for a follow-up CL
dedicated strictly to adding the corresponding automated tests.
**Exceptions:** Critical path features or security fixes where a lack of
immediate coverage introduces an unacceptable regression risk.
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Upstream:** T1 | Gerrit Submission and Labeling Workflow - *Reviewer
approval mechanisms and label enforcement dictate when a feature can merge
while deferring tests to a follow-up CL.*
* **Downstream:** T3 | CI Builder Environment and Execution Integrity -
*Pragmatically bounded unit and integration tests ensure reliable CI
pipeline execution without false-positive success markers.*
File diff suppressed because it is too large Load Diff
+32
View File
@@ -0,0 +1,32 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file contains version pins of a few tools used by run_tests.
# Pin resolved versions in the repo, to reduce trust in the CIPD backend.
#
# Most of these tools are generated via builders at
# https://ci.chromium.org/p/infra/g/infra/console
#
# For these, the git revision is the one of
# https://chromium.googlesource.com/infra/infra.git.
#
# To regenerate them (after modifying this file):
# cipd ensure-file-resolve -ensure-file cipd_manifest.txt
$ResolvedVersions cipd_manifest.versions
# Supported platforms. Feel free to add more as long as the tools work.
$VerifiedPlatform linux-amd64 linux-arm64 mac-amd64 mac-arm64
infra/3pp/tools/help2man/${platform} version:3@1.49.3
+18
View File
@@ -0,0 +1,18 @@
# This file is auto-generated by 'cipd ensure-file-resolve'.
# Do not modify manually. All changes will be overwritten.
infra/3pp/tools/help2man/linux-amd64
version:3@1.49.3
82ySLC4--mvjnmuwj_j_2odHLj5zFaNFpWvfCyMunY0C
infra/3pp/tools/help2man/linux-arm64
version:3@1.49.3
szfWfjI4RyCT6gIB48CeUOGG6_Un4wfbIb0Vvs5M9MUC
infra/3pp/tools/help2man/mac-amd64
version:3@1.49.3
t7iyQx5TnGbuJfj6zMTvjPdBl-Cb_w5papUh80NI2S8C
infra/3pp/tools/help2man/mac-arm64
version:3@1.49.3
t5eD-eLKRpnSifZdbi072U8bnPfGye1pIMmQ07nxJ7kC
+32 -20
View File
@@ -14,6 +14,7 @@
import os
import sys
from typing import Optional
import pager
@@ -84,28 +85,43 @@ def _Color(fg=None, bg=None, attr=None):
DEFAULT = None
def SetDefaultColoring(state):
class _CheckConsoleSentinel:
"""Sentinel for checking console coloring."""
# Placholder value that indicates we need to check if the user is in an
# interactive terminal session to determine if we turn on color or not.
_CHECK_CONSOLE = _CheckConsoleSentinel()
# https://git-scm.com/docs/git-config#Documentation/git-config.txt-colorui
_CONFIG_TO_COLOR_SETTING = {
"false": False,
"never": False,
"no": False,
"auto": _CHECK_CONSOLE,
"true": _CHECK_CONSOLE,
"yes": _CHECK_CONSOLE,
"always": True,
}
def SetDefaultColoring(state: Optional[str]) -> None:
"""Set coloring behavior to |state|.
This is useful for overriding config options via the command line.
"""
if state is None:
# Leave it alone -- return quick!
return
global DEFAULT
state = state.lower()
if state in ("auto",):
if isinstance(state, str):
state = state.lower()
if state in _CONFIG_TO_COLOR_SETTING:
DEFAULT = state
elif state in ("always", "yes", "true", True):
DEFAULT = "always"
elif state in ("never", "no", "false", False):
DEFAULT = "never"
class Coloring:
def __init__(self, config, section_type):
self._section = "color.%s" % section_type
self._section = f"color.{section_type}"
self._config = config
self._out = sys.stdout
@@ -114,16 +130,12 @@ class Coloring:
on = self._config.GetString(self._section)
if on is None:
on = self._config.GetString("color.ui")
if isinstance(on, str):
on = on.lower()
if on == "auto":
if pager.active or os.isatty(1):
self._on = True
else:
self._on = False
elif on in ("true", "always"):
self._on = True
else:
self._on = False
self._on = _CONFIG_TO_COLOR_SETTING.get(on, _CHECK_CONSOLE)
if self._on is _CHECK_CONSOLE:
self._on = pager.active or os.isatty(1)
def redirect(self, out):
self._out = out
+23 -1
View File
@@ -101,6 +101,11 @@ class Command:
def WantPager(self, _opt):
return False
@staticmethod
def is_multiprocessing_active() -> bool:
"""Whether the current process is a worker in a pool."""
return multiprocessing.current_process().name != "MainProcess"
def ReadEnvironmentOptions(self, opts):
"""Set options from environment variables."""
@@ -407,7 +412,8 @@ class Command:
for project in all_projects_list:
if submodules_ok or project.sync_s:
derived_projects.update(
(p.name, p) for p in project.GetDerivedSubprojects()
(p.RelPath(local=False), p)
for p in project.GetDerivedSubprojects()
)
all_projects_list.extend(derived_projects.values())
for project in all_projects_list:
@@ -511,6 +517,22 @@ class Command:
)
return result
def GetSmartSyncOverridePath(self, manifest=None) -> str:
"""Return the path where smart sync writes its override manifest."""
if manifest is None:
manifest = self.manifest
return os.path.join(
manifest.manifestProject.worktree, "smart_sync_override.xml"
)
def TryOverrideManifestWithSmartSync(self, manifest=None) -> None:
"""Override manifest with smart_sync_override.xml if it exists."""
if manifest is None:
manifest = self.manifest
smart_sync_manifest_path = self.GetSmartSyncOverridePath(manifest)
if os.path.isfile(smart_sync_manifest_path):
manifest.Override(smart_sync_manifest_path)
def ManifestList(self, opt):
"""Yields all of the manifests to traverse.
+449
View File
@@ -0,0 +1,449 @@
#compdef _repo repo
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
_repo() {
local context state line
typeset -A opt_args
local -a subcommands
subcommands=(
'abandon:Abandon a development branch'
'branches:View current topic branches'
'checkout:Checkout a branch'
'cherry-pick:Cherry-pick a change'
'diff:Show changes between commit and working tree'
'diffmanifests:Show differences between two manifests'
'download:Download and find a change'
'forall:Run a shell command in each project'
'gc:Garbage collect'
'grep:Print lines matching a pattern'
'help:Display help about a command'
'info:Get info about the manifest'
'init:Initialize a repo client'
'list:List projects and their names'
'manifest:Manifest management'
'overview:Display overview of topic branches'
'prune:Prune topic branches'
'rebase:Rebase local branches'
'selfupdate:Update repo'
'smartsync:Update working tree to latest known good revision'
'stage:Stage file(s) for commit'
'start:Start a new branch for development'
'status:Show the working tree status'
'sync:Update working tree'
'upload:Upload changes to the review server'
'version:Display version'
'wipe:Wipe uncommitted changes'
)
local -a global_opts
global_opts=(
'(-h --help)'{-h,--help}'[Show help message]'
'--help-all[Show help message for all commands]'
'(-p --paginate)'{-p,--paginate}'[Pipe all output into less]'
'--no-pager[Do not pipe output into a pager]'
'--color=[Control color output]:color:(always never auto)'
'--trace[Trace git command execution]'
'--trace-to-stderr[Trace git command execution to stderr]'
'--trace-python[Trace python execution]'
'--time[Time execute of command]'
'--version[Show version]'
'--show-toplevel[Show top level of workspace]'
'--event-log=[File to log events to]:file:_files'
'--git-trace2-event-log=[File to log git trace2 events to]:file:_files'
'--submanifest-path=[Path to submanifest]:path:_files'
)
_arguments -C \
${global_opts} \
'1: :->cmds' \
'*:: :->subcmds'
case ${state} in
cmds)
_describe -t commands 'repo command' subcommands
;;
subcmds)
local cmd=${words[1]}
curcontext="${curcontext%:*}-${cmd}:"
local -a common_opts
common_opts=(
'(-h --help)'{-h,--help}'[Show help message]'
'(-v --verbose)'{-v,--verbose}'[Show verbose messages]'
'(-q --quiet)'{-q,--quiet}'[Show only errors]'
'(-j --jobs)'{-j,--jobs=}'[Number of jobs to run in parallel]:jobs:'
'--outer-manifest[Use outer manifest]'
'--no-outer-manifest[Do not use outer manifest]'
'--this-manifest-only[Only use this manifest]'
'--no-this-manifest-only[Do not only use this manifest]'
'--all-manifests[Use all manifests]'
)
case ${cmd} in
abandon)
_arguments \
${common_opts} \
'--all[Abandon all branches]' \
'1: :->branch' \
'*: :->project'
;;
branches)
_arguments \
${common_opts} \
'*: :->project'
;;
checkout)
_arguments \
${common_opts} \
'1: :->branch' \
'*: :->project'
;;
cherry-pick)
_arguments \
${common_opts} \
'1: :_message "sha1"'
;;
diff)
_arguments \
${common_opts} \
'(-u --absolute)'{-u,--absolute}'[Show absolute paths]' \
'*: :->project'
;;
diffmanifests)
_arguments \
'--raw[Show raw diff]' \
'--no-color[Do not show color]' \
'--pretty-format=[Pretty format]:format:' \
'1: :_files -g "*.xml"' \
'2: :_files -g "*.xml"'
;;
download)
_arguments \
${common_opts} \
'(-b --branch)'{-b,--branch=}'[One or more branches to check]:branch:' \
'(-c --cherry-pick)'{-c,--cherry-pick}'[Cherry-pick the change]' \
'(-x --record-origin)'{-x,--record-origin}'[Record origin of cherry-pick]' \
'(-r --revert)'{-r,--revert}'[Revert the change]' \
'(-f --ff-only)'{-f,--ff-only}'[Force fast-forward]' \
'*: :_message "change[/patchset]"'
;;
forall)
_arguments \
${common_opts} \
'(-r --regex)'{-r,--regex}'[Execute command only on projects matching regex]' \
'(-i --inverse-regex)'{-i,--inverse-regex}'[Execute command only on projects not matching regex]' \
'(-g --groups)'{-g,--groups=}'[Execute command only on projects matching groups]:groups:' \
'(-c --command)'{-c,--command}'[Command to execute]' \
'(-e --abort-on-errors)'{-e,--abort-on-errors}'[Abort on errors]' \
'--ignore-missing[Ignore missing projects]' \
'--interactive[Run interactively]' \
'-p[Show project headers]' \
'*: :->project'
;;
gc)
_arguments \
${common_opts} \
'(-n --dry-run)'{-n,--dry-run}'[Dry run]' \
'(-y --yes)'{-y,--yes}'[Answer yes to all prompts]' \
'--repack[Repack objects]'
;;
grep)
_arguments \
${common_opts} \
'--cached[Search cached files]' \
'(-r --revision)'{-r,--revision=}'[Search in revision]:revision:' \
'-e[Pattern]' \
'(-i --ignore-case)'{-i,--ignore-case}'[Ignore case]' \
'(-a --text)'{-a,--text}'[Treat all files as text]' \
'-I[Do not match binary files]' \
'(-w --word-regexp)'{-w,--word-regexp}'[Match word boundaries]' \
'(-v --invert-match)'{-v,--invert-match}'[Invert match]' \
'(-G --basic-regexp)'{-G,--basic-regexp}'[Basic regexp]' \
'(-E --extended-regexp)'{-E,--extended-regexp}'[Extended regexp]' \
'(-F --fixed-strings)'{-F,--fixed-strings}'[Fixed strings]' \
'--all-match[All match]' \
'--and[And]' \
'--or[Or]' \
'--not[Not]' \
'-([Open paren]' \
'-)[Close paren]' \
'-n[Line number]' \
'-C[Context]:lines:' \
'-B[Before context]:lines:' \
'-A[After context]:lines:' \
'-l[Name only]' \
'--name-only[Name only]' \
'--files-with-matches[Files with matches]' \
'-L[Files without match]' \
'--files-without-match[Files without match]' \
'1: :->pattern' \
'*: :->project'
;;
help)
_arguments \
'(-a --all)'{-a,--all}'[Show all commands]' \
'--help-all[Show help message for all commands]' \
'1: :->help_cmds'
;;
info)
_arguments \
${common_opts} \
'(-d --diff)'{-d,--diff}'[Show diff]' \
'(-o --overview)'{-o,--overview}'[Show overview]' \
'(-c --current-branch)'{-c,--current-branch}'[Show current branch]' \
'--no-current-branch[Do not show current branch]' \
'(-l --local-only)'{-l,--local-only}'[Local only]' \
'*: :->project'
;;
init)
_arguments \
'(-u --manifest-url)'{-u,--manifest-url=}'[Manifest URL]:url:' \
'(-b --manifest-branch)'{-b,--manifest-branch=}'[Manifest branch]:branch:' \
'--manifest-upstream-branch=[Manifest upstream branch]:branch:' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'(-g --groups)'{-g,--groups=}'[Restrict manifest projects to groups]:groups:' \
'(-p --platform)'{-p,--platform=}'[Restrict manifest projects to platform]:platform:' \
'--submodules[Sync submodules]' \
'--standalone-manifest[Standalone manifest]' \
'--manifest-depth=[Manifest depth]:depth:' \
'(-c --current-branch)'{-c,--current-branch}'[Sync current branch only]' \
'--no-current-branch[Do not sync current branch only]' \
'--tags[Sync tags]' \
'--no-tags[Do not sync tags]' \
'--mirror[Mirror]' \
'--archive[Archive]' \
'--worktree[Worktree]' \
'--reference=[Reference repository]:repository:_files -/' \
'--dissociate[Dissociate from reference]' \
'--depth=[Depth]:depth:' \
'--partial-clone[Partial clone]' \
'--no-partial-clone[Do not partial clone]' \
'--partial-clone-exclude=[Exclude from partial clone]:projects:' \
'--clone-filter=[Clone filter]:filter:' \
'--use-superproject[Use superproject]' \
'--no-use-superproject[Do not use superproject]' \
'--clone-bundle[Use clone bundle]' \
'--no-clone-bundle[Do not use clone bundle]' \
'--git-lfs[Use git lfs]' \
'--no-git-lfs[Do not use git lfs]' \
'--repo-url=[Repo URL]:url:' \
'--repo-rev=[Repo revision]:revision:' \
'--no-repo-verify[Do not verify repo]' \
'--config-name[Use config name]'
;;
list)
_arguments \
${common_opts} \
'(-r --regex)'{-r,--regex}'[Filter by regex]' \
'(-g --groups)'{-g,--groups=}'[Filter by groups]:groups:' \
'(-a --all)'{-a,--all}'[Show all projects]' \
'(-n --name-only)'{-n,--name-only}'[Show only names]' \
'(-p --path-only)'{-p,--path-only}'[Show only paths]' \
'(-f --fullpath)'{-f,--fullpath}'[Show full paths]' \
'--relative-to=[Show paths relative to]:path:_files -/' \
'*: :->project'
;;
manifest)
_arguments \
'(-r --revision-as-HEAD)'{-r,--revision-as-HEAD}'[Save revisions as current HEAD]' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'--suppress-upstream-revision[Suppress upstream revision]' \
'--suppress-dest-branch[Suppress dest branch]' \
'--format=[Output format]:format:(xml json)' \
'--pretty[Pretty print]' \
'--no-local-manifests[Ignore local manifests]' \
'(-o --output-file)'{-o,--output-file=}'[Output file]:file:_files'
;;
overview)
_arguments \
${common_opts} \
'(-c --current-branch)'{-c,--current-branch}'[Show current branch]' \
'--no-current-branch[Do not show current branch]' \
'*: :->project'
;;
prune)
_arguments \
${common_opts} \
'*: :->project'
;;
rebase)
_arguments \
${common_opts} \
'--fail-fast[Fail fast]' \
'(-f --force-rebase)'{-f,--force-rebase}'[Force rebase]' \
'--no-ff[No fast forward]' \
'--autosquash[Autosquash]' \
'--whitespace=[Whitespace option]:option:' \
'--auto-stash[Auto stash]' \
'(-m --onto-manifest)'{-m,--onto-manifest}'[Rebase onto manifest]' \
'(-i --interactive)'{-i,--interactive}'[Interactive rebase]' \
'*: :->project'
;;
selfupdate)
_arguments \
'--no-repo-verify[Do not verify repo]'
;;
smartsync | sync)
_arguments \
${common_opts} \
'--jobs-network=[Number of network jobs]:jobs:' \
'--jobs-checkout=[Number of checkout jobs]:jobs:' \
'(-f --force-broken)'{-f,--force-broken}'[Continue sync even if a project fails]' \
'--fail-fast[Fail fast]' \
'--force-sync[Overwrite existing git directories]' \
'--force-checkout[Force checkout]' \
'--force-remove-dirty[Force remove dirty]' \
'--rebase[Rebase local branches]' \
'(-l --local-only)'{-l,--local-only}'[Only use local files]' \
'--no-manifest-update[Do not update manifest]' \
'--nmu[Do not update manifest]' \
'--interleaved[Interleave output]' \
'--no-interleaved[Do not interleave output]' \
'(-n --network-only)'{-n,--network-only}'[Only fetch]' \
'(-d --detach)'{-d,--detach}'[Detach HEAD]' \
'(-c --current-branch)'{-c,--current-branch}'[Fetch only current branch]' \
'--no-current-branch[Do not fetch only current branch]' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'--clone-bundle[Use clone bundle]' \
'--no-clone-bundle[Do not use clone bundle]' \
'(-u --manifest-server-username)'{-u,--manifest-server-username=}'[Username for manifest server]:username:' \
'(-p --manifest-server-password)'{-p,--manifest-server-password=}'[Password for manifest server]:password:' \
'--fetch-submodules[Fetch submodules]' \
'--use-superproject[Use superproject]' \
'--no-use-superproject[Do not use superproject]' \
'--tags[Sync tags]' \
'--no-tags[Do not sync tags]' \
'--optimized-fetch[Optimized fetch]' \
'--retry-fetches=[Retry fetches]:count:' \
'--prune[Prune]' \
'--no-prune[Do not prune]' \
'--auto-gc[Run auto gc]' \
'--no-auto-gc[Do not run auto gc]' \
'(-s --smart-sync)'{-s,--smart-sync}'[Smart sync]' \
'(-t --smart-tag)'{-t,--smart-tag=}'[Smart tag]:tag:' \
'--no-repo-verify[Do not verify repo]' \
'--no-verify[Do not verify]' \
'--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \
'*: :->project'
;;
stage)
_arguments \
${common_opts} \
'(-i --interactive)'{-i,--interactive}'[Interactive staging]' \
'*: :->project'
;;
start)
_arguments \
${common_opts} \
'--all[Start branch in all projects]' \
'(-r --rev --revision)'{-r,--rev=,--revision=}'[Revision]:revision:' \
'--head[Head]' \
'--HEAD[Head]' \
'1: :->newbranch' \
'*: :->project'
;;
status)
_arguments \
${common_opts} \
'(-o --orphans)'{-o,--orphans}'[Show orphans]' \
'*: :->project'
;;
upload)
_arguments \
${common_opts} \
'(-t --topic-branch)'{-t,--topic-branch}'[Topic branch]' \
'--topic=[Topic]:topic:' \
'--hashtag=[Hashtag]:hashtag:' \
'--ht=[Hashtag]:hashtag:' \
'--hashtag-branch[Hashtag branch]' \
'--htb[Hashtag branch]' \
'(-l --label)'{-l,--label=}'[Label]:label:' \
'--pd=[Patchset description]:description:' \
'--patchset-description=[Patchset description]:description:' \
'--re=[Reviewers]:reviewers:' \
'--reviewers=[Reviewers]:reviewers:' \
'--cc=[CC]:cc:' \
'--br=[Branch]:branch:' \
'--branch=[Branch]:branch:' \
'(-c --current-branch)'{-c,--current-branch}'[Upload current branch]' \
'--no-current-branch[Do not upload current branch]' \
'--ne[No emails]' \
'--no-emails[No emails]' \
'(-p --private)'{-p,--private}'[Private]' \
'(-w --wip)'{-w,--wip}'[Work in progress]' \
'(-r --ready)'{-r,--ready}'[Ready]' \
'(-o --push-option)'{-o,--push-option=}'[Push option]:option:' \
'(-D --destination --dest)'{-D,--destination=,--dest=}'[Destination]:destination:' \
'(-n --dry-run)'{-n,--dry-run}'[Dry run]' \
'(-y --yes)'{-y,--yes}'[Answer yes to all prompts]' \
'--ignore-untracked-files[Ignore untracked files]' \
'--no-ignore-untracked-files[Do not ignore untracked files]' \
'--no-cert-checks[Do not check certificates]' \
'--no-verify[Do not verify]' \
'--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \
'*: :->project'
;;
version)
_arguments ${common_opts}
;;
wipe)
_arguments \
${common_opts} \
'(-f --force)'{-f,--force}'[Force wipe]' \
'--force-uncommitted[Force uncommitted]' \
'--force-shared[Force shared]' \
'*: :->project'
;;
esac
# Handle states for positional arguments.
case ${state} in
branch)
[[ ${PREFIX} != -* ]] && _repo_branches
;;
project)
[[ ${PREFIX} != -* ]] && _repo_projects
;;
pattern)
_message 'pattern'
;;
newbranch)
_message 'new branch name'
;;
help_cmds)
_describe -t commands 'repo command' subcommands
;;
esac
;;
esac
}
_repo_branches() {
local -a branches
branches=(${(f)"$(_call_program branches repo branches 2>/dev/null | sed -E 's/^.*[[:space:]]([^[:space:]]+)[[:space:]]+\|.*$/\1/' | awk '{print $1}')"})
_describe -t branches 'branch' branches
}
_repo_projects() {
local -a projects
projects=(${(f)"$(_call_program projects repo list -n 2>/dev/null)"})
_describe -t projects 'project' projects
}
_repo "$@"
+63
View File
@@ -0,0 +1,63 @@
# Fetch Command Contract
The `repo.fetchcmd` configuration allows specifying a custom command to be
executed during `repo sync` to fetch objects, instead of using standard
`git fetch`. This is particularly useful in environments with virtualized
filesystems or lazy checkouts where fetching metadata and downloading file
contents should be decoupled.
## Configuration
To use this feature, set the following in `.repo/manifests.git/config`:
```ini
[repo]
fetchcmd = "your custom command here"
uselocalgitdirs = true
```
Setting `repo.fetchcmd` **requires** `repo.uselocalgitdirs` to be set to `true`.
## Environment Variables
The custom command is executed in a subshell populated with standard
project-context environment variables. For details on standard variables (such
as `REPO_PROJECT`, `REPO_PATH`, `REPO_PROJECT_FETCH_URL`, etc.), see the
Environment section in `repo help forall` or `subcmds/forall.py`.
The following environment variable is specific to `repo.fetchcmd`:
* `REPO_TREV`: The target revision resolved to a full commit hash.
## Contract
### Postconditions on exit 0
After the fetch command exits with status 0, `repo` expects the following
postconditions to be met:
1. `git cat-file -e REPO_TREV` succeeds (the commit must exist in the object
store).
2. The mapped local tracking ref (e.g. `refs/remotes/REPO_REMOTE/<branch>`
for a branch revision, or the tag ref itself for a tag) must point to
`REPO_TREV`.
3. `FETCH_HEAD` must point to `REPO_TREV`.
4. The commit graph from `REPO_TREV` must be reachable far enough to compute
merge bases with local branches.
### Invariants
* The command should be idempotent; fetching the same `REPO_TREV` twice should
be a no-op.
* Only `FETCH_HEAD` and `refs/remotes/*` should be modified to preserve
`repo sync --network-only` semantics. `HEAD` and local branches must not be
touched by the fetch command.
* Dirty worktree state must be preserved.
* The command is **not** executed for `MetaProject`s (i.e. the internal `repo`
repository itself at `.repo/repo` and the `manifests` repository at
`.repo/manifests`).
### Failure
* A non-zero exit status aborts the project's sync, and the command's stderr
is surfaced to the user.
* `repo` verifies the tracking ref and target reachability after exit 0. Any
mismatch is treated as a failure.
+1 -1
View File
@@ -182,7 +182,7 @@ User controlled settings are initialized when running `repo init`.
| user.email | `--config-name` | User's e-mail address; Copied into `.git/config` when checking out a new project |
| user.name | `--config-name` | User's name; Copied into `.git/config` when checking out a new project |
[partial git clones]: https://git-scm.com/docs/gitrepository-layout#_code_partialclone_code
[partial git clones]: https://git-scm.com/docs/partial-clone
[superproject]: https://en.wikibooks.org/wiki/Git/Submodules_and_Superprojects
### Repo hooks settings
+6
View File
@@ -58,6 +58,7 @@ following DTD:
<!ELEMENT manifest-server EMPTY>
<!ATTLIST manifest-server url CDATA #REQUIRED>
<!ATTLIST manifest-server helper CDATA #IMPLIED>
<!ELEMENT submanifest EMPTY>
<!ATTLIST submanifest name ID #REQUIRED>
@@ -239,6 +240,11 @@ XML RPC service.
See the [smart sync documentation](./smart-sync.md) for more details.
Attribute `url`: The URL of the manifest server.
Attribute `helper`: Optional name of a remote helper binary to execute to
resolve proxying or authentication for the manifest server.
### Element submanifest
+2 -1
View File
@@ -163,13 +163,14 @@ Example:
The `post-sync.py` file should be defined like:
```py
def main(repo_topdir=None, **kwargs):
def main(repo_topdir=None, sync_duration_seconds=None, **kwargs):
"""Main function invoked directly by repo.
We must use the name "main" as that is what repo requires.
Args:
repo_topdir: The absolute path to the top-level directory of the repo workspace.
sync_duration_seconds: The duration of the sync operation in seconds.
kwargs: Leave this here for forward-compatibility.
"""
```
+32 -1
View File
@@ -27,13 +27,44 @@ the [`<manifest-server>` element](manifest-format.md#Element-manifest_server)
element. This is how the client knows what service to talk to.
```xml
<manifest-server url="https://example.com/your/manifest/server/url" />
<manifest-server url="https://example.com/your/manifest/server/url"
helper="repo-remote-helper-name" />
```
If the URL starts with `persistent-`, then the
[`git-remote-persistent-https` helper](https://github.com/git/git/blob/HEAD/contrib/persistent-https/README)
is used to communicate with the server.
### Pluggable Remote Helpers
For custom proxying or authentication, Repo supports pluggable remote helpers.
You can declare a helper binary via the optional `helper` attribute on the
`<manifest-server>` element.
If the `helper` attribute is present in `<manifest-server>`:
1. Repo searches your system `PATH` for the specified helper binary (e.g.,
`repo-remote-sso`). If the helper cannot be found in `PATH`, Repo will raise
a `SmartSyncError` and abort the sync.
2. The helper is executed with the manifest server URL as its first argument:
```bash
<helper-binary-name> <url>
```
3. The helper must output a single-line JSON object on stdout and exit:
* On success, return `status: "ok"` and a loopback proxy URL (including
scheme, e.g., `http://127.0.0.1:999`):
```json
{"status": "ok", "message": "http://127.0.0.1:999"}
```
* On failure, return `status: "error"` and a detailed error message:
```json
{"status": "error", "message": "unauthorized"}
```
4. Repo routes the XML-RPC request through the returned proxy address.
**Timeout Constraint**: The remote helper must complete and exit within 10
seconds. If it hangs or exceeds this limit, Repo will force-terminate (`kill`)
the process and abort the synchronization.
## Credentials
Credentials may be specified directly in typical `username:password`
+1 -1
View File
@@ -145,7 +145,7 @@ You will have to specify this flag every time you upload.
[mintty]: https://github.com/mintty/mintty/issues/56
### repohooks always fail with an close_fds error.
### repohooks always fail with an close_fds error
When using the [reference repohooks project][repohooks] included in AOSP,
you might see errors like this when running `repo upload`:
+4 -3
View File
@@ -34,7 +34,7 @@ def fetch_file(url, verbose=False):
"""
scheme = urlparse(url).scheme
if scheme == "gs":
cmd = ["gsutil", "cat", url]
cmd = ["gcloud", "storage", "cat", url]
errors = []
try:
result = subprocess.run(
@@ -42,7 +42,7 @@ def fetch_file(url, verbose=False):
)
if result.stderr and verbose:
print(
'warning: non-fatal error running "gsutil": %s'
'warning: non-fatal error running "gcloud storage": %s'
% result.stderr,
file=sys.stderr,
)
@@ -50,7 +50,8 @@ def fetch_file(url, verbose=False):
except subprocess.CalledProcessError as e:
errors.append(e)
print(
'fatal: error running "gsutil": %s' % e.stderr, file=sys.stderr
'fatal: error running "gcloud storage": %s' % e.stderr,
file=sys.stderr,
)
raise FetchFileError(aggregate_errors=errors)
with urlopen(url) as f:
+8 -5
View File
@@ -40,7 +40,7 @@ from repo_trace import Trace
# that is saved in the config.
SYNC_STATE_PREFIX = "repo.syncstate."
ID_RE = re.compile(r"^[0-9a-f]{40}$")
ID_RE = re.compile(r"^[0-9a-f]{40,64}$")
REVIEW_CACHE = {}
@@ -49,8 +49,8 @@ def IsChange(rev):
return rev.startswith(R_CHANGES)
def IsId(rev):
return ID_RE.match(rev)
def IsId(rev: str) -> bool:
return bool(ID_RE.match(rev))
def IsTag(rev):
@@ -438,7 +438,7 @@ class GitConfig:
if p.Wait() == 0:
return p.stdout
else:
raise GitError(f"git config {str(args)}: {p.stderr}")
raise GitError(f"git {' '.join(command)}: {p.stderr}")
class RepoConfig(GitConfig):
@@ -724,7 +724,10 @@ class Remote:
def Save(self):
"""Save this remote to the configuration."""
self._Set("url", self.url)
if self.pushUrl is not None:
# projectname is initialized for projects listed in the manifest, but
# not for others (e.g. the manifest project). This class is used for
# all of them.
if self.pushUrl is not None and self.projectname is not None:
self._Set("pushurl", self.pushUrl + "/" + self.projectname)
else:
self._Set("pushurl", self.pushUrl)
+20 -5
View File
@@ -34,6 +34,7 @@ import urllib.parse
from git_command import git_require
from git_command import GitCommand
from git_config import IsId
from git_config import RepoConfig
from git_refs import GitRefs
import platform_utils
@@ -100,7 +101,7 @@ class Superproject:
self._manifest = manifest
self.name = name
self.remote = remote
self.revision = self._branch = revision
self.revision = revision
self._repodir = manifest.repodir
self._superproject_dir = superproject_dir
self._superproject_path = manifest.SubmanifestInfoDir(
@@ -132,6 +133,10 @@ class Superproject:
"""Set the _print_messages attribute."""
self._print_messages = value
def SetRevisionId(self, revision_id: str) -> None:
"""Set the revisionId of the superproject to sync to."""
self.revision = revision_id
@property
def commit_id(self):
"""Returns the commit ID of the superproject checkout."""
@@ -199,7 +204,8 @@ class Superproject:
def _LogMessagePrefix(self):
"""Returns the prefix string to be logged in each log message"""
return (
f"repo superproject branch: {self._branch} url: {self._remote_url}"
f"repo superproject revision: {self.revision} "
f"url: {self._remote_url}"
)
def _LogError(self, fmt, *inputs):
@@ -312,8 +318,15 @@ class Superproject:
if rev_commit:
cmd.extend(["--negotiation-tip", rev_commit])
if self._branch:
cmd += [self._branch + ":" + self._branch]
if self.revision:
# If revision is a commit hash, fetch it directly to avoid
# creating a local branch of the same name.
refspec = (
self.revision
if IsId(self.revision)
else f"{self.revision}:{self.revision}"
)
cmd.append(refspec)
p = GitCommand(
None,
cmd,
@@ -348,7 +361,7 @@ class Superproject:
)
return None
data = None
branch = "HEAD" if not self._branch else self._branch
branch = "HEAD" if not self.revision else self.revision
cmd = ["ls-tree", "-z", "-r", branch]
p = GitCommand(
@@ -400,6 +413,8 @@ class Superproject:
if not self._Init():
return SyncResult(False, should_exit)
if IsId(self.revision) and self.commit_id:
return SyncResult(True, False)
if not self._Fetch():
return SyncResult(False, should_exit)
if not self._quiet:
+7 -1
View File
@@ -195,7 +195,13 @@ class BaseEventLog:
def GetDataEventName(self, value):
"""Returns 'data-json' if the value is an array else returns 'data'."""
return "data-json" if value[0] == "[" and value[-1] == "]" else "data"
return (
"data-json"
if isinstance(value, str)
and value.startswith("[")
and value.endswith("]")
else "data"
)
def LogDataConfigEvents(self, config, prefix):
"""Append a 'data' event for each entry in |config| to the current log.
+1 -1
View File
@@ -25,7 +25,7 @@ from git_refs import HEAD
# The API we've documented to hook authors. Keep in sync with repo-hooks.md.
_API_ARGS = {
"pre-upload": {"project_list", "worktree_list"},
"post-sync": {"repo_topdir"},
"post-sync": {"repo_topdir", "sync_duration_seconds"},
}
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo abandon" "Repo Manual"
.TH REPO "1" "June 2026" "repo abandon" "Repo Manual"
.SH NAME
repo \- repo abandon - manual page for repo abandon
.SH SYNOPSIS
@@ -20,8 +20,7 @@ It is equivalent to "git branch \fB\-D\fR <branchname>".
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-\-all\fR
delete all branches in all projects
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo branches" "Repo Manual"
.TH REPO "1" "June 2026" "repo branches" "Repo Manual"
.SH NAME
repo \- repo branches - manual page for repo branches
.SH SYNOPSIS
@@ -46,8 +46,7 @@ is shown, then the branch appears in all projects.
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo checkout" "Repo Manual"
.TH REPO "1" "June 2026" "repo checkout" "Repo Manual"
.SH NAME
repo \- repo checkout - manual page for repo checkout
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Checkout a branch for development
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo diff" "Repo Manual"
.TH REPO "1" "June 2026" "repo diff" "Repo Manual"
.SH NAME
repo \- repo diff - manual page for repo diff
.SH SYNOPSIS
@@ -19,8 +19,7 @@ to the Unix 'patch' command.
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-u\fR, \fB\-\-absolute\fR
paths are relative to the repository root
+13 -11
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo forall" "Repo Manual"
.TH REPO "1" "June 2026" "repo forall" "Repo Manual"
.SH NAME
repo \- repo forall - manual page for repo forall
.SH SYNOPSIS
@@ -17,20 +17,16 @@ repo forall \fB\-r\fR str1 [str2] ... \fB\-c\fR <command> [<arg>...]
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-r\fR, \fB\-\-regex\fR
execute the command only on projects matching regex or
wildcard expression
execute the command only on projects matching regex or wildcard expression
.TP
\fB\-i\fR, \fB\-\-inverse\-regex\fR
execute the command only on projects not matching
regex or wildcard expression
execute the command only on projects not matching regex or wildcard expression
.TP
\fB\-g\fR GROUPS, \fB\-\-groups\fR=\fI\,GROUPS\/\fR
execute the command only on projects matching the
specified groups
execute the command only on projects matching the specified groups
.TP
\fB\-c\fR, \fB\-\-command\fR
command (and arguments) to execute
@@ -39,8 +35,7 @@ command (and arguments) to execute
abort if a command exits unsuccessfully
.TP
\fB\-\-ignore\-missing\fR
silently skip & do not exit non\-zero due missing
checkouts
silently skip & do not exit non\-zero due missing checkouts
.TP
\fB\-\-interactive\fR
force interactive usage
@@ -120,6 +115,13 @@ git command, use REPO_LREV.
REPO_RREV is the name of the revision from the manifest, exactly as written in
the manifest.
.PP
REPO_UPSTREAM is the name of the upstream branch as specified in the manifest.
.PP
REPO_DEST_BRANCH is the name of the destination branch for code review, as
specified in the manifest.
.PP
REPO_PROJECT_FETCH_URL is the full resolved fetch URL for the project.
.PP
REPO_COUNT is the total number of projects being iterated.
.PP
REPO_I is the current (1\-based) iteration count. Can be used in conjunction with
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "April 2025" "repo gc" "Repo Manual"
.TH REPO "1" "June 2026" "repo gc" "Repo Manual"
.SH NAME
repo \- repo gc - manual page for repo gc
.SH SYNOPSIS
@@ -21,8 +21,7 @@ do everything except actually delete
answer yes to all safe prompts
.TP
\fB\-\-repack\fR
repack all projects that use partial clone with
filter=blob:none
repack all projects that use partial clone with filter=blob:none
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo grep" "Repo Manual"
.TH REPO "1" "June 2026" "repo grep" "Repo Manual"
.SH NAME
repo \- repo grep - manual page for repo grep
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Print lines matching a pattern
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.SS Logging options:
.TP
\fB\-\-verbose\fR
+21 -4
View File
@@ -1,10 +1,10 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo info" "Repo Manual"
.TH REPO "1" "June 2026" "repo info" "Repo Manual"
.SH NAME
repo \- repo info - manual page for repo info
.SH SYNOPSIS
.B repo
\fI\,info \/\fR[\fI\,-dl\/\fR] [\fI\,-o \/\fR[\fI\,-c\/\fR]] [\fI\,<project>\/\fR...]
\fI\,info \/\fR[\fI\,-dl\/\fR] [\fI\,-o \/\fR[\fI\,-c\/\fR]] [\fI\,--format=<format>\/\fR] [\fI\,<project>\/\fR...]
.SH DESCRIPTION
Summary
.PP
@@ -14,13 +14,27 @@ Get info on the manifest branch, current branch or unmerged branches
\fB\-h\fR, \fB\-\-help\fR
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-d\fR, \fB\-\-diff\fR
show full info and commit diff including remote
branches
show full info and commit diff including remote branches
.TP
\fB\-o\fR, \fB\-\-overview\fR
show overview of all local commits
.TP
\fB\-\-include\-summary\fR
include manifest summary (default: true)
.TP
\fB\-\-no\-include\-summary\fR
exclude manifest summary
.TP
\fB\-\-include\-projects\fR
include project details (default: true)
.TP
\fB\-\-no\-include\-projects\fR
exclude project details
.TP
\fB\-c\fR, \fB\-\-current\-branch\fR
consider only checked out branches
.TP
@@ -29,6 +43,9 @@ consider all local branches
.TP
\fB\-l\fR, \fB\-\-local\-only\fR
disable all remote operations
.TP
\fB\-\-format\fR=\fI\,FORMAT\/\fR
output format: text, json (default: text)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+19 -30
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "September 2024" "repo init" "Repo Manual"
.TH REPO "1" "June 2026" "repo init" "Repo Manual"
.SH NAME
repo \- repo init - manual page for repo init
.SH SYNOPSIS
@@ -29,36 +29,29 @@ manifest repository location
manifest branch or revision (use HEAD for default)
.TP
\fB\-\-manifest\-upstream\-branch\fR=\fI\,BRANCH\/\fR
when a commit is provided to \fB\-\-manifest\-branch\fR, this
is the name of the git ref in which the commit can be
found
when a commit is provided to \fB\-\-manifest\-branch\fR, this is the name of the git ref in which the commit can be found
.TP
\fB\-m\fR NAME.xml, \fB\-\-manifest\-name\fR=\fI\,NAME\/\fR.xml
initial manifest file
.TP
\fB\-g\fR GROUP, \fB\-\-groups\fR=\fI\,GROUP\/\fR
restrict manifest projects to ones with specified
group(s) [default|all|G1,G2,G3|G4,\-G5,\-G6]
restrict manifest projects to ones with specified group(s) [default|all|G1,G2,G3|G4,\-G5,\-G6]
.TP
\fB\-p\fR PLATFORM, \fB\-\-platform\fR=\fI\,PLATFORM\/\fR
restrict manifest projects to ones with a specified
platform group [auto|all|none|linux|darwin|...]
restrict manifest projects to ones with a specified platform group [auto|all|none|linux|darwin|...]
.TP
\fB\-\-submodules\fR
sync any submodules associated with the manifest repo
.TP
\fB\-\-standalone\-manifest\fR
download the manifest as a static file rather then
create a git checkout of the manifest repo
download the manifest as a static file rather then create a git checkout of the manifest repo
.TP
\fB\-\-manifest\-depth\fR=\fI\,DEPTH\/\fR
create a shallow clone of the manifest repo with given
depth (0 for full clone); see git clone (default: 0)
create a shallow clone of the manifest repo with given depth (0 for full clone); see git clone (default: 0)
.SS Manifest (only) checkout options:
.TP
\fB\-c\fR, \fB\-\-current\-branch\fR
fetch only current manifest branch from server
(default)
fetch only current manifest branch from server (default)
.TP
\fB\-\-no\-current\-branch\fR
fetch all manifest branches from server
@@ -71,15 +64,16 @@ don't fetch tags in the manifest
.SS Checkout modes:
.TP
\fB\-\-mirror\fR
create a replica of the remote repositories rather
than a client working directory
create a replica of the remote repositories rather than a client working directory
.TP
\fB\-\-archive\fR
checkout an archive instead of a git repository for
each project. See git archive.
checkout an archive instead of a git repository for each project. See git archive.
.TP
\fB\-\-worktree\fR
use git\-worktree to manage projects
.TP
\fB\-\-use\-local\-gitdirs\fR
bypass .repo/projects/ and use standard Git layout in working tree
.SS Project checkout optimizations:
.TP
\fB\-\-reference\fR=\fI\,DIR\/\fR
@@ -92,33 +86,28 @@ dissociate from reference mirrors after clone
create a shallow clone with given depth; see git clone
.TP
\fB\-\-partial\-clone\fR
perform partial clone (https://gitscm.com/docs/gitrepositorylayout#_code_partialclone_code)
perform partial clone (https://git\-scm.com/docs/partial\-clone)
.TP
\fB\-\-no\-partial\-clone\fR
disable use of partial clone (https://gitscm.com/docs/gitrepositorylayout#_code_partialclone_code)
disable use of partial clone (https://git\-scm.com/docs/partial\-clone)
.TP
\fB\-\-partial\-clone\-exclude\fR=\fI\,PARTIAL_CLONE_EXCLUDE\/\fR
exclude the specified projects (a comma\-delimited
project names) from partial clone (https://gitscm.com/docs/gitrepositorylayout#_code_partialclone_code)
exclude the specified projects (a comma\-delimited project names) from partial clone (https://git\-scm.com/docs/partial\-clone)
.TP
\fB\-\-clone\-filter\fR=\fI\,CLONE_FILTER\/\fR
filter for use with \fB\-\-partial\-clone\fR [default:
blob:none]
filter for use with \fB\-\-partial\-clone\fR [default: blob:none]
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects;
implies \fB\-c\fR
use the manifest superproject to sync projects; implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-\-clone\-bundle\fR
enable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if
not \fB\-\-partial\-clone\fR)
enable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if not \fB\-\-partial\-clone\fR)
.TP
\fB\-\-no\-clone\-bundle\fR
disable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if
\fB\-\-partial\-clone\fR)
disable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if \fB\-\-partial\-clone\fR)
.TP
\fB\-\-git\-lfs\fR
enable Git LFS support
+5 -9
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo list" "Repo Manual"
.TH REPO "1" "June 2026" "repo list" "Repo Manual"
.SH NAME
repo \- repo list - manual page for repo list
.SH SYNOPSIS
@@ -17,12 +17,10 @@ repo list [\-f] \fB\-r\fR str1 [str2]...
show this help message and exit
.TP
\fB\-r\fR, \fB\-\-regex\fR
filter the project list based on regex or wildcard
matching of strings
filter the project list based on regex or wildcard matching of strings
.TP
\fB\-g\fR GROUPS, \fB\-\-groups\fR=\fI\,GROUPS\/\fR
filter the project list based on the groups the
project is in
filter the project list based on the groups the project is in
.TP
\fB\-a\fR, \fB\-\-all\fR
show projects regardless of checkout state
@@ -34,12 +32,10 @@ display only the name of the repository
display only the path of the repository
.TP
\fB\-f\fR, \fB\-\-fullpath\fR
display the full work tree path instead of the
relative path
display the full work tree path instead of the relative path
.TP
\fB\-\-relative\-to\fR=\fI\,PATH\/\fR
display paths relative to this one (default: top of
repo client checkout)
display paths relative to this one (default: top of repo client checkout)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+10 -9
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "April 2026" "repo manifest" "Repo Manual"
.TH REPO "1" "June 2026" "repo manifest" "Repo Manual"
.SH NAME
repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS
@@ -21,14 +21,10 @@ save revisions as current HEAD
temporary manifest to use for this sync
.TP
\fB\-\-suppress\-upstream\-revision\fR
if in \fB\-r\fR mode, do not write the upstream field (only
of use if the branch names for a sha1 manifest are
sensitive)
if in \fB\-r\fR mode, do not write the upstream field (only of use if the branch names for a sha1 manifest are sensitive)
.TP
\fB\-\-suppress\-dest\-branch\fR
if in \fB\-r\fR mode, do not write the dest\-branch field
(only of use if the branch names for a sha1 manifest
are sensitive)
if in \fB\-r\fR mode, do not write the dest\-branch field (only of use if the branch names for a sha1 manifest are sensitive)
.TP
\fB\-\-format\fR=\fI\,FORMAT\/\fR
output format: xml, json (default: xml)
@@ -40,8 +36,7 @@ format output for humans to read
ignore local manifests
.TP
\fB\-o\fR \-|NAME.xml, \fB\-\-output\-file\fR=\fI\,\-\/\fR|NAME.xml
file to save the manifest to. (Filename prefix for
multi\-tree.)
file to save the manifest to. (Filename prefix for multi\-tree.)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
@@ -138,6 +133,7 @@ include*)>
.IP
<!ELEMENT manifest\-server EMPTY>
<!ATTLIST manifest\-server url CDATA #REQUIRED>
<!ATTLIST manifest\-server helper CDATA #IMPLIED>
.IP
<!ELEMENT submanifest EMPTY>
<!ATTLIST submanifest name ID #REQUIRED>
@@ -347,6 +343,11 @@ specify the URL of a manifest server, which is an XML RPC service.
.PP
See the [smart sync documentation](./smart\-sync.md) for more details.
.PP
Attribute `url`: The URL of the manifest server.
.PP
Attribute `helper`: Optional name of a remote helper binary to execute to
resolve proxying or authentication for the manifest server.
.PP
Element submanifest
.PP
One or more submanifest elements may be specified. Each element describes a
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo prune" "Repo Manual"
.TH REPO "1" "June 2026" "repo prune" "Repo Manual"
.SH NAME
repo \- repo prune - manual page for repo prune
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Prune (delete) already merged topics
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -4
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo rebase" "Repo Manual"
.TH REPO "1" "June 2026" "repo rebase" "Repo Manual"
.SH NAME
repo \- repo rebase - manual page for repo rebase
.SH SYNOPSIS
@@ -33,9 +33,7 @@ pass \fB\-\-whitespace\fR to git rebase
stash local modifications before starting
.TP
\fB\-m\fR, \fB\-\-onto\-manifest\fR
rebase onto the manifest version instead of upstream
HEAD (this helps to make sure the local tree stays
consistent if you previously synced to a manifest)
rebase onto the manifest version instead of upstream HEAD (this helps to make sure the local tree stays consistent if you previously synced to a manifest)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+16 -28
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "August 2025" "repo smartsync" "Repo Manual"
.TH REPO "1" "June 2026" "repo smartsync" "Repo Manual"
.SH NAME
repo \- repo smartsync - manual page for repo smartsync
.SH SYNOPSIS
@@ -15,16 +15,13 @@ Update working tree to the latest known good revision
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-\-jobs\-network\fR=\fI\,JOBS\/\fR
number of network jobs to run in parallel (defaults to
\fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel
(defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-nointerleaved\fR is set
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future)
@@ -33,30 +30,22 @@ obsolete option (to be deleted in the future)
stop syncing after first error is hit
.TP
\fB\-\-force\-sync\fR
overwrite an existing git directory if it needs to
point to a different object directory. WARNING: this
may cause loss of data
overwrite an existing git directory if it needs to point to a different object directory. WARNING: this may cause loss of data
.TP
\fB\-\-force\-checkout\fR
force checkout even if it results in throwing away
uncommitted modifications. WARNING: this may cause
loss of data
force checkout even if it results in throwing away uncommitted modifications. WARNING: this may cause loss of data
.TP
\fB\-\-force\-remove\-dirty\fR
force remove projects with uncommitted modifications
if projects no longer exist in the manifest. WARNING:
this may cause loss of data
force remove projects with uncommitted modifications if projects no longer exist in the manifest. WARNING: this may cause loss of data
.TP
\fB\-\-rebase\fR
rebase local commits regardless of whether they are
published
rebase local commits regardless of whether they are published
.TP
\fB\-l\fR, \fB\-\-local\-only\fR
only update working tree, don't fetch
.TP
\fB\-\-no\-manifest\-update\fR, \fB\-\-nmu\fR
use the existing manifest checkout as\-is. (do not
update to the latest revision)
use the existing manifest checkout as\-is. (do not update to the latest revision)
.TP
\fB\-\-interleaved\fR
fetch and checkout projects in parallel (default)
@@ -95,12 +84,14 @@ password to authenticate with the manifest server
fetch submodules from server
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects;
implies \fB\-c\fR
use the manifest superproject to sync projects; implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-\-superproject\-revision\fR=\fI\,SUPERPROJECT_REVISION\/\fR
sync to superproject revision (applies to outer manifest)
.TP
\fB\-\-tags\fR
fetch tags
.TP
@@ -108,15 +99,13 @@ fetch tags
don't fetch tags (default)
.TP
\fB\-\-optimized\-fetch\fR
only fetch projects fixed to sha1 if revision does not
exist locally
only fetch projects fixed to sha1 if revision does not exist locally
.TP
\fB\-\-retry\-fetches\fR=\fI\,RETRY_FETCHES\/\fR
number of times to retry fetches on transient errors
.TP
\fB\-\-prune\fR
delete refs that no longer exist on the remote
(default)
delete refs that no longer exist on the remote (default)
.TP
\fB\-\-no\-prune\fR
do not delete refs that no longer exist on the remote
@@ -125,8 +114,7 @@ do not delete refs that no longer exist on the remote
run garbage collection on all synced projects
.TP
\fB\-\-no\-auto\-gc\fR
do not run garbage collection on any projects
(default)
do not run garbage collection on any projects (default)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo start" "Repo Manual"
.TH REPO "1" "June 2026" "repo start" "Repo Manual"
.SH NAME
repo \- repo start - manual page for repo start
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Start a new branch for development
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-\-all\fR
begin branch in all projects
+12 -5
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo status" "Repo Manual"
.TH REPO "1" "June 2026" "repo status" "Repo Manual"
.SH NAME
repo \- repo status - manual page for repo status
.SH SYNOPSIS
@@ -15,12 +15,10 @@ Show the working tree status
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-o\fR, \fB\-\-orphans\fR
include objects in working directory outside of repo
projects
include objects in working directory outside of repo projects
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
@@ -70,6 +68,15 @@ branch devwork
\fB\-m\fR
subcmds/status.py
.PP
If the branch is tracking an upstream branch, the number of commits ahead and/or
behind is also shown:
.TP
project repo/
branch devwork [ahead 1, behind 2]
.TP
\fB\-m\fR
subcmds/status.py
.PP
The first column explains how the staging area (index) differs from the last
commit (HEAD). Its values are always displayed in upper case and have the
following meanings:
+17 -30
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "August 2025" "repo sync" "Repo Manual"
.TH REPO "1" "June 2026" "repo sync" "Repo Manual"
.SH NAME
repo \- repo sync - manual page for repo sync
.SH SYNOPSIS
@@ -15,16 +15,13 @@ Update working tree to the latest revision
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-\-jobs\-network\fR=\fI\,JOBS\/\fR
number of network jobs to run in parallel (defaults to
\fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel
(defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-nointerleaved\fR is set
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future)
@@ -33,30 +30,22 @@ obsolete option (to be deleted in the future)
stop syncing after first error is hit
.TP
\fB\-\-force\-sync\fR
overwrite an existing git directory if it needs to
point to a different object directory. WARNING: this
may cause loss of data
overwrite an existing git directory if it needs to point to a different object directory. WARNING: this may cause loss of data
.TP
\fB\-\-force\-checkout\fR
force checkout even if it results in throwing away
uncommitted modifications. WARNING: this may cause
loss of data
force checkout even if it results in throwing away uncommitted modifications. WARNING: this may cause loss of data
.TP
\fB\-\-force\-remove\-dirty\fR
force remove projects with uncommitted modifications
if projects no longer exist in the manifest. WARNING:
this may cause loss of data
force remove projects with uncommitted modifications if projects no longer exist in the manifest. WARNING: this may cause loss of data
.TP
\fB\-\-rebase\fR
rebase local commits regardless of whether they are
published
rebase local commits regardless of whether they are published
.TP
\fB\-l\fR, \fB\-\-local\-only\fR
only update working tree, don't fetch
.TP
\fB\-\-no\-manifest\-update\fR, \fB\-\-nmu\fR
use the existing manifest checkout as\-is. (do not
update to the latest revision)
use the existing manifest checkout as\-is. (do not update to the latest revision)
.TP
\fB\-\-interleaved\fR
fetch and checkout projects in parallel (default)
@@ -95,12 +84,14 @@ password to authenticate with the manifest server
fetch submodules from server
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects;
implies \fB\-c\fR
use the manifest superproject to sync projects; implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-\-superproject\-revision\fR=\fI\,SUPERPROJECT_REVISION\/\fR
sync to superproject revision (applies to outer manifest)
.TP
\fB\-\-tags\fR
fetch tags
.TP
@@ -108,15 +99,13 @@ fetch tags
don't fetch tags (default)
.TP
\fB\-\-optimized\-fetch\fR
only fetch projects fixed to sha1 if revision does not
exist locally
only fetch projects fixed to sha1 if revision does not exist locally
.TP
\fB\-\-retry\-fetches\fR=\fI\,RETRY_FETCHES\/\fR
number of times to retry fetches on transient errors
.TP
\fB\-\-prune\fR
delete refs that no longer exist on the remote
(default)
delete refs that no longer exist on the remote (default)
.TP
\fB\-\-no\-prune\fR
do not delete refs that no longer exist on the remote
@@ -125,12 +114,10 @@ do not delete refs that no longer exist on the remote
run garbage collection on all synced projects
.TP
\fB\-\-no\-auto\-gc\fR
do not run garbage collection on any projects
(default)
do not run garbage collection on any projects (default)
.TP
\fB\-s\fR, \fB\-\-smart\-sync\fR
smart sync using manifest from the latest known good
build
smart sync using manifest from the latest known good build
.TP
\fB\-t\fR SMART_TAG, \fB\-\-smart\-tag\fR=\fI\,SMART_TAG\/\fR
smart sync using manifest from a known tag
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2024" "repo upload" "Repo Manual"
.TH REPO "1" "June 2026" "repo upload" "Repo Manual"
.SH NAME
repo \- repo upload - manual page for repo upload
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Upload changes for code review
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-t\fR, \fB\-\-topic\-branch\fR
set the topic to the local branch name
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "November 2025" "repo wipe" "Repo Manual"
.TH REPO "1" "June 2026" "repo wipe" "Repo Manual"
.SH NAME
repo \- repo wipe - manual page for repo wipe
.SH SYNOPSIS
@@ -21,8 +21,7 @@ force wipe shared projects and uncommitted changes
force wipe even if there are uncommitted changes
.TP
\fB\-\-force\-shared\fR
force wipe even if the project shares an object
directory
force wipe even if the project shares an object directory
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+3 -5
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "November 2025" "repo" "Repo Manual"
.TH REPO "1" "June 2026" "repo" "Repo Manual"
.SH NAME
repo \- repository management tool built on top of git
.SH SYNOPSIS
@@ -26,8 +26,7 @@ control color usage: auto, always, never
trace git command execution (REPO_TRACE=1)
.TP
\fB\-\-trace\-to\-stderr\fR
trace outputs go to stderr in addition to
\&.repo/TRACE_FILE
trace outputs go to stderr in addition to .repo/TRACE_FILE
.TP
\fB\-\-trace\-python\fR
trace python command execution
@@ -39,8 +38,7 @@ time repo command execution
display this version of repo
.TP
\fB\-\-show\-toplevel\fR
display the path of the top\-level directory of the
repo client checkout
display the path of the top\-level directory of the repo client checkout
.TP
\fB\-\-event\-log\fR=\fI\,EVENT_LOG\/\fR
filename of event log to append timeline to
+28 -8
View File
@@ -651,6 +651,8 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if self._manifest_server:
e = doc.createElement("manifest-server")
e.setAttribute("url", self._manifest_server)
if self._manifest_server_helper:
e.setAttribute("helper", self._manifest_server_helper)
root.appendChild(e)
root.appendChild(doc.createTextNode(""))
@@ -999,6 +1001,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
self._Load()
return self._manifest_server
@property
def manifest_server_helper(self):
self._Load()
return self._manifest_server_helper
@property
def CloneBundle(self):
clone_bundle = self.manifestProject.clone_bundle
@@ -1055,6 +1062,10 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
def UseGitWorktrees(self):
return self.manifestProject.use_worktree
@property
def UseLocalGitDirs(self):
return self.manifestProject.use_local_gitdirs
@property
def IsArchive(self):
return self.manifestProject.archive
@@ -1153,6 +1164,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
self._notice = None
self.branch = None
self._manifest_server = None
self._manifest_server_helper = None
def Load(self):
"""Read the manifest into memory."""
@@ -1422,11 +1434,13 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
for node in itertools.chain(*node_list):
if node.nodeName == "manifest-server":
url = self._reqatt(node, "url")
helper = node.getAttribute("helper") or None
if self._manifest_server is not None:
raise ManifestParseError(
"duplicate manifest-server in %s" % (self.manifestFile)
)
self._manifest_server = url
self._manifest_server_helper = helper
def recursively_add_projects(project):
projects = self._projects.setdefault(project.name, [])
@@ -2042,15 +2056,21 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
else:
namepath = f"{name}.git"
worktree = os.path.join(self.topdir, path).replace("\\", "/")
gitdir = os.path.join(self.subdir, "projects", "%s.git" % path)
# We allow people to mix git worktrees & non-git worktrees for now.
# This allows for in situ migration of repo clients.
if os.path.exists(gitdir) or not self.UseGitWorktrees:
objdir = os.path.join(self.repodir, "project-objects", namepath)
else:
use_git_worktrees = True
gitdir = os.path.join(self.repodir, "worktrees", namepath)
if self.UseLocalGitDirs:
gitdir = os.path.join(worktree, ".git")
objdir = gitdir
else:
gitdir = os.path.join(self.subdir, "projects", "%s.git" % path)
# We allow people to mix git worktrees & non-git worktrees for
# now. This allows for in situ migration of repo clients.
if os.path.exists(gitdir) or not self.UseGitWorktrees:
objdir = os.path.join(
self.repodir, "project-objects", namepath
)
else:
use_git_worktrees = True
gitdir = os.path.join(self.repodir, "worktrees", namepath)
objdir = gitdir
return relpath, worktree, gitdir, objdir, use_git_worktrees
def GetProjectsWithName(self, name, all_manifests=False):
+37
View File
@@ -222,6 +222,43 @@ def rmdir(path):
os.rmdir(_makelongpath(path))
def removedirs(path: str) -> None:
"""Remove a directory tree of symlinks and empty directories.
Walks |path| bottom-up without following symlinks. Removes symlinks
and empty directories. Stops at (and preserves) regular files and
non-empty directories. Silently succeeds if |path| does not exist.
Availability: Unix, Windows.
"""
if islink(path):
remove(path)
return
if not isdir(path):
return
for dirpath, dirnames, filenames in walk(path, topdown=False):
for name in dirnames:
entry = os.path.join(dirpath, name)
if islink(entry):
remove(entry)
else:
try:
rmdir(entry)
except OSError:
pass
for name in filenames:
entry = os.path.join(dirpath, name)
if islink(entry):
remove(entry)
try:
rmdir(path)
except OSError:
pass
def isdir(path):
"""os.path.isdir(path) wrapper with support for long paths on Windows.
+404 -74
View File
@@ -28,7 +28,7 @@ import sys
import tarfile
import tempfile
import time
from typing import List, NamedTuple, Optional
from typing import Dict, List, NamedTuple, Optional
import urllib.parse
from color import Coloring
@@ -453,9 +453,13 @@ class _LinkFile(NamedTuple):
platform_utils.readlink(absDest) != relSrc
):
try:
# Remove existing file first, since it might be read-only.
# Remove existing path first, since it might be read-only.
if os.path.lexists(absDest):
platform_utils.remove(absDest)
# removedirs handles symlinks, empty dirs, and nested
# trees of stale linkfile dests without deleting user
# content. Falls through to symlink() if the path was
# fully removed, or raises OSError if not.
platform_utils.removedirs(absDest)
else:
dest_dir = os.path.dirname(absDest)
if not platform_utils.isdir(dest_dir):
@@ -647,6 +651,51 @@ class Project:
return self.relpath
return os.path.join(self.manifest.path_prefix, self.relpath)
def GetEnvVars(self, local: bool = True) -> Dict[str, str]:
"""Get project-context environment variables.
Args:
local: If True, REPO_PATH is relative to the local (sub)manifest.
If False, it is relative to the outermost manifest.
Returns:
A dictionary mapping environment variable names to their values.
Environment Variables:
See the Environment section in `repo help forall` or
`subcmds/forall.py` for details on the available variables.
Note that `forall.py` also documents some extra variables that are
specific to how the `repo forall` command iterates over projects
(e.g., `REPO_COUNT` and `REPO_I`).
"""
env = {}
def setenv(name, val):
if val is None:
val = ""
env[name] = val
setenv("REPO_PROJECT", self.name)
setenv("REPO_OUTERPATH", self.manifest.path_prefix)
setenv("REPO_INNERPATH", self.relpath)
setenv("REPO_PATH", self.RelPath(local=local))
setenv("REPO_REMOTE", self.remote.name)
setenv("REPO_PROJECT_FETCH_URL", self.remote.url)
try:
lrev = "" if self.manifest.IsMirror else self.GetRevisionId()
except ManifestInvalidRevisionError:
lrev = ""
setenv("REPO_LREV", lrev)
setenv("REPO_RREV", self.revisionExpr)
setenv("REPO_UPSTREAM", self.upstream)
setenv("REPO_DEST_BRANCH", self.dest_branch)
for annotation in self.annotations:
setenv(f"REPO__{annotation.name}", annotation.value)
return env
def SetRevision(self, revisionExpr, revisionId=None):
"""Set revisionId based on revision expression and id"""
self.revisionExpr = revisionExpr
@@ -947,11 +996,32 @@ class Project:
out.nl()
return "DIRTY"
branch = self.CurrentBranch
if branch is None:
branch_name = self.CurrentBranch
if branch_name is None:
out.nobranch("(*** NO BRANCH ***)")
else:
out.branch("branch %s", branch)
branch_obj = self.GetBranch(branch_name)
ahead_behind = ""
try:
local_merge = branch_obj.LocalMerge
if local_merge:
left_right = self.work_git.rev_list(
"--left-right",
"--count",
f"{local_merge}...{R_HEADS}{branch_name}",
)
left, right = left_right[0].split()
behind = int(left)
ahead = int(right)
if ahead and behind:
ahead_behind = f" [ahead {ahead}, behind {behind}]"
elif ahead:
ahead_behind = f" [ahead {ahead}]"
elif behind:
ahead_behind = f" [behind {behind}]"
except GitError:
pass
out.branch("branch %s%s", branch_name, ahead_behind)
out.nl()
if rb:
@@ -1487,42 +1557,25 @@ class Project:
depth = None
clone_filter = clone_filter_for_depth
# See if we can skip the network fetch entirely.
custom_fetch_configured = (
self.manifest.manifestProject.use_local_gitdirs
and self.manifest.manifestProject.fetch_cmd
and not isinstance(self, MetaProject)
)
remote_fetched = False
if not (
optimized_fetch
and IsId(self.revisionExpr)
and self._CheckForImmutableRevision(
use_superproject=use_superproject
)
and (
not depth
or os.path.exists(os.path.join(self.gitdir, "shallow"))
)
):
if custom_fetch_configured:
# The custom fetch command is executed unconditionally on every sync
# because it is responsible for updating tracking refs and
# FETCH_HEAD.
remote_fetched = True
try:
if not self._RemoteFetch(
initial=is_new,
quiet=quiet,
verbose=verbose,
output_redir=output_redir,
alt_dir=alt_dir,
use_superproject=use_superproject,
current_branch_only=current_branch_only,
tags=tags,
prune=prune,
depth=depth,
submodules=submodules,
force_sync=force_sync,
ssh_proxy=ssh_proxy,
clone_filter=clone_filter,
retry_fetches=retry_fetches,
):
if not self._CustomFetch(verbose=verbose):
return SyncNetworkHalfResult(
remote_fetched,
SyncNetworkHalfError(
f"Unable to remote fetch project {self.name}",
f"Unable to fetch project {self.name} using "
"fetchcmd",
project=self.name,
),
)
@@ -1531,6 +1584,53 @@ class Project:
remote_fetched,
e,
)
else:
# See if we can skip the standard network fetch entirely.
has_shallow = os.path.exists(os.path.join(self.gitdir, "shallow"))
skip_fetch = (
optimized_fetch
and IsId(self.revisionExpr)
and self._CheckForImmutableRevision(
use_superproject=use_superproject
)
and (
has_shallow
or (not depth and not self._SharingProjectHasShallow())
)
)
if not skip_fetch:
remote_fetched = True
try:
if not self._RemoteFetch(
initial=is_new,
quiet=quiet,
verbose=verbose,
output_redir=output_redir,
alt_dir=alt_dir,
use_superproject=use_superproject,
current_branch_only=current_branch_only,
tags=tags,
prune=prune,
depth=depth,
submodules=submodules,
force_sync=force_sync,
ssh_proxy=ssh_proxy,
clone_filter=clone_filter,
retry_fetches=retry_fetches,
):
return SyncNetworkHalfResult(
remote_fetched,
SyncNetworkHalfError(
f"Unable to remote fetch project {self.name}",
project=self.name,
),
)
except RepoError as e:
return SyncNetworkHalfResult(
remote_fetched,
e,
)
mp = self.manifest.manifestProject
dissociate = mp.dissociate
@@ -1571,6 +1671,8 @@ class Project:
self._InitHooks()
def _CopyAndLinkFiles(self):
if not self.worktree or not platform_utils.isdir(self.worktree):
return
for copyfile in self.copyfiles:
copyfile._Copy()
for linkfile in self.linkfiles:
@@ -1595,6 +1697,18 @@ class Project:
f"revision {self.revisionExpr} in {self.name} not found"
)
def GetHeadRevisionId(self) -> Optional[str]:
"""Get the commit revision of the checked out HEAD.
Returns None if worktree is not checked out or HEAD cannot be resolved.
"""
if self.work_git:
try:
return self.work_git.rev_parse("HEAD")
except GitError:
pass
return None
def GetRevisionId(self, all_refs=None):
if self.revisionId:
return self.revisionId
@@ -1767,7 +1881,7 @@ class Project:
self, "leaving %s; does not track upstream", branch.name
)
try:
self._Checkout(revid, quiet=True)
self._Checkout(revid, force_checkout=force_checkout, quiet=True)
if submodules:
self._SyncSubmodules(quiet=True)
except GitError as e:
@@ -1949,28 +2063,51 @@ class Project:
Args:
verbose: Whether to show verbose messages.
force: Always delete tree even if dirty.
force: Always delete tree even if dirty or corrupted.
Returns:
True if the worktree was completely cleaned out.
"""
if self.IsDirty():
is_dirty = False
is_corrupted = False
try:
is_dirty = self.IsDirty()
except GitError:
is_corrupted = True
rel_path = self.RelPath(local=False)
if is_dirty or is_corrupted:
if force:
logger.warning(
"warning: %s: Removing dirty project: uncommitted changes "
"lost.",
self.RelPath(local=False),
)
if is_corrupted:
logger.warning(
"warning: %s: Removing corrupted project.",
rel_path,
)
else:
logger.warning(
"warning: %s: Removing dirty project: "
"uncommitted changes lost.",
rel_path,
)
else:
msg = (
"error: %s: Cannot remove project: uncommitted "
"changes are present.\n" % self.RelPath(local=False)
)
logger.error(msg)
raise DeleteDirtyWorktreeError(msg, project=self.name)
if is_corrupted:
msg = (
f"error: {rel_path}: Cannot remove project: "
"project is corrupted.\n"
)
logger.error(msg)
raise DeleteWorktreeError(msg, project=self.name)
else:
msg = (
f"error: {rel_path}: Cannot remove project: "
"uncommitted changes are present.\n"
)
logger.error(msg)
raise DeleteDirtyWorktreeError(msg, project=self.name)
if verbose:
print(f"{self.RelPath(local=False)}: Deleting obsolete checkout.")
print(f"{rel_path}: Deleting obsolete checkout.")
# Unlock and delink from the main worktree. We don't use git's worktree
# remove because it will recursively delete projects -- we handle that
@@ -1978,23 +2115,33 @@ class Project:
if self.use_git_worktrees:
needle = os.path.realpath(self.gitdir)
# Find the git worktree commondir under .repo/worktrees/.
output = self.bare_git.worktree("list", "--porcelain").splitlines()[
0
]
assert output.startswith("worktree "), output
commondir = output[9:]
# Walk each of the git worktrees to see where they point.
configs = os.path.join(commondir, "worktrees")
for name in os.listdir(configs):
gitdir = os.path.join(configs, name, "gitdir")
with open(gitdir) as fp:
relpath = fp.read().strip()
# Resolve the checkout path and see if it matches this project.
fullpath = os.path.realpath(
os.path.join(configs, name, relpath)
try:
output = self.bare_git.worktree(
"list", "--porcelain"
).splitlines()[0]
assert output.startswith("worktree "), output
commondir = output[9:]
# Walk each of the git worktrees to see where they point.
configs = os.path.join(commondir, "worktrees")
if os.path.exists(configs):
for name in os.listdir(configs):
gitdir = os.path.join(configs, name, "gitdir")
with open(gitdir) as fp:
relpath = fp.read().strip()
# Resolve the checkout path and see if it
# matches this project.
fullpath = os.path.realpath(
os.path.join(configs, name, relpath)
)
if fullpath == needle:
platform_utils.rmtree(os.path.join(configs, name))
except GitError as e:
logger.warning(
"warning: %s: Failed to list worktrees, skipping worktree "
"cleanup: %s",
rel_path,
e,
)
if fullpath == needle:
platform_utils.rmtree(os.path.join(configs, name))
# Delete the .git directory first, so we're less likely to have a
# partially working git repository around. There shouldn't be any git
@@ -2015,7 +2162,7 @@ class Project:
logger.error(
"error: %s: Failed to delete obsolete checkout; remove "
"manually, then run `repo sync -l`.",
self.RelPath(local=False),
rel_path,
)
raise DeleteWorktreeError(aggregate_errors=[e])
@@ -2079,7 +2226,7 @@ class Project:
logger.error(
"%s: Failed to delete obsolete checkout.\n",
" Remove manually, then run `repo sync -l`.",
self.RelPath(local=False),
rel_path,
)
raise DeleteWorktreeError(aggregate_errors=errors)
@@ -2444,7 +2591,7 @@ class Project:
result.extend(project.GetDerivedSubprojects())
continue
if url.startswith(".."):
if url.startswith(("./", "../")):
url = urllib.parse.urljoin("%s/" % self.remote.url, url)
remote = RemoteSpec(
self.remote.name,
@@ -2559,6 +2706,23 @@ class Project:
# There is no such persistent revision. We have to fetch it.
return False
def _SharingProjectHasShallow(self) -> bool:
"""Check if another project sharing this objdir has a "shallow" file.
If any project sharing this objdir has a shallow file in its gitdir,
then the shared objdir may be depth-limited, and every other project
sharing this objdir needs its own shallow file so that git knows
where history is truncated.
"""
other_projects = self.manifest.GetProjectsWithName(
self.name, all_manifests=True
)
for proj in other_projects:
if proj.objdir == self.objdir and proj.gitdir != self.gitdir:
if os.path.exists(os.path.join(proj.gitdir, "shallow")):
return True
return False
def _FetchArchive(self, tarpath, cwd=None):
cmd = ["archive", "-v", "-o", tarpath]
cmd.append("--remote=%s" % self.remote.url)
@@ -2575,6 +2739,114 @@ class Project:
)
command.Wait()
def _CustomFetch(self, verbose=False) -> bool:
"""Fetch using a custom command specified by repo.fetchcmd.
Populates the subshell with project-context environment variables,
including REPO_TREV (target revision resolved to a commit hash).
Expects the target commit to be reachable and tracking refs/FETCH_HEAD
to be updated upon a successful 0 exit code.
For a detailed contract, environment variables, and postconditions,
see docs/fetch-cmd.md.
"""
# Resolve REPO_TREV (target revision resolved to a full commit hash).
repo_trev = None
if self.revisionId and IsId(self.revisionId):
repo_trev = self.revisionId
if not repo_trev:
output = self._LsRemote(self.upstream or self.revisionExpr)
if output:
lines = output.splitlines()
if lines:
parts = lines[0].split()
if parts:
repo_trev = parts[0]
if not repo_trev:
logger.error("error: Cannot resolve REPO_TREV for %s", self.name)
return False
env = os.environ.copy()
env.update(self.GetEnvVars())
env["REPO_TREV"] = repo_trev
cmd_str = self.manifest.manifestProject.fetch_cmd
if verbose:
print(f"Running fetchcmd: {cmd_str} for {self.name}")
stdout = None if verbose else subprocess.PIPE
stderr = None if verbose else subprocess.PIPE
try:
p = subprocess.run(
cmd_str,
shell=True,
cwd=self.manifest.topdir,
env=env,
stdout=stdout,
stderr=stderr,
text=True,
)
if p.returncode != 0:
logger.error(
"error: fetchcmd failed with exit code %d", p.returncode
)
if not verbose:
logger.error("stderr: %s", p.stderr)
return False
except Exception as e:
logger.error("error: Failed to execute fetchcmd: %s", e)
return False
# Verify postconditions.
try:
self.bare_git.cat_file("-e", repo_trev)
except GitError:
logger.error(
"error: Postcondition failed: %s not found in object store",
repo_trev,
)
return False
try:
ref_name = self.GetRemote().ToLocal(self.revisionExpr)
except GitError as e:
logger.error("error: Failed to resolve tracking ref: %s", e)
return False
try:
resolved_ref = self.bare_git.rev_parse(ref_name)
if resolved_ref != repo_trev:
logger.error(
"error: Postcondition failed: %s is %s, expected %s",
ref_name,
resolved_ref,
repo_trev,
)
return False
except GitError:
logger.error("error: Postcondition failed: %s not found", ref_name)
return False
try:
fetch_head = self.bare_git.rev_parse("FETCH_HEAD")
if fetch_head != repo_trev:
logger.error(
"error: Postcondition failed: FETCH_HEAD is %s, "
"expected %s",
fetch_head,
repo_trev,
)
return False
except GitError:
logger.error("error: Postcondition failed: FETCH_HEAD not found")
return False
return True
def _RemoteFetch(
self,
name=None,
@@ -2607,7 +2879,7 @@ class Project:
if depth:
current_branch_only = True
is_sha1 = bool(IsId(self.revisionExpr))
is_sha1 = IsId(self.revisionExpr)
if current_branch_only:
if self.revisionExpr.startswith(R_TAGS):
@@ -2618,11 +2890,14 @@ class Project:
tag_name = self.upstream[len(R_TAGS) :]
if is_sha1 or tag_name is not None:
has_shallow = os.path.exists(
os.path.join(self.gitdir, "shallow")
)
if self._CheckForImmutableRevision(
use_superproject=use_superproject
) and (
not depth
or os.path.exists(os.path.join(self.gitdir, "shallow"))
has_shallow
or (not depth and not self._SharingProjectHasShallow())
):
if verbose:
print(
@@ -3682,7 +3957,11 @@ class Project:
self._MigrateOldSubmoduleDir()
# If using an old layout style (a directory), migrate it.
if not platform_utils.islink(dotgit) and platform_utils.isdir(dotgit):
if (
not platform_utils.islink(dotgit)
and platform_utils.isdir(dotgit)
and not self.manifest.UseLocalGitDirs
):
self._MigrateOldWorkTreeGitDir(dotgit, project=self.name)
init_dotgit = not os.path.lexists(dotgit)
@@ -3731,6 +4010,11 @@ class Project:
For submodule projects, create a '.git' file using the gitfile
mechanism, and for the rest, create a symbolic link.
"""
if self.manifest.UseLocalGitDirs and os.path.normpath(
self.gitdir
) == os.path.normpath(dotgit):
return
os.makedirs(self.worktree, exist_ok=True)
if self.parent:
_lwrite(
@@ -4484,6 +4768,16 @@ class ManifestProject(MetaProject):
"""Whether we use worktree."""
return self.config.GetBoolean("repo.worktree")
@property
def use_local_gitdirs(self):
"""Whether we use local gitdirs."""
return self.config.GetBoolean("repo.uselocalgitdirs")
@property
def fetch_cmd(self):
"""The fetch command to use."""
return self.config.GetString("repo.fetchcmd")
@property
def clone_bundle(self):
"""Whether we use clone_bundle."""
@@ -4638,6 +4932,7 @@ class ManifestProject(MetaProject):
this_manifest_only=False,
outer_manifest=True,
clone_filter_for_depth=None,
use_local_gitdirs=False,
):
"""Sync the manifest and all submanifests.
@@ -4868,6 +5163,41 @@ class ManifestProject(MetaProject):
self.use_git_worktrees = True
logger.warning("warning: --worktree is experimental!")
if use_local_gitdirs:
if mirror:
logger.error(
"fatal: --mirror and --use-local-gitdirs are incompatible"
)
return False
if worktree:
logger.error(
"fatal: --worktree and --use-local-gitdirs are incompatible"
)
return False
if archive:
logger.error(
"fatal: --archive and --use-local-gitdirs are incompatible"
)
return False
if not is_new and not self.use_local_gitdirs:
logger.error(
"fatal: --use-local-gitdirs is only supported when "
"initializing a new workspace."
)
return False
self.config.SetBoolean("repo.uselocalgitdirs", use_local_gitdirs)
if self.fetch_cmd and not (use_local_gitdirs or self.use_local_gitdirs):
logger.error(
"fatal: repo.fetchcmd is set but repo.uselocalgitdirs is "
"not enabled"
)
return False
if archive:
if is_new:
self.config.SetBoolean("repo.archive", archive)
+7 -7
View File
@@ -85,18 +85,18 @@ Repo launcher bucket:
gs://git-repo-downloads/
You should first upload it with a specific version:
gsutil cp -a public-read {opts.launcher} gs://git-repo-downloads/repo-{version}
gsutil cp -a public-read {opts.launcher}.asc gs://git-repo-downloads/repo-{version}.asc
gcloud storage cp --predefined-acl=publicRead {opts.launcher} gs://git-repo-downloads/repo-{version}
gcloud storage cp --predefined-acl=publicRead {opts.launcher}.asc gs://git-repo-downloads/repo-{version}.asc
Then to make it the public default:
gsutil cp -a public-read gs://git-repo-downloads/repo-{version} gs://git-repo-downloads/repo
gsutil cp -a public-read gs://git-repo-downloads/repo-{version}.asc gs://git-repo-downloads/repo.asc
gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo-{version} gs://git-repo-downloads/repo
gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo-{version}.asc gs://git-repo-downloads/repo.asc
NB: If a rollback is necessary, the GS bucket archives old versions, and may be
accessed by specifying their unique id number.
gsutil ls -la gs://git-repo-downloads/repo gs://git-repo-downloads/repo.asc
gsutil cp -a public-read gs://git-repo-downloads/repo#<unique id> gs://git-repo-downloads/repo
gsutil cp -a public-read gs://git-repo-downloads/repo.asc#<unique id> gs://git-repo-downloads/repo.asc
gcloud storage ls --long --all-versions gs://git-repo-downloads/repo gs://git-repo-downloads/repo.asc
gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo#<unique id> gs://git-repo-downloads/repo
gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo.asc#<unique id> gs://git-repo-downloads/repo.asc
""" # noqa: E501
)
+9 -6
View File
@@ -30,8 +30,7 @@ import tempfile
from typing import List
# NB: This script is currently imported by tests/ to unittest some logic.
assert sys.version_info >= (3, 6), "Python 3.6+ required"
assert sys.version_info >= (3, 9), "Release framework requires Python 3.9+"
THIS_FILE = Path(__file__).resolve()
@@ -66,13 +65,18 @@ def main(argv: List[str]) -> int:
parser = get_parser()
opts = parser.parse_args(argv)
if not shutil.which("help2man"):
help2man = ["help2man"]
cipd_help2man = TOPDIR / ".cipd_bin/bin/help2man"
if cipd_help2man.exists():
help2man = [cipd_help2man]
elif not shutil.which("help2man"):
sys.exit("Please install help2man to continue.")
# Let repo know we're generating man pages so it can avoid some dynamic
# behavior (like probing active number of CPUs). We use a weird name &
# value to make it less likely for users to set this var themselves.
os.environ["_REPO_GENERATE_MANPAGES_"] = " indeed! "
os.environ["COLUMNS"] = "10000"
# "repo branch" is an alias for "repo branches".
del subcmds.all_commands["branch"]
@@ -81,7 +85,7 @@ def main(argv: List[str]) -> int:
version = RepoSourceVersion()
cmdlist = [
[
"help2man",
*help2man,
"-N",
"-n",
f"repo {cmd} - manual page for repo {cmd}",
@@ -100,7 +104,7 @@ def main(argv: List[str]) -> int:
]
cmdlist.append(
[
"help2man",
*help2man,
"-N",
"-n",
"repository management tool built on top of git",
@@ -178,7 +182,6 @@ def replace_regex(data):
"""
regex = (
(r"(It was generated by help2man) [0-9.]+", r"\g<1>."),
(r"^\033\[[0-9;]*m([^\033]*)\033\[m", r"\g<1>"),
(r"^\.IP\n(.*:)\n", r".SS \g<1>\n"),
(r"^\.PP\nDescription", r".SH DETAILS"),
)
+32 -5
View File
@@ -129,7 +129,7 @@ if not REPO_REV:
BUG_URL = "https://issues.gerritcodereview.com/issues/new?component=1370071"
# increment this whenever we make important changes to this script
VERSION = (2, 54)
VERSION = (2, 65)
# increment this if the MAINTAINER_KEYS block is modified
KEYRING_VERSION = (2, 3)
@@ -379,6 +379,11 @@ def InitParser(parser):
action="store_true",
help="use git-worktree to manage projects",
)
group.add_option(
"--use-local-gitdirs",
action="store_true",
help="bypass .repo/projects/ and use standard Git layout in working tree",
)
# These are fundamentally different ways of structuring the checkout.
group = parser.add_option_group("Project checkout optimizations")
@@ -400,20 +405,20 @@ def InitParser(parser):
"--partial-clone",
action="store_true",
help="perform partial clone (https://git-scm.com/"
"docs/gitrepository-layout#_code_partialclone_code)",
"docs/partial-clone)",
)
group.add_option(
"--no-partial-clone",
action="store_false",
help="disable use of partial clone (https://git-scm.com/"
"docs/gitrepository-layout#_code_partialclone_code)",
"docs/partial-clone)",
)
group.add_option(
"--partial-clone-exclude",
action="store",
help="exclude the specified projects (a comma-delimited "
"project names) from partial clone (https://git-scm.com"
"/docs/gitrepository-layout#_code_partialclone_code)",
"/docs/partial-clone)",
)
group.add_option(
"--clone-filter",
@@ -742,6 +747,28 @@ def SetGitTrace2ParentSid(env=None):
_setenv(KEY, value, env=env)
def _NormalizeGpgHomePath(path: str) -> str:
"""Convert path for GPG on Windows if running in MSYS/MinGW.
If GPG is MSYS-based (common in Git Bash/Cygwin), it expects POSIX-like paths
(e.g. /c/Users/... instead of C:\\Users\\...). Native Windows Python uses
Windows-style paths, which MSYS GPG will incorrectly treat as relative paths.
We use cygpath to query the active MSYS/Cygwin mount configuration and resolve
the correct POSIX path.
"""
if platform.system() == "Windows":
try:
out = subprocess.check_output(
["cygpath", "-u", path],
stderr=subprocess.DEVNULL,
universal_newlines=True,
)
return out.strip()
except (OSError, subprocess.CalledProcessError):
pass
return path
def _setenv(key, value, env=None):
"""Set |key| in the OS environment |env| to |value|."""
if env is None:
@@ -1074,7 +1101,7 @@ def verify_rev(cwd, remote_ref, rev, quiet):
print(file=sys.stderr)
env = os.environ.copy()
_setenv("GNUPGHOME", gpg_dir, env)
_setenv("GNUPGHOME", _NormalizeGpgHomePath(gpg_dir), env)
run_git("tag", "-v", cur, cwd=cwd, env=env)
return "%s^0" % cur
+16 -5
View File
@@ -157,11 +157,6 @@ def run_check_metadata():
def run_update_manpages() -> int:
"""Returns the exit code from release/update-manpages."""
# Allow this to fail on CI, but not local devs.
if is_ci() and not shutil.which("help2man"):
print("update-manpages: help2man not found; skipping test")
return 0
argv = ["--check"]
log_cmd("release/update-manpages", argv)
return subprocess.run(
@@ -171,8 +166,24 @@ def run_update_manpages() -> int:
).returncode
def cipd_bootstrap() -> None:
"""Install packages via cipd if available."""
argv = ["ensure", "-root", ".cipd_bin", "-ensure-file", "cipd_manifest.txt"]
log_cmd("cipd", argv)
try:
subprocess.run(
["cipd"] + argv,
check=True,
cwd=ROOT_DIR,
)
except FileNotFoundError:
# Skip if the user doesn't have cipd from depot_tools.
return
def main(argv):
"""The main entry."""
cipd_bootstrap()
checks = (
functools.partial(run_pytest, argv),
functools.partial(run_pytest_py38, argv),
+1
View File
@@ -94,6 +94,7 @@ It is equivalent to "git branch -D <branchname>".
def Execute(self, opt, args):
nb = args[0].split()
self.TryOverrideManifestWithSmartSync()
err = collections.defaultdict(list)
success = collections.defaultdict(list)
aggregate_errors = []
+11 -27
View File
@@ -25,7 +25,6 @@ from color import Coloring
from command import Command
from command import DEFAULT_LOCAL_JOBS
from command import MirrorSafeCommand
from error import ManifestInvalidRevisionError
from repo_logging import RepoLogger
@@ -102,6 +101,14 @@ revision to a locally executed git command, use REPO_LREV.
REPO_RREV is the name of the revision from the manifest, exactly
as written in the manifest.
REPO_UPSTREAM is the name of the upstream branch as specified in the
manifest.
REPO_DEST_BRANCH is the name of the destination branch for code review,
as specified in the manifest.
REPO_PROJECT_FETCH_URL is the full resolved fetch URL for the project.
REPO_COUNT is the total number of projects being iterated.
REPO_I is the current (1-based) iteration count. Can be used in
@@ -236,13 +243,7 @@ without iterating through the remaining projects.
mirror = self.manifest.IsMirror
smart_sync_manifest_name = "smart_sync_override.xml"
smart_sync_manifest_path = os.path.join(
self.manifest.manifestProject.worktree, smart_sync_manifest_name
)
if os.path.isfile(smart_sync_manifest_path):
self.manifest.Override(smart_sync_manifest_path)
self.TryOverrideManifestWithSmartSync()
if opt.regex:
projects = self.FindProjects(args, all_manifests=all_trees)
@@ -339,25 +340,8 @@ def DoWork(project, mirror, opt, cmd, shell, cnt, config):
val = ""
env[name] = val
setenv("REPO_PROJECT", project.name)
setenv("REPO_OUTERPATH", project.manifest.path_prefix)
setenv("REPO_INNERPATH", project.relpath)
setenv("REPO_PATH", project.RelPath(local=opt.this_manifest_only))
setenv("REPO_REMOTE", project.remote.name)
try:
# If we aren't in a fully synced state and we don't have the ref the
# manifest wants, then this will fail. Ignore it for the purposes of
# this code.
lrev = "" if mirror else project.GetRevisionId()
except ManifestInvalidRevisionError:
lrev = ""
setenv("REPO_LREV", lrev)
setenv("REPO_RREV", project.revisionExpr)
setenv("REPO_UPSTREAM", project.upstream)
setenv("REPO_DEST_BRANCH", project.dest_branch)
setenv("REPO_I", str(cnt + 1))
for annotation in project.annotations:
setenv("REPO__%s" % (annotation.name), annotation.value)
env.update(project.GetEnvVars(local=opt.this_manifest_only))
env["REPO_I"] = str(cnt + 1)
if mirror:
setenv("GIT_DIR", project.gitdir)
+345 -124
View File
@@ -12,14 +12,41 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import enum
import functools
import io
import json
import optparse
import sys
from typing import Any, Dict, List, NamedTuple
from color import Coloring
from command import DEFAULT_LOCAL_JOBS
from command import PagedCommand
from git_refs import R_HEADS
from git_refs import R_M
class BranchInfo(NamedTuple):
"""Holds information about a branch in a project."""
relpath: str
name: str
commits: Any
date: str
is_current: bool
class OutputFormat(enum.Enum):
"""Type for the requested output format."""
# Human-readable text output.
TEXT = enum.auto()
# Machine-readable JSON output.
JSON = enum.auto()
class _Coloring(Coloring):
def __init__(self, config):
Coloring.__init__(self, config, "status")
@@ -27,10 +54,11 @@ class _Coloring(Coloring):
class Info(PagedCommand):
COMMON = True
PARALLEL_JOBS = DEFAULT_LOCAL_JOBS
helpSummary = (
"Get info on the manifest branch, current branch or unmerged branches"
)
helpUsage = "%prog [-dl] [-o [-c]] [<project>...]"
helpUsage = "%prog [-dl] [-o [-c]] [--format=<format>] [<project>...]"
def _Options(self, p):
p.add_option(
@@ -46,6 +74,30 @@ class Info(PagedCommand):
action="store_true",
help="show overview of all local commits",
)
p.add_option(
"--include-summary",
action="store_true",
default=True,
help="include manifest summary (default: true)",
)
p.add_option(
"--no-include-summary",
dest="include_summary",
action="store_false",
help="exclude manifest summary",
)
p.add_option(
"--include-projects",
action="store_true",
default=True,
help="include project details (default: true)",
)
p.add_option(
"--no-include-projects",
dest="include_projects",
action="store_false",
help="exclude project details",
)
p.add_option(
"-c",
"--current-branch",
@@ -72,8 +124,39 @@ class Info(PagedCommand):
action="store_true",
help="disable all remote operations",
)
formats = tuple(x.lower() for x in OutputFormat.__members__.keys())
p.add_option(
"--format",
default=OutputFormat.TEXT.name.lower(),
choices=formats,
help=f"output format: {', '.join(formats)} (default: %default)",
)
def WantPager(self, opt):
return OutputFormat[opt.format.upper()] == OutputFormat.TEXT
def ValidateOptions(self, opt, args):
output_format = OutputFormat[opt.format.upper()]
if output_format == OutputFormat.JSON:
if opt.all:
self.OptionParser.error("--diff is not supported with JSON")
if opt.overview:
self.OptionParser.error("--overview is not supported with JSON")
def Execute(self, opt, args):
if not opt.this_manifest_only:
self.manifest = self.manifest.outer_client
self.TryOverrideManifestWithSmartSync()
output_format = OutputFormat[opt.format.upper()]
if output_format == OutputFormat.JSON:
self._ExecuteJson(opt, args)
else:
self._ExecuteText(opt, args)
def _ExecuteText(self, opt, args) -> None:
"""Output info as human-readable text."""
self.out = _Coloring(self.client.globalConfig)
self.heading = self.out.printer("heading", attr="bold")
self.headtext = self.out.nofmt_printer("headtext", fg="yellow")
@@ -84,157 +167,295 @@ class Info(PagedCommand):
self.opt = opt
if not opt.this_manifest_only:
self.manifest = self.manifest.outer_client
manifestConfig = self.manifest.manifestProject.config
mergeBranch = manifestConfig.GetBranch("default").merge
manifestGroups = self.manifest.GetManifestGroupsStr()
if opt.include_summary:
self._printSummary()
self.heading("Manifest branch: ")
if self.manifest.default.revisionExpr:
self.headtext(self.manifest.default.revisionExpr)
self.out.nl()
self.heading("Manifest merge branch: ")
# The manifest might not have a merge branch if it isn't in a git repo,
# e.g. if `repo init --standalone-manifest` is used.
self.headtext(mergeBranch or "")
self.out.nl()
self.heading("Manifest groups: ")
self.headtext(manifestGroups)
self.out.nl()
sp = self.manifest.superproject
srev = sp.commit_id if sp and sp.commit_id else "None"
self.heading("Superproject revision: ")
self.headtext(srev)
self.out.nl()
self.printSeparator()
if not opt.overview:
if not opt.include_projects:
return
elif not opt.overview:
self._printDiffInfo(opt, args)
else:
self._printCommitOverview(opt, args)
def _getSummaryData(self) -> Dict[str, Any]:
"""Gather manifest summary data as a dict."""
manifestConfig = self.manifest.manifestProject.config
mergeBranch = manifestConfig.GetBranch("default").merge
manifestGroups = self.manifest.GetManifestGroupsStr()
sp = self.manifest.superproject
srev = sp.commit_id if sp and sp.commit_id else None
return {
"manifest_branch": self.manifest.default.revisionExpr or "",
"manifest_merge_branch": mergeBranch or "",
"manifest_groups": manifestGroups,
"superproject_revision": srev,
}
@classmethod
def _getProjectData(cls, project) -> Dict[str, Any]:
"""Gather project data as a dict."""
data = {
"name": project.name,
"mount_path": project.worktree,
"current_revision": project.GetHeadRevisionId()
or project.GetRevisionId(),
"manifest_revision": project.revisionExpr,
"local_branches": list(project.GetBranches()),
}
currentBranch = project.CurrentBranch
if currentBranch:
data["current_branch"] = currentBranch
return data
@classmethod
def _ProjectDataHelper(cls, project_idx: int) -> Dict[str, Any]:
"""Helper to get project data in parallel."""
project = cls.get_parallel_context()["projects"][project_idx]
return cls._getProjectData(project)
def _ExecuteJson(self, opt, args) -> None:
"""Output info as JSON."""
result = {}
if opt.include_summary:
result["summary"] = self._getSummaryData()
if opt.include_projects:
projs = self.GetProjects(
args, all_manifests=not opt.this_manifest_only
)
project_data = []
def _ProcessResults(_pool, _output, results):
project_data.extend(results)
with self.ParallelContext():
self.get_parallel_context()["projects"] = projs
self.ExecuteInParallel(
opt.jobs,
self._ProjectDataHelper,
range(len(projs)),
callback=_ProcessResults,
ordered=True,
chunksize=1,
)
result["projects"] = project_data
json_settings = {
# JSON style guide says Unicode characters are fully allowed.
"ensure_ascii": False,
# We use 2 space indent to match JSON style guide.
"indent": 2,
"separators": (",", ": "),
"sort_keys": True,
}
sys.stdout.write(json.dumps(result, **json_settings) + "\n")
def _printSummary(self) -> None:
"""Print manifest summary in text format."""
data = self._getSummaryData()
self.heading("Manifest branch: ")
self.headtext(data["manifest_branch"])
self.out.nl()
self.heading("Manifest merge branch: ")
self.headtext(data["manifest_merge_branch"])
self.out.nl()
self.heading("Manifest groups: ")
self.headtext(data["manifest_groups"])
self.out.nl()
self.heading("Superproject revision: ")
self.headtext(data["superproject_revision"] or "None")
self.out.nl()
self.printSeparator()
def printSeparator(self):
self.text("----------------------------")
self.out.nl()
@classmethod
def _DiffHelper(cls, project_idx: int, opt: Any) -> str:
"""Helper for ParallelContext to get diff info for a project."""
buf = io.StringIO()
project = cls.get_parallel_context()["projects"][project_idx]
config = cls.get_parallel_context()["config"]
out = _Coloring(config)
out.redirect(buf)
heading = out.printer("heading", attr="bold")
headtext = out.nofmt_printer("headtext", fg="yellow")
redtext = out.printer("redtext", fg="red")
sha = out.printer("sha", fg="yellow")
text = out.nofmt_printer("text")
dimtext = out.printer("dimtext", attr="dim")
heading("Project: ")
headtext(project.name)
out.nl()
heading("Mount path: ")
headtext(project.worktree)
out.nl()
heading("Current revision: ")
headtext(project.GetHeadRevisionId() or project.GetRevisionId())
out.nl()
currentBranch = project.CurrentBranch
if currentBranch:
heading("Current branch: ")
headtext(currentBranch)
out.nl()
heading("Manifest revision: ")
headtext(project.revisionExpr)
out.nl()
localBranches = list(project.GetBranches().keys())
heading("Local Branches: ")
redtext(str(len(localBranches)))
if localBranches:
text(" [")
text(", ".join(localBranches))
text("]")
out.nl()
if opt.all:
if not opt.local:
project.Sync_NetworkHalf(quiet=True, current_branch_only=True)
branch = project.manifest.manifestProject.config.GetBranch(
"default"
).merge
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS) :]
logTarget = R_M + branch
bareTmp = project.bare_git._bare
project.bare_git._bare = False
localCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
logTarget + "..",
"--",
)
originCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
".." + logTarget,
"--",
)
project.bare_git._bare = bareTmp
heading("Local Commits: ")
redtext(str(len(localCommits)))
dimtext(" (on current branch)")
out.nl()
for c in localCommits:
split = c.split()
sha(split[0] + " ")
text(" ".join(split[1:]))
out.nl()
text("----------------------------")
out.nl()
heading("Remote Commits: ")
redtext(str(len(originCommits)))
out.nl()
for c in originCommits:
split = c.split()
sha(split[0] + " ")
text(" ".join(split[1:]))
out.nl()
text("----------------------------")
out.nl()
return buf.getvalue()
def _printDiffInfo(self, opt, args):
# We let exceptions bubble up to main as they'll be well structured.
projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only)
for p in projs:
self.heading("Project: ")
self.headtext(p.name)
self.out.nl()
def _ProcessResults(_pool, _output, results):
for output in results:
if output:
print(output, end="")
self.heading("Mount path: ")
self.headtext(p.worktree)
self.out.nl()
with self.ParallelContext():
self.get_parallel_context()["projects"] = projs
self.get_parallel_context()[
"config"
] = self.manifest.manifestProject.config
self.heading("Current revision: ")
self.headtext(p.GetRevisionId())
self.out.nl()
self.ExecuteInParallel(
opt.jobs,
functools.partial(self._DiffHelper, opt=opt),
range(len(projs)),
callback=_ProcessResults,
ordered=True,
chunksize=1,
)
currentBranch = p.CurrentBranch
if currentBranch:
self.heading("Current branch: ")
self.headtext(currentBranch)
self.out.nl()
@classmethod
def _OverviewHelper(cls, project_idx: int, opt: Any) -> List[BranchInfo]:
"""Helper to get overview of uploadable branches."""
project = cls.get_parallel_context()["projects"][project_idx]
self.heading("Manifest revision: ")
self.headtext(p.revisionExpr)
self.out.nl()
branches = []
br = [project.GetUploadableBranch(x) for x in project.GetBranches()]
br = [x for x in br if x]
if opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch]
localBranches = list(p.GetBranches().keys())
self.heading("Local Branches: ")
self.redtext(str(len(localBranches)))
if localBranches:
self.text(" [")
self.text(", ".join(localBranches))
self.text("]")
self.out.nl()
if self.opt.all:
self.findRemoteLocalDiff(p)
self.printSeparator()
def findRemoteLocalDiff(self, project):
# Fetch all the latest commits.
if not self.opt.local:
project.Sync_NetworkHalf(quiet=True, current_branch_only=True)
branch = self.manifest.manifestProject.config.GetBranch("default").merge
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS) :]
logTarget = R_M + branch
bareTmp = project.bare_git._bare
project.bare_git._bare = False
localCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
logTarget + "..",
"--",
)
originCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
".." + logTarget,
"--",
)
project.bare_git._bare = bareTmp
self.heading("Local Commits: ")
self.redtext(str(len(localCommits)))
self.dimtext(" (on current branch)")
self.out.nl()
for c in localCommits:
split = c.split()
self.sha(split[0] + " ")
self.text(" ".join(split[1:]))
self.out.nl()
self.printSeparator()
self.heading("Remote Commits: ")
self.redtext(str(len(originCommits)))
self.out.nl()
for c in originCommits:
split = c.split()
self.sha(split[0] + " ")
self.text(" ".join(split[1:]))
self.out.nl()
for b in br:
branches.append(
BranchInfo(
relpath=project.RelPath(local=opt.this_manifest_only),
name=b.name,
commits=b.commits,
date=b.date,
is_current=b.name == project.CurrentBranch,
)
)
return branches
def _printCommitOverview(self, opt, args):
projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only)
all_branches = []
for project in self.GetProjects(
args, all_manifests=not opt.this_manifest_only
):
br = [project.GetUploadableBranch(x) for x in project.GetBranches()]
br = [x for x in br if x]
if self.opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch]
all_branches.extend(br)
def _ProcessResults(_pool, _output, results):
for branches in results:
all_branches.extend(branches)
with self.ParallelContext():
self.get_parallel_context()["projects"] = projs
self.ExecuteInParallel(
opt.jobs,
functools.partial(self._OverviewHelper, opt=opt),
range(len(projs)),
callback=_ProcessResults,
ordered=True,
chunksize=1,
)
if not all_branches:
return
self.out.nl()
self.heading("Projects Overview")
project = None
current_relpath = None
for branch in all_branches:
if project != branch.project:
project = branch.project
if current_relpath != branch.relpath:
current_relpath = branch.relpath
self.out.nl()
self.headtext(project.RelPath(local=opt.this_manifest_only))
self.headtext(current_relpath)
self.out.nl()
commits = branch.commits
@@ -242,7 +463,7 @@ class Info(PagedCommand):
self.text(
"%s %-33s (%2d commit%s, %s)"
% (
branch.name == project.CurrentBranch and "*" or " ",
branch.is_current and "*" or " ",
branch.name,
len(commits),
len(commits) != 1 and "s" or "",
+1
View File
@@ -169,6 +169,7 @@ to update the working directory files.
depth=opt.depth,
git_event_log=self.git_event_log,
manifest_name=opt.manifest_name,
use_local_gitdirs=opt.use_local_gitdirs,
):
manifest_name = opt.manifest_name
raise UpdateManifestError(
+14 -1
View File
@@ -16,7 +16,9 @@ import sys
from color import Coloring
from command import Command
from error import GitError
from git_command import GitCommand
from project import Project
from repo_logging import RepoLogger
@@ -30,6 +32,17 @@ class RebaseColoring(Coloring):
self.fail = self.printer("fail", fg="red")
def _ResolveOntoManifest(project: Project) -> str:
"""Resolve project's revisionExpr to a local tracking branch.
Falls back to the raw revisionExpr if ToLocal fails or raises GitError.
"""
try:
return project.GetRemote().ToLocal(project.revisionExpr)
except GitError:
return project.revisionExpr
class Rebase(Command):
COMMON = True
helpSummary = "Rebase local branches on upstream branch"
@@ -162,7 +175,7 @@ branch but need to incorporate new upstream changes "underneath" them.
args = common_args[:]
if opt.onto_manifest:
args.append("--onto")
args.append(project.revisionExpr)
args.append(_ResolveOntoManifest(project))
args.append(upbranch.LocalMerge)
+1
View File
@@ -104,6 +104,7 @@ revision specified in the manifest.
def Execute(self, opt, args):
nb = args[0]
self.TryOverrideManifestWithSmartSync()
err_projects = []
err = []
projects = []
+6
View File
@@ -54,6 +54,12 @@ project 'repo' on branch 'devwork':
project repo/ branch devwork
-m subcmds/status.py
If the branch is tracking an upstream branch, the number of commits
ahead and/or behind is also shown:
project repo/ branch devwork [ahead 1, behind 2]
-m subcmds/status.py
The first column explains how the staging area (index) differs from
the last commit (HEAD). Its values are always displayed in upper
case and have the following meanings:
+304 -65
View File
@@ -23,6 +23,8 @@ import netrc
import optparse
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import time
@@ -64,6 +66,7 @@ from error import SyncError
from error import UpdateManifestError
import event_log
from git_command import git_require
from git_command import GitCommand
from git_config import GetUrlCookieFile
from git_refs import HEAD
from git_refs import R_HEADS
@@ -95,19 +98,27 @@ logger = RepoLogger(__file__)
def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
"""Generate a sequence of checkouts that is safe to perform. The client
should checkout everything from n-th index before moving to n+1.
"""Generate a sequence of checkouts that is safe to perform.
The client should checkout everything from n-th index before moving to
n+1.
This is only useful if manifest contains nested projects.
E.g. if foo, foo/bar and foo/bar/baz are project paths, then foo needs to
finish before foo/bar can proceed, and foo/bar needs to finish before
foo/bar/baz."""
res = [[]]
current = res[0]
foo/bar/baz.
# depth_stack contains a current stack of parent paths.
Discovered submodules have an additional constraint: sibling submodules in
the same parent repository must not be checked out in parallel because they
all run `git submodule init` against the same parent .git/config.
"""
res = [[]]
# depth_stack contains the current stack of parent paths together with the
# effective checkout level assigned to each path.
depth_stack = []
submodule_parent_level = {}
# Checkouts are iterated in the hierarchical order. That way, it can easily
# be determined if the previous checkout is parent of the current checkout.
# We are splitting by the path separator so the final result is
@@ -118,21 +129,29 @@ def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
checkout_path = Path(checkout.relpath)
while depth_stack:
try:
checkout_path.relative_to(depth_stack[-1])
checkout_path.relative_to(depth_stack[-1][0])
except ValueError:
# Path.relative_to returns ValueError if paths are not relative.
# TODO(sokcevic): Switch to is_relative_to once min supported
# version is py3.9.
depth_stack.pop()
else:
if len(depth_stack) >= len(res):
# Another depth created.
res.append([])
break
current = res[len(depth_stack)]
level = depth_stack[-1][1] + 1 if depth_stack else 0
parent = checkout.parent
if parent is not None:
level = max(
level,
submodule_parent_level.get(parent.worktree, level - 1) + 1,
)
submodule_parent_level[parent.worktree] = level
if level >= len(res):
res.extend([] for _ in range(level + 1 - len(res)))
current = res[level]
current.append(checkout)
depth_stack.append(checkout_path)
depth_stack.append((checkout_path, level))
return res
@@ -565,6 +584,11 @@ later is required to fix a server side protocol bug.
dest="use_superproject",
help="disable use of manifest superprojects",
)
p.add_option(
"--superproject-revision",
action="store",
help="sync to superproject revision (applies to outer manifest)",
)
p.add_option("--tags", action="store_true", help="fetch tags")
p.add_option(
"--no-tags",
@@ -668,6 +692,24 @@ later is required to fix a server side protocol bug.
or opt.current_branch_only
)
def _ConfigureSuperproject(
self,
opt: optparse.Values,
manifest,
revision: Optional[str] = None,
) -> bool:
"""Configure superproject with options."""
if not manifest.superproject:
return False
manifest.superproject.SetQuiet(not opt.verbose)
print_messages = git_superproject.PrintMessages(
opt.use_superproject, manifest
)
manifest.superproject.SetPrintMessages(print_messages)
if revision:
manifest.superproject.SetRevisionId(revision)
return print_messages
def _UpdateProjectsRevisionId(
self, opt, args, superproject_logging_data, manifest
):
@@ -741,11 +783,7 @@ later is required to fix a server side protocol bug.
if not use_super:
continue
m.superproject.SetQuiet(not opt.verbose)
print_messages = git_superproject.PrintMessages(
opt.use_superproject, m
)
m.superproject.SetPrintMessages(print_messages)
print_messages = self._ConfigureSuperproject(opt, m)
update_result = m.superproject.UpdateProjectsRevisionId(
per_manifest[m.path_prefix], git_event_log=self.git_event_log
)
@@ -841,6 +879,8 @@ later is required to fix a server side protocol bug.
)
except KeyboardInterrupt:
logger.error("Keyboard interrupt while processing %s", project.name)
if not cls.is_multiprocessing_active():
raise
except GitError as e:
logger.error("error.GitError: Cannot fetch %s", e)
errors.append(e)
@@ -1104,6 +1144,8 @@ later is required to fix a server side protocol bug.
errors.extend(syncbuf.errors)
except KeyboardInterrupt:
logger.error("Keyboard interrupt while processing %s", project.name)
if not cls.is_multiprocessing_active():
raise
except GitError as e:
logger.error(
"error.GitError: Cannot checkout %s: %s", project.name, e
@@ -1459,7 +1501,11 @@ later is required to fix a server side protocol bug.
if not git_require((2, 23, 0)):
return
projects = [p for p in projects if p.clone_depth]
projects = [
p
for p in projects
if p.clone_depth and not p.stateless_prune_needed
]
if not projects:
return
@@ -1623,9 +1669,10 @@ later is required to fix a server side protocol bug.
new_paths = {}
new_linkfile_paths = []
new_copyfile_paths = []
for project in self.GetProjects(
projects = self.GetProjects(
None, missing_ok=True, manifest=manifest, all_manifests=False
):
)
for project in projects:
new_linkfile_paths.extend(x.dest for x in project.linkfiles)
new_copyfile_paths.extend(x.dest for x in project.copyfiles)
@@ -1661,29 +1708,136 @@ later is required to fix a server side protocol bug.
)
for need_remove_file in need_remove_files:
# Try to remove the updated copyfile or linkfile.
# So, if the file is not exist, nothing need to do.
platform_utils.remove(
os.path.join(self.client.topdir, need_remove_file),
missing_ok=True,
need_remove_path = os.path.join(
self.client.topdir, need_remove_file
)
if os.path.isfile(need_remove_path):
platform_utils.remove(need_remove_path)
else:
platform_utils.removedirs(need_remove_path)
# Also try to remove empty parent directories.
parent = os.path.dirname(need_remove_path)
while parent != self.client.topdir:
try:
os.rmdir(parent)
except OSError:
break
parent = os.path.dirname(parent)
# Create copy-link-files.json, save dest path of "copyfile" and
# "linkfile".
with open(copylinkfile_path, "w", encoding="utf-8") as fp:
json.dump(new_paths, fp)
# Retry linkfile/copyfile creation for all projects. In
# interleaved sync mode, _CopyAndLinkFiles runs before this
# cleanup, so linkfiles whose dest was blocked by an old
# directory may have failed. _CopyAndLinkFiles is idempotent
# and skips dests that are already correct.
for project in projects:
project._CopyAndLinkFiles()
return True
def _SmartSyncSetup(self, opt, smart_sync_manifest_path, manifest):
if not manifest.manifest_server:
raise SmartSyncError(
"error: cannot smart sync: no manifest server defined in "
"manifest"
)
def _ResolveManifestServerTransport(self, opt, manifest):
"""Resolves the manifest server URL and transport.
Returns:
Tuple[str, xmlrpc.client.Transport]: The resolved server URL and
transport.
Raises:
SmartSyncError: If resolution fails (e.g. helper missing, helper
error, unsupported scheme).
"""
manifest_server = manifest.manifest_server
if not opt.quiet:
print("Using manifest server %s" % manifest_server)
helper_binary = manifest.manifest_server_helper
if helper_binary:
if not shutil.which(helper_binary):
raise SmartSyncError(
f"error: helper binary '{helper_binary}' declared in "
"manifest was not found in your PATH."
)
if not opt.quiet:
print(f"Using remote helper {helper_binary}")
p = None
try:
p = subprocess.Popen(
[helper_binary, manifest_server],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
stdout, stderr = p.communicate(timeout=10)
output = stdout.strip()
stderr_content = stderr.strip() if stderr else ""
except subprocess.TimeoutExpired as e:
err_msg = f"helper {helper_binary} timed out after 10 seconds"
timeout_stderr = e.stderr.strip() if e.stderr else ""
if timeout_stderr:
err_msg += f". Stderr: {timeout_stderr}"
raise SmartSyncError(err_msg)
except OSError as e:
raise SmartSyncError(
"failed to start or communicate with helper "
f"{helper_binary}: {e}"
)
finally:
if p and p.poll() is None:
p.kill()
p.wait()
if p.returncode != 0:
err_msg = (
f"helper {helper_binary} exited with exit code "
f"{p.returncode}."
)
if stderr_content:
err_msg += f" Stderr: {stderr_content}"
raise SmartSyncError(err_msg)
try:
res = json.loads(output)
status = res.get("status")
msg = res.get("message")
except json.JSONDecodeError as e:
err_msg = (
f"failed to parse JSON from helper {helper_binary}: {e}. "
f"Output was: {output}"
)
if stderr_content:
err_msg += f"\nStderr was: {stderr_content}"
raise SmartSyncError(err_msg)
if status != "ok":
err_msg = f"helper {helper_binary} returned error: {msg}"
if stderr_content:
err_msg += f"\nStderr was: {stderr_content}"
raise SmartSyncError(err_msg)
proxy_url = msg
transport = PersistentTransport(manifest_server, proxy=proxy_url)
server_url = manifest_server
if server_url.startswith("persistent-"):
server_url = server_url[len("persistent-") :]
return server_url, transport
# Fallback path if helper isn't specified
scheme = urllib.parse.urlparse(manifest_server).scheme
if scheme not in (
"http",
"https",
"persistent-http",
"persistent-https",
):
raise SmartSyncError(
f"error: unsupported manifest server scheme '{scheme}'."
)
if "@" not in manifest_server:
username = None
@@ -1718,20 +1872,34 @@ later is required to fix a server side protocol bug.
)
transport = PersistentTransport(manifest_server)
if manifest_server.startswith("persistent-"):
manifest_server = manifest_server[len("persistent-") :]
server_url = manifest_server
if server_url.startswith("persistent-"):
server_url = server_url[len("persistent-") :]
return server_url, transport
def _SmartSyncSetup(self, opt, smart_sync_manifest_path, manifest):
if not manifest.manifest_server:
raise SmartSyncError(
"error: cannot smart sync: no manifest server defined in "
"manifest"
)
if not opt.quiet:
print("Using manifest server %s" % manifest.manifest_server)
server_url, transport = self._ResolveManifestServerTransport(
opt, manifest
)
# Changes in behavior should update docs/smart-sync.md accordingly.
try:
server = xmlrpc.client.Server(manifest_server, transport=transport)
server = xmlrpc.client.Server(server_url, transport=transport)
if opt.smart_sync:
branch = self._GetBranch(manifest.manifestProject)
target = None
if "SYNC_TARGET" in os.environ:
target = os.environ["SYNC_TARGET"]
[success, manifest_str] = server.GetApprovedManifest(
branch, target
)
elif (
"TARGET_PRODUCT" in os.environ
and "TARGET_BUILD_VARIANT" in os.environ
@@ -1742,9 +1910,6 @@ later is required to fix a server side protocol bug.
os.environ["TARGET_RELEASE"],
os.environ["TARGET_BUILD_VARIANT"],
)
[success, manifest_str] = server.GetApprovedManifest(
branch, target
)
elif (
"TARGET_PRODUCT" in os.environ
and "TARGET_BUILD_VARIANT" in os.environ
@@ -1753,6 +1918,8 @@ later is required to fix a server side protocol bug.
os.environ["TARGET_PRODUCT"],
os.environ["TARGET_BUILD_VARIANT"],
)
if target:
[success, manifest_str] = server.GetApprovedManifest(
branch, target
)
@@ -1774,25 +1941,34 @@ later is required to fix a server side protocol bug.
aggregate_errors=[e],
)
self._ReloadManifest(manifest_name, manifest)
else:
raise SmartSyncError(
"error: manifest server RPC call failed: %s" % manifest_str
)
return manifest_name
raise SmartSyncError(
"error: manifest server RPC call failed: %s" % manifest_str
)
except (OSError, xmlrpc.client.Fault) as e:
if manifest.manifest_server_helper:
raise SmartSyncError(
"error: failed to communicate with manifest server via "
f"helper: {e}"
)
raise SmartSyncError(
"error: cannot connect to manifest server %s:\n%s"
% (manifest.manifest_server, e),
aggregate_errors=[e],
)
except xmlrpc.client.ProtocolError as e:
if manifest.manifest_server_helper:
raise SmartSyncError(
"error: failed to communicate with manifest server via "
f"helper: {e}"
)
raise SmartSyncError(
"error: cannot connect to manifest server %s:\n%d %s"
% (manifest.manifest_server, e.errcode, e.errmsg),
aggregate_errors=[e],
)
return manifest_name
def _UpdateAllManifestProjects(self, opt, mp, manifest_name, errors):
"""Fetch & update the local manifest project.
@@ -1804,7 +1980,11 @@ later is required to fix a server side protocol bug.
mp: the manifestProject to query.
manifest_name: Manifest file to be reloaded.
"""
if not mp.standalone_manifest_url:
if opt.superproject_revision and mp.manifest == self.outer_manifest:
self._SyncToSuperprojectRev(
opt, mp.manifest, mp, manifest_name, errors
)
elif not mp.standalone_manifest_url:
self._UpdateManifestProject(opt, mp, manifest_name, errors)
if mp.manifest.submanifests:
@@ -1986,17 +2166,19 @@ later is required to fix a server side protocol bug.
def Execute(self, opt, args):
errors = []
start_time = time.time()
try:
self._ExecuteHelper(opt, args, errors)
sync_duration_seconds = time.time() - start_time
except (RepoExitError, RepoChangedException):
raise
except (KeyboardInterrupt, Exception) as e:
raise RepoUnhandledExceptionError(e, aggregate_errors=errors)
# Run post-sync hook only after successful sync
self._RunPostSyncHook(opt)
self._RunPostSyncHook(opt, sync_duration_seconds=sync_duration_seconds)
def _RunPostSyncHook(self, opt):
def _RunPostSyncHook(self, opt, sync_duration_seconds=None):
"""Run post-sync hook if configured in manifest <repo-hooks>."""
hook = RepoHook.FromSubcmd(
hook_type="post-sync",
@@ -2004,10 +2186,60 @@ later is required to fix a server side protocol bug.
opt=opt,
abort_if_user_denies=False,
)
success = hook.Run(repo_topdir=self.client.topdir)
success = hook.Run(
repo_topdir=self.client.topdir,
sync_duration_seconds=sync_duration_seconds,
)
if not success:
print("Warning: post-sync hook reported failure.")
def _SyncToSuperprojectRev(
self,
opt: optparse.Values,
manifest,
mp: Project,
manifest_name: Optional[str],
errors: List[Exception],
) -> None:
"""Sync to a specific superproject commit."""
if not manifest.superproject:
raise SyncError("superproject not defined in manifest")
self._ConfigureSuperproject(
opt, manifest, revision=opt.superproject_revision
)
sync_result = manifest.superproject.Sync(self.git_event_log)
if not sync_result.success:
raise SyncError("failed to sync superproject")
cmd = ["show", f"{opt.superproject_revision}:.supermanifest"]
p = GitCommand(
None,
cmd,
gitdir=manifest.superproject._work_git,
bare=True,
capture_stdout=True,
capture_stderr=True,
)
if p.Wait() != 0:
raise SyncError(
f"failed to read .supermanifest from superproject: {p.stderr}"
)
try:
_, _, manifest_commit = p.stdout.strip().split()
except ValueError:
raise SyncError("could not parse .supermanifest")
mp.SetRevision(manifest_commit)
try:
self._UpdateManifestProject(opt, mp, manifest_name, errors)
except UpdateManifestError as e:
raise SyncError(
"failed to sync manifest project", aggregate_errors=[e]
)
def _ExecuteHelper(self, opt, args, errors):
manifest = self.outer_manifest
if not opt.outer_manifest:
@@ -2017,9 +2249,7 @@ later is required to fix a server side protocol bug.
manifest.Override(opt.manifest_name)
manifest_name = opt.manifest_name
smart_sync_manifest_path = os.path.join(
manifest.manifestProject.worktree, "smart_sync_override.xml"
)
smart_sync_manifest_path = self.GetSmartSyncOverridePath(manifest)
if opt.clone_bundle is None:
opt.clone_bundle = manifest.CloneBundle
@@ -2068,7 +2298,7 @@ later is required to fix a server side protocol bug.
):
mp.ConfigureCloneFilterForDepth("blob:none")
if opt.mp_update:
if opt.mp_update or opt.superproject_revision:
self._UpdateAllManifestProjects(opt, mp, manifest_name, errors)
else:
print("Skipping update of local manifest project.")
@@ -2146,8 +2376,8 @@ later is required to fix a server side protocol bug.
for project_name in sorted(self._bloated_projects):
warn_msg = (
f'warning: Project "{project_name}" is accumulating '
'unoptimized data. Please run "repo sync --auto-gc" or '
'"repo gc --repack" to clean up.'
'unoptimized data. Please run "git repack -a -d" in the '
'project directory or "repo gc --repack" to clean up.'
)
self.git_event_log.ErrorEvent(warn_msg)
logger.warning(warn_msg)
@@ -2401,6 +2631,8 @@ later is required to fix a server side protocol bug.
logger.error(
"Keyboard interrupt while processing %s", project.name
)
if not cls.is_multiprocessing_active():
raise
except GitError as e:
fetch_errors.append(e)
logger.error("error.GitError: Cannot fetch %s", e)
@@ -2451,6 +2683,8 @@ later is required to fix a server side protocol bug.
logger.error(
"Keyboard interrupt while processing %s", project.name
)
if not cls.is_multiprocessing_active():
raise
except GitError as e:
checkout_errors.append(e)
logger.error(
@@ -2972,14 +3206,15 @@ class LocalSyncState:
# request to request like the normal transport, the real url
# is passed during initialization.
class PersistentTransport(xmlrpc.client.Transport):
def __init__(self, orig_host):
def __init__(self, orig_host, proxy=None):
super().__init__()
self.orig_host = orig_host
self.proxy = proxy
def request(self, host, handler, request_body, verbose=False):
with GetUrlCookieFile(self.orig_host, not verbose) as (
cookiefile,
proxy,
cookie_proxy,
):
# Python doesn't understand cookies with the #HttpOnly_ prefix
# Since we're only using them for HTTP, copy the file temporarily,
@@ -3005,10 +3240,11 @@ class PersistentTransport(xmlrpc.client.Transport):
else:
cookiejar = cookielib.CookieJar()
active_proxy = self.proxy or cookie_proxy
proxyhandler = urllib.request.ProxyHandler
if proxy:
if active_proxy:
proxyhandler = urllib.request.ProxyHandler(
{"http": proxy, "https": proxy}
{"http": active_proxy, "https": active_proxy}
)
opener = urllib.request.build_opener(
@@ -3021,13 +3257,16 @@ class PersistentTransport(xmlrpc.client.Transport):
scheme = parse_results.scheme
if scheme == "persistent-http":
scheme = "http"
if scheme == "persistent-https":
elif scheme == "persistent-https":
# If we're proxying through persistent-https, use http. The
# proxy itself will do the https.
if proxy:
if active_proxy:
scheme = "http"
else:
scheme = "https"
elif scheme not in ("http", "https"):
if active_proxy:
scheme = "http"
# Parse out any authentication information using the base class.
host, extra_headers, _ = self.get_host_info(parse_results.netloc)
+11
View File
@@ -18,6 +18,7 @@ import pathlib
import pytest
import color
import platform_utils
import repo_trace
@@ -28,6 +29,16 @@ def disable_repo_trace(tmp_path):
repo_trace._TRACE_FILE = str(tmp_path / "TRACE_FILE_from_test")
@pytest.fixture(autouse=True)
def restore_default_coloring() -> None:
"""Force disable color for reproducible test behavior.
The few tests that cover color behavior can still reset/change the color as
needed.
"""
color.SetDefaultColoring("never")
# adapted from pytest-home 0.5.1
def _set_home(monkeypatch, path: pathlib.Path):
"""
+81 -1
View File
@@ -14,6 +14,8 @@
"""Unittests for the color.py module."""
from unittest import mock
import pytest
import utils_for_test
@@ -24,9 +26,14 @@ import git_config
@pytest.fixture
def coloring() -> color.Coloring:
"""Create a Coloring object for testing."""
return _make_coloring("always")
def _make_coloring(default_state: str) -> color.Coloring:
"""Set the default color mode and return a Coloring using test config."""
config_fixture = utils_for_test.FIXTURES_DIR / "test.gitconfig"
config = git_config.GitConfig(config_fixture)
color.SetDefaultColoring("true")
color.SetDefaultColoring(default_state)
return color.Coloring(config, "status")
@@ -72,3 +79,76 @@ def test_Color_Parse_empty_entry(coloring: color.Coloring) -> None:
assert val == "\033[2;34;47m"
val = coloring._parse("empty", "green", "white", "bold")
assert val == "\033[1;32;47m"
class TestSetDefaultColoring:
"""Tests for SetDefaultColoring."""
def test_none_leaves_default_unchanged(self) -> None:
color.DEFAULT = "auto"
color.SetDefaultColoring(None)
assert color.DEFAULT == "auto"
@pytest.mark.parametrize(
"value, expected",
(
# auto/true/yes all store their lowercase form.
("auto", "auto"),
("Auto", "auto"),
("true", "true"),
("True", "true"),
("yes", "yes"),
("Yes", "yes"),
# "always" stores as "always".
("always", "always"),
("Always", "always"),
# never/no/false store their lowercase form.
("never", "never"),
("no", "no"),
("false", "false"),
),
)
def test_maps_to_expected(self, value: str, expected: str) -> None:
color.SetDefaultColoring(value)
assert color.DEFAULT == expected
def test_unrecognised_leaves_default_unchanged(self) -> None:
color.DEFAULT = "auto"
color.SetDefaultColoring("garbage")
assert color.DEFAULT == "auto"
class TestColoringInit:
"""Tests for Coloring.__init__ color mode logic."""
@pytest.mark.parametrize(
"state, isatty, pager_active, expected",
(
# "always" enables color unconditionally.
("always", False, False, True),
# "never" disables color unconditionally.
("never", True, True, False),
# auto/true/yes enable color only on a TTY or active pager.
("auto", True, False, True),
("auto", False, False, False),
("auto", False, True, True),
("true", True, False, True),
("true", False, False, False),
("true", False, True, True),
("yes", True, False, True),
("yes", False, False, False),
("yes", False, True, True),
),
)
def test_color_mode(
self,
state: str,
isatty: bool,
pager_active: bool,
expected: bool,
) -> None:
with mock.patch("os.isatty", return_value=isatty), mock.patch(
"pager.active", pager_active
):
c = _make_coloring(state)
assert c.is_on is expected
+88
View File
@@ -0,0 +1,88 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the command.py module."""
from command import Command
class FakeProject:
"""Minimal project double for Command.GetProjects tests."""
def __init__(
self,
name,
relpath,
*,
gitdir=None,
derived_subprojects=None,
sync_s=False,
):
self.name = name
self.relpath = relpath
self.gitdir = gitdir or f"/git/{relpath}"
self.sync_s = sync_s
self.Exists = True
self._derived_subprojects = derived_subprojects or []
def GetDerivedSubprojects(self):
return list(self._derived_subprojects)
def MatchesGroups(self, _groups):
return True
def RelPath(self, local=True):
return self.relpath
class FakeManifest:
"""Minimal manifest double for Command.GetProjects tests."""
def __init__(self, projects):
self.projects = projects
def GetManifestGroupsStr(self):
return "default"
def test_get_projects_keeps_derived_subprojects_for_repeated_repo():
"""Derived subprojects are keyed by checkout path, not repo identity."""
submodule_a = FakeProject(
"submodule",
"src/one/submodule",
gitdir="/shared/modules/submodule.git",
)
submodule_b = FakeProject(
"submodule",
"src/two/submodule",
gitdir="/shared/modules/submodule.git",
)
project_a = FakeProject(
"project",
"src/one",
derived_subprojects=[submodule_a],
sync_s=True,
)
project_b = FakeProject(
"project",
"src/two",
derived_subprojects=[submodule_b],
sync_s=True,
)
manifest = FakeManifest([project_a, project_b])
cmd = Command(manifest=manifest)
projects = cmd.GetProjects([])
assert set(projects) == {project_a, project_b, submodule_a, submodule_b}
+42
View File
@@ -226,3 +226,45 @@ def test_get_sync_analysis_state_data(rw_config_file: Path) -> None:
for key, value in TESTS:
assert sync_data[f"{git_config.SYNC_STATE_PREFIX}{key}"] == value
assert sync_data[f"{git_config.SYNC_STATE_PREFIX}main.synctime"]
def test_remote_save_with_push_url_without_projectname(
rw_config_file: Path,
) -> None:
"""Test saving a remote pushUrl when projectname is unset."""
config = git_config.GitConfig(str(rw_config_file))
remote = config.GetRemote("origin")
remote.url = "https://example.com/repo"
remote.pushUrl = "ssh://example.com"
remote.projectname = None
remote.Save()
written_config = git_config.GitConfig(str(rw_config_file))
assert (
written_config.GetString("remote.origin.pushurl") == "ssh://example.com"
)
@pytest.mark.parametrize(
"rev, expected",
(
("a" * 40, True),
("0" * 40, True),
("f" * 40, True),
("a" * 64, True),
("0" * 64, True),
("f" * 64, True),
("a" * 39, False),
("a" * 41, True),
("a" * 63, True),
("a" * 65, False),
("g" * 40, False),
("g" * 64, False),
("refs/heads/master", False),
("refs/tags/v1.0", False),
),
)
def test_is_id(rev: str, expected: bool) -> None:
"""Test IsId identifies both SHA-1 and SHA-256 hashes."""
assert git_config.IsId(rev) == expected
+1 -1
View File
@@ -190,7 +190,7 @@ class SuperprojectTestCase(unittest.TestCase):
),
revision="refs/heads/main",
)
with mock.patch.object(self._superproject, "_branch", "junk"):
with mock.patch.object(self._superproject, "revision", "junk"):
sync_result = self._superproject.Sync(self.git_event_log)
self.assertFalse(sync_result.success)
self.assertTrue(sync_result.fatal)
+2 -1
View File
@@ -315,6 +315,7 @@ def test_data_event_config(event_log: git_trace2_event_log.EventLog) -> None:
"repo.partialclone": "false",
"repo.syncstate.superproject.hassuperprojecttag": "true",
"repo.syncstate.superproject.sys.argv": ["--", "sync", "protobuf"],
"repo.syncstate.emptykey": "",
}
prefix_value = "prefix"
event_log.LogDataConfigEvents(config, prefix_value)
@@ -323,7 +324,7 @@ def test_data_event_config(event_log: git_trace2_event_log.EventLog) -> None:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 5
assert len(log_data) == 6
data_events = log_data[1:]
verify_common_keys(log_data[0], expected_event_name="version")
+47
View File
@@ -14,6 +14,9 @@
"""Unittests for the hooks.py module."""
from io import StringIO
import sys
import pytest
import hooks
@@ -58,3 +61,47 @@ def test_direct_interp(shebang: str, interp: str) -> None:
def test_env_interp(shebang: str, interp: str) -> None:
"""Lines whose shebang launches through `env`."""
assert hooks.RepoHook._ExtractInterpFromShebang(shebang) == interp
def test_post_sync_argument_validation() -> None:
"""Test that post-sync hook requires exact API arguments."""
class FakeProject:
def __init__(self):
self.worktree = "/some/path"
self.enabled_repo_hooks = ["post-sync"]
hook = hooks.RepoHook(
hook_type="post-sync",
hooks_project=FakeProject(),
repo_topdir="/topdir",
manifest_url="https://gerrit",
allow_all_hooks=True,
)
old_stderr = sys.stderr
sys.stderr = StringIO()
try:
# Call with missing arg `sync_duration_seconds`
res = hook.Run(repo_topdir="/topdir")
assert res is False
assert "hook 'post-sync' called incorrectly" in sys.stderr.getvalue()
# Mock _CheckHook and _ExecuteHook to test success path
hook._CheckHook = lambda: None
executed_kwargs = {}
def fake_execute(**kw):
executed_kwargs.update(kw)
hook._ExecuteHook = fake_execute
res = hook.Run(repo_topdir="/topdir", sync_duration_seconds=12.345)
assert res is True
assert executed_kwargs.get("sync_duration_seconds") == 12.345
finally:
sys.stderr = old_stderr
+20
View File
@@ -819,6 +819,26 @@ class TestProjectElement:
str(repo_client.topdir), ".repo", "projects", "..git"
)
def test_get_project_paths_local_gitdirs(
self, repo_client: RepoClient
) -> None:
"""Check GetProjectPaths with UseLocalGitDirs."""
manifest = repo_client.get_xml_manifest(
'<?xml version="1.0" encoding="UTF-8"?><manifest></manifest>'
)
manifest.manifestProject.config.SetBoolean("repo.uselocalgitdirs", True)
relpath, worktree, gitdir, objdir, use_git_worktrees = (
manifest.GetProjectPaths("foo", "bar", "origin")
)
assert os.path.normpath(gitdir) == os.path.normpath(
os.path.join(str(repo_client.topdir), "bar", ".git")
)
assert os.path.normpath(objdir) == os.path.normpath(
os.path.join(str(repo_client.topdir), "bar", ".git")
)
def test_bad_path_name_checks(self, repo_client: RepoClient) -> None:
"""Check handling of bad path & name attributes."""
+68
View File
@@ -46,3 +46,71 @@ def test_remove_missing_ok(tmp_path: Path) -> None:
path.touch()
platform_utils.remove(path, missing_ok=False)
assert not path.exists()
def test_removedirs_nonexistent(tmp_path: Path) -> None:
"""removedirs should silently succeed on nonexistent paths."""
platform_utils.removedirs(tmp_path / "does-not-exist")
def test_removedirs_symlink(tmp_path: Path) -> None:
"""removedirs should remove a symlink."""
link = tmp_path / "link"
link.symlink_to("target")
platform_utils.removedirs(link)
assert not link.exists()
def test_removedirs_empty_dir(tmp_path: Path) -> None:
"""removedirs should remove an empty directory."""
d = tmp_path / "empty"
d.mkdir()
platform_utils.removedirs(d)
assert not d.exists()
def test_removedirs_nested_empty_dirs(tmp_path: Path) -> None:
"""removedirs should remove nested empty directories."""
d = tmp_path / "a" / "b" / "c"
d.mkdir(parents=True)
platform_utils.removedirs(tmp_path / "a")
assert not (tmp_path / "a").exists()
def test_removedirs_symlinks_inside_dir(tmp_path: Path) -> None:
"""removedirs should remove symlinks inside a directory."""
d = tmp_path / "dir"
d.mkdir()
(d / "link1").symlink_to("target1")
(d / "link2").symlink_to("target2")
platform_utils.removedirs(d)
assert not d.exists()
def test_removedirs_preserves_user_files(tmp_path: Path) -> None:
"""removedirs should not delete regular files or their parent dirs."""
d = tmp_path / "dir"
d.mkdir()
(d / "link").symlink_to("target")
(d / "user-file.txt").write_text("keep me")
platform_utils.removedirs(d)
assert d.exists()
assert not (d / "link").exists()
assert (d / "user-file.txt").read_text() == "keep me"
def test_removedirs_deep_nested_with_symlinks(tmp_path: Path) -> None:
"""removedirs should handle deep nesting: sub/dir/target."""
d = tmp_path / "sub" / "dir"
d.mkdir(parents=True)
(d / "link").symlink_to("target")
platform_utils.removedirs(tmp_path / "sub")
assert not (tmp_path / "sub").exists()
def test_removedirs_regular_file_noop(tmp_path: Path) -> None:
"""removedirs should not delete a regular file."""
f = tmp_path / "file.txt"
f.write_text("data")
platform_utils.removedirs(f)
assert f.exists()
+605 -45
View File
@@ -104,6 +104,28 @@ class ProjectTests(unittest.TestCase):
"abcd00%21%21_%2b",
)
def test_get_head_revision_id(self):
"""Check GetHeadRevisionId behavior."""
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir)
# Initially unborn HEAD should return None.
self.assertIsNone(proj.GetHeadRevisionId())
# Create a commit.
with open(os.path.join(tempdir, "readme"), "w") as fp:
fp.write("hello")
proj.work_git.add("readme")
proj.work_git.commit("-m", "initial commit")
# HEAD should resolve to the commit SHA.
commit_sha = proj.work_git.rev_parse("HEAD")
self.assertEqual(commit_sha, proj.GetHeadRevisionId())
# Even if worktree is detached.
proj.work_git.checkout("HEAD~0")
self.assertEqual(commit_sha, proj.GetHeadRevisionId())
@unittest.skipUnless(
utils_for_test.supports_reftable(),
"git reftable support is required for this test",
@@ -127,6 +149,88 @@ class ProjectTests(unittest.TestCase):
).strip()
self.assertEqual(expected, fakeproj.work_git.GetHead())
def _get_derived_subproject_url(self, submodule_url):
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
class FakeManifest:
def __init__(self, topdir):
self.topdir = topdir
self.globalConfig = None
self.is_multimanifest = False
self.path_prefix = ""
self.paths = {}
def GetSubprojectName(self, parent, path):
return path
def GetSubprojectPaths(self, parent, name, path):
relpath = path
worktree = os.path.join(self.topdir, path)
gitdir = os.path.join(self.topdir, f"{path}.git")
objdir = os.path.join(self.topdir, f"{path}.obj")
os.makedirs(worktree, exist_ok=True)
os.makedirs(gitdir, exist_ok=True)
os.makedirs(objdir, exist_ok=True)
return relpath, worktree, gitdir, objdir
manifest = FakeManifest(tempdir)
worktree = os.path.join(tempdir, "parent")
gitdir = os.path.join(tempdir, "parent.git")
objdir = os.path.join(tempdir, "parent.obj")
os.makedirs(worktree)
os.makedirs(gitdir)
os.makedirs(objdir)
parent = project.Project(
manifest=manifest,
name="parent",
remote=project.RemoteSpec(
"origin", url="https://example.com/platform/superproject"
),
gitdir=gitdir,
objdir=objdir,
worktree=worktree,
relpath="parent",
revisionExpr="refs/heads/main",
revisionId=None,
)
def fake_get_submodules(current):
if current is parent:
return [("subrev", "child", submodule_url, "false")]
return []
with mock.patch.object(
project.Project, "_GetSubmodules", autospec=True
) as get_submodules:
get_submodules.side_effect = fake_get_submodules
result = parent.GetDerivedSubprojects()
self.assertEqual(1, len(result))
return result[0].remote.url
def test_derived_subproject_joins_only_git_relative_urls(self):
tests = (
(
"./submodule",
"https://example.com/platform/superproject/submodule",
),
("../sibling", "https://example.com/platform/sibling"),
)
for submodule_url, expected in tests:
with self.subTest(submodule_url=submodule_url):
self.assertEqual(
expected, self._get_derived_subproject_url(submodule_url)
)
def test_derived_subproject_leaves_dot_prefixed_names_unchanged(self):
for submodule_url in (".foo", "..bar"):
with self.subTest(submodule_url=submodule_url):
self.assertEqual(
submodule_url,
self._get_derived_subproject_url(submodule_url),
)
class CopyLinkTestCase(unittest.TestCase):
"""TestCase for stub repo client checkouts.
@@ -365,6 +469,47 @@ class LinkFile(CopyLinkTestCase):
os.path.join("git-project", "foo.txt"), os.readlink(dest)
)
def test_replace_empty_dir_with_symlink(self):
"""A linkfile should replace an empty real directory at the dest path.
This is the common case: the old linkfiles inside the directory were
already cleaned up by UpdateCopyLinkfileList, leaving an empty parent
directory behind.
"""
src_dir = os.path.join(self.worktree, "dot-llms")
os.makedirs(src_dir)
dest = os.path.join(self.topdir, "mydir")
os.makedirs(dest)
lf = self.LinkFile("dot-llms", "mydir")
lf._Link()
self.assertTrue(os.path.islink(dest))
self.assertEqual(
os.path.join("git-project", "dot-llms"), os.readlink(dest)
)
def test_nonempty_dir_not_clobbered(self):
"""A linkfile must not delete a non-empty directory.
If the user created files in a directory that a new linkfile wants
to replace, __linkIt should fail safely rather than deleting content.
"""
src_dir = os.path.join(self.worktree, "dot-llms")
os.makedirs(src_dir)
dest = os.path.join(self.topdir, "mydir")
os.makedirs(dest)
user_file = os.path.join(dest, "user-notes.txt")
self.touch(user_file)
lf = self.LinkFile("dot-llms", "mydir")
lf._Link()
# The directory should NOT be replaced — user content is preserved.
self.assertFalse(os.path.islink(dest))
self.assertTrue(os.path.isdir(dest))
self.assertTrue(os.path.exists(user_file))
class MigrateWorkTreeTests(unittest.TestCase):
"""Check _MigrateOldWorkTreeGitDir handling."""
@@ -534,6 +679,9 @@ class ManifestPropertiesFetchedCorrectly(unittest.TestCase):
fakeproj.config.SetBoolean("repo.worktree", False)
self.assertFalse(fakeproj.use_worktree)
fakeproj.config.SetBoolean("repo.uselocalgitdirs", False)
self.assertFalse(fakeproj.use_local_gitdirs)
fakeproj.config.SetBoolean("repo.clonebundle", False)
self.assertFalse(fakeproj.clone_bundle)
@@ -568,39 +716,137 @@ class ManifestPropertiesFetchedCorrectly(unittest.TestCase):
fakeproj.config.SetString("manifest.platform", "auto")
self.assertEqual(fakeproj.manifest_platform, "auto")
def test_sync_use_local_gitdirs_worktree_conflict(self):
"""Test that --use-local-gitdirs conflicts with --worktree."""
with utils_for_test.TempGitTree() as tempdir:
fakeproj = self.setUpManifest(tempdir)
class DummyManifest:
is_submanifest = False
def GetDefaultGroupsStr(self, with_platform=False):
return ""
fakeproj.manifest = DummyManifest()
result = fakeproj.Sync(use_local_gitdirs=True, worktree=True)
self.assertFalse(result)
def test_sync_use_local_gitdirs_archive_conflict(self):
"""Test that --use-local-gitdirs conflicts with --archive."""
with utils_for_test.TempGitTree() as tempdir:
fakeproj = self.setUpManifest(tempdir)
class DummyManifest:
is_submanifest = False
def GetDefaultGroupsStr(self, with_platform=False):
return ""
fakeproj.manifest = DummyManifest()
result = fakeproj.Sync(use_local_gitdirs=True, archive=True)
self.assertFalse(result)
def test_sync_use_local_gitdirs_mirror_conflict(self):
"""Test that --use-local-gitdirs conflicts with --mirror."""
with utils_for_test.TempGitTree() as tempdir:
fakeproj = self.setUpManifest(tempdir)
class DummyManifest:
is_submanifest = False
def GetDefaultGroupsStr(self, with_platform=False):
return ""
fakeproj.manifest = DummyManifest()
result = fakeproj.Sync(use_local_gitdirs=True, mirror=True)
self.assertFalse(result)
def test_delete_worktree_corrupted(self):
"""Test DeleteWorktree gracefully handles corrupted projects."""
for use_git_worktrees in (False, True):
with self.subTest(use_git_worktrees=use_git_worktrees):
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir)
os.makedirs(os.path.join(tempdir, "worktree"))
os.makedirs(os.path.join(tempdir, "gitdir"))
proj.worktree = os.path.join(tempdir, "worktree")
proj.gitdir = os.path.join(tempdir, "gitdir")
proj.use_git_worktrees = use_git_worktrees
with mock.patch.object(
proj,
"IsDirty",
side_effect=error.GitError("mock error"),
):
with self.assertRaises(project.DeleteWorktreeError):
proj.DeleteWorktree(force=False)
self.assertTrue(proj.DeleteWorktree(force=True))
self.assertFalse(os.path.exists(proj.worktree))
self.assertFalse(os.path.exists(proj.gitdir))
def _create_mock_project(
tempdir,
use_local_gitdirs=False,
fetch_cmd=None,
depth=None,
gitdir=None,
objdir=None,
revisionExpr="main",
sync_strategy=None,
):
manifest = mock.MagicMock()
manifest.manifestProject.use_local_gitdirs = use_local_gitdirs
manifest.manifestProject.fetch_cmd = fetch_cmd
manifest.manifestProject.depth = depth
manifest.manifestProject.dissociate = False
manifest.manifestProject.clone_filter = None
manifest.manifestProject.config.GetBoolean.return_value = False
manifest.is_multimanifest = False
manifest.IsMirror = False
manifest.topdir = tempdir
remote = mock.MagicMock()
remote.name = "origin"
remote.url = "http://example.com/repo"
if gitdir is None:
gitdir = os.path.join(tempdir, ".git")
if objdir is None:
objdir = os.path.join(tempdir, ".git")
proj = project.Project(
manifest=manifest,
name="test-project",
remote=remote,
gitdir=gitdir,
objdir=objdir,
worktree=tempdir,
relpath="test-project",
revisionExpr=revisionExpr,
revisionId=None,
sync_strategy=sync_strategy,
)
proj.bare_git = mock.MagicMock()
proj._LsRemote = mock.MagicMock(return_value="1234abcd\trefs/heads/main\n")
manifest.GetProjectsWithName.return_value = [proj]
return proj
class StatelessSyncTests(unittest.TestCase):
"""Tests for stateless sync strategy."""
def _get_project(self, tempdir):
manifest = mock.MagicMock()
manifest.manifestProject.depth = None
manifest.manifestProject.dissociate = False
manifest.manifestProject.clone_filter = None
manifest.is_multimanifest = False
manifest.manifestProject.config.GetBoolean.return_value = False
remote = mock.MagicMock()
remote.name = "origin"
remote.url = "http://"
proj = project.Project(
manifest=manifest,
name="test-project",
remote=remote,
gitdir=os.path.join(tempdir, ".git"),
objdir=os.path.join(tempdir, ".git"),
worktree=tempdir,
relpath="test-project",
revisionExpr="1234abcd",
revisionId=None,
sync_strategy="stateless",
proj = _create_mock_project(
tempdir, revisionExpr="1234abcd", sync_strategy="stateless"
)
proj._CheckForImmutableRevision = mock.MagicMock(return_value=False)
proj._LsRemote = mock.MagicMock(
return_value="1234abcd\trefs/heads/main\n"
)
proj.bare_git = mock.MagicMock()
proj.bare_git.rev_parse.return_value = "5678abcd"
proj.bare_git.rev_list.return_value = ["0"]
proj.IsDirty = mock.MagicMock(return_value=False)
@@ -663,6 +909,36 @@ class StatelessSyncTests(unittest.TestCase):
capture_stderr=True,
)
def test_sync_local_half_no_upstream_propagates_force_checkout(self):
"""Test Sync_LocalHalf forwards force_checkout when detaching."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj._InitWorkTree = mock.MagicMock()
proj.CleanPublishedCache = mock.MagicMock()
proj.GetRevisionId = mock.MagicMock(return_value="1234abcd")
proj._Checkout = mock.MagicMock()
proj._CopyAndLinkFiles = mock.MagicMock()
proj.work_git = mock.MagicMock()
proj.work_git.GetHead.return_value = "refs/heads/topic"
proj.bare_ref = mock.MagicMock()
proj.bare_ref.all = {"refs/heads/topic": "5678abcd"}
branch = mock.MagicMock()
branch.name = "topic"
branch.LocalMerge = False
proj.GetBranch = mock.MagicMock(return_value=branch)
syncbuf = project.SyncBuffer(proj.config)
proj.Sync_LocalHalf(syncbuf, force_checkout=True)
proj._Checkout.assert_called_once_with(
"1234abcd", force_checkout=True, quiet=True
)
proj._CopyAndLinkFiles.assert_called_once_with()
def test_sync_network_half_stateless_skips_if_stash(self):
"""Test stateless sync skips if stash exists."""
with utils_for_test.TempGitTree() as tempdir:
@@ -690,28 +966,12 @@ class SyncOptimizationTests(unittest.TestCase):
"""Tests for sync optimization logic involving shallow clones."""
def _get_project(self, tempdir, depth=None):
manifest = mock.MagicMock()
manifest.manifestProject.depth = depth
manifest.manifestProject.dissociate = False
manifest.manifestProject.clone_filter = None
manifest.is_multimanifest = False
manifest.manifestProject.config.GetBoolean.return_value = False
manifest.IsMirror = False
remote = mock.MagicMock()
remote.name = "origin"
remote.url = "http://"
proj = project.Project(
manifest=manifest,
name="test-project",
remote=remote,
proj = _create_mock_project(
tempdir,
depth=depth,
gitdir=os.path.join(tempdir, "gitdir"),
objdir=os.path.join(tempdir, "objdir"),
worktree=tempdir,
relpath="test-project",
revisionExpr="0123456789abcdef0123456789abcdef01234567",
revisionId=None,
)
proj._CheckForImmutableRevision = mock.MagicMock(return_value=True)
proj.DeleteWorktree = mock.MagicMock()
@@ -720,6 +980,31 @@ class SyncOptimizationTests(unittest.TestCase):
proj._InitMRef = mock.MagicMock()
return proj
def _create_sharing_project(self, tempdir, proj, share_objdir=True):
"""Create another project with the same name but a different gitdir.
Args:
share_objdir: a boolean, if True - the new project shares the same
objdir, if False - the new project has a different objdir.
"""
if share_objdir:
other_objdir = proj.objdir
else:
other_objdir = os.path.join(tempdir, "other_objdir")
other = project.Project(
manifest=proj.manifest,
name=proj.name,
remote=proj.remote,
gitdir=os.path.join(tempdir, "other_gitdir"),
objdir=other_objdir,
worktree=os.path.join(tempdir, "other_worktree"),
relpath="other-test-project",
revisionExpr=proj.revisionExpr,
revisionId=None,
)
proj.manifest.GetProjectsWithName.return_value.append(other)
return other
def test_sync_network_half_shallow_missing_fetches(self):
"""Test Sync_NetworkHalf fetches if shallow file is missing."""
with utils_for_test.TempGitTree() as tempdir:
@@ -754,6 +1039,72 @@ class SyncOptimizationTests(unittest.TestCase):
self.assertTrue(res.success)
proj._RemoteFetch.assert_not_called()
def test_sync_network_half_sharing_project_shallow_missing_fetches(
self,
):
"""Test Sync_NetworkHalf fetches when sharing project has shallow
file but this project does not."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
other = self._create_sharing_project(tempdir, proj)
os.makedirs(other.gitdir, exist_ok=True)
with open(os.path.join(other.gitdir, "shallow"), "w") as f:
f.write("")
proj._RemoteFetch = mock.MagicMock(return_value=True)
res = proj.Sync_NetworkHalf(optimized_fetch=True)
self.assertTrue(res.success)
proj._RemoteFetch.assert_called_once()
def test_sync_network_half_different_objdir_shallow_exists_skips(self):
"""Test Sync_NetworkHalf skips when same-name project has shallow file
but different objdir (like in a multi-manifest setup)."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
other = self._create_sharing_project(
tempdir, proj, share_objdir=False
)
os.makedirs(other.gitdir, exist_ok=True)
with open(os.path.join(other.gitdir, "shallow"), "w") as f:
f.write("")
proj._RemoteFetch = mock.MagicMock()
res = proj.Sync_NetworkHalf(optimized_fetch=True)
self.assertTrue(res.success)
proj._RemoteFetch.assert_not_called()
def test_sync_network_half_sharing_project_both_shallow_skips(self):
"""Test Sync_NetworkHalf skips when both this project and the sharing
project have shallow files."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
with open(os.path.join(proj.gitdir, "shallow"), "w") as f:
f.write("")
other = self._create_sharing_project(tempdir, proj)
os.makedirs(other.gitdir, exist_ok=True)
with open(os.path.join(other.gitdir, "shallow"), "w") as f:
f.write("")
proj._RemoteFetch = mock.MagicMock()
res = proj.Sync_NetworkHalf(optimized_fetch=True)
self.assertTrue(res.success)
proj._RemoteFetch.assert_not_called()
def test_remote_fetch_shallow_missing_fetches(self):
"""Test _RemoteFetch fetches if shallow file is missing."""
with utils_for_test.TempGitTree() as tempdir:
@@ -794,3 +1145,212 @@ class SyncOptimizationTests(unittest.TestCase):
self.assertTrue(res)
mock_git_cmd.assert_not_called()
def test_remote_fetch_sharing_project_shallow_missing_fetches(self):
"""Test _RemoteFetch fetches when sharing project has shallow file
but this project does not."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
other = self._create_sharing_project(tempdir, proj)
os.makedirs(other.gitdir, exist_ok=True)
with open(os.path.join(other.gitdir, "shallow"), "w") as f:
f.write("")
with mock.patch("project.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
res = proj._RemoteFetch(
current_branch_only=True,
depth=1,
use_superproject=False,
)
self.assertTrue(res)
mock_git_cmd.assert_called()
def test_remote_fetch_sharing_project_both_shallow_skips(self):
"""Test _RemoteFetch skips when both this project and the sharing
project have shallow files."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
with open(os.path.join(proj.gitdir, "shallow"), "w") as f:
f.write("")
other = self._create_sharing_project(tempdir, proj)
os.makedirs(other.gitdir, exist_ok=True)
with open(os.path.join(other.gitdir, "shallow"), "w") as f:
f.write("")
with mock.patch("project.GitCommand") as mock_git_cmd:
res = proj._RemoteFetch(
current_branch_only=True,
depth=1,
use_superproject=False,
)
self.assertTrue(res)
mock_git_cmd.assert_not_called()
class GetEnvVarsTests(unittest.TestCase):
"""Tests for GetEnvVars project environment variable generation."""
def _get_project(self, tempdir, revisionExpr="main"):
proj = _create_mock_project(tempdir, revisionExpr=revisionExpr)
proj.GetRevisionId = mock.MagicMock(return_value="1234abcd")
return proj
def test_get_env_vars_basic(self):
"""Test that all basic environment variables are set correctly."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.manifest.path_prefix = "sub-manifest"
proj.upstream = "upstream-branch"
proj.dest_branch = "dest-branch"
env = proj.GetEnvVars(local=True)
self.assertEqual(env["REPO_PROJECT"], "test-project")
self.assertEqual(env["REPO_OUTERPATH"], "sub-manifest")
self.assertEqual(env["REPO_INNERPATH"], "test-project")
self.assertEqual(env["REPO_PATH"], "test-project")
self.assertEqual(env["REPO_REMOTE"], "origin")
self.assertEqual(env["REPO_LREV"], "1234abcd")
self.assertEqual(env["REPO_RREV"], "main")
self.assertEqual(env["REPO_UPSTREAM"], "upstream-branch")
self.assertEqual(env["REPO_DEST_BRANCH"], "dest-branch")
self.assertEqual(
env["REPO_PROJECT_FETCH_URL"], "http://example.com/repo"
)
def test_get_env_vars_non_local(self):
"""Test environment variables generation with local=False."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.manifest.path_prefix = "sub-manifest"
env = proj.GetEnvVars(local=False)
# REPO_PATH should be relative to outermost manifest
# (sub-manifest/test-project)
self.assertEqual(env["REPO_PATH"], "sub-manifest/test-project")
def test_get_env_vars_mirror(self):
"""Test environment variables generation in mirror mode."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.manifest.IsMirror = True
env = proj.GetEnvVars()
# In mirror mode, REPO_LREV should be empty, and GetRevisionId must
# not be called
self.assertEqual(env["REPO_LREV"], "")
proj.GetRevisionId.assert_not_called()
def test_get_env_vars_annotations(self):
"""Test that project annotations are added correctly."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
annotation1 = mock.MagicMock()
annotation1.name = "key1"
annotation1.value = "value1"
annotation2 = mock.MagicMock()
annotation2.name = "key2"
annotation2.value = "value2"
proj.annotations = [annotation1, annotation2]
env = proj.GetEnvVars()
self.assertEqual(env["REPO__key1"], "value1")
self.assertEqual(env["REPO__key2"], "value2")
def test_get_env_vars_invalid_revision_graceful(self):
"""Test that invalid revision error is handled gracefully."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.GetRevisionId.side_effect = error.ManifestInvalidRevisionError(
"revision not found"
)
env = proj.GetEnvVars()
self.assertEqual(env["REPO_LREV"], "")
class FetchCmdTests(unittest.TestCase):
"""Tests for fetch_cmd feature."""
def setUpManifest(self, tempdir):
repodir = os.path.join(tempdir, ".repo")
manifest_dir = os.path.join(repodir, "manifests")
manifest_file = os.path.join(repodir, manifest_xml.MANIFEST_FILE_NAME)
os.mkdir(repodir)
os.mkdir(manifest_dir)
manifest = manifest_xml.XmlManifest(repodir, manifest_file)
return project.ManifestProject(
manifest, "test/manifest", os.path.join(tempdir, ".git"), tempdir
)
def _get_project(self, tempdir):
proj = _create_mock_project(
tempdir, use_local_gitdirs=True, fetch_cmd="echo hi"
)
proj.GetRevisionId = mock.MagicMock(return_value="1234abcd")
return proj
def test_fetch_cmd_execution(self):
"""Test that fetch_cmd is executed with correct environment."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.bare_git.rev_parse.return_value = "1234abcd"
mock_remote = mock.MagicMock()
mock_remote.ToLocal.return_value = "refs/remotes/origin/main"
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stderr="")
res = proj._CustomFetch()
self.assertTrue(res)
mock_run.assert_called_once()
args, kwargs = mock_run.call_args
self.assertEqual(args[0], "echo hi")
self.assertEqual(kwargs["shell"], True)
self.assertEqual(kwargs["cwd"], tempdir)
self.assertEqual(kwargs["env"]["REPO_TREV"], "1234abcd")
self.assertEqual(
kwargs["env"]["REPO_PROJECT_FETCH_URL"],
"http://example.com/repo",
)
def test_sync_fetch_cmd_requires_use_local_gitdirs(self):
"""Test that fetch_cmd requires use_local_gitdirs."""
with utils_for_test.TempGitTree() as tempdir:
fakeproj = self.setUpManifest(tempdir)
class DummyManifest:
is_submanifest = False
def GetDefaultGroupsStr(self, with_platform=False):
return ""
fakeproj.manifest = DummyManifest()
fakeproj.config.SetString("repo.fetchcmd", "echo hi")
fakeproj.config.SetBoolean("repo.uselocalgitdirs", False)
result = fakeproj.Sync(use_local_gitdirs=False)
self.assertFalse(result)
+9 -8
View File
@@ -17,12 +17,10 @@
import contextlib
import io
from pathlib import Path
from unittest import mock
import utils_for_test
import manifest_xml
import project
import subcmds
@@ -87,13 +85,16 @@ def test_forall_all_projects_called_once(tmp_path: Path) -> None:
opts, args = cmd.OptionParser.parse_args(["-c", "echo $REPO_PROJECT"])
opts.verbose = False
# Set revisionId directly so GetRevisionId() short-circuits without
# touching git. Using mock.patch.object on the class does not work
# with Python 3.14+, which defaults to "forkserver" on Linux —
# class-level patches do not survive into forkserver worker processes.
for proj in manifest.projects:
proj.revisionId = "refs/heads/main"
with contextlib.redirect_stdout(io.StringIO()) as stdout:
# Mock to not have the Execute fail on remote check.
with mock.patch.object(
project.Project, "GetRevisionId", return_value="refs/heads/main"
):
# Run the forall command.
cmd.Execute(opts, args)
# Run the forall command.
cmd.Execute(opts, args)
output = stdout.getvalue()
# Verify that we got every project name in the output.
+255
View File
@@ -0,0 +1,255 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the subcmds/info.py module."""
import json
from unittest import mock
import pytest
from subcmds import info
def _get_cmd() -> info.Info:
"""Build a mock-backed Info command for testing."""
manifest = mock.MagicMock()
manifest.default.revisionExpr = "refs/heads/main"
manifest.manifestProject.config.GetBranch.return_value.merge = (
"refs/heads/main"
)
manifest.GetManifestGroupsStr.return_value = "all"
manifest.superproject = None
manifest.outer_client = manifest
client = mock.MagicMock()
git_event_log = mock.MagicMock()
return info.Info(
manifest=manifest,
client=client,
git_event_log=git_event_log,
)
def test_include_options_default_true() -> None:
"""Both include options should default to True."""
opts, _ = _get_cmd().OptionParser.parse_args([])
assert opts.include_summary
assert opts.include_projects
def test_no_include_summary_parses() -> None:
"""--no-include-summary should set include_summary to False."""
opts, _ = _get_cmd().OptionParser.parse_args(["--no-include-summary"])
assert not opts.include_summary
def test_no_include_projects_parses() -> None:
"""--no-include-projects should set include_projects to False."""
opts, _ = _get_cmd().OptionParser.parse_args(["--no-include-projects"])
assert not opts.include_projects
def test_format_default_text() -> None:
"""Default format should be text."""
opts, _ = _get_cmd().OptionParser.parse_args([])
assert opts.format == "text"
def test_format_json_parses() -> None:
"""--format=json should be accepted."""
opts, _ = _get_cmd().OptionParser.parse_args(["--format=json"])
assert opts.format == "json"
def test_no_include_projects_skips_projects() -> None:
"""--no-include-projects should skip project iteration."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--no-include-projects"])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_not_called()
def test_no_include_summary_skips_summary() -> None:
"""--no-include-summary should not query or print manifest metadata."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--no-include-summary"])
with mock.patch.object(cmd, "_printDiffInfo"):
cmd.Execute(opts, args)
cmd.manifest.GetManifestGroupsStr.assert_not_called()
def test_default_calls_diff_info() -> None:
"""Default options should call _printDiffInfo."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args([])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_called_once_with(opts, args)
mock_overview.assert_not_called()
def test_overview_calls_commit_overview() -> None:
"""--overview should call _printCommitOverview, not _printDiffInfo."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--overview"])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_called_once_with(opts, args)
def test_no_include_projects_with_overview() -> None:
"""--no-include-projects should take priority over --overview."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--no-include-projects", "--overview"]
)
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_not_called()
def test_json_summary_only(capsys) -> None:
"""--format=json --no-include-projects should emit only summary."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--format=json", "--no-include-projects"]
)
cmd.Execute(opts, args)
data = json.loads(capsys.readouterr().out)
assert "summary" in data
assert "projects" not in data
assert data["summary"]["manifest_branch"] == "refs/heads/main"
assert data["summary"]["manifest_groups"] == "all"
def test_json_no_summary(capsys) -> None:
"""--format=json --no-include-summary should omit summary."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--format=json", "--no-include-summary", "--no-include-projects"]
)
cmd.Execute(opts, args)
data = json.loads(capsys.readouterr().out)
assert "summary" not in data
def test_json_rejects_diff() -> None:
"""--format=json --diff should be rejected."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--format=json", "--diff"])
with pytest.raises(SystemExit):
cmd.ValidateOptions(opts, args)
def test_json_rejects_overview() -> None:
"""--format=json --overview should be rejected."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--format=json", "--overview"])
with pytest.raises(SystemExit):
cmd.ValidateOptions(opts, args)
def test_json_disables_pager() -> None:
"""--format=json should disable the pager."""
cmd = _get_cmd()
opts, _ = cmd.OptionParser.parse_args(["--format=json"])
assert not cmd.WantPager(opts)
def test_text_enables_pager() -> None:
"""Default text format should enable the pager."""
cmd = _get_cmd()
opts, _ = cmd.OptionParser.parse_args([])
assert cmd.WantPager(opts)
def test_get_project_data_uses_head_revision() -> None:
"""_getProjectData should use GetHeadRevisionId if available."""
cmd = _get_cmd()
project = mock.MagicMock()
project.name = "foo"
project.worktree = "/path/to/foo"
project.revisionExpr = "refs/heads/main"
project.GetBranches.return_value = []
# GetHeadRevisionId() returns a SHA, it should be used.
project.GetHeadRevisionId.return_value = "head_sha_12345"
project.GetRevisionId.return_value = "manifest_sha_54321"
data = cmd._getProjectData(project)
assert data["current_revision"] == "head_sha_12345"
project.GetHeadRevisionId.assert_called_once()
# GetHeadRevisionId() is None, fall back to GetRevisionId().
project.GetHeadRevisionId.reset_mock()
project.GetHeadRevisionId.return_value = None
data = cmd._getProjectData(project)
assert data["current_revision"] == "manifest_sha_54321"
def test_json_with_projects(capsys) -> None:
"""--format=json should emit project data."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--format=json"])
opts.jobs = 1 # To avoid multiprocessing pickle issues with mocks
project = mock.MagicMock()
project.name = "foo"
project.worktree = "/path/to/foo"
project.revisionExpr = "refs/heads/main"
project.GetBranches.return_value = {"branch1": mock.MagicMock()}
project.GetHeadRevisionId.return_value = "head_sha_12345"
project.CurrentBranch = "branch1"
cmd.GetProjects = mock.MagicMock(return_value=[project])
cmd.Execute(opts, args)
data = json.loads(capsys.readouterr().out)
assert "projects" in data
assert len(data["projects"]) == 1
project_data = data["projects"][0]
assert project_data["name"] == "foo"
assert project_data["mount_path"] == "/path/to/foo"
assert project_data["current_revision"] == "head_sha_12345"
assert project_data["manifest_revision"] == "refs/heads/main"
assert project_data["local_branches"] == ["branch1"]
assert project_data["current_branch"] == "branch1"
+50
View File
@@ -0,0 +1,50 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the subcmds/rebase.py module."""
from unittest import mock
from error import GitError
from subcmds import rebase
def test_resolve_onto_manifest_success() -> None:
"""Test _ResolveOntoManifest when ToLocal succeeds."""
project = mock.MagicMock()
project.revisionExpr = "main"
remote = mock.MagicMock()
remote.ToLocal.return_value = "refs/remotes/goog/main"
project.GetRemote.return_value = remote
res = rebase._ResolveOntoManifest(project)
assert res == "refs/remotes/goog/main"
project.GetRemote.assert_called_once()
remote.ToLocal.assert_called_once_with("main")
def test_resolve_onto_manifest_fallback() -> None:
"""Test _ResolveOntoManifest when ToLocal raises GitError."""
project = mock.MagicMock()
project.revisionExpr = "main"
remote = mock.MagicMock()
remote.ToLocal.side_effect = GitError("Failed to resolve")
project.GetRemote.return_value = remote
res = rebase._ResolveOntoManifest(project)
assert res == "main"
project.GetRemote.assert_called_once()
remote.ToLocal.assert_called_once_with("main")
+453
View File
@@ -0,0 +1,453 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the status subcmd."""
import contextlib
import io
import os
from pathlib import Path
import subprocess
from typing import List, Tuple
from unittest import mock
import pytest
import utils_for_test
import manifest_xml
import subcmds
@pytest.fixture
def repo_client_checkout(
tmp_path: Path,
) -> Tuple[Path, manifest_xml.XmlManifest]:
"""Create a basic repo client checkout for status tests."""
# Create in a subdir to avoid noise (like the repo_trace file).
topdir = tmp_path / "client_checkout"
repodir = topdir / ".repo"
manifest_dir = repodir / "manifests"
manifest_file = repodir / manifest_xml.MANIFEST_FILE_NAME
repodir.mkdir(parents=True)
manifest_dir.mkdir()
gitdir = repodir / "manifests.git"
gitdir.mkdir()
(gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
verbose = false
"""
)
_init_temp_git_tree(manifest_dir)
manifest_file.write_text(
"""
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="proj" path="src/proj" />
</manifest>
""",
encoding="utf-8",
)
(repodir / "projects" / "src" / "proj.git").mkdir(parents=True)
(repodir / "project-objects" / "proj.git").mkdir(parents=True)
worktree = topdir / "src" / "proj"
worktree.parent.mkdir(parents=True, exist_ok=True)
_init_temp_git_tree(worktree)
manifest = manifest_xml.XmlManifest(str(repodir), str(manifest_file))
return topdir, manifest
def _init_temp_git_tree(git_dir: Path) -> None:
"""Create a new git checkout with an initial commit for testing."""
utils_for_test.init_git_tree(git_dir)
(git_dir / "README").write_text("init")
subprocess.check_call(["git", "add", "README"], cwd=git_dir)
subprocess.check_call(["git", "commit", "-q", "-m", "init"], cwd=git_dir)
def _run_status(manifest: manifest_xml.XmlManifest, argv: List[str]) -> None:
"""Run the status subcommand with parsed options against a test manifest."""
cmd = subcmds.status.Status()
cmd.manifest = manifest
cmd.client = mock.MagicMock(globalConfig=manifest.globalConfig)
opts, args = cmd.OptionParser.parse_args(argv + ["--jobs=1"])
cmd.CommonValidateOptions(opts, args)
cmd.Execute(opts, args)
def _status_lines(output: str) -> List[str]:
"""Normalize path separators and split command output into lines."""
return output.replace(os.sep, "/").splitlines()
def _assert_project_header(line: str, project_path: str, branch: str) -> None:
"""Assert a status project header line for a project and branch."""
expected = f"project {(project_path + '/ '):<40}branch {branch}"
assert line == expected
def _assert_project_header_with_ahead_behind(
line: str,
project_path: str,
branch: str,
ahead_behind: str,
) -> None:
"""Assert a status project header line includes ahead/behind info."""
suffix = f"branch {branch}{ahead_behind}"
expected = f"project {(project_path + '/ '):<40}{suffix}"
assert line == expected
def _assert_orphan_block(lines: List[str], expected: List[str]) -> None:
"""Assert orphan block header and entries, independent of entry ordering."""
assert lines
assert lines[0] == ("Objects not within a project (orphans)")
orphan_lines = lines[1:]
assert len(orphan_lines) == len(expected)
assert sorted(orphan_lines) == sorted(expected)
def test_orphans_basic(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify -o output includes project header and orphan block."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / "src" / "orphan_dir").mkdir(parents=True)
(topdir / "orphan.txt").write_text("data")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, ["-o"])
lines = _status_lines(stdout.getvalue())
_assert_project_header(lines[0], project_path, "main")
_assert_orphan_block(
lines[1:],
[
" --\torphan.txt",
" --\tsrc/orphan_dir/",
],
)
def test_empty_status_without_orphans(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify clean status without -o prints only the project header line."""
_, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, "main")
def test_status_without_orphans(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify modified tracked file appears in status output without -o."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / project_path / "README").write_text("updated")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 2
_assert_project_header(lines[0], project_path, "main")
assert lines[1] == " -m\tREADME"
def test_status_with_orphans_and_modified_file(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify modified-file status plus orphan block."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / project_path / "README").write_text("updated")
(topdir / "src" / "orphan_dir").mkdir(parents=True)
(topdir / "orphan.txt").write_text("data")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, ["-o"])
lines = _status_lines(stdout.getvalue())
_assert_project_header(lines[0], project_path, "main")
assert lines[1] == " -m\tREADME"
_assert_orphan_block(
lines[2:],
[
" --\torphan.txt",
" --\tsrc/orphan_dir/",
],
)
def test_empty_status_after_start_shows_started_branch(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify status shows the started branch name when the tree is clean."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
started_branch = "topic/test-status-branch"
subprocess.check_call(
["git", "checkout", "-q", "-b", started_branch], cwd=project_worktree
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, started_branch)
def _setup_remote_tracking_branch(
manifest: manifest_xml.XmlManifest,
branch_name: str,
) -> None:
"""Create a branch tracking a remote ref, like ``repo start``.
In a real repo checkout, ``repo start`` creates a branch that
tracks a remote tracking ref (e.g. refs/remotes/origin/main).
This sets up the same config in both the worktree and the
project gitdir (where repo reads its config from).
"""
proj = list(manifest.paths.values())[0]
worktree = Path(proj.worktree)
proj_gitdir = Path(proj.gitdir)
# Create the remote tracking ref in the worktree.
subprocess.check_call(
["git", "update-ref", "refs/remotes/origin/main", "main"],
cwd=worktree,
)
# Create the new branch from main in the worktree.
subprocess.check_call(
["git", "checkout", "-q", "-b", branch_name, "main"],
cwd=worktree,
)
# Write remote and branch config into the *project gitdir* config,
# which is where repo's Project.config reads from.
cfg = str(proj_gitdir / "config")
subprocess.check_call(
[
"git",
"config",
"-f",
cfg,
"remote.origin.url",
"http://localhost/fake",
],
)
subprocess.check_call(
[
"git",
"config",
"-f",
cfg,
"remote.origin.fetch",
"+refs/heads/*:refs/remotes/origin/*",
],
)
subprocess.check_call(
["git", "config", "-f", cfg, f"branch.{branch_name}.remote", "origin"],
)
subprocess.check_call(
[
"git",
"config",
"-f",
cfg,
f"branch.{branch_name}.merge",
"refs/heads/main",
],
)
def test_status_branch_ahead_of_upstream(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify status shows [ahead N] for local commits."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
_setup_remote_tracking_branch(manifest, "feature")
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "c1"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "c2"],
cwd=project_worktree,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header_with_ahead_behind(
lines[0], project_path, "feature", " [ahead 2]"
)
def test_status_branch_behind_upstream(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify status shows [behind N] when upstream is ahead."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
_setup_remote_tracking_branch(manifest, "feature")
# Advance the remote tracking ref past the feature branch.
subprocess.check_call(
["git", "checkout", "-q", "main"], cwd=project_worktree
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "upstream"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "update-ref", "refs/remotes/origin/main", "main"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "checkout", "-q", "feature"],
cwd=project_worktree,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header_with_ahead_behind(
lines[0], project_path, "feature", " [behind 1]"
)
def test_status_branch_ahead_and_behind(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify [ahead N, behind M] when branch has diverged."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
_setup_remote_tracking_branch(manifest, "feature")
# Add a local commit on feature.
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "local"],
cwd=project_worktree,
)
# Advance the remote tracking ref independently.
subprocess.check_call(
["git", "checkout", "-q", "main"], cwd=project_worktree
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "upstream"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "update-ref", "refs/remotes/origin/main", "main"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "checkout", "-q", "feature"],
cwd=project_worktree,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header_with_ahead_behind(
lines[0],
project_path,
"feature",
" [ahead 1, behind 1]",
)
def test_status_branch_no_tracking_no_ahead_behind(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify no ahead/behind when branch has no upstream."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
subprocess.check_call(
[
"git",
"checkout",
"-q",
"-b",
"orphan-branch",
"--no-track",
"main",
],
cwd=project_worktree,
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "c1"],
cwd=project_worktree,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, "orphan-branch")
def test_status_branch_synced_no_ahead_behind(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify no ahead/behind when branch is fully synced."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
_setup_remote_tracking_branch(manifest, "synced")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, "synced")
+664
View File
@@ -13,6 +13,7 @@
# limitations under the License.
"""Unittests for the subcmds/sync.py module."""
import json
import os
import shutil
import tempfile
@@ -339,6 +340,7 @@ class FakeProject:
self.name = name or relpath
self.objdir = objdir or relpath
self.worktree = relpath
self.parent = None
self.use_git_worktrees = False
self.UseAlternates = False
@@ -396,6 +398,65 @@ class SafeCheckoutOrder(unittest.TestCase):
],
)
def test_sibling_submodules_with_shared_parent_are_serialized(self):
parent = mock.Mock(worktree="/worktree/parent")
other_parent = mock.Mock(worktree="/worktree/other")
p_parent = FakeProject("parent")
p_other = FakeProject("other")
p_parent_sub1 = FakeProject("parent/sub1")
p_parent_sub1.parent = parent
p_parent_sub2 = FakeProject("parent/sub2")
p_parent_sub2.parent = parent
p_other_sub = FakeProject("other/sub")
p_other_sub.parent = other_parent
out = sync._SafeCheckoutOrder(
[p_parent_sub2, p_other_sub, p_parent, p_parent_sub1, p_other]
)
self.assertEqual(
out,
[
[p_other, p_parent],
[p_other_sub, p_parent_sub1],
[p_parent_sub2],
],
)
def test_nested_submodules_respect_delayed_parent_level(self):
parent = mock.Mock(worktree="/worktree/parent")
sub1 = mock.Mock(worktree="/worktree/parent/sub1")
sub2 = mock.Mock(worktree="/worktree/parent/sub2")
p_parent = FakeProject("parent")
p_parent_sub1 = FakeProject("parent/sub1")
p_parent_sub1.parent = parent
p_parent_sub1_nested = FakeProject("parent/sub1/nested")
p_parent_sub1_nested.parent = sub1
p_parent_sub2 = FakeProject("parent/sub2")
p_parent_sub2.parent = parent
p_parent_sub2_nested = FakeProject("parent/sub2/nested")
p_parent_sub2_nested.parent = sub2
out = sync._SafeCheckoutOrder(
[
p_parent_sub2_nested,
p_parent_sub2,
p_parent_sub1_nested,
p_parent,
p_parent_sub1,
]
)
self.assertEqual(
out,
[
[p_parent],
[p_parent_sub1],
[p_parent_sub1_nested, p_parent_sub2],
[p_parent_sub2_nested],
],
)
class Chunksize(unittest.TestCase):
"""Tests for _chunksize."""
@@ -477,6 +538,56 @@ class GetPreciousObjectsState(unittest.TestCase):
)
class KeyboardInterruptTest(unittest.TestCase):
"""Tests for KeyboardInterrupt handling in Sync operations."""
def setUp(self):
self.project = mock.MagicMock(name="project")
self.project.name = "project"
self.project.relpath = "proj"
self.project.manifest.IsArchive = False
self.opt = mock.Mock()
self.opt.quiet = True
self.opt.verbose = False
self.opt.tags = False
self.sync_dict = {}
self.get_parallel_context_mock = {
"projects": [self.project],
"sync_dict": self.sync_dict,
"ssh_proxy": None,
}
@mock.patch("subcmds.sync.Sync.is_multiprocessing_active")
def test_fetch_one_keyboard_interrupt_main_process(self, mock_is_active):
"""Test that _FetchOne re-raises KeyboardInterrupt if not worker."""
mock_is_active.return_value = False
self.project.Sync_NetworkHalf.side_effect = KeyboardInterrupt()
with mock.patch.object(
sync.Sync,
"get_parallel_context",
return_value=self.get_parallel_context_mock,
):
with self.assertRaises(KeyboardInterrupt):
sync.Sync._FetchOne(self.opt, 0)
@mock.patch("subcmds.sync.Sync.is_multiprocessing_active")
def test_fetch_one_keyboard_interrupt_worker_process(self, mock_is_active):
"""Test that _FetchOne suppresses KeyboardInterrupt in workers."""
mock_is_active.return_value = True
self.project.Sync_NetworkHalf.side_effect = KeyboardInterrupt()
with mock.patch.object(
sync.Sync,
"get_parallel_context",
return_value=self.get_parallel_context_mock,
):
result = sync.Sync._FetchOne(self.opt, 0)
self.assertFalse(result.success)
class CheckForBloatedProjects(unittest.TestCase):
"""Tests for Sync._CheckForBloatedProjects."""
@@ -489,6 +600,7 @@ class CheckForBloatedProjects(unittest.TestCase):
self.project.name = "project"
self.project.Exists = True
self.project.worktree = "worktree"
self.project.stateless_prune_needed = False
self.cmd.git_event_log = mock.MagicMock()
self.cmd._bloated_projects = []
@@ -530,6 +642,21 @@ class CheckForBloatedProjects(unittest.TestCase):
self.assertEqual(self.cmd._bloated_projects, ["project"])
@mock.patch("subcmds.sync.git_require")
@mock.patch("subcmds.sync.Progress")
def test_stateless_prune_excluded(self, mock_progress, mock_git_require):
"""Test that projects pruned for stateless sync are excluded."""
mock_git_require.return_value = True
self.project.stateless_prune_needed = True
self.cmd.ExecuteInParallel = mock.Mock()
with mock.patch.object(self.cmd, "ParallelContext"):
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertFalse(self.cmd.ExecuteInParallel.called)
self.assertEqual(self.cmd._bloated_projects, [])
class GCProjectsTest(unittest.TestCase):
"""Tests for Sync._GCProjects."""
@@ -1079,3 +1206,540 @@ class InterleavedSyncTest(unittest.TestCase):
self.assertTrue(result.checkout_success)
project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_not_called()
class UpdateCopyLinkfileListTest(unittest.TestCase):
"""Tests for Sync.UpdateCopyLinkfileList."""
def setUp(self):
self.tempdirobj = tempfile.TemporaryDirectory(prefix="repo_tests")
self.topdir = self.tempdirobj.name
self.repodir = os.path.join(self.topdir, ".repo")
os.makedirs(self.repodir)
manifest = mock.MagicMock()
manifest.subdir = self.repodir
self.manifest = manifest
git_event_log = mock.MagicMock(ErrorEvent=mock.Mock(return_value=None))
self.cmd = sync.Sync(
manifest=manifest,
outer_client=mock.MagicMock(),
git_event_log=git_event_log,
)
self.cmd.client = mock.MagicMock(topdir=self.topdir)
def tearDown(self):
self.tempdirobj.cleanup()
def _write_copylinkfile_json(self, data: dict) -> None:
path = os.path.join(self.repodir, "copy-link-files.json")
with open(path, "w") as f:
json.dump(data, f)
def _setup_projects(self, linkfile_dests: list) -> None:
project = mock.MagicMock()
project.linkfiles = [mock.MagicMock(dest=d) for d in linkfile_dests]
project.copyfiles = []
mock.patch.object(
self.cmd, "GetProjects", return_value=[project]
).start()
def test_removes_old_symlink_dest(self):
"""Old linkfile dests that are symlinks should be removed."""
old_dest = os.path.join(self.topdir, "old-link")
os.symlink("target", old_dest)
self._write_copylinkfile_json(
{"linkfile": ["old-link"], "copyfile": []}
)
self._setup_projects([])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(old_dest))
def test_does_not_delete_through_new_symlink(self):
"""Old dests that resolve through a new symlink must not delete files.
When the manifest changes from individual linkfiles inside a directory
to a single directory linkfile, and _CopyAndLinkFiles has already
created the symlink (interleaved mode), cleanup must not follow the
symlink and delete real project files.
"""
project_dir = os.path.join(self.topdir, "vendor", "tools", "llms")
os.makedirs(os.path.join(project_dir, "dot-llms", "rules"))
os.makedirs(os.path.join(project_dir, "dot-llms", "skills"))
with open(
os.path.join(project_dir, "dot-llms", "rules", "basics.md"), "w"
) as f:
f.write("# basics")
with open(
os.path.join(project_dir, "dot-llms", "skills", "repo.md"), "w"
) as f:
f.write("# repo")
# Simulate interleaved mode: .llms -> vendor/tools/llms/dot-llms.
llms_link = os.path.join(self.topdir, ".llms")
os.symlink("vendor/tools/llms/dot-llms", llms_link)
self._write_copylinkfile_json(
{"linkfile": [".llms/rules", ".llms/skills"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
# Real project files must still exist.
self.assertTrue(
os.path.exists(
os.path.join(project_dir, "dot-llms", "rules", "basics.md")
)
)
self.assertTrue(
os.path.exists(
os.path.join(project_dir, "dot-llms", "skills", "repo.md")
),
)
self.assertTrue(os.path.islink(llms_link))
def test_cleans_up_empty_parent_dirs(self):
"""After removing old dests, empty parent directories are removed."""
llms_dir = os.path.join(self.topdir, ".llms")
os.makedirs(llms_dir)
os.symlink(
"../vendor/tools/llms/rules", os.path.join(llms_dir, "rules")
)
os.symlink(
"../vendor/tools/llms/skills",
os.path.join(llms_dir, "skills"),
)
self._write_copylinkfile_json(
{"linkfile": [".llms/rules", ".llms/skills"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "rules")))
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "skills")))
# Parent directory should be removed since it's now empty.
self.assertFalse(os.path.exists(llms_dir))
def test_preserves_nonempty_parent_dirs(self):
"""Non-empty parent directories are preserved after old dest removal."""
llms_dir = os.path.join(self.topdir, ".llms")
os.makedirs(llms_dir)
os.symlink(
"../vendor/tools/llms/rules", os.path.join(llms_dir, "rules")
)
with open(os.path.join(llms_dir, "my-notes.txt"), "w") as f:
f.write("user content")
self._write_copylinkfile_json(
{"linkfile": [".llms/rules"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "rules")))
self.assertTrue(os.path.exists(os.path.join(llms_dir, "my-notes.txt")))
self.assertTrue(os.path.isdir(llms_dir))
class SyncToSuperprojectRevTests(unittest.TestCase):
"""Tests for Sync._SyncToSuperprojectRev."""
def setUp(self):
self.repodir = tempfile.mkdtemp(".repo")
self.manifest = mock.MagicMock(repodir=self.repodir)
self.manifest.superproject = mock.MagicMock()
self.manifest.path_prefix = ""
self.mp = mock.MagicMock()
self.cmd = sync.Sync(manifest=self.manifest)
self.cmd.outer_manifest = self.manifest
self.opt = mock.Mock()
self.opt.verbose = False
self.opt.superproject_revision = "deadbeef"
self.opt.mp_update = True
self.errors = []
def tearDown(self):
shutil.rmtree(self.repodir)
@mock.patch("subcmds.sync.GitCommand")
def test_successful_sync(self, mock_git_command):
"""Test successful sync to superproject rev."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 0
mock_git.stdout = "proj branch manifest_commit_hash\n"
mock_git_command.return_value = mock_git
with mock.patch.object(
self.cmd, "_UpdateManifestProject"
) as mock_update:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
mock_superproject.SetRevisionId.assert_called_with("deadbeef")
mock_superproject.Sync.assert_called_once()
mock_git_command.assert_called_once()
self.mp.SetRevision.assert_called_with("manifest_commit_hash")
mock_update.assert_called_once()
self.assertEqual(self.errors, [])
@mock.patch("subcmds.sync.GitCommand")
def test_parse_error(self, mock_git_command):
"""Test error when .supermanifest cannot be parsed."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 0
# Invalid format (not 3 parts)
mock_git.stdout = "invalid_content\n"
mock_git_command.return_value = mock_git
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("could not parse .supermanifest", str(e.exception))
@mock.patch("subcmds.sync.GitCommand")
def test_read_error(self, mock_git_command):
"""Test error when reading .supermanifest fails."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 1
mock_git.stderr = "git error"
mock_git_command.return_value = mock_git
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("failed to read .supermanifest", str(e.exception))
def test_no_superproject(self):
"""Test error when superproject is not defined."""
self.manifest.superproject = None
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("superproject not defined", str(e.exception))
@mock.patch("subcmds.sync.GitCommand")
def test_sync_failure(self, mock_git_command):
"""Test error when superproject sync fails."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=False)
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("failed to sync superproject", str(e.exception))
class UpdateAllManifestProjectsTests(unittest.TestCase):
"""Tests for Sync._UpdateAllManifestProjects."""
def setUp(self):
self.repodir = tempfile.mkdtemp(".repo")
self.manifest = mock.MagicMock(repodir=self.repodir)
self.manifest.superproject = mock.MagicMock()
self.manifest.path_prefix = ""
self.manifest.standalone_manifest_url = None
self.manifest.submanifests = {}
self.mp = mock.MagicMock()
self.mp.manifest = self.manifest
self.mp.standalone_manifest_url = None
self.cmd = sync.Sync(manifest=self.manifest)
self.cmd.outer_manifest = self.manifest
self.opt = mock.Mock()
self.opt.verbose = False
self.opt.superproject_revision = None
self.opt.mp_update = True
self.errors = []
def tearDown(self):
shutil.rmtree(self.repodir)
def test_superproject_revision_outer_manifest(self):
"""Test that _SyncToSuperprojectRev is called for outer manifest."""
self.opt.superproject_revision = "deadbeef"
with mock.patch.object(
self.cmd, "_SyncToSuperprojectRev"
) as mock_sync_to_rev:
self.cmd._UpdateAllManifestProjects(
self.opt, self.mp, "name", self.errors
)
mock_sync_to_rev.assert_called_once_with(
self.opt, self.manifest, self.mp, "name", self.errors
)
def test_superproject_revision_submanifest(self):
"""Test that _SyncToSuperprojectRev is NOT called for submanifest."""
self.opt.superproject_revision = "deadbeef"
submanifest = mock.MagicMock()
submanifest.path_prefix = "sub/"
submanifest.standalone_manifest_url = None
self.mp.manifest = submanifest
with mock.patch.object(
self.cmd, "_SyncToSuperprojectRev"
) as mock_sync_to_rev:
with mock.patch.object(
self.cmd, "_UpdateManifestProject"
) as mock_update_manifest:
self.cmd._UpdateAllManifestProjects(
self.opt, self.mp, "name", self.errors
)
mock_sync_to_rev.assert_not_called()
mock_update_manifest.assert_called_once()
class TestSmartSyncSetupRemoteHelper(unittest.TestCase):
"""Tests for _SmartSyncSetup with remote helpers."""
def setUp(self):
self.cmd = sync.Sync()
self.opt = mock.MagicMock()
self.opt.quiet = False
self.opt.smart_sync = True
self.manifest = mock.MagicMock()
self.smart_sync_manifest_path = "/fake/path/to/manifest.xml"
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
@mock.patch("xmlrpc.client.Server")
@mock.patch("subcmds.sync.PersistentTransport")
def test_smart_sync_setup_with_helper(
self, mock_transport_class, mock_server_class, mock_popen, mock_which
):
"""Test _SmartSyncSetup when a helper is present and succeeds."""
import subprocess
self.manifest.manifest_server = (
"persistent-https://android-smartsync.corp.google.com/"
"manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
# Mock subprocess to return a JSON with status ok and proxy address
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
'{"status":"ok","message":"http://127.0.0.1:999"}\n',
"",
)
mock_process.returncode = 0
mock_popen.return_value = mock_process
# Mock XML-RPC server call
mock_server = mock.MagicMock()
mock_server.GetApprovedManifest.return_value = [
True,
"<manifest></manifest>",
]
mock_server_class.return_value = mock_server
# Mock manifest project branch
self.cmd._GetBranch = mock.MagicMock(return_value="main")
self.cmd._ReloadManifest = mock.MagicMock()
# Mock open to avoid writing to disk
with mock.patch("builtins.open", mock.mock_open()):
manifest_name = self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
# Assertions
mock_which.assert_called_once_with("repo-remote-sso")
mock_popen.assert_called_once_with(
["repo-remote-sso", self.manifest.manifest_server],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
# Verify transport was created with the proxy returned by helper (with
# http:// prepended)
mock_transport_class.assert_called_once_with(
self.manifest.manifest_server, proxy="http://127.0.0.1:999"
)
# Verify Server was created with the same URL, with persistent- stripped
mock_server_class.assert_called_once_with(
"https://android-smartsync.corp.google.com/manifestserver",
transport=mock_transport_class.return_value,
)
self.assertEqual(manifest_name, "manifest.xml")
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
def test_smart_sync_setup_helper_error(self, mock_popen, mock_which):
"""Test _SmartSyncSetup when helper returns an error status."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
# Mock subprocess to return a JSON with status error
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
'{"status":"error","message":"uplink-helper failed"}\n',
"",
)
mock_process.returncode = 0
mock_popen.return_value = mock_process
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"helper repo-remote-sso returned error: uplink-helper failed",
str(context.exception),
)
@mock.patch("shutil.which")
def test_smart_sync_setup_missing_declared_helper(self, mock_which):
"""Test _SmartSyncSetup when helper declared in manifest is missing."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = None
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"helper binary 'repo-remote-sso' declared in manifest was not "
"found",
str(context.exception),
)
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
def test_smart_sync_setup_helper_exit_code_error(
self, mock_popen, mock_which
):
"""Test _SmartSyncSetup when helper exits with non-zero and stderr."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
# Mock subprocess: exit code 1, stderr, and no JSON on stdout
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
"",
"internal binary error occurred\n",
)
mock_process.returncode = 1
mock_popen.return_value = mock_process
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"helper repo-remote-sso exited with exit code 1. "
"Stderr: internal binary error occurred",
str(context.exception),
)
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
def test_smart_sync_setup_helper_json_decode_error_with_stderr(
self, mock_popen, mock_which
):
"""Test _SmartSyncSetup when helper returns invalid JSON and stderr."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
"not a json",
"some warning messages\n",
)
mock_process.returncode = 0
mock_popen.return_value = mock_process
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"failed to parse JSON from helper repo-remote-sso",
str(context.exception),
)
self.assertIn(
"Stderr was: some warning messages",
str(context.exception),
)
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
def test_smart_sync_setup_helper_error_with_stderr(
self, mock_popen, mock_which
):
"""Test _SmartSyncSetup when helper returns error status and stderr."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
'{"status":"error","message":"uplink-helper failed"}\n',
"debugging logs\n",
)
mock_process.returncode = 0
mock_popen.return_value = mock_process
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"helper repo-remote-sso returned error: uplink-helper failed",
str(context.exception),
)
self.assertIn(
"Stderr was: debugging logs",
str(context.exception),
)
-23
View File
@@ -1,23 +0,0 @@
# Copyright (C) 2022 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the update_manpages module."""
from release import update_manpages
def test_replace_regex() -> None:
"""Check that replace_regex works."""
data = "\n\033[1mSummary\033[m\n"
assert update_manpages.replace_regex(data) == "\nSummary\n"