Compare commits

...
59 Commits
Author SHA1 Message Date
Rahul Yadav d27d6829a8 project: check upstream ref ancestry for non-shallow clones
In _CheckForImmutableRevision, commit d9cc0a15 restricted upstream ref
validation strictly to superprojects (if use_superproject) to prevent
shallow clones with an upstream attribute from failing ancestry checks
and falling back to full clones.

However, restricting this check exclusively to superprojects broke
non-shallow projects with pinned immutable revisions (such as in Smart
Sync, `repo sync -t <BUILD_ID>`, or pinned manifests): When the target
commit already exists in the local Git object store (for instance,
prefetched into refs/prefetch/ by a background daemon or via shared
object dirs), _CheckForImmutableRevision returned True without verifying
that the local tracking ref (refs/remotes/<remote>/<upstream>) is
present and reaches the revision. As a result, _RemoteFetch skipped
fetching the upstream branch, leaving the local tracking ref stale or
missing. Subsequent `repo start` branches tracking that remote branch
diverged, causing `repo upload` to attempt uploading all intermediate
commits between the stale tracking ref and HEAD.

Restore upstream ancestry validation in _CheckForImmutableRevision for
non-shallow projects when upstream is specified. Also pass the sync
depth into _CheckForImmutableRevision call sites so shallow checkouts
continue to skip upstream verification and avoid triggering un-shallow
fallbacks.

Test: PYTHONPATH=. pytest tests/test_project.py
Change-Id: Ib5bdf41810bb06c8ed053447209ddc2e488f3913
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/624361
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-03 02:15:23 -07:00
Gavin Mak 0ea57e2eed project: gate the default-branch query by Git version
GIT_DEFAULT_BRANCH support was added to git var in Git 2.35. Skip that
guaranteed failure on older supported clients and read
init.defaultBranch directly, reducing their fallback from two processes
to one.

Bug: 553599402
Change-Id: I079e9e6057b56afa41b685317a106fc9cdbad8a2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623907
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-02 17:46:23 -07:00
Gavin Mak ba8ddf396c git_superproject: resolve one negotiation ref directly
The superproject only needs one branch tip or HEAD OID. Resolve that ref
with one quiet rev-parse instead of constructing GitRefs, which loads
the entire namespace and resolves HEAD in two or three processes on
older clients.

Bug: 553599402
Change-Id: I162d1133ebd1941e9de5aca43929fd5af17386ef
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623906
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:46:01 -07:00
Gavin Mak d88ce8d952 upload: reuse the worker current-branch result
Carry the current branch through _GatherOne so the parent can report a
failed current-branch upload without resolving HEAD a second time.

Bug: 553599402
Change-Id: I3fc54be6c10c30470e14f792ff68c0a396519327
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623905
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-02 17:45:55 -07:00
Gavin Mak 5e8d2a6e3a version: read describe and date in one Git call
Git 2.32 added the %(describe) pretty placeholder. Use one log format to
retrieve both repo's describe string and commit date on newer clients,
with the existing describe-plus-log path retained for older Git and
untagged output.

Bug: 553599402
Change-Id: I108346030677b2b90ef8bfb5acea1c45be2c85f2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623904
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-02 17:45:49 -07:00
Gavin Mak e59c9cde99 cherry_pick: resolve and read commits in one process
Send a typed revision expression to cat-file --batch over stdin and
parse its object header and exact byte length. This safely resolves the
commit OID and retrieves its raw message in one Git process instead of
separate rev-parse and cat-file calls.

Bug: 553599402
Change-Id: I764894323ec28e134f2523bd1d5694738b6b52d0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623903
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:43:55 -07:00
Gavin Mak 83428a9b26 rebase: delegate automatic stashing to Git
Git --autostash predates repo's minimum supported client. Pass it
directly to rebase instead of probing the index and separately running
stash and stash pop, reducing the dirty path from four Git processes to
one and correctly covering staged-only changes.

Bug: 553599402
Change-Id: I0d6d59b17996576310498e9378e49380d355236e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623902
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:43:48 -07:00
Gavin Mak 948abc85bc git_config: recognize only complete object IDs
Full Git object IDs are exactly 40 hexadecimal digits for SHA-1 or 64
for SHA-256. Stop treating every intermediate length as immutable, which
could bypass normal ref resolution for invalid revision strings.

Bug: 553599402
Change-Id: I41b4ce6b2bfe2a2d8b351c7f040ec0cf469b474b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623901
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:41:05 -07:00
kimhappy c63a2f92fa sync: tell apart same-path submanifest projects
Interleaved sync remembers which projects are done and which ones are
still pending by Project.relpath, and lists failing projects by it as
well. That path is relative to the project's own (sub)manifest though,
so once several manifests are synced together two projects can share
it, and the sets no longer tell them apart.

The damage is done when a project turns up in a later pass, e.g. a
submodule that is only derived once its parent has been synced: its
path is already recorded as finished, so it is left out of every pass
that follows and never synced, while sync still reports success. With
an outer manifest holding

  <project name="a" path="a" sync-s="true"/>

where a carries a submodule at b, and a submanifest at sub/ holding

  <project name="sub-ab" path="a/b"/>

a fresh `repo sync` checks out sub/a/b but never a/b. Failing projects
are reported under the same ambiguous path, and the stall detection
merges them too.

Use RelPath(local=opt.this_manifest_only) instead, which is unique
within the set of projects being synced and is what GetProjects() and
the other subcommands already use.

Spotted during the review of I30395b8a16a9154f60972e1f408a066af64ba77c.

Change-Id: I5fa694b968774667be6060d9722cb57acbd2b579
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622641
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-01 19:13:52 -07:00
Gavin Mak 0039e39000 project: carry current state from the ref snapshot
Attach the current marker computed by GetBranches to uploadable and
prunable branches. Overview and prune reporting can then reuse that
result instead of resolving HEAD once per displayed branch.

Bug: 553599402
Change-Id: Ie3e23d08d0caf66c47cc12d7295ec4b6a834560c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623184
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 13:26:22 -07:00
Gavin Mak 5f378458d2 project: reuse ref OIDs in branch maintenance
Load refs before resolving the current branch, compare cached HEAD and
manifest OIDs instead of walking their symmetric difference, and reuse
the snapshot when preserving bare HEAD. Apply the same HEAD reuse to
abandon and MetaProject checks, and bound the latter revision walk to
one commit.

Bug: 553599402
Change-Id: I715e65dcfa02c0df4aaa08da5b7bfbb04ca17725
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623183
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 13:25:40 -07:00
Gavin Mak d27034bf62 project: centralize safe branch and commit resolution
Resolve commit-ish values through one typed helper using rev-parse
--verify --quiet and, on Git 2.30+, --end-of-options. Reject option-like
revisions on older clients, reuse the helper for project and manifest
resolution, and validate user branch names with check-ref-format
--branch.

Bug: 553599402
Change-Id: I2dda49ff3e781cec14d3b84263a8dace06b4073f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623182
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-31 12:34:53 -07:00
Gavin Mak 6541729a18 project: reuse loaded refs when resolving HEAD
When the regular checkout has already loaded GitRefs, derive its current
branch and HEAD object from that snapshot instead of spawning
symbolic-ref or rev-parse again. Keep linked Git worktrees on their
worktree-specific HEAD path because the shared repository HEAD is not
authoritative for them.

Bug: 543851900
Bug: 553599402
Change-Id: Iec456be5f3c279865b4165a75d1c6df36e331dbc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623181
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 12:33:39 -07:00
Gavin Mak b85e76a86a git_refs: load HEAD with the ref snapshot on Git 2.45
Use for-each-ref --include-root-refs to resolve attached and detached
HEAD in the same process as refs/* on Git 2.45 and newer. Limit the
patterns to HEAD and refs so volatile pseudo-refs do not enter the
cache, and retain the existing fallback for older Git and unborn HEADs.

Bug: 543851900
Bug: 553599402
Change-Id: Ie091ca0757f21ec6b1e19d49e09d160e055ec639
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622981
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-08-31 11:57:39 -07:00
James Hawkins e5bbb5c9e6 upload: prune ref scans for untouched projects and specific branches
Benchmark (100 untouched projects):
* GetUploadableBranches: 0.985s -> 0.0004s (2,200x faster)
* repo upload (3,045 projects): saves ~2.5s of redundant ref scans

During `repo upload`, `GetUploadableBranches()` currently scans
`self._allrefs` across all projects in the workspace. In large
manifests (e.g. Android with 3,000+ projects), `self._allrefs`
executes `git for-each-ref`, `git symbolic-ref`, and full filesystem
mtime traversals on thousands of projects where no local branch was
ever started or modified.

Any branch that can be uploaded for review must have upstream tracking
configured via `[branch "..."]` sections in `.git/config`
(otherwise `branch.LocalMerge` is None and `GetUploadableBranch`
returns None).

This patch introduces early-pruning fast paths:
1. When no branch subsections exist in `.git/config`
   (or only 2-part keys like `branch.autosetupmerge`), return []
   immediately without touching `_allrefs`.
2. When a `selected_branch` is specified (e.g. `repo upload --br=...`),
   query `branch.LocalMerge`, `refs/heads/<branch>`, and
   `refs/published/<branch>` directly, skipping the whole-tree
   ref scan.
3. In `subcmds/upload.py`, optimize `_GetMergeBranch` to resolve the
   merge branch in-memory via `project.CurrentBranch` and
   `project.GetBranch()` instead of spawning two `GitCommand`
   processes.

Test: ./run_tests tests/test_project.py tests/test_subcmds_upload.py
Change-Id: I3c2c9d63a68a393d478e1fb50f548ec8b7d44639
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623021
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: James Hawkins <jhawkins@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
2026-08-28 16:07:01 -07:00
James Hawkins 4fe87617ff project: read HEAD directly in-memory to avoid subprocesses
Benchmark:
* repo upload frameworks/base: 2.84s -> 0.50s (-82.3%, 5.6x faster)
* repo upload (3,045 projects): 7.66s -> 5.24s (-31.6%, 2.42s saved)

From repo's inception through v2.56, GetHead() directly read the
`.git/HEAD` file in Python. In commit 52bab0ba ("project: Use git
rev-parse to read HEAD"), this was replaced with git subprocess calls
on the premise that git provides a dedicated command. However, in
large multi-project workspaces (such as Android with 3,000+ projects),
spawning thousands of git processes introduced severe latency
regressions during `repo upload` and `repo status`.

Furthermore, switching to subprocesses broke detached HEADs and
unborn branches (fixed in commits 7f7d70ef and 8c3585f3 by re-adding
the v2.56 file-reading logic as an error recovery fallback).

This patch restores fast in-memory reading as the primary path, while
adding modern defensive safeguards:
* Symbolic refs (`ref: refs/heads/...`): Strips whitespace and tabs
  and returns the ref directly in memory.
* Detached HEAD: Validates 40-char SHA-1 and 64-char SHA-256 commit
  hashes via git_config.IsId(), normalizing to lowercase.
* Symlinks: Detects filesystem symlinks via os.path.islink() and
  safely falls back to git symbolic-ref.
* Fallback: Catches (OSError, AssertionError) and transparently falls
  back to native git commands for reftables, unexpected layouts, or
  filesystem errors. Unifies recovery fallback parsing with the fast
  path (CRLF/tabs, lowercase hashes, and consistent RelPath errors).

In addition, this change substantially expands test coverage in
tests/test_project.py, adding comprehensive unit tests for symbolic
refs, whitespace/tabs, CRLF line endings, SHA-1, SHA-256, uppercase
hash normalization, symlinks, corrupted worktrees, and fallback
robustness.

Test: ./run_tests tests/test_project.py
Change-Id: Ib5c2530117c6939e4b9293feda81aa745c003c6a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623001
Commit-Queue: James Hawkins <jhawkins@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
2026-08-28 16:03:06 -07:00
kimhappy d7299422ae sync: skip submodules removed from their parent
The submodules to sync are derived from their parent before it is
fetched, so one that the parent's new revision no longer holds a gitlink
for is still synced. Interleaved sync then fails to check it out,
because `git submodule init` no longer knows that path:

  error: Cannot checkout a/c
  error: pathspec '.../a/c' did not match any file(s) known to git

Report such submodules while the gitlinks are read again, and leave them
out of the fetch and of the checkout, so the usual project list update
removes them from the working tree. Phased sync already left them out of
the checkout, but still fetched them.

Nothing is reported as removed while the gitlinks of a parent cannot be
read, since a missing gitlink cannot be told apart from a fetch that did
not happen.

Bug: 550074864
Change-Id: Ia429b35c12758a68e4df73665e5512a201af658a
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621681
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:54 -07:00
kimhappy 3f087a8dd9 sync: fetch submodules after their parent repository
Phased sync fetches every project at once, so a submodule is fetched at
the gitlink that was read before its parent was fetched. With -c that
stale revision is already present, the fetch is skipped as an immutable
revision, and the checkout then fails on the revision the reloaded
manifest resolved:

  error: Cannot checkout a/b
  fatal: bad object 5556dad99da5df7e53303c5732aacc8782493e03

Fetch in batches, holding a submodule back until the project holding its
gitlink is fetched, and read the gitlinks again between batches. A
manifest without submodules keeps its single batch.

Bug: 550074864
Change-Id: I04c2eb1ca81ab25b4df2c03c5dc048d2e4ecdcb8
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621663
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:49 -07:00
kimhappy 3a6e25af75 sync: check out submodules at the parent's new revision
Interleaved sync processes projects in hierarchical levels, so a
submodule is only handled after the project holding its gitlink has been
fetched and checked out. Its revision, however, is still the one read
before that parent was fetched, so the submodule is synced to the
revision of the previous sync and only catches up on the next one.

Read the gitlinks again right before a level is dispatched. The parent
is done by then, so its submodules are fetched and checked out at the
revision the parent now points at. Sibling submodules are spread over
several levels, so the gitlinks read for one level are carried over to
the next ones and every project is only read once per pass.

Bug: 550074864
Change-Id: I30395b8a16a9154f60972e1f408a066af64ba77c
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621662
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:43 -07:00
Gavin Mak 41c2597509 project: don't use dest-branch or tags as SHA-1 upstream fallback
https://gerrit-review.googlesource.com/614762 resolves a fallback
upstream for a SHA-1-pinned project that has no explicit upstream, so
--current-branch can narrow the fetch to one branch instead of fetching
all heads.

It drew that fallback from dest-branch and from the manifest <default>
upstream, dest-branch, and revision. dest-branch and tags are not valid
fallback candidates:

* dest-branch is the review destination for upload, not a fetch
  source.
* Tags are meant to be immutable snapshots of a single tagged commit,
  and fetching it cannot retrieve an arbitrary pinned SHA-1.

Restrict the fallback to branch heads resolved from the manifest
upstream and revision defaults, and drop dest-branch entirely.

Bug: 541240657
Bug: 544041102
Change-Id: I89ae86891ee166f4c340553ac5a9068efcc18461
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622541
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 12:43:00 -07:00
Rahul Yadav e6ad708009 hooks: add --fix option to auto-apply hook fixes
Pass the --fix flag as a keyword argument "fix" to the hook main
function. This allows hooks (such as git-repohooks) to decouple
automated fix application from the -y/--yes flag so that -y can
answer yes to upload confirmation prompts without triggering file
mutations.

Companion change in git-repohooks:
https://gerrit-review.googlesource.com/c/git-repohooks/+/621761

Bug: 546510319
Test: python3 -m pytest tests/test_hooks.py

Change-Id: If0288d4791fc0a2aba6e88854e3aa81b0923b664
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/619281
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-08-27 03:28:58 -07:00
Ram Peri 09914bcab7 Add detailed upload context to Trace2 telemetry
This patch intercepts the output during a successful upload
execution to capture the generated CL URLs. It logs a dynamically
constructed "repo.uploadstate" data event to the active trace2
log containing:
- Uploaded CL URLs
- Target remote name
- Source branch
- Modified files

Test:
1. ./run_tests
2. pytest tests/test_project.py
3. Manual verification:
- Created a dummy branch `test_upload_branch` in a project with local file changes.
- Invoked repo upload passing an explicit trace output directory:
  `repo --git-trace2-event-log=/tmp/trace2out upload --dry-run --no-verify art`
- Verified the injected payload successfully appeared on disk within the generated log:
  ```json
  {"event":"data",..."key":"repo.uploadstate/cls","value":""}
  {"event":"data",..."key":"repo.uploadstate/remote","value":"goog"}
  {"event":"data",..."key":"repo.uploadstate/branch","value":"test_upload_branch"}
  {"event":"data",..."key":"repo.uploadstate/files","value":"dummy_file.txt"}
  ```
  (cls is correctly blank on --dry-run but populates on real HTTP pushes)

Bug: 543953499
Change-Id: I9c402a32d01d156d42cf24eaa2e60e22710b5e6f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/616581
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Ram Peri <ramperi@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
2026-08-26 10:19:42 -07:00
kimhappy 3f1775607f project: allow re-reading submodule gitlinks
A discovered submodule becomes a project whose revision is the gitlink
read from its parent at manifest load time, which is before the parent
has been fetched. Remember which gitlink a submodule was derived from,
and add a way to read those gitlinks again, so that callers can resolve
a submodule revision once its parent is up-to-date.

A revision that is set does not have to be fetched, so verify its
objects are really there. Without them a submodule that was removed
cannot be told apart from one that was never fetched, which the caller
has to know about.

Bug: 550074864
Change-Id: I4f35d53607126508eec6f02158a3fb604917a199
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621661
Reviewed-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-08-25 19:39:51 -07:00
Gavin Mak b85886fa9f project: skip manifest default fallback for MetaProjects
Avoid loading manifest.xml during repo init when syncing a manifest
commit SHA. Override _GetUpstreamFallback and _SharingProjectHasShallow
in MetaProject to prevent premature manifest parsing.

Bug: 544041102
Change-Id: I7aa54a7c1282e5bfe811d59e977b44163a37653c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/617081
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-10 13:37:04 -07:00
Gavin Mak d9da609d8c project: preserve -c optimization when revision is a SHA-1
Avoid disabling --current-branch when syncing a SHA-1 revision without
an explicit project upstream (e.g., smart tags). Resolve a fallback
upstream from dest-branch or manifest defaults so -c only fetches the
target branch.

Bug: 541240657
Change-Id: Ib44b6a732131210e1ec3a3136747d1a19bc5aa18
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/614762
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-08-04 15:00:19 -07:00
Gavin Mak 4bec297eb6 command: Respect smart sync override declaratively by default
Introduce a `RESPECT_SMART_SYNC_OVERRIDE` class attribute to the base
`Command` class, defaulting to `True`. This allows subcommands to
automatically respect the smart sync override manifest if it exists.

The override is applied in `CommonValidateOptions` before any
subcommand-specific validation or execution occurs. The `sync` and
`init` commands explicitly opt out.

This ensures all workspace-aware subcommands consistently align with the
active smart sync override manifest. It also fixes a bug in
multi-manifest setups where running a command from a submanifest would
not apply the override to the outer manifest, causing inconsistency when
resolving projects across all manifests.

Bug: 279204331
Change-Id: I9426e90a13a77ce6bd94b4a82efda4d485cbe116
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/585081
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-08-04 11:38:03 -07:00
Brian Gan 54fa31cd84 project: derive HEAD fallback from git's own default branch
When the default branch cannot be determined dynamically, repo fell back
to a hardcoded "refs/heads/master". This can point at a branch that does
not exist on the server, since projects increasingly default to "main".
Rather than swap one hardcoded name for another, ask git itself what it
would use.

Bug: 483758905
Change-Id: Ic66712acea98f8e548a2d6d8211865ee5c416b4d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/612845
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-07-30 17:00:00 -07:00
Brian Gan 29b6630a65 project: make GetHead file-read fallback reftable-aware
When both `git symbolic-ref` and `git rev-parse` fail to resolve HEAD,
GetHead falls back to reading .git/HEAD directly. With the "reftable" ref
backend, .git/HEAD is only a stub pointing at the "refs/heads/.invalid"
placeholder while the real HEAD lives in the reftable stack. Reading the
stub returned that bogus placeholder value.

Detect the placeholder and treat HEAD as unresolvable (raising
NoManifestException) instead of returning the invalid ref.

Bug: 483758905
Change-Id: Ia0a825c0d1874686b98c5ddcdea6dc26c0d46784
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/612882
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-07-30 10:01:58 -07:00
Andrew Chant 0b82311632 sync: allow syncing groups with repo sync -g group
Similar to how repo init -g can restrict repo syncs to
a subset of the manifest globally, allow "repo sync -g" to only sync a
subset of projects from the manifest when running that specific
sync command.

Change-Id: I4929aad109de05c73a7db42bb27fd8d47eea32fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/609481
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Andrew Chant <achant@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Andrew Chant <achant@google.com>
2026-07-21 13:42:38 -07:00
Andrew Chant 06c4f9e1cc git_superproject: don't filter rewritten manifest
When creating the superproject manifest, take all groups
from the original manifest, including notdefault.

Using the ManifestGroupsStr will inadvertantly filter out
notdefault projects from the superproject manifest.

Change-Id: If3daf41e9a6572348de9182fe8679b90d3cec833
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/610021
Commit-Queue: Andrew Chant <achant@google.com>
Tested-by: Andrew Chant <achant@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-07-21 13:42:30 -07:00
Josef Malmström dd1130352b sync: Deprecate fetch-submodules flag names
The names for flags --fetch-submodules / --no-fetch-submodules are
misleading, since they impact the full sync operation (fetch and
checkout), not just the fetching.

Introduce new flags --recurse-submodules / --no-recurse-submodules
and treat the old ones as deprecated aliases.

Change-Id: I78339a3e0496a855c222c1869b27b578507886a7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/608881
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
2026-07-21 00:19:21 -07:00
Rahul Yadav eeba6f268d hooks: pass yes flag when available
Pass the -y flag as a keyword argument "yes" to the hook main function.
This allows upload hooks (such as auto-fixers) to automatically
apply fixes when the -y flag is passed, rather than prompting the user.

Bug: 498893733
Change-Id: I12096029aa3af471ba9175314d749deb0ebd1007
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/605261
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
2026-07-20 07:56:03 -07:00
Ajay Gupta 1729aaebae project: Skip superproject upstream check for MetaProjects
The superproject-gated upstream check in _CheckForImmutableRevision
only applies to user projects listed in the manifest. MetaProjects
(ManifestProject and RepoProject) never participate in a superproject
relationship, so evaluating git_superproject.UseSuperproject(...,
self.manifest) for them serves no purpose and, worse, calls the
manifest.superproject property which forces a manifest load.

During repo init, ManifestProject._ConfigureDependencies calls
self.Sync_NetworkHalf before manifest.xml has been linked into
.repo/. That reaches _CheckForImmutableRevision, which triggered the
manifest load and failed with:

  ManifestParseError: .../.repo/manifest.xml: [Errno 2] No such file
  or directory

breaking fresh repo init with SHA-based --manifest-branch combined
with --manifest-upstream-branch.

Factor the "should we consult the superproject for upstream?"
decision into a small overridable hook, _UseSuperprojectForUpstream.
Project's default delegates to git_superproject.UseSuperproject;
MetaProject overrides it to return False, localizing the
MetaProject-specific behavior to MetaProject.

Test verifies that calling _CheckForImmutableRevision on a
ManifestProject whose manifest.xml is not yet on disk returns False
without raising and does not create the file.

Change-Id: I22059109243d914036c06c6fe0081a5aba05da89
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574201
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Ajay Gupta <ajagup@qti.qualcomm.com>
Tested-by: Ajay Gupta <ajagup@qti.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
2026-07-16 11:51:00 -07:00
Josef Malmström 978adb7ea5 sync: Add CLI flag for globally disabling submodule fetch
A global setting for disabling fetching of submodules is useful
since this can currently otherwise only be done by modifying
the manifest, or by explicitly providing projects on command line.

Add this setting as --no-fetch-submodules to mirror the existing
--fetch-submodules.

Change-Id: Ic727c54f11a594aa52315751284b87138cf246bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/607641
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-07-16 01:19:51 -07:00
Brian Gan 0398c6718e color: Replace anonymous sentinel with named class
Replace the bare `object()` sentinel used for `_CHECK_CONSOLE` with an
instance of a dedicated `_CheckConsoleSentinel` class. This gives the
sentinel a meaningful repr and type, making it easier to identify in
debugging output and type checks compared to an opaque `<object>`.

Change-Id: I916521d47ba13207e29a2412e00f3576aff8afff
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/605021
Commit-Queue: Brian Gan <brgan@google.com>
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-07-07 14:55:56 -07:00
Gavin Mak 3bb4871c44 rebase: Resolve revisionExpr to tracking branch for --onto-manifest
When running `repo rebase -m` (`--onto-manifest`), the command uses the
raw `revisionExpr` from the manifest (e.g. `main`) directly as the
`--onto` target. This can fail or behave incorrectly if it should
reference the local tracking branch (e.g. `refs/remotes/<remote>/main`).

Resolve `project.revisionExpr` to its local tracking branch using
`project.GetRemote().ToLocal()`. Fall back to using the raw
`revisionExpr` value if the resolution fails (raising a `GitError`).

Bug: 532028666
Change-Id: I4c1bca1374a5842688be227f6aa2afffcdad5397
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/604941
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-07-07 13:40:52 -07:00
Brian Gan 35bbf701d0 color: Treat "true" and "yes" as "auto", not "always"
Per the git documentation for color.ui [1], setting color.ui to "true"
(or "yes") should behave identically to "auto", enabling color only
when output is written to a terminal or an active pager. Previously,
repo was equating "true" and "yes" with "always", which caused color
escape codes to be emitted unconditionally, even when output was piped
or redirected.

Replace the duplicated string-matching logic in SetDefaultColoring and
Coloring.__init__ with a single CONFIG_TO_COLOR_SETTING dict that maps
all git color config values to their behavior. This makes the mapping
easy to verify against the git docs and impossible to get out of sync
between the two call sites.

Added tests for SetDefaultColoring and Coloring.__init__ covering
all color mode values (auto, true, yes, always, never, no, false),
case insensitivity, TTY vs pipe behavior, active pager detection,
and unrecognised input.

[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-colorui

Bug: 295841573
Change-Id: I8a04b9c7e4154de37ed7518c010233039e0afdc9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602981
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-07-01 15:59:45 -07:00
Gavin Mak 881af15cdc agents: run review agents manually
The review agents are broken right now, so don't run them automatically.
They should still be runnable on demand.

Change-Id: I22e5ff46f917a49d0c267a0b1e8c63ae2165347d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/603461
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
2026-07-01 15:45:23 -07:00
Brian Gan fbc9c79192 tests: Fix test compatibility with Python 3.14 forkserver
Python 3.14 changed the default multiprocessing start method on Linux
from "fork" to "forkserver". The test_forall_all_projects_called_once
test uses mock.patch.object on Project.GetRevisionId, but class-level
mock patches do not survive into forkserver worker processes because
they start from a clean Python interpreter rather than inheriting the
parent's memory.

Replace the mock with setting revisionId directly on each Project
instance so GetRevisionId() short-circuits without touching git.
This works with any multiprocessing start method since the string
attribute is part of the Project objects stored in _parallel_context,
which is properly serialized to workers via initargs.

Bug: 425319437
Change-Id: Icd3bbd010921d7652bb2425fad85974df9198367
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602941
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-30 15:35:57 -07:00
Gavin Mak ead4b2d7aa sync: Implement fetchcmd for standard Git layouts
Introduce support for `repo.fetchcmd` configuration, allowing users to
specify a custom command to fetch project data instead of
`_RemoteFetch`.

When `repo.fetchcmd` is specified, `repo` will execute it instead of
`_RemoteFetch` during the network half of sync. Note that
`repo.fetchcmd` requires `repo.uselocalgitdirs` to be enabled.

The command is executed in a subshell with project-context environment
variables, including the new `REPO_TREV` (target revision resolved to a
commit hash) and `REPO_PROJECT_FETCH_URL`.

After execution, `repo` verifies that the target commit is available and
that the tracking ref and FETCH_HEAD are correctly updated.

Tested with:
```
> ~/git-repo/repo init -u https://android-review.googlesource.com/platform/manifest \
    --repo-url file:///usr/local/google/home/gavinmak/git-repo \
    --groups developers \
    --no-repo-verify \
    --use-local-gitdirs
...

> git config --file .repo/manifests.git/config repo.fetchcmd 'mkdir -p $REPO_PATH && cd $REPO_PATH && if [ ! -d .git ]; then git init && git remote add aosp $REPO_PROJECT_FETCH_URL; fi && git fetch aosp $REPO_TREV && git reset --hard $REPO_TREV && mkdir -p .git/refs/remotes/aosp && echo $REPO_TREV > .git/refs/remotes/aosp/main && echo $REPO_TREV > .git/FETCH_HEAD'

> ~/git-repo/repo sync -j32
warning: repo is not tracking a remote branch, so it will not receive updates; run `repo init --repo-rev=stable` to fix.
You are currently enrolled in Git submodules experiment (go/android-submodules-quickstart).  Use --no-use-superproject to override.

Syncing: 100% (4/4), done in 1m15.686s
Finalizing sync state...
repo sync has finished successfully.
```

Bug: 513329573
Change-Id: I754d3f3c78e86fdeee1a72115297a75b571bc497
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583883
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-30 13:20:15 -07:00
Brian Gan e7cac4bca6 status: Show ahead/behind info for local branches
When viewing `repo status`, it is difficult to distinguish between branches that have active unpushed changes and stale branches that are fully synced. Previously, developers had to run commands like `repo forall -c "git status"` to see their ahead/behind counts.

This change updates Project.PrintWorkTreeStatus to automatically calculate and display the number of commits a branch is ahead and/or behind its upstream tracking branch. We use `git rev-list --left-right --count` to fetch this information natively and efficiently.

If the branch is completely synced with upstream, no extra text is shown.

Added tests for ahead-only, behind-only, diverged, no-tracking, and fully-synced branch states.

Bug: 319412954
Change-Id: I23879b2d472c7a7e11d01b565428a84b1b4f09c1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602423
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-06-30 11:13:14 -07:00
Gavin Mak 91986011b0 info: Parallelize project data gathering for JSON output
https://gerrit-review.googlesource.com/c/git-repo/+/581921 parallelized
`repo info` for text output. This commit does the same for JSON output
format.

Benchmarked `repo info --format=json` on an Android workspace with ~3k
projects (N=3):
- Before (sequential): 1m 30s average
- After (parallelized): 46s average (~2x speedup)

Verified that the JSON output is identical before and after.

Bug: 526685287
Change-Id: If573223aba584f8b932f87d29e34ed565c5c930a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601861
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-30 09:19:35 -07:00
Rahul Yadav a27dbcdb7b git_trace2_event_log: Fix index out of range on empty config values
In git_trace2_event_log_base.py's GetDataEventName method, it parses
value to identify if it represents a JSON list. When a config key has an
empty string value, GetDataEventName evaluates value[0], which raises
IndexError: string index out of range.

This change fixes the crash by checking if the value is a string and using
startswith/endswith to check for JSON lists instead of direct indexing.

Test: PYTHONPATH=. pytest tests/test_git_trace2_event_log.py

Bug: 512518342
TAG=agy
CONV=ff5d70d7-e5b3-42b3-8f16-23b9e3070754

Change-Id: Ic40a8c6a22df57d0e97f268f6e1bc8a14a5024a4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602201
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
2026-06-30 04:44:38 -07:00
Gavin Mak 547dc9985c repo: Normalize GNUPGHOME path for MSYS GPG on Windows
Convert GNUPGHOME to a POSIX path using cygpath on Windows
so MSYS GPG can read it correctly.

Bug: 510840000
Change-Id: I7d25564dd6521f6e887ff45548f42984e709ef5e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601121
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-29 11:06:23 -07:00
Gavin Mak 88a7e88e54 info: Report actual checked-out HEAD revision
When `repo info` runs, the reported "Current revision" is resolved using
the manifest's target branch tracking ref (e.g. refs/remotes/goog/main).

Introduce Project.GetHeadRevisionId(), which gets the checked-out HEAD
commit in the worktree, and use it in `repo info` with a fallback to the
old behavior if the project is not checked out.

Bug: 526685287
Change-Id: I72280ce27daa210cada27d722a94e365644f06e0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/599481
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-29 10:50:02 -07:00
Gavin Mak c21a41c7cc git_config: Support SHA-256 object IDs
Update ID_RE to match both 40-char (SHA-1) and 64-char (SHA-256) IDs as
part of Git 3.0 support.

Bug: 483758905
Change-Id: Ie5d9a2a5df4c6e7adda7f1d89f1bfb7724846057
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/600341
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-26 15:40:55 -07:00
Brian Gan 6586efe79a man: disable line wrapping
When help2man runs the repo script to generate man pages, the argparse
module relies on the COLUMNS environment variable to wrap help text.
This wrapping can cause URLs and descriptions to be awkwardly broken
across lines. Setting COLUMNS="10000" prevents argparse from wrapping
the output, keeping URLs intact in the generated man pages.

Additionally, this fixes a broken git documentation URL fragment for
--partial-clone in the repo script.

Bug: 295374161
Change-Id: I0c79f37fbfe2bebe71ff90585f2e5e1f88ea33cb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601482
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-25 16:07:53 -07:00
Sainath Varanasi 3af9e2f146 project: fix sync of shallow projects sharing objdir
Repo sync fails when the following conditions are met:

* There are several checkouts of the same project
  in different paths.
* The checkouts are using git hashes as revisions
  (not branches).
* There is a clone-depth set on these projects.
* sync-c="true" is set in the manifest.
* The revision specified in the manifest
  has moved forward since the first repo init.

The sync fails because only the first gitdir gets the "shallow" file,
and subsequent dirs can't be synced.

Do not optimize away the fetch when the conditions above happen.

Simplified the boolean check in Sync_NetworkHalf and _RemoteFetch
using has_shallow, renamed loop variable to avoid shadowing.

Test: create a manifest matching conditions above, repo init,
forward the hash, and repo sync.
Test: added tests in test_project.py
Bug: 505072873
Originally-by: Elvira Khabirova <elvira.khabirova@volvocars.com>

Change-Id: I37c533c382e34fc5ddab489c5593b9e5d3875be2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601441
Tested-by: Sainath Varanasi <varanasisai@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Sainath Varanasi <varanasisai@google.com>
2026-06-25 15:28:46 -07:00
Gavin Mak d32b70275c agents: Add CRAG-generated review agents and skills
With cl/930783235, this change sets up two AI review agents that run
automatically on new changes.

Bug: 522929179
Change-Id: I315a7ec327dd30af842ec890818cf697756fd55c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/596721
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-24 13:31:45 -07:00
Nasser Grainawi 7f58543703 gitignore: Add AI agent files
Allow contributors to use local AI agent files without polluting the
repository with brand-specific files.

Further AI agent improvements will be done roughly following the same
framework being used for Chromium [1].

[1] https://chromium.googlesource.com/chromium/src/+/HEAD/agents

Change-Id: I791f318b4f9cad1705f94e05522089809097cc96
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/573601
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-06-22 16:19:30 -07:00
Rahul Yadav 39c0b60900 sync: Support pluggable remote helpers for smart sync manifest server.
Introduce support for pluggable remote helpers (declared via the
optional 'helper' attribute in <manifest-server>) to dynamically resolve
proxy addresses. Route the XML-RPC manifest server connection through
the resolved proxy.

Bug: b/517477903
Change-Id: I3b6b8ea2640bb077521df4b4a9e8a34a8c6ecdad
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591642
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-18 09:48:49 -07:00
Gavin Mak cd307a6089 project: Add REPO_PROJECT_FETCH_URL environment variable
Add REPO_PROJECT_FETCH_URL to Project.GetEnvVars(), which resolves to
the remote fetch URL of the project. This is useful for exposing the
URL to custom fetch commands or other external scripts.

Bug: 513329573
Change-Id: Ic2b0a83493934d16bb1152366ee4e1a2c35ea2dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/596121
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-16 10:30:47 -07:00
Josef Malmström 4b462634e0 sync: do not init sibling submodules in parallel
Initializing a submodule requires locking <parent>/.git/config.
As the implementation was currently doing this in parallel for
sibling submodules, it lead to a race condition causing
intermittent errors like:

error: could not lock config file .git/config: File exists

This commit enforces that sibling submodules are initialized
sequentially, eliminating the race condition.

Change-Id: I5ffb3de90276ba43e262d0e279a3d34324220b63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591241
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-16 00:21:27 -07:00
Gavin Mak d9d86fb595 repo: Bump launcher version to 2.65
https://gerrit-review.git.corp.google.com/c/git-repo/+/569001 didn't
update the repo launcher script version for the newly added
`--use-local-gitdirs` flag.

Bug: 513329573
Bug: 508146070
Change-Id: I07ad939a40466ed50e439c815608797689fd505f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/587381
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-06-11 10:39:45 -07:00
Gavin Mak f7a24df00c project: handle corrupted projects in DeleteWorktree
Catch GitError from IsDirty() during DeleteWorktree so corrupted
projects can be cleanly wiped instead of crashing.

Bug: 515415221
Change-Id: Ic03ae77c30a722f9aa06f2220747251e0aea4ab7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591001
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-05 09:16:22 -07:00
Gavin Mak e0bd39c691 project: Extract project envvar generation to GetEnvVars
Move project environment variable setup from subcmds/forall.py to a
reusable Project.GetEnvVars() helper method.

Bug: 513329573
Change-Id: I3b4b113aa5a086e5fa5eaf4461c7ce517d928610
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583881
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-27 12:58:02 -07:00
Xin Li c883613e31 sync: Skip copyfile/linkfile for unavailable projects
Commit 5534f16 made UpdateCopyLinkfileList to retry _CopyAndLinkFiles
for all projects in the manifest to handle directory-to-symlink
transistions. However, for partial sync's (e.g. repo sync <project>),
projects not included in the sync may not have their local directories
populated, attempts to perform the operation on unavailable projects
would result in dangling symlinks, or errors like:

error: Cannot copy file <source> to <destination>

Fix this by adding a check inside _CopyAndLinkFiles in project.py to
so these operations happen only when the project's worktree exists
and is a directory.

TAG=agy
CONV=90f5fca7-2199-4017-8f62-a010b0ab1dbf

Change-Id: I9ae1e8d62d8fd129cb4535e574c38339c87af441
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/587501
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
2026-05-27 10:36:32 -07:00
Gavin Mak 384c059f9e tests: Deduplicate test setup in test_project.py
Bug: 513329573
Change-Id: Id729c590a69d2e81d4bce2942c65bd0b3ce3a11a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583882
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Dan Willemsen <dwillemsen@google.com>
2026-05-26 10:29:06 -07:00
Gavin Mak b8531133de init: Add --use-local-gitdirs for standard Git layouts
Introduce --use-local-gitdirs to bypass repo's symlink-based layouts in
favor of standard local .git directories.

Bug: 513329573
Bug: 508146070
Change-Id: I53d1602e61be0b86964529bcbea3dc801471f9c9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569001
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Dan Willemsen <dwillemsen@google.com>
2026-05-26 10:26:25 -07:00
75 changed files with 12848 additions and 638 deletions
+5
View File
@@ -12,3 +12,8 @@ __pycache__
# PyCharm related
/.idea/
# AI tool related.
/AGENTS.md
/CLAUDE.md
/GEMINI.md
+42
View File
@@ -0,0 +1,42 @@
# git-repo AI Review Agents (WIP)
**Note:** This project is a work in progress and is subject to change.
This directory contains configurations and skills for AI review agents that
automatically analyze changes in the `git-repo` codebase.
These agents help maintain code quality, enforce style guidelines, and catch
common pitfalls before code is merged.
## Directory Structure
* [`agent_configs.txtpb`](agent_configs.txtpb): Defines the active AI
agents, their configurations, and which skills they are equipped with.
* [`skills/`](skills/): Contains the "skills" (rules, guidelines, and traps)
used by the agents.
* [`code_review_workflow/`](skills/code_review_workflow/SKILL.md):
Guidelines for code review processes, commit messages, and testing.
* [`core_internals/`](skills/core_internals/SKILL.md): Technical
guidelines for `git-repo` core logic (sync, manifest, git integration,
etc.).
## How It Works
The agents defined in `agent_configs.txtpb` are configured to run
automatically on new changes. They analyze the diffs against the rules defined
in their respective skills and provide feedback in the code review interface
(e.g., Gerrit).
## Contributing
To improve the agent's review quality or add new rules:
1. **Update existing skills**: Modify the `SKILL.md` files under `skills/` to
add new rules, "What" explanations, "Why" rationales, and "Traps"
(Don't/Do code examples).
2. **Add new skills**:
* Create a new directory under `skills/`.
* Add a `SKILL.md` following the established format (see existing skills
for reference).
* Register the new skill in `agent_configs.txtpb` by adding it to the
`skills` field of an agent configuration.
+26
View File
@@ -0,0 +1,26 @@
# proto-file: google/corp/android/engprod/codereviewagentconfiguration/v1/agent.proto
# proto-message: HostAgents
# Code Review Workflow Agent
configs {
id: "code-review-workflow"
display_name: "Code Review Workflow"
description: "Analyzes git-repo Gerrit submission labeling, commit metadata, Python linting, and testing strategy."
skills: "code_review_workflow"
include_filters {
project: "git-repo"
}
automatic: false
}
# Core Internals Agent
configs {
id: "core-internals"
display_name: "Core Internals"
description: "Analyzes git-repo synchronization, multiprocessing, manifest parsing, git integration, worktree layouts, and CLI commands."
skills: "core_internals"
include_filters {
project: "git-repo"
}
automatic: false
}
+968
View File
@@ -0,0 +1,968 @@
---
name: code-review-workflow
description: Provides guidance and best practices on Gerrit submission labeling, CI builder execution, Python code formatting/linting, commit metadata standardization, and testing strategy in git-repo.
---
# Code Review Workflow Engineering Guide
## Executive Summary
Welcome to the authoritative engineering guide for the Code Review Workflow.
This living repository exists to capture critical folk knowledge, prevent the
recurrence of historical failure modes, and enforce strict architectural and
procedural boundaries across our integration pipeline. By standardizing these
protocols, we ensure high development velocity while maintaining rock-solid
codebase stability and traceability.
This guide covers the complete lifecycle of a change list (CL) from local
development to automated submission. It defines the strict Gerrit labeling
mechanisms required to trigger the Commit-Queue, mandates comprehensive CI
builder environment checks, and enforces centralized Python static analysis.
Furthermore, it outlines uncompromising standards for atomic commit metadata and
pragmatic testing state isolation to guarantee that every integration is fully
bisectable and verifiable.
For incoming engineers, adherence to these mandates eliminates the friction of
stalled pipelines, unreviewable monolithic changes, and silent CI regressions.
Treat this guide as your primary roadmap for navigating the repository's strict
submission requirements, enabling seamless transitions from peer approval to
successfully integrated code.
## Summary
| Chapter Theme / Title | Scope & Objective |
| :------------------------------- | :---------------------------------------- |
| **Gerrit Submission and Labeling | Dictates strict access controls, review |
: Workflow** : enforcement protocols, and Gerrit :
: : labeling mechanisms required to advance :
: : changes through the CI pipeline, ensuring :
: : seamless transitions to automated :
: : integration via the Commit-Queue. :
| **CI Builder Environment and | Defines guidelines for ensuring build |
: Execution Integrity** : script resilience against missing :
: : dependencies and managing process :
: : execution contexts within LUCI and local :
: : testing environments to prevent silent :
: : builder failures. :
| **Python Code Formatting and | Governs the automated enforcement of |
: Linting** : Python style guidelines, mandating strict :
: : PEP-8 compliance, import sorting, and :
: : consistent string quoting to ensure :
: : codebase uniformity and prevent CI :
: : regressions. :
| **Commit Metadata and History | Establishes the structural composition |
: Standardization** : and metadata formatting of change lists :
: : (CLs) to ensure precise issue tracker :
: : integration, reliable CI/CD parsing, and :
: : an atomic, bisectable repository history. :
| **Testing Strategy and State | Outlines test implementation boundaries, |
: Isolation** : emphasizing pragmatic mocking limits to :
: : prevent false positives and detailing :
: : acceptable workflows for deferred test :
: : coverage while maintaining verification :
: : integrity. :
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
## Chapter: Gerrit Submission and Labeling Workflow
**Context:** This domain dictates the strict access controls, review enforcement
protocols, and specific Gerrit labeling mechanisms required to advance changes
through the CI pipeline. Adherence ensures seamless transitions from peer
approval to automated integration via the Commit-Queue.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / |
: : : : Trap :
| :-------- | :------------------------------ | :------- | :----------------- |
| **T1-01** | Explicit Labeling for Gerrit | High | Leaving a change |
: : Automated Submission : : idle after :
: : : : addressing :
: : : : comments or :
: : : : receiving a :
: : : : reviewer's LGTM, :
: : : : expecting the :
: : : : reviewer to merge :
: : : : it. :
| **T1-02** | Automated Submission via | Medium | Requesting a |
: : Commit-Queue (CQ) : : manual push or :
: : : : direct submit from :
: : : : repository :
: : : : maintainers after :
: : : : receiving code :
: : : : review approval. :
| **T1-03** | Gerrit Trusted Contributor | Medium | Relying on a |
: : Review Enforcement Verification : : standard +2 vote :
: : : : from a non-trusted :
: : : : contributor to :
: : : : fulfill strict :
: : : : Review-Enforcement :
: : : : requirements. :
| **T1-04** | Mandatory Gerrit Labels for | High | Acknowledging an |
: : Automated Submission : : approval but :
: : : : failing to apply :
: : : : the appropriate :
: : : : Gerrit labels to :
: : : : initiate the merge :
: : : : pipeline. :
| **T1-05** | Gerrit Automated Submission | Medium | Leaving an |
: : Triggers : : approved patchset :
: : : : idle and waiting :
: : : : for maintainers to :
: : : : manually merge it. :
| **T1-06** | Active Reviewer Rerouting for | Medium | Waiting weeks or |
: : Stalled Changes : : months for an :
: : : : inactive or OOO :
: : : : reviewer to :
: : : : respond to a :
: : : : patchset update. :
--------------------------------------------------------------------------------
### Rules
#### T1-01: Explicit Labeling for Gerrit Automated Submission
> **Rule:** Always apply `Verified+1` and `Commit-Queue+2` explicitly to trigger
> the final submission phase. Never assume a code approval automatically
> initiates the pipeline.
>
> **What:** Changes are not merged automatically upon receiving approval;
> contributors must explicitly set the `Verified+1` and `Commit-Queue+2` labels
> to trigger the final submission phase.
>
> **Applies To:** Gerrit review UI and change submission pipeline as defined in
> `CONTRIBUTING.md`.
>
> **Why:** Contributors often mistakenly assume an LGTM implies an immediate
> merge, leading to stalled changes. The project relies on explicitly triggering
> the Commit-Queue to finalize CI checks and perform the merge. Failing to
> adhere to this typically results in **Stalled Submission Pipeline**.
**Trap 1: Leaving a change idle after addressing comments or receiving a
reviewer's LGTM, expecting the reviewer to merge it.**
**Don't:**
* Waiting indefinitely after reviewer posts 'LGTM'.
**Do:**
* Vote `Verified+1` and `Commit-Queue+2` manually to submit the change to the
automated queue.
**Exceptions:** Contributors lacking trusted permissions must ping a repository
maintainer to apply the final `Commit-Queue+2` vote.
--------------------------------------------------------------------------------
#### T1-02: Automated Submission via Commit-Queue (CQ)
> **Rule:** Must utilize the Gerrit Commit-Queue (CQ) labeling system to merge
> code. Maintainers must never perform direct manual submissions.
>
> **What:** Merging code must be triggered via the Gerrit Commit-Queue (CQ)
> labeling system rather than relying on direct manual submission by
> maintainers.
>
> **Applies To:** Gerrit code review UI and CI/CD submission workflow.
>
> **Why:** Contributors would request maintainers to directly merge patches once
> approved, bypassing the automated commit-queue pipeline, which guarantees that
> final integration tests pass before pushing to the target branch. Failing to
> adhere to this typically results in **Bypassed CI / Direct Submit**.
**Trap 1: Requesting a manual push or direct submit from repository maintainers
after receiving code review approval.**
**Don't:**
* Leaving a comment: "I believe everything is ready for integrating this. So
if either of you can submit it, it would be appreciated."
**Do:**
* Applying the `Commit-Queue+2` (CQ+2) label in Gerrit, which delegates
testing and the final merge operation to the automated bot.
--------------------------------------------------------------------------------
#### T1-03: Gerrit Trusted Contributor Review Enforcement Verification
> **Rule:** Verify review enforcement requirements are satisfied by contributors
> within the explicitly configured trusted group. Never cast misleading +2 votes
> if you lack valid trusted group privileges.
>
> **What:** Gerrit submission requirements may mandate specific approval levels
> (e.g., two trusted contributors). Votes from users with +2 access who are not
> in the designated 'trusted' group do not satisfy the 'Review-Enforcement'
> submit requirement.
>
> **Applies To:** Gerrit repository administration and code review voting
> workflows.
>
> **Why:** Non-trusted contributors with +2 rights were casting +2 votes on
> changes. These votes did not fulfill the 'Two trusted contributors'
> Review-Enforcement requirement, leading to stalled submissions and confusion
> regarding why the UI showed a +2 but blocked submission. Failing to adhere to
> this typically results in **Blocked Submission / Silent Requirement Failure**.
**Trap 1: Relying on a standard +2 vote from a non-trusted contributor to
fulfill strict Review-Enforcement requirements.**
**Don't:**
* Leaving a +2 vote on a change as a non-trusted contributor, creating the
false appearance that the Review-Enforcement requirement has been partially
or fully met.
**Do:**
* Verifying the reviewer is in the explicitly configured trusted group for the
repository. If not, the reviewer should manually downgrade their invalid +2
vote to a +1 to clearly indicate that their vote does not count toward the
enforcement threshold.
**Exceptions:** Repositories where specific non-employee groups have been
explicitly added to the trusted administrators list.
--------------------------------------------------------------------------------
#### T1-04: Mandatory Gerrit Labels for Automated Submission
> **Rule:** Always apply `Verified+1` and `Commit-Queue+2` labels to initiate
> the CI merge process. Never leave an approved CL in a technically unlabeled
> state.
>
> **What:** A code change must receive explicit `Verified+1` and
> `Commit-Queue+2` labels by the author or reviewer to trigger the automated CI
> merge process.
>
> **Applies To:** Gerrit workflow / Merge execution phase.
>
> **Why:** Historically, leaving a Change List (CL) in an approved but unlabeled
> state causes the integration pipeline to stall indefinitely, requiring manual
> intervention or reviewer pinging to trigger the CI queue. Failing to adhere to
> this typically results in **Merge Pipeline Stall**.
**Trap 1: Acknowledging an approval but failing to apply the appropriate Gerrit
labels to initiate the merge pipeline.**
**Don't:**
* Leaving the CL in an approved state and waiting for auto-submission without
applying the `Verified+1` or `Commit-Queue+2` labels.
**Do:**
* Explicitly applying `Verified+1` (and `Commit-Queue+2` if ready) once
reviewers have approved the logic, to instruct the automation to merge the
code.
--------------------------------------------------------------------------------
#### T1-05: Gerrit Automated Submission Triggers
> **Rule:** Must actively signal patch readiness to Gerrit systems using proper
> label thresholds. Avoid leaving patchsets idle assuming upstream maintainer
> action.
>
> **What:** A patchset requires specific label thresholds ('Verified+1' and
> 'Commit-Queue+2') to trigger automated submission in the Gerrit workflow.
>
> **Applies To:** Gerrit review UI and automated CI/CD submission process for
> the git-repo codebase.
>
> **Why:** Contributors frequently asked how to integrate changes after
> receiving an approval, leading to stalled patches because the automated
> pipeline was not explicitly triggered. Failing to adhere to this typically
> results in **Stalled Patch Integration**.
**Trap 1: Leaving an approved patchset idle and waiting for maintainers to
manually merge it.**
**Don't:**
* Waiting indefinitely after receiving an 'LGTM' without setting workflow
labels.
**Do:**
* The patch author manually sets the 'Verified' flag (if locally tested) and
applies the 'Commit-Queue+2' vote to signal readiness for automated merge.
--------------------------------------------------------------------------------
#### T1-06: Active Reviewer Rerouting for Stalled Changes
> **Rule:** Actively reroute reviews stalled by unresponsive or out-of-office
> (OOO) primary reviewers. Must explicitly tag alternate maintainers and
> document the absence to prevent lifecycle stalls.
>
> **What:** If the primary reviewer is out-of-office (OOO) or unresponsive for
> an extended period, contributors must actively CC and reroute the review to
> another active maintainer.
>
> **Applies To:** Gerrit review cycle and reviewer assignment process.
>
> **Why:** Patchsets have historically stalled for over a month due to reviewers
> taking extended leave without actively delegating their review queues. Failing
> to adhere to this typically results in **Indefinite Review Stalls**.
**Trap 1: Waiting weeks or months for an inactive or OOO reviewer to respond to
a patchset update.**
**Don't:**
* Leaving a review assigned strictly to an unresponsive reviewer without
notifying other maintainers or attempting to escalate.
**Do:**
* Tag a new reviewer with 'PTAL' (Please Take A Look) in the thread,
explicitly noting the original reviewer's absence, and confirm alignment
with the original author.
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Upstream:** T4 | Python Code Formatting and Linting - *Proper formatting
and static analysis are enforced before changes become eligible for final
Gerrit review and automated integration.*
* **Upstream:** T5 | Commit Metadata and History Standardization - *Accurate
commit messaging and isolated history must be validated by reviewers prior
to receiving approval labels.*
* **Downstream:** T3 | CI Builder Environment and Execution Integrity -
*Triggering the Commit-Queue directly invokes downstream LUCI environments
to guarantee execution integrity prior to branch merge.*
## Chapter: CI Builder Environment and Execution Integrity
**Context:** This section defines strict guidelines for ensuring the resilience
of build scripts against missing dependencies and managing process execution
contexts within LUCI and local testing environments. Adherence guarantees robust
verification across diverse operating systems and CI pipelines while preventing
silent builder failures.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / Trap |
| :-------- | :------------------------ | :------- | :------------------------ |
| **T3-01** | Verification Against | High | Running a standard local |
: : Breaking Change Build : : `make` without testing :
: : Configurations : : strict configurations or :
: : : : breaking-change flags. :
| **T3-02** | Windows Developer Mode | Medium | Attempting to run full |
: : Requirements for Tool : : local verification on a :
: : Verification : : standard Windows user :
: : : : account. :
| **T3-03** | Graceful Degradation for | Medium | Assuming all local |
: : Missing Builder Utilities : : developer utilities exist :
: : : : in the strict CI builder :
: : : : environment and :
: : : : unconditionally executing :
: : : : them. :
| **T3-04** | Contextual Diagnostic | High | Observing a generic CI |
: : Logging for LUCI CI : : failure without isolating :
: : Failures : : the specific process :
: : : : execution context or :
: : : : dependency resolution :
: : : : step. :
--------------------------------------------------------------------------------
### Rules
#### T3-01: Verification Against Breaking Change Build Configurations
> **Rule:** Always explicitly test core build structure modifications with
> breaking changes enabled to ensure forward compatibility.
>
> **What:** When modifying core build structures, the build must be tested
> explicitly with breaking changes enabled to ensure forward compatibility and
> correct regeneration of generated files.
>
> **Applies To:** Local build environments and Makefile targets.
>
> **Why:** Changes might succeed in a standard default build but fail when
> breaking change toggles are activated, hiding underlying dependency or
> regeneration issues. Failing to adhere to this typically results in **Build
> Breakage / Stale Artifacts**.
**Trap 1: Running a standard local `make` without testing strict configurations
or breaking-change flags.**
**Don't:**
```bash
make -j
```
**Do:**
```bash
make -j WITH_BREAKING_CHANGES=1
```
--------------------------------------------------------------------------------
#### T3-02: Windows Developer Mode Requirements for Tool Verification
> **Rule:** Must execute local tool verification on Windows (gWindows) using an
> Administrator account to enable Developer Mode.
>
> **What:** Local verification of git-repo tooling on Windows (gWindows)
> explicitly requires the host environment to be running with Administrator
> privileges to enable Developer Mode.
>
> **Applies To:** Windows (gWindows) test environments verifying file system
> operations.
>
> **Why:** Without Developer Mode enabled (which necessitates Admin rights),
> features relying on advanced OS-level file system operations (like symlinks)
> cannot execute, permanently blocking full local test suite execution on
> standard accounts. Failing to adhere to this typically results in
> **Verification Blocked / OS Permission Error**.
**Trap 1: Attempting to run full local verification on a standard Windows user
account.**
**Don't:**
* Executing the test suite from a non-elevated command prompt on Windows
without Developer Mode.
**Do:**
* Elevate to an Administrator account to enable Developer Mode before
executing the test suite on gWindows.
--------------------------------------------------------------------------------
#### T3-03: Graceful Degradation for Missing Builder Utilities
> **Rule:** Always implement auto-skip logic for optional utilities in build
> scripts rather than hard-failing when unavailable on the CI builder.
>
> **What:** Build scripts and test suites must implement auto-skip logic for
> optional, environment-specific utilities rather than hard-failing when the
> utility is unavailable on the CI builder.
>
> **Applies To:** CI Builder environment scripts and test suites, specifically
> testing external CLI utilities (e.g., `help2man`).
>
> **Why:** When a required utility was not pre-installed on the CI builder
> image, the build hard-failed. Adding auto-skip logic allows the CI pipeline to
> remain unblocked while still providing local testing benefits for developers
> who have the tool installed. Failing to adhere to this typically results in
> **Build Failure / Blocked CI**.
**Trap 1: Assuming all local developer utilities exist in the strict CI builder
environment and unconditionally executing them.**
**Don't:**
```python
# BAD: Hard failure if utility is missing
subprocess.run(["help2man", "repo"], check=True)
```
**Do:**
```python
# GOOD: Auto-skip test if utility is missing in the environment
if not shutil.which("help2man"):
self.skipTest("help2man not installed")
subprocess.run(["help2man", "repo"], check=True)
```
**Exceptions:** Core dependencies required for fundamental build steps cannot be
skipped and must be installed on the bot image.
--------------------------------------------------------------------------------
#### T3-04: Contextual Diagnostic Logging for LUCI CI Failures
> **Rule:** Must investigate CI builder failures by extracting and analyzing
> full execution context logs to isolate environmental roadblocks.
>
> **What:** CI builder failures must be investigated using full execution
> context logs (e.g., LUCI context, vpython3 resolution, and retcode outputs) to
> isolate environmental roadblocks.
>
> **Applies To:** LUCI builder execution environment, vpython3 resolution, and
> CI pipeline debugging.
>
> **Why:** CI commands failed with `retcode 1` due to external factors like
> specific URLs being flagged as suspect by internal security tools, breaking
> the build environment. Failing to adhere to this typically results in **Silent
> Builder Failure**.
**Trap 1: Observing a generic CI failure without isolating the specific process
execution context or dependency resolution step.**
**Don't:**
* Restarting the CI pipeline blindly when a job fails with a generic retcode,
ignoring potential external network or security blockers.
**Do:**
* Extract the step-by-step LUCI context log, verify path resolution (e.g.,
CIPD packages), and explicitly document external blockers like security
flags in the review.
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Upstream:** T6 | Testing Strategy and State Isolation - *Test
implementation dictates how missing builder utilities are mocked or
gracefully skipped during execution.*
* **Downstream:** T1 | Gerrit Submission and Labeling Workflow - *Automated
Verified+1 labels rely entirely on the stable, unblocked execution of CI
builder pipelines.*
## Chapter: Python Code Formatting and Linting
**Context:** This domain governs the automated enforcement of Python style
guidelines, mandating strict PEP-8 compliance, import sorting, and consistent
string quoting. All Python modifications must pass centralized static analysis
pipelines before integration to ensure codebase uniformity and prevent CI
regressions.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / Trap |
| :-------- | :----------------------- | :------- | :------------------------ |
| **T4-01** | Automated Flake8 | Medium | Relying purely on manual |
: : Post-Submit Verification : : code review or sporadic :
: : : : local linting without a :
: : : : continuous integration :
: : : : check. :
| **T4-02** | Mandatory Python | High | Using single quotes for |
: : Formatting and Import : : strings and appending new :
: : Sorting : : imports to the bottom of :
: : : : the import block without :
: : : : alphabetical or :
: : : : categorical sorting. :
| **T4-03** | Strict Python Import | High | Mixing local application |
: : Ordering : : imports with standard :
: : : : library imports, causing :
: : : : linting tools to fail the :
: : : : CQ job. :
--------------------------------------------------------------------------------
### Rules
#### T4-01: Automated Flake8 Post-Submit Verification
> **Rule:** Always configure and maintain centralized CI workflows to
> automatically run static analysis and validate Python code styling
> post-submit.
>
> **What:** Static analysis and Python linting must be automated via a
> centralized CI pipeline (e.g., Flake8 post-submit workflows) to enforce
> consistent style and prevent basic errors.
>
> **Applies To:** All Python files in the git-repo codebase; specifically
> validated via `.github/workflows/flake8-postsubmit.yml`.
>
> **Why:** Relying strictly on manual code review to catch styling and linting
> violations is error-prone. Automation ensures a baseline of code quality on
> every code push without consuming human review cycles. Failing to adhere to
> this typically results in **Linting Regression / Style Violation**.
**Trap 1: Relying purely on manual code review or sporadic local linting without
a continuous integration check.**
**Don't:**
* Committing Python code without an active CI linting workflow configuration.
**Do:**
* Maintain `.github/workflows/flake8-postsubmit.yml` to automatically run
flake8 on target branches.
--------------------------------------------------------------------------------
#### T4-02: Mandatory Python Formatting and Import Sorting
> **Rule:** Must format Python code to enforce double-quoted strings and
> alphabetically sorted import blocks to satisfy automated formatting checks.
>
> **What:** Python code modifications must pass automated style and linting
> checks ('Verify git-repo CL'), which strictly enforce string quote conventions
> (preferring double quotes), import block sorting, and PEP-8 style formatting.
>
> **Applies To:** All Python source files modified in the git-repo codebase.
>
> **Why:** Developers submitting patches with single-quoted strings or unsorted
> imports triggered automated CI failures in the `Verify git-repo CL` job,
> completely blocking code submission until formatting tools were executed
> locally. Failing to adhere to this typically results in **CI Pipeline
> Failure**.
**Trap 1: Using single quotes for strings and appending new imports to the
bottom of the import block without alphabetical or categorical sorting.**
**Don't:**
```python
import sys
import os
msg = 'This is an error'
```
**Do:**
```python
import os
import sys
msg = "This is an error"
```
--------------------------------------------------------------------------------
#### T4-03: Strict Python Import Ordering
> **Rule:** Always segment and order Python imports strictly according to
> project standards (standard library, third-party, local) to prevent CQ
> pipeline failures.
>
> **What:** Python module imports must adhere strictly to the project's
> formatting rules (e.g., standard library, third-party, local module ordering)
> to pass automated Commit-Queue (CQ) checks.
>
> **Applies To:** Python source files.
>
> **Why:** Non-standard import blocks cause the automated CI/CQ linting pipeline
> to fail, completely blocking submission even if the core functional logic of
> the patch is flawless. Failing to adhere to this typically results in **CI
> Linting Failure**.
**Trap 1: Mixing local application imports with standard library imports,
causing linting tools to fail the CQ job.**
**Don't:**
```python
import sys
import my_local_module
import os
```
**Do:**
```python
import os
import sys
import my_local_module
```
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Upstream:** T3 | CI Builder Environment and Execution Integrity -
*Reliable CI builder environments must be available to execute the static
analysis and Python formatting verifications.*
* **Downstream:** T1 | Gerrit Submission and Labeling Workflow - *Formatting
and linting rules must be fully satisfied before automated mechanisms like
the Commit-Queue (CQ+2) will merge code into the repository.*
## Chapter: Commit Metadata and History Standardization
**Context:** This domain governs the structural composition and metadata
formatting of change lists (CLs) within the git-repo codebase. Strict adherence
ensures precise issue tracker integration, reliable CI/CD parsing, and atomic,
bisectable repository history.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / Trap |
| :-------- | :------------------------ | :------- | :------------------------ |
| **T5-01** | Strict Commit Message Bug | Medium | Providing free-text |
: : Tag Formatting : : descriptions, arbitrary :
: : : : prefixes, or non-standard :
: : : : bug references in the :
: : : : commit block. :
| **T5-02** | Atomic and Bisectable | High | Waiting for an entire |
: : Change Integration : : feature stack of multiple :
: : : : interdependent CLs to be :
: : : : approved before merging :
: : : : the base commits. :
| **T5-03** | Explicit Bug Tracker | Medium | Submitting a fix or |
: : Linking for Context : : revert without :
: : Restoration : : referencing the :
: : : : corresponding bug tracker :
: : : : issue detailing the :
: : : : specific regression or :
: : : : stack trace. :
| **T5-04** | Atomic Change List | Medium | Submitting a single large |
: : Decomposition : : CL that touches multiple :
: : : : isolated components or :
: : : : implements several :
: : : : distinct features :
: : : : simultaneously. :
--------------------------------------------------------------------------------
### Rules
#### T5-01: Strict Commit Message Bug Tag Formatting
> **Rule:** Must use the exact `Bug: <number>` syntax in commit messages to
> properly link issue trackers.
>
> **What:** Commit messages must link directly to issue trackers using the
> explicit 'Bug: <number>' syntax to allow reliable parsing by CI/CD and history
> tracking systems.
>
> **Applies To:** Commit messages across all git-repo changes.
>
> **Why:** Improperly formatted bug tags fail to link with the external issue
> tracker, severing historical context and breaking automated post-submit
> tracking workflows. Failing to adhere to this typically results in **Broken
> Traceability / Pre-submit Failure**.
**Trap 1: Providing free-text descriptions, arbitrary prefixes, or non-standard
bug references in the commit block.**
**Don't:**
```text
Fixes bug 486536908
Closes issue 486536908
```
**Do:**
```text
Bug: 486536908
```
--------------------------------------------------------------------------------
#### T5-02: Atomic and Bisectable Change Integration
> **Rule:** Always submit code incrementally as isolated, functional units
> rather than hoarding monolithic stacks.
>
> **What:** Code changes must be submitted incrementally as isolated, functional
> units rather than waiting to merge a massive interdependent stack all at once.
>
> **Applies To:** Git commit history, PR structuring, and stack-based code
> integration.
>
> **Why:** Contributors accustomed to integrating full monolithic stacks at once
> held off on landing initial, stable changes. This practice hinders the ability
> to isolate regressions via `git bisect` and prevents foundational code from
> "baking" in production. Failing to adhere to this typically results in
> **Bisection Breakage / Monolithic Rollbacks**.
**Trap 1: Waiting for an entire feature stack of multiple interdependent CLs to
be approved before merging the base commits.**
**Don't:**
* Holding all changes in a stack locally or in code review until the final
feature patch is approved, then landing 10+ patches simultaneously.
**Do:**
* Landing initial, independent CLs one-by-one as soon as they are approved.
Ensuring each commit is independently usable and does not break the build.
--------------------------------------------------------------------------------
#### T5-03: Explicit Bug Tracker Linking for Context Restoration
> **Rule:** Must include a direct URL to the relevant bug tracker issue
> documenting the failure traceback when submitting a regression fix or revert.
>
> **What:** When submitting a change (especially a revert or bug fix) addressing
> a specific runtime regression, the commit metadata or patchset-level comments
> must include a direct link to the bug tracker issue documenting the failure
> traceback.
>
> **Applies To:** Commit messages and patchset documentation during code
> reviews, particularly for reverts.
>
> **Why:** A previous commit caused a runtime regression (e.g., an
> AttributeError related to a missing object attribute). Without linking the
> specific issue containing the traceback, reviewers lacked the necessary
> context to justify restoring the previous codebase state. Failing to adhere to
> this typically results in **Undocumented Regression / Context Loss**.
**Trap 1: Submitting a fix or revert without referencing the corresponding bug
tracker issue detailing the specific regression or stack trace.**
**Don't:**
* Reverting a change with a vague description like "Fixing previous breakage"
or "Reverting due to pipeline failure" without providing the traceback
source.
**Do:**
* Linking the specific issue tracker URL containing the exact failure mode.
Example: "for more context, see
https://g-issues.gerritcodereview.com/issues/[ISSUE_ID]#comment4"
--------------------------------------------------------------------------------
#### T5-04: Atomic Change List Decomposition
> **Rule:** Never submit large, monolithic change lists; always decompose them
> into logically independent patchsets.
>
> **What:** Large, monolithic change lists (CLs) must be broken down into
> smaller, logically independent patchsets to ensure accurate review and
> historical bisectability.
>
> **Applies To:** Version control history and code review scoping.
>
> **Why:** Massive CLs heavily increase reviewer cognitive load, making thorough
> reviews impossible and complicating future `git bisect` operations when
> tracking down the origin of a regression. Failing to adhere to this typically
> results in **Unreviewable Monolithic Change**.
**Trap 1: Submitting a single large CL that touches multiple isolated components
or implements several distinct features simultaneously.**
**Don't:**
* A single CL containing sweeping refactoring, new feature implementation, and
unrelated bug fixes.
**Do:**
* Breaking the monolithic change into smaller, logically dependent or
independent CLs where each addresses one specific piece of the feature or
refactor.
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Downstream:** T1 | Gerrit Submission and Labeling Workflow - *Gerrit and
CI pipelines strictly rely on standardized commit metadata to link tracking
issues and depend on atomic patchsets to execute automated review and
verification correctly.*
## Chapter: Testing Strategy and State Isolation
**Context:** This chapter governs test implementation boundaries, emphasizing
pragmatic mocking limits to prevent false positives and detailing acceptable
workflows for deferred test coverage. Strict adherence ensures robust state
isolation and maintains development velocity without compromising verification
integrity.
### Summary
| Rule ID | Principle / Constraint | Priority | Primary Symptom / Trap |
| :-------- | :----------------------- | :------- | :------------------------- |
| **T6-01** | Pragmatic Mocking | Medium | Mocking the entire core |
: : Boundaries in Unit Tests : : state or framework :
: : : : dependencies just to force :
: : : : a unit test for a highly :
: : : : integrated function. :
| **T6-02** | Deferred Test | Medium | Submitting functional code |
: : Implementation via : : without matching test :
: : Follow-up : : coverage and stalling the :
: : : : merge while complex tests :
: : : : are written. :
--------------------------------------------------------------------------------
### Rules
#### T6-01: Pragmatic Mocking Boundaries in Unit Tests
> **Rule:** Always restrict unit tests to isolated methods and avoid aggressive
> mocking of core functionality to prevent brittle, false-positive verification.
>
> **What:** Do not aggressively mock core functionality in unit tests; restrict
> unit tests to isolated methods to avoid creating brittle tests based on false
> assumptions when an integration framework is unavailable.
>
> **Applies To:** Test suite implementation (Unit vs. Integration testing
> boundaries).
>
> **Why:** Over-mocking complex systems in unit tests leads to scenarios where
> tests pass but the core integration fails in production because the unit test
> mocks assumed incorrect behavior about the underlying environment. Failing to
> adhere to this typically results in **False Positive Test Passage**.
**Trap 1: Mocking the entire core state or framework dependencies just to force
a unit test for a highly integrated function.**
**Don't:**
* Mocking file systems, external processes, and global state heavily to test a
core workflow orchestrator in a unit test suite.
**Do:**
* Limiting unit tests strictly to isolated utility methods (e.g., adding
promisor files) and explicitly documenting testing gaps that require
integration test frameworks.
**Exceptions:** Isolated helper methods or purely functional data
transformations should be fully unit tested with appropriate mocked inputs.
--------------------------------------------------------------------------------
#### T6-02: Deferred Test Implementation via Follow-up
> **Rule:** Never stall critical feature merges indefinitely for test
> implementation if maintainers authorize formalized, immediate follow-up test
> coverage.
>
> **What:** New logic requires automated tests; however, reviewers may permit
> test coverage to be implemented in a subsequent follow-up CL to maintain
> development velocity.
>
> **Applies To:** Feature development, regression testing, and code review
> criteria.
>
> **Why:** Reviewers identified a lack of test coverage for new functionality
> but opted not to block the immediate patchset, instead formalizing the test
> requirement as a near-term follow-up task. Failing to adhere to this typically
> results in **Missing Test Coverage**.
**Trap 1: Submitting functional code without matching test coverage and stalling
the merge while complex tests are written.**
**Don't:**
* Blocking a necessary feature indefinitely due to missing unit tests when a
follow-up CL is viable and acceptable to maintainers.
**Do:**
* Approve the feature with an explicit, documented 'TODO' for a follow-up CL
dedicated strictly to adding the corresponding automated tests.
**Exceptions:** Critical path features or security fixes where a lack of
immediate coverage introduces an unacceptable regression risk.
--------------------------------------------------------------------------------
### Cross-Domain Dependencies
* **Upstream:** T1 | Gerrit Submission and Labeling Workflow - *Reviewer
approval mechanisms and label enforcement dictate when a feature can merge
while deferring tests to a follow-up CL.*
* **Downstream:** T3 | CI Builder Environment and Execution Integrity -
*Pragmatically bounded unit and integration tests ensure reliable CI
pipeline execution without false-positive success markers.*
File diff suppressed because it is too large Load Diff
+30 -19
View File
@@ -85,28 +85,43 @@ def _Color(fg=None, bg=None, attr=None):
DEFAULT = None
class _CheckConsoleSentinel:
"""Sentinel for checking console coloring."""
# Placholder value that indicates we need to check if the user is in an
# interactive terminal session to determine if we turn on color or not.
_CHECK_CONSOLE = _CheckConsoleSentinel()
# https://git-scm.com/docs/git-config#Documentation/git-config.txt-colorui
_CONFIG_TO_COLOR_SETTING = {
"false": False,
"never": False,
"no": False,
"auto": _CHECK_CONSOLE,
"true": _CHECK_CONSOLE,
"yes": _CHECK_CONSOLE,
"always": True,
}
def SetDefaultColoring(state: Optional[str]) -> None:
"""Set coloring behavior to |state|.
This is useful for overriding config options via the command line.
"""
if state is None:
# Leave it alone -- return quick!
return
global DEFAULT
state = state.lower()
if state in ("auto",):
if isinstance(state, str):
state = state.lower()
if state in _CONFIG_TO_COLOR_SETTING:
DEFAULT = state
elif state in ("always", "yes", "true"):
DEFAULT = "always"
elif state in ("never", "no", "false"):
DEFAULT = "never"
class Coloring:
def __init__(self, config, section_type):
self._section = "color.%s" % section_type
self._section = f"color.{section_type}"
self._config = config
self._out = sys.stdout
@@ -115,16 +130,12 @@ class Coloring:
on = self._config.GetString(self._section)
if on is None:
on = self._config.GetString("color.ui")
if isinstance(on, str):
on = on.lower()
if on == "auto":
if pager.active or os.isatty(1):
self._on = True
else:
self._on = False
elif on in ("true", "always"):
self._on = True
else:
self._on = False
self._on = _CONFIG_TO_COLOR_SETTING.get(on, _CHECK_CONSOLE)
if self._on is _CHECK_CONSOLE:
self._on = pager.active or os.isatty(1)
def redirect(self, out):
self._out = out
+26 -4
View File
@@ -17,6 +17,7 @@ import multiprocessing
import optparse
import os
import re
from typing import TYPE_CHECKING
from error import InvalidProjectGroupsError
from error import NoSuchProjectError
@@ -25,6 +26,10 @@ from event_log import EventLog
import progress
if TYPE_CHECKING:
from project import Project
# Are we generating man-pages?
GENERATE_MANPAGES = os.environ.get("_REPO_GENERATE_MANPAGES_") == " indeed! "
@@ -61,6 +66,10 @@ class Command:
# command to show short-vs-full summaries.
COMMON = False
# Whether this command should respect the smart sync override manifest if
# it exists.
RESPECT_SMART_SYNC_OVERRIDE = True
# Whether this command supports running in parallel. If greater than 0,
# it is the number of parallel jobs to default to.
PARALLEL_JOBS = None
@@ -242,6 +251,12 @@ class Command:
# from the user's perspective.
opt.outer_manifest = True
if self.RESPECT_SMART_SYNC_OVERRIDE:
if self.manifest:
self.TryOverrideManifestWithSmartSync(self.manifest)
if self.outer_manifest and self.outer_manifest != self.manifest:
self.TryOverrideManifestWithSmartSync(self.outer_manifest)
def ValidateOptions(self, opt, args):
"""Validate the user options & arguments before executing.
@@ -375,7 +390,7 @@ class Command:
manifest=None,
groups="",
missing_ok=False,
submodules_ok=False,
submodules_ok=None,
all_manifests=False,
):
"""A list of projects that match the arguments.
@@ -385,7 +400,9 @@ class Command:
manifest: an XmlManifest, the manifest to use, or None for default.
groups: a string, the manifest groups in use.
missing_ok: a boolean, whether to allow missing projects.
submodules_ok: a boolean, whether to allow submodules.
submodules_ok: whether to allow submodules. True allows them for
all projects, False disallows them for all projects, and None
defers to each project's sync-s setting.
all_manifests: a boolean, if True then all manifests and
submanifests are used. If False, then only the local
(sub)manifest is used.
@@ -403,6 +420,11 @@ class Command:
all_projects_list = manifest.projects
result = []
def should_include_submodules(project: "Project") -> bool:
if submodules_ok is None:
return project.sync_s
return submodules_ok
if not groups:
groups = manifest.GetManifestGroupsStr()
groups = [x for x in re.split(r"[,\s]+", groups) if x]
@@ -410,7 +432,7 @@ class Command:
if not args:
derived_projects = {}
for project in all_projects_list:
if submodules_ok or project.sync_s:
if should_include_submodules(project):
derived_projects.update(
(p.RelPath(local=False), p)
for p in project.GetDerivedSubprojects()
@@ -452,7 +474,7 @@ class Command:
if (
project
and not project.Derived
and (submodules_ok or project.sync_s)
and should_include_submodules(project)
):
search_again = False
for subproject in project.GetDerivedSubprojects():
+5 -1
View File
@@ -322,7 +322,10 @@ _repo() {
'--no-clone-bundle[Do not use clone bundle]' \
'(-u --manifest-server-username)'{-u,--manifest-server-username=}'[Username for manifest server]:username:' \
'(-p --manifest-server-password)'{-p,--manifest-server-password=}'[Password for manifest server]:password:' \
'--fetch-submodules[Fetch submodules]' \
'--recurse-submodules[Sync submodules]' \
'--no-recurse-submodules[Do not sync submodules]' \
'--fetch-submodules[Deprecated alias for --recurse-submodules]' \
'--no-fetch-submodules[Deprecated alias for --no-recurse-submodules]' \
'--use-superproject[Use superproject]' \
'--no-use-superproject[Do not use superproject]' \
'--tags[Sync tags]' \
@@ -397,6 +400,7 @@ _repo() {
'--no-verify[Do not verify]' \
'--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \
'--fix[Automatically fix]' \
'*: :->project'
;;
version)
+63
View File
@@ -0,0 +1,63 @@
# Fetch Command Contract
The `repo.fetchcmd` configuration allows specifying a custom command to be
executed during `repo sync` to fetch objects, instead of using standard
`git fetch`. This is particularly useful in environments with virtualized
filesystems or lazy checkouts where fetching metadata and downloading file
contents should be decoupled.
## Configuration
To use this feature, set the following in `.repo/manifests.git/config`:
```ini
[repo]
fetchcmd = "your custom command here"
uselocalgitdirs = true
```
Setting `repo.fetchcmd` **requires** `repo.uselocalgitdirs` to be set to `true`.
## Environment Variables
The custom command is executed in a subshell populated with standard
project-context environment variables. For details on standard variables (such
as `REPO_PROJECT`, `REPO_PATH`, `REPO_PROJECT_FETCH_URL`, etc.), see the
Environment section in `repo help forall` or `subcmds/forall.py`.
The following environment variable is specific to `repo.fetchcmd`:
* `REPO_TREV`: The target revision resolved to a full commit hash.
## Contract
### Postconditions on exit 0
After the fetch command exits with status 0, `repo` expects the following
postconditions to be met:
1. `git cat-file -e REPO_TREV` succeeds (the commit must exist in the object
store).
2. The mapped local tracking ref (e.g. `refs/remotes/REPO_REMOTE/<branch>`
for a branch revision, or the tag ref itself for a tag) must point to
`REPO_TREV`.
3. `FETCH_HEAD` must point to `REPO_TREV`.
4. The commit graph from `REPO_TREV` must be reachable far enough to compute
merge bases with local branches.
### Invariants
* The command should be idempotent; fetching the same `REPO_TREV` twice should
be a no-op.
* Only `FETCH_HEAD` and `refs/remotes/*` should be modified to preserve
`repo sync --network-only` semantics. `HEAD` and local branches must not be
touched by the fetch command.
* Dirty worktree state must be preserved.
* The command is **not** executed for `MetaProject`s (i.e. the internal `repo`
repository itself at `.repo/repo` and the `manifests` repository at
`.repo/manifests`).
### Failure
* A non-zero exit status aborts the project's sync, and the command's stderr
is surfaced to the user.
* `repo` verifies the tracking ref and target reachability after exit 0. Any
mismatch is treated as a failure.
+1 -1
View File
@@ -182,7 +182,7 @@ User controlled settings are initialized when running `repo init`.
| user.email | `--config-name` | User's e-mail address; Copied into `.git/config` when checking out a new project |
| user.name | `--config-name` | User's name; Copied into `.git/config` when checking out a new project |
[partial git clones]: https://git-scm.com/docs/gitrepository-layout#_code_partialclone_code
[partial git clones]: https://git-scm.com/docs/partial-clone
[superproject]: https://en.wikibooks.org/wiki/Git/Submodules_and_Superprojects
### Repo hooks settings
+6
View File
@@ -58,6 +58,7 @@ following DTD:
<!ELEMENT manifest-server EMPTY>
<!ATTLIST manifest-server url CDATA #REQUIRED>
<!ATTLIST manifest-server helper CDATA #IMPLIED>
<!ELEMENT submanifest EMPTY>
<!ATTLIST submanifest name ID #REQUIRED>
@@ -239,6 +240,11 @@ XML RPC service.
See the [smart sync documentation](./smart-sync.md) for more details.
Attribute `url`: The URL of the manifest server.
Attribute `helper`: Optional name of a remote helper binary to execute to
resolve proxying or authentication for the manifest server.
### Element submanifest
+18 -1
View File
@@ -83,6 +83,20 @@ then check it directly. Hooks should not normally modify the active git repo
the user. Although user interaction is discouraged in the common case, it can
be useful when deploying automatic fixes.
### Safe Prompts
If the repo command that triggered the hook supports a "yes" option (e.g.,
`repo upload --yes`), this option is propagated to the hook's `main` function
as `yes` parameter (defaulting to `False`). Hooks can use this to bypass
interactive confirmation prompts for safe non-modifying operations.
### Automated Fixes
If the repo command that triggered the hook supports a "fix" option (e.g.,
`repo upload --fix`), this option is propagated to the hook's `main` function
as `fix` parameter (defaulting to `False`). Hooks can use this to automatically
apply fixes without prompting the user.
### Shebang Handling
*** note
@@ -119,7 +133,7 @@ This hook runs when people run `repo upload`.
The `pre-upload.py` file should be defined like:
```py
def main(project_list, worktree_list=None, **kwargs):
def main(project_list, worktree_list=None, fix=False, yes=False, **kwargs):
"""Main function invoked directly by repo.
We must use the name "main" as that is what repo requires.
@@ -130,6 +144,9 @@ def main(project_list, worktree_list=None, **kwargs):
project_list, so that each entry in project_list matches with a
directory in worktree_list. If None, we will attempt to calculate
the directories automatically.
fix: Whether to automatically apply fixes without prompting.
yes: Whether to answer yes to all safe prompts (see
[Safe Prompts](#safe-prompts)).
kwargs: Leave this here for forward-compatibility.
"""
```
+32 -1
View File
@@ -27,13 +27,44 @@ the [`<manifest-server>` element](manifest-format.md#Element-manifest_server)
element. This is how the client knows what service to talk to.
```xml
<manifest-server url="https://example.com/your/manifest/server/url" />
<manifest-server url="https://example.com/your/manifest/server/url"
helper="repo-remote-helper-name" />
```
If the URL starts with `persistent-`, then the
[`git-remote-persistent-https` helper](https://github.com/git/git/blob/HEAD/contrib/persistent-https/README)
is used to communicate with the server.
### Pluggable Remote Helpers
For custom proxying or authentication, Repo supports pluggable remote helpers.
You can declare a helper binary via the optional `helper` attribute on the
`<manifest-server>` element.
If the `helper` attribute is present in `<manifest-server>`:
1. Repo searches your system `PATH` for the specified helper binary (e.g.,
`repo-remote-sso`). If the helper cannot be found in `PATH`, Repo will raise
a `SmartSyncError` and abort the sync.
2. The helper is executed with the manifest server URL as its first argument:
```bash
<helper-binary-name> <url>
```
3. The helper must output a single-line JSON object on stdout and exit:
* On success, return `status: "ok"` and a loopback proxy URL (including
scheme, e.g., `http://127.0.0.1:999`):
```json
{"status": "ok", "message": "http://127.0.0.1:999"}
```
* On failure, return `status: "error"` and a detailed error message:
```json
{"status": "error", "message": "unauthorized"}
```
4. Repo routes the XML-RPC request through the returned proxy address.
**Timeout Constraint**: The remote helper must complete and exit within 10
seconds. If it hangs or exceeds this limit, Repo will force-terminate (`kill`)
the process and abort the synchronization.
## Credentials
Credentials may be specified directly in typical `username:password`
+13
View File
@@ -70,6 +70,19 @@ class _GitCall:
git = _GitCall()
def IsValidBranchName(name: str) -> bool:
"""Return whether |name| is valid where Git expects a branch name."""
p = GitCommand(
None,
["check-ref-format", "--branch", name],
capture_stdout=True,
capture_stderr=True,
add_event_log=False,
log_as_error=False,
)
return p.Wait() == 0
def RepoSourceVersion():
"""Return the version of the repo.git tree."""
ver = getattr(RepoSourceVersion, "version", None)
+3 -3
View File
@@ -40,7 +40,7 @@ from repo_trace import Trace
# that is saved in the config.
SYNC_STATE_PREFIX = "repo.syncstate."
ID_RE = re.compile(r"^[0-9a-f]{40}$")
ID_RE = re.compile(r"^(?:[0-9a-f]{40}|[0-9a-f]{64})$")
REVIEW_CACHE = {}
@@ -49,8 +49,8 @@ def IsChange(rev):
return rev.startswith(R_CHANGES)
def IsId(rev):
return ID_RE.match(rev)
def IsId(rev: str) -> bool:
return bool(ID_RE.match(rev))
def IsTag(rev):
+37 -6
View File
@@ -14,6 +14,7 @@
import os
from git_command import git_require
from git_command import GitCommand
import platform_utils
from repo_trace import Trace
@@ -41,6 +42,17 @@ class GitRefs:
self._EnsureLoaded()
return self._phyref
@property
def head(self) -> str:
"""Return HEAD's symbolic target or detached object ID."""
self._EnsureLoaded()
return self._symref.get(HEAD) or self._phyref.get(HEAD, "")
@property
def is_loaded(self) -> bool:
"""Whether a ref snapshot has already been loaded."""
return self._phyref is not None
def get(self, name):
try:
return self.all[name]
@@ -87,8 +99,12 @@ class GitRefs:
self._symref = {}
self._mtime = {}
self._ReadRefs()
self._ReadSymbolicRef(HEAD)
root_refs_loaded = self._ReadRefs()
if not root_refs_loaded or (
HEAD not in self._phyref and HEAD not in self._symref
):
# --include-root-refs does not report an unborn HEAD.
self._ReadSymbolicRef(HEAD)
scan = self._symref
attempts = 0
@@ -113,18 +129,32 @@ class GitRefs:
"""Check if a ref_id is a null object ID."""
return ref_id and all(ch == "0" for ch in ref_id)
def _ReadRefs(self) -> None:
"""Read all references using git for-each-ref."""
def _ReadRefs(self) -> bool:
"""Read all references using git for-each-ref.
Returns:
Whether root refs, including HEAD when it exists, were loaded.
"""
include_root_refs = git_require((2, 45, 0))
cmd = [
"for-each-ref",
"--format=%(objectname)%00%(refname)%00%(symref)",
]
if include_root_refs:
cmd.insert(1, "--include-root-refs")
# Avoid caching volatile root refs such as ORIG_HEAD. HEAD and
# refs/* are the only namespaces GitRefs exposes to callers.
cmd.extend([HEAD, "refs"])
p = GitCommand(
None,
["for-each-ref", "--format=%(objectname)%00%(refname)%00%(symref)"],
cmd,
capture_stdout=True,
capture_stderr=True,
bare=True,
gitdir=self._gitdir,
)
if p.Wait() != 0:
return
return False
for line in p.stdout.splitlines():
ref_id, name, symref = line.split("\0")
@@ -132,6 +162,7 @@ class GitRefs:
self._symref[name] = symref
elif ref_id and not self._IsNullRef(ref_id):
self._phyref[name] = ref_id
return include_root_refs
def _ReadSymbolicRef(self, name: str) -> None:
"""Read a symbolic reference."""
+21 -4
View File
@@ -36,7 +36,6 @@ from git_command import git_require
from git_command import GitCommand
from git_config import IsId
from git_config import RepoConfig
from git_refs import GitRefs
import platform_utils
@@ -189,7 +188,7 @@ class Superproject:
if netloc:
parts = netloc.split("-review", 1)
host = parts[0]
rev = GitRefs(self._work_git).get("HEAD")
rev = self._GetRef("HEAD")
return f"{host}/{self.name}@{rev}"
return None
@@ -314,7 +313,10 @@ class Superproject:
# We use --negotiation-tip to speed up the fetch. Superproject branches
# do not share commits. So this lets git know it only needs to send
# commits reachable from the specified local refs.
rev_commit = GitRefs(self._work_git).get(f"refs/heads/{self.revision}")
negotiation_ref = self.revision
if negotiation_ref and not negotiation_ref.startswith("refs/"):
negotiation_ref = f"refs/heads/{negotiation_ref}"
rev_commit = self._GetRef(negotiation_ref) if negotiation_ref else ""
if rev_commit:
cmd.extend(["--negotiation-tip", rev_commit])
@@ -347,6 +349,21 @@ class Superproject:
return False
return True
def _GetRef(self, ref: str) -> str:
"""Resolve one local ref without loading the entire ref namespace."""
p = GitCommand(
None,
["rev-parse", "--verify", "--quiet", ref],
gitdir=self._work_git,
bare=True,
capture_stdout=True,
capture_stderr=True,
log_as_error=False,
)
if p.Wait() == 0:
return p.stdout.strip()
return ""
def _LsTree(self):
"""Gets the commit ids for all projects.
@@ -473,7 +490,7 @@ class Superproject:
)
return None
manifest_str = self._manifest.ToXml(
filter_groups=self._manifest.GetManifestGroupsStr(),
filter_groups="all",
omit_local=True,
).toxml()
manifest_path = self._manifest_path
+7 -1
View File
@@ -195,7 +195,13 @@ class BaseEventLog:
def GetDataEventName(self, value):
"""Returns 'data-json' if the value is an array else returns 'data'."""
return "data-json" if value[0] == "[" and value[-1] == "]" else "data"
return (
"data-json"
if isinstance(value, str)
and value.startswith("[")
and value.endswith("]")
else "data"
)
def LogDataConfigEvents(self, config, prefix):
"""Append a 'data' event for each entry in |config| to the current log.
+27 -3
View File
@@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import optparse
import os
import re
import sys
@@ -68,6 +69,8 @@ class RepoHook:
allow_all_hooks=False,
ignore_hooks=False,
abort_if_user_denies=False,
yes=False,
fix=False,
):
"""RepoHook constructor.
@@ -89,6 +92,8 @@ class RepoHook:
ignore_hooks: If True, then 'Do not abort action if hooks fail'.
abort_if_user_denies: If True, we'll abort running the hook if the
user doesn't allow us to run the hook.
yes: If True, then 'Yes' is assumed for any prompts.
fix: If True, then 'Fix' is assumed for any fixup prompts.
"""
self._hook_type = hook_type
self._hooks_project = hooks_project
@@ -99,6 +104,8 @@ class RepoHook:
self._allow_all_hooks = allow_all_hooks
self._ignore_hooks = ignore_hooks
self._abort_if_user_denies = abort_if_user_denies
self._yes = yes
self._fix = fix
# Store the full path to the script for convenience.
self._script_fullpath = None
@@ -374,8 +381,12 @@ class RepoHook:
# def main(project_list, **kwargs):
#
# This allows us to later expand the API without breaking old hooks.
kwargs = kwargs.copy()
kwargs["hook_should_take_kwargs"] = True
kwargs = {
**kwargs,
"hook_should_take_kwargs": True,
"fix": self._fix,
"yes": self._yes,
}
# See what version of python the hook has been written against.
data = open(self._script_fullpath).read()
@@ -497,12 +508,18 @@ class RepoHook:
"origin"
).url,
"bug_url": manifest.contactinfo.bugurl,
"yes": getattr(opt, "yes", False),
"fix": getattr(opt, "fix", False),
}
)
return cls(*args, **kwargs)
@staticmethod
def AddOptionGroup(parser, name):
def AddOptionGroup(
parser: optparse.OptionParser,
name: str,
allow_fix: bool = False,
) -> None:
"""Help options relating to the various hooks."""
# Note that verify and no-verify are NOT opposites of each other, which
@@ -526,3 +543,10 @@ class RepoHook:
action="store_true",
help="Do not abort if %s hooks fail." % name,
)
if allow_fix:
group.add_option(
"--fix",
action="store_true",
default=False,
help="Automatically apply %s fixes without prompting." % name,
)
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo abandon" "Repo Manual"
.TH REPO "1" "June 2026" "repo abandon" "Repo Manual"
.SH NAME
repo \- repo abandon - manual page for repo abandon
.SH SYNOPSIS
@@ -20,8 +20,7 @@ It is equivalent to "git branch \fB\-D\fR <branchname>".
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-\-all\fR
delete all branches in all projects
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo branches" "Repo Manual"
.TH REPO "1" "June 2026" "repo branches" "Repo Manual"
.SH NAME
repo \- repo branches - manual page for repo branches
.SH SYNOPSIS
@@ -46,8 +46,7 @@ is shown, then the branch appears in all projects.
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo checkout" "Repo Manual"
.TH REPO "1" "June 2026" "repo checkout" "Repo Manual"
.SH NAME
repo \- repo checkout - manual page for repo checkout
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Checkout a branch for development
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo diff" "Repo Manual"
.TH REPO "1" "June 2026" "repo diff" "Repo Manual"
.SH NAME
repo \- repo diff - manual page for repo diff
.SH SYNOPSIS
@@ -19,8 +19,7 @@ to the Unix 'patch' command.
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-u\fR, \fB\-\-absolute\fR
paths are relative to the repository root
+8 -11
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "May 2026" "repo forall" "Repo Manual"
.TH REPO "1" "June 2026" "repo forall" "Repo Manual"
.SH NAME
repo \- repo forall - manual page for repo forall
.SH SYNOPSIS
@@ -17,20 +17,16 @@ repo forall \fB\-r\fR str1 [str2] ... \fB\-c\fR <command> [<arg>...]
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-r\fR, \fB\-\-regex\fR
execute the command only on projects matching regex or
wildcard expression
execute the command only on projects matching regex or wildcard expression
.TP
\fB\-i\fR, \fB\-\-inverse\-regex\fR
execute the command only on projects not matching
regex or wildcard expression
execute the command only on projects not matching regex or wildcard expression
.TP
\fB\-g\fR GROUPS, \fB\-\-groups\fR=\fI\,GROUPS\/\fR
execute the command only on projects matching the
specified groups
execute the command only on projects matching the specified groups
.TP
\fB\-c\fR, \fB\-\-command\fR
command (and arguments) to execute
@@ -39,8 +35,7 @@ command (and arguments) to execute
abort if a command exits unsuccessfully
.TP
\fB\-\-ignore\-missing\fR
silently skip & do not exit non\-zero due missing
checkouts
silently skip & do not exit non\-zero due missing checkouts
.TP
\fB\-\-interactive\fR
force interactive usage
@@ -125,6 +120,8 @@ REPO_UPSTREAM is the name of the upstream branch as specified in the manifest.
REPO_DEST_BRANCH is the name of the destination branch for code review, as
specified in the manifest.
.PP
REPO_PROJECT_FETCH_URL is the full resolved fetch URL for the project.
.PP
REPO_COUNT is the total number of projects being iterated.
.PP
REPO_I is the current (1\-based) iteration count. Can be used in conjunction with
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "April 2025" "repo gc" "Repo Manual"
.TH REPO "1" "June 2026" "repo gc" "Repo Manual"
.SH NAME
repo \- repo gc - manual page for repo gc
.SH SYNOPSIS
@@ -21,8 +21,7 @@ do everything except actually delete
answer yes to all safe prompts
.TP
\fB\-\-repack\fR
repack all projects that use partial clone with
filter=blob:none
repack all projects that use partial clone with filter=blob:none
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo grep" "Repo Manual"
.TH REPO "1" "June 2026" "repo grep" "Repo Manual"
.SH NAME
repo \- repo grep - manual page for repo grep
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Print lines matching a pattern
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.SS Logging options:
.TP
\fB\-\-verbose\fR
+3 -5
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "May 2026" "repo info" "Repo Manual"
.TH REPO "1" "June 2026" "repo info" "Repo Manual"
.SH NAME
repo \- repo info - manual page for repo info
.SH SYNOPSIS
@@ -15,12 +15,10 @@ Get info on the manifest branch, current branch or unmerged branches
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-d\fR, \fB\-\-diff\fR
show full info and commit diff including remote
branches
show full info and commit diff including remote branches
.TP
\fB\-o\fR, \fB\-\-overview\fR
show overview of all local commits
+19 -30
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "September 2024" "repo init" "Repo Manual"
.TH REPO "1" "June 2026" "repo init" "Repo Manual"
.SH NAME
repo \- repo init - manual page for repo init
.SH SYNOPSIS
@@ -29,36 +29,29 @@ manifest repository location
manifest branch or revision (use HEAD for default)
.TP
\fB\-\-manifest\-upstream\-branch\fR=\fI\,BRANCH\/\fR
when a commit is provided to \fB\-\-manifest\-branch\fR, this
is the name of the git ref in which the commit can be
found
when a commit is provided to \fB\-\-manifest\-branch\fR, this is the name of the git ref in which the commit can be found
.TP
\fB\-m\fR NAME.xml, \fB\-\-manifest\-name\fR=\fI\,NAME\/\fR.xml
initial manifest file
.TP
\fB\-g\fR GROUP, \fB\-\-groups\fR=\fI\,GROUP\/\fR
restrict manifest projects to ones with specified
group(s) [default|all|G1,G2,G3|G4,\-G5,\-G6]
restrict manifest projects to ones with specified group(s) [default|all|G1,G2,G3|G4,\-G5,\-G6]
.TP
\fB\-p\fR PLATFORM, \fB\-\-platform\fR=\fI\,PLATFORM\/\fR
restrict manifest projects to ones with a specified
platform group [auto|all|none|linux|darwin|...]
restrict manifest projects to ones with a specified platform group [auto|all|none|linux|darwin|...]
.TP
\fB\-\-submodules\fR
sync any submodules associated with the manifest repo
.TP
\fB\-\-standalone\-manifest\fR
download the manifest as a static file rather then
create a git checkout of the manifest repo
download the manifest as a static file rather then create a git checkout of the manifest repo
.TP
\fB\-\-manifest\-depth\fR=\fI\,DEPTH\/\fR
create a shallow clone of the manifest repo with given
depth (0 for full clone); see git clone (default: 0)
create a shallow clone of the manifest repo with given depth (0 for full clone); see git clone (default: 0)
.SS Manifest (only) checkout options:
.TP
\fB\-c\fR, \fB\-\-current\-branch\fR
fetch only current manifest branch from server
(default)
fetch only current manifest branch from server (default)
.TP
\fB\-\-no\-current\-branch\fR
fetch all manifest branches from server
@@ -71,15 +64,16 @@ don't fetch tags in the manifest
.SS Checkout modes:
.TP
\fB\-\-mirror\fR
create a replica of the remote repositories rather
than a client working directory
create a replica of the remote repositories rather than a client working directory
.TP
\fB\-\-archive\fR
checkout an archive instead of a git repository for
each project. See git archive.
checkout an archive instead of a git repository for each project. See git archive.
.TP
\fB\-\-worktree\fR
use git\-worktree to manage projects
.TP
\fB\-\-use\-local\-gitdirs\fR
bypass .repo/projects/ and use standard Git layout in working tree
.SS Project checkout optimizations:
.TP
\fB\-\-reference\fR=\fI\,DIR\/\fR
@@ -92,33 +86,28 @@ dissociate from reference mirrors after clone
create a shallow clone with given depth; see git clone
.TP
\fB\-\-partial\-clone\fR
perform partial clone (https://gitscm.com/docs/gitrepositorylayout#_code_partialclone_code)
perform partial clone (https://git\-scm.com/docs/partial\-clone)
.TP
\fB\-\-no\-partial\-clone\fR
disable use of partial clone (https://gitscm.com/docs/gitrepositorylayout#_code_partialclone_code)
disable use of partial clone (https://git\-scm.com/docs/partial\-clone)
.TP
\fB\-\-partial\-clone\-exclude\fR=\fI\,PARTIAL_CLONE_EXCLUDE\/\fR
exclude the specified projects (a comma\-delimited
project names) from partial clone (https://gitscm.com/docs/gitrepositorylayout#_code_partialclone_code)
exclude the specified projects (a comma\-delimited project names) from partial clone (https://git\-scm.com/docs/partial\-clone)
.TP
\fB\-\-clone\-filter\fR=\fI\,CLONE_FILTER\/\fR
filter for use with \fB\-\-partial\-clone\fR [default:
blob:none]
filter for use with \fB\-\-partial\-clone\fR [default: blob:none]
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects;
implies \fB\-c\fR
use the manifest superproject to sync projects; implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-\-clone\-bundle\fR
enable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if
not \fB\-\-partial\-clone\fR)
enable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if not \fB\-\-partial\-clone\fR)
.TP
\fB\-\-no\-clone\-bundle\fR
disable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if
\fB\-\-partial\-clone\fR)
disable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if \fB\-\-partial\-clone\fR)
.TP
\fB\-\-git\-lfs\fR
enable Git LFS support
+5 -9
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo list" "Repo Manual"
.TH REPO "1" "June 2026" "repo list" "Repo Manual"
.SH NAME
repo \- repo list - manual page for repo list
.SH SYNOPSIS
@@ -17,12 +17,10 @@ repo list [\-f] \fB\-r\fR str1 [str2]...
show this help message and exit
.TP
\fB\-r\fR, \fB\-\-regex\fR
filter the project list based on regex or wildcard
matching of strings
filter the project list based on regex or wildcard matching of strings
.TP
\fB\-g\fR GROUPS, \fB\-\-groups\fR=\fI\,GROUPS\/\fR
filter the project list based on the groups the
project is in
filter the project list based on the groups the project is in
.TP
\fB\-a\fR, \fB\-\-all\fR
show projects regardless of checkout state
@@ -34,12 +32,10 @@ display only the name of the repository
display only the path of the repository
.TP
\fB\-f\fR, \fB\-\-fullpath\fR
display the full work tree path instead of the
relative path
display the full work tree path instead of the relative path
.TP
\fB\-\-relative\-to\fR=\fI\,PATH\/\fR
display paths relative to this one (default: top of
repo client checkout)
display paths relative to this one (default: top of repo client checkout)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+10 -9
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "April 2026" "repo manifest" "Repo Manual"
.TH REPO "1" "June 2026" "repo manifest" "Repo Manual"
.SH NAME
repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS
@@ -21,14 +21,10 @@ save revisions as current HEAD
temporary manifest to use for this sync
.TP
\fB\-\-suppress\-upstream\-revision\fR
if in \fB\-r\fR mode, do not write the upstream field (only
of use if the branch names for a sha1 manifest are
sensitive)
if in \fB\-r\fR mode, do not write the upstream field (only of use if the branch names for a sha1 manifest are sensitive)
.TP
\fB\-\-suppress\-dest\-branch\fR
if in \fB\-r\fR mode, do not write the dest\-branch field
(only of use if the branch names for a sha1 manifest
are sensitive)
if in \fB\-r\fR mode, do not write the dest\-branch field (only of use if the branch names for a sha1 manifest are sensitive)
.TP
\fB\-\-format\fR=\fI\,FORMAT\/\fR
output format: xml, json (default: xml)
@@ -40,8 +36,7 @@ format output for humans to read
ignore local manifests
.TP
\fB\-o\fR \-|NAME.xml, \fB\-\-output\-file\fR=\fI\,\-\/\fR|NAME.xml
file to save the manifest to. (Filename prefix for
multi\-tree.)
file to save the manifest to. (Filename prefix for multi\-tree.)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
@@ -138,6 +133,7 @@ include*)>
.IP
<!ELEMENT manifest\-server EMPTY>
<!ATTLIST manifest\-server url CDATA #REQUIRED>
<!ATTLIST manifest\-server helper CDATA #IMPLIED>
.IP
<!ELEMENT submanifest EMPTY>
<!ATTLIST submanifest name ID #REQUIRED>
@@ -347,6 +343,11 @@ specify the URL of a manifest server, which is an XML RPC service.
.PP
See the [smart sync documentation](./smart\-sync.md) for more details.
.PP
Attribute `url`: The URL of the manifest server.
.PP
Attribute `helper`: Optional name of a remote helper binary to execute to
resolve proxying or authentication for the manifest server.
.PP
Element submanifest
.PP
One or more submanifest elements may be specified. Each element describes a
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo prune" "Repo Manual"
.TH REPO "1" "June 2026" "repo prune" "Repo Manual"
.SH NAME
repo \- repo prune - manual page for repo prune
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Prune (delete) already merged topics
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -4
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo rebase" "Repo Manual"
.TH REPO "1" "June 2026" "repo rebase" "Repo Manual"
.SH NAME
repo \- repo rebase - manual page for repo rebase
.SH SYNOPSIS
@@ -33,9 +33,7 @@ pass \fB\-\-whitespace\fR to git rebase
stash local modifications before starting
.TP
\fB\-m\fR, \fB\-\-onto\-manifest\fR
rebase onto the manifest version instead of upstream
HEAD (this helps to make sure the local tree stays
consistent if you previously synced to a manifest)
rebase onto the manifest version instead of upstream HEAD (this helps to make sure the local tree stays consistent if you previously synced to a manifest)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+22 -32
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "May 2026" "repo smartsync" "Repo Manual"
.TH REPO "1" "July 2026" "repo smartsync" "Repo Manual"
.SH NAME
repo \- repo smartsync - manual page for repo smartsync
.SH SYNOPSIS
@@ -15,16 +15,13 @@ Update working tree to the latest known good revision
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-\-jobs\-network\fR=\fI\,JOBS\/\fR
number of network jobs to run in parallel (defaults to
\fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel
(defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-nointerleaved\fR is set
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future)
@@ -33,30 +30,22 @@ obsolete option (to be deleted in the future)
stop syncing after first error is hit
.TP
\fB\-\-force\-sync\fR
overwrite an existing git directory if it needs to
point to a different object directory. WARNING: this
may cause loss of data
overwrite an existing git directory if it needs to point to a different object directory. WARNING: this may cause loss of data
.TP
\fB\-\-force\-checkout\fR
force checkout even if it results in throwing away
uncommitted modifications. WARNING: this may cause
loss of data
force checkout even if it results in throwing away uncommitted modifications. WARNING: this may cause loss of data
.TP
\fB\-\-force\-remove\-dirty\fR
force remove projects with uncommitted modifications
if projects no longer exist in the manifest. WARNING:
this may cause loss of data
force remove projects with uncommitted modifications if projects no longer exist in the manifest. WARNING: this may cause loss of data
.TP
\fB\-\-rebase\fR
rebase local commits regardless of whether they are
published
rebase local commits regardless of whether they are published
.TP
\fB\-l\fR, \fB\-\-local\-only\fR
only update working tree, don't fetch
.TP
\fB\-\-no\-manifest\-update\fR, \fB\-\-nmu\fR
use the existing manifest checkout as\-is. (do not
update to the latest revision)
use the existing manifest checkout as\-is. (do not update to the latest revision)
.TP
\fB\-\-interleaved\fR
fetch and checkout projects in parallel (default)
@@ -79,6 +68,9 @@ fetch all branches from server
\fB\-m\fR NAME.xml, \fB\-\-manifest\-name\fR=\fI\,NAME\/\fR.xml
temporary manifest to use for this sync
.TP
\fB\-g\fR GROUP, \fB\-\-groups\fR=\fI\,GROUP\/\fR
sync projects matching the specific groups. Not persistent unlike when used on init
.TP
\fB\-\-clone\-bundle\fR
enable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS
.TP
@@ -91,19 +83,20 @@ username to authenticate with the manifest server
\fB\-p\fR MANIFEST_SERVER_PASSWORD, \fB\-\-manifest\-server\-password\fR=\fI\,MANIFEST_SERVER_PASSWORD\/\fR
password to authenticate with the manifest server
.TP
\fB\-\-fetch\-submodules\fR
fetch submodules from server
\fB\-\-recurse\-submodules\fR
sync submodules from server
.TP
\fB\-\-no\-recurse\-submodules\fR
don't sync submodules from server
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects;
implies \fB\-c\fR
use the manifest superproject to sync projects; implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-\-superproject\-revision\fR=\fI\,SUPERPROJECT_REVISION\/\fR
sync to superproject revision (applies to outer
manifest)
sync to superproject revision (applies to outer manifest)
.TP
\fB\-\-tags\fR
fetch tags
@@ -112,15 +105,13 @@ fetch tags
don't fetch tags (default)
.TP
\fB\-\-optimized\-fetch\fR
only fetch projects fixed to sha1 if revision does not
exist locally
only fetch projects fixed to sha1 if revision does not exist locally
.TP
\fB\-\-retry\-fetches\fR=\fI\,RETRY_FETCHES\/\fR
number of times to retry fetches on transient errors
.TP
\fB\-\-prune\fR
delete refs that no longer exist on the remote
(default)
delete refs that no longer exist on the remote (default)
.TP
\fB\-\-no\-prune\fR
do not delete refs that no longer exist on the remote
@@ -129,8 +120,7 @@ do not delete refs that no longer exist on the remote
run garbage collection on all synced projects
.TP
\fB\-\-no\-auto\-gc\fR
do not run garbage collection on any projects
(default)
do not run garbage collection on any projects (default)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo start" "Repo Manual"
.TH REPO "1" "June 2026" "repo start" "Repo Manual"
.SH NAME
repo \- repo start - manual page for repo start
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Start a new branch for development
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-\-all\fR
begin branch in all projects
+12 -5
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo status" "Repo Manual"
.TH REPO "1" "June 2026" "repo status" "Repo Manual"
.SH NAME
repo \- repo status - manual page for repo status
.SH SYNOPSIS
@@ -15,12 +15,10 @@ Show the working tree status
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-o\fR, \fB\-\-orphans\fR
include objects in working directory outside of repo
projects
include objects in working directory outside of repo projects
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
@@ -70,6 +68,15 @@ branch devwork
\fB\-m\fR
subcmds/status.py
.PP
If the branch is tracking an upstream branch, the number of commits ahead and/or
behind is also shown:
.TP
project repo/
branch devwork [ahead 1, behind 2]
.TP
\fB\-m\fR
subcmds/status.py
.PP
The first column explains how the staging area (index) differs from the last
commit (HEAD). Its values are always displayed in upper case and have the
following meanings:
+29 -36
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "May 2026" "repo sync" "Repo Manual"
.TH REPO "1" "July 2026" "repo sync" "Repo Manual"
.SH NAME
repo \- repo sync - manual page for repo sync
.SH SYNOPSIS
@@ -15,16 +15,13 @@ Update working tree to the latest revision
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-\-jobs\-network\fR=\fI\,JOBS\/\fR
number of network jobs to run in parallel (defaults to
\fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel
(defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-nointerleaved\fR is set
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future)
@@ -33,30 +30,22 @@ obsolete option (to be deleted in the future)
stop syncing after first error is hit
.TP
\fB\-\-force\-sync\fR
overwrite an existing git directory if it needs to
point to a different object directory. WARNING: this
may cause loss of data
overwrite an existing git directory if it needs to point to a different object directory. WARNING: this may cause loss of data
.TP
\fB\-\-force\-checkout\fR
force checkout even if it results in throwing away
uncommitted modifications. WARNING: this may cause
loss of data
force checkout even if it results in throwing away uncommitted modifications. WARNING: this may cause loss of data
.TP
\fB\-\-force\-remove\-dirty\fR
force remove projects with uncommitted modifications
if projects no longer exist in the manifest. WARNING:
this may cause loss of data
force remove projects with uncommitted modifications if projects no longer exist in the manifest. WARNING: this may cause loss of data
.TP
\fB\-\-rebase\fR
rebase local commits regardless of whether they are
published
rebase local commits regardless of whether they are published
.TP
\fB\-l\fR, \fB\-\-local\-only\fR
only update working tree, don't fetch
.TP
\fB\-\-no\-manifest\-update\fR, \fB\-\-nmu\fR
use the existing manifest checkout as\-is. (do not
update to the latest revision)
use the existing manifest checkout as\-is. (do not update to the latest revision)
.TP
\fB\-\-interleaved\fR
fetch and checkout projects in parallel (default)
@@ -79,6 +68,9 @@ fetch all branches from server
\fB\-m\fR NAME.xml, \fB\-\-manifest\-name\fR=\fI\,NAME\/\fR.xml
temporary manifest to use for this sync
.TP
\fB\-g\fR GROUP, \fB\-\-groups\fR=\fI\,GROUP\/\fR
sync projects matching the specific groups. Not persistent unlike when used on init
.TP
\fB\-\-clone\-bundle\fR
enable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS
.TP
@@ -91,19 +83,20 @@ username to authenticate with the manifest server
\fB\-p\fR MANIFEST_SERVER_PASSWORD, \fB\-\-manifest\-server\-password\fR=\fI\,MANIFEST_SERVER_PASSWORD\/\fR
password to authenticate with the manifest server
.TP
\fB\-\-fetch\-submodules\fR
fetch submodules from server
\fB\-\-recurse\-submodules\fR
sync submodules from server
.TP
\fB\-\-no\-recurse\-submodules\fR
don't sync submodules from server
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects;
implies \fB\-c\fR
use the manifest superproject to sync projects; implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-\-superproject\-revision\fR=\fI\,SUPERPROJECT_REVISION\/\fR
sync to superproject revision (applies to outer
manifest)
sync to superproject revision (applies to outer manifest)
.TP
\fB\-\-tags\fR
fetch tags
@@ -112,15 +105,13 @@ fetch tags
don't fetch tags (default)
.TP
\fB\-\-optimized\-fetch\fR
only fetch projects fixed to sha1 if revision does not
exist locally
only fetch projects fixed to sha1 if revision does not exist locally
.TP
\fB\-\-retry\-fetches\fR=\fI\,RETRY_FETCHES\/\fR
number of times to retry fetches on transient errors
.TP
\fB\-\-prune\fR
delete refs that no longer exist on the remote
(default)
delete refs that no longer exist on the remote (default)
.TP
\fB\-\-no\-prune\fR
do not delete refs that no longer exist on the remote
@@ -129,12 +120,10 @@ do not delete refs that no longer exist on the remote
run garbage collection on all synced projects
.TP
\fB\-\-no\-auto\-gc\fR
do not run garbage collection on any projects
(default)
do not run garbage collection on any projects (default)
.TP
\fB\-s\fR, \fB\-\-smart\-sync\fR
smart sync using manifest from the latest known good
build
smart sync using manifest from the latest known good build
.TP
\fB\-t\fR SMART_TAG, \fB\-\-smart\-tag\fR=\fI\,SMART_TAG\/\fR
smart sync using manifest from a known tag
@@ -229,8 +218,12 @@ bootstrap a new Git repository from a resumeable bundle file on a content
delivery network. This may be necessary if there are problems with the local
Python HTTP client or proxy configuration, but the Git binary works.
.PP
The \fB\-\-fetch\-submodules\fR option enables fetching Git submodules of a project from
server.
The \fB\-\-recurse\-submodules\fR option enables syncing Git submodules of all projects
from the server. The \fB\-\-no\-recurse\-submodules\fR option disables syncing Git
submodules, even when a project has sync\-s="true" in the manifest.
.PP
The \fB\-\-fetch\-submodules\fR and \fB\-\-no\-fetch\-submodules\fR options are deprecated aliases
for \fB\-\-recurse\-submodules\fR and \fB\-\-no\-recurse\-submodules\fR, respectively.
.PP
The \fB\-c\fR/\-\-current\-branch option can be used to only fetch objects that are on the
branch specified by a project's revision.
+5 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2024" "repo upload" "Repo Manual"
.TH REPO "1" "August 2026" "repo upload" "Repo Manual"
.SH NAME
repo \- repo upload - manual page for repo upload
.SH SYNOPSIS
@@ -15,8 +15,7 @@ Upload changes for code review
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
number of jobs to run in parallel (default: based on number of CPU cores)
.TP
\fB\-t\fR, \fB\-\-topic\-branch\fR
set the topic to the local branch name
@@ -113,6 +112,9 @@ Run the pre\-upload hook without prompting.
.TP
\fB\-\-ignore\-hooks\fR
Do not abort if pre\-upload hooks fail.
.TP
\fB\-\-fix\fR
Automatically apply pre\-upload fixes without prompting.
.PP
Run `repo help upload` to view the detailed manual.
.SH DETAILS
+2 -3
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "November 2025" "repo wipe" "Repo Manual"
.TH REPO "1" "June 2026" "repo wipe" "Repo Manual"
.SH NAME
repo \- repo wipe - manual page for repo wipe
.SH SYNOPSIS
@@ -21,8 +21,7 @@ force wipe shared projects and uncommitted changes
force wipe even if there are uncommitted changes
.TP
\fB\-\-force\-shared\fR
force wipe even if the project shares an object
directory
force wipe even if the project shares an object directory
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+3 -5
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "November 2025" "repo" "Repo Manual"
.TH REPO "1" "June 2026" "repo" "Repo Manual"
.SH NAME
repo \- repository management tool built on top of git
.SH SYNOPSIS
@@ -26,8 +26,7 @@ control color usage: auto, always, never
trace git command execution (REPO_TRACE=1)
.TP
\fB\-\-trace\-to\-stderr\fR
trace outputs go to stderr in addition to
\&.repo/TRACE_FILE
trace outputs go to stderr in addition to .repo/TRACE_FILE
.TP
\fB\-\-trace\-python\fR
trace python command execution
@@ -39,8 +38,7 @@ time repo command execution
display this version of repo
.TP
\fB\-\-show\-toplevel\fR
display the path of the top\-level directory of the
repo client checkout
display the path of the top\-level directory of the repo client checkout
.TP
\fB\-\-event\-log\fR=\fI\,EVENT_LOG\/\fR
filename of event log to append timeline to
+30 -10
View File
@@ -651,6 +651,8 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if self._manifest_server:
e = doc.createElement("manifest-server")
e.setAttribute("url", self._manifest_server)
if self._manifest_server_helper:
e.setAttribute("helper", self._manifest_server_helper)
root.appendChild(e)
root.appendChild(doc.createTextNode(""))
@@ -690,9 +692,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
e.setAttribute("remote", remoteName)
if peg_rev:
if self.IsMirror:
value = p.bare_git.rev_parse(p.revisionExpr + "^0")
value = p.bare_git.ResolveCommit(p.revisionExpr)
else:
value = p.work_git.rev_parse(HEAD + "^0")
value = p.work_git.ResolveCommit(HEAD)
e.setAttribute("revision", value)
if peg_rev_upstream:
if p.upstream:
@@ -999,6 +1001,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
self._Load()
return self._manifest_server
@property
def manifest_server_helper(self):
self._Load()
return self._manifest_server_helper
@property
def CloneBundle(self):
clone_bundle = self.manifestProject.clone_bundle
@@ -1055,6 +1062,10 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
def UseGitWorktrees(self):
return self.manifestProject.use_worktree
@property
def UseLocalGitDirs(self):
return self.manifestProject.use_local_gitdirs
@property
def IsArchive(self):
return self.manifestProject.archive
@@ -1153,6 +1164,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
self._notice = None
self.branch = None
self._manifest_server = None
self._manifest_server_helper = None
def Load(self):
"""Read the manifest into memory."""
@@ -1422,11 +1434,13 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
for node in itertools.chain(*node_list):
if node.nodeName == "manifest-server":
url = self._reqatt(node, "url")
helper = node.getAttribute("helper") or None
if self._manifest_server is not None:
raise ManifestParseError(
"duplicate manifest-server in %s" % (self.manifestFile)
)
self._manifest_server = url
self._manifest_server_helper = helper
def recursively_add_projects(project):
projects = self._projects.setdefault(project.name, [])
@@ -2042,15 +2056,21 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
else:
namepath = f"{name}.git"
worktree = os.path.join(self.topdir, path).replace("\\", "/")
gitdir = os.path.join(self.subdir, "projects", "%s.git" % path)
# We allow people to mix git worktrees & non-git worktrees for now.
# This allows for in situ migration of repo clients.
if os.path.exists(gitdir) or not self.UseGitWorktrees:
objdir = os.path.join(self.repodir, "project-objects", namepath)
else:
use_git_worktrees = True
gitdir = os.path.join(self.repodir, "worktrees", namepath)
if self.UseLocalGitDirs:
gitdir = os.path.join(worktree, ".git")
objdir = gitdir
else:
gitdir = os.path.join(self.subdir, "projects", "%s.git" % path)
# We allow people to mix git worktrees & non-git worktrees for
# now. This allows for in situ migration of repo clients.
if os.path.exists(gitdir) or not self.UseGitWorktrees:
objdir = os.path.join(
self.repodir, "project-objects", namepath
)
else:
use_git_worktrees = True
gitdir = os.path.join(self.repodir, "worktrees", namepath)
objdir = gitdir
return relpath, worktree, gitdir, objdir, use_git_worktrees
def GetProjectsWithName(self, name, all_manifests=False):
+772 -157
View File
File diff suppressed because it is too large Load Diff
+1
View File
@@ -76,6 +76,7 @@ def main(argv: List[str]) -> int:
# behavior (like probing active number of CPUs). We use a weird name &
# value to make it less likely for users to set this var themselves.
os.environ["_REPO_GENERATE_MANPAGES_"] = " indeed! "
os.environ["COLUMNS"] = "10000"
# "repo branch" is an alias for "repo branches".
del subcmds.all_commands["branch"]
+32 -5
View File
@@ -129,7 +129,7 @@ if not REPO_REV:
BUG_URL = "https://issues.gerritcodereview.com/issues/new?component=1370071"
# increment this whenever we make important changes to this script
VERSION = (2, 54)
VERSION = (2, 65)
# increment this if the MAINTAINER_KEYS block is modified
KEYRING_VERSION = (2, 3)
@@ -379,6 +379,11 @@ def InitParser(parser):
action="store_true",
help="use git-worktree to manage projects",
)
group.add_option(
"--use-local-gitdirs",
action="store_true",
help="bypass .repo/projects/ and use standard Git layout in working tree",
)
# These are fundamentally different ways of structuring the checkout.
group = parser.add_option_group("Project checkout optimizations")
@@ -400,20 +405,20 @@ def InitParser(parser):
"--partial-clone",
action="store_true",
help="perform partial clone (https://git-scm.com/"
"docs/gitrepository-layout#_code_partialclone_code)",
"docs/partial-clone)",
)
group.add_option(
"--no-partial-clone",
action="store_false",
help="disable use of partial clone (https://git-scm.com/"
"docs/gitrepository-layout#_code_partialclone_code)",
"docs/partial-clone)",
)
group.add_option(
"--partial-clone-exclude",
action="store",
help="exclude the specified projects (a comma-delimited "
"project names) from partial clone (https://git-scm.com"
"/docs/gitrepository-layout#_code_partialclone_code)",
"/docs/partial-clone)",
)
group.add_option(
"--clone-filter",
@@ -742,6 +747,28 @@ def SetGitTrace2ParentSid(env=None):
_setenv(KEY, value, env=env)
def _NormalizeGpgHomePath(path: str) -> str:
"""Convert path for GPG on Windows if running in MSYS/MinGW.
If GPG is MSYS-based (common in Git Bash/Cygwin), it expects POSIX-like paths
(e.g. /c/Users/... instead of C:\\Users\\...). Native Windows Python uses
Windows-style paths, which MSYS GPG will incorrectly treat as relative paths.
We use cygpath to query the active MSYS/Cygwin mount configuration and resolve
the correct POSIX path.
"""
if platform.system() == "Windows":
try:
out = subprocess.check_output(
["cygpath", "-u", path],
stderr=subprocess.DEVNULL,
universal_newlines=True,
)
return out.strip()
except (OSError, subprocess.CalledProcessError):
pass
return path
def _setenv(key, value, env=None):
"""Set |key| in the OS environment |env| to |value|."""
if env is None:
@@ -1074,7 +1101,7 @@ def verify_rev(cwd, remote_ref, rev, quiet):
print(file=sys.stderr)
env = os.environ.copy()
_setenv("GNUPGHOME", gpg_dir, env)
_setenv("GNUPGHOME", _NormalizeGpgHomePath(gpg_dir), env)
run_git("tag", "-v", cur, cwd=cwd, env=env)
return "%s^0" % cur
+2 -5
View File
@@ -20,7 +20,7 @@ from command import Command
from command import DEFAULT_LOCAL_JOBS
from error import RepoError
from error import RepoExitError
from git_command import git
from git_command import IsValidBranchName
from progress import Progress
from repo_logging import RepoLogger
@@ -58,9 +58,7 @@ It is equivalent to "git branch -D <branchname>".
if not opt.all:
branches = args[0].split()
invalid_branches = [
x for x in branches if not git.check_ref_format(f"heads/{x}")
]
invalid_branches = [x for x in branches if not IsValidBranchName(x)]
if invalid_branches:
self.OptionParser.error(
@@ -94,7 +92,6 @@ It is equivalent to "git branch -D <branchname>".
def Execute(self, opt, args):
nb = args[0].split()
self.TryOverrideManifestWithSmartSync()
err = collections.defaultdict(list)
success = collections.defaultdict(list)
aggregate_errors = []
+45 -30
View File
@@ -14,6 +14,7 @@
import re
import sys
from typing import Tuple
from command import Command
from error import GitError
@@ -43,36 +44,8 @@ change id will be added.
def Execute(self, opt, args):
reference = args[0]
p = GitCommand(
None,
["rev-parse", "--verify", reference],
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
try:
p.Wait()
except GitError:
logger.error(p.stderr)
raise
sha1 = p.stdout.strip()
p = GitCommand(
None,
["cat-file", "commit", sha1],
capture_stdout=True,
verify_command=True,
)
try:
p.Wait()
except GitError:
logger.error("error: Failed to retrieve old commit message")
raise
old_msg = self._StripHeader(p.stdout)
sha1, commit = self._ResolveReference(reference)
old_msg = self._StripHeader(commit)
p = GitCommand(
None,
@@ -117,6 +90,48 @@ change id will be added.
logger.error("error: Failed to update commit message")
raise
def _ResolveReference(self, reference: str) -> Tuple[str, str]:
"""Resolve a commit and read it through one cat-file batch request."""
expression = f"{reference}^{{commit}}"
p = GitCommand(
None,
["cat-file", "--batch"],
input=expression + "\n",
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
try:
p.Wait()
header, separator, output = p.stdout.partition("\n")
if not separator:
raise ValueError("missing cat-file header")
if header.endswith(" missing") or header.endswith(" ambiguous"):
raise GitError(f"commit {reference} not found")
parts = header.split(" ", 2)
if len(parts) != 3 or parts[1] != "commit":
raise ValueError(
f"unexpected object type {parts[1]!r}"
if len(parts) >= 2
else "invalid header"
)
sha1, _object_type, _size = parts
if not output.endswith("\n"):
raise ValueError("truncated cat-file object")
commit = output[:-1]
except (GitError, ValueError) as e:
logger.error(
"error: Failed to resolve or read commit %s", reference
)
if isinstance(e, GitError):
raise
raise GitError(str(e)) from e
return sha1, commit
def _IsChangeId(self, line):
return CHANGE_ID_RE.match(line)
+4 -22
View File
@@ -25,7 +25,6 @@ from color import Coloring
from command import Command
from command import DEFAULT_LOCAL_JOBS
from command import MirrorSafeCommand
from error import ManifestInvalidRevisionError
from repo_logging import RepoLogger
@@ -108,6 +107,8 @@ manifest.
REPO_DEST_BRANCH is the name of the destination branch for code review,
as specified in the manifest.
REPO_PROJECT_FETCH_URL is the full resolved fetch URL for the project.
REPO_COUNT is the total number of projects being iterated.
REPO_I is the current (1-based) iteration count. Can be used in
@@ -242,8 +243,6 @@ without iterating through the remaining projects.
mirror = self.manifest.IsMirror
self.TryOverrideManifestWithSmartSync()
if opt.regex:
projects = self.FindProjects(args, all_manifests=all_trees)
elif opt.inverse_regex:
@@ -339,25 +338,8 @@ def DoWork(project, mirror, opt, cmd, shell, cnt, config):
val = ""
env[name] = val
setenv("REPO_PROJECT", project.name)
setenv("REPO_OUTERPATH", project.manifest.path_prefix)
setenv("REPO_INNERPATH", project.relpath)
setenv("REPO_PATH", project.RelPath(local=opt.this_manifest_only))
setenv("REPO_REMOTE", project.remote.name)
try:
# If we aren't in a fully synced state and we don't have the ref the
# manifest wants, then this will fail. Ignore it for the purposes of
# this code.
lrev = "" if mirror else project.GetRevisionId()
except ManifestInvalidRevisionError:
lrev = ""
setenv("REPO_LREV", lrev)
setenv("REPO_RREV", project.revisionExpr)
setenv("REPO_UPSTREAM", project.upstream)
setenv("REPO_DEST_BRANCH", project.dest_branch)
setenv("REPO_I", str(cnt + 1))
for annotation in project.annotations:
setenv("REPO__%s" % (annotation.name), annotation.value)
env.update(project.GetEnvVars(local=opt.this_manifest_only))
env["REPO_I"] = str(cnt + 1)
if mirror:
setenv("GIT_DIR", project.gitdir)
+27 -6
View File
@@ -147,8 +147,6 @@ class Info(PagedCommand):
if not opt.this_manifest_only:
self.manifest = self.manifest.outer_client
self.TryOverrideManifestWithSmartSync()
output_format = OutputFormat[opt.format.upper()]
if output_format == OutputFormat.JSON:
self._ExecuteJson(opt, args)
@@ -191,12 +189,14 @@ class Info(PagedCommand):
"superproject_revision": srev,
}
def _getProjectData(self, project) -> Dict[str, Any]:
@classmethod
def _getProjectData(cls, project) -> Dict[str, Any]:
"""Gather project data as a dict."""
data = {
"name": project.name,
"mount_path": project.worktree,
"current_revision": project.GetRevisionId(),
"current_revision": project.GetHeadRevisionId()
or project.GetRevisionId(),
"manifest_revision": project.revisionExpr,
"local_branches": list(project.GetBranches()),
}
@@ -205,6 +205,12 @@ class Info(PagedCommand):
data["current_branch"] = currentBranch
return data
@classmethod
def _ProjectDataHelper(cls, project_idx: int) -> Dict[str, Any]:
"""Helper to get project data in parallel."""
project = cls.get_parallel_context()["projects"][project_idx]
return cls._getProjectData(project)
def _ExecuteJson(self, opt, args) -> None:
"""Output info as JSON."""
result = {}
@@ -214,7 +220,22 @@ class Info(PagedCommand):
projs = self.GetProjects(
args, all_manifests=not opt.this_manifest_only
)
result["projects"] = [self._getProjectData(p) for p in projs]
project_data = []
def _ProcessResults(_pool, _output, results):
project_data.extend(results)
with self.ParallelContext():
self.get_parallel_context()["projects"] = projs
self.ExecuteInParallel(
opt.jobs,
self._ProjectDataHelper,
range(len(projs)),
callback=_ProcessResults,
ordered=True,
chunksize=1,
)
result["projects"] = project_data
json_settings = {
# JSON style guide says Unicode characters are fully allowed.
@@ -273,7 +294,7 @@ class Info(PagedCommand):
out.nl()
heading("Current revision: ")
headtext(project.GetRevisionId())
headtext(project.GetHeadRevisionId() or project.GetRevisionId())
out.nl()
currentBranch = project.CurrentBranch
+2
View File
@@ -33,6 +33,7 @@ _REPO_ALLOW_SHALLOW = os.environ.get("REPO_ALLOW_SHALLOW")
class Init(InteractiveCommand, MirrorSafeCommand):
COMMON = True
RESPECT_SMART_SYNC_OVERRIDE = False
MULTI_MANIFEST_SUPPORT = True
helpSummary = "Initialize a repo client checkout in the current directory"
helpUsage = """
@@ -169,6 +170,7 @@ to update the working directory files.
depth=opt.depth,
git_event_log=self.git_event_log,
manifest_name=opt.manifest_name,
use_local_gitdirs=opt.use_local_gitdirs,
):
manifest_name = opt.manifest_name
raise UpdateManifestError(
+9 -4
View File
@@ -59,10 +59,15 @@ are displayed.
for project in self.GetProjects(
args, all_manifests=not opt.this_manifest_only
):
br = [project.GetUploadableBranch(x) for x in project.GetBranches()]
br = [x for x in br if x]
local_branches = project.GetBranches()
br = []
for name, branch in local_branches.items():
uploadable = project.GetUploadableBranch(name)
if uploadable:
uploadable.branch.current = branch.current
br.append(uploadable)
if opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch]
br = [x for x in br if x.current]
all_branches.extend(br)
if not all_branches:
@@ -97,7 +102,7 @@ are displayed.
print(
"%s %-33s (%2d commit%s, %s)"
% (
branch.name == project.CurrentBranch and "*" or " ",
branch.current and "*" or " ",
branch.name,
len(commits),
len(commits) != 1 and "s" or " ",
+1 -1
View File
@@ -80,7 +80,7 @@ class Prune(PagedCommand):
print(
"%s %-33s "
% (
branch.name == project.CurrentBranch and "*" or " ",
branch.current and "*" or " ",
branch.name,
),
end="",
+16 -20
View File
@@ -16,7 +16,9 @@ import sys
from color import Coloring
from command import Command
from error import GitError
from git_command import GitCommand
from project import Project
from repo_logging import RepoLogger
@@ -30,6 +32,17 @@ class RebaseColoring(Coloring):
self.fail = self.printer("fail", fg="red")
def _ResolveOntoManifest(project: Project) -> str:
"""Resolve project's revisionExpr to a local tracking branch.
Falls back to the raw revisionExpr if ToLocal fails or raises GitError.
"""
try:
return project.GetRemote().ToLocal(project.revisionExpr)
except GitError:
return project.revisionExpr
class Rebase(Command):
COMMON = True
helpSummary = "Rebase local branches on upstream branch"
@@ -126,6 +139,8 @@ branch but need to incorporate new upstream changes "underneath" them.
common_args.append("--autosquash")
if opt.interactive:
common_args.append("-i")
if opt.auto_stash:
common_args.append("--autostash")
config = self.manifest.manifestProject.config
out = RebaseColoring(config)
@@ -162,7 +177,7 @@ branch but need to incorporate new upstream changes "underneath" them.
args = common_args[:]
if opt.onto_manifest:
args.append("--onto")
args.append(project.revisionExpr)
args.append(_ResolveOntoManifest(project))
args.append(upbranch.LocalMerge)
@@ -175,29 +190,10 @@ branch but need to incorporate new upstream changes "underneath" them.
out.nl()
out.flush()
needs_stash = False
if opt.auto_stash:
stash_args = ["update-index", "--refresh", "-q"]
if GitCommand(project, stash_args).Wait() != 0:
needs_stash = True
# Dirty index, requires stash...
stash_args = ["stash"]
if GitCommand(project, stash_args).Wait() != 0:
ret += 1
continue
if GitCommand(project, args).Wait() != 0:
ret += 1
continue
if needs_stash:
stash_args.append("pop")
stash_args.append("--quiet")
if GitCommand(project, stash_args).Wait() != 0:
ret += 1
if ret:
msg_fmt = "%d projects had errors"
self.git_event_log.ErrorEvent(msg_fmt % (ret), msg_fmt)
+2 -3
View File
@@ -18,7 +18,7 @@ from typing import NamedTuple
from command import Command
from command import DEFAULT_LOCAL_JOBS
from error import RepoExitError
from git_command import git
from git_command import IsValidBranchName
from git_config import IsImmutable
from progress import Progress
from repo_logging import RepoLogger
@@ -75,7 +75,7 @@ revision specified in the manifest.
self.Usage()
nb = args[0]
if not git.check_ref_format("heads/%s" % nb):
if not IsValidBranchName(nb):
self.OptionParser.error("'%s' is not a valid name" % nb)
@classmethod
@@ -104,7 +104,6 @@ revision specified in the manifest.
def Execute(self, opt, args):
nb = args[0]
self.TryOverrideManifestWithSmartSync()
err_projects = []
err = []
projects = []
+6
View File
@@ -54,6 +54,12 @@ project 'repo' on branch 'devwork':
project repo/ branch devwork
-m subcmds/status.py
If the branch is tracking an upstream branch, the number of commits
ahead and/or behind is also shown:
project repo/ branch devwork [ahead 1, behind 2]
-m subcmds/status.py
The first column explains how the staging area (index) differs from
the last commit (HEAD). Its values are always displayed in upper
case and have the following meanings:
+365 -61
View File
@@ -23,10 +23,12 @@ import netrc
import optparse
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import time
from typing import List, NamedTuple, Optional, Set, Tuple, Union
from typing import Dict, List, NamedTuple, Optional, Set, Tuple, Union
import urllib.error
import urllib.parse
import urllib.request
@@ -96,19 +98,27 @@ logger = RepoLogger(__file__)
def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
"""Generate a sequence of checkouts that is safe to perform. The client
should checkout everything from n-th index before moving to n+1.
"""Generate a sequence of checkouts that is safe to perform.
The client should checkout everything from n-th index before moving to
n+1.
This is only useful if manifest contains nested projects.
E.g. if foo, foo/bar and foo/bar/baz are project paths, then foo needs to
finish before foo/bar can proceed, and foo/bar needs to finish before
foo/bar/baz."""
res = [[]]
current = res[0]
foo/bar/baz.
# depth_stack contains a current stack of parent paths.
Discovered submodules have an additional constraint: sibling submodules in
the same parent repository must not be checked out in parallel because they
all run `git submodule init` against the same parent .git/config.
"""
res = [[]]
# depth_stack contains the current stack of parent paths together with the
# effective checkout level assigned to each path.
depth_stack = []
submodule_parent_level = {}
# Checkouts are iterated in the hierarchical order. That way, it can easily
# be determined if the previous checkout is parent of the current checkout.
# We are splitting by the path separator so the final result is
@@ -119,25 +129,110 @@ def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
checkout_path = Path(checkout.relpath)
while depth_stack:
try:
checkout_path.relative_to(depth_stack[-1])
checkout_path.relative_to(depth_stack[-1][0])
except ValueError:
# Path.relative_to returns ValueError if paths are not relative.
# TODO(sokcevic): Switch to is_relative_to once min supported
# version is py3.9.
depth_stack.pop()
else:
if len(depth_stack) >= len(res):
# Another depth created.
res.append([])
break
current = res[len(depth_stack)]
level = depth_stack[-1][1] + 1 if depth_stack else 0
parent = checkout.parent
if parent is not None:
level = max(
level,
submodule_parent_level.get(parent.worktree, level - 1) + 1,
)
submodule_parent_level[parent.worktree] = level
if level >= len(res):
res.extend([] for _ in range(level + 1 - len(res)))
current = res[level]
current.append(checkout)
depth_stack.append(checkout_path)
depth_stack.append((checkout_path, level))
return res
def _ParentFirstBatches(projects: List[Project]) -> List[List[Project]]:
"""Group |projects| so that a parent is fetched before its submodules.
A discovered submodule can only be fetched at the right revision once the
project holding its gitlink has been fetched, so it is held back to a later
batch than its parent. Projects that are not discovered submodules all end
up in the first batch, which keeps manifests without submodules on a single
batch.
"""
batches = collections.defaultdict(list)
for project in projects:
depth = 0
ancestor = project
while ancestor.Derived and ancestor.parent:
depth += 1
ancestor = ancestor.parent
batches[depth].append(project)
return [batches[depth] for depth in sorted(batches)]
def _RefreshDerivedRevisions(
projects: List[Project],
submodule_revisions: Optional[Dict[Project, Dict[str, str]]] = None,
) -> List[Project]:
"""Re-resolve the gitlinks of the discovered submodules in |projects|.
The revision of a discovered submodule is read from its parent when the
manifest is loaded, so it is stale as soon as the parent gets fetched. It
has to be resolved again once the parent is up-to-date, and before the
submodule itself is fetched and checked out.
Args:
projects: The projects whose discovered submodules to resolve.
submodule_revisions: Gitlinks already read, keyed by the project
holding them. Passing the same dict for project sets that follow
the same fetches, e.g. the levels of one checkout order, keeps a
project from being read more than once.
Returns:
The submodules that their parent no longer holds a gitlink for.
"""
if submodule_revisions is None:
submodule_revisions = {}
subprojects_by_parent = collections.defaultdict(list)
for project in projects:
if project.Derived and project.parent:
subprojects_by_parent[project.parent].append(project)
removed = []
for parent, subprojects in subprojects_by_parent.items():
revisions = submodule_revisions.get(parent)
if revisions is None:
revisions = parent.GetSubmoduleRevisions()
if revisions is None:
# Leave the submodules of a parent we cannot read alone.
continue
submodule_revisions[parent] = revisions
for subproject in subprojects:
rev = revisions.get(subproject.gitlink_path)
if rev:
subproject.SetRevision(rev, revisionId=rev)
else:
removed.append(subproject)
return removed
def _WithoutProjects(
projects: List[Project], unwanted: List[Project]
) -> List[Project]:
"""Return |projects| without the projects in |unwanted|."""
if not unwanted:
return projects
dropped = set(unwanted)
return [p for p in projects if p not in dropped]
def _chunksize(projects: int, jobs: int) -> int:
"""Calculate chunk size for the given number of projects and jobs."""
return min(max(1, projects // jobs), WORKER_BATCH_SIZE)
@@ -206,7 +301,8 @@ class _SyncResult(NamedTuple):
Attributes:
project_index (int): The index of the project in the shared list.
relpath (str): The project's relative path from the repo client top.
relpath (str): The project's path relative to the tree being synced.
Unlike Project.relpath, it is unique across submanifests.
remote_fetched (bool): True if the remote was actually queried.
fetch_success (bool): True if the fetch operation was successful.
fetch_errors (List[Exception]): The Exceptions from a failed fetch.
@@ -294,6 +390,7 @@ class TeeStringIO(io.StringIO):
class Sync(Command, MirrorSafeCommand):
COMMON = True
RESPECT_SMART_SYNC_OVERRIDE = False
MULTI_MANIFEST_SUPPORT = True
helpSummary = "Update working tree to the latest revision"
helpUsage = """
@@ -360,8 +457,12 @@ resumeable bundle file on a content delivery network. This
may be necessary if there are problems with the local Python
HTTP client or proxy configuration, but the Git binary works.
The --fetch-submodules option enables fetching Git submodules
of a project from server.
The --recurse-submodules option enables syncing Git submodules of all projects
from the server. The --no-recurse-submodules option disables syncing Git
submodules, even when a project has sync-s="true" in the manifest.
The --fetch-submodules and --no-fetch-submodules options are deprecated aliases
for --recurse-submodules and --no-recurse-submodules, respectively.
The -c/--current-branch option can be used to only fetch objects that
are on the branch specified by a project's revision.
@@ -409,6 +510,15 @@ later is required to fix a server side protocol bug.
_JOBS_WARN_THRESHOLD = 100
@staticmethod
def _deprecated_submodules_option(option, opt_str, _value, parser):
enabled = opt_str == "--fetch-submodules"
replacement = (
"--recurse-submodules" if enabled else "--no-recurse-submodules"
)
logger.warning("%s is deprecated; use %s instead", opt_str, replacement)
setattr(parser.values, option.dest, enabled)
def _Options(self, p, show_smart=True):
p.add_option(
"--jobs-network",
@@ -527,6 +637,13 @@ later is required to fix a server side protocol bug.
help="temporary manifest to use for this sync",
metavar="NAME.xml",
)
p.add_option(
"-g",
"--groups",
help="sync projects matching the specific groups. Not persistent "
"unlike when used on init",
metavar="GROUP",
)
p.add_option(
"--clone-bundle",
action="store_true",
@@ -551,9 +668,29 @@ later is required to fix a server side protocol bug.
help="password to authenticate with the manifest server",
)
p.add_option(
"--fetch-submodules",
"--recurse-submodules",
action="store_true",
help="fetch submodules from server",
help="sync submodules from server",
)
p.add_option(
"--no-recurse-submodules",
dest="recurse_submodules",
action="store_false",
help="don't sync submodules from server",
)
p.add_option(
"--fetch-submodules",
dest="recurse_submodules",
action="callback",
callback=self._deprecated_submodules_option,
help=optparse.SUPPRESS_HELP,
)
p.add_option(
"--no-fetch-submodules",
dest="recurse_submodules",
action="callback",
callback=self._deprecated_submodules_option,
help=optparse.SUPPRESS_HELP,
)
p.add_option(
"--use-superproject",
@@ -723,8 +860,9 @@ later is required to fix a server side protocol bug.
all_projects = self.GetProjects(
args,
groups=opt.groups,
missing_ok=True,
submodules_ok=opt.fetch_submodules,
submodules_ok=opt.recurse_submodules,
manifest=manifest,
all_manifests=not opt.this_manifest_only,
)
@@ -1007,6 +1145,40 @@ later is required to fix a server side protocol bug.
return _FetchResult(ret, fetched)
def _FetchParentFirst(
self,
projects: List[Project],
opt: optparse.Values,
err_event: _threading.Event,
ssh_proxy: ssh.ProxyManager,
errors: List[Exception],
) -> _FetchResult:
"""Fetch |projects|, holding submodules back until their parent is done.
Args:
projects: Projects to fetch.
opt: Program options returned from optparse. See _Options().
err_event: Whether an error was hit while processing.
ssh_proxy: SSH manager for clients & masters.
errors: A list to accumulate errors.
Returns:
_FetchResult for all the batches combined.
"""
success = True
fetched = set()
for batch in _ParentFirstBatches(projects):
batch = _WithoutProjects(batch, _RefreshDerivedRevisions(batch))
if not batch:
continue
batch.sort(key=self._fetch_times.Get, reverse=True)
result = self._Fetch(batch, opt, err_event, ssh_proxy, errors)
success = success and result.success
fetched.update(result.projects)
if not success and opt.fail_fast:
break
return _FetchResult(success, fetched)
def _FetchMain(
self, opt, args, all_projects, err_event, ssh_proxy, manifest, errors
):
@@ -1023,12 +1195,10 @@ later is required to fix a server side protocol bug.
Returns:
List of all projects that should be checked out.
"""
to_fetch = []
to_fetch.extend(all_projects)
to_fetch.sort(key=self._fetch_times.Get, reverse=True)
try:
result = self._Fetch(to_fetch, opt, err_event, ssh_proxy, errors)
result = self._FetchParentFirst(
all_projects, opt, err_event, ssh_proxy, errors
)
success = result.success
fetched = result.projects
if not success:
@@ -1052,8 +1222,9 @@ later is required to fix a server side protocol bug.
self._ReloadManifest(None, manifest)
all_projects = self.GetProjects(
args,
groups=opt.groups,
missing_ok=True,
submodules_ok=opt.fetch_submodules,
submodules_ok=opt.recurse_submodules,
manifest=manifest,
all_manifests=not opt.this_manifest_only,
)
@@ -1070,7 +1241,9 @@ later is required to fix a server side protocol bug.
if previously_missing_set == missing_set:
break
previously_missing_set = missing_set
result = self._Fetch(missing, opt, err_event, ssh_proxy, errors)
result = self._FetchParentFirst(
missing, opt, err_event, ssh_proxy, errors
)
success = result.success
new_fetched = result.projects
if not success:
@@ -1722,16 +1895,104 @@ later is required to fix a server side protocol bug.
return True
def _SmartSyncSetup(self, opt, smart_sync_manifest_path, manifest):
if not manifest.manifest_server:
raise SmartSyncError(
"error: cannot smart sync: no manifest server defined in "
"manifest"
)
def _ResolveManifestServerTransport(self, opt, manifest):
"""Resolves the manifest server URL and transport.
Returns:
Tuple[str, xmlrpc.client.Transport]: The resolved server URL and
transport.
Raises:
SmartSyncError: If resolution fails (e.g. helper missing, helper
error, unsupported scheme).
"""
manifest_server = manifest.manifest_server
if not opt.quiet:
print("Using manifest server %s" % manifest_server)
helper_binary = manifest.manifest_server_helper
if helper_binary:
if not shutil.which(helper_binary):
raise SmartSyncError(
f"error: helper binary '{helper_binary}' declared in "
"manifest was not found in your PATH."
)
if not opt.quiet:
print(f"Using remote helper {helper_binary}")
p = None
try:
p = subprocess.Popen(
[helper_binary, manifest_server],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
stdout, stderr = p.communicate(timeout=10)
output = stdout.strip()
stderr_content = stderr.strip() if stderr else ""
except subprocess.TimeoutExpired as e:
err_msg = f"helper {helper_binary} timed out after 10 seconds"
timeout_stderr = e.stderr.strip() if e.stderr else ""
if timeout_stderr:
err_msg += f". Stderr: {timeout_stderr}"
raise SmartSyncError(err_msg)
except OSError as e:
raise SmartSyncError(
"failed to start or communicate with helper "
f"{helper_binary}: {e}"
)
finally:
if p and p.poll() is None:
p.kill()
p.wait()
if p.returncode != 0:
err_msg = (
f"helper {helper_binary} exited with exit code "
f"{p.returncode}."
)
if stderr_content:
err_msg += f" Stderr: {stderr_content}"
raise SmartSyncError(err_msg)
try:
res = json.loads(output)
status = res.get("status")
msg = res.get("message")
except json.JSONDecodeError as e:
err_msg = (
f"failed to parse JSON from helper {helper_binary}: {e}. "
f"Output was: {output}"
)
if stderr_content:
err_msg += f"\nStderr was: {stderr_content}"
raise SmartSyncError(err_msg)
if status != "ok":
err_msg = f"helper {helper_binary} returned error: {msg}"
if stderr_content:
err_msg += f"\nStderr was: {stderr_content}"
raise SmartSyncError(err_msg)
proxy_url = msg
transport = PersistentTransport(manifest_server, proxy=proxy_url)
server_url = manifest_server
if server_url.startswith("persistent-"):
server_url = server_url[len("persistent-") :]
return server_url, transport
# Fallback path if helper isn't specified
scheme = urllib.parse.urlparse(manifest_server).scheme
if scheme not in (
"http",
"https",
"persistent-http",
"persistent-https",
):
raise SmartSyncError(
f"error: unsupported manifest server scheme '{scheme}'."
)
if "@" not in manifest_server:
username = None
@@ -1766,20 +2027,34 @@ later is required to fix a server side protocol bug.
)
transport = PersistentTransport(manifest_server)
if manifest_server.startswith("persistent-"):
manifest_server = manifest_server[len("persistent-") :]
server_url = manifest_server
if server_url.startswith("persistent-"):
server_url = server_url[len("persistent-") :]
return server_url, transport
def _SmartSyncSetup(self, opt, smart_sync_manifest_path, manifest):
if not manifest.manifest_server:
raise SmartSyncError(
"error: cannot smart sync: no manifest server defined in "
"manifest"
)
if not opt.quiet:
print("Using manifest server %s" % manifest.manifest_server)
server_url, transport = self._ResolveManifestServerTransport(
opt, manifest
)
# Changes in behavior should update docs/smart-sync.md accordingly.
try:
server = xmlrpc.client.Server(manifest_server, transport=transport)
server = xmlrpc.client.Server(server_url, transport=transport)
if opt.smart_sync:
branch = self._GetBranch(manifest.manifestProject)
target = None
if "SYNC_TARGET" in os.environ:
target = os.environ["SYNC_TARGET"]
[success, manifest_str] = server.GetApprovedManifest(
branch, target
)
elif (
"TARGET_PRODUCT" in os.environ
and "TARGET_BUILD_VARIANT" in os.environ
@@ -1790,9 +2065,6 @@ later is required to fix a server side protocol bug.
os.environ["TARGET_RELEASE"],
os.environ["TARGET_BUILD_VARIANT"],
)
[success, manifest_str] = server.GetApprovedManifest(
branch, target
)
elif (
"TARGET_PRODUCT" in os.environ
and "TARGET_BUILD_VARIANT" in os.environ
@@ -1801,6 +2073,8 @@ later is required to fix a server side protocol bug.
os.environ["TARGET_PRODUCT"],
os.environ["TARGET_BUILD_VARIANT"],
)
if target:
[success, manifest_str] = server.GetApprovedManifest(
branch, target
)
@@ -1822,25 +2096,34 @@ later is required to fix a server side protocol bug.
aggregate_errors=[e],
)
self._ReloadManifest(manifest_name, manifest)
else:
raise SmartSyncError(
"error: manifest server RPC call failed: %s" % manifest_str
)
return manifest_name
raise SmartSyncError(
"error: manifest server RPC call failed: %s" % manifest_str
)
except (OSError, xmlrpc.client.Fault) as e:
if manifest.manifest_server_helper:
raise SmartSyncError(
"error: failed to communicate with manifest server via "
f"helper: {e}"
)
raise SmartSyncError(
"error: cannot connect to manifest server %s:\n%s"
% (manifest.manifest_server, e),
aggregate_errors=[e],
)
except xmlrpc.client.ProtocolError as e:
if manifest.manifest_server_helper:
raise SmartSyncError(
"error: failed to communicate with manifest server via "
f"helper: {e}"
)
raise SmartSyncError(
"error: cannot connect to manifest server %s:\n%d %s"
% (manifest.manifest_server, e.errcode, e.errmsg),
aggregate_errors=[e],
)
return manifest_name
def _UpdateAllManifestProjects(self, opt, mp, manifest_name, errors):
"""Fetch & update the local manifest project.
@@ -2189,8 +2472,9 @@ later is required to fix a server side protocol bug.
all_projects = self.GetProjects(
args,
groups=opt.groups,
missing_ok=True,
submodules_ok=opt.fetch_submodules,
submodules_ok=opt.recurse_submodules,
manifest=manifest,
all_manifests=not opt.this_manifest_only,
)
@@ -2586,7 +2870,7 @@ later is required to fix a server side protocol bug.
return _SyncResult(
project_index=project_index,
relpath=project.relpath,
relpath=project.RelPath(local=opt.this_manifest_only),
fetch_success=fetch_success,
remote_fetched=remote_fetched,
checkout_success=checkout_success,
@@ -2734,6 +3018,10 @@ later is required to fix a server side protocol bug.
self._interleaved_err_checkout = False
self._interleaved_err_checkout_results = []
# Project.relpath is relative to its own (sub)manifest, so it does not
# tell apart projects of different manifests being synced together.
_RelPath = lambda p: p.RelPath(local=opt.this_manifest_only)
err_event = multiprocessing.Event()
finished_relpaths = set()
project_list = list(all_projects)
@@ -2770,13 +3058,13 @@ later is required to fix a server side protocol bug.
projects_to_sync = [
p
for p in project_list
if p.relpath not in finished_relpaths
if _RelPath(p) not in finished_relpaths
]
if not projects_to_sync:
break
pending_relpaths = {
p.relpath for p in projects_to_sync
_RelPath(p) for p in projects_to_sync
}
if previously_pending_relpaths == pending_relpaths:
stalled_projects_str = "\n".join(
@@ -2805,12 +3093,22 @@ later is required to fix a server side protocol bug.
# projects in one level can be processed in
# parallel, but we must wait for a level to complete
# before starting the next.
submodule_revisions = {}
for level_projects in _SafeCheckoutOrder(
projects_to_sync
):
if not level_projects:
continue
level_projects = _WithoutProjects(
level_projects,
_RefreshDerivedRevisions(
level_projects, submodule_revisions
),
)
if not level_projects:
continue
objdir_project_map = collections.defaultdict(
list
)
@@ -2852,8 +3150,9 @@ later is required to fix a server side protocol bug.
self._ReloadManifest(None, manifest)
project_list = self.GetProjects(
args,
groups=opt.groups,
missing_ok=True,
submodules_ok=opt.fetch_submodules,
submodules_ok=opt.recurse_submodules,
manifest=manifest,
all_manifests=not opt.this_manifest_only,
)
@@ -3078,14 +3377,15 @@ class LocalSyncState:
# request to request like the normal transport, the real url
# is passed during initialization.
class PersistentTransport(xmlrpc.client.Transport):
def __init__(self, orig_host):
def __init__(self, orig_host, proxy=None):
super().__init__()
self.orig_host = orig_host
self.proxy = proxy
def request(self, host, handler, request_body, verbose=False):
with GetUrlCookieFile(self.orig_host, not verbose) as (
cookiefile,
proxy,
cookie_proxy,
):
# Python doesn't understand cookies with the #HttpOnly_ prefix
# Since we're only using them for HTTP, copy the file temporarily,
@@ -3111,10 +3411,11 @@ class PersistentTransport(xmlrpc.client.Transport):
else:
cookiejar = cookielib.CookieJar()
active_proxy = self.proxy or cookie_proxy
proxyhandler = urllib.request.ProxyHandler
if proxy:
if active_proxy:
proxyhandler = urllib.request.ProxyHandler(
{"http": proxy, "https": proxy}
{"http": active_proxy, "https": active_proxy}
)
opener = urllib.request.build_opener(
@@ -3127,13 +3428,16 @@ class PersistentTransport(xmlrpc.client.Transport):
scheme = parse_results.scheme
if scheme == "persistent-http":
scheme = "http"
if scheme == "persistent-https":
elif scheme == "persistent-https":
# If we're proxying through persistent-https, use http. The
# proxy itself will do the https.
if proxy:
if active_proxy:
scheme = "http"
else:
scheme = "https"
elif scheme not in ("http", "https"):
if active_proxy:
scheme = "http"
# Parse out any authentication information using the base class.
host, extra_headers, _ = self.get_host_info(parse_results.netloc)
+17 -22
View File
@@ -25,7 +25,6 @@ from editor import Editor
from error import GitError
from error import SilentRepoExitError
from error import UploadError
from git_command import GitCommand
from git_refs import R_HEADS
import git_superproject
from hooks import RepoHook
@@ -379,7 +378,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
default=True,
help="disable verifying ssl certs (unsafe)",
)
RepoHook.AddOptionGroup(p, "pre-upload")
RepoHook.AddOptionGroup(p, "pre-upload", allow_fix=True)
def _SingleBranch(self, opt, branch, people):
project = branch.project
@@ -649,6 +648,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
validate_certs=opt.validate_certs,
push_options=push_options,
patchset_description=opt.patchset_description,
git_event_log=self.git_event_log,
)
branch.uploaded = True
@@ -704,36 +704,31 @@ Gerrit Code Review: https://www.gerritcodereview.com/
raise UploadExitError(aggregate_errors=aggregate_errors)
def _GetMergeBranch(self, project, local_branch=None):
"""Get the merge branch name for a local branch.
Resolves the merge branch in-memory via project configuration to
avoid git subprocess overhead during upload.
"""
if local_branch is None:
p = GitCommand(
project,
["rev-parse", "--abbrev-ref", "HEAD"],
capture_stdout=True,
capture_stderr=True,
)
p.Wait()
local_branch = p.stdout.strip()
p = GitCommand(
project,
["config", "--get", "branch.%s.merge" % local_branch],
capture_stdout=True,
capture_stderr=True,
)
p.Wait()
merge_branch = p.stdout.strip()
return merge_branch
local_branch = project.CurrentBranch
if local_branch:
branch = project.GetBranch(local_branch)
if branch.merge:
return branch.merge
return ""
@classmethod
def _GatherOne(cls, opt, project_idx):
"""Figure out the upload status for |project|."""
project = cls.get_parallel_context()["projects"][project_idx]
cbr = None
if opt.current_branch:
cbr = project.CurrentBranch
up_branch = project.GetUploadableBranch(cbr)
avail = [up_branch] if up_branch else None
else:
avail = project.GetUploadableBranches(opt.branch)
return (project_idx, avail)
return (project_idx, avail, cbr)
def Execute(self, opt, args):
projects = self.GetProjects(
@@ -743,7 +738,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
def _ProcessResults(_pool, _out, results):
pending = []
for result in results:
project_idx, avail = result
project_idx, avail, current_branch = result
project = projects[project_idx]
if avail is None:
logger.error(
@@ -751,7 +746,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
"You might be able to fix the branch by running:\n"
" git branch --set-upstream-to m/%s",
project.RelPath(local=opt.this_manifest_only),
project.CurrentBranch,
current_branch,
project.manifest.branch,
)
elif avail:
+20 -2
View File
@@ -14,10 +14,12 @@
import platform
import sys
from typing import Any, Tuple
from command import Command
from command import MirrorSafeCommand
from git_command import git
from git_command import git_require
from git_command import RepoSourceVersion
from git_command import user_agent
from git_refs import HEAD
@@ -34,6 +36,22 @@ class Version(Command, MirrorSafeCommand):
%prog
"""
@staticmethod
def _RepoVersion(project: Any) -> Tuple[str, str]:
"""Return repo's describe string and commit date."""
if git_require((2, 32, 0)):
output = project.bare_git.log(
"-1", "--format=%(describe)%n%cD", HEAD
)
description, commit_date = output.rstrip("\n").split("\n", 1)
if description:
return description, commit_date
return (
project.bare_git.describe(HEAD),
project.bare_git.log("-1", "--format=%cD", HEAD),
)
def Execute(self, opt, args):
rp = self.manifest.repoProject
rem = rp.GetRemote()
@@ -41,11 +59,11 @@ class Version(Command, MirrorSafeCommand):
# These might not be the same. Report them both.
src_ver = RepoSourceVersion()
rp_ver = rp.bare_git.describe(HEAD)
rp_ver, commit_date = self._RepoVersion(rp)
print(f"repo version {rp_ver}")
print(f" (from {rem.url})")
print(f" (tracking {branch.merge})")
print(f" ({rp.bare_git.log('-1', '--format=%cD', HEAD)})")
print(f" ({commit_date})")
if self.wrapper_path is not None:
print(f"repo launcher version {self.wrapper_version}")
+81 -1
View File
@@ -14,6 +14,8 @@
"""Unittests for the color.py module."""
from unittest import mock
import pytest
import utils_for_test
@@ -24,9 +26,14 @@ import git_config
@pytest.fixture
def coloring() -> color.Coloring:
"""Create a Coloring object for testing."""
return _make_coloring("always")
def _make_coloring(default_state: str) -> color.Coloring:
"""Set the default color mode and return a Coloring using test config."""
config_fixture = utils_for_test.FIXTURES_DIR / "test.gitconfig"
config = git_config.GitConfig(config_fixture)
color.SetDefaultColoring("true")
color.SetDefaultColoring(default_state)
return color.Coloring(config, "status")
@@ -72,3 +79,76 @@ def test_Color_Parse_empty_entry(coloring: color.Coloring) -> None:
assert val == "\033[2;34;47m"
val = coloring._parse("empty", "green", "white", "bold")
assert val == "\033[1;32;47m"
class TestSetDefaultColoring:
"""Tests for SetDefaultColoring."""
def test_none_leaves_default_unchanged(self) -> None:
color.DEFAULT = "auto"
color.SetDefaultColoring(None)
assert color.DEFAULT == "auto"
@pytest.mark.parametrize(
"value, expected",
(
# auto/true/yes all store their lowercase form.
("auto", "auto"),
("Auto", "auto"),
("true", "true"),
("True", "true"),
("yes", "yes"),
("Yes", "yes"),
# "always" stores as "always".
("always", "always"),
("Always", "always"),
# never/no/false store their lowercase form.
("never", "never"),
("no", "no"),
("false", "false"),
),
)
def test_maps_to_expected(self, value: str, expected: str) -> None:
color.SetDefaultColoring(value)
assert color.DEFAULT == expected
def test_unrecognised_leaves_default_unchanged(self) -> None:
color.DEFAULT = "auto"
color.SetDefaultColoring("garbage")
assert color.DEFAULT == "auto"
class TestColoringInit:
"""Tests for Coloring.__init__ color mode logic."""
@pytest.mark.parametrize(
"state, isatty, pager_active, expected",
(
# "always" enables color unconditionally.
("always", False, False, True),
# "never" disables color unconditionally.
("never", True, True, False),
# auto/true/yes enable color only on a TTY or active pager.
("auto", True, False, True),
("auto", False, False, False),
("auto", False, True, True),
("true", True, False, True),
("true", False, False, False),
("true", False, True, True),
("yes", True, False, True),
("yes", False, False, False),
("yes", False, True, True),
),
)
def test_color_mode(
self,
state: str,
isatty: bool,
pager_active: bool,
expected: bool,
) -> None:
with mock.patch("os.isatty", return_value=isatty), mock.patch(
"pager.active", pager_active
):
c = _make_coloring(state)
assert c.is_on is expected
+31
View File
@@ -14,6 +14,8 @@
"""Unittests for the command.py module."""
import pytest
from command import Command
@@ -86,3 +88,32 @@ def test_get_projects_keeps_derived_subprojects_for_repeated_repo():
projects = cmd.GetProjects([])
assert set(projects) == {project_a, project_b, submodule_a, submodule_b}
@pytest.mark.parametrize(
"submodules_ok, sync_s, includes_submodule",
[
(None, False, False),
(None, True, True),
(True, False, True),
(True, True, True),
(False, False, False),
(False, True, False),
],
)
def test_get_projects_submodule_override(
submodules_ok, sync_s, includes_submodule
):
"""The CLI override takes precedence over a project's sync-s setting."""
submodule = FakeProject("submodule", "project/submodule")
project = FakeProject(
"project",
"project",
derived_subprojects=[submodule],
sync_s=sync_s,
)
cmd = Command(manifest=FakeManifest([project]))
projects = cmd.GetProjects([], submodules_ok=submodules_ok)
assert (submodule in projects) is includes_submodule
+18
View File
@@ -231,6 +231,24 @@ class GitCommandStreamLogsTest(unittest.TestCase):
class GitCallUnitTest(unittest.TestCase):
"""Tests the _GitCall class (via git_command.git)."""
def test_valid_branch_name_uses_branch_mode(self) -> None:
"""Branch validation applies Git's branch-specific restrictions."""
command = mock.MagicMock()
command.Wait.return_value = 1
with mock.patch.object(
git_command, "GitCommand", return_value=command
) as check:
self.assertFalse(git_command.IsValidBranchName("-topic"))
check.assert_called_once_with(
None,
["check-ref-format", "--branch", "-topic"],
capture_stdout=True,
capture_stderr=True,
add_event_log=False,
log_as_error=False,
)
def test_version_tuple(self):
"""Check git.version_tuple() handling."""
ver = git_command.git.version_tuple()
+24
View File
@@ -244,3 +244,27 @@ def test_remote_save_with_push_url_without_projectname(
assert (
written_config.GetString("remote.origin.pushurl") == "ssh://example.com"
)
@pytest.mark.parametrize(
"rev, expected",
(
("a" * 40, True),
("0" * 40, True),
("f" * 40, True),
("a" * 64, True),
("0" * 64, True),
("f" * 64, True),
("a" * 39, False),
("a" * 41, False),
("a" * 63, False),
("a" * 65, False),
("g" * 40, False),
("g" * 64, False),
("refs/heads/master", False),
("refs/tags/v1.0", False),
),
)
def test_is_id(rev: str, expected: bool) -> None:
"""Test IsId identifies both SHA-1 and SHA-256 hashes."""
assert git_config.IsId(rev) == expected
+148 -2
View File
@@ -17,6 +17,8 @@
import os
from pathlib import Path
import subprocess
from typing import Any, List
from unittest import mock
import pytest
import utils_for_test
@@ -24,7 +26,7 @@ import utils_for_test
import git_refs
def _run(repo, *args):
def _run(repo: str, *args: str) -> str:
return subprocess.run(
["git", "-C", repo, *args],
stdout=subprocess.PIPE,
@@ -34,7 +36,7 @@ def _run(repo, *args):
).stdout.strip()
def _init_repo(tmp_path, reftable=False):
def _init_repo(tmp_path: Path, reftable: bool = False) -> str:
repo = os.path.join(tmp_path, "repo")
ref_format = "reftable" if reftable else "files"
utils_for_test.init_git_tree(repo, ref_format=ref_format)
@@ -57,10 +59,154 @@ def test_reads_refs(tmp_path, reftable):
branch = _run(repo, "symbolic-ref", "--short", "HEAD")
head = _run(repo, "rev-parse", "HEAD")
assert refs.symref("HEAD") == f"refs/heads/{branch}"
assert refs.head == f"refs/heads/{branch}"
assert refs.get("HEAD") == head
assert refs.get(f"refs/heads/{branch}") == head
@pytest.mark.parametrize("reftable", [False, True])
def test_reads_detached_head(tmp_path: Path, reftable: bool) -> None:
if reftable and not utils_for_test.supports_reftable():
pytest.skip("reftable not supported")
repo = _init_repo(tmp_path, reftable=reftable)
head = _run(repo, "rev-parse", "HEAD")
_run(repo, "checkout", "--detach", head)
refs = git_refs.GitRefs(os.path.join(repo, ".git"))
assert refs.symref("HEAD") == ""
assert refs.head == head
assert refs.get("HEAD") == head
def test_reads_head_with_root_refs_in_one_command(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Git 2.45 and newer include HEAD in the ref snapshot."""
head = "1" * 40
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **_kwargs: Any
) -> None:
commands.append(cmdv)
self.stdout = (
f"{head}\0HEAD\0refs/heads/main\n"
f"{head}\0refs/heads/main\0\n"
)
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
with mock.patch.object(refs, "_ReadSymbolicRef") as read_head:
assert refs.get("HEAD") == head
assert commands == [
[
"for-each-ref",
"--include-root-refs",
"--format=%(objectname)%00%(refname)%00%(symref)",
"HEAD",
"refs",
]
]
read_head.assert_not_called()
def test_root_ref_snapshot_falls_back_for_unborn_head(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""An unborn HEAD still uses symbolic-ref after the ref snapshot."""
class FakeGitCommand:
def __init__(
self, _project: Any, _cmdv: List[str], **_kwargs: Any
) -> None:
self.stdout = ""
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.symref("HEAD") == "refs/heads/main"
def test_old_git_keeps_separate_head_fallback(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Git before 2.45 uses the original for-each-ref and HEAD calls."""
head = "1" * 40
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **_kwargs: Any
) -> None:
commands.append(cmdv)
self.stdout = f"{head}\0refs/heads/main\0\n"
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: False)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.get("HEAD") == head
assert commands == [
[
"for-each-ref",
"--format=%(objectname)%00%(refname)%00%(symref)",
]
]
def test_for_each_ref_failure_falls_back_to_symbolic_ref(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""When for-each-ref fails, HEAD is still resolved via symbolic-ref."""
class FakeGitCommand:
def __init__(
self, _project: Any, _cmdv: List[str], **_kwargs: Any
) -> None:
self.stdout = ""
def Wait(self) -> int:
return 1
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.symref("HEAD") == "refs/heads/main"
@pytest.mark.parametrize("reftable", [False, True])
def test_updates_when_refs_change(tmp_path, reftable):
if reftable and not utils_for_test.supports_reftable():
+24 -4
View File
@@ -542,14 +542,15 @@ class SuperprojectTestCase(unittest.TestCase):
with mock.patch(
"git_superproject.GitCommand", autospec=True
) as mock_git_command:
with mock.patch(
"git_superproject.GitRefs.get", autospec=True
) as mock_git_refs:
with mock.patch.object(
self._superproject, "_GetRef"
) as get_ref:
instance = mock_git_command.return_value
instance.Wait.return_value = 0
mock_git_refs.side_effect = ["", "1234"]
get_ref.side_effect = ["", "1234"]
self.assertTrue(self._superproject._Fetch())
get_ref.assert_called_with("refs/heads/main")
self.assertEqual(
# TODO: Once we require Python 3.8+,
# use 'mock_git_command.call_args.args'.
@@ -572,6 +573,7 @@ class SuperprojectTestCase(unittest.TestCase):
# If branch for revision exists, set as --negotiation-tip.
self.assertTrue(self._superproject._Fetch())
get_ref.assert_called_with("refs/heads/main")
self.assertEqual(
# TODO: Once we require Python 3.8+,
# use 'mock_git_command.call_args.args'.
@@ -593,3 +595,21 @@ class SuperprojectTestCase(unittest.TestCase):
],
),
)
def test_GetRef_resolves_only_the_requested_ref(self) -> None:
command = mock.MagicMock(stdout="1234\n")
command.Wait.return_value = 0
with mock.patch(
"git_superproject.GitCommand", return_value=command
) as git_command:
self.assertEqual("1234", self._superproject._GetRef("HEAD"))
git_command.assert_called_once_with(
None,
["rev-parse", "--verify", "--quiet", "HEAD"],
gitdir=self._superproject._work_git,
bare=True,
capture_stdout=True,
capture_stderr=True,
log_as_error=False,
)
+2 -1
View File
@@ -315,6 +315,7 @@ def test_data_event_config(event_log: git_trace2_event_log.EventLog) -> None:
"repo.partialclone": "false",
"repo.syncstate.superproject.hassuperprojecttag": "true",
"repo.syncstate.superproject.sys.argv": ["--", "sync", "protobuf"],
"repo.syncstate.emptykey": "",
}
prefix_value = "prefix"
event_log.LogDataConfigEvents(config, prefix_value)
@@ -323,7 +324,7 @@ def test_data_event_config(event_log: git_trace2_event_log.EventLog) -> None:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 5
assert len(log_data) == 6
data_events = log_data[1:]
verify_common_keys(log_data[0], expected_event_name="version")
+96
View File
@@ -15,6 +15,7 @@
"""Unittests for the hooks.py module."""
from io import StringIO
from pathlib import Path
import sys
import pytest
@@ -105,3 +106,98 @@ def test_post_sync_argument_validation() -> None:
finally:
sys.stderr = old_stderr
@pytest.mark.parametrize("yes_val", (True, False))
def test_repo_upload_yes_arg(tmp_path: Path, yes_val: bool) -> None:
"""Test that yes is passed in kwargs during hook execution."""
class FakeProject:
def __init__(self, worktree: str) -> None:
self.worktree = worktree
self.enabled_repo_hooks = ["pre-upload"]
self.config = None
hook_file = tmp_path / "pre-upload.py"
hook_content = """
def main(project_list, **kwargs):
project_list.append(kwargs.get("yes"))
"""
hook_file.write_text(hook_content)
hook = hooks.RepoHook(
hook_type="pre-upload",
hooks_project=FakeProject(str(tmp_path)),
repo_topdir=str(tmp_path),
manifest_url="https://gerrit",
allow_all_hooks=True,
yes=yes_val,
)
project_list = []
res = hook.Run(project_list=project_list, worktree_list=[])
assert res is True
assert project_list == [yes_val]
@pytest.mark.parametrize("fix_val", (True, False))
def test_repo_upload_fix_arg(tmp_path: Path, fix_val: bool) -> None:
"""Test that fix is passed in kwargs during hook execution."""
class FakeProject:
def __init__(self, worktree: str) -> None:
self.worktree = worktree
self.enabled_repo_hooks = ["pre-upload"]
self.config = None
hook_file = tmp_path / "pre-upload.py"
hook_content = """
def main(project_list, **kwargs):
project_list.append(kwargs.get("fix"))
"""
hook_file.write_text(hook_content)
hook = hooks.RepoHook(
hook_type="pre-upload",
hooks_project=FakeProject(str(tmp_path)),
repo_topdir=str(tmp_path),
manifest_url="https://gerrit",
allow_all_hooks=True,
fix=fix_val,
)
project_list = []
res = hook.Run(project_list=project_list, worktree_list=[])
assert res is True
assert project_list == [fix_val]
def test_from_subcmd_without_fix_option() -> None:
"""Test that FromSubcmd works when opt does not have fix attribute."""
class Remote:
url = "https://gerrit"
class FakeManifest:
repo_hooks_project = None
topdir = "/fake/topdir"
class manifestProject:
@staticmethod
def GetRemote(name: str) -> "Remote":
return Remote()
class contactinfo:
bugurl = "https://bugs"
class FakeOpt:
bypass_hooks = False
allow_all_hooks = False
ignore_hooks = False
hook = hooks.RepoHook.FromSubcmd(FakeManifest(), FakeOpt(), "post-sync")
assert hook._fix is False
+20
View File
@@ -819,6 +819,26 @@ class TestProjectElement:
str(repo_client.topdir), ".repo", "projects", "..git"
)
def test_get_project_paths_local_gitdirs(
self, repo_client: RepoClient
) -> None:
"""Check GetProjectPaths with UseLocalGitDirs."""
manifest = repo_client.get_xml_manifest(
'<?xml version="1.0" encoding="UTF-8"?><manifest></manifest>'
)
manifest.manifestProject.config.SetBoolean("repo.uselocalgitdirs", True)
relpath, worktree, gitdir, objdir, use_git_worktrees = (
manifest.GetProjectPaths("foo", "bar", "origin")
)
assert os.path.normpath(gitdir) == os.path.normpath(
os.path.join(str(repo_client.topdir), "bar", ".git")
)
assert os.path.normpath(objdir) == os.path.normpath(
os.path.join(str(repo_client.topdir), "bar", ".git")
)
def test_bad_path_name_checks(self, repo_client: RepoClient) -> None:
"""Check handling of bad path & name attributes."""
+1940 -50
View File
File diff suppressed because it is too large Load Diff
+85
View File
@@ -0,0 +1,85 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for subcmds/cherry_pick.py."""
from unittest import mock
import pytest
from error import GitError
from git_command import GitCommand
from subcmds import cherry_pick
def test_resolve_reference_uses_one_typed_batch_request(
monkeypatch: pytest.MonkeyPatch,
) -> None:
oid = "1" * 40
commit = "tree " + "2" * 40 + "\n\nSubject 🚀\n\nBody\n"
output = oid + " commit " + str(len(commit.encode("utf-8"))) + "\n"
output += commit + "\n"
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = output
command.stderr = ""
command.Wait.return_value = 0
run_git = mock.create_autospec(GitCommand, return_value=command)
monkeypatch.setattr(cherry_pick, "GitCommand", run_git)
resolved, contents = cherry_pick.CherryPick()._ResolveReference("topic")
assert resolved == oid
assert contents == commit
run_git.assert_called_once_with(
None,
["cat-file", "--batch"],
input="topic^{commit}\n",
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
def test_resolve_reference_rejects_missing_object(
monkeypatch: pytest.MonkeyPatch,
) -> None:
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = "topic^{commit} missing\n"
command.stderr = ""
command.Wait.return_value = 0
monkeypatch.setattr(
cherry_pick,
"GitCommand",
mock.create_autospec(GitCommand, return_value=command),
)
with pytest.raises(GitError, match="commit topic not found"):
cherry_pick.CherryPick()._ResolveReference("topic")
def test_resolve_reference_rejects_ambiguous_object(
monkeypatch: pytest.MonkeyPatch,
) -> None:
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = "topic^{commit} ambiguous\n"
command.stderr = ""
command.Wait.return_value = 0
monkeypatch.setattr(
cherry_pick,
"GitCommand",
mock.create_autospec(GitCommand, return_value=command),
)
with pytest.raises(GitError, match="commit topic not found"):
cherry_pick.CherryPick()._ResolveReference("topic")
+9 -8
View File
@@ -17,12 +17,10 @@
import contextlib
import io
from pathlib import Path
from unittest import mock
import utils_for_test
import manifest_xml
import project
import subcmds
@@ -87,13 +85,16 @@ def test_forall_all_projects_called_once(tmp_path: Path) -> None:
opts, args = cmd.OptionParser.parse_args(["-c", "echo $REPO_PROJECT"])
opts.verbose = False
# Set revisionId directly so GetRevisionId() short-circuits without
# touching git. Using mock.patch.object on the class does not work
# with Python 3.14+, which defaults to "forkserver" on Linux —
# class-level patches do not survive into forkserver worker processes.
for proj in manifest.projects:
proj.revisionId = "refs/heads/main"
with contextlib.redirect_stdout(io.StringIO()) as stdout:
# Mock to not have the Execute fail on remote check.
with mock.patch.object(
project.Project, "GetRevisionId", return_value="refs/heads/main"
):
# Run the forall command.
cmd.Execute(opts, args)
# Run the forall command.
cmd.Execute(opts, args)
output = stdout.getvalue()
# Verify that we got every project name in the output.
+1 -1
View File
@@ -24,7 +24,7 @@ class GcCommand(unittest.TestCase):
"""Tests for gc command."""
def setUp(self):
self.cmd = gc.Gc()
self.cmd = gc.Gc(manifest=mock.MagicMock())
self.opt, self.args = self.cmd.OptionParser.parse_args([])
self.opt.this_manifest_only = False
self.opt.repack = False
+54
View File
@@ -199,3 +199,57 @@ def test_text_enables_pager() -> None:
cmd = _get_cmd()
opts, _ = cmd.OptionParser.parse_args([])
assert cmd.WantPager(opts)
def test_get_project_data_uses_head_revision() -> None:
"""_getProjectData should use GetHeadRevisionId if available."""
cmd = _get_cmd()
project = mock.MagicMock()
project.name = "foo"
project.worktree = "/path/to/foo"
project.revisionExpr = "refs/heads/main"
project.GetBranches.return_value = []
# GetHeadRevisionId() returns a SHA, it should be used.
project.GetHeadRevisionId.return_value = "head_sha_12345"
project.GetRevisionId.return_value = "manifest_sha_54321"
data = cmd._getProjectData(project)
assert data["current_revision"] == "head_sha_12345"
project.GetHeadRevisionId.assert_called_once()
# GetHeadRevisionId() is None, fall back to GetRevisionId().
project.GetHeadRevisionId.reset_mock()
project.GetHeadRevisionId.return_value = None
data = cmd._getProjectData(project)
assert data["current_revision"] == "manifest_sha_54321"
def test_json_with_projects(capsys) -> None:
"""--format=json should emit project data."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--format=json"])
opts.jobs = 1 # To avoid multiprocessing pickle issues with mocks
project = mock.MagicMock()
project.name = "foo"
project.worktree = "/path/to/foo"
project.revisionExpr = "refs/heads/main"
project.GetBranches.return_value = {"branch1": mock.MagicMock()}
project.GetHeadRevisionId.return_value = "head_sha_12345"
project.CurrentBranch = "branch1"
cmd.GetProjects = mock.MagicMock(return_value=[project])
cmd.Execute(opts, args)
data = json.loads(capsys.readouterr().out)
assert "projects" in data
assert len(data["projects"]) == 1
project_data = data["projects"][0]
assert project_data["name"] == "foo"
assert project_data["mount_path"] == "/path/to/foo"
assert project_data["current_revision"] == "head_sha_12345"
assert project_data["manifest_revision"] == "refs/heads/main"
assert project_data["local_branches"] == ["branch1"]
assert project_data["current_branch"] == "branch1"
+91
View File
@@ -0,0 +1,91 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the subcmds/rebase.py module."""
import contextlib
import io
from types import SimpleNamespace
from unittest import mock
from error import GitError
from subcmds import rebase
def test_resolve_onto_manifest_success() -> None:
"""Test _ResolveOntoManifest when ToLocal succeeds."""
project = mock.MagicMock()
project.revisionExpr = "main"
remote = mock.MagicMock()
remote.ToLocal.return_value = "refs/remotes/goog/main"
project.GetRemote.return_value = remote
res = rebase._ResolveOntoManifest(project)
assert res == "refs/remotes/goog/main"
project.GetRemote.assert_called_once()
remote.ToLocal.assert_called_once_with("main")
def test_resolve_onto_manifest_fallback() -> None:
"""Test _ResolveOntoManifest when ToLocal raises GitError."""
project = mock.MagicMock()
project.revisionExpr = "main"
remote = mock.MagicMock()
remote.ToLocal.side_effect = GitError("Failed to resolve")
project.GetRemote.return_value = remote
res = rebase._ResolveOntoManifest(project)
assert res == "main"
project.GetRemote.assert_called_once()
remote.ToLocal.assert_called_once_with("main")
def test_execute_delegates_autostash_to_rebase() -> None:
"""--auto-stash is one rebase process, including staged-only changes."""
cmd = rebase.Rebase()
cmd.manifest = mock.MagicMock()
cmd.git_event_log = mock.MagicMock()
project = mock.MagicMock()
project.CurrentBranch = "topic"
project.RelPath.return_value = "project"
branch = mock.MagicMock()
branch.LocalMerge = "refs/remotes/origin/main"
project.GetBranch.return_value = branch
cmd.GetProjects = mock.MagicMock(return_value=[project])
opt = SimpleNamespace(
interactive=False,
fail_fast=False,
whitespace=None,
quiet=False,
force_rebase=False,
ff=True,
autosquash=False,
auto_stash=True,
onto_manifest=False,
this_manifest_only=False,
)
git_command = mock.MagicMock()
git_command.Wait.return_value = 0
with mock.patch.object(
rebase, "GitCommand", return_value=git_command
) as run_git, contextlib.redirect_stdout(io.StringIO()):
assert cmd.Execute(opt, []) == 0
run_git.assert_called_once_with(
project,
["rebase", "--autostash", "refs/remotes/origin/main"],
)
+233
View File
@@ -107,6 +107,18 @@ def _assert_project_header(line: str, project_path: str, branch: str) -> None:
assert line == expected
def _assert_project_header_with_ahead_behind(
line: str,
project_path: str,
branch: str,
ahead_behind: str,
) -> None:
"""Assert a status project header line includes ahead/behind info."""
suffix = f"branch {branch}{ahead_behind}"
expected = f"project {(project_path + '/ '):<40}{suffix}"
assert line == expected
def _assert_orphan_block(lines: List[str], expected: List[str]) -> None:
"""Assert orphan block header and entries, independent of entry ordering."""
assert lines
@@ -218,3 +230,224 @@ def test_empty_status_after_start_shows_started_branch(
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, started_branch)
def _setup_remote_tracking_branch(
manifest: manifest_xml.XmlManifest,
branch_name: str,
) -> None:
"""Create a branch tracking a remote ref, like ``repo start``.
In a real repo checkout, ``repo start`` creates a branch that
tracks a remote tracking ref (e.g. refs/remotes/origin/main).
This sets up the same config in both the worktree and the
project gitdir (where repo reads its config from).
"""
proj = list(manifest.paths.values())[0]
worktree = Path(proj.worktree)
proj_gitdir = Path(proj.gitdir)
# Create the remote tracking ref in the worktree.
subprocess.check_call(
["git", "update-ref", "refs/remotes/origin/main", "main"],
cwd=worktree,
)
# Create the new branch from main in the worktree.
subprocess.check_call(
["git", "checkout", "-q", "-b", branch_name, "main"],
cwd=worktree,
)
# Write remote and branch config into the *project gitdir* config,
# which is where repo's Project.config reads from.
cfg = str(proj_gitdir / "config")
subprocess.check_call(
[
"git",
"config",
"-f",
cfg,
"remote.origin.url",
"http://localhost/fake",
],
)
subprocess.check_call(
[
"git",
"config",
"-f",
cfg,
"remote.origin.fetch",
"+refs/heads/*:refs/remotes/origin/*",
],
)
subprocess.check_call(
["git", "config", "-f", cfg, f"branch.{branch_name}.remote", "origin"],
)
subprocess.check_call(
[
"git",
"config",
"-f",
cfg,
f"branch.{branch_name}.merge",
"refs/heads/main",
],
)
def test_status_branch_ahead_of_upstream(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify status shows [ahead N] for local commits."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
_setup_remote_tracking_branch(manifest, "feature")
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "c1"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "c2"],
cwd=project_worktree,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header_with_ahead_behind(
lines[0], project_path, "feature", " [ahead 2]"
)
def test_status_branch_behind_upstream(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify status shows [behind N] when upstream is ahead."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
_setup_remote_tracking_branch(manifest, "feature")
# Advance the remote tracking ref past the feature branch.
subprocess.check_call(
["git", "checkout", "-q", "main"], cwd=project_worktree
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "upstream"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "update-ref", "refs/remotes/origin/main", "main"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "checkout", "-q", "feature"],
cwd=project_worktree,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header_with_ahead_behind(
lines[0], project_path, "feature", " [behind 1]"
)
def test_status_branch_ahead_and_behind(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify [ahead N, behind M] when branch has diverged."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
_setup_remote_tracking_branch(manifest, "feature")
# Add a local commit on feature.
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "local"],
cwd=project_worktree,
)
# Advance the remote tracking ref independently.
subprocess.check_call(
["git", "checkout", "-q", "main"], cwd=project_worktree
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "upstream"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "update-ref", "refs/remotes/origin/main", "main"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "checkout", "-q", "feature"],
cwd=project_worktree,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header_with_ahead_behind(
lines[0],
project_path,
"feature",
" [ahead 1, behind 1]",
)
def test_status_branch_no_tracking_no_ahead_behind(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify no ahead/behind when branch has no upstream."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
subprocess.check_call(
[
"git",
"checkout",
"-q",
"-b",
"orphan-branch",
"--no-track",
"main",
],
cwd=project_worktree,
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "c1"],
cwd=project_worktree,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, "orphan-branch")
def test_status_branch_synced_no_ahead_behind(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify no ahead/behind when branch is fully synced."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
_setup_remote_tracking_branch(manifest, "synced")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, "synced")
+856 -3
View File
@@ -14,10 +14,13 @@
"""Unittests for the subcmds/sync.py module."""
import json
import optparse
import os
from pathlib import Path
import shutil
import tempfile
import time
from typing import Dict, List, Optional
import unittest
from unittest import mock
@@ -26,10 +29,51 @@ import pytest
import command
from error import GitError
from error import RepoExitError
import manifest_xml
from project import SyncNetworkHalfResult
from subcmds import sync
@pytest.mark.parametrize(
"cli_args, expected",
[
([], None),
(["--recurse-submodules"], True),
(["--no-recurse-submodules"], False),
(["--fetch-submodules"], True),
(["--no-fetch-submodules"], False),
],
)
def test_recurse_submodules_option(cli_args, expected):
"""The submodule flags preserve an unset manifest-driven state."""
cmd = sync.Sync()
opts, _ = cmd.OptionParser.parse_args(cli_args)
assert opts.recurse_submodules is expected
@pytest.mark.parametrize(
"old_flag, new_flag",
[
("--fetch-submodules", "--recurse-submodules"),
("--no-fetch-submodules", "--no-recurse-submodules"),
],
)
def test_recurse_submodules_option_deprecation(old_flag, new_flag):
"""The old submodule flags warn and direct users to their replacements."""
cmd = sync.Sync()
with mock.patch.object(sync.logger, "warning") as warning:
cmd.OptionParser.parse_args([old_flag])
warning.assert_called_once_with(
"%s is deprecated; use %s instead", old_flag, new_flag
)
assert old_flag not in cmd.OptionParser.format_help()
@pytest.mark.parametrize(
"use_superproject, cli_args, result",
[
@@ -56,6 +100,120 @@ def test_get_current_branch_only(use_superproject, cli_args, result):
assert cmd._GetCurrentBranchOnly(opts, cmd.manifest) == result
@pytest.mark.parametrize(
"cli_args, expected_groups",
[
([], None),
(["-g", "groupA"], "groupA"),
(["--groups=groupB,groupC"], "groupB,groupC"),
],
)
def test_groups_option_parsing(cli_args, expected_groups):
"""Test --groups / -g option parsing."""
cmd = sync.Sync()
opts, _ = cmd.OptionParser.parse_args(cli_args)
assert opts.groups == expected_groups
def _create_manifest_with_groups(topdir: Path) -> manifest_xml.XmlManifest:
"""Create a test XmlManifest with projects assigned to various groups."""
repodir = topdir / ".repo"
manifest_dir = repodir / "manifests"
manifest_file = repodir / manifest_xml.MANIFEST_FILE_NAME
repodir.mkdir(exist_ok=True)
manifest_dir.mkdir(exist_ok=True)
gitdir = repodir / "manifests.git"
gitdir.mkdir(exist_ok=True)
(gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
""",
encoding="utf-8",
)
manifest_file.write_text(
"""
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="proj_g1" path="path_g1" groups="group1" />
<project name="proj_g2" path="path_g2" groups="group2" />
<project name="proj_g1_g2" path="path_g1_g2"
groups="group1,group2" />
<project name="proj_default" path="path_default" />
<project name="proj_notdefault" path="path_notdefault"
groups="notdefault" />
</manifest>
""",
encoding="utf-8",
)
for p in [
"proj_g1",
"proj_g2",
"proj_g1_g2",
"proj_default",
"proj_notdefault",
]:
(repodir / "projects" / f"{p}.git").mkdir(parents=True, exist_ok=True)
return manifest_xml.XmlManifest(str(repodir), str(manifest_file))
@pytest.mark.parametrize(
"cli_args, expected_projects",
[
(["-g", "group1"], ["proj_g1", "proj_g1_g2"]),
(["-g", "group2"], ["proj_g2", "proj_g1_g2"]),
(["-g", "group1,group2"], ["proj_g1", "proj_g1_g2", "proj_g2"]),
(["-g", "default,-group1"], ["proj_default", "proj_g2"]),
([], ["proj_default", "proj_g1", "proj_g1_g2", "proj_g2"]),
],
)
def test_sync_groups_manifest_filtering(
tmp_path: Path, cli_args, expected_projects
):
"""Test that repo sync -g selects only matching projects."""
manifest = _create_manifest_with_groups(tmp_path)
cmd = sync.Sync()
cmd.manifest = manifest
opts, args = cmd.OptionParser.parse_args(cli_args)
projects = cmd.GetProjects(args, groups=opts.groups, missing_ok=True)
project_names = sorted([p.name for p in projects])
assert project_names == sorted(expected_projects)
def test_sync_update_projects_revision_id_respects_groups(tmp_path: Path):
"""Test that _UpdateProjectsRevisionId filters projects using opt.groups."""
manifest = _create_manifest_with_groups(tmp_path)
cmd = sync.Sync()
cmd.manifest = manifest
superproject = mock.MagicMock()
superproject.UpdateProjectsRevisionId.return_value = mock.MagicMock(
manifest_path=None
)
manifest._superproject = superproject
opts, args = cmd.OptionParser.parse_args(["-g", "group1"])
opts.verbose = False
opts.fetch_submodules = False
opts.this_manifest_only = True
opts.local_only = False
with mock.patch.object(
cmd, "GetProjects", wraps=cmd.GetProjects
) as spy_get_projects:
with mock.patch.object(cmd, "ManifestList", return_value=[manifest]):
cmd._UpdateProjectsRevisionId(opts, args, {}, manifest)
spy_get_projects.assert_called_once()
_, kwargs = spy_get_projects.call_args
assert kwargs.get("groups") == "group1"
# Used to patch os.cpu_count() for reliable results.
OS_CPU_COUNT = 24
@@ -335,11 +493,26 @@ class LocalSyncState(unittest.TestCase):
class FakeProject:
def __init__(self, relpath, name=None, objdir=None):
def __init__(
self,
relpath: str,
name: Optional[str] = None,
objdir: Optional[str] = None,
parent: Optional["FakeProject"] = None,
is_derived: bool = False,
revisionId: Optional[str] = None,
gitlink_path: Optional[str] = None,
path_prefix: str = "",
) -> None:
self.relpath = relpath
self.path_prefix = path_prefix
self.name = name or relpath
self.objdir = objdir or relpath
self.worktree = relpath
self.parent = parent
self.is_derived = is_derived
self.revisionId = revisionId
self.gitlink_path = gitlink_path
self.use_git_worktrees = False
self.UseAlternates = False
@@ -348,8 +521,20 @@ class FakeProject:
self.config = mock.MagicMock()
self.EnableRepositoryExtension = mock.MagicMock()
def RelPath(self, local=None):
return self.relpath
@property
def Derived(self) -> bool:
return self.is_derived
def SetRevision(
self, revisionExpr: str, revisionId: Optional[str] = None
) -> None:
self.revisionExpr = revisionExpr
self.revisionId = revisionId or revisionExpr
def RelPath(self, local: bool = True) -> str:
if local:
return self.relpath
return os.path.join(self.path_prefix, self.relpath)
def __str__(self):
return f"project: {self.relpath}"
@@ -397,6 +582,218 @@ class SafeCheckoutOrder(unittest.TestCase):
],
)
def test_sibling_submodules_with_shared_parent_are_serialized(self):
parent = mock.Mock(worktree="/worktree/parent")
other_parent = mock.Mock(worktree="/worktree/other")
p_parent = FakeProject("parent")
p_other = FakeProject("other")
p_parent_sub1 = FakeProject("parent/sub1")
p_parent_sub1.parent = parent
p_parent_sub2 = FakeProject("parent/sub2")
p_parent_sub2.parent = parent
p_other_sub = FakeProject("other/sub")
p_other_sub.parent = other_parent
out = sync._SafeCheckoutOrder(
[p_parent_sub2, p_other_sub, p_parent, p_parent_sub1, p_other]
)
self.assertEqual(
out,
[
[p_other, p_parent],
[p_other_sub, p_parent_sub1],
[p_parent_sub2],
],
)
def test_nested_submodules_respect_delayed_parent_level(self):
parent = mock.Mock(worktree="/worktree/parent")
sub1 = mock.Mock(worktree="/worktree/parent/sub1")
sub2 = mock.Mock(worktree="/worktree/parent/sub2")
p_parent = FakeProject("parent")
p_parent_sub1 = FakeProject("parent/sub1")
p_parent_sub1.parent = parent
p_parent_sub1_nested = FakeProject("parent/sub1/nested")
p_parent_sub1_nested.parent = sub1
p_parent_sub2 = FakeProject("parent/sub2")
p_parent_sub2.parent = parent
p_parent_sub2_nested = FakeProject("parent/sub2/nested")
p_parent_sub2_nested.parent = sub2
out = sync._SafeCheckoutOrder(
[
p_parent_sub2_nested,
p_parent_sub2,
p_parent_sub1_nested,
p_parent,
p_parent_sub1,
]
)
self.assertEqual(
out,
[
[p_parent],
[p_parent_sub1],
[p_parent_sub1_nested, p_parent_sub2],
[p_parent_sub2_nested],
],
)
class ParentFirstBatches(unittest.TestCase):
def test_no_submodules(self) -> None:
p_a = FakeProject("a")
p_a_b = FakeProject("a/b")
out = sync._ParentFirstBatches([p_a, p_a_b])
self.assertEqual(out, [[p_a, p_a_b]])
def test_submodules_follow_their_parent(self) -> None:
p_a = FakeProject("a")
p_a_b = FakeProject("a/b", parent=p_a, is_derived=True)
p_a_b_c = FakeProject("a/b/c", parent=p_a_b, is_derived=True)
out = sync._ParentFirstBatches([p_a_b_c, p_a, p_a_b])
self.assertEqual(out, [[p_a], [p_a_b], [p_a_b_c]])
class RefreshDerivedRevisions(unittest.TestCase):
def _parent_with_submodules(self, **gitlinks: str) -> FakeProject:
p_a = FakeProject("a")
p_a.GetSubmoduleRevisions = mock.Mock(return_value=gitlinks)
return p_a
def _submodule(self, parent: FakeProject, path: str) -> FakeProject:
return FakeProject(
f"a/{path}", parent=parent, is_derived=True, gitlink_path=path
)
def test_reads_each_parent_once(self) -> None:
p_a = self._parent_with_submodules(b="beef1234", c="cafe1234")
p_a_b = self._submodule(p_a, "b")
p_a_c = self._submodule(p_a, "c")
sync._RefreshDerivedRevisions([p_a, p_a_b, p_a_c])
p_a.GetSubmoduleRevisions.assert_called_once_with()
self.assertEqual(p_a_b.revisionId, "beef1234")
self.assertEqual(p_a_c.revisionId, "cafe1234")
def test_reuses_gitlinks_read_for_an_earlier_level(self) -> None:
p_a = self._parent_with_submodules(b="beef1234", c="cafe1234")
p_a_b = self._submodule(p_a, "b")
p_a_c = self._submodule(p_a, "c")
submodule_revisions = {}
sync._RefreshDerivedRevisions([p_a_b], submodule_revisions)
sync._RefreshDerivedRevisions([p_a_c], submodule_revisions)
p_a.GetSubmoduleRevisions.assert_called_once_with()
self.assertEqual(p_a_c.revisionId, "cafe1234")
def test_tells_apart_projects_with_the_same_path(self) -> None:
# Paths are relative to their own (sub)manifest, so two projects can
# share one.
first = self._parent_with_submodules(b="beef1234")
second = self._parent_with_submodules(b="cafe1234")
first_sub = self._submodule(first, "b")
second_sub = self._submodule(second, "b")
submodule_revisions = {}
sync._RefreshDerivedRevisions([first_sub], submodule_revisions)
sync._RefreshDerivedRevisions([second_sub], submodule_revisions)
self.assertEqual(first_sub.revisionId, "beef1234")
self.assertEqual(second_sub.revisionId, "cafe1234")
def test_ignores_projects_from_the_manifest(self) -> None:
p_a = self._parent_with_submodules()
sync._RefreshDerivedRevisions([p_a])
p_a.GetSubmoduleRevisions.assert_not_called()
def test_reports_submodules_removed_from_their_parent(self) -> None:
p_a = self._parent_with_submodules(b="beef1234")
p_a_c = self._submodule(p_a, "c")
removed = sync._RefreshDerivedRevisions([p_a, p_a_c])
self.assertEqual(removed, [p_a_c])
def test_keeps_submodules_of_an_unreadable_parent(self) -> None:
p_a = FakeProject("a")
p_a.GetSubmoduleRevisions = mock.Mock(return_value=None)
p_a_b = FakeProject(
"a/b",
parent=p_a,
is_derived=True,
revisionId="stale",
gitlink_path="b",
)
removed = sync._RefreshDerivedRevisions([p_a, p_a_b])
self.assertEqual(removed, [])
self.assertEqual(p_a_b.revisionId, "stale")
class FetchParentFirst(unittest.TestCase):
def test_submodules_are_fetched_after_their_parent(self) -> None:
cmd = sync.Sync()
cmd._fetch_times = mock.Mock()
cmd._fetch_times.Get = mock.Mock(return_value=0)
calls = []
p_a = FakeProject("a")
def fake_read() -> Dict[str, str]:
calls.append(("read gitlinks of", p_a.relpath))
return {"b": "beef1234"}
p_a.GetSubmoduleRevisions = mock.Mock(side_effect=fake_read)
p_a_b = FakeProject(
"a/b", parent=p_a, is_derived=True, gitlink_path="b"
)
def fake_fetch(
projects: List[FakeProject], *_args: object
) -> sync._FetchResult:
calls.append(("fetch", [p.relpath for p in projects]))
return sync._FetchResult(True, {p.objdir for p in projects})
opt = mock.Mock(fail_fast=False)
with mock.patch.object(cmd, "_Fetch", side_effect=fake_fetch):
result = cmd._FetchParentFirst([p_a_b, p_a], opt, None, None, [])
self.assertTrue(result.success)
self.assertEqual(result.projects, {"a", "a/b"})
self.assertEqual(
calls,
[
("fetch", ["a"]),
("read gitlinks of", "a"),
("fetch", ["a/b"]),
],
)
class WithoutProjects(unittest.TestCase):
def test_drops_the_unwanted_projects(self) -> None:
p_a = FakeProject("a")
p_a_b = FakeProject("a/b")
self.assertEqual(sync._WithoutProjects([p_a, p_a_b], [p_a_b]), [p_a])
self.assertEqual(sync._WithoutProjects([p_a, p_a_b], []), [p_a, p_a_b])
def test_keeps_projects_with_the_same_path(self) -> None:
# Paths are relative to their own (sub)manifest, so two projects can
# share one.
first = FakeProject("a/b")
second = FakeProject("a/b")
self.assertEqual(
sync._WithoutProjects([first, second], [second]), [first]
)
class Chunksize(unittest.TestCase):
"""Tests for _chunksize."""
@@ -732,6 +1129,19 @@ class SyncCommand(unittest.TestCase):
self.assertIn(self.sync_local_half_error, e.aggregate_errors)
self.assertIn(self.sync_network_half_error, e.aggregate_errors)
def test_groups_passed_to_get_projects(self):
"""Ensure Execute passes opt.groups to GetProjects."""
self.opt.groups = "my_group"
self.opt.mp_update = False
with mock.patch.object(self.cmd, "_UpdateRepoProject"):
with mock.patch.object(self.cmd, "_ValidateOptionsWithManifest"):
with mock.patch.object(self.cmd, "_SyncInterleaved"):
with mock.patch.object(self.cmd, "_RunPostSyncHook"):
self.cmd.Execute(self.opt, [])
self.cmd.GetProjects.assert_called()
_, kwargs = self.cmd.GetProjects.call_args
self.assertEqual(kwargs.get("groups"), "my_group")
class SyncUpdateRepoProject(unittest.TestCase):
"""Tests for Sync._UpdateRepoProject."""
@@ -929,6 +1339,203 @@ class InterleavedSyncTest(unittest.TestCase):
execute_mock.assert_called_once()
def test_interleaved_refreshes_submodule_revision(self) -> None:
"""Test submodules are synced at the revision of the fetched parent."""
opt, args = self.cmd.OptionParser.parse_args(["--interleaved", "-j4"])
opt.quiet = True
submodule = FakeProject(
"projA/sub",
name="projA_sub",
objdir="objA_sub",
parent=self.projA,
is_derived=True,
revisionId="stale",
gitlink_path="sub",
)
all_projects = [self.projA, submodule]
mock.patch.object(
self.cmd, "GetProjects", return_value=all_projects
).start()
self.projA.GetSubmoduleRevisions = mock.Mock(
return_value={"sub": "fetched"}
)
synced = []
def execute_side_effect(
jobs: int,
target: object,
work_items: List[List[int]],
**kwargs: object,
) -> bool:
synced_relpaths_set = kwargs["callback"].args[0]
projects_in_pass = self.cmd.get_parallel_context()["projects"]
for item in work_items:
for project_idx in item:
project = projects_in_pass[project_idx]
synced.append((project.relpath, project.revisionId))
synced_relpaths_set.add(project.relpath)
return True
mock.patch.object(
self.cmd, "ExecuteInParallel", side_effect=execute_side_effect
).start()
self.cmd._SyncInterleaved(
opt,
args,
[],
self.manifest,
self.manifest.manifestProject,
all_projects,
{},
)
self.assertIn(("projA/sub", "fetched"), synced)
def test_interleaved_skips_removed_submodule(self) -> None:
"""Test submodules dropped by their parent are not checked out."""
opt, args = self.cmd.OptionParser.parse_args(["--interleaved", "-j4"])
opt.quiet = True
submodule = FakeProject(
"projA/sub",
name="projA_sub",
objdir="objA_sub",
parent=self.projA,
is_derived=True,
revisionId="stale",
gitlink_path="sub",
)
# The parent no longer holds a gitlink for the submodule.
self.projA.GetSubmoduleRevisions = mock.Mock(return_value={})
# The reloaded manifest no longer derives the removed submodule.
mock.patch.object(
self.cmd, "GetProjects", return_value=[self.projA]
).start()
synced = []
def execute_side_effect(
jobs: int,
target: object,
work_items: List[List[int]],
**kwargs: object,
) -> bool:
synced_relpaths_set = kwargs["callback"].args[0]
projects_in_pass = self.cmd.get_parallel_context()["projects"]
for item in work_items:
for project_idx in item:
project = projects_in_pass[project_idx]
synced.append(project.relpath)
synced_relpaths_set.add(project.relpath)
return True
mock.patch.object(
self.cmd, "ExecuteInParallel", side_effect=execute_side_effect
).start()
self.cmd._SyncInterleaved(
opt,
args,
[],
self.manifest,
self.manifest.manifestProject,
[self.projA, submodule],
{},
)
self.assertEqual(synced, ["projA"])
def _make_syncable(self, project: FakeProject) -> FakeProject:
project.Sync_NetworkHalf = mock.Mock(
return_value=SyncNetworkHalfResult(error=None, remote_fetched=True)
)
project.Sync_LocalHalf = mock.Mock()
return project
def _run_interleaved(
self,
opt: optparse.Values,
initial_projects: List[FakeProject],
reloaded_projects: List[FakeProject],
) -> None:
"""Run _SyncInterleaved with the real workers and callback.
|initial_projects| make up the first pass, |reloaded_projects| every
later one, the way reloading the manifest between passes does.
"""
mock.patch.object(
self.cmd, "GetProjects", return_value=reloaded_projects
).start()
mock.patch.object(self.cmd, "event_log").start()
def execute_side_effect(
jobs: int,
target: object,
work_items: List[List[int]],
**kwargs: object,
) -> bool:
results = [target(item) for item in work_items]
return kwargs["callback"](None, kwargs["output"], results)
mock.patch.object(
self.cmd, "ExecuteInParallel", side_effect=execute_side_effect
).start()
with mock.patch("subcmds.sync.SyncBuffer") as mock_sync_buffer:
mock_sync_buffer.return_value.Finish.return_value = True
mock_sync_buffer.return_value.errors = []
self.cmd._SyncInterleaved(
opt,
[],
[],
self.manifest,
self.manifest.manifestProject,
initial_projects,
{},
)
def test_interleaved_syncs_same_path_projects_of_every_manifest(
self,
) -> None:
"""Test a project is not skipped because another shares its path."""
opt = self._get_opts(["--interleaved", "-j4"])
outer = self._make_syncable(
FakeProject("foo", name="outer", objdir="a")
)
sub = self._make_syncable(
FakeProject("foo", name="sub", objdir="b", path_prefix="sub")
)
# |sub| is only discovered once the manifest is reloaded after the
# first pass has synced |outer|.
self._run_interleaved(opt, [outer], [outer, sub])
outer.Sync_LocalHalf.assert_called_once()
sub.Sync_LocalHalf.assert_called_once()
def test_interleaved_reports_failures_by_a_unique_path(self) -> None:
"""Test failing projects are listed by a path that is theirs alone."""
opt = self._get_opts(["--interleaved", "-j4"])
outer = self._make_syncable(
FakeProject("foo", name="outer", objdir="a")
)
sub = self._make_syncable(
FakeProject("foo", name="sub", objdir="b", path_prefix="sub")
)
sub.Sync_LocalHalf.side_effect = GitError("checkout failed")
self.cmd.git_event_log = mock.MagicMock()
with self.assertRaises(sync.SyncError):
self._run_interleaved(opt, [outer, sub], [outer, sub])
self.assertEqual(
self.cmd._interleaved_err_checkout_results, ["sub/foo"]
)
def test_interleaved_shared_objdir_serial(self):
"""Test that projects with shared objdir are processed serially."""
opt, args = self.cmd.OptionParser.parse_args(["--interleaved", "-j4"])
@@ -1023,6 +1630,23 @@ class InterleavedSyncTest(unittest.TestCase):
project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_called_once()
def test_worker_reports_a_path_unique_across_manifests(self) -> None:
"""Test _SyncResult.relpath tells apart same-path projects."""
project = FakeProject("foo", objdir="objA", path_prefix="sub")
self._make_syncable(project)
self.mock_context["projects"] = [project]
for this_manifest_only, expected in ((False, "sub/foo"), (True, "foo")):
with self.subTest(this_manifest_only=this_manifest_only):
opt = self._get_opts()
opt.this_manifest_only = this_manifest_only
with mock.patch("subcmds.sync.SyncBuffer") as mock_sync_buffer:
mock_sync_buffer.return_value.Finish.return_value = True
mock_sync_buffer.return_value.errors = []
result_obj = self.cmd._SyncProjectList(opt, [0])
self.assertEqual(result_obj.results[0].relpath, expected)
def test_worker_fetch_fails(self):
"""Test _SyncProjectList with a failed fetch."""
opt = self._get_opts()
@@ -1454,3 +2078,232 @@ class UpdateAllManifestProjectsTests(unittest.TestCase):
)
mock_sync_to_rev.assert_not_called()
mock_update_manifest.assert_called_once()
class TestSmartSyncSetupRemoteHelper(unittest.TestCase):
"""Tests for _SmartSyncSetup with remote helpers."""
def setUp(self):
self.cmd = sync.Sync()
self.opt = mock.MagicMock()
self.opt.quiet = False
self.opt.smart_sync = True
self.manifest = mock.MagicMock()
self.smart_sync_manifest_path = "/fake/path/to/manifest.xml"
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
@mock.patch("xmlrpc.client.Server")
@mock.patch("subcmds.sync.PersistentTransport")
def test_smart_sync_setup_with_helper(
self, mock_transport_class, mock_server_class, mock_popen, mock_which
):
"""Test _SmartSyncSetup when a helper is present and succeeds."""
import subprocess
self.manifest.manifest_server = (
"persistent-https://android-smartsync.corp.google.com/"
"manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
# Mock subprocess to return a JSON with status ok and proxy address
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
'{"status":"ok","message":"http://127.0.0.1:999"}\n',
"",
)
mock_process.returncode = 0
mock_popen.return_value = mock_process
# Mock XML-RPC server call
mock_server = mock.MagicMock()
mock_server.GetApprovedManifest.return_value = [
True,
"<manifest></manifest>",
]
mock_server_class.return_value = mock_server
# Mock manifest project branch
self.cmd._GetBranch = mock.MagicMock(return_value="main")
self.cmd._ReloadManifest = mock.MagicMock()
# Mock open to avoid writing to disk
with mock.patch("builtins.open", mock.mock_open()):
manifest_name = self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
# Assertions
mock_which.assert_called_once_with("repo-remote-sso")
mock_popen.assert_called_once_with(
["repo-remote-sso", self.manifest.manifest_server],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
# Verify transport was created with the proxy returned by helper (with
# http:// prepended)
mock_transport_class.assert_called_once_with(
self.manifest.manifest_server, proxy="http://127.0.0.1:999"
)
# Verify Server was created with the same URL, with persistent- stripped
mock_server_class.assert_called_once_with(
"https://android-smartsync.corp.google.com/manifestserver",
transport=mock_transport_class.return_value,
)
self.assertEqual(manifest_name, "manifest.xml")
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
def test_smart_sync_setup_helper_error(self, mock_popen, mock_which):
"""Test _SmartSyncSetup when helper returns an error status."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
# Mock subprocess to return a JSON with status error
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
'{"status":"error","message":"uplink-helper failed"}\n',
"",
)
mock_process.returncode = 0
mock_popen.return_value = mock_process
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"helper repo-remote-sso returned error: uplink-helper failed",
str(context.exception),
)
@mock.patch("shutil.which")
def test_smart_sync_setup_missing_declared_helper(self, mock_which):
"""Test _SmartSyncSetup when helper declared in manifest is missing."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = None
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"helper binary 'repo-remote-sso' declared in manifest was not "
"found",
str(context.exception),
)
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
def test_smart_sync_setup_helper_exit_code_error(
self, mock_popen, mock_which
):
"""Test _SmartSyncSetup when helper exits with non-zero and stderr."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
# Mock subprocess: exit code 1, stderr, and no JSON on stdout
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
"",
"internal binary error occurred\n",
)
mock_process.returncode = 1
mock_popen.return_value = mock_process
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"helper repo-remote-sso exited with exit code 1. "
"Stderr: internal binary error occurred",
str(context.exception),
)
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
def test_smart_sync_setup_helper_json_decode_error_with_stderr(
self, mock_popen, mock_which
):
"""Test _SmartSyncSetup when helper returns invalid JSON and stderr."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
"not a json",
"some warning messages\n",
)
mock_process.returncode = 0
mock_popen.return_value = mock_process
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"failed to parse JSON from helper repo-remote-sso",
str(context.exception),
)
self.assertIn(
"Stderr was: some warning messages",
str(context.exception),
)
@mock.patch("shutil.which")
@mock.patch("subprocess.Popen")
def test_smart_sync_setup_helper_error_with_stderr(
self, mock_popen, mock_which
):
"""Test _SmartSyncSetup when helper returns error status and stderr."""
self.manifest.manifest_server = (
"http://android-smartsync.corp.google.com/manifestserver"
)
self.manifest.manifest_server_helper = "repo-remote-sso"
mock_which.return_value = "/fake/bin/repo-remote-sso"
mock_process = mock.MagicMock()
mock_process.communicate.return_value = (
'{"status":"error","message":"uplink-helper failed"}\n',
"debugging logs\n",
)
mock_process.returncode = 0
mock_popen.return_value = mock_process
with self.assertRaises(sync.SmartSyncError) as context:
self.cmd._SmartSyncSetup(
self.opt, self.smart_sync_manifest_path, self.manifest
)
self.assertIn(
"helper repo-remote-sso returned error: uplink-helper failed",
str(context.exception),
)
self.assertIn(
"Stderr was: debugging logs",
str(context.exception),
)
+54
View File
@@ -63,3 +63,57 @@ def test_UploadAndReport_UnhandledError(cmd: upload.Upload) -> None:
with mock.patch.object(cmd, "_UploadBranch", side_effect=UnexpectedError):
with pytest.raises(UnexpectedError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
def test_GetMergeBranch_explicit_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch reads branch.merge for explicit local_branch."""
mock_project = mock.MagicMock()
mock_branch = mock.MagicMock()
mock_branch.merge = "refs/heads/main"
mock_project.GetBranch.return_value = mock_branch
res = cmd._GetMergeBranch(mock_project, local_branch="feature")
assert res == "refs/heads/main"
mock_project.GetBranch.assert_called_once_with("feature")
def test_GetMergeBranch_current_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch falls back to project.CurrentBranch."""
mock_project = mock.MagicMock()
mock_project.CurrentBranch = "auto-cbr"
mock_branch = mock.MagicMock()
mock_branch.merge = "refs/heads/upstream-main"
mock_project.GetBranch.return_value = mock_branch
res = cmd._GetMergeBranch(mock_project, local_branch=None)
assert res == "refs/heads/upstream-main"
mock_project.GetBranch.assert_called_once_with("auto-cbr")
def test_GetMergeBranch_none_when_no_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch returns empty string when detached HEAD."""
mock_project = mock.MagicMock()
mock_project.CurrentBranch = None
res = cmd._GetMergeBranch(mock_project, local_branch=None)
assert res == ""
def test_GatherOne_returns_resolved_current_branch(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Upload error reporting reuses the branch gathered by the worker."""
project = mock.MagicMock()
project.CurrentBranch = "topic"
branch = mock.sentinel.branch
project.GetUploadableBranch.return_value = branch
monkeypatch.setattr(
upload.Upload,
"get_parallel_context",
lambda: {"projects": [project]},
)
opt = mock.MagicMock(current_branch=True)
assert upload.Upload._GatherOne(opt, 0) == (0, [branch], "topic")
project.GetUploadableBranch.assert_called_once_with("topic")
+52
View File
@@ -0,0 +1,52 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for subcmds/version.py."""
from unittest import mock
import pytest
from subcmds import version
def test_repo_version_uses_one_pretty_format_call(
monkeypatch: pytest.MonkeyPatch,
) -> None:
project = mock.MagicMock()
project.bare_git.log.return_value = "v2.0-1-g12345678\nTue, 25 Aug\n"
monkeypatch.setattr(version, "git_require", lambda _version: True)
result = version.Version._RepoVersion(project)
assert result == ("v2.0-1-g12345678", "Tue, 25 Aug")
project.bare_git.log.assert_called_once_with(
"-1", "--format=%(describe)%n%cD", "HEAD"
)
project.bare_git.describe.assert_not_called()
def test_repo_version_keeps_old_git_fallback(
monkeypatch: pytest.MonkeyPatch,
) -> None:
project = mock.MagicMock()
project.bare_git.describe.return_value = "v2.0"
project.bare_git.log.return_value = "Tue, 25 Aug"
monkeypatch.setattr(version, "git_require", lambda _version: False)
result = version.Version._RepoVersion(project)
assert result == ("v2.0", "Tue, 25 Aug")
project.bare_git.describe.assert_called_once_with("HEAD")
project.bare_git.log.assert_called_once_with("-1", "--format=%cD", "HEAD")