Compare commits

..
67 Commits
Author SHA1 Message Date
Insaf Latypov a0bc2b14a3 git_config: Fix bugs in processing RepoConfig.
- Automatically create all the parent directories for `git confing
  --file ...`
- Prevent `RepoConfig` from inheriting caches from `GitConfig`. These
  two are different files.

Bug: 569927750
Test: ./run_tests
Change-Id: I9c60cecff75c1fb62be3ff63425d9618c0f56b06
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/643263
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Insaf Latypov <saferif@google.com>
Commit-Queue: Insaf Latypov <saferif@google.com>
2026-10-14 07:49:21 -07:00
Rahul Yadav 3448df3954 sync: Add GetApprovedManifestV2 with named request options
Smart sync calls GetApprovedManifest(branch[, target]) over XML-RPC,
which only supports positional arguments.  Adding new request options
that way does not scale.

Add a GetApprovedManifestV2 method that takes a single struct of named
options and returns a struct:

  request:  {branch, target?, best_effort?}
  response: {success, manifest_xml?, error_message?}

Servers must ignore unknown options, so new options (e.g. criteria for
selecting a good build) can be added without another method.

The first new option is `best_effort`.  repo sets it when smart sync was
enabled implicitly by the manifest's sync-smartsync="true" default
rather than by -s.  The server may decline a best_effort request (e.g.
for rollout or cohort gating) by returning success=false.  The client
still owns the fallback policy: declined implicit requests fall back to
a regular sync, while explicit -s requests fail as before.

If the server does not implement GetApprovedManifestV2, repo falls back
to the legacy GetApprovedManifest call.

Bug: 545137090
Test: ./run_tests
Change-Id: I9297b728df26516012cb93dffc91d39c1fb0458f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/636623
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
2026-10-14 03:30:36 -07:00
Brian Gan a110680ff3 repo: tolerate a transiently missing stable branch
The automated release process deletes the stable branch and then
recreates it. A client that fetches during that window can't find it:
`repo init` fails to resolve --repo-rev, and the daily self-update in
`repo sync` fails with "Cannot fetch repo tool".

In the launcher, if --repo-rev doesn't resolve after the initial fetch,
wait 10 seconds and refetch once before giving up. `resolve_repo_rev`
gains a `report_errors` flag so this first check doesn't print an error
that the refetch then fixes. Bump the launcher version to 2.69 so
existing users pick this up.

For fetches of the repo project itself (sync's self-update and
`repo selfupdate`), wait 10 seconds instead of 4 before retrying. The
delay is now a `RETRY_SLEEP_INITIAL_SEC` class attribute that
`RepoProject` overrides, so other projects keep the 4 second delay. This
replaces the unused `retry_sleep_initial_sec` parameter of
`_RemoteFetch`.

Bug: 566270786
Change-Id: I23cb810258522b5aa9eae9d7013dbb7904ebcad8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/643265
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-13 13:55:40 -07:00
Rahul Yadav 178b66ee3e sync: keep smart sync override with -l
Splitting a sync into `repo sync -n` followed by `repo sync -l` should
end up in the same state as a single `repo sync`. With smart sync it
does not always: -n fetches the revisions pinned by the smart sync
manifest and saves it as smart_sync_override.xml, but the following -l
deletes that override and checks out ToT instead, unless the manifest
sets sync-smartsync="true". So `repo sync -s -n` + `repo sync -l` lands
on ToT, while `repo sync -s` lands on the pinned revisions.

Without an explicit -s or -t, -l does not contact the manifest server,
so it cannot choose new pins itself; the only consistent choice is to
keep the override the previous -n left behind. Do that regardless of
the manifest default.

Syncs that choose a manifest themselves still replace or remove the
override as before: any sync without -l, and -l combined with -s, -t,
-m, --superproject-revision or --no-smart-sync. Smart sync stays
non-sticky for regular syncs, and `repo sync -l --no-smart-sync` still
discards the override to check out ToT.

Bug: 569927750
Change-Id: Id06ff88e9f3183fae32e993e86630fc3e8de71a1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/640965
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-12 02:14:30 -07:00
Gavin Mak beda65323e sync: skip clone.bundle when repo.fetchcmd is configured
In Sync_NetworkHalf, clone_bundle defaults to True on initial sync when
self.objdir does not exist yet, and _ApplyCloneBundle is called before
custom_fetch_configured is checked.

For remotes with HTTP/HTTPS URLs that serve /clone.bundle, this causes
repo sync to download and unpack the full clone.bundle into .git/objects
before invoking repo.fetchcmd.

Skip _ApplyCloneBundle when custom_fetch_configured is enabled so
repo.fetchcmd exclusively handles object fetching.

Bug: 543101585
Change-Id: If41a775a75589ad5a23106163f9932e2940b72d2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641685
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-10-09 14:09:50 -07:00
Gavin Mak 02e07525f9 project: peel annotated tags in GetRevisionId with cached refs
When all_refs (e.g. self.bare_ref.all in Sync_LocalHalf) is passed to
GetRevisionId, looking up a tag ref in all_refs returns the tag object
SHA for annotated tags rather than the peeled commit SHA.

In Sync_LocalHalf, this causes repo.reprojectcmd to receive a tag SHA in
REPO_TREV and DetachHead to fail with 'trying to write non-commit object
to branch HEAD', and causes standard checkouts to re-checkout on every
subsequent sync because HEAD (the commit SHA) never equals revid (the
tag object SHA).

Skip the all_refs fast path for refs/tags/* so tags always fall through
to bare_git.ResolveCommit, which peels to the underlying commit SHA.

Bug: 543101585
Change-Id: Ie4afa3cee9149e2a8438bd8fb4a8026a210848d8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641683
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-10-09 14:09:44 -07:00
Gavin Mak 2e8a3805bb sync: don't run git ls-remote before fetchcmd on unpinned refs
When a project's revision is an unpinned branch or tag and superprojects
are not used (so self.revisionId is unset), _CustomFetch previously ran
git ls-remote against the Git remote before invoking repo.fetchcmd just
to pre-populate REPO_TREV.

This added a per-project network round-trip before fetchcmd and forced
branch resolution against the Git remote rather than letting fetchcmd
resolve REPO_RREV from its own server (just as standard git fetch does
in _RemoteFetch).

Only populate REPO_TREV before fetchcmd when the target commit SHA is
already known (via self.revisionId from superproject or a SHA-pinned
revisionExpr). When REPO_RREV is an unpinned branch or tag, leave
REPO_TREV empty, let fetchcmd resolve REPO_RREV and update the local
tracking ref and FETCH_HEAD, and verify afterward that the tracking ref
and FETCH_HEAD resolve to the same valid commit in the object store.

Bug: 543101585
Change-Id: I482bf4c022d0e669e5620d4517581ac193983da0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/640423
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-10-09 14:09:37 -07:00
Becky Siegel 277759b9e1 agents: enable generic multi-perspective code review agent
Configure git-repo-code-review-agent with use_default_subagents: true in
agents/agent_configs.txtpb so groupagent-grev-review uses the generic
gerrit_reviewer multi-perspective review framework.

Bug: 555706086
Change-Id: I079681b00e71cb886dc2343692d88b01d58a51ea
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641904
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Becky Siegel <beckysiegel@google.com>
2026-10-09 13:10:58 -07:00
Brian Gan 9593a76fd1 sync: keep --superproject-revision across manifest reloads
When --superproject-revision moves `.repo/manifests`, the manifest is
reloaded, discarding the pinned superproject. _UpdateProjectsRevisionId
then read gitlinks from the superproject's default branch instead,
pinning failed, and projects synced unpinned.

Apply the revision in _ConfigureSuperproject so it is re-applied after
every reload.

Verified with https://paste.googleplex.com/5889974172516352

Bug: 571245807
Change-Id: I4d782d229b128c0f15081d11b722d8901709458c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641743
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-09 12:56:07 -07:00
Brian Gan 9293cda144 sync: roll back repo when stable is rewound
The documented way to roll back a bad release is to force push an older
release to stable, and docs/release-process.md says clients will then
automatically downgrade. _PostRepoFetch already supports switching to an
older commit, but it only runs when `RepoProject.HasChanges` is true,
and that used the `MetaProject` check of whether the remote has commits
that HEAD lacks. After a rewind it doesn't, so clients that had already
upgraded stayed on the bad release.

Make `RepoProject.HasChanges` true whenever the remote revision differs
from HEAD, in either direction. `ManifestProject` keeps the forward-only
check so local manifest commits are still rebased rather than dropped.

Bug: 571204396
Change-Id: I248507f434d2552ce11a9afabe9a6335cb24ddbd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/640130
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-09 09:49:49 -07:00
Rahul Yadav 340b587e94 sync: Support SYNC_BRANCH to override the smart sync branch
`repo sync -s` always sends the manifest branch to the manifest server
as the `branch` argument of GetApprovedManifest. Some servers map one
manifest branch to several server-side (CI) branches, and the only way
to pick among them was indirectly through the target.

Honor a SYNC_BRANCH environment variable, mirroring SYNC_TARGET: if set
and non-empty, send it instead of the manifest branch. An empty value
is treated as unset. -t/--smart-tag is unaffected.

Bug: 568962967
Test: ./run_tests
Change-Id: I26700e8997c21487f83011258100fd91a5c87508
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/639123
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
2026-10-09 07:00:47 -07:00
Rahul Yadav 0a4e68f9d2 sync: don't let --nmu affect smart sync
--no-manifest-update has always meant "use the existing manifest
checkout as-is": it skips updating .repo/manifests and nothing else.
Smart sync pins do not come from that checkout but from the manifest
server, and an explicit `-s --nmu` already works that way.

The manifest-driven default smart sync (sync-smartsync="true") instead
treated --nmu like -l: it disabled the implicit smart sync and kept an
existing smart_sync_override.xml. So on such manifests `repo sync --nmu`
fetched and checked out the previous pins (or ToT if there were none)
instead of doing what `repo sync` does minus the manifest checkout
update.

Make --nmu orthogonal to smart sync again: it no longer disables the
default smart sync, and it no longer keeps the override. Both have to
change together; otherwise a failed implicit smart sync under --nmu
would fall back to ToT while leaving the override on disk. Only -l,
which skips fetching, keeps the override.

Bug: 569927750
Change-Id: I33730363df485d72cd86ae99bf5eb8399b58e81a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641264
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
2026-10-09 07:00:22 -07:00
Mike Frysinger 7bba4ee47e vpython: update to uv spec format
Change-Id: I18dca6f8c9c1f9234cfee640f780f1f2e4462ab8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/635623
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-10-05 10:40:31 -07:00
Rahul Yadav d53a4278f5 sync: Support manifest-driven default smart sync
Add support for enabling smart sync by default via the
sync-smartsync="true" attribute on the manifest <default> element.

When sync-smartsync is set to true in the manifest defaults:
- Plain `repo sync` triggers Smart Sync automatically.
- If that implicit Smart Sync fails (manifest server unreachable, or
  the returned manifest cannot be parsed), repo silently falls back to
  a regular ToT sync. The reason is logged with --verbose.
- `--no-smart-sync` CLI flag overrides the manifest default and
  syncs to ToT.
- Passing explicit `-s` / `--smart-sync` forces Smart Sync as before,
  and its failures are still fatal.
- `-t`, `--smart-tag`, `-m`, `--manifest-name`, and
  `--superproject-revision` disable the default smart sync.
- `-l` / `--local-only` and `--no-manifest-update` skip the manifest
  server and reuse the existing smart_sync_override.xml, if any.

Update docs, completion, regenerate man pages, and add tests for XML
parsing, serialization, CLI option resolution, and the fallback and
reuse paths.

Bug: 545137090
Change-Id: If75569c58dd525d170e9801e70ccbf02206a52f2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/617641
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-01 03:13:33 -07:00
Victor Pushkarev 5d4149f32e gc: add --keep-manifest-projects
By default, `repo gc` treats projects excluded by the configured
manifest groups as unused. This matches the active project set, but it
can remove cached Git data for projects that might be used again after
changing groups.

Add `--keep-manifest-projects` to keep Git data for projects that are
still in the manifests, even if the current groups exclude them or their
checkout is missing. Git data for projects no longer in the manifests
can still be removed.

Change-Id: I63ae18ce1c6acc2463be86f81921476b12fb27d7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/629741
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-30 01:11:40 -07:00
Ram Peri cf499e312a sync: record superproject sync analysis state again
Since the multi-manifest change (bdcba7d), _UpdateProjectsRevisionId
rebinds its superproject_logging_data parameter to a new local dict.
The caller keeps its own empty dict and passes it to
UpdateSyncAnalysisState, so repo.syncstate.superproject.* is never
written to the manifest config or logged to trace2.

Drop the rebinding so the caller's dict is populated.

Also record superproject, haslocalmanifests and hassuperprojecttag
when no manifest declares a superproject. Before bdcba7d these keys
were logged on every sync. SyncAnalysisState does not remove old
keys, so without this a checkout that moves to a manifest without a
superproject keeps reporting the values from its last superproject
sync.

Bug: 565957530
Test: ./run_tests
Change-Id: I8796e3a34b9cf4786383d25ecc3ebf0c5409caa3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634843
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Tested-by: Ram Peri <ramperi@google.com>
2026-09-28 15:34:54 -07:00
Mike Frysinger cc0546bc71 isort: print diff on failures
This should help users figure out how to fix problems.

Change-Id: I6346e163d06f9a0e52147689c5b329fe7f1366d2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/635343
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-28 10:55:16 -07:00
Victor Pushkarev ccf9ce39b8 sync: keep local sync state paths relative to repo root
`LocalSyncState` uses `Project.relpath` to identify projects, but that
path is relative to each project's own manifest. Projects in different
manifests can have the same relpath, so their fetch and checkout
timestamps overwrite each other. This can make a later partial sync look
like a full sync.

Pruning has a related problem. It treats the stored paths as relative to
the current manifest's topdir, so when sync starts from the outermost
manifest it can look for submanifest projects in the wrong place and
remove valid state.

Use `Project.RelPath(local=False)` to identify projects in local sync
state and check stored paths from the repo root when pruning.

Add tests for same-relpath projects, partial-sync detection, and pruning
when sync starts from either the outermost manifest or a submanifest.

Change-Id: I24485c315f1277a14913443674245cedd20820fe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632421
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-26 01:17:45 -07:00
Ram Peri 9066409c2d init: don't suggest --no-use-superproject on failure
When the superproject can't be fetched, repo init warns and tells users
to re-run repo init with --no-use-superproject. That choice is saved in
the checkout config (repo.superproject), so following it disables the
superproject for that checkout until it is re-inited, even though these
failures are often transient.

Drop the recommendation and keep the rest of the warning.

This is the repo init counterpart of reverting
https://gerrit-review.googlesource.com/c/git-repo/+/305622 for repo sync
(https://gerrit-review.googlesource.com/c/git-repo/+/634804).

Bug: 565845573
Change-Id: I91a6e682ef1d1c6dee07bc7969de88a681e1d239
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634784
Tested-by: Ram Peri <ramperi@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
2026-09-24 13:54:06 -07:00
Ram Peri 2908f596c5 Revert "sync: Recommend using --no-use-superproject if sync fails."
This reverts commit f907ced0fe.
Original change: https://gerrit-review.googlesource.com/c/git-repo/+/305622

Reason for revert: the hint was added when a superproject failure made
repo sync exit with an error. Since 8db30d6 the failure is non-fatal
and sync falls back to fetching normally, and these failures are often
transient (e.g. on 2026-09-23 a manifest change added
external/slackhq/compose-lints before the superproject had a commit
for it, until the change was reverted ~2h later). Following the hint
means users keep passing --no-use-superproject and bypass the
superproject even after the failure is gone.

The revert does not apply cleanly after the black reformat, so the
conflict was resolved by restoring the original message text.

Bug: 565845573
Change-Id: I5efadedd4d3d0002df50493fa29554ba9626fe82
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634804
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Tested-by: Ram Peri <ramperi@google.com>
2026-09-24 13:53:54 -07:00
Gavin Mak e1e215a14e project: don't plan a stateless prune without a worktree
Skip stateless prune planning when a project's worktree directory
does not exist, preventing fetch failures on subsequent syncs.

Also run _LsRemote() bare so it does not depend on worktree
existence.

Bug: 565047698
Change-Id: I385cba4e06813d9113ec7ddc963e8499acffaec5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634426
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-23 13:27:06 -07:00
Gavin Mak 175198b0aa sync: skip the bloat check for projects without a worktree
Skip shallow clone bloat checks during --network-only syncs and
filter out projects whose worktree directory does not exist on disk.

Also return None quietly from _GetStatusSnapshot() when a project
lacks an existing worktree directory.

Bug: 565047698
Change-Id: Ia4c7db36a2bd78bc89e7b72f7a492b901b50efa7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634425
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-23 13:27:01 -07:00
Gavin Mak 310c99571f project: don't rerun git status after a failed snapshot
When a status snapshot fails, fall back directly to _IsDirtyLegacy()
instead of IsDirty() to avoid running git status twice.

Expose Project.GetDirtyAndHead() so sync bloat checks can inspect
worktree state without invoking private Project methods.

Bug: 565047698
Change-Id: I6432ed82da88778eec9d151fd5b70bedb1efc62b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634424
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-23 13:26:55 -07:00
Fredrik de Groot e7d4d97ffc Fix fail in test_repo_trace if trace is disabled
If env has REPO_TRACE 0, then the test produces:
FileNotFoundError: No such file or directory: TRACE_FILE_from_test'
because it assumes trace is always enabled.

Improve by enabling trace for this test class.

Also modify the repo_trace.SetTrace API to take an optional
True/False if needed, still defaults to True though.

Change-Id: I2bb4632eb45ee529fc1785833e0bafd9c36f0875
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/445541
Commit-Queue: Fredrik de Groot <fredrik.de.groot@haleytek.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Fredrik de Groot <fredrik.de.groot@haleytek.com>
2026-09-22 23:43:15 -07:00
Gavin Mak 8c7e0a683e info: reuse branch snapshots and combine history walks
Derive the current branch from GetBranches so attached HEAD and refs are
resolved once, including overview reporting. Replace the two
complementary rev-list calls in --all output with one --left-right
symmetric-difference walk and partition its records in memory.

Bug: 553599402
Change-Id: I01d5db804cbf591ecc0e08d6605971f0f5a38538
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634005
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-21 13:40:06 -07:00
Gavin Mak f6f5946422 project: avoid unnecessary sync revision enumeration
Skip the detached-history walk when its count will not be printed, ask
rev-list --count for upstream gain, and cap the published-but-unmerged
existence probe at one commit. These preserve the decisions while
reducing traversal and output work.

Bug: 553599402
Change-Id: I30f769c9aa5269428ece15ffcb45c50993d7c4b8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634004
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-21 13:40:01 -07:00
Gavin Mak 6321b26685 project: make fast-forward merges explicit
Every caller expects _FastForward to reject a merge commit. Always pass
--ff-only and remove the optional parameter so a violated ancestry
assumption fails instead of silently creating a merge.

Bug: 553599402
Change-Id: I04690b5ca675cb7d3062df08d5e20cd82f5b9e82
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634003
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-21 13:39:56 -07:00
Gavin Mak 7bffc72b5b project: fold normal stash detection into status
When Git 2.35 or newer supplies a stash header, use the same status
snapshot for stateless dirty and stash checks. Retain rev-parse for
earlier clients or when the status snapshot is unavailable.

Bug: 543851900
Bug: 553599402
Change-Id: I6e310459d9c1a3ba7978f7907ae6b7a9bc611749
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632146
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-21 13:39:51 -07:00
Gavin Mak aaadd5da35 sync: reuse status HEAD in the bloat scan
The shallow-project bloat check already needs dirty state. Request
branch metadata in that same porcelain-v2 snapshot and use branch.oid
instead of launching a separate rev-parse, with the existing calls
retained as a fallback.

Bug: 543851900
Bug: 553599402
Change-Id: I3e7068c2e14c025a53adf92bfe8483b876cfda9d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632145
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-21 13:39:21 -07:00
Gavin Mak c9448f986e project: render worktree status from one snapshot
Use porcelain-v2 branch and path records to replace the status display
fan-out with one Git process on Git 2.11 and newer. Preserve the legacy
renderer for older clients, force rename and ahead/behind behavior where
supported, and fall back to the old divergence query only when status
cannot supply branch.ab.

Bug: 543851900
Bug: 553599402
Change-Id: Ic760360f4f48ab9320abf69dbbb7310dab4ac649
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632144
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-21 13:39:16 -07:00
Gavin Mak 530258d08e project: use one status snapshot for dirty checks
On Git 2.11 and newer, replace update-index, diff-index, diff-files, and
ls-files fan-out with one porcelain-v2 snapshot for IsDirty,
UncommittedFiles, and HasChanges. Preserve category ordering and
duplicate staged-plus-unstaged paths, and keep the exact legacy path for
older Git or a failed snapshot.

Bug: 543851900
Bug: 553599402
Change-Id: I309e39977e1f1d4ee6115f488d6aabb83ec3d83c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632143
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-21 13:37:38 -07:00
Gavin Mak c1566487a8 git_status: parse one porcelain v2 worktree snapshot
Add a strict, NUL-aware parser for porcelain v2 records, including
ordinary changes, renames, conflicts, untracked paths, branch
divergence, and stash metadata. The reader is gated on Git 2.11, opts
into newer flags only when supported, and preserves arbitrary pathname
bytes.

Bug: 543851900
Bug: 553599402
Change-Id: Ibae3e1056fd9866f3cb6d490745d377cda1e8fef
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632142
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-21 13:36:55 -07:00
Gavin Mak 22f820de17 git_command: allow lossless capture of machine output
NUL-delimited Git formats may contain path bytes that are not valid
UTF-8. Add an opt-in byte capture mode so parsers can preserve those
bytes instead of receiving backslash replacements, while continuing to
decode stderr for diagnostics.

Bug: 543851900
Bug: 553599402
Change-Id: Ia9dfcde065a0f6f72c785a4b96896f346e17510a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632141
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-21 13:36:47 -07:00
Sam Saccone 4c37f58806 project: pass --no-filter and --no-tags in DownloadPatchSet
In partial clone checkouts (e.g. --filter=blob:none or blob:limit=10k),
DownloadPatchSet previously invoked `git fetch <remote> <ref>` without
explicit filter overrides. Consequently, git fetch applied the
configured partial clone filter from the remote and omitted the blobs
for the incoming change.

When repo download subsequently checks out, cherry-picks, or merges the
downloaded commit, Git discovers that the required blobs are missing
locally and triggers on-demand unadvertised object fetches to the
promisor remote (POST /git-upload-pack with `want <blob_sha>`). On large
repositories, server-side reachability validation for unadvertised
objects can result in expensive graph traversals, proxy/gateway timeouts
(such as HTTP 502 or connection drops), and client fetch failures:
  fatal: expected 'packfile'
  fatal: could not fetch <sha> from promisor remote

Pass --no-filter (when Git >= 2.17.0) and --no-tags when fetching patch
sets in DownloadPatchSet. Because the change ref is an advertised ref
tip, fetching with --no-filter allows the server to generate a complete
packfile containing all required blobs upfront without requiring
unadvertised object reachability checks. All blobs are present in the
local object database prior to checkout or cherry-pick, completely
preventing on-demand promisor queries.

TAG=agy
CONV=bdc489e3-b20b-4ef3-b32e-549dbe21fc6c

Google-Bug-Id: b/532920848
Test: pytest tests/test_project.py
Change-Id: Ic6118e4edeb754f0467e0580fd1d0965a6553153
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/625621
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Sam Saccone <samccone@google.com>
Commit-Queue: Sam Saccone <samccone@google.com>
2026-09-21 09:19:39 -07:00
Gavin Mak c86ddc1628 project: rename UncommitedFiles to UncommittedFiles
Fix typo in UncommitedFiles across project.py and HasChanges.

Bug: 553599402
Change-Id: I2b9b606b4c9a47fd998e50cca976545fcd783771
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632881
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-17 17:21:32 -07:00
Victor Pushkarev c638b54e19 command: preserve project filters in regex selection
`FindProjects` gets its candidate projects from `GetProjects` and then
applies regex matching. However, it only forwards `all_manifests`, so
other project filters are lost on regex paths.

As a result, `repo forall -r/-i --groups` falls back to the configured
group selection instead of using the requested groups.
Similarly, `repo list -r --groups` ignores the requested groups, and
`repo list -r --all` still excludes projects without a checkout.

Pass `groups` and `missing_ok` through `FindProjects`, and forward the
existing options from `forall` and `list`. When no group override is
given, keep using each manifest's effective groups.

Add regression tests for `FindProjects` and both command paths.

Change-Id: I5dfc22172fdd81de5312a84433867300a1342aec
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/631281
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-16 04:54:02 -07:00
Victor Pushkarev fb4a91060e tests: avoid losing trace2 socket readiness signal
The trace2 socket test uses a Condition to wait for its server to start
listening. If the server calls notify() before the test starts waiting,
the notification is lost and the test waits for the full 120-second
timeout before continuing.

Use an Event for the readiness signal instead.
Once the server starts listening, the event stays set, so the test can
continue immediately even if the server becomes ready before the test
starts waiting.

Change-Id: I0e8e192d997c8bd785bcefa5d5b91493f93863e8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/630861
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-16 04:53:31 -07:00
Victor Pushkarev 4b3ada1781 sync: keep generated jobs-checkout help host-independent
The default shown for `--jobs-checkout` comes from `DEFAULT_LOCAL_JOBS`,
which depends on the host CPU count. As a result, regenerating the sync
and smartsync manpages on a different machine can change their output
even though repo's behavior has not changed.

Use shared CPU-based help formatting for `--jobs` and `--jobs-checkout`.
Interactive help still shows the actual default, while generated
manpages describe it without including a host-specific value.
Runtime job selection is unchanged.

Add coverage for both help modes and regenerate the affected manpages.

Change-Id: Ia60e6fcc17b6f4618c52a76217c128eddead8848
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/625381
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-16 04:51:54 -07:00
Gavin Mak a468ea7752 sync: implement reprojectcmd for the local half
Add `repo.reprojectcmd`, the checkout counterpart to `repo.fetchcmd`.
When set, `repo sync` runs this command instead of Git to materialize a
project's index and worktree at the target tree, while `repo` handles
ref updates directly via `git update-ref`.

This replaces Git's tree materialization steps: detaching HEAD,
fast-forwarding, and hard-resetting. Rebasing is not delegated, and the
command is skipped if HEAD is already at the target, if HEAD is ahead of
the target during fast-forward, or for MetaProjects.

The command runs in a subshell with project environment variables (such
as `REPO_TREV`). Before running, `repo` ensures no operation is in
progress and no staged changes exist. Worktree collision detection is
delegated to the command (preserving benign unstaged/untracked edits).
Afterward, `repo` verifies that HEAD was untouched and that the index
matches the target tree.

Like `repo.fetchcmd`, this requires `repo.uselocalgitdirs`. Nested
projects and submodules are unsupported; `repo sync` fails if the
manifest contains any while `repo.reprojectcmd` is enabled.

Verified end-to-end with repo init using local-gitdirs, repo.fetchcmd,
and repo.reprojectcmd ('git -C $REPO_PATH read-tree -m -u $REPO_TREV'):
  * Verified detached HEAD checkout and correct reflog generation across
    projects.
  * Verified benign unstaged edits and untracked files survive checkout.
  * Verified conflicting untracked files fail with exit 128 without
    clobbering worktree.
  * Verified staged changes fail upfront before reprojectcmd is executed.

Bug: 513329573
Change-Id: I964d24d22dccffc05a9b991ad69f3a7e93268c01
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/626281
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-15 12:53:01 -07:00
Brian Gan c2c330ba4a project: report why refreshing the index failed
"repo status" aborts the whole tree when one project fails, without
saying why. The three callers that refresh the index stat cache pass -q,
which suppresses git's diagnosis when it cannot create the index lock;
git then exits 128 with both streams empty. A stale lock, a permissions
problem on the gitdir and a full disk all reach that same silent exit,
yet each needs a different remedy.

Route the three identical call sites through one helper that keeps -q
for the refresh itself. When that fails, it repeats the command without
-q to obtain git's diagnosis. The command still fails, which is correct,
but the user is now told why.

Keeping -q on the common path matters: without it git exits 1 whenever
paths hold uncommitted changes, which would report a failure to
telemetry for every modified project, even if we treat it as benign in
the code. The retry accepts that 1, since --unmerged makes git skip
conflicted paths rather than count them, so nothing else yields it.

The similar call in subcmds/sync.py is unchanged: it passes different
flags, without --unmerged, so exit 1 cannot be assumed benign there.

Bug: 560289756
Change-Id: I2e1024c84454f86076081f2d7f155728b1aa35c6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/630641
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-09-14 13:27:19 -07:00
Victor Pushkarev cc88be34d2 command: fix group selection across submanifests
When GetProjects walks all manifests without explicit groups, it
resolves the effective groups once from the selected manifest and reuses
them for projects from child manifests. This can ignore a child
manifest's own group selection and include projects that should be
excluded.

When no groups are specified, resolve them from each project's owning
manifest. Explicit groups still apply to all projects in the call, and
single-manifest behavior remains unchanged.

Add regression tests for per-manifest group selection, explicit
overrides, and name lookup across manifests.

Bug: 382086294
Change-Id: Id5c7e52410431b86b395f489cfd494ca05620975
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/624821
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-11 01:54:27 -07:00
Victor Pushkarev 16cfb53e0a release: update-manpages: restore __main__ guard
The update-manpages wrapper calls update_manpages.main() unconditionally.
With spawn and forkserver, multiprocessing re-executes the directly run
main script as __mp_main__.
The wrapper then calls main() again and tries to create another Pool
while the child is still starting.

Protect the entrypoint with __main__ guard. On Linux, this was mostly
hidden while fork was the default start method; Python 3.14 now defaults
to forkserver there. Normal command-line behavior is unchanged.

Add a regression test that mirrors multiprocessing's __mp_main__
execution path and checks that it does not call update_manpages.main().

Change-Id: I28fdb18ccf2e4d61605cc072b4de52b6eb561335
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/626841
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-11 01:52:38 -07:00
Fredrik de Groot 578b57c975 manifest_xml: Improve error message on base check fail
Previous error had a scent of alien blob to it
due to dumping the list in a single long line, so
improve print for human readability.

Example of new fail:

error: in `manifest`: detected base-revision mismatch, updates needed:
remove-project name build/make:
  base fccb6b3e884b4b964e795f5f8865f6bbb4722b5a vs revision 6e9a0c16f20d1cfcf08b8b9cb42301b6c9712862
remove-project path build/soong:
  base 16b3c9ca0679bb376966866afb849322c8aec79f vs revision b7490575c16b87d861a532b03bd1b2d076b6fa0b
extend-project name platform/system/sepolicy:
  base c2652856e9f0e0433d78cfb481efe392d6f5dbc4 vs revision cf9a73f92f503ffc4bd3e726164babfb40a9fcaf

Test: repo manifest > /dev/null && echo 'Nice =)'
      on various good and bad test manifests
Change-Id: I8d741e11c69ef9e4f8cf09e32aa0868b6daa5acd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/619921
Tested-by: Fredrik de Groot <fredrik.de.groot@haleytek.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Fredrik de Groot <fredrik.de.groot@haleytek.com>
2026-09-10 23:17:12 -07:00
Arif Kasim fe2c23f8e2 upload: bypass branch editor with explicit branch and -y
When uploading across multiple projects with --yes and an explicit
branch specified (via --current-branch / -c or --branch / --br),
repo previously still opened the interactive branch selection editor.

If an explicit branch and --yes are provided, bypass the interactive
editor and upload all matching pending branches directly.

Bug: 553024110
Test: python3 -m pytest tests/
Test: flake8 subcmds/upload.py tests/test_subcmds_upload.py
Test: isort --check subcmds/upload.py tests/test_subcmds_upload.py
Change-Id: I104619603f6e0539d05b01d3eb0292a9145d58db
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622661
Tested-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-09 12:10:24 -07:00
Rahul Yadav d27d6829a8 project: check upstream ref ancestry for non-shallow clones
In _CheckForImmutableRevision, commit d9cc0a15 restricted upstream ref
validation strictly to superprojects (if use_superproject) to prevent
shallow clones with an upstream attribute from failing ancestry checks
and falling back to full clones.

However, restricting this check exclusively to superprojects broke
non-shallow projects with pinned immutable revisions (such as in Smart
Sync, `repo sync -t <BUILD_ID>`, or pinned manifests): When the target
commit already exists in the local Git object store (for instance,
prefetched into refs/prefetch/ by a background daemon or via shared
object dirs), _CheckForImmutableRevision returned True without verifying
that the local tracking ref (refs/remotes/<remote>/<upstream>) is
present and reaches the revision. As a result, _RemoteFetch skipped
fetching the upstream branch, leaving the local tracking ref stale or
missing. Subsequent `repo start` branches tracking that remote branch
diverged, causing `repo upload` to attempt uploading all intermediate
commits between the stale tracking ref and HEAD.

Restore upstream ancestry validation in _CheckForImmutableRevision for
non-shallow projects when upstream is specified. Also pass the sync
depth into _CheckForImmutableRevision call sites so shallow checkouts
continue to skip upstream verification and avoid triggering un-shallow
fallbacks.

Test: PYTHONPATH=. pytest tests/test_project.py
Change-Id: Ib5bdf41810bb06c8ed053447209ddc2e488f3913
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/624361
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-03 02:15:23 -07:00
Gavin Mak 0ea57e2eed project: gate the default-branch query by Git version
GIT_DEFAULT_BRANCH support was added to git var in Git 2.35. Skip that
guaranteed failure on older supported clients and read
init.defaultBranch directly, reducing their fallback from two processes
to one.

Bug: 553599402
Change-Id: I079e9e6057b56afa41b685317a106fc9cdbad8a2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623907
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-02 17:46:23 -07:00
Gavin Mak ba8ddf396c git_superproject: resolve one negotiation ref directly
The superproject only needs one branch tip or HEAD OID. Resolve that ref
with one quiet rev-parse instead of constructing GitRefs, which loads
the entire namespace and resolves HEAD in two or three processes on
older clients.

Bug: 553599402
Change-Id: I162d1133ebd1941e9de5aca43929fd5af17386ef
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623906
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:46:01 -07:00
Gavin Mak d88ce8d952 upload: reuse the worker current-branch result
Carry the current branch through _GatherOne so the parent can report a
failed current-branch upload without resolving HEAD a second time.

Bug: 553599402
Change-Id: I3fc54be6c10c30470e14f792ff68c0a396519327
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623905
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-02 17:45:55 -07:00
Gavin Mak 5e8d2a6e3a version: read describe and date in one Git call
Git 2.32 added the %(describe) pretty placeholder. Use one log format to
retrieve both repo's describe string and commit date on newer clients,
with the existing describe-plus-log path retained for older Git and
untagged output.

Bug: 553599402
Change-Id: I108346030677b2b90ef8bfb5acea1c45be2c85f2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623904
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-02 17:45:49 -07:00
Gavin Mak e59c9cde99 cherry_pick: resolve and read commits in one process
Send a typed revision expression to cat-file --batch over stdin and
parse its object header and exact byte length. This safely resolves the
commit OID and retrieves its raw message in one Git process instead of
separate rev-parse and cat-file calls.

Bug: 553599402
Change-Id: I764894323ec28e134f2523bd1d5694738b6b52d0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623903
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:43:55 -07:00
Gavin Mak 83428a9b26 rebase: delegate automatic stashing to Git
Git --autostash predates repo's minimum supported client. Pass it
directly to rebase instead of probing the index and separately running
stash and stash pop, reducing the dirty path from four Git processes to
one and correctly covering staged-only changes.

Bug: 553599402
Change-Id: I0d6d59b17996576310498e9378e49380d355236e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623902
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:43:48 -07:00
Gavin Mak 948abc85bc git_config: recognize only complete object IDs
Full Git object IDs are exactly 40 hexadecimal digits for SHA-1 or 64
for SHA-256. Stop treating every intermediate length as immutable, which
could bypass normal ref resolution for invalid revision strings.

Bug: 553599402
Change-Id: I41b4ce6b2bfe2a2d8b351c7f040ec0cf469b474b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623901
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:41:05 -07:00
kimhappy c63a2f92fa sync: tell apart same-path submanifest projects
Interleaved sync remembers which projects are done and which ones are
still pending by Project.relpath, and lists failing projects by it as
well. That path is relative to the project's own (sub)manifest though,
so once several manifests are synced together two projects can share
it, and the sets no longer tell them apart.

The damage is done when a project turns up in a later pass, e.g. a
submodule that is only derived once its parent has been synced: its
path is already recorded as finished, so it is left out of every pass
that follows and never synced, while sync still reports success. With
an outer manifest holding

  <project name="a" path="a" sync-s="true"/>

where a carries a submodule at b, and a submanifest at sub/ holding

  <project name="sub-ab" path="a/b"/>

a fresh `repo sync` checks out sub/a/b but never a/b. Failing projects
are reported under the same ambiguous path, and the stall detection
merges them too.

Use RelPath(local=opt.this_manifest_only) instead, which is unique
within the set of projects being synced and is what GetProjects() and
the other subcommands already use.

Spotted during the review of I30395b8a16a9154f60972e1f408a066af64ba77c.

Change-Id: I5fa694b968774667be6060d9722cb57acbd2b579
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622641
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-01 19:13:52 -07:00
Gavin Mak 0039e39000 project: carry current state from the ref snapshot
Attach the current marker computed by GetBranches to uploadable and
prunable branches. Overview and prune reporting can then reuse that
result instead of resolving HEAD once per displayed branch.

Bug: 553599402
Change-Id: Ie3e23d08d0caf66c47cc12d7295ec4b6a834560c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623184
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 13:26:22 -07:00
Gavin Mak 5f378458d2 project: reuse ref OIDs in branch maintenance
Load refs before resolving the current branch, compare cached HEAD and
manifest OIDs instead of walking their symmetric difference, and reuse
the snapshot when preserving bare HEAD. Apply the same HEAD reuse to
abandon and MetaProject checks, and bound the latter revision walk to
one commit.

Bug: 553599402
Change-Id: I715e65dcfa02c0df4aaa08da5b7bfbb04ca17725
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623183
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 13:25:40 -07:00
Gavin Mak d27034bf62 project: centralize safe branch and commit resolution
Resolve commit-ish values through one typed helper using rev-parse
--verify --quiet and, on Git 2.30+, --end-of-options. Reject option-like
revisions on older clients, reuse the helper for project and manifest
resolution, and validate user branch names with check-ref-format
--branch.

Bug: 553599402
Change-Id: I2dda49ff3e781cec14d3b84263a8dace06b4073f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623182
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-31 12:34:53 -07:00
Gavin Mak 6541729a18 project: reuse loaded refs when resolving HEAD
When the regular checkout has already loaded GitRefs, derive its current
branch and HEAD object from that snapshot instead of spawning
symbolic-ref or rev-parse again. Keep linked Git worktrees on their
worktree-specific HEAD path because the shared repository HEAD is not
authoritative for them.

Bug: 543851900
Bug: 553599402
Change-Id: Iec456be5f3c279865b4165a75d1c6df36e331dbc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623181
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 12:33:39 -07:00
Gavin Mak b85e76a86a git_refs: load HEAD with the ref snapshot on Git 2.45
Use for-each-ref --include-root-refs to resolve attached and detached
HEAD in the same process as refs/* on Git 2.45 and newer. Limit the
patterns to HEAD and refs so volatile pseudo-refs do not enter the
cache, and retain the existing fallback for older Git and unborn HEADs.

Bug: 543851900
Bug: 553599402
Change-Id: Ie091ca0757f21ec6b1e19d49e09d160e055ec639
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622981
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-08-31 11:57:39 -07:00
James Hawkins e5bbb5c9e6 upload: prune ref scans for untouched projects and specific branches
Benchmark (100 untouched projects):
* GetUploadableBranches: 0.985s -> 0.0004s (2,200x faster)
* repo upload (3,045 projects): saves ~2.5s of redundant ref scans

During `repo upload`, `GetUploadableBranches()` currently scans
`self._allrefs` across all projects in the workspace. In large
manifests (e.g. Android with 3,000+ projects), `self._allrefs`
executes `git for-each-ref`, `git symbolic-ref`, and full filesystem
mtime traversals on thousands of projects where no local branch was
ever started or modified.

Any branch that can be uploaded for review must have upstream tracking
configured via `[branch "..."]` sections in `.git/config`
(otherwise `branch.LocalMerge` is None and `GetUploadableBranch`
returns None).

This patch introduces early-pruning fast paths:
1. When no branch subsections exist in `.git/config`
   (or only 2-part keys like `branch.autosetupmerge`), return []
   immediately without touching `_allrefs`.
2. When a `selected_branch` is specified (e.g. `repo upload --br=...`),
   query `branch.LocalMerge`, `refs/heads/<branch>`, and
   `refs/published/<branch>` directly, skipping the whole-tree
   ref scan.
3. In `subcmds/upload.py`, optimize `_GetMergeBranch` to resolve the
   merge branch in-memory via `project.CurrentBranch` and
   `project.GetBranch()` instead of spawning two `GitCommand`
   processes.

Test: ./run_tests tests/test_project.py tests/test_subcmds_upload.py
Change-Id: I3c2c9d63a68a393d478e1fb50f548ec8b7d44639
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623021
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: James Hawkins <jhawkins@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
2026-08-28 16:07:01 -07:00
James Hawkins 4fe87617ff project: read HEAD directly in-memory to avoid subprocesses
Benchmark:
* repo upload frameworks/base: 2.84s -> 0.50s (-82.3%, 5.6x faster)
* repo upload (3,045 projects): 7.66s -> 5.24s (-31.6%, 2.42s saved)

From repo's inception through v2.56, GetHead() directly read the
`.git/HEAD` file in Python. In commit 52bab0ba ("project: Use git
rev-parse to read HEAD"), this was replaced with git subprocess calls
on the premise that git provides a dedicated command. However, in
large multi-project workspaces (such as Android with 3,000+ projects),
spawning thousands of git processes introduced severe latency
regressions during `repo upload` and `repo status`.

Furthermore, switching to subprocesses broke detached HEADs and
unborn branches (fixed in commits 7f7d70ef and 8c3585f3 by re-adding
the v2.56 file-reading logic as an error recovery fallback).

This patch restores fast in-memory reading as the primary path, while
adding modern defensive safeguards:
* Symbolic refs (`ref: refs/heads/...`): Strips whitespace and tabs
  and returns the ref directly in memory.
* Detached HEAD: Validates 40-char SHA-1 and 64-char SHA-256 commit
  hashes via git_config.IsId(), normalizing to lowercase.
* Symlinks: Detects filesystem symlinks via os.path.islink() and
  safely falls back to git symbolic-ref.
* Fallback: Catches (OSError, AssertionError) and transparently falls
  back to native git commands for reftables, unexpected layouts, or
  filesystem errors. Unifies recovery fallback parsing with the fast
  path (CRLF/tabs, lowercase hashes, and consistent RelPath errors).

In addition, this change substantially expands test coverage in
tests/test_project.py, adding comprehensive unit tests for symbolic
refs, whitespace/tabs, CRLF line endings, SHA-1, SHA-256, uppercase
hash normalization, symlinks, corrupted worktrees, and fallback
robustness.

Test: ./run_tests tests/test_project.py
Change-Id: Ib5c2530117c6939e4b9293feda81aa745c003c6a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623001
Commit-Queue: James Hawkins <jhawkins@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
2026-08-28 16:03:06 -07:00
kimhappy d7299422ae sync: skip submodules removed from their parent
The submodules to sync are derived from their parent before it is
fetched, so one that the parent's new revision no longer holds a gitlink
for is still synced. Interleaved sync then fails to check it out,
because `git submodule init` no longer knows that path:

  error: Cannot checkout a/c
  error: pathspec '.../a/c' did not match any file(s) known to git

Report such submodules while the gitlinks are read again, and leave them
out of the fetch and of the checkout, so the usual project list update
removes them from the working tree. Phased sync already left them out of
the checkout, but still fetched them.

Nothing is reported as removed while the gitlinks of a parent cannot be
read, since a missing gitlink cannot be told apart from a fetch that did
not happen.

Bug: 550074864
Change-Id: Ia429b35c12758a68e4df73665e5512a201af658a
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621681
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:54 -07:00
kimhappy 3f087a8dd9 sync: fetch submodules after their parent repository
Phased sync fetches every project at once, so a submodule is fetched at
the gitlink that was read before its parent was fetched. With -c that
stale revision is already present, the fetch is skipped as an immutable
revision, and the checkout then fails on the revision the reloaded
manifest resolved:

  error: Cannot checkout a/b
  fatal: bad object 5556dad99da5df7e53303c5732aacc8782493e03

Fetch in batches, holding a submodule back until the project holding its
gitlink is fetched, and read the gitlinks again between batches. A
manifest without submodules keeps its single batch.

Bug: 550074864
Change-Id: I04c2eb1ca81ab25b4df2c03c5dc048d2e4ecdcb8
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621663
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:49 -07:00
kimhappy 3a6e25af75 sync: check out submodules at the parent's new revision
Interleaved sync processes projects in hierarchical levels, so a
submodule is only handled after the project holding its gitlink has been
fetched and checked out. Its revision, however, is still the one read
before that parent was fetched, so the submodule is synced to the
revision of the previous sync and only catches up on the next one.

Read the gitlinks again right before a level is dispatched. The parent
is done by then, so its submodules are fetched and checked out at the
revision the parent now points at. Sibling submodules are spread over
several levels, so the gitlinks read for one level are carried over to
the next ones and every project is only read once per pass.

Bug: 550074864
Change-Id: I30395b8a16a9154f60972e1f408a066af64ba77c
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621662
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:43 -07:00
Gavin Mak 41c2597509 project: don't use dest-branch or tags as SHA-1 upstream fallback
https://gerrit-review.googlesource.com/614762 resolves a fallback
upstream for a SHA-1-pinned project that has no explicit upstream, so
--current-branch can narrow the fetch to one branch instead of fetching
all heads.

It drew that fallback from dest-branch and from the manifest <default>
upstream, dest-branch, and revision. dest-branch and tags are not valid
fallback candidates:

* dest-branch is the review destination for upload, not a fetch
  source.
* Tags are meant to be immutable snapshots of a single tagged commit,
  and fetching it cannot retrieve an arbitrary pinned SHA-1.

Restrict the fallback to branch heads resolved from the manifest
upstream and revision defaults, and drop dest-branch entirely.

Bug: 541240657
Bug: 544041102
Change-Id: I89ae86891ee166f4c340553ac5a9068efcc18461
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622541
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 12:43:00 -07:00
Rahul Yadav e6ad708009 hooks: add --fix option to auto-apply hook fixes
Pass the --fix flag as a keyword argument "fix" to the hook main
function. This allows hooks (such as git-repohooks) to decouple
automated fix application from the -y/--yes flag so that -y can
answer yes to upload confirmation prompts without triggering file
mutations.

Companion change in git-repohooks:
https://gerrit-review.googlesource.com/c/git-repohooks/+/621761

Bug: 546510319
Test: python3 -m pytest tests/test_hooks.py

Change-Id: If0288d4791fc0a2aba6e88854e3aa81b0923b664
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/619281
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-08-27 03:28:58 -07:00
Ram Peri 09914bcab7 Add detailed upload context to Trace2 telemetry
This patch intercepts the output during a successful upload
execution to capture the generated CL URLs. It logs a dynamically
constructed "repo.uploadstate" data event to the active trace2
log containing:
- Uploaded CL URLs
- Target remote name
- Source branch
- Modified files

Test:
1. ./run_tests
2. pytest tests/test_project.py
3. Manual verification:
- Created a dummy branch `test_upload_branch` in a project with local file changes.
- Invoked repo upload passing an explicit trace output directory:
  `repo --git-trace2-event-log=/tmp/trace2out upload --dry-run --no-verify art`
- Verified the injected payload successfully appeared on disk within the generated log:
  ```json
  {"event":"data",..."key":"repo.uploadstate/cls","value":""}
  {"event":"data",..."key":"repo.uploadstate/remote","value":"goog"}
  {"event":"data",..."key":"repo.uploadstate/branch","value":"test_upload_branch"}
  {"event":"data",..."key":"repo.uploadstate/files","value":"dummy_file.txt"}
  ```
  (cls is correctly blank on --dry-run but populates on real HTTP pushes)

Bug: 543953499
Change-Id: I9c402a32d01d156d42cf24eaa2e60e22710b5e6f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/616581
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Ram Peri <ramperi@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
2026-08-26 10:19:42 -07:00
kimhappy 3f1775607f project: allow re-reading submodule gitlinks
A discovered submodule becomes a project whose revision is the gitlink
read from its parent at manifest load time, which is before the parent
has been fetched. Remember which gitlink a submodule was derived from,
and add a way to read those gitlinks again, so that callers can resolve
a submodule revision once its parent is up-to-date.

A revision that is set does not have to be fetched, so verify its
objects are really there. Without them a submodule that was removed
cannot be told apart from one that was never fetched, which the caller
has to know about.

Bug: 550074864
Change-Id: I4f35d53607126508eec6f02158a3fb604917a199
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621661
Reviewed-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-08-25 19:39:51 -07:00
66 changed files with 8577 additions and 760 deletions

No files matched your search

+13
View File
@@ -24,3 +24,16 @@ configs {
}
automatic: false
}
# Default Multi-Perspective Code Review Agent
configs {
id: "git-repo-code-review-agent"
display_name: "Code Review Agent"
description: "Default multi-perspective review agent for git-repo."
include_filters {
project: "git-repo"
path_regex: ".*"
}
use_default_subagents: true
tags: "code-review-agent"
}
+68 -17
View File
@@ -17,7 +17,7 @@ import multiprocessing
import optparse
import os
import re
from typing import TYPE_CHECKING
from typing import List, Optional, TYPE_CHECKING
from error import InvalidProjectGroupsError
from error import NoSuchProjectError
@@ -152,6 +152,20 @@ class Command:
self._Options(self._optparse)
return self._optparse
@staticmethod
def _GetHelpForCpuJobCount(
default_jobs: Optional[int] = None,
) -> str:
"""Return CPU-based job help, with an explicit default when needed.
Specify default_jobs when additional logic computes effective default.
"""
if GENERATE_MANPAGES:
return "based on number of CPU cores"
default = "%default" if default_jobs is None else str(default_jobs)
return f"{default}; based on number of CPU cores"
def _CommonOptions(self, p, opt_v=True):
"""Initialize the option parser with common options.
@@ -176,11 +190,7 @@ class Command:
)
if self.PARALLEL_JOBS is not None:
default = "based on number of CPU cores"
if not GENERATE_MANPAGES:
# Only include active cpu count if we aren't generating man
# pages.
default = f"%default; {default}"
default = self._GetHelpForCpuJobCount()
p.add_option(
"-j",
"--jobs",
@@ -398,7 +408,11 @@ class Command:
Args:
args: a list of (case-insensitive) strings, projects to search for.
manifest: an XmlManifest, the manifest to use, or None for default.
groups: a string, the manifest groups in use.
groups: a string, the manifest group selection to apply.
Non-empty values apply to all candidate projects in this call.
When empty or omitted, single-manifest calls use the selected
manifest's effective groups; all-manifest calls use each
candidate project's owning manifest's effective groups.
missing_ok: a boolean, whether to allow missing projects.
submodules_ok: whether to allow submodules. True allows them for
all projects, False disallows them for all projects, and None
@@ -425,9 +439,33 @@ class Command:
return project.sync_s
return submodules_ok
if not groups:
groups = manifest.GetManifestGroupsStr()
groups = [x for x in re.split(r"[,\s]+", groups) if x]
def parse_groups(value: str) -> List[str]:
return [x for x in re.split(r"[,\s]+", value) if x]
if groups:
groups_for_all_projects = parse_groups(groups)
elif all_manifests:
# In all-manifest mode, each project uses its owning
# manifest's effective groups.
groups_for_all_projects = None
else:
groups_for_all_projects = parse_groups(
manifest.GetManifestGroupsStr()
)
groups_by_manifest = {}
def matches_groups(project: "Project") -> bool:
if groups_for_all_projects is not None:
return project.MatchesGroups(groups_for_all_projects)
project_manifest = project.manifest
if project_manifest not in groups_by_manifest:
groups_by_manifest[project_manifest] = parse_groups(
project_manifest.GetManifestGroupsStr()
)
return project.MatchesGroups(groups_by_manifest[project_manifest])
if not args:
derived_projects = {}
@@ -439,9 +477,7 @@ class Command:
)
all_projects_list.extend(derived_projects.values())
for project in all_projects_list:
if (missing_ok or project.Exists) and project.MatchesGroups(
groups
):
if (missing_ok or project.Exists) and matches_groups(project):
result.append(project)
else:
self._ResetPathToProjectMap(all_projects_list)
@@ -455,7 +491,7 @@ class Command:
for project in manifest.GetProjectsWithName(
arg, all_manifests=all_manifests
)
if project.MatchesGroups(groups)
if matches_groups(project)
]
if not projects:
@@ -498,7 +534,7 @@ class Command:
"%s (%s)"
% (arg, project.RelPath(local=not all_manifests))
)
if not project.MatchesGroups(groups):
if not matches_groups(project):
raise InvalidProjectGroupsError(arg)
result.extend(projects)
@@ -509,7 +545,14 @@ class Command:
result.sort(key=_getpath)
return result
def FindProjects(self, args, inverse=False, all_manifests=False):
def FindProjects(
self,
args: List[str],
inverse: bool = False,
all_manifests: bool = False,
groups: Optional[str] = "",
missing_ok: Optional[bool] = False,
) -> List["Project"]:
"""Find projects from command line arguments.
Args:
@@ -519,10 +562,18 @@ class Command:
all_manifests: a boolean, if True then all manifests and
submanifests are used. If False, then only the local
(sub)manifest is used.
groups: a string specifying manifest groups. If empty or None, use
each manifest's effective groups.
missing_ok: a boolean, whether to allow missing projects.
"""
result = []
patterns = [re.compile(r"%s" % a, re.IGNORECASE) for a in args]
for project in self.GetProjects("", all_manifests=all_manifests):
for project in self.GetProjects(
"",
groups=groups,
missing_ok=missing_ok,
all_manifests=all_manifests,
):
paths = [project.name, project.RelPath(local=not all_manifests)]
for pattern in patterns:
match = any(pattern.search(x) for x in paths)
+2
View File
@@ -337,6 +337,7 @@ _repo() {
'--auto-gc[Run auto gc]' \
'--no-auto-gc[Do not run auto gc]' \
'(-s --smart-sync)'{-s,--smart-sync}'[Smart sync]' \
'--no-smart-sync[Do not smart sync]' \
'(-t --smart-tag)'{-t,--smart-tag=}'[Smart tag]:tag:' \
'--no-repo-verify[Do not verify repo]' \
'--no-verify[Do not verify]' \
@@ -400,6 +401,7 @@ _repo() {
'--no-verify[Do not verify]' \
'--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \
'--fix[Automatically fix]' \
'*: :->project'
;;
version)
+21 -14
View File
@@ -6,6 +6,9 @@ executed during `repo sync` to fetch objects, instead of using standard
filesystems or lazy checkouts where fetching metadata and downloading file
contents should be decoupled.
The checkout half of a sync has a counterpart, `repo.reprojectcmd`; see
`docs/reproject-cmd.md`.
## Configuration
To use this feature, set the following in `.repo/manifests.git/config`:
@@ -25,7 +28,10 @@ Environment section in `repo help forall` or `subcmds/forall.py`.
The following environment variable is specific to `repo.fetchcmd`:
* `REPO_TREV`: The target revision resolved to a full commit hash.
* `REPO_TREV`: The target commit hash when the revision is already pinned to
a commit (e.g. via a superproject or a commit SHA in the manifest), or
empty when `REPO_RREV` is an unpinned branch or tag to be resolved by the
fetch command.
## Contract
@@ -34,22 +40,23 @@ The following environment variable is specific to `repo.fetchcmd`:
After the fetch command exits with status 0, `repo` expects the following
postconditions to be met:
1. `git cat-file -e REPO_TREV` succeeds (the commit must exist in the object
store).
2. The mapped local tracking ref (e.g. `refs/remotes/REPO_REMOTE/<branch>`
for a branch revision, or the tag ref itself for a tag) must point to
`REPO_TREV`.
3. `FETCH_HEAD` must point to `REPO_TREV`.
4. The commit graph from `REPO_TREV` must be reachable far enough to compute
merge bases with local branches.
1. When the manifest revision names a branch or tag, the mapped local tracking
ref (e.g. `refs/remotes/REPO_REMOTE/<branch>` for a branch, or
`refs/tags/<tag>` for a tag) must resolve to a valid commit in the object
store (`git rev-parse --verify <ref>^{commit}`), and must equal `REPO_TREV`
if `REPO_TREV` was non-empty.
2. `FETCH_HEAD` must resolve to the same target commit in the object store
(`git rev-parse --verify FETCH_HEAD^{commit}`).
3. The commit graph from the target commit must be reachable far enough to
compute merge bases with local branches.
### Invariants
* The command should be idempotent; fetching the same `REPO_TREV` twice should
be a no-op.
* Only `FETCH_HEAD` and `refs/remotes/*` should be modified to preserve
`repo sync --network-only` semantics. `HEAD` and local branches must not be
touched by the fetch command.
* The command should be idempotent; repeating the same fetch should be a
no-op.
* Only `FETCH_HEAD`, `refs/remotes/*`, and `refs/tags/*` should be modified to
preserve `repo sync --network-only` semantics. `HEAD` and local branches
must not be touched by the fetch command.
* Dirty worktree state must be preserved.
* The command is **not** executed for `MetaProject`s (i.e. the internal `repo`
repository itself at `.repo/repo` and the `manifests` repository at
+13 -10
View File
@@ -46,18 +46,19 @@ following DTD:
<!ATTLIST remote revision CDATA #IMPLIED>
<!ELEMENT default EMPTY>
<!ATTLIST default remote IDREF #IMPLIED>
<!ATTLIST default revision CDATA #IMPLIED>
<!ATTLIST default dest-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync-j CDATA #IMPLIED>
<!ATTLIST default sync-j-max CDATA #IMPLIED>
<!ATTLIST default sync-c CDATA #IMPLIED>
<!ATTLIST default sync-s CDATA #IMPLIED>
<!ATTLIST default sync-tags CDATA #IMPLIED>
<!ATTLIST default remote IDREF #IMPLIED>
<!ATTLIST default revision CDATA #IMPLIED>
<!ATTLIST default dest-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync-j CDATA #IMPLIED>
<!ATTLIST default sync-j-max CDATA #IMPLIED>
<!ATTLIST default sync-c CDATA #IMPLIED>
<!ATTLIST default sync-s CDATA #IMPLIED>
<!ATTLIST default sync-tags CDATA #IMPLIED>
<!ATTLIST default sync-smartsync CDATA #IMPLIED>
<!ELEMENT manifest-server EMPTY>
<!ATTLIST manifest-server url CDATA #REQUIRED>
<!ATTLIST manifest-server url CDATA #REQUIRED>
<!ATTLIST manifest-server helper CDATA #IMPLIED>
<!ELEMENT submanifest EMPTY>
@@ -231,6 +232,8 @@ Attribute `sync-tags`: Set to false to only sync the given Git
branch (specified in the `revision` attribute) rather than
the other ref tags.
Attribute `sync-smartsync`: Set to true to enable smart sync by default.
### Element manifest-server
+10 -2
View File
@@ -88,7 +88,14 @@ be useful when deploying automatic fixes.
If the repo command that triggered the hook supports a "yes" option (e.g.,
`repo upload --yes`), this option is propagated to the hook's `main` function
as `yes` parameter (defaulting to `False`). Hooks can use this to bypass
interactive confirmation prompts when they can automatically fix issues.
interactive confirmation prompts for safe non-modifying operations.
### Automated Fixes
If the repo command that triggered the hook supports a "fix" option (e.g.,
`repo upload --fix`), this option is propagated to the hook's `main` function
as `fix` parameter (defaulting to `False`). Hooks can use this to automatically
apply fixes without prompting the user.
### Shebang Handling
@@ -126,7 +133,7 @@ This hook runs when people run `repo upload`.
The `pre-upload.py` file should be defined like:
```py
def main(project_list, worktree_list=None, yes=False, **kwargs):
def main(project_list, worktree_list=None, fix=False, yes=False, **kwargs):
"""Main function invoked directly by repo.
We must use the name "main" as that is what repo requires.
@@ -137,6 +144,7 @@ def main(project_list, worktree_list=None, yes=False, **kwargs):
project_list, so that each entry in project_list matches with a
directory in worktree_list. If None, we will attempt to calculate
the directories automatically.
fix: Whether to automatically apply fixes without prompting.
yes: Whether to answer yes to all safe prompts (see
[Safe Prompts](#safe-prompts)).
kwargs: Leave this here for forward-compatibility.
+126
View File
@@ -0,0 +1,126 @@
# Reproject Command Contract
The `repo.reprojectcmd` configuration names a command that `repo sync` runs
instead of Git to move a project's index and worktree to the tree of the
target commit. It is the checkout-side counterpart of `repo.fetchcmd` (see
`docs/fetch-cmd.md`): together they let an external tool take over both the
network fetch and the materialization of a project. This is useful on
virtualized filesystems that address content by hash, where a tree can be
materialized far faster than `git checkout` can write every file.
The command only materializes the tree. `repo` then makes the ref write that
Git would have made, using `git update-ref`.
## Configuration
To use this feature, set the following in `.repo/manifests.git/config`:
```ini
[repo]
reprojectcmd = "your custom command here"
uselocalgitdirs = true
```
Setting `repo.reprojectcmd` **requires** `repo.uselocalgitdirs` to be set to
`true`.
For reference, this command does with Git what `repo` would otherwise do
itself:
```ini
[repo]
reprojectcmd = "git -C $REPO_PATH read-tree -m -u $REPO_TREV"
uselocalgitdirs = true
```
The one-tree merge applies the change to the target, keeps local changes to
every other path, and refuses to overwrite a modified or untracked file, so it
enforces the preconditions below by itself. It also works for a project that
has nothing checked out yet.
## Environment Variables
The command is executed in a subshell, from the root of the client, populated
with standard project-context environment variables. For details on standard
variables (such as `REPO_PROJECT`, `REPO_PATH`, `REPO_REMOTE`, etc.), see the
Environment section in `repo help forall` or `subcmds/forall.py`.
The variables the command typically needs are:
* `REPO_PATH`: The project path relative to the root of the client.
* `REPO_TREV`: The target revision resolved to a full commit hash. Match this
commit's tree.
There is no force mode: a project that would need one never reaches the
command (see the preconditions below).
## When the command runs
`repo sync` already classifies each project and picks a Git operation. The
command replaces the three that are a materialization of a target tree:
1. The checkout that detaches HEAD at the target. This is the common case: a
project on a detached HEAD, a project on a branch that does not track
upstream, and `repo sync -d`.
2. The fast-forward of the checked out branch to the target.
3. The hard reset of the checked out branch to the target, when the commits
it carried were dropped upstream.
After the command exits 0, `repo` writes the ref itself: it detaches `HEAD` at
`REPO_TREV`, or moves the checked out branch to `REPO_TREV`.
The command is **not** run:
* When `HEAD` already names `REPO_TREV`.
* At the fast-forward step when `HEAD` is ahead of `REPO_TREV`, where Git's
merge would be a no-op.
* For a rebase. A branch carrying local commits has them replayed onto the
target by `git rebase`, which is not a materialization of a target tree.
* For `MetaProject`s (i.e. the internal `repo` repository itself at
`.repo/repo` and the `manifests` repository at `.repo/manifests`).
## Contract
### Preconditions
Before invoking the command, `repo` ensures that:
* The index has no staged changes (the index matches `HEAD`, or is empty on an unborn `HEAD`).
* No rebase, cherry-pick, merge, or revert is in progress.
Detecting collisions with untracked files or unstaged working-tree modifications is the responsibility of the reproject command itself (e.g. via `git read-tree -m -u $REPO_TREV` or a custom virtual filesystem checkout tool). If local changes collide with the target tree, the command must abort with a non-zero exit code. Local modifications and untracked files outside the diff between `HEAD` and `REPO_TREV` must be preserved.
### Postconditions on exit 0
After the command exits with status 0, `repo` expects the following
postconditions to be met:
1. `git diff-index --quiet --cached REPO_TREV^{tree}` exits 0 (the index
matches the target tree).
2. `HEAD` still names what it did before the command, and its resolved commit
object ID has not changed.
### Invariants
The command may modify the worktree and the index, and may write project-local
Git config. The command must:
* Apply the change from `HEAD`'s tree to `REPO_TREV`'s tree and leave every
other path alone. Local modifications and untracked files outside that
change must survive: the command applies a diff, it does not reset the
tree.
* Not write any ref, including `HEAD` and `ORIG_HEAD`. `repo` owns every ref
write.
* Not create or replace `.git/`, and not touch anything under `.repo/`.
* Not require the Git remote, to preserve `repo sync --local-only`.
* Be idempotent. Running it twice on the same target is a no-op.
### Failure
* A non-zero exit status, a failed precondition or a failed postcondition
fails that project's sync, and the command's or Git's output is surfaced
to the user.
* Other projects continue, and `repo sync` exits non-zero.
## Limitations
Nested projects are out of scope: a project whose path lies inside another
project's path, a `<project>` nested in another `<project>` in the manifest,
and a submodule discovered with `sync-s` or `--recurse-submodules`. `repo sync`
fails if the manifest has one while `repo.reprojectcmd` is set.
+80 -4
View File
@@ -95,6 +95,36 @@ example would do. Google's internal server uses Python's
The manifest server should implement the following RPC methods.
### GetApprovedManifestV2
> `GetApprovedManifestV2(request: dict) -> dict`
Like [GetApprovedManifest](#GetApprovedManifest), but takes a single struct of
named options so new options can be added without changing the method signature.
Unset options are omitted from the struct. Servers should ignore options they do
not understand.
Request options:
* `branch` (`str`, required): Same as in `GetApprovedManifest`.
* `target` (`str`, optional): Same as in `GetApprovedManifest`.
* `best_effort` (`bool`, optional, default `False`): The client can proceed
without a manifest, so the server may decline the request (e.g. during a
staged rollout or to shed load). Repo sets this for Smart Syncs enabled by
the manifest's `sync-smartsync` default, and falls back to a regular ToT
sync if the request is declined. Servers must not decline requests without
this option for these reasons.
Response fields:
* `success` (`bool`): Whether a manifest was returned.
* `manifest_xml` (`str`): The manifest. Required if `success` is true.
* `error_message` (`str`): Why the request failed or was declined. Repo shows
it to the user like `GetApprovedManifest` errors.
If the server does not implement this method, repo calls `GetApprovedManifest`
instead and drops the options it cannot express.
### GetApprovedManifest
> `GetApprovedManifest(branch: str, target: Optional[str]) -> str`
@@ -138,12 +168,53 @@ copy into `.repo/manifests/smart_sync_override.xml` so users can examine it.
The next time `repo sync` is run, this file is automatically replaced or removed
based on the current set of options.
### --smart-sync
With `-l`/`--local-only` and no explicit `-s`/`--smart-sync` or
`-t`/`--smart-tag`, repo does not contact the manifest server. Instead, it keeps
using the existing `smart_sync_override.xml`, if there is one, whether it came
from an explicit `-s`/`--smart-sync` or from the manifest's `sync-smartsync`
default. This way, `repo sync -n` followed by `repo sync -l` ends up on the same
revisions as the equivalent single `repo sync` (e.g. `repo sync -s -n` +
`repo sync -l` matches `repo sync -s`). Pass `--no-smart-sync` to discard it and
use the default manifest instead.
Repo will call `GetApprovedManifest(branch[, target])`.
`--no-manifest-update` only skips updating the manifest checkout. It does not
affect Smart Sync: the default Smart Sync from `sync-smartsync` (see below)
still runs, and an existing `smart_sync_override.xml` is replaced or removed as
in a regular sync.
### sync-smartsync (Manifest Attribute)
The manifest can enable Smart Sync by default for all `repo sync` invocations
by setting `sync-smartsync="true"` on the `<default>` element:
```xml
<default sync-smartsync="true" ... />
```
When enabled in the manifest, `repo sync` will automatically perform a Smart
Sync unless explicitly overridden on the command line. Since the user did not
ask for a Smart Sync, any failure to obtain a manifest from the manifest server
(e.g. the server is unreachable, or it returns a manifest that cannot be parsed)
is silently ignored and `repo sync` falls back to a regular ToT sync; pass
`--verbose` to see the reason. Repo marks these requests as `best_effort` (see
[GetApprovedManifestV2](#GetApprovedManifestV2)), so the server may also decline
them, e.g. during a staged rollout. This is only a fallback for the
manifest-driven default: an explicit `-s`/`--smart-sync` or `-t`/`--smart-tag`
still fails the sync, as before. Options that specify an explicit target or
manifest source (such as `-t`/`--smart-tag`, `-m`/`--manifest-name` or
`--superproject-revision`) will also disable the default Smart Sync behavior.
### --smart-sync / -s
Explicitly enables Smart Sync. Repo will call
`GetApprovedManifestV2({branch[, target]})`, or
`GetApprovedManifest(branch[, target])` if the server does not implement it.
The `branch` is determined by the current manifest branch as specified by
`--manifest-branch=BRANCH` when running `repo init`.
`--manifest-branch=BRANCH` when running `repo init`. If `${SYNC_BRANCH}` is
defined and non-empty, its value is sent instead. This allows selecting a
branch other than the one the checkout tracks, e.g. a server-side (CI) branch
name that builds from the same manifest branch.
The `target` is defined by environment variables in the order below. If none
of them match, then `target` is omitted. These variables were decided as they
@@ -155,6 +226,11 @@ match the settings Android build environments automatically setup.
3. `${TARGET_PRODUCT}-${TARGET_BUILD_VARIANT}`: If these variables are all
defined, then they are merged with `-` and used.
### --smart-tag=TAG
### --no-smart-sync
Explicitly disables Smart Sync, overriding any `sync-smartsync="true"` setting
declared in the manifest `<default>` element.
### --smart-tag=TAG / -t
Repo will call `GetManifest(TAG)`.
+56 -8
View File
@@ -70,6 +70,19 @@ class _GitCall:
git = _GitCall()
def IsValidBranchName(name: str) -> bool:
"""Return whether |name| is valid where Git expects a branch name."""
p = GitCommand(
None,
["check-ref-format", "--branch", name],
capture_stdout=True,
capture_stderr=True,
add_event_log=False,
log_as_error=False,
)
return p.Wait() == 0
def RepoSourceVersion():
"""Return the version of the repo.git tree."""
ver = getattr(RepoSourceVersion, "version", None)
@@ -270,6 +283,7 @@ class GitCommand:
bare=False,
input=None,
capture_stdout=False,
capture_stdout_bytes: bool = False,
capture_stderr=False,
merge_output=False,
disable_editor=False,
@@ -291,6 +305,12 @@ class GitCommand:
self.cmdv = cmdv
self.verify_command = verify_command
self.stdout, self.stderr = None, None
if capture_stdout_bytes:
if merge_output:
raise ValueError(
"capture_stdout_bytes cannot be combined with merge_output"
)
capture_stdout = True
# Git on Windows wants its paths only using / for reliability.
if platform_utils.isWindows():
@@ -334,6 +354,7 @@ class GitCommand:
command,
env,
capture_stdout=capture_stdout,
capture_stdout_bytes=capture_stdout_bytes,
capture_stderr=capture_stderr,
merge_output=merge_output,
ssh_proxy=ssh_proxy,
@@ -367,6 +388,7 @@ class GitCommand:
command,
env,
capture_stdout=False,
capture_stdout_bytes: bool = False,
capture_stderr=False,
merge_output=False,
ssh_proxy=None,
@@ -399,6 +421,10 @@ class GitCommand:
# See go/tee-repo-stderr for more context.
tee_stderr = False
kwargs = {"encoding": "utf-8", "errors": "backslashreplace"}
if capture_stdout_bytes:
kwargs = {}
if isinstance(input, str):
input = input.encode("utf-8", "surrogateescape")
if not (stdin or stdout or stderr):
tee_stderr = True
# stderr will be written back to sys.stderr even though it is
@@ -477,6 +503,10 @@ class GitCommand:
self.stderr = self._Tee(p.stderr, sys.stderr)
else:
self.stdout, self.stderr = p.communicate(input=input)
if capture_stdout_bytes and isinstance(self.stderr, bytes):
self.stderr = self.stderr.decode(
"utf-8", "backslashreplace"
).replace("\r\n", "\n")
finally:
if ssh_proxy:
ssh_proxy.remove_client(p)
@@ -528,17 +558,35 @@ class GitCommand:
env.pop(key, None)
return env
def VerifyCommand(self):
def VerifyCommand(self) -> None:
if self.rc == 0:
return None
stdout = (
"\n".join(self.stdout.split("\n")[:GIT_ERROR_STDOUT_LINES])
if self.stdout
else None
)
raw_stdout = self.stdout
if isinstance(raw_stdout, bytes):
first_records = re.split(
rb"\r\n|[\r\n\0]", raw_stdout, maxsplit=GIT_ERROR_STDOUT_LINES
)[:GIT_ERROR_STDOUT_LINES]
stdout = (
"\n".join(
r.decode("utf-8", "backslashreplace") for r in first_records
)
if raw_stdout
else None
)
elif raw_stdout:
first_records = re.split(
r"\r\n|[\r\n\0]", raw_stdout, maxsplit=GIT_ERROR_STDOUT_LINES
)[:GIT_ERROR_STDOUT_LINES]
stdout = "\n".join(first_records)
else:
stdout = None
raw_stderr = self.stderr
if isinstance(raw_stderr, bytes):
raw_stderr = raw_stderr.decode("utf-8", "backslashreplace")
stderr = (
"\n".join(self.stderr.split("\n")[:GIT_ERROR_STDERR_LINES])
if self.stderr
"\n".join(raw_stderr.split("\n")[:GIT_ERROR_STDERR_LINES])
if raw_stderr
else None
)
project = self.project.name if self.project else None
+13 -1
View File
@@ -40,7 +40,7 @@ from repo_trace import Trace
# that is saved in the config.
SYNC_STATE_PREFIX = "repo.syncstate."
ID_RE = re.compile(r"^[0-9a-f]{40,64}$")
ID_RE = re.compile(r"^(?:[0-9a-f]{40}|[0-9a-f]{64})$")
REVIEW_CACHE = {}
@@ -266,6 +266,13 @@ class GitConfig:
except KeyError:
old = []
if self.file != self._SYSTEM_CONFIG:
# `git config --file` does not create missing parent directories
# (e.g. ~/.repoconfig/ on first use).
parent_dir = os.path.dirname(self.file)
if parent_dir:
os.makedirs(parent_dir, exist_ok=True)
if value is None:
if old:
del self._cache[key]
@@ -444,6 +451,11 @@ class GitConfig:
class RepoConfig(GitConfig):
"""User settings for repo itself."""
# Keep separate caches from GitConfig, otherwise ForUser()/ForSystem()
# would return the already-cached ~/.gitconfig instance of the parent.
_ForUser = None
_ForSystem = None
@staticmethod
def _getUserConfig():
repo_config_dir = os.getenv("REPO_CONFIG_DIR", os.path.expanduser("~"))
+37 -6
View File
@@ -14,6 +14,7 @@
import os
from git_command import git_require
from git_command import GitCommand
import platform_utils
from repo_trace import Trace
@@ -41,6 +42,17 @@ class GitRefs:
self._EnsureLoaded()
return self._phyref
@property
def head(self) -> str:
"""Return HEAD's symbolic target or detached object ID."""
self._EnsureLoaded()
return self._symref.get(HEAD) or self._phyref.get(HEAD, "")
@property
def is_loaded(self) -> bool:
"""Whether a ref snapshot has already been loaded."""
return self._phyref is not None
def get(self, name):
try:
return self.all[name]
@@ -87,8 +99,12 @@ class GitRefs:
self._symref = {}
self._mtime = {}
self._ReadRefs()
self._ReadSymbolicRef(HEAD)
root_refs_loaded = self._ReadRefs()
if not root_refs_loaded or (
HEAD not in self._phyref and HEAD not in self._symref
):
# --include-root-refs does not report an unborn HEAD.
self._ReadSymbolicRef(HEAD)
scan = self._symref
attempts = 0
@@ -113,18 +129,32 @@ class GitRefs:
"""Check if a ref_id is a null object ID."""
return ref_id and all(ch == "0" for ch in ref_id)
def _ReadRefs(self) -> None:
"""Read all references using git for-each-ref."""
def _ReadRefs(self) -> bool:
"""Read all references using git for-each-ref.
Returns:
Whether root refs, including HEAD when it exists, were loaded.
"""
include_root_refs = git_require((2, 45, 0))
cmd = [
"for-each-ref",
"--format=%(objectname)%00%(refname)%00%(symref)",
]
if include_root_refs:
cmd.insert(1, "--include-root-refs")
# Avoid caching volatile root refs such as ORIG_HEAD. HEAD and
# refs/* are the only namespaces GitRefs exposes to callers.
cmd.extend([HEAD, "refs"])
p = GitCommand(
None,
["for-each-ref", "--format=%(objectname)%00%(refname)%00%(symref)"],
cmd,
capture_stdout=True,
capture_stderr=True,
bare=True,
gitdir=self._gitdir,
)
if p.Wait() != 0:
return
return False
for line in p.stdout.splitlines():
ref_id, name, symref = line.split("\0")
@@ -132,6 +162,7 @@ class GitRefs:
self._symref[name] = symref
elif ref_id and not self._IsNullRef(ref_id):
self._phyref[name] = ref_id
return include_root_refs
def _ReadSymbolicRef(self, name: str) -> None:
"""Read a symbolic reference."""
+259
View File
@@ -0,0 +1,259 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Read a worktree's state from one machine-readable git status snapshot."""
from collections import OrderedDict
import os
from typing import Iterator, List, Optional, TYPE_CHECKING
from git_command import git_require
from git_command import GitCommand
if TYPE_CHECKING:
from project import Project
class StatusEntry:
"""The state of one path on one side of the index."""
def __init__(
self,
path: str,
status: str,
src_path: Optional[str] = None,
level: Optional[str] = None,
) -> None:
self.path = path
self.status = status
self.src_path = src_path
self.level = level
class StatusSnapshot:
"""A consistent view of worktree, index, and branch state."""
def __init__(self) -> None:
self.index_changes = OrderedDict()
self.worktree_changes = OrderedDict()
self.untracked = []
self.branch_oid = None
self.branch_head = None
self.upstream = None
self.ahead = 0
self.behind = 0
self.has_ahead_behind = False
self.stash_count = 0
@property
def current_branch(self) -> Optional[str]:
if self.branch_head in (None, "(detached)", "(unknown)"):
return None
return self.branch_head
def is_dirty(self, consider_untracked: bool = True) -> bool:
return bool(
self.index_changes
or self.worktree_changes
or (consider_untracked and self.untracked)
)
def GetStatus(
project: "Project",
gitdir: str,
untracked_files: str = "all",
branch: bool = False,
ahead_behind: bool = False,
show_stash: bool = False,
) -> StatusSnapshot:
"""Return one machine-readable status snapshot for |project|."""
if not git_require((2, 11, 0)):
raise UnsupportedStatusError("porcelain v2 requires Git 2.11")
cmd = [
"status",
"--porcelain=v2",
"-z",
"--ignore-submodules=all",
f"--untracked-files={untracked_files}",
]
if branch:
cmd.append("--branch")
if git_require((2, 17, 0)):
cmd.append(
"--ahead-behind" if ahead_behind else "--no-ahead-behind"
)
if git_require((2, 18, 0)):
# Match the existing staged diff's explicit rename detection even if
# status.renames is disabled in the user's config.
cmd.append("--renames")
if show_stash and git_require((2, 35, 0)):
cmd.append("--show-stash")
p = GitCommand(
project,
cmd,
bare=False,
gitdir=gitdir,
capture_stdout=True,
capture_stdout_bytes=True,
capture_stderr=True,
verify_command=True,
)
p.Wait()
return ParsePorcelainV2(p.stdout)
def _Path(value: bytes) -> str:
"""Decode a Git pathname without losing undecodable bytes."""
return os.fsdecode(value)
def _Status(value: int) -> str:
"""Normalize Git's unchanged markers for repo's status display."""
char = chr(value)
return "" if char == "." else char
def _Records(output: bytes) -> Iterator[bytes]:
if not output:
return iter(())
if not output.endswith(b"\0"):
raise StatusParseError("porcelain v2 output is not NUL terminated")
records = output.split(b"\0")
if not records[-1]:
records.pop()
return iter(records)
class StatusParseError(ValueError):
"""Raised when machine-readable status output is malformed."""
class UnsupportedStatusError(RuntimeError):
"""Raised when the Git client cannot produce porcelain v2."""
def _Fields(record: bytes, count: int) -> List[bytes]:
fields = record.split(b" ", count - 1)
if len(fields) != count:
raise StatusParseError(f"malformed porcelain v2 record: {record!r}")
return fields
def _AddTracked(
status: StatusSnapshot,
path: str,
xy: bytes,
src_path: Optional[str] = None,
level: Optional[str] = None,
) -> None:
index_status = _Status(xy[0])
worktree_status = _Status(xy[1])
if index_status:
status.index_changes[path] = StatusEntry(
path,
index_status,
src_path=src_path if index_status in ("R", "C") else None,
level=level if index_status in ("R", "C") else None,
)
if worktree_status:
status.worktree_changes[path] = StatusEntry(
path,
worktree_status,
src_path=src_path if worktree_status in ("R", "C") else None,
level=level if worktree_status in ("R", "C") else None,
)
def ParsePorcelainV2(output: bytes) -> StatusSnapshot:
"""Parse ``git status --porcelain=v2 -z --branch`` output."""
status = StatusSnapshot()
records = _Records(output)
for record in records:
kind = record[:1]
if kind == b"#":
try:
key, value = record[2:].split(b" ", 1)
except ValueError as e:
raise StatusParseError(
f"malformed porcelain v2 header: {record!r}"
) from e
if key == b"branch.oid":
value = value.decode("ascii")
status.branch_oid = None if value == "(initial)" else value
elif key == b"branch.head":
status.branch_head = _Path(value)
elif key == b"branch.upstream":
status.upstream = _Path(value)
elif key == b"branch.ab":
try:
value = value.decode("ascii")
ahead, behind = value.split()
if ahead != "+?" and behind != "-?":
status.ahead = int(ahead)
status.behind = -int(behind)
status.has_ahead_behind = True
except ValueError as e:
raise StatusParseError(
f"malformed porcelain v2 branch.ab record: {record!r}"
) from e
elif key == b"stash":
status.stash_count = int(value.decode("ascii"))
continue
if kind == b"1":
fields = _Fields(record, 9)
xy = fields[1]
if len(xy) != 2:
raise StatusParseError(f"invalid status pair: {xy!r}")
path = _Path(fields[8])
_AddTracked(status, path, xy)
elif kind == b"2":
fields = _Fields(record, 10)
xy = fields[1]
if len(xy) != 2:
raise StatusParseError(f"invalid status pair: {xy!r}")
score = fields[8][1:].lstrip(b"0") or b"0"
try:
src_path = _Path(next(records))
except StopIteration as e:
raise StatusParseError(
"rename record has no source path"
) from e
path = _Path(fields[9])
_AddTracked(
status,
path,
xy,
src_path=src_path,
level=score.decode("ascii"),
)
elif kind == b"u":
fields = _Fields(record, 11)
path = _Path(fields[10])
# The old diff-index/diff-files pair reported unmerged paths on
# both sides, regardless of porcelain's more specific XY pair.
status.index_changes[path] = StatusEntry(path, "U")
status.worktree_changes[path] = StatusEntry(path, "U")
elif kind == b"?":
status.untracked.append(_Path(record[2:]))
elif kind == b"!":
continue
else:
raise StatusParseError(
f"unknown porcelain v2 record type: {record!r}"
)
return status
+20 -3
View File
@@ -36,7 +36,6 @@ from git_command import git_require
from git_command import GitCommand
from git_config import IsId
from git_config import RepoConfig
from git_refs import GitRefs
import platform_utils
@@ -189,7 +188,7 @@ class Superproject:
if netloc:
parts = netloc.split("-review", 1)
host = parts[0]
rev = GitRefs(self._work_git).get("HEAD")
rev = self._GetRef("HEAD")
return f"{host}/{self.name}@{rev}"
return None
@@ -314,7 +313,10 @@ class Superproject:
# We use --negotiation-tip to speed up the fetch. Superproject branches
# do not share commits. So this lets git know it only needs to send
# commits reachable from the specified local refs.
rev_commit = GitRefs(self._work_git).get(f"refs/heads/{self.revision}")
negotiation_ref = self.revision
if negotiation_ref and not negotiation_ref.startswith("refs/"):
negotiation_ref = f"refs/heads/{negotiation_ref}"
rev_commit = self._GetRef(negotiation_ref) if negotiation_ref else ""
if rev_commit:
cmd.extend(["--negotiation-tip", rev_commit])
@@ -347,6 +349,21 @@ class Superproject:
return False
return True
def _GetRef(self, ref: str) -> str:
"""Resolve one local ref without loading the entire ref namespace."""
p = GitCommand(
None,
["rev-parse", "--verify", "--quiet", ref],
gitdir=self._work_git,
bare=True,
capture_stdout=True,
capture_stderr=True,
log_as_error=False,
)
if p.Wait() == 0:
return p.stdout.strip()
return ""
def _LsTree(self):
"""Gets the commit ids for all projects.
+18 -1
View File
@@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import optparse
import os
import re
import sys
@@ -69,6 +70,7 @@ class RepoHook:
ignore_hooks=False,
abort_if_user_denies=False,
yes=False,
fix=False,
):
"""RepoHook constructor.
@@ -91,6 +93,7 @@ class RepoHook:
abort_if_user_denies: If True, we'll abort running the hook if the
user doesn't allow us to run the hook.
yes: If True, then 'Yes' is assumed for any prompts.
fix: If True, then 'Fix' is assumed for any fixup prompts.
"""
self._hook_type = hook_type
self._hooks_project = hooks_project
@@ -102,6 +105,7 @@ class RepoHook:
self._ignore_hooks = ignore_hooks
self._abort_if_user_denies = abort_if_user_denies
self._yes = yes
self._fix = fix
# Store the full path to the script for convenience.
self._script_fullpath = None
@@ -380,6 +384,7 @@ class RepoHook:
kwargs = {
**kwargs,
"hook_should_take_kwargs": True,
"fix": self._fix,
"yes": self._yes,
}
@@ -504,12 +509,17 @@ class RepoHook:
).url,
"bug_url": manifest.contactinfo.bugurl,
"yes": getattr(opt, "yes", False),
"fix": getattr(opt, "fix", False),
}
)
return cls(*args, **kwargs)
@staticmethod
def AddOptionGroup(parser, name):
def AddOptionGroup(
parser: optparse.OptionParser,
name: str,
allow_fix: bool = False,
) -> None:
"""Help options relating to the various hooks."""
# Note that verify and no-verify are NOT opposites of each other, which
@@ -533,3 +543,10 @@ class RepoHook:
action="store_true",
help="Do not abort if %s hooks fail." % name,
)
if allow_fix:
group.add_option(
"--fix",
action="store_true",
default=False,
help="Automatically apply %s fixes without prompting." % name,
)
+5
View File
@@ -104,6 +104,11 @@ elif sys.version_info < MIN_PYTHON_VERSION_SOFT:
KEYBOARD_INTERRUPT_EXIT = 128 + signal.SIGINT
MAX_PRINT_ERRORS = 5
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(errors="surrogateescape")
if hasattr(sys.stderr, "reconfigure"):
sys.stderr.reconfigure(errors="surrogateescape")
global_options = optparse.OptionParser(
usage="repo [-p|--paginate|--no-pager] COMMAND [ARGS]",
add_help_option=False,
+12 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2026" "repo gc" "Repo Manual"
.TH REPO "1" "September 2026" "repo gc" "Repo Manual"
.SH NAME
repo \- repo gc - manual page for repo gc
.SH SYNOPSIS
@@ -22,6 +22,9 @@ answer yes to all safe prompts
.TP
\fB\-\-repack\fR
repack all projects that use partial clone with filter=blob:none
.TP
\fB\-\-keep\-manifest\-projects\fR
keep Git data for projects still in the manifests
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
@@ -44,3 +47,11 @@ only operate on this (sub)manifest
operate on this manifest and its submanifests
.PP
Run `repo help gc` to view the detailed manual.
.SH DETAILS
.PP
The 'repo gc' command removes internal Git data for projects that are no longer
active in the current configuration.
.PP
Projects excluded by the configured manifest groups are inactive by default. Use
\fB\-\-keep\-manifest\-projects\fR to keep Git data for projects that are still in the
manifests.
+14 -11
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2026" "repo manifest" "Repo Manual"
.TH REPO "1" "August 2026" "repo manifest" "Repo Manual"
.SH NAME
repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS
@@ -121,18 +121,19 @@ include*)>
<!ATTLIST remote revision CDATA #IMPLIED>
.IP
<!ELEMENT default EMPTY>
<!ATTLIST default remote IDREF #IMPLIED>
<!ATTLIST default revision CDATA #IMPLIED>
<!ATTLIST default dest\-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync\-j CDATA #IMPLIED>
<!ATTLIST default sync\-j\-max CDATA #IMPLIED>
<!ATTLIST default sync\-c CDATA #IMPLIED>
<!ATTLIST default sync\-s CDATA #IMPLIED>
<!ATTLIST default sync\-tags CDATA #IMPLIED>
<!ATTLIST default remote IDREF #IMPLIED>
<!ATTLIST default revision CDATA #IMPLIED>
<!ATTLIST default dest\-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync\-j CDATA #IMPLIED>
<!ATTLIST default sync\-j\-max CDATA #IMPLIED>
<!ATTLIST default sync\-c CDATA #IMPLIED>
<!ATTLIST default sync\-s CDATA #IMPLIED>
<!ATTLIST default sync\-tags CDATA #IMPLIED>
<!ATTLIST default sync\-smartsync CDATA #IMPLIED>
.IP
<!ELEMENT manifest\-server EMPTY>
<!ATTLIST manifest\-server url CDATA #REQUIRED>
<!ATTLIST manifest\-server url CDATA #REQUIRED>
<!ATTLIST manifest\-server helper CDATA #IMPLIED>
.IP
<!ELEMENT submanifest EMPTY>
@@ -336,6 +337,8 @@ Attribute `sync\-s`: Set to true to also sync sub\-projects.
Attribute `sync\-tags`: Set to false to only sync the given Git branch (specified
in the `revision` attribute) rather than the other ref tags.
.PP
Attribute `sync\-smartsync`: Set to true to enable smart sync by default.
.PP
Element manifest\-server
.PP
At most one manifest\-server may be specified. The url attribute is used to
+2 -2
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2026" "repo smartsync" "Repo Manual"
.TH REPO "1" "September 2026" "repo smartsync" "Repo Manual"
.SH NAME
repo \- repo smartsync - manual page for repo smartsync
.SH SYNOPSIS
@@ -21,7 +21,7 @@ number of jobs to run in parallel (default: based on number of CPU cores)
number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-no\-interleaved\fR is set
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or based on number of CPU cores). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future)
+5 -2
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2026" "repo sync" "Repo Manual"
.TH REPO "1" "September 2026" "repo sync" "Repo Manual"
.SH NAME
repo \- repo sync - manual page for repo sync
.SH SYNOPSIS
@@ -21,7 +21,7 @@ number of jobs to run in parallel (default: based on number of CPU cores)
number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-no\-interleaved\fR is set
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or based on number of CPU cores). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future)
@@ -125,6 +125,9 @@ do not run garbage collection on any projects (default)
\fB\-s\fR, \fB\-\-smart\-sync\fR
smart sync using manifest from the latest known good build
.TP
\fB\-\-no\-smart\-sync\fR
disable smart sync and sync to ToT instead
.TP
\fB\-t\fR SMART_TAG, \fB\-\-smart\-tag\fR=\fI\,SMART_TAG\/\fR
smart sync using manifest from a known tag
.SS Logging options:
+4 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2026" "repo upload" "Repo Manual"
.TH REPO "1" "August 2026" "repo upload" "Repo Manual"
.SH NAME
repo \- repo upload - manual page for repo upload
.SH SYNOPSIS
@@ -112,6 +112,9 @@ Run the pre\-upload hook without prompting.
.TP
\fB\-\-ignore\-hooks\fR
Do not abort if pre\-upload hooks fail.
.TP
\fB\-\-fix\fR
Automatically apply pre\-upload fixes without prompting.
.PP
Run `repo help upload` to view the detailed manual.
.SH DETAILS
+30 -18
View File
@@ -159,6 +159,7 @@ class _Default:
sync_c = False
sync_s = False
sync_tags = True
sync_smartsync = False
def __eq__(self, other):
if not isinstance(other, _Default):
@@ -483,6 +484,17 @@ class XmlManifest:
self.Unload()
self._Load()
def ClearOverride(self) -> None:
"""Stop overriding the manifest, reverting to the default one.
This is the inverse of Override(), and is a no-op if no override is in
effect. It is useful to recover from a failed Override(), which
registers the override before parsing it.
"""
self._outer_client.manifestFileOverrides.pop(self.path_prefix, None)
self._load_local_manifests = True
self.Unload()
def Link(self, name):
"""Update the repo metadata to use a different manifest."""
self.Override(name)
@@ -644,6 +656,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if not d.sync_tags:
have_default = True
e.setAttribute("sync-tags", "false")
if d.sync_smartsync:
have_default = True
e.setAttribute("sync-smartsync", "true")
if have_default:
root.appendChild(e)
root.appendChild(doc.createTextNode(""))
@@ -692,9 +707,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
e.setAttribute("remote", remoteName)
if peg_rev:
if self.IsMirror:
value = p.bare_git.rev_parse(p.revisionExpr + "^0")
value = p.bare_git.ResolveCommit(p.revisionExpr)
else:
value = p.work_git.rev_parse(HEAD + "^0")
value = p.work_git.ResolveCommit(HEAD)
e.setAttribute("revision", value)
if peg_rev_upstream:
if p.upstream:
@@ -1519,9 +1534,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if base_revision:
if p.revisionExpr != base_revision:
failed_revision_changes.append(
"extend-project name %s mismatch base "
"%s vs revision %s"
% (name, base_revision, p.revisionExpr)
f"extend-project name {name}:\n "
f"base {base_revision} vs "
f"revision {p.revisionExpr}"
)
p.SetRevision(revision)
@@ -1622,9 +1637,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if base_revision:
if p.revisionExpr != base_revision:
failed_revision_changes.append(
"remove-project name %s mismatch base "
"%s vs revision %s"
% (name, base_revision, p.revisionExpr)
f"remove-project name {name}:\n "
f"base {base_revision} vs "
f"revision {p.revisionExpr}"
)
del self._paths[p.relpath]
if not removed_project:
@@ -1636,13 +1651,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if base_revision:
if p.revisionExpr != base_revision:
failed_revision_changes.append(
"remove-project path %s mismatch base "
"%s vs revision %s"
% (
p.relpath,
base_revision,
p.revisionExpr,
)
f"remove-project path {p.relpath}:\n "
f"base {base_revision} vs "
f"revision {p.revisionExpr}"
)
self._projects[projname].remove(p)
del self._paths[p.relpath]
@@ -1664,10 +1675,10 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
)
if failed_revision_changes:
fail_string = "\n".join(failed_revision_changes)
raise ManifestParseError(
"revision base check failed, rebase patches and update "
"base revs for: ",
failed_revision_changes,
f"detected base-revision mismatch, updates needed:\n"
f"{fail_string}",
)
# Store repo hooks project information.
@@ -1793,6 +1804,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
d.sync_c = XmlBool(node, "sync-c", False)
d.sync_s = XmlBool(node, "sync-s", False)
d.sync_tags = XmlBool(node, "sync-tags", True)
d.sync_smartsync = XmlBool(node, "sync-smartsync", False)
return d
def _ParseNotice(self, node):
+974 -236
View File
File diff suppressed because it is too large. Load diff
+2 -1
View File
@@ -23,4 +23,5 @@ import sys
import update_manpages
sys.exit(update_manpages.main(sys.argv[1:]))
if __name__ == "__main__":
sys.exit(update_manpages.main(sys.argv[1:]))
+54 -18
View File
@@ -26,7 +26,7 @@ import platform
import shlex
import subprocess
import sys
from typing import NamedTuple
from typing import NamedTuple, Tuple
# These should never be newer than the main.py version since this needs to be a
@@ -129,7 +129,7 @@ if not REPO_REV:
BUG_URL = "https://issues.gerritcodereview.com/issues/new?component=1370071"
# increment this whenever we make important changes to this script
VERSION = (2, 65)
VERSION = (2, 69)
# increment this if the MAINTAINER_KEYS block is modified
KEYRING_VERSION = (2, 3)
@@ -220,6 +220,10 @@ repodir = ".repo" # name of repo's private directory
S_repo = "repo" # special repo repository
S_manifests = "manifests" # special manifest repository
REPO_MAIN = S_repo + "/main.py" # main script
# The automated release process deletes the stable branch and then recreates it
# at the new release, so a fetch can briefly miss it. Wait this long and
# refetch once before giving up on --repo-rev.
REV_REFETCH_DELAY_SEC = 10
import errno
@@ -228,6 +232,7 @@ import optparse
import re
import shutil
import stat
import time
import urllib.error
import urllib.request
@@ -631,6 +636,7 @@ def _Init(args):
dst = dst_final + ".tmp"
shutil.rmtree(dst, ignore_errors=True)
_Clone(url, dst, opt.clone_bundle, opt.quiet, opt.verbose)
_RefetchIfRevMissing(url, dst, rev, opt.quiet, opt.verbose)
remote_ref, rev = check_repo_rev(
dst, rev, opt.repo_verify, quiet=opt.quiet
@@ -1005,7 +1011,43 @@ def _Clone(url, cwd, clone_bundle, quiet, verbose):
_Fetch(url, cwd, "origin", quiet, verbose)
def resolve_repo_rev(cwd, committish):
def _RefetchIfRevMissing(
url: str, cwd: str, rev: str, quiet: bool, verbose: bool
) -> None:
"""Refetch once if |rev| doesn't resolve.
This doesn't fail if |rev| still doesn't resolve; check_repo_rev reports
that.
Any unresolvable |rev| is retried, not just branches. Telling a branch
apart from a tag or commit isn't possible while it's missing, and the only
cost for a bad --repo-rev is a short delay before the same error.
Args:
url: The URL of the repo source.
cwd: The git checkout to fetch into.
rev: The --repo-rev to wait for.
quiet: Whether to suppress fetch output.
verbose: Whether to show verbose fetch output.
"""
try:
resolve_repo_rev(cwd, rev, report_errors=False)
return
except CloneFailure:
pass
# Print even with --quiet since this explains the delay.
print(
f'repo: warning: unable to resolve "{rev}"; '
f"refetching in {REV_REFETCH_DELAY_SEC} seconds",
file=sys.stderr,
)
time.sleep(REV_REFETCH_DELAY_SEC)
_Fetch(url, cwd, "origin", quiet, verbose)
def resolve_repo_rev(
cwd: str, committish: str, report_errors: bool = True
) -> Tuple[str, str]:
"""Figure out what REPO_REV represents.
We support:
@@ -1016,6 +1058,7 @@ def resolve_repo_rev(cwd, committish):
Args:
cwd: The git checkout to run in.
committish: The REPO_REV argument to resolve.
report_errors: Whether to print an error if it can't be resolved.
Returns:
A tuple of (remote ref, commit) as makes sense for the committish.
@@ -1034,17 +1077,18 @@ def resolve_repo_rev(cwd, committish):
)
return None if ret.returncode else ret.stdout.strip()
def error(msg: str) -> CloneFailure:
if report_errors:
print(f"repo: error: {msg}", file=sys.stderr)
return CloneFailure()
# An explicit branch.
if committish.startswith("refs/heads/"):
remote_ref = committish
committish = committish[len("refs/heads/") :]
rev = resolve("refs/remotes/origin/%s" % committish)
if rev is None:
print(
f'repo: error: unknown branch "{committish}"',
file=sys.stderr,
)
raise CloneFailure()
raise error(f'unknown branch "{committish}"')
return (remote_ref, rev)
# An explicit tag.
@@ -1053,11 +1097,7 @@ def resolve_repo_rev(cwd, committish):
committish = committish[len("refs/tags/") :]
rev = resolve(remote_ref)
if rev is None:
print(
f'repo: error: unknown tag "{committish}"',
file=sys.stderr,
)
raise CloneFailure()
raise error(f'unknown tag "{committish}"')
return (remote_ref, rev)
# See if it's a short branch name.
@@ -1076,11 +1116,7 @@ def resolve_repo_rev(cwd, committish):
return (rev, rev)
# Give up!
print(
f'repo: error: unable to resolve "{committish}"',
file=sys.stderr,
)
raise CloneFailure()
raise error(f'unable to resolve "{committish}"')
def verify_rev(cwd, remote_ref, rev, quiet):
+3 -3
View File
@@ -57,10 +57,10 @@ def SetTraceToStderr():
_TRACE_TO_STDERR = True
def SetTrace():
"""Enables tracing."""
def SetTrace(value: bool = True) -> None:
"""Enables by default, or disables tracing."""
global _TRACE
_TRACE = True
_TRACE = value
def _SetTraceFile(quiet):
+13 -1
View File
@@ -13,6 +13,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# /// script
# requires-python = '>=3.11,<3.12'
# dependencies = [
# 'pytest==8.3.4',
# 'black==25.1.0',
# 'flake8==6.0.0',
# 'flake8-bugbear==24.12.12',
# 'flake8-comprehensions==3.16.0',
# 'isort==5.10.1'
# ]
# ///
"""Wrapper to run linters and pytest with the right settings."""
import functools
@@ -135,7 +147,7 @@ def run_isort():
cwd=ROOT_DIR,
)
argv = ["--check", ROOT_DIR]
argv = ["--check", "--diff", ROOT_DIR]
log_cmd("isort", argv)
return subprocess.run(
[sys.executable, "-m", "isort"] + argv,
+101
View File
@@ -0,0 +1,101 @@
attrs==24.2.0 \
--hash=sha256:37c4653e51383dc1249dba15635d59011ca11714a8e61db8cd6dc73011e4b603 \
--hash=sha256:81921eb96de3191c8258c199618104dd27ac608d9366f5e35d011eae1867ede2
# via flake8-bugbear
black==25.1.0 \
--hash=sha256:95e8176dae143ba9097f351d174fdaf0ccd29efb414b362ae3fd72bf0f710717 \
--hash=sha256:c9ed033e41998dd45d6f19bf9923e2444cb401185ccb26d1a2101a9049637c1f
click==8.0.3 \
--hash=sha256:353f466495adaeb40b6b5f592f9f91cb22372351c84caeb068132442a4518ef3 \
--hash=sha256:353f466495adaeb40b6b5f592f9f91cb22372351c84caeb068132442a4518ef3 \
--hash=sha256:b0a0492d07c8e7e4a88ee153ca6553d357a0746792d6868bfca79dc1a344bb1b
# via black
colorama==0.4.6 ; sys_platform == 'win32' \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 \
--hash=sha256:8a5a9494346d7ae2d8f599e4661738007d2df56dbbf4ccbbf7a6a23679a3a302
# via
# click
# pytest
flake8==6.0.0 \
--hash=sha256:9b5bf0ebd06aff1601e38dfc4e91f9b99ba3cdf446e74ee25a9cbbe01aef81ee \
--hash=sha256:f34b5cb8bf908ff49e4673d3dbc557b2381a14132b637d53e9d3a59816f4b6ba
# via
# flake8-bugbear
# flake8-comprehensions
flake8-bugbear==24.12.12 \
--hash=sha256:1b6967436f65ca22a42e5373aaa6f2d87966ade9aa38d4baf2a1be550767545e
flake8-comprehensions==3.16.0 \
--hash=sha256:7c1eadc9d22e765f39857798febe7766b4d9c519793c6c149e3e13bf99693f70
iniconfig==1.1.1 \
--hash=sha256:011e24c64b7f47f6ebd835bb12a743f2fbe9a26d4cecaa7f53bc4f35ee9da8b3 \
--hash=sha256:9d81514370059186276e9a1799e5a9752e4c060233c254d6954b53fad2ff8a5e
# via pytest
isort==5.10.1 \
--hash=sha256:6f62d78e2f89b4500b080fe3a81690850cd254227f27f75c3a0c491a1f351ba7 \
--hash=sha256:ec0da0d88470e89ade75bf1e2ea7b4349ec906f95c84aff08e4d41dea6864d25
mccabe==0.7.0 \
--hash=sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e \
--hash=sha256:b85abd0a321cdd357d0be34adbf49cf01453a857a23e94d44612ec6f159cb261
# via flake8
mypy-extensions==0.4.3 \
--hash=sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d \
--hash=sha256:38cbac35a502c4f13b9d9b1d06258fa51e01da4f5828b171b4d1200d00378e7c
# via black
packaging==23.0 \
--hash=sha256:37cb5b4759d17ae6bad78ee3347f0ee1057452895fcd1bbff48fb12908ce00a1 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2
# via
# black
# pytest
pathspec==0.9.0 \
--hash=sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a \
--hash=sha256:aaf90f4ad38a16bd3ebe0fa64da7f6ee01819c881120db68cc6440941a9adc08
# via black
platformdirs==2.5.2 \
--hash=sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788 \
--hash=sha256:e257a04b5d0528cb9836b766c8a16a85368e249cba7ae42888a403976ccd7d2d
# via black
pluggy==1.5.0 \
--hash=sha256:40da5bf96b15e359e280fd760a80c11d7a8ea20c3c296c3f3e09f8be67cec10b \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669
# via pytest
py==1.11.0 \
--hash=sha256:607c53218732647dff4acdfcd50cb62615cedf612e72d1724fb1a0cc6405b378 \
--hash=sha256:607c53218732647dff4acdfcd50cb62615cedf612e72d1724fb1a0cc6405b378
pycodestyle==2.10.0 \
--hash=sha256:8a4eaf0d0495c7395bdab3589ac2db602797d76207242c17d470186815706610 \
--hash=sha256:aacf8d28ecea10fad31b2d14943a42d85d007f4d2fee8f212fc05751c8460647
# via flake8
pyflakes==3.0.1 \
--hash=sha256:a968b8baa43926124b692c7e76279bc5fa8c6ef9a2281e4a2a475be69f954bdf \
--hash=sha256:ec55bf7fe21fff7f1ad2f7da62363d749e2a470500eab1b555334b67aa1ef8cf
# via flake8
pyparsing==3.0.7 \
--hash=sha256:943c220ca64e93940b74dd070152ed9ef1402bc3b28fd1f899843bee34e83708 \
--hash=sha256:a6c06a88f252e6c322f65faf8f418b16213b51bdfaece0524c1c1bc30c63c484
pytest==8.3.4 \
--hash=sha256:00cba7845d2dccfe5403025a688d60ceae228b1ddaa866d2eb86b2edb6923e73 \
--hash=sha256:50e16d954148559c9a74109af1eaf0c945ba2d8f30f0a3d3335edde19788b6f6
toml==0.10.1 \
--hash=sha256:4099063d9b50219bf09bfa693ab8dcd96ce585cee9fdadce5a48a08ee23b0646 \
--hash=sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88
tomli==2.0.1 \
--hash=sha256:864f310e021d68415d6ba45b23e29808f99703b471d884267d6942eecdf8d6c5 \
--hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc
typing-extensions==4.3.0 \
--hash=sha256:09126dab54422810a541e078c37e62024e7260a022d4f67c075a6707d9553649 \
--hash=sha256:25642c956049920a5aa49edcdd6ab1e06d7e5d467fc00e0506c44ac86fbfca02
-135
View File
@@ -1,135 +0,0 @@
# This is a vpython "spec" file.
#
# Read more about `vpython` and how to modify this file here:
# https://chromium.googlesource.com/infra/infra/+/main/doc/users/vpython.md
# List of available wheels:
# https://chromium.googlesource.com/infra/infra/+/main/infra/tools/dockerbuild/wheels.md
python_version: "3.11"
wheel: <
name: "infra/python/wheels/pytest-py3"
version: "version:8.3.4"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/py-py2_py3"
version: "version:1.11.0"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/iniconfig-py3"
version: "version:1.1.1"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/packaging-py3"
version: "version:23.0"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/pluggy-py3"
version: "version:1.5.0"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/toml-py3"
version: "version:0.10.1"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/pyparsing-py3"
version: "version:3.0.7"
>
# Required by pytest==8.3.4 and flake8-bugbear==24.12.12
wheel: <
name: "infra/python/wheels/attrs-py3"
version: "version:24.2.0"
>
# NB: Keep in sync with constraints.txt.
wheel: <
name: "infra/python/wheels/black-py3"
version: "version:25.1.0"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/mypy-extensions-py3"
version: "version:0.4.3"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/tomli-py3"
version: "version:2.0.1"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/platformdirs-py3"
version: "version:2.5.2"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/pathspec-py3"
version: "version:0.9.0"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/typing-extensions-py3"
version: "version:4.3.0"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/click-py3"
version: "version:8.0.3"
>
wheel: <
name: "infra/python/wheels/flake8-py2_py3"
version: "version:6.0.0"
>
# Required by flake8==6.0.0
wheel: <
name: "infra/python/wheels/mccabe-py2_py3"
version: "version:0.7.0"
>
# Required by flake8==6.0.0
wheel: <
name: "infra/python/wheels/pyflakes-py2_py3"
version: "version:3.0.1"
>
# Required by flake8==6.0.0
wheel: <
name: "infra/python/wheels/pycodestyle-py2_py3"
version: "version:2.10.0"
>
wheel: <
name: "infra/python/wheels/flake8-bugbear-py3"
version: "version:24.12.12"
>
wheel: <
name: "infra/python/wheels/flake8-comprehensions-py3"
version: "version:3.16.0"
>
wheel: <
name: "infra/python/wheels/isort-py3"
version: "version:5.10.1"
>
+2 -4
View File
@@ -20,7 +20,7 @@ from command import Command
from command import DEFAULT_LOCAL_JOBS
from error import RepoError
from error import RepoExitError
from git_command import git
from git_command import IsValidBranchName
from progress import Progress
from repo_logging import RepoLogger
@@ -58,9 +58,7 @@ It is equivalent to "git branch -D <branchname>".
if not opt.all:
branches = args[0].split()
invalid_branches = [
x for x in branches if not git.check_ref_format(f"heads/{x}")
]
invalid_branches = [x for x in branches if not IsValidBranchName(x)]
if invalid_branches:
self.OptionParser.error(
+45 -30
View File
@@ -14,6 +14,7 @@
import re
import sys
from typing import Tuple
from command import Command
from error import GitError
@@ -43,36 +44,8 @@ change id will be added.
def Execute(self, opt, args):
reference = args[0]
p = GitCommand(
None,
["rev-parse", "--verify", reference],
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
try:
p.Wait()
except GitError:
logger.error(p.stderr)
raise
sha1 = p.stdout.strip()
p = GitCommand(
None,
["cat-file", "commit", sha1],
capture_stdout=True,
verify_command=True,
)
try:
p.Wait()
except GitError:
logger.error("error: Failed to retrieve old commit message")
raise
old_msg = self._StripHeader(p.stdout)
sha1, commit = self._ResolveReference(reference)
old_msg = self._StripHeader(commit)
p = GitCommand(
None,
@@ -117,6 +90,48 @@ change id will be added.
logger.error("error: Failed to update commit message")
raise
def _ResolveReference(self, reference: str) -> Tuple[str, str]:
"""Resolve a commit and read it through one cat-file batch request."""
expression = f"{reference}^{{commit}}"
p = GitCommand(
None,
["cat-file", "--batch"],
input=expression + "\n",
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
try:
p.Wait()
header, separator, output = p.stdout.partition("\n")
if not separator:
raise ValueError("missing cat-file header")
if header.endswith(" missing") or header.endswith(" ambiguous"):
raise GitError(f"commit {reference} not found")
parts = header.split(" ", 2)
if len(parts) != 3 or parts[1] != "commit":
raise ValueError(
f"unexpected object type {parts[1]!r}"
if len(parts) >= 2
else "invalid header"
)
sha1, _object_type, _size = parts
if not output.endswith("\n"):
raise ValueError("truncated cat-file object")
commit = output[:-1]
except (GitError, ValueError) as e:
logger.error(
"error: Failed to resolve or read commit %s", reference
)
if isinstance(e, GitError):
raise
raise GitError(str(e)) from e
return sha1, commit
def _IsChangeId(self, line):
return CHANGE_ID_RE.match(line)
+1 -1
View File
@@ -195,7 +195,7 @@ If no project is specified try to use current directory as a project.
elif opt.revert:
project._Revert(dl.commit)
elif opt.ffonly:
project._FastForward(dl.commit, ffonly=True)
project._FastForward(dl.commit)
else:
if opt.branch:
project.StartBranch(opt.branch, revision=dl.commit)
+6 -2
View File
@@ -244,10 +244,14 @@ without iterating through the remaining projects.
mirror = self.manifest.IsMirror
if opt.regex:
projects = self.FindProjects(args, all_manifests=all_trees)
projects = self.FindProjects(
args,
groups=opt.groups,
all_manifests=all_trees,
)
elif opt.inverse_regex:
projects = self.FindProjects(
args, inverse=True, all_manifests=all_trees
args, inverse=True, groups=opt.groups, all_manifests=all_trees
)
else:
projects = self.GetProjects(
+25 -6
View File
@@ -28,6 +28,14 @@ class Gc(Command):
helpSummary = "Cleaning up internal repo and Git state."
helpUsage = """
%prog
"""
helpDescription = """
The '%prog' command removes internal Git data for projects that are no
longer active in the current configuration.
Projects excluded by the configured manifest groups are inactive by
default. Use --keep-manifest-projects to keep Git data for projects that
are still in the manifests.
"""
def _Options(self, p):
@@ -53,6 +61,12 @@ class Gc(Command):
help="repack all projects that use partial clone with "
"filter=blob:none",
)
p.add_option(
"--keep-manifest-projects",
default=False,
action="store_true",
help="keep Git data for projects still in the manifests",
)
def _find_git_to_delete(
self, to_keep: Set[str], start_dir: str
@@ -292,16 +306,21 @@ class Gc(Command):
args, all_manifests=not opt.this_manifest_only
)
# If the user specified projects, fetch the global list separately
# to avoid deleting untargeted projects.
if args:
all_projects = self.GetProjects(
if opt.keep_manifest_projects:
projects_to_keep = self.GetProjects(
[],
groups="all",
missing_ok=True,
all_manifests=not opt.this_manifest_only,
)
elif args:
projects_to_keep = self.GetProjects(
[], all_manifests=not opt.this_manifest_only
)
else:
all_projects = projects
projects_to_keep = projects
ret = self.delete_unused_projects(all_projects, opt)
ret = self.delete_unused_projects(projects_to_keep, opt)
if ret != 0:
return ret
+58 -26
View File
@@ -18,7 +18,7 @@ import io
import json
import optparse
import sys
from typing import Any, Dict, List, NamedTuple
from typing import Any, Dict, List, NamedTuple, Optional, Tuple
from color import Coloring
from command import DEFAULT_LOCAL_JOBS
@@ -189,18 +189,26 @@ class Info(PagedCommand):
"superproject_revision": srev,
}
@staticmethod
def _GetCurrentBranch(branches: Dict[str, Any]) -> Optional[str]:
"""Return the name of the current branch from a GetBranches mapping."""
return next(
(name for name, branch in branches.items() if branch.current), None
)
@classmethod
def _getProjectData(cls, project) -> Dict[str, Any]:
"""Gather project data as a dict."""
branches = project.GetBranches()
currentBranch = cls._GetCurrentBranch(branches)
data = {
"name": project.name,
"mount_path": project.worktree,
"current_revision": project.GetHeadRevisionId()
or project.GetRevisionId(),
"manifest_revision": project.revisionExpr,
"local_branches": list(project.GetBranches()),
"local_branches": list(branches),
}
currentBranch = project.CurrentBranch
if currentBranch:
data["current_branch"] = currentBranch
return data
@@ -285,6 +293,9 @@ class Info(PagedCommand):
text = out.nofmt_printer("text")
dimtext = out.printer("dimtext", attr="dim")
branches = project.GetBranches()
currentBranch = cls._GetCurrentBranch(branches)
heading("Project: ")
headtext(project.name)
out.nl()
@@ -297,7 +308,6 @@ class Info(PagedCommand):
headtext(project.GetHeadRevisionId() or project.GetRevisionId())
out.nl()
currentBranch = project.CurrentBranch
if currentBranch:
heading("Current branch: ")
headtext(currentBranch)
@@ -307,7 +317,7 @@ class Info(PagedCommand):
headtext(project.revisionExpr)
out.nl()
localBranches = list(project.GetBranches().keys())
localBranches = list(branches)
heading("Local Branches: ")
redtext(str(len(localBranches)))
if localBranches:
@@ -327,25 +337,10 @@ class Info(PagedCommand):
branch = branch[len(R_HEADS) :]
logTarget = R_M + branch
bareTmp = project.bare_git._bare
project.bare_git._bare = False
localCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
logTarget + "..",
"--",
localCommits, originCommits = cls._GetDiffCommits(
project, logTarget
)
originCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
".." + logTarget,
"--",
)
project.bare_git._bare = bareTmp
heading("Local Commits: ")
redtext(str(len(localCommits)))
dimtext(" (on current branch)")
@@ -375,6 +370,35 @@ class Info(PagedCommand):
return buf.getvalue()
@classmethod
def _GetDiffCommits(
cls, project: Any, log_target: str
) -> Tuple[List[str], List[str]]:
"""Return local-only and remote-only commits from one history walk."""
git = getattr(project, "work_git", None) or getattr(
project, "bare_git", None
)
if git is None:
return [], []
commits = git.rev_list(
"--left-right",
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
f"HEAD...{log_target}",
"--",
)
if isinstance(commits, str):
commits = commits.splitlines()
local = []
remote = []
for commit in commits:
if commit.startswith("<"):
local.append(commit[1:])
elif commit.startswith(">"):
remote.append(commit[1:])
return local, remote
def _printDiffInfo(self, opt, args):
projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only)
@@ -404,10 +428,18 @@ class Info(PagedCommand):
project = cls.get_parallel_context()["projects"][project_idx]
branches = []
br = [project.GetUploadableBranch(x) for x in project.GetBranches()]
br = [x for x in br if x]
local_branches = project.GetBranches()
current_branch = cls._GetCurrentBranch(local_branches)
if opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch]
candidate_branches = (
[current_branch]
if current_branch and current_branch in local_branches
else []
)
else:
candidate_branches = local_branches
br = [project.GetUploadableBranch(x) for x in candidate_branches]
br = [x for x in br if x]
for b in br:
branches.append(
@@ -416,7 +448,7 @@ class Info(PagedCommand):
name=b.name,
commits=b.commits,
date=b.date,
is_current=b.name == project.CurrentBranch,
is_current=b.name == current_branch,
)
)
return branches
+4 -1
View File
@@ -109,7 +109,10 @@ This is similar to running: repo forall -c 'echo "$REPO_PATH : $REPO_PROJECT"'.
)
else:
projects = self.FindProjects(
args, all_manifests=not opt.this_manifest_only
args,
groups=opt.groups,
missing_ok=opt.all,
all_manifests=not opt.this_manifest_only,
)
def _getpath(x):
+9 -4
View File
@@ -59,10 +59,15 @@ are displayed.
for project in self.GetProjects(
args, all_manifests=not opt.this_manifest_only
):
br = [project.GetUploadableBranch(x) for x in project.GetBranches()]
br = [x for x in br if x]
local_branches = project.GetBranches()
br = []
for name, branch in local_branches.items():
uploadable = project.GetUploadableBranch(name)
if uploadable:
uploadable.branch.current = branch.current
br.append(uploadable)
if opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch]
br = [x for x in br if x.current]
all_branches.extend(br)
if not all_branches:
@@ -97,7 +102,7 @@ are displayed.
print(
"%s %-33s (%2d commit%s, %s)"
% (
branch.name == project.CurrentBranch and "*" or " ",
branch.current and "*" or " ",
branch.name,
len(commits),
len(commits) != 1 and "s" or " ",
+1 -1
View File
@@ -80,7 +80,7 @@ class Prune(PagedCommand):
print(
"%s %-33s "
% (
branch.name == project.CurrentBranch and "*" or " ",
branch.current and "*" or " ",
branch.name,
),
end="",
+2 -19
View File
@@ -139,6 +139,8 @@ branch but need to incorporate new upstream changes "underneath" them.
common_args.append("--autosquash")
if opt.interactive:
common_args.append("-i")
if opt.auto_stash:
common_args.append("--autostash")
config = self.manifest.manifestProject.config
out = RebaseColoring(config)
@@ -188,29 +190,10 @@ branch but need to incorporate new upstream changes "underneath" them.
out.nl()
out.flush()
needs_stash = False
if opt.auto_stash:
stash_args = ["update-index", "--refresh", "-q"]
if GitCommand(project, stash_args).Wait() != 0:
needs_stash = True
# Dirty index, requires stash...
stash_args = ["stash"]
if GitCommand(project, stash_args).Wait() != 0:
ret += 1
continue
if GitCommand(project, args).Wait() != 0:
ret += 1
continue
if needs_stash:
stash_args.append("pop")
stash_args.append("--quiet")
if GitCommand(project, stash_args).Wait() != 0:
ret += 1
if ret:
msg_fmt = "%d projects had errors"
self.git_event_log.ErrorEvent(msg_fmt % (ret), msg_fmt)
+2 -2
View File
@@ -18,7 +18,7 @@ from typing import NamedTuple
from command import Command
from command import DEFAULT_LOCAL_JOBS
from error import RepoExitError
from git_command import git
from git_command import IsValidBranchName
from git_config import IsImmutable
from progress import Progress
from repo_logging import RepoLogger
@@ -75,7 +75,7 @@ revision specified in the manifest.
self.Usage()
nb = args[0]
if not git.check_ref_format("heads/%s" % nb):
if not IsValidBranchName(nb):
self.OptionParser.error("'%s' is not a valid name" % nb)
@classmethod
+425 -55
View File
@@ -28,7 +28,7 @@ import subprocess
import sys
import tempfile
import time
from typing import List, NamedTuple, Optional, Set, Tuple, Union
from typing import Dict, List, NamedTuple, Optional, Set, Tuple, Union
import urllib.error
import urllib.parse
import urllib.request
@@ -58,6 +58,7 @@ from command import DEFAULT_LOCAL_JOBS
from command import MirrorSafeCommand
from command import WORKER_BATCH_SIZE
from error import GitError
from error import ManifestParseError
from error import RepoChangedException
from error import RepoError
from error import RepoExitError
@@ -72,6 +73,7 @@ from git_refs import HEAD
from git_refs import R_HEADS
import git_superproject
from hooks import RepoHook
from manifest_xml import XmlManifest
import platform_utils
from progress import elapsed_str
from progress import jobs_str
@@ -96,6 +98,69 @@ _BLOAT_SIZE_GARBAGE_THRESHOLD_KB = 1 * 1024 * 1024 # 1 GiB in KiB
logger = RepoLogger(__file__)
# Manifest server RPC that takes a single struct of named options. Servers
# that do not implement it are called via the positional GetApprovedManifest.
_GET_APPROVED_MANIFEST_V2 = "GetApprovedManifestV2"
def _IsMethodNotFoundFault(fault: xmlrpc.client.Fault, method: str) -> bool:
"""Whether |fault| means the server does not implement |method|."""
if fault.faultCode == xmlrpc.client.METHOD_NOT_FOUND:
return True
# Python's SimpleXMLRPCDispatcher uses faultCode 1 with this message.
return f'method "{method}" is not supported' in str(fault.faultString)
def _GetApprovedManifest(
server: xmlrpc.client.ServerProxy,
branch: str,
target: Optional[str],
best_effort: bool,
) -> Tuple[bool, str]:
"""Requests the approved manifest from the manifest server.
Args:
server: The manifest server proxy.
branch: The manifest branch.
target: The build target, if any.
best_effort: The client can proceed without a manifest (e.g. by
falling back to ToT), so the server may decline the request.
Returns:
A (success, manifest XML or error message) tuple.
"""
request = {"branch": branch}
if target:
request["target"] = target
if best_effort:
request["best_effort"] = True
try:
response = server.GetApprovedManifestV2(request)
except xmlrpc.client.Fault as e:
if not _IsMethodNotFoundFault(e, _GET_APPROVED_MANIFEST_V2):
raise
# Older server: |best_effort| has no equivalent and is dropped.
if target:
success, manifest_str = server.GetApprovedManifest(branch, target)
else:
success, manifest_str = server.GetApprovedManifest(branch)
return success, manifest_str
if not isinstance(response, dict):
raise SmartSyncError(
f"error: invalid {_GET_APPROVED_MANIFEST_V2} response: {response!r}"
)
if response.get("success"):
manifest_xml = response.get("manifest_xml")
if not isinstance(manifest_xml, str):
raise SmartSyncError(
f"error: {_GET_APPROVED_MANIFEST_V2} response has invalid "
f"manifest_xml: {manifest_xml!r}"
)
return True, manifest_xml
return False, response.get("error_message", "unknown error")
def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
"""Generate a sequence of checkouts that is safe to perform.
@@ -156,6 +221,88 @@ def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
return res
def _NestedProjects(projects: List[Project]) -> List[Project]:
"""Return the projects in |projects| living inside another one's path."""
return [p for level in _SafeCheckoutOrder(projects)[1:] for p in level]
def _ParentFirstBatches(projects: List[Project]) -> List[List[Project]]:
"""Group |projects| so that a parent is fetched before its submodules.
A discovered submodule can only be fetched at the right revision once the
project holding its gitlink has been fetched, so it is held back to a later
batch than its parent. Projects that are not discovered submodules all end
up in the first batch, which keeps manifests without submodules on a single
batch.
"""
batches = collections.defaultdict(list)
for project in projects:
depth = 0
ancestor = project
while ancestor.Derived and ancestor.parent:
depth += 1
ancestor = ancestor.parent
batches[depth].append(project)
return [batches[depth] for depth in sorted(batches)]
def _RefreshDerivedRevisions(
projects: List[Project],
submodule_revisions: Optional[Dict[Project, Dict[str, str]]] = None,
) -> List[Project]:
"""Re-resolve the gitlinks of the discovered submodules in |projects|.
The revision of a discovered submodule is read from its parent when the
manifest is loaded, so it is stale as soon as the parent gets fetched. It
has to be resolved again once the parent is up-to-date, and before the
submodule itself is fetched and checked out.
Args:
projects: The projects whose discovered submodules to resolve.
submodule_revisions: Gitlinks already read, keyed by the project
holding them. Passing the same dict for project sets that follow
the same fetches, e.g. the levels of one checkout order, keeps a
project from being read more than once.
Returns:
The submodules that their parent no longer holds a gitlink for.
"""
if submodule_revisions is None:
submodule_revisions = {}
subprojects_by_parent = collections.defaultdict(list)
for project in projects:
if project.Derived and project.parent:
subprojects_by_parent[project.parent].append(project)
removed = []
for parent, subprojects in subprojects_by_parent.items():
revisions = submodule_revisions.get(parent)
if revisions is None:
revisions = parent.GetSubmoduleRevisions()
if revisions is None:
# Leave the submodules of a parent we cannot read alone.
continue
submodule_revisions[parent] = revisions
for subproject in subprojects:
rev = revisions.get(subproject.gitlink_path)
if rev:
subproject.SetRevision(rev, revisionId=rev)
else:
removed.append(subproject)
return removed
def _WithoutProjects(
projects: List[Project], unwanted: List[Project]
) -> List[Project]:
"""Return |projects| without the projects in |unwanted|."""
if not unwanted:
return projects
dropped = set(unwanted)
return [p for p in projects if p not in dropped]
def _chunksize(projects: int, jobs: int) -> int:
"""Calculate chunk size for the given number of projects and jobs."""
return min(max(1, projects // jobs), WORKER_BATCH_SIZE)
@@ -224,7 +371,8 @@ class _SyncResult(NamedTuple):
Attributes:
project_index (int): The index of the project in the shared list.
relpath (str): The project's relative path from the repo client top.
relpath (str): The project's path relative to the tree being synced.
Unlike Project.relpath, it is unique across submanifests.
remote_fetched (bool): True if the remote was actually queried.
fetch_success (bool): True if the fetch operation was successful.
fetch_errors (List[Exception]): The Exceptions from a failed fetch.
@@ -450,6 +598,8 @@ later is required to fix a server side protocol bug.
help="number of network jobs to run in parallel (defaults to "
"--jobs or 1). Ignored unless --no-interleaved is set",
)
jobs_checkout_default = self._GetHelpForCpuJobCount(DEFAULT_LOCAL_JOBS)
p.add_option(
"--jobs-checkout",
default=None,
@@ -457,7 +607,7 @@ later is required to fix a server side protocol bug.
metavar="JOBS",
help=(
"number of local checkout jobs to run in parallel (defaults "
f"to --jobs or {DEFAULT_LOCAL_JOBS}). Ignored unless "
f"to --jobs or {jobs_checkout_default}). Ignored unless "
"--no-interleaved is set"
),
)
@@ -678,9 +828,16 @@ later is required to fix a server side protocol bug.
"-s",
"--smart-sync",
action="store_true",
default=None,
help="smart sync using manifest from the latest known good "
"build",
)
p.add_option(
"--no-smart-sync",
dest="smart_sync",
action="store_false",
help="disable smart sync and sync to ToT instead",
)
p.add_option(
"-t",
"--smart-tag",
@@ -733,13 +890,42 @@ later is required to fix a server side protocol bug.
or opt.current_branch_only
)
def _GetSuperprojectRevision(
self, opt: optparse.Values, manifest: XmlManifest
) -> Optional[str]:
"""Returns the --superproject-revision to apply to manifest.
The option only applies to the outer manifest.
Args:
opt: Program options returned from optparse. See _Options().
manifest: The manifest to check.
Returns:
The requested superproject revision, or None if manifest's
superproject should not be pinned.
"""
if manifest == self.outer_manifest:
return opt.superproject_revision
return None
def _ConfigureSuperproject(
self,
opt: optparse.Values,
manifest,
revision: Optional[str] = None,
self, opt: optparse.Values, manifest: XmlManifest
) -> bool:
"""Configure superproject with options."""
"""Configure superproject with options.
The configuration lives on the manifest's Superproject object, which
is discarded whenever the manifest is unloaded (see _ReloadManifest
and XmlManifest.Unload). Callers must therefore configure the
superproject again after every manifest reload.
Args:
opt: Program options returned from optparse. See _Options().
manifest: The manifest whose superproject should be configured.
Returns:
Whether superproject messages should be printed.
"""
if not manifest.superproject:
return False
manifest.superproject.SetQuiet(not opt.verbose)
@@ -747,6 +933,7 @@ later is required to fix a server side protocol bug.
opt.use_superproject, manifest
)
manifest.superproject.SetPrintMessages(print_messages)
revision = self._GetSuperprojectRevision(opt, manifest)
if revision:
manifest.superproject.SetRevisionId(revision)
return print_messages
@@ -772,6 +959,11 @@ later is required to fix a server side protocol bug.
m.superproject for m in manifest.all_children
)
if not have_superproject:
superproject_logging_data.update(
superproject=False,
haslocalmanifests=bool(manifest.HasLocalManifests),
hassuperprojecttag=False,
)
return
if opt.local_only and manifest.superproject:
@@ -796,7 +988,6 @@ later is required to fix a server side protocol bug.
for p in all_projects:
per_manifest[p.manifest.path_prefix].append(p)
superproject_logging_data = {}
need_unload = False
for m in self.ManifestList(opt):
if m.path_prefix not in per_manifest:
@@ -837,12 +1028,8 @@ later is required to fix a server side protocol bug.
else:
if print_messages:
logger.warning(
"%s: warning: Update of revisionId from superproject "
"has failed, repo sync will not use superproject to "
"fetch the source. Please resync with the "
"--no-use-superproject option to avoid this repo "
"warning.",
m.path_prefix,
"error: Update of revsionId from superproject "
"has failed"
)
if update_result.fatal and opt.use_superproject is not None:
raise SuperprojectError()
@@ -1067,6 +1254,40 @@ later is required to fix a server side protocol bug.
return _FetchResult(ret, fetched)
def _FetchParentFirst(
self,
projects: List[Project],
opt: optparse.Values,
err_event: _threading.Event,
ssh_proxy: ssh.ProxyManager,
errors: List[Exception],
) -> _FetchResult:
"""Fetch |projects|, holding submodules back until their parent is done.
Args:
projects: Projects to fetch.
opt: Program options returned from optparse. See _Options().
err_event: Whether an error was hit while processing.
ssh_proxy: SSH manager for clients & masters.
errors: A list to accumulate errors.
Returns:
_FetchResult for all the batches combined.
"""
success = True
fetched = set()
for batch in _ParentFirstBatches(projects):
batch = _WithoutProjects(batch, _RefreshDerivedRevisions(batch))
if not batch:
continue
batch.sort(key=self._fetch_times.Get, reverse=True)
result = self._Fetch(batch, opt, err_event, ssh_proxy, errors)
success = success and result.success
fetched.update(result.projects)
if not success and opt.fail_fast:
break
return _FetchResult(success, fetched)
def _FetchMain(
self, opt, args, all_projects, err_event, ssh_proxy, manifest, errors
):
@@ -1083,12 +1304,10 @@ later is required to fix a server side protocol bug.
Returns:
List of all projects that should be checked out.
"""
to_fetch = []
to_fetch.extend(all_projects)
to_fetch.sort(key=self._fetch_times.Get, reverse=True)
try:
result = self._Fetch(to_fetch, opt, err_event, ssh_proxy, errors)
result = self._FetchParentFirst(
all_projects, opt, err_event, ssh_proxy, errors
)
success = result.success
fetched = result.projects
if not success:
@@ -1131,7 +1350,9 @@ later is required to fix a server side protocol bug.
if previously_missing_set == missing_set:
break
previously_missing_set = missing_set
result = self._Fetch(missing, opt, err_event, ssh_proxy, errors)
result = self._FetchParentFirst(
missing, opt, err_event, ssh_proxy, errors
)
success = result.success
new_fetched = result.projects
if not success:
@@ -1488,16 +1709,15 @@ later is required to fix a server side protocol bug.
"""
project = cls.get_parallel_context()["projects"][project_index]
if not project.Exists or not project.worktree:
return None
# Only check dirty or locally modified projects. These can't be
# freshly cloned and will accumulate garbage.
try:
is_dirty = project.IsDirty(consider_untracked=True)
is_dirty, head_rev = project.GetDirtyAndHead()
if head_rev is None:
# An unborn branch has no HEAD to compare, so skip it.
return None
manifest_rev = project.GetRevisionId(project.bare_ref.all)
head_rev = project.work_git.rev_parse(HEAD)
has_local_commits = manifest_rev != head_rev
if not (is_dirty or has_local_commits):
@@ -1538,16 +1758,27 @@ later is required to fix a server side protocol bug.
run 'git count-objects -v' and warn if the repository is accumulating
excessive pack files or garbage.
"""
# --network-only promises not to touch worktrees, but git status and
# update-index --refresh can both rewrite the index.
if opt.network_only:
return
# We only care about bloated projects if we have a git version that
# supports --no-auto-gc (2.23.0+) since what we use to disable auto-gc
# in Project._RemoteFetch.
if not git_require((2, 23, 0)):
return
# Skip projects with no worktree on disk, e.g. ones only ever synced
# with --network-only.
projects = [
p
for p in projects
if p.clone_depth and not p.stateless_prune_needed
if p.clone_depth
and not p.stateless_prune_needed
and p.worktree
and p.Exists
and platform_utils.isdir(p.worktree)
]
if not projects:
return
@@ -1921,7 +2152,13 @@ later is required to fix a server side protocol bug.
return server_url, transport
def _SmartSyncSetup(self, opt, smart_sync_manifest_path, manifest):
def _SmartSyncSetup(
self,
opt: optparse.Values,
smart_sync_manifest_path: str,
manifest: XmlManifest,
best_effort: bool = False,
) -> str:
if not manifest.manifest_server:
raise SmartSyncError(
"error: cannot smart sync: no manifest server defined in "
@@ -1939,7 +2176,12 @@ later is required to fix a server side protocol bug.
try:
server = xmlrpc.client.Server(server_url, transport=transport)
if opt.smart_sync:
branch = self._GetBranch(manifest.manifestProject)
# SYNC_BRANCH overrides the branch sent to the manifest server
# (e.g. to select a specific CI branch that builds the same
# manifest branch). An empty value is treated as unset.
branch = os.environ.get("SYNC_BRANCH")
if not branch:
branch = self._GetBranch(manifest.manifestProject)
target = None
if "SYNC_TARGET" in os.environ:
target = os.environ["SYNC_TARGET"]
@@ -1962,12 +2204,9 @@ later is required to fix a server side protocol bug.
os.environ["TARGET_BUILD_VARIANT"],
)
if target:
[success, manifest_str] = server.GetApprovedManifest(
branch, target
)
else:
[success, manifest_str] = server.GetApprovedManifest(branch)
success, manifest_str = _GetApprovedManifest(
server, branch, target, best_effort
)
else:
assert opt.smart_tag
[success, manifest_str] = server.GetManifest(opt.smart_tag)
@@ -2023,7 +2262,7 @@ later is required to fix a server side protocol bug.
mp: the manifestProject to query.
manifest_name: Manifest file to be reloaded.
"""
if opt.superproject_revision and mp.manifest == self.outer_manifest:
if self._GetSuperprojectRevision(opt, mp.manifest):
self._SyncToSuperprojectRev(
opt, mp.manifest, mp, manifest_name, errors
)
@@ -2119,10 +2358,6 @@ later is required to fix a server side protocol bug.
if opt.manifest_name and opt.smart_tag:
self.OptionParser.error("cannot combine -m and -t")
if opt.manifest_server_username or opt.manifest_server_password:
if not (opt.smart_sync or opt.smart_tag):
self.OptionParser.error(
"-u and -p may only be combined with -s or -t"
)
if None in [
opt.manifest_server_username,
opt.manifest_server_password,
@@ -2239,18 +2474,19 @@ later is required to fix a server side protocol bug.
def _SyncToSuperprojectRev(
self,
opt: optparse.Values,
manifest,
manifest: XmlManifest,
mp: Project,
manifest_name: Optional[str],
errors: List[Exception],
) -> None:
"""Sync to a specific superproject commit."""
"""Sync to a specific superproject commit.
Only valid for the outer manifest; see _GetSuperprojectRevision.
"""
if not manifest.superproject:
raise SyncError("superproject not defined in manifest")
self._ConfigureSuperproject(
opt, manifest, revision=opt.superproject_revision
)
self._ConfigureSuperproject(opt, manifest)
sync_result = manifest.superproject.Sync(self.git_event_log)
if not sync_result.success:
@@ -2283,6 +2519,33 @@ later is required to fix a server side protocol bug.
"failed to sync manifest project", aggregate_errors=[e]
)
def _ResolveSmartSyncOption(
self, opt: optparse.Values, manifest: XmlManifest
) -> bool:
"""Resolve opt.smart_sync from the CLI flags and manifest default.
Returns:
True if smart_sync was enabled implicitly by the manifest default,
False otherwise.
"""
implicit = (
opt.smart_sync is None
and not opt.smart_tag
and not opt.manifest_name
and not opt.superproject_revision
and not opt.local_only
and getattr(manifest.default, "sync_smartsync", False) is True
)
if opt.smart_sync is None:
opt.smart_sync = implicit
if (
opt.manifest_server_username or opt.manifest_server_password
) and not (opt.smart_sync or opt.smart_tag):
raise SmartSyncError("-u and -p may only be combined with -s or -t")
return implicit
def _ExecuteHelper(self, opt, args, errors):
manifest = self.outer_manifest
if not opt.outer_manifest:
@@ -2297,11 +2560,62 @@ later is required to fix a server side protocol bug.
if opt.clone_bundle is None:
opt.clone_bundle = manifest.CloneBundle
# Without an explicit smart sync option, -l does not contact the
# manifest server, so it keeps the smart sync manifest the client last
# synced to, whether that came from -s or from the manifest's
# sync-smartsync default. This keeps `sync -n` + `sync -l` equivalent
# to a single sync. Computed before _ResolveSmartSyncOption, which
# makes an unset opt.smart_sync indistinguishable from --no-smart-sync.
reuse_smart_sync_override = (
opt.local_only
and opt.smart_sync is None
and not opt.smart_tag
and not opt.manifest_name
and not opt.superproject_revision
and os.path.isfile(smart_sync_manifest_path)
)
implicit_smart_sync = self._ResolveSmartSyncOption(opt, manifest)
if opt.smart_sync or opt.smart_tag:
manifest_name = self._SmartSyncSetup(
opt, smart_sync_manifest_path, manifest
)
else:
try:
manifest_name = self._SmartSyncSetup(
opt,
smart_sync_manifest_path,
manifest,
best_effort=implicit_smart_sync,
)
except (SmartSyncError, ManifestParseError) as e:
if not implicit_smart_sync:
raise
# The user did not ask for a smart sync, so failing to get a
# usable manifest from the server should not fail the sync.
# Fall back to a regular ToT sync instead.
opt.smart_sync = False
# Override() records the override before parsing it, so an
# unparsable manifest would otherwise leave the client pinned
# to a file it cannot load.
manifest.ClearOverride()
if opt.verbose:
logger.warning(
"warning: smart sync failed; falling back to ToT: %s", e
)
elif reuse_smart_sync_override:
manifest_name = os.path.basename(smart_sync_manifest_path)
try:
self._ReloadManifest(manifest_name, manifest)
except ManifestParseError as e:
reuse_smart_sync_override = False
manifest_name = None
manifest.ClearOverride()
if opt.verbose:
logger.warning(
"warning: failed to load existing smart sync manifest; "
"falling back to ToT: %s",
e,
)
if not (opt.smart_sync or opt.smart_tag or reuse_smart_sync_override):
if os.path.isfile(smart_sync_manifest_path):
try:
platform_utils.remove(smart_sync_manifest_path)
@@ -2366,6 +2680,7 @@ later is required to fix a server side protocol bug.
manifest=manifest,
all_manifests=not opt.this_manifest_only,
)
self._CheckReprojectCmdNesting(opt, args, manifest, all_projects)
# Log the repo projects by existing and new.
existing = [x for x in all_projects if x.Exists]
@@ -2429,6 +2744,46 @@ later is required to fix a server side protocol bug.
if not opt.quiet:
print("repo sync has finished successfully.")
def _CheckReprojectCmdNesting(
self,
opt: optparse.Values,
args: List[str],
manifest: XmlManifest,
all_projects: List[Project],
) -> None:
"""Fail when repo.reprojectcmd is used on a manifest nesting projects.
The command materializes a project's tree without Git, so nothing
keeps it from clobbering a project or submodule checked out inside
that tree. See docs/reproject-cmd.md.
"""
if not any(p.UseReprojectCmd for p in all_projects):
return
projects = all_projects
if args:
# Nesting is a property of the manifest, not of the projects
# picked on the command line.
projects = self.GetProjects(
[],
groups=opt.groups,
missing_ok=True,
submodules_ok=opt.recurse_submodules,
manifest=manifest,
all_manifests=not opt.this_manifest_only,
)
nested = _NestedProjects(projects)
if not nested:
return
e = SyncError(
"error: repo.reprojectcmd does not support nested projects or "
"submodules; found:\n"
+ "\n".join(
f" - {p.RelPath(local=opt.this_manifest_only)}" for p in nested
)
)
logger.error(e)
raise e
def _CreateSyncProgressThread(
self, pm: Progress, stop_event: _threading.Event
) -> _threading.Thread:
@@ -2758,7 +3113,7 @@ later is required to fix a server side protocol bug.
return _SyncResult(
project_index=project_index,
relpath=project.relpath,
relpath=project.RelPath(local=opt.this_manifest_only),
fetch_success=fetch_success,
remote_fetched=remote_fetched,
checkout_success=checkout_success,
@@ -2906,6 +3261,10 @@ later is required to fix a server side protocol bug.
self._interleaved_err_checkout = False
self._interleaved_err_checkout_results = []
# Project.relpath is relative to its own (sub)manifest, so it does not
# tell apart projects of different manifests being synced together.
_RelPath = lambda p: p.RelPath(local=opt.this_manifest_only)
err_event = multiprocessing.Event()
finished_relpaths = set()
project_list = list(all_projects)
@@ -2942,13 +3301,13 @@ later is required to fix a server side protocol bug.
projects_to_sync = [
p
for p in project_list
if p.relpath not in finished_relpaths
if _RelPath(p) not in finished_relpaths
]
if not projects_to_sync:
break
pending_relpaths = {
p.relpath for p in projects_to_sync
_RelPath(p) for p in projects_to_sync
}
if previously_pending_relpaths == pending_relpaths:
stalled_projects_str = "\n".join(
@@ -2977,12 +3336,22 @@ later is required to fix a server side protocol bug.
# projects in one level can be processed in
# parallel, but we must wait for a level to complete
# before starting the next.
submodule_revisions = {}
for level_projects in _SafeCheckoutOrder(
projects_to_sync
):
if not level_projects:
continue
level_projects = _WithoutProjects(
level_projects,
_RefreshDerivedRevisions(
level_projects, submodule_revisions
),
)
if not level_projects:
continue
objdir_project_map = collections.defaultdict(
list
)
@@ -3184,13 +3553,13 @@ class LocalSyncState:
def _Get(self, project, key):
self._Load()
p = project.relpath
p = project.RelPath(local=False)
if p not in self._state:
return
return self._state[p].get(key)
def _Set(self, project, key):
p = project.relpath
p = project.RelPath(local=False)
if p not in self._state:
self._state[p] = {}
self._state[p][key] = self._time
@@ -3218,8 +3587,9 @@ class LocalSyncState:
if not self._state:
return
delete = set()
outer_topdir = self._manifest.outer_client.topdir
for path in self._state:
gitdir = os.path.join(self._manifest.topdir, path, ".git")
gitdir = os.path.join(outer_topdir, path, ".git")
if not os.path.exists(gitdir) or os.path.islink(gitdir):
delete.add(path)
if not delete:
+29 -22
View File
@@ -25,7 +25,6 @@ from editor import Editor
from error import GitError
from error import SilentRepoExitError
from error import UploadError
from git_command import GitCommand
from git_refs import R_HEADS
import git_superproject
from hooks import RepoHook
@@ -379,7 +378,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
default=True,
help="disable verifying ssl certs (unsafe)",
)
RepoHook.AddOptionGroup(p, "pre-upload")
RepoHook.AddOptionGroup(p, "pre-upload", allow_fix=True)
def _SingleBranch(self, opt, branch, people):
project = branch.project
@@ -436,6 +435,18 @@ Gerrit Code Review: https://www.gerritcodereview.com/
self._UploadAndReport(opt, [branch], people)
def _MultipleBranches(self, opt, pending, people):
if opt.yes and (opt.current_branch or opt.branch):
todo = [
branch
for _, avail in pending
for branch in avail
if branch is not None
]
if not todo:
_die("nothing ready for upload")
self._UploadAndReport(opt, todo, people)
return
projects = {}
branches = {}
@@ -649,6 +660,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
validate_certs=opt.validate_certs,
push_options=push_options,
patchset_description=opt.patchset_description,
git_event_log=self.git_event_log,
)
branch.uploaded = True
@@ -704,36 +716,31 @@ Gerrit Code Review: https://www.gerritcodereview.com/
raise UploadExitError(aggregate_errors=aggregate_errors)
def _GetMergeBranch(self, project, local_branch=None):
"""Get the merge branch name for a local branch.
Resolves the merge branch in-memory via project configuration to
avoid git subprocess overhead during upload.
"""
if local_branch is None:
p = GitCommand(
project,
["rev-parse", "--abbrev-ref", "HEAD"],
capture_stdout=True,
capture_stderr=True,
)
p.Wait()
local_branch = p.stdout.strip()
p = GitCommand(
project,
["config", "--get", "branch.%s.merge" % local_branch],
capture_stdout=True,
capture_stderr=True,
)
p.Wait()
merge_branch = p.stdout.strip()
return merge_branch
local_branch = project.CurrentBranch
if local_branch:
branch = project.GetBranch(local_branch)
if branch.merge:
return branch.merge
return ""
@classmethod
def _GatherOne(cls, opt, project_idx):
"""Figure out the upload status for |project|."""
project = cls.get_parallel_context()["projects"][project_idx]
cbr = None
if opt.current_branch:
cbr = project.CurrentBranch
up_branch = project.GetUploadableBranch(cbr)
avail = [up_branch] if up_branch else None
else:
avail = project.GetUploadableBranches(opt.branch)
return (project_idx, avail)
return (project_idx, avail, cbr)
def Execute(self, opt, args):
projects = self.GetProjects(
@@ -743,7 +750,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
def _ProcessResults(_pool, _out, results):
pending = []
for result in results:
project_idx, avail = result
project_idx, avail, current_branch = result
project = projects[project_idx]
if avail is None:
logger.error(
@@ -751,7 +758,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
"You might be able to fix the branch by running:\n"
" git branch --set-upstream-to m/%s",
project.RelPath(local=opt.this_manifest_only),
project.CurrentBranch,
current_branch,
project.manifest.branch,
)
elif avail:
+20 -2
View File
@@ -14,10 +14,12 @@
import platform
import sys
from typing import Any, Tuple
from command import Command
from command import MirrorSafeCommand
from git_command import git
from git_command import git_require
from git_command import RepoSourceVersion
from git_command import user_agent
from git_refs import HEAD
@@ -34,6 +36,22 @@ class Version(Command, MirrorSafeCommand):
%prog
"""
@staticmethod
def _RepoVersion(project: Any) -> Tuple[str, str]:
"""Return repo's describe string and commit date."""
if git_require((2, 32, 0)):
output = project.bare_git.log(
"-1", "--format=%(describe)%n%cD", HEAD
)
description, commit_date = output.rstrip("\n").split("\n", 1)
if description:
return description, commit_date
return (
project.bare_git.describe(HEAD),
project.bare_git.log("-1", "--format=%cD", HEAD),
)
def Execute(self, opt, args):
rp = self.manifest.repoProject
rem = rp.GetRemote()
@@ -41,11 +59,11 @@ class Version(Command, MirrorSafeCommand):
# These might not be the same. Report them both.
src_ver = RepoSourceVersion()
rp_ver = rp.bare_git.describe(HEAD)
rp_ver, commit_date = self._RepoVersion(rp)
print(f"repo version {rp_ver}")
print(f" (from {rem.url})")
print(f" (tracking {branch.merge})")
print(f" ({rp.bare_git.log('-1', '--format=%cD', HEAD)})")
print(f" ({commit_date})")
if self.wrapper_path is not None:
print(f"repo launcher version {self.wrapper_version}")
+1
View File
@@ -26,6 +26,7 @@ import repo_trace
@pytest.fixture(autouse=True)
def disable_repo_trace(tmp_path):
"""Set an environment marker to relax certain strict checks for test code.""" # noqa: E501
repo_trace.SetTrace(False)
repo_trace._TRACE_FILE = str(tmp_path / "TRACE_FILE_from_test")
+234 -4
View File
@@ -14,6 +14,8 @@
"""Unittests for the command.py module."""
from typing import Iterable, List, Optional
import pytest
from command import Command
@@ -30,12 +32,15 @@ class FakeProject:
gitdir=None,
derived_subprojects=None,
sync_s=False,
exists: bool = True,
):
self.name = name
self.relpath = relpath
self.worktree = f"/work/{relpath}"
self.manifest = None
self.gitdir = gitdir or f"/git/{relpath}"
self.sync_s = sync_s
self.Exists = True
self.Exists = exists
self._derived_subprojects = derived_subprojects or []
def GetDerivedSubprojects(self):
@@ -51,11 +56,59 @@ class FakeProject:
class FakeManifest:
"""Minimal manifest double for Command.GetProjects tests."""
def __init__(self, projects):
self.projects = projects
def __init__(
self,
projects: Iterable[FakeProject],
*,
all_projects: Optional[Iterable[FakeProject]] = None,
effective_groups: str = "default",
):
self.projects = list(projects)
self.all_projects = (
list(self.projects) if all_projects is None else list(all_projects)
)
self._effective_groups = effective_groups
self.path_prefix = ""
# all_projects may include projects owned by child manifests,
# so only set this manifest on its direct projects.
for project in self.projects:
self._set_project_manifest(project)
def _set_project_manifest(self, project: FakeProject) -> None:
project.manifest = self
for subproject in project.GetDerivedSubprojects():
self._set_project_manifest(subproject)
def GetManifestGroupsStr(self):
return "default"
return self._effective_groups
def GetProjectsWithName(
self, name: str, all_manifests: bool = False
) -> List[FakeProject]:
projects = self.all_projects if all_manifests else self.projects
return [project for project in projects if project.name == name]
class GroupMatchingFakeProject(FakeProject):
"""Fake project with predictable group matches for GetProjects tests.
This lets the tests check which groups GetProjects uses without
reimplementing Project.MatchesGroups.
"""
def __init__(
self,
name: str,
relpath: str,
*,
matching_groups: Iterable[str],
):
super().__init__(name, relpath)
self._matching_groups = set(matching_groups)
def MatchesGroups(self, groups: Iterable[str]) -> bool:
return bool(self._matching_groups.intersection(groups))
def test_get_projects_keeps_derived_subprojects_for_repeated_repo():
@@ -117,3 +170,180 @@ def test_get_projects_submodule_override(
projects = cmd.GetProjects([], submodules_ok=submodules_ok)
assert (submodule in projects) is includes_submodule
@pytest.mark.parametrize(
("groups", "expected_relpaths"),
[
(None, ["outer", "sub/child"]),
("", ["outer", "sub/child"]),
("override-group", ["sub/override"]),
],
ids=("groups-omitted", "groups-empty", "explicit-override"),
)
def test_get_projects_uses_groups_from_each_manifest_unless_overridden(
groups: Optional[str],
expected_relpaths: List[str],
) -> None:
"""Use each manifest's effective groups unless the caller overrides them."""
outer_project = GroupMatchingFakeProject(
"outer",
"outer",
matching_groups={"outer-group"},
)
# Both child projects also match "outer". Reusing the outer manifest's
# groups would therefore select both child projects.
child_project = GroupMatchingFakeProject(
"child",
"sub/child",
matching_groups={"outer-group", "child-group"},
)
override_project = GroupMatchingFakeProject(
"override",
"sub/override",
matching_groups={"outer-group", "override-group"},
)
child_manifest = FakeManifest(
[child_project, override_project],
effective_groups="child-group",
)
outer_manifest = FakeManifest(
[outer_project],
all_projects=[outer_project, *child_manifest.projects],
effective_groups="outer-group",
)
cmd = Command(manifest=outer_manifest)
projects = cmd.GetProjects(
[],
manifest=outer_manifest,
groups=groups,
all_manifests=True,
)
assert [project.relpath for project in projects] == expected_relpaths
def test_get_projects_by_name_uses_groups_from_each_manifest() -> None:
"""Name matches use the groups from each project's owning manifest."""
outer_project = GroupMatchingFakeProject(
"shared",
"outer/shared",
matching_groups={"outer-group"},
)
child_project = GroupMatchingFakeProject(
"shared",
"sub/shared",
matching_groups={"child-group"},
)
child_manifest = FakeManifest(
[child_project],
effective_groups="child-group",
)
outer_manifest = FakeManifest(
[outer_project],
all_projects=[outer_project, *child_manifest.projects],
effective_groups="outer-group",
)
cmd = Command(manifest=outer_manifest)
projects = cmd.GetProjects(
["shared"],
manifest=outer_manifest,
all_manifests=True,
)
assert [project.relpath for project in projects] == [
"outer/shared",
"sub/shared",
]
def test_find_projects_uses_groups_from_each_manifest() -> None:
"""Use each manifest's effective groups for regex selection."""
outer_project = GroupMatchingFakeProject(
"match-outer",
"outer",
matching_groups={"outer-group"},
)
child_project = GroupMatchingFakeProject(
"match-child",
"sub/child",
matching_groups={"child-group"},
)
excluded_project = GroupMatchingFakeProject(
"match-excluded",
"sub/excluded",
matching_groups={"other-group"},
)
child_manifest = FakeManifest(
[child_project, excluded_project],
effective_groups="child-group",
)
child_manifest.path_prefix = "sub"
outer_manifest = FakeManifest(
[outer_project],
all_projects=[
outer_project,
*child_manifest.projects,
],
effective_groups="outer-group",
)
outer_manifest.outer_client = outer_manifest
cmd = Command(manifest=outer_manifest)
projects = cmd.FindProjects(["match"], all_manifests=True)
assert [project.relpath for project in projects] == [
"outer",
"sub/child",
]
def test_find_projects_uses_explicit_groups() -> None:
"""Use explicit groups for regex selection."""
default_project = GroupMatchingFakeProject(
"default",
"default",
matching_groups={"default-group"},
)
override_project = GroupMatchingFakeProject(
"override",
"override",
matching_groups={"override-group"},
)
manifest = FakeManifest(
[default_project, override_project],
effective_groups="default-group",
)
cmd = Command(manifest=manifest)
projects = cmd.FindProjects(
["override"],
groups="override-group",
)
assert projects == [override_project]
def test_find_projects_allows_missing_projects() -> None:
"""Allow regex selection to include projects without a checkout."""
project = FakeProject(
"missing",
"missing",
exists=False,
)
cmd = Command(manifest=FakeManifest([project]))
projects = cmd.FindProjects(
["missing"],
missing_ok=True,
)
assert projects == [project]
+154
View File
@@ -123,6 +123,7 @@ class GitCommandStreamLogsTest(unittest.TestCase):
"""Tests the GitCommand class stderr log streaming cases."""
def setUp(self):
_ = git_command.user_agent.git
self.mock_process = mock.MagicMock()
self.mock_process.communicate.return_value = (None, None)
self.mock_process.wait.return_value = 0
@@ -228,9 +229,162 @@ class GitCommandStreamLogsTest(unittest.TestCase):
self.assertEqual(cmd.stderr, logs)
class GitCommandCaptureBytesTest(unittest.TestCase):
"""Tests the GitCommand class byte capture cases."""
def setUp(self) -> None:
_ = git_command.user_agent.git
self.mock_process = mock.MagicMock()
self.mock_process.communicate.return_value = (None, None)
self.mock_process.wait.return_value = 0
self.mock_popen = mock.MagicMock()
self.mock_popen.return_value = self.mock_process
mock.patch("subprocess.Popen", self.mock_popen).start()
def tearDown(self) -> None:
mock.patch.stopall()
def test_captures_stdout_as_bytes(self) -> None:
self.mock_process.communicate.return_value = (b"\xff\x00", b"error\r\n")
cmd = git_command.GitCommand(
None,
["status"],
capture_stdout=True,
capture_stdout_bytes=True,
capture_stderr=True,
)
self.mock_popen.assert_called_once_with(
["git", "status"],
cwd=None,
env=mock.ANY,
stdin=None,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
self.assertEqual(cmd.stdout, b"\xff\x00")
self.assertEqual(cmd.stderr, "error\n")
def test_capture_stdout_bytes_auto_enables_capture_stdout(self) -> None:
self.mock_process.communicate.return_value = (b"output", b"")
cmd = git_command.GitCommand(
None,
["status"],
capture_stdout_bytes=True,
)
self.mock_popen.assert_called_once_with(
["git", "status"],
cwd=None,
env=mock.ANY,
stdin=None,
stdout=subprocess.PIPE,
stderr=None,
)
self.assertEqual(cmd.stdout, b"output")
def test_capture_stdout_bytes_with_merge_output_raises(self) -> None:
with self.assertRaises(ValueError):
git_command.GitCommand(
None,
["status"],
capture_stdout_bytes=True,
merge_output=True,
)
def test_captures_stdout_as_bytes_encodes_str_input(self) -> None:
self.mock_process.communicate.return_value = (b"output", b"")
git_command.GitCommand(
None,
["status"],
input="hello world",
capture_stdout_bytes=True,
)
self.mock_process.communicate.assert_called_once_with(
input=b"hello world"
)
def test_captures_stdout_as_bytes_encodes_surrogate_input(self) -> None:
self.mock_process.communicate.return_value = (b"output", b"")
git_command.GitCommand(
None,
["status"],
input="file_\udcff.txt",
capture_stdout_bytes=True,
)
self.mock_process.communicate.assert_called_once_with(
input=b"file_\xff.txt"
)
def test_captures_stdout_as_bytes_passes_bytes_input(self) -> None:
self.mock_process.communicate.return_value = (b"output", b"")
git_command.GitCommand(
None,
["status"],
input=b"raw_\xff.txt",
capture_stdout_bytes=True,
)
self.mock_process.communicate.assert_called_once_with(
input=b"raw_\xff.txt"
)
def test_verify_command_truncates_nul_delimited_stdout(self) -> None:
cmd = git_command.GitCommand(
None,
["status"],
capture_stdout_bytes=True,
)
cmd.rc = 1
cmd.stdout = b"first_file\0second_file\0third_file"
cmd.stderr = "stderr"
with self.assertRaises(git_command.GitCommandError) as cm:
cmd.VerifyCommand()
self.assertEqual(cm.exception.git_stdout, "first_file")
def test_verify_command_decodes_bytes_stdout(self) -> None:
cmd = git_command.GitCommand(
None,
["status"],
capture_stdout_bytes=True,
)
cmd.rc = 1
cmd.stdout = b"error\xff\nline2"
cmd.stderr = "stderr"
with self.assertRaises(git_command.GitCommandError) as cm:
cmd.VerifyCommand()
self.assertEqual(cm.exception.git_stdout, "error\\xff")
class GitCallUnitTest(unittest.TestCase):
"""Tests the _GitCall class (via git_command.git)."""
def test_valid_branch_name_uses_branch_mode(self) -> None:
"""Branch validation applies Git's branch-specific restrictions."""
command = mock.MagicMock()
command.Wait.return_value = 1
with mock.patch.object(
git_command, "GitCommand", return_value=command
) as check:
self.assertFalse(git_command.IsValidBranchName("-topic"))
check.assert_called_once_with(
None,
["check-ref-format", "--branch", "-topic"],
capture_stdout=True,
capture_stderr=True,
add_event_log=False,
log_as_error=False,
)
def test_version_tuple(self):
"""Check git.version_tuple() handling."""
ver = git_command.git.version_tuple()
+22 -2
View File
@@ -256,8 +256,8 @@ def test_remote_save_with_push_url_without_projectname(
("0" * 64, True),
("f" * 64, True),
("a" * 39, False),
("a" * 41, True),
("a" * 63, True),
("a" * 41, False),
("a" * 63, False),
("a" * 65, False),
("g" * 40, False),
("g" * 64, False),
@@ -268,3 +268,23 @@ def test_remote_save_with_push_url_without_projectname(
def test_is_id(rev: str, expected: bool) -> None:
"""Test IsId identifies both SHA-1 and SHA-256 hashes."""
assert git_config.IsId(rev) == expected
def test_repo_config_for_user_is_not_shared_with_git_config(
monkeypatch: pytest.MonkeyPatch, tmp_home_dir: Path
) -> None:
"""RepoConfig.ForUser() must not reuse GitConfig.ForUser()'s cache."""
monkeypatch.setattr(git_config.GitConfig, "_ForUser", None)
monkeypatch.setattr(git_config.RepoConfig, "_ForUser", None)
monkeypatch.delenv("REPO_CONFIG_DIR", raising=False)
# Populate the GitConfig cache first, as manifest loading does.
git_cfg = git_config.GitConfig.ForUser()
repo_cfg = git_config.RepoConfig.ForUser()
assert repo_cfg is not git_cfg
assert git_cfg.file == str(tmp_home_dir / ".gitconfig")
assert repo_cfg.file == str(tmp_home_dir / ".repoconfig" / "config")
# And the RepoConfig instance is cached on its own class.
assert git_config.RepoConfig.ForUser() is repo_cfg
assert git_config.GitConfig.ForUser() is git_cfg
+148 -2
View File
@@ -17,6 +17,8 @@
import os
from pathlib import Path
import subprocess
from typing import Any, List
from unittest import mock
import pytest
import utils_for_test
@@ -24,7 +26,7 @@ import utils_for_test
import git_refs
def _run(repo, *args):
def _run(repo: str, *args: str) -> str:
return subprocess.run(
["git", "-C", repo, *args],
stdout=subprocess.PIPE,
@@ -34,7 +36,7 @@ def _run(repo, *args):
).stdout.strip()
def _init_repo(tmp_path, reftable=False):
def _init_repo(tmp_path: Path, reftable: bool = False) -> str:
repo = os.path.join(tmp_path, "repo")
ref_format = "reftable" if reftable else "files"
utils_for_test.init_git_tree(repo, ref_format=ref_format)
@@ -57,10 +59,154 @@ def test_reads_refs(tmp_path, reftable):
branch = _run(repo, "symbolic-ref", "--short", "HEAD")
head = _run(repo, "rev-parse", "HEAD")
assert refs.symref("HEAD") == f"refs/heads/{branch}"
assert refs.head == f"refs/heads/{branch}"
assert refs.get("HEAD") == head
assert refs.get(f"refs/heads/{branch}") == head
@pytest.mark.parametrize("reftable", [False, True])
def test_reads_detached_head(tmp_path: Path, reftable: bool) -> None:
if reftable and not utils_for_test.supports_reftable():
pytest.skip("reftable not supported")
repo = _init_repo(tmp_path, reftable=reftable)
head = _run(repo, "rev-parse", "HEAD")
_run(repo, "checkout", "--detach", head)
refs = git_refs.GitRefs(os.path.join(repo, ".git"))
assert refs.symref("HEAD") == ""
assert refs.head == head
assert refs.get("HEAD") == head
def test_reads_head_with_root_refs_in_one_command(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Git 2.45 and newer include HEAD in the ref snapshot."""
head = "1" * 40
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **_kwargs: Any
) -> None:
commands.append(cmdv)
self.stdout = (
f"{head}\0HEAD\0refs/heads/main\n"
f"{head}\0refs/heads/main\0\n"
)
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
with mock.patch.object(refs, "_ReadSymbolicRef") as read_head:
assert refs.get("HEAD") == head
assert commands == [
[
"for-each-ref",
"--include-root-refs",
"--format=%(objectname)%00%(refname)%00%(symref)",
"HEAD",
"refs",
]
]
read_head.assert_not_called()
def test_root_ref_snapshot_falls_back_for_unborn_head(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""An unborn HEAD still uses symbolic-ref after the ref snapshot."""
class FakeGitCommand:
def __init__(
self, _project: Any, _cmdv: List[str], **_kwargs: Any
) -> None:
self.stdout = ""
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.symref("HEAD") == "refs/heads/main"
def test_old_git_keeps_separate_head_fallback(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Git before 2.45 uses the original for-each-ref and HEAD calls."""
head = "1" * 40
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **_kwargs: Any
) -> None:
commands.append(cmdv)
self.stdout = f"{head}\0refs/heads/main\0\n"
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: False)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.get("HEAD") == head
assert commands == [
[
"for-each-ref",
"--format=%(objectname)%00%(refname)%00%(symref)",
]
]
def test_for_each_ref_failure_falls_back_to_symbolic_ref(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""When for-each-ref fails, HEAD is still resolved via symbolic-ref."""
class FakeGitCommand:
def __init__(
self, _project: Any, _cmdv: List[str], **_kwargs: Any
) -> None:
self.stdout = ""
def Wait(self) -> int:
return 1
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.symref("HEAD") == "refs/heads/main"
@pytest.mark.parametrize("reftable", [False, True])
def test_updates_when_refs_change(tmp_path, reftable):
if reftable and not utils_for_test.supports_reftable():
+210
View File
@@ -0,0 +1,210 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the git_status.py module."""
import os
from typing import Any, List
from unittest import mock
import pytest
import git_status
def test_parse_porcelain_v2_branch_and_paths() -> None:
output = (
b"# branch.oid " + b"1" * 40 + b"\0"
b"# branch.head topic\0"
b"# branch.upstream origin/main\0"
b"# branch.ab +2 -3\0"
b"# stash 1\0"
b"1 M. N... 100644 100644 100644 "
+ b"1" * 40
+ b" "
+ b"2" * 40
+ b" staged name\0"
b"1 .M N... 100644 100644 100644 "
+ b"1" * 40
+ b" "
+ b"2" * 40
+ b" worktree name\0"
b"2 R. N... 100644 100644 100644 "
+ b"1" * 40
+ b" "
+ b"2" * 40
+ b" R075 renamed\0old name\0"
b"? untracked\0"
)
status = git_status.ParsePorcelainV2(output)
assert status.current_branch == "topic"
assert status.upstream == "origin/main"
assert (status.ahead, status.behind, status.stash_count) == (2, 3, 1)
assert status.index_changes["staged name"].status == "M"
assert status.worktree_changes["worktree name"].status == "M"
renamed = status.index_changes["renamed"]
assert (renamed.src_path, renamed.level) == ("old name", "75")
assert status.untracked == ["untracked"]
def test_parse_porcelain_v2_unmerged_and_non_utf8_path() -> None:
path = b"bad-\xff-name"
output = (
b"u UU N... 100644 100644 100644 100644 "
+ b"1" * 40
+ b" "
+ b"2" * 40
+ b" "
+ b"3" * 40
+ b" "
+ path
+ b"\0"
)
status = git_status.ParsePorcelainV2(output)
decoded = os.fsdecode(path)
assert status.index_changes[decoded].status == "U"
assert status.worktree_changes[decoded].status == "U"
assert os.fsencode(status.index_changes[decoded].path) == path
def test_untracked_only_respects_consider_untracked() -> None:
status = git_status.ParsePorcelainV2(b"? new file\0")
assert status.is_dirty()
assert not status.is_dirty(consider_untracked=False)
def test_branch_headers_preserve_non_ascii_names() -> None:
branch = "tópico"
status = git_status.ParsePorcelainV2(
b"# branch.oid " + b"1" * 40 + b"\0"
b"# branch.head " + os.fsencode(branch) + b"\0"
)
assert status.current_branch == branch
def test_quick_ahead_behind_is_recorded_as_unknown() -> None:
status = git_status.ParsePorcelainV2(b"# branch.ab +? -?\0")
assert (status.ahead, status.behind) == (0, 0)
assert not status.has_ahead_behind
def test_unknown_head_is_not_a_current_branch() -> None:
status = git_status.ParsePorcelainV2(b"# branch.head (unknown)\0")
assert status.current_branch is None
def test_malformed_output_is_rejected() -> None:
with pytest.raises(git_status.StatusParseError):
git_status.ParsePorcelainV2(b"2 R. truncated\0")
def test_malformed_branch_ab_is_rejected() -> None:
with pytest.raises(git_status.StatusParseError):
git_status.ParsePorcelainV2(b"# branch.ab not-a-valid-ab\0")
def test_ignored_records_are_skipped() -> None:
status = git_status.ParsePorcelainV2(b"! ignored_file\0")
assert not status.is_dirty()
assert status.untracked == []
def test_get_status_uses_versioned_machine_options(
monkeypatch: pytest.MonkeyPatch,
) -> None:
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **kwargs: Any
) -> None:
commands.append((cmdv, kwargs))
self.stdout = b""
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_status, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_status, "git_require", lambda _version: True)
git_status.GetStatus(
mock.sentinel.project,
mock.sentinel.gitdir,
untracked_files="no",
branch=True,
ahead_behind=True,
show_stash=True,
)
cmd, kwargs = commands[0]
assert cmd == [
"status",
"--porcelain=v2",
"-z",
"--ignore-submodules=all",
"--untracked-files=no",
"--branch",
"--ahead-behind",
"--renames",
"--show-stash",
]
assert kwargs["capture_stdout_bytes"]
def test_get_status_rejects_git_before_2_11(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(git_status, "git_require", lambda _version: False)
with pytest.raises(git_status.UnsupportedStatusError):
git_status.GetStatus(mock.sentinel.project, mock.sentinel.gitdir)
def test_get_status_omits_stash_header_before_2_35(
monkeypatch: pytest.MonkeyPatch,
) -> None:
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **_kwargs: Any
) -> None:
commands.append(cmdv)
self.stdout = b""
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_status, "GitCommand", FakeGitCommand)
monkeypatch.setattr(
git_status,
"git_require",
lambda version: version <= (2, 34, 0),
)
git_status.GetStatus(
mock.sentinel.project,
mock.sentinel.gitdir,
show_stash=True,
)
assert "--show-stash" not in commands[0]
+24 -4
View File
@@ -542,14 +542,15 @@ class SuperprojectTestCase(unittest.TestCase):
with mock.patch(
"git_superproject.GitCommand", autospec=True
) as mock_git_command:
with mock.patch(
"git_superproject.GitRefs.get", autospec=True
) as mock_git_refs:
with mock.patch.object(
self._superproject, "_GetRef"
) as get_ref:
instance = mock_git_command.return_value
instance.Wait.return_value = 0
mock_git_refs.side_effect = ["", "1234"]
get_ref.side_effect = ["", "1234"]
self.assertTrue(self._superproject._Fetch())
get_ref.assert_called_with("refs/heads/main")
self.assertEqual(
# TODO: Once we require Python 3.8+,
# use 'mock_git_command.call_args.args'.
@@ -572,6 +573,7 @@ class SuperprojectTestCase(unittest.TestCase):
# If branch for revision exists, set as --negotiation-tip.
self.assertTrue(self._superproject._Fetch())
get_ref.assert_called_with("refs/heads/main")
self.assertEqual(
# TODO: Once we require Python 3.8+,
# use 'mock_git_command.call_args.args'.
@@ -593,3 +595,21 @@ class SuperprojectTestCase(unittest.TestCase):
],
),
)
def test_GetRef_resolves_only_the_requested_ref(self) -> None:
command = mock.MagicMock(stdout="1234\n")
command.Wait.return_value = 0
with mock.patch(
"git_superproject.GitCommand", return_value=command
) as git_command:
self.assertEqual("1234", self._superproject._GetRef("HEAD"))
git_command.assert_called_once_with(
None,
["rev-parse", "--verify", "--quiet", "HEAD"],
gitdir=self._superproject._work_git,
bare=True,
capture_stdout=True,
capture_stderr=True,
log_as_error=False,
)
+5 -9
View File
@@ -33,7 +33,7 @@ import platform_utils
def server_logging_thread(
socket_path: str,
server_ready: threading.Condition,
server_ready: threading.Event,
received_traces: List[str],
) -> None:
"""Helper function to receive logs over a Unix domain socket.
@@ -43,8 +43,7 @@ def server_logging_thread(
Args:
socket_path: path to a Unix domain socket on which to listen for traces
server_ready: a threading.Condition used to signal to the caller that
this thread is ready to accept connections
server_ready: event set when the server is ready to accept connections
received_traces: a list to which received traces will be appended (after
decoding to a utf-8 string).
"""
@@ -53,8 +52,7 @@ def server_logging_thread(
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock:
sock.bind(socket_path)
sock.listen(0)
with server_ready:
server_ready.notify()
server_ready.set()
with sock.accept()[0] as conn:
while True:
recved = conn.recv(4096)
@@ -404,7 +402,7 @@ def test_write_socket(event_log: git_trace2_event_log.EventLog) -> None:
received_traces: List[str] = []
with tempfile.TemporaryDirectory(prefix="test_server_sockets") as tempdir:
socket_path = os.path.join(tempdir, "server.sock")
server_ready = threading.Condition()
server_ready = threading.Event()
# Start "server" listening on Unix domain socket at socket_path.
server_thread = threading.Thread(
target=server_logging_thread,
@@ -412,9 +410,7 @@ def test_write_socket(event_log: git_trace2_event_log.EventLog) -> None:
)
try:
server_thread.start()
with server_ready:
server_ready.wait(timeout=120)
server_ready.wait(timeout=120)
event_log.StartEvent([])
path = event_log.Write(path=f"af_unix:{socket_path}")
+64 -2
View File
@@ -15,6 +15,7 @@
"""Unittests for the hooks.py module."""
from io import StringIO
from pathlib import Path
import sys
import pytest
@@ -108,11 +109,11 @@ def test_post_sync_argument_validation() -> None:
@pytest.mark.parametrize("yes_val", (True, False))
def test_repo_upload_yes_arg(tmp_path, yes_val: bool) -> None:
def test_repo_upload_yes_arg(tmp_path: Path, yes_val: bool) -> None:
"""Test that yes is passed in kwargs during hook execution."""
class FakeProject:
def __init__(self, worktree):
def __init__(self, worktree: str) -> None:
self.worktree = worktree
self.enabled_repo_hooks = ["pre-upload"]
self.config = None
@@ -139,3 +140,64 @@ def main(project_list, **kwargs):
assert res is True
assert project_list == [yes_val]
@pytest.mark.parametrize("fix_val", (True, False))
def test_repo_upload_fix_arg(tmp_path: Path, fix_val: bool) -> None:
"""Test that fix is passed in kwargs during hook execution."""
class FakeProject:
def __init__(self, worktree: str) -> None:
self.worktree = worktree
self.enabled_repo_hooks = ["pre-upload"]
self.config = None
hook_file = tmp_path / "pre-upload.py"
hook_content = """
def main(project_list, **kwargs):
project_list.append(kwargs.get("fix"))
"""
hook_file.write_text(hook_content)
hook = hooks.RepoHook(
hook_type="pre-upload",
hooks_project=FakeProject(str(tmp_path)),
repo_topdir=str(tmp_path),
manifest_url="https://gerrit",
allow_all_hooks=True,
fix=fix_val,
)
project_list = []
res = hook.Run(project_list=project_list, worktree_list=[])
assert res is True
assert project_list == [fix_val]
def test_from_subcmd_without_fix_option() -> None:
"""Test that FromSubcmd works when opt does not have fix attribute."""
class Remote:
url = "https://gerrit"
class FakeManifest:
repo_hooks_project = None
topdir = "/fake/topdir"
class manifestProject:
@staticmethod
def GetRemote(name: str) -> "Remote":
return Remote()
class contactinfo:
bugurl = "https://bugs"
class FakeOpt:
bypass_hooks = False
allow_all_hooks = False
ignore_hooks = False
hook = hooks.RepoHook.FromSubcmd(FakeManifest(), FakeOpt(), "post-sync")
assert hook._fix is False
+56
View File
@@ -420,6 +420,62 @@ class TestXmlManifest:
)
manifest.ToXml()
def test_sync_smartsync(self, repo_client: RepoClient) -> None:
"""Check sync-smartsync handling."""
manifest = repo_client.get_xml_manifest(
"<manifest><default /></manifest>"
)
assert manifest.default.sync_smartsync is False
manifest = repo_client.get_xml_manifest(
'<manifest><default sync-smartsync="true" /></manifest>'
)
assert manifest.default.sync_smartsync is True
assert (
manifest.ToXml().toxml() == '<?xml version="1.0" ?>'
'<manifest><default sync-smartsync="true"/></manifest>'
)
manifest = repo_client.get_xml_manifest(
'<manifest><default sync-smartsync="false" /></manifest>'
)
assert manifest.default.sync_smartsync is False
assert manifest.ToXml().toxml() == '<?xml version="1.0" ?><manifest/>'
def test_clear_override(self, repo_client: RepoClient) -> None:
"""Check ClearOverride reverts to the default manifest."""
manifest_xml_fmt = (
'<manifest><remote name="r" fetch="." />'
'<default remote="r" revision="main" />'
'<project name="%s" /></manifest>'
)
manifest = repo_client.get_xml_manifest(manifest_xml_fmt % "default")
(repo_client.manifest_dir / "good.xml").write_text(
manifest_xml_fmt % "override", encoding="utf-8"
)
(repo_client.manifest_dir / "bad.xml").write_text(
"<manifest>", encoding="utf-8"
)
# No override in effect: a no-op.
manifest.ClearOverride()
assert list(manifest.paths) == ["default"]
manifest.Override("good.xml")
assert list(manifest.paths) == ["override"]
manifest.ClearOverride()
assert not manifest.manifestFileOverrides
assert list(manifest.paths) == ["default"]
# Override() registers the override before parsing it, so a failed
# Override() leaves the client pinned until ClearOverride().
with pytest.raises(error.ManifestParseError):
manifest.Override("bad.xml")
with pytest.raises(error.ManifestParseError):
manifest.Load()
manifest.ClearOverride()
assert list(manifest.paths) == ["default"]
class TestIncludeElement:
"""Tests for <include>."""
+2803 -44
View File
File diff suppressed because it is too large. Load diff
+43
View File
@@ -0,0 +1,43 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests for the release/update-manpages wrapper."""
import runpy
import sys
from types import ModuleType
import pytest
import utils_for_test
UPDATE_MANPAGES_SCRIPT = (
utils_for_test.THIS_DIR.parent / "release" / "update-manpages"
)
def test_wrapper_does_not_run_main_for_multiprocessing_child(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Do not rerun main when multiprocessing re-executes the wrapper."""
fake_update_manpages = ModuleType("update_manpages")
# Mock this because the real module requires newer Python versions than
# our unittest framework does.
monkeypatch.setitem(sys.modules, "update_manpages", fake_update_manpages)
runpy.run_path(
str(UPDATE_MANPAGES_SCRIPT),
run_name="__mp_main__",
)
+3
View File
@@ -25,6 +25,9 @@ def test_trace_max_size_enforced(monkeypatch: pytest.MonkeyPatch) -> None:
"""Check Trace behavior."""
content = "git chicken"
# Enable trace for the test, in case users have it disabled.
monkeypatch.setattr(repo_trace, "_TRACE", True)
with repo_trace.Trace(content, first_trace=True):
pass
first_trace_size = os.path.getsize(repo_trace._TRACE_FILE)
+85
View File
@@ -0,0 +1,85 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for subcmds/cherry_pick.py."""
from unittest import mock
import pytest
from error import GitError
from git_command import GitCommand
from subcmds import cherry_pick
def test_resolve_reference_uses_one_typed_batch_request(
monkeypatch: pytest.MonkeyPatch,
) -> None:
oid = "1" * 40
commit = "tree " + "2" * 40 + "\n\nSubject 🚀\n\nBody\n"
output = oid + " commit " + str(len(commit.encode("utf-8"))) + "\n"
output += commit + "\n"
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = output
command.stderr = ""
command.Wait.return_value = 0
run_git = mock.create_autospec(GitCommand, return_value=command)
monkeypatch.setattr(cherry_pick, "GitCommand", run_git)
resolved, contents = cherry_pick.CherryPick()._ResolveReference("topic")
assert resolved == oid
assert contents == commit
run_git.assert_called_once_with(
None,
["cat-file", "--batch"],
input="topic^{commit}\n",
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
def test_resolve_reference_rejects_missing_object(
monkeypatch: pytest.MonkeyPatch,
) -> None:
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = "topic^{commit} missing\n"
command.stderr = ""
command.Wait.return_value = 0
monkeypatch.setattr(
cherry_pick,
"GitCommand",
mock.create_autospec(GitCommand, return_value=command),
)
with pytest.raises(GitError, match="commit topic not found"):
cherry_pick.CherryPick()._ResolveReference("topic")
def test_resolve_reference_rejects_ambiguous_object(
monkeypatch: pytest.MonkeyPatch,
) -> None:
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = "topic^{commit} ambiguous\n"
command.stderr = ""
command.Wait.return_value = 0
monkeypatch.setattr(
cherry_pick,
"GitCommand",
mock.create_autospec(GitCommand, return_value=command),
)
with pytest.raises(GitError, match="commit topic not found"):
cherry_pick.CherryPick()._ResolveReference("topic")
+56
View File
@@ -17,7 +17,9 @@
import contextlib
import io
from pathlib import Path
from unittest import mock
import pytest
import utils_for_test
import manifest_xml
@@ -105,3 +107,57 @@ def test_forall_all_projects_called_once(tmp_path: Path) -> None:
line_count = sum(1 for x in output.splitlines() if x)
# Verify that we didn't get more lines than expected.
assert line_count == 8
@pytest.mark.parametrize(
("regex_option", "inverse"),
[
("-r", False),
("-i", True),
],
ids=("regex", "inverse-regex"),
)
def test_forall_regex_modes_pass_groups_to_find_projects(
tmp_path: Path,
regex_option: str,
inverse: bool,
) -> None:
"""Pass --groups through in regex modes."""
manifest = _create_manifest_with_8_projects(tmp_path)
cmd = subcmds.forall.Forall()
cmd.manifest = manifest
opts, args = cmd.OptionParser.parse_args(
[
regex_option,
"--groups",
"special",
"project",
"-c",
"true",
]
)
with mock.patch.object(
cmd,
"FindProjects",
return_value=[],
) as find_projects, mock.patch.object(
cmd,
"ExecuteInParallel",
return_value=0,
):
cmd.Execute(opts, args)
expected_kwargs = {
"groups": "special",
"all_manifests": True,
}
if inverse:
expected_kwargs["inverse"] = True
find_projects.assert_called_once_with(
["project"],
**expected_kwargs,
)
+107
View File
@@ -14,9 +14,11 @@
"""Unittests for the subcmds/gc.py module."""
from pathlib import Path
import unittest
from unittest import mock
import manifest_xml
from subcmds import gc
@@ -80,3 +82,108 @@ class GcCommand(unittest.TestCase):
ret = self.cmd.Execute(self.opt, [])
self.assertEqual(ret, 1)
self.mock_repack.assert_not_called()
def test_gc_keep_manifest_projects(tmp_path: Path) -> None:
"""Keep inactive projects that are still in the manifest."""
repodir = tmp_path / ".repo"
manifest_dir = repodir / "manifests"
manifest_file = repodir / manifest_xml.MANIFEST_FILE_NAME
repodir.mkdir()
manifest_dir.mkdir()
manifest_gitdir = repodir / "manifests.git"
manifest_gitdir.mkdir()
(manifest_gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
""",
encoding="utf-8",
)
manifest_file.write_text(
"""\
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="included" groups="active-group" />
<project name="excluded" groups="other-group" />
</manifest>
""",
encoding="utf-8",
)
manifest = manifest_xml.XmlManifest(str(repodir), str(manifest_file))
manifest.manifestProject.config.SetString(
"manifest.groups",
"active-group",
)
projects = {project.name: project for project in manifest.projects}
included = projects["included"]
excluded = projects["excluded"]
Path(included.gitdir).mkdir(parents=True)
Path(included.objdir).mkdir(parents=True)
# Simulate an inactive project with only its object cache left.
Path(excluded.objdir).mkdir(parents=True)
assert not excluded.Exists
orphan_gitdir = repodir / "projects" / "orphan.git"
orphan_objdir = repodir / "project-objects" / "orphan.git"
orphan_gitdir.mkdir()
orphan_objdir.mkdir()
cmd = gc.Gc(repodir=str(repodir), manifest=manifest)
opt, args = cmd.OptionParser.parse_args(
["--yes", "--keep-manifest-projects"]
)
opt.quiet = True
selected_projects = cmd.GetProjects([], all_manifests=True)
assert included in selected_projects
assert excluded not in selected_projects
cmd.Execute(opt, args)
assert included.Exists
assert Path(excluded.objdir).exists()
assert not orphan_gitdir.exists()
assert not orphan_objdir.exists()
def test_gc_keep_manifest_projects_preserves_repack_targets() -> None:
"""Keep manifest projects without expanding repack targets."""
cmd = gc.Gc()
opt, _ = cmd.OptionParser.parse_args(
["--keep-manifest-projects", "--repack"]
)
opt.this_manifest_only = False
with mock.patch.object(
cmd,
"GetProjects",
side_effect=[["projA"], ["manifest_projects"]],
) as get_projects, mock.patch.object(
cmd, "delete_unused_projects", return_value=0
) as delete, mock.patch.object(
cmd, "repack_projects", return_value=0
) as repack:
cmd.Execute(opt, ["projA"])
get_projects.assert_has_calls(
[
mock.call(["projA"], all_manifests=True),
mock.call(
[],
groups="all",
missing_ok=True,
all_manifests=True,
),
]
)
delete.assert_called_once_with(["manifest_projects"], opt)
repack.assert_called_once_with(["projA"], opt)
+116 -2
View File
@@ -208,7 +208,7 @@ def test_get_project_data_uses_head_revision() -> None:
project.name = "foo"
project.worktree = "/path/to/foo"
project.revisionExpr = "refs/heads/main"
project.GetBranches.return_value = []
project.GetBranches.return_value = {}
# GetHeadRevisionId() returns a SHA, it should be used.
project.GetHeadRevisionId.return_value = "head_sha_12345"
@@ -235,7 +235,9 @@ def test_json_with_projects(capsys) -> None:
project.name = "foo"
project.worktree = "/path/to/foo"
project.revisionExpr = "refs/heads/main"
project.GetBranches.return_value = {"branch1": mock.MagicMock()}
branch = mock.MagicMock()
branch.current = True
project.GetBranches.return_value = {"branch1": branch}
project.GetHeadRevisionId.return_value = "head_sha_12345"
project.CurrentBranch = "branch1"
@@ -253,3 +255,115 @@ def test_json_with_projects(capsys) -> None:
assert project_data["manifest_revision"] == "refs/heads/main"
assert project_data["local_branches"] == ["branch1"]
assert project_data["current_branch"] == "branch1"
def test_diff_commits_uses_one_left_right_walk() -> None:
"""Local and remote commits are partitioned from one rev-list."""
project = mock.MagicMock()
project.work_git.rev_list.return_value = [
"<11111111 local commit",
">22222222 remote commit",
]
local, remote = info.Info._GetDiffCommits(project, "refs/remotes/m/main")
assert local == ["11111111 local commit"]
assert remote == ["22222222 remote commit"]
project.work_git.rev_list.assert_called_once_with(
"--left-right",
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
"HEAD...refs/remotes/m/main",
"--",
)
def test_diff_commits_falls_back_to_bare_git_when_no_worktree() -> None:
"""Bare or worktree-less projects fall back to bare_git for history walk."""
project = mock.MagicMock()
project.work_git = None
project.bare_git.rev_list.return_value = [
"<11111111 local commit",
">22222222 remote commit",
]
local, remote = info.Info._GetDiffCommits(project, "refs/remotes/m/main")
assert local == ["11111111 local commit"]
assert remote == ["22222222 remote commit"]
project.bare_git.rev_list.assert_called_once_with(
"--left-right",
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
"HEAD...refs/remotes/m/main",
"--",
)
def test_diff_commits_empty_output() -> None:
"""Empty rev-list output produces empty local and remote commit lists."""
project = mock.MagicMock()
project.work_git.rev_list.return_value = []
local, remote = info.Info._GetDiffCommits(project, "refs/remotes/m/main")
assert local == []
assert remote == []
def test_get_current_branch() -> None:
"""_GetCurrentBranch identifies the branch with current=True."""
b1 = mock.MagicMock(current=False)
b2 = mock.MagicMock(current=True)
assert info.Info._GetCurrentBranch({"b1": b1, "b2": b2}) == "b2"
assert info.Info._GetCurrentBranch({"b1": b1}) is None
assert info.Info._GetCurrentBranch({}) is None
def test_overview_helper_current_branch_filters_before_uploadable(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""_OverviewHelper only checks uploadable state for the current branch."""
project = mock.MagicMock()
project.RelPath.return_value = "proj"
b1 = mock.MagicMock(current=False)
b2 = mock.MagicMock(current=True)
project.GetBranches.return_value = {"b1": b1, "b2": b2}
uploadable = mock.MagicMock(commits=["c1"], date="2026-09-21")
uploadable.name = "b2"
project.GetUploadableBranch.return_value = uploadable
monkeypatch.setattr(
info.Info,
"get_parallel_context",
lambda: {"projects": [project]},
)
opt = mock.MagicMock(current_branch=True, this_manifest_only=False)
result = info.Info._OverviewHelper(0, opt)
project.GetUploadableBranch.assert_called_once_with("b2")
assert len(result) == 1
assert result[0].name == "b2"
assert result[0].is_current is True
def test_overview_helper_current_branch_detached_head_skips_uploadable(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""_OverviewHelper skips GetUploadableBranch when detached with -b."""
project = mock.MagicMock()
b1 = mock.MagicMock(current=False)
project.GetBranches.return_value = {"b1": b1}
monkeypatch.setattr(
info.Info,
"get_parallel_context",
lambda: {"projects": [project]},
)
opt = mock.MagicMock(current_branch=True, this_manifest_only=False)
result = info.Info._OverviewHelper(0, opt)
project.GetUploadableBranch.assert_not_called()
assert result == []
+57
View File
@@ -0,0 +1,57 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the list subcmd."""
from typing import List, Optional
from unittest import mock
import pytest
import subcmds
@pytest.mark.parametrize(
("extra_args", "expected_groups", "expected_missing_ok"),
[
(["--groups", "special"], "special", None),
(["--all"], None, True),
],
ids=("groups", "all"),
)
def test_list_regex_passes_groups_and_all(
extra_args: List[str],
expected_groups: Optional[str],
expected_missing_ok: Optional[bool],
) -> None:
"""Pass --groups and --all through in regex mode."""
cmd = subcmds.list.List()
opts, args = cmd.OptionParser.parse_args(
["--regex", *extra_args, "project"]
)
with mock.patch.object(
cmd,
"FindProjects",
return_value=[],
) as find_projects:
cmd.Execute(opts, args)
find_projects.assert_called_once_with(
["project"],
groups=expected_groups,
missing_ok=expected_missing_ok,
all_manifests=True,
)
+41
View File
@@ -14,6 +14,9 @@
"""Unittests for the subcmds/rebase.py module."""
import contextlib
import io
from types import SimpleNamespace
from unittest import mock
from error import GitError
@@ -48,3 +51,41 @@ def test_resolve_onto_manifest_fallback() -> None:
assert res == "main"
project.GetRemote.assert_called_once()
remote.ToLocal.assert_called_once_with("main")
def test_execute_delegates_autostash_to_rebase() -> None:
"""--auto-stash is one rebase process, including staged-only changes."""
cmd = rebase.Rebase()
cmd.manifest = mock.MagicMock()
cmd.git_event_log = mock.MagicMock()
project = mock.MagicMock()
project.CurrentBranch = "topic"
project.RelPath.return_value = "project"
branch = mock.MagicMock()
branch.LocalMerge = "refs/remotes/origin/main"
project.GetBranch.return_value = branch
cmd.GetProjects = mock.MagicMock(return_value=[project])
opt = SimpleNamespace(
interactive=False,
fail_fast=False,
whitespace=None,
quiet=False,
force_rebase=False,
ff=True,
autosquash=False,
auto_stash=True,
onto_manifest=False,
this_manifest_only=False,
)
git_command = mock.MagicMock()
git_command.Wait.return_value = 0
with mock.patch.object(
rebase, "GitCommand", return_value=git_command
) as run_git, contextlib.redirect_stdout(io.StringIO()):
assert cmd.Execute(opt, []) == 0
run_git.assert_called_once_with(
project,
["rebase", "--autostash", "refs/remotes/origin/main"],
)
+111
View File
@@ -185,6 +185,96 @@ def test_status_without_orphans(
assert lines[1] == " -m\tREADME"
def test_status_staged_and_unstaged_same_path(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""A path changed on both sides of the index renders both states."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
readme = project_worktree / "README"
readme.write_text("staged")
subprocess.check_call(["git", "add", "README"], cwd=project_worktree)
readme.write_text("unstaged")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert lines[1] == " Mm\tREADME"
def test_status_forces_staged_rename_detection(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""The snapshot preserves rename scores despite user status config."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
subprocess.check_call(
["git", "config", "status.renames", "false"], cwd=project_worktree
)
subprocess.check_call(
["git", "mv", "README", "RENAMED"], cwd=project_worktree
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert lines[1] == " R-\tREADME => RENAMED (100%)"
def test_detached_clean_status_is_suppressed(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""A clean detached checkout keeps returning CLEAN without output."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
subprocess.check_call(
["git", "checkout", "-q", "--detach", "HEAD"],
cwd=topdir / project_path,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
assert _status_lines(stdout.getvalue()) == [
"nothing to commit (working directory clean)"
]
def test_status_unmerged_path_matches_legacy_display(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Porcelain-v2 unmerged records render as index U and worktree u."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
worktree = topdir / project_path
subprocess.check_call(
["git", "checkout", "-q", "-b", "other"], cwd=worktree
)
(worktree / "README").write_text("other")
subprocess.check_call(["git", "commit", "-qam", "other"], cwd=worktree)
subprocess.check_call(["git", "checkout", "-q", "main"], cwd=worktree)
(worktree / "README").write_text("main")
subprocess.check_call(["git", "commit", "-qam", "main"], cwd=worktree)
merge = subprocess.run(
["git", "merge", "other"],
cwd=worktree,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
assert merge.returncode != 0
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert lines[1] == " Uu\tREADME"
def test_status_with_orphans_and_modified_file(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
@@ -304,6 +394,10 @@ def test_status_branch_ahead_of_upstream(
project_worktree = topdir / project_path
_setup_remote_tracking_branch(manifest, "feature")
subprocess.check_call(
["git", "config", "status.aheadBehind", "false"],
cwd=project_worktree,
)
subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "c1"],
cwd=project_worktree,
@@ -451,3 +545,20 @@ def test_status_branch_synced_no_ahead_behind(
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, "synced")
def test_status_non_utf8_path(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Non-UTF-8 pathnames render without crashing."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
bad_path = project_worktree / os.fsdecode(b"bad-\xff-name")
bad_path.write_bytes(b"content")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert any("bad-" in line for line in lines)
+1375 -21
View File
File diff suppressed because it is too large. Load diff
+170
View File
@@ -14,6 +14,7 @@
"""Unittests for the subcmds/upload.py module."""
from typing import List, Optional
from unittest import mock
import pytest
@@ -63,3 +64,172 @@ def test_UploadAndReport_UnhandledError(cmd: upload.Upload) -> None:
with mock.patch.object(cmd, "_UploadBranch", side_effect=UnexpectedError):
with pytest.raises(UnexpectedError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
def test_GetMergeBranch_explicit_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch reads branch.merge for explicit local_branch."""
mock_project = mock.MagicMock()
mock_branch = mock.MagicMock()
mock_branch.merge = "refs/heads/main"
mock_project.GetBranch.return_value = mock_branch
res = cmd._GetMergeBranch(mock_project, local_branch="feature")
assert res == "refs/heads/main"
mock_project.GetBranch.assert_called_once_with("feature")
def test_GetMergeBranch_current_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch falls back to project.CurrentBranch."""
mock_project = mock.MagicMock()
mock_project.CurrentBranch = "auto-cbr"
mock_branch = mock.MagicMock()
mock_branch.merge = "refs/heads/upstream-main"
mock_project.GetBranch.return_value = mock_branch
res = cmd._GetMergeBranch(mock_project, local_branch=None)
assert res == "refs/heads/upstream-main"
mock_project.GetBranch.assert_called_once_with("auto-cbr")
def test_GetMergeBranch_none_when_no_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch returns empty string when detached HEAD."""
mock_project = mock.MagicMock()
mock_project.CurrentBranch = None
res = cmd._GetMergeBranch(mock_project, local_branch=None)
assert res == ""
def test_GatherOne_returns_resolved_current_branch(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Upload error reporting reuses the branch gathered by the worker."""
project = mock.MagicMock()
project.CurrentBranch = "topic"
branch = mock.sentinel.branch
project.GetUploadableBranch.return_value = branch
monkeypatch.setattr(
upload.Upload,
"get_parallel_context",
lambda: {"projects": [project]},
)
opt = mock.MagicMock(current_branch=True)
assert upload.Upload._GatherOne(opt, 0) == (0, [branch], "topic")
project.GetUploadableBranch.assert_called_once_with("topic")
def _create_mock_branch(
name: str = "main",
commits: Optional[List[str]] = None,
project_relpath: str = "project-a",
) -> mock.MagicMock:
"""Helper to construct a mock ReviewableBranch."""
branch = mock.MagicMock()
branch.name = name
branch.commits = commits if commits is not None else ["commit1"]
project = mock.MagicMock()
project.RelPath.return_value = project_relpath
branch.project = project
return branch
def test_MultipleBranches_yes_with_current_branch_flag_bypasses_editor(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with --yes and -c flag bypasses editor."""
opt, _ = cmd.OptionParser.parse_args(["-c", "-y"])
branch1 = _create_mock_branch("b1", project_relpath="p1")
branch2 = _create_mock_branch("b2", project_relpath="p2")
pending = [(branch1.project, [branch1]), (branch2.project, [branch2])]
with mock.patch.object(cmd, "_UploadAndReport") as mock_upload, mock.patch(
"editor.Editor.EditString"
) as mock_edit:
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_not_called()
mock_upload.assert_called_once_with(
opt, [branch1, branch2], _STUB_PEOPLE
)
def test_MultipleBranches_yes_with_cbr_flag_bypasses_editor(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with --yes and --cbr flag bypasses editor."""
opt, _ = cmd.OptionParser.parse_args(["--cbr", "--yes"])
branch1 = _create_mock_branch("b1", project_relpath="p1")
branch2 = _create_mock_branch("b2", project_relpath="p2")
pending = [(branch1.project, [branch1]), (branch2.project, [branch2])]
with mock.patch.object(cmd, "_UploadAndReport") as mock_upload, mock.patch(
"editor.Editor.EditString"
) as mock_edit:
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_not_called()
mock_upload.assert_called_once_with(
opt, [branch1, branch2], _STUB_PEOPLE
)
def test_MultipleBranches_yes_with_branch_flag_bypasses_editor(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with --yes and --br flag bypasses editor."""
opt, _ = cmd.OptionParser.parse_args(["--br", "feature", "-y"])
branch1 = _create_mock_branch("feature", project_relpath="p1")
branch2 = _create_mock_branch("feature", project_relpath="p2")
pending = [(branch1.project, [branch1]), (branch2.project, [branch2])]
with mock.patch.object(cmd, "_UploadAndReport") as mock_upload, mock.patch(
"editor.Editor.EditString"
) as mock_edit:
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_not_called()
mock_upload.assert_called_once_with(
opt, [branch1, branch2], _STUB_PEOPLE
)
def test_MultipleBranches_yes_with_branch_flag_empty_pending_dies(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with --yes and empty pending branches dies."""
opt, _ = cmd.OptionParser.parse_args(["--br", "feature", "-y"])
mock_project = mock.MagicMock()
pending = [(mock_project, [])]
with pytest.raises(
upload.UploadExitError, match="nothing ready for upload"
):
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
def test_MultipleBranches_yes_without_branch_or_cbr_uses_editor(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with -y but no -c/--br falls back to editor."""
opt, _ = cmd.OptionParser.parse_args(["-y"])
branch1 = _create_mock_branch("b1", project_relpath="p1")
branch1.date = "2026-08-26"
mock_remote = mock.MagicMock()
branch1.project.dest_branch = None
branch1.project.revisionExpr = "refs/heads/main"
branch_config = mock.MagicMock()
branch_config.remote = mock_remote
branch1.project.GetBranch.return_value = branch_config
pending = [(branch1.project, [branch1])]
edited_script = (
"project p1/:\n"
" branch b1 ( 1 commit, 2026-08-26) to remote branch "
"refs/heads/main:\n"
)
with mock.patch.object(cmd, "_UploadAndReport") as mock_upload, mock.patch(
"editor.Editor.EditString", return_value=edited_script
) as mock_edit:
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_called_once()
mock_upload.assert_called_once_with(opt, [branch1], _STUB_PEOPLE)
+52
View File
@@ -0,0 +1,52 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for subcmds/version.py."""
from unittest import mock
import pytest
from subcmds import version
def test_repo_version_uses_one_pretty_format_call(
monkeypatch: pytest.MonkeyPatch,
) -> None:
project = mock.MagicMock()
project.bare_git.log.return_value = "v2.0-1-g12345678\nTue, 25 Aug\n"
monkeypatch.setattr(version, "git_require", lambda _version: True)
result = version.Version._RepoVersion(project)
assert result == ("v2.0-1-g12345678", "Tue, 25 Aug")
project.bare_git.log.assert_called_once_with(
"-1", "--format=%(describe)%n%cD", "HEAD"
)
project.bare_git.describe.assert_not_called()
def test_repo_version_keeps_old_git_fallback(
monkeypatch: pytest.MonkeyPatch,
) -> None:
project = mock.MagicMock()
project.bare_git.describe.return_value = "v2.0"
project.bare_git.log.return_value = "Tue, 25 Aug"
monkeypatch.setattr(version, "git_require", lambda _version: False)
result = version.Version._RepoVersion(project)
assert result == ("v2.0", "Tue, 25 Aug")
project.bare_git.describe.assert_called_once_with("HEAD")
project.bare_git.log.assert_called_once_with("-1", "--format=%cD", "HEAD")
+56
View File
@@ -515,6 +515,62 @@ class TestResolveRepoRev:
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper.resolve_repo_rev(git_checkout.git_dir, "boooooooya")
@pytest.mark.parametrize(
"committish",
["refs/heads/unknown", "refs/tags/unknown", "boooooooya"],
)
@pytest.mark.parametrize("report_errors", [True, False])
def test_report_errors(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
capsys: pytest.CaptureFixture,
committish: str,
report_errors: bool,
) -> None:
"""Check errors are only printed when requested."""
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, committish, report_errors=report_errors
)
assert ("repo: error:" in capsys.readouterr().err) == report_errors
class TestRefetchIfRevMissing:
"""Check _RefetchIfRevMissing behavior."""
@pytest.mark.parametrize("missing", [False, True])
def test_refetch(
self,
repo_wrapper: wrapper.Wrapper,
capsys: pytest.CaptureFixture,
missing: bool,
) -> None:
"""Check it refetches once only if the rev doesn't resolve."""
mock_resolve = mock.Mock(
side_effect=repo_wrapper.CloneFailure() if missing else None
)
with mock.patch.object(repo_wrapper, "_Fetch") as mock_fetch:
with mock.patch.object(repo_wrapper.time, "sleep") as mock_sleep:
with mock.patch.object(
repo_wrapper, "resolve_repo_rev", mock_resolve
):
repo_wrapper._RefetchIfRevMissing(
"url", "/dst", "stable", False, False
)
# It resolves quietly; check_repo_rev reports the error.
mock_resolve.assert_called_once_with(
"/dst", "stable", report_errors=False
)
fetches = int(missing)
fetch = mock.call("url", "/dst", "origin", False, False)
assert mock_fetch.call_args_list == [fetch] * fetches
delay = repo_wrapper.REV_REFETCH_DELAY_SEC
assert mock_sleep.call_args_list == [mock.call(delay)] * fetches
warnings = capsys.readouterr().err.count(f"refetching in {delay} ")
assert warnings == fetches
class TestCheckRepoVerify:
"""Check check_repo_verify behavior."""