Compare commits

...
44 Commits
Author SHA1 Message Date
Gavin Mak 8c7e0a683e info: reuse branch snapshots and combine history walks
Derive the current branch from GetBranches so attached HEAD and refs are
resolved once, including overview reporting. Replace the two
complementary rev-list calls in --all output with one --left-right
symmetric-difference walk and partition its records in memory.

Bug: 553599402
Change-Id: I01d5db804cbf591ecc0e08d6605971f0f5a38538
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634005
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-21 13:40:06 -07:00
Gavin Mak f6f5946422 project: avoid unnecessary sync revision enumeration
Skip the detached-history walk when its count will not be printed, ask
rev-list --count for upstream gain, and cap the published-but-unmerged
existence probe at one commit. These preserve the decisions while
reducing traversal and output work.

Bug: 553599402
Change-Id: I30f769c9aa5269428ece15ffcb45c50993d7c4b8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634004
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-21 13:40:01 -07:00
Gavin Mak 6321b26685 project: make fast-forward merges explicit
Every caller expects _FastForward to reject a merge commit. Always pass
--ff-only and remove the optional parameter so a violated ancestry
assumption fails instead of silently creating a merge.

Bug: 553599402
Change-Id: I04690b5ca675cb7d3062df08d5e20cd82f5b9e82
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634003
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-21 13:39:56 -07:00
Gavin Mak 7bffc72b5b project: fold normal stash detection into status
When Git 2.35 or newer supplies a stash header, use the same status
snapshot for stateless dirty and stash checks. Retain rev-parse for
earlier clients or when the status snapshot is unavailable.

Bug: 543851900
Bug: 553599402
Change-Id: I6e310459d9c1a3ba7978f7907ae6b7a9bc611749
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632146
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-21 13:39:51 -07:00
Gavin Mak aaadd5da35 sync: reuse status HEAD in the bloat scan
The shallow-project bloat check already needs dirty state. Request
branch metadata in that same porcelain-v2 snapshot and use branch.oid
instead of launching a separate rev-parse, with the existing calls
retained as a fallback.

Bug: 543851900
Bug: 553599402
Change-Id: I3e7068c2e14c025a53adf92bfe8483b876cfda9d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632145
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-21 13:39:21 -07:00
Gavin Mak c9448f986e project: render worktree status from one snapshot
Use porcelain-v2 branch and path records to replace the status display
fan-out with one Git process on Git 2.11 and newer. Preserve the legacy
renderer for older clients, force rename and ahead/behind behavior where
supported, and fall back to the old divergence query only when status
cannot supply branch.ab.

Bug: 543851900
Bug: 553599402
Change-Id: Ic760360f4f48ab9320abf69dbbb7310dab4ac649
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632144
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-21 13:39:16 -07:00
Gavin Mak 530258d08e project: use one status snapshot for dirty checks
On Git 2.11 and newer, replace update-index, diff-index, diff-files, and
ls-files fan-out with one porcelain-v2 snapshot for IsDirty,
UncommittedFiles, and HasChanges. Preserve category ordering and
duplicate staged-plus-unstaged paths, and keep the exact legacy path for
older Git or a failed snapshot.

Bug: 543851900
Bug: 553599402
Change-Id: I309e39977e1f1d4ee6115f488d6aabb83ec3d83c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632143
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-21 13:37:38 -07:00
Gavin Mak c1566487a8 git_status: parse one porcelain v2 worktree snapshot
Add a strict, NUL-aware parser for porcelain v2 records, including
ordinary changes, renames, conflicts, untracked paths, branch
divergence, and stash metadata. The reader is gated on Git 2.11, opts
into newer flags only when supported, and preserves arbitrary pathname
bytes.

Bug: 543851900
Bug: 553599402
Change-Id: Ibae3e1056fd9866f3cb6d490745d377cda1e8fef
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632142
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-21 13:36:55 -07:00
Gavin Mak 22f820de17 git_command: allow lossless capture of machine output
NUL-delimited Git formats may contain path bytes that are not valid
UTF-8. Add an opt-in byte capture mode so parsers can preserve those
bytes instead of receiving backslash replacements, while continuing to
decode stderr for diagnostics.

Bug: 543851900
Bug: 553599402
Change-Id: Ia9dfcde065a0f6f72c785a4b96896f346e17510a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632141
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-21 13:36:47 -07:00
Sam Saccone 4c37f58806 project: pass --no-filter and --no-tags in DownloadPatchSet
In partial clone checkouts (e.g. --filter=blob:none or blob:limit=10k),
DownloadPatchSet previously invoked `git fetch <remote> <ref>` without
explicit filter overrides. Consequently, git fetch applied the
configured partial clone filter from the remote and omitted the blobs
for the incoming change.

When repo download subsequently checks out, cherry-picks, or merges the
downloaded commit, Git discovers that the required blobs are missing
locally and triggers on-demand unadvertised object fetches to the
promisor remote (POST /git-upload-pack with `want <blob_sha>`). On large
repositories, server-side reachability validation for unadvertised
objects can result in expensive graph traversals, proxy/gateway timeouts
(such as HTTP 502 or connection drops), and client fetch failures:
  fatal: expected 'packfile'
  fatal: could not fetch <sha> from promisor remote

Pass --no-filter (when Git >= 2.17.0) and --no-tags when fetching patch
sets in DownloadPatchSet. Because the change ref is an advertised ref
tip, fetching with --no-filter allows the server to generate a complete
packfile containing all required blobs upfront without requiring
unadvertised object reachability checks. All blobs are present in the
local object database prior to checkout or cherry-pick, completely
preventing on-demand promisor queries.

TAG=agy
CONV=bdc489e3-b20b-4ef3-b32e-549dbe21fc6c

Google-Bug-Id: b/532920848
Test: pytest tests/test_project.py
Change-Id: Ic6118e4edeb754f0467e0580fd1d0965a6553153
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/625621
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Sam Saccone <samccone@google.com>
Commit-Queue: Sam Saccone <samccone@google.com>
2026-09-21 09:19:39 -07:00
Gavin Mak c86ddc1628 project: rename UncommitedFiles to UncommittedFiles
Fix typo in UncommitedFiles across project.py and HasChanges.

Bug: 553599402
Change-Id: I2b9b606b4c9a47fd998e50cca976545fcd783771
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632881
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-17 17:21:32 -07:00
Victor Pushkarev c638b54e19 command: preserve project filters in regex selection
`FindProjects` gets its candidate projects from `GetProjects` and then
applies regex matching. However, it only forwards `all_manifests`, so
other project filters are lost on regex paths.

As a result, `repo forall -r/-i --groups` falls back to the configured
group selection instead of using the requested groups.
Similarly, `repo list -r --groups` ignores the requested groups, and
`repo list -r --all` still excludes projects without a checkout.

Pass `groups` and `missing_ok` through `FindProjects`, and forward the
existing options from `forall` and `list`. When no group override is
given, keep using each manifest's effective groups.

Add regression tests for `FindProjects` and both command paths.

Change-Id: I5dfc22172fdd81de5312a84433867300a1342aec
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/631281
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-16 04:54:02 -07:00
Victor Pushkarev fb4a91060e tests: avoid losing trace2 socket readiness signal
The trace2 socket test uses a Condition to wait for its server to start
listening. If the server calls notify() before the test starts waiting,
the notification is lost and the test waits for the full 120-second
timeout before continuing.

Use an Event for the readiness signal instead.
Once the server starts listening, the event stays set, so the test can
continue immediately even if the server becomes ready before the test
starts waiting.

Change-Id: I0e8e192d997c8bd785bcefa5d5b91493f93863e8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/630861
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-16 04:53:31 -07:00
Victor Pushkarev 4b3ada1781 sync: keep generated jobs-checkout help host-independent
The default shown for `--jobs-checkout` comes from `DEFAULT_LOCAL_JOBS`,
which depends on the host CPU count. As a result, regenerating the sync
and smartsync manpages on a different machine can change their output
even though repo's behavior has not changed.

Use shared CPU-based help formatting for `--jobs` and `--jobs-checkout`.
Interactive help still shows the actual default, while generated
manpages describe it without including a host-specific value.
Runtime job selection is unchanged.

Add coverage for both help modes and regenerate the affected manpages.

Change-Id: Ia60e6fcc17b6f4618c52a76217c128eddead8848
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/625381
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-16 04:51:54 -07:00
Gavin Mak a468ea7752 sync: implement reprojectcmd for the local half
Add `repo.reprojectcmd`, the checkout counterpart to `repo.fetchcmd`.
When set, `repo sync` runs this command instead of Git to materialize a
project's index and worktree at the target tree, while `repo` handles
ref updates directly via `git update-ref`.

This replaces Git's tree materialization steps: detaching HEAD,
fast-forwarding, and hard-resetting. Rebasing is not delegated, and the
command is skipped if HEAD is already at the target, if HEAD is ahead of
the target during fast-forward, or for MetaProjects.

The command runs in a subshell with project environment variables (such
as `REPO_TREV`). Before running, `repo` ensures no operation is in
progress and no staged changes exist. Worktree collision detection is
delegated to the command (preserving benign unstaged/untracked edits).
Afterward, `repo` verifies that HEAD was untouched and that the index
matches the target tree.

Like `repo.fetchcmd`, this requires `repo.uselocalgitdirs`. Nested
projects and submodules are unsupported; `repo sync` fails if the
manifest contains any while `repo.reprojectcmd` is enabled.

Verified end-to-end with repo init using local-gitdirs, repo.fetchcmd,
and repo.reprojectcmd ('git -C $REPO_PATH read-tree -m -u $REPO_TREV'):
  * Verified detached HEAD checkout and correct reflog generation across
    projects.
  * Verified benign unstaged edits and untracked files survive checkout.
  * Verified conflicting untracked files fail with exit 128 without
    clobbering worktree.
  * Verified staged changes fail upfront before reprojectcmd is executed.

Bug: 513329573
Change-Id: I964d24d22dccffc05a9b991ad69f3a7e93268c01
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/626281
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-15 12:53:01 -07:00
Brian Gan c2c330ba4a project: report why refreshing the index failed
"repo status" aborts the whole tree when one project fails, without
saying why. The three callers that refresh the index stat cache pass -q,
which suppresses git's diagnosis when it cannot create the index lock;
git then exits 128 with both streams empty. A stale lock, a permissions
problem on the gitdir and a full disk all reach that same silent exit,
yet each needs a different remedy.

Route the three identical call sites through one helper that keeps -q
for the refresh itself. When that fails, it repeats the command without
-q to obtain git's diagnosis. The command still fails, which is correct,
but the user is now told why.

Keeping -q on the common path matters: without it git exits 1 whenever
paths hold uncommitted changes, which would report a failure to
telemetry for every modified project, even if we treat it as benign in
the code. The retry accepts that 1, since --unmerged makes git skip
conflicted paths rather than count them, so nothing else yields it.

The similar call in subcmds/sync.py is unchanged: it passes different
flags, without --unmerged, so exit 1 cannot be assumed benign there.

Bug: 560289756
Change-Id: I2e1024c84454f86076081f2d7f155728b1aa35c6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/630641
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-09-14 13:27:19 -07:00
Victor Pushkarev cc88be34d2 command: fix group selection across submanifests
When GetProjects walks all manifests without explicit groups, it
resolves the effective groups once from the selected manifest and reuses
them for projects from child manifests. This can ignore a child
manifest's own group selection and include projects that should be
excluded.

When no groups are specified, resolve them from each project's owning
manifest. Explicit groups still apply to all projects in the call, and
single-manifest behavior remains unchanged.

Add regression tests for per-manifest group selection, explicit
overrides, and name lookup across manifests.

Bug: 382086294
Change-Id: Id5c7e52410431b86b395f489cfd494ca05620975
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/624821
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-11 01:54:27 -07:00
Victor Pushkarev 16cfb53e0a release: update-manpages: restore __main__ guard
The update-manpages wrapper calls update_manpages.main() unconditionally.
With spawn and forkserver, multiprocessing re-executes the directly run
main script as __mp_main__.
The wrapper then calls main() again and tries to create another Pool
while the child is still starting.

Protect the entrypoint with __main__ guard. On Linux, this was mostly
hidden while fork was the default start method; Python 3.14 now defaults
to forkserver there. Normal command-line behavior is unchanged.

Add a regression test that mirrors multiprocessing's __mp_main__
execution path and checks that it does not call update_manpages.main().

Change-Id: I28fdb18ccf2e4d61605cc072b4de52b6eb561335
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/626841
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-11 01:52:38 -07:00
Fredrik de Groot 578b57c975 manifest_xml: Improve error message on base check fail
Previous error had a scent of alien blob to it
due to dumping the list in a single long line, so
improve print for human readability.

Example of new fail:

error: in `manifest`: detected base-revision mismatch, updates needed:
remove-project name build/make:
  base fccb6b3e884b4b964e795f5f8865f6bbb4722b5a vs revision 6e9a0c16f20d1cfcf08b8b9cb42301b6c9712862
remove-project path build/soong:
  base 16b3c9ca0679bb376966866afb849322c8aec79f vs revision b7490575c16b87d861a532b03bd1b2d076b6fa0b
extend-project name platform/system/sepolicy:
  base c2652856e9f0e0433d78cfb481efe392d6f5dbc4 vs revision cf9a73f92f503ffc4bd3e726164babfb40a9fcaf

Test: repo manifest > /dev/null && echo 'Nice =)'
      on various good and bad test manifests
Change-Id: I8d741e11c69ef9e4f8cf09e32aa0868b6daa5acd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/619921
Tested-by: Fredrik de Groot <fredrik.de.groot@haleytek.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Fredrik de Groot <fredrik.de.groot@haleytek.com>
2026-09-10 23:17:12 -07:00
Arif Kasim fe2c23f8e2 upload: bypass branch editor with explicit branch and -y
When uploading across multiple projects with --yes and an explicit
branch specified (via --current-branch / -c or --branch / --br),
repo previously still opened the interactive branch selection editor.

If an explicit branch and --yes are provided, bypass the interactive
editor and upload all matching pending branches directly.

Bug: 553024110
Test: python3 -m pytest tests/
Test: flake8 subcmds/upload.py tests/test_subcmds_upload.py
Test: isort --check subcmds/upload.py tests/test_subcmds_upload.py
Change-Id: I104619603f6e0539d05b01d3eb0292a9145d58db
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622661
Tested-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-09 12:10:24 -07:00
Rahul Yadav d27d6829a8 project: check upstream ref ancestry for non-shallow clones
In _CheckForImmutableRevision, commit d9cc0a15 restricted upstream ref
validation strictly to superprojects (if use_superproject) to prevent
shallow clones with an upstream attribute from failing ancestry checks
and falling back to full clones.

However, restricting this check exclusively to superprojects broke
non-shallow projects with pinned immutable revisions (such as in Smart
Sync, `repo sync -t <BUILD_ID>`, or pinned manifests): When the target
commit already exists in the local Git object store (for instance,
prefetched into refs/prefetch/ by a background daemon or via shared
object dirs), _CheckForImmutableRevision returned True without verifying
that the local tracking ref (refs/remotes/<remote>/<upstream>) is
present and reaches the revision. As a result, _RemoteFetch skipped
fetching the upstream branch, leaving the local tracking ref stale or
missing. Subsequent `repo start` branches tracking that remote branch
diverged, causing `repo upload` to attempt uploading all intermediate
commits between the stale tracking ref and HEAD.

Restore upstream ancestry validation in _CheckForImmutableRevision for
non-shallow projects when upstream is specified. Also pass the sync
depth into _CheckForImmutableRevision call sites so shallow checkouts
continue to skip upstream verification and avoid triggering un-shallow
fallbacks.

Test: PYTHONPATH=. pytest tests/test_project.py
Change-Id: Ib5bdf41810bb06c8ed053447209ddc2e488f3913
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/624361
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-03 02:15:23 -07:00
Gavin Mak 0ea57e2eed project: gate the default-branch query by Git version
GIT_DEFAULT_BRANCH support was added to git var in Git 2.35. Skip that
guaranteed failure on older supported clients and read
init.defaultBranch directly, reducing their fallback from two processes
to one.

Bug: 553599402
Change-Id: I079e9e6057b56afa41b685317a106fc9cdbad8a2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623907
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-02 17:46:23 -07:00
Gavin Mak ba8ddf396c git_superproject: resolve one negotiation ref directly
The superproject only needs one branch tip or HEAD OID. Resolve that ref
with one quiet rev-parse instead of constructing GitRefs, which loads
the entire namespace and resolves HEAD in two or three processes on
older clients.

Bug: 553599402
Change-Id: I162d1133ebd1941e9de5aca43929fd5af17386ef
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623906
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:46:01 -07:00
Gavin Mak d88ce8d952 upload: reuse the worker current-branch result
Carry the current branch through _GatherOne so the parent can report a
failed current-branch upload without resolving HEAD a second time.

Bug: 553599402
Change-Id: I3fc54be6c10c30470e14f792ff68c0a396519327
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623905
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-02 17:45:55 -07:00
Gavin Mak 5e8d2a6e3a version: read describe and date in one Git call
Git 2.32 added the %(describe) pretty placeholder. Use one log format to
retrieve both repo's describe string and commit date on newer clients,
with the existing describe-plus-log path retained for older Git and
untagged output.

Bug: 553599402
Change-Id: I108346030677b2b90ef8bfb5acea1c45be2c85f2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623904
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-02 17:45:49 -07:00
Gavin Mak e59c9cde99 cherry_pick: resolve and read commits in one process
Send a typed revision expression to cat-file --batch over stdin and
parse its object header and exact byte length. This safely resolves the
commit OID and retrieves its raw message in one Git process instead of
separate rev-parse and cat-file calls.

Bug: 553599402
Change-Id: I764894323ec28e134f2523bd1d5694738b6b52d0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623903
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:43:55 -07:00
Gavin Mak 83428a9b26 rebase: delegate automatic stashing to Git
Git --autostash predates repo's minimum supported client. Pass it
directly to rebase instead of probing the index and separately running
stash and stash pop, reducing the dirty path from four Git processes to
one and correctly covering staged-only changes.

Bug: 553599402
Change-Id: I0d6d59b17996576310498e9378e49380d355236e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623902
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:43:48 -07:00
Gavin Mak 948abc85bc git_config: recognize only complete object IDs
Full Git object IDs are exactly 40 hexadecimal digits for SHA-1 or 64
for SHA-256. Stop treating every intermediate length as immutable, which
could bypass normal ref resolution for invalid revision strings.

Bug: 553599402
Change-Id: I41b4ce6b2bfe2a2d8b351c7f040ec0cf469b474b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623901
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-09-02 17:41:05 -07:00
kimhappy c63a2f92fa sync: tell apart same-path submanifest projects
Interleaved sync remembers which projects are done and which ones are
still pending by Project.relpath, and lists failing projects by it as
well. That path is relative to the project's own (sub)manifest though,
so once several manifests are synced together two projects can share
it, and the sets no longer tell them apart.

The damage is done when a project turns up in a later pass, e.g. a
submodule that is only derived once its parent has been synced: its
path is already recorded as finished, so it is left out of every pass
that follows and never synced, while sync still reports success. With
an outer manifest holding

  <project name="a" path="a" sync-s="true"/>

where a carries a submodule at b, and a submanifest at sub/ holding

  <project name="sub-ab" path="a/b"/>

a fresh `repo sync` checks out sub/a/b but never a/b. Failing projects
are reported under the same ambiguous path, and the stall detection
merges them too.

Use RelPath(local=opt.this_manifest_only) instead, which is unique
within the set of projects being synced and is what GetProjects() and
the other subcommands already use.

Spotted during the review of I30395b8a16a9154f60972e1f408a066af64ba77c.

Change-Id: I5fa694b968774667be6060d9722cb57acbd2b579
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622641
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-01 19:13:52 -07:00
Gavin Mak 0039e39000 project: carry current state from the ref snapshot
Attach the current marker computed by GetBranches to uploadable and
prunable branches. Overview and prune reporting can then reuse that
result instead of resolving HEAD once per displayed branch.

Bug: 553599402
Change-Id: Ie3e23d08d0caf66c47cc12d7295ec4b6a834560c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623184
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 13:26:22 -07:00
Gavin Mak 5f378458d2 project: reuse ref OIDs in branch maintenance
Load refs before resolving the current branch, compare cached HEAD and
manifest OIDs instead of walking their symmetric difference, and reuse
the snapshot when preserving bare HEAD. Apply the same HEAD reuse to
abandon and MetaProject checks, and bound the latter revision walk to
one commit.

Bug: 553599402
Change-Id: I715e65dcfa02c0df4aaa08da5b7bfbb04ca17725
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623183
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 13:25:40 -07:00
Gavin Mak d27034bf62 project: centralize safe branch and commit resolution
Resolve commit-ish values through one typed helper using rev-parse
--verify --quiet and, on Git 2.30+, --end-of-options. Reject option-like
revisions on older clients, reuse the helper for project and manifest
resolution, and validate user branch names with check-ref-format
--branch.

Bug: 553599402
Change-Id: I2dda49ff3e781cec14d3b84263a8dace06b4073f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623182
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-31 12:34:53 -07:00
Gavin Mak 6541729a18 project: reuse loaded refs when resolving HEAD
When the regular checkout has already loaded GitRefs, derive its current
branch and HEAD object from that snapshot instead of spawning
symbolic-ref or rev-parse again. Keep linked Git worktrees on their
worktree-specific HEAD path because the shared repository HEAD is not
authoritative for them.

Bug: 543851900
Bug: 553599402
Change-Id: Iec456be5f3c279865b4165a75d1c6df36e331dbc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623181
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 12:33:39 -07:00
Gavin Mak b85e76a86a git_refs: load HEAD with the ref snapshot on Git 2.45
Use for-each-ref --include-root-refs to resolve attached and detached
HEAD in the same process as refs/* on Git 2.45 and newer. Limit the
patterns to HEAD and refs so volatile pseudo-refs do not enter the
cache, and retain the existing fallback for older Git and unborn HEADs.

Bug: 543851900
Bug: 553599402
Change-Id: Ie091ca0757f21ec6b1e19d49e09d160e055ec639
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622981
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-08-31 11:57:39 -07:00
James Hawkins e5bbb5c9e6 upload: prune ref scans for untouched projects and specific branches
Benchmark (100 untouched projects):
* GetUploadableBranches: 0.985s -> 0.0004s (2,200x faster)
* repo upload (3,045 projects): saves ~2.5s of redundant ref scans

During `repo upload`, `GetUploadableBranches()` currently scans
`self._allrefs` across all projects in the workspace. In large
manifests (e.g. Android with 3,000+ projects), `self._allrefs`
executes `git for-each-ref`, `git symbolic-ref`, and full filesystem
mtime traversals on thousands of projects where no local branch was
ever started or modified.

Any branch that can be uploaded for review must have upstream tracking
configured via `[branch "..."]` sections in `.git/config`
(otherwise `branch.LocalMerge` is None and `GetUploadableBranch`
returns None).

This patch introduces early-pruning fast paths:
1. When no branch subsections exist in `.git/config`
   (or only 2-part keys like `branch.autosetupmerge`), return []
   immediately without touching `_allrefs`.
2. When a `selected_branch` is specified (e.g. `repo upload --br=...`),
   query `branch.LocalMerge`, `refs/heads/<branch>`, and
   `refs/published/<branch>` directly, skipping the whole-tree
   ref scan.
3. In `subcmds/upload.py`, optimize `_GetMergeBranch` to resolve the
   merge branch in-memory via `project.CurrentBranch` and
   `project.GetBranch()` instead of spawning two `GitCommand`
   processes.

Test: ./run_tests tests/test_project.py tests/test_subcmds_upload.py
Change-Id: I3c2c9d63a68a393d478e1fb50f548ec8b7d44639
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623021
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: James Hawkins <jhawkins@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
2026-08-28 16:07:01 -07:00
James Hawkins 4fe87617ff project: read HEAD directly in-memory to avoid subprocesses
Benchmark:
* repo upload frameworks/base: 2.84s -> 0.50s (-82.3%, 5.6x faster)
* repo upload (3,045 projects): 7.66s -> 5.24s (-31.6%, 2.42s saved)

From repo's inception through v2.56, GetHead() directly read the
`.git/HEAD` file in Python. In commit 52bab0ba ("project: Use git
rev-parse to read HEAD"), this was replaced with git subprocess calls
on the premise that git provides a dedicated command. However, in
large multi-project workspaces (such as Android with 3,000+ projects),
spawning thousands of git processes introduced severe latency
regressions during `repo upload` and `repo status`.

Furthermore, switching to subprocesses broke detached HEADs and
unborn branches (fixed in commits 7f7d70ef and 8c3585f3 by re-adding
the v2.56 file-reading logic as an error recovery fallback).

This patch restores fast in-memory reading as the primary path, while
adding modern defensive safeguards:
* Symbolic refs (`ref: refs/heads/...`): Strips whitespace and tabs
  and returns the ref directly in memory.
* Detached HEAD: Validates 40-char SHA-1 and 64-char SHA-256 commit
  hashes via git_config.IsId(), normalizing to lowercase.
* Symlinks: Detects filesystem symlinks via os.path.islink() and
  safely falls back to git symbolic-ref.
* Fallback: Catches (OSError, AssertionError) and transparently falls
  back to native git commands for reftables, unexpected layouts, or
  filesystem errors. Unifies recovery fallback parsing with the fast
  path (CRLF/tabs, lowercase hashes, and consistent RelPath errors).

In addition, this change substantially expands test coverage in
tests/test_project.py, adding comprehensive unit tests for symbolic
refs, whitespace/tabs, CRLF line endings, SHA-1, SHA-256, uppercase
hash normalization, symlinks, corrupted worktrees, and fallback
robustness.

Test: ./run_tests tests/test_project.py
Change-Id: Ib5c2530117c6939e4b9293feda81aa745c003c6a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623001
Commit-Queue: James Hawkins <jhawkins@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
2026-08-28 16:03:06 -07:00
kimhappy d7299422ae sync: skip submodules removed from their parent
The submodules to sync are derived from their parent before it is
fetched, so one that the parent's new revision no longer holds a gitlink
for is still synced. Interleaved sync then fails to check it out,
because `git submodule init` no longer knows that path:

  error: Cannot checkout a/c
  error: pathspec '.../a/c' did not match any file(s) known to git

Report such submodules while the gitlinks are read again, and leave them
out of the fetch and of the checkout, so the usual project list update
removes them from the working tree. Phased sync already left them out of
the checkout, but still fetched them.

Nothing is reported as removed while the gitlinks of a parent cannot be
read, since a missing gitlink cannot be told apart from a fetch that did
not happen.

Bug: 550074864
Change-Id: Ia429b35c12758a68e4df73665e5512a201af658a
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621681
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:54 -07:00
kimhappy 3f087a8dd9 sync: fetch submodules after their parent repository
Phased sync fetches every project at once, so a submodule is fetched at
the gitlink that was read before its parent was fetched. With -c that
stale revision is already present, the fetch is skipped as an immutable
revision, and the checkout then fails on the revision the reloaded
manifest resolved:

  error: Cannot checkout a/b
  fatal: bad object 5556dad99da5df7e53303c5732aacc8782493e03

Fetch in batches, holding a submodule back until the project holding its
gitlink is fetched, and read the gitlinks again between batches. A
manifest without submodules keeps its single batch.

Bug: 550074864
Change-Id: I04c2eb1ca81ab25b4df2c03c5dc048d2e4ecdcb8
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621663
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:49 -07:00
kimhappy 3a6e25af75 sync: check out submodules at the parent's new revision
Interleaved sync processes projects in hierarchical levels, so a
submodule is only handled after the project holding its gitlink has been
fetched and checked out. Its revision, however, is still the one read
before that parent was fetched, so the submodule is synced to the
revision of the previous sync and only catches up on the next one.

Read the gitlinks again right before a level is dispatched. The parent
is done by then, so its submodules are fetched and checked out at the
revision the parent now points at. Sibling submodules are spread over
several levels, so the gitlinks read for one level are carried over to
the next ones and every project is only read once per pass.

Bug: 550074864
Change-Id: I30395b8a16a9154f60972e1f408a066af64ba77c
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621662
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:43 -07:00
Gavin Mak 41c2597509 project: don't use dest-branch or tags as SHA-1 upstream fallback
https://gerrit-review.googlesource.com/614762 resolves a fallback
upstream for a SHA-1-pinned project that has no explicit upstream, so
--current-branch can narrow the fetch to one branch instead of fetching
all heads.

It drew that fallback from dest-branch and from the manifest <default>
upstream, dest-branch, and revision. dest-branch and tags are not valid
fallback candidates:

* dest-branch is the review destination for upload, not a fetch
  source.
* Tags are meant to be immutable snapshots of a single tagged commit,
  and fetching it cannot retrieve an arbitrary pinned SHA-1.

Restrict the fallback to branch heads resolved from the manifest
upstream and revision defaults, and drop dest-branch entirely.

Bug: 541240657
Bug: 544041102
Change-Id: I89ae86891ee166f4c340553ac5a9068efcc18461
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622541
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 12:43:00 -07:00
Rahul Yadav e6ad708009 hooks: add --fix option to auto-apply hook fixes
Pass the --fix flag as a keyword argument "fix" to the hook main
function. This allows hooks (such as git-repohooks) to decouple
automated fix application from the -y/--yes flag so that -y can
answer yes to upload confirmation prompts without triggering file
mutations.

Companion change in git-repohooks:
https://gerrit-review.googlesource.com/c/git-repohooks/+/621761

Bug: 546510319
Test: python3 -m pytest tests/test_hooks.py

Change-Id: If0288d4791fc0a2aba6e88854e3aa81b0923b664
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/619281
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-08-27 03:28:58 -07:00
Ram Peri 09914bcab7 Add detailed upload context to Trace2 telemetry
This patch intercepts the output during a successful upload
execution to capture the generated CL URLs. It logs a dynamically
constructed "repo.uploadstate" data event to the active trace2
log containing:
- Uploaded CL URLs
- Target remote name
- Source branch
- Modified files

Test:
1. ./run_tests
2. pytest tests/test_project.py
3. Manual verification:
- Created a dummy branch `test_upload_branch` in a project with local file changes.
- Invoked repo upload passing an explicit trace output directory:
  `repo --git-trace2-event-log=/tmp/trace2out upload --dry-run --no-verify art`
- Verified the injected payload successfully appeared on disk within the generated log:
  ```json
  {"event":"data",..."key":"repo.uploadstate/cls","value":""}
  {"event":"data",..."key":"repo.uploadstate/remote","value":"goog"}
  {"event":"data",..."key":"repo.uploadstate/branch","value":"test_upload_branch"}
  {"event":"data",..."key":"repo.uploadstate/files","value":"dummy_file.txt"}
  ```
  (cls is correctly blank on --dry-run but populates on real HTTP pushes)

Bug: 543953499
Change-Id: I9c402a32d01d156d42cf24eaa2e60e22710b5e6f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/616581
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Ram Peri <ramperi@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
2026-08-26 10:19:42 -07:00
kimhappy 3f1775607f project: allow re-reading submodule gitlinks
A discovered submodule becomes a project whose revision is the gitlink
read from its parent at manifest load time, which is before the parent
has been fetched. Remember which gitlink a submodule was derived from,
and add a way to read those gitlinks again, so that callers can resolve
a submodule revision once its parent is up-to-date.

A revision that is set does not have to be fetched, so verify its
objects are really there. Without them a submodule that was removed
cannot be told apart from one that was never fetched, which the caller
has to know about.

Bug: 550074864
Change-Id: I4f35d53607126508eec6f02158a3fb604917a199
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621661
Reviewed-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-08-25 19:39:51 -07:00
Gavin Mak b85886fa9f project: skip manifest default fallback for MetaProjects
Avoid loading manifest.xml during repo init when syncing a manifest
commit SHA. Override _GetUpstreamFallback and _SharingProjectHasShallow
in MetaProject to prevent premature manifest parsing.

Bug: 544041102
Change-Id: I7aa54a7c1282e5bfe811d59e977b44163a37653c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/617081
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-10 13:37:04 -07:00
50 changed files with 6515 additions and 424 deletions
+68 -17
View File
@@ -17,7 +17,7 @@ import multiprocessing
import optparse import optparse
import os import os
import re import re
from typing import TYPE_CHECKING from typing import List, Optional, TYPE_CHECKING
from error import InvalidProjectGroupsError from error import InvalidProjectGroupsError
from error import NoSuchProjectError from error import NoSuchProjectError
@@ -152,6 +152,20 @@ class Command:
self._Options(self._optparse) self._Options(self._optparse)
return self._optparse return self._optparse
@staticmethod
def _GetHelpForCpuJobCount(
default_jobs: Optional[int] = None,
) -> str:
"""Return CPU-based job help, with an explicit default when needed.
Specify default_jobs when additional logic computes effective default.
"""
if GENERATE_MANPAGES:
return "based on number of CPU cores"
default = "%default" if default_jobs is None else str(default_jobs)
return f"{default}; based on number of CPU cores"
def _CommonOptions(self, p, opt_v=True): def _CommonOptions(self, p, opt_v=True):
"""Initialize the option parser with common options. """Initialize the option parser with common options.
@@ -176,11 +190,7 @@ class Command:
) )
if self.PARALLEL_JOBS is not None: if self.PARALLEL_JOBS is not None:
default = "based on number of CPU cores" default = self._GetHelpForCpuJobCount()
if not GENERATE_MANPAGES:
# Only include active cpu count if we aren't generating man
# pages.
default = f"%default; {default}"
p.add_option( p.add_option(
"-j", "-j",
"--jobs", "--jobs",
@@ -398,7 +408,11 @@ class Command:
Args: Args:
args: a list of (case-insensitive) strings, projects to search for. args: a list of (case-insensitive) strings, projects to search for.
manifest: an XmlManifest, the manifest to use, or None for default. manifest: an XmlManifest, the manifest to use, or None for default.
groups: a string, the manifest groups in use. groups: a string, the manifest group selection to apply.
Non-empty values apply to all candidate projects in this call.
When empty or omitted, single-manifest calls use the selected
manifest's effective groups; all-manifest calls use each
candidate project's owning manifest's effective groups.
missing_ok: a boolean, whether to allow missing projects. missing_ok: a boolean, whether to allow missing projects.
submodules_ok: whether to allow submodules. True allows them for submodules_ok: whether to allow submodules. True allows them for
all projects, False disallows them for all projects, and None all projects, False disallows them for all projects, and None
@@ -425,9 +439,33 @@ class Command:
return project.sync_s return project.sync_s
return submodules_ok return submodules_ok
if not groups: def parse_groups(value: str) -> List[str]:
groups = manifest.GetManifestGroupsStr() return [x for x in re.split(r"[,\s]+", value) if x]
groups = [x for x in re.split(r"[,\s]+", groups) if x]
if groups:
groups_for_all_projects = parse_groups(groups)
elif all_manifests:
# In all-manifest mode, each project uses its owning
# manifest's effective groups.
groups_for_all_projects = None
else:
groups_for_all_projects = parse_groups(
manifest.GetManifestGroupsStr()
)
groups_by_manifest = {}
def matches_groups(project: "Project") -> bool:
if groups_for_all_projects is not None:
return project.MatchesGroups(groups_for_all_projects)
project_manifest = project.manifest
if project_manifest not in groups_by_manifest:
groups_by_manifest[project_manifest] = parse_groups(
project_manifest.GetManifestGroupsStr()
)
return project.MatchesGroups(groups_by_manifest[project_manifest])
if not args: if not args:
derived_projects = {} derived_projects = {}
@@ -439,9 +477,7 @@ class Command:
) )
all_projects_list.extend(derived_projects.values()) all_projects_list.extend(derived_projects.values())
for project in all_projects_list: for project in all_projects_list:
if (missing_ok or project.Exists) and project.MatchesGroups( if (missing_ok or project.Exists) and matches_groups(project):
groups
):
result.append(project) result.append(project)
else: else:
self._ResetPathToProjectMap(all_projects_list) self._ResetPathToProjectMap(all_projects_list)
@@ -455,7 +491,7 @@ class Command:
for project in manifest.GetProjectsWithName( for project in manifest.GetProjectsWithName(
arg, all_manifests=all_manifests arg, all_manifests=all_manifests
) )
if project.MatchesGroups(groups) if matches_groups(project)
] ]
if not projects: if not projects:
@@ -498,7 +534,7 @@ class Command:
"%s (%s)" "%s (%s)"
% (arg, project.RelPath(local=not all_manifests)) % (arg, project.RelPath(local=not all_manifests))
) )
if not project.MatchesGroups(groups): if not matches_groups(project):
raise InvalidProjectGroupsError(arg) raise InvalidProjectGroupsError(arg)
result.extend(projects) result.extend(projects)
@@ -509,7 +545,14 @@ class Command:
result.sort(key=_getpath) result.sort(key=_getpath)
return result return result
def FindProjects(self, args, inverse=False, all_manifests=False): def FindProjects(
self,
args: List[str],
inverse: bool = False,
all_manifests: bool = False,
groups: Optional[str] = "",
missing_ok: Optional[bool] = False,
) -> List["Project"]:
"""Find projects from command line arguments. """Find projects from command line arguments.
Args: Args:
@@ -519,10 +562,18 @@ class Command:
all_manifests: a boolean, if True then all manifests and all_manifests: a boolean, if True then all manifests and
submanifests are used. If False, then only the local submanifests are used. If False, then only the local
(sub)manifest is used. (sub)manifest is used.
groups: a string specifying manifest groups. If empty or None, use
each manifest's effective groups.
missing_ok: a boolean, whether to allow missing projects.
""" """
result = [] result = []
patterns = [re.compile(r"%s" % a, re.IGNORECASE) for a in args] patterns = [re.compile(r"%s" % a, re.IGNORECASE) for a in args]
for project in self.GetProjects("", all_manifests=all_manifests): for project in self.GetProjects(
"",
groups=groups,
missing_ok=missing_ok,
all_manifests=all_manifests,
):
paths = [project.name, project.RelPath(local=not all_manifests)] paths = [project.name, project.RelPath(local=not all_manifests)]
for pattern in patterns: for pattern in patterns:
match = any(pattern.search(x) for x in paths) match = any(pattern.search(x) for x in paths)
+1
View File
@@ -400,6 +400,7 @@ _repo() {
'--no-verify[Do not verify]' \ '--no-verify[Do not verify]' \
'--verify[Verify]' \ '--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \ '--ignore-hooks[Ignore hooks]' \
'--fix[Automatically fix]' \
'*: :->project' '*: :->project'
;; ;;
version) version)
+3
View File
@@ -6,6 +6,9 @@ executed during `repo sync` to fetch objects, instead of using standard
filesystems or lazy checkouts where fetching metadata and downloading file filesystems or lazy checkouts where fetching metadata and downloading file
contents should be decoupled. contents should be decoupled.
The checkout half of a sync has a counterpart, `repo.reprojectcmd`; see
`docs/reproject-cmd.md`.
## Configuration ## Configuration
To use this feature, set the following in `.repo/manifests.git/config`: To use this feature, set the following in `.repo/manifests.git/config`:
+10 -2
View File
@@ -88,7 +88,14 @@ be useful when deploying automatic fixes.
If the repo command that triggered the hook supports a "yes" option (e.g., If the repo command that triggered the hook supports a "yes" option (e.g.,
`repo upload --yes`), this option is propagated to the hook's `main` function `repo upload --yes`), this option is propagated to the hook's `main` function
as `yes` parameter (defaulting to `False`). Hooks can use this to bypass as `yes` parameter (defaulting to `False`). Hooks can use this to bypass
interactive confirmation prompts when they can automatically fix issues. interactive confirmation prompts for safe non-modifying operations.
### Automated Fixes
If the repo command that triggered the hook supports a "fix" option (e.g.,
`repo upload --fix`), this option is propagated to the hook's `main` function
as `fix` parameter (defaulting to `False`). Hooks can use this to automatically
apply fixes without prompting the user.
### Shebang Handling ### Shebang Handling
@@ -126,7 +133,7 @@ This hook runs when people run `repo upload`.
The `pre-upload.py` file should be defined like: The `pre-upload.py` file should be defined like:
```py ```py
def main(project_list, worktree_list=None, yes=False, **kwargs): def main(project_list, worktree_list=None, fix=False, yes=False, **kwargs):
"""Main function invoked directly by repo. """Main function invoked directly by repo.
We must use the name "main" as that is what repo requires. We must use the name "main" as that is what repo requires.
@@ -137,6 +144,7 @@ def main(project_list, worktree_list=None, yes=False, **kwargs):
project_list, so that each entry in project_list matches with a project_list, so that each entry in project_list matches with a
directory in worktree_list. If None, we will attempt to calculate directory in worktree_list. If None, we will attempt to calculate
the directories automatically. the directories automatically.
fix: Whether to automatically apply fixes without prompting.
yes: Whether to answer yes to all safe prompts (see yes: Whether to answer yes to all safe prompts (see
[Safe Prompts](#safe-prompts)). [Safe Prompts](#safe-prompts)).
kwargs: Leave this here for forward-compatibility. kwargs: Leave this here for forward-compatibility.
+126
View File
@@ -0,0 +1,126 @@
# Reproject Command Contract
The `repo.reprojectcmd` configuration names a command that `repo sync` runs
instead of Git to move a project's index and worktree to the tree of the
target commit. It is the checkout-side counterpart of `repo.fetchcmd` (see
`docs/fetch-cmd.md`): together they let an external tool take over both the
network fetch and the materialization of a project. This is useful on
virtualized filesystems that address content by hash, where a tree can be
materialized far faster than `git checkout` can write every file.
The command only materializes the tree. `repo` then makes the ref write that
Git would have made, using `git update-ref`.
## Configuration
To use this feature, set the following in `.repo/manifests.git/config`:
```ini
[repo]
reprojectcmd = "your custom command here"
uselocalgitdirs = true
```
Setting `repo.reprojectcmd` **requires** `repo.uselocalgitdirs` to be set to
`true`.
For reference, this command does with Git what `repo` would otherwise do
itself:
```ini
[repo]
reprojectcmd = "git -C $REPO_PATH read-tree -m -u $REPO_TREV"
uselocalgitdirs = true
```
The one-tree merge applies the change to the target, keeps local changes to
every other path, and refuses to overwrite a modified or untracked file, so it
enforces the preconditions below by itself. It also works for a project that
has nothing checked out yet.
## Environment Variables
The command is executed in a subshell, from the root of the client, populated
with standard project-context environment variables. For details on standard
variables (such as `REPO_PROJECT`, `REPO_PATH`, `REPO_REMOTE`, etc.), see the
Environment section in `repo help forall` or `subcmds/forall.py`.
The variables the command typically needs are:
* `REPO_PATH`: The project path relative to the root of the client.
* `REPO_TREV`: The target revision resolved to a full commit hash. Match this
commit's tree.
There is no force mode: a project that would need one never reaches the
command (see the preconditions below).
## When the command runs
`repo sync` already classifies each project and picks a Git operation. The
command replaces the three that are a materialization of a target tree:
1. The checkout that detaches HEAD at the target. This is the common case: a
project on a detached HEAD, a project on a branch that does not track
upstream, and `repo sync -d`.
2. The fast-forward of the checked out branch to the target.
3. The hard reset of the checked out branch to the target, when the commits
it carried were dropped upstream.
After the command exits 0, `repo` writes the ref itself: it detaches `HEAD` at
`REPO_TREV`, or moves the checked out branch to `REPO_TREV`.
The command is **not** run:
* When `HEAD` already names `REPO_TREV`.
* At the fast-forward step when `HEAD` is ahead of `REPO_TREV`, where Git's
merge would be a no-op.
* For a rebase. A branch carrying local commits has them replayed onto the
target by `git rebase`, which is not a materialization of a target tree.
* For `MetaProject`s (i.e. the internal `repo` repository itself at
`.repo/repo` and the `manifests` repository at `.repo/manifests`).
## Contract
### Preconditions
Before invoking the command, `repo` ensures that:
* The index has no staged changes (the index matches `HEAD`, or is empty on an unborn `HEAD`).
* No rebase, cherry-pick, merge, or revert is in progress.
Detecting collisions with untracked files or unstaged working-tree modifications is the responsibility of the reproject command itself (e.g. via `git read-tree -m -u $REPO_TREV` or a custom virtual filesystem checkout tool). If local changes collide with the target tree, the command must abort with a non-zero exit code. Local modifications and untracked files outside the diff between `HEAD` and `REPO_TREV` must be preserved.
### Postconditions on exit 0
After the command exits with status 0, `repo` expects the following
postconditions to be met:
1. `git diff-index --quiet --cached REPO_TREV^{tree}` exits 0 (the index
matches the target tree).
2. `HEAD` still names what it did before the command, and its resolved commit
object ID has not changed.
### Invariants
The command may modify the worktree and the index, and may write project-local
Git config. The command must:
* Apply the change from `HEAD`'s tree to `REPO_TREV`'s tree and leave every
other path alone. Local modifications and untracked files outside that
change must survive: the command applies a diff, it does not reset the
tree.
* Not write any ref, including `HEAD` and `ORIG_HEAD`. `repo` owns every ref
write.
* Not create or replace `.git/`, and not touch anything under `.repo/`.
* Not require the Git remote, to preserve `repo sync --local-only`.
* Be idempotent. Running it twice on the same target is a no-op.
### Failure
* A non-zero exit status, a failed precondition or a failed postcondition
fails that project's sync, and the command's or Git's output is surfaced
to the user.
* Other projects continue, and `repo sync` exits non-zero.
## Limitations
Nested projects are out of scope: a project whose path lies inside another
project's path, a `<project>` nested in another `<project>` in the manifest,
and a submodule discovered with `sync-s` or `--recurse-submodules`. `repo sync`
fails if the manifest has one while `repo.reprojectcmd` is set.
+56 -8
View File
@@ -70,6 +70,19 @@ class _GitCall:
git = _GitCall() git = _GitCall()
def IsValidBranchName(name: str) -> bool:
"""Return whether |name| is valid where Git expects a branch name."""
p = GitCommand(
None,
["check-ref-format", "--branch", name],
capture_stdout=True,
capture_stderr=True,
add_event_log=False,
log_as_error=False,
)
return p.Wait() == 0
def RepoSourceVersion(): def RepoSourceVersion():
"""Return the version of the repo.git tree.""" """Return the version of the repo.git tree."""
ver = getattr(RepoSourceVersion, "version", None) ver = getattr(RepoSourceVersion, "version", None)
@@ -270,6 +283,7 @@ class GitCommand:
bare=False, bare=False,
input=None, input=None,
capture_stdout=False, capture_stdout=False,
capture_stdout_bytes: bool = False,
capture_stderr=False, capture_stderr=False,
merge_output=False, merge_output=False,
disable_editor=False, disable_editor=False,
@@ -291,6 +305,12 @@ class GitCommand:
self.cmdv = cmdv self.cmdv = cmdv
self.verify_command = verify_command self.verify_command = verify_command
self.stdout, self.stderr = None, None self.stdout, self.stderr = None, None
if capture_stdout_bytes:
if merge_output:
raise ValueError(
"capture_stdout_bytes cannot be combined with merge_output"
)
capture_stdout = True
# Git on Windows wants its paths only using / for reliability. # Git on Windows wants its paths only using / for reliability.
if platform_utils.isWindows(): if platform_utils.isWindows():
@@ -334,6 +354,7 @@ class GitCommand:
command, command,
env, env,
capture_stdout=capture_stdout, capture_stdout=capture_stdout,
capture_stdout_bytes=capture_stdout_bytes,
capture_stderr=capture_stderr, capture_stderr=capture_stderr,
merge_output=merge_output, merge_output=merge_output,
ssh_proxy=ssh_proxy, ssh_proxy=ssh_proxy,
@@ -367,6 +388,7 @@ class GitCommand:
command, command,
env, env,
capture_stdout=False, capture_stdout=False,
capture_stdout_bytes: bool = False,
capture_stderr=False, capture_stderr=False,
merge_output=False, merge_output=False,
ssh_proxy=None, ssh_proxy=None,
@@ -399,6 +421,10 @@ class GitCommand:
# See go/tee-repo-stderr for more context. # See go/tee-repo-stderr for more context.
tee_stderr = False tee_stderr = False
kwargs = {"encoding": "utf-8", "errors": "backslashreplace"} kwargs = {"encoding": "utf-8", "errors": "backslashreplace"}
if capture_stdout_bytes:
kwargs = {}
if isinstance(input, str):
input = input.encode("utf-8", "surrogateescape")
if not (stdin or stdout or stderr): if not (stdin or stdout or stderr):
tee_stderr = True tee_stderr = True
# stderr will be written back to sys.stderr even though it is # stderr will be written back to sys.stderr even though it is
@@ -477,6 +503,10 @@ class GitCommand:
self.stderr = self._Tee(p.stderr, sys.stderr) self.stderr = self._Tee(p.stderr, sys.stderr)
else: else:
self.stdout, self.stderr = p.communicate(input=input) self.stdout, self.stderr = p.communicate(input=input)
if capture_stdout_bytes and isinstance(self.stderr, bytes):
self.stderr = self.stderr.decode(
"utf-8", "backslashreplace"
).replace("\r\n", "\n")
finally: finally:
if ssh_proxy: if ssh_proxy:
ssh_proxy.remove_client(p) ssh_proxy.remove_client(p)
@@ -528,17 +558,35 @@ class GitCommand:
env.pop(key, None) env.pop(key, None)
return env return env
def VerifyCommand(self): def VerifyCommand(self) -> None:
if self.rc == 0: if self.rc == 0:
return None return None
stdout = ( raw_stdout = self.stdout
"\n".join(self.stdout.split("\n")[:GIT_ERROR_STDOUT_LINES]) if isinstance(raw_stdout, bytes):
if self.stdout first_records = re.split(
else None rb"\r\n|[\r\n\0]", raw_stdout, maxsplit=GIT_ERROR_STDOUT_LINES
) )[:GIT_ERROR_STDOUT_LINES]
stdout = (
"\n".join(
r.decode("utf-8", "backslashreplace") for r in first_records
)
if raw_stdout
else None
)
elif raw_stdout:
first_records = re.split(
r"\r\n|[\r\n\0]", raw_stdout, maxsplit=GIT_ERROR_STDOUT_LINES
)[:GIT_ERROR_STDOUT_LINES]
stdout = "\n".join(first_records)
else:
stdout = None
raw_stderr = self.stderr
if isinstance(raw_stderr, bytes):
raw_stderr = raw_stderr.decode("utf-8", "backslashreplace")
stderr = ( stderr = (
"\n".join(self.stderr.split("\n")[:GIT_ERROR_STDERR_LINES]) "\n".join(raw_stderr.split("\n")[:GIT_ERROR_STDERR_LINES])
if self.stderr if raw_stderr
else None else None
) )
project = self.project.name if self.project else None project = self.project.name if self.project else None
+1 -1
View File
@@ -40,7 +40,7 @@ from repo_trace import Trace
# that is saved in the config. # that is saved in the config.
SYNC_STATE_PREFIX = "repo.syncstate." SYNC_STATE_PREFIX = "repo.syncstate."
ID_RE = re.compile(r"^[0-9a-f]{40,64}$") ID_RE = re.compile(r"^(?:[0-9a-f]{40}|[0-9a-f]{64})$")
REVIEW_CACHE = {} REVIEW_CACHE = {}
+37 -6
View File
@@ -14,6 +14,7 @@
import os import os
from git_command import git_require
from git_command import GitCommand from git_command import GitCommand
import platform_utils import platform_utils
from repo_trace import Trace from repo_trace import Trace
@@ -41,6 +42,17 @@ class GitRefs:
self._EnsureLoaded() self._EnsureLoaded()
return self._phyref return self._phyref
@property
def head(self) -> str:
"""Return HEAD's symbolic target or detached object ID."""
self._EnsureLoaded()
return self._symref.get(HEAD) or self._phyref.get(HEAD, "")
@property
def is_loaded(self) -> bool:
"""Whether a ref snapshot has already been loaded."""
return self._phyref is not None
def get(self, name): def get(self, name):
try: try:
return self.all[name] return self.all[name]
@@ -87,8 +99,12 @@ class GitRefs:
self._symref = {} self._symref = {}
self._mtime = {} self._mtime = {}
self._ReadRefs() root_refs_loaded = self._ReadRefs()
self._ReadSymbolicRef(HEAD) if not root_refs_loaded or (
HEAD not in self._phyref and HEAD not in self._symref
):
# --include-root-refs does not report an unborn HEAD.
self._ReadSymbolicRef(HEAD)
scan = self._symref scan = self._symref
attempts = 0 attempts = 0
@@ -113,18 +129,32 @@ class GitRefs:
"""Check if a ref_id is a null object ID.""" """Check if a ref_id is a null object ID."""
return ref_id and all(ch == "0" for ch in ref_id) return ref_id and all(ch == "0" for ch in ref_id)
def _ReadRefs(self) -> None: def _ReadRefs(self) -> bool:
"""Read all references using git for-each-ref.""" """Read all references using git for-each-ref.
Returns:
Whether root refs, including HEAD when it exists, were loaded.
"""
include_root_refs = git_require((2, 45, 0))
cmd = [
"for-each-ref",
"--format=%(objectname)%00%(refname)%00%(symref)",
]
if include_root_refs:
cmd.insert(1, "--include-root-refs")
# Avoid caching volatile root refs such as ORIG_HEAD. HEAD and
# refs/* are the only namespaces GitRefs exposes to callers.
cmd.extend([HEAD, "refs"])
p = GitCommand( p = GitCommand(
None, None,
["for-each-ref", "--format=%(objectname)%00%(refname)%00%(symref)"], cmd,
capture_stdout=True, capture_stdout=True,
capture_stderr=True, capture_stderr=True,
bare=True, bare=True,
gitdir=self._gitdir, gitdir=self._gitdir,
) )
if p.Wait() != 0: if p.Wait() != 0:
return return False
for line in p.stdout.splitlines(): for line in p.stdout.splitlines():
ref_id, name, symref = line.split("\0") ref_id, name, symref = line.split("\0")
@@ -132,6 +162,7 @@ class GitRefs:
self._symref[name] = symref self._symref[name] = symref
elif ref_id and not self._IsNullRef(ref_id): elif ref_id and not self._IsNullRef(ref_id):
self._phyref[name] = ref_id self._phyref[name] = ref_id
return include_root_refs
def _ReadSymbolicRef(self, name: str) -> None: def _ReadSymbolicRef(self, name: str) -> None:
"""Read a symbolic reference.""" """Read a symbolic reference."""
+259
View File
@@ -0,0 +1,259 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Read a worktree's state from one machine-readable git status snapshot."""
from collections import OrderedDict
import os
from typing import Iterator, List, Optional, TYPE_CHECKING
from git_command import git_require
from git_command import GitCommand
if TYPE_CHECKING:
from project import Project
class StatusEntry:
"""The state of one path on one side of the index."""
def __init__(
self,
path: str,
status: str,
src_path: Optional[str] = None,
level: Optional[str] = None,
) -> None:
self.path = path
self.status = status
self.src_path = src_path
self.level = level
class StatusSnapshot:
"""A consistent view of worktree, index, and branch state."""
def __init__(self) -> None:
self.index_changes = OrderedDict()
self.worktree_changes = OrderedDict()
self.untracked = []
self.branch_oid = None
self.branch_head = None
self.upstream = None
self.ahead = 0
self.behind = 0
self.has_ahead_behind = False
self.stash_count = 0
@property
def current_branch(self) -> Optional[str]:
if self.branch_head in (None, "(detached)", "(unknown)"):
return None
return self.branch_head
def is_dirty(self, consider_untracked: bool = True) -> bool:
return bool(
self.index_changes
or self.worktree_changes
or (consider_untracked and self.untracked)
)
def GetStatus(
project: "Project",
gitdir: str,
untracked_files: str = "all",
branch: bool = False,
ahead_behind: bool = False,
show_stash: bool = False,
) -> StatusSnapshot:
"""Return one machine-readable status snapshot for |project|."""
if not git_require((2, 11, 0)):
raise UnsupportedStatusError("porcelain v2 requires Git 2.11")
cmd = [
"status",
"--porcelain=v2",
"-z",
"--ignore-submodules=all",
f"--untracked-files={untracked_files}",
]
if branch:
cmd.append("--branch")
if git_require((2, 17, 0)):
cmd.append(
"--ahead-behind" if ahead_behind else "--no-ahead-behind"
)
if git_require((2, 18, 0)):
# Match the existing staged diff's explicit rename detection even if
# status.renames is disabled in the user's config.
cmd.append("--renames")
if show_stash and git_require((2, 35, 0)):
cmd.append("--show-stash")
p = GitCommand(
project,
cmd,
bare=False,
gitdir=gitdir,
capture_stdout=True,
capture_stdout_bytes=True,
capture_stderr=True,
verify_command=True,
)
p.Wait()
return ParsePorcelainV2(p.stdout)
def _Path(value: bytes) -> str:
"""Decode a Git pathname without losing undecodable bytes."""
return os.fsdecode(value)
def _Status(value: int) -> str:
"""Normalize Git's unchanged markers for repo's status display."""
char = chr(value)
return "" if char == "." else char
def _Records(output: bytes) -> Iterator[bytes]:
if not output:
return iter(())
if not output.endswith(b"\0"):
raise StatusParseError("porcelain v2 output is not NUL terminated")
records = output.split(b"\0")
if not records[-1]:
records.pop()
return iter(records)
class StatusParseError(ValueError):
"""Raised when machine-readable status output is malformed."""
class UnsupportedStatusError(RuntimeError):
"""Raised when the Git client cannot produce porcelain v2."""
def _Fields(record: bytes, count: int) -> List[bytes]:
fields = record.split(b" ", count - 1)
if len(fields) != count:
raise StatusParseError(f"malformed porcelain v2 record: {record!r}")
return fields
def _AddTracked(
status: StatusSnapshot,
path: str,
xy: bytes,
src_path: Optional[str] = None,
level: Optional[str] = None,
) -> None:
index_status = _Status(xy[0])
worktree_status = _Status(xy[1])
if index_status:
status.index_changes[path] = StatusEntry(
path,
index_status,
src_path=src_path if index_status in ("R", "C") else None,
level=level if index_status in ("R", "C") else None,
)
if worktree_status:
status.worktree_changes[path] = StatusEntry(
path,
worktree_status,
src_path=src_path if worktree_status in ("R", "C") else None,
level=level if worktree_status in ("R", "C") else None,
)
def ParsePorcelainV2(output: bytes) -> StatusSnapshot:
"""Parse ``git status --porcelain=v2 -z --branch`` output."""
status = StatusSnapshot()
records = _Records(output)
for record in records:
kind = record[:1]
if kind == b"#":
try:
key, value = record[2:].split(b" ", 1)
except ValueError as e:
raise StatusParseError(
f"malformed porcelain v2 header: {record!r}"
) from e
if key == b"branch.oid":
value = value.decode("ascii")
status.branch_oid = None if value == "(initial)" else value
elif key == b"branch.head":
status.branch_head = _Path(value)
elif key == b"branch.upstream":
status.upstream = _Path(value)
elif key == b"branch.ab":
try:
value = value.decode("ascii")
ahead, behind = value.split()
if ahead != "+?" and behind != "-?":
status.ahead = int(ahead)
status.behind = -int(behind)
status.has_ahead_behind = True
except ValueError as e:
raise StatusParseError(
f"malformed porcelain v2 branch.ab record: {record!r}"
) from e
elif key == b"stash":
status.stash_count = int(value.decode("ascii"))
continue
if kind == b"1":
fields = _Fields(record, 9)
xy = fields[1]
if len(xy) != 2:
raise StatusParseError(f"invalid status pair: {xy!r}")
path = _Path(fields[8])
_AddTracked(status, path, xy)
elif kind == b"2":
fields = _Fields(record, 10)
xy = fields[1]
if len(xy) != 2:
raise StatusParseError(f"invalid status pair: {xy!r}")
score = fields[8][1:].lstrip(b"0") or b"0"
try:
src_path = _Path(next(records))
except StopIteration as e:
raise StatusParseError(
"rename record has no source path"
) from e
path = _Path(fields[9])
_AddTracked(
status,
path,
xy,
src_path=src_path,
level=score.decode("ascii"),
)
elif kind == b"u":
fields = _Fields(record, 11)
path = _Path(fields[10])
# The old diff-index/diff-files pair reported unmerged paths on
# both sides, regardless of porcelain's more specific XY pair.
status.index_changes[path] = StatusEntry(path, "U")
status.worktree_changes[path] = StatusEntry(path, "U")
elif kind == b"?":
status.untracked.append(_Path(record[2:]))
elif kind == b"!":
continue
else:
raise StatusParseError(
f"unknown porcelain v2 record type: {record!r}"
)
return status
+20 -3
View File
@@ -36,7 +36,6 @@ from git_command import git_require
from git_command import GitCommand from git_command import GitCommand
from git_config import IsId from git_config import IsId
from git_config import RepoConfig from git_config import RepoConfig
from git_refs import GitRefs
import platform_utils import platform_utils
@@ -189,7 +188,7 @@ class Superproject:
if netloc: if netloc:
parts = netloc.split("-review", 1) parts = netloc.split("-review", 1)
host = parts[0] host = parts[0]
rev = GitRefs(self._work_git).get("HEAD") rev = self._GetRef("HEAD")
return f"{host}/{self.name}@{rev}" return f"{host}/{self.name}@{rev}"
return None return None
@@ -314,7 +313,10 @@ class Superproject:
# We use --negotiation-tip to speed up the fetch. Superproject branches # We use --negotiation-tip to speed up the fetch. Superproject branches
# do not share commits. So this lets git know it only needs to send # do not share commits. So this lets git know it only needs to send
# commits reachable from the specified local refs. # commits reachable from the specified local refs.
rev_commit = GitRefs(self._work_git).get(f"refs/heads/{self.revision}") negotiation_ref = self.revision
if negotiation_ref and not negotiation_ref.startswith("refs/"):
negotiation_ref = f"refs/heads/{negotiation_ref}"
rev_commit = self._GetRef(negotiation_ref) if negotiation_ref else ""
if rev_commit: if rev_commit:
cmd.extend(["--negotiation-tip", rev_commit]) cmd.extend(["--negotiation-tip", rev_commit])
@@ -347,6 +349,21 @@ class Superproject:
return False return False
return True return True
def _GetRef(self, ref: str) -> str:
"""Resolve one local ref without loading the entire ref namespace."""
p = GitCommand(
None,
["rev-parse", "--verify", "--quiet", ref],
gitdir=self._work_git,
bare=True,
capture_stdout=True,
capture_stderr=True,
log_as_error=False,
)
if p.Wait() == 0:
return p.stdout.strip()
return ""
def _LsTree(self): def _LsTree(self):
"""Gets the commit ids for all projects. """Gets the commit ids for all projects.
+18 -1
View File
@@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import optparse
import os import os
import re import re
import sys import sys
@@ -69,6 +70,7 @@ class RepoHook:
ignore_hooks=False, ignore_hooks=False,
abort_if_user_denies=False, abort_if_user_denies=False,
yes=False, yes=False,
fix=False,
): ):
"""RepoHook constructor. """RepoHook constructor.
@@ -91,6 +93,7 @@ class RepoHook:
abort_if_user_denies: If True, we'll abort running the hook if the abort_if_user_denies: If True, we'll abort running the hook if the
user doesn't allow us to run the hook. user doesn't allow us to run the hook.
yes: If True, then 'Yes' is assumed for any prompts. yes: If True, then 'Yes' is assumed for any prompts.
fix: If True, then 'Fix' is assumed for any fixup prompts.
""" """
self._hook_type = hook_type self._hook_type = hook_type
self._hooks_project = hooks_project self._hooks_project = hooks_project
@@ -102,6 +105,7 @@ class RepoHook:
self._ignore_hooks = ignore_hooks self._ignore_hooks = ignore_hooks
self._abort_if_user_denies = abort_if_user_denies self._abort_if_user_denies = abort_if_user_denies
self._yes = yes self._yes = yes
self._fix = fix
# Store the full path to the script for convenience. # Store the full path to the script for convenience.
self._script_fullpath = None self._script_fullpath = None
@@ -380,6 +384,7 @@ class RepoHook:
kwargs = { kwargs = {
**kwargs, **kwargs,
"hook_should_take_kwargs": True, "hook_should_take_kwargs": True,
"fix": self._fix,
"yes": self._yes, "yes": self._yes,
} }
@@ -504,12 +509,17 @@ class RepoHook:
).url, ).url,
"bug_url": manifest.contactinfo.bugurl, "bug_url": manifest.contactinfo.bugurl,
"yes": getattr(opt, "yes", False), "yes": getattr(opt, "yes", False),
"fix": getattr(opt, "fix", False),
} }
) )
return cls(*args, **kwargs) return cls(*args, **kwargs)
@staticmethod @staticmethod
def AddOptionGroup(parser, name): def AddOptionGroup(
parser: optparse.OptionParser,
name: str,
allow_fix: bool = False,
) -> None:
"""Help options relating to the various hooks.""" """Help options relating to the various hooks."""
# Note that verify and no-verify are NOT opposites of each other, which # Note that verify and no-verify are NOT opposites of each other, which
@@ -533,3 +543,10 @@ class RepoHook:
action="store_true", action="store_true",
help="Do not abort if %s hooks fail." % name, help="Do not abort if %s hooks fail." % name,
) )
if allow_fix:
group.add_option(
"--fix",
action="store_true",
default=False,
help="Automatically apply %s fixes without prompting." % name,
)
+5
View File
@@ -104,6 +104,11 @@ elif sys.version_info < MIN_PYTHON_VERSION_SOFT:
KEYBOARD_INTERRUPT_EXIT = 128 + signal.SIGINT KEYBOARD_INTERRUPT_EXIT = 128 + signal.SIGINT
MAX_PRINT_ERRORS = 5 MAX_PRINT_ERRORS = 5
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(errors="surrogateescape")
if hasattr(sys.stderr, "reconfigure"):
sys.stderr.reconfigure(errors="surrogateescape")
global_options = optparse.OptionParser( global_options = optparse.OptionParser(
usage="repo [-p|--paginate|--no-pager] COMMAND [ARGS]", usage="repo [-p|--paginate|--no-pager] COMMAND [ARGS]",
add_help_option=False, add_help_option=False,
+2 -2
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man. .\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2026" "repo smartsync" "Repo Manual" .TH REPO "1" "September 2026" "repo smartsync" "Repo Manual"
.SH NAME .SH NAME
repo \- repo smartsync - manual page for repo smartsync repo \- repo smartsync - manual page for repo smartsync
.SH SYNOPSIS .SH SYNOPSIS
@@ -21,7 +21,7 @@ number of jobs to run in parallel (default: based on number of CPU cores)
number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP .TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR \fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-no\-interleaved\fR is set number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or based on number of CPU cores). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP .TP
\fB\-f\fR, \fB\-\-force\-broken\fR \fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future) obsolete option (to be deleted in the future)
+2 -2
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man. .\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2026" "repo sync" "Repo Manual" .TH REPO "1" "September 2026" "repo sync" "Repo Manual"
.SH NAME .SH NAME
repo \- repo sync - manual page for repo sync repo \- repo sync - manual page for repo sync
.SH SYNOPSIS .SH SYNOPSIS
@@ -21,7 +21,7 @@ number of jobs to run in parallel (default: based on number of CPU cores)
number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set number of network jobs to run in parallel (defaults to \fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP .TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR \fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-no\-interleaved\fR is set number of local checkout jobs to run in parallel (defaults to \fB\-\-jobs\fR or based on number of CPU cores). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP .TP
\fB\-f\fR, \fB\-\-force\-broken\fR \fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future) obsolete option (to be deleted in the future)
+4 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man. .\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2026" "repo upload" "Repo Manual" .TH REPO "1" "August 2026" "repo upload" "Repo Manual"
.SH NAME .SH NAME
repo \- repo upload - manual page for repo upload repo \- repo upload - manual page for repo upload
.SH SYNOPSIS .SH SYNOPSIS
@@ -112,6 +112,9 @@ Run the pre\-upload hook without prompting.
.TP .TP
\fB\-\-ignore\-hooks\fR \fB\-\-ignore\-hooks\fR
Do not abort if pre\-upload hooks fail. Do not abort if pre\-upload hooks fail.
.TP
\fB\-\-fix\fR
Automatically apply pre\-upload fixes without prompting.
.PP .PP
Run `repo help upload` to view the detailed manual. Run `repo help upload` to view the detailed manual.
.SH DETAILS .SH DETAILS
+14 -18
View File
@@ -692,9 +692,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
e.setAttribute("remote", remoteName) e.setAttribute("remote", remoteName)
if peg_rev: if peg_rev:
if self.IsMirror: if self.IsMirror:
value = p.bare_git.rev_parse(p.revisionExpr + "^0") value = p.bare_git.ResolveCommit(p.revisionExpr)
else: else:
value = p.work_git.rev_parse(HEAD + "^0") value = p.work_git.ResolveCommit(HEAD)
e.setAttribute("revision", value) e.setAttribute("revision", value)
if peg_rev_upstream: if peg_rev_upstream:
if p.upstream: if p.upstream:
@@ -1519,9 +1519,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if base_revision: if base_revision:
if p.revisionExpr != base_revision: if p.revisionExpr != base_revision:
failed_revision_changes.append( failed_revision_changes.append(
"extend-project name %s mismatch base " f"extend-project name {name}:\n "
"%s vs revision %s" f"base {base_revision} vs "
% (name, base_revision, p.revisionExpr) f"revision {p.revisionExpr}"
) )
p.SetRevision(revision) p.SetRevision(revision)
@@ -1622,9 +1622,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if base_revision: if base_revision:
if p.revisionExpr != base_revision: if p.revisionExpr != base_revision:
failed_revision_changes.append( failed_revision_changes.append(
"remove-project name %s mismatch base " f"remove-project name {name}:\n "
"%s vs revision %s" f"base {base_revision} vs "
% (name, base_revision, p.revisionExpr) f"revision {p.revisionExpr}"
) )
del self._paths[p.relpath] del self._paths[p.relpath]
if not removed_project: if not removed_project:
@@ -1636,13 +1636,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if base_revision: if base_revision:
if p.revisionExpr != base_revision: if p.revisionExpr != base_revision:
failed_revision_changes.append( failed_revision_changes.append(
"remove-project path %s mismatch base " f"remove-project path {p.relpath}:\n "
"%s vs revision %s" f"base {base_revision} vs "
% ( f"revision {p.revisionExpr}"
p.relpath,
base_revision,
p.revisionExpr,
)
) )
self._projects[projname].remove(p) self._projects[projname].remove(p)
del self._paths[p.relpath] del self._paths[p.relpath]
@@ -1664,10 +1660,10 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
) )
if failed_revision_changes: if failed_revision_changes:
fail_string = "\n".join(failed_revision_changes)
raise ManifestParseError( raise ManifestParseError(
"revision base check failed, rebase patches and update " f"detected base-revision mismatch, updates needed:\n"
"base revs for: ", f"{fail_string}",
failed_revision_changes,
) )
# Store repo hooks project information. # Store repo hooks project information.
+886 -167
View File
File diff suppressed because it is too large Load Diff
+2 -1
View File
@@ -23,4 +23,5 @@ import sys
import update_manpages import update_manpages
sys.exit(update_manpages.main(sys.argv[1:])) if __name__ == "__main__":
sys.exit(update_manpages.main(sys.argv[1:]))
+2 -4
View File
@@ -20,7 +20,7 @@ from command import Command
from command import DEFAULT_LOCAL_JOBS from command import DEFAULT_LOCAL_JOBS
from error import RepoError from error import RepoError
from error import RepoExitError from error import RepoExitError
from git_command import git from git_command import IsValidBranchName
from progress import Progress from progress import Progress
from repo_logging import RepoLogger from repo_logging import RepoLogger
@@ -58,9 +58,7 @@ It is equivalent to "git branch -D <branchname>".
if not opt.all: if not opt.all:
branches = args[0].split() branches = args[0].split()
invalid_branches = [ invalid_branches = [x for x in branches if not IsValidBranchName(x)]
x for x in branches if not git.check_ref_format(f"heads/{x}")
]
if invalid_branches: if invalid_branches:
self.OptionParser.error( self.OptionParser.error(
+45 -30
View File
@@ -14,6 +14,7 @@
import re import re
import sys import sys
from typing import Tuple
from command import Command from command import Command
from error import GitError from error import GitError
@@ -43,36 +44,8 @@ change id will be added.
def Execute(self, opt, args): def Execute(self, opt, args):
reference = args[0] reference = args[0]
sha1, commit = self._ResolveReference(reference)
p = GitCommand( old_msg = self._StripHeader(commit)
None,
["rev-parse", "--verify", reference],
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
try:
p.Wait()
except GitError:
logger.error(p.stderr)
raise
sha1 = p.stdout.strip()
p = GitCommand(
None,
["cat-file", "commit", sha1],
capture_stdout=True,
verify_command=True,
)
try:
p.Wait()
except GitError:
logger.error("error: Failed to retrieve old commit message")
raise
old_msg = self._StripHeader(p.stdout)
p = GitCommand( p = GitCommand(
None, None,
@@ -117,6 +90,48 @@ change id will be added.
logger.error("error: Failed to update commit message") logger.error("error: Failed to update commit message")
raise raise
def _ResolveReference(self, reference: str) -> Tuple[str, str]:
"""Resolve a commit and read it through one cat-file batch request."""
expression = f"{reference}^{{commit}}"
p = GitCommand(
None,
["cat-file", "--batch"],
input=expression + "\n",
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
try:
p.Wait()
header, separator, output = p.stdout.partition("\n")
if not separator:
raise ValueError("missing cat-file header")
if header.endswith(" missing") or header.endswith(" ambiguous"):
raise GitError(f"commit {reference} not found")
parts = header.split(" ", 2)
if len(parts) != 3 or parts[1] != "commit":
raise ValueError(
f"unexpected object type {parts[1]!r}"
if len(parts) >= 2
else "invalid header"
)
sha1, _object_type, _size = parts
if not output.endswith("\n"):
raise ValueError("truncated cat-file object")
commit = output[:-1]
except (GitError, ValueError) as e:
logger.error(
"error: Failed to resolve or read commit %s", reference
)
if isinstance(e, GitError):
raise
raise GitError(str(e)) from e
return sha1, commit
def _IsChangeId(self, line): def _IsChangeId(self, line):
return CHANGE_ID_RE.match(line) return CHANGE_ID_RE.match(line)
+1 -1
View File
@@ -195,7 +195,7 @@ If no project is specified try to use current directory as a project.
elif opt.revert: elif opt.revert:
project._Revert(dl.commit) project._Revert(dl.commit)
elif opt.ffonly: elif opt.ffonly:
project._FastForward(dl.commit, ffonly=True) project._FastForward(dl.commit)
else: else:
if opt.branch: if opt.branch:
project.StartBranch(opt.branch, revision=dl.commit) project.StartBranch(opt.branch, revision=dl.commit)
+6 -2
View File
@@ -244,10 +244,14 @@ without iterating through the remaining projects.
mirror = self.manifest.IsMirror mirror = self.manifest.IsMirror
if opt.regex: if opt.regex:
projects = self.FindProjects(args, all_manifests=all_trees) projects = self.FindProjects(
args,
groups=opt.groups,
all_manifests=all_trees,
)
elif opt.inverse_regex: elif opt.inverse_regex:
projects = self.FindProjects( projects = self.FindProjects(
args, inverse=True, all_manifests=all_trees args, inverse=True, groups=opt.groups, all_manifests=all_trees
) )
else: else:
projects = self.GetProjects( projects = self.GetProjects(
+58 -26
View File
@@ -18,7 +18,7 @@ import io
import json import json
import optparse import optparse
import sys import sys
from typing import Any, Dict, List, NamedTuple from typing import Any, Dict, List, NamedTuple, Optional, Tuple
from color import Coloring from color import Coloring
from command import DEFAULT_LOCAL_JOBS from command import DEFAULT_LOCAL_JOBS
@@ -189,18 +189,26 @@ class Info(PagedCommand):
"superproject_revision": srev, "superproject_revision": srev,
} }
@staticmethod
def _GetCurrentBranch(branches: Dict[str, Any]) -> Optional[str]:
"""Return the name of the current branch from a GetBranches mapping."""
return next(
(name for name, branch in branches.items() if branch.current), None
)
@classmethod @classmethod
def _getProjectData(cls, project) -> Dict[str, Any]: def _getProjectData(cls, project) -> Dict[str, Any]:
"""Gather project data as a dict.""" """Gather project data as a dict."""
branches = project.GetBranches()
currentBranch = cls._GetCurrentBranch(branches)
data = { data = {
"name": project.name, "name": project.name,
"mount_path": project.worktree, "mount_path": project.worktree,
"current_revision": project.GetHeadRevisionId() "current_revision": project.GetHeadRevisionId()
or project.GetRevisionId(), or project.GetRevisionId(),
"manifest_revision": project.revisionExpr, "manifest_revision": project.revisionExpr,
"local_branches": list(project.GetBranches()), "local_branches": list(branches),
} }
currentBranch = project.CurrentBranch
if currentBranch: if currentBranch:
data["current_branch"] = currentBranch data["current_branch"] = currentBranch
return data return data
@@ -285,6 +293,9 @@ class Info(PagedCommand):
text = out.nofmt_printer("text") text = out.nofmt_printer("text")
dimtext = out.printer("dimtext", attr="dim") dimtext = out.printer("dimtext", attr="dim")
branches = project.GetBranches()
currentBranch = cls._GetCurrentBranch(branches)
heading("Project: ") heading("Project: ")
headtext(project.name) headtext(project.name)
out.nl() out.nl()
@@ -297,7 +308,6 @@ class Info(PagedCommand):
headtext(project.GetHeadRevisionId() or project.GetRevisionId()) headtext(project.GetHeadRevisionId() or project.GetRevisionId())
out.nl() out.nl()
currentBranch = project.CurrentBranch
if currentBranch: if currentBranch:
heading("Current branch: ") heading("Current branch: ")
headtext(currentBranch) headtext(currentBranch)
@@ -307,7 +317,7 @@ class Info(PagedCommand):
headtext(project.revisionExpr) headtext(project.revisionExpr)
out.nl() out.nl()
localBranches = list(project.GetBranches().keys()) localBranches = list(branches)
heading("Local Branches: ") heading("Local Branches: ")
redtext(str(len(localBranches))) redtext(str(len(localBranches)))
if localBranches: if localBranches:
@@ -327,25 +337,10 @@ class Info(PagedCommand):
branch = branch[len(R_HEADS) :] branch = branch[len(R_HEADS) :]
logTarget = R_M + branch logTarget = R_M + branch
bareTmp = project.bare_git._bare localCommits, originCommits = cls._GetDiffCommits(
project.bare_git._bare = False project, logTarget
localCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
logTarget + "..",
"--",
) )
originCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
".." + logTarget,
"--",
)
project.bare_git._bare = bareTmp
heading("Local Commits: ") heading("Local Commits: ")
redtext(str(len(localCommits))) redtext(str(len(localCommits)))
dimtext(" (on current branch)") dimtext(" (on current branch)")
@@ -375,6 +370,35 @@ class Info(PagedCommand):
return buf.getvalue() return buf.getvalue()
@classmethod
def _GetDiffCommits(
cls, project: Any, log_target: str
) -> Tuple[List[str], List[str]]:
"""Return local-only and remote-only commits from one history walk."""
git = getattr(project, "work_git", None) or getattr(
project, "bare_git", None
)
if git is None:
return [], []
commits = git.rev_list(
"--left-right",
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
f"HEAD...{log_target}",
"--",
)
if isinstance(commits, str):
commits = commits.splitlines()
local = []
remote = []
for commit in commits:
if commit.startswith("<"):
local.append(commit[1:])
elif commit.startswith(">"):
remote.append(commit[1:])
return local, remote
def _printDiffInfo(self, opt, args): def _printDiffInfo(self, opt, args):
projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only) projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only)
@@ -404,10 +428,18 @@ class Info(PagedCommand):
project = cls.get_parallel_context()["projects"][project_idx] project = cls.get_parallel_context()["projects"][project_idx]
branches = [] branches = []
br = [project.GetUploadableBranch(x) for x in project.GetBranches()] local_branches = project.GetBranches()
br = [x for x in br if x] current_branch = cls._GetCurrentBranch(local_branches)
if opt.current_branch: if opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch] candidate_branches = (
[current_branch]
if current_branch and current_branch in local_branches
else []
)
else:
candidate_branches = local_branches
br = [project.GetUploadableBranch(x) for x in candidate_branches]
br = [x for x in br if x]
for b in br: for b in br:
branches.append( branches.append(
@@ -416,7 +448,7 @@ class Info(PagedCommand):
name=b.name, name=b.name,
commits=b.commits, commits=b.commits,
date=b.date, date=b.date,
is_current=b.name == project.CurrentBranch, is_current=b.name == current_branch,
) )
) )
return branches return branches
+4 -1
View File
@@ -109,7 +109,10 @@ This is similar to running: repo forall -c 'echo "$REPO_PATH : $REPO_PROJECT"'.
) )
else: else:
projects = self.FindProjects( projects = self.FindProjects(
args, all_manifests=not opt.this_manifest_only args,
groups=opt.groups,
missing_ok=opt.all,
all_manifests=not opt.this_manifest_only,
) )
def _getpath(x): def _getpath(x):
+9 -4
View File
@@ -59,10 +59,15 @@ are displayed.
for project in self.GetProjects( for project in self.GetProjects(
args, all_manifests=not opt.this_manifest_only args, all_manifests=not opt.this_manifest_only
): ):
br = [project.GetUploadableBranch(x) for x in project.GetBranches()] local_branches = project.GetBranches()
br = [x for x in br if x] br = []
for name, branch in local_branches.items():
uploadable = project.GetUploadableBranch(name)
if uploadable:
uploadable.branch.current = branch.current
br.append(uploadable)
if opt.current_branch: if opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch] br = [x for x in br if x.current]
all_branches.extend(br) all_branches.extend(br)
if not all_branches: if not all_branches:
@@ -97,7 +102,7 @@ are displayed.
print( print(
"%s %-33s (%2d commit%s, %s)" "%s %-33s (%2d commit%s, %s)"
% ( % (
branch.name == project.CurrentBranch and "*" or " ", branch.current and "*" or " ",
branch.name, branch.name,
len(commits), len(commits),
len(commits) != 1 and "s" or " ", len(commits) != 1 and "s" or " ",
+1 -1
View File
@@ -80,7 +80,7 @@ class Prune(PagedCommand):
print( print(
"%s %-33s " "%s %-33s "
% ( % (
branch.name == project.CurrentBranch and "*" or " ", branch.current and "*" or " ",
branch.name, branch.name,
), ),
end="", end="",
+2 -19
View File
@@ -139,6 +139,8 @@ branch but need to incorporate new upstream changes "underneath" them.
common_args.append("--autosquash") common_args.append("--autosquash")
if opt.interactive: if opt.interactive:
common_args.append("-i") common_args.append("-i")
if opt.auto_stash:
common_args.append("--autostash")
config = self.manifest.manifestProject.config config = self.manifest.manifestProject.config
out = RebaseColoring(config) out = RebaseColoring(config)
@@ -188,29 +190,10 @@ branch but need to incorporate new upstream changes "underneath" them.
out.nl() out.nl()
out.flush() out.flush()
needs_stash = False
if opt.auto_stash:
stash_args = ["update-index", "--refresh", "-q"]
if GitCommand(project, stash_args).Wait() != 0:
needs_stash = True
# Dirty index, requires stash...
stash_args = ["stash"]
if GitCommand(project, stash_args).Wait() != 0:
ret += 1
continue
if GitCommand(project, args).Wait() != 0: if GitCommand(project, args).Wait() != 0:
ret += 1 ret += 1
continue continue
if needs_stash:
stash_args.append("pop")
stash_args.append("--quiet")
if GitCommand(project, stash_args).Wait() != 0:
ret += 1
if ret: if ret:
msg_fmt = "%d projects had errors" msg_fmt = "%d projects had errors"
self.git_event_log.ErrorEvent(msg_fmt % (ret), msg_fmt) self.git_event_log.ErrorEvent(msg_fmt % (ret), msg_fmt)
+2 -2
View File
@@ -18,7 +18,7 @@ from typing import NamedTuple
from command import Command from command import Command
from command import DEFAULT_LOCAL_JOBS from command import DEFAULT_LOCAL_JOBS
from error import RepoExitError from error import RepoExitError
from git_command import git from git_command import IsValidBranchName
from git_config import IsImmutable from git_config import IsImmutable
from progress import Progress from progress import Progress
from repo_logging import RepoLogger from repo_logging import RepoLogger
@@ -75,7 +75,7 @@ revision specified in the manifest.
self.Usage() self.Usage()
nb = args[0] nb = args[0]
if not git.check_ref_format("heads/%s" % nb): if not IsValidBranchName(nb):
self.OptionParser.error("'%s' is not a valid name" % nb) self.OptionParser.error("'%s' is not a valid name" % nb)
@classmethod @classmethod
+202 -14
View File
@@ -28,7 +28,7 @@ import subprocess
import sys import sys
import tempfile import tempfile
import time import time
from typing import List, NamedTuple, Optional, Set, Tuple, Union from typing import Dict, List, NamedTuple, Optional, Set, Tuple, Union
import urllib.error import urllib.error
import urllib.parse import urllib.parse
import urllib.request import urllib.request
@@ -72,6 +72,7 @@ from git_refs import HEAD
from git_refs import R_HEADS from git_refs import R_HEADS
import git_superproject import git_superproject
from hooks import RepoHook from hooks import RepoHook
from manifest_xml import XmlManifest
import platform_utils import platform_utils
from progress import elapsed_str from progress import elapsed_str
from progress import jobs_str from progress import jobs_str
@@ -156,6 +157,88 @@ def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
return res return res
def _NestedProjects(projects: List[Project]) -> List[Project]:
"""Return the projects in |projects| living inside another one's path."""
return [p for level in _SafeCheckoutOrder(projects)[1:] for p in level]
def _ParentFirstBatches(projects: List[Project]) -> List[List[Project]]:
"""Group |projects| so that a parent is fetched before its submodules.
A discovered submodule can only be fetched at the right revision once the
project holding its gitlink has been fetched, so it is held back to a later
batch than its parent. Projects that are not discovered submodules all end
up in the first batch, which keeps manifests without submodules on a single
batch.
"""
batches = collections.defaultdict(list)
for project in projects:
depth = 0
ancestor = project
while ancestor.Derived and ancestor.parent:
depth += 1
ancestor = ancestor.parent
batches[depth].append(project)
return [batches[depth] for depth in sorted(batches)]
def _RefreshDerivedRevisions(
projects: List[Project],
submodule_revisions: Optional[Dict[Project, Dict[str, str]]] = None,
) -> List[Project]:
"""Re-resolve the gitlinks of the discovered submodules in |projects|.
The revision of a discovered submodule is read from its parent when the
manifest is loaded, so it is stale as soon as the parent gets fetched. It
has to be resolved again once the parent is up-to-date, and before the
submodule itself is fetched and checked out.
Args:
projects: The projects whose discovered submodules to resolve.
submodule_revisions: Gitlinks already read, keyed by the project
holding them. Passing the same dict for project sets that follow
the same fetches, e.g. the levels of one checkout order, keeps a
project from being read more than once.
Returns:
The submodules that their parent no longer holds a gitlink for.
"""
if submodule_revisions is None:
submodule_revisions = {}
subprojects_by_parent = collections.defaultdict(list)
for project in projects:
if project.Derived and project.parent:
subprojects_by_parent[project.parent].append(project)
removed = []
for parent, subprojects in subprojects_by_parent.items():
revisions = submodule_revisions.get(parent)
if revisions is None:
revisions = parent.GetSubmoduleRevisions()
if revisions is None:
# Leave the submodules of a parent we cannot read alone.
continue
submodule_revisions[parent] = revisions
for subproject in subprojects:
rev = revisions.get(subproject.gitlink_path)
if rev:
subproject.SetRevision(rev, revisionId=rev)
else:
removed.append(subproject)
return removed
def _WithoutProjects(
projects: List[Project], unwanted: List[Project]
) -> List[Project]:
"""Return |projects| without the projects in |unwanted|."""
if not unwanted:
return projects
dropped = set(unwanted)
return [p for p in projects if p not in dropped]
def _chunksize(projects: int, jobs: int) -> int: def _chunksize(projects: int, jobs: int) -> int:
"""Calculate chunk size for the given number of projects and jobs.""" """Calculate chunk size for the given number of projects and jobs."""
return min(max(1, projects // jobs), WORKER_BATCH_SIZE) return min(max(1, projects // jobs), WORKER_BATCH_SIZE)
@@ -224,7 +307,8 @@ class _SyncResult(NamedTuple):
Attributes: Attributes:
project_index (int): The index of the project in the shared list. project_index (int): The index of the project in the shared list.
relpath (str): The project's relative path from the repo client top. relpath (str): The project's path relative to the tree being synced.
Unlike Project.relpath, it is unique across submanifests.
remote_fetched (bool): True if the remote was actually queried. remote_fetched (bool): True if the remote was actually queried.
fetch_success (bool): True if the fetch operation was successful. fetch_success (bool): True if the fetch operation was successful.
fetch_errors (List[Exception]): The Exceptions from a failed fetch. fetch_errors (List[Exception]): The Exceptions from a failed fetch.
@@ -450,6 +534,8 @@ later is required to fix a server side protocol bug.
help="number of network jobs to run in parallel (defaults to " help="number of network jobs to run in parallel (defaults to "
"--jobs or 1). Ignored unless --no-interleaved is set", "--jobs or 1). Ignored unless --no-interleaved is set",
) )
jobs_checkout_default = self._GetHelpForCpuJobCount(DEFAULT_LOCAL_JOBS)
p.add_option( p.add_option(
"--jobs-checkout", "--jobs-checkout",
default=None, default=None,
@@ -457,7 +543,7 @@ later is required to fix a server side protocol bug.
metavar="JOBS", metavar="JOBS",
help=( help=(
"number of local checkout jobs to run in parallel (defaults " "number of local checkout jobs to run in parallel (defaults "
f"to --jobs or {DEFAULT_LOCAL_JOBS}). Ignored unless " f"to --jobs or {jobs_checkout_default}). Ignored unless "
"--no-interleaved is set" "--no-interleaved is set"
), ),
) )
@@ -1067,6 +1153,40 @@ later is required to fix a server side protocol bug.
return _FetchResult(ret, fetched) return _FetchResult(ret, fetched)
def _FetchParentFirst(
self,
projects: List[Project],
opt: optparse.Values,
err_event: _threading.Event,
ssh_proxy: ssh.ProxyManager,
errors: List[Exception],
) -> _FetchResult:
"""Fetch |projects|, holding submodules back until their parent is done.
Args:
projects: Projects to fetch.
opt: Program options returned from optparse. See _Options().
err_event: Whether an error was hit while processing.
ssh_proxy: SSH manager for clients & masters.
errors: A list to accumulate errors.
Returns:
_FetchResult for all the batches combined.
"""
success = True
fetched = set()
for batch in _ParentFirstBatches(projects):
batch = _WithoutProjects(batch, _RefreshDerivedRevisions(batch))
if not batch:
continue
batch.sort(key=self._fetch_times.Get, reverse=True)
result = self._Fetch(batch, opt, err_event, ssh_proxy, errors)
success = success and result.success
fetched.update(result.projects)
if not success and opt.fail_fast:
break
return _FetchResult(success, fetched)
def _FetchMain( def _FetchMain(
self, opt, args, all_projects, err_event, ssh_proxy, manifest, errors self, opt, args, all_projects, err_event, ssh_proxy, manifest, errors
): ):
@@ -1083,12 +1203,10 @@ later is required to fix a server side protocol bug.
Returns: Returns:
List of all projects that should be checked out. List of all projects that should be checked out.
""" """
to_fetch = []
to_fetch.extend(all_projects)
to_fetch.sort(key=self._fetch_times.Get, reverse=True)
try: try:
result = self._Fetch(to_fetch, opt, err_event, ssh_proxy, errors) result = self._FetchParentFirst(
all_projects, opt, err_event, ssh_proxy, errors
)
success = result.success success = result.success
fetched = result.projects fetched = result.projects
if not success: if not success:
@@ -1131,7 +1249,9 @@ later is required to fix a server side protocol bug.
if previously_missing_set == missing_set: if previously_missing_set == missing_set:
break break
previously_missing_set = missing_set previously_missing_set = missing_set
result = self._Fetch(missing, opt, err_event, ssh_proxy, errors) result = self._FetchParentFirst(
missing, opt, err_event, ssh_proxy, errors
)
success = result.success success = result.success
new_fetched = result.projects new_fetched = result.projects
if not success: if not success:
@@ -1494,10 +1614,23 @@ later is required to fix a server side protocol bug.
# Only check dirty or locally modified projects. These can't be # Only check dirty or locally modified projects. These can't be
# freshly cloned and will accumulate garbage. # freshly cloned and will accumulate garbage.
try: try:
is_dirty = project.IsDirty(consider_untracked=True) status = project._GetStatusSnapshot(
untracked_files="normal", branch=True
)
if status is not None:
is_dirty = status.is_dirty(consider_untracked=True)
head_rev = status.branch_oid
else:
is_dirty = project.IsDirty(consider_untracked=True)
head_rev = project.work_git.rev_parse(HEAD)
if head_rev is None:
# Porcelain v2 reports an unborn branch as "(initial)". The
# legacy rev-parse path failed here and skipped the bloat
# calculation, so preserve that behavior.
return None
manifest_rev = project.GetRevisionId(project.bare_ref.all) manifest_rev = project.GetRevisionId(project.bare_ref.all)
head_rev = project.work_git.rev_parse(HEAD)
has_local_commits = manifest_rev != head_rev has_local_commits = manifest_rev != head_rev
if not (is_dirty or has_local_commits): if not (is_dirty or has_local_commits):
@@ -2366,6 +2499,7 @@ later is required to fix a server side protocol bug.
manifest=manifest, manifest=manifest,
all_manifests=not opt.this_manifest_only, all_manifests=not opt.this_manifest_only,
) )
self._CheckReprojectCmdNesting(opt, args, manifest, all_projects)
# Log the repo projects by existing and new. # Log the repo projects by existing and new.
existing = [x for x in all_projects if x.Exists] existing = [x for x in all_projects if x.Exists]
@@ -2429,6 +2563,46 @@ later is required to fix a server side protocol bug.
if not opt.quiet: if not opt.quiet:
print("repo sync has finished successfully.") print("repo sync has finished successfully.")
def _CheckReprojectCmdNesting(
self,
opt: optparse.Values,
args: List[str],
manifest: XmlManifest,
all_projects: List[Project],
) -> None:
"""Fail when repo.reprojectcmd is used on a manifest nesting projects.
The command materializes a project's tree without Git, so nothing
keeps it from clobbering a project or submodule checked out inside
that tree. See docs/reproject-cmd.md.
"""
if not any(p.UseReprojectCmd for p in all_projects):
return
projects = all_projects
if args:
# Nesting is a property of the manifest, not of the projects
# picked on the command line.
projects = self.GetProjects(
[],
groups=opt.groups,
missing_ok=True,
submodules_ok=opt.recurse_submodules,
manifest=manifest,
all_manifests=not opt.this_manifest_only,
)
nested = _NestedProjects(projects)
if not nested:
return
e = SyncError(
"error: repo.reprojectcmd does not support nested projects or "
"submodules; found:\n"
+ "\n".join(
f" - {p.RelPath(local=opt.this_manifest_only)}" for p in nested
)
)
logger.error(e)
raise e
def _CreateSyncProgressThread( def _CreateSyncProgressThread(
self, pm: Progress, stop_event: _threading.Event self, pm: Progress, stop_event: _threading.Event
) -> _threading.Thread: ) -> _threading.Thread:
@@ -2758,7 +2932,7 @@ later is required to fix a server side protocol bug.
return _SyncResult( return _SyncResult(
project_index=project_index, project_index=project_index,
relpath=project.relpath, relpath=project.RelPath(local=opt.this_manifest_only),
fetch_success=fetch_success, fetch_success=fetch_success,
remote_fetched=remote_fetched, remote_fetched=remote_fetched,
checkout_success=checkout_success, checkout_success=checkout_success,
@@ -2906,6 +3080,10 @@ later is required to fix a server side protocol bug.
self._interleaved_err_checkout = False self._interleaved_err_checkout = False
self._interleaved_err_checkout_results = [] self._interleaved_err_checkout_results = []
# Project.relpath is relative to its own (sub)manifest, so it does not
# tell apart projects of different manifests being synced together.
_RelPath = lambda p: p.RelPath(local=opt.this_manifest_only)
err_event = multiprocessing.Event() err_event = multiprocessing.Event()
finished_relpaths = set() finished_relpaths = set()
project_list = list(all_projects) project_list = list(all_projects)
@@ -2942,13 +3120,13 @@ later is required to fix a server side protocol bug.
projects_to_sync = [ projects_to_sync = [
p p
for p in project_list for p in project_list
if p.relpath not in finished_relpaths if _RelPath(p) not in finished_relpaths
] ]
if not projects_to_sync: if not projects_to_sync:
break break
pending_relpaths = { pending_relpaths = {
p.relpath for p in projects_to_sync _RelPath(p) for p in projects_to_sync
} }
if previously_pending_relpaths == pending_relpaths: if previously_pending_relpaths == pending_relpaths:
stalled_projects_str = "\n".join( stalled_projects_str = "\n".join(
@@ -2977,12 +3155,22 @@ later is required to fix a server side protocol bug.
# projects in one level can be processed in # projects in one level can be processed in
# parallel, but we must wait for a level to complete # parallel, but we must wait for a level to complete
# before starting the next. # before starting the next.
submodule_revisions = {}
for level_projects in _SafeCheckoutOrder( for level_projects in _SafeCheckoutOrder(
projects_to_sync projects_to_sync
): ):
if not level_projects: if not level_projects:
continue continue
level_projects = _WithoutProjects(
level_projects,
_RefreshDerivedRevisions(
level_projects, submodule_revisions
),
)
if not level_projects:
continue
objdir_project_map = collections.defaultdict( objdir_project_map = collections.defaultdict(
list list
) )
+29 -22
View File
@@ -25,7 +25,6 @@ from editor import Editor
from error import GitError from error import GitError
from error import SilentRepoExitError from error import SilentRepoExitError
from error import UploadError from error import UploadError
from git_command import GitCommand
from git_refs import R_HEADS from git_refs import R_HEADS
import git_superproject import git_superproject
from hooks import RepoHook from hooks import RepoHook
@@ -379,7 +378,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
default=True, default=True,
help="disable verifying ssl certs (unsafe)", help="disable verifying ssl certs (unsafe)",
) )
RepoHook.AddOptionGroup(p, "pre-upload") RepoHook.AddOptionGroup(p, "pre-upload", allow_fix=True)
def _SingleBranch(self, opt, branch, people): def _SingleBranch(self, opt, branch, people):
project = branch.project project = branch.project
@@ -436,6 +435,18 @@ Gerrit Code Review: https://www.gerritcodereview.com/
self._UploadAndReport(opt, [branch], people) self._UploadAndReport(opt, [branch], people)
def _MultipleBranches(self, opt, pending, people): def _MultipleBranches(self, opt, pending, people):
if opt.yes and (opt.current_branch or opt.branch):
todo = [
branch
for _, avail in pending
for branch in avail
if branch is not None
]
if not todo:
_die("nothing ready for upload")
self._UploadAndReport(opt, todo, people)
return
projects = {} projects = {}
branches = {} branches = {}
@@ -649,6 +660,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
validate_certs=opt.validate_certs, validate_certs=opt.validate_certs,
push_options=push_options, push_options=push_options,
patchset_description=opt.patchset_description, patchset_description=opt.patchset_description,
git_event_log=self.git_event_log,
) )
branch.uploaded = True branch.uploaded = True
@@ -704,36 +716,31 @@ Gerrit Code Review: https://www.gerritcodereview.com/
raise UploadExitError(aggregate_errors=aggregate_errors) raise UploadExitError(aggregate_errors=aggregate_errors)
def _GetMergeBranch(self, project, local_branch=None): def _GetMergeBranch(self, project, local_branch=None):
"""Get the merge branch name for a local branch.
Resolves the merge branch in-memory via project configuration to
avoid git subprocess overhead during upload.
"""
if local_branch is None: if local_branch is None:
p = GitCommand( local_branch = project.CurrentBranch
project, if local_branch:
["rev-parse", "--abbrev-ref", "HEAD"], branch = project.GetBranch(local_branch)
capture_stdout=True, if branch.merge:
capture_stderr=True, return branch.merge
) return ""
p.Wait()
local_branch = p.stdout.strip()
p = GitCommand(
project,
["config", "--get", "branch.%s.merge" % local_branch],
capture_stdout=True,
capture_stderr=True,
)
p.Wait()
merge_branch = p.stdout.strip()
return merge_branch
@classmethod @classmethod
def _GatherOne(cls, opt, project_idx): def _GatherOne(cls, opt, project_idx):
"""Figure out the upload status for |project|.""" """Figure out the upload status for |project|."""
project = cls.get_parallel_context()["projects"][project_idx] project = cls.get_parallel_context()["projects"][project_idx]
cbr = None
if opt.current_branch: if opt.current_branch:
cbr = project.CurrentBranch cbr = project.CurrentBranch
up_branch = project.GetUploadableBranch(cbr) up_branch = project.GetUploadableBranch(cbr)
avail = [up_branch] if up_branch else None avail = [up_branch] if up_branch else None
else: else:
avail = project.GetUploadableBranches(opt.branch) avail = project.GetUploadableBranches(opt.branch)
return (project_idx, avail) return (project_idx, avail, cbr)
def Execute(self, opt, args): def Execute(self, opt, args):
projects = self.GetProjects( projects = self.GetProjects(
@@ -743,7 +750,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
def _ProcessResults(_pool, _out, results): def _ProcessResults(_pool, _out, results):
pending = [] pending = []
for result in results: for result in results:
project_idx, avail = result project_idx, avail, current_branch = result
project = projects[project_idx] project = projects[project_idx]
if avail is None: if avail is None:
logger.error( logger.error(
@@ -751,7 +758,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
"You might be able to fix the branch by running:\n" "You might be able to fix the branch by running:\n"
" git branch --set-upstream-to m/%s", " git branch --set-upstream-to m/%s",
project.RelPath(local=opt.this_manifest_only), project.RelPath(local=opt.this_manifest_only),
project.CurrentBranch, current_branch,
project.manifest.branch, project.manifest.branch,
) )
elif avail: elif avail:
+20 -2
View File
@@ -14,10 +14,12 @@
import platform import platform
import sys import sys
from typing import Any, Tuple
from command import Command from command import Command
from command import MirrorSafeCommand from command import MirrorSafeCommand
from git_command import git from git_command import git
from git_command import git_require
from git_command import RepoSourceVersion from git_command import RepoSourceVersion
from git_command import user_agent from git_command import user_agent
from git_refs import HEAD from git_refs import HEAD
@@ -34,6 +36,22 @@ class Version(Command, MirrorSafeCommand):
%prog %prog
""" """
@staticmethod
def _RepoVersion(project: Any) -> Tuple[str, str]:
"""Return repo's describe string and commit date."""
if git_require((2, 32, 0)):
output = project.bare_git.log(
"-1", "--format=%(describe)%n%cD", HEAD
)
description, commit_date = output.rstrip("\n").split("\n", 1)
if description:
return description, commit_date
return (
project.bare_git.describe(HEAD),
project.bare_git.log("-1", "--format=%cD", HEAD),
)
def Execute(self, opt, args): def Execute(self, opt, args):
rp = self.manifest.repoProject rp = self.manifest.repoProject
rem = rp.GetRemote() rem = rp.GetRemote()
@@ -41,11 +59,11 @@ class Version(Command, MirrorSafeCommand):
# These might not be the same. Report them both. # These might not be the same. Report them both.
src_ver = RepoSourceVersion() src_ver = RepoSourceVersion()
rp_ver = rp.bare_git.describe(HEAD) rp_ver, commit_date = self._RepoVersion(rp)
print(f"repo version {rp_ver}") print(f"repo version {rp_ver}")
print(f" (from {rem.url})") print(f" (from {rem.url})")
print(f" (tracking {branch.merge})") print(f" (tracking {branch.merge})")
print(f" ({rp.bare_git.log('-1', '--format=%cD', HEAD)})") print(f" ({commit_date})")
if self.wrapper_path is not None: if self.wrapper_path is not None:
print(f"repo launcher version {self.wrapper_version}") print(f"repo launcher version {self.wrapper_version}")
+234 -4
View File
@@ -14,6 +14,8 @@
"""Unittests for the command.py module.""" """Unittests for the command.py module."""
from typing import Iterable, List, Optional
import pytest import pytest
from command import Command from command import Command
@@ -30,12 +32,15 @@ class FakeProject:
gitdir=None, gitdir=None,
derived_subprojects=None, derived_subprojects=None,
sync_s=False, sync_s=False,
exists: bool = True,
): ):
self.name = name self.name = name
self.relpath = relpath self.relpath = relpath
self.worktree = f"/work/{relpath}"
self.manifest = None
self.gitdir = gitdir or f"/git/{relpath}" self.gitdir = gitdir or f"/git/{relpath}"
self.sync_s = sync_s self.sync_s = sync_s
self.Exists = True self.Exists = exists
self._derived_subprojects = derived_subprojects or [] self._derived_subprojects = derived_subprojects or []
def GetDerivedSubprojects(self): def GetDerivedSubprojects(self):
@@ -51,11 +56,59 @@ class FakeProject:
class FakeManifest: class FakeManifest:
"""Minimal manifest double for Command.GetProjects tests.""" """Minimal manifest double for Command.GetProjects tests."""
def __init__(self, projects): def __init__(
self.projects = projects self,
projects: Iterable[FakeProject],
*,
all_projects: Optional[Iterable[FakeProject]] = None,
effective_groups: str = "default",
):
self.projects = list(projects)
self.all_projects = (
list(self.projects) if all_projects is None else list(all_projects)
)
self._effective_groups = effective_groups
self.path_prefix = ""
# all_projects may include projects owned by child manifests,
# so only set this manifest on its direct projects.
for project in self.projects:
self._set_project_manifest(project)
def _set_project_manifest(self, project: FakeProject) -> None:
project.manifest = self
for subproject in project.GetDerivedSubprojects():
self._set_project_manifest(subproject)
def GetManifestGroupsStr(self): def GetManifestGroupsStr(self):
return "default" return self._effective_groups
def GetProjectsWithName(
self, name: str, all_manifests: bool = False
) -> List[FakeProject]:
projects = self.all_projects if all_manifests else self.projects
return [project for project in projects if project.name == name]
class GroupMatchingFakeProject(FakeProject):
"""Fake project with predictable group matches for GetProjects tests.
This lets the tests check which groups GetProjects uses without
reimplementing Project.MatchesGroups.
"""
def __init__(
self,
name: str,
relpath: str,
*,
matching_groups: Iterable[str],
):
super().__init__(name, relpath)
self._matching_groups = set(matching_groups)
def MatchesGroups(self, groups: Iterable[str]) -> bool:
return bool(self._matching_groups.intersection(groups))
def test_get_projects_keeps_derived_subprojects_for_repeated_repo(): def test_get_projects_keeps_derived_subprojects_for_repeated_repo():
@@ -117,3 +170,180 @@ def test_get_projects_submodule_override(
projects = cmd.GetProjects([], submodules_ok=submodules_ok) projects = cmd.GetProjects([], submodules_ok=submodules_ok)
assert (submodule in projects) is includes_submodule assert (submodule in projects) is includes_submodule
@pytest.mark.parametrize(
("groups", "expected_relpaths"),
[
(None, ["outer", "sub/child"]),
("", ["outer", "sub/child"]),
("override-group", ["sub/override"]),
],
ids=("groups-omitted", "groups-empty", "explicit-override"),
)
def test_get_projects_uses_groups_from_each_manifest_unless_overridden(
groups: Optional[str],
expected_relpaths: List[str],
) -> None:
"""Use each manifest's effective groups unless the caller overrides them."""
outer_project = GroupMatchingFakeProject(
"outer",
"outer",
matching_groups={"outer-group"},
)
# Both child projects also match "outer". Reusing the outer manifest's
# groups would therefore select both child projects.
child_project = GroupMatchingFakeProject(
"child",
"sub/child",
matching_groups={"outer-group", "child-group"},
)
override_project = GroupMatchingFakeProject(
"override",
"sub/override",
matching_groups={"outer-group", "override-group"},
)
child_manifest = FakeManifest(
[child_project, override_project],
effective_groups="child-group",
)
outer_manifest = FakeManifest(
[outer_project],
all_projects=[outer_project, *child_manifest.projects],
effective_groups="outer-group",
)
cmd = Command(manifest=outer_manifest)
projects = cmd.GetProjects(
[],
manifest=outer_manifest,
groups=groups,
all_manifests=True,
)
assert [project.relpath for project in projects] == expected_relpaths
def test_get_projects_by_name_uses_groups_from_each_manifest() -> None:
"""Name matches use the groups from each project's owning manifest."""
outer_project = GroupMatchingFakeProject(
"shared",
"outer/shared",
matching_groups={"outer-group"},
)
child_project = GroupMatchingFakeProject(
"shared",
"sub/shared",
matching_groups={"child-group"},
)
child_manifest = FakeManifest(
[child_project],
effective_groups="child-group",
)
outer_manifest = FakeManifest(
[outer_project],
all_projects=[outer_project, *child_manifest.projects],
effective_groups="outer-group",
)
cmd = Command(manifest=outer_manifest)
projects = cmd.GetProjects(
["shared"],
manifest=outer_manifest,
all_manifests=True,
)
assert [project.relpath for project in projects] == [
"outer/shared",
"sub/shared",
]
def test_find_projects_uses_groups_from_each_manifest() -> None:
"""Use each manifest's effective groups for regex selection."""
outer_project = GroupMatchingFakeProject(
"match-outer",
"outer",
matching_groups={"outer-group"},
)
child_project = GroupMatchingFakeProject(
"match-child",
"sub/child",
matching_groups={"child-group"},
)
excluded_project = GroupMatchingFakeProject(
"match-excluded",
"sub/excluded",
matching_groups={"other-group"},
)
child_manifest = FakeManifest(
[child_project, excluded_project],
effective_groups="child-group",
)
child_manifest.path_prefix = "sub"
outer_manifest = FakeManifest(
[outer_project],
all_projects=[
outer_project,
*child_manifest.projects,
],
effective_groups="outer-group",
)
outer_manifest.outer_client = outer_manifest
cmd = Command(manifest=outer_manifest)
projects = cmd.FindProjects(["match"], all_manifests=True)
assert [project.relpath for project in projects] == [
"outer",
"sub/child",
]
def test_find_projects_uses_explicit_groups() -> None:
"""Use explicit groups for regex selection."""
default_project = GroupMatchingFakeProject(
"default",
"default",
matching_groups={"default-group"},
)
override_project = GroupMatchingFakeProject(
"override",
"override",
matching_groups={"override-group"},
)
manifest = FakeManifest(
[default_project, override_project],
effective_groups="default-group",
)
cmd = Command(manifest=manifest)
projects = cmd.FindProjects(
["override"],
groups="override-group",
)
assert projects == [override_project]
def test_find_projects_allows_missing_projects() -> None:
"""Allow regex selection to include projects without a checkout."""
project = FakeProject(
"missing",
"missing",
exists=False,
)
cmd = Command(manifest=FakeManifest([project]))
projects = cmd.FindProjects(
["missing"],
missing_ok=True,
)
assert projects == [project]
+154
View File
@@ -123,6 +123,7 @@ class GitCommandStreamLogsTest(unittest.TestCase):
"""Tests the GitCommand class stderr log streaming cases.""" """Tests the GitCommand class stderr log streaming cases."""
def setUp(self): def setUp(self):
_ = git_command.user_agent.git
self.mock_process = mock.MagicMock() self.mock_process = mock.MagicMock()
self.mock_process.communicate.return_value = (None, None) self.mock_process.communicate.return_value = (None, None)
self.mock_process.wait.return_value = 0 self.mock_process.wait.return_value = 0
@@ -228,9 +229,162 @@ class GitCommandStreamLogsTest(unittest.TestCase):
self.assertEqual(cmd.stderr, logs) self.assertEqual(cmd.stderr, logs)
class GitCommandCaptureBytesTest(unittest.TestCase):
"""Tests the GitCommand class byte capture cases."""
def setUp(self) -> None:
_ = git_command.user_agent.git
self.mock_process = mock.MagicMock()
self.mock_process.communicate.return_value = (None, None)
self.mock_process.wait.return_value = 0
self.mock_popen = mock.MagicMock()
self.mock_popen.return_value = self.mock_process
mock.patch("subprocess.Popen", self.mock_popen).start()
def tearDown(self) -> None:
mock.patch.stopall()
def test_captures_stdout_as_bytes(self) -> None:
self.mock_process.communicate.return_value = (b"\xff\x00", b"error\r\n")
cmd = git_command.GitCommand(
None,
["status"],
capture_stdout=True,
capture_stdout_bytes=True,
capture_stderr=True,
)
self.mock_popen.assert_called_once_with(
["git", "status"],
cwd=None,
env=mock.ANY,
stdin=None,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
self.assertEqual(cmd.stdout, b"\xff\x00")
self.assertEqual(cmd.stderr, "error\n")
def test_capture_stdout_bytes_auto_enables_capture_stdout(self) -> None:
self.mock_process.communicate.return_value = (b"output", b"")
cmd = git_command.GitCommand(
None,
["status"],
capture_stdout_bytes=True,
)
self.mock_popen.assert_called_once_with(
["git", "status"],
cwd=None,
env=mock.ANY,
stdin=None,
stdout=subprocess.PIPE,
stderr=None,
)
self.assertEqual(cmd.stdout, b"output")
def test_capture_stdout_bytes_with_merge_output_raises(self) -> None:
with self.assertRaises(ValueError):
git_command.GitCommand(
None,
["status"],
capture_stdout_bytes=True,
merge_output=True,
)
def test_captures_stdout_as_bytes_encodes_str_input(self) -> None:
self.mock_process.communicate.return_value = (b"output", b"")
git_command.GitCommand(
None,
["status"],
input="hello world",
capture_stdout_bytes=True,
)
self.mock_process.communicate.assert_called_once_with(
input=b"hello world"
)
def test_captures_stdout_as_bytes_encodes_surrogate_input(self) -> None:
self.mock_process.communicate.return_value = (b"output", b"")
git_command.GitCommand(
None,
["status"],
input="file_\udcff.txt",
capture_stdout_bytes=True,
)
self.mock_process.communicate.assert_called_once_with(
input=b"file_\xff.txt"
)
def test_captures_stdout_as_bytes_passes_bytes_input(self) -> None:
self.mock_process.communicate.return_value = (b"output", b"")
git_command.GitCommand(
None,
["status"],
input=b"raw_\xff.txt",
capture_stdout_bytes=True,
)
self.mock_process.communicate.assert_called_once_with(
input=b"raw_\xff.txt"
)
def test_verify_command_truncates_nul_delimited_stdout(self) -> None:
cmd = git_command.GitCommand(
None,
["status"],
capture_stdout_bytes=True,
)
cmd.rc = 1
cmd.stdout = b"first_file\0second_file\0third_file"
cmd.stderr = "stderr"
with self.assertRaises(git_command.GitCommandError) as cm:
cmd.VerifyCommand()
self.assertEqual(cm.exception.git_stdout, "first_file")
def test_verify_command_decodes_bytes_stdout(self) -> None:
cmd = git_command.GitCommand(
None,
["status"],
capture_stdout_bytes=True,
)
cmd.rc = 1
cmd.stdout = b"error\xff\nline2"
cmd.stderr = "stderr"
with self.assertRaises(git_command.GitCommandError) as cm:
cmd.VerifyCommand()
self.assertEqual(cm.exception.git_stdout, "error\\xff")
class GitCallUnitTest(unittest.TestCase): class GitCallUnitTest(unittest.TestCase):
"""Tests the _GitCall class (via git_command.git).""" """Tests the _GitCall class (via git_command.git)."""
def test_valid_branch_name_uses_branch_mode(self) -> None:
"""Branch validation applies Git's branch-specific restrictions."""
command = mock.MagicMock()
command.Wait.return_value = 1
with mock.patch.object(
git_command, "GitCommand", return_value=command
) as check:
self.assertFalse(git_command.IsValidBranchName("-topic"))
check.assert_called_once_with(
None,
["check-ref-format", "--branch", "-topic"],
capture_stdout=True,
capture_stderr=True,
add_event_log=False,
log_as_error=False,
)
def test_version_tuple(self): def test_version_tuple(self):
"""Check git.version_tuple() handling.""" """Check git.version_tuple() handling."""
ver = git_command.git.version_tuple() ver = git_command.git.version_tuple()
+2 -2
View File
@@ -256,8 +256,8 @@ def test_remote_save_with_push_url_without_projectname(
("0" * 64, True), ("0" * 64, True),
("f" * 64, True), ("f" * 64, True),
("a" * 39, False), ("a" * 39, False),
("a" * 41, True), ("a" * 41, False),
("a" * 63, True), ("a" * 63, False),
("a" * 65, False), ("a" * 65, False),
("g" * 40, False), ("g" * 40, False),
("g" * 64, False), ("g" * 64, False),
+148 -2
View File
@@ -17,6 +17,8 @@
import os import os
from pathlib import Path from pathlib import Path
import subprocess import subprocess
from typing import Any, List
from unittest import mock
import pytest import pytest
import utils_for_test import utils_for_test
@@ -24,7 +26,7 @@ import utils_for_test
import git_refs import git_refs
def _run(repo, *args): def _run(repo: str, *args: str) -> str:
return subprocess.run( return subprocess.run(
["git", "-C", repo, *args], ["git", "-C", repo, *args],
stdout=subprocess.PIPE, stdout=subprocess.PIPE,
@@ -34,7 +36,7 @@ def _run(repo, *args):
).stdout.strip() ).stdout.strip()
def _init_repo(tmp_path, reftable=False): def _init_repo(tmp_path: Path, reftable: bool = False) -> str:
repo = os.path.join(tmp_path, "repo") repo = os.path.join(tmp_path, "repo")
ref_format = "reftable" if reftable else "files" ref_format = "reftable" if reftable else "files"
utils_for_test.init_git_tree(repo, ref_format=ref_format) utils_for_test.init_git_tree(repo, ref_format=ref_format)
@@ -57,10 +59,154 @@ def test_reads_refs(tmp_path, reftable):
branch = _run(repo, "symbolic-ref", "--short", "HEAD") branch = _run(repo, "symbolic-ref", "--short", "HEAD")
head = _run(repo, "rev-parse", "HEAD") head = _run(repo, "rev-parse", "HEAD")
assert refs.symref("HEAD") == f"refs/heads/{branch}" assert refs.symref("HEAD") == f"refs/heads/{branch}"
assert refs.head == f"refs/heads/{branch}"
assert refs.get("HEAD") == head assert refs.get("HEAD") == head
assert refs.get(f"refs/heads/{branch}") == head assert refs.get(f"refs/heads/{branch}") == head
@pytest.mark.parametrize("reftable", [False, True])
def test_reads_detached_head(tmp_path: Path, reftable: bool) -> None:
if reftable and not utils_for_test.supports_reftable():
pytest.skip("reftable not supported")
repo = _init_repo(tmp_path, reftable=reftable)
head = _run(repo, "rev-parse", "HEAD")
_run(repo, "checkout", "--detach", head)
refs = git_refs.GitRefs(os.path.join(repo, ".git"))
assert refs.symref("HEAD") == ""
assert refs.head == head
assert refs.get("HEAD") == head
def test_reads_head_with_root_refs_in_one_command(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Git 2.45 and newer include HEAD in the ref snapshot."""
head = "1" * 40
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **_kwargs: Any
) -> None:
commands.append(cmdv)
self.stdout = (
f"{head}\0HEAD\0refs/heads/main\n"
f"{head}\0refs/heads/main\0\n"
)
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
with mock.patch.object(refs, "_ReadSymbolicRef") as read_head:
assert refs.get("HEAD") == head
assert commands == [
[
"for-each-ref",
"--include-root-refs",
"--format=%(objectname)%00%(refname)%00%(symref)",
"HEAD",
"refs",
]
]
read_head.assert_not_called()
def test_root_ref_snapshot_falls_back_for_unborn_head(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""An unborn HEAD still uses symbolic-ref after the ref snapshot."""
class FakeGitCommand:
def __init__(
self, _project: Any, _cmdv: List[str], **_kwargs: Any
) -> None:
self.stdout = ""
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.symref("HEAD") == "refs/heads/main"
def test_old_git_keeps_separate_head_fallback(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Git before 2.45 uses the original for-each-ref and HEAD calls."""
head = "1" * 40
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **_kwargs: Any
) -> None:
commands.append(cmdv)
self.stdout = f"{head}\0refs/heads/main\0\n"
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: False)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.get("HEAD") == head
assert commands == [
[
"for-each-ref",
"--format=%(objectname)%00%(refname)%00%(symref)",
]
]
def test_for_each_ref_failure_falls_back_to_symbolic_ref(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""When for-each-ref fails, HEAD is still resolved via symbolic-ref."""
class FakeGitCommand:
def __init__(
self, _project: Any, _cmdv: List[str], **_kwargs: Any
) -> None:
self.stdout = ""
def Wait(self) -> int:
return 1
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
assert name == "HEAD"
refs._symref[name] = "refs/heads/main"
monkeypatch.setattr(refs, "_ReadSymbolicRef", read_head)
assert refs.symref("HEAD") == "refs/heads/main"
@pytest.mark.parametrize("reftable", [False, True]) @pytest.mark.parametrize("reftable", [False, True])
def test_updates_when_refs_change(tmp_path, reftable): def test_updates_when_refs_change(tmp_path, reftable):
if reftable and not utils_for_test.supports_reftable(): if reftable and not utils_for_test.supports_reftable():
+210
View File
@@ -0,0 +1,210 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the git_status.py module."""
import os
from typing import Any, List
from unittest import mock
import pytest
import git_status
def test_parse_porcelain_v2_branch_and_paths() -> None:
output = (
b"# branch.oid " + b"1" * 40 + b"\0"
b"# branch.head topic\0"
b"# branch.upstream origin/main\0"
b"# branch.ab +2 -3\0"
b"# stash 1\0"
b"1 M. N... 100644 100644 100644 "
+ b"1" * 40
+ b" "
+ b"2" * 40
+ b" staged name\0"
b"1 .M N... 100644 100644 100644 "
+ b"1" * 40
+ b" "
+ b"2" * 40
+ b" worktree name\0"
b"2 R. N... 100644 100644 100644 "
+ b"1" * 40
+ b" "
+ b"2" * 40
+ b" R075 renamed\0old name\0"
b"? untracked\0"
)
status = git_status.ParsePorcelainV2(output)
assert status.current_branch == "topic"
assert status.upstream == "origin/main"
assert (status.ahead, status.behind, status.stash_count) == (2, 3, 1)
assert status.index_changes["staged name"].status == "M"
assert status.worktree_changes["worktree name"].status == "M"
renamed = status.index_changes["renamed"]
assert (renamed.src_path, renamed.level) == ("old name", "75")
assert status.untracked == ["untracked"]
def test_parse_porcelain_v2_unmerged_and_non_utf8_path() -> None:
path = b"bad-\xff-name"
output = (
b"u UU N... 100644 100644 100644 100644 "
+ b"1" * 40
+ b" "
+ b"2" * 40
+ b" "
+ b"3" * 40
+ b" "
+ path
+ b"\0"
)
status = git_status.ParsePorcelainV2(output)
decoded = os.fsdecode(path)
assert status.index_changes[decoded].status == "U"
assert status.worktree_changes[decoded].status == "U"
assert os.fsencode(status.index_changes[decoded].path) == path
def test_untracked_only_respects_consider_untracked() -> None:
status = git_status.ParsePorcelainV2(b"? new file\0")
assert status.is_dirty()
assert not status.is_dirty(consider_untracked=False)
def test_branch_headers_preserve_non_ascii_names() -> None:
branch = "tópico"
status = git_status.ParsePorcelainV2(
b"# branch.oid " + b"1" * 40 + b"\0"
b"# branch.head " + os.fsencode(branch) + b"\0"
)
assert status.current_branch == branch
def test_quick_ahead_behind_is_recorded_as_unknown() -> None:
status = git_status.ParsePorcelainV2(b"# branch.ab +? -?\0")
assert (status.ahead, status.behind) == (0, 0)
assert not status.has_ahead_behind
def test_unknown_head_is_not_a_current_branch() -> None:
status = git_status.ParsePorcelainV2(b"# branch.head (unknown)\0")
assert status.current_branch is None
def test_malformed_output_is_rejected() -> None:
with pytest.raises(git_status.StatusParseError):
git_status.ParsePorcelainV2(b"2 R. truncated\0")
def test_malformed_branch_ab_is_rejected() -> None:
with pytest.raises(git_status.StatusParseError):
git_status.ParsePorcelainV2(b"# branch.ab not-a-valid-ab\0")
def test_ignored_records_are_skipped() -> None:
status = git_status.ParsePorcelainV2(b"! ignored_file\0")
assert not status.is_dirty()
assert status.untracked == []
def test_get_status_uses_versioned_machine_options(
monkeypatch: pytest.MonkeyPatch,
) -> None:
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **kwargs: Any
) -> None:
commands.append((cmdv, kwargs))
self.stdout = b""
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_status, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_status, "git_require", lambda _version: True)
git_status.GetStatus(
mock.sentinel.project,
mock.sentinel.gitdir,
untracked_files="no",
branch=True,
ahead_behind=True,
show_stash=True,
)
cmd, kwargs = commands[0]
assert cmd == [
"status",
"--porcelain=v2",
"-z",
"--ignore-submodules=all",
"--untracked-files=no",
"--branch",
"--ahead-behind",
"--renames",
"--show-stash",
]
assert kwargs["capture_stdout_bytes"]
def test_get_status_rejects_git_before_2_11(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(git_status, "git_require", lambda _version: False)
with pytest.raises(git_status.UnsupportedStatusError):
git_status.GetStatus(mock.sentinel.project, mock.sentinel.gitdir)
def test_get_status_omits_stash_header_before_2_35(
monkeypatch: pytest.MonkeyPatch,
) -> None:
commands = []
class FakeGitCommand:
def __init__(
self, _project: Any, cmdv: List[str], **_kwargs: Any
) -> None:
commands.append(cmdv)
self.stdout = b""
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_status, "GitCommand", FakeGitCommand)
monkeypatch.setattr(
git_status,
"git_require",
lambda version: version <= (2, 34, 0),
)
git_status.GetStatus(
mock.sentinel.project,
mock.sentinel.gitdir,
show_stash=True,
)
assert "--show-stash" not in commands[0]
+24 -4
View File
@@ -542,14 +542,15 @@ class SuperprojectTestCase(unittest.TestCase):
with mock.patch( with mock.patch(
"git_superproject.GitCommand", autospec=True "git_superproject.GitCommand", autospec=True
) as mock_git_command: ) as mock_git_command:
with mock.patch( with mock.patch.object(
"git_superproject.GitRefs.get", autospec=True self._superproject, "_GetRef"
) as mock_git_refs: ) as get_ref:
instance = mock_git_command.return_value instance = mock_git_command.return_value
instance.Wait.return_value = 0 instance.Wait.return_value = 0
mock_git_refs.side_effect = ["", "1234"] get_ref.side_effect = ["", "1234"]
self.assertTrue(self._superproject._Fetch()) self.assertTrue(self._superproject._Fetch())
get_ref.assert_called_with("refs/heads/main")
self.assertEqual( self.assertEqual(
# TODO: Once we require Python 3.8+, # TODO: Once we require Python 3.8+,
# use 'mock_git_command.call_args.args'. # use 'mock_git_command.call_args.args'.
@@ -572,6 +573,7 @@ class SuperprojectTestCase(unittest.TestCase):
# If branch for revision exists, set as --negotiation-tip. # If branch for revision exists, set as --negotiation-tip.
self.assertTrue(self._superproject._Fetch()) self.assertTrue(self._superproject._Fetch())
get_ref.assert_called_with("refs/heads/main")
self.assertEqual( self.assertEqual(
# TODO: Once we require Python 3.8+, # TODO: Once we require Python 3.8+,
# use 'mock_git_command.call_args.args'. # use 'mock_git_command.call_args.args'.
@@ -593,3 +595,21 @@ class SuperprojectTestCase(unittest.TestCase):
], ],
), ),
) )
def test_GetRef_resolves_only_the_requested_ref(self) -> None:
command = mock.MagicMock(stdout="1234\n")
command.Wait.return_value = 0
with mock.patch(
"git_superproject.GitCommand", return_value=command
) as git_command:
self.assertEqual("1234", self._superproject._GetRef("HEAD"))
git_command.assert_called_once_with(
None,
["rev-parse", "--verify", "--quiet", "HEAD"],
gitdir=self._superproject._work_git,
bare=True,
capture_stdout=True,
capture_stderr=True,
log_as_error=False,
)
+5 -9
View File
@@ -33,7 +33,7 @@ import platform_utils
def server_logging_thread( def server_logging_thread(
socket_path: str, socket_path: str,
server_ready: threading.Condition, server_ready: threading.Event,
received_traces: List[str], received_traces: List[str],
) -> None: ) -> None:
"""Helper function to receive logs over a Unix domain socket. """Helper function to receive logs over a Unix domain socket.
@@ -43,8 +43,7 @@ def server_logging_thread(
Args: Args:
socket_path: path to a Unix domain socket on which to listen for traces socket_path: path to a Unix domain socket on which to listen for traces
server_ready: a threading.Condition used to signal to the caller that server_ready: event set when the server is ready to accept connections
this thread is ready to accept connections
received_traces: a list to which received traces will be appended (after received_traces: a list to which received traces will be appended (after
decoding to a utf-8 string). decoding to a utf-8 string).
""" """
@@ -53,8 +52,7 @@ def server_logging_thread(
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock:
sock.bind(socket_path) sock.bind(socket_path)
sock.listen(0) sock.listen(0)
with server_ready: server_ready.set()
server_ready.notify()
with sock.accept()[0] as conn: with sock.accept()[0] as conn:
while True: while True:
recved = conn.recv(4096) recved = conn.recv(4096)
@@ -404,7 +402,7 @@ def test_write_socket(event_log: git_trace2_event_log.EventLog) -> None:
received_traces: List[str] = [] received_traces: List[str] = []
with tempfile.TemporaryDirectory(prefix="test_server_sockets") as tempdir: with tempfile.TemporaryDirectory(prefix="test_server_sockets") as tempdir:
socket_path = os.path.join(tempdir, "server.sock") socket_path = os.path.join(tempdir, "server.sock")
server_ready = threading.Condition() server_ready = threading.Event()
# Start "server" listening on Unix domain socket at socket_path. # Start "server" listening on Unix domain socket at socket_path.
server_thread = threading.Thread( server_thread = threading.Thread(
target=server_logging_thread, target=server_logging_thread,
@@ -412,9 +410,7 @@ def test_write_socket(event_log: git_trace2_event_log.EventLog) -> None:
) )
try: try:
server_thread.start() server_thread.start()
server_ready.wait(timeout=120)
with server_ready:
server_ready.wait(timeout=120)
event_log.StartEvent([]) event_log.StartEvent([])
path = event_log.Write(path=f"af_unix:{socket_path}") path = event_log.Write(path=f"af_unix:{socket_path}")
+64 -2
View File
@@ -15,6 +15,7 @@
"""Unittests for the hooks.py module.""" """Unittests for the hooks.py module."""
from io import StringIO from io import StringIO
from pathlib import Path
import sys import sys
import pytest import pytest
@@ -108,11 +109,11 @@ def test_post_sync_argument_validation() -> None:
@pytest.mark.parametrize("yes_val", (True, False)) @pytest.mark.parametrize("yes_val", (True, False))
def test_repo_upload_yes_arg(tmp_path, yes_val: bool) -> None: def test_repo_upload_yes_arg(tmp_path: Path, yes_val: bool) -> None:
"""Test that yes is passed in kwargs during hook execution.""" """Test that yes is passed in kwargs during hook execution."""
class FakeProject: class FakeProject:
def __init__(self, worktree): def __init__(self, worktree: str) -> None:
self.worktree = worktree self.worktree = worktree
self.enabled_repo_hooks = ["pre-upload"] self.enabled_repo_hooks = ["pre-upload"]
self.config = None self.config = None
@@ -139,3 +140,64 @@ def main(project_list, **kwargs):
assert res is True assert res is True
assert project_list == [yes_val] assert project_list == [yes_val]
@pytest.mark.parametrize("fix_val", (True, False))
def test_repo_upload_fix_arg(tmp_path: Path, fix_val: bool) -> None:
"""Test that fix is passed in kwargs during hook execution."""
class FakeProject:
def __init__(self, worktree: str) -> None:
self.worktree = worktree
self.enabled_repo_hooks = ["pre-upload"]
self.config = None
hook_file = tmp_path / "pre-upload.py"
hook_content = """
def main(project_list, **kwargs):
project_list.append(kwargs.get("fix"))
"""
hook_file.write_text(hook_content)
hook = hooks.RepoHook(
hook_type="pre-upload",
hooks_project=FakeProject(str(tmp_path)),
repo_topdir=str(tmp_path),
manifest_url="https://gerrit",
allow_all_hooks=True,
fix=fix_val,
)
project_list = []
res = hook.Run(project_list=project_list, worktree_list=[])
assert res is True
assert project_list == [fix_val]
def test_from_subcmd_without_fix_option() -> None:
"""Test that FromSubcmd works when opt does not have fix attribute."""
class Remote:
url = "https://gerrit"
class FakeManifest:
repo_hooks_project = None
topdir = "/fake/topdir"
class manifestProject:
@staticmethod
def GetRemote(name: str) -> "Remote":
return Remote()
class contactinfo:
bugurl = "https://bugs"
class FakeOpt:
bypass_hooks = False
allow_all_hooks = False
ignore_hooks = False
hook = hooks.RepoHook.FromSubcmd(FakeManifest(), FakeOpt(), "post-sync")
assert hook._fix is False
+2489 -38
View File
File diff suppressed because it is too large Load Diff
+43
View File
@@ -0,0 +1,43 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests for the release/update-manpages wrapper."""
import runpy
import sys
from types import ModuleType
import pytest
import utils_for_test
UPDATE_MANPAGES_SCRIPT = (
utils_for_test.THIS_DIR.parent / "release" / "update-manpages"
)
def test_wrapper_does_not_run_main_for_multiprocessing_child(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Do not rerun main when multiprocessing re-executes the wrapper."""
fake_update_manpages = ModuleType("update_manpages")
# Mock this because the real module requires newer Python versions than
# our unittest framework does.
monkeypatch.setitem(sys.modules, "update_manpages", fake_update_manpages)
runpy.run_path(
str(UPDATE_MANPAGES_SCRIPT),
run_name="__mp_main__",
)
+85
View File
@@ -0,0 +1,85 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for subcmds/cherry_pick.py."""
from unittest import mock
import pytest
from error import GitError
from git_command import GitCommand
from subcmds import cherry_pick
def test_resolve_reference_uses_one_typed_batch_request(
monkeypatch: pytest.MonkeyPatch,
) -> None:
oid = "1" * 40
commit = "tree " + "2" * 40 + "\n\nSubject 🚀\n\nBody\n"
output = oid + " commit " + str(len(commit.encode("utf-8"))) + "\n"
output += commit + "\n"
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = output
command.stderr = ""
command.Wait.return_value = 0
run_git = mock.create_autospec(GitCommand, return_value=command)
monkeypatch.setattr(cherry_pick, "GitCommand", run_git)
resolved, contents = cherry_pick.CherryPick()._ResolveReference("topic")
assert resolved == oid
assert contents == commit
run_git.assert_called_once_with(
None,
["cat-file", "--batch"],
input="topic^{commit}\n",
capture_stdout=True,
capture_stderr=True,
verify_command=True,
)
def test_resolve_reference_rejects_missing_object(
monkeypatch: pytest.MonkeyPatch,
) -> None:
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = "topic^{commit} missing\n"
command.stderr = ""
command.Wait.return_value = 0
monkeypatch.setattr(
cherry_pick,
"GitCommand",
mock.create_autospec(GitCommand, return_value=command),
)
with pytest.raises(GitError, match="commit topic not found"):
cherry_pick.CherryPick()._ResolveReference("topic")
def test_resolve_reference_rejects_ambiguous_object(
monkeypatch: pytest.MonkeyPatch,
) -> None:
command = mock.create_autospec(GitCommand, instance=True)
command.stdout = "topic^{commit} ambiguous\n"
command.stderr = ""
command.Wait.return_value = 0
monkeypatch.setattr(
cherry_pick,
"GitCommand",
mock.create_autospec(GitCommand, return_value=command),
)
with pytest.raises(GitError, match="commit topic not found"):
cherry_pick.CherryPick()._ResolveReference("topic")
+56
View File
@@ -17,7 +17,9 @@
import contextlib import contextlib
import io import io
from pathlib import Path from pathlib import Path
from unittest import mock
import pytest
import utils_for_test import utils_for_test
import manifest_xml import manifest_xml
@@ -105,3 +107,57 @@ def test_forall_all_projects_called_once(tmp_path: Path) -> None:
line_count = sum(1 for x in output.splitlines() if x) line_count = sum(1 for x in output.splitlines() if x)
# Verify that we didn't get more lines than expected. # Verify that we didn't get more lines than expected.
assert line_count == 8 assert line_count == 8
@pytest.mark.parametrize(
("regex_option", "inverse"),
[
("-r", False),
("-i", True),
],
ids=("regex", "inverse-regex"),
)
def test_forall_regex_modes_pass_groups_to_find_projects(
tmp_path: Path,
regex_option: str,
inverse: bool,
) -> None:
"""Pass --groups through in regex modes."""
manifest = _create_manifest_with_8_projects(tmp_path)
cmd = subcmds.forall.Forall()
cmd.manifest = manifest
opts, args = cmd.OptionParser.parse_args(
[
regex_option,
"--groups",
"special",
"project",
"-c",
"true",
]
)
with mock.patch.object(
cmd,
"FindProjects",
return_value=[],
) as find_projects, mock.patch.object(
cmd,
"ExecuteInParallel",
return_value=0,
):
cmd.Execute(opts, args)
expected_kwargs = {
"groups": "special",
"all_manifests": True,
}
if inverse:
expected_kwargs["inverse"] = True
find_projects.assert_called_once_with(
["project"],
**expected_kwargs,
)
+116 -2
View File
@@ -208,7 +208,7 @@ def test_get_project_data_uses_head_revision() -> None:
project.name = "foo" project.name = "foo"
project.worktree = "/path/to/foo" project.worktree = "/path/to/foo"
project.revisionExpr = "refs/heads/main" project.revisionExpr = "refs/heads/main"
project.GetBranches.return_value = [] project.GetBranches.return_value = {}
# GetHeadRevisionId() returns a SHA, it should be used. # GetHeadRevisionId() returns a SHA, it should be used.
project.GetHeadRevisionId.return_value = "head_sha_12345" project.GetHeadRevisionId.return_value = "head_sha_12345"
@@ -235,7 +235,9 @@ def test_json_with_projects(capsys) -> None:
project.name = "foo" project.name = "foo"
project.worktree = "/path/to/foo" project.worktree = "/path/to/foo"
project.revisionExpr = "refs/heads/main" project.revisionExpr = "refs/heads/main"
project.GetBranches.return_value = {"branch1": mock.MagicMock()} branch = mock.MagicMock()
branch.current = True
project.GetBranches.return_value = {"branch1": branch}
project.GetHeadRevisionId.return_value = "head_sha_12345" project.GetHeadRevisionId.return_value = "head_sha_12345"
project.CurrentBranch = "branch1" project.CurrentBranch = "branch1"
@@ -253,3 +255,115 @@ def test_json_with_projects(capsys) -> None:
assert project_data["manifest_revision"] == "refs/heads/main" assert project_data["manifest_revision"] == "refs/heads/main"
assert project_data["local_branches"] == ["branch1"] assert project_data["local_branches"] == ["branch1"]
assert project_data["current_branch"] == "branch1" assert project_data["current_branch"] == "branch1"
def test_diff_commits_uses_one_left_right_walk() -> None:
"""Local and remote commits are partitioned from one rev-list."""
project = mock.MagicMock()
project.work_git.rev_list.return_value = [
"<11111111 local commit",
">22222222 remote commit",
]
local, remote = info.Info._GetDiffCommits(project, "refs/remotes/m/main")
assert local == ["11111111 local commit"]
assert remote == ["22222222 remote commit"]
project.work_git.rev_list.assert_called_once_with(
"--left-right",
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
"HEAD...refs/remotes/m/main",
"--",
)
def test_diff_commits_falls_back_to_bare_git_when_no_worktree() -> None:
"""Bare or worktree-less projects fall back to bare_git for history walk."""
project = mock.MagicMock()
project.work_git = None
project.bare_git.rev_list.return_value = [
"<11111111 local commit",
">22222222 remote commit",
]
local, remote = info.Info._GetDiffCommits(project, "refs/remotes/m/main")
assert local == ["11111111 local commit"]
assert remote == ["22222222 remote commit"]
project.bare_git.rev_list.assert_called_once_with(
"--left-right",
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
"HEAD...refs/remotes/m/main",
"--",
)
def test_diff_commits_empty_output() -> None:
"""Empty rev-list output produces empty local and remote commit lists."""
project = mock.MagicMock()
project.work_git.rev_list.return_value = []
local, remote = info.Info._GetDiffCommits(project, "refs/remotes/m/main")
assert local == []
assert remote == []
def test_get_current_branch() -> None:
"""_GetCurrentBranch identifies the branch with current=True."""
b1 = mock.MagicMock(current=False)
b2 = mock.MagicMock(current=True)
assert info.Info._GetCurrentBranch({"b1": b1, "b2": b2}) == "b2"
assert info.Info._GetCurrentBranch({"b1": b1}) is None
assert info.Info._GetCurrentBranch({}) is None
def test_overview_helper_current_branch_filters_before_uploadable(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""_OverviewHelper only checks uploadable state for the current branch."""
project = mock.MagicMock()
project.RelPath.return_value = "proj"
b1 = mock.MagicMock(current=False)
b2 = mock.MagicMock(current=True)
project.GetBranches.return_value = {"b1": b1, "b2": b2}
uploadable = mock.MagicMock(commits=["c1"], date="2026-09-21")
uploadable.name = "b2"
project.GetUploadableBranch.return_value = uploadable
monkeypatch.setattr(
info.Info,
"get_parallel_context",
lambda: {"projects": [project]},
)
opt = mock.MagicMock(current_branch=True, this_manifest_only=False)
result = info.Info._OverviewHelper(0, opt)
project.GetUploadableBranch.assert_called_once_with("b2")
assert len(result) == 1
assert result[0].name == "b2"
assert result[0].is_current is True
def test_overview_helper_current_branch_detached_head_skips_uploadable(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""_OverviewHelper skips GetUploadableBranch when detached with -b."""
project = mock.MagicMock()
b1 = mock.MagicMock(current=False)
project.GetBranches.return_value = {"b1": b1}
monkeypatch.setattr(
info.Info,
"get_parallel_context",
lambda: {"projects": [project]},
)
opt = mock.MagicMock(current_branch=True, this_manifest_only=False)
result = info.Info._OverviewHelper(0, opt)
project.GetUploadableBranch.assert_not_called()
assert result == []
+57
View File
@@ -0,0 +1,57 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the list subcmd."""
from typing import List, Optional
from unittest import mock
import pytest
import subcmds
@pytest.mark.parametrize(
("extra_args", "expected_groups", "expected_missing_ok"),
[
(["--groups", "special"], "special", None),
(["--all"], None, True),
],
ids=("groups", "all"),
)
def test_list_regex_passes_groups_and_all(
extra_args: List[str],
expected_groups: Optional[str],
expected_missing_ok: Optional[bool],
) -> None:
"""Pass --groups and --all through in regex mode."""
cmd = subcmds.list.List()
opts, args = cmd.OptionParser.parse_args(
["--regex", *extra_args, "project"]
)
with mock.patch.object(
cmd,
"FindProjects",
return_value=[],
) as find_projects:
cmd.Execute(opts, args)
find_projects.assert_called_once_with(
["project"],
groups=expected_groups,
missing_ok=expected_missing_ok,
all_manifests=True,
)
+41
View File
@@ -14,6 +14,9 @@
"""Unittests for the subcmds/rebase.py module.""" """Unittests for the subcmds/rebase.py module."""
import contextlib
import io
from types import SimpleNamespace
from unittest import mock from unittest import mock
from error import GitError from error import GitError
@@ -48,3 +51,41 @@ def test_resolve_onto_manifest_fallback() -> None:
assert res == "main" assert res == "main"
project.GetRemote.assert_called_once() project.GetRemote.assert_called_once()
remote.ToLocal.assert_called_once_with("main") remote.ToLocal.assert_called_once_with("main")
def test_execute_delegates_autostash_to_rebase() -> None:
"""--auto-stash is one rebase process, including staged-only changes."""
cmd = rebase.Rebase()
cmd.manifest = mock.MagicMock()
cmd.git_event_log = mock.MagicMock()
project = mock.MagicMock()
project.CurrentBranch = "topic"
project.RelPath.return_value = "project"
branch = mock.MagicMock()
branch.LocalMerge = "refs/remotes/origin/main"
project.GetBranch.return_value = branch
cmd.GetProjects = mock.MagicMock(return_value=[project])
opt = SimpleNamespace(
interactive=False,
fail_fast=False,
whitespace=None,
quiet=False,
force_rebase=False,
ff=True,
autosquash=False,
auto_stash=True,
onto_manifest=False,
this_manifest_only=False,
)
git_command = mock.MagicMock()
git_command.Wait.return_value = 0
with mock.patch.object(
rebase, "GitCommand", return_value=git_command
) as run_git, contextlib.redirect_stdout(io.StringIO()):
assert cmd.Execute(opt, []) == 0
run_git.assert_called_once_with(
project,
["rebase", "--autostash", "refs/remotes/origin/main"],
)
+111
View File
@@ -185,6 +185,96 @@ def test_status_without_orphans(
assert lines[1] == " -m\tREADME" assert lines[1] == " -m\tREADME"
def test_status_staged_and_unstaged_same_path(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""A path changed on both sides of the index renders both states."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
readme = project_worktree / "README"
readme.write_text("staged")
subprocess.check_call(["git", "add", "README"], cwd=project_worktree)
readme.write_text("unstaged")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert lines[1] == " Mm\tREADME"
def test_status_forces_staged_rename_detection(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""The snapshot preserves rename scores despite user status config."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
subprocess.check_call(
["git", "config", "status.renames", "false"], cwd=project_worktree
)
subprocess.check_call(
["git", "mv", "README", "RENAMED"], cwd=project_worktree
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert lines[1] == " R-\tREADME => RENAMED (100%)"
def test_detached_clean_status_is_suppressed(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""A clean detached checkout keeps returning CLEAN without output."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
subprocess.check_call(
["git", "checkout", "-q", "--detach", "HEAD"],
cwd=topdir / project_path,
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
assert _status_lines(stdout.getvalue()) == [
"nothing to commit (working directory clean)"
]
def test_status_unmerged_path_matches_legacy_display(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Porcelain-v2 unmerged records render as index U and worktree u."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
worktree = topdir / project_path
subprocess.check_call(
["git", "checkout", "-q", "-b", "other"], cwd=worktree
)
(worktree / "README").write_text("other")
subprocess.check_call(["git", "commit", "-qam", "other"], cwd=worktree)
subprocess.check_call(["git", "checkout", "-q", "main"], cwd=worktree)
(worktree / "README").write_text("main")
subprocess.check_call(["git", "commit", "-qam", "main"], cwd=worktree)
merge = subprocess.run(
["git", "merge", "other"],
cwd=worktree,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
assert merge.returncode != 0
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert lines[1] == " Uu\tREADME"
def test_status_with_orphans_and_modified_file( def test_status_with_orphans_and_modified_file(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest], repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None: ) -> None:
@@ -304,6 +394,10 @@ def test_status_branch_ahead_of_upstream(
project_worktree = topdir / project_path project_worktree = topdir / project_path
_setup_remote_tracking_branch(manifest, "feature") _setup_remote_tracking_branch(manifest, "feature")
subprocess.check_call(
["git", "config", "status.aheadBehind", "false"],
cwd=project_worktree,
)
subprocess.check_call( subprocess.check_call(
["git", "commit", "-q", "--allow-empty", "-m", "c1"], ["git", "commit", "-q", "--allow-empty", "-m", "c1"],
cwd=project_worktree, cwd=project_worktree,
@@ -451,3 +545,20 @@ def test_status_branch_synced_no_ahead_behind(
lines = _status_lines(stdout.getvalue()) lines = _status_lines(stdout.getvalue())
assert len(lines) == 1 assert len(lines) == 1
_assert_project_header(lines[0], project_path, "synced") _assert_project_header(lines[0], project_path, "synced")
def test_status_non_utf8_path(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Non-UTF-8 pathnames render without crashing."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
bad_path = project_worktree / os.fsdecode(b"bad-\xff-name")
bad_path.write_bytes(b"content")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert any("bad-" in line for line in lines)
+559 -4
View File
@@ -13,12 +13,15 @@
# limitations under the License. # limitations under the License.
"""Unittests for the subcmds/sync.py module.""" """Unittests for the subcmds/sync.py module."""
import contextlib
import json import json
import optparse
import os import os
from pathlib import Path from pathlib import Path
import shutil import shutil
import tempfile import tempfile
import time import time
from typing import Dict, List, Optional, Tuple
import unittest import unittest
from unittest import mock from unittest import mock
@@ -27,6 +30,7 @@ import pytest
import command import command
from error import GitError from error import GitError
from error import RepoExitError from error import RepoExitError
import git_status
import manifest_xml import manifest_xml
from project import SyncNetworkHalfResult from project import SyncNetworkHalfResult
from subcmds import sync from subcmds import sync
@@ -212,6 +216,31 @@ def test_sync_update_projects_revision_id_respects_groups(tmp_path: Path):
assert kwargs.get("groups") == "group1" assert kwargs.get("groups") == "group1"
@pytest.mark.parametrize(
"generate_manpages, expected_default",
[
(False, "7; based on number of CPU cores"),
(True, "based on number of CPU cores"),
],
ids=("interactive", "manpages"),
)
def test_jobs_checkout_help_default(
generate_manpages: bool,
expected_default: str,
) -> None:
"""Test checkout-jobs default help in interactive and manpage modes."""
with mock.patch.object(sync, "DEFAULT_LOCAL_JOBS", 7), mock.patch.object(
command,
"GENERATE_MANPAGES",
generate_manpages,
):
help_text = " ".join(sync.Sync().OptionParser.format_help().split())
assert f"defaults to --jobs or {expected_default}" in help_text
if generate_manpages:
assert "defaults to --jobs or 7" not in help_text
# Used to patch os.cpu_count() for reliable results. # Used to patch os.cpu_count() for reliable results.
OS_CPU_COUNT = 24 OS_CPU_COUNT = 24
@@ -491,22 +520,49 @@ class LocalSyncState(unittest.TestCase):
class FakeProject: class FakeProject:
def __init__(self, relpath, name=None, objdir=None): def __init__(
self,
relpath: str,
name: Optional[str] = None,
objdir: Optional[str] = None,
parent: Optional["FakeProject"] = None,
is_derived: bool = False,
revisionId: Optional[str] = None,
gitlink_path: Optional[str] = None,
path_prefix: str = "",
) -> None:
self.relpath = relpath self.relpath = relpath
self.path_prefix = path_prefix
self.name = name or relpath self.name = name or relpath
self.objdir = objdir or relpath self.objdir = objdir or relpath
self.worktree = relpath self.worktree = relpath
self.parent = None self.parent = parent
self.is_derived = is_derived
self.revisionId = revisionId
self.gitlink_path = gitlink_path
self.use_git_worktrees = False self.use_git_worktrees = False
self.UseAlternates = False self.UseAlternates = False
self.UseReprojectCmd = False
self.manifest = mock.MagicMock() self.manifest = mock.MagicMock()
self.manifest.GetProjectsWithName.return_value = [self] self.manifest.GetProjectsWithName.return_value = [self]
self.config = mock.MagicMock() self.config = mock.MagicMock()
self.EnableRepositoryExtension = mock.MagicMock() self.EnableRepositoryExtension = mock.MagicMock()
def RelPath(self, local=None): @property
return self.relpath def Derived(self) -> bool:
return self.is_derived
def SetRevision(
self, revisionExpr: str, revisionId: Optional[str] = None
) -> None:
self.revisionExpr = revisionExpr
self.revisionId = revisionId or revisionExpr
def RelPath(self, local: bool = True) -> str:
if local:
return self.relpath
return os.path.join(self.path_prefix, self.relpath)
def __str__(self): def __str__(self):
return f"project: {self.relpath}" return f"project: {self.relpath}"
@@ -614,6 +670,180 @@ class SafeCheckoutOrder(unittest.TestCase):
) )
class NestedProjects(unittest.TestCase):
def test_flat_manifest(self) -> None:
p_foo = FakeProject("foo")
p_foo_bar = FakeProject("foo-bar")
self.assertEqual(sync._NestedProjects([p_foo, p_foo_bar]), [])
def test_nested_paths(self) -> None:
p_foo = FakeProject("foo")
p_foo_bar = FakeProject("foo/bar")
p_foo_bar_baz = FakeProject("foo/bar/baz")
self.assertEqual(
sync._NestedProjects([p_foo_bar_baz, p_foo, p_foo_bar]),
[p_foo_bar, p_foo_bar_baz],
)
def test_submodule_of_a_parent(self) -> None:
parent = FakeProject("foo")
sub = FakeProject("foo/sub", parent=parent, is_derived=True)
self.assertEqual(sync._NestedProjects([parent, sub]), [sub])
class ParentFirstBatches(unittest.TestCase):
def test_no_submodules(self) -> None:
p_a = FakeProject("a")
p_a_b = FakeProject("a/b")
out = sync._ParentFirstBatches([p_a, p_a_b])
self.assertEqual(out, [[p_a, p_a_b]])
def test_submodules_follow_their_parent(self) -> None:
p_a = FakeProject("a")
p_a_b = FakeProject("a/b", parent=p_a, is_derived=True)
p_a_b_c = FakeProject("a/b/c", parent=p_a_b, is_derived=True)
out = sync._ParentFirstBatches([p_a_b_c, p_a, p_a_b])
self.assertEqual(out, [[p_a], [p_a_b], [p_a_b_c]])
class RefreshDerivedRevisions(unittest.TestCase):
def _parent_with_submodules(self, **gitlinks: str) -> FakeProject:
p_a = FakeProject("a")
p_a.GetSubmoduleRevisions = mock.Mock(return_value=gitlinks)
return p_a
def _submodule(self, parent: FakeProject, path: str) -> FakeProject:
return FakeProject(
f"a/{path}", parent=parent, is_derived=True, gitlink_path=path
)
def test_reads_each_parent_once(self) -> None:
p_a = self._parent_with_submodules(b="beef1234", c="cafe1234")
p_a_b = self._submodule(p_a, "b")
p_a_c = self._submodule(p_a, "c")
sync._RefreshDerivedRevisions([p_a, p_a_b, p_a_c])
p_a.GetSubmoduleRevisions.assert_called_once_with()
self.assertEqual(p_a_b.revisionId, "beef1234")
self.assertEqual(p_a_c.revisionId, "cafe1234")
def test_reuses_gitlinks_read_for_an_earlier_level(self) -> None:
p_a = self._parent_with_submodules(b="beef1234", c="cafe1234")
p_a_b = self._submodule(p_a, "b")
p_a_c = self._submodule(p_a, "c")
submodule_revisions = {}
sync._RefreshDerivedRevisions([p_a_b], submodule_revisions)
sync._RefreshDerivedRevisions([p_a_c], submodule_revisions)
p_a.GetSubmoduleRevisions.assert_called_once_with()
self.assertEqual(p_a_c.revisionId, "cafe1234")
def test_tells_apart_projects_with_the_same_path(self) -> None:
# Paths are relative to their own (sub)manifest, so two projects can
# share one.
first = self._parent_with_submodules(b="beef1234")
second = self._parent_with_submodules(b="cafe1234")
first_sub = self._submodule(first, "b")
second_sub = self._submodule(second, "b")
submodule_revisions = {}
sync._RefreshDerivedRevisions([first_sub], submodule_revisions)
sync._RefreshDerivedRevisions([second_sub], submodule_revisions)
self.assertEqual(first_sub.revisionId, "beef1234")
self.assertEqual(second_sub.revisionId, "cafe1234")
def test_ignores_projects_from_the_manifest(self) -> None:
p_a = self._parent_with_submodules()
sync._RefreshDerivedRevisions([p_a])
p_a.GetSubmoduleRevisions.assert_not_called()
def test_reports_submodules_removed_from_their_parent(self) -> None:
p_a = self._parent_with_submodules(b="beef1234")
p_a_c = self._submodule(p_a, "c")
removed = sync._RefreshDerivedRevisions([p_a, p_a_c])
self.assertEqual(removed, [p_a_c])
def test_keeps_submodules_of_an_unreadable_parent(self) -> None:
p_a = FakeProject("a")
p_a.GetSubmoduleRevisions = mock.Mock(return_value=None)
p_a_b = FakeProject(
"a/b",
parent=p_a,
is_derived=True,
revisionId="stale",
gitlink_path="b",
)
removed = sync._RefreshDerivedRevisions([p_a, p_a_b])
self.assertEqual(removed, [])
self.assertEqual(p_a_b.revisionId, "stale")
class FetchParentFirst(unittest.TestCase):
def test_submodules_are_fetched_after_their_parent(self) -> None:
cmd = sync.Sync()
cmd._fetch_times = mock.Mock()
cmd._fetch_times.Get = mock.Mock(return_value=0)
calls = []
p_a = FakeProject("a")
def fake_read() -> Dict[str, str]:
calls.append(("read gitlinks of", p_a.relpath))
return {"b": "beef1234"}
p_a.GetSubmoduleRevisions = mock.Mock(side_effect=fake_read)
p_a_b = FakeProject(
"a/b", parent=p_a, is_derived=True, gitlink_path="b"
)
def fake_fetch(
projects: List[FakeProject], *_args: object
) -> sync._FetchResult:
calls.append(("fetch", [p.relpath for p in projects]))
return sync._FetchResult(True, {p.objdir for p in projects})
opt = mock.Mock(fail_fast=False)
with mock.patch.object(cmd, "_Fetch", side_effect=fake_fetch):
result = cmd._FetchParentFirst([p_a_b, p_a], opt, None, None, [])
self.assertTrue(result.success)
self.assertEqual(result.projects, {"a", "a/b"})
self.assertEqual(
calls,
[
("fetch", ["a"]),
("read gitlinks of", "a"),
("fetch", ["a/b"]),
],
)
class WithoutProjects(unittest.TestCase):
def test_drops_the_unwanted_projects(self) -> None:
p_a = FakeProject("a")
p_a_b = FakeProject("a/b")
self.assertEqual(sync._WithoutProjects([p_a, p_a_b], [p_a_b]), [p_a])
self.assertEqual(sync._WithoutProjects([p_a, p_a_b], []), [p_a, p_a_b])
def test_keeps_projects_with_the_same_path(self) -> None:
# Paths are relative to their own (sub)manifest, so two projects can
# share one.
first = FakeProject("a/b")
second = FakeProject("a/b")
self.assertEqual(
sync._WithoutProjects([first, second], [second]), [first]
)
class Chunksize(unittest.TestCase): class Chunksize(unittest.TestCase):
"""Tests for _chunksize.""" """Tests for _chunksize."""
@@ -760,6 +990,42 @@ class CheckForBloatedProjects(unittest.TestCase):
self.cmd.git_event_log = mock.MagicMock() self.cmd.git_event_log = mock.MagicMock()
self.cmd._bloated_projects = [] self.cmd._bloated_projects = []
def test_one_project_reuses_status_head_oid(self) -> None:
"""The bloat scan gets dirty state and HEAD from one snapshot."""
status = git_status.StatusSnapshot()
status.branch_oid = "local"
self.project._GetStatusSnapshot.return_value = status
self.project.GetRevisionId.return_value = "manifest"
self.project.bare_git.count_objects.return_value = (
"packs: 0\nsize-pack: 0\nsize-garbage: 0\n"
)
with mock.patch.object(
sync.Sync,
"get_parallel_context",
return_value={"projects": [self.project]},
):
self.assertIsNone(self.cmd._CheckOneBloatedProject(0))
self.project.IsDirty.assert_not_called()
self.project.work_git.rev_parse.assert_not_called()
self.project.bare_git.count_objects.assert_called_once_with("-v")
def test_one_unborn_project_skips_bloat_check(self) -> None:
"""A porcelain initial branch behaves like failed rev-parse HEAD."""
status = git_status.StatusSnapshot()
status.index_changes["staged"] = git_status.StatusEntry("staged", "M")
self.project._GetStatusSnapshot.return_value = status
with mock.patch.object(
sync.Sync,
"get_parallel_context",
return_value={"projects": [self.project]},
):
self.assertIsNone(self.cmd._CheckOneBloatedProject(0))
self.project.GetRevisionId.assert_not_called()
self.project.bare_git.count_objects.assert_not_called()
@mock.patch("subcmds.sync.git_require") @mock.patch("subcmds.sync.git_require")
def test_git_version_unsupported(self, mock_git_require): def test_git_version_unsupported(self, mock_git_require):
"""Test that it returns early if git version is unsupported.""" """Test that it returns early if git version is unsupported."""
@@ -902,7 +1168,10 @@ class SyncCommand(unittest.TestCase):
self.project = p = mock.MagicMock( self.project = p = mock.MagicMock(
use_git_worktrees=False, use_git_worktrees=False,
UseAlternates=False, UseAlternates=False,
UseReprojectCmd=False,
name="project", name="project",
relpath="rel_path",
parent=None,
Sync_NetworkHalf=Sync_NetworkHalf, Sync_NetworkHalf=Sync_NetworkHalf,
Sync_LocalHalf=Sync_LocalHalf, Sync_LocalHalf=Sync_LocalHalf,
RelPath=mock.Mock(return_value="rel_path"), RelPath=mock.Mock(return_value="rel_path"),
@@ -961,6 +1230,78 @@ class SyncCommand(unittest.TestCase):
_, kwargs = self.cmd.GetProjects.call_args _, kwargs = self.cmd.GetProjects.call_args
self.assertEqual(kwargs.get("groups"), "my_group") self.assertEqual(kwargs.get("groups"), "my_group")
def _ExecuteUntilSync(
self, args: List[str]
) -> Tuple[mock.MagicMock, mock.MagicMock]:
"""Run Execute up to the sync itself, returning the sync mocks."""
self.opt.mp_update = False
with contextlib.ExitStack() as stack:
for name in (
"_UpdateRepoProject",
"_UpdateProjectsRevisionId",
"_ValidateOptionsWithManifest",
"_RunPostSyncHook",
):
stack.enter_context(mock.patch.object(self.cmd, name))
phased = stack.enter_context(
mock.patch.object(self.cmd, "_SyncPhased")
)
interleaved = stack.enter_context(
mock.patch.object(self.cmd, "_SyncInterleaved")
)
self.cmd.Execute(self.opt, args)
return phased, interleaved
def test_reproject_cmd_allows_a_flat_manifest(self) -> None:
"""Ensure repo.reprojectcmd syncs a manifest without nesting."""
self.project.UseReprojectCmd = True
phased, interleaved = self._ExecuteUntilSync([])
self.assertTrue(phased.called or interleaved.called)
def test_reproject_cmd_rejects_nested_projects(self) -> None:
"""Ensure repo.reprojectcmd fails a manifest with nested projects."""
p_foo = FakeProject("foo")
p_foo_bar = FakeProject("foo/bar")
p_foo.UseReprojectCmd = p_foo_bar.UseReprojectCmd = True
self.cmd.GetProjects.return_value = [p_foo, p_foo_bar]
with self.assertRaises(sync.SyncError) as e:
self._ExecuteUntilSync([])
self.assertIn("foo/bar", str(e.exception))
self.assertNotIn(" - foo\n", str(e.exception))
def test_reproject_cmd_rejects_a_submodule(self) -> None:
"""Ensure repo.reprojectcmd fails a manifest with a submodule."""
p_foo = FakeProject("foo")
p_sub = FakeProject("foo/sub", parent=p_foo, is_derived=True)
p_foo.UseReprojectCmd = p_sub.UseReprojectCmd = True
self.cmd.GetProjects.return_value = [p_foo, p_sub]
with self.assertRaises(sync.SyncError) as e:
self._ExecuteUntilSync([])
self.assertIn("foo/sub", str(e.exception))
def test_reproject_cmd_checks_the_whole_manifest(self) -> None:
"""Ensure nesting is checked beyond the projects given as args."""
p_foo = FakeProject("foo")
p_foo_bar = FakeProject("foo/bar")
p_foo.UseReprojectCmd = p_foo_bar.UseReprojectCmd = True
self.cmd.GetProjects.side_effect = lambda args, **kwargs: (
[p_foo_bar] if args else [p_foo, p_foo_bar]
)
with self.assertRaises(sync.SyncError):
self._ExecuteUntilSync(["foo/bar"])
self.assertEqual(self.cmd.GetProjects.call_count, 2)
_, kwargs = self.cmd.GetProjects.call_args
self.assertEqual(kwargs.get("missing_ok"), True)
def test_reproject_cmd_off_ignores_nested_projects(self) -> None:
"""Ensure nesting is only checked with repo.reprojectcmd in use."""
projects = [FakeProject("foo"), FakeProject("foo/bar")]
for p in projects:
p.Exists = False
self.cmd.GetProjects.return_value = projects
phased, interleaved = self._ExecuteUntilSync([])
self.assertTrue(phased.called or interleaved.called)
class SyncUpdateRepoProject(unittest.TestCase): class SyncUpdateRepoProject(unittest.TestCase):
"""Tests for Sync._UpdateRepoProject.""" """Tests for Sync._UpdateRepoProject."""
@@ -1158,6 +1499,203 @@ class InterleavedSyncTest(unittest.TestCase):
execute_mock.assert_called_once() execute_mock.assert_called_once()
def test_interleaved_refreshes_submodule_revision(self) -> None:
"""Test submodules are synced at the revision of the fetched parent."""
opt, args = self.cmd.OptionParser.parse_args(["--interleaved", "-j4"])
opt.quiet = True
submodule = FakeProject(
"projA/sub",
name="projA_sub",
objdir="objA_sub",
parent=self.projA,
is_derived=True,
revisionId="stale",
gitlink_path="sub",
)
all_projects = [self.projA, submodule]
mock.patch.object(
self.cmd, "GetProjects", return_value=all_projects
).start()
self.projA.GetSubmoduleRevisions = mock.Mock(
return_value={"sub": "fetched"}
)
synced = []
def execute_side_effect(
jobs: int,
target: object,
work_items: List[List[int]],
**kwargs: object,
) -> bool:
synced_relpaths_set = kwargs["callback"].args[0]
projects_in_pass = self.cmd.get_parallel_context()["projects"]
for item in work_items:
for project_idx in item:
project = projects_in_pass[project_idx]
synced.append((project.relpath, project.revisionId))
synced_relpaths_set.add(project.relpath)
return True
mock.patch.object(
self.cmd, "ExecuteInParallel", side_effect=execute_side_effect
).start()
self.cmd._SyncInterleaved(
opt,
args,
[],
self.manifest,
self.manifest.manifestProject,
all_projects,
{},
)
self.assertIn(("projA/sub", "fetched"), synced)
def test_interleaved_skips_removed_submodule(self) -> None:
"""Test submodules dropped by their parent are not checked out."""
opt, args = self.cmd.OptionParser.parse_args(["--interleaved", "-j4"])
opt.quiet = True
submodule = FakeProject(
"projA/sub",
name="projA_sub",
objdir="objA_sub",
parent=self.projA,
is_derived=True,
revisionId="stale",
gitlink_path="sub",
)
# The parent no longer holds a gitlink for the submodule.
self.projA.GetSubmoduleRevisions = mock.Mock(return_value={})
# The reloaded manifest no longer derives the removed submodule.
mock.patch.object(
self.cmd, "GetProjects", return_value=[self.projA]
).start()
synced = []
def execute_side_effect(
jobs: int,
target: object,
work_items: List[List[int]],
**kwargs: object,
) -> bool:
synced_relpaths_set = kwargs["callback"].args[0]
projects_in_pass = self.cmd.get_parallel_context()["projects"]
for item in work_items:
for project_idx in item:
project = projects_in_pass[project_idx]
synced.append(project.relpath)
synced_relpaths_set.add(project.relpath)
return True
mock.patch.object(
self.cmd, "ExecuteInParallel", side_effect=execute_side_effect
).start()
self.cmd._SyncInterleaved(
opt,
args,
[],
self.manifest,
self.manifest.manifestProject,
[self.projA, submodule],
{},
)
self.assertEqual(synced, ["projA"])
def _make_syncable(self, project: FakeProject) -> FakeProject:
project.Sync_NetworkHalf = mock.Mock(
return_value=SyncNetworkHalfResult(error=None, remote_fetched=True)
)
project.Sync_LocalHalf = mock.Mock()
return project
def _run_interleaved(
self,
opt: optparse.Values,
initial_projects: List[FakeProject],
reloaded_projects: List[FakeProject],
) -> None:
"""Run _SyncInterleaved with the real workers and callback.
|initial_projects| make up the first pass, |reloaded_projects| every
later one, the way reloading the manifest between passes does.
"""
mock.patch.object(
self.cmd, "GetProjects", return_value=reloaded_projects
).start()
mock.patch.object(self.cmd, "event_log").start()
def execute_side_effect(
jobs: int,
target: object,
work_items: List[List[int]],
**kwargs: object,
) -> bool:
results = [target(item) for item in work_items]
return kwargs["callback"](None, kwargs["output"], results)
mock.patch.object(
self.cmd, "ExecuteInParallel", side_effect=execute_side_effect
).start()
with mock.patch("subcmds.sync.SyncBuffer") as mock_sync_buffer:
mock_sync_buffer.return_value.Finish.return_value = True
mock_sync_buffer.return_value.errors = []
self.cmd._SyncInterleaved(
opt,
[],
[],
self.manifest,
self.manifest.manifestProject,
initial_projects,
{},
)
def test_interleaved_syncs_same_path_projects_of_every_manifest(
self,
) -> None:
"""Test a project is not skipped because another shares its path."""
opt = self._get_opts(["--interleaved", "-j4"])
outer = self._make_syncable(
FakeProject("foo", name="outer", objdir="a")
)
sub = self._make_syncable(
FakeProject("foo", name="sub", objdir="b", path_prefix="sub")
)
# |sub| is only discovered once the manifest is reloaded after the
# first pass has synced |outer|.
self._run_interleaved(opt, [outer], [outer, sub])
outer.Sync_LocalHalf.assert_called_once()
sub.Sync_LocalHalf.assert_called_once()
def test_interleaved_reports_failures_by_a_unique_path(self) -> None:
"""Test failing projects are listed by a path that is theirs alone."""
opt = self._get_opts(["--interleaved", "-j4"])
outer = self._make_syncable(
FakeProject("foo", name="outer", objdir="a")
)
sub = self._make_syncable(
FakeProject("foo", name="sub", objdir="b", path_prefix="sub")
)
sub.Sync_LocalHalf.side_effect = GitError("checkout failed")
self.cmd.git_event_log = mock.MagicMock()
with self.assertRaises(sync.SyncError):
self._run_interleaved(opt, [outer, sub], [outer, sub])
self.assertEqual(
self.cmd._interleaved_err_checkout_results, ["sub/foo"]
)
def test_interleaved_shared_objdir_serial(self): def test_interleaved_shared_objdir_serial(self):
"""Test that projects with shared objdir are processed serially.""" """Test that projects with shared objdir are processed serially."""
opt, args = self.cmd.OptionParser.parse_args(["--interleaved", "-j4"]) opt, args = self.cmd.OptionParser.parse_args(["--interleaved", "-j4"])
@@ -1252,6 +1790,23 @@ class InterleavedSyncTest(unittest.TestCase):
project.Sync_NetworkHalf.assert_called_once() project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_called_once() project.Sync_LocalHalf.assert_called_once()
def test_worker_reports_a_path_unique_across_manifests(self) -> None:
"""Test _SyncResult.relpath tells apart same-path projects."""
project = FakeProject("foo", objdir="objA", path_prefix="sub")
self._make_syncable(project)
self.mock_context["projects"] = [project]
for this_manifest_only, expected in ((False, "sub/foo"), (True, "foo")):
with self.subTest(this_manifest_only=this_manifest_only):
opt = self._get_opts()
opt.this_manifest_only = this_manifest_only
with mock.patch("subcmds.sync.SyncBuffer") as mock_sync_buffer:
mock_sync_buffer.return_value.Finish.return_value = True
mock_sync_buffer.return_value.errors = []
result_obj = self.cmd._SyncProjectList(opt, [0])
self.assertEqual(result_obj.results[0].relpath, expected)
def test_worker_fetch_fails(self): def test_worker_fetch_fails(self):
"""Test _SyncProjectList with a failed fetch.""" """Test _SyncProjectList with a failed fetch."""
opt = self._get_opts() opt = self._get_opts()
+170
View File
@@ -14,6 +14,7 @@
"""Unittests for the subcmds/upload.py module.""" """Unittests for the subcmds/upload.py module."""
from typing import List, Optional
from unittest import mock from unittest import mock
import pytest import pytest
@@ -63,3 +64,172 @@ def test_UploadAndReport_UnhandledError(cmd: upload.Upload) -> None:
with mock.patch.object(cmd, "_UploadBranch", side_effect=UnexpectedError): with mock.patch.object(cmd, "_UploadBranch", side_effect=UnexpectedError):
with pytest.raises(UnexpectedError): with pytest.raises(UnexpectedError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE) cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
def test_GetMergeBranch_explicit_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch reads branch.merge for explicit local_branch."""
mock_project = mock.MagicMock()
mock_branch = mock.MagicMock()
mock_branch.merge = "refs/heads/main"
mock_project.GetBranch.return_value = mock_branch
res = cmd._GetMergeBranch(mock_project, local_branch="feature")
assert res == "refs/heads/main"
mock_project.GetBranch.assert_called_once_with("feature")
def test_GetMergeBranch_current_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch falls back to project.CurrentBranch."""
mock_project = mock.MagicMock()
mock_project.CurrentBranch = "auto-cbr"
mock_branch = mock.MagicMock()
mock_branch.merge = "refs/heads/upstream-main"
mock_project.GetBranch.return_value = mock_branch
res = cmd._GetMergeBranch(mock_project, local_branch=None)
assert res == "refs/heads/upstream-main"
mock_project.GetBranch.assert_called_once_with("auto-cbr")
def test_GetMergeBranch_none_when_no_branch(cmd: upload.Upload) -> None:
"""Verify _GetMergeBranch returns empty string when detached HEAD."""
mock_project = mock.MagicMock()
mock_project.CurrentBranch = None
res = cmd._GetMergeBranch(mock_project, local_branch=None)
assert res == ""
def test_GatherOne_returns_resolved_current_branch(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Upload error reporting reuses the branch gathered by the worker."""
project = mock.MagicMock()
project.CurrentBranch = "topic"
branch = mock.sentinel.branch
project.GetUploadableBranch.return_value = branch
monkeypatch.setattr(
upload.Upload,
"get_parallel_context",
lambda: {"projects": [project]},
)
opt = mock.MagicMock(current_branch=True)
assert upload.Upload._GatherOne(opt, 0) == (0, [branch], "topic")
project.GetUploadableBranch.assert_called_once_with("topic")
def _create_mock_branch(
name: str = "main",
commits: Optional[List[str]] = None,
project_relpath: str = "project-a",
) -> mock.MagicMock:
"""Helper to construct a mock ReviewableBranch."""
branch = mock.MagicMock()
branch.name = name
branch.commits = commits if commits is not None else ["commit1"]
project = mock.MagicMock()
project.RelPath.return_value = project_relpath
branch.project = project
return branch
def test_MultipleBranches_yes_with_current_branch_flag_bypasses_editor(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with --yes and -c flag bypasses editor."""
opt, _ = cmd.OptionParser.parse_args(["-c", "-y"])
branch1 = _create_mock_branch("b1", project_relpath="p1")
branch2 = _create_mock_branch("b2", project_relpath="p2")
pending = [(branch1.project, [branch1]), (branch2.project, [branch2])]
with mock.patch.object(cmd, "_UploadAndReport") as mock_upload, mock.patch(
"editor.Editor.EditString"
) as mock_edit:
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_not_called()
mock_upload.assert_called_once_with(
opt, [branch1, branch2], _STUB_PEOPLE
)
def test_MultipleBranches_yes_with_cbr_flag_bypasses_editor(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with --yes and --cbr flag bypasses editor."""
opt, _ = cmd.OptionParser.parse_args(["--cbr", "--yes"])
branch1 = _create_mock_branch("b1", project_relpath="p1")
branch2 = _create_mock_branch("b2", project_relpath="p2")
pending = [(branch1.project, [branch1]), (branch2.project, [branch2])]
with mock.patch.object(cmd, "_UploadAndReport") as mock_upload, mock.patch(
"editor.Editor.EditString"
) as mock_edit:
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_not_called()
mock_upload.assert_called_once_with(
opt, [branch1, branch2], _STUB_PEOPLE
)
def test_MultipleBranches_yes_with_branch_flag_bypasses_editor(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with --yes and --br flag bypasses editor."""
opt, _ = cmd.OptionParser.parse_args(["--br", "feature", "-y"])
branch1 = _create_mock_branch("feature", project_relpath="p1")
branch2 = _create_mock_branch("feature", project_relpath="p2")
pending = [(branch1.project, [branch1]), (branch2.project, [branch2])]
with mock.patch.object(cmd, "_UploadAndReport") as mock_upload, mock.patch(
"editor.Editor.EditString"
) as mock_edit:
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_not_called()
mock_upload.assert_called_once_with(
opt, [branch1, branch2], _STUB_PEOPLE
)
def test_MultipleBranches_yes_with_branch_flag_empty_pending_dies(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with --yes and empty pending branches dies."""
opt, _ = cmd.OptionParser.parse_args(["--br", "feature", "-y"])
mock_project = mock.MagicMock()
pending = [(mock_project, [])]
with pytest.raises(
upload.UploadExitError, match="nothing ready for upload"
):
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
def test_MultipleBranches_yes_without_branch_or_cbr_uses_editor(
cmd: upload.Upload,
) -> None:
"""_MultipleBranches with -y but no -c/--br falls back to editor."""
opt, _ = cmd.OptionParser.parse_args(["-y"])
branch1 = _create_mock_branch("b1", project_relpath="p1")
branch1.date = "2026-08-26"
mock_remote = mock.MagicMock()
branch1.project.dest_branch = None
branch1.project.revisionExpr = "refs/heads/main"
branch_config = mock.MagicMock()
branch_config.remote = mock_remote
branch1.project.GetBranch.return_value = branch_config
pending = [(branch1.project, [branch1])]
edited_script = (
"project p1/:\n"
" branch b1 ( 1 commit, 2026-08-26) to remote branch "
"refs/heads/main:\n"
)
with mock.patch.object(cmd, "_UploadAndReport") as mock_upload, mock.patch(
"editor.Editor.EditString", return_value=edited_script
) as mock_edit:
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_called_once()
mock_upload.assert_called_once_with(opt, [branch1], _STUB_PEOPLE)
+52
View File
@@ -0,0 +1,52 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for subcmds/version.py."""
from unittest import mock
import pytest
from subcmds import version
def test_repo_version_uses_one_pretty_format_call(
monkeypatch: pytest.MonkeyPatch,
) -> None:
project = mock.MagicMock()
project.bare_git.log.return_value = "v2.0-1-g12345678\nTue, 25 Aug\n"
monkeypatch.setattr(version, "git_require", lambda _version: True)
result = version.Version._RepoVersion(project)
assert result == ("v2.0-1-g12345678", "Tue, 25 Aug")
project.bare_git.log.assert_called_once_with(
"-1", "--format=%(describe)%n%cD", "HEAD"
)
project.bare_git.describe.assert_not_called()
def test_repo_version_keeps_old_git_fallback(
monkeypatch: pytest.MonkeyPatch,
) -> None:
project = mock.MagicMock()
project.bare_git.describe.return_value = "v2.0"
project.bare_git.log.return_value = "Tue, 25 Aug"
monkeypatch.setattr(version, "git_require", lambda _version: False)
result = version.Version._RepoVersion(project)
assert result == ("v2.0", "Tue, 25 Aug")
project.bare_git.describe.assert_called_once_with("HEAD")
project.bare_git.log.assert_called_once_with("-1", "--format=%cD", "HEAD")