Compare commits

..
48 Commits
Author SHA1 Message Date
Mike Frysinger 338ddb7d83 style: use import git_refs everywhere
Align with Google style guides and `import git_refs` as a module
rather than import symbols out of it.

Change-Id: I4e33ebdf3b06acd94494f32cdf39aa58427cd4b2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648703
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-10-27 02:00:11 -07:00
Gavin Mak 8c316d7c98 upload: suppress git push progress when quiet or non-TTY
`Project.UploadForReview()` unconditionally passed `--progress` to
`git push`, forcing client and remote progress meters (`Counting
objects`, `remote: Resolving deltas: 3%..100%`) even on non-TTY
stderr or when `-q`/`--quiet` was passed. Meanwhile, `repo upload`
ignored `opt.quiet` completely.

Move `--progress` injection for `push` into `GitCommand` alongside
`fetch` and `clone`, enabling it only when `sys.stderr.isatty()` and
none of `--progress`, `--no-progress`, `--quiet`, or `-q` are set.

Forward `opt.quiet` from `repo upload` to `Project.UploadForReview()`
and pass `--no-progress` to `git push` rather than `--quiet` so
Gerrit `remote: https://...` review URLs are still emitted and parsed
by `_FindGerritUrls()`. Also suppress the non-error `[OK]` summary
banner when `-q`/`--quiet` is set.

Bug: 578114287
Change-Id: I88b7b055dcca0db80dae6962886a693fe717c741
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/649663
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-10-26 14:51:52 -07:00
Brian Gan e3e8e8a3b0 tests: e2e: add tests for more repo init options
Cover `--reference` from a local mirror (with & without `--dissociate`),
`--standalone-manifest` with a manifest file served over HTTP, and
`--manifest-depth`.

Teach the server to serve static files (`GitHost.publish_file`), let
`manifest_xml` take an absolute `fetch` URL, and add `move_aside` to
make git dirs disappear mid-test (deleting them fails on Windows).

Bug: 566270944
Change-Id: Id60a19613a627f76d8d9125570dd5ce0cdfff9aa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648549
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-26 13:52:54 -07:00
Brian Gan cad9f591a4 tests: e2e: add tests for sync failures
Cover projects missing on the server (incl. `--fail-fast`), missing
project & manifest revisions, and local changes that conflict with an
incoming change.  The tests check that repo exits non-zero with a useful
error, leaves the rest of the tree usable, and recovers once the problem
is fixed.

Let `RepoClient.init` return failures instead of raising.

Bug: 566270944
Change-Id: I84e2b8c0bb6012678add77360b446d4bea010c78
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648548
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-26 13:52:43 -07:00
Brian Gan 24197b8ae1 tests: e2e: add tests for manifest groups
Cover re-running `repo init -g` to add & remove projects,
`<include groups=...>`, `platform-*` groups & `--platform`, and groups
combined with a superproject.

Let `RepoClient.list_projects` pass args to `repo list`.

Bug: 566270944
Change-Id: Id333078fd18d38962d8b725ae9bd87f4976c3112
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648547
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-26 13:52:38 -07:00
Brian Gan bfb9869b89 tests: e2e: add tests for partial clones & the Android setup
Cover the go/repo-init flags used across Android (superproject, `-c`,
`--partial-clone`, `--partial-clone-exclude` & `--clone-filter`)
followed by the go/sync flags, under Git 2 & Git 3 defaults: projects
are pinned to the superproject, filtered blobs stay on the server,
excluded projects are full clones, and incremental syncs skip projects
whose pin didn't move.  Also cover the default filter, and `sync -n`
then `sync -l` with a partial clone (where `-l` has to fetch blobs on
demand).

Add harness helpers to read a checkout's git config, list the repos a
client contacted, and create blobs that only exist in history.

Bug: 566270944
Change-Id: I77d2cabc1d4f5db2ea139c7825ba87638c6db1e7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648546
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
2026-10-26 13:00:28 -07:00
Brian Gan 887f9a9014 tests: e2e: add tests for network-only & local-only syncs
Cover `repo sync -n` & `-l` on existing checkouts (interleaved &
phased), projects that were never fetched, and two-phase syncs with a
superproject.  `-l` must not touch the network, which the tests check by
taking the server offline.  Also cover tags (`--no-tags` & `sync-tags`),
`--prune`, `--force-sync`, `--force-remove-dirty` & `-d`.

Teach the harness to simulate outages (`GitHost.offline`) and to collect
the requests made during a block (`GitHost.watch_requests`), and add a
few `RepoClient` helpers.

Bug: 566270944
Change-Id: I96b4729a7d4e5b58bc57fb46a060beda35549979
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648545
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-26 12:57:09 -07:00
Brian Gan dc1811ba23 tests: e2e: add tests for init, sync & common workflows
Cover the core `repo init` & `repo sync` options (current branch only,
clone bundles, shallow clones, groups, local manifests & manifest
names), multi-step workflows (two-phase syncs, mirrors, incremental
syncs, local work, switching manifest branches, selfupdate, ...), and
manifests that broke users in the past.  Some tests also run under Git 3
defaults.

Add `RepoClient` helpers to list remote-tracking branches & checkouts,
and document the test layout in the README.

Bug: 566270944
Change-Id: Iac333216e0f27e489e64f4eb853189f860a91f20
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648544
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-26 12:53:32 -07:00
Brian Gan 9aac80dafc tests: e2e: fix harness on Windows, CI & inside git checkouts
Normalize the `\r\n` that Python on Windows writes to pipes in repo's
output so tests can match multi-line messages.  Pass a str to
`os.readlink`, which only accepts Path objects on Windows since Python
3.8.  Set `GIT_CEILING_DIRECTORIES` for the client so git never finds a
repo above the test dir (e.g. when running with `--basetemp=./tmp` from
a checkout).

Run the real git rather than a `git` wrapper that's first on PATH: the
real git's exec dir (from `git --exec-path`) has a copy of git, so put
it first on PATH.  LUCI's wrapper, for one, retries server errors for
minutes, which hangs tests that simulate them.

Bug: 566270944
Change-Id: Ia0ed5285852638e533632a7272f154239f1e834f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648543
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-26 12:50:26 -07:00
Mike Frysinger f4ce9572d1 style: use import git_command everywhere
Align with Google style guides and `import git_command` as a module
rather than import symbols out of it.

Change-Id: If80d95203e240dd6591e465ef75500c72401d2bf
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/648503
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-10-23 17:00:18 -07:00
Mike Frysinger cc1f41d782 style: use import repo_logging everywhere
Align with Google style guides and `import repo_logging` as a module
rather than import symbols out of it.

Change-Id: Ie13c1f13be00dbf6dd3814a8780baf6453972237
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647823
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-10-23 09:49:36 -07:00
Mike Frysinger f7d252f5dc style: use import repo_trace everywhere
Align with Google style guides and `import repo_trace` as a module
rather than import symbols out of it.

Change-Id: I6bea5e7a68775e93b6b7fcd218db666350aff690
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647803
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-23 09:01:00 -07:00
Mike Frysinger f95e7cd899 style: use import ctypes everywhere
Align with Google style guides and `import ctypes` as a module rather
than import symbols out of it.

Change-Id: I256495f9aaf4566284c2e2209a0e40268ed1aa37
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647603
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Insaf Latypov <saferif@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-10-22 17:44:42 -07:00
Mike Frysinger 2c04b98c60 style: use import error everywhere
Align with Google style guides and `import error` as a module rather
than import classes out of it.

Change-Id: I8686fbe7887bd6ceac7a07877af63934f0d0ddd2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647563
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Insaf Latypov <saferif@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-10-22 17:44:11 -07:00
Brian Gan 4212588c20 tests: e2e: add pytest fixtures & e2e marker
Add tests/e2e/conftest.py with fixtures for writing end-to-end tests: a
session-wide git server, the published code under test, a per-test
namespace of server repos, and a repo client with an isolated HOME &
git config.  Parametrize `git_defaults` to emulate Git 3 defaults
(reftable & `main` as the default branch) on older git.

Tests under tests/e2e/ are marked `e2e` and run after the unit tests in
`./run_tests`.  Use `-m e2e` or `-m 'not e2e'` to select them.  They're
skipped when `git http-backend` is unavailable, git is older than 2.28,
or not running from a git checkout.

Bug: 566270944
Change-Id: Ib61e4dd7acc41a65beec4d74f1635a25dce4e3be
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647643
Commit-Queue: Brian Gan <brgan@google.com>
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-22 16:43:23 -07:00
Brian Gan be06f8dd01 tests: e2e: add a client to run repo against the local server
Add `RepoClient`, which runs a standalone copy of the `repo` launcher as
a subprocess the same way end users do, and `publish_repo_source`, which
publishes the code under test (the current checkout, including
uncommitted changes) as a repo on the server so `repo init` installs it
into .repo/repo/ over HTTP.

Also add `GIT_DEFAULTS` so tests can emulate Git 3 defaults (reftable &
`main` as the default branch) on older git.

Bug: 566270944
Change-Id: I01ef53362b7812f4cb3e012509350980413cf0a8

Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647505
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-22 16:43:19 -07:00
Brian Gan 74d887ea86 tests: e2e: add a local smart HTTP git server
Add `GitHttpServer`, which serves the bare repositories under a
directory over smart HTTP by running `git http-backend` behind a small
threaded HTTP server on 127.0.0.1.  Like Gerrit/GoB, it also serves
static `<repo>/clone.bundle` files.  It records the requests it receives
so tests can check what clients fetched.

Add `GitHost`, which gives each test its own namespace of repos on a
shared server, with helpers to create & update a manifest repo.

Bug: 566270944
Change-Id: I35ff0b9ae743ed7a8d3c775525986e662ef03de0

Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647504
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-22 16:43:14 -07:00
Brian Gan 29d644a5c0 tests: e2e: add helpers to create bare git repos
This is the first piece of a hermetic end-to-end test harness for
running the real `repo` against a local git server.

Add tests/e2e/harness.py with `ServerRepo`, which creates bare
repositories, commits (including gitlinks for superprojects), branches,
tags & clone bundles without needing a worktree.  Commits are written
with `git fast-import` using a fixed identity & a monotonic fake clock
so they're cheap & strictly ordered.  Also add helpers to run git with
the user's git environment stripped & to write git config files
directly.

Bug: 566270944
Change-Id: Ifa9cf84529a699987bf6b21bbe94bd43483d1e73

Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647503
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Tested-by: Brian Gan <brgan@google.com>
2026-10-22 16:43:09 -07:00
Brian Gan a93741993b tests: fail skipped tests in CQ
A skip in CQ usually means the builder is missing a dependency (e.g. git
without reftable support), which silently drops coverage while the check
still passes.  Add a `--fail-on-skip` pytest option that turns skips
into failures, and have run_tests pass it in CI alongside the existing
`skip_cq` deselection.  Tests that shouldn't run in CQ keep using the
`skip_cq` marker.

Bug: 576728033
Change-Id: I2eb8ab168859a9a13d05ba7c771bb455c082f512
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647626
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
2026-10-22 16:43:05 -07:00
Insaf Latypov 9991aa6177 config: add subcommand to manage ~/.repoconfig/config
Per-user settings such as sync.smartsync live in ~/.repoconfig/config,
but reading or changing them required knowing the file location and
running `git config --file ~/.repoconfig/config ...`.

Add `repo config` as a thin wrapper around RepoConfig.ForUser(),
following the subcommand style of modern `git config`:
* `repo config get <name>` prints the value(s), exiting 1 if unset.
* `repo config set <name> <value>` sets it.
* `repo config unset <name>` removes it; a missing key is a no-op.
* `repo config list` shows all settings.

Values are stored as strings; `--type=bool|int` validates and
canonicalizes them on set and get, like `git config --type`.  `-z` makes
get and list NUL-terminated for scripting.  Setting names are validated
by `git config` itself when writing.

Factor the value parsing out of GitConfig.GetBoolean()/GetInt() into
ParseBoolean()/ParseInt() so the command and the readers agree, and
accept on/off/1/0 as booleans like git does.

Only the per-user file is handled; per-checkout settings in
.repo/manifests.git/config are out of scope.  Like other subcommands, it
runs from inside a repo checkout.

Update docs/repoconfig.md to use the new command in its examples and
add zsh completion and the man page.

Bug: 569927750
Test: ./run_tests
Change-Id: I940d0318549fe9d79f6f9173139f0b96a706f1bf
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/645727
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Insaf Latypov <saferif@google.com>
Commit-Queue: Insaf Latypov <saferif@google.com>
2026-10-22 08:34:46 -07:00
Brian Gan d744eb15f3 project: accept v3 git bundles in _IsValidBundle
Git 2.29+ introduced v3 bundle headers (`# v3 git bundle`) for SHA-256
and filtered bundles. Allow both v2 and v3 bundle headers when
validating downloaded `clone.bundle` files.

Bug: 576377949
Change-Id: Ic5ac94bf535a801d948f8932213e62a6cd132cac
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/647123
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-21 16:04:30 -07:00
LaMont Jones f0124d0fd5 sync, init: support repo.smartsync config and --head alias
Allow users to configure Smart Sync as the default for a checkout:

* Add `ManifestProject.smart_sync` backed by
  `self.config.GetBoolean("repo.smartsync")`.
* Add `-s`/`--smart-sync` and `--no-smart-sync`/`--head` to `repo init`
  to persist `repo.smartsync` in the manifest project config.
* Add `--head` as an alias for `--no-smart-sync` in `repo sync` and
  `repo init`.
* Update `Sync._ResolveSmartSyncOption` so `ManifestProject.smart_sync`
  takes precedence over `<default sync-smartsync="true" />` while
  remaining overridable by per-invocation `repo sync` CLI flags.

Bug: 569927750
Test: pytest tests/test_project.py tests/test_subcmds_init.py tests/test_subcmds_sync.py
Change-Id: Ib40efc530336efb59a28acfed3d582713e9d67b2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/643303
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: LaMont Jones <lamontjones@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2026-10-21 11:22:58 -07:00
Insaf Latypov 46e9866ca0 sync: Provide a CLI command to opt in/out of smartsync for a user.
Add a boolean `sync.smartsync` setting in the per-user repo config
(~/.repoconfig/config):
* true: smart sync by default, as if -s were passed.
* false: never smart sync by default, even if the manifest enables it.
* unset: follow the manifest default, as before.

Add docs/repoconfig.md as the user facing documentation for
~/.repoconfig/config and its settings, and point smart-sync.md and the
`repo sync` help at it.

Bug: 569927750
Test: ./run_tests
Change-Id: Ideb4119fcb5e90f8bb35c493d9e4f8bec7878b77
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/643264
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Insaf Latypov <saferif@google.com>
Tested-by: Insaf Latypov <saferif@google.com>
2026-10-19 12:20:06 -07:00
Rahul Yadav 404aec7b53 upload: reject --fix with --ignore-hooks or --no-verify
--fix asks the pre-upload hook to apply its fixups, but:
* --no-verify skips the hook entirely, so --fix silently does nothing.
* --ignore-hooks uploads the original commits even if the hook fails,
  so any fixups applied to the working tree are not part of the
  upload. Whether --fix should then apply fixups or be ignored is
  ambiguous.

Reject both combinations up front, before any hook runs, instead of
picking an interpretation.

Bug: 562921767
Test: python3 -m pytest tests/test_subcmds_upload.py
Change-Id: Ib00ae69004642f2e763212b82e7c39f3b4da52eb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/642664
Tested-by: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
2026-10-16 00:18:33 -07:00
Insaf Latypov a0bc2b14a3 git_config: Fix bugs in processing RepoConfig.
- Automatically create all the parent directories for `git confing
  --file ...`
- Prevent `RepoConfig` from inheriting caches from `GitConfig`. These
  two are different files.

Bug: 569927750
Test: ./run_tests
Change-Id: I9c60cecff75c1fb62be3ff63425d9618c0f56b06
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/643263
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Insaf Latypov <saferif@google.com>
Commit-Queue: Insaf Latypov <saferif@google.com>
2026-10-14 07:49:21 -07:00
Rahul Yadav 3448df3954 sync: Add GetApprovedManifestV2 with named request options
Smart sync calls GetApprovedManifest(branch[, target]) over XML-RPC,
which only supports positional arguments.  Adding new request options
that way does not scale.

Add a GetApprovedManifestV2 method that takes a single struct of named
options and returns a struct:

  request:  {branch, target?, best_effort?}
  response: {success, manifest_xml?, error_message?}

Servers must ignore unknown options, so new options (e.g. criteria for
selecting a good build) can be added without another method.

The first new option is `best_effort`.  repo sets it when smart sync was
enabled implicitly by the manifest's sync-smartsync="true" default
rather than by -s.  The server may decline a best_effort request (e.g.
for rollout or cohort gating) by returning success=false.  The client
still owns the fallback policy: declined implicit requests fall back to
a regular sync, while explicit -s requests fail as before.

If the server does not implement GetApprovedManifestV2, repo falls back
to the legacy GetApprovedManifest call.

Bug: 545137090
Test: ./run_tests
Change-Id: I9297b728df26516012cb93dffc91d39c1fb0458f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/636623
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
2026-10-14 03:30:36 -07:00
Brian Gan a110680ff3 repo: tolerate a transiently missing stable branch
The automated release process deletes the stable branch and then
recreates it. A client that fetches during that window can't find it:
`repo init` fails to resolve --repo-rev, and the daily self-update in
`repo sync` fails with "Cannot fetch repo tool".

In the launcher, if --repo-rev doesn't resolve after the initial fetch,
wait 10 seconds and refetch once before giving up. `resolve_repo_rev`
gains a `report_errors` flag so this first check doesn't print an error
that the refetch then fixes. Bump the launcher version to 2.69 so
existing users pick this up.

For fetches of the repo project itself (sync's self-update and
`repo selfupdate`), wait 10 seconds instead of 4 before retrying. The
delay is now a `RETRY_SLEEP_INITIAL_SEC` class attribute that
`RepoProject` overrides, so other projects keep the 4 second delay. This
replaces the unused `retry_sleep_initial_sec` parameter of
`_RemoteFetch`.

Bug: 566270786
Change-Id: I23cb810258522b5aa9eae9d7013dbb7904ebcad8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/643265
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-13 13:55:40 -07:00
Rahul Yadav 178b66ee3e sync: keep smart sync override with -l
Splitting a sync into `repo sync -n` followed by `repo sync -l` should
end up in the same state as a single `repo sync`. With smart sync it
does not always: -n fetches the revisions pinned by the smart sync
manifest and saves it as smart_sync_override.xml, but the following -l
deletes that override and checks out ToT instead, unless the manifest
sets sync-smartsync="true". So `repo sync -s -n` + `repo sync -l` lands
on ToT, while `repo sync -s` lands on the pinned revisions.

Without an explicit -s or -t, -l does not contact the manifest server,
so it cannot choose new pins itself; the only consistent choice is to
keep the override the previous -n left behind. Do that regardless of
the manifest default.

Syncs that choose a manifest themselves still replace or remove the
override as before: any sync without -l, and -l combined with -s, -t,
-m, --superproject-revision or --no-smart-sync. Smart sync stays
non-sticky for regular syncs, and `repo sync -l --no-smart-sync` still
discards the override to check out ToT.

Bug: 569927750
Change-Id: Id06ff88e9f3183fae32e993e86630fc3e8de71a1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/640965
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-12 02:14:30 -07:00
Gavin Mak beda65323e sync: skip clone.bundle when repo.fetchcmd is configured
In Sync_NetworkHalf, clone_bundle defaults to True on initial sync when
self.objdir does not exist yet, and _ApplyCloneBundle is called before
custom_fetch_configured is checked.

For remotes with HTTP/HTTPS URLs that serve /clone.bundle, this causes
repo sync to download and unpack the full clone.bundle into .git/objects
before invoking repo.fetchcmd.

Skip _ApplyCloneBundle when custom_fetch_configured is enabled so
repo.fetchcmd exclusively handles object fetching.

Bug: 543101585
Change-Id: If41a775a75589ad5a23106163f9932e2940b72d2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641685
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-10-09 14:09:50 -07:00
Gavin Mak 02e07525f9 project: peel annotated tags in GetRevisionId with cached refs
When all_refs (e.g. self.bare_ref.all in Sync_LocalHalf) is passed to
GetRevisionId, looking up a tag ref in all_refs returns the tag object
SHA for annotated tags rather than the peeled commit SHA.

In Sync_LocalHalf, this causes repo.reprojectcmd to receive a tag SHA in
REPO_TREV and DetachHead to fail with 'trying to write non-commit object
to branch HEAD', and causes standard checkouts to re-checkout on every
subsequent sync because HEAD (the commit SHA) never equals revid (the
tag object SHA).

Skip the all_refs fast path for refs/tags/* so tags always fall through
to bare_git.ResolveCommit, which peels to the underlying commit SHA.

Bug: 543101585
Change-Id: Ie4afa3cee9149e2a8438bd8fb4a8026a210848d8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641683
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-10-09 14:09:44 -07:00
Gavin Mak 2e8a3805bb sync: don't run git ls-remote before fetchcmd on unpinned refs
When a project's revision is an unpinned branch or tag and superprojects
are not used (so self.revisionId is unset), _CustomFetch previously ran
git ls-remote against the Git remote before invoking repo.fetchcmd just
to pre-populate REPO_TREV.

This added a per-project network round-trip before fetchcmd and forced
branch resolution against the Git remote rather than letting fetchcmd
resolve REPO_RREV from its own server (just as standard git fetch does
in _RemoteFetch).

Only populate REPO_TREV before fetchcmd when the target commit SHA is
already known (via self.revisionId from superproject or a SHA-pinned
revisionExpr). When REPO_RREV is an unpinned branch or tag, leave
REPO_TREV empty, let fetchcmd resolve REPO_RREV and update the local
tracking ref and FETCH_HEAD, and verify afterward that the tracking ref
and FETCH_HEAD resolve to the same valid commit in the object store.

Bug: 543101585
Change-Id: I482bf4c022d0e669e5620d4517581ac193983da0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/640423
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-10-09 14:09:37 -07:00
Becky Siegel 277759b9e1 agents: enable generic multi-perspective code review agent
Configure git-repo-code-review-agent with use_default_subagents: true in
agents/agent_configs.txtpb so groupagent-grev-review uses the generic
gerrit_reviewer multi-perspective review framework.

Bug: 555706086
Change-Id: I079681b00e71cb886dc2343692d88b01d58a51ea
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641904
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Becky Siegel <beckysiegel@google.com>
2026-10-09 13:10:58 -07:00
Brian Gan 9593a76fd1 sync: keep --superproject-revision across manifest reloads
When --superproject-revision moves `.repo/manifests`, the manifest is
reloaded, discarding the pinned superproject. _UpdateProjectsRevisionId
then read gitlinks from the superproject's default branch instead,
pinning failed, and projects synced unpinned.

Apply the revision in _ConfigureSuperproject so it is re-applied after
every reload.

Verified with https://paste.googleplex.com/5889974172516352

Bug: 571245807
Change-Id: I4d782d229b128c0f15081d11b722d8901709458c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641743
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-09 12:56:07 -07:00
Brian Gan 9293cda144 sync: roll back repo when stable is rewound
The documented way to roll back a bad release is to force push an older
release to stable, and docs/release-process.md says clients will then
automatically downgrade. _PostRepoFetch already supports switching to an
older commit, but it only runs when `RepoProject.HasChanges` is true,
and that used the `MetaProject` check of whether the remote has commits
that HEAD lacks. After a rewind it doesn't, so clients that had already
upgraded stayed on the bad release.

Make `RepoProject.HasChanges` true whenever the remote revision differs
from HEAD, in either direction. `ManifestProject` keeps the forward-only
check so local manifest commits are still rebased rather than dropped.

Bug: 571204396
Change-Id: I248507f434d2552ce11a9afabe9a6335cb24ddbd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/640130
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-10-09 09:49:49 -07:00
Rahul Yadav 340b587e94 sync: Support SYNC_BRANCH to override the smart sync branch
`repo sync -s` always sends the manifest branch to the manifest server
as the `branch` argument of GetApprovedManifest. Some servers map one
manifest branch to several server-side (CI) branches, and the only way
to pick among them was indirectly through the target.

Honor a SYNC_BRANCH environment variable, mirroring SYNC_TARGET: if set
and non-empty, send it instead of the manifest branch. An empty value
is treated as unset. -t/--smart-tag is unaffected.

Bug: 568962967
Test: ./run_tests
Change-Id: I26700e8997c21487f83011258100fd91a5c87508
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/639123
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
2026-10-09 07:00:47 -07:00
Rahul Yadav 0a4e68f9d2 sync: don't let --nmu affect smart sync
--no-manifest-update has always meant "use the existing manifest
checkout as-is": it skips updating .repo/manifests and nothing else.
Smart sync pins do not come from that checkout but from the manifest
server, and an explicit `-s --nmu` already works that way.

The manifest-driven default smart sync (sync-smartsync="true") instead
treated --nmu like -l: it disabled the implicit smart sync and kept an
existing smart_sync_override.xml. So on such manifests `repo sync --nmu`
fetched and checked out the previous pins (or ToT if there were none)
instead of doing what `repo sync` does minus the manifest checkout
update.

Make --nmu orthogonal to smart sync again: it no longer disables the
default smart sync, and it no longer keeps the override. Both have to
change together; otherwise a failed implicit smart sync under --nmu
would fall back to ToT while leaving the override on disk. Only -l,
which skips fetching, keeps the override.

Bug: 569927750
Change-Id: I33730363df485d72cd86ae99bf5eb8399b58e81a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/641264
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
2026-10-09 07:00:22 -07:00
Mike Frysinger 7bba4ee47e vpython: update to uv spec format
Change-Id: I18dca6f8c9c1f9234cfee640f780f1f2e4462ab8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/635623
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-10-05 10:40:31 -07:00
Rahul Yadav d53a4278f5 sync: Support manifest-driven default smart sync
Add support for enabling smart sync by default via the
sync-smartsync="true" attribute on the manifest <default> element.

When sync-smartsync is set to true in the manifest defaults:
- Plain `repo sync` triggers Smart Sync automatically.
- If that implicit Smart Sync fails (manifest server unreachable, or
  the returned manifest cannot be parsed), repo silently falls back to
  a regular ToT sync. The reason is logged with --verbose.
- `--no-smart-sync` CLI flag overrides the manifest default and
  syncs to ToT.
- Passing explicit `-s` / `--smart-sync` forces Smart Sync as before,
  and its failures are still fatal.
- `-t`, `--smart-tag`, `-m`, `--manifest-name`, and
  `--superproject-revision` disable the default smart sync.
- `-l` / `--local-only` and `--no-manifest-update` skip the manifest
  server and reuse the existing smart_sync_override.xml, if any.

Update docs, completion, regenerate man pages, and add tests for XML
parsing, serialization, CLI option resolution, and the fallback and
reuse paths.

Bug: 545137090
Change-Id: If75569c58dd525d170e9801e70ccbf02206a52f2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/617641
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-10-01 03:13:33 -07:00
Victor Pushkarev 5d4149f32e gc: add --keep-manifest-projects
By default, `repo gc` treats projects excluded by the configured
manifest groups as unused. This matches the active project set, but it
can remove cached Git data for projects that might be used again after
changing groups.

Add `--keep-manifest-projects` to keep Git data for projects that are
still in the manifests, even if the current groups exclude them or their
checkout is missing. Git data for projects no longer in the manifests
can still be removed.

Change-Id: I63ae18ce1c6acc2463be86f81921476b12fb27d7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/629741
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-30 01:11:40 -07:00
Ram Peri cf499e312a sync: record superproject sync analysis state again
Since the multi-manifest change (bdcba7d), _UpdateProjectsRevisionId
rebinds its superproject_logging_data parameter to a new local dict.
The caller keeps its own empty dict and passes it to
UpdateSyncAnalysisState, so repo.syncstate.superproject.* is never
written to the manifest config or logged to trace2.

Drop the rebinding so the caller's dict is populated.

Also record superproject, haslocalmanifests and hassuperprojecttag
when no manifest declares a superproject. Before bdcba7d these keys
were logged on every sync. SyncAnalysisState does not remove old
keys, so without this a checkout that moves to a manifest without a
superproject keeps reporting the values from its last superproject
sync.

Bug: 565957530
Test: ./run_tests
Change-Id: I8796e3a34b9cf4786383d25ecc3ebf0c5409caa3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634843
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Tested-by: Ram Peri <ramperi@google.com>
2026-09-28 15:34:54 -07:00
Mike Frysinger cc0546bc71 isort: print diff on failures
This should help users figure out how to fix problems.

Change-Id: I6346e163d06f9a0e52147689c5b329fe7f1366d2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/635343
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-09-28 10:55:16 -07:00
Victor Pushkarev ccf9ce39b8 sync: keep local sync state paths relative to repo root
`LocalSyncState` uses `Project.relpath` to identify projects, but that
path is relative to each project's own manifest. Projects in different
manifests can have the same relpath, so their fetch and checkout
timestamps overwrite each other. This can make a later partial sync look
like a full sync.

Pruning has a related problem. It treats the stored paths as relative to
the current manifest's topdir, so when sync starts from the outermost
manifest it can look for submanifest projects in the wrong place and
remove valid state.

Use `Project.RelPath(local=False)` to identify projects in local sync
state and check stored paths from the repo root when pruning.

Add tests for same-relpath projects, partial-sync detection, and pruning
when sync starts from either the outermost manifest or a submanifest.

Change-Id: I24485c315f1277a14913443674245cedd20820fe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/632421
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Victor Pushkarev <corvinus.v@gmail.com>
Commit-Queue: Victor Pushkarev <corvinus.v@gmail.com>
2026-09-26 01:17:45 -07:00
Ram Peri 9066409c2d init: don't suggest --no-use-superproject on failure
When the superproject can't be fetched, repo init warns and tells users
to re-run repo init with --no-use-superproject. That choice is saved in
the checkout config (repo.superproject), so following it disables the
superproject for that checkout until it is re-inited, even though these
failures are often transient.

Drop the recommendation and keep the rest of the warning.

This is the repo init counterpart of reverting
https://gerrit-review.googlesource.com/c/git-repo/+/305622 for repo sync
(https://gerrit-review.googlesource.com/c/git-repo/+/634804).

Bug: 565845573
Change-Id: I91a6e682ef1d1c6dee07bc7969de88a681e1d239
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634784
Tested-by: Ram Peri <ramperi@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
2026-09-24 13:54:06 -07:00
Ram Peri 2908f596c5 Revert "sync: Recommend using --no-use-superproject if sync fails."
This reverts commit f907ced0fe.
Original change: https://gerrit-review.googlesource.com/c/git-repo/+/305622

Reason for revert: the hint was added when a superproject failure made
repo sync exit with an error. Since 8db30d6 the failure is non-fatal
and sync falls back to fetching normally, and these failures are often
transient (e.g. on 2026-09-23 a manifest change added
external/slackhq/compose-lints before the superproject had a commit
for it, until the change was reverted ~2h later). Following the hint
means users keep passing --no-use-superproject and bypass the
superproject even after the failure is gone.

The revert does not apply cleanly after the black reformat, so the
conflict was resolved by restoring the original message text.

Bug: 565845573
Change-Id: I5efadedd4d3d0002df50493fa29554ba9626fe82
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634804
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Tested-by: Ram Peri <ramperi@google.com>
2026-09-24 13:53:54 -07:00
Gavin Mak e1e215a14e project: don't plan a stateless prune without a worktree
Skip stateless prune planning when a project's worktree directory
does not exist, preventing fetch failures on subsequent syncs.

Also run _LsRemote() bare so it does not depend on worktree
existence.

Bug: 565047698
Change-Id: I385cba4e06813d9113ec7ddc963e8499acffaec5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634426
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-09-23 13:27:06 -07:00
Gavin Mak 175198b0aa sync: skip the bloat check for projects without a worktree
Skip shallow clone bloat checks during --network-only syncs and
filter out projects whose worktree directory does not exist on disk.

Also return None quietly from _GetStatusSnapshot() when a project
lacks an existing worktree directory.

Bug: 565047698
Change-Id: Ia4c7db36a2bd78bc89e7b72f7a492b901b50efa7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634425
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-23 13:27:01 -07:00
Gavin Mak 310c99571f project: don't rerun git status after a failed snapshot
When a status snapshot fails, fall back directly to _IsDirtyLegacy()
instead of IsDirty() to avoid running git status twice.

Expose Project.GetDirtyAndHead() so sync bloat checks can inspect
worktree state without invoking private Project methods.

Bug: 565047698
Change-Id: I6432ed82da88778eec9d151fd5b70bedb1efc62b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/634424
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-09-23 13:26:55 -07:00
Fredrik de Groot e7d4d97ffc Fix fail in test_repo_trace if trace is disabled
If env has REPO_TRACE 0, then the test produces:
FileNotFoundError: No such file or directory: TRACE_FILE_from_test'
because it assumes trace is always enabled.

Improve by enabling trace for this test class.

Also modify the repo_trace.SetTrace API to take an optional
True/False if needed, still defaults to True though.

Change-Id: I2bb4632eb45ee529fc1785833e0bafd9c36f0875
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/445541
Commit-Queue: Fredrik de Groot <fredrik.de.groot@haleytek.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Fredrik de Groot <fredrik.de.groot@haleytek.com>
2026-09-22 23:43:15 -07:00
89 changed files with 8051 additions and 1242 deletions

No files matched your search

+13
View File
@@ -24,3 +24,16 @@ configs {
}
automatic: false
}
# Default Multi-Perspective Code Review Agent
configs {
id: "git-repo-code-review-agent"
display_name: "Code Review Agent"
description: "Default multi-perspective review agent for git-repo."
include_filters {
project: "git-repo"
path_regex: ".*"
}
use_default_subagents: true
tags: "code-review-agent"
}
+2 -2
View File
@@ -2004,10 +2004,10 @@ def _Run(self, *cmd):
**Do:**
```python
from git_command import GitCommand
import git_command
def _Run(self, cmd):
p = GitCommand(
p = git_command.GitCommand(
None,
cmd,
capture_stdout=True,
+5 -7
View File
@@ -19,9 +19,7 @@ import os
import re
from typing import List, Optional, TYPE_CHECKING
from error import InvalidProjectGroupsError
from error import NoSuchProjectError
from error import RepoExitError
import error
from event_log import EventLog
import progress
@@ -48,7 +46,7 @@ WORKER_BATCH_SIZE = 32
DEFAULT_LOCAL_JOBS = min(os.cpu_count(), 8)
class UsageError(RepoExitError):
class UsageError(error.RepoExitError):
"""Exception thrown with invalid command usage."""
@@ -526,16 +524,16 @@ class Command:
projects = [project]
if not projects:
raise NoSuchProjectError(arg)
raise error.NoSuchProjectError(arg)
for project in projects:
if not missing_ok and not project.Exists:
raise NoSuchProjectError(
raise error.NoSuchProjectError(
"%s (%s)"
% (arg, project.RelPath(local=not all_manifests))
)
if not matches_groups(project):
raise InvalidProjectGroupsError(arg)
raise error.InvalidProjectGroupsError(arg)
result.extend(projects)
+15
View File
@@ -24,6 +24,7 @@ _repo() {
'branches:View current topic branches'
'checkout:Checkout a branch'
'cherry-pick:Cherry-pick a change'
'config:Read and write per-user repo settings'
'diff:Show changes between commit and working tree'
'diffmanifests:Show differences between two manifests'
'download:Download and find a change'
@@ -117,6 +118,15 @@ _repo() {
${common_opts} \
'1: :_message "sha1"'
;;
config)
_arguments \
${common_opts} \
'--type=[Validate and canonicalize the value as this type]:type:(bool int)' \
'(-z --null)'{-z,--null}'[Terminate output with NUL]' \
'1: :(get set unset list)' \
'2: :_message "name"' \
'3: :_message "value"'
;;
diff)
_arguments \
${common_opts} \
@@ -236,6 +246,9 @@ _repo() {
'--clone-filter=[Clone filter]:filter:' \
'--use-superproject[Use superproject]' \
'--no-use-superproject[Do not use superproject]' \
'(-s --smart-sync)'{-s,--smart-sync}'[Default to smart sync]' \
'--no-smart-sync[Do not default to smart sync]' \
'--head[Do not default to smart sync]' \
'--clone-bundle[Use clone bundle]' \
'--no-clone-bundle[Do not use clone bundle]' \
'--git-lfs[Use git lfs]' \
@@ -337,6 +350,8 @@ _repo() {
'--auto-gc[Run auto gc]' \
'--no-auto-gc[Do not run auto gc]' \
'(-s --smart-sync)'{-s,--smart-sync}'[Smart sync]' \
'--no-smart-sync[Do not smart sync]' \
'--head[Do not smart sync]' \
'(-t --smart-tag)'{-t,--smart-tag=}'[Smart tag]:tag:' \
'--no-repo-verify[Do not verify repo]' \
'--no-verify[Do not verify]' \
+18 -14
View File
@@ -28,7 +28,10 @@ Environment section in `repo help forall` or `subcmds/forall.py`.
The following environment variable is specific to `repo.fetchcmd`:
* `REPO_TREV`: The target revision resolved to a full commit hash.
* `REPO_TREV`: The target commit hash when the revision is already pinned to
a commit (e.g. via a superproject or a commit SHA in the manifest), or
empty when `REPO_RREV` is an unpinned branch or tag to be resolved by the
fetch command.
## Contract
@@ -37,22 +40,23 @@ The following environment variable is specific to `repo.fetchcmd`:
After the fetch command exits with status 0, `repo` expects the following
postconditions to be met:
1. `git cat-file -e REPO_TREV` succeeds (the commit must exist in the object
store).
2. The mapped local tracking ref (e.g. `refs/remotes/REPO_REMOTE/<branch>`
for a branch revision, or the tag ref itself for a tag) must point to
`REPO_TREV`.
3. `FETCH_HEAD` must point to `REPO_TREV`.
4. The commit graph from `REPO_TREV` must be reachable far enough to compute
merge bases with local branches.
1. When the manifest revision names a branch or tag, the mapped local tracking
ref (e.g. `refs/remotes/REPO_REMOTE/<branch>` for a branch, or
`refs/tags/<tag>` for a tag) must resolve to a valid commit in the object
store (`git rev-parse --verify <ref>^{commit}`), and must equal `REPO_TREV`
if `REPO_TREV` was non-empty.
2. `FETCH_HEAD` must resolve to the same target commit in the object store
(`git rev-parse --verify FETCH_HEAD^{commit}`).
3. The commit graph from the target commit must be reachable far enough to
compute merge bases with local branches.
### Invariants
* The command should be idempotent; fetching the same `REPO_TREV` twice should
be a no-op.
* Only `FETCH_HEAD` and `refs/remotes/*` should be modified to preserve
`repo sync --network-only` semantics. `HEAD` and local branches must not be
touched by the fetch command.
* The command should be idempotent; repeating the same fetch should be a
no-op.
* Only `FETCH_HEAD`, `refs/remotes/*`, and `refs/tags/*` should be modified to
preserve `repo sync --network-only` semantics. `HEAD` and local branches
must not be touched by the fetch command.
* Dirty worktree state must be preserved.
* The command is **not** executed for `MetaProject`s (i.e. the internal `repo`
repository itself at `.repo/repo` and the `manifests` repository at
+3
View File
@@ -176,6 +176,7 @@ User controlled settings are initialized when running `repo init`.
| repo.partialclone | `--partial-clone` | Create [partial git clones] |
| repo.partialcloneexclude | `--partial-clone-exclude` | Comma-delimited list of project names (not paths) to exclude while using [partial git clones] |
| repo.reference | `--reference` | Reference repo client checkout |
| repo.smartsync | `--smart-sync` | Default to [Smart Sync](./smart-sync.md) when syncing |
| repo.submodules | `--submodules` | Sync git submodules |
| repo.superproject | `--use-superproject` | Sync [superproject] |
| repo.worktree | `--worktree` | Use [git worktree] for checkouts |
@@ -253,6 +254,8 @@ setting `REPO_CONFIG_DIR`.
* `.repoconfig/`: Repo's per-user directory for all random config files/state.
* `.repoconfig/config`: Per-user settings using [git-config] file format.
Managed with `repo config`; see [repoconfig.md](./repoconfig.md) for the
available settings.
* `.repoconfig/keyring-version`: Cache file for checking if the gnupg subdir
has all the same keys as the repo launcher. Used to avoid running gpg
constantly as that can be quite slow.
+13 -10
View File
@@ -46,18 +46,19 @@ following DTD:
<!ATTLIST remote revision CDATA #IMPLIED>
<!ELEMENT default EMPTY>
<!ATTLIST default remote IDREF #IMPLIED>
<!ATTLIST default revision CDATA #IMPLIED>
<!ATTLIST default dest-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync-j CDATA #IMPLIED>
<!ATTLIST default sync-j-max CDATA #IMPLIED>
<!ATTLIST default sync-c CDATA #IMPLIED>
<!ATTLIST default sync-s CDATA #IMPLIED>
<!ATTLIST default sync-tags CDATA #IMPLIED>
<!ATTLIST default remote IDREF #IMPLIED>
<!ATTLIST default revision CDATA #IMPLIED>
<!ATTLIST default dest-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync-j CDATA #IMPLIED>
<!ATTLIST default sync-j-max CDATA #IMPLIED>
<!ATTLIST default sync-c CDATA #IMPLIED>
<!ATTLIST default sync-s CDATA #IMPLIED>
<!ATTLIST default sync-tags CDATA #IMPLIED>
<!ATTLIST default sync-smartsync CDATA #IMPLIED>
<!ELEMENT manifest-server EMPTY>
<!ATTLIST manifest-server url CDATA #REQUIRED>
<!ATTLIST manifest-server url CDATA #REQUIRED>
<!ATTLIST manifest-server helper CDATA #IMPLIED>
<!ELEMENT submanifest EMPTY>
@@ -231,6 +232,8 @@ Attribute `sync-tags`: Set to false to only sync the given Git
branch (specified in the `revision` attribute) rather than
the other ref tags.
Attribute `sync-smartsync`: Set to true to enable smart sync by default.
### Element manifest-server
+96
View File
@@ -0,0 +1,96 @@
# repoconfig: User Configuration
Repo keeps a small number of settings that apply to the current user rather
than to one checkout. They live in the per-user configuration file
`~/.repoconfig/config`, which is separate from both git's own `~/.gitconfig`
and the per-checkout settings stored in `.repo/manifests.git/config` (see
[internal-fs-layout.md](./internal-fs-layout.md#Global-settings) for those).
[TOC]
## Location
The file is `.repoconfig/config` inside the user's home directory. The parent
directory can be changed by setting the `REPO_CONFIG_DIR` environment variable,
in which case repo reads `$REPO_CONFIG_DIR/.repoconfig/config` instead.
The file is created on demand; it's fine for it to be missing.
## Format
The file uses the [git-config] syntax. The easiest way to manage it is the
`repo config` command:
```sh
# Set a value.
repo config set sync.smartsync true
# Read a value.
repo config get sync.smartsync
# Remove a value.
repo config unset sync.smartsync
# Show all settings.
repo config list
```
Like other repo subcommands, `repo config` must be run from inside a repo
checkout. The file can also be edited by hand:
```ini
[sync]
smartsync = true
```
Boolean settings accept `true`/`yes` and `false`/`no`.
## Settings
### alias.*
`alias.<name>` defines `repo <name>` as a shortcut for another subcommand,
optionally with arguments, in the same way as git aliases:
```ini
[alias]
ss = sync -s
st = status -o
```
Aliases defined in a checkout's `.repo/config` take precedence over per-user
ones. Existing subcommands cannot be overridden by an alias.
### sync
#### smartsync
`sync.smartsync` is a boolean that lets a user decide once, for all their
checkouts, whether `repo sync` should [Smart Sync](./smart-sync.md) by
default: `true` always Smart Syncs, as if `-s`/`--smart-sync` were passed;
`false` never does by default, even if the manifest sets
`sync-smartsync="true"`; unset follows the manifest.
The effective behavior is resolved in this order; the first match wins:
1. Command line: `-s`/`--smart-sync`, `--no-smart-sync`, or `-t`/`--smart-tag`.
2. User config: `sync.smartsync` in `~/.repoconfig/config`.
3. Manifest: `sync-smartsync="true"` on the `<default>` element.
4. Otherwise, a regular ToT sync.
So `sync.smartsync=false` turns off the manifest-driven default, but an
explicit `repo sync -s`, `repo sync -t TAG` or `repo smartsync` still works.
Likewise `sync.smartsync=true` is overridden by `--no-smart-sync`.
A Smart Sync enabled by `sync.smartsync=true` is treated as an explicit
request: it is subject to the same restrictions as `-s` (it is skipped when
`-l`/`--local-only`, `-m`/`--manifest-name`, `-t`/`--smart-tag` or
`--superproject-revision` is used), and a failure to obtain a manifest from
the server fails the sync instead of silently falling back to ToT.
`sync.smartsync` does not affect `-l`/`--local-only`: a local-only sync keeps
whatever `smart_sync_override.xml` the previous network sync left behind. See
[smart-sync.md](./smart-sync.md#Client-Options) for details.
[git-config]: https://git-scm.com/docs/git-config
+118 -4
View File
@@ -95,6 +95,36 @@ example would do. Google's internal server uses Python's
The manifest server should implement the following RPC methods.
### GetApprovedManifestV2
> `GetApprovedManifestV2(request: dict) -> dict`
Like [GetApprovedManifest](#GetApprovedManifest), but takes a single struct of
named options so new options can be added without changing the method signature.
Unset options are omitted from the struct. Servers should ignore options they do
not understand.
Request options:
* `branch` (`str`, required): Same as in `GetApprovedManifest`.
* `target` (`str`, optional): Same as in `GetApprovedManifest`.
* `best_effort` (`bool`, optional, default `False`): The client can proceed
without a manifest, so the server may decline the request (e.g. during a
staged rollout or to shed load). Repo sets this for Smart Syncs enabled by
the manifest's `sync-smartsync` default, and falls back to a regular ToT
sync if the request is declined. Servers must not decline requests without
this option for these reasons.
Response fields:
* `success` (`bool`): Whether a manifest was returned.
* `manifest_xml` (`str`): The manifest. Required if `success` is true.
* `error_message` (`str`): Why the request failed or was declined. Repo shows
it to the user like `GetApprovedManifest` errors.
If the server does not implement this method, repo calls `GetApprovedManifest`
instead and drops the options it cannot express.
### GetApprovedManifest
> `GetApprovedManifest(branch: str, target: Optional[str]) -> str`
@@ -138,12 +168,79 @@ copy into `.repo/manifests/smart_sync_override.xml` so users can examine it.
The next time `repo sync` is run, this file is automatically replaced or removed
based on the current set of options.
### --smart-sync
With `-l`/`--local-only` and no explicit `-s`/`--smart-sync` or
`-t`/`--smart-tag`, repo does not contact the manifest server. Instead, it keeps
using the existing `smart_sync_override.xml`, if there is one, whether it came
from an explicit `-s`/`--smart-sync` or from the manifest's `sync-smartsync`
default. This way, `repo sync -n` followed by `repo sync -l` ends up on the same
revisions as the equivalent single `repo sync` (e.g. `repo sync -s -n` +
`repo sync -l` matches `repo sync -s`). Pass `--no-smart-sync` to discard it and
use the default manifest instead.
Repo will call `GetApprovedManifest(branch[, target])`.
`--no-manifest-update` only skips updating the manifest checkout. It does not
affect Smart Sync: the default Smart Sync from `sync-smartsync` (see below)
still runs, and an existing `smart_sync_override.xml` is replaced or removed as
in a regular sync.
### `repo init --[no-]smart-sync`
Users can configure Smart Sync as the default for a checkout by passing
`-s`/`--smart-sync` (or `--no-smart-sync`/`--head`) to `repo init` (this can
also be re-run in an existing checkout to change the setting).
When resolving whether to perform a Smart Sync by default, `repo sync` uses the
following precedence order (highest to lowest):
1. Per-invocation CLI flags (`-s`/`--smart-sync`, `--no-smart-sync`/`--head`,
or target selection flags like `-t`, `-m`, `--superproject-revision`, `-l`).
2. Checkout `repo init` setting (`repo init --[no-]smart-sync`).
3. [Per-user `sync.smartsync` setting](#Per_user-default-sync_smartsync).
4. Manifest [`sync-smartsync`](#sync-smartsync-manifest-attribute) setting.
Running `repo init --no-smart-sync` (or `repo init --head`) disables default
Smart Sync for the checkout even when enabled by the user setting or manifest.
As with the [manifest setting](#sync-smartsync-manifest-attribute), a
checkout-enabled default Smart Sync falls back to a regular ToT sync if the
manifest does not define a `<manifest-server>` or the manifest server cannot
provide a usable manifest.
### sync-smartsync (Manifest Attribute)
The manifest can enable Smart Sync by default for all `repo sync` invocations
by setting `sync-smartsync="true"` on the `<default>` element:
```xml
<default sync-smartsync="true" ... />
```
When enabled in the manifest, `repo sync` will automatically perform a Smart
Sync unless explicitly overridden on the command line. Since the user did not
ask for a Smart Sync, any failure to obtain a manifest from the manifest server
(e.g. the server is unreachable, or it returns a manifest that cannot be parsed)
is silently ignored and `repo sync` falls back to a regular ToT sync; pass
`--verbose` to see the reason. Repo marks these requests as `best_effort` (see
[GetApprovedManifestV2](#GetApprovedManifestV2)), so the server may also decline
them, e.g. during a staged rollout. This is only a fallback for the
manifest-driven default: an explicit `-s`/`--smart-sync` or `-t`/`--smart-tag`
still fails the sync, as before. Options that specify an explicit target or
manifest source (such as `-t`/`--smart-tag`, `-m`/`--manifest-name` or
`--superproject-revision`) will also disable the default Smart Sync behavior.
Users can override this manifest default on their workstation with the
`sync.smartsync` user setting; see
[Per-user default](#Per_user-default-sync_smartsync) below.
### --smart-sync / -s
Explicitly enables Smart Sync. Repo will call
`GetApprovedManifestV2({branch[, target]})`, or
`GetApprovedManifest(branch[, target])` if the server does not implement it.
The `branch` is determined by the current manifest branch as specified by
`--manifest-branch=BRANCH` when running `repo init`.
`--manifest-branch=BRANCH` when running `repo init`. If `${SYNC_BRANCH}` is
defined and non-empty, its value is sent instead. This allows selecting a
branch other than the one the checkout tracks, e.g. a server-side (CI) branch
name that builds from the same manifest branch.
The `target` is defined by environment variables in the order below. If none
of them match, then `target` is omitted. These variables were decided as they
@@ -155,6 +252,23 @@ match the settings Android build environments automatically setup.
3. `${TARGET_PRODUCT}-${TARGET_BUILD_VARIANT}`: If these variables are all
defined, then they are merged with `-` and used.
### --smart-tag=TAG
### --no-smart-sync / --head
Explicitly disables Smart Sync, overriding any `repo.smartsync` git config or
`sync-smartsync="true"` setting declared in the manifest `<default>` element.
### Per-user default (sync.smartsync)
A user can decide once, for all their checkouts, whether `repo sync` should
Smart Sync by default by setting the boolean `sync.smartsync` key in their
[per-user repo config](./repoconfig.md#smartsync).
Command line flags (`-s`, `--no-smart-sync`, `-t`) override the user setting,
which in turn overrides the manifest's `sync-smartsync` default. A Smart Sync
enabled this way is treated as an explicit request, so it is subject to the
same restrictions as `-s` and fails the sync if no manifest can be obtained
from the server instead of silently falling back to ToT.
### --smart-tag=TAG / -t
Repo will call `GetManifest(TAG)`.
+5 -3
View File
@@ -18,7 +18,7 @@ import subprocess
import sys
import tempfile
from error import EditorError
import error
import platform_utils
@@ -104,9 +104,11 @@ least one of these before using this command.""", # noqa: E501
try:
rc = subprocess.Popen(args, shell=shell).wait()
except OSError as e:
raise EditorError(f"editor failed, {str(e)}: {editor} {path}")
raise error.EditorError(
f"editor failed, {str(e)}: {editor} {path}"
)
if rc != 0:
raise EditorError(
raise error.EditorError(
"editor failed with exit status %d: %s %s"
% (rc, editor, path)
)
+2 -2
View File
@@ -19,10 +19,10 @@ import sys
from urllib.parse import urlparse
from urllib.request import urlopen
from error import RepoExitError
import error
class FetchFileError(RepoExitError):
class FetchFileError(error.RepoExitError):
"""Exit error when fetch_file fails."""
+20 -17
View File
@@ -20,14 +20,11 @@ import subprocess
import sys
from typing import Any, Optional
from error import GitError
from error import RepoExitError
import error
from git_trace2_event_log_base import BaseEventLog
import platform_utils
from repo_logging import RepoLogger
from repo_trace import IsTrace
from repo_trace import REPO_TRACE
from repo_trace import Trace
import repo_logging
import repo_trace
from wrapper import Wrapper
@@ -43,7 +40,7 @@ GIT_ERROR_STDOUT_LINES = 1
GIT_ERROR_STDERR_LINES = 10
INVALID_GIT_EXIT_CODE = 126
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class _GitCall:
@@ -335,11 +332,17 @@ class GitCommand:
if command_name in ("fetch", "clone"):
env["GIT_TERMINAL_PROMPT"] = "0"
# Need to use the --progress flag for fetch/clone so output will be
# displayed as by default git only does progress output if stderr is
# a TTY.
if command_name in ("fetch", "clone", "push"):
# Need to use the --progress flag for fetch/clone/push so output
# will be displayed as by default git only does progress output if
# stderr is a TTY.
if sys.stderr.isatty():
if "--progress" not in cmdv and "--quiet" not in cmdv:
if (
"--progress" not in cmdv
and "--no-progress" not in cmdv
and "--quiet" not in cmdv
and "-q" not in cmdv
):
command.append("--progress")
command.extend(cmdv[1:])
@@ -433,7 +436,7 @@ class GitCommand:
kwargs = {}
dbg = ""
if IsTrace():
if repo_trace.IsTrace():
global LAST_CWD
global LAST_GITDIR
@@ -470,7 +473,7 @@ class GitCommand:
elif stderr == subprocess.STDOUT:
dbg += " 2>&1"
with Trace(
with repo_trace.Trace(
"git command %s %s with debug: %s", LAST_GITDIR, command, dbg
):
try:
@@ -547,7 +550,7 @@ class GitCommand:
"""
env = os.environ.copy()
for key in (
REPO_TRACE,
repo_trace.REPO_TRACE,
GIT_DIR,
"GIT_ALTERNATE_OBJECT_DIRECTORIES",
"GIT_OBJECT_DIRECTORY",
@@ -604,14 +607,14 @@ class GitCommand:
return self.rc
class GitRequireError(RepoExitError):
class GitRequireError(error.RepoExitError):
"""Error raised when git version is unavailable or invalid."""
def __init__(self, message, exit_code: int = INVALID_GIT_EXIT_CODE):
super().__init__(message, exit_code=exit_code)
class GitCommandError(GitError):
class GitCommandError(error.GitError):
"""
Error raised from a failed git command.
Note that GitError can refer to any Git related error (e.g. branch not
@@ -691,7 +694,7 @@ class GitCommandError(GitError):
return string
class GitPopenCommandError(GitError):
class GitPopenCommandError(error.GitError):
"""
Error raised when subprocess.Popen fails for a GitCommand
"""
+97 -50
View File
@@ -26,14 +26,11 @@ from typing import Union
import urllib.error
import urllib.request
from error import GitError
from error import UploadError
from git_command import GitCommand
from git_refs import R_CHANGES
from git_refs import R_HEADS
from git_refs import R_TAGS
import error
import git_command
import git_refs
import platform_utils
from repo_trace import Trace
import repo_trace
# Prefix that is prepended to all the keys of SyncAnalysisState's data
@@ -46,7 +43,7 @@ REVIEW_CACHE = {}
def IsChange(rev):
return rev.startswith(R_CHANGES)
return rev.startswith(git_refs.R_CHANGES)
def IsId(rev: str) -> bool:
@@ -54,13 +51,63 @@ def IsId(rev: str) -> bool:
def IsTag(rev):
return rev.startswith(R_TAGS)
return rev.startswith(git_refs.R_TAGS)
def IsImmutable(rev):
return IsChange(rev) or IsId(rev) or IsTag(rev)
def ParseBoolean(value: str) -> Union[bool, None]:
"""Parses a git config boolean value.
Accepts the same spellings as `git config --type=bool`.
https://git-scm.com/docs/git-config#Documentation/git-config.txt-boolean
Returns:
True/False, or None if |value| is not a valid boolean.
"""
v = value.strip().lower()
if v in ("true", "yes", "on", "1"):
return True
if v in ("false", "no", "off", "0"):
return False
return None
def ParseInt(value: str) -> Union[int, None]:
"""Parses a git config integer value.
Accepts the same syntax as `git config --type=int`: an optional k, m or
g suffix (multiples of 1024) and hex with a 0x prefix.
https://git-scm.com/docs/git-config#Documentation/git-config.txt-integer
Returns:
The integer, or None if |value| is not a valid integer.
"""
v = value.strip().lower()
mult = 1
if v.endswith("k"):
v = v[:-1]
mult = 1024
elif v.endswith("m"):
v = v[:-1]
mult = 1024 * 1024
elif v.endswith("g"):
v = v[:-1]
mult = 1024 * 1024 * 1024
base = 10
if v.startswith("0x"):
base = 16
try:
return int(v, base=base) * mult
except ValueError:
return None
def _key(name):
parts = name.split(".")
if len(parts) < 2:
@@ -152,32 +199,15 @@ class GitConfig:
v = self.GetString(name)
if v is None:
return None
v = v.strip()
mult = 1
if v.endswith("k"):
v = v[:-1]
mult = 1024
elif v.endswith("m"):
v = v[:-1]
mult = 1024 * 1024
elif v.endswith("g"):
v = v[:-1]
mult = 1024 * 1024 * 1024
base = 10
if v.startswith("0x"):
base = 16
try:
return int(v, base=base) * mult
except ValueError:
ret = ParseInt(v)
if ret is None:
print(
f"warning: expected {name} to represent an integer, got {v} "
"instead",
file=sys.stderr,
)
return None
return ret
def DumpConfigDict(self):
"""Returns the current configuration dict.
@@ -197,24 +227,25 @@ class GitConfig:
def GetBoolean(self, name: str) -> Union[bool, None]:
"""Returns a boolean from the configuration file.
This follows the git config syntax.
Returns:
None: The value was not defined, or is not a boolean.
True: The value was set to true or yes.
False: The value was set to false or no.
True: The value was set to true, yes, on or 1.
False: The value was set to false, no, off or 0.
"""
v = self.GetString(name)
if v is None:
return None
v = v.lower()
if v in ("true", "yes"):
return True
if v in ("false", "no"):
return False
print(
f"warning: expected {name} to represent a boolean, got {v} instead",
file=sys.stderr,
)
return None
ret = ParseBoolean(v)
if ret is None:
print(
f"warning: expected {name} to represent a boolean, got {v} "
"instead",
file=sys.stderr,
)
return ret
def SetBoolean(self, name, value):
"""Set the truthy value for a key."""
@@ -266,6 +297,13 @@ class GitConfig:
except KeyError:
old = []
if self.file != self._SYSTEM_CONFIG:
# `git config --file` does not create missing parent directories
# (e.g. ~/.repoconfig/ on first use).
parent_dir = os.path.dirname(self.file)
if parent_dir:
os.makedirs(parent_dir, exist_ok=True)
if value is None:
if old:
del self._cache[key]
@@ -387,7 +425,7 @@ class GitConfig:
except OSError:
return None
try:
with Trace(": parsing %s", self.file):
with repo_trace.Trace(": parsing %s", self.file):
with open(self._json) as fd:
return json.load(fd)
except (OSError, ValueError):
@@ -434,16 +472,23 @@ class GitConfig:
command = ["config", "--file", self.file, "--includes"]
command.extend(args)
p = GitCommand(None, command, capture_stdout=True, capture_stderr=True)
p = git_command.GitCommand(
None, command, capture_stdout=True, capture_stderr=True
)
if p.Wait() == 0:
return p.stdout
else:
raise GitError(f"git {' '.join(command)}: {p.stderr}")
raise error.GitError(f"git {' '.join(command)}: {p.stderr}")
class RepoConfig(GitConfig):
"""User settings for repo itself."""
# Keep separate caches from GitConfig, otherwise ForUser()/ForSystem()
# would return the already-cached ~/.gitconfig instance of the parent.
_ForUser = None
_ForSystem = None
@staticmethod
def _getUserConfig():
repo_config_dir = os.getenv("REPO_CONFIG_DIR", os.path.expanduser("~"))
@@ -671,11 +716,13 @@ class Remote:
userEmail, host, port
)
except urllib.error.HTTPError as e:
raise UploadError(f"{self.review}: {str(e)}")
raise error.UploadError(f"{self.review}: {str(e)}")
except urllib.error.URLError as e:
raise UploadError(f"{self.review}: {str(e)}")
raise error.UploadError(f"{self.review}: {str(e)}")
except http.client.HTTPException as e:
raise UploadError(f"{self.review}: {e.__class__.__name__}")
raise error.UploadError(
f"{self.review}: {e.__class__.__name__}"
)
REVIEW_CACHE[u] = self._review_url
return self._review_url + self.projectname
@@ -692,16 +739,16 @@ class Remote:
return rev
if not rev.startswith("refs/"):
rev = R_HEADS + rev
rev = git_refs.R_HEADS + rev
for spec in self.fetch:
if spec.SourceMatches(rev):
return spec.MapSource(rev)
if not rev.startswith(R_HEADS):
if not rev.startswith(git_refs.R_HEADS):
return rev
raise GitError(
raise error.GitError(
"%s: remote %s does not have %s"
% (self.projectname, self.name, rev)
)
+8 -9
View File
@@ -14,10 +14,9 @@
import os
from git_command import git_require
from git_command import GitCommand
import git_command
import platform_utils
from repo_trace import Trace
import repo_trace
HEAD = "HEAD"
@@ -82,7 +81,7 @@ class GitRefs:
self._LoadAll()
def _NeedUpdate(self):
with Trace(": scan refs %s", self._gitdir):
with repo_trace.Trace(": scan refs %s", self._gitdir):
for name, mtime in self._mtime.items():
try:
if mtime != os.path.getmtime(
@@ -94,7 +93,7 @@ class GitRefs:
return False
def _LoadAll(self):
with Trace(": load refs %s", self._gitdir):
with repo_trace.Trace(": load refs %s", self._gitdir):
self._phyref = {}
self._symref = {}
self._mtime = {}
@@ -135,7 +134,7 @@ class GitRefs:
Returns:
Whether root refs, including HEAD when it exists, were loaded.
"""
include_root_refs = git_require((2, 45, 0))
include_root_refs = git_command.git_require((2, 45, 0))
cmd = [
"for-each-ref",
"--format=%(objectname)%00%(refname)%00%(symref)",
@@ -145,7 +144,7 @@ class GitRefs:
# Avoid caching volatile root refs such as ORIG_HEAD. HEAD and
# refs/* are the only namespaces GitRefs exposes to callers.
cmd.extend([HEAD, "refs"])
p = GitCommand(
p = git_command.GitCommand(
None,
cmd,
capture_stdout=True,
@@ -166,7 +165,7 @@ class GitRefs:
def _ReadSymbolicRef(self, name: str) -> None:
"""Read a symbolic reference."""
p = GitCommand(
p = git_command.GitCommand(
None,
["symbolic-ref", "-q", name],
capture_stdout=True,
@@ -180,7 +179,7 @@ class GitRefs:
self._symref[name] = ref
return
p = GitCommand(
p = git_command.GitCommand(
None,
["rev-parse", "--verify", "-q", name],
capture_stdout=True,
+6 -7
View File
@@ -18,8 +18,7 @@ from collections import OrderedDict
import os
from typing import Iterator, List, Optional, TYPE_CHECKING
from git_command import git_require
from git_command import GitCommand
import git_command
if TYPE_CHECKING:
@@ -80,7 +79,7 @@ def GetStatus(
show_stash: bool = False,
) -> StatusSnapshot:
"""Return one machine-readable status snapshot for |project|."""
if not git_require((2, 11, 0)):
if not git_command.git_require((2, 11, 0)):
raise UnsupportedStatusError("porcelain v2 requires Git 2.11")
cmd = [
"status",
@@ -91,18 +90,18 @@ def GetStatus(
]
if branch:
cmd.append("--branch")
if git_require((2, 17, 0)):
if git_command.git_require((2, 17, 0)):
cmd.append(
"--ahead-behind" if ahead_behind else "--no-ahead-behind"
)
if git_require((2, 18, 0)):
if git_command.git_require((2, 18, 0)):
# Match the existing staged diff's explicit rename detection even if
# status.renames is disabled in the user's config.
cmd.append("--renames")
if show_stash and git_require((2, 35, 0)):
if show_stash and git_command.git_require((2, 35, 0)):
cmd.append("--show-stash")
p = GitCommand(
p = git_command.GitCommand(
project,
cmd,
bare=False,
+7 -8
View File
@@ -32,8 +32,7 @@ import time
from typing import NamedTuple
import urllib.parse
from git_command import git_require
from git_command import GitCommand
import git_command
from git_config import IsId
from git_config import RepoConfig
import platform_utils
@@ -140,7 +139,7 @@ class Superproject:
def commit_id(self):
"""Returns the commit ID of the superproject checkout."""
cmd = ["rev-parse", self.revision]
p = GitCommand(
p = git_command.GitCommand(
None, # project
cmd,
gitdir=self._work_git,
@@ -242,7 +241,7 @@ class Superproject:
try:
cmd = ["init", "--bare", tmp_git_name]
p = GitCommand(
p = git_command.GitCommand(
None,
cmd,
cwd=self._superproject_path,
@@ -292,7 +291,7 @@ class Superproject:
if not os.path.exists(self._work_git):
self._LogWarning("git fetch missing directory: {}", self._work_git)
return False
if not git_require((2, 28, 0)):
if not git_command.git_require((2, 28, 0)):
self._LogWarning(
"superproject requires a git version 2.28 or later"
)
@@ -329,7 +328,7 @@ class Superproject:
else f"{self.revision}:{self.revision}"
)
cmd.append(refspec)
p = GitCommand(
p = git_command.GitCommand(
None,
cmd,
gitdir=self._work_git,
@@ -351,7 +350,7 @@ class Superproject:
def _GetRef(self, ref: str) -> str:
"""Resolve one local ref without loading the entire ref namespace."""
p = GitCommand(
p = git_command.GitCommand(
None,
["rev-parse", "--verify", "--quiet", ref],
gitdir=self._work_git,
@@ -381,7 +380,7 @@ class Superproject:
branch = "HEAD" if not self.revision else self.revision
cmd = ["ls-tree", "-z", "-r", branch]
p = GitCommand(
p = git_command.GitCommand(
None,
cmd,
gitdir=self._work_git,
+5 -4
View File
@@ -14,8 +14,7 @@
"""Event logging in the git trace2 EVENT format."""
from git_command import GetEventTargetPath
from git_command import RepoSourceVersion
import git_command
from git_trace2_event_log_base import BaseEventLog
@@ -37,7 +36,9 @@ class EventLog(BaseEventLog):
"""
def __init__(self, **kwargs):
super().__init__(repo_source_version=RepoSourceVersion(), **kwargs)
super().__init__(
repo_source_version=git_command.RepoSourceVersion(), **kwargs
)
def Write(self, path=None, **kwargs):
if path is None:
@@ -45,4 +46,4 @@ class EventLog(BaseEventLog):
return super().Write(path=path, **kwargs)
def _GetEventTargetPath(self):
return GetEventTargetPath()
return git_command.GetEventTargetPath()
+12 -10
View File
@@ -19,8 +19,8 @@ import sys
import traceback
import urllib.parse
from error import HookError
from git_refs import HEAD
import error
import git_refs
# The API we've documented to hook authors. Keep in sync with repo-hooks.md.
@@ -142,7 +142,7 @@ class RepoHook:
# NOTE: Local (non-committed) changes will not be factored into this
# hash. I think this is OK, since we're really only worried about
# warning the user about upstream changes.
return self._hooks_project.work_git.rev_parse(HEAD)
return self._hooks_project.work_git.rev_parse(git_refs.HEAD)
def _GetMustVerb(self):
"""Return 'must' if the hook is required; 'should' if not."""
@@ -228,7 +228,7 @@ class RepoHook:
# For anything else, we'll assume no approval.
if self._abort_if_user_denies:
raise HookError(
raise error.HookError(
"You must allow the %s hook or use --no-verify."
% self._hook_type
)
@@ -326,14 +326,16 @@ class RepoHook:
try:
exec(compile(data, self._script_fullpath, "exec"), context)
except Exception:
raise HookError(
raise error.HookError(
"%s\nFailed to import %s hook; see traceback above."
% (traceback.format_exc(), self._hook_type)
)
# Running the script should have defined a main() function.
if "main" not in context:
raise HookError('Missing main() in: "%s"' % self._script_fullpath)
raise error.HookError(
'Missing main() in: "%s"' % self._script_fullpath
)
# Call the main function in the hook. If the hook should cause the
# build to fail, it will raise an Exception. We'll catch that convert
@@ -341,7 +343,7 @@ class RepoHook:
try:
context["main"](**kwargs)
except Exception:
raise HookError(
raise error.HookError(
"%s\nFailed to run main() for %s hook; see traceback "
"above." % (traceback.format_exc(), self._hook_type)
)
@@ -394,7 +396,7 @@ class RepoHook:
if interp:
prog = os.path.basename(interp)
if prog.startswith("python2"):
raise HookError("Python 2 is not supported")
raise error.HookError("Python 2 is not supported")
# Run the hook by importing directly.
self._ExecuteHookViaImport(data, context, **kwargs)
@@ -406,7 +408,7 @@ class RepoHook:
def _CheckHook(self):
# Bail with a nice error if we can't find the hook.
if not os.path.isfile(self._script_fullpath):
raise HookError(
raise error.HookError(
"Couldn't find repo hook: %s" % self._script_fullpath
)
@@ -473,7 +475,7 @@ class RepoHook:
% (self._hook_type, str(e)),
file=sys.stderr,
)
except HookError as e:
except error.HookError as e:
passed = False
print("ERROR: %s" % str(e), file=sys.stderr)
+30 -42
View File
@@ -33,7 +33,7 @@ import time
from typing import Optional
import urllib.request
from repo_logging import RepoLogger
import repo_logging
try:
@@ -45,34 +45,22 @@ from color import SetDefaultColoring
from command import InteractiveCommand
from command import MirrorSafeCommand
from editor import Editor
from error import DownloadError
from error import InvalidProjectGroupsError
from error import ManifestInvalidRevisionError
from error import ManifestParseError
from error import NoManifestException
from error import NoSuchProjectError
from error import RepoChangedException
from error import RepoError
from error import RepoExitError
from error import RepoUnhandledExceptionError
from error import SilentRepoExitError
import error
import event_log
from git_command import user_agent
import git_command
from git_config import RepoConfig
from git_trace2_event_log import EventLog
from manifest_xml import RepoClient
from pager import RunPager
from pager import TerminatePager
from repo_trace import SetTrace
from repo_trace import SetTraceToStderr
from repo_trace import Trace
import repo_trace
from subcmds import all_commands
from subcmds.version import Version
from wrapper import Wrapper
from wrapper import WrapperPath
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
# NB: These do not need to be kept in sync with the repo launcher script.
@@ -279,7 +267,7 @@ class _Repo:
run = (
lambda: self._RunLong(name, gopts, argv, git_trace2_event_log) or 0
)
with Trace(
with repo_trace.Trace(
"starting new command: %s [sid=%s]",
", ".join([name] + argv),
git_trace2_event_log.full_sid,
@@ -435,7 +423,7 @@ class _Repo:
try:
copts, cargs = cmd.OptionParser.parse_args(argv)
copts = cmd.ReadEnvironmentOptions(copts)
except NoManifestException as e:
except error.NoManifestException as e:
logger.error("error: in `%s`: %s", " ".join([name] + argv), e)
logger.error(
"error: manifest missing or unreadable -- please run init"
@@ -511,23 +499,23 @@ class _Repo:
KeyboardInterrupt,
SystemExit,
Exception,
RepoExitError,
error.RepoExitError,
) as e:
ok = isinstance(e, SystemExit) and not e.code
exception_name = type(e).__name__
if isinstance(e, RepoUnhandledExceptionError):
if isinstance(e, error.RepoUnhandledExceptionError):
exception_name = type(e.error).__name__
if isinstance(e, RepoExitError):
if isinstance(e, error.RepoExitError):
aggregated_errors = e.aggregate_errors or []
for error in aggregated_errors:
for err in aggregated_errors:
project = None
if isinstance(error, RepoError):
project = error.project
if isinstance(err, error.RepoError):
project = err.project
error_info = json.dumps(
{
"ErrorType": type(error).__name__,
"ErrorType": type(err).__name__,
"Project": str(project),
"Message": str(error),
"Message": str(err),
}
)
git_trace2_event_log.ErrorEvent(
@@ -542,24 +530,24 @@ class _Repo:
try:
execute_command()
except (
DownloadError,
ManifestInvalidRevisionError,
ManifestParseError,
NoManifestException,
error.DownloadError,
error.ManifestInvalidRevisionError,
error.ManifestParseError,
error.NoManifestException,
) as e:
logger.error("error: in `%s`: %s", " ".join([name] + argv), e)
if isinstance(e, NoManifestException):
if isinstance(e, error.NoManifestException):
logger.error(
"error: manifest missing or unreadable -- please run init"
)
result = e.exit_code
except NoSuchProjectError as e:
except error.NoSuchProjectError as e:
if e.name:
logger.error("error: project %s not found", e.name)
else:
logger.error("error: no project in current directory")
result = e.exit_code
except InvalidProjectGroupsError as e:
except error.InvalidProjectGroupsError as e:
if e.name:
logger.error(
"error: project group must be enabled for project %s",
@@ -578,7 +566,7 @@ class _Repo:
except KeyboardInterrupt:
result = KEYBOARD_INTERRUPT_EXIT
raise
except RepoExitError as e:
except error.RepoExitError as e:
result = e.exit_code
raise
except Exception:
@@ -712,11 +700,11 @@ def _PruneOptions(argv, opt):
class _UserAgentHandler(urllib.request.BaseHandler):
def http_request(self, req):
req.add_header("User-Agent", user_agent.repo)
req.add_header("User-Agent", git_command.user_agent.repo)
return req
def https_request(self, req):
req.add_header("User-Agent", user_agent.repo)
req.add_header("User-Agent", git_command.user_agent.repo)
return req
@@ -943,20 +931,20 @@ def _Main(argv):
name, gopts, argv = repo._ParseArgs(argv)
if gopts.trace:
SetTrace()
repo_trace.SetTrace()
if gopts.trace_to_stderr:
SetTraceToStderr()
repo_trace.SetTraceToStderr()
result = repo._Run(name, gopts, argv) or 0
except RepoExitError as e:
if not isinstance(e, SilentRepoExitError):
except error.RepoExitError as e:
if not isinstance(e, error.SilentRepoExitError):
logger.log_aggregated_errors(e)
result = e.exit_code
except KeyboardInterrupt:
print("aborted by user", file=sys.stderr)
result = KEYBOARD_INTERRUPT_EXIT
except RepoChangedException as rce:
except error.RepoChangedException as rce:
# If repo changed, re-exec ourselves.
argv = list(sys.argv)
argv.extend(rce.extra_args)
+60
View File
@@ -0,0 +1,60 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "October 2026" "repo config" "Repo Manual"
.SH NAME
repo \- repo config - manual page for repo config
.SH SYNOPSIS
.B repo
\fI\,config \/\fR[\fI\,--type=<type>\/\fR] [\fI\,-z\/\fR] \fI\,get <name>\/\fR
.SH DESCRIPTION
Summary
.PP
Read and write per\-user repo settings
.PP
repo config [\-\-type=<type>] set <name> <value>
repo config unset <name>
repo config [\-z] list
.SH OPTIONS
.TP
\fB\-h\fR, \fB\-\-help\fR
show this help message and exit
.TP
\fB\-\-type\fR=\fI\,TYPE\/\fR
validate and canonicalize the value as this type for get and set (bool, int)
.TP
\fB\-z\fR, \fB\-\-null\fR
terminate output of get and list with NUL instead of newline, and separate key and value with a newline
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
show all output
.TP
\fB\-q\fR, \fB\-\-quiet\fR
only show errors
.SS Multi\-manifest options:
.TP
\fB\-\-outer\-manifest\fR
operate starting at the outermost manifest
.TP
\fB\-\-no\-outer\-manifest\fR
do not operate on outer manifests
.TP
\fB\-\-this\-manifest\-only\fR
only operate on this (sub)manifest
.TP
\fB\-\-no\-this\-manifest\-only\fR, \fB\-\-all\-manifests\fR
operate on this manifest and its submanifests
.PP
Run `repo help config` to view the detailed manual.
.SH DETAILS
.PP
The 'repo config' command reads and writes repo's own per\-user settings. These
are separate from git's config and from a checkout's settings; see
docs/repoconfig.md for where they live and which settings repo understands.
.PP
Settings are named <section>.<name>, e.g. sync.smartsync. 'get' prints every
value of the setting and exits 1 without output if it is not set, like `git
config`.
.PP
Values are stored as strings. With \fB\-\-type\fR, 'set' validates the value and stores
it in canonical form (e.g. "on" as "true", "1k" as "1024"), and 'get' prints the
canonical form, like `git config \fB\-\-type\fR`.
+12 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2026" "repo gc" "Repo Manual"
.TH REPO "1" "September 2026" "repo gc" "Repo Manual"
.SH NAME
repo \- repo gc - manual page for repo gc
.SH SYNOPSIS
@@ -22,6 +22,9 @@ answer yes to all safe prompts
.TP
\fB\-\-repack\fR
repack all projects that use partial clone with filter=blob:none
.TP
\fB\-\-keep\-manifest\-projects\fR
keep Git data for projects still in the manifests
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
@@ -44,3 +47,11 @@ only operate on this (sub)manifest
operate on this manifest and its submanifests
.PP
Run `repo help gc` to view the detailed manual.
.SH DETAILS
.PP
The 'repo gc' command removes internal Git data for projects that are no longer
active in the current configuration.
.PP
Projects excluded by the configured manifest groups are inactive by default. Use
\fB\-\-keep\-manifest\-projects\fR to keep Git data for projects that are still in the
manifests.
+6
View File
@@ -103,6 +103,12 @@ use the manifest superproject to sync projects; implies \fB\-c\fR
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-s\fR, \fB\-\-smart\-sync\fR
default to smart sync when syncing projects
.TP
\fB\-\-no\-smart\-sync\fR, \fB\-\-head\fR
do not default to smart sync when syncing projects
.TP
\fB\-\-clone\-bundle\fR
enable use of \fI\,/clone.bundle\/\fP on HTTP/HTTPS (default if not \fB\-\-partial\-clone\fR)
.TP
+14 -11
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2026" "repo manifest" "Repo Manual"
.TH REPO "1" "August 2026" "repo manifest" "Repo Manual"
.SH NAME
repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS
@@ -121,18 +121,19 @@ include*)>
<!ATTLIST remote revision CDATA #IMPLIED>
.IP
<!ELEMENT default EMPTY>
<!ATTLIST default remote IDREF #IMPLIED>
<!ATTLIST default revision CDATA #IMPLIED>
<!ATTLIST default dest\-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync\-j CDATA #IMPLIED>
<!ATTLIST default sync\-j\-max CDATA #IMPLIED>
<!ATTLIST default sync\-c CDATA #IMPLIED>
<!ATTLIST default sync\-s CDATA #IMPLIED>
<!ATTLIST default sync\-tags CDATA #IMPLIED>
<!ATTLIST default remote IDREF #IMPLIED>
<!ATTLIST default revision CDATA #IMPLIED>
<!ATTLIST default dest\-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync\-j CDATA #IMPLIED>
<!ATTLIST default sync\-j\-max CDATA #IMPLIED>
<!ATTLIST default sync\-c CDATA #IMPLIED>
<!ATTLIST default sync\-s CDATA #IMPLIED>
<!ATTLIST default sync\-tags CDATA #IMPLIED>
<!ATTLIST default sync\-smartsync CDATA #IMPLIED>
.IP
<!ELEMENT manifest\-server EMPTY>
<!ATTLIST manifest\-server url CDATA #REQUIRED>
<!ATTLIST manifest\-server url CDATA #REQUIRED>
<!ATTLIST manifest\-server helper CDATA #IMPLIED>
.IP
<!ELEMENT submanifest EMPTY>
@@ -336,6 +337,8 @@ Attribute `sync\-s`: Set to true to also sync sub\-projects.
Attribute `sync\-tags`: Set to false to only sync the given Git branch (specified
in the `revision` attribute) rather than the other ref tags.
.PP
Attribute `sync\-smartsync`: Set to true to enable smart sync by default.
.PP
Element manifest\-server
.PP
At most one manifest\-server may be specified. The url attribute is used to
+8 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "September 2026" "repo sync" "Repo Manual"
.TH REPO "1" "October 2026" "repo sync" "Repo Manual"
.SH NAME
repo \- repo sync - manual page for repo sync
.SH SYNOPSIS
@@ -125,6 +125,9 @@ do not run garbage collection on any projects (default)
\fB\-s\fR, \fB\-\-smart\-sync\fR
smart sync using manifest from the latest known good build
.TP
\fB\-\-no\-smart\-sync\fR, \fB\-\-head\fR
disable smart sync and sync to ToT instead
.TP
\fB\-t\fR SMART_TAG, \fB\-\-smart\-tag\fR=\fI\,SMART_TAG\/\fR
smart sync using manifest from a known tag
.SS Logging options:
@@ -185,6 +188,10 @@ The \fB\-s\fR/\-\-smart\-sync option can be used to sync to a known good build a
specified by the manifest\-server element in the current manifest. The
\fB\-t\fR/\-\-smart\-tag option is similar and allows you to specify a custom tag/label.
.PP
Whether 'repo sync' smart syncs by default can also be controlled with the
sync.smartsync per\-user setting; see docs/repoconfig.md. Explicit
\fB\-s\fR/\-\-no\-smart\-sync/\-t flags always override it.
.PP
The \fB\-u\fR/\-\-manifest\-server\-username and \fB\-p\fR/\-\-manifest\-server\-password options can
be used to specify a username and password to authenticate with the manifest
server when using the \fB\-s\fR or \fB\-t\fR option.
+4 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2026" "repo" "Repo Manual"
.TH REPO "1" "October 2026" "repo" "Repo Manual"
.SH NAME
repo \- repository management tool built on top of git
.SH SYNOPSIS
@@ -65,6 +65,9 @@ Checkout a branch for development
cherry\-pick
Cherry\-pick a change.
.TP
config
Read and write per\-user repo settings
.TP
diff
Show changes between commit and working tree
.TP
+79 -60
View File
@@ -21,12 +21,9 @@ import sys
import urllib.parse
import xml.dom.minidom
from error import ManifestInvalidPathError
from error import ManifestInvalidRevisionError
from error import ManifestParseError
import error
from git_config import GitConfig
from git_refs import HEAD
from git_refs import R_HEADS
import git_refs
from git_superproject import Superproject
import platform_utils
from project import Annotation
@@ -114,7 +111,9 @@ def XmlInt(node, attr, default=None):
try:
return int(value)
except ValueError:
raise ManifestParseError(f'manifest: invalid {attr}="{value}" integer')
raise error.ManifestParseError(
f'manifest: invalid {attr}="{value}" integer'
)
def normalize_url(url: str) -> str:
@@ -159,6 +158,7 @@ class _Default:
sync_c = False
sync_s = False
sync_tags = True
sync_smartsync = False
def __eq__(self, other):
if not isinstance(other, _Default):
@@ -289,7 +289,7 @@ class _XmlSubmanifest:
self.annotations = []
outer_client = parent._outer_client or parent
if self.remote and not self.project:
raise ManifestParseError(
raise error.ManifestParseError(
f"Submanifest {name}: must specify project when remote is "
"given."
)
@@ -423,7 +423,7 @@ class XmlManifest:
# This avoids a trailing os.path.sep when submanifest_path is empty.
self.topdir = os.path.join(self.topdir, submanifest_path)
if manifest_file != os.path.abspath(manifest_file):
raise ManifestParseError("manifest_file must be abspath")
raise error.ManifestParseError("manifest_file must be abspath")
self.manifestFile = manifest_file
if not outer_client or outer_client == self:
# manifestFileOverrides only exists in the outer_client's manifest,
@@ -437,7 +437,9 @@ class XmlManifest:
if submanifest_path and not outer_client:
# If passing a submanifest_path, there must be an outer_client.
raise ManifestParseError(f"Bad call to {self.__class__.__name__}")
raise error.ManifestParseError(
f"Bad call to {self.__class__.__name__}"
)
# If self._outer_client is None, this is not a checkout that supports
# multi-tree.
@@ -476,13 +478,24 @@ class XmlManifest:
if path is None:
path = os.path.join(self.manifestProject.worktree, name)
if not os.path.isfile(path):
raise ManifestParseError("manifest %s not found" % name)
raise error.ManifestParseError("manifest %s not found" % name)
self._load_local_manifests = load_local_manifests
self._outer_client.manifestFileOverrides[self.path_prefix] = path
self.Unload()
self._Load()
def ClearOverride(self) -> None:
"""Stop overriding the manifest, reverting to the default one.
This is the inverse of Override(), and is a no-op if no override is in
effect. It is useful to recover from a failed Override(), which
registers the override before parsing it.
"""
self._outer_client.manifestFileOverrides.pop(self.path_prefix, None)
self._load_local_manifests = True
self.Unload()
def Link(self, name):
"""Update the repo metadata to use a different manifest."""
self.Override(name)
@@ -644,6 +657,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if not d.sync_tags:
have_default = True
e.setAttribute("sync-tags", "false")
if d.sync_smartsync:
have_default = True
e.setAttribute("sync-smartsync", "true")
if have_default:
root.appendChild(e)
root.appendChild(doc.createTextNode(""))
@@ -694,7 +710,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if self.IsMirror:
value = p.bare_git.ResolveCommit(p.revisionExpr)
else:
value = p.work_git.ResolveCommit(HEAD)
value = p.work_git.ResolveCommit(git_refs.HEAD)
e.setAttribute("revision", value)
if peg_rev_upstream:
if p.upstream:
@@ -848,7 +864,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
elif child.nodeName in MULTI_ELEMENTS:
ret.setdefault(child.nodeName, []).append(element)
else:
raise ManifestParseError(
raise error.ManifestParseError(
f'Unhandled element "{child.nodeName}"'
)
@@ -1173,7 +1189,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
def _Load(self, initial_client=None, submanifest_depth=0):
if submanifest_depth > MAX_SUBMANIFEST_DEPTH:
raise ManifestParseError(
raise error.ManifestParseError(
"maximum submanifest depth %d exceeded." % MAX_SUBMANIFEST_DEPTH
)
if not self._loaded:
@@ -1191,8 +1207,8 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
try:
m = self.manifestProject
b = m.GetBranch(m.CurrentBranch).merge
if b is not None and b.startswith(R_HEADS):
b = b[len(R_HEADS) :]
if b is not None and b.startswith(git_refs.R_HEADS):
b = b[len(git_refs.R_HEADS) :]
self.branch = b
parent_groups = self.parent_groups.copy()
@@ -1245,7 +1261,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
try:
self._ParseManifest(nodes)
except ManifestParseError as e:
except error.ManifestParseError as e:
# There was a problem parsing, unload ourselves in case they
# catch this error and try again later, we will show the
# correct error
@@ -1307,10 +1323,12 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
try:
root = xml.dom.minidom.parse(path)
except (OSError, xml.parsers.expat.ExpatError) as e:
raise ManifestParseError(f"error parsing manifest {path}: {e}")
raise error.ManifestParseError(
f"error parsing manifest {path}: {e}"
)
if not root or not root.childNodes:
raise ManifestParseError(f"no root node in {path}")
raise error.ManifestParseError(f"no root node in {path}")
for manifest in root.childNodes:
if (
@@ -1319,7 +1337,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
):
break
else:
raise ManifestParseError(f"no <manifest> in {path}")
raise error.ManifestParseError(f"no <manifest> in {path}")
nodes = []
for node in manifest.childNodes:
@@ -1336,7 +1354,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if restrict_includes:
msg = self._CheckLocalPath(name)
if msg:
raise ManifestInvalidPathError(
raise error.ManifestInvalidPathError(
f'<include> invalid "name": {name}: {msg}'
)
include_groups = (parent_groups or set()).copy()
@@ -1346,7 +1364,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
)
fp = os.path.join(include_root, name)
if not os.path.isfile(fp):
raise ManifestParseError(
raise error.ManifestParseError(
"include [%s/]%s doesn't exist or isn't a file"
% (include_root, name)
)
@@ -1358,10 +1376,10 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
)
# should isolate this to the exact exception, but that's
# tricky. actual parsing implementation may vary.
except (RuntimeError, ManifestParseError):
except (RuntimeError, error.ManifestParseError):
raise
except Exception as e:
raise ManifestParseError(
raise error.ManifestParseError(
f"failed parsing included manifest {name}: {e}"
)
else:
@@ -1385,7 +1403,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if remote:
if remote.name in self._remotes:
if remote != self._remotes[remote.name]:
raise ManifestParseError(
raise error.ManifestParseError(
"remote %s already exists with different "
"attributes" % (remote.name)
)
@@ -1401,7 +1419,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if self._default is None:
self._default = new_default
elif not emptyDefault and new_default != self._default:
raise ManifestParseError(
raise error.ManifestParseError(
"duplicate default in %s" % (self.manifestFile)
)
@@ -1415,7 +1433,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if submanifest:
if submanifest.name in self._submanifests:
if submanifest != self._submanifests[submanifest.name]:
raise ManifestParseError(
raise error.ManifestParseError(
"submanifest %s already exists with different "
"attributes" % (submanifest.name)
)
@@ -1426,7 +1444,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
for node in itertools.chain(*node_list):
if node.nodeName == "notice":
if self._notice is not None:
raise ManifestParseError(
raise error.ManifestParseError(
"duplicate notice in %s" % (self.manifestFile)
)
self._notice = self._ParseNotice(node)
@@ -1436,7 +1454,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
url = self._reqatt(node, "url")
helper = node.getAttribute("helper") or None
if self._manifest_server is not None:
raise ManifestParseError(
raise error.ManifestParseError(
"duplicate manifest-server in %s" % (self.manifestFile)
)
self._manifest_server = url
@@ -1445,18 +1463,18 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
def recursively_add_projects(project):
projects = self._projects.setdefault(project.name, [])
if project.relpath is None:
raise ManifestParseError(
raise error.ManifestParseError(
"missing path for %s in %s"
% (project.name, self.manifestFile)
)
if project.relpath in self._paths:
raise ManifestParseError(
raise error.ManifestParseError(
"duplicate path %s in %s"
% (project.relpath, self.manifestFile)
)
for tree in submanifest_paths:
if project.relpath.startswith(tree):
raise ManifestParseError(
raise error.ManifestParseError(
"project %s conflicts with submanifest path %s"
% (project.relpath, tree)
)
@@ -1476,7 +1494,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
name = self._reqatt(node, "name")
if name not in self._projects:
raise ManifestParseError(
raise error.ManifestParseError(
"extend-project element specifies non-existent "
"project: %s" % name
)
@@ -1498,7 +1516,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
named_projects = self._projects[name]
if dest_path and not path and len(named_projects) > 1:
raise ManifestParseError(
raise error.ManifestParseError(
"extend-project cannot use dest-path when "
"matching multiple projects: %s" % name
)
@@ -1555,7 +1573,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if node.nodeName == "repo-hooks":
# Only one project can be the hooks project
if repo_hooks_project is not None:
raise ManifestParseError(
raise error.ManifestParseError(
"duplicate repo-hooks in %s" % (self.manifestFile)
)
@@ -1569,7 +1587,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
name = self._reqatt(node, "name")
# There can only be one superproject.
if self._superproject:
raise ManifestParseError(
raise error.ManifestParseError(
"duplicate superproject in %s" % (self.manifestFile)
)
remote_name = node.getAttribute("remote")
@@ -1578,7 +1596,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
else:
remote = self._get_remote(node)
if remote is None:
raise ManifestParseError(
raise error.ManifestParseError(
"no remote for superproject %s within %s"
% (name, self.manifestFile)
)
@@ -1586,7 +1604,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if not revision:
revision = self._default.revisionExpr
if not revision:
raise ManifestParseError(
raise error.ManifestParseError(
"no revision for superproject %s within %s"
% (name, self.manifestFile)
)
@@ -1609,7 +1627,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
# Name or path needed.
if not name and not path:
raise ManifestParseError(
raise error.ManifestParseError(
"remove-project must have name and/or path"
)
@@ -1654,14 +1672,14 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
repo_hooks_project = None
if not removed_project and not XmlBool(node, "optional", False):
raise ManifestParseError(
raise error.ManifestParseError(
"remove-project element specifies non-existent "
"project: %s" % node.toxml()
)
if failed_revision_changes:
fail_string = "\n".join(failed_revision_changes)
raise ManifestParseError(
raise error.ManifestParseError(
f"detected base-revision mismatch, updates needed:\n"
f"{fail_string}",
)
@@ -1672,12 +1690,12 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
try:
repo_hooks_projects = self._projects[repo_hooks_project]
except KeyError:
raise ManifestParseError(
raise error.ManifestParseError(
"project %s not found for repo-hooks" % (repo_hooks_project)
)
if len(repo_hooks_projects) != 1:
raise ManifestParseError(
raise error.ManifestParseError(
"internal error parsing repo-hooks in %s"
% (self.manifestFile)
)
@@ -1689,7 +1707,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
name = None
m_url = m.GetRemote().url
if m_url.endswith("/.git"):
raise ManifestParseError("refusing to mirror %s" % m_url)
raise error.ManifestParseError("refusing to mirror %s" % m_url)
if self._default and self._default.remote:
url = self._default.remote.resolvedFetchUrl
@@ -1774,14 +1792,14 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
d.sync_j = XmlInt(node, "sync-j", None)
if d.sync_j is not None and d.sync_j <= 0:
raise ManifestParseError(
raise error.ManifestParseError(
'%s: sync-j must be greater than 0, not "%s"'
% (self.manifestFile, d.sync_j)
)
d.sync_j_max = XmlInt(node, "sync-j-max", None)
if d.sync_j_max is not None and d.sync_j_max <= 0:
raise ManifestParseError(
raise error.ManifestParseError(
'%s: sync-j-max must be greater than 0, not "%s"'
% (self.manifestFile, d.sync_j_max)
)
@@ -1789,6 +1807,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
d.sync_c = XmlBool(node, "sync-c", False)
d.sync_s = XmlBool(node, "sync-s", False)
d.sync_tags = XmlBool(node, "sync-tags", True)
d.sync_smartsync = XmlBool(node, "sync-smartsync", False)
return d
def _ParseNotice(self, node):
@@ -1856,20 +1875,20 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if revision:
msg = self._CheckLocalPath(revision.split("/")[-1])
if msg:
raise ManifestInvalidPathError(
raise error.ManifestInvalidPathError(
'<submanifest> invalid "revision": %s: %s'
% (revision, msg)
)
else:
msg = self._CheckLocalPath(name)
if msg:
raise ManifestInvalidPathError(
raise error.ManifestInvalidPathError(
f'<submanifest> invalid "name": {name}: {msg}'
)
else:
msg = self._CheckLocalPath(path)
if msg:
raise ManifestInvalidPathError(
raise error.ManifestInvalidPathError(
f'<submanifest> invalid "path": {path}: {msg}'
)
@@ -1904,7 +1923,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
name = self._reqatt(node, "name")
msg = self._CheckLocalPath(name, dir_ok=True)
if msg:
raise ManifestInvalidPathError(
raise error.ManifestInvalidPathError(
f'<project> invalid "name": {name}: {msg}'
)
if parent:
@@ -1914,7 +1933,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if remote is None:
remote = self._default.remote
if remote is None:
raise ManifestParseError(
raise error.ManifestParseError(
f"no remote for project {name} within {self.manifestFile}"
)
@@ -1922,7 +1941,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if not revisionExpr:
revisionExpr = self._default.revisionExpr
if not revisionExpr:
raise ManifestParseError(
raise error.ManifestParseError(
"no revision for project %s within %s"
% (name, self.manifestFile)
)
@@ -1935,7 +1954,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
# Project.Sync_LocalHalf.
msg = self._CheckLocalPath(path, dir_ok=True, cwd_dot_ok=True)
if msg:
raise ManifestInvalidPathError(
raise error.ManifestInvalidPathError(
f'<project> invalid "path": {path}: {msg}'
)
@@ -1946,7 +1965,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
clone_depth = XmlInt(node, "clone-depth")
if clone_depth is not None and clone_depth <= 0:
raise ManifestParseError(
raise error.ManifestParseError(
'%s: clone-depth must be greater than 0, not "%s"'
% (self.manifestFile, clone_depth)
)
@@ -2240,7 +2259,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
# It is relative to the top of the repo client checkout.
msg = cls._CheckLocalPath(dest)
if msg:
raise ManifestInvalidPathError(
raise error.ManifestInvalidPathError(
f'<{element}> invalid "dest": {dest}: {msg}'
)
@@ -2251,7 +2270,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
src, dir_ok=is_linkfile, cwd_dot_ok=is_linkfile
)
if msg:
raise ManifestInvalidPathError(
raise error.ManifestInvalidPathError(
f'<{element}> invalid "src": {src}: {msg}'
)
@@ -2280,10 +2299,10 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
value = self._reqatt(node, "value")
try:
keep = self._reqatt(node, "keep").lower()
except ManifestParseError:
except error.ManifestParseError:
keep = "true"
if keep != "true" and keep != "false":
raise ManifestParseError(
raise error.ManifestParseError(
'optional "keep" attribute must be ' '"true" or "false"'
)
element.AddAnnotation(name, value, keep)
@@ -2295,7 +2314,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
v = self._remotes.get(name)
if not v:
raise ManifestParseError(
raise error.ManifestParseError(
f"remote {name} not defined in {self.manifestFile}"
)
return v
@@ -2306,7 +2325,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
"""
v = node.getAttribute(attname)
if not v:
raise ManifestParseError(
raise error.ManifestParseError(
"no %s in <%s> within %s"
% (attname, node.nodeName, self.manifestFile)
)
@@ -2344,7 +2363,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
try:
fromRevId = fromProj.GetCommitRevisionId()
toRevId = toProj.GetCommitRevisionId()
except ManifestInvalidRevisionError:
except error.ManifestInvalidRevisionError:
diff["unreachable"].append((fromProj, toProj))
else:
if fromRevId != toRevId:
+62 -80
View File
@@ -12,32 +12,14 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from ctypes import addressof
from ctypes import byref
from ctypes import c_buffer
from ctypes import c_ubyte
from ctypes import FormatError
from ctypes import get_last_error
from ctypes import Structure
from ctypes import Union
from ctypes import WinDLL
from ctypes import WinError
from ctypes.wintypes import BOOL
from ctypes.wintypes import BOOLEAN
from ctypes.wintypes import DWORD
from ctypes.wintypes import HANDLE
from ctypes.wintypes import LPCWSTR
from ctypes.wintypes import LPDWORD
from ctypes.wintypes import LPVOID
from ctypes.wintypes import ULONG
from ctypes.wintypes import USHORT
from ctypes.wintypes import WCHAR
import ctypes
from ctypes import wintypes
import errno
kernel32 = WinDLL("kernel32", use_last_error=True)
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
UCHAR = c_ubyte
UCHAR = ctypes.c_ubyte
# Win32 error codes
ERROR_SUCCESS = 0
@@ -46,11 +28,11 @@ ERROR_PRIVILEGE_NOT_HELD = 1314
# Win32 API entry points
CreateSymbolicLinkW = kernel32.CreateSymbolicLinkW
CreateSymbolicLinkW.restype = BOOLEAN
CreateSymbolicLinkW.restype = wintypes.BOOLEAN
CreateSymbolicLinkW.argtypes = (
LPCWSTR, # lpSymlinkFileName In
LPCWSTR, # lpTargetFileName In
DWORD, # dwFlags In
wintypes.LPCWSTR, # lpSymlinkFileName In
wintypes.LPCWSTR, # lpTargetFileName In
wintypes.DWORD, # dwFlags In
)
# Symbolic link creation flags
@@ -61,44 +43,44 @@ SYMBOLIC_LINK_FLAG_DIRECTORY = 0x01
SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE = 0x02
GetFileAttributesW = kernel32.GetFileAttributesW
GetFileAttributesW.restype = DWORD
GetFileAttributesW.argtypes = (LPCWSTR,) # lpFileName In
GetFileAttributesW.restype = wintypes.DWORD
GetFileAttributesW.argtypes = (wintypes.LPCWSTR,) # lpFileName In
INVALID_FILE_ATTRIBUTES = 0xFFFFFFFF
FILE_ATTRIBUTE_REPARSE_POINT = 0x00400
CreateFileW = kernel32.CreateFileW
CreateFileW.restype = HANDLE
CreateFileW.restype = wintypes.HANDLE
CreateFileW.argtypes = (
LPCWSTR, # lpFileName In
DWORD, # dwDesiredAccess In
DWORD, # dwShareMode In
LPVOID, # lpSecurityAttributes In_opt
DWORD, # dwCreationDisposition In
DWORD, # dwFlagsAndAttributes In
HANDLE, # hTemplateFile In_opt
wintypes.LPCWSTR, # lpFileName In
wintypes.DWORD, # dwDesiredAccess In
wintypes.DWORD, # dwShareMode In
wintypes.LPVOID, # lpSecurityAttributes In_opt
wintypes.DWORD, # dwCreationDisposition In
wintypes.DWORD, # dwFlagsAndAttributes In
wintypes.HANDLE, # hTemplateFile In_opt
)
CloseHandle = kernel32.CloseHandle
CloseHandle.restype = BOOL
CloseHandle.argtypes = (HANDLE,) # hObject In
CloseHandle.restype = wintypes.BOOL
CloseHandle.argtypes = (wintypes.HANDLE,) # hObject In
INVALID_HANDLE_VALUE = HANDLE(-1).value
INVALID_HANDLE_VALUE = wintypes.HANDLE(-1).value
OPEN_EXISTING = 3
FILE_FLAG_BACKUP_SEMANTICS = 0x02000000
FILE_FLAG_OPEN_REPARSE_POINT = 0x00200000
DeviceIoControl = kernel32.DeviceIoControl
DeviceIoControl.restype = BOOL
DeviceIoControl.restype = wintypes.BOOL
DeviceIoControl.argtypes = (
HANDLE, # hDevice In
DWORD, # dwIoControlCode In
LPVOID, # lpInBuffer In_opt
DWORD, # nInBufferSize In
LPVOID, # lpOutBuffer Out_opt
DWORD, # nOutBufferSize In
LPDWORD, # lpBytesReturned Out_opt
LPVOID, # lpOverlapped Inout_opt
wintypes.HANDLE, # hDevice In
wintypes.DWORD, # dwIoControlCode In
wintypes.LPVOID, # lpInBuffer In_opt
wintypes.DWORD, # nInBufferSize In
wintypes.LPVOID, # lpOutBuffer Out_opt
wintypes.DWORD, # nOutBufferSize In
wintypes.LPDWORD, # lpBytesReturned Out_opt
wintypes.LPVOID, # lpOverlapped Inout_opt
)
# Device I/O control flags and options
@@ -108,45 +90,45 @@ IO_REPARSE_TAG_SYMLINK = 0xA000000C
MAXIMUM_REPARSE_DATA_BUFFER_SIZE = 0x4000
class GENERIC_REPARSE_BUFFER(Structure):
class GENERIC_REPARSE_BUFFER(ctypes.Structure):
_fields_ = (("DataBuffer", UCHAR * 1),)
class SYMBOLIC_LINK_REPARSE_BUFFER(Structure):
class SYMBOLIC_LINK_REPARSE_BUFFER(ctypes.Structure):
_fields_ = (
("SubstituteNameOffset", USHORT),
("SubstituteNameLength", USHORT),
("PrintNameOffset", USHORT),
("PrintNameLength", USHORT),
("Flags", ULONG),
("PathBuffer", WCHAR * 1),
("SubstituteNameOffset", wintypes.USHORT),
("SubstituteNameLength", wintypes.USHORT),
("PrintNameOffset", wintypes.USHORT),
("PrintNameLength", wintypes.USHORT),
("Flags", wintypes.ULONG),
("PathBuffer", wintypes.WCHAR * 1),
)
@property
def PrintName(self):
arrayt = WCHAR * (self.PrintNameLength // 2)
arrayt = wintypes.WCHAR * (self.PrintNameLength // 2)
offset = type(self).PathBuffer.offset + self.PrintNameOffset
return arrayt.from_address(addressof(self) + offset).value
return arrayt.from_address(ctypes.addressof(self) + offset).value
class MOUNT_POINT_REPARSE_BUFFER(Structure):
class MOUNT_POINT_REPARSE_BUFFER(ctypes.Structure):
_fields_ = (
("SubstituteNameOffset", USHORT),
("SubstituteNameLength", USHORT),
("PrintNameOffset", USHORT),
("PrintNameLength", USHORT),
("PathBuffer", WCHAR * 1),
("SubstituteNameOffset", wintypes.USHORT),
("SubstituteNameLength", wintypes.USHORT),
("PrintNameOffset", wintypes.USHORT),
("PrintNameLength", wintypes.USHORT),
("PathBuffer", wintypes.WCHAR * 1),
)
@property
def PrintName(self):
arrayt = WCHAR * (self.PrintNameLength // 2)
arrayt = wintypes.WCHAR * (self.PrintNameLength // 2)
offset = type(self).PathBuffer.offset + self.PrintNameOffset
return arrayt.from_address(addressof(self) + offset).value
return arrayt.from_address(ctypes.addressof(self) + offset).value
class REPARSE_DATA_BUFFER(Structure):
class REPARSE_BUFFER(Union):
class REPARSE_DATA_BUFFER(ctypes.Structure):
class REPARSE_BUFFER(ctypes.Union):
_fields_ = (
("SymbolicLinkReparseBuffer", SYMBOLIC_LINK_REPARSE_BUFFER),
("MountPointReparseBuffer", MOUNT_POINT_REPARSE_BUFFER),
@@ -154,9 +136,9 @@ class REPARSE_DATA_BUFFER(Structure):
)
_fields_ = (
("ReparseTag", ULONG),
("ReparseDataLength", USHORT),
("Reserved", USHORT),
("ReparseTag", wintypes.ULONG),
("ReparseDataLength", wintypes.USHORT),
("Reserved", wintypes.USHORT),
("ReparseBuffer", REPARSE_BUFFER),
)
_anonymous_ = ("ReparseBuffer",)
@@ -182,8 +164,8 @@ def _create_symlink(source, link_name, dwFlags):
# "the unprivileged create flag is unsupported below Windows 10 (1703,
# v10.0.14972). retry without it."
if not CreateSymbolicLinkW(link_name, source, dwFlags):
code = get_last_error()
error_desc = FormatError(code).strip()
code = ctypes.get_last_error()
error_desc = ctypes.FormatError(code).strip()
if code == ERROR_PRIVILEGE_NOT_HELD:
raise OSError(errno.EPERM, error_desc, link_name)
_raise_winerror(code, f'Error creating symbolic link "{link_name}"')
@@ -208,10 +190,10 @@ def readlink(path):
)
if reparse_point_handle == INVALID_HANDLE_VALUE:
_raise_winerror(
get_last_error(), f'Error opening symbolic link "{path}"'
ctypes.get_last_error(), f'Error opening symbolic link "{path}"'
)
target_buffer = c_buffer(MAXIMUM_REPARSE_DATA_BUFFER_SIZE)
n_bytes_returned = DWORD()
target_buffer = ctypes.c_buffer(MAXIMUM_REPARSE_DATA_BUFFER_SIZE)
n_bytes_returned = wintypes.DWORD()
io_result = DeviceIoControl(
reparse_point_handle,
FSCTL_GET_REPARSE_POINT,
@@ -219,13 +201,13 @@ def readlink(path):
0,
target_buffer,
len(target_buffer),
byref(n_bytes_returned),
ctypes.byref(n_bytes_returned),
None,
)
CloseHandle(reparse_point_handle)
if not io_result:
_raise_winerror(
get_last_error(), f'Error reading symbolic link "{path}"'
ctypes.get_last_error(), f'Error reading symbolic link "{path}"'
)
rdb = REPARSE_DATA_BUFFER.from_buffer(target_buffer)
if rdb.ReparseTag == IO_REPARSE_TAG_SYMLINK:
@@ -239,6 +221,6 @@ def readlink(path):
def _raise_winerror(code, error_desc):
win_error_desc = FormatError(code).strip()
win_error_desc = ctypes.FormatError(code).strip()
error_desc = f"{error_desc}: {win_error_desc}"
raise WinError(code, error_desc)
raise ctypes.WinError(code, error_desc)
+4 -4
View File
@@ -22,7 +22,7 @@ try:
except ImportError:
import dummy_threading as _threading
from repo_trace import IsTraceToStderr
import repo_trace
# Capture the original stderr stream. We use this exclusively for progress
@@ -166,7 +166,7 @@ class Progress:
msg = self._last_msg
self._last_msg = msg
if not _TTY or IsTraceToStderr() or self._quiet:
if not _TTY or repo_trace.IsTraceToStderr() or self._quiet:
return
elapsed_sec = time.time() - self._start
@@ -211,7 +211,7 @@ class Progress:
The progress bar is then redrawn on the next line.
"""
if not _TTY or IsTraceToStderr() or self._quiet:
if not _TTY or repo_trace.IsTraceToStderr() or self._quiet:
return
# Erase the current line, print the message with a newline,
@@ -227,7 +227,7 @@ class Progress:
self._ended = True
self._update_event.set()
if not _TTY or IsTraceToStderr() or self._quiet:
if not _TTY or repo_trace.IsTraceToStderr() or self._quiet:
return
duration = duration_str(time.time() - self._start)
+375 -303
View File
File diff suppressed because it is too large. Load diff
+1
View File
@@ -43,4 +43,5 @@ single_line_exclusions = ['abc', 'collections.abc', 'typing']
[tool.pytest.ini_options]
markers = """
skip_cq: Skip tests in the CQ. Should be rarely used!
e2e: Hermetic end-to-end tests (tests/e2e/). Deselect with -m 'not e2e'.
"""
+2 -2
View File
@@ -39,7 +39,7 @@ MANDIR = TOPDIR.joinpath("man")
# Load repo local modules.
sys.path.insert(0, str(TOPDIR))
from git_command import RepoSourceVersion
import git_command
import subcmds
@@ -82,7 +82,7 @@ def main(argv: List[str]) -> int:
del subcmds.all_commands["branch"]
(MANDIR / "repo-branch.1").write_text(".so man1/repo-branches.1")
version = RepoSourceVersion()
version = git_command.RepoSourceVersion()
cmdlist = [
[
*help2man,
+68 -18
View File
@@ -26,7 +26,7 @@ import platform
import shlex
import subprocess
import sys
from typing import NamedTuple
from typing import NamedTuple, Tuple
# These should never be newer than the main.py version since this needs to be a
@@ -129,7 +129,7 @@ if not REPO_REV:
BUG_URL = "https://issues.gerritcodereview.com/issues/new?component=1370071"
# increment this whenever we make important changes to this script
VERSION = (2, 65)
VERSION = (2, 71)
# increment this if the MAINTAINER_KEYS block is modified
KEYRING_VERSION = (2, 3)
@@ -220,6 +220,10 @@ repodir = ".repo" # name of repo's private directory
S_repo = "repo" # special repo repository
S_manifests = "manifests" # special manifest repository
REPO_MAIN = S_repo + "/main.py" # main script
# The automated release process deletes the stable branch and then recreates it
# at the new release, so a fetch can briefly miss it. Wait this long and
# refetch once before giving up on --repo-rev.
REV_REFETCH_DELAY_SEC = 10
import errno
@@ -228,6 +232,7 @@ import optparse
import re
import shutil
import stat
import time
import urllib.error
import urllib.request
@@ -438,6 +443,20 @@ def InitParser(parser):
dest="use_superproject",
help="disable use of manifest superprojects",
)
group.add_option(
"-s",
"--smart-sync",
action="store_true",
default=None,
help="default to smart sync when syncing projects",
)
group.add_option(
"--no-smart-sync",
"--head",
action="store_false",
dest="smart_sync",
help="do not default to smart sync when syncing projects",
)
group.add_option(
"--clone-bundle",
action="store_true",
@@ -631,6 +650,7 @@ def _Init(args):
dst = dst_final + ".tmp"
shutil.rmtree(dst, ignore_errors=True)
_Clone(url, dst, opt.clone_bundle, opt.quiet, opt.verbose)
_RefetchIfRevMissing(url, dst, rev, opt.quiet, opt.verbose)
remote_ref, rev = check_repo_rev(
dst, rev, opt.repo_verify, quiet=opt.quiet
@@ -1005,7 +1025,43 @@ def _Clone(url, cwd, clone_bundle, quiet, verbose):
_Fetch(url, cwd, "origin", quiet, verbose)
def resolve_repo_rev(cwd, committish):
def _RefetchIfRevMissing(
url: str, cwd: str, rev: str, quiet: bool, verbose: bool
) -> None:
"""Refetch once if |rev| doesn't resolve.
This doesn't fail if |rev| still doesn't resolve; check_repo_rev reports
that.
Any unresolvable |rev| is retried, not just branches. Telling a branch
apart from a tag or commit isn't possible while it's missing, and the only
cost for a bad --repo-rev is a short delay before the same error.
Args:
url: The URL of the repo source.
cwd: The git checkout to fetch into.
rev: The --repo-rev to wait for.
quiet: Whether to suppress fetch output.
verbose: Whether to show verbose fetch output.
"""
try:
resolve_repo_rev(cwd, rev, report_errors=False)
return
except CloneFailure:
pass
# Print even with --quiet since this explains the delay.
print(
f'repo: warning: unable to resolve "{rev}"; '
f"refetching in {REV_REFETCH_DELAY_SEC} seconds",
file=sys.stderr,
)
time.sleep(REV_REFETCH_DELAY_SEC)
_Fetch(url, cwd, "origin", quiet, verbose)
def resolve_repo_rev(
cwd: str, committish: str, report_errors: bool = True
) -> Tuple[str, str]:
"""Figure out what REPO_REV represents.
We support:
@@ -1016,6 +1072,7 @@ def resolve_repo_rev(cwd, committish):
Args:
cwd: The git checkout to run in.
committish: The REPO_REV argument to resolve.
report_errors: Whether to print an error if it can't be resolved.
Returns:
A tuple of (remote ref, commit) as makes sense for the committish.
@@ -1034,17 +1091,18 @@ def resolve_repo_rev(cwd, committish):
)
return None if ret.returncode else ret.stdout.strip()
def error(msg: str) -> CloneFailure:
if report_errors:
print(f"repo: error: {msg}", file=sys.stderr)
return CloneFailure()
# An explicit branch.
if committish.startswith("refs/heads/"):
remote_ref = committish
committish = committish[len("refs/heads/") :]
rev = resolve("refs/remotes/origin/%s" % committish)
if rev is None:
print(
f'repo: error: unknown branch "{committish}"',
file=sys.stderr,
)
raise CloneFailure()
raise error(f'unknown branch "{committish}"')
return (remote_ref, rev)
# An explicit tag.
@@ -1053,11 +1111,7 @@ def resolve_repo_rev(cwd, committish):
committish = committish[len("refs/tags/") :]
rev = resolve(remote_ref)
if rev is None:
print(
f'repo: error: unknown tag "{committish}"',
file=sys.stderr,
)
raise CloneFailure()
raise error(f'unknown tag "{committish}"')
return (remote_ref, rev)
# See if it's a short branch name.
@@ -1076,11 +1130,7 @@ def resolve_repo_rev(cwd, committish):
return (rev, rev)
# Give up!
print(
f'repo: error: unable to resolve "{committish}"',
file=sys.stderr,
)
raise CloneFailure()
raise error(f'unable to resolve "{committish}"')
def verify_rev(cwd, remote_ref, rev, quiet):
+2 -2
View File
@@ -17,7 +17,7 @@
import logging
from color import Coloring
from error import RepoExitError
import error
SEPARATOR = "=" * 80
@@ -71,7 +71,7 @@ class RepoLogger(logging.Logger):
handler.setFormatter(_LogColoringFormatter(config))
self.addHandler(handler)
def log_aggregated_errors(self, err: RepoExitError):
def log_aggregated_errors(self, err: error.RepoExitError) -> None:
"""Print all aggregated logs."""
self.error(SEPARATOR)
+3 -3
View File
@@ -57,10 +57,10 @@ def SetTraceToStderr():
_TRACE_TO_STDERR = True
def SetTrace():
"""Enables tracing."""
def SetTrace(value: bool = True) -> None:
"""Enables by default, or disables tracing."""
global _TRACE
_TRACE = True
_TRACE = value
def _SetTraceFile(quiet):
+22 -5
View File
@@ -13,6 +13,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# /// script
# requires-python = '>=3.11,<3.12'
# dependencies = [
# 'pytest==8.3.4',
# 'black==25.1.0',
# 'flake8==6.0.0',
# 'flake8-bugbear==24.12.12',
# 'flake8-comprehensions==3.16.0',
# 'isort==5.10.1'
# ]
# ///
"""Wrapper to run linters and pytest with the right settings."""
import functools
@@ -42,10 +54,16 @@ def is_ci() -> bool:
return os.getenv("LUCI_CQ") == "yes"
def ci_pytest_args() -> list[str]:
"""Returns extra pytest args for CI runs."""
if not is_ci():
return []
return ["-m", "not skip_cq", "--fail-on-skip"]
def run_pytest(argv: list[str]) -> int:
"""Returns the exit code from pytest."""
if is_ci():
argv = ["-m", "not skip_cq"] + argv
argv = ci_pytest_args() + argv
log_cmd("pytest", argv)
return subprocess.run(
@@ -57,8 +75,7 @@ def run_pytest(argv: list[str]) -> int:
def run_pytest_py38(argv: list[str]) -> int:
"""Returns the exit code from pytest under Python 3.8."""
if is_ci():
argv = ["-m", "not skip_cq"] + argv
argv = ci_pytest_args() + argv
log_cmd("[vpython 3.8] pytest", argv)
try:
@@ -135,7 +152,7 @@ def run_isort():
cwd=ROOT_DIR,
)
argv = ["--check", ROOT_DIR]
argv = ["--check", "--diff", ROOT_DIR]
log_cmd("isort", argv)
return subprocess.run(
[sys.executable, "-m", "isort"] + argv,
+101
View File
@@ -0,0 +1,101 @@
attrs==24.2.0 \
--hash=sha256:37c4653e51383dc1249dba15635d59011ca11714a8e61db8cd6dc73011e4b603 \
--hash=sha256:81921eb96de3191c8258c199618104dd27ac608d9366f5e35d011eae1867ede2
# via flake8-bugbear
black==25.1.0 \
--hash=sha256:95e8176dae143ba9097f351d174fdaf0ccd29efb414b362ae3fd72bf0f710717 \
--hash=sha256:c9ed033e41998dd45d6f19bf9923e2444cb401185ccb26d1a2101a9049637c1f
click==8.0.3 \
--hash=sha256:353f466495adaeb40b6b5f592f9f91cb22372351c84caeb068132442a4518ef3 \
--hash=sha256:353f466495adaeb40b6b5f592f9f91cb22372351c84caeb068132442a4518ef3 \
--hash=sha256:b0a0492d07c8e7e4a88ee153ca6553d357a0746792d6868bfca79dc1a344bb1b
# via black
colorama==0.4.6 ; sys_platform == 'win32' \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 \
--hash=sha256:8a5a9494346d7ae2d8f599e4661738007d2df56dbbf4ccbbf7a6a23679a3a302
# via
# click
# pytest
flake8==6.0.0 \
--hash=sha256:9b5bf0ebd06aff1601e38dfc4e91f9b99ba3cdf446e74ee25a9cbbe01aef81ee \
--hash=sha256:f34b5cb8bf908ff49e4673d3dbc557b2381a14132b637d53e9d3a59816f4b6ba
# via
# flake8-bugbear
# flake8-comprehensions
flake8-bugbear==24.12.12 \
--hash=sha256:1b6967436f65ca22a42e5373aaa6f2d87966ade9aa38d4baf2a1be550767545e
flake8-comprehensions==3.16.0 \
--hash=sha256:7c1eadc9d22e765f39857798febe7766b4d9c519793c6c149e3e13bf99693f70
iniconfig==1.1.1 \
--hash=sha256:011e24c64b7f47f6ebd835bb12a743f2fbe9a26d4cecaa7f53bc4f35ee9da8b3 \
--hash=sha256:9d81514370059186276e9a1799e5a9752e4c060233c254d6954b53fad2ff8a5e
# via pytest
isort==5.10.1 \
--hash=sha256:6f62d78e2f89b4500b080fe3a81690850cd254227f27f75c3a0c491a1f351ba7 \
--hash=sha256:ec0da0d88470e89ade75bf1e2ea7b4349ec906f95c84aff08e4d41dea6864d25
mccabe==0.7.0 \
--hash=sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e \
--hash=sha256:b85abd0a321cdd357d0be34adbf49cf01453a857a23e94d44612ec6f159cb261
# via flake8
mypy-extensions==0.4.3 \
--hash=sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d \
--hash=sha256:38cbac35a502c4f13b9d9b1d06258fa51e01da4f5828b171b4d1200d00378e7c
# via black
packaging==23.0 \
--hash=sha256:37cb5b4759d17ae6bad78ee3347f0ee1057452895fcd1bbff48fb12908ce00a1 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2 \
--hash=sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2
# via
# black
# pytest
pathspec==0.9.0 \
--hash=sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a \
--hash=sha256:aaf90f4ad38a16bd3ebe0fa64da7f6ee01819c881120db68cc6440941a9adc08
# via black
platformdirs==2.5.2 \
--hash=sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788 \
--hash=sha256:e257a04b5d0528cb9836b766c8a16a85368e249cba7ae42888a403976ccd7d2d
# via black
pluggy==1.5.0 \
--hash=sha256:40da5bf96b15e359e280fd760a80c11d7a8ea20c3c296c3f3e09f8be67cec10b \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669 \
--hash=sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669
# via pytest
py==1.11.0 \
--hash=sha256:607c53218732647dff4acdfcd50cb62615cedf612e72d1724fb1a0cc6405b378 \
--hash=sha256:607c53218732647dff4acdfcd50cb62615cedf612e72d1724fb1a0cc6405b378
pycodestyle==2.10.0 \
--hash=sha256:8a4eaf0d0495c7395bdab3589ac2db602797d76207242c17d470186815706610 \
--hash=sha256:aacf8d28ecea10fad31b2d14943a42d85d007f4d2fee8f212fc05751c8460647
# via flake8
pyflakes==3.0.1 \
--hash=sha256:a968b8baa43926124b692c7e76279bc5fa8c6ef9a2281e4a2a475be69f954bdf \
--hash=sha256:ec55bf7fe21fff7f1ad2f7da62363d749e2a470500eab1b555334b67aa1ef8cf
# via flake8
pyparsing==3.0.7 \
--hash=sha256:943c220ca64e93940b74dd070152ed9ef1402bc3b28fd1f899843bee34e83708 \
--hash=sha256:a6c06a88f252e6c322f65faf8f418b16213b51bdfaece0524c1c1bc30c63c484
pytest==8.3.4 \
--hash=sha256:00cba7845d2dccfe5403025a688d60ceae228b1ddaa866d2eb86b2edb6923e73 \
--hash=sha256:50e16d954148559c9a74109af1eaf0c945ba2d8f30f0a3d3335edde19788b6f6
toml==0.10.1 \
--hash=sha256:4099063d9b50219bf09bfa693ab8dcd96ce585cee9fdadce5a48a08ee23b0646 \
--hash=sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88
tomli==2.0.1 \
--hash=sha256:864f310e021d68415d6ba45b23e29808f99703b471d884267d6942eecdf8d6c5 \
--hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc
typing-extensions==4.3.0 \
--hash=sha256:09126dab54422810a541e078c37e62024e7260a022d4f67c075a6707d9553649 \
--hash=sha256:25642c956049920a5aa49edcdd6ab1e06d7e5d467fc00e0506c44ac86fbfca02
-135
View File
@@ -1,135 +0,0 @@
# This is a vpython "spec" file.
#
# Read more about `vpython` and how to modify this file here:
# https://chromium.googlesource.com/infra/infra/+/main/doc/users/vpython.md
# List of available wheels:
# https://chromium.googlesource.com/infra/infra/+/main/infra/tools/dockerbuild/wheels.md
python_version: "3.11"
wheel: <
name: "infra/python/wheels/pytest-py3"
version: "version:8.3.4"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/py-py2_py3"
version: "version:1.11.0"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/iniconfig-py3"
version: "version:1.1.1"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/packaging-py3"
version: "version:23.0"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/pluggy-py3"
version: "version:1.5.0"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/toml-py3"
version: "version:0.10.1"
>
# Required by pytest==8.3.4
wheel: <
name: "infra/python/wheels/pyparsing-py3"
version: "version:3.0.7"
>
# Required by pytest==8.3.4 and flake8-bugbear==24.12.12
wheel: <
name: "infra/python/wheels/attrs-py3"
version: "version:24.2.0"
>
# NB: Keep in sync with constraints.txt.
wheel: <
name: "infra/python/wheels/black-py3"
version: "version:25.1.0"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/mypy-extensions-py3"
version: "version:0.4.3"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/tomli-py3"
version: "version:2.0.1"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/platformdirs-py3"
version: "version:2.5.2"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/pathspec-py3"
version: "version:0.9.0"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/typing-extensions-py3"
version: "version:4.3.0"
>
# Required by black==25.1.0
wheel: <
name: "infra/python/wheels/click-py3"
version: "version:8.0.3"
>
wheel: <
name: "infra/python/wheels/flake8-py2_py3"
version: "version:6.0.0"
>
# Required by flake8==6.0.0
wheel: <
name: "infra/python/wheels/mccabe-py2_py3"
version: "version:0.7.0"
>
# Required by flake8==6.0.0
wheel: <
name: "infra/python/wheels/pyflakes-py2_py3"
version: "version:3.0.1"
>
# Required by flake8==6.0.0
wheel: <
name: "infra/python/wheels/pycodestyle-py2_py3"
version: "version:2.10.0"
>
wheel: <
name: "infra/python/wheels/flake8-bugbear-py3"
version: "version:24.12.12"
>
wheel: <
name: "infra/python/wheels/flake8-comprehensions-py3"
version: "version:3.16.0"
>
wheel: <
name: "infra/python/wheels/isort-py3"
version: "version:5.10.1"
>
+7 -5
View File
@@ -24,9 +24,9 @@ import sys
import tempfile
import time
from git_command import git
import git_command
import platform_utils
from repo_trace import Trace
import repo_trace
PROXY_PATH = os.path.join(os.path.dirname(__file__), "git_ssh")
@@ -201,7 +201,9 @@ class ProxyManager:
# is the same as 'myhost.com' where "User git" is setup in the user's
# ~/.ssh/config file.
check_command = command_base + ["-O", "check"]
with Trace("Call to ssh (check call): %s", " ".join(check_command)):
with repo_trace.Trace(
"Call to ssh (check call): %s", " ".join(check_command)
):
try:
check_process = subprocess.Popen(
check_command,
@@ -250,7 +252,7 @@ class ProxyManager:
)
p = None
try:
with Trace("Call to ssh: %s", " ".join(command)):
with repo_trace.Trace("Call to ssh: %s", " ".join(command)):
p = subprocess.Popen(command)
except Exception as e:
self._master_broken.value = True
@@ -357,7 +359,7 @@ def _get_git_protocol_version() -> str:
if e.returncode == 1:
# Exit code 1 means that the git config key was not found.
# Try to imitate the defaults that git would have used.
git_version = git.version_tuple()
git_version = git_command.git.version_tuple()
if git_version >= (2, 26, 0):
# Since git version 2.26, protocol v2 is the default.
return "2"
+9 -8
View File
@@ -18,17 +18,16 @@ import itertools
from command import Command
from command import DEFAULT_LOCAL_JOBS
from error import RepoError
from error import RepoExitError
from git_command import IsValidBranchName
import error
import git_command
from progress import Progress
from repo_logging import RepoLogger
import repo_logging
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class AbandonError(RepoExitError):
class AbandonError(error.RepoExitError):
"""Exit error when abandon command fails."""
@@ -58,7 +57,9 @@ It is equivalent to "git branch -D <branchname>".
if not opt.all:
branches = args[0].split()
invalid_branches = [x for x in branches if not IsValidBranchName(x)]
invalid_branches = [
x for x in branches if not git_command.IsValidBranchName(x)
]
if invalid_branches:
self.OptionParser.error(
@@ -82,7 +83,7 @@ It is equivalent to "git branch -D <branchname>".
status = None
try:
status = project.AbandonBranch(name)
except RepoError as e:
except error.RepoError as e:
status = False
errors.append(e)
if status is not None:
+9 -10
View File
@@ -17,13 +17,12 @@ from typing import NamedTuple
from command import Command
from command import DEFAULT_LOCAL_JOBS
from error import GitError
from error import RepoExitError
import error
from progress import Progress
from repo_logging import RepoLogger
import repo_logging
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class CheckoutBranchResult(NamedTuple):
@@ -33,11 +32,11 @@ class CheckoutBranchResult(NamedTuple):
error: Exception
class CheckoutCommandError(RepoExitError):
class CheckoutCommandError(error.RepoExitError):
"""Exception thrown when checkout command fails."""
class MissingBranchError(RepoExitError):
class MissingBranchError(error.RepoExitError):
"""Exception thrown when no project has specified branch."""
@@ -64,14 +63,14 @@ The command is equivalent to:
@classmethod
def _ExecuteOne(cls, nb, project_idx):
"""Checkout one project."""
error = None
err = None
result = None
project = cls.get_parallel_context()["projects"][project_idx]
try:
result = project.CheckoutBranch(nb)
except GitError as e:
error = e
return CheckoutBranchResult(result, project_idx, error)
except error.GitError as e:
err = e
return CheckoutBranchResult(result, project_idx, err)
def Execute(self, opt, args):
nb = args[0]
+13 -13
View File
@@ -17,13 +17,13 @@ import sys
from typing import Tuple
from command import Command
from error import GitError
from git_command import GitCommand
from repo_logging import RepoLogger
import error
import git_command
import repo_logging
CHANGE_ID_RE = re.compile(r"^\s*Change-Id: I([0-9a-f]{40})\s*$")
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class CherryPick(Command):
@@ -47,7 +47,7 @@ change id will be added.
sha1, commit = self._ResolveReference(reference)
old_msg = self._StripHeader(commit)
p = GitCommand(
p = git_command.GitCommand(
None,
["cherry-pick", sha1],
capture_stdout=True,
@@ -57,7 +57,7 @@ change id will be added.
try:
p.Wait()
except GitError as e:
except error.GitError as e:
logger.error(e)
logger.warning(
"NOTE: When committing (please see above) and editing the "
@@ -76,7 +76,7 @@ change id will be added.
# the commit message.
new_msg = self._Reformat(old_msg, sha1)
p = GitCommand(
p = git_command.GitCommand(
None,
["commit", "--amend", "-F", "-"],
input=new_msg,
@@ -86,14 +86,14 @@ change id will be added.
)
try:
p.Wait()
except GitError:
except error.GitError:
logger.error("error: Failed to update commit message")
raise
def _ResolveReference(self, reference: str) -> Tuple[str, str]:
"""Resolve a commit and read it through one cat-file batch request."""
expression = f"{reference}^{{commit}}"
p = GitCommand(
p = git_command.GitCommand(
None,
["cat-file", "--batch"],
input=expression + "\n",
@@ -107,7 +107,7 @@ change id will be added.
if not separator:
raise ValueError("missing cat-file header")
if header.endswith(" missing") or header.endswith(" ambiguous"):
raise GitError(f"commit {reference} not found")
raise error.GitError(f"commit {reference} not found")
parts = header.split(" ", 2)
if len(parts) != 3 or parts[1] != "commit":
@@ -122,13 +122,13 @@ change id will be added.
raise ValueError("truncated cat-file object")
commit = output[:-1]
except (GitError, ValueError) as e:
except (error.GitError, ValueError) as e:
logger.error(
"error: Failed to resolve or read commit %s", reference
)
if isinstance(e, GitError):
if isinstance(e, error.GitError):
raise
raise GitError(str(e)) from e
raise error.GitError(str(e)) from e
return sha1, commit
+185
View File
@@ -0,0 +1,185 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import optparse
from typing import List, Optional
import command
import error
import git_config
class Error(error.RepoExitError):
"""Exit error when the config command fails."""
class NotSetError(error.SilentRepoExitError):
"""Exit quietly when a requested setting is not set, like `git config`."""
# Subcommand name -> number of positional arguments it takes.
_ACTIONS = {
"get": 1,
"set": 2,
"unset": 1,
"list": 0,
}
def _CanonicalizeBool(name: str, value: str) -> str:
"""Returns |value| as "true"/"false", or raises Error."""
ret = git_config.ParseBoolean(value)
if ret is None:
raise Error(f"error: {name}: '{value}' is not a boolean")
return "true" if ret else "false"
def _CanonicalizeInt(name: str, value: str) -> str:
"""Returns |value| as a plain decimal integer, or raises Error."""
ret = git_config.ParseInt(value)
if ret is None:
raise Error(f"error: {name}: '{value}' is not an integer")
return str(ret)
# --type name -> function turning a value into its canonical string form.
_TYPES = {
"bool": _CanonicalizeBool,
"int": _CanonicalizeInt,
}
class Config(command.Command, command.MirrorSafeCommand):
"""Read and write repo's per-user settings."""
COMMON = True
MULTI_MANIFEST_SUPPORT = True
helpSummary = "Read and write per-user repo settings"
helpUsage = """
%prog [--type=<type>] [-z] get <name>
%prog [--type=<type>] set <name> <value>
%prog unset <name>
%prog [-z] list
"""
helpDescription = """
The '%prog' command reads and writes repo's own per-user settings. These
are separate from git's config and from a checkout's settings; see
docs/repoconfig.md for where they live and which settings repo understands.
Settings are named <section>.<name>, e.g. sync.smartsync. 'get' prints
every value of the setting and exits 1 without output if it is not set,
like `git config`.
Values are stored as strings. With --type, 'set' validates the value and
stores it in canonical form (e.g. "on" as "true", "1k" as "1024"), and
'get' prints the canonical form, like `git config --type`.
"""
def _Options(self, p: optparse.OptionParser) -> None:
p.add_option(
"--type",
choices=sorted(_TYPES),
help="validate and canonicalize the value as this type for "
"get and set (%s)" % ", ".join(sorted(_TYPES)),
)
p.add_option(
"-z",
"--null",
action="store_true",
help="terminate output of get and list with NUL instead of "
"newline, and separate key and value with a newline",
)
def ValidateOptions(self, opt: optparse.Values, args: List[str]) -> None:
if not args:
self.Usage()
action, rest = args[0], args[1:]
if action not in _ACTIONS:
self.OptionParser.error(f"unknown action '{action}'")
if len(rest) != _ACTIONS[action]:
self.Usage()
if opt.type and action not in ("get", "set"):
self.OptionParser.error("--type only applies to get and set")
if opt.null and action not in ("get", "list"):
self.OptionParser.error("-z/--null only applies to get and list")
def Execute(self, opt: optparse.Values, args: List[str]) -> None:
config = git_config.RepoConfig.ForUser()
action, rest = args[0], args[1:]
dispatch = {
"get": self._ExecuteGet,
"set": self._ExecuteSet,
"unset": self._ExecuteUnset,
"list": self._ExecuteList,
}[action]
dispatch(opt, config, *rest)
@staticmethod
def _Print(opt: optparse.Values, text: str) -> None:
"""Prints |text| terminated by NUL with -z, else by newline."""
print(text, end="\0" if opt.null else "\n")
@staticmethod
def _Format(opt: optparse.Values, name: str, value: Optional[str]) -> str:
"""Formats a value for output, canonicalizing it per --type."""
if value is None:
# A key without `=` in the file has no value; git prints a blank.
return ""
if opt.type:
return _TYPES[opt.type](name, value)
return value
def _ExecuteGet(
self, opt: optparse.Values, config: git_config.RepoConfig, name: str
) -> None:
values = config.GetString(name, all_keys=True)
if not values:
raise NotSetError(exit_code=1)
for value in values:
self._Print(opt, self._Format(opt, name, value))
def _ExecuteSet(
self,
opt: optparse.Values,
config: git_config.RepoConfig,
name: str,
value: str,
) -> None:
if opt.type:
value = _TYPES[opt.type](name, value)
self._Write(config, name, value)
def _ExecuteUnset(
self, opt: optparse.Values, config: git_config.RepoConfig, name: str
) -> None:
self._Write(config, name, None)
def _ExecuteList(
self, opt: optparse.Values, config: git_config.RepoConfig
) -> None:
sep = "\n" if opt.null else "="
for key in sorted(config.DumpConfigDict()):
for value in config.GetString(key, all_keys=True):
self._Print(opt, key if value is None else f"{key}{sep}{value}")
@staticmethod
def _Write(
config: git_config.RepoConfig, name: str, value: Optional[str]
) -> None:
"""Writes |value| (None to delete) via `git config`, which also
validates the name."""
try:
config.SetString(name, value)
except error.GitError as e:
raise Error(str(e))
+8 -10
View File
@@ -16,17 +16,15 @@ import re
import sys
from command import Command
from error import GitError
from error import NoSuchProjectError
from error import RepoExitError
from repo_logging import RepoLogger
import error
import repo_logging
CHANGE_RE = re.compile(r"^([1-9][0-9]*)(?:[/\.-]([1-9][0-9]*))?$")
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class DownloadCommandError(RepoExitError):
class DownloadCommandError(error.RepoExitError):
"""Error raised when download command fails."""
@@ -107,7 +105,7 @@ If no project is specified try to use current directory as a project.
# If the cwd is one of the projects, assume they want that.
try:
project = self.GetProjects(".")[0]
except NoSuchProjectError:
except error.NoSuchProjectError:
project = None
if project not in projects:
logger.error(
@@ -121,7 +119,7 @@ If no project is specified try to use current directory as a project.
project.RelPath(local=opt.this_manifest_only),
project.revisionExpr,
)
raise NoSuchProjectError()
raise error.NoSuchProjectError()
else:
project = projects[0]
print("Defaulting to cwd project", project.name)
@@ -143,7 +141,7 @@ If no project is specified try to use current directory as a project.
try:
self._ExecuteHelper(opt, args)
except Exception as e:
if isinstance(e, RepoExitError):
if isinstance(e, error.RepoExitError):
raise e
raise DownloadCommandError(aggregate_errors=[e])
@@ -202,7 +200,7 @@ If no project is specified try to use current directory as a project.
else:
project._Checkout(dl.commit)
except GitError:
except error.GitError:
logger.error(
"[%s] Could not complete the %s of %s",
project.name,
+2 -2
View File
@@ -25,10 +25,10 @@ from color import Coloring
from command import Command
from command import DEFAULT_LOCAL_JOBS
from command import MirrorSafeCommand
from repo_logging import RepoLogger
import repo_logging
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
_CAN_COLOR = [
"branch",
"diff",
+34 -16
View File
@@ -16,8 +16,7 @@ import os
from typing import List, Set
from command import Command
from git_command import git_require
from git_command import GitCommand
import git_command
import platform_utils
from progress import Progress
from project import Project
@@ -28,6 +27,14 @@ class Gc(Command):
helpSummary = "Cleaning up internal repo and Git state."
helpUsage = """
%prog
"""
helpDescription = """
The '%prog' command removes internal Git data for projects that are no
longer active in the current configuration.
Projects excluded by the configured manifest groups are inactive by
default. Use --keep-manifest-projects to keep Git data for projects that
are still in the manifests.
"""
def _Options(self, p):
@@ -53,6 +60,12 @@ class Gc(Command):
help="repack all projects that use partial clone with "
"filter=blob:none",
)
p.add_option(
"--keep-manifest-projects",
default=False,
action="store_true",
help="keep Git data for projects still in the manifests",
)
def _find_git_to_delete(
self, to_keep: Set[str], start_dir: str
@@ -186,13 +199,13 @@ class Gc(Command):
# Prepare workspace for repacking - remove all unreachable refs and
# their objects.
GitCommand(
git_command.GitCommand(
project,
["reflog", "expire", "--expire-unreachable=all"],
verify_command=True,
).Wait()
pm.update(msg=f"{project.name} | gc", inc=0)
GitCommand(
git_command.GitCommand(
project,
["gc"],
verify_command=True,
@@ -200,7 +213,7 @@ class Gc(Command):
# Get all objects that are reachable from the remote, and pack them.
pm.update(msg=f"{project.name} | generating list of objects", inc=0)
remote_objects_cmd = GitCommand(
remote_objects_cmd = git_command.GitCommand(
project,
[
"rev-list",
@@ -215,7 +228,7 @@ class Gc(Command):
)
# Get all local objects and pack them.
local_head_objects_cmd = GitCommand(
local_head_objects_cmd = git_command.GitCommand(
project,
[
"rev-list",
@@ -226,7 +239,7 @@ class Gc(Command):
capture_stdout=True,
verify_command=True,
)
local_objects_cmd = GitCommand(
local_objects_cmd = git_command.GitCommand(
project,
[
"rev-list",
@@ -246,7 +259,7 @@ class Gc(Command):
remote_objects_cmd.Wait()
pm.update(msg=f"{project.name} | remote repack", inc=0)
GitCommand(
git_command.GitCommand(
project,
["pack-objects", os.path.join(pack_dir, "pack")],
input=remote_objects_cmd.stdout,
@@ -262,7 +275,7 @@ class Gc(Command):
local_objects_cmd.Wait()
pm.update(msg=f"{project.name} | local repack", inc=0)
GitCommand(
git_command.GitCommand(
project,
["pack-objects", os.path.join(pack_dir, "pack")],
input=local_head_objects_cmd.stdout + local_objects_cmd.stdout,
@@ -292,21 +305,26 @@ class Gc(Command):
args, all_manifests=not opt.this_manifest_only
)
# If the user specified projects, fetch the global list separately
# to avoid deleting untargeted projects.
if args:
all_projects = self.GetProjects(
if opt.keep_manifest_projects:
projects_to_keep = self.GetProjects(
[],
groups="all",
missing_ok=True,
all_manifests=not opt.this_manifest_only,
)
elif args:
projects_to_keep = self.GetProjects(
[], all_manifests=not opt.this_manifest_only
)
else:
all_projects = projects
projects_to_keep = projects
ret = self.delete_unused_projects(all_projects, opt)
ret = self.delete_unused_projects(projects_to_keep, opt)
if ret != 0:
return ret
if opt.repack:
git_require((2, 17, 0), fail=True, msg="--repack")
git_command.git_require((2, 17, 0), fail=True, msg="--repack")
ret = self.repack_projects(projects, opt)
return ret
+13 -15
View File
@@ -19,14 +19,12 @@ from typing import NamedTuple
from color import Coloring
from command import DEFAULT_LOCAL_JOBS
from command import PagedCommand
from error import GitError
from error import InvalidArgumentsError
from error import SilentRepoExitError
from git_command import GitCommand
from repo_logging import RepoLogger
import error
import git_command
import repo_logging
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class GrepColoring(Coloring):
@@ -43,10 +41,10 @@ class ExecuteOneResult(NamedTuple):
rc: int
stdout: str
stderr: str
error: GitError
error: error.GitError
class GrepCommandError(SilentRepoExitError):
class GrepCommandError(error.SilentRepoExitError):
"""Grep command failure. Since Grep command
output already outputs errors ensure that
aggregate errors exit silently."""
@@ -265,7 +263,7 @@ contain a line that matches both expressions:
"""Process one project."""
project = cls.get_parallel_context()["projects"][project_idx]
try:
p = GitCommand(
p = git_command.GitCommand(
project,
cmd_argv,
bare=False,
@@ -273,16 +271,16 @@ contain a line that matches both expressions:
capture_stderr=True,
verify_command=True,
)
except GitError as e:
except error.GitError as e:
return ExecuteOneResult(project_idx, -1, None, str(e), e)
try:
error = None
err = None
rc = p.Wait()
except GitError as e:
except error.GitError as e:
rc = 1
error = e
return ExecuteOneResult(project_idx, rc, p.stdout, p.stderr, error)
err = e
return ExecuteOneResult(project_idx, rc, p.stdout, p.stderr, err)
@staticmethod
def _ProcessResults(
@@ -377,7 +375,7 @@ contain a line that matches both expressions:
if "--cached" in cmd_argv:
msg = "fatal: cannot combine --cached and --revision"
logger.error(msg)
raise InvalidArgumentsError(msg)
raise error.InvalidArgumentsError(msg)
have_rev = True
cmd_argv.extend(opt.revision)
cmd_argv.append("--")
+2 -2
View File
@@ -19,12 +19,12 @@ import textwrap
from color import Coloring
from command import MirrorSafeCommand
from command import PagedCommand
from error import RepoExitError
import error
from subcmds import all_commands
from wrapper import Wrapper
class InvalidHelpCommand(RepoExitError):
class InvalidHelpCommand(error.RepoExitError):
"""Invalid command passed into help."""
+4 -5
View File
@@ -23,8 +23,7 @@ from typing import Any, Dict, List, NamedTuple, Optional, Tuple
from color import Coloring
from command import DEFAULT_LOCAL_JOBS
from command import PagedCommand
from git_refs import R_HEADS
from git_refs import R_M
import git_refs
class BranchInfo(NamedTuple):
@@ -333,9 +332,9 @@ class Info(PagedCommand):
branch = project.manifest.manifestProject.config.GetBranch(
"default"
).merge
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS) :]
logTarget = R_M + branch
if branch.startswith(git_refs.R_HEADS):
branch = branch[len(git_refs.R_HEADS) :]
logTarget = git_refs.R_M + branch
localCommits, originCommits = cls._GetDiffCommits(
project, logTarget
+12 -10
View File
@@ -18,15 +18,14 @@ import sys
from color import Coloring
from command import InteractiveCommand
from command import MirrorSafeCommand
from error import RepoUnhandledExceptionError
from error import UpdateManifestError
from git_command import git_require
from repo_logging import RepoLogger
import error
import git_command
import repo_logging
from wrapper import Wrapper
from wrapper import WrapperDir
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
_REPO_ALLOW_SHALLOW = os.environ.get("REPO_ALLOW_SHALLOW")
@@ -164,6 +163,7 @@ to update the working directory files.
clone_bundle=opt.clone_bundle,
git_lfs=opt.git_lfs,
use_superproject=opt.use_superproject,
smart_sync=opt.smart_sync,
verbose=opt.verbose,
current_branch_only=opt.current_branch_only,
tags=opt.tags,
@@ -173,7 +173,7 @@ to update the working directory files.
use_local_gitdirs=opt.use_local_gitdirs,
):
manifest_name = opt.manifest_name
raise UpdateManifestError(
raise error.UpdateManifestError(
f"Unable to sync manifest {manifest_name}"
)
@@ -347,9 +347,9 @@ to update the working directory files.
wrapper = Wrapper()
reqs = wrapper.Requirements.from_dir(WrapperDir())
git_require(reqs.get_hard_ver("git"), fail=True)
git_command.git_require(reqs.get_hard_ver("git"), fail=True)
min_git_version_soft = reqs.get_soft_ver("git")
if not git_require(min_git_version_soft):
if not git_command.git_require(min_git_version_soft):
logger.warning(
"repo: warning: git-%s+ will soon be required; "
"please upgrade your version of git to maintain "
@@ -378,7 +378,7 @@ to update the working directory files.
err_msg = "fatal: double check your --repo-rev setting."
logger.error(err_msg)
self.git_event_log.ErrorEvent(err_msg)
raise RepoUnhandledExceptionError(e)
raise error.RepoUnhandledExceptionError(e)
branch = rp.GetBranch("default")
branch.merge = remote_ref
@@ -388,7 +388,9 @@ to update the working directory files.
if opt.worktree:
# Older versions of git supported worktree, but had dangerous gc
# bugs.
git_require((2, 15, 0), fail=True, msg="git gc worktree corruption")
git_command.git_require(
(2, 15, 0), fail=True, msg="git gc worktree corruption"
)
# Provide a short notice that we're reinitializing an existing checkout.
# Sometimes developers might not realize that they're in one, or that
+2 -2
View File
@@ -19,10 +19,10 @@ import os
import sys
from command import PagedCommand
from repo_logging import RepoLogger
import repo_logging
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class OutputFormat(enum.Enum):
+6 -6
View File
@@ -16,13 +16,13 @@ import sys
from color import Coloring
from command import Command
from error import GitError
from git_command import GitCommand
import error
import git_command
from project import Project
from repo_logging import RepoLogger
import repo_logging
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class RebaseColoring(Coloring):
@@ -39,7 +39,7 @@ def _ResolveOntoManifest(project: Project) -> str:
"""
try:
return project.GetRemote().ToLocal(project.revisionExpr)
except GitError:
except error.GitError:
return project.revisionExpr
@@ -190,7 +190,7 @@ branch but need to incorporate new upstream changes "underneath" them.
out.nl()
out.flush()
if GitCommand(project, args).Wait() != 0:
if git_command.GitCommand(project, args).Wait() != 0:
ret += 1
continue
+4 -4
View File
@@ -16,16 +16,16 @@ import optparse
from command import Command
from command import MirrorSafeCommand
from error import RepoExitError
from repo_logging import RepoLogger
import error
import repo_logging
from subcmds.sync import _PostRepoFetch
from subcmds.sync import _PostRepoUpgrade
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class SelfupdateError(RepoExitError):
class SelfupdateError(error.RepoExitError):
"""Exit error for failed selfupdate command."""
+4 -4
View File
@@ -16,11 +16,11 @@ import sys
from color import Coloring
from command import InteractiveCommand
from git_command import GitCommand
from repo_logging import RepoLogger
import git_command
import repo_logging
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class _ProjectList(Coloring):
@@ -129,5 +129,5 @@ The '%prog' command stages files to prepare the next commit.
def _AddI(project):
p = GitCommand(project, ["add", "--interactive"], bare=False)
p = git_command.GitCommand(project, ["add", "--interactive"], bare=False)
p.Wait()
+9 -9
View File
@@ -17,14 +17,14 @@ from typing import NamedTuple
from command import Command
from command import DEFAULT_LOCAL_JOBS
from error import RepoExitError
from git_command import IsValidBranchName
import error
import git_command
from git_config import IsImmutable
from progress import Progress
from repo_logging import RepoLogger
import repo_logging
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class ExecuteOneResult(NamedTuple):
@@ -32,7 +32,7 @@ class ExecuteOneResult(NamedTuple):
error: Exception
class StartError(RepoExitError):
class StartError(error.RepoExitError):
"""Exit error for failed start command."""
@@ -75,7 +75,7 @@ revision specified in the manifest.
self.Usage()
nb = args[0]
if not IsValidBranchName(nb):
if not git_command.IsValidBranchName(nb):
self.OptionParser.error("'%s' is not a valid name" % nb)
@classmethod
@@ -85,7 +85,7 @@ revision specified in the manifest.
# a change, then we can't push back to it. Substitute with
# dest_branch, if defined; or with manifest default revision instead.
branch_merge = ""
error = None
err = None
project = cls.get_parallel_context()["projects"][project_idx]
if IsImmutable(project.revisionExpr):
if project.dest_branch:
@@ -99,8 +99,8 @@ revision specified in the manifest.
)
except Exception as e:
logger.error("error: unable to checkout %s: %s", project.name, e)
error = e
return ExecuteOneResult(project_idx, error)
err = e
return ExecuteOneResult(project_idx, err)
def Execute(self, opt, args):
nb = args[0]
+308 -101
View File
@@ -57,19 +57,12 @@ from command import Command
from command import DEFAULT_LOCAL_JOBS
from command import MirrorSafeCommand
from command import WORKER_BATCH_SIZE
from error import GitError
from error import RepoChangedException
from error import RepoError
from error import RepoExitError
from error import RepoUnhandledExceptionError
from error import SyncError
from error import UpdateManifestError
import error
import event_log
from git_command import git_require
from git_command import GitCommand
import git_command
from git_config import GetUrlCookieFile
from git_refs import HEAD
from git_refs import R_HEADS
from git_config import RepoConfig
import git_refs
import git_superproject
from hooks import RepoHook
from manifest_xml import XmlManifest
@@ -81,8 +74,8 @@ from project import DeleteWorktreeError
from project import Project
from project import RemoteSpec
from project import SyncBuffer
from repo_logging import RepoLogger
from repo_trace import Trace
import repo_logging
import repo_trace
import ssh
from wrapper import Wrapper
@@ -95,7 +88,70 @@ _BLOAT_PACK_COUNT_THRESHOLD = 10
_BLOAT_SIZE_PACK_THRESHOLD_KB = 10 * 1024 * 1024 # 10 GiB in KiB
_BLOAT_SIZE_GARBAGE_THRESHOLD_KB = 1 * 1024 * 1024 # 1 GiB in KiB
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
# Manifest server RPC that takes a single struct of named options. Servers
# that do not implement it are called via the positional GetApprovedManifest.
_GET_APPROVED_MANIFEST_V2 = "GetApprovedManifestV2"
def _IsMethodNotFoundFault(fault: xmlrpc.client.Fault, method: str) -> bool:
"""Whether |fault| means the server does not implement |method|."""
if fault.faultCode == xmlrpc.client.METHOD_NOT_FOUND:
return True
# Python's SimpleXMLRPCDispatcher uses faultCode 1 with this message.
return f'method "{method}" is not supported' in str(fault.faultString)
def _GetApprovedManifest(
server: xmlrpc.client.ServerProxy,
branch: str,
target: Optional[str],
best_effort: bool,
) -> Tuple[bool, str]:
"""Requests the approved manifest from the manifest server.
Args:
server: The manifest server proxy.
branch: The manifest branch.
target: The build target, if any.
best_effort: The client can proceed without a manifest (e.g. by
falling back to ToT), so the server may decline the request.
Returns:
A (success, manifest XML or error message) tuple.
"""
request = {"branch": branch}
if target:
request["target"] = target
if best_effort:
request["best_effort"] = True
try:
response = server.GetApprovedManifestV2(request)
except xmlrpc.client.Fault as e:
if not _IsMethodNotFoundFault(e, _GET_APPROVED_MANIFEST_V2):
raise
# Older server: |best_effort| has no equivalent and is dropped.
if target:
success, manifest_str = server.GetApprovedManifest(branch, target)
else:
success, manifest_str = server.GetApprovedManifest(branch)
return success, manifest_str
if not isinstance(response, dict):
raise SmartSyncError(
f"error: invalid {_GET_APPROVED_MANIFEST_V2} response: {response!r}"
)
if response.get("success"):
manifest_xml = response.get("manifest_xml")
if not isinstance(manifest_xml, str):
raise SmartSyncError(
f"error: {_GET_APPROVED_MANIFEST_V2} response has invalid "
f"manifest_xml: {manifest_xml!r}"
)
return True, manifest_xml
return False, response.get("error_message", "unknown error")
def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
@@ -353,19 +409,19 @@ class _InterleavedSyncResult(NamedTuple):
results: List[_SyncResult]
class SuperprojectError(SyncError):
class SuperprojectError(error.SyncError):
"""Superproject sync repo."""
class SyncFailFastError(SyncError):
class SyncFailFastError(error.SyncError):
"""Sync exit error when --fail-fast set."""
class SmartSyncError(SyncError):
class SmartSyncError(error.SyncError):
"""Smart sync exit error."""
class ManifestInterruptError(RepoError):
class ManifestInterruptError(error.RepoError):
"""Aggregate Error to be logged when a user interrupts a manifest update."""
def __init__(self, output, **kwargs):
@@ -427,6 +483,10 @@ build as specified by the manifest-server element in the current
manifest. The -t/--smart-tag option is similar and allows you to
specify a custom tag/label.
Whether '%prog' smart syncs by default can also be controlled with the
sync.smartsync per-user setting; see docs/repoconfig.md. Explicit
-s/--no-smart-sync/-t flags always override it.
The -u/--manifest-server-username and -p/--manifest-server-password
options can be used to specify a username and password to authenticate
with the manifest server when using the -s or -t option.
@@ -764,9 +824,17 @@ later is required to fix a server side protocol bug.
"-s",
"--smart-sync",
action="store_true",
default=None,
help="smart sync using manifest from the latest known good "
"build",
)
p.add_option(
"--no-smart-sync",
"--head",
dest="smart_sync",
action="store_false",
help="disable smart sync and sync to ToT instead",
)
p.add_option(
"-t",
"--smart-tag",
@@ -797,8 +865,8 @@ later is required to fix a server side protocol bug.
"""
b = manifest_project.GetBranch(manifest_project.CurrentBranch)
branch = b.merge
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS) :]
if branch.startswith(git_refs.R_HEADS):
branch = branch[len(git_refs.R_HEADS) :]
return branch
@classmethod
@@ -819,13 +887,42 @@ later is required to fix a server side protocol bug.
or opt.current_branch_only
)
def _GetSuperprojectRevision(
self, opt: optparse.Values, manifest: XmlManifest
) -> Optional[str]:
"""Returns the --superproject-revision to apply to manifest.
The option only applies to the outer manifest.
Args:
opt: Program options returned from optparse. See _Options().
manifest: The manifest to check.
Returns:
The requested superproject revision, or None if manifest's
superproject should not be pinned.
"""
if manifest == self.outer_manifest:
return opt.superproject_revision
return None
def _ConfigureSuperproject(
self,
opt: optparse.Values,
manifest,
revision: Optional[str] = None,
self, opt: optparse.Values, manifest: XmlManifest
) -> bool:
"""Configure superproject with options."""
"""Configure superproject with options.
The configuration lives on the manifest's Superproject object, which
is discarded whenever the manifest is unloaded (see _ReloadManifest
and XmlManifest.Unload). Callers must therefore configure the
superproject again after every manifest reload.
Args:
opt: Program options returned from optparse. See _Options().
manifest: The manifest whose superproject should be configured.
Returns:
Whether superproject messages should be printed.
"""
if not manifest.superproject:
return False
manifest.superproject.SetQuiet(not opt.verbose)
@@ -833,6 +930,7 @@ later is required to fix a server side protocol bug.
opt.use_superproject, manifest
)
manifest.superproject.SetPrintMessages(print_messages)
revision = self._GetSuperprojectRevision(opt, manifest)
if revision:
manifest.superproject.SetRevisionId(revision)
return print_messages
@@ -858,6 +956,11 @@ later is required to fix a server side protocol bug.
m.superproject for m in manifest.all_children
)
if not have_superproject:
superproject_logging_data.update(
superproject=False,
haslocalmanifests=bool(manifest.HasLocalManifests),
hassuperprojecttag=False,
)
return
if opt.local_only and manifest.superproject:
@@ -882,7 +985,6 @@ later is required to fix a server side protocol bug.
for p in all_projects:
per_manifest[p.manifest.path_prefix].append(p)
superproject_logging_data = {}
need_unload = False
for m in self.ManifestList(opt):
if m.path_prefix not in per_manifest:
@@ -923,12 +1025,8 @@ later is required to fix a server side protocol bug.
else:
if print_messages:
logger.warning(
"%s: warning: Update of revisionId from superproject "
"has failed, repo sync will not use superproject to "
"fetch the source. Please resync with the "
"--no-use-superproject option to avoid this repo "
"warning.",
m.path_prefix,
"error: Update of revsionId from superproject "
"has failed"
)
if update_result.fatal and opt.use_superproject is not None:
raise SuperprojectError()
@@ -1009,7 +1107,7 @@ later is required to fix a server side protocol bug.
logger.error("Keyboard interrupt while processing %s", project.name)
if not cls.is_multiprocessing_active():
raise
except GitError as e:
except error.GitError as e:
logger.error("error.GitError: Cannot fetch %s", e)
errors.append(e)
except Exception as e:
@@ -1215,7 +1313,7 @@ later is required to fix a server side protocol bug.
if opt.network_only:
# Bail out now; the rest touches the working tree.
if err_event.is_set():
e = SyncError(
e = error.SyncError(
"error: Exited sync due to fetch errors.",
aggregate_errors=errors,
)
@@ -1309,7 +1407,7 @@ later is required to fix a server side protocol bug.
logger.error("Keyboard interrupt while processing %s", project.name)
if not cls.is_multiprocessing_active():
raise
except GitError as e:
except error.GitError as e:
logger.error(
"error.GitError: Cannot checkout %s: %s", project.name, e
)
@@ -1465,7 +1563,7 @@ later is required to fix a server side protocol bug.
if expected != actual:
# If this is unexpected, log it and repair.
Trace(
repo_trace.Trace(
f"{relpath} expected preciousObjects={expected}, got {actual}"
)
if expected:
@@ -1475,7 +1573,7 @@ later is required to fix a server side protocol bug.
% (relpath, project.name)
)
if git_require((2, 7, 0)):
if git_command.git_require((2, 7, 0)):
project.EnableRepositoryExtension("preciousObjects")
else:
# This isn't perfect, but it's the best we can do with old
@@ -1568,7 +1666,7 @@ later is required to fix a server side protocol bug.
cls._RunOneGC(bare_git._project, config=config)
else:
bare_git.pack_refs(config=config)
except GitError:
except error.GitError:
err_event.set()
except Exception:
err_event.set()
@@ -1608,26 +1706,12 @@ later is required to fix a server side protocol bug.
"""
project = cls.get_parallel_context()["projects"][project_index]
if not project.Exists or not project.worktree:
return None
# Only check dirty or locally modified projects. These can't be
# freshly cloned and will accumulate garbage.
try:
status = project._GetStatusSnapshot(
untracked_files="normal", branch=True
)
if status is not None:
is_dirty = status.is_dirty(consider_untracked=True)
head_rev = status.branch_oid
else:
is_dirty = project.IsDirty(consider_untracked=True)
head_rev = project.work_git.rev_parse(HEAD)
is_dirty, head_rev = project.GetDirtyAndHead()
if head_rev is None:
# Porcelain v2 reports an unborn branch as "(initial)". The
# legacy rev-parse path failed here and skipped the bloat
# calculation, so preserve that behavior.
# An unborn branch has no HEAD to compare, so skip it.
return None
manifest_rev = project.GetRevisionId(project.bare_ref.all)
@@ -1671,16 +1755,27 @@ later is required to fix a server side protocol bug.
run 'git count-objects -v' and warn if the repository is accumulating
excessive pack files or garbage.
"""
# --network-only promises not to touch worktrees, but git status and
# update-index --refresh can both rewrite the index.
if opt.network_only:
return
# We only care about bloated projects if we have a git version that
# supports --no-auto-gc (2.23.0+) since what we use to disable auto-gc
# in Project._RemoteFetch.
if not git_require((2, 23, 0)):
if not git_command.git_require((2, 23, 0)):
return
# Skip projects with no worktree on disk, e.g. ones only ever synced
# with --network-only.
projects = [
p
for p in projects
if p.clone_depth and not p.stateless_prune_needed
if p.clone_depth
and not p.stateless_prune_needed
and p.worktree
and p.Exists
and platform_utils.isdir(p.worktree)
]
if not projects:
return
@@ -2054,7 +2149,13 @@ later is required to fix a server side protocol bug.
return server_url, transport
def _SmartSyncSetup(self, opt, smart_sync_manifest_path, manifest):
def _SmartSyncSetup(
self,
opt: optparse.Values,
smart_sync_manifest_path: str,
manifest: XmlManifest,
best_effort: bool = False,
) -> str:
if not manifest.manifest_server:
raise SmartSyncError(
"error: cannot smart sync: no manifest server defined in "
@@ -2072,7 +2173,12 @@ later is required to fix a server side protocol bug.
try:
server = xmlrpc.client.Server(server_url, transport=transport)
if opt.smart_sync:
branch = self._GetBranch(manifest.manifestProject)
# SYNC_BRANCH overrides the branch sent to the manifest server
# (e.g. to select a specific CI branch that builds the same
# manifest branch). An empty value is treated as unset.
branch = os.environ.get("SYNC_BRANCH")
if not branch:
branch = self._GetBranch(manifest.manifestProject)
target = None
if "SYNC_TARGET" in os.environ:
target = os.environ["SYNC_TARGET"]
@@ -2095,12 +2201,9 @@ later is required to fix a server side protocol bug.
os.environ["TARGET_BUILD_VARIANT"],
)
if target:
[success, manifest_str] = server.GetApprovedManifest(
branch, target
)
else:
[success, manifest_str] = server.GetApprovedManifest(branch)
success, manifest_str = _GetApprovedManifest(
server, branch, target, best_effort
)
else:
assert opt.smart_tag
[success, manifest_str] = server.GetManifest(opt.smart_tag)
@@ -2156,7 +2259,7 @@ later is required to fix a server side protocol bug.
mp: the manifestProject to query.
manifest_name: Manifest file to be reloaded.
"""
if opt.superproject_revision and mp.manifest == self.outer_manifest:
if self._GetSuperprojectRevision(opt, mp.manifest):
self._SyncToSuperprojectRev(
opt, mp.manifest, mp, manifest_name, errors
)
@@ -2234,7 +2337,7 @@ later is required to fix a server side protocol bug.
mp, event_log.TASK_SYNC_LOCAL, start, time.time(), clean
)
if not clean:
raise UpdateManifestError(aggregate_errors=errors)
raise error.UpdateManifestError(aggregate_errors=errors)
self._ReloadManifest(manifest_name, mp.manifest)
def ValidateOptions(self, opt, args):
@@ -2252,10 +2355,6 @@ later is required to fix a server side protocol bug.
if opt.manifest_name and opt.smart_tag:
self.OptionParser.error("cannot combine -m and -t")
if opt.manifest_server_username or opt.manifest_server_password:
if not (opt.smart_sync or opt.smart_tag):
self.OptionParser.error(
"-u and -p may only be combined with -s or -t"
)
if None in [
opt.manifest_server_username,
opt.manifest_server_password,
@@ -2346,10 +2445,10 @@ later is required to fix a server side protocol bug.
try:
self._ExecuteHelper(opt, args, errors)
sync_duration_seconds = time.time() - start_time
except (RepoExitError, RepoChangedException):
except (error.RepoExitError, error.RepoChangedException):
raise
except (KeyboardInterrupt, Exception) as e:
raise RepoUnhandledExceptionError(e, aggregate_errors=errors)
raise error.RepoUnhandledExceptionError(e, aggregate_errors=errors)
# Run post-sync hook only after successful sync
self._RunPostSyncHook(opt, sync_duration_seconds=sync_duration_seconds)
@@ -2372,25 +2471,26 @@ later is required to fix a server side protocol bug.
def _SyncToSuperprojectRev(
self,
opt: optparse.Values,
manifest,
manifest: XmlManifest,
mp: Project,
manifest_name: Optional[str],
errors: List[Exception],
) -> None:
"""Sync to a specific superproject commit."""
if not manifest.superproject:
raise SyncError("superproject not defined in manifest")
"""Sync to a specific superproject commit.
self._ConfigureSuperproject(
opt, manifest, revision=opt.superproject_revision
)
Only valid for the outer manifest; see _GetSuperprojectRevision.
"""
if not manifest.superproject:
raise error.SyncError("superproject not defined in manifest")
self._ConfigureSuperproject(opt, manifest)
sync_result = manifest.superproject.Sync(self.git_event_log)
if not sync_result.success:
raise SyncError("failed to sync superproject")
raise error.SyncError("failed to sync superproject")
cmd = ["show", f"{opt.superproject_revision}:.supermanifest"]
p = GitCommand(
p = git_command.GitCommand(
None,
cmd,
gitdir=manifest.superproject._work_git,
@@ -2399,23 +2499,75 @@ later is required to fix a server side protocol bug.
capture_stderr=True,
)
if p.Wait() != 0:
raise SyncError(
raise error.SyncError(
f"failed to read .supermanifest from superproject: {p.stderr}"
)
try:
_, _, manifest_commit = p.stdout.strip().split()
except ValueError:
raise SyncError("could not parse .supermanifest")
raise error.SyncError("could not parse .supermanifest")
mp.SetRevision(manifest_commit)
try:
self._UpdateManifestProject(opt, mp, manifest_name, errors)
except UpdateManifestError as e:
raise SyncError(
except error.UpdateManifestError as e:
raise error.SyncError(
"failed to sync manifest project", aggregate_errors=[e]
)
def _ResolveSmartSyncOption(
self, opt: optparse.Values, manifest: XmlManifest
) -> bool:
"""Resolve opt.smart_sync from CLI flags, config, and manifest.
Precedence: CLI flags > per-checkout settings > per-user settings >
per-manifest settings.
See docs/repoconfig.md and docs/smart-sync.md for details.
Returns:
True if smart_sync was enabled implicitly by checkout or manifest
default, False otherwise. Smart sync enabled via per-user settings
counts as an explicit user request, not implicit.
"""
checkout_pref = getattr(manifest.manifestProject, "smart_sync", None)
if not isinstance(checkout_pref, bool):
checkout_pref = None
user_pref = RepoConfig.ForUser().GetBoolean("sync.smartsync")
default_allowed = (
opt.smart_sync is None
and not opt.smart_tag
and not opt.manifest_name
and not opt.superproject_revision
and not opt.local_only
)
default_smart_sync = (
checkout_pref
if checkout_pref is not None
else getattr(manifest.default, "sync_smartsync", False) is True
)
implicit = (
default_allowed
and (checkout_pref is not None or user_pref is None)
and default_smart_sync
)
if opt.smart_sync is None:
if checkout_pref is not None:
opt.smart_sync = implicit
elif user_pref is False:
opt.smart_sync = False
elif user_pref is True:
opt.smart_sync = default_allowed
else:
opt.smart_sync = implicit
if (
opt.manifest_server_username or opt.manifest_server_password
) and not (opt.smart_sync or opt.smart_tag):
raise SmartSyncError("-u and -p may only be combined with -s or -t")
return implicit
def _ExecuteHelper(self, opt, args, errors):
manifest = self.outer_manifest
if not opt.outer_manifest:
@@ -2430,11 +2582,62 @@ later is required to fix a server side protocol bug.
if opt.clone_bundle is None:
opt.clone_bundle = manifest.CloneBundle
# Without an explicit smart sync option, -l does not contact the
# manifest server, so it keeps the smart sync manifest the client last
# synced to, whether that came from -s or from the manifest's
# sync-smartsync default. This keeps `sync -n` + `sync -l` equivalent
# to a single sync. Computed before _ResolveSmartSyncOption, which
# makes an unset opt.smart_sync indistinguishable from --no-smart-sync.
reuse_smart_sync_override = (
opt.local_only
and opt.smart_sync is None
and not opt.smart_tag
and not opt.manifest_name
and not opt.superproject_revision
and os.path.isfile(smart_sync_manifest_path)
)
implicit_smart_sync = self._ResolveSmartSyncOption(opt, manifest)
if opt.smart_sync or opt.smart_tag:
manifest_name = self._SmartSyncSetup(
opt, smart_sync_manifest_path, manifest
)
else:
try:
manifest_name = self._SmartSyncSetup(
opt,
smart_sync_manifest_path,
manifest,
best_effort=implicit_smart_sync,
)
except (SmartSyncError, error.ManifestParseError) as e:
if not implicit_smart_sync:
raise
# The user did not ask for a smart sync, so failing to get a
# usable manifest from the server should not fail the sync.
# Fall back to a regular ToT sync instead.
opt.smart_sync = False
# Override() records the override before parsing it, so an
# unparsable manifest would otherwise leave the client pinned
# to a file it cannot load.
manifest.ClearOverride()
if opt.verbose:
logger.warning(
"warning: smart sync failed; falling back to ToT: %s", e
)
elif reuse_smart_sync_override:
manifest_name = os.path.basename(smart_sync_manifest_path)
try:
self._ReloadManifest(manifest_name, manifest)
except error.ManifestParseError as e:
reuse_smart_sync_override = False
manifest_name = None
manifest.ClearOverride()
if opt.verbose:
logger.warning(
"warning: failed to load existing smart sync manifest; "
"falling back to ToT: %s",
e,
)
if not (opt.smart_sync or opt.smart_tag or reuse_smart_sync_override):
if os.path.isfile(smart_sync_manifest_path):
try:
platform_utils.remove(smart_sync_manifest_path)
@@ -2593,7 +2796,7 @@ later is required to fix a server side protocol bug.
nested = _NestedProjects(projects)
if not nested:
return
e = SyncError(
e = error.SyncError(
"error: repo.reprojectcmd does not support nested projects or "
"submodules; found:\n"
+ "\n".join(
@@ -2717,7 +2920,7 @@ later is required to fix a server side protocol bug.
logger.error(
'Try re-running with "-j1 --fail-fast" to exit at the first error.'
)
raise SyncError(aggregate_errors=errors)
raise error.SyncError(aggregate_errors=errors)
def _SyncPhased(
self,
@@ -2851,7 +3054,7 @@ later is required to fix a server side protocol bug.
)
if not cls.is_multiprocessing_active():
raise
except GitError as e:
except error.GitError as e:
fetch_errors.append(e)
logger.error("error.GitError: Cannot fetch %s", e)
except Exception as e:
@@ -2903,7 +3106,7 @@ later is required to fix a server side protocol bug.
)
if not cls.is_multiprocessing_active():
raise
except GitError as e:
except error.GitError as e:
checkout_errors.append(e)
logger.error(
"error.GitError: Cannot checkout %s: %s",
@@ -3271,7 +3474,7 @@ def _PostRepoFetch(rp, repo_verify=True, verbose=False):
wrapper = Wrapper()
try:
rev = rp.bare_git.describe(rp.GetRevisionId())
except GitError:
except error.GitError:
rev = None
_, new_rev = wrapper.check_repo_rev(
rp.gitdir, rev, repo_verify=repo_verify
@@ -3289,15 +3492,18 @@ def _PostRepoFetch(rp, repo_verify=True, verbose=False):
# Refresh index since reset --keep won't do it.
rp.work_git.update_index("-q", "--refresh")
rp.work_git.reset("--keep", new_rev)
except GitError as e:
raise RepoUnhandledExceptionError(e)
except error.GitError as e:
raise error.RepoUnhandledExceptionError(e)
print("info: Restarting repo with latest version")
raise RepoChangedException(["--repo-upgraded"])
raise error.RepoChangedException(["--repo-upgraded"])
else:
logger.warning("warning: Skipped upgrade to unverified version")
else:
if verbose:
print("repo version %s is current" % rp.work_git.describe(HEAD))
print(
"repo version %s is current"
% rp.work_git.describe(git_refs.HEAD)
)
class _FetchTimes:
@@ -3372,13 +3578,13 @@ class LocalSyncState:
def _Get(self, project, key):
self._Load()
p = project.relpath
p = project.RelPath(local=False)
if p not in self._state:
return
return self._state[p].get(key)
def _Set(self, project, key):
p = project.relpath
p = project.RelPath(local=False)
if p not in self._state:
self._state[p] = {}
self._state[p][key] = self._time
@@ -3406,8 +3612,9 @@ class LocalSyncState:
if not self._state:
return
delete = set()
outer_topdir = self._manifest.outer_client.topdir
for path in self._state:
gitdir = os.path.join(self._manifest.topdir, path, ".git")
gitdir = os.path.join(outer_topdir, path, ".git")
if not os.path.exists(gitdir) or os.path.islink(gitdir):
delete.add(path)
if not delete:
+43 -25
View File
@@ -22,22 +22,20 @@ from typing import List
from command import DEFAULT_LOCAL_JOBS
from command import InteractiveCommand
from editor import Editor
from error import GitError
from error import SilentRepoExitError
from error import UploadError
from git_refs import R_HEADS
import error
import git_refs
import git_superproject
from hooks import RepoHook
from project import ReviewableBranch
from repo_logging import RepoLogger
import repo_logging
from subcmds.sync import LocalSyncState
_DEFAULT_UNUSUAL_COMMIT_THRESHOLD = 5
logger = RepoLogger(__file__)
logger = repo_logging.RepoLogger(__file__)
class UploadExitError(SilentRepoExitError):
class UploadExitError(error.SilentRepoExitError):
"""Indicates that there is an upload command error requiring a sys exit."""
@@ -609,11 +607,11 @@ Gerrit Code Review: https://www.gerritcodereview.com/
)
full_dest = destination
if not full_dest.startswith(R_HEADS):
full_dest = R_HEADS + full_dest
if not full_dest.startswith(git_refs.R_HEADS):
full_dest = git_refs.R_HEADS + full_dest
full_revision = branch.project.revisionExpr
if not full_revision.startswith(R_HEADS):
full_revision = R_HEADS + full_revision
if not full_revision.startswith(git_refs.R_HEADS):
full_revision = git_refs.R_HEADS + full_revision
# If the merge branch of the local branch is different from
# the project's revision AND destination, this might not be
@@ -661,6 +659,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
push_options=push_options,
patchset_description=opt.patchset_description,
git_event_log=self.git_event_log,
quiet=opt.quiet,
)
branch.uploaded = True
@@ -671,15 +670,17 @@ Gerrit Code Review: https://www.gerritcodereview.com/
for branch in todo:
try:
self._UploadBranch(opt, branch, people)
except (UploadError, GitError) as e:
except (error.UploadError, error.GitError) as e:
self.git_event_log.ErrorEvent(f"upload error: {e}")
branch.error = e
aggregate_errors.append(e)
branch.uploaded = False
have_errors = True
print(file=sys.stderr)
print("-" * 70, file=sys.stderr)
quiet = getattr(opt, "quiet", False)
if not quiet or have_errors:
print(file=sys.stderr)
print("-" * 70, file=sys.stderr)
if have_errors:
for branch in todo:
@@ -700,17 +701,18 @@ Gerrit Code Review: https://www.gerritcodereview.com/
)
print()
for branch in todo:
if branch.uploaded:
print(
"[OK ] %-15s %s"
% (
branch.project.RelPath(local=opt.this_manifest_only)
+ "/",
branch.name,
),
file=sys.stderr,
)
if not quiet:
for branch in todo:
if branch.uploaded:
print(
"[OK ] %-15s %s"
% (
branch.project.RelPath(local=opt.this_manifest_only)
+ "/",
branch.name,
),
file=sys.stderr,
)
if have_errors:
raise UploadExitError(aggregate_errors=aggregate_errors)
@@ -742,6 +744,22 @@ Gerrit Code Review: https://www.gerritcodereview.com/
avail = project.GetUploadableBranches(opt.branch)
return (project_idx, avail, cbr)
def ValidateOptions(self, opt: optparse.Values, args: List[str]) -> None:
if opt.fix:
if opt.ignore_hooks:
self.OptionParser.error(
"--fix and --ignore-hooks are mutually exclusive options: "
"with --ignore-hooks, the original commits are uploaded, "
"so fixups would only modify the working tree and would "
"not be part of the upload"
)
if opt.bypass_hooks:
self.OptionParser.error(
"--fix and --no-verify are mutually exclusive options: "
"--no-verify skips the pre-upload hook, so --fix would "
"have no effect"
)
def Execute(self, opt, args):
projects = self.GetProjects(
args, all_manifests=not opt.this_manifest_only
+10 -13
View File
@@ -18,11 +18,8 @@ from typing import Any, Tuple
from command import Command
from command import MirrorSafeCommand
from git_command import git
from git_command import git_require
from git_command import RepoSourceVersion
from git_command import user_agent
from git_refs import HEAD
import git_command
import git_refs
from wrapper import Wrapper
@@ -39,17 +36,17 @@ class Version(Command, MirrorSafeCommand):
@staticmethod
def _RepoVersion(project: Any) -> Tuple[str, str]:
"""Return repo's describe string and commit date."""
if git_require((2, 32, 0)):
if git_command.git_require((2, 32, 0)):
output = project.bare_git.log(
"-1", "--format=%(describe)%n%cD", HEAD
"-1", "--format=%(describe)%n%cD", git_refs.HEAD
)
description, commit_date = output.rstrip("\n").split("\n", 1)
if description:
return description, commit_date
return (
project.bare_git.describe(HEAD),
project.bare_git.log("-1", "--format=%cD", HEAD),
project.bare_git.describe(git_refs.HEAD),
project.bare_git.log("-1", "--format=%cD", git_refs.HEAD),
)
def Execute(self, opt, args):
@@ -58,7 +55,7 @@ class Version(Command, MirrorSafeCommand):
branch = rp.GetBranch("default")
# These might not be the same. Report them both.
src_ver = RepoSourceVersion()
src_ver = git_command.RepoSourceVersion()
rp_ver, commit_date = self._RepoVersion(rp)
print(f"repo version {rp_ver}")
print(f" (from {rem.url})")
@@ -72,9 +69,9 @@ class Version(Command, MirrorSafeCommand):
if src_ver != rp_ver:
print(f" (currently at {src_ver})")
print(f"repo User-Agent {user_agent.repo}")
print(f"git {git.version_tuple().full}")
print(f"git User-Agent {user_agent.git}")
print(f"repo User-Agent {git_command.user_agent.repo}")
print(f"git {git_command.git.version_tuple().full}")
print(f"git User-Agent {git_command.user_agent.git}")
print(f"Python {sys.version}")
uname = platform.uname()
if sys.version_info.major < 3:
+3 -4
View File
@@ -17,13 +17,12 @@ import sys
from typing import List
from command import Command
from error import GitError
from error import RepoExitError
import error
import platform_utils
from project import DeleteWorktreeError
class Error(RepoExitError):
class Error(error.RepoExitError):
"""Exit error when wipe command fails."""
@@ -101,7 +100,7 @@ Examples:
try:
if project.HasChanges():
uncommitted_projects.append(project.name)
except GitError:
except error.GitError:
uncommitted_projects.append(f"{project.name} (corrupted)")
users = objdir_map.get(project.objdir, {project.relpath})
+2
View File
@@ -14,6 +14,8 @@ should be written using [pytest] only.
[subcmds/forall.py](../subcmds/forall.py) module.
* [conftest.py](./conftest.py): Custom pytest fixtures for sharing.
* [utils_for_test.py](./utils_for_test.py): Helpers for sharing in tests.
* [e2e/](./e2e/): Hermetic end-to-end tests that run the real `repo` against
a local git server.
[pytest]: https://pytest.org/
+36
View File
@@ -15,6 +15,7 @@
"""Common fixtures for pytests."""
import pathlib
from typing import Any, Generator
import pytest
@@ -23,9 +24,44 @@ import platform_utils
import repo_trace
def pytest_addoption(parser: pytest.Parser) -> None:
parser.addoption(
"--fail-on-skip",
action="store_true",
help="fail skipped tests (run_tests sets this in CQ)",
)
@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(
item: pytest.Item,
) -> Generator[None, Any, None]:
"""Fail skipped tests with --fail-on-skip.
A skip in CQ usually means the builder is missing a dependency (e.g. a new
enough git), which would otherwise silently drop coverage. Tests that
should not run in CQ are deselected via the `skip_cq` marker instead.
"""
outcome = yield
report = outcome.get_result()
if (
item.config.getoption("fail_on_skip")
and report.skipped
# xfail outcomes are also reported as skipped.
and not hasattr(report, "wasxfail")
):
reason = report.longrepr
if isinstance(reason, tuple):
# (path, lineno, reason)
reason = reason[2]
report.outcome = "failed"
report.longrepr = f"Not allowed with --fail-on-skip: {reason}"
@pytest.fixture(autouse=True)
def disable_repo_trace(tmp_path):
"""Set an environment marker to relax certain strict checks for test code.""" # noqa: E501
repo_trace.SetTrace(False)
repo_trace._TRACE_FILE = str(tmp_path / "TRACE_FILE_from_test")
+37
View File
@@ -0,0 +1,37 @@
# Repo End-to-End Tests
These tests run the real `repo` launcher & code (as subprocesses) against a
local git smart HTTP server. They are hermetic: everything is served from
`127.0.0.1` out of temporary directories, and the user's git config &
environment are ignored.
They run as part of `./run_tests` by default, after the unit tests.
## Running
```sh
# Everything (unit + e2e).
./run_tests
# Only the e2e tests.
./run_tests -m e2e
# Everything but the e2e tests.
./run_tests -m 'not e2e'
```
The suite is skipped automatically when `git http-backend` isn't
available, when git is older than 2.28 (need default branch support and
superproject support), or when not run from a git checkout.
## Layout
* `harness.py`: the git server, server-side repos & the repo client.
* `test_init_sync.py`: core `repo init` & `repo sync` options.
* `test_partial_clone.py`: partial clones, incl. the standard Android
configuration from go/repo-init & go/sync.
* `test_sync_modes.py`: `sync -n`/`-l` & the other sync flags.
* `test_groups.py`: manifest groups.
* `test_failures.py`: failure paths (missing projects, dirty trees, ...).
* `test_workflows.py`: multi-step workflows.
* `test_manifest_regressions.py`: manifests that broke users in the past.
+136
View File
@@ -0,0 +1,136 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Fixtures for the hermetic end-to-end tests.
Tests normally only need the `host` fixture (to create server repos) and the
`client` fixture (to run repo). Override the `git_defaults` fixture via
parametrize to run under Git 3 defaults.
"""
import itertools
from pathlib import Path
from typing import Dict, Iterator, List
import harness
import pytest
# Run before pytest's own `-m` deselection so `-m e2e` & `-m 'not e2e'` see the
# marker.
@pytest.hookimpl(tryfirst=True)
def pytest_collection_modifyitems(items: List[pytest.Item]) -> None:
"""Mark tests in this directory as `e2e` & run them after unit tests.
Note: pytest passes every collected item (not just this directory's), so
this reorders the whole session.
"""
this_dir = Path(__file__).resolve().parent
for item in items:
if this_dir in item.path.resolve().parents:
item.add_marker(pytest.mark.e2e)
# Stable sort: unit tests keep their order & come first, then E2E tests.
items.sort(key=lambda item: item.get_closest_marker("e2e") is not None)
@pytest.fixture(scope="session", autouse=True)
def _require_e2e_support() -> None:
"""Skip the suite when the host can't run it."""
# The code under test is published from the git checkout (e.g. this isn't
# the case when running from an unpacked sdist).
if not (harness.ROOT_DIR / ".git").exists():
pytest.skip("E2E tests must run from a git checkout")
if not harness.has_http_backend():
pytest.skip("git http-backend is not available")
if harness.git_version() < (2, 28, 0):
pytest.skip("E2E tests require git 2.28+")
@pytest.fixture(scope="session")
def git_server(
tmp_path_factory: pytest.TempPathFactory,
) -> Iterator[harness.GitHttpServer]:
"""A git smart HTTP server shared by all tests in the session."""
with harness.serve(tmp_path_factory.mktemp("git-server")) as server:
yield server
@pytest.fixture(scope="session")
def launcher(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""A standalone copy of the repo launcher."""
return harness.install_launcher(tmp_path_factory.mktemp("bin"))
@pytest.fixture(scope="session")
def repo_source(git_server: harness.GitHttpServer) -> harness.ServerRepo:
"""The code under test, published as a repo on `git_server`."""
return harness.publish_repo_source(
git_server.root / "git-repo.git", f"{git_server.url}/git-repo"
)
NAMESPACE_IDS = itertools.count()
@pytest.fixture
def host(git_server: harness.GitHttpServer) -> harness.GitHost:
"""A per-test namespace for creating repos on the server."""
return harness.GitHost(git_server, f"t{next(NAMESPACE_IDS)}")
@pytest.fixture
def git_defaults() -> str:
"""Client git defaults to emulate; a key of harness.GIT_DEFAULTS."""
return "git2"
@pytest.fixture
def client_env(tmp_path: Path, git_defaults: str) -> Dict[str, str]:
"""Isolated environment for running repo & git as a client."""
home = tmp_path / "home"
home.mkdir()
settings = {
"user.name": "E2E User",
"user.email": "user@example.com",
"color.ui": "false",
# Don't let background maintenance race with assertions.
"maintenance.auto": "false",
"gc.auto": "0",
}
if git_defaults == "git3" and harness.git_version() < (2, 47, 0):
# init.defaultRefFormat & init.defaultObjectFormat landed in git 2.47.
pytest.skip("git is too old to emulate Git 3 defaults")
settings.update(harness.GIT_DEFAULTS[git_defaults])
harness.write_config(home / ".gitconfig", settings)
# Python resolves `~` via USERPROFILE on Windows. The ceiling keeps git
# from finding a repo above the test dir (e.g. with `--basetemp=./tmp`).
return dict(
harness.clean_environ(),
HOME=str(home),
USERPROFILE=str(home),
GIT_CEILING_DIRECTORIES=str(tmp_path),
)
@pytest.fixture
def client(
tmp_path: Path,
launcher: Path,
repo_source: harness.ServerRepo,
client_env: Dict[str, str],
) -> harness.RepoClient:
"""A repo client checkout directory that uses the code under test."""
return harness.RepoClient(
tmp_path / "client", launcher, client_env, repo_source.url
)
+1232
View File
File diff suppressed because it is too large. Load diff
+195
View File
@@ -0,0 +1,195 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""E2E tests for failure paths: sync must fail loudly & leave a usable tree.
Note: repo retries failed fetches once after a few seconds, so each failing
fetch costs about 5s.
"""
from typing import List
from harness import GitHost
from harness import manifest_xml
from harness import RepoClient
import pytest
@pytest.mark.parametrize(
"sync_args",
[
pytest.param(["-j1"], id="interleaved"),
pytest.param(["-j1", "--fail-fast"], id="fail-fast"),
],
)
def test_missing_project(
host: GitHost, client: RepoClient, sync_args: List[str]
) -> None:
"""A project missing on the server fails the sync with a useful error.
By default the other projects are still synced & creating the missing repo
lets the next sync recover; with `--fail-fast` the sync stops at the first
failure without contacting later repos.
"""
host.create_repo("platform/foo", {"f": "foo"})
host.create_repo("platform/bar", {"f": "bar"})
# Paths sort the missing project first, so `-j1` reaches it first.
host.create_manifest(
manifest_xml(
'<project name="platform/missing" path="missing" />',
'<project name="platform/foo" path="present-foo" />',
'<project name="platform/bar" path="present-bar" />',
)
)
client.init(host.manifest_url)
with host.watch_requests() as requests:
result = client.sync(*sync_args, check=False)
assert result.returncode != 0
# The failing fetch & git's reason are reported either way.
assert "on platform/missing failed" in result.output
assert "not found" in result.output
assert not (client.top / "missing").exists()
# Repo requested the missing project (it's a 404, not a client bug).
assert {r.status for r in host.requests_for("platform/missing")} == {404}
if "--fail-fast" in sync_args:
assert "SyncFailFastError" in result.stderr
assert client.checkouts() == set()
assert host.requested_repos(requests) == {
"platform/manifest",
"platform/missing",
}
return
assert client.checkouts() == {"present-bar", "present-foo"}
assert "error: Unable to fully sync the tree" in result.stderr
assert "Failing repos (network):\nmissing" in result.stderr
assert 'Try re-running with "-j1 --fail-fast"' in result.stderr
# Once the missing project exists on the server, the next sync recovers.
host.create_repo("platform/missing", {"f": "missing"})
client.sync(*sync_args)
assert client.checkouts() == {"missing", "present-bar", "present-foo"}
assert (client.top / "missing/f").read_text() == "missing"
@pytest.mark.parametrize(
"revision,error",
[
(
"nonexistent",
"couldn't find remote ref refs/heads/nonexistent",
),
# A pinned commit that was never pushed.
("1" * 40, "1" * 40),
],
ids=["branch", "commit"],
)
def test_missing_project_revision(
host: GitHost, client: RepoClient, revision: str, error: str
) -> None:
"""A manifest revision that doesn't exist on the server fails the sync."""
foo = host.create_repo("platform/foo", {"f": "foo"})
host.create_manifest(
manifest_xml(
f'<project name="platform/foo" path="foo" revision="{revision}" />',
'<project name="platform/foo" path="ok" />',
)
)
client.init(host.manifest_url)
result = client.sync(check=False)
assert result.returncode != 0
assert "error: Unable to fully sync the tree" in result.stderr
assert "Failing repos (network):\nfoo" in result.stderr
assert error in result.stderr
assert not (client.top / "foo").exists()
assert client.head("ok") == foo.rev_parse("main")
# Fixing the missing branch on the server recovers on the next sync.
if revision == "nonexistent":
fixed = foo.commit({"f": "fixed"}, branch="nonexistent")
client.sync()
assert client.head("foo") == fixed
def test_missing_manifest_branch(host: GitHost, client: RepoClient) -> None:
"""`init -b` with a branch the manifest repo doesn't have fails cleanly.
The half-initialized manifest checkout is removed so a corrected `init`
starts from scratch.
"""
host.create_repo("platform/foo", {"f": "foo"})
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
result = client.init(host.manifest_url, "-b", "nonexistent", check=False)
assert result.returncode != 0
assert "couldn't find remote ref refs/heads/nonexistent" in result.output
assert "fatal: cannot obtain manifest" in result.stderr
assert not (client.repodir / "manifests.git").exists()
client.init(host.manifest_url, "-b", "main")
client.sync()
assert (client.top / "platform/foo/f").read_text() == "foo"
@pytest.mark.parametrize("on_branch", [False, True])
def test_dirty_worktree_blocks_sync(
host: GitHost, client: RepoClient, on_branch: bool
) -> None:
"""Uncommitted changes that conflict with upstream fail the checkout.
The local change is preserved and the fetched objects are kept, so the
user can resolve it and run `repo sync -l`. On a detached HEAD,
`--force-checkout` discards the local change instead.
"""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_manifest(
manifest_xml('<project name="platform/foo" path="foo" />')
)
client.init(host.manifest_url)
client.sync()
old = foo.rev_parse("main")
if on_branch:
client.run("start", "topic", "--all")
(client.top / "foo/f").write_text("local")
latest = foo.commit({"f": "2"})
result = client.sync(check=False)
assert result.returncode != 0
assert "error: Checking out local projects failed" in result.stderr
assert "Failing repos (checkout):\nfoo" in result.stderr
if on_branch:
assert "contains uncommitted changes" in result.stderr
else:
assert f"platform/foo checkout {latest}" in result.stderr
assert (client.top / "foo/f").read_text() == "local"
assert client.head("foo") == old
assert client.git("foo", "rev-parse", "refs/remotes/origin/main") == latest
if not on_branch:
client.sync("--force-checkout")
else:
# Discarding the edit & running `sync -l` recovers without touching
# the network.
client.git("foo", "checkout", "--", "f")
with host.watch_requests() as requests, host.offline():
client.sync("-l")
assert requests == []
assert client.head("foo") == latest
assert (client.top / "foo/f").read_text() == "2"
+201
View File
@@ -0,0 +1,201 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""E2E tests for manifest groups beyond `repo init -g`."""
import platform
from typing import Dict, List, Optional, Set
from harness import GitHost
from harness import Gitlink
from harness import manifest_xml
from harness import RepoClient
from harness import ServerRepo
import pytest
# `a` & `b` are in the default group; `c` & `d` are not.
GROUPED_MANIFEST = manifest_xml(
'<project name="platform/a" path="a" />',
'<project name="platform/b" path="b" groups="g1" />',
'<project name="platform/c" path="c" groups="g2,notdefault" />',
'<project name="platform/d" path="d" groups="notdefault" />',
)
def create_grouped_projects(host: GitHost) -> Dict[str, ServerRepo]:
"""Create the projects of `GROUPED_MANIFEST` & the manifest itself."""
repos = {}
for name in "abcd":
repos[name] = host.create_repo(f"platform/{name}", {"f": name})
host.create_manifest(GROUPED_MANIFEST)
return repos
def listed(client: RepoClient, *args: str) -> Set[str]:
"""The project paths printed by `repo list <args>`."""
return set(client.list_projects(*args))
def test_reinit_groups(host: GitHost, client: RepoClient) -> None:
"""Re-running `init -g` adds & removes projects on the next sync."""
create_grouped_projects(host)
this_platform = f"platform-{platform.system().lower()}"
client.init(host.manifest_url)
client.sync()
assert client.checkouts() == {"a", "b"}
assert client.config(".repo/manifests", "manifest.groups") is None
client.init(host.manifest_url, "-g", "default,g2")
client.sync()
assert client.checkouts() == {"a", "b", "c"}
assert (
client.config(".repo/manifests", "manifest.groups")
== f"default,g2,{this_platform}"
)
client.init(host.manifest_url, "-g", "g2")
client.sync()
assert client.checkouts() == {"c"}
assert client.list_projects() == {"c": "platform/c"}
assert (
client.config(".repo/manifests", "manifest.groups")
== f"g2,{this_platform}"
)
# The object dirs of removed projects are kept for a later re-add.
assert (client.repodir / "project-objects/platform/a.git").is_dir()
# Back to the default groups.
client.init(host.manifest_url, "-g", "default")
client.sync()
assert client.checkouts() == {"a", "b"}
assert (client.top / "a/f").read_text() == "a"
@pytest.mark.parametrize(
"init_args,expected",
[
([], {"base"}),
(["-g", "default,extra"], {"base", "extra", "extra2"}),
(["-g", "default,extra,-g2"], {"base", "extra"}),
(["-g", "g2"], {"extra2"}),
],
)
def test_include_groups(
host: GitHost, client: RepoClient, init_args: List[str], expected: Set[str]
) -> None:
"""`<include groups=...>` adds its groups to every included project."""
for name in ("base", "extra", "extra2"):
host.create_repo(f"platform/{name}", {"f": name})
host.create_manifest(
{
"default.xml": manifest_xml(
'<project name="platform/base" path="base" />',
'<include name="extra.xml" groups="notdefault,extra" />',
),
"extra.xml": (
"<manifest>\n"
' <project name="platform/extra" path="extra" />\n'
' <project name="platform/extra2" path="extra2" '
'groups="g2" />\n'
"</manifest>\n"
),
}
)
client.init(host.manifest_url, *init_args)
client.sync()
assert client.checkouts() == expected
assert set(client.list_projects()) == expected
# The include's groups are merged with the project's own.
assert listed(client, "-a", "-g", "extra") == {"extra", "extra2"}
assert listed(client, "-a", "-g", "g2") == {"extra2"}
@pytest.mark.parametrize(
"init_args,platforms",
[
# None means the platform the test runs on.
([], None),
(["--platform=all"], {"linux", "darwin", "windows"}),
(["--platform=none"], set()),
(["-p", "darwin"], {"darwin"}),
(["-g", "default,platform-linux", "--platform=none"], {"linux"}),
],
)
def test_platform_groups(
host: GitHost,
client: RepoClient,
init_args: List[str],
platforms: Optional[Set[str]],
) -> None:
"""`platform-*` groups follow the host OS unless `--platform` is given."""
names = ("linux", "darwin", "windows")
host.create_repo("platform/base", {"f": "base"})
for name in names:
host.create_repo(f"platform/{name}", {"f": name})
host.create_manifest(
manifest_xml(
'<project name="platform/base" path="base" />',
*(
f'<project name="platform/{name}" path="{name}" '
f'groups="notdefault,platform-{name}" />'
for name in names
),
)
)
if platforms is None:
platforms = {platform.system().lower()}
client.init(host.manifest_url, *init_args)
client.sync()
assert client.checkouts() == {"base"} | platforms
assert set(client.list_projects()) == {"base"} | platforms
def test_groups_with_superproject(host: GitHost, client: RepoClient) -> None:
"""Group selection applies on top of superproject pinning."""
pins = {}
for name in "ab":
repo = host.create_repo(f"platform/{name}", {"f": "v1"})
pins[name] = repo.rev_parse("main")
repo.commit({"f": "v2"})
host.create_repo(
"superproject", {name: Gitlink(rev) for name, rev in pins.items()}
)
host.create_manifest(
manifest_xml(
'<superproject name="superproject" />',
'<project name="platform/a" path="a" />',
'<project name="platform/b" path="b" groups="notdefault,g1" />',
)
)
client.init(host.manifest_url, "--use-superproject")
assert client.sync().stderr == ""
assert client.checkouts() == {"a"}
assert client.head("a") == pins["a"]
assert host.requests_for("platform/b") == []
client.init(host.manifest_url, "-g", "all")
assert client.sync().stderr == ""
assert client.checkouts() == {"a", "b"}
assert client.head("b") == pins["b"]
assert (client.top / "b/f").read_text() == "v1"
+209
View File
@@ -0,0 +1,209 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests for the E2E harness helpers themselves."""
import http.client
from pathlib import Path
from typing import Mapping, Tuple
import urllib.parse
import harness
import pytest
def _http_request(
server: harness.GitHttpServer, method: str, path: str
) -> Tuple[int, bytes]:
"""Send a request to `server` & return the response status & body."""
url = urllib.parse.urlsplit(server.url)
conn = http.client.HTTPConnection(url.hostname, url.port, timeout=30)
try:
conn.request(method, path)
resp = conn.getresponse()
return resp.status, resp.read()
finally:
conn.close()
@pytest.mark.parametrize(
"key,value",
[
("core.bare", "true"),
('url.C:\\src\\#1;"x".insteadOf', 'C:\\dst\\ #2;"y"'),
("alias.multi", "a\nb"),
],
)
def test_write_config(tmp_path: Path, key: str, value: str) -> None:
"""Subsections & values round-trip through git's config parser."""
path = tmp_path / "config"
harness.write_config(path, {key: value})
got = harness.run_git("config", "--file", str(path), "--get", key)
assert got == value
@pytest.mark.parametrize(
"files",
[
{"foo/bar": "x", "foo": None},
{"foo": None, "foo/bar": "x"},
],
)
def test_commit_replace_file_with_dir(
tmp_path: Path, files: Mapping[str, harness.FileContent]
) -> None:
"""Deletions apply before additions regardless of dict order."""
repo = harness.ServerRepo.create(tmp_path / "repo.git", "unused")
repo.commit({"foo": "file"})
repo.commit(files)
assert repo.git("ls-tree", "-r", "--name-only", "main") == "foo/bar"
def test_snapshot_worktree_host_excludes(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""The developer's global excludes apply despite the isolated HOME."""
host_home = tmp_path / "host-home"
host_home.mkdir()
excludes = host_home / "excludes"
excludes.write_text("venv/\n")
# Set excludesFile explicitly so a system-wide one can't take precedence.
harness.write_config(
host_home / ".gitconfig", {"core.excludesFile": excludes.as_posix()}
)
monkeypatch.setattr(harness, "HOST_ENVIRON", {"HOME": str(host_home)})
root = tmp_path / "checkout"
harness.run_git("init", "--quiet", str(root))
(root / ".gitignore").write_text("*.pyc\n")
(root / "main.py").write_text("")
(root / "main.pyc").write_text("")
(root / "venv").mkdir()
(root / "venv" / "lib.py").write_text("")
assert sorted(harness._snapshot_worktree(root)) == [".gitignore", "main.py"]
@pytest.mark.parametrize(
"chunked,expect_continue",
[(False, False), (True, False), (True, True)],
)
def test_http_post_body(
git_server: harness.GitHttpServer, chunked: bool, expect_continue: bool
) -> None:
"""POST bodies reach git http-backend however curl encodes them."""
name = f"post-{int(chunked)}{int(expect_continue)}.git"
repo = harness.ServerRepo.create(git_server.root / name, "unused")
commit = repo.commit({"file": "x"})
# A protocol v2 ls-refs request: a command pkt-line & a flush-pkt.
body = b"0014command=ls-refs\n0000"
headers = {
"Content-Type": "application/x-git-upload-pack-request",
"Git-Protocol": "version=2",
}
if expect_continue:
headers["Expect"] = "100-continue"
url = urllib.parse.urlsplit(git_server.url)
conn = http.client.HTTPConnection(url.hostname, url.port, timeout=30)
try:
# http.client skips any interim `100 Continue` response.
conn.request(
"POST",
f"/{name}/git-upload-pack",
body=iter([body[:10], body[10:]]) if chunked else body,
headers=headers,
encode_chunked=chunked,
)
resp = conn.getresponse()
assert resp.status == 200
assert f"{commit} refs/heads/main".encode() in resp.read()
finally:
conn.close()
def test_static_files(git_server: harness.GitHttpServer) -> None:
"""Published files are served as-is, and nothing outside of them."""
host = harness.GitHost(git_server, "static-files")
host.create_repo("repo", {"f": "x"})
url = host.publish_file("dir/default.xml", "<manifest />\n")
path = urllib.parse.urlsplit(url).path
assert _http_request(git_server, "GET", path) == (200, b"<manifest />\n")
# Publishing again replaces the file.
assert host.publish_file("dir/default.xml", b"v2") == url
assert _http_request(git_server, "GET", path) == (200, b"v2")
static = f"/{host.prefix}/{harness.STATIC_DIR}"
for bad_path in (
f"{static}/missing.xml",
f"{static}/dir",
f"{static}/../repo.git/config",
f"{static}/%2e%2e/repo.git/config",
f"{static}//etc/passwd",
):
assert _http_request(git_server, "GET", bad_path)[0] == 404, bad_path
assert _http_request(git_server, "POST", path)[0] == 404
def test_offline(git_server: harness.GitHttpServer) -> None:
"""Requests fail while a namespace is offline, and work again after."""
host = harness.GitHost(git_server, "offline")
repo = host.create_repo("platform/repo", {"f": "x"})
other = harness.GitHost(git_server, "online")
other_repo = other.create_repo("repo", {"f": "x"})
with host.offline():
with pytest.raises(AssertionError, match="503"):
harness.run_git("ls-remote", repo.url)
# Other namespaces are unaffected.
assert "refs/heads/main" in harness.run_git("ls-remote", other_repo.url)
assert {r.status for r in host.requests()} == {503}
# The outage ends even if the context exits with an exception.
with pytest.raises(RuntimeError):
with host.offline():
raise RuntimeError("boom")
assert "refs/heads/main" in harness.run_git("ls-remote", repo.url)
def test_requested_repos(git_server: harness.GitHttpServer) -> None:
"""`requested_repos()` & `watch_requests()` track repo requests."""
host = harness.GitHost(git_server, "requested")
nested = host.create_repo("platform/nested", {"f": "x"})
top = host.create_repo("top", {"f": "x"})
host.publish_file("file.xml", "x")
harness.run_git("ls-remote", nested.url)
with host.watch_requests() as watched:
harness.run_git("ls-remote", top.url)
_http_request(git_server, "GET", f"/{host.prefix}/top/clone.bundle")
_http_request(git_server, "GET", f"/{host.prefix}/_static/file.xml")
assert host.requested_repos() == {"platform/nested", "top"}
assert host.requested_repos(watched) == {"top"}
assert host.requests_for("platform/nested", watched) == []
def test_client_config(tmp_path: Path) -> None:
"""`RepoClient.config()` tells unset keys apart from errors."""
# Don't let git find a repo above tmp_path (e.g. with --basetemp).
env = dict(harness.clean_environ(), GIT_CEILING_DIRECTORIES=str(tmp_path))
client = harness.RepoClient(tmp_path / "client", harness.LAUNCHER, env, "")
repo = client.top / "repo"
harness.run_git("init", "-q", str(repo))
harness.run_git("config", "-f", str(repo / ".git/config"), "a.key", "val")
(client.top / "not-a-repo").mkdir()
assert client.config("repo", "a.key") == "val"
assert client.config("repo", "a.unset") is None
with pytest.raises(AssertionError, match="failed in not-a-repo"):
client.config("not-a-repo", "a.key")
+355
View File
@@ -0,0 +1,355 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""E2E tests for the core `repo init` & `repo sync` options."""
from pathlib import Path
from typing import List, Set
from harness import GIT_DEFAULTS_PARAMS
from harness import GitHost
from harness import manifest_xml
from harness import move_aside
from harness import RepoClient
import pytest
@pytest.mark.parametrize("git_defaults", GIT_DEFAULTS_PARAMS)
def test_init_sync_basic(host: GitHost, client: RepoClient) -> None:
"""The basic `repo init && repo sync` flow."""
foo = host.create_repo("platform/foo", {"foo.txt": "foo"})
host.create_repo("platform/bar", {"bar.txt": "bar"})
host.create_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
'<project name="platform/bar" path="sub/bar" />',
)
)
client.init(host.manifest_url)
client.sync()
assert (client.top / "foo" / "foo.txt").read_text() == "foo"
assert (client.top / "sub" / "bar" / "bar.txt").read_text() == "bar"
assert client.head("foo") == foo.rev_parse("main")
assert client.list_projects() == {
"foo": "platform/foo",
"sub/bar": "platform/bar",
}
# The code under test is what got installed & is running.
assert "v99.0-e2e" in client.run("version").stdout
@pytest.mark.parametrize(
"init_args,sync_args,manifest_refs,project_refs",
[
# `init -c` is the default for the manifest project.
([], [], {"main"}, {"main", "other"}),
(["--no-current-branch"], [], {"main", "other"}, {"main", "other"}),
([], ["-c"], {"main"}, {"main"}),
],
)
def test_current_branch(
host: GitHost,
client: RepoClient,
init_args: List[str],
sync_args: List[str],
manifest_refs: Set[str],
project_refs: Set[str],
) -> None:
"""`init` limits the manifest fetch; `sync -c` limits projects."""
foo = host.create_repo("platform/foo", {"foo.txt": "main"})
foo.commit({"foo.txt": "other"}, branch="other")
manifest = host.create_manifest(
manifest_xml('<project name="platform/foo" />')
)
manifest.commit({"README": "other"}, branch="other")
client.init(host.manifest_url, *init_args)
assert client.remote_branches(".repo/manifests") == manifest_refs
client.sync(*sync_args)
assert client.remote_branches("platform/foo") == project_refs
assert (client.top / "platform/foo/foo.txt").read_text() == "main"
@pytest.mark.parametrize(
"sync_args,expect_bundle", [([], True), (["--no-clone-bundle"], False)]
)
def test_clone_bundle(
host: GitHost,
client: RepoClient,
sync_args: List[str],
expect_bundle: bool,
) -> None:
"""repo bootstraps new projects from clone.bundle when available."""
foo = host.create_repo("platform/foo", {"foo.txt": "foo"})
# A commit that only the bundle has, to prove the bundle was applied.
bundle_only = foo.commit({"x": "bundle"}, branch="bundle-only")
foo.write_bundle()
foo.git("update-ref", "-d", "refs/heads/bundle-only")
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url)
client.sync(*sync_args)
bundle_gets = [
r
for r in host.requests()
if r.path.endswith("/platform/foo/clone.bundle")
]
assert [r.status for r in bundle_gets] == ([200] if expect_bundle else [])
assert client.has_object("platform/foo", bundle_only) == expect_bundle
assert client.head("platform/foo") == foo.rev_parse("main")
@pytest.mark.parametrize(
"init_args,project_attrs",
[
(["--depth=1"], ""),
([], 'clone-depth="1"'),
],
)
def test_shallow(
host: GitHost,
client: RepoClient,
init_args: List[str],
project_attrs: str,
) -> None:
"""Shallow clones via `--depth` and manifest `clone-depth`."""
foo = host.create_repo("platform/foo", {"foo.txt": "1"})
foo.commit({"foo.txt": "2"})
foo.commit({"foo.txt": "3"})
host.create_manifest(
manifest_xml(f'<project name="platform/foo" {project_attrs} />')
)
client.init(host.manifest_url, *init_args)
client.sync()
path = "platform/foo"
assert client.is_shallow(path)
assert client.git(path, "rev-list", "--count", "HEAD") == "1"
assert (client.top / path / "foo.txt").read_text() == "3"
@pytest.mark.parametrize(
"init_args,expected",
[
([], {"a", "b"}),
(["-g", "all"], {"a", "b", "c", "d"}),
(["-g", "g2"], {"c"}),
(["-g", "default,-g1"], {"a"}),
(["-g", "default,g2"], {"a", "b", "c"}),
(["-g", "name:platform/d"], {"d"}),
],
)
def test_groups(
host: GitHost, client: RepoClient, init_args: List[str], expected: Set[str]
) -> None:
"""Manifest groups select which projects get synced."""
for name in "abcd":
host.create_repo(f"platform/{name}", {"f": name})
host.create_manifest(
manifest_xml(
'<project name="platform/a" path="a" />',
'<project name="platform/b" path="b" groups="g1" />',
'<project name="platform/c" path="c" groups="g2,notdefault" />',
'<project name="platform/d" path="d" groups="notdefault" />',
)
)
client.init(host.manifest_url, *init_args)
client.sync()
assert set(client.list_projects()) == expected
assert client.checkouts() == expected
def test_local_manifests(host: GitHost, client: RepoClient) -> None:
""".repo/local_manifests/ can add, remove & extend projects."""
foo = host.create_repo("platform/foo", {"f": "main"})
stable = foo.commit({"f": "stable"}, branch="stable")
host.create_repo("platform/bar", {"f": "bar"})
host.create_repo("vendor/extra", {"f": "extra"})
host.create_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
'<project name="platform/bar" path="bar" />',
)
)
client.init(host.manifest_url)
local = client.repodir / "local_manifests"
local.mkdir()
(local / "local.xml").write_text(
"""<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remove-project name="platform/bar" />
<extend-project name="platform/foo" revision="stable" />
<project name="vendor/extra" path="extra" />
</manifest>
"""
)
client.sync()
assert client.list_projects() == {
"extra": "vendor/extra",
"foo": "platform/foo",
}
assert client.head("foo") == stable
assert not (client.top / "bar").exists()
def test_manifest_name(host: GitHost, client: RepoClient) -> None:
"""`-m` selects a manifest file other than default.xml."""
host.create_repo("platform/foo", {"f": "foo"})
host.create_repo("platform/bar", {"f": "bar"})
host.create_manifest(
{
"default.xml": manifest_xml('<project name="platform/foo" />'),
"other.xml": manifest_xml('<project name="platform/bar" />'),
}
)
client.init(host.manifest_url, "-m", "other.xml")
client.sync()
assert client.list_projects() == {"platform/bar": "platform/bar"}
@pytest.mark.parametrize("dissociate", [False, True])
def test_reference_mirror(
host: GitHost, client: RepoClient, dissociate: bool
) -> None:
"""`init --reference` borrows objects from a local `--mirror` checkout.
With `--dissociate` the objects are copied so the mirror can go away.
"""
foo = host.create_repo("platform/foo", {"f": "foo"})
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
mirror = RepoClient(
client.top.parent / "mirror",
client.launcher,
client.env,
client.repo_url,
)
mirror.init(host.manifest_url, "--mirror")
mirror.sync()
if dissociate:
# Dissociating the manifest repo itself currently fails (its
# preciousObjects setting blocks `git repack -a -d`), so only reference
# the mirror's projects.
move_aside(mirror.repodir / "manifests.git")
init_args = ["--reference", str(mirror.top)]
if dissociate:
init_args.append("--dissociate")
client.init(host.manifest_url, *init_args)
client.sync()
path = "platform/foo"
assert client.head(path) == foo.rev_parse("main")
assert (client.top / path / "f").read_text() == "foo"
assert client.has_object(path, foo.rev_parse("main"))
objdir = client.repodir / "project-objects/platform/foo.git"
alternates = objdir / "objects/info/alternates"
assert client.config(".repo/manifests", "repo.reference") == str(mirror.top)
if dissociate:
assert not alternates.exists()
assert client.config(".repo/manifests", "repo.dissociate") == "true"
else:
expected = mirror.top / "platform/foo.git/objects"
assert Path(alternates.read_text().strip()) == expected
# Nothing had to be downloaded or copied into the local object dir.
counts = client.git(".", f"--git-dir={objdir}", "count-objects", "-v")
assert "count: 0\n" in counts
assert "in-pack: 0\n" in counts
# Updates still come from the server (the mirror isn't synced).
latest = foo.commit({"f": "2"})
client.sync()
assert client.head(path) == latest
# If the mirror goes away, `--dissociate` checkouts still have all
# historical objects, whereas shared checkouts lost the borrowed ones.
move_aside(mirror.top)
if dissociate:
assert client.git(path, "rev-list", "--count", "HEAD") == "2"
client.git(path, "rev-list", "--objects", "HEAD")
else:
with pytest.raises(AssertionError, match="rev-list"):
client.git(path, "rev-list", "--objects", "HEAD")
def test_standalone_manifest(host: GitHost, client: RepoClient) -> None:
"""`init --standalone-manifest` fetches a static manifest file over HTTP.
There's no manifest checkout to update, so syncs only fetch the projects.
"""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_repo("platform/bar", {"f": "bar"})
xml = manifest_xml('<project name="platform/foo" />', fetch=host.url)
manifest_url = host.publish_file("default.xml", xml)
client.init(manifest_url, "--standalone-manifest")
client.sync()
path = "platform/foo"
assert client.head(path) == foo.rev_parse("main")
assert (client.repodir / "manifests/default.xml").read_text() == xml
assert (
client.config(".repo/manifests.git", "manifest.standalone")
== manifest_url
)
# Updating the static file on the server does not affect `repo sync`,
# which only fetches the project.
host.publish_file(
"default.xml",
manifest_xml(
'<project name="platform/foo" />',
'<project name="platform/bar" />',
fetch=host.url,
),
)
latest = foo.commit({"f": "2"})
with host.watch_requests() as requests:
client.sync()
assert client.head(path) == latest
assert client.list_projects() == {"platform/foo": "platform/foo"}
assert requests == host.requests_for("platform/foo", requests)
# Re-initializing without a manifest URL isn't allowed.
result = client.run("init", "--no-repo-verify", check=False)
assert result.returncode != 0
assert "cannot be re-initialized without --manifest-url/-u" in result.stderr
def test_manifest_depth(host: GitHost, client: RepoClient) -> None:
"""`init --manifest-depth` shallow-clones the manifest repo only."""
foo = host.create_repo("platform/foo", {"f": "1"})
foo.commit({"f": "2"})
host.create_manifest(manifest_xml())
host.update_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url, "--manifest-depth=1")
manifests = ".repo/manifests"
assert client.is_shallow(manifests)
assert client.git(manifests, "rev-list", "--count", "HEAD") == "1"
# `--nmu` skips updating the manifest project, keeping it shallow while
# syncing projects at full depth.
client.sync("--nmu")
assert client.is_shallow(manifests)
assert not client.is_shallow("platform/foo")
assert client.git("platform/foo", "rev-list", "--count", "HEAD") == "2"
+387
View File
@@ -0,0 +1,387 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""E2E regression tests for manifests that broke users in the past.
Each test names the commit and/or bug that fixed the original breakage.
"""
import os
from typing import List
from harness import GitHost
from harness import Gitlink
from harness import manifest_xml
from harness import RepoClient
import pytest
def test_empty_manifest(host: GitHost, client: RepoClient) -> None:
"""A manifest without projects syncs (616e314, b/377546300)."""
host.create_manifest(manifest_xml())
client.init(host.manifest_url)
client.sync()
assert client.list_projects() == {}
def test_local_extend_project_with_superproject(
host: GitHost, client: RepoClient
) -> None:
"""Local `extend-project` works with superproject overrides.
Regression test for 3c0e67b (b/470374343).
"""
foo = host.create_repo("platform/foo", {"f": "v1"})
pinned = foo.rev_parse("main")
host.create_repo("superproject", {"foo": Gitlink(pinned)})
foo.commit({"f": "v2"})
host.create_manifest(
manifest_xml(
'<superproject name="superproject" />',
'<project name="platform/foo" path="foo" />',
)
)
client.init(host.manifest_url, "--use-superproject")
local = client.repodir / "local_manifests"
local.mkdir()
(local / "local.xml").write_text(
"<manifest>\n"
' <extend-project name="platform/foo" groups="extra" />\n'
"</manifest>\n"
)
client.sync("--use-superproject")
assert client.head("foo") == pinned
# 3af9e2f fixed both the regular & the --optimized-fetch skip-fetch checks.
@pytest.mark.parametrize("sync_args", [[], ["--optimized-fetch"]])
def test_same_project_shallow_pinned_paths(
host: GitHost, client: RepoClient, sync_args: List[str]
) -> None:
"""One project at two paths pinned to SHAs with sync-c, one shallow.
The non-shallow path shares the shallow path's (depth-limited) object
store, so its fetch must not be skipped as already up to date, or its
history is left truncated.
Regression test for 3af9e2f (b/505072873).
"""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_manifest(manifest_xml())
def pin(rev: str) -> None:
host.update_manifest(
manifest_xml(
f'<project name="platform/foo" path="a" revision="{rev}" '
'clone-depth="1" />',
f'<project name="platform/foo" path="b" revision="{rev}" />',
default='remote="origin" revision="main" sync-c="true"',
)
)
pin(foo.commit({"f": "2"}))
client.init(host.manifest_url)
client.sync(*sync_args)
# Skip a commit so the shallow fetch alone would leave a gap in the shared
# history.
foo.commit({"f": "3"})
latest = foo.commit({"f": "4"})
pin(latest)
client.sync(*sync_args)
assert client.is_shallow("a")
for path in ("a", "b"):
assert client.head(path) == latest
# Fails to traverse the shared, truncated history if `path`'s fetch
# was skipped.
client.git(path, "rev-list", "HEAD")
def test_shallow_pinned_with_upstream(
host: GitHost, client: RepoClient
) -> None:
"""clone-depth + upstream + SHA revisions stay shallow.
The bug only triggered when the upstream branch was ahead of the pinned
SHA, so each pin leaves `main` one commit ahead.
Regression test for d9cc0a1 (b/427093249).
"""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_manifest(manifest_xml())
def pin(rev: str) -> None:
host.update_manifest(
manifest_xml(
f'<project name="platform/foo" revision="{rev}" '
'upstream="main" clone-depth="1" />'
)
)
path = "platform/foo"
pinned = foo.commit({"f": "2"})
foo.commit({"f": "2 ahead"})
pin(pinned)
client.init(host.manifest_url)
client.sync()
assert client.head(path) == pinned
assert client.git(path, "rev-list", "--count", "HEAD") == "1"
pinned = foo.commit({"f": "3"})
foo.commit({"f": "3 ahead"})
pin(pinned)
client.sync()
assert client.head(path) == pinned
assert client.git(path, "rev-list", "--count", "HEAD") == "1"
def test_pinned_revision_refreshes_upstream(
host: GitHost, client: RepoClient
) -> None:
"""Pinned SHAs already present locally still refresh the upstream ref.
Regression test for d27d682: when the pinned commit was already in the
local object store (e.g. prefetched), `sync -c` skipped the fetch as an
immutable revision, so the stale tracking ref of the upstream branch was
never updated.
"""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_manifest(manifest_xml())
def pin(rev: str) -> None:
host.update_manifest(
manifest_xml(
f'<project name="platform/foo" revision="{rev}" '
'upstream="main" />',
# The skip only happens when fetching the current branch only.
default='remote="origin" revision="main" sync-c="true"',
)
)
initial = foo.rev_parse("main")
pin(initial)
client.init(host.manifest_url)
client.sync()
path = "platform/foo"
second = foo.commit({"f": "2"})
latest = foo.commit({"f": "3"})
# Emulate a background prefetch that doesn't touch the tracking refs.
client.git(
path, "fetch", "-q", "--refmap=", "origin", "+main:refs/prefetch/main"
)
assert client.has_object(path, second)
assert client.git(path, "rev-parse", "refs/remotes/origin/main") == initial
pin(second)
client.sync()
assert client.head(path) == second
assert client.git(path, "rev-parse", "refs/remotes/origin/main") == latest
def test_nested_projects(host: GitHost, client: RepoClient) -> None:
"""Nested project paths are checked out and removed in the right order.
Regression test for 4679022 (b/325119758).
"""
foo = host.create_repo("platform/foo", {"f": "foo", "bar/f": "old"})
host.create_repo("platform/bar", {"f": "foo/bar"})
host.create_repo("platform/foo-bar", {"f": "foo-bar"})
# aaa shares foo/bar's project, so foo/bar is grouped with it & checked out
# early unless the checkout order puts it after its parent foo.
host.create_manifest(
manifest_xml(
'<project name="platform/bar" path="aaa" />',
'<project name="platform/foo" path="foo" />',
'<project name="platform/foo-bar" path="foo-bar" />',
)
)
client.init(host.manifest_url)
client.sync()
# Move foo's bar/f into the new nested project foo/bar.
foo.commit({"bar/f": None})
host.update_manifest(
manifest_xml(
'<project name="platform/bar" path="aaa" />',
'<project name="platform/foo" path="foo" />',
'<project name="platform/bar" path="foo/bar" />',
'<project name="platform/foo-bar" path="foo-bar" />',
)
)
client.sync("-j1")
for path in ("foo", "foo/bar", "foo-bar"):
assert (client.top / path / "f").read_text() == path
assert client.git("foo/bar", "status", "--porcelain") == ""
host.update_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
'<project name="platform/foo-bar" path="foo-bar" />',
)
)
client.sync()
assert set(client.list_projects()) == {"foo", "foo-bar"}
assert not (client.top / "foo/bar").exists()
assert (client.top / "foo/f").read_text() == "foo"
def test_copyfile_linkfile(host: GitHost, client: RepoClient) -> None:
"""copyfile and linkfile are created, updated and cleaned up.
linkfile symlinks are relative (0184dcc) so trees can be moved.
"""
foo = host.create_repo(
"platform/foo", {"copy.txt": "v1", "dir/link.txt": "link"}
)
host.create_manifest(
manifest_xml(
'<project name="platform/foo" path="foo">'
'<copyfile src="copy.txt" dest="copied.txt" />'
'<linkfile src="dir/link.txt" dest="links/link.txt" />'
"</project>"
)
)
client.init(host.manifest_url)
client.sync()
link = client.top / "links/link.txt"
# Python <3.8 on Windows rejects Path objects here.
target = os.readlink(str(link))
assert target == os.path.join("..", "foo", "dir", "link.txt")
assert link.read_text() == "link"
assert (client.top / "copied.txt").read_text() == "v1"
foo.commit({"copy.txt": "v2"})
host.update_manifest(
manifest_xml(
'<project name="platform/foo" path="foo">'
'<copyfile src="copy.txt" dest="copied.txt" />'
"</project>"
)
)
client.sync()
assert (client.top / "copied.txt").read_text() == "v2"
assert not os.path.lexists(link)
@pytest.mark.parametrize("sync_args", [[], ["--no-interleaved"]])
def test_submodules(
host: GitHost, client: RepoClient, sync_args: List[str]
) -> None:
"""sync-s checks out & updates submodules, incl. for repeated projects.
Regression tests for 4623264 (b/444366154, `git submodule init` in
interleaved sync), 1b4e7a0 (the same repo checked out at multiple paths),
and 3a6e25a/3f087a8 (b/550074864, submodules follow their parent's new
revision in the same interleaved & phased sync).
"""
sub = host.create_repo("platform/sub", {"s": "1"})
gitmodules = '[submodule "sub"]\n\tpath = sub\n\turl = ../sub\n'
foo = host.create_repo(
"platform/foo",
{".gitmodules": gitmodules, "sub": Gitlink(sub.rev_parse("main"))},
)
paths = ("a", "b")
# Track a branch, as gitlinks are only stale (b/550074864) when the
# parent's revision resolves before it's fetched (unlike new pinned SHAs).
# With sync-c, phased syncs skip fetching the stale (present) revision.
host.create_manifest(
manifest_xml(
*(
f'<project name="platform/foo" path="{p}" sync-s="true" '
'sync-c="true" />'
for p in paths
)
)
)
client.init(host.manifest_url)
client.sync(*sync_args)
for path in paths:
assert (client.top / path / "sub/s").read_text() == "1"
assert client.git(path, "config", "--get", "submodule.sub.url")
latest = foo.commit({"sub": Gitlink(sub.commit({"s": "2"}))})
client.sync(*sync_args)
for path in paths:
assert client.head(path) == latest
assert (client.top / path / "sub/s").read_text() == "2"
# The default layout reads HEAD from the ref cache instead of GetHead().
@pytest.mark.parametrize("init_args", [[], ["--worktree"]])
def test_prune_detached_head(
host: GitHost, client: RepoClient, init_args: List[str]
) -> None:
"""`repo prune` handles projects with a detached HEAD.
Regression test for 7f7d70e (b/434077990).
"""
host.create_repo("platform/foo", {"f": "foo"})
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url, *init_args)
client.sync()
path = "platform/foo"
client.run("start", "topic", "--all")
client.git(path, "checkout", "-q", "--detach")
client.run("prune")
assert not client.git(path, "branch", "--list", "topic")
def test_submanifest_groups(host: GitHost, client: RepoClient) -> None:
"""Submanifests apply their own default groups.
Regression test for cc88be3 (b/382086294).
"""
for name in ("parent", "child", "child-extra"):
host.create_repo(f"platform/{name}", {"f": name})
host.create_repo(
"platform/child-manifest",
{
"default.xml": manifest_xml(
'<project name="platform/child" path="child" groups="g1" />',
'<project name="platform/child-extra" path="extra" />',
)
},
)
host.create_manifest(
manifest_xml(
'<project name="platform/parent" path="parent" />',
'<submanifest name="sub" project="platform/child-manifest" '
'revision="main" path="sub" default-groups="g1" />',
)
)
client.init(host.manifest_url)
client.sync()
assert client.list_projects() == {
"parent": "platform/parent",
"sub/child": "platform/child",
}
assert (client.top / "sub/child/f").read_text() == "child"
assert not (client.top / "sub/extra").exists()
+294
View File
@@ -0,0 +1,294 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""E2E tests for partial clones, incl. the standard Android configuration."""
import os
from typing import Dict, List, NamedTuple, Tuple
from harness import GIT_DEFAULTS_PARAMS
from harness import GitHost
from harness import Gitlink
from harness import manifest_xml
from harness import RepoClient
from harness import ServerRepo
import pytest
# The go/repo-init flags. The filter is 10M there; 10k keeps the test data
# small without changing the semantics.
ANDROID_INIT_ARGS = [
"-c",
"-b",
"main",
"--use-superproject",
"--partial-clone",
"--partial-clone-exclude=platform/frameworks/base",
"--clone-filter=blob:limit=10k",
]
# The go/sync flags.
ANDROID_SYNC_ARGS = ["-c", "-j20", "--no-tags", "--optimized-fetch", "--prune"]
# Larger than the 10k filter above, smaller than git's default big file
# threshold so it's still delta compressed & packed normally.
BIG = "x" * 20_000
# The git config key holding a partial clone's filter.
FILTER_KEY = "remote.origin.partialclonefilter"
def create_with_big_history(host: GitHost, name: str) -> Tuple[ServerRepo, str]:
"""Create `name` with a large blob that only exists in history.
Returns:
The repo and the id of the historical large blob. The tip commit has
a different large blob (`big2.bin`) and a small file.
"""
repo = host.create_repo(name, {"big.bin": BIG, "small": "s"})
big_blob = repo.git("rev-parse", "main:big.bin")
repo.commit({"big.bin": None, "big2.bin": BIG + "2"})
return repo, big_blob
class AndroidTree(NamedTuple):
"""The server side of `create_android_tree`, keyed by project path."""
repos: Dict[str, ServerRepo]
superproject: ServerRepo
# The commit the superproject pins each project to.
pins: Dict[str, str]
# A large blob that only exists in each project's history.
blobs: Dict[str, str]
def create_android_tree(host: GitHost) -> AndroidTree:
"""Create a small AOSP-like tree with a superproject.
`platform/foo` is a regular project and `platform/frameworks/base` is the
one go/repo-init excludes from the partial clone. Both have moved past
the superproject's pins.
"""
repos, pins, blobs = {}, {}, {}
for path in ("foo", "frameworks/base"):
name = f"platform/{path}"
repos[path], blobs[path] = create_with_big_history(host, name)
pins[path] = repos[path].rev_parse("main")
superproject = host.create_repo(
"platform/superproject",
{path: Gitlink(rev) for path, rev in pins.items()},
)
for repo in repos.values():
repo.commit({"small": "ahead"})
repo.commit({"small": "other"}, branch="other")
host.create_manifest(
manifest_xml(
'<superproject name="platform/superproject" />',
'<project name="platform/foo" path="foo" />',
'<project name="platform/frameworks/base" '
'path="frameworks/base" />',
)
)
return AndroidTree(repos, superproject, pins, blobs)
def bundle_requests(host: GitHost) -> Dict[str, List[int]]:
"""Map repo name to the status codes of its `clone.bundle` requests."""
result: Dict[str, List[int]] = {}
for r in host.requests():
if r.path.endswith("/clone.bundle"):
name = r.path[len(f"/{host.prefix}/") : -len("/clone.bundle")]
result.setdefault(name, []).append(r.status)
return result
@pytest.mark.parametrize("git_defaults", GIT_DEFAULTS_PARAMS)
def test_android_init_sync(host: GitHost, client: RepoClient) -> None:
"""The go/repo-init flag combination followed by `repo sync -c -j32`.
Covers superproject pinning, `-c`, the clone filter on included projects,
no filter on the `--partial-clone-exclude`d project, and that historical
large blobs stay on the server while small historical blobs are fetched.
"""
tree = create_android_tree(host)
small_history_blob = tree.repos["foo"].commit_history_only_blob(
"small_old.txt", b"small history"
)
new_foo_pin = tree.repos["foo"].rev_parse("main")
tree.superproject.commit({"foo": Gitlink(new_foo_pin)})
tree.repos["foo"].commit({"small": "unpinned"})
pins = dict(tree.pins, foo=new_foo_pin)
client.init(host.manifest_url, *ANDROID_INIT_ARGS)
assert client.sync("-c", "-j32").stderr == ""
# init persisted the settings later syncs depend on.
manifests = ".repo/manifests"
assert client.config(manifests, "repo.superproject") == "true"
assert client.config(manifests, "repo.partialclone") == "true"
assert client.config(manifests, "repo.clonefilter") == "blob:limit=10k"
assert (
client.config(manifests, "repo.partialcloneexclude")
== "platform/frameworks/base"
)
for path, pinned in pins.items():
assert client.head(path) == pinned
assert (client.top / path / "big2.bin").read_text() == BIG + "2"
assert client.remote_branches(path) == {"main"}
override = client.repodir / "exp-superproject/superproject_override.xml"
assert override.exists()
# The included project is a partial clone (git normalizes the size), so
# the large historical blob was never downloaded, while the small
# historical blob (under the 10k limit) was downloaded.
assert client.config("foo", "extensions.partialclone") == "origin"
assert client.config("foo", FILTER_KEY) == "blob:limit=10240"
assert not client.has_object("foo", tree.blobs["foo"])
assert client.has_object("foo", small_history_blob)
# The excluded project is a full clone.
assert client.config("frameworks/base", "extensions.partialclone") is None
assert client.config("frameworks/base", FILTER_KEY) is None
assert client.has_object("frameworks/base", tree.blobs["frameworks/base"])
# Partial clones disable clone bundles, but repo still requests one for
# excluded projects (this server has none).
assert bundle_requests(host) == {"platform/frameworks/base": [404]}
@pytest.mark.parametrize("git_defaults", GIT_DEFAULTS_PARAMS)
def test_android_sync_flags(host: GitHost, client: RepoClient) -> None:
"""The go/sync flags on an initial & an incremental sync.
`-c -jN --no-tags --optimized-fetch --prune` on top of the go/repo-init
configuration. The incremental sync must fetch a project whose pin moved
and skip one whose pin didn't.
"""
tree = create_android_tree(host)
tree.repos["foo"].tag("v1", tree.pins["foo"])
client.init(host.manifest_url, *ANDROID_INIT_ARGS)
assert client.sync(*ANDROID_SYNC_ARGS).stderr == ""
for path, pinned in tree.pins.items():
assert client.head(path) == pinned
assert client.remote_branches(path) == {"main"}
assert client.tags(path) == set()
# Move foo's pin forward (on main) & tag it; frameworks/base keeps its pin.
new_pin = tree.repos["foo"].commit({"small": "new pin"})
tree.repos["foo"].tag("v2", new_pin)
tree.superproject.commit({"foo": Gitlink(new_pin)})
with host.watch_requests() as requests:
assert client.sync(*ANDROID_SYNC_ARGS).stderr == ""
assert client.head("foo") == new_pin
assert client.head("frameworks/base") == tree.pins["frameworks/base"]
assert client.tags("foo") == set()
assert (client.top / "foo/small").read_text() == "new pin"
# A project already at its (immutable) pin isn't fetched.
assert host.requested_repos(requests) == {
"platform/manifest",
"platform/superproject",
"platform/foo",
}
def test_partial_clone_default_filter(
host: GitHost, client: RepoClient
) -> None:
"""`--partial-clone` alone uses `blob:none` & disables clone bundles."""
foo, big_blob = create_with_big_history(host, "platform/foo")
small_history = foo.commit_history_only_blob("old_small.txt", b"tiny")
small_tip = foo.git("rev-parse", "main:small")
foo.write_bundle()
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url, "--partial-clone")
client.sync()
path = "platform/foo"
assert client.config(".repo/manifests", "repo.partialclone") == "true"
assert client.config(".repo/manifests", "repo.clonefilter") == "blob:none"
assert client.config(".repo/manifests", FILTER_KEY) == "blob:none"
assert client.config(path, FILTER_KEY) == "blob:none"
assert client.config(path, "extensions.partialclone") == "origin"
# Blobs of the checked out tree were fetched on demand; history wasn't
# (even tiny blobs).
assert (client.top / path / "big2.bin").read_text() == BIG + "2"
assert client.has_object(path, small_tip)
assert not client.has_object(path, big_blob)
assert not client.has_object(path, small_history)
assert bundle_requests(host) == {}
def test_partial_clone_two_phase_sync(
host: GitHost, client: RepoClient
) -> None:
"""`sync -n` then `sync -l` with a partial clone (initial & incremental).
The network phase fetches no blobs, so the local phase has to fetch the
checked out blobs on demand: `-l` isn't offline for partial clones. When
the server is unreachable, `sync -l` fails cleanly & can be retried.
"""
foo, big_blob = create_with_big_history(host, "platform/foo")
small_blob = foo.git("rev-parse", "main:small")
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url, "--partial-clone")
assert client.sync("-n").stderr == ""
path = "platform/foo"
assert not (client.top / path).exists()
gitdir = f"--git-dir={client.repodir / 'projects' / path}.git"
assert client.git(
".", gitdir, "rev-parse", "refs/remotes/origin/main"
) == foo.rev_parse("main")
objects = client.git(
".",
gitdir,
"cat-file",
"--batch-check=%(objectname)",
"--batch-all-objects",
).split()
assert foo.rev_parse("main") in objects
assert small_blob not in objects
# When the server is offline, `sync -l` fails cleanly without leaving a
# broken worktree behind.
with host.offline():
failed = client.sync("-l", check=False)
assert failed.returncode != 0
assert "Cannot initialize work tree for platform/foo" in failed.stderr
assert not os.path.lexists(client.top / path / ".git")
with host.watch_requests() as requests:
client.sync("-l")
assert client.head(path) == foo.rev_parse("main")
assert (client.top / path / "big2.bin").read_text() == BIG + "2"
assert client.has_object(path, small_blob)
assert not client.has_object(path, big_blob)
# The on-demand blob fetches are the only traffic of the local sync.
assert host.requested_repos(requests) == {"platform/foo"}
# Incremental `-n` then `-l` on the existing partial-clone checkout.
v2 = foo.commit({"small": "s2"})
s2_blob = foo.git("rev-parse", "main:small")
client.sync("-n")
assert (client.top / path / "small").read_text() == "s"
assert not client.has_object(path, s2_blob)
with host.watch_requests() as inc_requests:
client.sync("-l")
assert client.head(path) == v2
assert (client.top / path / "small").read_text() == "s2"
assert client.has_object(path, s2_blob)
assert host.requested_repos(inc_requests) == {"platform/foo"}
+339
View File
@@ -0,0 +1,339 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""E2E tests for `repo sync` modes (`-n`, `-l`) & sync flags."""
from typing import List
from harness import GIT_DEFAULTS_PARAMS
from harness import GitHost
from harness import Gitlink
from harness import manifest_xml
from harness import RepoClient
import pytest
# Interleaved (the default) & phased syncs implement `-n` & `-l` separately.
SYNC_MODES = [[], ["--no-interleaved"]]
@pytest.mark.parametrize("mode_args", SYNC_MODES, ids=["interleaved", "phased"])
def test_network_only_then_local_only(
host: GitHost, client: RepoClient, mode_args: List[str]
) -> None:
"""`sync -n` on an existing checkout updates objects & remote refs only.
The worktree, HEAD & local branches don't change until `sync -l`, which
checks out what `-n` fetched without touching the network.
"""
foo = host.create_repo("platform/foo", {"f": "1"})
bar = host.create_repo("platform/bar", {"f": "1"})
host.create_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
'<project name="platform/bar" path="bar" />',
)
)
client.init(host.manifest_url)
client.sync(*mode_args)
old_foo = foo.rev_parse("main")
# Local work on a topic branch in bar.
client.run("start", "topic", "bar")
(client.top / "bar/local").write_text("local")
client.git("bar", "add", "local")
client.git("bar", "commit", "-q", "-m", "local change")
local_commit = client.head("bar")
fetched = {"foo": foo.commit({"f": "2"}), "bar": bar.commit({"f": "2"})}
assert client.sync("-n", *mode_args).stderr == ""
assert client.head("foo") == old_foo
assert (client.top / "foo/f").read_text() == "1"
assert client.head("bar") == local_commit
assert client.git("bar", "symbolic-ref", "--short", "HEAD") == "topic"
for path, commit in fetched.items():
assert client.has_object(path, commit)
assert client.git(path, "rev-parse", "refs/remotes/origin/main") == (
commit
)
assert client.git(path, "status", "--porcelain") == ""
# The server moves on, but `-l` must only use what `-n` fetched.
foo.commit({"f": "3"})
with host.watch_requests() as requests, host.offline():
client.sync("-l", *mode_args)
assert requests == []
assert client.head("foo") == fetched["foo"]
assert (client.top / "foo/f").read_text() == "2"
# The local commit on `bar` was rebased onto what `-n` fetched.
assert client.git("bar", "symbolic-ref", "--short", "HEAD") == "topic"
assert client.git("bar", "rev-parse", "HEAD~1") == fetched["bar"]
assert (client.top / "bar/local").read_text() == "local"
def test_local_only_unfetched_project(
host: GitHost, client: RepoClient
) -> None:
"""`sync -l` fails clearly for a project that was never fetched.
The other projects are still checked out, no network requests are made,
and the error says how to fix it.
"""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_repo("platform/bar", {"f": "bar"})
host.create_manifest(
manifest_xml('<project name="platform/foo" path="foo" />')
)
client.init(host.manifest_url)
client.sync()
v2 = foo.commit({"f": "2"})
host.update_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
'<project name="platform/bar" path="bar" />',
)
)
# Update the manifest & fetch foo, but not the new project.
client.sync("-n", "platform/foo")
assert not (client.repodir / "projects/bar.git").exists()
with host.watch_requests() as requests, host.offline():
result = client.sync("-l", check=False)
assert requests == []
assert result.returncode != 0
assert (
"Cannot checkout platform/bar due to missing network sync; Run "
"`repo sync -n platform/bar` first." in result.stderr
)
assert not (client.top / "bar").exists()
# The already-fetched project still got checked out.
assert client.head("foo") == v2
assert (client.top / "foo/f").read_text() == "2"
# Fetching the missing project fixes it.
client.sync("-n", "platform/bar")
with host.watch_requests() as requests, host.offline():
client.sync("-l")
assert requests == []
assert (client.top / "bar/f").read_text() == "bar"
assert client.list_projects() == {
"bar": "platform/bar",
"foo": "platform/foo",
}
@pytest.mark.parametrize("git_defaults", GIT_DEFAULTS_PARAMS)
def test_two_phase_sync_superproject(host: GitHost, client: RepoClient) -> None:
"""`sync -n` pins projects via the superproject; `sync -l` reuses the pins.
The local phase must not hit the network, even to re-read the
superproject after the server's superproject moved on.
"""
foo = host.create_repo("platform/foo", {"f": "v1"})
pinned = foo.rev_parse("main")
superproject = host.create_repo("superproject", {"foo": Gitlink(pinned)})
foo.commit({"f": "v2"})
host.create_manifest(
manifest_xml(
'<superproject name="superproject" />',
'<project name="platform/foo" path="foo" />',
)
)
client.init(host.manifest_url, "--use-superproject")
assert client.sync("-n").stderr == ""
assert not (client.top / "foo").exists()
assert (
client.repodir / "exp-superproject/superproject_override.xml"
).exists()
# Advance the superproject on the server before running `sync -l`.
new_pin = foo.commit({"f": "v3"})
superproject.commit({"foo": Gitlink(new_pin)})
with host.watch_requests() as requests, host.offline():
client.sync("-l")
assert requests == []
assert client.head("foo") == pinned
assert (client.top / "foo/f").read_text() == "v1"
# The next two-phase sync picks up the new superproject pin.
client.sync("-n")
assert client.head("foo") == pinned
with host.watch_requests() as requests, host.offline():
client.sync("-l")
assert requests == []
assert client.head("foo") == new_pin
assert (client.top / "foo/f").read_text() == "v3"
@pytest.mark.parametrize(
"sync_args,project_attrs,expect_tags",
[
# Despite `repo sync --help` calling `--no-tags` the default.
([], "", True),
# `-c` only limits branches: tags on other branches are still fetched
# unless `--no-tags` is passed.
(["-c"], "", True),
(["--tags"], "", True),
(["-c", "--no-tags"], "", False),
(["-c"], 'sync-tags="false"', False),
(["-c", "--tags"], 'sync-tags="false"', True),
],
)
def test_tags(
host: GitHost,
client: RepoClient,
sync_args: List[str],
project_attrs: str,
expect_tags: bool,
) -> None:
"""Tags are fetched unless `--no-tags` or the manifest says `sync-tags`."""
foo = host.create_repo("platform/foo", {"f": "1"})
foo.tag("v1", "main")
# A tag that isn't on the synced branch, like release tags usually are.
foo.commit({"f": "release"}, branch="release")
foo.tag("v2", "release")
host.create_manifest(
manifest_xml(f'<project name="platform/foo" {project_attrs} />')
)
client.init(host.manifest_url)
client.sync(*sync_args)
assert client.tags("platform/foo") == (
{"v1", "v2"} if expect_tags else set()
)
expected_branches = {"main"} if "-c" in sync_args else {"main", "release"}
assert client.remote_branches("platform/foo") == expected_branches
def test_prune(host: GitHost, client: RepoClient) -> None:
"""Remote-tracking branches deleted on the server are pruned by default.
`--no-prune` keeps them.
"""
foo = host.create_repo("platform/foo", {"f": "1"})
foo.commit({"f": "stale"}, branch="stale")
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url)
client.sync()
path = "platform/foo"
assert client.remote_branches(path) == {"main", "stale"}
foo.git("update-ref", "-d", "refs/heads/stale")
client.sync("--no-prune")
assert client.remote_branches(path) == {"main", "stale"}
client.sync()
assert client.remote_branches(path) == {"main"}
def test_force_sync(host: GitHost, client: RepoClient) -> None:
"""`--force-sync` re-creates a path whose project changed in the manifest.
The old gitdir points at another project's object dir, which sync refuses
to overwrite unless forced.
"""
host.create_repo("platform/foo", {"f": "foo"})
foo_v2 = host.create_repo("platform/foo-v2", {"f": "foo-v2"})
host.create_manifest(
manifest_xml('<project name="platform/foo" path="foo" />')
)
client.init(host.manifest_url)
client.sync()
host.update_manifest(
manifest_xml('<project name="platform/foo-v2" path="foo" />')
)
result = client.sync(check=False)
assert result.returncode != 0
assert "--force-sync not enabled" in result.stderr
assert "`repo sync --force-sync foo`" in result.stderr
assert (client.top / "foo/f").read_text() == "foo"
client.sync("--force-sync")
assert client.head("foo") == foo_v2.rev_parse("main")
assert (client.top / "foo/f").read_text() == "foo-v2"
assert client.list_projects() == {"foo": "platform/foo-v2"}
assert client.git("foo", "status", "--porcelain") == ""
def test_force_remove_dirty(host: GitHost, client: RepoClient) -> None:
"""Removed projects with uncommitted changes need `--force-remove-dirty`."""
host.create_repo("platform/foo", {"f": "foo"})
host.create_repo("platform/bar", {"f": "bar"})
host.create_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
'<project name="platform/bar" path="bar" />',
)
)
client.init(host.manifest_url)
client.sync()
(client.top / "bar/f").write_text("dirty")
host.update_manifest(
manifest_xml('<project name="platform/foo" path="foo" />')
)
# Repeat to check that the project stays tracked in project.list (rather
# than forgotten & left behind) after the first failure.
for _ in range(2):
result = client.sync(check=False)
assert result.returncode != 0
assert (
"bar: Cannot remove project: uncommitted changes are present"
in result.stderr
)
assert (client.top / "bar/f").read_text() == "dirty"
result = client.sync("--force-remove-dirty")
assert "bar: Removing dirty project" in result.stderr
assert not (client.top / "bar").exists()
assert client.list_projects() == {"foo": "platform/foo"}
def test_detach(host: GitHost, client: RepoClient) -> None:
"""`sync -d` detaches projects on a topic branch at the manifest revision.
The topic branch & its local commit are left alone.
"""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_manifest(
manifest_xml('<project name="platform/foo" path="foo" />')
)
client.init(host.manifest_url)
client.sync()
client.run("start", "topic", "--all")
(client.top / "foo/local").write_text("local")
client.git("foo", "add", "local")
client.git("foo", "commit", "-q", "-m", "local change")
local = client.head("foo")
latest = foo.commit({"f": "2"})
client.sync("-d")
assert client.is_detached("foo")
assert client.head("foo") == latest
assert (client.top / "foo/f").read_text() == "2"
assert not (client.top / "foo/local").exists()
assert client.git("foo", "rev-parse", "topic") == local
# A regular sync afterwards stays detached.
client.sync()
assert client.is_detached("foo")
+261
View File
@@ -0,0 +1,261 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""E2E tests for multi-step workflows, many from past point releases."""
import os
import re
import time
from typing import List
from harness import GIT_DEFAULTS_PARAMS
from harness import GitHost
from harness import manifest_xml
from harness import RepoClient
from harness import run_git
from harness import ServerRepo
import pytest
@pytest.mark.parametrize("git_defaults", GIT_DEFAULTS_PARAMS)
@pytest.mark.parametrize(
"project_attrs",
["", 'clone-depth="1"', 'sync-strategy="stateless"'],
)
def test_two_phase_sync(
host: GitHost, client: RepoClient, project_attrs: str
) -> None:
"""`repo sync -n` then `repo sync -l` before worktrees exist.
Regression tests for b/565047698 (v2.68.1, incl. e1e215a for stateless
syncs).
"""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_manifest(
manifest_xml(
f'<project name="platform/foo" path="foo" {project_attrs} />'
)
)
client.init(host.manifest_url)
# The b/565047698 regression only logs a warning & still exits 0, so also
# require a clean stderr.
assert client.sync("-n").stderr == ""
assert not (client.top / "foo").exists()
# A second network-only sync (after upstream moved) with no worktree yet.
latest = foo.commit({"f": "2"})
assert client.sync("-n").stderr == ""
assert not (client.top / "foo").exists()
client.sync("-l")
assert client.head("foo") == latest
assert (client.top / "foo/f").read_text() == "2"
# And the checkout keeps working with regular syncs afterwards.
latest = foo.commit({"f": "3"})
client.sync()
assert client.head("foo") == latest
@pytest.mark.parametrize(
"extra_init_args", [[], ["--manifest-upstream-branch", "main"]]
)
def test_init_manifest_commit(
host: GitHost, client: RepoClient, extra_init_args: List[str]
) -> None:
"""`repo init -b <sha>` pins the manifest to a raw commit.
Regression test for b/544041102 (v2.66.1).
"""
host.create_repo("platform/foo", {"f": "foo"})
host.create_repo("platform/bar", {"f": "bar"})
manifest = host.create_manifest(
manifest_xml('<project name="platform/foo" />')
)
pinned = manifest.rev_parse("main")
host.update_manifest(
manifest_xml(
'<project name="platform/foo" />',
'<project name="platform/bar" />',
)
)
client.init(host.manifest_url, "-b", pinned, *extra_init_args)
client.sync()
assert client.list_projects() == {"platform/foo": "platform/foo"}
assert client.git(".repo/manifests", "rev-parse", "HEAD") == pinned
@pytest.mark.parametrize("git_defaults", GIT_DEFAULTS_PARAMS)
def test_mirror(host: GitHost, client: RepoClient) -> None:
"""`repo init --mirror && repo sync` creates bare mirrors (v2.57.1)."""
foo = host.create_repo("platform/foo", {"f": "1"})
foo.commit({"f": "other"}, branch="other")
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url, "--mirror")
client.sync()
mirror = client.top / "platform/foo.git"
def mirror_rev(ref: str) -> str:
return run_git(f"--git-dir={mirror}", "rev-parse", ref, env=client.env)
assert not (client.top / "platform/foo").exists()
assert mirror_rev("refs/heads/main") == foo.rev_parse("main")
assert mirror_rev("refs/heads/other") == foo.rev_parse("other")
latest = foo.commit({"f": "2"})
client.sync()
assert mirror_rev("refs/heads/main") == latest
def test_incremental_sync(host: GitHost, client: RepoClient) -> None:
"""Syncs pick up new commits, new projects & removed projects."""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_repo("platform/bar", {"f": "bar"})
host.create_repo("platform/baz", {"f": "baz"})
host.create_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
'<project name="platform/bar" path="bar" />',
)
)
client.init(host.manifest_url)
client.sync()
latest = foo.commit({"f": "2", "new": "new"})
host.update_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
'<project name="platform/baz" path="baz" />',
)
)
client.sync()
assert client.head("foo") == latest
assert (client.top / "foo/new").read_text() == "new"
assert (client.top / "baz/f").read_text() == "baz"
assert not (client.top / "bar").exists()
assert client.list_projects() == {
"baz": "platform/baz",
"foo": "platform/foo",
}
def test_rebase_local_work(host: GitHost, client: RepoClient) -> None:
"""Local commits on a `repo start` branch are rebased by sync."""
foo = host.create_repo("platform/foo", {"f": "1"})
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url)
client.sync()
path = "platform/foo"
client.run("start", "topic", "--all")
(client.top / path / "local").write_text("local")
client.git(path, "add", "local")
client.git(path, "commit", "-q", "-m", "local change")
latest = foo.commit({"f": "2"})
client.sync()
assert client.git(path, "symbolic-ref", "--short", "HEAD") == "topic"
assert client.git(path, "rev-parse", "HEAD~1") == latest
assert (client.top / path / "local").read_text() == "local"
def test_switch_manifest_branch(host: GitHost, client: RepoClient) -> None:
"""Re-running `repo init -b` on an existing checkout switches branches."""
foo = host.create_repo("platform/foo", {"f": "main"})
release = foo.commit({"f": "release"}, branch="release")
host.create_manifest(
manifest_xml('<project name="platform/foo" path="foo" />')
)
host.update_manifest(
manifest_xml(
'<project name="platform/foo" path="foo" />',
default='remote="origin" revision="release"',
),
branch="release",
)
client.init(host.manifest_url)
client.sync()
assert client.head("foo") == foo.rev_parse("main")
client.init(host.manifest_url, "-b", "release")
client.sync()
assert client.head("foo") == release
@pytest.mark.parametrize(
"command",
[
["selfupdate", "--no-repo-verify"],
["sync", "--no-repo-verify"],
],
)
def test_selfupdate(
host: GitHost,
client: RepoClient,
repo_source: ServerRepo,
command: List[str],
) -> None:
"""repo upgrades itself when its source moves forward."""
source = host.fork(repo_source, "git-repo")
client.repo_url = source.url
host.create_repo("platform/foo", {"f": "foo"})
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
# Start from an existing synced checkout before publishing the repo update.
client.init(host.manifest_url)
client.sync()
latest = source.commit({"E2E_SELFUPDATE_MARKER": "new"})
# sync only checks for updates once a day.
fetch_head = client.repodir / "repo/.git/FETCH_HEAD"
old = time.time() - 2 * 24 * 60 * 60
os.utime(fetch_head, (old, old))
result = client.run(*command)
assert "A new version of repo is available" in result.stderr
assert client.git(".repo/repo", "rev-parse", "HEAD") == latest
assert (client.repodir / "repo/E2E_SELFUPDATE_MARKER").exists()
# The restarted (upgraded) repo keeps working.
assert "v99.0-e2e-1-g" in client.run("version").stdout
def test_commands_after_sync(host: GitHost, client: RepoClient) -> None:
"""Common read-only commands work on a freshly synced checkout."""
host.create_repo("platform/foo", {"f": "foo"})
host.create_manifest(manifest_xml('<project name="platform/foo" />'))
client.init(host.manifest_url)
client.sync()
for command, expected in (
(["status"], r"nothing to commit"),
(["info"], r"Manifest branch: main"),
(["branches"], r"\(no branches\)"),
# No local changes, so no output.
(["diff"], r"\A\Z"),
(["forall", "-c", "git", "log", "-1"], r"platform/foo: initial"),
# Projects are pinned to commit ids.
(["manifest", "-r"], r'revision="[0-9a-f]{40}"'),
):
output = client.run(*command).output
assert re.search(expected, output), (
f"`repo {' '.join(command)}` output {output!r} "
f"did not match {expected!r}"
)
+23
View File
@@ -201,6 +201,7 @@ class GitCommandStreamLogsTest(unittest.TestCase):
@mock.patch("sys.stderr")
def test_streams_stderr_when_no_stream_is_set(self, mock_stderr):
mock_stderr.isatty.return_value = False
logs = "\n".join(
[
"Enumerating objects: 5, done.",
@@ -228,6 +229,28 @@ class GitCommandStreamLogsTest(unittest.TestCase):
mock_stderr.write.assert_called_once_with(logs)
self.assertEqual(cmd.stderr, logs)
@mock.patch("sys.stderr")
def test_push_injects_progress_only_when_tty_and_not_suppressed(
self, mock_stderr: mock.MagicMock
) -> None:
mock_stderr.isatty.return_value = True
self.mock_process.stderr = io.BufferedReader(io.BytesIO(b""))
git_command.GitCommand(None, ["push"])
self.mock_popen.assert_called_once()
cmd = self.mock_popen.call_args[0][0]
self.assertEqual(cmd, ["git", "push", "--progress"])
@mock.patch("sys.stderr")
def test_push_does_not_inject_progress_when_no_progress_set(
self, mock_stderr: mock.MagicMock
) -> None:
mock_stderr.isatty.return_value = True
self.mock_process.stderr = io.BufferedReader(io.BytesIO(b""))
git_command.GitCommand(None, ["push", "--no-progress"])
self.mock_popen.assert_called_once()
cmd = self.mock_popen.call_args[0][0]
self.assertEqual(cmd, ["git", "push", "--no-progress"])
class GitCommandCaptureBytesTest(unittest.TestCase):
"""Tests the GitCommand class byte capture cases."""
+79 -1
View File
@@ -15,7 +15,7 @@
"""Unittests for the git_config.py module."""
from pathlib import Path
from typing import Any
from typing import Any, Optional
import pytest
import utils_for_test
@@ -268,3 +268,81 @@ def test_remote_save_with_push_url_without_projectname(
def test_is_id(rev: str, expected: bool) -> None:
"""Test IsId identifies both SHA-1 and SHA-256 hashes."""
assert git_config.IsId(rev) == expected
def test_repo_config_for_user_is_not_shared_with_git_config(
monkeypatch: pytest.MonkeyPatch, tmp_home_dir: Path
) -> None:
"""RepoConfig.ForUser() must not reuse GitConfig.ForUser()'s cache."""
monkeypatch.setattr(git_config.GitConfig, "_ForUser", None)
monkeypatch.setattr(git_config.RepoConfig, "_ForUser", None)
monkeypatch.delenv("REPO_CONFIG_DIR", raising=False)
# Populate the GitConfig cache first, as manifest loading does.
git_cfg = git_config.GitConfig.ForUser()
repo_cfg = git_config.RepoConfig.ForUser()
assert repo_cfg is not git_cfg
assert git_cfg.file == str(tmp_home_dir / ".gitconfig")
assert repo_cfg.file == str(tmp_home_dir / ".repoconfig" / "config")
# And the RepoConfig instance is cached on its own class.
assert git_config.RepoConfig.ForUser() is repo_cfg
assert git_config.GitConfig.ForUser() is git_cfg
@pytest.mark.parametrize(
"value, expected",
(
("true", True),
("True", True),
("yes", True),
("on", True),
("1", True),
(" true ", True),
("false", False),
("No", False),
("off", False),
("0", False),
("oops", None),
("2", None),
("", None),
),
)
def test_parse_boolean(value: str, expected: Optional[bool]) -> None:
"""Test ParseBoolean accepts the spellings `git config --type=bool` does."""
assert git_config.ParseBoolean(value) is expected
@pytest.mark.parametrize(
"value, expected",
(
("10", 10),
("-7", -7),
("10k", 10 * 1024),
("10M", 10 * 1024 * 1024),
("10g", 10 * 1024 * 1024 * 1024),
("0x10", 16),
("0x10k", 16 * 1024),
(" 10 ", 10),
("oops", None),
("1x", None),
("", None),
),
)
def test_parse_int(value: str, expected: Optional[int]) -> None:
"""Test ParseInt accepts the syntax `git config --type=int` does."""
assert git_config.ParseInt(value) == expected
@pytest.mark.parametrize(
"value, expected",
(("on", True), ("off", False), ("1", True), ("0", False)),
)
def test_get_boolean_git_spellings(
tmp_path: Path, value: str, expected: bool
) -> None:
"""Test GetBoolean accepts on/off/1/0 like git does."""
config_file = tmp_path / "config"
config_file.write_text(f"[section]\n\tkey = {value}\n")
config = git_config.GitConfig(config_file)
assert config.GetBoolean("section.key") is expected
+9 -8
View File
@@ -23,6 +23,7 @@ from unittest import mock
import pytest
import utils_for_test
import git_command
import git_refs
@@ -99,8 +100,8 @@ def test_reads_head_with_root_refs_in_one_command(
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
monkeypatch.setattr(git_command, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_command, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
with mock.patch.object(refs, "_ReadSymbolicRef") as read_head:
assert refs.get("HEAD") == head
@@ -131,8 +132,8 @@ def test_root_ref_snapshot_falls_back_for_unborn_head(
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
monkeypatch.setattr(git_command, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_command, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
@@ -161,8 +162,8 @@ def test_old_git_keeps_separate_head_fallback(
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: False)
monkeypatch.setattr(git_command, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_command, "git_require", lambda _version: False)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
@@ -194,8 +195,8 @@ def test_for_each_ref_failure_falls_back_to_symbolic_ref(
def Wait(self) -> int:
return 1
monkeypatch.setattr(git_refs, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_refs, "git_require", lambda _version: True)
monkeypatch.setattr(git_command, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_command, "git_require", lambda _version: True)
refs = git_refs.GitRefs("/nonexistent")
def read_head(name: str) -> None:
+6 -5
View File
@@ -20,6 +20,7 @@ from unittest import mock
import pytest
import git_command
import git_status
@@ -143,8 +144,8 @@ def test_get_status_uses_versioned_machine_options(
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_status, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_status, "git_require", lambda _version: True)
monkeypatch.setattr(git_command, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_command, "git_require", lambda _version: True)
git_status.GetStatus(
mock.sentinel.project,
@@ -173,7 +174,7 @@ def test_get_status_uses_versioned_machine_options(
def test_get_status_rejects_git_before_2_11(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(git_status, "git_require", lambda _version: False)
monkeypatch.setattr(git_command, "git_require", lambda _version: False)
with pytest.raises(git_status.UnsupportedStatusError):
git_status.GetStatus(mock.sentinel.project, mock.sentinel.gitdir)
@@ -194,9 +195,9 @@ def test_get_status_omits_stash_header_before_2_35(
def Wait(self) -> int:
return 0
monkeypatch.setattr(git_status, "GitCommand", FakeGitCommand)
monkeypatch.setattr(git_command, "GitCommand", FakeGitCommand)
monkeypatch.setattr(
git_status,
git_command,
"git_require",
lambda version: version <= (2, 34, 0),
)
+5 -5
View File
@@ -465,7 +465,7 @@ class SuperprojectTestCase(unittest.TestCase):
"""Test _Init succeeds and creates the work git dir."""
self.assertFalse(os.path.exists(self._superproject._work_git))
with mock.patch(
"git_superproject.GitCommand", autospec=True
"git_command.GitCommand", autospec=True
) as mock_git_command:
instance = mock_git_command.return_value
instance.Wait.return_value = 0
@@ -492,7 +492,7 @@ class SuperprojectTestCase(unittest.TestCase):
os.mkdir(self._superproject._superproject_path)
os.mkdir(self._superproject._work_git)
with mock.patch(
"git_superproject.GitCommand", autospec=True
"git_command.GitCommand", autospec=True
) as mock_git_command:
self.assertTrue(self._superproject._Init())
mock_git_command.assert_not_called()
@@ -500,7 +500,7 @@ class SuperprojectTestCase(unittest.TestCase):
def test_Init_failure_cleans_up(self):
"""Test _Init cleans up the temporary directory if 'git init' fails."""
with mock.patch(
"git_superproject.GitCommand", autospec=True
"git_command.GitCommand", autospec=True
) as mock_git_command:
instance = mock_git_command.return_value
instance.Wait.return_value = 1
@@ -540,7 +540,7 @@ class SuperprojectTestCase(unittest.TestCase):
os.mkdir(self._superproject._work_git)
with mock.patch.object(self._superproject, "_Init", return_value=True):
with mock.patch(
"git_superproject.GitCommand", autospec=True
"git_command.GitCommand", autospec=True
) as mock_git_command:
with mock.patch.object(
self._superproject, "_GetRef"
@@ -600,7 +600,7 @@ class SuperprojectTestCase(unittest.TestCase):
command = mock.MagicMock(stdout="1234\n")
command.Wait.return_value = 0
with mock.patch(
"git_superproject.GitCommand", return_value=command
"git_command.GitCommand", return_value=command
) as git_command:
self.assertEqual("1234", self._superproject._GetRef("HEAD"))
+56
View File
@@ -420,6 +420,62 @@ class TestXmlManifest:
)
manifest.ToXml()
def test_sync_smartsync(self, repo_client: RepoClient) -> None:
"""Check sync-smartsync handling."""
manifest = repo_client.get_xml_manifest(
"<manifest><default /></manifest>"
)
assert manifest.default.sync_smartsync is False
manifest = repo_client.get_xml_manifest(
'<manifest><default sync-smartsync="true" /></manifest>'
)
assert manifest.default.sync_smartsync is True
assert (
manifest.ToXml().toxml() == '<?xml version="1.0" ?>'
'<manifest><default sync-smartsync="true"/></manifest>'
)
manifest = repo_client.get_xml_manifest(
'<manifest><default sync-smartsync="false" /></manifest>'
)
assert manifest.default.sync_smartsync is False
assert manifest.ToXml().toxml() == '<?xml version="1.0" ?><manifest/>'
def test_clear_override(self, repo_client: RepoClient) -> None:
"""Check ClearOverride reverts to the default manifest."""
manifest_xml_fmt = (
'<manifest><remote name="r" fetch="." />'
'<default remote="r" revision="main" />'
'<project name="%s" /></manifest>'
)
manifest = repo_client.get_xml_manifest(manifest_xml_fmt % "default")
(repo_client.manifest_dir / "good.xml").write_text(
manifest_xml_fmt % "override", encoding="utf-8"
)
(repo_client.manifest_dir / "bad.xml").write_text(
"<manifest>", encoding="utf-8"
)
# No override in effect: a no-op.
manifest.ClearOverride()
assert list(manifest.paths) == ["default"]
manifest.Override("good.xml")
assert list(manifest.paths) == ["override"]
manifest.ClearOverride()
assert not manifest.manifestFileOverrides
assert list(manifest.paths) == ["default"]
# Override() registers the override before parsing it, so a failed
# Override() leaves the client pinned until ClearOverride().
with pytest.raises(error.ManifestParseError):
manifest.Override("bad.xml")
with pytest.raises(error.ManifestParseError):
manifest.Load()
manifest.ClearOverride()
assert list(manifest.paths) == ["default"]
class TestIncludeElement:
"""Tests for <include>."""
+505 -54
View File
@@ -15,12 +15,13 @@
"""Unittests for the project.py module."""
import contextlib
import io
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple
from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple, Type
import unittest
from unittest import mock
@@ -30,6 +31,7 @@ import utils_for_test
import error
import git_command
import git_config
import git_refs
import git_status
import git_trace2_event_log
import manifest_xml
@@ -113,6 +115,19 @@ class ReviewableBranchTests(unittest.TestCase):
_, kwargs = proj.UploadForReview.call_args
self.assertEqual(kwargs.get("git_event_log"), mock_event_log)
def test_upload_for_review_forwards_quiet(self) -> None:
"""Check ReviewableBranch.UploadForReview forwards quiet flag."""
proj = mock.MagicMock(spec=project.Project)
branch = mock.MagicMock()
branch.name = "work"
rb = project.ReviewableBranch(proj, branch, "main")
rb.UploadForReview(people=([], []), quiet=True)
proj.UploadForReview.assert_called_once()
_, kwargs = proj.UploadForReview.call_args
self.assertTrue(kwargs.get("quiet"))
def test_current_comes_from_branch_snapshot(self) -> None:
"""Reviewable branches carry current state into report consumers."""
branch = mock.MagicMock()
@@ -215,7 +230,7 @@ class ProjectTests(unittest.TestCase):
"""Check UploadForReview emits repo.uploadstate trace2 data events."""
proj, _ = self._create_project_for_upload_test()
with mock.patch("project.GitCommand") as mock_git_cmd, mock.patch(
with mock.patch("git_command.GitCommand") as mock_git_cmd, mock.patch(
"project.ReviewableBranch"
) as mock_rb_cls:
mock_cmd = mock.MagicMock()
@@ -255,7 +270,7 @@ class ProjectTests(unittest.TestCase):
"""Check event emission when stderr contains no review URLs."""
proj, _ = self._create_project_for_upload_test()
with mock.patch("project.GitCommand") as mock_git_cmd, mock.patch(
with mock.patch("git_command.GitCommand") as mock_git_cmd, mock.patch(
"project.ReviewableBranch"
) as mock_rb_cls:
mock_cmd = mock.MagicMock()
@@ -289,7 +304,7 @@ class ProjectTests(unittest.TestCase):
"""Check UploadForReview succeeds when git_event_log is None."""
proj, _ = self._create_project_for_upload_test()
with mock.patch("project.GitCommand") as mock_git_cmd, mock.patch(
with mock.patch("git_command.GitCommand") as mock_git_cmd, mock.patch(
"project.ReviewableBranch"
) as mock_rb_cls:
mock_cmd = mock.MagicMock()
@@ -312,7 +327,7 @@ class ProjectTests(unittest.TestCase):
"""Check exceptions during tracing are caught and do not fail upload."""
proj, _ = self._create_project_for_upload_test()
with mock.patch("project.GitCommand") as mock_git_cmd, mock.patch(
with mock.patch("git_command.GitCommand") as mock_git_cmd, mock.patch(
"project.ReviewableBranch"
) as mock_rb_cls, mock.patch("project.logger") as mock_logger:
mock_cmd = mock.MagicMock()
@@ -337,7 +352,7 @@ class ProjectTests(unittest.TestCase):
"""Check integration with real git_trace2_event_log.EventLog."""
proj, _ = self._create_project_for_upload_test()
with mock.patch("project.GitCommand") as mock_git_cmd, mock.patch(
with mock.patch("git_command.GitCommand") as mock_git_cmd, mock.patch(
"project.ReviewableBranch"
) as mock_rb_cls:
mock_cmd = mock.MagicMock()
@@ -375,6 +390,36 @@ class ProjectTests(unittest.TestCase):
data_map.get("repo.uploadstate/files"), "file1.py,file2.py"
)
def test_upload_for_review_progress_flag(self) -> None:
"""Check UploadForReview passes --no-progress iff quiet=True."""
for quiet in (False, True):
with self.subTest(quiet=quiet):
proj, _ = self._create_project_for_upload_test()
with mock.patch(
"git_command.GitCommand"
) as mock_git_cmd, mock.patch("project.ReviewableBranch"):
mock_cmd = mock.MagicMock()
mock_cmd.Wait.return_value = 0
mock_cmd.stderr = ""
mock_git_cmd.return_value = mock_cmd
project.Project.UploadForReview(
proj,
branch="test-branch",
dryrun=True,
quiet=quiet,
)
mock_git_cmd.assert_called_once()
_, args, _ = mock_git_cmd.mock_calls[0]
cmd = args[1]
if quiet:
self.assertIn("--no-progress", cmd)
else:
self.assertNotIn("--no-progress", cmd)
self.assertNotIn("--progress", cmd)
self.assertNotIn("--quiet", cmd)
def test_get_head_revision_id(self):
"""Check GetHeadRevisionId behavior."""
with utils_for_test.TempGitTree() as tempdir:
@@ -407,9 +452,9 @@ class ProjectTests(unittest.TestCase):
command.Wait.return_value = 0
with mock.patch.object(
project, "git_require", return_value=True
git_command, "git_require", return_value=True
), mock.patch.object(
project, "GitCommand", return_value=command
git_command, "GitCommand", return_value=command
) as cmd:
self.assertEqual("1" * 40, git.ResolveCommit("topic"))
@@ -436,7 +481,7 @@ class ProjectTests(unittest.TestCase):
proj.name = "project"
git = project.Project._GitGetByExec(proj, bare=True, gitdir="gitdir")
with mock.patch.object(project, "git_require", return_value=False):
with mock.patch.object(git_command, "git_require", return_value=False):
with self.assertRaises(error.GitError):
git.ResolveCommit("--not-a-revision")
@@ -713,6 +758,18 @@ class ProjectTests(unittest.TestCase):
):
self.assertIsNone(proj._GetStatusSnapshot())
def test_get_status_snapshot_missing_worktree_is_quiet(self) -> None:
"""A missing worktree skips git status without a warning."""
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir)
proj.worktree = os.path.join(tempdir, "missing")
with mock.patch.object(git_status, "GetStatus") as mock_get_status:
with mock.patch.object(project, "logger") as mock_logger:
self.assertIsNone(proj._GetStatusSnapshot())
mock_get_status.assert_not_called()
mock_logger.warning.assert_not_called()
def test_dirty_or_stash_uses_status_stash_header(self) -> None:
"""A normal stash is detected without a second Git process."""
with utils_for_test.TempGitTree() as tempdir:
@@ -722,7 +779,9 @@ class ProjectTests(unittest.TestCase):
proj._GetStatusSnapshot = mock.MagicMock(return_value=status)
proj.HasStash = mock.MagicMock()
with mock.patch.object(project, "git_require", return_value=True):
with mock.patch.object(
git_command, "git_require", return_value=True
):
self.assertTrue(proj._HasDirtyOrStash())
proj.HasStash.assert_not_called()
@@ -735,7 +794,9 @@ class ProjectTests(unittest.TestCase):
proj._GetStatusSnapshot = mock.MagicMock(return_value=status)
proj.HasStash = mock.MagicMock()
with mock.patch.object(project, "git_require", return_value=True):
with mock.patch.object(
git_command, "git_require", return_value=True
):
self.assertFalse(proj._HasDirtyOrStash())
proj.HasStash.assert_not_called()
@@ -748,7 +809,9 @@ class ProjectTests(unittest.TestCase):
proj._GetStatusSnapshot = mock.MagicMock(return_value=status)
proj.HasStash = mock.MagicMock(return_value=True)
with mock.patch.object(project, "git_require", return_value=False):
with mock.patch.object(
git_command, "git_require", return_value=False
):
self.assertTrue(proj._HasDirtyOrStash())
proj._GetStatusSnapshot.assert_called_once_with(
@@ -756,6 +819,66 @@ class ProjectTests(unittest.TestCase):
)
proj.HasStash.assert_called_once_with()
def test_dirty_and_head_agree_across_status_paths(self) -> None:
"""Porcelain v2 and legacy plumbing report the same state."""
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir)
def check(expected: Tuple[bool, Optional[str]]) -> None:
for use_status in (True, False):
with self.subTest(expected=expected, status=use_status):
with mock.patch.object(
git_command, "git_require", return_value=use_status
):
self.assertEqual(expected, proj.GetDirtyAndHead())
check((False, None))
Path(tempdir, "untracked").write_text("new")
check((True, None))
Path(tempdir, "tracked").write_text("initial")
proj.work_git.add("tracked")
proj.work_git.commit("-m", "initial")
head = proj.work_git.rev_parse("HEAD")
check((True, head))
os.remove(os.path.join(tempdir, "untracked"))
check((False, head))
def test_dirty_and_head_fallback_skips_second_snapshot(self) -> None:
"""A failed snapshot goes straight to the legacy plumbing."""
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir)
proj._GetStatusSnapshot = mock.MagicMock(return_value=None)
proj._IsDirtyLegacy = mock.MagicMock(return_value=False)
proj.work_git = mock.MagicMock()
proj.work_git.rev_parse.return_value = "head"
self.assertEqual((False, "head"), proj.GetDirtyAndHead())
proj._GetStatusSnapshot.assert_called_once_with(
untracked_files="normal", branch=True
)
proj._IsDirtyLegacy.assert_called_once_with(consider_untracked=True)
proj.work_git.rev_parse.assert_called_once_with("HEAD")
def test_dirty_or_stash_fallback_skips_second_snapshot(self) -> None:
"""A failed snapshot goes straight to the legacy dirty check."""
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir)
proj._GetStatusSnapshot = mock.MagicMock(return_value=None)
proj._IsDirtyLegacy = mock.MagicMock(return_value=False)
proj.HasStash = mock.MagicMock(return_value=False)
with mock.patch.object(
git_command, "git_require", return_value=True
):
self.assertFalse(proj._HasDirtyOrStash())
proj._GetStatusSnapshot.assert_called_once_with(
untracked_files="normal", show_stash=True
)
proj._IsDirtyLegacy.assert_called_once_with(consider_untracked=True)
def test_old_git_dirty_check_uses_legacy_plumbing(self) -> None:
"""Git clients before 2.11 retain the existing dirty-check path."""
with utils_for_test.TempGitTree() as tempdir:
@@ -763,7 +886,9 @@ class ProjectTests(unittest.TestCase):
proj.work_git = mock.MagicMock()
proj.work_git.DiffZ.side_effect = [{}, {"tracked": mock.sentinel}]
with mock.patch.object(project, "git_require", return_value=False):
with mock.patch.object(
git_command, "git_require", return_value=False
):
self.assertTrue(proj.IsDirty())
proj.work_git.update_index.assert_called_once_with(
@@ -780,7 +905,7 @@ class ProjectTests(unittest.TestCase):
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir)
with mock.patch.object(
project, "GitCommand", autospec=True
git_command, "GitCommand", autospec=True
) as run_git:
run_git.return_value.Wait.return_value = 0
proj._FastForward("revision")
@@ -1268,7 +1393,7 @@ class ProjectTests(unittest.TestCase):
hex41 = "a" * 41
proj.SetRevision(hex41)
with mock.patch("project.GitCommand") as mock_git:
with mock.patch("git_command.GitCommand") as mock_git:
mock_cmd = mock.MagicMock()
mock_cmd.Wait.return_value = 0
mock_git.return_value = mock_cmd
@@ -1751,6 +1876,9 @@ class ManifestPropertiesFetchedCorrectly(unittest.TestCase):
fakeproj.config.SetBoolean("repo.superproject", False)
self.assertFalse(fakeproj.use_superproject)
fakeproj.config.SetBoolean("repo.smartsync", False)
self.assertFalse(fakeproj.smart_sync)
fakeproj.config.SetBoolean("repo.partialclone", False)
self.assertFalse(fakeproj.partial_clone)
@@ -1968,8 +2096,8 @@ def test_default_branch_fallback(
project._DefaultBranchFallback.cache_clear()
try:
with mock.patch.object(
project, "git_require", return_value=supports_git_var
), mock.patch.object(project, "GitCommand", FakeGitCommand):
git_command, "git_require", return_value=supports_git_var
), mock.patch.object(git_command, "GitCommand", FakeGitCommand):
assert project._DefaultBranchFallback() == expected_ref
assert seen == expected_calls
finally:
@@ -1991,6 +2119,87 @@ def test_metaproject_has_changes_bounds_revision_walk() -> None:
meta._revlist.assert_called_once_with("-1", "^HEAD", "remote")
@pytest.mark.parametrize(
"project_cls,revision,expected",
(
(project.RepoProject, "refs/remotes/origin/current", False),
(project.RepoProject, "refs/remotes/origin/newer", True),
(project.RepoProject, "refs/remotes/origin/divergent", True),
# A release rollback rewinds stable, and repo itself has to follow.
(project.RepoProject, "refs/remotes/origin/older", True),
# --repo-rev can pin repo to an annotated tag, which must be peeled.
(project.RepoProject, "refs/tags/current-tag", False),
# Other MetaProjects only update when the remote has new commits.
(project.ManifestProject, "refs/remotes/origin/current", False),
(project.ManifestProject, "refs/remotes/origin/newer", True),
(project.ManifestProject, "refs/remotes/origin/divergent", True),
(project.ManifestProject, "refs/remotes/origin/older", False),
),
ids=lambda v: v.__name__ if isinstance(v, type) else None,
)
def test_metaproject_has_changes(
project_cls: Type[project.MetaProject], revision: str, expected: bool
) -> None:
"""HasChanges compares the tracked revision with the checked out one."""
with utils_for_test.TempGitTree() as tempdir:
def git(*args: str) -> str:
return subprocess.run(
["git", "-C", tempdir, *args],
check=True,
capture_output=True,
text=True,
).stdout.strip()
def commit(msg: str) -> str:
git("commit", "-q", "--allow-empty", "-m", msg)
return git("rev-parse", "HEAD")
# older -> current -> newer, plus divergent branching off older.
revs = {name: commit(name) for name in ("older", "current", "newer")}
git("checkout", "-q", "--detach", revs["older"])
revs["divergent"] = commit("divergent")
git("checkout", "-q", "main")
git("reset", "-q", "--hard", revs["current"])
for name, rev in revs.items():
git("update-ref", f"refs/remotes/origin/{name}", rev)
git("tag", "-a", "-m", "current-tag", "current-tag", revs["current"])
proj = _create_meta_project(tempdir, project_cls)
proj.revisionExpr = revision
assert proj.HasChanges is expected
@pytest.mark.parametrize(
"project_cls,expected_sleep",
(
(project.ManifestProject, 4.0),
# Repo waits longer for a briefly deleted stable branch to come back.
(project.RepoProject, 10.0),
),
ids=lambda v: v.__name__ if isinstance(v, type) else None,
)
def test_remote_fetch_retry_sleep(
project_cls: Type[project.MetaProject], expected_sleep: float
) -> None:
"""A failed fetch waits RETRY_SLEEP_INITIAL_SEC before retrying."""
with utils_for_test.TempGitTree() as tempdir:
proj = _create_meta_project(tempdir, project_cls)
proj.config.GetRemote("origin").ResetFetch()
proj.revisionExpr = "refs/heads/stable"
with mock.patch.object(git_command, "GitCommand") as mock_git:
# What git does while stable is deleted.
mock_git.return_value.Wait.return_value = 128
mock_git.return_value.stdout = (
"fatal: couldn't find remote ref refs/heads/stable\n"
)
with mock.patch.object(project.time, "sleep") as mock_sleep:
assert not proj._RemoteFetch(output_redir=io.StringIO())
assert mock_sleep.call_args_list == [mock.call(expected_sleep)]
@pytest.mark.parametrize(
"state,expect_failure",
[
@@ -2198,7 +2407,7 @@ class StatelessSyncTests(unittest.TestCase):
proj.work_git = mock.MagicMock()
syncbuf = project.SyncBuffer(proj.config)
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
@@ -2319,7 +2528,7 @@ class StatelessSyncTests(unittest.TestCase):
proj.bare_ref.head = "refs/heads/topic"
proj.work_git = mock.MagicMock()
proj.work_git.GetHead.return_value = "refs/heads/topic"
proj.work_git.merge_base.side_effect = project.GitError("diverged")
proj.work_git.merge_base.side_effect = error.GitError("diverged")
branch = mock.MagicMock()
branch.name = "topic"
branch.LocalMerge = "refs/remotes/origin/main"
@@ -2350,6 +2559,33 @@ class StatelessSyncTests(unittest.TestCase):
self.assertTrue(res.success)
self.assertFalse(getattr(proj, "stateless_prune_needed", False))
def test_sync_network_half_stateless_skips_without_worktree(self) -> None:
"""Test stateless sync doesn't prune a project with no worktree."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.worktree = os.path.join(tempdir, "missing")
proj._HasDirtyOrStash = mock.MagicMock(return_value=False)
res = proj.Sync_NetworkHalf()
self.assertTrue(res.success)
self.assertFalse(proj.stateless_prune_needed)
proj._LsRemote.assert_not_called()
proj._HasDirtyOrStash.assert_not_called()
def test_ls_remote_runs_without_worktree(self) -> None:
"""Test ls-remote only needs the gitdir's remote config."""
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir)
proj.work_git.commit("--allow-empty", "-m", "initial")
proj.work_git.config("remote.origin.url", tempdir)
head = proj.work_git.rev_parse("HEAD")
proj.worktree = os.path.join(tempdir, "missing")
# _create_mock_project() stubs this out.
del proj._LsRemote
self.assertEqual(f"{head}\tHEAD\n", proj._LsRemote("HEAD"))
def test_sync_network_half_stateless_skips_if_local_commits(self):
"""Test stateless sync skips if there are local-only commits."""
with utils_for_test.TempGitTree() as tempdir:
@@ -2520,7 +2756,7 @@ class SyncOptimizationTests(unittest.TestCase):
if os.path.exists(shallow_path):
os.unlink(shallow_path)
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
@@ -2543,7 +2779,7 @@ class SyncOptimizationTests(unittest.TestCase):
with open(os.path.join(proj.gitdir, "shallow"), "w") as f:
f.write("")
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
res = proj._RemoteFetch(
current_branch_only=True,
depth=1,
@@ -2566,7 +2802,7 @@ class SyncOptimizationTests(unittest.TestCase):
with open(os.path.join(other.gitdir, "shallow"), "w") as f:
f.write("")
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
@@ -2595,7 +2831,7 @@ class SyncOptimizationTests(unittest.TestCase):
with open(os.path.join(other.gitdir, "shallow"), "w") as f:
f.write("")
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
res = proj._RemoteFetch(
current_branch_only=True,
depth=1,
@@ -2789,7 +3025,7 @@ class SyncOptimizationTests(unittest.TestCase):
mock_remote.PreConnectFetch.return_value = True
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
@@ -2828,7 +3064,7 @@ class SyncOptimizationTests(unittest.TestCase):
mock_remote.PreConnectFetch.return_value = True
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
@@ -2870,7 +3106,7 @@ class SyncOptimizationTests(unittest.TestCase):
mock_remote.PreConnectFetch.return_value = True
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
@@ -2918,7 +3154,7 @@ class SyncOptimizationTests(unittest.TestCase):
mock_remote.PreConnectFetch.return_value = True
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
@@ -3020,7 +3256,7 @@ class SyncOptimizationTests(unittest.TestCase):
return inst
with mock.patch(
"project.GitCommand", side_effect=_fake_git_command
"git_command.GitCommand", side_effect=_fake_git_command
):
res = proj._RemoteFetch(current_branch_only=True)
@@ -3128,8 +3364,11 @@ class GetEnvVarsTests(unittest.TestCase):
self.assertEqual(env["REPO_LREV"], "")
def _create_manifest_project(tempdir: str) -> project.ManifestProject:
"""Return a ManifestProject for a new .repo/ under |tempdir|."""
def _create_meta_project(
tempdir: str,
project_cls: Type[project.MetaProject] = project.ManifestProject,
) -> project.MetaProject:
"""Return a |project_cls| for a new .repo/ under |tempdir|."""
repodir = os.path.join(tempdir, ".repo")
manifest_dir = os.path.join(repodir, "manifests")
manifest_file = os.path.join(repodir, manifest_xml.MANIFEST_FILE_NAME)
@@ -3137,7 +3376,7 @@ def _create_manifest_project(tempdir: str) -> project.ManifestProject:
os.mkdir(manifest_dir)
manifest = manifest_xml.XmlManifest(repodir, manifest_file)
return project.ManifestProject(
return project_cls(
manifest, "test/manifest", os.path.join(tempdir, ".git"), tempdir
)
@@ -3146,7 +3385,7 @@ class FetchCmdTests(unittest.TestCase):
"""Tests for fetch_cmd feature."""
def setUpManifest(self, tempdir):
return _create_manifest_project(tempdir)
return _create_meta_project(tempdir)
def _get_project(self, tempdir):
proj = _create_mock_project(
@@ -3155,12 +3394,13 @@ class FetchCmdTests(unittest.TestCase):
proj.GetRevisionId = mock.MagicMock(return_value="1234abcd")
return proj
def test_fetch_cmd_execution(self):
"""Test that fetch_cmd is executed with correct environment."""
def test_fetch_cmd_execution_unpinned_branch(self) -> None:
"""Test fetch_cmd with an unpinned branch does not call ls-remote."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.bare_git.rev_parse.return_value = "1234abcd"
proj.revisionId = None
proj.revisionExpr = "main"
proj.bare_git.ResolveCommit.return_value = "1234abcd"
mock_remote = mock.MagicMock()
mock_remote.ToLocal.return_value = "refs/remotes/origin/main"
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
@@ -3171,15 +3411,115 @@ class FetchCmdTests(unittest.TestCase):
self.assertTrue(res)
mock_run.assert_called_once()
proj._LsRemote.assert_not_called()
args, kwargs = mock_run.call_args
self.assertEqual(args[0], "echo hi")
self.assertEqual(kwargs["shell"], True)
self.assertEqual(kwargs["cwd"], tempdir)
self.assertEqual(kwargs["env"]["REPO_TREV"], "1234abcd")
self.assertEqual(kwargs["env"]["REPO_TREV"], "")
self.assertEqual(kwargs["env"]["REPO_RREV"], "main")
self.assertEqual(
kwargs["env"]["REPO_PROJECT_FETCH_URL"],
"http://example.com/repo",
)
proj.bare_git.ResolveCommit.assert_any_call(
"refs/remotes/origin/main"
)
proj.bare_git.ResolveCommit.assert_any_call("FETCH_HEAD")
def test_fetch_cmd_execution_pinned_revision_id(self) -> None:
"""Test fetch_cmd populates REPO_TREV when revisionId is set."""
with utils_for_test.TempGitTree() as tempdir:
sha = "0123456789abcdef0123456789abcdef01234567"
proj = self._get_project(tempdir)
proj.revisionId = sha
proj.revisionExpr = "main"
proj.bare_git.ResolveCommit.return_value = sha
mock_remote = mock.MagicMock()
mock_remote.ToLocal.return_value = "refs/remotes/origin/main"
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stderr="")
res = proj._CustomFetch()
self.assertTrue(res)
proj._LsRemote.assert_not_called()
_, kwargs = mock_run.call_args
self.assertEqual(kwargs["env"]["REPO_TREV"], sha)
def test_fetch_cmd_execution_pinned_revision_expr_sha(self) -> None:
"""Test fetch_cmd populates REPO_TREV when revisionExpr is a SHA."""
with utils_for_test.TempGitTree() as tempdir:
sha = "0123456789abcdef0123456789abcdef01234567"
proj = self._get_project(tempdir)
proj.revisionId = None
proj.revisionExpr = sha
proj.bare_git.ResolveCommit.return_value = sha
proj.GetRemote = mock.MagicMock()
with mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stderr="")
res = proj._CustomFetch()
self.assertTrue(res)
proj._LsRemote.assert_not_called()
proj.GetRemote.assert_not_called()
_, kwargs = mock_run.call_args
self.assertEqual(kwargs["env"]["REPO_TREV"], sha)
proj.bare_git.ResolveCommit.assert_called_once_with("FETCH_HEAD")
def test_fetch_cmd_postcondition_fails_when_fetch_head_mismatches_ref(
self,
) -> None:
"""Test _CustomFetch fails when FETCH_HEAD disagrees with the ref."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.revisionId = None
proj.revisionExpr = "main"
proj.bare_git.ResolveCommit.side_effect = [
"11111111" * 5,
"22222222" * 5,
]
mock_remote = mock.MagicMock()
mock_remote.ToLocal.return_value = "refs/remotes/origin/main"
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stderr="")
self.assertFalse(proj._CustomFetch())
def test_fetch_cmd_postcondition_fails_when_ref_mismatches_pinned_trev(
self,
) -> None:
"""Test _CustomFetch fails when the ref disagrees with REPO_TREV."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.revisionId = "11111111" * 5
proj.revisionExpr = "main"
proj.bare_git.ResolveCommit.return_value = "22222222" * 5
mock_remote = mock.MagicMock()
mock_remote.ToLocal.return_value = "refs/remotes/origin/main"
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stderr="")
self.assertFalse(proj._CustomFetch())
def test_fetch_cmd_postcondition_fails_when_ref_missing(self) -> None:
"""Test _CustomFetch fails when tracking ref or FETCH_HEAD fails."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.revisionId = None
proj.revisionExpr = "main"
proj.bare_git.ResolveCommit.side_effect = error.GitError("missing")
mock_remote = mock.MagicMock()
mock_remote.ToLocal.return_value = "refs/remotes/origin/main"
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
with mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stderr="")
self.assertFalse(proj._CustomFetch())
def test_sync_fetch_cmd_requires_use_local_gitdirs(self):
"""Test that fetch_cmd requires use_local_gitdirs."""
@@ -3200,6 +3540,31 @@ class FetchCmdTests(unittest.TestCase):
result = fakeproj.Sync(use_local_gitdirs=False)
self.assertFalse(result)
def test_sync_network_half_skips_clone_bundle_with_fetch_cmd(self) -> None:
"""Test Sync_NetworkHalf skips clone.bundle when fetchcmd is set."""
with tempfile.TemporaryDirectory() as tempdir:
worktree = os.path.join(tempdir, "new-proj")
gitdir = os.path.join(worktree, ".git")
proj = _create_mock_project(
worktree,
use_local_gitdirs=True,
fetch_cmd="echo hi",
gitdir=gitdir,
objdir=gitdir,
)
proj._InitGitDir = mock.MagicMock()
proj._InitRemote = mock.MagicMock()
proj._InitMRef = mock.MagicMock()
proj._CustomFetch = mock.MagicMock(return_value=True)
proj._ApplyCloneBundle = mock.MagicMock(return_value=True)
res = proj.Sync_NetworkHalf(clone_bundle=True, is_new=True)
self.assertIsNone(res.error)
self.assertTrue(res.remote_fetched)
proj._ApplyCloneBundle.assert_not_called()
proj._CustomFetch.assert_called_once()
class ReprojectCmdTests(unittest.TestCase):
"""Tests for the repo.reprojectcmd feature."""
@@ -3267,7 +3632,7 @@ class ReprojectCmdTests(unittest.TestCase):
) -> mock.MagicMock:
"""Run _Reproject against mocked Git; return the subprocess mock."""
with mock.patch(
"project.GitCommand", side_effect=self._git_command(**outputs)
"git_command.GitCommand", side_effect=self._git_command(**outputs)
), mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stdout="")
proj._Reproject(self.REVID)
@@ -3278,7 +3643,7 @@ class ReprojectCmdTests(unittest.TestCase):
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
with mock.patch(
"project.GitCommand", side_effect=self._git_command()
"git_command.GitCommand", side_effect=self._git_command()
), mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stdout="")
proj._Reproject(self.REVID)
@@ -3338,7 +3703,7 @@ class ReprojectCmdTests(unittest.TestCase):
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
with mock.patch(
"project.GitCommand", side_effect=self._git_command()
"git_command.GitCommand", side_effect=self._git_command()
), mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(
returncode=3, stdout="disk on fire\n"
@@ -3354,7 +3719,7 @@ class ReprojectCmdTests(unittest.TestCase):
proj = self._get_project(tempdir)
proj.work_git.GetHead.side_effect = [self.HEAD_ID, self.REVID]
with mock.patch(
"project.GitCommand", side_effect=self._git_command()
"git_command.GitCommand", side_effect=self._git_command()
), mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stdout="")
with self.assertRaises(project.LocalSyncFail) as e:
@@ -3372,7 +3737,7 @@ class ReprojectCmdTests(unittest.TestCase):
self.REVID, # HEAD after command
]
with mock.patch(
"project.GitCommand", side_effect=self._git_command()
"git_command.GitCommand", side_effect=self._git_command()
), mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stdout="")
with self.assertRaises(project.LocalSyncFail) as e:
@@ -3384,7 +3749,7 @@ class ReprojectCmdTests(unittest.TestCase):
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
with mock.patch(
"project.GitCommand",
"git_command.GitCommand",
side_effect=self._git_command(diff="lib/a.c\n"),
), mock.patch("subprocess.run") as mock_run:
mock_run.return_value = mock.MagicMock(returncode=0, stdout="")
@@ -3428,8 +3793,10 @@ class ReprojectCmdTests(unittest.TestCase):
proj.bare_ref = mock.MagicMock()
if branch:
proj.bare_ref.head = project.R_HEADS + branch
proj.bare_ref.all = {project.R_HEADS + branch: head} if head else {}
proj.bare_ref.head = git_refs.R_HEADS + branch
proj.bare_ref.all = (
{git_refs.R_HEADS + branch: head} if head else {}
)
# A branch that does not track upstream; tests that need one
# tracking upstream replace this with _tracking_branch().
proj.GetBranch = mock.MagicMock(
@@ -3442,7 +3809,7 @@ class ReprojectCmdTests(unittest.TestCase):
def _revlist(*args: Any, **kwargs: Any) -> List[str]:
if kwargs.get("format"):
return list(local_changes)
if project.not_rev(project.HEAD) in args:
if project.not_rev(git_refs.HEAD) in args:
if "--count" in args:
return [str(len(upstream_gain))]
return list(upstream_gain)
@@ -3545,7 +3912,7 @@ class ReprojectCmdTests(unittest.TestCase):
self.assertTrue(syncbuf.Finish())
proj._Reproject.assert_called_once_with(self.REVID, verbose=False)
proj.work_git.UpdateRef.assert_called_once_with(
project.HEAD,
git_refs.HEAD,
self.REVID,
old=self.HEAD_ID,
message=f"merge {self.REVID}: Fast-forward",
@@ -3592,7 +3959,7 @@ class ReprojectCmdTests(unittest.TestCase):
self.assertTrue(syncbuf.Finish())
proj._Reproject.assert_called_once_with(self.REVID, verbose=False)
proj.work_git.UpdateRef.assert_called_once_with(
project.HEAD,
git_refs.HEAD,
self.REVID,
old=self.HEAD_ID,
message=f"reset: moving to {self.REVID}",
@@ -3653,7 +4020,7 @@ class ReprojectCmdTests(unittest.TestCase):
def test_metaproject_never_uses_the_command(self) -> None:
"""Test .repo/manifests and .repo/repo are checked out by Git."""
with utils_for_test.TempGitTree() as tempdir:
fakeproj = _create_manifest_project(tempdir)
fakeproj = _create_meta_project(tempdir)
fakeproj.config.SetString("repo.reprojectcmd", "echo hi")
fakeproj.config.SetBoolean("repo.uselocalgitdirs", True)
self.assertFalse(fakeproj.UseReprojectCmd)
@@ -3661,7 +4028,7 @@ class ReprojectCmdTests(unittest.TestCase):
def test_sync_reproject_cmd_requires_use_local_gitdirs(self) -> None:
"""Test that repo.reprojectcmd requires repo.uselocalgitdirs."""
with utils_for_test.TempGitTree() as tempdir:
fakeproj = _create_manifest_project(tempdir)
fakeproj = _create_meta_project(tempdir)
class DummyManifest:
is_submanifest = False
@@ -3679,6 +4046,51 @@ class ReprojectCmdTests(unittest.TestCase):
result = fakeproj.Sync(use_local_gitdirs=False)
self.assertFalse(result)
def test_get_revision_id_peels_tags_even_with_all_refs(self) -> None:
"""Test GetRevisionId resolves tags via ResolveCommit, not all_refs."""
with utils_for_test.TempGitTree() as tempdir:
proj = _create_mock_project(tempdir, use_local_gitdirs=True)
proj.revisionId = None
mock_remote = mock.MagicMock()
proj.GetRemote = mock.MagicMock(return_value=mock_remote)
proj.bare_git.ResolveCommit.return_value = self.REVID
# Branch ref in all_refs returns cached commit SHA directly.
proj.revisionExpr = "main"
mock_remote.ToLocal.return_value = "refs/remotes/origin/main"
all_refs = {
"refs/remotes/origin/main": self.HEAD_ID,
"refs/tags/v1.0": self.OTHER_ID,
}
self.assertEqual(proj.GetRevisionId(all_refs), self.HEAD_ID)
proj.bare_git.ResolveCommit.assert_not_called()
# Tag ref in all_refs may be an annotated tag object SHA, so it
# must fall through to ResolveCommit to peel to a commit SHA.
proj.revisionExpr = "refs/tags/v1.0"
mock_remote.ToLocal.return_value = "refs/tags/v1.0"
self.assertEqual(proj.GetRevisionId(all_refs), self.REVID)
proj.bare_git.ResolveCommit.assert_called_once_with(
"refs/tags/v1.0"
)
# GetCommitRevisionId skips _allrefs for tags and pinned revisionId.
with mock.patch.object(
project.Project, "_allrefs", new_callable=mock.PropertyMock
) as mock_allrefs:
proj.bare_git.ResolveCommit.reset_mock()
self.assertEqual(proj.GetCommitRevisionId(), self.REVID)
mock_allrefs.assert_not_called()
proj.bare_git.ResolveCommit.assert_called_once_with(
"refs/tags/v1.0"
)
proj.revisionId = self.HEAD_ID
proj.bare_git.ResolveCommit.reset_mock()
self.assertEqual(proj.GetCommitRevisionId(), self.HEAD_ID)
mock_allrefs.assert_not_called()
proj.bare_git.ResolveCommit.assert_not_called()
class ReprojectCmdGitTests(unittest.TestCase):
"""Tests running the reprojectcmd contract against a real Git checkout."""
@@ -3951,6 +4363,45 @@ class ReprojectCmdGitTests(unittest.TestCase):
[" M keep.txt", "?? junk"],
)
def test_sync_local_half_checks_out_an_annotated_tag(self) -> None:
"""Test Sync_LocalHalf peels an annotated tag to its commit SHA."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as topdir:
marker = os.path.join(topdir, "ran")
proj, revid = self._make_client(
topdir, f"{self.READ_TREE} && touch {marker}"
)
remote = proj.remote.url
worktree = proj.worktree
self._git(
remote,
"-c",
"user.name=Test",
"-c",
"user.email=test@example.com",
"tag",
"-a",
"-m",
"release v1.0",
"v1.0",
"HEAD",
)
self._git(
worktree, "fetch", "-q", remote, "refs/tags/v1.0:refs/tags/v1.0"
)
proj.revisionExpr = "refs/tags/v1.0"
proj.revisionId = None
clean, errors = self._sync(proj)
self.assertTrue(clean, errors)
self.assertTrue(os.path.exists(marker))
self.assertEqual(self._git(worktree, "rev-parse", "HEAD"), revid)
# Syncing a second time sees HEAD == revid and skips reprojectcmd.
os.remove(marker)
clean, errors = self._sync(proj)
self.assertTrue(clean, errors)
self.assertFalse(os.path.exists(marker))
class DownloadPatchSetTests(unittest.TestCase):
"""Tests for Project.DownloadPatchSet."""
@@ -3966,9 +4417,9 @@ class DownloadPatchSetTests(unittest.TestCase):
def test_download_patch_set_with_no_filter_and_no_tags(self) -> None:
"""Test DownloadPatchSet includes --no-filter and --no-tags."""
with mock.patch(
"project.git_require", return_value=True
"git_command.git_require", return_value=True
) as mock_git_require:
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
@@ -3994,8 +4445,8 @@ class DownloadPatchSetTests(unittest.TestCase):
def test_download_patch_set_legacy_git_omits_no_filter(self) -> None:
"""Test DownloadPatchSet omits --no-filter on git < 2.17."""
with mock.patch("project.git_require", return_value=False):
with mock.patch("project.GitCommand") as mock_git_cmd:
with mock.patch("git_command.git_require", return_value=False):
with mock.patch("git_command.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
+9 -9
View File
@@ -23,16 +23,16 @@ from unittest import mock
import pytest
from color import SetDefaultColoring
from error import RepoExitError
from repo_logging import RepoLogger
import error
import repo_logging
@mock.patch.object(RepoLogger, "error")
@mock.patch.object(repo_logging.RepoLogger, "error")
def test_log_aggregated_errors_logs_aggregated_errors(mock_error) -> None:
"""Test if log_aggregated_errors logs a list of aggregated errors."""
logger = RepoLogger(__name__)
logger = repo_logging.RepoLogger(__name__)
logger.log_aggregated_errors(
RepoExitError(
error.RepoExitError(
aggregate_errors=[
Exception("foo"),
Exception("bar"),
@@ -57,11 +57,11 @@ def test_log_aggregated_errors_logs_aggregated_errors(mock_error) -> None:
)
@mock.patch.object(RepoLogger, "error")
@mock.patch.object(repo_logging.RepoLogger, "error")
def test_log_aggregated_errors_logs_single_error(mock_error) -> None:
"""Test if log_aggregated_errors logs empty aggregated_errors."""
logger = RepoLogger(__name__)
logger.log_aggregated_errors(RepoExitError())
logger = repo_logging.RepoLogger(__name__)
logger.log_aggregated_errors(error.RepoExitError())
mock_error.assert_has_calls(
[
@@ -99,7 +99,7 @@ def test_log_with_format_string(level: int) -> None:
output = io.StringIO()
with contextlib.redirect_stderr(output):
logger = RepoLogger(__name__)
logger = repo_logging.RepoLogger(__name__)
logger.log(level, "%s", "100% pass")
assert re.search(
+3
View File
@@ -25,6 +25,9 @@ def test_trace_max_size_enforced(monkeypatch: pytest.MonkeyPatch) -> None:
"""Check Trace behavior."""
content = "git chicken"
# Enable trace for the test, in case users have it disabled.
monkeypatch.setattr(repo_trace, "_TRACE", True)
with repo_trace.Trace(content, first_trace=True):
pass
first_trace_size = os.path.getsize(repo_trace._TRACE_FILE)
+13 -13
View File
@@ -18,8 +18,8 @@ from unittest import mock
import pytest
from error import GitError
from git_command import GitCommand
import error
import git_command
from subcmds import cherry_pick
@@ -30,12 +30,12 @@ def test_resolve_reference_uses_one_typed_batch_request(
commit = "tree " + "2" * 40 + "\n\nSubject 🚀\n\nBody\n"
output = oid + " commit " + str(len(commit.encode("utf-8"))) + "\n"
output += commit + "\n"
command = mock.create_autospec(GitCommand, instance=True)
command = mock.create_autospec(git_command.GitCommand, instance=True)
command.stdout = output
command.stderr = ""
command.Wait.return_value = 0
run_git = mock.create_autospec(GitCommand, return_value=command)
monkeypatch.setattr(cherry_pick, "GitCommand", run_git)
run_git = mock.create_autospec(git_command.GitCommand, return_value=command)
monkeypatch.setattr(git_command, "GitCommand", run_git)
resolved, contents = cherry_pick.CherryPick()._ResolveReference("topic")
@@ -54,32 +54,32 @@ def test_resolve_reference_uses_one_typed_batch_request(
def test_resolve_reference_rejects_missing_object(
monkeypatch: pytest.MonkeyPatch,
) -> None:
command = mock.create_autospec(GitCommand, instance=True)
command = mock.create_autospec(git_command.GitCommand, instance=True)
command.stdout = "topic^{commit} missing\n"
command.stderr = ""
command.Wait.return_value = 0
monkeypatch.setattr(
cherry_pick,
git_command,
"GitCommand",
mock.create_autospec(GitCommand, return_value=command),
mock.create_autospec(git_command.GitCommand, return_value=command),
)
with pytest.raises(GitError, match="commit topic not found"):
with pytest.raises(error.GitError, match="commit topic not found"):
cherry_pick.CherryPick()._ResolveReference("topic")
def test_resolve_reference_rejects_ambiguous_object(
monkeypatch: pytest.MonkeyPatch,
) -> None:
command = mock.create_autospec(GitCommand, instance=True)
command = mock.create_autospec(git_command.GitCommand, instance=True)
command.stdout = "topic^{commit} ambiguous\n"
command.stderr = ""
command.Wait.return_value = 0
monkeypatch.setattr(
cherry_pick,
git_command,
"GitCommand",
mock.create_autospec(GitCommand, return_value=command),
mock.create_autospec(git_command.GitCommand, return_value=command),
)
with pytest.raises(GitError, match="commit topic not found"):
with pytest.raises(error.GitError, match="commit topic not found"):
cherry_pick.CherryPick()._ResolveReference("topic")
+343
View File
@@ -0,0 +1,343 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the subcmds/config.py module."""
from pathlib import Path
from typing import Iterator, List
from unittest import mock
import pytest
import command
import error
import git_config
from subcmds import config
@pytest.mark.parametrize(
"argv",
[
[],
["get"],
["get", "a.b", "c"],
["set"],
["set", "a.b"],
["set", "a.b", "c", "d"],
["unset"],
["unset", "a.b", "c"],
["list", "a.b"],
],
)
def test_validate_options_usage(argv: List[str]) -> None:
"""Wrong number of arguments for an action shows usage."""
cmd = config.Config()
opts, args = cmd.OptionParser.parse_args(argv)
with pytest.raises(command.UsageError):
cmd.ValidateOptions(opts, args)
@pytest.mark.parametrize(
"argv",
[
["bogus"],
["a.b", "c"],
["--type=bool", "unset", "a.b"],
["--type=bool", "list"],
["-z", "set", "a.b", "c"],
["-z", "unset", "a.b"],
],
)
def test_validate_options_error(argv: List[str]) -> None:
"""Unknown actions and inapplicable options are rejected."""
cmd = config.Config()
opts, args = cmd.OptionParser.parse_args(argv)
with pytest.raises(SystemExit):
cmd.ValidateOptions(opts, args)
def test_validate_options_unknown_type() -> None:
"""--type only accepts the supported types."""
cmd = config.Config()
with pytest.raises(SystemExit):
cmd.OptionParser.parse_args(["--type=path", "get", "a.b"])
@pytest.mark.parametrize(
"argv",
[
["get", "sync.smartsync"],
["set", "sync.smartsync", "true"],
["unset", "sync.smartsync"],
["list"],
["--type=bool", "get", "sync.smartsync"],
["--type=int", "set", "a.b", "1"],
["-z", "get", "sync.smartsync"],
["--null", "list"],
],
)
def test_validate_options_accepts(argv: List[str]) -> None:
"""Valid invocations pass validation."""
cmd = config.Config()
opts, args = cmd.OptionParser.parse_args(argv)
cmd.ValidateOptions(opts, args)
@pytest.fixture
def cfg(tmp_path: Path) -> Iterator[git_config.RepoConfig]:
"""A RepoConfig backed by a temp file, used as the per-user config."""
cfg = git_config.RepoConfig(configfile=tmp_path / "config")
with mock.patch.object(git_config.RepoConfig, "ForUser", return_value=cfg):
yield cfg
def run(argv: List[str], capsys: pytest.CaptureFixture) -> str:
"""Runs `repo config` with |argv| and returns its stdout."""
cmd = config.Config()
opts, args = cmd.OptionParser.parse_args(argv)
cmd.ValidateOptions(opts, args)
capsys.readouterr()
cmd.Execute(opts, args)
return capsys.readouterr().out
def test_set_creates_file(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""Setting a value writes it to the config file."""
run(["set", "sync.smartsync", "true"], capsys)
assert "smartsync = true" in Path(cfg.file).read_text()
assert cfg.GetBoolean("sync.smartsync") is True
def test_set_then_get(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""Getting a value prints what was set."""
run(["set", "sync.smartsync", "false"], capsys)
assert run(["get", "sync.smartsync"], capsys) == "false\n"
def test_set_overwrites(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""Setting an existing key replaces its value."""
run(["set", "sync.smartsync", "true"], capsys)
run(["set", "sync.smartsync", "false"], capsys)
assert run(["get", "sync.smartsync"], capsys) == "false\n"
assert Path(cfg.file).read_text().count("smartsync") == 1
def test_set_invalid_name(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""git's own validation of the name is surfaced as a clean error."""
with pytest.raises(config.Error) as e:
run(["set", "nodot", "true"], capsys)
assert "nodot" in str(e.value)
@pytest.mark.parametrize(
"value, expected",
[
("true", "true"),
("Yes", "true"),
("on", "true"),
("1", "true"),
("false", "false"),
("No", "false"),
("off", "false"),
("0", "false"),
],
)
def test_set_type_bool(
cfg: git_config.RepoConfig,
capsys: pytest.CaptureFixture,
value: str,
expected: str,
) -> None:
"""--type=bool canonicalizes accepted spellings on set."""
run(["--type=bool", "set", "sync.smartsync", value], capsys)
assert run(["get", "sync.smartsync"], capsys) == f"{expected}\n"
@pytest.mark.parametrize(
"value, expected",
[
("42", "42"),
("1k", "1024"),
("2M", "2097152"),
("1g", "1073741824"),
("0x10", "16"),
],
)
def test_set_type_int(
cfg: git_config.RepoConfig,
capsys: pytest.CaptureFixture,
value: str,
expected: str,
) -> None:
"""--type=int canonicalizes accepted spellings on set."""
run(["--type=int", "set", "section.num", value], capsys)
assert run(["get", "section.num"], capsys) == f"{expected}\n"
@pytest.mark.parametrize(
"argv",
[
["--type=bool", "set", "sync.smartsync", "maybe"],
["--type=bool", "set", "sync.smartsync", "2"],
["--type=int", "set", "section.num", "lots"],
["--type=int", "set", "section.num", "1x"],
["--type=int", "set", "section.num", ""],
],
)
def test_set_type_rejects_bad_values(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture, argv: List[str]
) -> None:
"""--type rejects values that do not parse and writes nothing."""
with pytest.raises(config.Error):
run(argv, capsys)
assert not Path(cfg.file).exists()
def test_get_type_canonicalizes(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""--type on get prints the canonical form of a stored value."""
Path(cfg.file).write_text("[a]\n\tb = Yes\n\tn = 1k\n")
assert run(["--type=bool", "get", "a.b"], capsys) == "true\n"
assert run(["--type=int", "get", "a.n"], capsys) == "1024\n"
with pytest.raises(config.Error):
run(["--type=int", "get", "a.b"], capsys)
def test_get_missing_exits_quietly(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""Reading an unset key exits 1 without output, like git config."""
cmd = config.Config()
opts, args = cmd.OptionParser.parse_args(["get", "sync.smartsync"])
with pytest.raises(config.NotSetError) as e:
cmd.Execute(opts, args)
assert e.value.exit_code == 1
assert capsys.readouterr().out == ""
def test_get_multiple_values(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""All values of a multi-valued key are printed."""
cfg.SetString("section.multi", ["one", "two"])
assert run(["get", "section.multi"], capsys) == "one\ntwo\n"
def test_get_valueless_key(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""A key present without a value prints an empty line, like git."""
Path(cfg.file).write_text("[sync]\n\tsmartsync\n")
assert run(["get", "sync.smartsync"], capsys) == "\n"
def test_get_null(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""-z terminates each value with NUL."""
cfg.SetString("section.multi", ["one", "two"])
assert run(["-z", "get", "section.multi"], capsys) == "one\0two\0"
def test_unset(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""unset removes the key."""
run(["set", "sync.smartsync", "true"], capsys)
run(["unset", "sync.smartsync"], capsys)
assert cfg.GetString("sync.smartsync") is None
assert "smartsync" not in Path(cfg.file).read_text()
def test_unset_missing_is_noop(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""unset on a missing key does not fail or create the file."""
run(["unset", "sync.smartsync"], capsys)
assert not Path(cfg.file).exists()
def test_list(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""list prints all keys sorted as key=value, with every value."""
run(["set", "sync.smartsync", "true"], capsys)
run(["set", "alias.ss", "sync -s"], capsys)
cfg.SetString("section.multi", ["one", "two"])
assert run(["list"], capsys) == (
"alias.ss=sync -s\n"
"section.multi=one\n"
"section.multi=two\n"
"sync.smartsync=true\n"
)
def test_list_null(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""-z separates key and value with a newline and entries with NUL."""
Path(cfg.file).write_text("[alias]\n\tss = sync -s\n[sync]\n\tsmartsync\n")
assert run(["-z", "list"], capsys) == "alias.ss\nsync -s\0sync.smartsync\0"
def test_list_valueless_key(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""list prints a bare key for a key without a value, like git."""
Path(cfg.file).write_text("[sync]\n\tsmartsync\n")
assert run(["list"], capsys) == "sync.smartsync\n"
def test_list_empty(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""list on a missing file prints nothing."""
assert run(["list"], capsys) == ""
def test_value_with_spaces(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""Values containing spaces round trip."""
run(["set", "alias.ss", "sync -s -j 4"], capsys)
assert run(["get", "alias.ss"], capsys) == "sync -s -j 4\n"
def test_key_is_case_insensitive(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""Section and name are case-insensitive, like git."""
run(["set", "Sync.SmartSync", "true"], capsys)
assert run(["get", "sync.smartsync"], capsys) == "true\n"
def test_write_failure_is_reported(
cfg: git_config.RepoConfig, capsys: pytest.CaptureFixture
) -> None:
"""A git failure while writing becomes a clean error, not a traceback."""
with mock.patch.object(
cfg, "SetString", side_effect=error.GitError("locked")
):
with pytest.raises(config.Error) as e:
run(["set", "sync.smartsync", "true"], capsys)
assert "locked" in str(e.value)
+107
View File
@@ -14,9 +14,11 @@
"""Unittests for the subcmds/gc.py module."""
from pathlib import Path
import unittest
from unittest import mock
import manifest_xml
from subcmds import gc
@@ -80,3 +82,108 @@ class GcCommand(unittest.TestCase):
ret = self.cmd.Execute(self.opt, [])
self.assertEqual(ret, 1)
self.mock_repack.assert_not_called()
def test_gc_keep_manifest_projects(tmp_path: Path) -> None:
"""Keep inactive projects that are still in the manifest."""
repodir = tmp_path / ".repo"
manifest_dir = repodir / "manifests"
manifest_file = repodir / manifest_xml.MANIFEST_FILE_NAME
repodir.mkdir()
manifest_dir.mkdir()
manifest_gitdir = repodir / "manifests.git"
manifest_gitdir.mkdir()
(manifest_gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
""",
encoding="utf-8",
)
manifest_file.write_text(
"""\
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="included" groups="active-group" />
<project name="excluded" groups="other-group" />
</manifest>
""",
encoding="utf-8",
)
manifest = manifest_xml.XmlManifest(str(repodir), str(manifest_file))
manifest.manifestProject.config.SetString(
"manifest.groups",
"active-group",
)
projects = {project.name: project for project in manifest.projects}
included = projects["included"]
excluded = projects["excluded"]
Path(included.gitdir).mkdir(parents=True)
Path(included.objdir).mkdir(parents=True)
# Simulate an inactive project with only its object cache left.
Path(excluded.objdir).mkdir(parents=True)
assert not excluded.Exists
orphan_gitdir = repodir / "projects" / "orphan.git"
orphan_objdir = repodir / "project-objects" / "orphan.git"
orphan_gitdir.mkdir()
orphan_objdir.mkdir()
cmd = gc.Gc(repodir=str(repodir), manifest=manifest)
opt, args = cmd.OptionParser.parse_args(
["--yes", "--keep-manifest-projects"]
)
opt.quiet = True
selected_projects = cmd.GetProjects([], all_manifests=True)
assert included in selected_projects
assert excluded not in selected_projects
cmd.Execute(opt, args)
assert included.Exists
assert Path(excluded.objdir).exists()
assert not orphan_gitdir.exists()
assert not orphan_objdir.exists()
def test_gc_keep_manifest_projects_preserves_repack_targets() -> None:
"""Keep manifest projects without expanding repack targets."""
cmd = gc.Gc()
opt, _ = cmd.OptionParser.parse_args(
["--keep-manifest-projects", "--repack"]
)
opt.this_manifest_only = False
with mock.patch.object(
cmd,
"GetProjects",
side_effect=[["projA"], ["manifest_projects"]],
) as get_projects, mock.patch.object(
cmd, "delete_unused_projects", return_value=0
) as delete, mock.patch.object(
cmd, "repack_projects", return_value=0
) as repack:
cmd.Execute(opt, ["projA"])
get_projects.assert_has_calls(
[
mock.call(["projA"], all_manifests=True),
mock.call(
[],
groups="all",
missing_ok=True,
all_manifests=True,
),
]
)
delete.assert_called_once_with(["manifest_projects"], opt)
repack.assert_called_once_with(["projA"], opt)
+7 -1
View File
@@ -23,7 +23,13 @@ from subcmds import init
@pytest.mark.parametrize(
"argv",
([],),
(
[],
["-s"],
["--smart-sync"],
["--no-smart-sync"],
["--head"],
),
)
def test_cli_parser_good(argv: List[str]) -> None:
"""Check valid command line options."""
+6 -5
View File
@@ -19,7 +19,8 @@ import io
from types import SimpleNamespace
from unittest import mock
from error import GitError
import error
import git_command
from subcmds import rebase
@@ -44,7 +45,7 @@ def test_resolve_onto_manifest_fallback() -> None:
project.revisionExpr = "main"
remote = mock.MagicMock()
remote.ToLocal.side_effect = GitError("Failed to resolve")
remote.ToLocal.side_effect = error.GitError("Failed to resolve")
project.GetRemote.return_value = remote
res = rebase._ResolveOntoManifest(project)
@@ -77,11 +78,11 @@ def test_execute_delegates_autostash_to_rebase() -> None:
onto_manifest=False,
this_manifest_only=False,
)
git_command = mock.MagicMock()
git_command.Wait.return_value = 0
command = mock.MagicMock()
command.Wait.return_value = 0
with mock.patch.object(
rebase, "GitCommand", return_value=git_command
git_command, "GitCommand", return_value=command
) as run_git, contextlib.redirect_stdout(io.StringIO()):
assert cmd.Execute(opt, []) == 0
+1051 -56
View File
File diff suppressed because it is too large. Load diff
+101 -4
View File
@@ -19,8 +19,7 @@ from unittest import mock
import pytest
from error import GitError
from error import UploadError
import error
from subcmds import upload
@@ -45,7 +44,9 @@ def cmd() -> upload.Upload:
def test_UploadAndReport_UploadError(cmd: upload.Upload) -> None:
"""Check UploadExitError raised when UploadError encountered."""
opt, _ = cmd.OptionParser.parse_args([])
with mock.patch.object(cmd, "_UploadBranch", side_effect=UploadError("")):
with mock.patch.object(
cmd, "_UploadBranch", side_effect=error.UploadError("")
):
with pytest.raises(upload.UploadExitError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
@@ -53,7 +54,9 @@ def test_UploadAndReport_UploadError(cmd: upload.Upload) -> None:
def test_UploadAndReport_GitError(cmd: upload.Upload) -> None:
"""Check UploadExitError raised when GitError encountered."""
opt, _ = cmd.OptionParser.parse_args([])
with mock.patch.object(cmd, "_UploadBranch", side_effect=GitError("")):
with mock.patch.object(
cmd, "_UploadBranch", side_effect=error.GitError("")
):
with pytest.raises(upload.UploadExitError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
@@ -233,3 +236,97 @@ def test_MultipleBranches_yes_without_branch_or_cbr_uses_editor(
cmd._MultipleBranches(opt, pending, _STUB_PEOPLE)
mock_edit.assert_called_once()
mock_upload.assert_called_once_with(opt, [branch1], _STUB_PEOPLE)
@pytest.mark.parametrize(
"argv, expected",
(
(["--fix", "--ignore-hooks"], "not be part of the upload"),
(["--fix", "--no-verify"], "--fix would have no effect"),
(
["--fix", "--ignore-hooks", "--no-verify"],
"--fix and --ignore-hooks are mutually exclusive",
),
),
)
def test_ValidateOptions_fix_conflicts(
cmd: upload.Upload,
capsys: pytest.CaptureFixture,
argv: List[str],
expected: str,
) -> None:
"""Check --fix is rejected with --ignore-hooks or --no-verify."""
opt, args = cmd.OptionParser.parse_args(argv)
with pytest.raises(SystemExit):
cmd.ValidateOptions(opt, args)
assert expected in capsys.readouterr().err
@pytest.mark.parametrize(
"argv",
(
[],
["--fix"],
["--fix", "--verify"],
["--fix", "-y"],
["--ignore-hooks"],
["--no-verify"],
["--ignore-hooks", "--no-verify"],
),
)
def test_ValidateOptions_allowed(cmd: upload.Upload, argv: List[str]) -> None:
"""Check valid hook option combinations are accepted."""
opt, args = cmd.OptionParser.parse_args(argv)
cmd.ValidateOptions(opt, args)
@pytest.mark.parametrize("argv,expected_quiet", [([], False), (["-q"], True)])
def test_UploadBranch_forwards_quiet(
cmd: upload.Upload, argv: List[str], expected_quiet: bool
) -> None:
"""Check _UploadBranch forwards opt.quiet to branch.UploadForReview."""
opt, _ = cmd.OptionParser.parse_args(argv)
cmd.CommonValidateOptions(opt, [])
branch = mock.MagicMock()
branch.project.dest_branch = None
with mock.patch("git_superproject.UseSuperproject", return_value=False):
cmd._UploadBranch(opt, branch, _STUB_PEOPLE)
branch.UploadForReview.assert_called_once()
_, kwargs = branch.UploadForReview.call_args
assert kwargs["quiet"] is expected_quiet
def test_UploadAndReport_quiet_suppresses_ok_summary(
cmd: upload.Upload, capsys: pytest.CaptureFixture
) -> None:
"""Check -q suppresses non-error [OK] summary output."""
opt, _ = cmd.OptionParser.parse_args(["-q"])
cmd.CommonValidateOptions(opt, [])
branch = mock.MagicMock()
branch.name = "topic"
branch.project.RelPath.return_value = "proj"
with mock.patch.object(cmd, "_UploadBranch"):
cmd._UploadAndReport(opt, [branch], _STUB_PEOPLE)
assert capsys.readouterr().err == ""
def test_UploadAndReport_quiet_still_prints_errors(
cmd: upload.Upload, capsys: pytest.CaptureFixture
) -> None:
"""Check -q still prints [FAILED] error summary when upload fails."""
opt, _ = cmd.OptionParser.parse_args(["-q"])
cmd.CommonValidateOptions(opt, [])
branch = mock.MagicMock()
branch.name = "topic"
branch.project.RelPath.return_value = "proj"
with mock.patch.object(
cmd, "_UploadBranch", side_effect=error.UploadError("boom")
):
with pytest.raises(upload.UploadExitError):
cmd._UploadAndReport(opt, [branch], _STUB_PEOPLE)
err = capsys.readouterr().err
assert "[FAILED]" in err
assert "[OK ]" not in err
+3 -2
View File
@@ -18,6 +18,7 @@ from unittest import mock
import pytest
import git_command
from subcmds import version
@@ -26,7 +27,7 @@ def test_repo_version_uses_one_pretty_format_call(
) -> None:
project = mock.MagicMock()
project.bare_git.log.return_value = "v2.0-1-g12345678\nTue, 25 Aug\n"
monkeypatch.setattr(version, "git_require", lambda _version: True)
monkeypatch.setattr(git_command, "git_require", lambda _version: True)
result = version.Version._RepoVersion(project)
@@ -43,7 +44,7 @@ def test_repo_version_keeps_old_git_fallback(
project = mock.MagicMock()
project.bare_git.describe.return_value = "v2.0"
project.bare_git.log.return_value = "Tue, 25 Aug"
monkeypatch.setattr(version, "git_require", lambda _version: False)
monkeypatch.setattr(git_command, "git_require", lambda _version: False)
result = version.Version._RepoVersion(project)
+56
View File
@@ -515,6 +515,62 @@ class TestResolveRepoRev:
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper.resolve_repo_rev(git_checkout.git_dir, "boooooooya")
@pytest.mark.parametrize(
"committish",
["refs/heads/unknown", "refs/tags/unknown", "boooooooya"],
)
@pytest.mark.parametrize("report_errors", [True, False])
def test_report_errors(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
capsys: pytest.CaptureFixture,
committish: str,
report_errors: bool,
) -> None:
"""Check errors are only printed when requested."""
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, committish, report_errors=report_errors
)
assert ("repo: error:" in capsys.readouterr().err) == report_errors
class TestRefetchIfRevMissing:
"""Check _RefetchIfRevMissing behavior."""
@pytest.mark.parametrize("missing", [False, True])
def test_refetch(
self,
repo_wrapper: wrapper.Wrapper,
capsys: pytest.CaptureFixture,
missing: bool,
) -> None:
"""Check it refetches once only if the rev doesn't resolve."""
mock_resolve = mock.Mock(
side_effect=repo_wrapper.CloneFailure() if missing else None
)
with mock.patch.object(repo_wrapper, "_Fetch") as mock_fetch:
with mock.patch.object(repo_wrapper.time, "sleep") as mock_sleep:
with mock.patch.object(
repo_wrapper, "resolve_repo_rev", mock_resolve
):
repo_wrapper._RefetchIfRevMissing(
"url", "/dst", "stable", False, False
)
# It resolves quietly; check_repo_rev reports the error.
mock_resolve.assert_called_once_with(
"/dst", "stable", report_errors=False
)
fetches = int(missing)
fetch = mock.call("url", "/dst", "origin", False, False)
assert mock_fetch.call_args_list == [fetch] * fetches
delay = repo_wrapper.REV_REFETCH_DELAY_SEC
assert mock_sleep.call_args_list == [mock.call(delay)] * fetches
warnings = capsys.readouterr().err.count(f"refetching in {delay} ")
assert warnings == fetches
class TestCheckRepoVerify:
"""Check check_repo_verify behavior."""