Compare commits

...

45 Commits

Author SHA1 Message Date
Gavin Mak 2d54384a5e sync: Add --superproject-rev flag to sync to specific revision
Allow syncing the outer manifest to a state defined by a specific
superproject revision. It updates the superproject, reads the manifest
commit from .supermanifest, and checks out the outer manifest project
to that commit.

Submanifests are then processed normally, allowing them to be updated
to the revisions specified in the new outer manifest state.

Bug: 416589884
Change-Id: I304c37a2b8794f9b74cb7e5e209a8a93762bdb52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/576321
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-20 12:28:55 -07:00
Josef Malmström 1b4e7a04be Fix submodules not synced for repeated repo
If I check out e.g. multiple branches of the same repo in one
manifest, only the submodules of one checkout are synced.

Fix this by adding the relative path  of the checkout as a key
to the mapping of derived projects, preventing overwrite.

The fix was originally proposed here:
https://issues.gerritcodereview.com/issues/40013218

Bug: 40013218
Change-Id: Ia86518ccf2c0af7bd7e4daf8d703a55b7e10ba52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581062
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
2026-05-20 06:05:57 -07:00
gurusai-voleti a94c9e2b52 Automated: Migrate gerrit/git-repo from gsutil to gcloud storage
Bug: 486536908
Change-Id: I248b093e189a3784b8959e837a5d66857f1ce0fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/577201
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Guru sai rama subbarao Voleti (xWF) <gvoleti@google.com>
Commit-Queue: Guru sai rama subbarao Voleti (xWF) <gvoleti@google.com>
2026-05-14 21:31:49 -07:00
Gavin Mak 51021fb209 abandon/start/info: Make them respect smart sync override
Call TryOverrideManifestWithSmartSync in start, abandon, and info
commands.

This ensures they pick up the pinned revisions from the smart sync
override manifest if it exists, rather than falling back to ToT from
the default manifest.

Bug: 279204331
Change-Id: I637f054a77773805daf0bf9cf5a712d82a5959b4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582503
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-14 13:19:55 -07:00
Gavin Mak d453273f06 command: Move smart sync override logic to Command base class
Deduplicate the logic for loading `smart_sync_override.xml` by
moving it from `forall.py` to the `Command` base class. This allows
other commands to reuse it to respect smart tags.

Bug: 279204331
Change-Id: I6f2f03995266c2a68c3225cacb92b2f580a89178
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582502
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-14 13:19:26 -07:00
Mike Frysinger 0129ac1c46 docs: drop period in headers
We don't do this in most places, so standardize the outliers.

Change-Id: I80c1ef16c5fa7355dcb7797bc8e56852b22c24d5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583641
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-14 09:03:56 -07:00
Mike Frysinger bf8c59f4a0 run_tests: help2man: update to latest release
Change-Id: I30e4a6452ac5504b782fe545a59ca1f58ac70385
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583621
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-05-14 09:03:52 -07:00
Josef Malmström 003f0407fc Fix missing None check in Remote.Save
This code was causing an exception in cases where `pushUrl` was set
but `projectname` was not.

This can happen when `pushUrl` is set for the manifest repo which does
not have a `projectname`. For example:

repo init --manifest-url ssh://url.to/my/manifest

cd .repo/manifests
git config remote.origin.pushurl ssh://url.to/my/manifest

repo init --manifest-url ssh://url.to/my/manifest --repo-rev main

The last `repo init` invocation causes an error.

Change-Id: Ibb68c8446880cfbac22feee595d1fd1b678c7ade
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579162
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-13 00:00:27 -07:00
Josef Malmström 1db50e49fb Add support for self referencing submodules
When working with relative submodule paths, The "./" needs special
handling similar to "../".

See information on:
https://git-scm.com/docs/git-submodule
Which currently states:
"<repository> is the URL of the new submodule’s origin repository.
This may be either an absolute URL, or (if it begins with ./ or ../),
the location relative to the superproject’s default remote
repository (Please note that to specify a repository foo.git which is
located right next to a superproject bar.git, you’ll have to use
../foo.git instead of ./foo.git - as one might expect when following
the rules for relative URLs - because the evaluation of relative URLs
in Git is identical to that of relative directories)."

The implementation also was not handling file/directory names
starting with "." or "..". Explicitly look for "./" and "../"
instead.

Change-Id: I8ae68d61fb0cbb1624183b175236e98a36e4afdb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579182
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
2026-05-13 00:00:11 -07:00
Mike Frysinger c97a306fe7 release: update-manpages: revert color filtering
We pull a pinned help2man from cipd now, so we should get stable
behavior between developers.  That means the color filtering should
not be necessary anymore, and we can drop the logic & unittest.

Change-Id: Ib53e1ce7f8d610d7f624c9a019c79dc5f438ac0d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582402
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-12 11:50:38 -07:00
Carlos Fernandez 5534f164d6 linkfile: Handle directory-to-symlink transitions safely
When a manifest changes from individual linkfiles inside a directory
(e.g. dest=".llms/rules", dest=".llms/skills") to a single linkfile
for the whole directory (e.g. dest=".llms", src="dot-llms"), two
things need to happen:

1. __linkIt must replace a real directory with a symlink.  Use
   os.rmdir() instead of platform_utils.remove() for real directories.
   rmdir only removes empty directories, so user-created content is
   never deleted.

2. UpdateCopyLinkfileList must handle the cleanup correctly:
   - Use os.rmdir() for directories (safe for non-empty)
   - Remove empty parent directories after cleaning old dests
   - Retry _CopyAndLinkFiles for all projects, since in interleaved
     sync mode _CopyAndLinkFiles runs before cleanup and may have
     failed because the directory was not yet empty

Change-Id: I0437b80beab98bce064cea81c11c47d699be91aa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569243
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
2026-05-12 11:10:43 -07:00
Mike Frysinger 67e52a120b run_tests: leverage cipd when available for help2man
This tool isn't installed on CI bot images so we've been skipping it,
but this is causing people to not run tests locally, and ignore errors.
Use cipd to pull the tool in when available.

Then revert a recent man change that the tool rejects.

Change-Id: I1030d0070fd5a624656eba7434ae6ec99b2e3f2d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582401
Reviewed-by: Greg Edelston <gredelston@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-05-12 10:45:13 -07:00
Gavin Mak d5037230e9 sync: Re-raise KeyboardInterrupt in main process
When running sync -j1, worker functions run directly in the main
process. Swallowing KeyboardInterrupt causes the loop to continue to the
next project instead of aborting.

Re-raise KeyboardInterrupt if running in the MainProcess, while
maintaining the suppression of stack traces in worker processes.

Bug: 468170157
Change-Id: I156d66bc209a265f7fa25eea0eb88737d1b51a34
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581342
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-05-12 10:41:47 -07:00
Mike Frysinger 7cc99b24c1 git_config: fix error message command output
str() inside an f-string is redundant.  Actually join the array together
as a string to show a proper error message, and include the full command
line, not the extra options passed in.

Before:
error.GitError: git config ('--null', '--list'): fatal: unable to read config file 'xxx': No such file or directory

After:
error.GitError: git config --system --includes --null --list: fatal: unable to read config file 'xxx': No such file or directory

Change-Id: I8983389aa2e0de7808991e73e636b77810f04c4b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582341
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-12 10:04:15 -07:00
Greg Edelston 12ad396f67 info: Parallelize repo info to improve performance
For a large checkout like chromiumos or android, `repo info` takes a
really long time! On my machine it took ~6 minutes. On a randomly
selected ChromiumOS cq-orchestrator build it took 4.1 minutes:
https://ci.chromium.org/b/8682060180498819729. This adds up to a lot of
wasted runtime for both humans and bots.

The problem is that `repo info` was single-threaded, which causes poor
performance when the checkout has 1000+ projects. We already have a
pattern for parallelization; let's use it.

BUG=None
TEST=Manually run, ensure no diff

Change-Id: I6b82b9495eb2a0e602a142dd3a16f09217871e1b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581921
Tested-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-12 09:59:07 -07:00
Gavin Mak a6bc1b7cf0 sync: Exclude stateless sync pruned projects from bloat check
If a project has been pruned for stateless sync, it has already
undergone aggressive garbage collection. There should be no bloat
to check for.

Change-Id: I9233a4611e05b7a0b7c097827d6f408144f7380d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581841
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-11 19:36:59 -07:00
Gavin Mak a98e422113 completion: document installation and usage in README
Add a "Shell Completion" section to README.md to document how to install
and use `completion.bash` and `completion.zsh`.

Change-Id: Ibf6c81043af6c24d45ea2dc6a6caa26d98ab7374
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581261
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-08 17:41:44 -07:00
Gavin Mak 5d8585012f sync: Suggest "git repack -a -d" in bloat warning
The warning suggests running "repo sync --auto-gc" which runs "git gc
--auto". Git may decide not to repack if its internal thresholds are not
met, leaving the warning active even after running the suggested
command.

"git repack -a -d" forces all reachable objects into a single pack and
deletes redundant packs, reducing the pack count to 1. This guarantees
that the warning (which triggers when pack count exceeds 10) goes away.

Bug: 505755299
Change-Id: I10163ef8efb7f3b7c5055378ad95051974d11b88
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580821
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-05-08 14:12:22 -07:00
Kamal Sacranie e8deeabf2a Add zsh completion
This PR adds `completion.zsh` to allow for relatively intelligent
completion of `repo` command-line arguments for all commands of the
`repo` CLI.

The `completion.zsh` can be added to your zsh completions by copying
(and renaming) the completion script to a directory that you have added
to your zsh completion path.

```
cp completion.zsh $ZSH_COMPLETION_DIRECTORY/_repo
```

You must name the file `_repo` for it to register as completion for
`repo`.

You can add a directory to your zsh completion by updating your `fpath`
variable before calling `compinit` as follows:

```
fpath=( /path/to/zsh-completions/dir $fpath )
```

Future work: Generate this file using the subcommand classes and python
reflection.

Change-Id: I6a4e785c1efcf9076bd693976ac03578836b691b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579561
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Kamal Sacranie <sacranie@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-08 07:46:00 -07:00
Gavin Mak 11428ae984 forall: Document REPO_UPSTREAM and REPO_DEST_BRANCH envvars
Change-Id: I74365295152f8828587c6b4ed93029efc6000881
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580761
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-07 20:27:08 -07:00
Carlos Fernandez 27d2232eb3 info: add --format and --include-summary/--include-projects options
Add --format={text,json} to produce machine-readable output, and
boolean options to control which sections are displayed:
  --include-summary / --no-include-summary (default: on)
  --include-projects / --no-include-projects (default: on)

The JSON output respects the include flags, so callers can request
only the fields they need (e.g. `repo info --format=json
--no-include-projects` for manifest metadata only).

Change-Id: I9641bc4023b630d9c61c5170eb86e5f3b787236f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569203
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
2026-05-07 19:23:24 -07:00
Josef Malmström e3eadd3728 sync: Fix force_checkout propagation
The force_checkout parameter was not propagated in all calls to
Checkout in Sync_LocalHalf.

Without this, repo sync --force-checkout can still fail for projects
currently on a local branch with no upstream/tracking configuration,
because the detach-to-manifest checkout was executed without -f,
leaving local modifications or untracked files able to block sync.

Change-Id: I58551388e2f906c4db96e220707a369057a71c24
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579181
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-06 08:53:27 -07:00
Gavin Mak 12cfc6036a git_superproject: Remove redundant _branch variable
Both variables were initialized to the same value and never modified.

Bug: 416589884
Change-Id: Iaa1ffffb4543d4cd9391ac679d9234fe01678861
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/576921
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-04-28 10:49:54 -07:00
Nasser Grainawi 134eeb024b tests: Add tests for repo status output
Build out status subcommand unit coverage using a minimal fake repo
checkout wired through XmlManifest.

The new tests verify:
- clean status output prints the expected project header
- modified tracked files appear with the expected status marker
- `-o` output includes the orphan section and orphan entries
- branch names shown in status reflect a started non-default branch

Change-Id: Ia7c22593d0bbdc4aed81faeb168b846f3e4016ab
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558501
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-23 17:52:09 -07:00
Nasser Grainawi 03fb18109f color: type SetDefaultColoring and drop bool states
Also add a test fixture to always disable coloring so that we get
reproducible test behavior. The few tests that want to check color
behavior (e.g. test_color.py) can still reset/change color values as
needed.

Change-Id: I1808139a63e0862c29bf81d7097e885ca8561040
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/573621
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
2026-04-22 17:05:55 -07:00
Ram Peri 5af71ce907 Add timing keyword argument for hooks
Measure the duration of the sync operation in the Execute method of the
Sync command and pass it to post-sync hooks as a standard keyword
argument (`sync_duration_seconds`).

Updates based on code review:
- Update _API_ARGS in hooks.py to allow sync_duration_seconds for post-sync hooks.
- Do not cast sync_duration_seconds to int for better granularity.
- Update docs/repo-hooks.md to document sync_duration_seconds.
- Add unit test for argument validation in test_hooks.py.

Test: Ran run_tests using venv python, all 554 tests passed.
Bug: TBD
Change-Id: Ie29e002a5d283460d993ad96c224dbf4b6d7985c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/575021
Tested-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-21 17:10:07 -07:00
Gavin Mak baa281d99e sync: Refactor to use _RunOneGC and fix config leakage
Extract _RunOneGC to handle GC on a single project. This refactoring
makes it easier to invoke GC from parallel worker tasks.

Also, avoid modifying the passed-in config dictionary in _RunOneGC by
creating a local copy, preventing unintended side effects on other
commands sharing the same config.

Bug: 498290329
Change-Id: I7b77ed6629b14b5ee3322870b9c6c8ce2bfd6ea2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574923
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-20 15:34:07 -07:00
Gavin Mak 7e9079b7cf sync: Switch to using self._bloated_projects
Store bloated projects in self._bloated_projects and print warnings at
the end of execution. This sets up for moving the check to workers.

Bug: 498290329
Change-Id: I993f1fd741db2994d480994861588eb18f6c5503
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574922
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-20 14:07:14 -07:00
Gavin Mak 61bd6b3ccb tests: Add tests for _CheckForBloatedProjects and _GCProjects
Change-Id: I6790a13929b8fc06a3197ca405dedf08d39d54c9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574926
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-20 13:31:41 -07:00
Marty Heavey 32e7327ca6 upload: Clarify partial sync message on hook failure
Demote the partial sync warning from error to info and rephrase it to
be a tip rather than an error. This prevents users from thinking that
a partial sync is the cause of their hook failures when it is often a linting failure.

The message now suggests that a full sync might help if there are
cross-project dependencies, instead of implying it will fix any issue.

Change-Id: I5d8c52b53ac315aa9f145ed069798bf201fa0815
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574262
Tested-by: Marty Heavey <mheavey@google.com>
Commit-Queue: Marty Heavey <mheavey@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-20 01:52:25 -07:00
Gavin Mak 4f707ff91e sync: Provide feedback during post-sync operations
After the main sync progress bar finishes, there's a pause while some
post-sync operations run. Print something to provide feedback so the
user doesn't think repo has hung.

Bug: 503869525
Change-Id: I695fd560e60dcb394e6844a56c8a336ca1f71c74
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574425
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-04-17 15:47:15 -07:00
Gavin Mak e2671c184e progress: Ignore updates after progress ends
This addresses the occasional "..working.." message at the end of sync.

Bug: 503869525
Change-Id: I489d29fa8ae588d77abb7fee5f157800d4abb368
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574482
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
2026-04-17 15:43:14 -07:00
Becky Siegel b43a20b859 project: Avoid skipping fetches for shallow clones without .git/shallow
When optimizing fetches for projects with immutable revisions, the fetch
should not be skipped if the project is configured for a shallow clone
(depth > 0) but the .git/shallow file is missing. The absence of the
.git/shallow file means the repository is not a shallow clone, or the
shallow clone is incomplete, so a fetch is necessary to ensure the
revision is present.

Bug: 503081454
Change-Id: Ic3549612bcd69050a926652ee4e522c79ad8124c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/573821
Tested-by: Becky Siegel <beckysiegel@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Becky Siegel <beckysiegel@google.com>
2026-04-17 09:16:36 -07:00
Miyako.Enei 8869a30283 project: Drop --no-deref from update-ref --stdin
repo calls `git update-ref --stdin` when updating multiple refs during
repo init and repo sync. Historically, `--no-deref` was also passed.

Older Git 2.17 which we still support rejects the combination of
`--stdin` and `--no-deref`, emitting a usage error even when the stdin
input is valid.

The `--no-deref` option is only meaningful when updating symbolic refs
such as HEAD. The stdin-based update-ref path only operates on explicit
refs (tags, remote refs, alternates) and never symbolic refs.

Remove the unnecessary option to restore compatibility with Git 2.17
while preserving identical behavior on newer Git versions.

Tested with:
  - Git 2.17.1
  - Git 2.34.1

Change-Id: I22001de03800f5699b26a40bc1fb1fec002ed048
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/571721
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Enei <miyako.enei@alpsalpine.com>
Tested-by: Enei <miyako.enei@alpsalpine.com>
2026-04-12 16:39:28 -07:00
Gavin Mak 3b0eebeccf project: implement stateless sync pruning logic
Implement in-situ shallow re-fetching and garbage collection logic.
Enables repositories with sync-strategy="stateless" to reclaim disk
space by running reflog expire and git gc --prune=now if the working
tree is clean and has no local commits.

Bug: 498730431
Change-Id: I940bdc9b74da29d3f7b13566667dcddea769ebd3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/568463
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-09 12:09:40 -07:00
Gavin Mak 00991bfb42 manifest: Add sync-strategy attribute to project elements
The only supported sync-strategy is "stateless". The intent is to keep
the local workspace as small as possible by not keeping history during
syncs. This prevents disk space waste for projects with large binaries
where we only care about the current version.

A follow up change will implement the logic.

Bug: 498730431
Change-Id: I84a436a9ca2492893163c6cfda6c28dc62a568f0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/568462
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-09 12:09:28 -07:00
Nasser Grainawi e8338b54bd tests: Convert forall subcmd test to pytest
Rewrite tests/test_subcmds_forall.py from unittest.TestCase to pytest
function-style tests to match the surrounding test suite conventions.

Replace setUp/tearDown and class-based helpers with tmp_path-based
setup, switch stdout capture to contextlib.redirect_stdout, and keep the
existing behavior checks intact (all eight projects are invoked exactly
once).

Change-Id: I9243f3461aa6850f867bdb864f4a34c442f817f6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569821
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-04-09 11:51:47 -07:00
Gavin Mak 951666fb23 gc: Fix hang during repack in partial clones
Add `--missing=allow-promisor` to `git rev-list` calls in
`repack_projects`. This prevents Git from auto-fetching missing objects
from the promisor remote, which can cause stalls due to sequential
network requests.

Also add a Git version check to ensure Git is at least 2.17.0 before
running `--repack`, as `--missing=allow-promisor` was introduced in that
version.

Bug: 500133631
Change-Id: I2dcf9b46fac4c6a53a3c2a46f06f61d6aec40f2f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/570361
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
2026-04-08 12:54:39 -07:00
Carlos Fernandez 854b330967 test_wrapper: add test for repo script executable permission
Add a test to verify that the repo launcher script has the
executable bit set, guarding against accidental permission changes.

Change-Id: I314658b57ed174673188fbbc5962d9fdeefac97d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569242
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
2026-04-06 14:16:04 -07:00
Mike Frysinger 654690e1b8 tests: convert more tests to pytest
Change-Id: Id4d48b61dc435564c336385bbc4944eb475d1942
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569443
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-06 11:36:39 -07:00
Mike Frysinger ac2be4c089 tests: convert __file__ usage to pathlib
Change-Id: I2408b0ac97629f0d5fc92779b78bf1ff159a6f83
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569442
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-04-06 11:15:58 -07:00
Mike Frysinger 3d819e8e3e tests: unify fixture() helper with Path constant
Change-Id: I63751042391f5cc3e06af7067bc83d67bd0716dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569441
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-06 11:15:14 -07:00
Carlos Fernandez 573983948a Fix all flake8 warnings from newer flake8-bugbear and flake8-comprehensions
Address warnings introduced by flake8-bugbear 24.12.12 and
flake8-comprehensions 3.16.0:

- C408: Replace dict()/list() calls with literal {} and []
- C413: Remove unnecessary list() around sorted()
- C414: Remove unnecessary list() inside sorted()
- C419: Suppress intentional list comprehension in all() (noqa)
- B001: Replace bare except with except Exception
- B006: Replace mutable default arguments with None
- B010: Replace setattr() with direct attribute assignment
- B017: Use RuntimeError instead of Exception in tests
- B019: Suppress lru_cache on methods for long-lived objects (noqa)
- B033: Remove duplicate item in set literal

Change-Id: If4693d3e946200bbc22f689f7b94da604addcb80
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/566321
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-03 07:50:52 -07:00
Gavin Mak 3f3c681a02 project: Refactor GetHead to use symbolic-ref first
Simplify branch resolution and optimize unborn branch detection by
prioritizing symbolic-ref over rev-parse.

Change-Id: Ic62dcb87cd051dafb00d520b1157be2e32abc2ab
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/563222
Reviewed-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-02 13:57:05 -07:00
Sam Saccone 242e97d9dd Implement command forgiveness with autocorrect
Similar to `git`, when a user types an unknown command like `repo tart`,
we now use `difflib.get_close_matches` to suggest similar commands.

If `help.autocorrect` is set in the git config, it will optionally
prompt the user to automatically run the assumed command, or wait
for a configured delay before executing it.

Verification Steps:
1. Created a dummy repo project locally.
2. Verified `help.autocorrect=0|false|off|no|show` suggests
   command and exits.
3. Verified `help.autocorrect=1|true|on|yes|immediate`
   automatically runs suggestion.
4. Verified `help.autocorrect=<number>` runs after
   `<number>*0.1` seconds.
5. Verified `help.autocorrect=never` exits immediately without
   suggestions.
6. Verified `help.autocorrect=prompt` asks user to accept [y/n]
   and handles correctly.

BUG: b/489753302

Change-Id: I6dcd63229cbd7badf5404459b48690c68f5b4857
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558021
Tested-by: Sam Saccone <samccone@google.com>
Commit-Queue: Sam Saccone <samccone@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-03-24 15:47:37 -07:00
60 changed files with 4655 additions and 1647 deletions
+1
View File
@@ -6,6 +6,7 @@ __pycache__
/dist /dist
.repopickle_* .repopickle_*
/repoc /repoc
/.cipd_bin
/.tox /.tox
/.venv /.venv
+2 -2
View File
@@ -24,7 +24,7 @@ However there are some differences, so please review and familiarize
yourself with the following relevant bits. yourself with the following relevant bits.
## Make separate commits for logically separate changes. ## Make separate commits for logically separate changes
Unless your patch is really trivial, you should not be sending out a patch that Unless your patch is really trivial, you should not be sending out a patch that
was generated between your working tree and your commit head. was generated between your working tree and your commit head.
@@ -118,7 +118,7 @@ your patch. It is virtually impossible to remove a patch once it
has been applied and pushed out. has been applied and pushed out.
## Sending your patches. ## Sending your patches
Do not email your patches to anyone. Do not email your patches to anyone.
+37
View File
@@ -49,6 +49,43 @@ $ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/.bin/repo
$ chmod a+rx ~/.bin/repo $ chmod a+rx ~/.bin/repo
``` ```
## Shell Completion
Repo includes completion scripts for Bash and Zsh.
### Bash
To enable completion in Bash, source `completion.bash` in your `~/.bashrc`:
```sh
source /path/to/git-repo/completion.bash
```
### Zsh
To enable completion in Zsh, you can either:
1. Copy or symlink `completion.zsh` to a file named `_repo` in a directory in your `$fpath`:
```sh
mkdir -p ~/.zsh/completion
# You can copy the file:
cp /path/to/git-repo/completion.zsh ~/.zsh/completion/_repo
# Or symlink it:
ln -s /path/to/git-repo/completion.zsh ~/.zsh/completion/_repo
```
Then add that directory to your `fpath` in `~/.zshrc` before `compinit`:
```zsh
fpath=(~/.zsh/completion $fpath)
autoload -Uz compinit
compinit
```
2. Or source the file directly and call `compdef` in your `~/.zshrc`:
```zsh
source /path/to/git-repo/completion.zsh
compdef _repo repo
```
[new-bug]: https://issues.gerritcodereview.com/issues/new?component=1370071 [new-bug]: https://issues.gerritcodereview.com/issues/new?component=1370071
[issue tracker]: https://issues.gerritcodereview.com/issues?q=is:open%20componentid:1370071 [issue tracker]: https://issues.gerritcodereview.com/issues?q=is:open%20componentid:1370071
+32
View File
@@ -0,0 +1,32 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file contains version pins of a few tools used by run_tests.
# Pin resolved versions in the repo, to reduce trust in the CIPD backend.
#
# Most of these tools are generated via builders at
# https://ci.chromium.org/p/infra/g/infra/console
#
# For these, the git revision is the one of
# https://chromium.googlesource.com/infra/infra.git.
#
# To regenerate them (after modifying this file):
# cipd ensure-file-resolve -ensure-file cipd_manifest.txt
$ResolvedVersions cipd_manifest.versions
# Supported platforms. Feel free to add more as long as the tools work.
$VerifiedPlatform linux-amd64 linux-arm64 mac-amd64 mac-arm64
infra/3pp/tools/help2man/${platform} version:3@1.49.3
+18
View File
@@ -0,0 +1,18 @@
# This file is auto-generated by 'cipd ensure-file-resolve'.
# Do not modify manually. All changes will be overwritten.
infra/3pp/tools/help2man/linux-amd64
version:3@1.49.3
82ySLC4--mvjnmuwj_j_2odHLj5zFaNFpWvfCyMunY0C
infra/3pp/tools/help2man/linux-arm64
version:3@1.49.3
szfWfjI4RyCT6gIB48CeUOGG6_Un4wfbIb0Vvs5M9MUC
infra/3pp/tools/help2man/mac-amd64
version:3@1.49.3
t7iyQx5TnGbuJfj6zMTvjPdBl-Cb_w5papUh80NI2S8C
infra/3pp/tools/help2man/mac-arm64
version:3@1.49.3
t5eD-eLKRpnSifZdbi072U8bnPfGye1pIMmQ07nxJ7kC
+4 -3
View File
@@ -14,6 +14,7 @@
import os import os
import sys import sys
from typing import Optional
import pager import pager
@@ -84,7 +85,7 @@ def _Color(fg=None, bg=None, attr=None):
DEFAULT = None DEFAULT = None
def SetDefaultColoring(state): def SetDefaultColoring(state: Optional[str]) -> None:
"""Set coloring behavior to |state|. """Set coloring behavior to |state|.
This is useful for overriding config options via the command line. This is useful for overriding config options via the command line.
@@ -97,9 +98,9 @@ def SetDefaultColoring(state):
state = state.lower() state = state.lower()
if state in ("auto",): if state in ("auto",):
DEFAULT = state DEFAULT = state
elif state in ("always", "yes", "true", True): elif state in ("always", "yes", "true"):
DEFAULT = "always" DEFAULT = "always"
elif state in ("never", "no", "false", False): elif state in ("never", "no", "false"):
DEFAULT = "never" DEFAULT = "never"
+23 -1
View File
@@ -101,6 +101,11 @@ class Command:
def WantPager(self, _opt): def WantPager(self, _opt):
return False return False
@staticmethod
def is_multiprocessing_active() -> bool:
"""Whether the current process is a worker in a pool."""
return multiprocessing.current_process().name != "MainProcess"
def ReadEnvironmentOptions(self, opts): def ReadEnvironmentOptions(self, opts):
"""Set options from environment variables.""" """Set options from environment variables."""
@@ -407,7 +412,8 @@ class Command:
for project in all_projects_list: for project in all_projects_list:
if submodules_ok or project.sync_s: if submodules_ok or project.sync_s:
derived_projects.update( derived_projects.update(
(p.name, p) for p in project.GetDerivedSubprojects() (p.RelPath(local=False), p)
for p in project.GetDerivedSubprojects()
) )
all_projects_list.extend(derived_projects.values()) all_projects_list.extend(derived_projects.values())
for project in all_projects_list: for project in all_projects_list:
@@ -511,6 +517,22 @@ class Command:
) )
return result return result
def GetSmartSyncOverridePath(self, manifest=None) -> str:
"""Return the path where smart sync writes its override manifest."""
if manifest is None:
manifest = self.manifest
return os.path.join(
manifest.manifestProject.worktree, "smart_sync_override.xml"
)
def TryOverrideManifestWithSmartSync(self, manifest=None) -> None:
"""Override manifest with smart_sync_override.xml if it exists."""
if manifest is None:
manifest = self.manifest
smart_sync_manifest_path = self.GetSmartSyncOverridePath(manifest)
if os.path.isfile(smart_sync_manifest_path):
manifest.Override(smart_sync_manifest_path)
def ManifestList(self, opt): def ManifestList(self, opt):
"""Yields all of the manifests to traverse. """Yields all of the manifests to traverse.
+449
View File
@@ -0,0 +1,449 @@
#compdef _repo repo
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
_repo() {
local context state line
typeset -A opt_args
local -a subcommands
subcommands=(
'abandon:Abandon a development branch'
'branches:View current topic branches'
'checkout:Checkout a branch'
'cherry-pick:Cherry-pick a change'
'diff:Show changes between commit and working tree'
'diffmanifests:Show differences between two manifests'
'download:Download and find a change'
'forall:Run a shell command in each project'
'gc:Garbage collect'
'grep:Print lines matching a pattern'
'help:Display help about a command'
'info:Get info about the manifest'
'init:Initialize a repo client'
'list:List projects and their names'
'manifest:Manifest management'
'overview:Display overview of topic branches'
'prune:Prune topic branches'
'rebase:Rebase local branches'
'selfupdate:Update repo'
'smartsync:Update working tree to latest known good revision'
'stage:Stage file(s) for commit'
'start:Start a new branch for development'
'status:Show the working tree status'
'sync:Update working tree'
'upload:Upload changes to the review server'
'version:Display version'
'wipe:Wipe uncommitted changes'
)
local -a global_opts
global_opts=(
'(-h --help)'{-h,--help}'[Show help message]'
'--help-all[Show help message for all commands]'
'(-p --paginate)'{-p,--paginate}'[Pipe all output into less]'
'--no-pager[Do not pipe output into a pager]'
'--color=[Control color output]:color:(always never auto)'
'--trace[Trace git command execution]'
'--trace-to-stderr[Trace git command execution to stderr]'
'--trace-python[Trace python execution]'
'--time[Time execute of command]'
'--version[Show version]'
'--show-toplevel[Show top level of workspace]'
'--event-log=[File to log events to]:file:_files'
'--git-trace2-event-log=[File to log git trace2 events to]:file:_files'
'--submanifest-path=[Path to submanifest]:path:_files'
)
_arguments -C \
${global_opts} \
'1: :->cmds' \
'*:: :->subcmds'
case ${state} in
cmds)
_describe -t commands 'repo command' subcommands
;;
subcmds)
local cmd=${words[1]}
curcontext="${curcontext%:*}-${cmd}:"
local -a common_opts
common_opts=(
'(-h --help)'{-h,--help}'[Show help message]'
'(-v --verbose)'{-v,--verbose}'[Show verbose messages]'
'(-q --quiet)'{-q,--quiet}'[Show only errors]'
'(-j --jobs)'{-j,--jobs=}'[Number of jobs to run in parallel]:jobs:'
'--outer-manifest[Use outer manifest]'
'--no-outer-manifest[Do not use outer manifest]'
'--this-manifest-only[Only use this manifest]'
'--no-this-manifest-only[Do not only use this manifest]'
'--all-manifests[Use all manifests]'
)
case ${cmd} in
abandon)
_arguments \
${common_opts} \
'--all[Abandon all branches]' \
'1: :->branch' \
'*: :->project'
;;
branches)
_arguments \
${common_opts} \
'*: :->project'
;;
checkout)
_arguments \
${common_opts} \
'1: :->branch' \
'*: :->project'
;;
cherry-pick)
_arguments \
${common_opts} \
'1: :_message "sha1"'
;;
diff)
_arguments \
${common_opts} \
'(-u --absolute)'{-u,--absolute}'[Show absolute paths]' \
'*: :->project'
;;
diffmanifests)
_arguments \
'--raw[Show raw diff]' \
'--no-color[Do not show color]' \
'--pretty-format=[Pretty format]:format:' \
'1: :_files -g "*.xml"' \
'2: :_files -g "*.xml"'
;;
download)
_arguments \
${common_opts} \
'(-b --branch)'{-b,--branch=}'[One or more branches to check]:branch:' \
'(-c --cherry-pick)'{-c,--cherry-pick}'[Cherry-pick the change]' \
'(-x --record-origin)'{-x,--record-origin}'[Record origin of cherry-pick]' \
'(-r --revert)'{-r,--revert}'[Revert the change]' \
'(-f --ff-only)'{-f,--ff-only}'[Force fast-forward]' \
'*: :_message "change[/patchset]"'
;;
forall)
_arguments \
${common_opts} \
'(-r --regex)'{-r,--regex}'[Execute command only on projects matching regex]' \
'(-i --inverse-regex)'{-i,--inverse-regex}'[Execute command only on projects not matching regex]' \
'(-g --groups)'{-g,--groups=}'[Execute command only on projects matching groups]:groups:' \
'(-c --command)'{-c,--command}'[Command to execute]' \
'(-e --abort-on-errors)'{-e,--abort-on-errors}'[Abort on errors]' \
'--ignore-missing[Ignore missing projects]' \
'--interactive[Run interactively]' \
'-p[Show project headers]' \
'*: :->project'
;;
gc)
_arguments \
${common_opts} \
'(-n --dry-run)'{-n,--dry-run}'[Dry run]' \
'(-y --yes)'{-y,--yes}'[Answer yes to all prompts]' \
'--repack[Repack objects]'
;;
grep)
_arguments \
${common_opts} \
'--cached[Search cached files]' \
'(-r --revision)'{-r,--revision=}'[Search in revision]:revision:' \
'-e[Pattern]' \
'(-i --ignore-case)'{-i,--ignore-case}'[Ignore case]' \
'(-a --text)'{-a,--text}'[Treat all files as text]' \
'-I[Do not match binary files]' \
'(-w --word-regexp)'{-w,--word-regexp}'[Match word boundaries]' \
'(-v --invert-match)'{-v,--invert-match}'[Invert match]' \
'(-G --basic-regexp)'{-G,--basic-regexp}'[Basic regexp]' \
'(-E --extended-regexp)'{-E,--extended-regexp}'[Extended regexp]' \
'(-F --fixed-strings)'{-F,--fixed-strings}'[Fixed strings]' \
'--all-match[All match]' \
'--and[And]' \
'--or[Or]' \
'--not[Not]' \
'-([Open paren]' \
'-)[Close paren]' \
'-n[Line number]' \
'-C[Context]:lines:' \
'-B[Before context]:lines:' \
'-A[After context]:lines:' \
'-l[Name only]' \
'--name-only[Name only]' \
'--files-with-matches[Files with matches]' \
'-L[Files without match]' \
'--files-without-match[Files without match]' \
'1: :->pattern' \
'*: :->project'
;;
help)
_arguments \
'(-a --all)'{-a,--all}'[Show all commands]' \
'--help-all[Show help message for all commands]' \
'1: :->help_cmds'
;;
info)
_arguments \
${common_opts} \
'(-d --diff)'{-d,--diff}'[Show diff]' \
'(-o --overview)'{-o,--overview}'[Show overview]' \
'(-c --current-branch)'{-c,--current-branch}'[Show current branch]' \
'--no-current-branch[Do not show current branch]' \
'(-l --local-only)'{-l,--local-only}'[Local only]' \
'*: :->project'
;;
init)
_arguments \
'(-u --manifest-url)'{-u,--manifest-url=}'[Manifest URL]:url:' \
'(-b --manifest-branch)'{-b,--manifest-branch=}'[Manifest branch]:branch:' \
'--manifest-upstream-branch=[Manifest upstream branch]:branch:' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'(-g --groups)'{-g,--groups=}'[Restrict manifest projects to groups]:groups:' \
'(-p --platform)'{-p,--platform=}'[Restrict manifest projects to platform]:platform:' \
'--submodules[Sync submodules]' \
'--standalone-manifest[Standalone manifest]' \
'--manifest-depth=[Manifest depth]:depth:' \
'(-c --current-branch)'{-c,--current-branch}'[Sync current branch only]' \
'--no-current-branch[Do not sync current branch only]' \
'--tags[Sync tags]' \
'--no-tags[Do not sync tags]' \
'--mirror[Mirror]' \
'--archive[Archive]' \
'--worktree[Worktree]' \
'--reference=[Reference repository]:repository:_files -/' \
'--dissociate[Dissociate from reference]' \
'--depth=[Depth]:depth:' \
'--partial-clone[Partial clone]' \
'--no-partial-clone[Do not partial clone]' \
'--partial-clone-exclude=[Exclude from partial clone]:projects:' \
'--clone-filter=[Clone filter]:filter:' \
'--use-superproject[Use superproject]' \
'--no-use-superproject[Do not use superproject]' \
'--clone-bundle[Use clone bundle]' \
'--no-clone-bundle[Do not use clone bundle]' \
'--git-lfs[Use git lfs]' \
'--no-git-lfs[Do not use git lfs]' \
'--repo-url=[Repo URL]:url:' \
'--repo-rev=[Repo revision]:revision:' \
'--no-repo-verify[Do not verify repo]' \
'--config-name[Use config name]'
;;
list)
_arguments \
${common_opts} \
'(-r --regex)'{-r,--regex}'[Filter by regex]' \
'(-g --groups)'{-g,--groups=}'[Filter by groups]:groups:' \
'(-a --all)'{-a,--all}'[Show all projects]' \
'(-n --name-only)'{-n,--name-only}'[Show only names]' \
'(-p --path-only)'{-p,--path-only}'[Show only paths]' \
'(-f --fullpath)'{-f,--fullpath}'[Show full paths]' \
'--relative-to=[Show paths relative to]:path:_files -/' \
'*: :->project'
;;
manifest)
_arguments \
'(-r --revision-as-HEAD)'{-r,--revision-as-HEAD}'[Save revisions as current HEAD]' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'--suppress-upstream-revision[Suppress upstream revision]' \
'--suppress-dest-branch[Suppress dest branch]' \
'--format=[Output format]:format:(xml json)' \
'--pretty[Pretty print]' \
'--no-local-manifests[Ignore local manifests]' \
'(-o --output-file)'{-o,--output-file=}'[Output file]:file:_files'
;;
overview)
_arguments \
${common_opts} \
'(-c --current-branch)'{-c,--current-branch}'[Show current branch]' \
'--no-current-branch[Do not show current branch]' \
'*: :->project'
;;
prune)
_arguments \
${common_opts} \
'*: :->project'
;;
rebase)
_arguments \
${common_opts} \
'--fail-fast[Fail fast]' \
'(-f --force-rebase)'{-f,--force-rebase}'[Force rebase]' \
'--no-ff[No fast forward]' \
'--autosquash[Autosquash]' \
'--whitespace=[Whitespace option]:option:' \
'--auto-stash[Auto stash]' \
'(-m --onto-manifest)'{-m,--onto-manifest}'[Rebase onto manifest]' \
'(-i --interactive)'{-i,--interactive}'[Interactive rebase]' \
'*: :->project'
;;
selfupdate)
_arguments \
'--no-repo-verify[Do not verify repo]'
;;
smartsync | sync)
_arguments \
${common_opts} \
'--jobs-network=[Number of network jobs]:jobs:' \
'--jobs-checkout=[Number of checkout jobs]:jobs:' \
'(-f --force-broken)'{-f,--force-broken}'[Continue sync even if a project fails]' \
'--fail-fast[Fail fast]' \
'--force-sync[Overwrite existing git directories]' \
'--force-checkout[Force checkout]' \
'--force-remove-dirty[Force remove dirty]' \
'--rebase[Rebase local branches]' \
'(-l --local-only)'{-l,--local-only}'[Only use local files]' \
'--no-manifest-update[Do not update manifest]' \
'--nmu[Do not update manifest]' \
'--interleaved[Interleave output]' \
'--no-interleaved[Do not interleave output]' \
'(-n --network-only)'{-n,--network-only}'[Only fetch]' \
'(-d --detach)'{-d,--detach}'[Detach HEAD]' \
'(-c --current-branch)'{-c,--current-branch}'[Fetch only current branch]' \
'--no-current-branch[Do not fetch only current branch]' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'--clone-bundle[Use clone bundle]' \
'--no-clone-bundle[Do not use clone bundle]' \
'(-u --manifest-server-username)'{-u,--manifest-server-username=}'[Username for manifest server]:username:' \
'(-p --manifest-server-password)'{-p,--manifest-server-password=}'[Password for manifest server]:password:' \
'--fetch-submodules[Fetch submodules]' \
'--use-superproject[Use superproject]' \
'--no-use-superproject[Do not use superproject]' \
'--tags[Sync tags]' \
'--no-tags[Do not sync tags]' \
'--optimized-fetch[Optimized fetch]' \
'--retry-fetches=[Retry fetches]:count:' \
'--prune[Prune]' \
'--no-prune[Do not prune]' \
'--auto-gc[Run auto gc]' \
'--no-auto-gc[Do not run auto gc]' \
'(-s --smart-sync)'{-s,--smart-sync}'[Smart sync]' \
'(-t --smart-tag)'{-t,--smart-tag=}'[Smart tag]:tag:' \
'--no-repo-verify[Do not verify repo]' \
'--no-verify[Do not verify]' \
'--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \
'*: :->project'
;;
stage)
_arguments \
${common_opts} \
'(-i --interactive)'{-i,--interactive}'[Interactive staging]' \
'*: :->project'
;;
start)
_arguments \
${common_opts} \
'--all[Start branch in all projects]' \
'(-r --rev --revision)'{-r,--rev=,--revision=}'[Revision]:revision:' \
'--head[Head]' \
'--HEAD[Head]' \
'1: :->newbranch' \
'*: :->project'
;;
status)
_arguments \
${common_opts} \
'(-o --orphans)'{-o,--orphans}'[Show orphans]' \
'*: :->project'
;;
upload)
_arguments \
${common_opts} \
'(-t --topic-branch)'{-t,--topic-branch}'[Topic branch]' \
'--topic=[Topic]:topic:' \
'--hashtag=[Hashtag]:hashtag:' \
'--ht=[Hashtag]:hashtag:' \
'--hashtag-branch[Hashtag branch]' \
'--htb[Hashtag branch]' \
'(-l --label)'{-l,--label=}'[Label]:label:' \
'--pd=[Patchset description]:description:' \
'--patchset-description=[Patchset description]:description:' \
'--re=[Reviewers]:reviewers:' \
'--reviewers=[Reviewers]:reviewers:' \
'--cc=[CC]:cc:' \
'--br=[Branch]:branch:' \
'--branch=[Branch]:branch:' \
'(-c --current-branch)'{-c,--current-branch}'[Upload current branch]' \
'--no-current-branch[Do not upload current branch]' \
'--ne[No emails]' \
'--no-emails[No emails]' \
'(-p --private)'{-p,--private}'[Private]' \
'(-w --wip)'{-w,--wip}'[Work in progress]' \
'(-r --ready)'{-r,--ready}'[Ready]' \
'(-o --push-option)'{-o,--push-option=}'[Push option]:option:' \
'(-D --destination --dest)'{-D,--destination=,--dest=}'[Destination]:destination:' \
'(-n --dry-run)'{-n,--dry-run}'[Dry run]' \
'(-y --yes)'{-y,--yes}'[Answer yes to all prompts]' \
'--ignore-untracked-files[Ignore untracked files]' \
'--no-ignore-untracked-files[Do not ignore untracked files]' \
'--no-cert-checks[Do not check certificates]' \
'--no-verify[Do not verify]' \
'--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \
'*: :->project'
;;
version)
_arguments ${common_opts}
;;
wipe)
_arguments \
${common_opts} \
'(-f --force)'{-f,--force}'[Force wipe]' \
'--force-uncommitted[Force uncommitted]' \
'--force-shared[Force shared]' \
'*: :->project'
;;
esac
# Handle states for positional arguments.
case ${state} in
branch)
[[ ${PREFIX} != -* ]] && _repo_branches
;;
project)
[[ ${PREFIX} != -* ]] && _repo_projects
;;
pattern)
_message 'pattern'
;;
newbranch)
_message 'new branch name'
;;
help_cmds)
_describe -t commands 'repo command' subcommands
;;
esac
;;
esac
}
_repo_branches() {
local -a branches
branches=(${(f)"$(_call_program branches repo branches 2>/dev/null | sed -E 's/^.*[[:space:]]([^[:space:]]+)[[:space:]]+\|.*$/\1/' | awk '{print $1}')"})
_describe -t branches 'branch' branches
}
_repo_projects() {
local -a projects
projects=(${(f)"$(_call_program projects repo list -n 2>/dev/null)"})
_describe -t projects 'project' projects
}
_repo "$@"
+29 -12
View File
@@ -73,18 +73,19 @@ following DTD:
project*, project*,
copyfile*, copyfile*,
linkfile*)> linkfile*)>
<!ATTLIST project name CDATA #REQUIRED> <!ATTLIST project name CDATA #REQUIRED>
<!ATTLIST project path CDATA #IMPLIED> <!ATTLIST project path CDATA #IMPLIED>
<!ATTLIST project remote IDREF #IMPLIED> <!ATTLIST project remote IDREF #IMPLIED>
<!ATTLIST project revision CDATA #IMPLIED> <!ATTLIST project revision CDATA #IMPLIED>
<!ATTLIST project dest-branch CDATA #IMPLIED> <!ATTLIST project dest-branch CDATA #IMPLIED>
<!ATTLIST project groups CDATA #IMPLIED> <!ATTLIST project groups CDATA #IMPLIED>
<!ATTLIST project sync-c CDATA #IMPLIED> <!ATTLIST project sync-c CDATA #IMPLIED>
<!ATTLIST project sync-s CDATA #IMPLIED> <!ATTLIST project sync-s CDATA #IMPLIED>
<!ATTLIST project sync-tags CDATA #IMPLIED> <!ATTLIST project sync-tags CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED> <!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project clone-depth CDATA #IMPLIED> <!ATTLIST project clone-depth CDATA #IMPLIED>
<!ATTLIST project force-path CDATA #IMPLIED> <!ATTLIST project force-path CDATA #IMPLIED>
<!ATTLIST project sync-strategy CDATA #IMPLIED>
<!ELEMENT annotation EMPTY> <!ELEMENT annotation EMPTY>
<!ATTLIST annotation name CDATA #REQUIRED> <!ATTLIST annotation name CDATA #REQUIRED>
@@ -389,6 +390,22 @@ rather than the `name` attribute. This attribute only applies to the
local mirrors syncing, it will be ignored when syncing the projects in a local mirrors syncing, it will be ignored when syncing the projects in a
client working directory. client working directory.
Attribute `sync-strategy`: Set the sync strategy used when fetching this
project. Currently the only supported value is `stateless`. When set to
`stateless`, repo will run a reflog expiration and aggressive garbage collection
at the end of the sync process. This is useful for projects that contain
large binary files and use `clone-depth="1"`, where garbage can accumulate
as binaries are added, deleted, or modified across successive syncs.
During a stateless sync, repo checks the following before cleaning up:
1. The project does not share an object directory with other projects.
2. The working tree is clean (no uncommitted changes, no untracked files).
3. There are no unpushed local commits.
4. There is no Git stash.
If any of these conditions are not met, repo falls back to a standard
sync without garbage collection.
### Element extend-project ### Element extend-project
Modify the attributes of the named project. Modify the attributes of the named project.
+2 -1
View File
@@ -163,13 +163,14 @@ Example:
The `post-sync.py` file should be defined like: The `post-sync.py` file should be defined like:
```py ```py
def main(repo_topdir=None, **kwargs): def main(repo_topdir=None, sync_duration_seconds=None, **kwargs):
"""Main function invoked directly by repo. """Main function invoked directly by repo.
We must use the name "main" as that is what repo requires. We must use the name "main" as that is what repo requires.
Args: Args:
repo_topdir: The absolute path to the top-level directory of the repo workspace. repo_topdir: The absolute path to the top-level directory of the repo workspace.
sync_duration_seconds: The duration of the sync operation in seconds.
kwargs: Leave this here for forward-compatibility. kwargs: Leave this here for forward-compatibility.
""" """
``` ```
+1 -1
View File
@@ -145,7 +145,7 @@ You will have to specify this flag every time you upload.
[mintty]: https://github.com/mintty/mintty/issues/56 [mintty]: https://github.com/mintty/mintty/issues/56
### repohooks always fail with an close_fds error. ### repohooks always fail with an close_fds error
When using the [reference repohooks project][repohooks] included in AOSP, When using the [reference repohooks project][repohooks] included in AOSP,
you might see errors like this when running `repo upload`: you might see errors like this when running `repo upload`:
+4 -3
View File
@@ -34,7 +34,7 @@ def fetch_file(url, verbose=False):
""" """
scheme = urlparse(url).scheme scheme = urlparse(url).scheme
if scheme == "gs": if scheme == "gs":
cmd = ["gsutil", "cat", url] cmd = ["gcloud", "storage", "cat", url]
errors = [] errors = []
try: try:
result = subprocess.run( result = subprocess.run(
@@ -42,7 +42,7 @@ def fetch_file(url, verbose=False):
) )
if result.stderr and verbose: if result.stderr and verbose:
print( print(
'warning: non-fatal error running "gsutil": %s' 'warning: non-fatal error running "gcloud storage": %s'
% result.stderr, % result.stderr,
file=sys.stderr, file=sys.stderr,
) )
@@ -50,7 +50,8 @@ def fetch_file(url, verbose=False):
except subprocess.CalledProcessError as e: except subprocess.CalledProcessError as e:
errors.append(e) errors.append(e)
print( print(
'fatal: error running "gsutil": %s' % e.stderr, file=sys.stderr 'fatal: error running "gcloud storage": %s' % e.stderr,
file=sys.stderr,
) )
raise FetchFileError(aggregate_errors=errors) raise FetchFileError(aggregate_errors=errors)
with urlopen(url) as f: with urlopen(url) as f:
+3 -3
View File
@@ -47,7 +47,7 @@ logger = RepoLogger(__file__)
class _GitCall: class _GitCall:
@functools.lru_cache(maxsize=None) @functools.lru_cache(maxsize=None) # noqa: B019
def version_tuple(self): def version_tuple(self):
ret = Wrapper().ParseGitVersion() ret = Wrapper().ParseGitVersion()
if ret is None: if ret is None:
@@ -95,7 +95,7 @@ def RepoSourceVersion():
ver = ver[1:] ver = ver[1:]
else: else:
ver = "unknown" ver = "unknown"
setattr(RepoSourceVersion, "version", ver) RepoSourceVersion.version = ver
return ver return ver
@@ -611,7 +611,7 @@ class GitCommandError(GitError):
self.git_stderr = git_stderr self.git_stderr = git_stderr
@property @property
@functools.lru_cache(maxsize=None) @functools.lru_cache(maxsize=None) # noqa: B019
def suggestion(self): def suggestion(self):
"""Returns helpful next steps for the given stderr.""" """Returns helpful next steps for the given stderr."""
if not self.git_stderr: if not self.git_stderr:
+7 -4
View File
@@ -42,7 +42,7 @@ SYNC_STATE_PREFIX = "repo.syncstate."
ID_RE = re.compile(r"^[0-9a-f]{40}$") ID_RE = re.compile(r"^[0-9a-f]{40}$")
REVIEW_CACHE = dict() REVIEW_CACHE = {}
def IsChange(rev): def IsChange(rev):
@@ -111,7 +111,7 @@ class GitConfig:
return cls(configfile=os.path.join(gitdir, "config"), defaults=defaults) return cls(configfile=os.path.join(gitdir, "config"), defaults=defaults)
def __init__(self, configfile, defaults=None, jsonFile=None): def __init__(self, configfile, defaults=None, jsonFile=None):
self.file = configfile self.file = str(configfile)
self.defaults = defaults self.defaults = defaults
self._cache_dict = None self._cache_dict = None
self._section_dict = None self._section_dict = None
@@ -438,7 +438,7 @@ class GitConfig:
if p.Wait() == 0: if p.Wait() == 0:
return p.stdout return p.stdout
else: else:
raise GitError(f"git config {str(args)}: {p.stderr}") raise GitError(f"git {' '.join(command)}: {p.stderr}")
class RepoConfig(GitConfig): class RepoConfig(GitConfig):
@@ -724,7 +724,10 @@ class Remote:
def Save(self): def Save(self):
"""Save this remote to the configuration.""" """Save this remote to the configuration."""
self._Set("url", self.url) self._Set("url", self.url)
if self.pushUrl is not None: # projectname is initialized for projects listed in the manifest, but
# not for others (e.g. the manifest project). This class is used for
# all of them.
if self.pushUrl is not None and self.projectname is not None:
self._Set("pushurl", self.pushUrl + "/" + self.projectname) self._Set("pushurl", self.pushUrl + "/" + self.projectname)
else: else:
self._Set("pushurl", self.pushUrl) self._Set("pushurl", self.pushUrl)
+20 -5
View File
@@ -34,6 +34,7 @@ import urllib.parse
from git_command import git_require from git_command import git_require
from git_command import GitCommand from git_command import GitCommand
from git_config import IsId
from git_config import RepoConfig from git_config import RepoConfig
from git_refs import GitRefs from git_refs import GitRefs
import platform_utils import platform_utils
@@ -100,7 +101,7 @@ class Superproject:
self._manifest = manifest self._manifest = manifest
self.name = name self.name = name
self.remote = remote self.remote = remote
self.revision = self._branch = revision self.revision = revision
self._repodir = manifest.repodir self._repodir = manifest.repodir
self._superproject_dir = superproject_dir self._superproject_dir = superproject_dir
self._superproject_path = manifest.SubmanifestInfoDir( self._superproject_path = manifest.SubmanifestInfoDir(
@@ -132,6 +133,10 @@ class Superproject:
"""Set the _print_messages attribute.""" """Set the _print_messages attribute."""
self._print_messages = value self._print_messages = value
def SetRevisionId(self, revision_id: str) -> None:
"""Set the revisionId of the superproject to sync to."""
self.revision = revision_id
@property @property
def commit_id(self): def commit_id(self):
"""Returns the commit ID of the superproject checkout.""" """Returns the commit ID of the superproject checkout."""
@@ -199,7 +204,8 @@ class Superproject:
def _LogMessagePrefix(self): def _LogMessagePrefix(self):
"""Returns the prefix string to be logged in each log message""" """Returns the prefix string to be logged in each log message"""
return ( return (
f"repo superproject branch: {self._branch} url: {self._remote_url}" f"repo superproject revision: {self.revision} "
f"url: {self._remote_url}"
) )
def _LogError(self, fmt, *inputs): def _LogError(self, fmt, *inputs):
@@ -312,8 +318,15 @@ class Superproject:
if rev_commit: if rev_commit:
cmd.extend(["--negotiation-tip", rev_commit]) cmd.extend(["--negotiation-tip", rev_commit])
if self._branch: if self.revision:
cmd += [self._branch + ":" + self._branch] # If revision is a commit hash, fetch it directly to avoid
# creating a local branch of the same name.
refspec = (
self.revision
if IsId(self.revision)
else f"{self.revision}:{self.revision}"
)
cmd.append(refspec)
p = GitCommand( p = GitCommand(
None, None,
cmd, cmd,
@@ -348,7 +361,7 @@ class Superproject:
) )
return None return None
data = None data = None
branch = "HEAD" if not self._branch else self._branch branch = "HEAD" if not self.revision else self.revision
cmd = ["ls-tree", "-z", "-r", branch] cmd = ["ls-tree", "-z", "-r", branch]
p = GitCommand( p = GitCommand(
@@ -400,6 +413,8 @@ class Superproject:
if not self._Init(): if not self._Init():
return SyncResult(False, should_exit) return SyncResult(False, should_exit)
if IsId(self.revision) and self.commit_id:
return SyncResult(True, False)
if not self._Fetch(): if not self._Fetch():
return SyncResult(False, should_exit) return SyncResult(False, should_exit)
if not self._quiet: if not self._quiet:
+1 -1
View File
@@ -25,7 +25,7 @@ from git_refs import HEAD
# The API we've documented to hook authors. Keep in sync with repo-hooks.md. # The API we've documented to hook authors. Keep in sync with repo-hooks.md.
_API_ARGS = { _API_ARGS = {
"pre-upload": {"project_list", "worktree_list"}, "pre-upload": {"project_list", "worktree_list"},
"post-sync": {"repo_topdir"}, "post-sync": {"repo_topdir", "sync_duration_seconds"},
} }
+113 -13
View File
@@ -19,6 +19,7 @@ People shouldn't run this directly; instead, they should use the `repo` wrapper
which takes care of execing this entry point. which takes care of execing this entry point.
""" """
import difflib
import getpass import getpass
import json import json
import netrc import netrc
@@ -29,6 +30,7 @@ import signal
import sys import sys
import textwrap import textwrap
import time import time
from typing import Optional
import urllib.request import urllib.request
from repo_logging import RepoLogger from repo_logging import RepoLogger
@@ -292,6 +294,102 @@ class _Repo:
result = run() result = run()
return result return result
def _autocorrect_command_name(
self, name: str, config: RepoConfig
) -> Optional[str]:
"""Autocorrect command name based on user's git config."""
close_commands = difflib.get_close_matches(
name, self.commands.keys(), n=5, cutoff=0.7
)
if not close_commands:
logger.error(
"repo: '%s' is not a repo command. See 'repo help'.", name
)
return None
assumed = close_commands[0]
autocorrect = config.GetString("help.autocorrect")
# If there are multiple close matches, git won't automatically run one.
# We'll always prompt instead of guessing.
if len(close_commands) > 1:
autocorrect = "prompt"
# Handle git configuration boolean values:
# 0, "false", "off", "no", "show": show suggestion (default)
# 1, "true", "on", "yes", "immediate": run suggestion immediately
# "never": don't run or show any suggested command
# "prompt": show the suggestion and prompt for confirmation
# positive number > 1: run suggestion after specified deciseconds
if autocorrect is None:
autocorrect = "0"
autocorrect = autocorrect.lower()
if autocorrect in ("0", "false", "off", "no", "show"):
autocorrect = 0
elif autocorrect in ("true", "on", "yes", "immediate"):
autocorrect = -1 # immediate
elif autocorrect == "never":
return None
elif autocorrect == "prompt":
logger.warning(
"You called a repo command named "
"'%s', which does not exist.",
name,
)
try:
resp = input(f"Run '{assumed}' instead [y/N]? ")
if resp.lower().startswith("y"):
return assumed
except (KeyboardInterrupt, EOFError):
pass
return None
else:
try:
autocorrect = int(autocorrect)
except ValueError:
autocorrect = 0
if autocorrect != 0:
if autocorrect < 0:
logger.warning(
"You called a repo command named "
"'%s', which does not exist.\n"
"Continuing assuming that "
"you meant '%s'.",
name,
assumed,
)
else:
delay = autocorrect * 0.1
logger.warning(
"You called a repo command named "
"'%s', which does not exist.\n"
"Continuing in %.1f seconds, assuming "
"that you meant '%s'.",
name,
delay,
assumed,
)
try:
time.sleep(delay)
except KeyboardInterrupt:
return None
return assumed
logger.error(
"repo: '%s' is not a repo command. See 'repo help'.", name
)
logger.warning(
"The most similar command%s\n\t%s",
"s are" if len(close_commands) > 1 else " is",
"\n\t".join(close_commands),
)
return None
def _RunLong(self, name, gopts, argv, git_trace2_event_log): def _RunLong(self, name, gopts, argv, git_trace2_event_log):
"""Execute the (longer running) requested subcommand.""" """Execute the (longer running) requested subcommand."""
result = 0 result = 0
@@ -306,20 +404,22 @@ class _Repo:
outer_client=outer_client, outer_client=outer_client,
) )
try: if name not in self.commands:
cmd = self.commands[name]( corrected_name = self._autocorrect_command_name(
repodir=self.repodir, name, outer_client.globalConfig
client=repo_client,
manifest=repo_client.manifest,
outer_client=outer_client,
outer_manifest=outer_client.manifest,
git_event_log=git_trace2_event_log,
) )
except KeyError: if not corrected_name:
logger.error( return 1
"repo: '%s' is not a repo command. See 'repo help'.", name name = corrected_name
)
return 1 cmd = self.commands[name](
repodir=self.repodir,
client=repo_client,
manifest=repo_client.manifest,
outer_client=outer_client,
outer_manifest=outer_client.manifest,
git_event_log=git_trace2_event_log,
)
Editor.globalConfig = cmd.client.globalConfig Editor.globalConfig = cmd.client.globalConfig
+6 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man. .\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo forall" "Repo Manual" .TH REPO "1" "May 2026" "repo forall" "Repo Manual"
.SH NAME .SH NAME
repo \- repo forall - manual page for repo forall repo \- repo forall - manual page for repo forall
.SH SYNOPSIS .SH SYNOPSIS
@@ -120,6 +120,11 @@ git command, use REPO_LREV.
REPO_RREV is the name of the revision from the manifest, exactly as written in REPO_RREV is the name of the revision from the manifest, exactly as written in
the manifest. the manifest.
.PP .PP
REPO_UPSTREAM is the name of the upstream branch as specified in the manifest.
.PP
REPO_DEST_BRANCH is the name of the destination branch for code review, as
specified in the manifest.
.PP
REPO_COUNT is the total number of projects being iterated. REPO_COUNT is the total number of projects being iterated.
.PP .PP
REPO_I is the current (1\-based) iteration count. Can be used in conjunction with REPO_I is the current (1\-based) iteration count. Can be used in conjunction with
+21 -2
View File
@@ -1,10 +1,10 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man. .\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo info" "Repo Manual" .TH REPO "1" "May 2026" "repo info" "Repo Manual"
.SH NAME .SH NAME
repo \- repo info - manual page for repo info repo \- repo info - manual page for repo info
.SH SYNOPSIS .SH SYNOPSIS
.B repo .B repo
\fI\,info \/\fR[\fI\,-dl\/\fR] [\fI\,-o \/\fR[\fI\,-c\/\fR]] [\fI\,<project>\/\fR...] \fI\,info \/\fR[\fI\,-dl\/\fR] [\fI\,-o \/\fR[\fI\,-c\/\fR]] [\fI\,--format=<format>\/\fR] [\fI\,<project>\/\fR...]
.SH DESCRIPTION .SH DESCRIPTION
Summary Summary
.PP .PP
@@ -14,6 +14,10 @@ Get info on the manifest branch, current branch or unmerged branches
\fB\-h\fR, \fB\-\-help\fR \fB\-h\fR, \fB\-\-help\fR
show this help message and exit show this help message and exit
.TP .TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
.TP
\fB\-d\fR, \fB\-\-diff\fR \fB\-d\fR, \fB\-\-diff\fR
show full info and commit diff including remote show full info and commit diff including remote
branches branches
@@ -21,6 +25,18 @@ branches
\fB\-o\fR, \fB\-\-overview\fR \fB\-o\fR, \fB\-\-overview\fR
show overview of all local commits show overview of all local commits
.TP .TP
\fB\-\-include\-summary\fR
include manifest summary (default: true)
.TP
\fB\-\-no\-include\-summary\fR
exclude manifest summary
.TP
\fB\-\-include\-projects\fR
include project details (default: true)
.TP
\fB\-\-no\-include\-projects\fR
exclude project details
.TP
\fB\-c\fR, \fB\-\-current\-branch\fR \fB\-c\fR, \fB\-\-current\-branch\fR
consider only checked out branches consider only checked out branches
.TP .TP
@@ -29,6 +45,9 @@ consider all local branches
.TP .TP
\fB\-l\fR, \fB\-\-local\-only\fR \fB\-l\fR, \fB\-\-local\-only\fR
disable all remote operations disable all remote operations
.TP
\fB\-\-format\fR=\fI\,FORMAT\/\fR
output format: text, json (default: text)
.SS Logging options: .SS Logging options:
.TP .TP
\fB\-v\fR, \fB\-\-verbose\fR \fB\-v\fR, \fB\-\-verbose\fR
+41 -9
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man. .\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "March 2026" "repo manifest" "Repo Manual" .TH REPO "1" "April 2026" "repo manifest" "Repo Manual"
.SH NAME .SH NAME
repo \- repo manifest - manual page for repo manifest repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS .SH SYNOPSIS
@@ -165,15 +165,32 @@ IDREF #IMPLIED>
.TP .TP
<!ATTLIST project revision <!ATTLIST project revision
CDATA #IMPLIED> CDATA #IMPLIED>
.TP
<!ATTLIST project dest\-branch
CDATA #IMPLIED>
.TP
<!ATTLIST project groups
CDATA #IMPLIED>
.TP
<!ATTLIST project sync\-c
CDATA #IMPLIED>
.TP
<!ATTLIST project sync\-s
CDATA #IMPLIED>
.TP
<!ATTLIST project sync\-tags
CDATA #IMPLIED>
.TP
<!ATTLIST project upstream
CDATA #IMPLIED>
.TP
<!ATTLIST project clone\-depth
CDATA #IMPLIED>
.TP
<!ATTLIST project force\-path
CDATA #IMPLIED>
.IP .IP
<!ATTLIST project dest\-branch CDATA #IMPLIED> <!ATTLIST project sync\-strategy CDATA #IMPLIED>
<!ATTLIST project groups CDATA #IMPLIED>
<!ATTLIST project sync\-c CDATA #IMPLIED>
<!ATTLIST project sync\-s CDATA #IMPLIED>
<!ATTLIST project sync\-tags CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project clone\-depth CDATA #IMPLIED>
<!ATTLIST project force\-path CDATA #IMPLIED>
.IP .IP
<!ELEMENT annotation EMPTY> <!ELEMENT annotation EMPTY>
<!ATTLIST annotation name CDATA #REQUIRED> <!ATTLIST annotation name CDATA #REQUIRED>
@@ -469,6 +486,21 @@ mirror repository according to its `path` attribute (if supplied) rather than
the `name` attribute. This attribute only applies to the local mirrors syncing, the `name` attribute. This attribute only applies to the local mirrors syncing,
it will be ignored when syncing the projects in a client working directory. it will be ignored when syncing the projects in a client working directory.
.PP .PP
Attribute `sync\-strategy`: Set the sync strategy used when fetching this
project. Currently the only supported value is `stateless`. When set to
`stateless`, repo will run a reflog expiration and aggressive garbage collection
at the end of the sync process. This is useful for projects that contain large
binary files and use `clone\-depth="1"`, where garbage can accumulate as binaries
are added, deleted, or modified across successive syncs.
.PP
During a stateless sync, repo checks the following before cleaning up: 1. The
project does not share an object directory with other projects. 2. The working
tree is clean (no uncommitted changes, no untracked files). 3. There are no
unpushed local commits. 4. There is no Git stash.
.PP
If any of these conditions are not met, repo falls back to a standard sync
without garbage collection.
.PP
Element extend\-project Element extend\-project
.PP .PP
Modify the attributes of the named project. Modify the attributes of the named project.
+5 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man. .\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "August 2025" "repo smartsync" "Repo Manual" .TH REPO "1" "May 2026" "repo smartsync" "Repo Manual"
.SH NAME .SH NAME
repo \- repo smartsync - manual page for repo smartsync repo \- repo smartsync - manual page for repo smartsync
.SH SYNOPSIS .SH SYNOPSIS
@@ -101,6 +101,10 @@ implies \fB\-c\fR
\fB\-\-no\-use\-superproject\fR \fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects disable use of manifest superprojects
.TP .TP
\fB\-\-superproject\-revision\fR=\fI\,SUPERPROJECT_REVISION\/\fR
sync to superproject revision (applies to outer
manifest)
.TP
\fB\-\-tags\fR \fB\-\-tags\fR
fetch tags fetch tags
.TP .TP
+5 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man. .\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "August 2025" "repo sync" "Repo Manual" .TH REPO "1" "May 2026" "repo sync" "Repo Manual"
.SH NAME .SH NAME
repo \- repo sync - manual page for repo sync repo \- repo sync - manual page for repo sync
.SH SYNOPSIS .SH SYNOPSIS
@@ -101,6 +101,10 @@ implies \fB\-c\fR
\fB\-\-no\-use\-superproject\fR \fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects disable use of manifest superprojects
.TP .TP
\fB\-\-superproject\-revision\fR=\fI\,SUPERPROJECT_REVISION\/\fR
sync to superproject revision (applies to outer
manifest)
.TP
\fB\-\-tags\fR \fB\-\-tags\fR
fetch tags fetch tags
.TP .TP
+8 -3
View File
@@ -759,14 +759,17 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if p.clone_depth: if p.clone_depth:
e.setAttribute("clone-depth", str(p.clone_depth)) e.setAttribute("clone-depth", str(p.clone_depth))
if p.sync_strategy:
e.setAttribute("sync-strategy", str(p.sync_strategy))
self._output_manifest_project_extras(p, e) self._output_manifest_project_extras(p, e)
if p.subprojects: if p.subprojects:
subprojects = {subp.name for subp in p.subprojects} subprojects = {subp.name for subp in p.subprojects}
output_projects(p, e, list(sorted(subprojects))) output_projects(p, e, sorted(subprojects))
projects = {p.name for p in self._paths.values() if not p.parent} projects = {p.name for p in self._paths.values() if not p.parent}
output_projects(None, root, list(sorted(projects))) output_projects(None, root, sorted(projects))
if self._repo_hooks_project: if self._repo_hooks_project:
root.appendChild(doc.createTextNode("")) root.appendChild(doc.createTextNode(""))
@@ -823,7 +826,6 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
"submanifest", "submanifest",
# These are children of 'project' nodes. # These are children of 'project' nodes.
"annotation", "annotation",
"project",
"copyfile", "copyfile",
"linkfile", "linkfile",
} }
@@ -1939,6 +1941,8 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
% (self.manifestFile, clone_depth) % (self.manifestFile, clone_depth)
) )
sync_strategy = node.getAttribute("sync-strategy") or None
dest_branch = ( dest_branch = (
node.getAttribute("dest-branch") or self._default.destBranchExpr node.getAttribute("dest-branch") or self._default.destBranchExpr
) )
@@ -1985,6 +1989,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
sync_s=sync_s, sync_s=sync_s,
sync_tags=sync_tags, sync_tags=sync_tags,
clone_depth=clone_depth, clone_depth=clone_depth,
sync_strategy=sync_strategy,
upstream=upstream, upstream=upstream,
parent=parent, parent=parent,
dest_branch=dest_branch, dest_branch=dest_branch,
+37
View File
@@ -222,6 +222,43 @@ def rmdir(path):
os.rmdir(_makelongpath(path)) os.rmdir(_makelongpath(path))
def removedirs(path: str) -> None:
"""Remove a directory tree of symlinks and empty directories.
Walks |path| bottom-up without following symlinks. Removes symlinks
and empty directories. Stops at (and preserves) regular files and
non-empty directories. Silently succeeds if |path| does not exist.
Availability: Unix, Windows.
"""
if islink(path):
remove(path)
return
if not isdir(path):
return
for dirpath, dirnames, filenames in walk(path, topdown=False):
for name in dirnames:
entry = os.path.join(dirpath, name)
if islink(entry):
remove(entry)
else:
try:
rmdir(entry)
except OSError:
pass
for name in filenames:
entry = os.path.join(dirpath, name)
if islink(entry):
remove(entry)
try:
rmdir(path)
except OSError:
pass
def isdir(path): def isdir(path):
"""os.path.isdir(path) wrapper with support for long paths on Windows. """os.path.isdir(path) wrapper with support for long paths on Windows.
+2
View File
@@ -159,6 +159,8 @@ class Progress:
inc: The number of items completed. inc: The number of items completed.
msg: The message to display. If None, use the last message. msg: The message to display. If None, use the last message.
""" """
if self._ended:
return
self._done += inc self._done += inc
if msg is None: if msg is None:
msg = self._last_msg msg = self._last_msg
+134 -34
View File
@@ -225,7 +225,7 @@ class ReviewableBranch:
@property @property
def unabbrev_commits(self): def unabbrev_commits(self):
r = dict() r = {}
for commit in self.project.bare_git.rev_list( for commit in self.project.bare_git.rev_list(
not_rev(self.base), R_HEADS + self.name, "--" not_rev(self.base), R_HEADS + self.name, "--"
): ):
@@ -453,9 +453,13 @@ class _LinkFile(NamedTuple):
platform_utils.readlink(absDest) != relSrc platform_utils.readlink(absDest) != relSrc
): ):
try: try:
# Remove existing file first, since it might be read-only. # Remove existing path first, since it might be read-only.
if os.path.lexists(absDest): if os.path.lexists(absDest):
platform_utils.remove(absDest) # removedirs handles symlinks, empty dirs, and nested
# trees of stale linkfile dests without deleting user
# content. Falls through to symlink() if the path was
# fully removed, or raises OSError if not.
platform_utils.removedirs(absDest)
else: else:
dest_dir = os.path.dirname(absDest) dest_dir = os.path.dirname(absDest)
if not platform_utils.isdir(dest_dir): if not platform_utils.isdir(dest_dir):
@@ -553,11 +557,12 @@ class Project:
revisionExpr, revisionExpr,
revisionId, revisionId,
rebase=True, rebase=True,
groups=set(), groups=None,
sync_c=False, sync_c=False,
sync_s=False, sync_s=False,
sync_tags=True, sync_tags=True,
clone_depth=None, clone_depth=None,
sync_strategy=None,
upstream=None, upstream=None,
parent=None, parent=None,
use_git_worktrees=False, use_git_worktrees=False,
@@ -605,11 +610,12 @@ class Project:
self.SetRevision(revisionExpr, revisionId=revisionId) self.SetRevision(revisionExpr, revisionId=revisionId)
self.rebase = rebase self.rebase = rebase
self.groups = groups self.groups = groups if groups is not None else set()
self.sync_c = sync_c self.sync_c = sync_c
self.sync_s = sync_s self.sync_s = sync_s
self.sync_tags = sync_tags self.sync_tags = sync_tags
self.clone_depth = clone_depth self.clone_depth = clone_depth
self.sync_strategy = sync_strategy
self.upstream = upstream self.upstream = upstream
self.parent = parent self.parent = parent
# NB: Do not use this setting in __init__ to change behavior so that the # NB: Do not use this setting in __init__ to change behavior so that the
@@ -627,6 +633,7 @@ class Project:
self.linkfiles = {} self.linkfiles = {}
self.annotations = [] self.annotations = []
self.dest_branch = dest_branch self.dest_branch = dest_branch
self.stateless_prune_needed = False
# This will be filled in if a project is later identified to be the # This will be filled in if a project is later identified to be the
# project containing repo hooks. # project containing repo hooks.
@@ -756,6 +763,18 @@ class Project:
return True return True
return False return False
def HasStash(self) -> bool:
"""Returns True if there is a stash in the repository."""
p = GitCommand(
self,
["rev-parse", "--verify", "refs/stash"],
bare=True,
capture_stdout=True,
capture_stderr=True,
log_as_error=False,
)
return p.Wait() == 0
_userident_name = None _userident_name = None
_userident_email = None _userident_email = None
@@ -943,7 +962,7 @@ class Project:
out.important("prior sync failed; rebase still in progress") out.important("prior sync failed; rebase still in progress")
out.nl() out.nl()
paths = list() paths = []
paths.extend(di.keys()) paths.extend(di.keys())
paths.extend(df.keys()) paths.extend(df.keys())
paths.extend(do) paths.extend(do)
@@ -1239,6 +1258,67 @@ class Project:
logger.error("error: Cannot extract archive %s: %s", tarpath, e) logger.error("error: Cannot extract archive %s: %s", tarpath, e)
return False return False
def _ShouldStatelessPrune(
self, use_superproject: Optional[bool] = None
) -> bool:
"""Determines if a stateless prune should be performed.
Stateless pruning reclaims space by running a reflog expiration and
garbage collection instead of an incremental fetch. It is only performed
if the repository is clean and has no local-only state.
"""
if not self.Exists:
return False
if self._CheckForImmutableRevision(use_superproject=use_superproject):
return False
# Query the target hash from remote to see if we are up-to-date.
target_hash = None
if IsId(self.revisionExpr):
target_hash = self.revisionExpr
else:
output = self._LsRemote(self.upstream or self.revisionExpr)
if output:
target_hash = output.splitlines()[0].split()[0]
if not target_hash:
return False
try:
local_head = self.bare_git.rev_parse("HEAD")
except GitError:
local_head = None
if target_hash == local_head:
return False
# Skip if sharing objects with other projects.
shares_objdir = self.UseAlternates or self.use_git_worktrees
if not shares_objdir:
for p in self.manifest.GetProjectsWithName(self.name):
if p != self and p.objdir == self.objdir:
shares_objdir = True
break
if shares_objdir:
return False
# Skip if HEAD contains any unpushed local commits.
try:
local_commits = self.bare_git.rev_list(
"--count", "HEAD", "--not", "--remotes", "--tags"
)
if int(local_commits[0]) > 0:
return False
except (GitError, IndexError, ValueError):
return False
if self.IsDirty(consider_untracked=True) or self.HasStash():
return False
return True
def Sync_NetworkHalf( def Sync_NetworkHalf(
self, self,
quiet=False, quiet=False,
@@ -1257,7 +1337,7 @@ class Project:
submodules=False, submodules=False,
ssh_proxy=None, ssh_proxy=None,
clone_filter=None, clone_filter=None,
partial_clone_exclude=set(), partial_clone_exclude=None,
clone_filter_for_depth=None, clone_filter_for_depth=None,
): ):
"""Perform only the network IO portion of the sync process. """Perform only the network IO portion of the sync process.
@@ -1310,10 +1390,17 @@ class Project:
if clone_bundle and os.path.exists(self.objdir): if clone_bundle and os.path.exists(self.objdir):
clone_bundle = False clone_bundle = False
if partial_clone_exclude is None:
partial_clone_exclude = set()
if self.name in partial_clone_exclude: if self.name in partial_clone_exclude:
clone_bundle = True clone_bundle = True
clone_filter = None clone_filter = None
if self.sync_strategy == "stateless" and self._ShouldStatelessPrune(
use_superproject
):
self.stateless_prune_needed = True
if is_new is None: if is_new is None:
is_new = not self.Exists is_new = not self.Exists
if is_new: if is_new:
@@ -1412,6 +1499,10 @@ class Project:
and self._CheckForImmutableRevision( and self._CheckForImmutableRevision(
use_superproject=use_superproject use_superproject=use_superproject
) )
and (
not depth
or os.path.exists(os.path.join(self.gitdir, "shallow"))
)
): ):
remote_fetched = True remote_fetched = True
try: try:
@@ -1598,6 +1689,23 @@ class Project:
def _dosubmodules(): def _dosubmodules():
self._SyncSubmodules(quiet=True) self._SyncSubmodules(quiet=True)
def _doprune() -> None:
"""Expire reflogs and run prune-now GC for stateless sync."""
GitCommand(
self,
["reflog", "expire", "--expire=all", "--all"],
bare=True,
).Wait()
p = GitCommand(
self,
["gc", "--prune=now"],
bare=True,
capture_stdout=True,
capture_stderr=True,
)
if p.Wait() != 0:
logger.warning("warn: %s: stateless gc failed", self.name)
head = self.work_git.GetHead() head = self.work_git.GetHead()
if head.startswith(R_HEADS): if head.startswith(R_HEADS):
branch = head[len(R_HEADS) :] branch = head[len(R_HEADS) :]
@@ -1643,6 +1751,8 @@ class Project:
fail(e) fail(e)
return return
self._CopyAndLinkFiles() self._CopyAndLinkFiles()
if self.stateless_prune_needed:
syncbuf.later2(self, _doprune, not verbose)
return return
if head == revid: if head == revid:
@@ -1661,7 +1771,7 @@ class Project:
self, "leaving %s; does not track upstream", branch.name self, "leaving %s; does not track upstream", branch.name
) )
try: try:
self._Checkout(revid, quiet=True) self._Checkout(revid, force_checkout=force_checkout, quiet=True)
if submodules: if submodules:
self._SyncSubmodules(quiet=True) self._SyncSubmodules(quiet=True)
except GitError as e: except GitError as e:
@@ -1789,6 +1899,9 @@ class Project:
if submodules: if submodules:
syncbuf.later1(self, _dosubmodules, not verbose) syncbuf.later1(self, _dosubmodules, not verbose)
if self.stateless_prune_needed:
syncbuf.later2(self, _doprune, not verbose)
def AddCopyFile(self, src, dest, topdir): def AddCopyFile(self, src, dest, topdir):
"""Mark |src| for copying to |dest| (relative to |topdir|). """Mark |src| for copying to |dest| (relative to |topdir|).
@@ -2335,7 +2448,7 @@ class Project:
result.extend(project.GetDerivedSubprojects()) result.extend(project.GetDerivedSubprojects())
continue continue
if url.startswith(".."): if url.startswith(("./", "../")):
url = urllib.parse.urljoin("%s/" % self.remote.url, url) url = urllib.parse.urljoin("%s/" % self.remote.url, url)
remote = RemoteSpec( remote = RemoteSpec(
self.remote.name, self.remote.name,
@@ -2511,6 +2624,9 @@ class Project:
if is_sha1 or tag_name is not None: if is_sha1 or tag_name is not None:
if self._CheckForImmutableRevision( if self._CheckForImmutableRevision(
use_superproject=use_superproject use_superproject=use_superproject
) and (
not depth
or os.path.exists(os.path.join(self.gitdir, "shallow"))
): ):
if verbose: if verbose:
print( print(
@@ -2568,7 +2684,7 @@ class Project:
if update_ref_cmds: if update_ref_cmds:
GitCommand( GitCommand(
self, self,
["update-ref", "--no-deref", "--stdin"], ["update-ref", "--stdin"],
bare=True, bare=True,
input="".join(update_ref_cmds), input="".join(update_ref_cmds),
).Wait() ).Wait()
@@ -2816,7 +2932,7 @@ class Project:
) )
GitCommand( GitCommand(
self, self,
["update-ref", "--no-deref", "--stdin"], ["update-ref", "--stdin"],
bare=True, bare=True,
input=delete_cmds, input=delete_cmds,
log_as_error=False, log_as_error=False,
@@ -3964,30 +4080,14 @@ class Project:
def GetHead(self): def GetHead(self):
"""Return the ref that HEAD points to.""" """Return the ref that HEAD points to."""
try: try:
symbolic_head = self.rev_parse("--symbolic-full-name", HEAD) return self.symbolic_ref("-q", HEAD, log_as_error=False)
if symbolic_head == HEAD: except GitError:
# Detached HEAD. Return the commit SHA instead. pass
return self.rev_parse(HEAD)
return symbolic_head
except GitError as e:
# `git rev-parse --symbolic-full-name HEAD` will fail for unborn
# branches, so try symbolic-ref before falling back to raw file
# parsing.
try:
p = GitCommand(
self._project,
["symbolic-ref", "-q", HEAD],
bare=True,
gitdir=self._gitdir,
capture_stdout=True,
capture_stderr=True,
log_as_error=False,
)
if p.Wait() == 0:
return p.stdout.rstrip("\n")
except GitError:
pass
try:
# If symbolic-ref fails, try to treat as detached HEAD.
return self.rev_parse(HEAD)
except GitError as e:
logger.warning( logger.warning(
"project %s: unparseable HEAD; trying to recover.\n" "project %s: unparseable HEAD; trying to recover.\n"
"Check that HEAD ref in .git/HEAD is valid. The error " "Check that HEAD ref in .git/HEAD is valid. The error "
+1 -1
View File
@@ -106,7 +106,7 @@ def check_path(opts: argparse.Namespace, path: Path) -> bool:
def check_paths(opts: argparse.Namespace, paths: list[Path]) -> bool: def check_paths(opts: argparse.Namespace, paths: list[Path]) -> bool:
"""Check all the paths.""" """Check all the paths."""
# NB: Use list comprehension and not a generator so we check all paths. # NB: Use list comprehension and not a generator so we check all paths.
return all([check_path(opts, x) for x in paths]) return all([check_path(opts, x) for x in paths]) # noqa: C419
def find_files(opts: argparse.Namespace) -> list[Path]: def find_files(opts: argparse.Namespace) -> list[Path]:
+7 -7
View File
@@ -85,18 +85,18 @@ Repo launcher bucket:
gs://git-repo-downloads/ gs://git-repo-downloads/
You should first upload it with a specific version: You should first upload it with a specific version:
gsutil cp -a public-read {opts.launcher} gs://git-repo-downloads/repo-{version} gcloud storage cp --predefined-acl=publicRead {opts.launcher} gs://git-repo-downloads/repo-{version}
gsutil cp -a public-read {opts.launcher}.asc gs://git-repo-downloads/repo-{version}.asc gcloud storage cp --predefined-acl=publicRead {opts.launcher}.asc gs://git-repo-downloads/repo-{version}.asc
Then to make it the public default: Then to make it the public default:
gsutil cp -a public-read gs://git-repo-downloads/repo-{version} gs://git-repo-downloads/repo gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo-{version} gs://git-repo-downloads/repo
gsutil cp -a public-read gs://git-repo-downloads/repo-{version}.asc gs://git-repo-downloads/repo.asc gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo-{version}.asc gs://git-repo-downloads/repo.asc
NB: If a rollback is necessary, the GS bucket archives old versions, and may be NB: If a rollback is necessary, the GS bucket archives old versions, and may be
accessed by specifying their unique id number. accessed by specifying their unique id number.
gsutil ls -la gs://git-repo-downloads/repo gs://git-repo-downloads/repo.asc gcloud storage ls --long --all-versions gs://git-repo-downloads/repo gs://git-repo-downloads/repo.asc
gsutil cp -a public-read gs://git-repo-downloads/repo#<unique id> gs://git-repo-downloads/repo gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo#<unique id> gs://git-repo-downloads/repo
gsutil cp -a public-read gs://git-repo-downloads/repo.asc#<unique id> gs://git-repo-downloads/repo.asc gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo.asc#<unique id> gs://git-repo-downloads/repo.asc
""" # noqa: E501 """ # noqa: E501
) )
+8 -6
View File
@@ -30,8 +30,7 @@ import tempfile
from typing import List from typing import List
# NB: This script is currently imported by tests/ to unittest some logic. assert sys.version_info >= (3, 9), "Release framework requires Python 3.9+"
assert sys.version_info >= (3, 6), "Python 3.6+ required"
THIS_FILE = Path(__file__).resolve() THIS_FILE = Path(__file__).resolve()
@@ -66,7 +65,11 @@ def main(argv: List[str]) -> int:
parser = get_parser() parser = get_parser()
opts = parser.parse_args(argv) opts = parser.parse_args(argv)
if not shutil.which("help2man"): help2man = ["help2man"]
cipd_help2man = TOPDIR / ".cipd_bin/bin/help2man"
if cipd_help2man.exists():
help2man = [cipd_help2man]
elif not shutil.which("help2man"):
sys.exit("Please install help2man to continue.") sys.exit("Please install help2man to continue.")
# Let repo know we're generating man pages so it can avoid some dynamic # Let repo know we're generating man pages so it can avoid some dynamic
@@ -81,7 +84,7 @@ def main(argv: List[str]) -> int:
version = RepoSourceVersion() version = RepoSourceVersion()
cmdlist = [ cmdlist = [
[ [
"help2man", *help2man,
"-N", "-N",
"-n", "-n",
f"repo {cmd} - manual page for repo {cmd}", f"repo {cmd} - manual page for repo {cmd}",
@@ -100,7 +103,7 @@ def main(argv: List[str]) -> int:
] ]
cmdlist.append( cmdlist.append(
[ [
"help2man", *help2man,
"-N", "-N",
"-n", "-n",
"repository management tool built on top of git", "repository management tool built on top of git",
@@ -178,7 +181,6 @@ def replace_regex(data):
""" """
regex = ( regex = (
(r"(It was generated by help2man) [0-9.]+", r"\g<1>."), (r"(It was generated by help2man) [0-9.]+", r"\g<1>."),
(r"^\033\[[0-9;]*m([^\033]*)\033\[m", r"\g<1>"),
(r"^\.IP\n(.*:)\n", r".SS \g<1>\n"), (r"^\.IP\n(.*:)\n", r".SS \g<1>\n"),
(r"^\.PP\nDescription", r".SH DETAILS"), (r"^\.PP\nDescription", r".SH DETAILS"),
) )
+16 -5
View File
@@ -157,11 +157,6 @@ def run_check_metadata():
def run_update_manpages() -> int: def run_update_manpages() -> int:
"""Returns the exit code from release/update-manpages.""" """Returns the exit code from release/update-manpages."""
# Allow this to fail on CI, but not local devs.
if is_ci() and not shutil.which("help2man"):
print("update-manpages: help2man not found; skipping test")
return 0
argv = ["--check"] argv = ["--check"]
log_cmd("release/update-manpages", argv) log_cmd("release/update-manpages", argv)
return subprocess.run( return subprocess.run(
@@ -171,8 +166,24 @@ def run_update_manpages() -> int:
).returncode ).returncode
def cipd_bootstrap() -> None:
"""Install packages via cipd if available."""
argv = ["ensure", "-root", ".cipd_bin", "-ensure-file", "cipd_manifest.txt"]
log_cmd("cipd", argv)
try:
subprocess.run(
["cipd"] + argv,
check=True,
cwd=ROOT_DIR,
)
except FileNotFoundError:
# Skip if the user doesn't have cipd from depot_tools.
return
def main(argv): def main(argv):
"""The main entry.""" """The main entry."""
cipd_bootstrap()
checks = ( checks = (
functools.partial(run_pytest, argv), functools.partial(run_pytest, argv),
functools.partial(run_pytest_py38, argv), functools.partial(run_pytest_py38, argv),
+13 -3
View File
@@ -48,10 +48,10 @@ wheel: <
version: "version:3.0.7" version: "version:3.0.7"
> >
# Required by pytest==8.3.4 # Required by pytest==8.3.4 and flake8-bugbear==24.12.12
wheel: < wheel: <
name: "infra/python/wheels/attrs-py2_py3" name: "infra/python/wheels/attrs-py3"
version: "version:21.4.0" version: "version:24.2.0"
> >
# NB: Keep in sync with constraints.txt. # NB: Keep in sync with constraints.txt.
@@ -119,6 +119,16 @@ wheel: <
version: "version:2.10.0" version: "version:2.10.0"
> >
wheel: <
name: "infra/python/wheels/flake8-bugbear-py3"
version: "version:24.12.12"
>
wheel: <
name: "infra/python/wheels/flake8-comprehensions-py3"
version: "version:3.16.0"
>
wheel: < wheel: <
name: "infra/python/wheels/isort-py3" name: "infra/python/wheels/isort-py3"
version: "version:5.10.1" version: "version:5.10.1"
+1 -1
View File
@@ -149,7 +149,7 @@ class ProxyManager:
while True: while True:
try: try:
procs.pop(0) procs.pop(0)
except: # noqa: E722 except IndexError:
break break
def close(self): def close(self):
+1
View File
@@ -94,6 +94,7 @@ It is equivalent to "git branch -D <branchname>".
def Execute(self, opt, args): def Execute(self, opt, args):
nb = args[0].split() nb = args[0].split()
self.TryOverrideManifestWithSmartSync()
err = collections.defaultdict(list) err = collections.defaultdict(list)
success = collections.defaultdict(list) success = collections.defaultdict(list)
aggregate_errors = [] aggregate_errors = []
+7 -7
View File
@@ -102,6 +102,12 @@ revision to a locally executed git command, use REPO_LREV.
REPO_RREV is the name of the revision from the manifest, exactly REPO_RREV is the name of the revision from the manifest, exactly
as written in the manifest. as written in the manifest.
REPO_UPSTREAM is the name of the upstream branch as specified in the
manifest.
REPO_DEST_BRANCH is the name of the destination branch for code review,
as specified in the manifest.
REPO_COUNT is the total number of projects being iterated. REPO_COUNT is the total number of projects being iterated.
REPO_I is the current (1-based) iteration count. Can be used in REPO_I is the current (1-based) iteration count. Can be used in
@@ -236,13 +242,7 @@ without iterating through the remaining projects.
mirror = self.manifest.IsMirror mirror = self.manifest.IsMirror
smart_sync_manifest_name = "smart_sync_override.xml" self.TryOverrideManifestWithSmartSync()
smart_sync_manifest_path = os.path.join(
self.manifest.manifestProject.worktree, smart_sync_manifest_name
)
if os.path.isfile(smart_sync_manifest_path):
self.manifest.Override(smart_sync_manifest_path)
if opt.regex: if opt.regex:
projects = self.FindProjects(args, all_manifests=all_trees) projects = self.FindProjects(args, all_manifests=all_trees)
+13 -4
View File
@@ -16,6 +16,7 @@ import os
from typing import List, Set from typing import List, Set
from command import Command from command import Command
from git_command import git_require
from git_command import GitCommand from git_command import GitCommand
import platform_utils import platform_utils
from progress import Progress from progress import Progress
@@ -204,6 +205,7 @@ class Gc(Command):
[ [
"rev-list", "rev-list",
"--objects", "--objects",
"--missing=allow-promisor",
f"--remotes={project.remote.name}", f"--remotes={project.remote.name}",
"--filter=blob:none", "--filter=blob:none",
"--tags", "--tags",
@@ -215,7 +217,12 @@ class Gc(Command):
# Get all local objects and pack them. # Get all local objects and pack them.
local_head_objects_cmd = GitCommand( local_head_objects_cmd = GitCommand(
project, project,
["rev-list", "--objects", "HEAD^{tree}"], [
"rev-list",
"--objects",
"--missing=allow-promisor",
"HEAD^{tree}",
],
capture_stdout=True, capture_stdout=True,
verify_command=True, verify_command=True,
) )
@@ -224,6 +231,7 @@ class Gc(Command):
[ [
"rev-list", "rev-list",
"--objects", "--objects",
"--missing=allow-promisor",
"--all", "--all",
"--reflog", "--reflog",
"--indexed-objects", "--indexed-objects",
@@ -297,7 +305,8 @@ class Gc(Command):
if ret != 0: if ret != 0:
return ret return ret
if not opt.repack: if opt.repack:
return git_require((2, 17, 0), fail=True, msg="--repack")
ret = self.repack_projects(projects, opt)
return self.repack_projects(projects, opt) return ret
+1 -1
View File
@@ -93,7 +93,7 @@ contain a line that matches both expressions:
pt = getattr(parser.values, "cmd_argv", None) pt = getattr(parser.values, "cmd_argv", None)
if pt is None: if pt is None:
pt = [] pt = []
setattr(parser.values, "cmd_argv", pt) parser.values.cmd_argv = pt
if opt_str == "-(": if opt_str == "-(":
pt.append("(") pt.append("(")
+3 -5
View File
@@ -59,7 +59,7 @@ Displays detailed usage information about a command.
def PrintAllCommandsBody(self): def PrintAllCommandsBody(self):
print("The complete list of recognized repo commands is:") print("The complete list of recognized repo commands is:")
commandNames = list(sorted(all_commands)) commandNames = sorted(all_commands)
self._PrintCommands(commandNames) self._PrintCommands(commandNames)
print( print(
"See 'repo help <command>' for more information on a " "See 'repo help <command>' for more information on a "
@@ -74,10 +74,8 @@ Displays detailed usage information about a command.
def PrintCommonCommandsBody(self): def PrintCommonCommandsBody(self):
print("The most commonly used repo commands are:") print("The most commonly used repo commands are:")
commandNames = list( commandNames = sorted(
sorted( name for name, command in all_commands.items() if command.COMMON
name for name, command in all_commands.items() if command.COMMON
)
) )
self._PrintCommands(commandNames) self._PrintCommands(commandNames)
+322 -124
View File
@@ -12,14 +12,41 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import enum
import functools
import io
import json
import optparse import optparse
import sys
from typing import Any, Dict, List, NamedTuple
from color import Coloring from color import Coloring
from command import DEFAULT_LOCAL_JOBS
from command import PagedCommand from command import PagedCommand
from git_refs import R_HEADS from git_refs import R_HEADS
from git_refs import R_M from git_refs import R_M
class BranchInfo(NamedTuple):
"""Holds information about a branch in a project."""
relpath: str
name: str
commits: Any
date: str
is_current: bool
class OutputFormat(enum.Enum):
"""Type for the requested output format."""
# Human-readable text output.
TEXT = enum.auto()
# Machine-readable JSON output.
JSON = enum.auto()
class _Coloring(Coloring): class _Coloring(Coloring):
def __init__(self, config): def __init__(self, config):
Coloring.__init__(self, config, "status") Coloring.__init__(self, config, "status")
@@ -27,10 +54,11 @@ class _Coloring(Coloring):
class Info(PagedCommand): class Info(PagedCommand):
COMMON = True COMMON = True
PARALLEL_JOBS = DEFAULT_LOCAL_JOBS
helpSummary = ( helpSummary = (
"Get info on the manifest branch, current branch or unmerged branches" "Get info on the manifest branch, current branch or unmerged branches"
) )
helpUsage = "%prog [-dl] [-o [-c]] [<project>...]" helpUsage = "%prog [-dl] [-o [-c]] [--format=<format>] [<project>...]"
def _Options(self, p): def _Options(self, p):
p.add_option( p.add_option(
@@ -46,6 +74,30 @@ class Info(PagedCommand):
action="store_true", action="store_true",
help="show overview of all local commits", help="show overview of all local commits",
) )
p.add_option(
"--include-summary",
action="store_true",
default=True,
help="include manifest summary (default: true)",
)
p.add_option(
"--no-include-summary",
dest="include_summary",
action="store_false",
help="exclude manifest summary",
)
p.add_option(
"--include-projects",
action="store_true",
default=True,
help="include project details (default: true)",
)
p.add_option(
"--no-include-projects",
dest="include_projects",
action="store_false",
help="exclude project details",
)
p.add_option( p.add_option(
"-c", "-c",
"--current-branch", "--current-branch",
@@ -72,8 +124,39 @@ class Info(PagedCommand):
action="store_true", action="store_true",
help="disable all remote operations", help="disable all remote operations",
) )
formats = tuple(x.lower() for x in OutputFormat.__members__.keys())
p.add_option(
"--format",
default=OutputFormat.TEXT.name.lower(),
choices=formats,
help=f"output format: {', '.join(formats)} (default: %default)",
)
def WantPager(self, opt):
return OutputFormat[opt.format.upper()] == OutputFormat.TEXT
def ValidateOptions(self, opt, args):
output_format = OutputFormat[opt.format.upper()]
if output_format == OutputFormat.JSON:
if opt.all:
self.OptionParser.error("--diff is not supported with JSON")
if opt.overview:
self.OptionParser.error("--overview is not supported with JSON")
def Execute(self, opt, args): def Execute(self, opt, args):
if not opt.this_manifest_only:
self.manifest = self.manifest.outer_client
self.TryOverrideManifestWithSmartSync()
output_format = OutputFormat[opt.format.upper()]
if output_format == OutputFormat.JSON:
self._ExecuteJson(opt, args)
else:
self._ExecuteText(opt, args)
def _ExecuteText(self, opt, args) -> None:
"""Output info as human-readable text."""
self.out = _Coloring(self.client.globalConfig) self.out = _Coloring(self.client.globalConfig)
self.heading = self.out.printer("heading", attr="bold") self.heading = self.out.printer("heading", attr="bold")
self.headtext = self.out.nofmt_printer("headtext", fg="yellow") self.headtext = self.out.nofmt_printer("headtext", fg="yellow")
@@ -84,157 +167,272 @@ class Info(PagedCommand):
self.opt = opt self.opt = opt
if not opt.this_manifest_only: if opt.include_summary:
self.manifest = self.manifest.outer_client self._printSummary()
manifestConfig = self.manifest.manifestProject.config
mergeBranch = manifestConfig.GetBranch("default").merge
manifestGroups = self.manifest.GetManifestGroupsStr()
self.heading("Manifest branch: ") if not opt.include_projects:
if self.manifest.default.revisionExpr: return
self.headtext(self.manifest.default.revisionExpr) elif not opt.overview:
self.out.nl()
self.heading("Manifest merge branch: ")
# The manifest might not have a merge branch if it isn't in a git repo,
# e.g. if `repo init --standalone-manifest` is used.
self.headtext(mergeBranch or "")
self.out.nl()
self.heading("Manifest groups: ")
self.headtext(manifestGroups)
self.out.nl()
sp = self.manifest.superproject
srev = sp.commit_id if sp and sp.commit_id else "None"
self.heading("Superproject revision: ")
self.headtext(srev)
self.out.nl()
self.printSeparator()
if not opt.overview:
self._printDiffInfo(opt, args) self._printDiffInfo(opt, args)
else: else:
self._printCommitOverview(opt, args) self._printCommitOverview(opt, args)
def _getSummaryData(self) -> Dict[str, Any]:
"""Gather manifest summary data as a dict."""
manifestConfig = self.manifest.manifestProject.config
mergeBranch = manifestConfig.GetBranch("default").merge
manifestGroups = self.manifest.GetManifestGroupsStr()
sp = self.manifest.superproject
srev = sp.commit_id if sp and sp.commit_id else None
return {
"manifest_branch": self.manifest.default.revisionExpr or "",
"manifest_merge_branch": mergeBranch or "",
"manifest_groups": manifestGroups,
"superproject_revision": srev,
}
def _getProjectData(self, project) -> Dict[str, Any]:
"""Gather project data as a dict."""
data = {
"name": project.name,
"mount_path": project.worktree,
"current_revision": project.GetRevisionId(),
"manifest_revision": project.revisionExpr,
"local_branches": list(project.GetBranches()),
}
currentBranch = project.CurrentBranch
if currentBranch:
data["current_branch"] = currentBranch
return data
def _ExecuteJson(self, opt, args) -> None:
"""Output info as JSON."""
result = {}
if opt.include_summary:
result["summary"] = self._getSummaryData()
if opt.include_projects:
projs = self.GetProjects(
args, all_manifests=not opt.this_manifest_only
)
result["projects"] = [self._getProjectData(p) for p in projs]
json_settings = {
# JSON style guide says Unicode characters are fully allowed.
"ensure_ascii": False,
# We use 2 space indent to match JSON style guide.
"indent": 2,
"separators": (",", ": "),
"sort_keys": True,
}
sys.stdout.write(json.dumps(result, **json_settings) + "\n")
def _printSummary(self) -> None:
"""Print manifest summary in text format."""
data = self._getSummaryData()
self.heading("Manifest branch: ")
self.headtext(data["manifest_branch"])
self.out.nl()
self.heading("Manifest merge branch: ")
self.headtext(data["manifest_merge_branch"])
self.out.nl()
self.heading("Manifest groups: ")
self.headtext(data["manifest_groups"])
self.out.nl()
self.heading("Superproject revision: ")
self.headtext(data["superproject_revision"] or "None")
self.out.nl()
self.printSeparator()
def printSeparator(self): def printSeparator(self):
self.text("----------------------------") self.text("----------------------------")
self.out.nl() self.out.nl()
@classmethod
def _DiffHelper(cls, project_idx: int, opt: Any) -> str:
"""Helper for ParallelContext to get diff info for a project."""
buf = io.StringIO()
project = cls.get_parallel_context()["projects"][project_idx]
config = cls.get_parallel_context()["config"]
out = _Coloring(config)
out.redirect(buf)
heading = out.printer("heading", attr="bold")
headtext = out.nofmt_printer("headtext", fg="yellow")
redtext = out.printer("redtext", fg="red")
sha = out.printer("sha", fg="yellow")
text = out.nofmt_printer("text")
dimtext = out.printer("dimtext", attr="dim")
heading("Project: ")
headtext(project.name)
out.nl()
heading("Mount path: ")
headtext(project.worktree)
out.nl()
heading("Current revision: ")
headtext(project.GetRevisionId())
out.nl()
currentBranch = project.CurrentBranch
if currentBranch:
heading("Current branch: ")
headtext(currentBranch)
out.nl()
heading("Manifest revision: ")
headtext(project.revisionExpr)
out.nl()
localBranches = list(project.GetBranches().keys())
heading("Local Branches: ")
redtext(str(len(localBranches)))
if localBranches:
text(" [")
text(", ".join(localBranches))
text("]")
out.nl()
if opt.all:
if not opt.local:
project.Sync_NetworkHalf(quiet=True, current_branch_only=True)
branch = project.manifest.manifestProject.config.GetBranch(
"default"
).merge
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS) :]
logTarget = R_M + branch
bareTmp = project.bare_git._bare
project.bare_git._bare = False
localCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
logTarget + "..",
"--",
)
originCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
".." + logTarget,
"--",
)
project.bare_git._bare = bareTmp
heading("Local Commits: ")
redtext(str(len(localCommits)))
dimtext(" (on current branch)")
out.nl()
for c in localCommits:
split = c.split()
sha(split[0] + " ")
text(" ".join(split[1:]))
out.nl()
text("----------------------------")
out.nl()
heading("Remote Commits: ")
redtext(str(len(originCommits)))
out.nl()
for c in originCommits:
split = c.split()
sha(split[0] + " ")
text(" ".join(split[1:]))
out.nl()
text("----------------------------")
out.nl()
return buf.getvalue()
def _printDiffInfo(self, opt, args): def _printDiffInfo(self, opt, args):
# We let exceptions bubble up to main as they'll be well structured.
projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only) projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only)
for p in projs: def _ProcessResults(_pool, _output, results):
self.heading("Project: ") for output in results:
self.headtext(p.name) if output:
self.out.nl() print(output, end="")
self.heading("Mount path: ") with self.ParallelContext():
self.headtext(p.worktree) self.get_parallel_context()["projects"] = projs
self.out.nl() self.get_parallel_context()[
"config"
] = self.manifest.manifestProject.config
self.heading("Current revision: ") self.ExecuteInParallel(
self.headtext(p.GetRevisionId()) opt.jobs,
self.out.nl() functools.partial(self._DiffHelper, opt=opt),
range(len(projs)),
callback=_ProcessResults,
ordered=True,
chunksize=1,
)
currentBranch = p.CurrentBranch @classmethod
if currentBranch: def _OverviewHelper(cls, project_idx: int, opt: Any) -> List[BranchInfo]:
self.heading("Current branch: ") """Helper to get overview of uploadable branches."""
self.headtext(currentBranch) project = cls.get_parallel_context()["projects"][project_idx]
self.out.nl()
self.heading("Manifest revision: ") branches = []
self.headtext(p.revisionExpr) br = [project.GetUploadableBranch(x) for x in project.GetBranches()]
self.out.nl() br = [x for x in br if x]
if opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch]
localBranches = list(p.GetBranches().keys()) for b in br:
self.heading("Local Branches: ") branches.append(
self.redtext(str(len(localBranches))) BranchInfo(
if localBranches: relpath=project.RelPath(local=opt.this_manifest_only),
self.text(" [") name=b.name,
self.text(", ".join(localBranches)) commits=b.commits,
self.text("]") date=b.date,
self.out.nl() is_current=b.name == project.CurrentBranch,
)
if self.opt.all: )
self.findRemoteLocalDiff(p) return branches
self.printSeparator()
def findRemoteLocalDiff(self, project):
# Fetch all the latest commits.
if not self.opt.local:
project.Sync_NetworkHalf(quiet=True, current_branch_only=True)
branch = self.manifest.manifestProject.config.GetBranch("default").merge
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS) :]
logTarget = R_M + branch
bareTmp = project.bare_git._bare
project.bare_git._bare = False
localCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
logTarget + "..",
"--",
)
originCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
".." + logTarget,
"--",
)
project.bare_git._bare = bareTmp
self.heading("Local Commits: ")
self.redtext(str(len(localCommits)))
self.dimtext(" (on current branch)")
self.out.nl()
for c in localCommits:
split = c.split()
self.sha(split[0] + " ")
self.text(" ".join(split[1:]))
self.out.nl()
self.printSeparator()
self.heading("Remote Commits: ")
self.redtext(str(len(originCommits)))
self.out.nl()
for c in originCommits:
split = c.split()
self.sha(split[0] + " ")
self.text(" ".join(split[1:]))
self.out.nl()
def _printCommitOverview(self, opt, args): def _printCommitOverview(self, opt, args):
projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only)
all_branches = [] all_branches = []
for project in self.GetProjects(
args, all_manifests=not opt.this_manifest_only def _ProcessResults(_pool, _output, results):
): for branches in results:
br = [project.GetUploadableBranch(x) for x in project.GetBranches()] all_branches.extend(branches)
br = [x for x in br if x]
if self.opt.current_branch: with self.ParallelContext():
br = [x for x in br if x.name == project.CurrentBranch] self.get_parallel_context()["projects"] = projs
all_branches.extend(br)
self.ExecuteInParallel(
opt.jobs,
functools.partial(self._OverviewHelper, opt=opt),
range(len(projs)),
callback=_ProcessResults,
ordered=True,
chunksize=1,
)
if not all_branches: if not all_branches:
return return
self.out.nl() self.out.nl()
self.heading("Projects Overview") self.heading("Projects Overview")
project = None current_relpath = None
for branch in all_branches: for branch in all_branches:
if project != branch.project: if current_relpath != branch.relpath:
project = branch.project current_relpath = branch.relpath
self.out.nl() self.out.nl()
self.headtext(project.RelPath(local=opt.this_manifest_only)) self.headtext(current_relpath)
self.out.nl() self.out.nl()
commits = branch.commits commits = branch.commits
@@ -242,7 +440,7 @@ class Info(PagedCommand):
self.text( self.text(
"%s %-33s (%2d commit%s, %s)" "%s %-33s (%2d commit%s, %s)"
% ( % (
branch.name == project.CurrentBranch and "*" or " ", branch.is_current and "*" or " ",
branch.name, branch.name,
len(commits), len(commits),
len(commits) != 1 and "s" or "", len(commits) != 1 and "s" or "",
+1
View File
@@ -104,6 +104,7 @@ revision specified in the manifest.
def Execute(self, opt, args): def Execute(self, opt, args):
nb = args[0] nb = args[0]
self.TryOverrideManifestWithSmartSync()
err_projects = [] err_projects = []
err = [] err = []
projects = [] projects = []
+166 -42
View File
@@ -64,6 +64,7 @@ from error import SyncError
from error import UpdateManifestError from error import UpdateManifestError
import event_log import event_log
from git_command import git_require from git_command import git_require
from git_command import GitCommand
from git_config import GetUrlCookieFile from git_config import GetUrlCookieFile
from git_refs import HEAD from git_refs import HEAD
from git_refs import R_HEADS from git_refs import R_HEADS
@@ -565,6 +566,11 @@ later is required to fix a server side protocol bug.
dest="use_superproject", dest="use_superproject",
help="disable use of manifest superprojects", help="disable use of manifest superprojects",
) )
p.add_option(
"--superproject-revision",
action="store",
help="sync to superproject revision (applies to outer manifest)",
)
p.add_option("--tags", action="store_true", help="fetch tags") p.add_option("--tags", action="store_true", help="fetch tags")
p.add_option( p.add_option(
"--no-tags", "--no-tags",
@@ -668,6 +674,24 @@ later is required to fix a server side protocol bug.
or opt.current_branch_only or opt.current_branch_only
) )
def _ConfigureSuperproject(
self,
opt: optparse.Values,
manifest,
revision: Optional[str] = None,
) -> bool:
"""Configure superproject with options."""
if not manifest.superproject:
return False
manifest.superproject.SetQuiet(not opt.verbose)
print_messages = git_superproject.PrintMessages(
opt.use_superproject, manifest
)
manifest.superproject.SetPrintMessages(print_messages)
if revision:
manifest.superproject.SetRevisionId(revision)
return print_messages
def _UpdateProjectsRevisionId( def _UpdateProjectsRevisionId(
self, opt, args, superproject_logging_data, manifest self, opt, args, superproject_logging_data, manifest
): ):
@@ -741,11 +765,7 @@ later is required to fix a server side protocol bug.
if not use_super: if not use_super:
continue continue
m.superproject.SetQuiet(not opt.verbose) print_messages = self._ConfigureSuperproject(opt, m)
print_messages = git_superproject.PrintMessages(
opt.use_superproject, m
)
m.superproject.SetPrintMessages(print_messages)
update_result = m.superproject.UpdateProjectsRevisionId( update_result = m.superproject.UpdateProjectsRevisionId(
per_manifest[m.path_prefix], git_event_log=self.git_event_log per_manifest[m.path_prefix], git_event_log=self.git_event_log
) )
@@ -841,6 +861,8 @@ later is required to fix a server side protocol bug.
) )
except KeyboardInterrupt: except KeyboardInterrupt:
logger.error("Keyboard interrupt while processing %s", project.name) logger.error("Keyboard interrupt while processing %s", project.name)
if not cls.is_multiprocessing_active():
raise
except GitError as e: except GitError as e:
logger.error("error.GitError: Cannot fetch %s", e) logger.error("error.GitError: Cannot fetch %s", e)
errors.append(e) errors.append(e)
@@ -947,7 +969,7 @@ later is required to fix a server side protocol bug.
"sync_dict" "sync_dict"
] = multiprocessing.Manager().dict() ] = multiprocessing.Manager().dict()
objdir_project_map = dict() objdir_project_map = {}
for index, project in enumerate(projects): for index, project in enumerate(projects):
objdir_project_map.setdefault(project.objdir, []).append(index) objdir_project_map.setdefault(project.objdir, []).append(index)
projects_list = list(objdir_project_map.values()) projects_list = list(objdir_project_map.values())
@@ -1104,6 +1126,8 @@ later is required to fix a server side protocol bug.
errors.extend(syncbuf.errors) errors.extend(syncbuf.errors)
except KeyboardInterrupt: except KeyboardInterrupt:
logger.error("Keyboard interrupt while processing %s", project.name) logger.error("Keyboard interrupt while processing %s", project.name)
if not cls.is_multiprocessing_active():
raise
except GitError as e: except GitError as e:
logger.error( logger.error(
"error.GitError: Cannot checkout %s: %s", project.name, e "error.GitError: Cannot checkout %s: %s", project.name, e
@@ -1244,14 +1268,15 @@ later is required to fix a server side protocol bug.
return False return False
def _SetPreciousObjectsState(self, project: Project, opt): @classmethod
def _SetPreciousObjectsState(cls, project: Project, opt):
"""Correct the preciousObjects state for the project. """Correct the preciousObjects state for the project.
Args: Args:
project: the project to examine, and possibly correct. project: the project to examine, and possibly correct.
opt: options given to sync. opt: options given to sync.
""" """
expected = self._GetPreciousObjectsState(project, opt) expected = cls._GetPreciousObjectsState(project, opt)
actual = ( actual = (
project.config.GetBoolean("extensions.preciousObjects") or False project.config.GetBoolean("extensions.preciousObjects") or False
) )
@@ -1285,7 +1310,22 @@ later is required to fix a server side protocol bug.
project.config.SetString("extensions.preciousObjects", None) project.config.SetString("extensions.preciousObjects", None)
project.config.SetString("gc.pruneExpire", None) project.config.SetString("gc.pruneExpire", None)
def _GCProjects(self, projects, opt, err_event): @staticmethod
def _RunOneGC(project: Project, config: Optional[dict] = None) -> None:
"""Run auto GC on a single project."""
local_config = {}
if config:
local_config.update(config)
local_config["gc.autoDetach"] = "false"
project.bare_git.gc("--auto", config=local_config)
@classmethod
def _GCProjects(
cls,
projects: List[Project],
opt: optparse.Values,
err_event: _threading.Event,
) -> None:
"""Perform garbage collection. """Perform garbage collection.
If We are skipping garbage collection (opt.auto_gc not set), we still If We are skipping garbage collection (opt.auto_gc not set), we still
@@ -1295,7 +1335,7 @@ later is required to fix a server side protocol bug.
if not opt.auto_gc: if not opt.auto_gc:
# Just repair preciousObjects state, and return. # Just repair preciousObjects state, and return.
for project in projects: for project in projects:
self._SetPreciousObjectsState(project, opt) cls._SetPreciousObjectsState(project, opt)
return return
pm = Progress( pm = Progress(
@@ -1305,9 +1345,8 @@ later is required to fix a server side protocol bug.
tidy_dirs = {} tidy_dirs = {}
for project in projects: for project in projects:
self._SetPreciousObjectsState(project, opt) cls._SetPreciousObjectsState(project, opt)
project.config.SetString("gc.autoDetach", "false")
# Only call git gc once per objdir, but call pack-refs for the # Only call git gc once per objdir, but call pack-refs for the
# remainder. # remainder.
if project.objdir not in tidy_dirs: if project.objdir not in tidy_dirs:
@@ -1328,7 +1367,7 @@ later is required to fix a server side protocol bug.
pm.update(msg=bare_git._project.name) pm.update(msg=bare_git._project.name)
if run_gc: if run_gc:
bare_git.gc("--auto") cls._RunOneGC(bare_git._project)
else: else:
bare_git.pack_refs() bare_git.pack_refs()
pm.end() pm.end()
@@ -1345,7 +1384,7 @@ later is required to fix a server side protocol bug.
try: try:
try: try:
if run_gc: if run_gc:
bare_git.gc("--auto", config=config) cls._RunOneGC(bare_git._project, config=config)
else: else:
bare_git.pack_refs(config=config) bare_git.pack_refs(config=config)
except GitError: except GitError:
@@ -1444,11 +1483,14 @@ later is required to fix a server side protocol bug.
if not git_require((2, 23, 0)): if not git_require((2, 23, 0)):
return return
projects = [p for p in projects if p.clone_depth] projects = [
p
for p in projects
if p.clone_depth and not p.stateless_prune_needed
]
if not projects: if not projects:
return return
bloated_projects = []
pm = Progress( pm = Progress(
"Checking for bloat", len(projects), delay=False, quiet=opt.quiet "Checking for bloat", len(projects), delay=False, quiet=opt.quiet
) )
@@ -1456,7 +1498,7 @@ later is required to fix a server side protocol bug.
def _ProcessResults(pool, pm, results): def _ProcessResults(pool, pm, results):
for result in results: for result in results:
if result: if result:
bloated_projects.append(result) self._bloated_projects.append(result)
pm.update(msg="") pm.update(msg="")
with self.ParallelContext(): with self.ParallelContext():
@@ -1471,15 +1513,6 @@ later is required to fix a server side protocol bug.
) )
pm.end() pm.end()
for project_name in bloated_projects:
warn_msg = (
f'warning: Project "{project_name}" is accumulating '
'unoptimized data. Please run "repo sync --auto-gc" or '
'"repo gc --repack" to clean up.'
)
self.git_event_log.ErrorEvent(warn_msg)
logger.warning(warn_msg)
def _UpdateRepoProject(self, opt, manifest, errors): def _UpdateRepoProject(self, opt, manifest, errors):
"""Fetch the repo project and check for updates.""" """Fetch the repo project and check for updates."""
if opt.local_only: if opt.local_only:
@@ -1618,9 +1651,10 @@ later is required to fix a server side protocol bug.
new_paths = {} new_paths = {}
new_linkfile_paths = [] new_linkfile_paths = []
new_copyfile_paths = [] new_copyfile_paths = []
for project in self.GetProjects( projects = self.GetProjects(
None, missing_ok=True, manifest=manifest, all_manifests=False None, missing_ok=True, manifest=manifest, all_manifests=False
): )
for project in projects:
new_linkfile_paths.extend(x.dest for x in project.linkfiles) new_linkfile_paths.extend(x.dest for x in project.linkfiles)
new_copyfile_paths.extend(x.dest for x in project.copyfiles) new_copyfile_paths.extend(x.dest for x in project.copyfiles)
@@ -1656,17 +1690,36 @@ later is required to fix a server side protocol bug.
) )
for need_remove_file in need_remove_files: for need_remove_file in need_remove_files:
# Try to remove the updated copyfile or linkfile. need_remove_path = os.path.join(
# So, if the file is not exist, nothing need to do. self.client.topdir, need_remove_file
platform_utils.remove(
os.path.join(self.client.topdir, need_remove_file),
missing_ok=True,
) )
if os.path.isfile(need_remove_path):
platform_utils.remove(need_remove_path)
else:
platform_utils.removedirs(need_remove_path)
# Also try to remove empty parent directories.
parent = os.path.dirname(need_remove_path)
while parent != self.client.topdir:
try:
os.rmdir(parent)
except OSError:
break
parent = os.path.dirname(parent)
# Create copy-link-files.json, save dest path of "copyfile" and # Create copy-link-files.json, save dest path of "copyfile" and
# "linkfile". # "linkfile".
with open(copylinkfile_path, "w", encoding="utf-8") as fp: with open(copylinkfile_path, "w", encoding="utf-8") as fp:
json.dump(new_paths, fp) json.dump(new_paths, fp)
# Retry linkfile/copyfile creation for all projects. In
# interleaved sync mode, _CopyAndLinkFiles runs before this
# cleanup, so linkfiles whose dest was blocked by an old
# directory may have failed. _CopyAndLinkFiles is idempotent
# and skips dests that are already correct.
for project in projects:
project._CopyAndLinkFiles()
return True return True
def _SmartSyncSetup(self, opt, smart_sync_manifest_path, manifest): def _SmartSyncSetup(self, opt, smart_sync_manifest_path, manifest):
@@ -1799,7 +1852,11 @@ later is required to fix a server side protocol bug.
mp: the manifestProject to query. mp: the manifestProject to query.
manifest_name: Manifest file to be reloaded. manifest_name: Manifest file to be reloaded.
""" """
if not mp.standalone_manifest_url: if opt.superproject_revision and mp.manifest == self.outer_manifest:
self._SyncToSuperprojectRev(
opt, mp.manifest, mp, manifest_name, errors
)
elif not mp.standalone_manifest_url:
self._UpdateManifestProject(opt, mp, manifest_name, errors) self._UpdateManifestProject(opt, mp, manifest_name, errors)
if mp.manifest.submanifests: if mp.manifest.submanifests:
@@ -1981,17 +2038,19 @@ later is required to fix a server side protocol bug.
def Execute(self, opt, args): def Execute(self, opt, args):
errors = [] errors = []
start_time = time.time()
try: try:
self._ExecuteHelper(opt, args, errors) self._ExecuteHelper(opt, args, errors)
sync_duration_seconds = time.time() - start_time
except (RepoExitError, RepoChangedException): except (RepoExitError, RepoChangedException):
raise raise
except (KeyboardInterrupt, Exception) as e: except (KeyboardInterrupt, Exception) as e:
raise RepoUnhandledExceptionError(e, aggregate_errors=errors) raise RepoUnhandledExceptionError(e, aggregate_errors=errors)
# Run post-sync hook only after successful sync # Run post-sync hook only after successful sync
self._RunPostSyncHook(opt) self._RunPostSyncHook(opt, sync_duration_seconds=sync_duration_seconds)
def _RunPostSyncHook(self, opt): def _RunPostSyncHook(self, opt, sync_duration_seconds=None):
"""Run post-sync hook if configured in manifest <repo-hooks>.""" """Run post-sync hook if configured in manifest <repo-hooks>."""
hook = RepoHook.FromSubcmd( hook = RepoHook.FromSubcmd(
hook_type="post-sync", hook_type="post-sync",
@@ -1999,10 +2058,60 @@ later is required to fix a server side protocol bug.
opt=opt, opt=opt,
abort_if_user_denies=False, abort_if_user_denies=False,
) )
success = hook.Run(repo_topdir=self.client.topdir) success = hook.Run(
repo_topdir=self.client.topdir,
sync_duration_seconds=sync_duration_seconds,
)
if not success: if not success:
print("Warning: post-sync hook reported failure.") print("Warning: post-sync hook reported failure.")
def _SyncToSuperprojectRev(
self,
opt: optparse.Values,
manifest,
mp: Project,
manifest_name: Optional[str],
errors: List[Exception],
) -> None:
"""Sync to a specific superproject commit."""
if not manifest.superproject:
raise SyncError("superproject not defined in manifest")
self._ConfigureSuperproject(
opt, manifest, revision=opt.superproject_revision
)
sync_result = manifest.superproject.Sync(self.git_event_log)
if not sync_result.success:
raise SyncError("failed to sync superproject")
cmd = ["show", f"{opt.superproject_revision}:.supermanifest"]
p = GitCommand(
None,
cmd,
gitdir=manifest.superproject._work_git,
bare=True,
capture_stdout=True,
capture_stderr=True,
)
if p.Wait() != 0:
raise SyncError(
f"failed to read .supermanifest from superproject: {p.stderr}"
)
try:
_, _, manifest_commit = p.stdout.strip().split()
except ValueError:
raise SyncError("could not parse .supermanifest")
mp.SetRevision(manifest_commit)
try:
self._UpdateManifestProject(opt, mp, manifest_name, errors)
except UpdateManifestError as e:
raise SyncError(
"failed to sync manifest project", aggregate_errors=[e]
)
def _ExecuteHelper(self, opt, args, errors): def _ExecuteHelper(self, opt, args, errors):
manifest = self.outer_manifest manifest = self.outer_manifest
if not opt.outer_manifest: if not opt.outer_manifest:
@@ -2012,9 +2121,7 @@ later is required to fix a server side protocol bug.
manifest.Override(opt.manifest_name) manifest.Override(opt.manifest_name)
manifest_name = opt.manifest_name manifest_name = opt.manifest_name
smart_sync_manifest_path = os.path.join( smart_sync_manifest_path = self.GetSmartSyncOverridePath(manifest)
manifest.manifestProject.worktree, "smart_sync_override.xml"
)
if opt.clone_bundle is None: if opt.clone_bundle is None:
opt.clone_bundle = manifest.CloneBundle opt.clone_bundle = manifest.CloneBundle
@@ -2063,7 +2170,7 @@ later is required to fix a server side protocol bug.
): ):
mp.ConfigureCloneFilterForDepth("blob:none") mp.ConfigureCloneFilterForDepth("blob:none")
if opt.mp_update: if opt.mp_update or opt.superproject_revision:
self._UpdateAllManifestProjects(opt, mp, manifest_name, errors) self._UpdateAllManifestProjects(opt, mp, manifest_name, errors)
else: else:
print("Skipping update of local manifest project.") print("Skipping update of local manifest project.")
@@ -2097,6 +2204,7 @@ later is required to fix a server side protocol bug.
self._fetch_times = _FetchTimes(manifest) self._fetch_times = _FetchTimes(manifest)
self._local_sync_state = LocalSyncState(manifest) self._local_sync_state = LocalSyncState(manifest)
self._bloated_projects = []
if opt.interleaved: if opt.interleaved:
sync_method = self._SyncInterleaved sync_method = self._SyncInterleaved
@@ -2113,6 +2221,9 @@ later is required to fix a server side protocol bug.
superproject_logging_data, superproject_logging_data,
) )
if not opt.quiet:
print("Finalizing sync state...")
# Log the previous sync analysis state from the config. # Log the previous sync analysis state from the config.
self.git_event_log.LogDataConfigEvents( self.git_event_log.LogDataConfigEvents(
mp.config.GetSyncAnalysisStateData(), "previous_sync_state" mp.config.GetSyncAnalysisStateData(), "previous_sync_state"
@@ -2134,6 +2245,15 @@ later is required to fix a server side protocol bug.
if existing: if existing:
self._CheckForBloatedProjects(all_projects, opt) self._CheckForBloatedProjects(all_projects, opt)
for project_name in sorted(self._bloated_projects):
warn_msg = (
f'warning: Project "{project_name}" is accumulating '
'unoptimized data. Please run "git repack -a -d" in the '
'project directory or "repo gc --repack" to clean up.'
)
self.git_event_log.ErrorEvent(warn_msg)
logger.warning(warn_msg)
if not opt.quiet: if not opt.quiet:
print("repo sync has finished successfully.") print("repo sync has finished successfully.")
@@ -2383,6 +2503,8 @@ later is required to fix a server side protocol bug.
logger.error( logger.error(
"Keyboard interrupt while processing %s", project.name "Keyboard interrupt while processing %s", project.name
) )
if not cls.is_multiprocessing_active():
raise
except GitError as e: except GitError as e:
fetch_errors.append(e) fetch_errors.append(e)
logger.error("error.GitError: Cannot fetch %s", e) logger.error("error.GitError: Cannot fetch %s", e)
@@ -2433,6 +2555,8 @@ later is required to fix a server side protocol bug.
logger.error( logger.error(
"Keyboard interrupt while processing %s", project.name "Keyboard interrupt while processing %s", project.name
) )
if not cls.is_multiprocessing_active():
raise
except GitError as e: except GitError as e:
checkout_errors.append(e) checkout_errors.append(e)
logger.error( logger.error(
@@ -2657,7 +2781,7 @@ later is required to fix a server side protocol bug.
if previously_pending_relpaths == pending_relpaths: if previously_pending_relpaths == pending_relpaths:
stalled_projects_str = "\n".join( stalled_projects_str = "\n".join(
f" - {path}" f" - {path}"
for path in sorted(list(pending_relpaths)) for path in sorted(pending_relpaths)
) )
logger.error( logger.error(
"The following projects failed and could " "The following projects failed and could "
+4 -3
View File
@@ -803,9 +803,10 @@ Gerrit Code Review: https://www.gerritcodereview.com/
project_list=pending_proj_names, worktree_list=pending_worktrees project_list=pending_proj_names, worktree_list=pending_worktrees
): ):
if LocalSyncState(manifest).IsPartiallySynced(): if LocalSyncState(manifest).IsPartiallySynced():
logger.error( logger.info(
"Partially synced tree detected. Syncing all projects " "Tip: A partially synced tree was detected. "
"may resolve issues you're seeing." "If this failure involves cross-project dependencies, "
"a full `repo sync` might help."
) )
ret = 1 ret = 1
if ret: if ret:
+11
View File
@@ -18,6 +18,7 @@ import pathlib
import pytest import pytest
import color
import platform_utils import platform_utils
import repo_trace import repo_trace
@@ -28,6 +29,16 @@ def disable_repo_trace(tmp_path):
repo_trace._TRACE_FILE = str(tmp_path / "TRACE_FILE_from_test") repo_trace._TRACE_FILE = str(tmp_path / "TRACE_FILE_from_test")
@pytest.fixture(autouse=True)
def restore_default_coloring() -> None:
"""Force disable color for reproducible test behavior.
The few tests that cover color behavior can still reset/change the color as
needed.
"""
color.SetDefaultColoring("never")
# adapted from pytest-home 0.5.1 # adapted from pytest-home 0.5.1
def _set_home(monkeypatch, path: pathlib.Path): def _set_home(monkeypatch, path: pathlib.Path):
""" """
+2 -8
View File
@@ -14,23 +14,17 @@
"""Unittests for the color.py module.""" """Unittests for the color.py module."""
import os
import pytest import pytest
import utils_for_test
import color import color
import git_config import git_config
def fixture(*paths: str) -> str:
"""Return a path relative to test/fixtures."""
return os.path.join(os.path.dirname(__file__), "fixtures", *paths)
@pytest.fixture @pytest.fixture
def coloring() -> color.Coloring: def coloring() -> color.Coloring:
"""Create a Coloring object for testing.""" """Create a Coloring object for testing."""
config_fixture = fixture("test.gitconfig") config_fixture = utils_for_test.FIXTURES_DIR / "test.gitconfig"
config = git_config.GitConfig(config_fixture) config = git_config.GitConfig(config_fixture)
color.SetDefaultColoring("true") color.SetDefaultColoring("true")
return color.Coloring(config, "status") return color.Coloring(config, "status")
+88
View File
@@ -0,0 +1,88 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the command.py module."""
from command import Command
class FakeProject:
"""Minimal project double for Command.GetProjects tests."""
def __init__(
self,
name,
relpath,
*,
gitdir=None,
derived_subprojects=None,
sync_s=False,
):
self.name = name
self.relpath = relpath
self.gitdir = gitdir or f"/git/{relpath}"
self.sync_s = sync_s
self.Exists = True
self._derived_subprojects = derived_subprojects or []
def GetDerivedSubprojects(self):
return list(self._derived_subprojects)
def MatchesGroups(self, _groups):
return True
def RelPath(self, local=True):
return self.relpath
class FakeManifest:
"""Minimal manifest double for Command.GetProjects tests."""
def __init__(self, projects):
self.projects = projects
def GetManifestGroupsStr(self):
return "default"
def test_get_projects_keeps_derived_subprojects_for_repeated_repo():
"""Derived subprojects are keyed by checkout path, not repo identity."""
submodule_a = FakeProject(
"submodule",
"src/one/submodule",
gitdir="/shared/modules/submodule.git",
)
submodule_b = FakeProject(
"submodule",
"src/two/submodule",
gitdir="/shared/modules/submodule.git",
)
project_a = FakeProject(
"project",
"src/one",
derived_subprojects=[submodule_a],
sync_s=True,
)
project_b = FakeProject(
"project",
"src/two",
derived_subprojects=[submodule_b],
sync_s=True,
)
manifest = FakeManifest([project_a, project_b])
cmd = Command(manifest=manifest)
projects = cmd.GetProjects([])
assert set(projects) == {project_a, project_b, submodule_a, submodule_b}
+21 -8
View File
@@ -14,24 +14,19 @@
"""Unittests for the git_config.py module.""" """Unittests for the git_config.py module."""
import os
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
import pytest import pytest
import utils_for_test
import git_config import git_config
def fixture_path(*paths: str) -> str:
"""Return a path relative to test/fixtures."""
return os.path.join(os.path.dirname(__file__), "fixtures", *paths)
@pytest.fixture @pytest.fixture
def readonly_config() -> git_config.GitConfig: def readonly_config() -> git_config.GitConfig:
"""Create a GitConfig object using the test.gitconfig fixture.""" """Create a GitConfig object using the test.gitconfig fixture."""
config_fixture = fixture_path("test.gitconfig") config_fixture = utils_for_test.FIXTURES_DIR / "test.gitconfig"
return git_config.GitConfig(config_fixture) return git_config.GitConfig(config_fixture)
@@ -63,7 +58,7 @@ def test_get_string_with_true_value(
def test_get_string_from_missing_file() -> None: def test_get_string_from_missing_file() -> None:
"""Test missing config file.""" """Test missing config file."""
config_fixture = fixture_path("not.present.gitconfig") config_fixture = utils_for_test.FIXTURES_DIR / "not.present.gitconfig"
config = git_config.GitConfig(config_fixture) config = git_config.GitConfig(config_fixture)
val = config.GetString("empty") val = config.GetString("empty")
assert val is None assert val is None
@@ -231,3 +226,21 @@ def test_get_sync_analysis_state_data(rw_config_file: Path) -> None:
for key, value in TESTS: for key, value in TESTS:
assert sync_data[f"{git_config.SYNC_STATE_PREFIX}{key}"] == value assert sync_data[f"{git_config.SYNC_STATE_PREFIX}{key}"] == value
assert sync_data[f"{git_config.SYNC_STATE_PREFIX}main.synctime"] assert sync_data[f"{git_config.SYNC_STATE_PREFIX}main.synctime"]
def test_remote_save_with_push_url_without_projectname(
rw_config_file: Path,
) -> None:
"""Test saving a remote pushUrl when projectname is unset."""
config = git_config.GitConfig(str(rw_config_file))
remote = config.GetRemote("origin")
remote.url = "https://example.com/repo"
remote.pushUrl = "ssh://example.com"
remote.projectname = None
remote.Save()
written_config = git_config.GitConfig(str(rw_config_file))
assert (
written_config.GetString("remote.origin.pushurl") == "ssh://example.com"
)
+1 -1
View File
@@ -190,7 +190,7 @@ class SuperprojectTestCase(unittest.TestCase):
), ),
revision="refs/heads/main", revision="refs/heads/main",
) )
with mock.patch.object(self._superproject, "_branch", "junk"): with mock.patch.object(self._superproject, "revision", "junk"):
sync_result = self._superproject.Sync(self.git_event_log) sync_result = self._superproject.Sync(self.git_event_log)
self.assertFalse(sync_result.success) self.assertFalse(sync_result.success)
self.assertTrue(sync_result.fatal) self.assertTrue(sync_result.fatal)
+390 -363
View File
@@ -18,17 +18,24 @@ import contextlib
import io import io
import json import json
import os import os
import re
import socket import socket
import tempfile import tempfile
import threading import threading
import unittest from typing import Any, Dict, List, Optional
from unittest import mock from unittest import mock
import pytest
import git_trace2_event_log import git_trace2_event_log
import platform_utils import platform_utils
def serverLoggingThread(socket_path, server_ready, received_traces): def server_logging_thread(
socket_path: str,
server_ready: threading.Condition,
received_traces: List[str],
) -> None:
"""Helper function to receive logs over a Unix domain socket. """Helper function to receive logs over a Unix domain socket.
Appends received messages on the provided socket and appends to Appends received messages on the provided socket and appends to
@@ -57,405 +64,425 @@ def serverLoggingThread(socket_path, server_ready, received_traces):
received_traces.extend(data.decode("utf-8").splitlines()) received_traces.extend(data.decode("utf-8").splitlines())
class EventLogTestCase(unittest.TestCase): PARENT_SID_KEY = "GIT_TRACE2_PARENT_SID"
"""TestCase for the EventLog module.""" PARENT_SID_VALUE = "parent_sid"
SELF_SID_REGEX = r"repo-\d+T\d+Z-.*"
FULL_SID_REGEX = rf"^{PARENT_SID_VALUE}/{SELF_SID_REGEX}"
PARENT_SID_KEY = "GIT_TRACE2_PARENT_SID"
PARENT_SID_VALUE = "parent_sid"
SELF_SID_REGEX = r"repo-\d+T\d+Z-.*"
FULL_SID_REGEX = rf"^{PARENT_SID_VALUE}/{SELF_SID_REGEX}"
def setUp(self): @pytest.fixture
"""Load the event_log module every time.""" def event_log() -> git_trace2_event_log.EventLog:
self._event_log = None """Fixture for the EventLog module."""
# By default we initialize with the expected case where # By default we initialize with the expected case where
# repo launches us (so GIT_TRACE2_PARENT_SID is set). # repo launches us (so GIT_TRACE2_PARENT_SID is set).
env = { env = {PARENT_SID_KEY: PARENT_SID_VALUE}
self.PARENT_SID_KEY: self.PARENT_SID_VALUE, return git_trace2_event_log.EventLog(env=env)
}
self._event_log = git_trace2_event_log.EventLog(env=env)
self._log_data = None
def verifyCommonKeys(
self, log_entry, expected_event_name=None, full_sid=True def verify_common_keys(
log_entry: Dict[str, Any],
expected_event_name: Optional[str] = None,
full_sid: bool = True,
) -> None:
"""Helper function to verify common event log keys."""
assert "event" in log_entry
assert "sid" in log_entry
assert "thread" in log_entry
assert "time" in log_entry
# Do basic data format validation.
if expected_event_name:
assert expected_event_name == log_entry["event"]
if full_sid:
assert re.match(FULL_SID_REGEX, log_entry["sid"])
else:
assert re.match(SELF_SID_REGEX, log_entry["sid"])
assert re.match(r"^\d+-\d+-\d+T\d+:\d+:\d+\.\d+\+00:00$", log_entry["time"])
def read_log(log_path: str) -> List[Dict[str, Any]]:
"""Helper function to read log data into a list."""
log_data = []
with open(log_path, mode="rb") as f:
for line in f:
log_data.append(json.loads(line))
return log_data
def remove_prefix(s: str, prefix: str) -> str:
"""Return a copy string after removing |prefix| from |s|, if present or
the original string."""
if s.startswith(prefix):
return s[len(prefix) :]
else:
return s
def test_initial_state_with_parent_sid(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is set by parent."""
assert re.match(FULL_SID_REGEX, event_log.full_sid)
def test_initial_state_no_parent_sid() -> None:
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is not set."""
# Setup an empty environment dict (no parent sid).
event_log = git_trace2_event_log.EventLog(env={})
assert re.match(SELF_SID_REGEX, event_log.full_sid)
def test_version_event(event_log: git_trace2_event_log.EventLog) -> None:
"""Test 'version' event data is valid.
Verify that the 'version' event is written even when no other
events are added.
Expected event log:
<version event>
"""
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
# A log with no added events should only have the version entry.
assert len(log_data) == 1
version_event = log_data[0]
verify_common_keys(version_event, expected_event_name="version")
# Check for 'version' event specific fields.
assert "evt" in version_event
assert "exe" in version_event
# Verify "evt" version field is a string.
assert isinstance(version_event["evt"], str)
def test_start_event(event_log: git_trace2_event_log.EventLog) -> None:
"""Test and validate 'start' event data is valid.
Expected event log:
<version event>
<start event>
"""
event_log.StartEvent([])
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
start_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(start_event, expected_event_name="start")
# Check for 'start' event specific fields.
assert "argv" in start_event
assert isinstance(start_event["argv"], list)
def test_exit_event_result_none(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test 'exit' event data is valid when result is None.
We expect None result to be converted to 0 in the exit event data.
Expected event log:
<version event>
<exit event>
"""
event_log.ExitEvent(None)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
exit_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(exit_event, expected_event_name="exit")
# Check for 'exit' event specific fields.
assert "code" in exit_event
# 'None' result should convert to 0 (successful) return code.
assert exit_event["code"] == 0
def test_exit_event_result_integer(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test 'exit' event data is valid when result is an integer.
Expected event log:
<version event>
<exit event>
"""
event_log.ExitEvent(2)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
exit_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(exit_event, expected_event_name="exit")
# Check for 'exit' event specific fields.
assert "code" in exit_event
assert exit_event["code"] == 2
def test_command_event(event_log: git_trace2_event_log.EventLog) -> None:
"""Test and validate 'command' event data is valid.
Expected event log:
<version event>
<command event>
"""
event_log.CommandEvent(name="repo", subcommands=["init", "this"])
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
command_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(command_event, expected_event_name="cmd_name")
# Check for 'command' event specific fields.
assert "name" in command_event
assert command_event["name"] == "repo-init-this"
def test_def_params_event_repo_config(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test 'def_params' event data outputs only repo config keys.
Expected event log:
<version event>
<def_param event>
<def_param event>
"""
config = {
"git.foo": "bar",
"repo.partialclone": "true",
"repo.partialclonefilter": "blob:none",
}
event_log.DefParamRepoEvents(config)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 3
def_param_events = log_data[1:]
verify_common_keys(log_data[0], expected_event_name="version")
for event in def_param_events:
verify_common_keys(event, expected_event_name="def_param")
# Check for 'def_param' event specific fields.
assert "param" in event
assert "value" in event
assert event["param"].startswith("repo.")
def test_def_params_event_no_repo_config(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test 'def_params' event data won't output non-repo config keys.
Expected event log:
<version event>
"""
config = {
"git.foo": "bar",
"git.core.foo2": "baz",
}
event_log.DefParamRepoEvents(config)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 1
verify_common_keys(log_data[0], expected_event_name="version")
def test_data_event_config(event_log: git_trace2_event_log.EventLog) -> None:
"""Test 'data' event data outputs all config keys.
Expected event log:
<version event>
<data event>
<data event>
"""
config = {
"git.foo": "bar",
"repo.partialclone": "false",
"repo.syncstate.superproject.hassuperprojecttag": "true",
"repo.syncstate.superproject.sys.argv": ["--", "sync", "protobuf"],
}
prefix_value = "prefix"
event_log.LogDataConfigEvents(config, prefix_value)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 5
data_events = log_data[1:]
verify_common_keys(log_data[0], expected_event_name="version")
for event in data_events:
verify_common_keys(event)
# Check for 'data' event specific fields.
assert "key" in event
assert "value" in event
key = event["key"]
key = remove_prefix(key, f"{prefix_value}/")
value = event["value"]
assert event_log.GetDataEventName(value) == event["event"]
assert key in config
assert value == config[key]
def test_error_event(event_log: git_trace2_event_log.EventLog) -> None:
"""Test and validate 'error' event data is valid.
Expected event log:
<version event>
<error event>
"""
msg = "invalid option: --cahced"
fmt = "invalid option: %s"
event_log.ErrorEvent(msg, fmt)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
error_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(error_event, expected_event_name="error")
# Check for 'error' event specific fields.
assert "msg" in error_event
assert "fmt" in error_event
assert error_event["msg"] == f"RepoErrorEvent:{msg}"
assert error_event["fmt"] == f"RepoErrorEvent:{fmt}"
def test_write_with_filename(event_log: git_trace2_event_log.EventLog) -> None:
"""Test Write() with a path to a file exits with None."""
assert event_log.Write(path="path/to/file") is None
def test_write_with_git_config(
tmp_path,
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test Write() uses the git config path when 'git config' call succeeds."""
with mock.patch.object(
event_log,
"_GetEventTargetPath",
return_value=str(tmp_path),
): ):
"""Helper function to verify common event log keys.""" assert os.path.dirname(event_log.Write()) == str(tmp_path)
self.assertIn("event", log_entry)
self.assertIn("sid", log_entry)
self.assertIn("thread", log_entry)
self.assertIn("time", log_entry)
# Do basic data format validation.
if expected_event_name: def test_write_no_git_config(event_log: git_trace2_event_log.EventLog) -> None:
self.assertEqual(expected_event_name, log_entry["event"]) """Test Write() with no git config variable present exits with None."""
if full_sid: with mock.patch.object(event_log, "_GetEventTargetPath", return_value=None):
self.assertRegex(log_entry["sid"], self.FULL_SID_REGEX) assert event_log.Write() is None
else:
self.assertRegex(log_entry["sid"], self.SELF_SID_REGEX)
self.assertRegex( def test_write_non_string(event_log: git_trace2_event_log.EventLog) -> None:
log_entry["time"], r"^\d+-\d+-\d+T\d+:\d+:\d+\.\d+\+00:00$" """Test Write() with non-string type for |path| throws TypeError."""
with pytest.raises(TypeError):
event_log.Write(path=1234)
@pytest.mark.skipif(
not hasattr(socket, "AF_UNIX"), reason="Requires AF_UNIX sockets"
)
def test_write_socket(event_log: git_trace2_event_log.EventLog) -> None:
"""Test Write() with Unix domain socket and validate received traces."""
received_traces: List[str] = []
with tempfile.TemporaryDirectory(prefix="test_server_sockets") as tempdir:
socket_path = os.path.join(tempdir, "server.sock")
server_ready = threading.Condition()
# Start "server" listening on Unix domain socket at socket_path.
server_thread = threading.Thread(
target=server_logging_thread,
args=(socket_path, server_ready, received_traces),
) )
try:
server_thread.start()
def readLog(self, log_path): with server_ready:
"""Helper function to read log data into a list.""" server_ready.wait(timeout=120)
log_data = []
with open(log_path, mode="rb") as f:
for line in f:
log_data.append(json.loads(line))
return log_data
def remove_prefix(self, s, prefix): event_log.StartEvent([])
"""Return a copy string after removing |prefix| from |s|, if present or path = event_log.Write(path=f"af_unix:{socket_path}")
the original string.""" finally:
if s.startswith(prefix): server_thread.join(timeout=5)
return s[len(prefix) :]
else:
return s
def test_initial_state_with_parent_sid(self): assert path == f"af_unix:stream:{socket_path}"
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is set by parent.""" assert len(received_traces) == 2
self.assertRegex(self._event_log.full_sid, self.FULL_SID_REGEX) version_event = json.loads(received_traces[0])
start_event = json.loads(received_traces[1])
def test_initial_state_no_parent_sid(self): verify_common_keys(version_event, expected_event_name="version")
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is not set.""" verify_common_keys(start_event, expected_event_name="start")
# Setup an empty environment dict (no parent sid). # Check for 'start' event specific fields.
self._event_log = git_trace2_event_log.EventLog(env={}) assert "argv" in start_event
self.assertRegex(self._event_log.full_sid, self.SELF_SID_REGEX) assert isinstance(start_event["argv"], list)
def test_version_event(self):
"""Test 'version' event data is valid.
Verify that the 'version' event is written even when no other
events are addded.
Expected event log:
<version event>
"""
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
# A log with no added events should only have the version entry.
self.assertEqual(len(self._log_data), 1)
version_event = self._log_data[0]
self.verifyCommonKeys(version_event, expected_event_name="version")
# Check for 'version' event specific fields.
self.assertIn("evt", version_event)
self.assertIn("exe", version_event)
# Verify "evt" version field is a string.
self.assertIsInstance(version_event["evt"], str)
def test_start_event(self):
"""Test and validate 'start' event data is valid.
Expected event log:
<version event>
<start event>
"""
self._event_log.StartEvent([])
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
start_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(start_event, expected_event_name="start")
# Check for 'start' event specific fields.
self.assertIn("argv", start_event)
self.assertTrue(isinstance(start_event["argv"], list))
def test_exit_event_result_none(self):
"""Test 'exit' event data is valid when result is None.
We expect None result to be converted to 0 in the exit event data.
Expected event log:
<version event>
<exit event>
"""
self._event_log.ExitEvent(None)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
exit_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(exit_event, expected_event_name="exit")
# Check for 'exit' event specific fields.
self.assertIn("code", exit_event)
# 'None' result should convert to 0 (successful) return code.
self.assertEqual(exit_event["code"], 0)
def test_exit_event_result_integer(self):
"""Test 'exit' event data is valid when result is an integer.
Expected event log:
<version event>
<exit event>
"""
self._event_log.ExitEvent(2)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
exit_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(exit_event, expected_event_name="exit")
# Check for 'exit' event specific fields.
self.assertIn("code", exit_event)
self.assertEqual(exit_event["code"], 2)
def test_command_event(self):
"""Test and validate 'command' event data is valid.
Expected event log:
<version event>
<command event>
"""
self._event_log.CommandEvent(name="repo", subcommands=["init", "this"])
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
command_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(command_event, expected_event_name="cmd_name")
# Check for 'command' event specific fields.
self.assertIn("name", command_event)
self.assertEqual(command_event["name"], "repo-init-this")
def test_def_params_event_repo_config(self):
"""Test 'def_params' event data outputs only repo config keys.
Expected event log:
<version event>
<def_param event>
<def_param event>
"""
config = {
"git.foo": "bar",
"repo.partialclone": "true",
"repo.partialclonefilter": "blob:none",
}
self._event_log.DefParamRepoEvents(config)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 3)
def_param_events = self._log_data[1:]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
for event in def_param_events:
self.verifyCommonKeys(event, expected_event_name="def_param")
# Check for 'def_param' event specific fields.
self.assertIn("param", event)
self.assertIn("value", event)
self.assertTrue(event["param"].startswith("repo."))
def test_def_params_event_no_repo_config(self):
"""Test 'def_params' event data won't output non-repo config keys.
Expected event log:
<version event>
"""
config = {
"git.foo": "bar",
"git.core.foo2": "baz",
}
self._event_log.DefParamRepoEvents(config)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 1)
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
def test_data_event_config(self):
"""Test 'data' event data outputs all config keys.
Expected event log:
<version event>
<data event>
<data event>
"""
config = {
"git.foo": "bar",
"repo.partialclone": "false",
"repo.syncstate.superproject.hassuperprojecttag": "true",
"repo.syncstate.superproject.sys.argv": ["--", "sync", "protobuf"],
}
prefix_value = "prefix"
self._event_log.LogDataConfigEvents(config, prefix_value)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 5)
data_events = self._log_data[1:]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
for event in data_events:
self.verifyCommonKeys(event)
# Check for 'data' event specific fields.
self.assertIn("key", event)
self.assertIn("value", event)
key = event["key"]
key = self.remove_prefix(key, f"{prefix_value}/")
value = event["value"]
self.assertEqual(
self._event_log.GetDataEventName(value), event["event"]
)
self.assertTrue(key in config and value == config[key])
def test_error_event(self):
"""Test and validate 'error' event data is valid.
Expected event log:
<version event>
<error event>
"""
msg = "invalid option: --cahced"
fmt = "invalid option: %s"
self._event_log.ErrorEvent(msg, fmt)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
error_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(error_event, expected_event_name="error")
# Check for 'error' event specific fields.
self.assertIn("msg", error_event)
self.assertIn("fmt", error_event)
self.assertEqual(error_event["msg"], f"RepoErrorEvent:{msg}")
self.assertEqual(error_event["fmt"], f"RepoErrorEvent:{fmt}")
def test_write_with_filename(self):
"""Test Write() with a path to a file exits with None."""
self.assertIsNone(self._event_log.Write(path="path/to/file"))
def test_write_with_git_config(self):
"""Test Write() uses the git config path when 'git config' call
succeeds."""
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
with mock.patch.object(
self._event_log,
"_GetEventTargetPath",
return_value=tempdir,
):
self.assertEqual(
os.path.dirname(self._event_log.Write()), tempdir
)
def test_write_no_git_config(self):
"""Test Write() with no git config variable present exits with None."""
with mock.patch.object(
self._event_log, "_GetEventTargetPath", return_value=None
):
self.assertIsNone(self._event_log.Write())
def test_write_non_string(self):
"""Test Write() with non-string type for |path| throws TypeError."""
with self.assertRaises(TypeError):
self._event_log.Write(path=1234)
@unittest.skipIf(not hasattr(socket, "AF_UNIX"), "Requires AF_UNIX sockets")
def test_write_socket(self):
"""Test Write() with Unix domain socket for |path| and validate received
traces."""
received_traces = []
with tempfile.TemporaryDirectory(
prefix="test_server_sockets"
) as tempdir:
socket_path = os.path.join(tempdir, "server.sock")
server_ready = threading.Condition()
# Start "server" listening on Unix domain socket at socket_path.
server_thread = threading.Thread(
target=serverLoggingThread,
args=(socket_path, server_ready, received_traces),
)
try:
server_thread.start()
with server_ready:
server_ready.wait(timeout=120)
self._event_log.StartEvent([])
path = self._event_log.Write(path=f"af_unix:{socket_path}")
finally:
server_thread.join(timeout=5)
self.assertEqual(path, f"af_unix:stream:{socket_path}")
self.assertEqual(len(received_traces), 2)
version_event = json.loads(received_traces[0])
start_event = json.loads(received_traces[1])
self.verifyCommonKeys(version_event, expected_event_name="version")
self.verifyCommonKeys(start_event, expected_event_name="start")
# Check for 'start' event specific fields.
self.assertIn("argv", start_event)
self.assertIsInstance(start_event["argv"], list)
class EventLogVerboseTestCase(unittest.TestCase): class TestEventLogVerbose:
"""TestCase for the EventLog module verbose logging.""" """TestCase for the EventLog module verbose logging."""
def setUp(self): def test_write_socket_error_no_verbose(self) -> None:
self._event_log = git_trace2_event_log.EventLog(env={})
def test_write_socket_error_no_verbose(self):
"""Test Write() suppression of socket errors when not verbose.""" """Test Write() suppression of socket errors when not verbose."""
self._event_log.verbose = False event_log = git_trace2_event_log.EventLog(env={})
event_log.verbose = False
with contextlib.redirect_stderr( with contextlib.redirect_stderr(
io.StringIO() io.StringIO()
) as mock_stderr, mock.patch("socket.socket", side_effect=OSError): ) as mock_stderr, mock.patch("socket.socket", side_effect=OSError):
self._event_log.Write(path="af_unix:stream:/tmp/test_sock") event_log.Write(path="af_unix:stream:/tmp/test_sock")
self.assertEqual(mock_stderr.getvalue(), "") assert mock_stderr.getvalue() == ""
def test_write_socket_error_verbose(self): def test_write_socket_error_verbose(self) -> None:
"""Test Write() printing of socket errors when verbose.""" """Test Write() printing of socket errors when verbose."""
self._event_log.verbose = True event_log = git_trace2_event_log.EventLog(env={})
event_log.verbose = True
with contextlib.redirect_stderr( with contextlib.redirect_stderr(
io.StringIO() io.StringIO()
) as mock_stderr, mock.patch( ) as mock_stderr, mock.patch(
"socket.socket", side_effect=OSError("Mock error") "socket.socket", side_effect=OSError("Mock error")
): ):
self._event_log.Write(path="af_unix:stream:/tmp/test_sock") event_log.Write(path="af_unix:stream:/tmp/test_sock")
self.assertIn( assert (
"git trace2 logging failed: Mock error", "git trace2 logging failed: Mock error"
mock_stderr.getvalue(), in mock_stderr.getvalue()
) )
def test_write_file_error_no_verbose(self): def test_write_file_error_no_verbose(self) -> None:
"""Test Write() suppression of file errors when not verbose.""" """Test Write() suppression of file errors when not verbose."""
self._event_log.verbose = False event_log = git_trace2_event_log.EventLog(env={})
event_log.verbose = False
with contextlib.redirect_stderr( with contextlib.redirect_stderr(
io.StringIO() io.StringIO()
) as mock_stderr, mock.patch( ) as mock_stderr, mock.patch(
"tempfile.NamedTemporaryFile", side_effect=FileExistsError "tempfile.NamedTemporaryFile", side_effect=FileExistsError
): ):
self._event_log.Write(path="/tmp") event_log.Write(path="/tmp")
self.assertEqual(mock_stderr.getvalue(), "") assert mock_stderr.getvalue() == ""
def test_write_file_error_verbose(self): def test_write_file_error_verbose(self) -> None:
"""Test Write() printing of file errors when verbose.""" """Test Write() printing of file errors when verbose."""
self._event_log.verbose = True event_log = git_trace2_event_log.EventLog(env={})
event_log.verbose = True
with contextlib.redirect_stderr( with contextlib.redirect_stderr(
io.StringIO() io.StringIO()
) as mock_stderr, mock.patch( ) as mock_stderr, mock.patch(
"tempfile.NamedTemporaryFile", "tempfile.NamedTemporaryFile",
side_effect=FileExistsError("Mock error"), side_effect=FileExistsError("Mock error"),
): ):
self._event_log.Write(path="/tmp") event_log.Write(path="/tmp")
self.assertIn( assert (
"git trace2 logging failed: FileExistsError", "git trace2 logging failed: FileExistsError"
mock_stderr.getvalue(), in mock_stderr.getvalue()
) )
+47
View File
@@ -14,6 +14,9 @@
"""Unittests for the hooks.py module.""" """Unittests for the hooks.py module."""
from io import StringIO
import sys
import pytest import pytest
import hooks import hooks
@@ -58,3 +61,47 @@ def test_direct_interp(shebang: str, interp: str) -> None:
def test_env_interp(shebang: str, interp: str) -> None: def test_env_interp(shebang: str, interp: str) -> None:
"""Lines whose shebang launches through `env`.""" """Lines whose shebang launches through `env`."""
assert hooks.RepoHook._ExtractInterpFromShebang(shebang) == interp assert hooks.RepoHook._ExtractInterpFromShebang(shebang) == interp
def test_post_sync_argument_validation() -> None:
"""Test that post-sync hook requires exact API arguments."""
class FakeProject:
def __init__(self):
self.worktree = "/some/path"
self.enabled_repo_hooks = ["post-sync"]
hook = hooks.RepoHook(
hook_type="post-sync",
hooks_project=FakeProject(),
repo_topdir="/topdir",
manifest_url="https://gerrit",
allow_all_hooks=True,
)
old_stderr = sys.stderr
sys.stderr = StringIO()
try:
# Call with missing arg `sync_duration_seconds`
res = hook.Run(repo_topdir="/topdir")
assert res is False
assert "hook 'post-sync' called incorrectly" in sys.stderr.getvalue()
# Mock _CheckHook and _ExecuteHook to test success path
hook._CheckHook = lambda: None
executed_kwargs = {}
def fake_execute(**kw):
executed_kwargs.update(kw)
hook._ExecuteHook = fake_execute
res = hook.Run(repo_topdir="/topdir", sync_duration_seconds=12.345)
assert res is True
assert executed_kwargs.get("sync_duration_seconds") == 12.345
finally:
sys.stderr = old_stderr
+166
View File
@@ -0,0 +1,166 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests for the main repo script and subcommand routing."""
from unittest import mock
import pytest
from main import _Repo
@pytest.fixture(name="repo")
def fixture_repo():
repo = _Repo("repodir")
# Overriding the command list here ensures that we are only testing
# against a fixed set of commands, reducing fragility to new
# subcommands being added to the main repo tool.
repo.commands = {"start": None, "sync": None, "smart": None}
return repo
@pytest.fixture(name="mock_config")
def fixture_mock_config():
return mock.MagicMock()
@mock.patch("time.sleep")
def test_autocorrect_delay(mock_sleep, repo, mock_config):
"""Test autocorrect with positive delay."""
mock_config.GetString.return_value = "10"
res = repo._autocorrect_command_name("tart", mock_config)
mock_config.GetString.assert_called_with("help.autocorrect")
mock_sleep.assert_called_with(1.0)
assert res == "start"
@mock.patch("time.sleep")
def test_autocorrect_delay_one(mock_sleep, repo, mock_config):
"""Test autocorrect with '1' (0.1s delay, not immediate)."""
mock_config.GetString.return_value = "1"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_called_with(0.1)
assert res == "start"
@mock.patch("time.sleep", side_effect=KeyboardInterrupt())
def test_autocorrect_delay_interrupt(mock_sleep, repo, mock_config):
"""Test autocorrect handles KeyboardInterrupt during delay."""
mock_config.GetString.return_value = "10"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_called_with(1.0)
assert res is None
@mock.patch("time.sleep")
def test_autocorrect_immediate(mock_sleep, repo, mock_config):
"""Test autocorrect with immediate/negative delay."""
# Test numeric negative.
mock_config.GetString.return_value = "-1"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_not_called()
assert res == "start"
# Test string boolean "true".
mock_config.GetString.return_value = "true"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_not_called()
assert res == "start"
# Test string boolean "yes".
mock_config.GetString.return_value = "YES"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_not_called()
assert res == "start"
# Test string boolean "immediate".
mock_config.GetString.return_value = "Immediate"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_not_called()
assert res == "start"
def test_autocorrect_zero_or_show(repo, mock_config):
"""Test autocorrect with zero delay (suggestions only)."""
# Test numeric zero.
mock_config.GetString.return_value = "0"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
# Test string boolean "false".
mock_config.GetString.return_value = "False"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
# Test string boolean "show".
mock_config.GetString.return_value = "show"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
def test_autocorrect_never(repo, mock_config):
"""Test autocorrect with 'never'."""
mock_config.GetString.return_value = "never"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
@mock.patch("builtins.input", return_value="y")
def test_autocorrect_prompt_yes(mock_input, repo, mock_config):
"""Test autocorrect with prompt and user answers yes."""
mock_config.GetString.return_value = "prompt"
res = repo._autocorrect_command_name("tart", mock_config)
assert res == "start"
@mock.patch("builtins.input", return_value="n")
def test_autocorrect_prompt_no(mock_input, repo, mock_config):
"""Test autocorrect with prompt and user answers no."""
mock_config.GetString.return_value = "prompt"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
@mock.patch("builtins.input", return_value="y")
def test_autocorrect_multiple_candidates(mock_input, repo, mock_config):
"""Test autocorrect with multiple matches forces a prompt."""
mock_config.GetString.return_value = "10" # Normally just delay
# 'snart' matches both 'start' and 'smart' with > 0.7 ratio
res = repo._autocorrect_command_name("snart", mock_config)
# Because there are multiple candidates, it should prompt
mock_input.assert_called_once()
assert res == "start"
@mock.patch("builtins.input", side_effect=KeyboardInterrupt())
def test_autocorrect_prompt_interrupt(mock_input, repo, mock_config):
"""Test autocorrect with prompt and user interrupts."""
mock_config.GetString.return_value = "prompt"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
+559 -498
View File
File diff suppressed because it is too large Load Diff
+68
View File
@@ -46,3 +46,71 @@ def test_remove_missing_ok(tmp_path: Path) -> None:
path.touch() path.touch()
platform_utils.remove(path, missing_ok=False) platform_utils.remove(path, missing_ok=False)
assert not path.exists() assert not path.exists()
def test_removedirs_nonexistent(tmp_path: Path) -> None:
"""removedirs should silently succeed on nonexistent paths."""
platform_utils.removedirs(tmp_path / "does-not-exist")
def test_removedirs_symlink(tmp_path: Path) -> None:
"""removedirs should remove a symlink."""
link = tmp_path / "link"
link.symlink_to("target")
platform_utils.removedirs(link)
assert not link.exists()
def test_removedirs_empty_dir(tmp_path: Path) -> None:
"""removedirs should remove an empty directory."""
d = tmp_path / "empty"
d.mkdir()
platform_utils.removedirs(d)
assert not d.exists()
def test_removedirs_nested_empty_dirs(tmp_path: Path) -> None:
"""removedirs should remove nested empty directories."""
d = tmp_path / "a" / "b" / "c"
d.mkdir(parents=True)
platform_utils.removedirs(tmp_path / "a")
assert not (tmp_path / "a").exists()
def test_removedirs_symlinks_inside_dir(tmp_path: Path) -> None:
"""removedirs should remove symlinks inside a directory."""
d = tmp_path / "dir"
d.mkdir()
(d / "link1").symlink_to("target1")
(d / "link2").symlink_to("target2")
platform_utils.removedirs(d)
assert not d.exists()
def test_removedirs_preserves_user_files(tmp_path: Path) -> None:
"""removedirs should not delete regular files or their parent dirs."""
d = tmp_path / "dir"
d.mkdir()
(d / "link").symlink_to("target")
(d / "user-file.txt").write_text("keep me")
platform_utils.removedirs(d)
assert d.exists()
assert not (d / "link").exists()
assert (d / "user-file.txt").read_text() == "keep me"
def test_removedirs_deep_nested_with_symlinks(tmp_path: Path) -> None:
"""removedirs should handle deep nesting: sub/dir/target."""
d = tmp_path / "sub" / "dir"
d.mkdir(parents=True)
(d / "link").symlink_to("target")
platform_utils.removedirs(tmp_path / "sub")
assert not (tmp_path / "sub").exists()
def test_removedirs_regular_file_noop(tmp_path: Path) -> None:
"""removedirs should not delete a regular file."""
f = tmp_path / "file.txt"
f.write_text("data")
platform_utils.removedirs(f)
assert f.exists()
+382
View File
@@ -17,10 +17,12 @@
import contextlib import contextlib
import os import os
from pathlib import Path from pathlib import Path
import shutil
import subprocess import subprocess
import tempfile import tempfile
from typing import Optional from typing import Optional
import unittest import unittest
from unittest import mock
import utils_for_test import utils_for_test
@@ -125,6 +127,88 @@ class ProjectTests(unittest.TestCase):
).strip() ).strip()
self.assertEqual(expected, fakeproj.work_git.GetHead()) self.assertEqual(expected, fakeproj.work_git.GetHead())
def _get_derived_subproject_url(self, submodule_url):
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
class FakeManifest:
def __init__(self, topdir):
self.topdir = topdir
self.globalConfig = None
self.is_multimanifest = False
self.path_prefix = ""
self.paths = {}
def GetSubprojectName(self, parent, path):
return path
def GetSubprojectPaths(self, parent, name, path):
relpath = path
worktree = os.path.join(self.topdir, path)
gitdir = os.path.join(self.topdir, f"{path}.git")
objdir = os.path.join(self.topdir, f"{path}.obj")
os.makedirs(worktree, exist_ok=True)
os.makedirs(gitdir, exist_ok=True)
os.makedirs(objdir, exist_ok=True)
return relpath, worktree, gitdir, objdir
manifest = FakeManifest(tempdir)
worktree = os.path.join(tempdir, "parent")
gitdir = os.path.join(tempdir, "parent.git")
objdir = os.path.join(tempdir, "parent.obj")
os.makedirs(worktree)
os.makedirs(gitdir)
os.makedirs(objdir)
parent = project.Project(
manifest=manifest,
name="parent",
remote=project.RemoteSpec(
"origin", url="https://example.com/platform/superproject"
),
gitdir=gitdir,
objdir=objdir,
worktree=worktree,
relpath="parent",
revisionExpr="refs/heads/main",
revisionId=None,
)
def fake_get_submodules(current):
if current is parent:
return [("subrev", "child", submodule_url, "false")]
return []
with mock.patch.object(
project.Project, "_GetSubmodules", autospec=True
) as get_submodules:
get_submodules.side_effect = fake_get_submodules
result = parent.GetDerivedSubprojects()
self.assertEqual(1, len(result))
return result[0].remote.url
def test_derived_subproject_joins_only_git_relative_urls(self):
tests = (
(
"./submodule",
"https://example.com/platform/superproject/submodule",
),
("../sibling", "https://example.com/platform/sibling"),
)
for submodule_url, expected in tests:
with self.subTest(submodule_url=submodule_url):
self.assertEqual(
expected, self._get_derived_subproject_url(submodule_url)
)
def test_derived_subproject_leaves_dot_prefixed_names_unchanged(self):
for submodule_url in (".foo", "..bar"):
with self.subTest(submodule_url=submodule_url):
self.assertEqual(
submodule_url,
self._get_derived_subproject_url(submodule_url),
)
class CopyLinkTestCase(unittest.TestCase): class CopyLinkTestCase(unittest.TestCase):
"""TestCase for stub repo client checkouts. """TestCase for stub repo client checkouts.
@@ -363,6 +447,47 @@ class LinkFile(CopyLinkTestCase):
os.path.join("git-project", "foo.txt"), os.readlink(dest) os.path.join("git-project", "foo.txt"), os.readlink(dest)
) )
def test_replace_empty_dir_with_symlink(self):
"""A linkfile should replace an empty real directory at the dest path.
This is the common case: the old linkfiles inside the directory were
already cleaned up by UpdateCopyLinkfileList, leaving an empty parent
directory behind.
"""
src_dir = os.path.join(self.worktree, "dot-llms")
os.makedirs(src_dir)
dest = os.path.join(self.topdir, "mydir")
os.makedirs(dest)
lf = self.LinkFile("dot-llms", "mydir")
lf._Link()
self.assertTrue(os.path.islink(dest))
self.assertEqual(
os.path.join("git-project", "dot-llms"), os.readlink(dest)
)
def test_nonempty_dir_not_clobbered(self):
"""A linkfile must not delete a non-empty directory.
If the user created files in a directory that a new linkfile wants
to replace, __linkIt should fail safely rather than deleting content.
"""
src_dir = os.path.join(self.worktree, "dot-llms")
os.makedirs(src_dir)
dest = os.path.join(self.topdir, "mydir")
os.makedirs(dest)
user_file = os.path.join(dest, "user-notes.txt")
self.touch(user_file)
lf = self.LinkFile("dot-llms", "mydir")
lf._Link()
# The directory should NOT be replaced — user content is preserved.
self.assertFalse(os.path.islink(dest))
self.assertTrue(os.path.isdir(dest))
self.assertTrue(os.path.exists(user_file))
class MigrateWorkTreeTests(unittest.TestCase): class MigrateWorkTreeTests(unittest.TestCase):
"""Check _MigrateOldWorkTreeGitDir handling.""" """Check _MigrateOldWorkTreeGitDir handling."""
@@ -565,3 +690,260 @@ class ManifestPropertiesFetchedCorrectly(unittest.TestCase):
fakeproj.config.SetString("manifest.platform", "auto") fakeproj.config.SetString("manifest.platform", "auto")
self.assertEqual(fakeproj.manifest_platform, "auto") self.assertEqual(fakeproj.manifest_platform, "auto")
class StatelessSyncTests(unittest.TestCase):
"""Tests for stateless sync strategy."""
def _get_project(self, tempdir):
manifest = mock.MagicMock()
manifest.manifestProject.depth = None
manifest.manifestProject.dissociate = False
manifest.manifestProject.clone_filter = None
manifest.is_multimanifest = False
manifest.manifestProject.config.GetBoolean.return_value = False
remote = mock.MagicMock()
remote.name = "origin"
remote.url = "http://"
proj = project.Project(
manifest=manifest,
name="test-project",
remote=remote,
gitdir=os.path.join(tempdir, ".git"),
objdir=os.path.join(tempdir, ".git"),
worktree=tempdir,
relpath="test-project",
revisionExpr="1234abcd",
revisionId=None,
sync_strategy="stateless",
)
proj._CheckForImmutableRevision = mock.MagicMock(return_value=False)
proj._LsRemote = mock.MagicMock(
return_value="1234abcd\trefs/heads/main\n"
)
proj.bare_git = mock.MagicMock()
proj.bare_git.rev_parse.return_value = "5678abcd"
proj.bare_git.rev_list.return_value = ["0"]
proj.IsDirty = mock.MagicMock(return_value=False)
proj.GetBranches = mock.MagicMock(return_value=[])
proj.DeleteWorktree = mock.MagicMock()
proj._InitGitDir = mock.MagicMock()
proj._RemoteFetch = mock.MagicMock(return_value=True)
proj._InitRemote = mock.MagicMock()
proj._InitMRef = mock.MagicMock()
return proj
def test_sync_network_half_stateless_prune_needed(self):
"""Test stateless sync queues prune when needed."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
res = proj.Sync_NetworkHalf()
self.assertTrue(res.success)
proj.DeleteWorktree.assert_not_called()
self.assertTrue(proj.stateless_prune_needed)
proj._RemoteFetch.assert_called_once()
def test_sync_local_half_stateless_prune(self):
"""Test stateless GC pruning is queued in Sync_LocalHalf."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.stateless_prune_needed = True
proj._Checkout = mock.MagicMock()
proj._InitWorkTree = mock.MagicMock()
proj.IsRebaseInProgress = mock.MagicMock(return_value=False)
proj.IsCherryPickInProgress = mock.MagicMock(return_value=False)
proj.bare_ref = mock.MagicMock()
proj.bare_ref.all = {}
proj.GetRevisionId = mock.MagicMock(return_value="1234abcd")
proj._CopyAndLinkFiles = mock.MagicMock()
proj.work_git = mock.MagicMock()
proj.work_git.GetHead.return_value = "5678abcd"
syncbuf = project.SyncBuffer(proj.config)
with mock.patch("project.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
proj.Sync_LocalHalf(syncbuf)
syncbuf.Finish()
self.assertEqual(mock_git_cmd.call_count, 2)
mock_git_cmd.assert_any_call(
proj, ["reflog", "expire", "--expire=all", "--all"], bare=True
)
mock_git_cmd.assert_any_call(
proj,
["gc", "--prune=now"],
bare=True,
capture_stdout=True,
capture_stderr=True,
)
def test_sync_local_half_no_upstream_propagates_force_checkout(self):
"""Test Sync_LocalHalf forwards force_checkout when detaching."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj._InitWorkTree = mock.MagicMock()
proj.CleanPublishedCache = mock.MagicMock()
proj.GetRevisionId = mock.MagicMock(return_value="1234abcd")
proj._Checkout = mock.MagicMock()
proj._CopyAndLinkFiles = mock.MagicMock()
proj.work_git = mock.MagicMock()
proj.work_git.GetHead.return_value = "refs/heads/topic"
proj.bare_ref = mock.MagicMock()
proj.bare_ref.all = {"refs/heads/topic": "5678abcd"}
branch = mock.MagicMock()
branch.name = "topic"
branch.LocalMerge = False
proj.GetBranch = mock.MagicMock(return_value=branch)
syncbuf = project.SyncBuffer(proj.config)
proj.Sync_LocalHalf(syncbuf, force_checkout=True)
proj._Checkout.assert_called_once_with(
"1234abcd", force_checkout=True, quiet=True
)
proj._CopyAndLinkFiles.assert_called_once_with()
def test_sync_network_half_stateless_skips_if_stash(self):
"""Test stateless sync skips if stash exists."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.HasStash = mock.MagicMock(return_value=True)
res = proj.Sync_NetworkHalf()
self.assertTrue(res.success)
self.assertFalse(getattr(proj, "stateless_prune_needed", False))
def test_sync_network_half_stateless_skips_if_local_commits(self):
"""Test stateless sync skips if there are local-only commits."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.bare_git.rev_list.return_value = ["1"]
res = proj.Sync_NetworkHalf()
self.assertTrue(res.success)
self.assertFalse(getattr(proj, "stateless_prune_needed", False))
class SyncOptimizationTests(unittest.TestCase):
"""Tests for sync optimization logic involving shallow clones."""
def _get_project(self, tempdir, depth=None):
manifest = mock.MagicMock()
manifest.manifestProject.depth = depth
manifest.manifestProject.dissociate = False
manifest.manifestProject.clone_filter = None
manifest.is_multimanifest = False
manifest.manifestProject.config.GetBoolean.return_value = False
manifest.IsMirror = False
remote = mock.MagicMock()
remote.name = "origin"
remote.url = "http://"
proj = project.Project(
manifest=manifest,
name="test-project",
remote=remote,
gitdir=os.path.join(tempdir, "gitdir"),
objdir=os.path.join(tempdir, "objdir"),
worktree=tempdir,
relpath="test-project",
revisionExpr="0123456789abcdef0123456789abcdef01234567",
revisionId=None,
)
proj._CheckForImmutableRevision = mock.MagicMock(return_value=True)
proj.DeleteWorktree = mock.MagicMock()
proj._InitGitDir = mock.MagicMock()
proj._InitRemote = mock.MagicMock()
proj._InitMRef = mock.MagicMock()
return proj
def test_sync_network_half_shallow_missing_fetches(self):
"""Test Sync_NetworkHalf fetches if shallow file is missing."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
# Ensure gitdir does not exist to simulate new project
if os.path.exists(proj.gitdir):
shutil.rmtree(proj.gitdir)
shallow_path = os.path.join(proj.gitdir, "shallow")
if os.path.exists(shallow_path):
os.unlink(shallow_path)
proj._RemoteFetch = mock.MagicMock(return_value=True)
res = proj.Sync_NetworkHalf(optimized_fetch=True)
self.assertTrue(res.success)
proj._RemoteFetch.assert_called_once()
def test_sync_network_half_shallow_exists_skips(self):
"""Test Sync_NetworkHalf skips fetch if shallow file exists."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
with open(os.path.join(proj.gitdir, "shallow"), "w") as f:
f.write("")
proj._RemoteFetch = mock.MagicMock()
res = proj.Sync_NetworkHalf(optimized_fetch=True)
self.assertTrue(res.success)
proj._RemoteFetch.assert_not_called()
def test_remote_fetch_shallow_missing_fetches(self):
"""Test _RemoteFetch fetches if shallow file is missing."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
shallow_path = os.path.join(proj.gitdir, "shallow")
if os.path.exists(shallow_path):
os.unlink(shallow_path)
with mock.patch("project.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
res = proj._RemoteFetch(
current_branch_only=True,
depth=1,
use_superproject=False,
)
self.assertTrue(res)
mock_git_cmd.assert_called()
def test_remote_fetch_shallow_exists_skips(self):
"""Test _RemoteFetch skips fetch if shallow file exists."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
with open(os.path.join(proj.gitdir, "shallow"), "w") as f:
f.write("")
with mock.patch("project.GitCommand") as mock_git_cmd:
res = proj._RemoteFetch(
current_branch_only=True,
depth=1,
use_superproject=False,
)
self.assertTrue(res)
mock_git_cmd.assert_not_called()
+65 -97
View File
@@ -14,11 +14,9 @@
"""Unittests for the forall subcmd.""" """Unittests for the forall subcmd."""
from io import StringIO import contextlib
import os import io
from shutil import rmtree from pathlib import Path
import tempfile
import unittest
from unittest import mock from unittest import mock
import utils_for_test import utils_for_test
@@ -28,111 +26,81 @@ import project
import subcmds import subcmds
class AllCommands(unittest.TestCase): def _create_manifest_with_8_projects(
"""Check registered all_commands.""" topdir: Path,
) -> manifest_xml.XmlManifest:
"""Create a setup of 8 projects to execute forall."""
repodir = topdir / ".repo"
manifest_dir = repodir / "manifests"
manifest_file = repodir / manifest_xml.MANIFEST_FILE_NAME
def setUp(self): repodir.mkdir()
"""Common setup.""" manifest_dir.mkdir()
self.tempdirobj = tempfile.TemporaryDirectory(prefix="forall_tests")
self.tempdir = self.tempdirobj.name
self.repodir = os.path.join(self.tempdir, ".repo")
self.manifest_dir = os.path.join(self.repodir, "manifests")
self.manifest_file = os.path.join(
self.repodir, manifest_xml.MANIFEST_FILE_NAME
)
self.local_manifest_dir = os.path.join(
self.repodir, manifest_xml.LOCAL_MANIFESTS_DIR_NAME
)
os.mkdir(self.repodir)
os.mkdir(self.manifest_dir)
def tearDown(self): # Set up a manifest git dir for parsing to work.
"""Common teardown.""" gitdir = repodir / "manifests.git"
rmtree(self.tempdir, ignore_errors=True) gitdir.mkdir()
(gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
verbose = false
"""
)
def getXmlManifestWith8Projects(self): # Add the manifest data.
"""Create and return a setup of 8 projects with enough dummy manifest_file.write_text(
files and setup to execute forall.""" """
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="project1" path="tests/path1" />
<project name="project2" path="tests/path2" />
<project name="project3" path="tests/path3" />
<project name="project4" path="tests/path4" />
<project name="project5" path="tests/path5" />
<project name="project6" path="tests/path6" />
<project name="project7" path="tests/path7" />
<project name="project8" path="tests/path8" />
</manifest>
""",
encoding="utf-8",
)
# Set up a manifest git dir for parsing to work # Set up 8 empty projects to match the manifest.
gitdir = os.path.join(self.repodir, "manifests.git") for x in range(1, 9):
os.mkdir(gitdir) (repodir / "projects" / "tests" / f"path{x}.git").mkdir(parents=True)
with open(os.path.join(gitdir, "config"), "w") as fp: (repodir / "project-objects" / f"project{x}.git").mkdir(parents=True)
fp.write( git_path = topdir / "tests" / f"path{x}"
"""[remote "origin"] utils_for_test.init_git_tree(git_path)
url = https://localhost:0/manifest
verbose = false
"""
)
# Add the manifest data return manifest_xml.XmlManifest(str(repodir), str(manifest_file))
manifest_data = """
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="project1" path="tests/path1" />
<project name="project2" path="tests/path2" />
<project name="project3" path="tests/path3" />
<project name="project4" path="tests/path4" />
<project name="project5" path="tests/path5" />
<project name="project6" path="tests/path6" />
<project name="project7" path="tests/path7" />
<project name="project8" path="tests/path8" />
</manifest>
"""
with open(self.manifest_file, "w", encoding="utf-8") as fp:
fp.write(manifest_data)
# Set up 8 empty projects to match the manifest
for x in range(1, 9):
os.makedirs(
os.path.join(
self.repodir, "projects/tests/path" + str(x) + ".git"
)
)
os.makedirs(
os.path.join(
self.repodir, "project-objects/project" + str(x) + ".git"
)
)
git_path = os.path.join(self.tempdir, "tests/path" + str(x))
utils_for_test.init_git_tree(git_path)
return manifest_xml.XmlManifest(self.repodir, self.manifest_file) def test_forall_all_projects_called_once(tmp_path: Path) -> None:
"""Test that all projects get a command run once each."""
manifest = _create_manifest_with_8_projects(tmp_path)
# Use mock to capture stdout from the forall run cmd = subcmds.forall.Forall()
@unittest.mock.patch("sys.stdout", new_callable=StringIO) cmd.manifest = manifest
def test_forall_all_projects_called_once(self, mock_stdout):
"""Test that all projects get a command run once each."""
manifest_with_8_projects = self.getXmlManifestWith8Projects() # Use echo project names as the test of forall.
opts, args = cmd.OptionParser.parse_args(["-c", "echo $REPO_PROJECT"])
opts.verbose = False
cmd = subcmds.forall.Forall() with contextlib.redirect_stdout(io.StringIO()) as stdout:
cmd.manifest = manifest_with_8_projects # Mock to not have the Execute fail on remote check.
# Use echo project names as the test of forall
opts, args = cmd.OptionParser.parse_args(["-c", "echo $REPO_PROJECT"])
opts.verbose = False
# Mock to not have the Execute fail on remote check
with mock.patch.object( with mock.patch.object(
project.Project, "GetRevisionId", return_value="refs/heads/main" project.Project, "GetRevisionId", return_value="refs/heads/main"
): ):
# Run the forall command # Run the forall command.
cmd.Execute(opts, args) cmd.Execute(opts, args)
# Verify that we got every project name in the prints output = stdout.getvalue()
for x in range(1, 9): # Verify that we got every project name in the output.
self.assertIn("project" + str(x), mock_stdout.getvalue()) for x in range(1, 9):
assert f"project{x}" in output
# Split the captured output into lines to count them # Split the captured output into lines to count them.
line_count = 0 line_count = sum(1 for x in output.splitlines() if x)
for line in mock_stdout.getvalue().split("\n"): # Verify that we didn't get more lines than expected.
# A commented out print to stderr as a reminder assert line_count == 8
# that stdout is mocked, include sys and uncomment if needed
# print(line, file=sys.stderr)
if len(line) > 0:
line_count += 1
# Verify that we didn't get more lines than expected
assert line_count == 8
+201
View File
@@ -0,0 +1,201 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the subcmds/info.py module."""
import json
from unittest import mock
import pytest
from subcmds import info
def _get_cmd() -> info.Info:
"""Build a mock-backed Info command for testing."""
manifest = mock.MagicMock()
manifest.default.revisionExpr = "refs/heads/main"
manifest.manifestProject.config.GetBranch.return_value.merge = (
"refs/heads/main"
)
manifest.GetManifestGroupsStr.return_value = "all"
manifest.superproject = None
manifest.outer_client = manifest
client = mock.MagicMock()
git_event_log = mock.MagicMock()
return info.Info(
manifest=manifest,
client=client,
git_event_log=git_event_log,
)
def test_include_options_default_true() -> None:
"""Both include options should default to True."""
opts, _ = _get_cmd().OptionParser.parse_args([])
assert opts.include_summary
assert opts.include_projects
def test_no_include_summary_parses() -> None:
"""--no-include-summary should set include_summary to False."""
opts, _ = _get_cmd().OptionParser.parse_args(["--no-include-summary"])
assert not opts.include_summary
def test_no_include_projects_parses() -> None:
"""--no-include-projects should set include_projects to False."""
opts, _ = _get_cmd().OptionParser.parse_args(["--no-include-projects"])
assert not opts.include_projects
def test_format_default_text() -> None:
"""Default format should be text."""
opts, _ = _get_cmd().OptionParser.parse_args([])
assert opts.format == "text"
def test_format_json_parses() -> None:
"""--format=json should be accepted."""
opts, _ = _get_cmd().OptionParser.parse_args(["--format=json"])
assert opts.format == "json"
def test_no_include_projects_skips_projects() -> None:
"""--no-include-projects should skip project iteration."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--no-include-projects"])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_not_called()
def test_no_include_summary_skips_summary() -> None:
"""--no-include-summary should not query or print manifest metadata."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--no-include-summary"])
with mock.patch.object(cmd, "_printDiffInfo"):
cmd.Execute(opts, args)
cmd.manifest.GetManifestGroupsStr.assert_not_called()
def test_default_calls_diff_info() -> None:
"""Default options should call _printDiffInfo."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args([])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_called_once_with(opts, args)
mock_overview.assert_not_called()
def test_overview_calls_commit_overview() -> None:
"""--overview should call _printCommitOverview, not _printDiffInfo."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--overview"])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_called_once_with(opts, args)
def test_no_include_projects_with_overview() -> None:
"""--no-include-projects should take priority over --overview."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--no-include-projects", "--overview"]
)
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_not_called()
def test_json_summary_only(capsys) -> None:
"""--format=json --no-include-projects should emit only summary."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--format=json", "--no-include-projects"]
)
cmd.Execute(opts, args)
data = json.loads(capsys.readouterr().out)
assert "summary" in data
assert "projects" not in data
assert data["summary"]["manifest_branch"] == "refs/heads/main"
assert data["summary"]["manifest_groups"] == "all"
def test_json_no_summary(capsys) -> None:
"""--format=json --no-include-summary should omit summary."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--format=json", "--no-include-summary", "--no-include-projects"]
)
cmd.Execute(opts, args)
data = json.loads(capsys.readouterr().out)
assert "summary" not in data
def test_json_rejects_diff() -> None:
"""--format=json --diff should be rejected."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--format=json", "--diff"])
with pytest.raises(SystemExit):
cmd.ValidateOptions(opts, args)
def test_json_rejects_overview() -> None:
"""--format=json --overview should be rejected."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--format=json", "--overview"])
with pytest.raises(SystemExit):
cmd.ValidateOptions(opts, args)
def test_json_disables_pager() -> None:
"""--format=json should disable the pager."""
cmd = _get_cmd()
opts, _ = cmd.OptionParser.parse_args(["--format=json"])
assert not cmd.WantPager(opts)
def test_text_enables_pager() -> None:
"""Default text format should enable the pager."""
cmd = _get_cmd()
opts, _ = cmd.OptionParser.parse_args([])
assert cmd.WantPager(opts)
+220
View File
@@ -0,0 +1,220 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the status subcmd."""
import contextlib
import io
import os
from pathlib import Path
import subprocess
from typing import List, Tuple
from unittest import mock
import pytest
import utils_for_test
import manifest_xml
import subcmds
@pytest.fixture
def repo_client_checkout(
tmp_path: Path,
) -> Tuple[Path, manifest_xml.XmlManifest]:
"""Create a basic repo client checkout for status tests."""
# Create in a subdir to avoid noise (like the repo_trace file).
topdir = tmp_path / "client_checkout"
repodir = topdir / ".repo"
manifest_dir = repodir / "manifests"
manifest_file = repodir / manifest_xml.MANIFEST_FILE_NAME
repodir.mkdir(parents=True)
manifest_dir.mkdir()
gitdir = repodir / "manifests.git"
gitdir.mkdir()
(gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
verbose = false
"""
)
_init_temp_git_tree(manifest_dir)
manifest_file.write_text(
"""
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="proj" path="src/proj" />
</manifest>
""",
encoding="utf-8",
)
(repodir / "projects" / "src" / "proj.git").mkdir(parents=True)
(repodir / "project-objects" / "proj.git").mkdir(parents=True)
worktree = topdir / "src" / "proj"
worktree.parent.mkdir(parents=True, exist_ok=True)
_init_temp_git_tree(worktree)
manifest = manifest_xml.XmlManifest(str(repodir), str(manifest_file))
return topdir, manifest
def _init_temp_git_tree(git_dir: Path) -> None:
"""Create a new git checkout with an initial commit for testing."""
utils_for_test.init_git_tree(git_dir)
(git_dir / "README").write_text("init")
subprocess.check_call(["git", "add", "README"], cwd=git_dir)
subprocess.check_call(["git", "commit", "-q", "-m", "init"], cwd=git_dir)
def _run_status(manifest: manifest_xml.XmlManifest, argv: List[str]) -> None:
"""Run the status subcommand with parsed options against a test manifest."""
cmd = subcmds.status.Status()
cmd.manifest = manifest
cmd.client = mock.MagicMock(globalConfig=manifest.globalConfig)
opts, args = cmd.OptionParser.parse_args(argv + ["--jobs=1"])
cmd.CommonValidateOptions(opts, args)
cmd.Execute(opts, args)
def _status_lines(output: str) -> List[str]:
"""Normalize path separators and split command output into lines."""
return output.replace(os.sep, "/").splitlines()
def _assert_project_header(line: str, project_path: str, branch: str) -> None:
"""Assert a status project header line for a project and branch."""
expected = f"project {(project_path + '/ '):<40}branch {branch}"
assert line == expected
def _assert_orphan_block(lines: List[str], expected: List[str]) -> None:
"""Assert orphan block header and entries, independent of entry ordering."""
assert lines
assert lines[0] == ("Objects not within a project (orphans)")
orphan_lines = lines[1:]
assert len(orphan_lines) == len(expected)
assert sorted(orphan_lines) == sorted(expected)
def test_orphans_basic(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify -o output includes project header and orphan block."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / "src" / "orphan_dir").mkdir(parents=True)
(topdir / "orphan.txt").write_text("data")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, ["-o"])
lines = _status_lines(stdout.getvalue())
_assert_project_header(lines[0], project_path, "main")
_assert_orphan_block(
lines[1:],
[
" --\torphan.txt",
" --\tsrc/orphan_dir/",
],
)
def test_empty_status_without_orphans(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify clean status without -o prints only the project header line."""
_, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, "main")
def test_status_without_orphans(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify modified tracked file appears in status output without -o."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / project_path / "README").write_text("updated")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 2
_assert_project_header(lines[0], project_path, "main")
assert lines[1] == " -m\tREADME"
def test_status_with_orphans_and_modified_file(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify modified-file status plus orphan block."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / project_path / "README").write_text("updated")
(topdir / "src" / "orphan_dir").mkdir(parents=True)
(topdir / "orphan.txt").write_text("data")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, ["-o"])
lines = _status_lines(stdout.getvalue())
_assert_project_header(lines[0], project_path, "main")
assert lines[1] == " -m\tREADME"
_assert_orphan_block(
lines[2:],
[
" --\torphan.txt",
" --\tsrc/orphan_dir/",
],
)
def test_empty_status_after_start_shows_started_branch(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify status shows the started branch name when the tree is clean."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
started_branch = "topic/test-status-branch"
subprocess.check_call(
["git", "checkout", "-q", "-b", started_branch], cwd=project_worktree
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, started_branch)
+484
View File
@@ -13,6 +13,7 @@
# limitations under the License. # limitations under the License.
"""Unittests for the subcmds/sync.py module.""" """Unittests for the subcmds/sync.py module."""
import json
import os import os
import shutil import shutil
import tempfile import tempfile
@@ -477,6 +478,181 @@ class GetPreciousObjectsState(unittest.TestCase):
) )
class KeyboardInterruptTest(unittest.TestCase):
"""Tests for KeyboardInterrupt handling in Sync operations."""
def setUp(self):
self.project = mock.MagicMock(name="project")
self.project.name = "project"
self.project.relpath = "proj"
self.project.manifest.IsArchive = False
self.opt = mock.Mock()
self.opt.quiet = True
self.opt.verbose = False
self.opt.tags = False
self.sync_dict = {}
self.get_parallel_context_mock = {
"projects": [self.project],
"sync_dict": self.sync_dict,
"ssh_proxy": None,
}
@mock.patch("subcmds.sync.Sync.is_multiprocessing_active")
def test_fetch_one_keyboard_interrupt_main_process(self, mock_is_active):
"""Test that _FetchOne re-raises KeyboardInterrupt if not worker."""
mock_is_active.return_value = False
self.project.Sync_NetworkHalf.side_effect = KeyboardInterrupt()
with mock.patch.object(
sync.Sync,
"get_parallel_context",
return_value=self.get_parallel_context_mock,
):
with self.assertRaises(KeyboardInterrupt):
sync.Sync._FetchOne(self.opt, 0)
@mock.patch("subcmds.sync.Sync.is_multiprocessing_active")
def test_fetch_one_keyboard_interrupt_worker_process(self, mock_is_active):
"""Test that _FetchOne suppresses KeyboardInterrupt in workers."""
mock_is_active.return_value = True
self.project.Sync_NetworkHalf.side_effect = KeyboardInterrupt()
with mock.patch.object(
sync.Sync,
"get_parallel_context",
return_value=self.get_parallel_context_mock,
):
result = sync.Sync._FetchOne(self.opt, 0)
self.assertFalse(result.success)
class CheckForBloatedProjects(unittest.TestCase):
"""Tests for Sync._CheckForBloatedProjects."""
def setUp(self):
self.cmd = sync.Sync()
self.opt = mock.Mock()
self.opt.quiet = True
self.opt.jobs = 1
self.project = mock.MagicMock(clone_depth="1")
self.project.name = "project"
self.project.Exists = True
self.project.worktree = "worktree"
self.project.stateless_prune_needed = False
self.cmd.git_event_log = mock.MagicMock()
self.cmd._bloated_projects = []
@mock.patch("subcmds.sync.git_require")
def test_git_version_unsupported(self, mock_git_require):
"""Test that it returns early if git version is unsupported."""
mock_git_require.return_value = False
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertFalse(self.cmd.git_event_log.ErrorEvent.called)
@mock.patch("subcmds.sync.git_require")
def test_no_projects(self, mock_git_require):
"""Test that it returns early if no projects have clone_depth."""
mock_git_require.return_value = True
self.project.clone_depth = None
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertFalse(self.cmd.git_event_log.ErrorEvent.called)
@mock.patch("subcmds.sync.git_require")
@mock.patch("subcmds.sync.Progress")
def test_bloated_project_found(self, mock_progress, mock_git_require):
"""Test that it adds project to _bloated_projects."""
mock_git_require.return_value = True
self.cmd.get_parallel_context = mock.Mock(
return_value={"projects": [self.project]}
)
def mock_execute_in_parallel(
jobs, func, work_items, callback, **kwargs
):
callback(None, mock.Mock(), ["project"])
return True
self.cmd.ExecuteInParallel = mock_execute_in_parallel
with mock.patch.object(self.cmd, "ParallelContext"):
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertEqual(self.cmd._bloated_projects, ["project"])
@mock.patch("subcmds.sync.git_require")
@mock.patch("subcmds.sync.Progress")
def test_stateless_prune_excluded(self, mock_progress, mock_git_require):
"""Test that projects pruned for stateless sync are excluded."""
mock_git_require.return_value = True
self.project.stateless_prune_needed = True
self.cmd.ExecuteInParallel = mock.Mock()
with mock.patch.object(self.cmd, "ParallelContext"):
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertFalse(self.cmd.ExecuteInParallel.called)
self.assertEqual(self.cmd._bloated_projects, [])
class GCProjectsTest(unittest.TestCase):
"""Tests for Sync._GCProjects."""
def setUp(self):
self.cmd = sync.Sync()
self.opt = mock.Mock()
self.opt.quiet = True
self.opt.auto_gc = True
self.opt.jobs = 1
self.project = mock.MagicMock()
self.project.name = "project"
self.project.objdir = "objdir"
self.project.gitdir = "gitdir"
self.project.bare_git = mock.MagicMock()
self.project.bare_git._project = self.project
self.cmd.git_event_log = mock.MagicMock()
@mock.patch("subcmds.sync.Progress")
def test_GCProjects_skip_gc(self, mock_progress):
"""Test that it skips GC if opt.auto_gc is False."""
self.opt.auto_gc = False
with mock.patch.object(
sync.Sync, "_SetPreciousObjectsState"
) as mock_set_state:
self.cmd._GCProjects([self.project], self.opt, None)
mock_set_state.assert_called_once_with(self.project, self.opt)
self.assertFalse(self.project.bare_git.gc.called)
@mock.patch("subcmds.sync.Progress")
def test_GCProjects_sequential(self, mock_progress):
"""Test sequential GC (jobs < 2)."""
with mock.patch.object(sync.Sync, "_SetPreciousObjectsState"):
self.cmd._GCProjects([self.project], self.opt, None)
self.project.bare_git.gc.assert_called_once_with(
"--auto", config={"gc.autoDetach": "false"}
)
# Verify that gc.autoDetach was not permanently set in config.
for call in self.project.config.SetString.call_args_list:
self.assertNotEqual(call.args[0], "gc.autoDetach")
@mock.patch("subcmds.sync.Progress")
def test_GCProjects_parallel(self, mock_progress):
"""Test parallel GC (jobs >= 2)."""
self.opt.jobs = 2
with mock.patch.object(sync.Sync, "_SetPreciousObjectsState"):
with mock.patch("subcmds.sync._threading.Thread") as mock_thread:
mock_t = mock.MagicMock()
mock_thread.return_value = mock_t
err_event = mock.Mock()
err_event.is_set.return_value = False
self.cmd._GCProjects([self.project], self.opt, err_event)
self.assertTrue(mock_thread.called)
class SyncCommand(unittest.TestCase): class SyncCommand(unittest.TestCase):
"""Tests for cmd.Execute.""" """Tests for cmd.Execute."""
@@ -970,3 +1146,311 @@ class InterleavedSyncTest(unittest.TestCase):
self.assertTrue(result.checkout_success) self.assertTrue(result.checkout_success)
project.Sync_NetworkHalf.assert_called_once() project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_not_called() project.Sync_LocalHalf.assert_not_called()
class UpdateCopyLinkfileListTest(unittest.TestCase):
"""Tests for Sync.UpdateCopyLinkfileList."""
def setUp(self):
self.tempdirobj = tempfile.TemporaryDirectory(prefix="repo_tests")
self.topdir = self.tempdirobj.name
self.repodir = os.path.join(self.topdir, ".repo")
os.makedirs(self.repodir)
manifest = mock.MagicMock()
manifest.subdir = self.repodir
self.manifest = manifest
git_event_log = mock.MagicMock(ErrorEvent=mock.Mock(return_value=None))
self.cmd = sync.Sync(
manifest=manifest,
outer_client=mock.MagicMock(),
git_event_log=git_event_log,
)
self.cmd.client = mock.MagicMock(topdir=self.topdir)
def tearDown(self):
self.tempdirobj.cleanup()
def _write_copylinkfile_json(self, data: dict) -> None:
path = os.path.join(self.repodir, "copy-link-files.json")
with open(path, "w") as f:
json.dump(data, f)
def _setup_projects(self, linkfile_dests: list) -> None:
project = mock.MagicMock()
project.linkfiles = [mock.MagicMock(dest=d) for d in linkfile_dests]
project.copyfiles = []
mock.patch.object(
self.cmd, "GetProjects", return_value=[project]
).start()
def test_removes_old_symlink_dest(self):
"""Old linkfile dests that are symlinks should be removed."""
old_dest = os.path.join(self.topdir, "old-link")
os.symlink("target", old_dest)
self._write_copylinkfile_json(
{"linkfile": ["old-link"], "copyfile": []}
)
self._setup_projects([])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(old_dest))
def test_does_not_delete_through_new_symlink(self):
"""Old dests that resolve through a new symlink must not delete files.
When the manifest changes from individual linkfiles inside a directory
to a single directory linkfile, and _CopyAndLinkFiles has already
created the symlink (interleaved mode), cleanup must not follow the
symlink and delete real project files.
"""
project_dir = os.path.join(self.topdir, "vendor", "tools", "llms")
os.makedirs(os.path.join(project_dir, "dot-llms", "rules"))
os.makedirs(os.path.join(project_dir, "dot-llms", "skills"))
with open(
os.path.join(project_dir, "dot-llms", "rules", "basics.md"), "w"
) as f:
f.write("# basics")
with open(
os.path.join(project_dir, "dot-llms", "skills", "repo.md"), "w"
) as f:
f.write("# repo")
# Simulate interleaved mode: .llms -> vendor/tools/llms/dot-llms.
llms_link = os.path.join(self.topdir, ".llms")
os.symlink("vendor/tools/llms/dot-llms", llms_link)
self._write_copylinkfile_json(
{"linkfile": [".llms/rules", ".llms/skills"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
# Real project files must still exist.
self.assertTrue(
os.path.exists(
os.path.join(project_dir, "dot-llms", "rules", "basics.md")
)
)
self.assertTrue(
os.path.exists(
os.path.join(project_dir, "dot-llms", "skills", "repo.md")
),
)
self.assertTrue(os.path.islink(llms_link))
def test_cleans_up_empty_parent_dirs(self):
"""After removing old dests, empty parent directories are removed."""
llms_dir = os.path.join(self.topdir, ".llms")
os.makedirs(llms_dir)
os.symlink(
"../vendor/tools/llms/rules", os.path.join(llms_dir, "rules")
)
os.symlink(
"../vendor/tools/llms/skills",
os.path.join(llms_dir, "skills"),
)
self._write_copylinkfile_json(
{"linkfile": [".llms/rules", ".llms/skills"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "rules")))
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "skills")))
# Parent directory should be removed since it's now empty.
self.assertFalse(os.path.exists(llms_dir))
def test_preserves_nonempty_parent_dirs(self):
"""Non-empty parent directories are preserved after old dest removal."""
llms_dir = os.path.join(self.topdir, ".llms")
os.makedirs(llms_dir)
os.symlink(
"../vendor/tools/llms/rules", os.path.join(llms_dir, "rules")
)
with open(os.path.join(llms_dir, "my-notes.txt"), "w") as f:
f.write("user content")
self._write_copylinkfile_json(
{"linkfile": [".llms/rules"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "rules")))
self.assertTrue(os.path.exists(os.path.join(llms_dir, "my-notes.txt")))
self.assertTrue(os.path.isdir(llms_dir))
class SyncToSuperprojectRevTests(unittest.TestCase):
"""Tests for Sync._SyncToSuperprojectRev."""
def setUp(self):
self.repodir = tempfile.mkdtemp(".repo")
self.manifest = mock.MagicMock(repodir=self.repodir)
self.manifest.superproject = mock.MagicMock()
self.manifest.path_prefix = ""
self.mp = mock.MagicMock()
self.cmd = sync.Sync(manifest=self.manifest)
self.cmd.outer_manifest = self.manifest
self.opt = mock.Mock()
self.opt.verbose = False
self.opt.superproject_revision = "deadbeef"
self.opt.mp_update = True
self.errors = []
def tearDown(self):
shutil.rmtree(self.repodir)
@mock.patch("subcmds.sync.GitCommand")
def test_successful_sync(self, mock_git_command):
"""Test successful sync to superproject rev."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 0
mock_git.stdout = "proj branch manifest_commit_hash\n"
mock_git_command.return_value = mock_git
with mock.patch.object(
self.cmd, "_UpdateManifestProject"
) as mock_update:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
mock_superproject.SetRevisionId.assert_called_with("deadbeef")
mock_superproject.Sync.assert_called_once()
mock_git_command.assert_called_once()
self.mp.SetRevision.assert_called_with("manifest_commit_hash")
mock_update.assert_called_once()
self.assertEqual(self.errors, [])
@mock.patch("subcmds.sync.GitCommand")
def test_parse_error(self, mock_git_command):
"""Test error when .supermanifest cannot be parsed."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 0
# Invalid format (not 3 parts)
mock_git.stdout = "invalid_content\n"
mock_git_command.return_value = mock_git
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("could not parse .supermanifest", str(e.exception))
@mock.patch("subcmds.sync.GitCommand")
def test_read_error(self, mock_git_command):
"""Test error when reading .supermanifest fails."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 1
mock_git.stderr = "git error"
mock_git_command.return_value = mock_git
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("failed to read .supermanifest", str(e.exception))
def test_no_superproject(self):
"""Test error when superproject is not defined."""
self.manifest.superproject = None
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("superproject not defined", str(e.exception))
@mock.patch("subcmds.sync.GitCommand")
def test_sync_failure(self, mock_git_command):
"""Test error when superproject sync fails."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=False)
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("failed to sync superproject", str(e.exception))
class UpdateAllManifestProjectsTests(unittest.TestCase):
"""Tests for Sync._UpdateAllManifestProjects."""
def setUp(self):
self.repodir = tempfile.mkdtemp(".repo")
self.manifest = mock.MagicMock(repodir=self.repodir)
self.manifest.superproject = mock.MagicMock()
self.manifest.path_prefix = ""
self.manifest.standalone_manifest_url = None
self.manifest.submanifests = {}
self.mp = mock.MagicMock()
self.mp.manifest = self.manifest
self.mp.standalone_manifest_url = None
self.cmd = sync.Sync(manifest=self.manifest)
self.cmd.outer_manifest = self.manifest
self.opt = mock.Mock()
self.opt.verbose = False
self.opt.superproject_revision = None
self.opt.mp_update = True
self.errors = []
def tearDown(self):
shutil.rmtree(self.repodir)
def test_superproject_revision_outer_manifest(self):
"""Test that _SyncToSuperprojectRev is called for outer manifest."""
self.opt.superproject_revision = "deadbeef"
with mock.patch.object(
self.cmd, "_SyncToSuperprojectRev"
) as mock_sync_to_rev:
self.cmd._UpdateAllManifestProjects(
self.opt, self.mp, "name", self.errors
)
mock_sync_to_rev.assert_called_once_with(
self.opt, self.manifest, self.mp, "name", self.errors
)
def test_superproject_revision_submanifest(self):
"""Test that _SyncToSuperprojectRev is NOT called for submanifest."""
self.opt.superproject_revision = "deadbeef"
submanifest = mock.MagicMock()
submanifest.path_prefix = "sub/"
submanifest.standalone_manifest_url = None
self.mp.manifest = submanifest
with mock.patch.object(
self.cmd, "_SyncToSuperprojectRev"
) as mock_sync_to_rev:
with mock.patch.object(
self.cmd, "_UpdateManifestProject"
) as mock_update_manifest:
self.cmd._UpdateAllManifestProjects(
self.opt, self.mp, "name", self.errors
)
mock_sync_to_rev.assert_not_called()
mock_update_manifest.assert_called_once()
+33 -38
View File
@@ -14,9 +14,10 @@
"""Unittests for the subcmds/upload.py module.""" """Unittests for the subcmds/upload.py module."""
import unittest
from unittest import mock from unittest import mock
import pytest
from error import GitError from error import GitError
from error import UploadError from error import UploadError
from subcmds import upload from subcmds import upload
@@ -26,45 +27,39 @@ class UnexpectedError(Exception):
"""An exception not expected by upload command.""" """An exception not expected by upload command."""
class UploadCommand(unittest.TestCase): # A stub people list (reviewers, cc).
"""Check registered all_commands.""" _STUB_PEOPLE = ([], [])
def setUp(self):
self.cmd = upload.Upload()
self.branch = mock.MagicMock()
self.people = mock.MagicMock()
self.opt, _ = self.cmd.OptionParser.parse_args([])
mock.patch.object(
self.cmd, "_AppendAutoList", return_value=None
).start()
mock.patch.object(self.cmd, "git_event_log").start()
def tearDown(self): @pytest.fixture
mock.patch.stopall() def cmd() -> upload.Upload:
"""Fixture to provide an Upload command instance with mocked methods."""
cmd = upload.Upload()
with mock.patch.object(
cmd, "_AppendAutoList", return_value=None
), mock.patch.object(cmd, "git_event_log"):
yield cmd
def test_UploadAndReport_UploadError(self):
"""Check UploadExitError raised when UploadError encountered."""
side_effect = UploadError("upload error")
with mock.patch.object(
self.cmd, "_UploadBranch", side_effect=side_effect
):
with self.assertRaises(upload.UploadExitError):
self.cmd._UploadAndReport(self.opt, [self.branch], self.people)
def test_UploadAndReport_GitError(self): def test_UploadAndReport_UploadError(cmd: upload.Upload) -> None:
"""Check UploadExitError raised when GitError encountered.""" """Check UploadExitError raised when UploadError encountered."""
side_effect = GitError("some git error") opt, _ = cmd.OptionParser.parse_args([])
with mock.patch.object( with mock.patch.object(cmd, "_UploadBranch", side_effect=UploadError("")):
self.cmd, "_UploadBranch", side_effect=side_effect with pytest.raises(upload.UploadExitError):
): cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
with self.assertRaises(upload.UploadExitError):
self.cmd._UploadAndReport(self.opt, [self.branch], self.people)
def test_UploadAndReport_UnhandledError(self):
"""Check UnexpectedError passed through.""" def test_UploadAndReport_GitError(cmd: upload.Upload) -> None:
side_effect = UnexpectedError("some os error") """Check UploadExitError raised when GitError encountered."""
with mock.patch.object( opt, _ = cmd.OptionParser.parse_args([])
self.cmd, "_UploadBranch", side_effect=side_effect with mock.patch.object(cmd, "_UploadBranch", side_effect=GitError("")):
): with pytest.raises(upload.UploadExitError):
with self.assertRaises(type(side_effect)): cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
self.cmd._UploadAndReport(self.opt, [self.branch], self.people)
def test_UploadAndReport_UnhandledError(cmd: upload.Upload) -> None:
"""Check UnexpectedError passed through."""
opt, _ = cmd.OptionParser.parse_args([])
with mock.patch.object(cmd, "_UploadBranch", side_effect=UnexpectedError):
with pytest.raises(UnexpectedError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
-23
View File
@@ -1,23 +0,0 @@
# Copyright (C) 2022 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the update_manpages module."""
from release import update_manpages
def test_replace_regex() -> None:
"""Check that replace_regex works."""
data = "\n\033[1mSummary\033[m\n"
assert update_manpages.replace_regex(data) == "\nSummary\n"
+343 -302
View File
@@ -19,267 +19,303 @@ import os
import re import re
import subprocess import subprocess
import sys import sys
import tempfile
import unittest
from unittest import mock from unittest import mock
import pytest
import utils_for_test import utils_for_test
import main import main
import wrapper import wrapper
def fixture(*paths): @pytest.fixture(autouse=True)
"""Return a path relative to tests/fixtures.""" def reset_wrapper() -> None:
return os.path.join(os.path.dirname(__file__), "fixtures", *paths) """Reset the wrapper module every time."""
wrapper.Wrapper.cache_clear()
class RepoWrapperTestCase(unittest.TestCase): @pytest.fixture
"""TestCase for the wrapper module.""" def repo_wrapper() -> wrapper.Wrapper:
"""Fixture for the wrapper module."""
def setUp(self): return wrapper.Wrapper()
"""Load the wrapper module every time."""
wrapper.Wrapper.cache_clear()
self.wrapper = wrapper.Wrapper()
class RepoWrapperUnitTest(RepoWrapperTestCase): class GitCheckout:
"""Class to hold git checkout info for tests."""
def __init__(self, git_dir, rev_list):
self.git_dir = git_dir
self.rev_list = rev_list
@pytest.fixture(scope="module")
def git_checkout(tmp_path_factory) -> GitCheckout:
"""Fixture for tests that use a real/small git checkout.
Create a repo to operate on, but do it once per-test-run.
"""
tempdir = tmp_path_factory.mktemp("repo-rev-tests")
run_git = wrapper.Wrapper().run_git
remote = os.path.join(tempdir, "remote")
os.mkdir(remote)
utils_for_test.init_git_tree(remote)
run_git("commit", "--allow-empty", "-minit", cwd=remote)
run_git("branch", "stable", cwd=remote)
run_git("tag", "v1.0", cwd=remote)
run_git("commit", "--allow-empty", "-m2nd commit", cwd=remote)
rev_list = run_git("rev-list", "HEAD", cwd=remote).stdout.splitlines()
run_git("init", cwd=tempdir)
run_git(
"fetch",
remote,
"+refs/heads/*:refs/remotes/origin/*",
cwd=tempdir,
)
yield GitCheckout(tempdir, rev_list)
class TestRepoWrapper:
"""Tests helper functions in the repo wrapper""" """Tests helper functions in the repo wrapper"""
def test_version(self): def test_version(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Make sure _Version works.""" """Make sure _Version works."""
with self.assertRaises(SystemExit) as e: with pytest.raises(SystemExit) as e:
with mock.patch("sys.stdout", new_callable=io.StringIO) as stdout: with mock.patch("sys.stdout", new_callable=io.StringIO) as stdout:
with mock.patch( with mock.patch(
"sys.stderr", new_callable=io.StringIO "sys.stderr", new_callable=io.StringIO
) as stderr: ) as stderr:
self.wrapper._Version() repo_wrapper._Version()
self.assertEqual(0, e.exception.code) assert e.value.code == 0
self.assertEqual("", stderr.getvalue()) assert stderr.getvalue() == ""
self.assertIn("repo launcher version", stdout.getvalue()) assert "repo launcher version" in stdout.getvalue()
def test_python_constraints(self): def test_python_constraints(self, repo_wrapper: wrapper.Wrapper) -> None:
"""The launcher should never require newer than main.py.""" """The launcher should never require newer than main.py."""
self.assertGreaterEqual( assert (
main.MIN_PYTHON_VERSION_HARD, self.wrapper.MIN_PYTHON_VERSION_HARD main.MIN_PYTHON_VERSION_HARD >= repo_wrapper.MIN_PYTHON_VERSION_HARD
) )
self.assertGreaterEqual( assert (
main.MIN_PYTHON_VERSION_SOFT, self.wrapper.MIN_PYTHON_VERSION_SOFT main.MIN_PYTHON_VERSION_SOFT >= repo_wrapper.MIN_PYTHON_VERSION_SOFT
) )
# Make sure the versions are themselves in sync. # Make sure the versions are themselves in sync.
self.assertGreaterEqual( assert (
self.wrapper.MIN_PYTHON_VERSION_SOFT, repo_wrapper.MIN_PYTHON_VERSION_SOFT
self.wrapper.MIN_PYTHON_VERSION_HARD, >= repo_wrapper.MIN_PYTHON_VERSION_HARD
) )
def test_init_parser(self): def test_repo_script_is_executable(self) -> None:
"""The repo launcher script should be executable."""
repo_path = utils_for_test.THIS_DIR.parent / "repo"
assert os.access(repo_path, os.X_OK), f"{repo_path} is not executable"
def test_init_parser(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Make sure 'init' GetParser works.""" """Make sure 'init' GetParser works."""
parser = self.wrapper.GetParser() parser = repo_wrapper.GetParser()
opts, args = parser.parse_args([]) opts, args = parser.parse_args([])
self.assertEqual([], args) assert args == []
self.assertIsNone(opts.manifest_url) assert opts.manifest_url is None
class SetGitTrace2ParentSid(RepoWrapperTestCase): class TestSetGitTrace2ParentSid:
"""Check SetGitTrace2ParentSid behavior.""" """Check SetGitTrace2ParentSid behavior."""
KEY = "GIT_TRACE2_PARENT_SID" KEY = "GIT_TRACE2_PARENT_SID"
VALID_FORMAT = re.compile(r"^repo-[0-9]{8}T[0-9]{6}Z-P[0-9a-f]{8}$") VALID_FORMAT = re.compile(r"^repo-[0-9]{8}T[0-9]{6}Z-P[0-9a-f]{8}$")
def test_first_set(self): def test_first_set(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Test env var not yet set.""" """Test env var not yet set."""
env = {} env = {}
self.wrapper.SetGitTrace2ParentSid(env) repo_wrapper.SetGitTrace2ParentSid(env)
self.assertIn(self.KEY, env) assert self.KEY in env
value = env[self.KEY] value = env[self.KEY]
self.assertRegex(value, self.VALID_FORMAT) assert self.VALID_FORMAT.match(value)
def test_append(self): def test_append(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Test env var is appended.""" """Test env var is appended."""
env = {self.KEY: "pfx"} env = {self.KEY: "pfx"}
self.wrapper.SetGitTrace2ParentSid(env) repo_wrapper.SetGitTrace2ParentSid(env)
self.assertIn(self.KEY, env) assert self.KEY in env
value = env[self.KEY] value = env[self.KEY]
self.assertTrue(value.startswith("pfx/")) assert value.startswith("pfx/")
self.assertRegex(value[4:], self.VALID_FORMAT) assert self.VALID_FORMAT.match(value[4:])
def test_global_context(self): def test_global_context(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check os.environ gets updated by default.""" """Check os.environ gets updated by default."""
os.environ.pop(self.KEY, None) os.environ.pop(self.KEY, None)
self.wrapper.SetGitTrace2ParentSid() repo_wrapper.SetGitTrace2ParentSid()
self.assertIn(self.KEY, os.environ) assert self.KEY in os.environ
value = os.environ[self.KEY] value = os.environ[self.KEY]
self.assertRegex(value, self.VALID_FORMAT) assert self.VALID_FORMAT.match(value)
class RunCommand(RepoWrapperTestCase): class TestRunCommand:
"""Check run_command behavior.""" """Check run_command behavior."""
def test_capture(self): def test_capture(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check capture_output handling.""" """Check capture_output handling."""
ret = self.wrapper.run_command(["echo", "hi"], capture_output=True) ret = repo_wrapper.run_command(["echo", "hi"], capture_output=True)
# echo command appends OS specific linesep, but on Windows + Git Bash # echo command appends OS specific linesep, but on Windows + Git Bash
# we get UNIX ending, so we allow both. # we get UNIX ending, so we allow both.
self.assertIn(ret.stdout, ["hi" + os.linesep, "hi\n"]) assert ret.stdout in ["hi" + os.linesep, "hi\n"]
def test_check(self): def test_check(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check check handling.""" """Check check handling."""
self.wrapper.run_command(["true"], check=False) repo_wrapper.run_command(["true"], check=False)
self.wrapper.run_command(["true"], check=True) repo_wrapper.run_command(["true"], check=True)
self.wrapper.run_command(["false"], check=False) repo_wrapper.run_command(["false"], check=False)
with self.assertRaises(subprocess.CalledProcessError): with pytest.raises(subprocess.CalledProcessError):
self.wrapper.run_command(["false"], check=True) repo_wrapper.run_command(["false"], check=True)
class RunGit(RepoWrapperTestCase): class TestRunGit:
"""Check run_git behavior.""" """Check run_git behavior."""
def test_capture(self): def test_capture(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check capture_output handling.""" """Check capture_output handling."""
ret = self.wrapper.run_git("--version") ret = repo_wrapper.run_git("--version")
self.assertIn("git", ret.stdout) assert "git" in ret.stdout
def test_check(self): def test_check(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check check handling.""" """Check check handling."""
with self.assertRaises(self.wrapper.CloneFailure): with pytest.raises(repo_wrapper.CloneFailure):
self.wrapper.run_git("--version-asdfasdf") repo_wrapper.run_git("--version-asdfasdf")
self.wrapper.run_git("--version-asdfasdf", check=False) repo_wrapper.run_git("--version-asdfasdf", check=False)
class ParseGitVersion(RepoWrapperTestCase): class TestParseGitVersion:
"""Check ParseGitVersion behavior.""" """Check ParseGitVersion behavior."""
def test_autoload(self): def test_autoload(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check we can load the version from the live git.""" """Check we can load the version from the live git."""
ret = self.wrapper.ParseGitVersion() assert repo_wrapper.ParseGitVersion() is not None
self.assertIsNotNone(ret)
def test_bad_ver(self): def test_bad_ver(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check handling of bad git versions.""" """Check handling of bad git versions."""
ret = self.wrapper.ParseGitVersion(ver_str="asdf") assert repo_wrapper.ParseGitVersion(ver_str="asdf") is None
self.assertIsNone(ret)
def test_normal_ver(self): def test_normal_ver(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check handling of normal git versions.""" """Check handling of normal git versions."""
ret = self.wrapper.ParseGitVersion(ver_str="git version 2.25.1") ret = repo_wrapper.ParseGitVersion(ver_str="git version 2.25.1")
self.assertEqual(2, ret.major) assert ret.major == 2
self.assertEqual(25, ret.minor) assert ret.minor == 25
self.assertEqual(1, ret.micro) assert ret.micro == 1
self.assertEqual("2.25.1", ret.full) assert ret.full == "2.25.1"
def test_extended_ver(self): def test_extended_ver(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check handling of extended distro git versions.""" """Check handling of extended distro git versions."""
ret = self.wrapper.ParseGitVersion( ret = repo_wrapper.ParseGitVersion(
ver_str="git version 1.30.50.696.g5e7596f4ac-goog" ver_str="git version 1.30.50.696.g5e7596f4ac-goog"
) )
self.assertEqual(1, ret.major) assert ret.major == 1
self.assertEqual(30, ret.minor) assert ret.minor == 30
self.assertEqual(50, ret.micro) assert ret.micro == 50
self.assertEqual("1.30.50.696.g5e7596f4ac-goog", ret.full) assert ret.full == "1.30.50.696.g5e7596f4ac-goog"
class CheckGitVersion(RepoWrapperTestCase): class TestCheckGitVersion:
"""Check _CheckGitVersion behavior.""" """Check _CheckGitVersion behavior."""
def test_unknown(self): def test_unknown(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Unknown versions should abort.""" """Unknown versions should abort."""
with mock.patch.object( with mock.patch.object(
self.wrapper, "ParseGitVersion", return_value=None repo_wrapper, "ParseGitVersion", return_value=None
): ):
with self.assertRaises(self.wrapper.CloneFailure): with pytest.raises(repo_wrapper.CloneFailure):
self.wrapper._CheckGitVersion() repo_wrapper._CheckGitVersion()
def test_old(self): def test_old(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Old versions should abort.""" """Old versions should abort."""
with mock.patch.object( with mock.patch.object(
self.wrapper, repo_wrapper,
"ParseGitVersion", "ParseGitVersion",
return_value=self.wrapper.GitVersion(1, 0, 0, "1.0.0"), return_value=repo_wrapper.GitVersion(1, 0, 0, "1.0.0"),
): ):
with self.assertRaises(self.wrapper.CloneFailure): with pytest.raises(repo_wrapper.CloneFailure):
self.wrapper._CheckGitVersion() repo_wrapper._CheckGitVersion()
def test_new(self): def test_new(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Newer versions should run fine.""" """Newer versions should run fine."""
with mock.patch.object( with mock.patch.object(
self.wrapper, repo_wrapper,
"ParseGitVersion", "ParseGitVersion",
return_value=self.wrapper.GitVersion(100, 0, 0, "100.0.0"), return_value=repo_wrapper.GitVersion(100, 0, 0, "100.0.0"),
): ):
self.wrapper._CheckGitVersion() repo_wrapper._CheckGitVersion()
class Requirements(RepoWrapperTestCase): class TestRequirements:
"""Check Requirements handling.""" """Check Requirements handling."""
def test_missing_file(self): def test_missing_file(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Don't crash if the file is missing (old version).""" """Don't crash if the file is missing (old version)."""
testdir = os.path.dirname(os.path.realpath(__file__)) assert (
self.assertIsNone(self.wrapper.Requirements.from_dir(testdir)) repo_wrapper.Requirements.from_dir(utils_for_test.THIS_DIR) is None
self.assertIsNone( )
self.wrapper.Requirements.from_file( assert (
os.path.join(testdir, "xxxxxxxxxxxxxxxxxxxxxxxx") repo_wrapper.Requirements.from_file(
utils_for_test.THIS_DIR / "xxxxxxxxxxxxxxxxxxxxxxxx"
) )
is None
) )
def test_corrupt_data(self): def test_corrupt_data(self, repo_wrapper: wrapper.Wrapper) -> None:
"""If the file can't be parsed, don't blow up.""" """If the file can't be parsed, don't blow up."""
self.assertIsNone(self.wrapper.Requirements.from_file(__file__)) assert repo_wrapper.Requirements.from_file(__file__) is None
self.assertIsNone(self.wrapper.Requirements.from_data(b"x")) assert repo_wrapper.Requirements.from_data(b"x") is None
def test_valid_data(self): def test_valid_data(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Make sure we can parse the file we ship.""" """Make sure we can parse the file we ship."""
self.assertIsNotNone(self.wrapper.Requirements.from_data(b"{}")) assert repo_wrapper.Requirements.from_data(b"{}") is not None
rootdir = os.path.dirname(os.path.dirname(os.path.realpath(__file__))) rootdir = utils_for_test.THIS_DIR.parent
self.assertIsNotNone(self.wrapper.Requirements.from_dir(rootdir)) assert repo_wrapper.Requirements.from_dir(rootdir) is not None
self.assertIsNotNone( assert (
self.wrapper.Requirements.from_file( repo_wrapper.Requirements.from_file(rootdir / "requirements.json")
os.path.join(rootdir, "requirements.json") is not None
)
) )
def test_format_ver(self): def test_format_ver(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check format_ver can format.""" """Check format_ver can format."""
self.assertEqual( assert repo_wrapper.Requirements._format_ver((1, 2, 3)) == "1.2.3"
"1.2.3", self.wrapper.Requirements._format_ver((1, 2, 3)) assert repo_wrapper.Requirements._format_ver([1]) == "1"
)
self.assertEqual("1", self.wrapper.Requirements._format_ver([1]))
def test_assert_all_unknown(self): def test_assert_all_unknown(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all works with incompatible file.""" """Check assert_all works with incompatible file."""
reqs = self.wrapper.Requirements({}) reqs = repo_wrapper.Requirements({})
reqs.assert_all() reqs.assert_all()
def test_assert_all_new_repo(self): def test_assert_all_new_repo(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all accepts new enough repo.""" """Check assert_all accepts new enough repo."""
reqs = self.wrapper.Requirements({"repo": {"hard": [1, 0]}}) reqs = repo_wrapper.Requirements({"repo": {"hard": [1, 0]}})
reqs.assert_all() reqs.assert_all()
def test_assert_all_old_repo(self): def test_assert_all_old_repo(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all rejects old repo.""" """Check assert_all rejects old repo."""
reqs = self.wrapper.Requirements({"repo": {"hard": [99999, 0]}}) reqs = repo_wrapper.Requirements({"repo": {"hard": [99999, 0]}})
with self.assertRaises(SystemExit): with pytest.raises(SystemExit):
reqs.assert_all() reqs.assert_all()
def test_assert_all_new_python(self): def test_assert_all_new_python(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all accepts new enough python.""" """Check assert_all accepts new enough python."""
reqs = self.wrapper.Requirements({"python": {"hard": sys.version_info}}) reqs = repo_wrapper.Requirements({"python": {"hard": sys.version_info}})
reqs.assert_all() reqs.assert_all()
def test_assert_all_old_python(self): def test_assert_all_old_python(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all rejects old python.""" """Check assert_all rejects old python."""
reqs = self.wrapper.Requirements({"python": {"hard": [99999, 0]}}) reqs = repo_wrapper.Requirements({"python": {"hard": [99999, 0]}})
with self.assertRaises(SystemExit): with pytest.raises(SystemExit):
reqs.assert_all() reqs.assert_all()
def test_assert_ver_unknown(self): def test_assert_ver_unknown(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_ver works with incompatible file.""" """Check assert_ver works with incompatible file."""
reqs = self.wrapper.Requirements({}) reqs = repo_wrapper.Requirements({})
reqs.assert_ver("xxx", (1, 0)) reqs.assert_ver("xxx", (1, 0))
def test_assert_ver_new(self): def test_assert_ver_new(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_ver allows new enough versions.""" """Check assert_ver allows new enough versions."""
reqs = self.wrapper.Requirements( reqs = repo_wrapper.Requirements(
{"git": {"hard": [1, 0], "soft": [2, 0]}} {"git": {"hard": [1, 0], "soft": [2, 0]}}
) )
reqs.assert_ver("git", (1, 0)) reqs.assert_ver("git", (1, 0))
@@ -287,274 +323,279 @@ class Requirements(RepoWrapperTestCase):
reqs.assert_ver("git", (2, 0)) reqs.assert_ver("git", (2, 0))
reqs.assert_ver("git", (2, 5)) reqs.assert_ver("git", (2, 5))
def test_assert_ver_old(self): def test_assert_ver_old(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_ver rejects old versions.""" """Check assert_ver rejects old versions."""
reqs = self.wrapper.Requirements( reqs = repo_wrapper.Requirements(
{"git": {"hard": [1, 0], "soft": [2, 0]}} {"git": {"hard": [1, 0], "soft": [2, 0]}}
) )
with self.assertRaises(SystemExit): with pytest.raises(SystemExit):
reqs.assert_ver("git", (0, 5)) reqs.assert_ver("git", (0, 5))
class NeedSetupGnuPG(RepoWrapperTestCase): class TestNeedSetupGnuPG:
"""Check NeedSetupGnuPG behavior.""" """Check NeedSetupGnuPG behavior."""
def test_missing_dir(self): def test_missing_dir(self, tmp_path, repo_wrapper: wrapper.Wrapper) -> None:
"""The ~/.repoconfig tree doesn't exist yet.""" """The ~/.repoconfig tree doesn't exist yet."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir: repo_wrapper.home_dot_repo = str(tmp_path / "foo")
self.wrapper.home_dot_repo = os.path.join(tempdir, "foo") assert repo_wrapper.NeedSetupGnuPG()
self.assertTrue(self.wrapper.NeedSetupGnuPG())
def test_missing_keyring(self): def test_missing_keyring(
self, tmp_path, repo_wrapper: wrapper.Wrapper
) -> None:
"""The keyring-version file doesn't exist yet.""" """The keyring-version file doesn't exist yet."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir: repo_wrapper.home_dot_repo = str(tmp_path)
self.wrapper.home_dot_repo = tempdir assert repo_wrapper.NeedSetupGnuPG()
self.assertTrue(self.wrapper.NeedSetupGnuPG())
def test_empty_keyring(self): def test_empty_keyring(
self, tmp_path, repo_wrapper: wrapper.Wrapper
) -> None:
"""The keyring-version file exists, but is empty.""" """The keyring-version file exists, but is empty."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir: repo_wrapper.home_dot_repo = str(tmp_path)
self.wrapper.home_dot_repo = tempdir (tmp_path / "keyring-version").write_text("")
with open(os.path.join(tempdir, "keyring-version"), "w"): assert repo_wrapper.NeedSetupGnuPG()
pass
self.assertTrue(self.wrapper.NeedSetupGnuPG())
def test_old_keyring(self): def test_old_keyring(self, tmp_path, repo_wrapper: wrapper.Wrapper) -> None:
"""The keyring-version file exists, but it's old.""" """The keyring-version file exists, but it's old."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir: repo_wrapper.home_dot_repo = str(tmp_path)
self.wrapper.home_dot_repo = tempdir (tmp_path / "keyring-version").write_text("1.0\n")
with open(os.path.join(tempdir, "keyring-version"), "w") as fp: assert repo_wrapper.NeedSetupGnuPG()
fp.write("1.0\n")
self.assertTrue(self.wrapper.NeedSetupGnuPG())
def test_new_keyring(self): def test_new_keyring(self, tmp_path, repo_wrapper: wrapper.Wrapper) -> None:
"""The keyring-version file exists, and is up-to-date.""" """The keyring-version file exists, and is up-to-date."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir: repo_wrapper.home_dot_repo = str(tmp_path)
self.wrapper.home_dot_repo = tempdir (tmp_path / "keyring-version").write_text("1000.0\n")
with open(os.path.join(tempdir, "keyring-version"), "w") as fp: assert not repo_wrapper.NeedSetupGnuPG()
fp.write("1000.0\n")
self.assertFalse(self.wrapper.NeedSetupGnuPG())
class SetupGnuPG(RepoWrapperTestCase): class TestSetupGnuPG:
"""Check SetupGnuPG behavior.""" """Check SetupGnuPG behavior."""
def test_full(self): def test_full(self, tmp_path, repo_wrapper: wrapper.Wrapper) -> None:
"""Make sure it works completely.""" """Make sure it works completely."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir: repo_wrapper.home_dot_repo = str(tmp_path)
self.wrapper.home_dot_repo = tempdir repo_wrapper.gpg_dir = str(tmp_path / "gnupg")
self.wrapper.gpg_dir = os.path.join( assert repo_wrapper.SetupGnuPG(True)
self.wrapper.home_dot_repo, "gnupg" data = (tmp_path / "keyring-version").read_text()
) assert (
self.assertTrue(self.wrapper.SetupGnuPG(True)) ".".join(str(x) for x in repo_wrapper.KEYRING_VERSION)
with open(os.path.join(tempdir, "keyring-version")) as fp: == data.strip()
data = fp.read() )
self.assertEqual(
".".join(str(x) for x in self.wrapper.KEYRING_VERSION),
data.strip(),
)
class VerifyRev(RepoWrapperTestCase): class TestVerifyRev:
"""Check verify_rev behavior.""" """Check verify_rev behavior."""
def test_verify_passes(self): def test_verify_passes(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check when we have a valid signed tag.""" """Check when we have a valid signed tag."""
desc_result = subprocess.CompletedProcess([], 0, "v1.0\n", "") desc_result = subprocess.CompletedProcess([], 0, "v1.0\n", "")
gpg_result = subprocess.CompletedProcess([], 0, "", "") gpg_result = subprocess.CompletedProcess([], 0, "", "")
with mock.patch.object( with mock.patch.object(
self.wrapper, "run_git", side_effect=(desc_result, gpg_result) repo_wrapper, "run_git", side_effect=(desc_result, gpg_result)
): ):
ret = self.wrapper.verify_rev( ret = repo_wrapper.verify_rev(
"/", "refs/heads/stable", "1234", True "/", "refs/heads/stable", "1234", True
) )
self.assertEqual("v1.0^0", ret) assert ret == "v1.0^0"
def test_unsigned_commit(self): def test_unsigned_commit(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check we fall back to signed tag when we have an unsigned commit.""" """Check we fall back to signed tag when we have an unsigned commit."""
desc_result = subprocess.CompletedProcess([], 0, "v1.0-10-g1234\n", "") desc_result = subprocess.CompletedProcess([], 0, "v1.0-10-g1234\n", "")
gpg_result = subprocess.CompletedProcess([], 0, "", "") gpg_result = subprocess.CompletedProcess([], 0, "", "")
with mock.patch.object( with mock.patch.object(
self.wrapper, "run_git", side_effect=(desc_result, gpg_result) repo_wrapper, "run_git", side_effect=(desc_result, gpg_result)
): ):
ret = self.wrapper.verify_rev( ret = repo_wrapper.verify_rev(
"/", "refs/heads/stable", "1234", True "/", "refs/heads/stable", "1234", True
) )
self.assertEqual("v1.0^0", ret) assert ret == "v1.0^0"
def test_verify_fails(self): def test_verify_fails(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check we fall back to signed tag when we have an unsigned commit.""" """Check we fall back to signed tag when we have an unsigned commit."""
desc_result = subprocess.CompletedProcess([], 0, "v1.0-10-g1234\n", "") desc_result = subprocess.CompletedProcess([], 0, "v1.0-10-g1234\n", "")
gpg_result = Exception gpg_result = RuntimeError
with mock.patch.object( with mock.patch.object(
self.wrapper, "run_git", side_effect=(desc_result, gpg_result) repo_wrapper, "run_git", side_effect=(desc_result, gpg_result)
): ):
with self.assertRaises(Exception): with pytest.raises(RuntimeError):
self.wrapper.verify_rev("/", "refs/heads/stable", "1234", True) repo_wrapper.verify_rev("/", "refs/heads/stable", "1234", True)
class GitCheckoutTestCase(RepoWrapperTestCase): class TestResolveRepoRev:
"""Tests that use a real/small git checkout."""
GIT_DIR = None
REV_LIST = None
@classmethod
def setUpClass(cls):
# Create a repo to operate on, but do it once per-class.
cls.tempdirobj = tempfile.TemporaryDirectory(prefix="repo-rev-tests")
cls.GIT_DIR = cls.tempdirobj.name
run_git = wrapper.Wrapper().run_git
remote = os.path.join(cls.GIT_DIR, "remote")
os.mkdir(remote)
utils_for_test.init_git_tree(remote)
run_git("commit", "--allow-empty", "-minit", cwd=remote)
run_git("branch", "stable", cwd=remote)
run_git("tag", "v1.0", cwd=remote)
run_git("commit", "--allow-empty", "-m2nd commit", cwd=remote)
cls.REV_LIST = run_git(
"rev-list", "HEAD", cwd=remote
).stdout.splitlines()
run_git("init", cwd=cls.GIT_DIR)
run_git(
"fetch",
remote,
"+refs/heads/*:refs/remotes/origin/*",
cwd=cls.GIT_DIR,
)
@classmethod
def tearDownClass(cls):
if not cls.tempdirobj:
return
cls.tempdirobj.cleanup()
class ResolveRepoRev(GitCheckoutTestCase):
"""Check resolve_repo_rev behavior.""" """Check resolve_repo_rev behavior."""
def test_explicit_branch(self): def test_explicit_branch(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check refs/heads/branch argument.""" """Check refs/heads/branch argument."""
rrev, lrev = self.wrapper.resolve_repo_rev( rrev, lrev = repo_wrapper.resolve_repo_rev(
self.GIT_DIR, "refs/heads/stable" git_checkout.git_dir, "refs/heads/stable"
) )
self.assertEqual("refs/heads/stable", rrev) assert rrev == "refs/heads/stable"
self.assertEqual(self.REV_LIST[1], lrev) assert lrev == git_checkout.rev_list[1]
with self.assertRaises(self.wrapper.CloneFailure): with pytest.raises(repo_wrapper.CloneFailure):
self.wrapper.resolve_repo_rev(self.GIT_DIR, "refs/heads/unknown") repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, "refs/heads/unknown"
)
def test_explicit_tag(self): def test_explicit_tag(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check refs/tags/tag argument.""" """Check refs/tags/tag argument."""
rrev, lrev = self.wrapper.resolve_repo_rev( rrev, lrev = repo_wrapper.resolve_repo_rev(
self.GIT_DIR, "refs/tags/v1.0" git_checkout.git_dir, "refs/tags/v1.0"
) )
self.assertEqual("refs/tags/v1.0", rrev) assert rrev == "refs/tags/v1.0"
self.assertEqual(self.REV_LIST[1], lrev) assert lrev == git_checkout.rev_list[1]
with self.assertRaises(self.wrapper.CloneFailure): with pytest.raises(repo_wrapper.CloneFailure):
self.wrapper.resolve_repo_rev(self.GIT_DIR, "refs/tags/unknown") repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, "refs/tags/unknown"
)
def test_branch_name(self): def test_branch_name(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check branch argument.""" """Check branch argument."""
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, "stable") rrev, lrev = repo_wrapper.resolve_repo_rev(
self.assertEqual("refs/heads/stable", rrev) git_checkout.git_dir, "stable"
self.assertEqual(self.REV_LIST[1], lrev) )
assert rrev == "refs/heads/stable"
assert lrev == git_checkout.rev_list[1]
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, "main") rrev, lrev = repo_wrapper.resolve_repo_rev(git_checkout.git_dir, "main")
self.assertEqual("refs/heads/main", rrev) assert rrev == "refs/heads/main"
self.assertEqual(self.REV_LIST[0], lrev) assert lrev == git_checkout.rev_list[0]
def test_tag_name(self): def test_tag_name(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check tag argument.""" """Check tag argument."""
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, "v1.0") rrev, lrev = repo_wrapper.resolve_repo_rev(git_checkout.git_dir, "v1.0")
self.assertEqual("refs/tags/v1.0", rrev) assert rrev == "refs/tags/v1.0"
self.assertEqual(self.REV_LIST[1], lrev) assert lrev == git_checkout.rev_list[1]
def test_full_commit(self): def test_full_commit(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check specific commit argument.""" """Check specific commit argument."""
commit = self.REV_LIST[0] commit = git_checkout.rev_list[0]
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, commit) rrev, lrev = repo_wrapper.resolve_repo_rev(git_checkout.git_dir, commit)
self.assertEqual(commit, rrev) assert rrev == commit
self.assertEqual(commit, lrev) assert lrev == commit
def test_partial_commit(self): def test_partial_commit(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check specific (partial) commit argument.""" """Check specific (partial) commit argument."""
commit = self.REV_LIST[0][0:20] commit = git_checkout.rev_list[0][0:20]
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, commit) rrev, lrev = repo_wrapper.resolve_repo_rev(git_checkout.git_dir, commit)
self.assertEqual(self.REV_LIST[0], rrev) assert rrev == git_checkout.rev_list[0]
self.assertEqual(self.REV_LIST[0], lrev) assert lrev == git_checkout.rev_list[0]
def test_unknown(self): def test_unknown(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check unknown ref/commit argument.""" """Check unknown ref/commit argument."""
with self.assertRaises(self.wrapper.CloneFailure): with pytest.raises(repo_wrapper.CloneFailure):
self.wrapper.resolve_repo_rev(self.GIT_DIR, "boooooooya") repo_wrapper.resolve_repo_rev(git_checkout.git_dir, "boooooooya")
class CheckRepoVerify(RepoWrapperTestCase): class TestCheckRepoVerify:
"""Check check_repo_verify behavior.""" """Check check_repo_verify behavior."""
def test_no_verify(self): def test_no_verify(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Always fail with --no-repo-verify.""" """Always fail with --no-repo-verify."""
self.assertFalse(self.wrapper.check_repo_verify(False)) assert not repo_wrapper.check_repo_verify(False)
def test_gpg_initialized(self): def test_gpg_initialized(
self,
repo_wrapper: wrapper.Wrapper,
) -> None:
"""Should pass if gpg is setup already.""" """Should pass if gpg is setup already."""
with mock.patch.object( with mock.patch.object(
self.wrapper, "NeedSetupGnuPG", return_value=False repo_wrapper, "NeedSetupGnuPG", return_value=False
): ):
self.assertTrue(self.wrapper.check_repo_verify(True)) assert repo_wrapper.check_repo_verify(True)
def test_need_gpg_setup(self): def test_need_gpg_setup(
self,
repo_wrapper: wrapper.Wrapper,
) -> None:
"""Should pass/fail based on gpg setup.""" """Should pass/fail based on gpg setup."""
with mock.patch.object( with mock.patch.object(
self.wrapper, "NeedSetupGnuPG", return_value=True repo_wrapper, "NeedSetupGnuPG", return_value=True
): ):
with mock.patch.object(self.wrapper, "SetupGnuPG") as m: with mock.patch.object(repo_wrapper, "SetupGnuPG") as m:
m.return_value = True m.return_value = True
self.assertTrue(self.wrapper.check_repo_verify(True)) assert repo_wrapper.check_repo_verify(True)
m.return_value = False m.return_value = False
self.assertFalse(self.wrapper.check_repo_verify(True)) assert not repo_wrapper.check_repo_verify(True)
class CheckRepoRev(GitCheckoutTestCase): class TestCheckRepoRev:
"""Check check_repo_rev behavior.""" """Check check_repo_rev behavior."""
def test_verify_works(self): def test_verify_works(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Should pass when verification passes.""" """Should pass when verification passes."""
with mock.patch.object( with mock.patch.object(
self.wrapper, "check_repo_verify", return_value=True repo_wrapper, "check_repo_verify", return_value=True
): ):
with mock.patch.object( with mock.patch.object(
self.wrapper, "verify_rev", return_value="12345" repo_wrapper, "verify_rev", return_value="12345"
): ):
rrev, lrev = self.wrapper.check_repo_rev(self.GIT_DIR, "stable") rrev, lrev = repo_wrapper.check_repo_rev(
self.assertEqual("refs/heads/stable", rrev) git_checkout.git_dir, "stable"
self.assertEqual("12345", lrev) )
assert rrev == "refs/heads/stable"
assert lrev == "12345"
def test_verify_fails(self): def test_verify_fails(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Should fail when verification fails.""" """Should fail when verification fails."""
with mock.patch.object( with mock.patch.object(
self.wrapper, "check_repo_verify", return_value=True repo_wrapper, "check_repo_verify", return_value=True
): ):
with mock.patch.object( with mock.patch.object(
self.wrapper, "verify_rev", side_effect=Exception repo_wrapper, "verify_rev", side_effect=RuntimeError
): ):
with self.assertRaises(Exception): with pytest.raises(RuntimeError):
self.wrapper.check_repo_rev(self.GIT_DIR, "stable") repo_wrapper.check_repo_rev(git_checkout.git_dir, "stable")
def test_verify_ignore(self): def test_verify_ignore(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Should pass when verification is disabled.""" """Should pass when verification is disabled."""
with mock.patch.object( with mock.patch.object(
self.wrapper, "verify_rev", side_effect=Exception repo_wrapper, "verify_rev", side_effect=RuntimeError
): ):
rrev, lrev = self.wrapper.check_repo_rev( rrev, lrev = repo_wrapper.check_repo_rev(
self.GIT_DIR, "stable", repo_verify=False git_checkout.git_dir, "stable", repo_verify=False
) )
self.assertEqual("refs/heads/stable", rrev) assert rrev == "refs/heads/stable"
self.assertEqual(self.REV_LIST[1], lrev) assert lrev == git_checkout.rev_list[1]
+5
View File
@@ -27,6 +27,11 @@ from typing import Optional, Union
import git_command import git_command
THIS_FILE = Path(__file__).resolve()
THIS_DIR = THIS_FILE.parent
FIXTURES_DIR = THIS_DIR / "fixtures"
def init_git_tree( def init_git_tree(
path: Union[str, Path], path: Union[str, Path],
ref_format: Optional[str] = None, ref_format: Optional[str] = None,