Compare commits

..

112 Commits

Author SHA1 Message Date
Gavin Mak 2d54384a5e sync: Add --superproject-rev flag to sync to specific revision
Allow syncing the outer manifest to a state defined by a specific
superproject revision. It updates the superproject, reads the manifest
commit from .supermanifest, and checks out the outer manifest project
to that commit.

Submanifests are then processed normally, allowing them to be updated
to the revisions specified in the new outer manifest state.

Bug: 416589884
Change-Id: I304c37a2b8794f9b74cb7e5e209a8a93762bdb52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/576321
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-20 12:28:55 -07:00
Josef Malmström 1b4e7a04be Fix submodules not synced for repeated repo
If I check out e.g. multiple branches of the same repo in one
manifest, only the submodules of one checkout are synced.

Fix this by adding the relative path  of the checkout as a key
to the mapping of derived projects, preventing overwrite.

The fix was originally proposed here:
https://issues.gerritcodereview.com/issues/40013218

Bug: 40013218
Change-Id: Ia86518ccf2c0af7bd7e4daf8d703a55b7e10ba52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581062
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
2026-05-20 06:05:57 -07:00
gurusai-voleti a94c9e2b52 Automated: Migrate gerrit/git-repo from gsutil to gcloud storage
Bug: 486536908
Change-Id: I248b093e189a3784b8959e837a5d66857f1ce0fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/577201
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Guru sai rama subbarao Voleti (xWF) <gvoleti@google.com>
Commit-Queue: Guru sai rama subbarao Voleti (xWF) <gvoleti@google.com>
2026-05-14 21:31:49 -07:00
Gavin Mak 51021fb209 abandon/start/info: Make them respect smart sync override
Call TryOverrideManifestWithSmartSync in start, abandon, and info
commands.

This ensures they pick up the pinned revisions from the smart sync
override manifest if it exists, rather than falling back to ToT from
the default manifest.

Bug: 279204331
Change-Id: I637f054a77773805daf0bf9cf5a712d82a5959b4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582503
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-14 13:19:55 -07:00
Gavin Mak d453273f06 command: Move smart sync override logic to Command base class
Deduplicate the logic for loading `smart_sync_override.xml` by
moving it from `forall.py` to the `Command` base class. This allows
other commands to reuse it to respect smart tags.

Bug: 279204331
Change-Id: I6f2f03995266c2a68c3225cacb92b2f580a89178
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582502
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-14 13:19:26 -07:00
Mike Frysinger 0129ac1c46 docs: drop period in headers
We don't do this in most places, so standardize the outliers.

Change-Id: I80c1ef16c5fa7355dcb7797bc8e56852b22c24d5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583641
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-14 09:03:56 -07:00
Mike Frysinger bf8c59f4a0 run_tests: help2man: update to latest release
Change-Id: I30e4a6452ac5504b782fe545a59ca1f58ac70385
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583621
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-05-14 09:03:52 -07:00
Josef Malmström 003f0407fc Fix missing None check in Remote.Save
This code was causing an exception in cases where `pushUrl` was set
but `projectname` was not.

This can happen when `pushUrl` is set for the manifest repo which does
not have a `projectname`. For example:

repo init --manifest-url ssh://url.to/my/manifest

cd .repo/manifests
git config remote.origin.pushurl ssh://url.to/my/manifest

repo init --manifest-url ssh://url.to/my/manifest --repo-rev main

The last `repo init` invocation causes an error.

Change-Id: Ibb68c8446880cfbac22feee595d1fd1b678c7ade
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579162
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-13 00:00:27 -07:00
Josef Malmström 1db50e49fb Add support for self referencing submodules
When working with relative submodule paths, The "./" needs special
handling similar to "../".

See information on:
https://git-scm.com/docs/git-submodule
Which currently states:
"<repository> is the URL of the new submodule’s origin repository.
This may be either an absolute URL, or (if it begins with ./ or ../),
the location relative to the superproject’s default remote
repository (Please note that to specify a repository foo.git which is
located right next to a superproject bar.git, you’ll have to use
../foo.git instead of ./foo.git - as one might expect when following
the rules for relative URLs - because the evaluation of relative URLs
in Git is identical to that of relative directories)."

The implementation also was not handling file/directory names
starting with "." or "..". Explicitly look for "./" and "../"
instead.

Change-Id: I8ae68d61fb0cbb1624183b175236e98a36e4afdb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579182
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
2026-05-13 00:00:11 -07:00
Mike Frysinger c97a306fe7 release: update-manpages: revert color filtering
We pull a pinned help2man from cipd now, so we should get stable
behavior between developers.  That means the color filtering should
not be necessary anymore, and we can drop the logic & unittest.

Change-Id: Ib53e1ce7f8d610d7f624c9a019c79dc5f438ac0d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582402
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-12 11:50:38 -07:00
Carlos Fernandez 5534f164d6 linkfile: Handle directory-to-symlink transitions safely
When a manifest changes from individual linkfiles inside a directory
(e.g. dest=".llms/rules", dest=".llms/skills") to a single linkfile
for the whole directory (e.g. dest=".llms", src="dot-llms"), two
things need to happen:

1. __linkIt must replace a real directory with a symlink.  Use
   os.rmdir() instead of platform_utils.remove() for real directories.
   rmdir only removes empty directories, so user-created content is
   never deleted.

2. UpdateCopyLinkfileList must handle the cleanup correctly:
   - Use os.rmdir() for directories (safe for non-empty)
   - Remove empty parent directories after cleaning old dests
   - Retry _CopyAndLinkFiles for all projects, since in interleaved
     sync mode _CopyAndLinkFiles runs before cleanup and may have
     failed because the directory was not yet empty

Change-Id: I0437b80beab98bce064cea81c11c47d699be91aa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569243
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
2026-05-12 11:10:43 -07:00
Mike Frysinger 67e52a120b run_tests: leverage cipd when available for help2man
This tool isn't installed on CI bot images so we've been skipping it,
but this is causing people to not run tests locally, and ignore errors.
Use cipd to pull the tool in when available.

Then revert a recent man change that the tool rejects.

Change-Id: I1030d0070fd5a624656eba7434ae6ec99b2e3f2d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582401
Reviewed-by: Greg Edelston <gredelston@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-05-12 10:45:13 -07:00
Gavin Mak d5037230e9 sync: Re-raise KeyboardInterrupt in main process
When running sync -j1, worker functions run directly in the main
process. Swallowing KeyboardInterrupt causes the loop to continue to the
next project instead of aborting.

Re-raise KeyboardInterrupt if running in the MainProcess, while
maintaining the suppression of stack traces in worker processes.

Bug: 468170157
Change-Id: I156d66bc209a265f7fa25eea0eb88737d1b51a34
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581342
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-05-12 10:41:47 -07:00
Mike Frysinger 7cc99b24c1 git_config: fix error message command output
str() inside an f-string is redundant.  Actually join the array together
as a string to show a proper error message, and include the full command
line, not the extra options passed in.

Before:
error.GitError: git config ('--null', '--list'): fatal: unable to read config file 'xxx': No such file or directory

After:
error.GitError: git config --system --includes --null --list: fatal: unable to read config file 'xxx': No such file or directory

Change-Id: I8983389aa2e0de7808991e73e636b77810f04c4b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582341
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-05-12 10:04:15 -07:00
Greg Edelston 12ad396f67 info: Parallelize repo info to improve performance
For a large checkout like chromiumos or android, `repo info` takes a
really long time! On my machine it took ~6 minutes. On a randomly
selected ChromiumOS cq-orchestrator build it took 4.1 minutes:
https://ci.chromium.org/b/8682060180498819729. This adds up to a lot of
wasted runtime for both humans and bots.

The problem is that `repo info` was single-threaded, which causes poor
performance when the checkout has 1000+ projects. We already have a
pattern for parallelization; let's use it.

BUG=None
TEST=Manually run, ensure no diff

Change-Id: I6b82b9495eb2a0e602a142dd3a16f09217871e1b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581921
Tested-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-12 09:59:07 -07:00
Gavin Mak a6bc1b7cf0 sync: Exclude stateless sync pruned projects from bloat check
If a project has been pruned for stateless sync, it has already
undergone aggressive garbage collection. There should be no bloat
to check for.

Change-Id: I9233a4611e05b7a0b7c097827d6f408144f7380d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581841
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-11 19:36:59 -07:00
Gavin Mak a98e422113 completion: document installation and usage in README
Add a "Shell Completion" section to README.md to document how to install
and use `completion.bash` and `completion.zsh`.

Change-Id: Ibf6c81043af6c24d45ea2dc6a6caa26d98ab7374
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581261
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-08 17:41:44 -07:00
Gavin Mak 5d8585012f sync: Suggest "git repack -a -d" in bloat warning
The warning suggests running "repo sync --auto-gc" which runs "git gc
--auto". Git may decide not to repack if its internal thresholds are not
met, leaving the warning active even after running the suggested
command.

"git repack -a -d" forces all reachable objects into a single pack and
deletes redundant packs, reducing the pack count to 1. This guarantees
that the warning (which triggers when pack count exceeds 10) goes away.

Bug: 505755299
Change-Id: I10163ef8efb7f3b7c5055378ad95051974d11b88
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580821
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-05-08 14:12:22 -07:00
Kamal Sacranie e8deeabf2a Add zsh completion
This PR adds `completion.zsh` to allow for relatively intelligent
completion of `repo` command-line arguments for all commands of the
`repo` CLI.

The `completion.zsh` can be added to your zsh completions by copying
(and renaming) the completion script to a directory that you have added
to your zsh completion path.

```
cp completion.zsh $ZSH_COMPLETION_DIRECTORY/_repo
```

You must name the file `_repo` for it to register as completion for
`repo`.

You can add a directory to your zsh completion by updating your `fpath`
variable before calling `compinit` as follows:

```
fpath=( /path/to/zsh-completions/dir $fpath )
```

Future work: Generate this file using the subcommand classes and python
reflection.

Change-Id: I6a4e785c1efcf9076bd693976ac03578836b691b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579561
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Kamal Sacranie <sacranie@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-08 07:46:00 -07:00
Gavin Mak 11428ae984 forall: Document REPO_UPSTREAM and REPO_DEST_BRANCH envvars
Change-Id: I74365295152f8828587c6b4ed93029efc6000881
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580761
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-07 20:27:08 -07:00
Carlos Fernandez 27d2232eb3 info: add --format and --include-summary/--include-projects options
Add --format={text,json} to produce machine-readable output, and
boolean options to control which sections are displayed:
  --include-summary / --no-include-summary (default: on)
  --include-projects / --no-include-projects (default: on)

The JSON output respects the include flags, so callers can request
only the fields they need (e.g. `repo info --format=json
--no-include-projects` for manifest metadata only).

Change-Id: I9641bc4023b630d9c61c5170eb86e5f3b787236f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569203
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
2026-05-07 19:23:24 -07:00
Josef Malmström e3eadd3728 sync: Fix force_checkout propagation
The force_checkout parameter was not propagated in all calls to
Checkout in Sync_LocalHalf.

Without this, repo sync --force-checkout can still fail for projects
currently on a local branch with no upstream/tracking configuration,
because the detach-to-manifest checkout was executed without -f,
leaving local modifications or untracked files able to block sync.

Change-Id: I58551388e2f906c4db96e220707a369057a71c24
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579181
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-06 08:53:27 -07:00
Gavin Mak 12cfc6036a git_superproject: Remove redundant _branch variable
Both variables were initialized to the same value and never modified.

Bug: 416589884
Change-Id: Iaa1ffffb4543d4cd9391ac679d9234fe01678861
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/576921
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-04-28 10:49:54 -07:00
Nasser Grainawi 134eeb024b tests: Add tests for repo status output
Build out status subcommand unit coverage using a minimal fake repo
checkout wired through XmlManifest.

The new tests verify:
- clean status output prints the expected project header
- modified tracked files appear with the expected status marker
- `-o` output includes the orphan section and orphan entries
- branch names shown in status reflect a started non-default branch

Change-Id: Ia7c22593d0bbdc4aed81faeb168b846f3e4016ab
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558501
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-23 17:52:09 -07:00
Nasser Grainawi 03fb18109f color: type SetDefaultColoring and drop bool states
Also add a test fixture to always disable coloring so that we get
reproducible test behavior. The few tests that want to check color
behavior (e.g. test_color.py) can still reset/change color values as
needed.

Change-Id: I1808139a63e0862c29bf81d7097e885ca8561040
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/573621
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
2026-04-22 17:05:55 -07:00
Ram Peri 5af71ce907 Add timing keyword argument for hooks
Measure the duration of the sync operation in the Execute method of the
Sync command and pass it to post-sync hooks as a standard keyword
argument (`sync_duration_seconds`).

Updates based on code review:
- Update _API_ARGS in hooks.py to allow sync_duration_seconds for post-sync hooks.
- Do not cast sync_duration_seconds to int for better granularity.
- Update docs/repo-hooks.md to document sync_duration_seconds.
- Add unit test for argument validation in test_hooks.py.

Test: Ran run_tests using venv python, all 554 tests passed.
Bug: TBD
Change-Id: Ie29e002a5d283460d993ad96c224dbf4b6d7985c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/575021
Tested-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-21 17:10:07 -07:00
Gavin Mak baa281d99e sync: Refactor to use _RunOneGC and fix config leakage
Extract _RunOneGC to handle GC on a single project. This refactoring
makes it easier to invoke GC from parallel worker tasks.

Also, avoid modifying the passed-in config dictionary in _RunOneGC by
creating a local copy, preventing unintended side effects on other
commands sharing the same config.

Bug: 498290329
Change-Id: I7b77ed6629b14b5ee3322870b9c6c8ce2bfd6ea2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574923
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-20 15:34:07 -07:00
Gavin Mak 7e9079b7cf sync: Switch to using self._bloated_projects
Store bloated projects in self._bloated_projects and print warnings at
the end of execution. This sets up for moving the check to workers.

Bug: 498290329
Change-Id: I993f1fd741db2994d480994861588eb18f6c5503
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574922
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-20 14:07:14 -07:00
Gavin Mak 61bd6b3ccb tests: Add tests for _CheckForBloatedProjects and _GCProjects
Change-Id: I6790a13929b8fc06a3197ca405dedf08d39d54c9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574926
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-20 13:31:41 -07:00
Marty Heavey 32e7327ca6 upload: Clarify partial sync message on hook failure
Demote the partial sync warning from error to info and rephrase it to
be a tip rather than an error. This prevents users from thinking that
a partial sync is the cause of their hook failures when it is often a linting failure.

The message now suggests that a full sync might help if there are
cross-project dependencies, instead of implying it will fix any issue.

Change-Id: I5d8c52b53ac315aa9f145ed069798bf201fa0815
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574262
Tested-by: Marty Heavey <mheavey@google.com>
Commit-Queue: Marty Heavey <mheavey@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-20 01:52:25 -07:00
Gavin Mak 4f707ff91e sync: Provide feedback during post-sync operations
After the main sync progress bar finishes, there's a pause while some
post-sync operations run. Print something to provide feedback so the
user doesn't think repo has hung.

Bug: 503869525
Change-Id: I695fd560e60dcb394e6844a56c8a336ca1f71c74
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574425
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-04-17 15:47:15 -07:00
Gavin Mak e2671c184e progress: Ignore updates after progress ends
This addresses the occasional "..working.." message at the end of sync.

Bug: 503869525
Change-Id: I489d29fa8ae588d77abb7fee5f157800d4abb368
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574482
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
2026-04-17 15:43:14 -07:00
Becky Siegel b43a20b859 project: Avoid skipping fetches for shallow clones without .git/shallow
When optimizing fetches for projects with immutable revisions, the fetch
should not be skipped if the project is configured for a shallow clone
(depth > 0) but the .git/shallow file is missing. The absence of the
.git/shallow file means the repository is not a shallow clone, or the
shallow clone is incomplete, so a fetch is necessary to ensure the
revision is present.

Bug: 503081454
Change-Id: Ic3549612bcd69050a926652ee4e522c79ad8124c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/573821
Tested-by: Becky Siegel <beckysiegel@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Becky Siegel <beckysiegel@google.com>
2026-04-17 09:16:36 -07:00
Miyako.Enei 8869a30283 project: Drop --no-deref from update-ref --stdin
repo calls `git update-ref --stdin` when updating multiple refs during
repo init and repo sync. Historically, `--no-deref` was also passed.

Older Git 2.17 which we still support rejects the combination of
`--stdin` and `--no-deref`, emitting a usage error even when the stdin
input is valid.

The `--no-deref` option is only meaningful when updating symbolic refs
such as HEAD. The stdin-based update-ref path only operates on explicit
refs (tags, remote refs, alternates) and never symbolic refs.

Remove the unnecessary option to restore compatibility with Git 2.17
while preserving identical behavior on newer Git versions.

Tested with:
  - Git 2.17.1
  - Git 2.34.1

Change-Id: I22001de03800f5699b26a40bc1fb1fec002ed048
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/571721
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Enei <miyako.enei@alpsalpine.com>
Tested-by: Enei <miyako.enei@alpsalpine.com>
2026-04-12 16:39:28 -07:00
Gavin Mak 3b0eebeccf project: implement stateless sync pruning logic
Implement in-situ shallow re-fetching and garbage collection logic.
Enables repositories with sync-strategy="stateless" to reclaim disk
space by running reflog expire and git gc --prune=now if the working
tree is clean and has no local commits.

Bug: 498730431
Change-Id: I940bdc9b74da29d3f7b13566667dcddea769ebd3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/568463
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-09 12:09:40 -07:00
Gavin Mak 00991bfb42 manifest: Add sync-strategy attribute to project elements
The only supported sync-strategy is "stateless". The intent is to keep
the local workspace as small as possible by not keeping history during
syncs. This prevents disk space waste for projects with large binaries
where we only care about the current version.

A follow up change will implement the logic.

Bug: 498730431
Change-Id: I84a436a9ca2492893163c6cfda6c28dc62a568f0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/568462
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-09 12:09:28 -07:00
Nasser Grainawi e8338b54bd tests: Convert forall subcmd test to pytest
Rewrite tests/test_subcmds_forall.py from unittest.TestCase to pytest
function-style tests to match the surrounding test suite conventions.

Replace setUp/tearDown and class-based helpers with tmp_path-based
setup, switch stdout capture to contextlib.redirect_stdout, and keep the
existing behavior checks intact (all eight projects are invoked exactly
once).

Change-Id: I9243f3461aa6850f867bdb864f4a34c442f817f6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569821
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-04-09 11:51:47 -07:00
Gavin Mak 951666fb23 gc: Fix hang during repack in partial clones
Add `--missing=allow-promisor` to `git rev-list` calls in
`repack_projects`. This prevents Git from auto-fetching missing objects
from the promisor remote, which can cause stalls due to sequential
network requests.

Also add a Git version check to ensure Git is at least 2.17.0 before
running `--repack`, as `--missing=allow-promisor` was introduced in that
version.

Bug: 500133631
Change-Id: I2dcf9b46fac4c6a53a3c2a46f06f61d6aec40f2f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/570361
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
2026-04-08 12:54:39 -07:00
Carlos Fernandez 854b330967 test_wrapper: add test for repo script executable permission
Add a test to verify that the repo launcher script has the
executable bit set, guarding against accidental permission changes.

Change-Id: I314658b57ed174673188fbbc5962d9fdeefac97d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569242
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
2026-04-06 14:16:04 -07:00
Mike Frysinger 654690e1b8 tests: convert more tests to pytest
Change-Id: Id4d48b61dc435564c336385bbc4944eb475d1942
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569443
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-06 11:36:39 -07:00
Mike Frysinger ac2be4c089 tests: convert __file__ usage to pathlib
Change-Id: I2408b0ac97629f0d5fc92779b78bf1ff159a6f83
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569442
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-04-06 11:15:58 -07:00
Mike Frysinger 3d819e8e3e tests: unify fixture() helper with Path constant
Change-Id: I63751042391f5cc3e06af7067bc83d67bd0716dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569441
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-06 11:15:14 -07:00
Carlos Fernandez 573983948a Fix all flake8 warnings from newer flake8-bugbear and flake8-comprehensions
Address warnings introduced by flake8-bugbear 24.12.12 and
flake8-comprehensions 3.16.0:

- C408: Replace dict()/list() calls with literal {} and []
- C413: Remove unnecessary list() around sorted()
- C414: Remove unnecessary list() inside sorted()
- C419: Suppress intentional list comprehension in all() (noqa)
- B001: Replace bare except with except Exception
- B006: Replace mutable default arguments with None
- B010: Replace setattr() with direct attribute assignment
- B017: Use RuntimeError instead of Exception in tests
- B019: Suppress lru_cache on methods for long-lived objects (noqa)
- B033: Remove duplicate item in set literal

Change-Id: If4693d3e946200bbc22f689f7b94da604addcb80
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/566321
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-03 07:50:52 -07:00
Gavin Mak 3f3c681a02 project: Refactor GetHead to use symbolic-ref first
Simplify branch resolution and optimize unborn branch detection by
prioritizing symbolic-ref over rev-parse.

Change-Id: Ic62dcb87cd051dafb00d520b1157be2e32abc2ab
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/563222
Reviewed-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-02 13:57:05 -07:00
Sam Saccone 242e97d9dd Implement command forgiveness with autocorrect
Similar to `git`, when a user types an unknown command like `repo tart`,
we now use `difflib.get_close_matches` to suggest similar commands.

If `help.autocorrect` is set in the git config, it will optionally
prompt the user to automatically run the assumed command, or wait
for a configured delay before executing it.

Verification Steps:
1. Created a dummy repo project locally.
2. Verified `help.autocorrect=0|false|off|no|show` suggests
   command and exits.
3. Verified `help.autocorrect=1|true|on|yes|immediate`
   automatically runs suggestion.
4. Verified `help.autocorrect=<number>` runs after
   `<number>*0.1` seconds.
5. Verified `help.autocorrect=never` exits immediately without
   suggestions.
6. Verified `help.autocorrect=prompt` asks user to accept [y/n]
   and handles correctly.

BUG: b/489753302

Change-Id: I6dcd63229cbd7badf5404459b48690c68f5b4857
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558021
Tested-by: Sam Saccone <samccone@google.com>
Commit-Queue: Sam Saccone <samccone@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-03-24 15:47:37 -07:00
Mike Frysinger ade45de770 docs: windows: mention Developer Mode for symlinks
This is probably better than recommending Administrator access.

Change-Id: Ic916f15fe03f7fa1e03c685265b4774bfc1279c2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/563581
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-03-20 14:07:06 -07:00
Lucas Tanure 0251fb33c4 project: don't re-shallow manually unshallowed repos during sync
If a user has manually unshallowed a repo (e.g. via
`git fetch --unshallow`), the absence of the `shallow` file in the
gitdir indicates a full clone. Re-applying depth during a subsequent
sync would undo the user's intent. Skip re-shallowing in this case
by clearing depth when the project is not new and no shallow file
is present.

Change-Id: I4ee0e78018de9078fe1bd77a9615613ef0c40d33
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558743
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Lucas Alves <ltanure@gmail.com>
Tested-by: Lucas Alves <ltanure@gmail.com>
Reviewed-by: Lucas Alves <ltanure@gmail.com>
2026-03-20 10:08:40 -07:00
Jacky Liu 0176586544 Use git_superproject.UseSuperproject() everywhere
Currently somewhere use git_superproject.UseSuperproject(), which checks
both the manifest config and user's config, and otherwhere use
manifest.manifestProject.use_superproject, which only checks the
manifest config. This causes Inconsistent behaviors for users who do not
set --use-superproject when doing repo init but have
repo.superprojectChoice in their git config.

Replace where using manifest.manifestProject.use_superproject with
git_superproject.UseSuperproject() to respect user's config and avoid
inconsistency.

Bug: 454514213
Change-Id: I1f734235cdd67b8a6915f1d05967d1aaa4d03f2a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/561801
Commit-Queue: Jacky Liu <qsliu@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jacky Liu <qsliu@google.com>
2026-03-18 21:03:07 -07:00
Mike Frysinger 582804a59e pydev: drop Python 2 reference
Not sure who uses this anymore, but might as well delete obviously
wrong content.

Change-Id: I5cdf1cf699c81b7db32b400f371134d21f474743
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/563161
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-03-18 12:01:37 -07:00
Mike Frysinger afc3d55d39 isort: merge config into pyproject.toml
Change-Id: I3a50de04897789c7b2f291882faf1c862645b054
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/563141
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-03-18 12:01:25 -07:00
Mike Frysinger f24bc7aed5 tests: switch some test modules to pytest
Change-Id: I524b5ff2d77f8232f94e21921b00ba4027d2ac4f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/563081
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-03-18 11:13:26 -07:00
Gavin Mak 83b8ebdbbe git_superproject: avoid re-initing bare repo
Running sync with reftable on a files-backed workspace fails to re-init
the superproject dir with:
```
fatal: could not open
'.../.repo/exp-superproject/<hash>-superproject.git/refs/heads' for writing:
Is a directory
```

Bug: 476209856
Change-Id: Ie8473d66069aafefa5661bd3ea8e73b2b27c6a38
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550981
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-03-17 14:30:35 -07:00
Gavin Mak a0abfd7339 project: resolve unborn HEAD robustly in reftable repos
Use `git symbolic-ref` to resolve HEAD before trying to parse .git/HEAD
directly which is unreliable for reftable repos.

Bug: 476209856
Change-Id: I60185d945c5b43c871945c0126cfdf52194e745d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550762
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-03-17 14:30:20 -07:00
Gavin Mak 403fedfeb5 project: support reftable anchors in worktree .git migration
The reftable backend creates real refs/ and reftable/ dirs. Update
_MigrateOldWorkTreeGitDir to expect these dirs and remove them.

Bug: 476209856
Change-Id: I4700da70cb466e25ecbc51ba4de9a906b8716bd8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550761
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-03-17 14:30:01 -07:00
Gavin Mak f14c577fce project: avoid direct packed-refs writes during fetch
Replace raw file manipulation with native `git update-ref` commands
inside a try/finally block to ensure temp refs are created/cleaned up
regardless of storage format.

Bug: 476209856
Change-Id: I228e81d3d3b323328260f6672075193421c8dc47
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550421
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-03-17 14:29:38 -07:00
Gavin Mak 67881c0c3b git_refs: read refs via git plumbing for files/reftable
Replace direct `packed-refs` file parsing with `git for-each-ref`
plumbing to support both `files` and `reftable` backends.

Bug: 476209856
Change-Id: I2ad8ff8f3382426600f15370c997f9bc17165485
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550401
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-03-17 14:28:39 -07:00
Mike Frysinger 551087cd98 tests: add a util module for sharing code
We've started duplicating code among test modules.  Start a common
utils module to hold that, and migrate over TempGitTree to start.

Change-Id: I10b2abd133535c90fbda4d6686602d7e5861d875
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/559041
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-03-12 20:09:19 -07:00
Mike Frysinger 8da56a0cc5 man: refresh after recent changes
Change-Id: Ibd60f89406e89255b3284413442b1d9c0ccbfb6d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/559601
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jeffery Miller <jefferymiller@google.com>
2026-03-10 07:35:40 -07:00
Jeffery Miller 0f01cd24e9 docs: Document support for child elements in extend-project
Clarify the existence and behavior of child elements when added to
extend-project.

Change-Id: Id9f270166c8498d4051495b9a1f68360f66e9143
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/553742
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jeffery Miller <jefferymiller@google.com>
Commit-Queue: Jeffery Miller <jefferymiller@google.com>
2026-02-19 08:38:56 -08:00
Jeffery Miller 1ee98667cc tests: Add extend-project test for additional annotations
Multiple annotations can exist for the same name when
extending projects. Add a test case to show this behavior.

Change-Id: I12bbd25e642c7e615e32f66a1c364a39ac81902c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/553906
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Jeffery Miller <jefferymiller@google.com>
Tested-by: Jeffery Miller <jefferymiller@google.com>
2026-02-19 08:35:03 -08:00
Jordan Esh 6f9622fe1c sync: Remove dependency on ssh if not needed
When running on a machine without the `ssh` command, repo sync would fail even if no ssh or ssh proxy was required. Use exception handling inside ssh.ProxyManager to more gracefully handle the case where ssh is not installed.

Bug: 467714011
Change-Id: I602a0819638ead4d02de88b750839bc3d70549ce
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/535141
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Jordan Esh <esh.jordan@gmail.com>
Commit-Queue: Jordan Esh <esh.jordan@gmail.com>
2026-02-11 17:00:07 -08:00
Gavin Mak 5cb0251248 gc: fix untargeted projects being deleted
`delete_unused_projects` needs a full list of active projects to figure
out which orphaned .git dirs need to be deleted. Otherwise it thinks
that only the projects specified in args are active.

Bug: 447626164
Change-Id: I02beebf6a01c77742a8db78221452d71cd78ea73
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550061
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-02-09 08:34:42 -08:00
Gavin Mak a214fd31bd manifest: Introduce sync-j-max attribute to cap sync jobs
Add a way for manifest owners to limit how many sync jobs run in
parallel.

Bug: 481100878
Change-Id: Ia6cbe02cbc83c9e414b53b8d14fe5e7e1b802505
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/548963
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-02-06 13:32:28 -08:00
Sam Saccone 62cd0de6cf Make git trace2 logging errors conditional on verbose mode.
Add a verbose attribute to the EventLog class, defaulting to False.
Error messages printed to sys.stderr within the EventLog.Write method
are now guarded by this verbose flag. In main.py, set EventLog.verbose
to True if the command-line --verbose option is used. This prevents
trace2 logging failures from being printed to stderr unless verbose
output is explicitly requested.

PROMPT=convert all git trace2 logging print messages to verbose only
logging

BUG: b/479811034
Change-Id: I8757ee52117d766f2f3ec47856db64cc4f51143c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/547542
Tested-by: Sam Saccone <samccone@google.com>
Reviewed-by: Julia Tuttle <juliatuttle@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-02-03 11:53:48 -08:00
Mike Frysinger b60512a75a run_tests: log tool versions
Change-Id: I4eee58786bae6d442773c63fa937fb11eda1e2f0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/547863
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-01-29 12:32:46 -08:00
Mike Frysinger 5d88972390 Revert "init: change --manifest-depth default to 1"
This reverts commit 622a5bf9c2.

CrOS infra is failing to sync code now for some reason.
Until we can investigate further, pull this back out.

Bug: 475668525
Bug: 468033850
Change-Id: I35a8623a95336df1be27ea870afbfc8065609f01
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/545141
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-01-21 12:32:59 -08:00
Gavin Mak 3c0e67bbc5 manifest_xml: prevent extend-project from inheriting local groups
When extending a project in a local manifest, the project inherits the
`local:` group. This causes the superproject override logic (which omits
projects with `local:` groups) to incorrectly exclude the project from
the override manifest. This leads to "extend-project element specifies
non-existent project" errors during sync reload.

Fix this by stripping `local:` groups from extended projects, ensuring
they remain visible to superproject overrides while still allowing other
inherited groups to persist.

Bug: 470374343
Change-Id: I1a057ebffebc11a19dc14dde7cc13b9f18cdd0a3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/543222
Reviewed-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-01-15 10:27:32 -08:00
Nico Wald 3b7b20ac1d CONTRIBUTING: fix HTTP password URL
Change-Id: I7ae085896fe951c2b1c662689fa111a0661f988d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/539762
Tested-by: Nico Wald <nicowald@mac.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Nico Wald <nicowald@mac.com>
2026-01-12 09:48:47 -08:00
Gavin Mak e71a8c6dd8 project: disable auto-gc for depth=1 in git config
During sync, `git checkout` can trigger fetch for missing objects in
partial clones. This internal fetch can trigger `git maintenance` or
`git gc` and cause delays during the local checkout phase. Set
maintenance.auto to false and gc.auto to 0 in during `_InitRemote` if
`depth=1` to ensure that implicit fetches spawned by git skip GC.

Bug: 379111283
Change-Id: I6b22a4867f29b6e9598746cb752820a84dc2aeb6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/540681
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-01-08 11:33:40 -08:00
Mike Frysinger c687b5df9e run_tests/release: require Python 3.9+
While we support running `repo` on clients with older Python versions,
we don't need to hold the runners & release code back.  These are only
used by repo devs on their systems to develop & release repo.

Python 3.9 was picked due to its typing changs which we've already
started using in this code.

Change-Id: I6f8885c84298760514c25abeb1fccb0338947bf4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/539801
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-01-06 11:36:26 -08:00
Mike Frysinger 1dd9c57a28 tests: drop tox support
This hasn't been working out as well as we'd hope.  Tox relies on
the system having Python versions installed which distros don't
tend to carry anymore.  Our custom run_tests leverages vpython
when possible to run stable Python 3.8 & 3.11 versions which is
providing an OK level of coverage in practice.

Change-Id: Ida517f7be47ca95703e43bc0af5a24dd70c0467e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/540001
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-01-06 11:32:42 -08:00
Mike Frysinger 4525c2e0ad github: add black check action
Change-Id: Ic87c1c5c72fb8a01108146c1f9d78466acb57278
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/540021
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-01-06 11:00:32 -08:00
Mike Frysinger 45dcd738b7 tests: skip AF_UNIX tests when unavailable
UNIX sockets aren't available under Windows, so skip the test.

Change-Id: Ic4ca22d161c6dee628352aad07ac6aaceb472ac2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/540002
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2026-01-06 10:17:53 -08:00
Mike Frysinger 1dad86dc00 check-metadata: skip files that do not exist
If the files don't exist, then they can't have errors, so skip checking.

Change-Id: I3ed4be4912b253c5454df41d690cb33dfe191289
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/540003
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2026-01-06 10:17:32 -08:00
Mike Frysinger 622a5bf9c2 init: change --manifest-depth default to 1
Most users do not care about the manifest history in .repo/manifests/.
Let's change the default to 1 so things work smoothly for most people
most of the time.  For the rare folks who want the full history, they
can add --manifest-depth=0 to their `repo init`.

This has no effect on existing checkouts.

Spot checking Android & CrOS manifests shows significant speedups.
Full history can take O(10's seconds) to O(minutes) while depth of 1
takes constant time of O(~5 seconds).

Bug: 468033850
Change-Id: I4b8ed62a8a636babcc5226552badb69600d0c353
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/535481
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-01-05 06:36:08 -08:00
Gavin Mak 871e4c7ed1 sync: skip bloat check if fresh sync
Initial syncs won't have accumulated any garbage.

Bug: 379111283
Change-Id: I04b2ecde3e33f1f055038861a2705ab6aabb36d1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/536083
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-15 15:24:45 -08:00
Gavin Mak 5b0b5513d6 project: only use --no-auto-gc for git 2.23.0+
The flag for git fetch was introduced in git 2.23.0. Also skip the bloat
check after sync if using an older version.

Bug: 468589976
Bug: 379111283
Change-Id: Ib53e5494350c71a83906e5219d3a8c2b654e531f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/536082
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-15 11:32:49 -08:00
Gavin Mak b5991d7128 sync: Add heuristic warning for bloated shallow repositories
For clone-depth="1" repositories that are dirty or have local commits,
add a check at the end of sync to detect excessive git object
accumulation.

This prevents silent performance degradation and disk exhaustion in
large prebuilts repos where automatic GC is typically disabled from
https://gerrit.googlesource.com/git-repo/+/7f87c54043ce9a35a5bb60a09ee846f9d7070352

Bug: 379111283
Change-Id: I376f38e1555cc6e906d852f6e63dc1c8f6331b4f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/534701
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-10 11:34:40 -08:00
Gavin Mak 7f87c54043 project: disable auto-gc on fetch for projects with clone-depth=1
This prevents GC hangs on repos with large binaries by skipping implicit
GC during network fetch, using clone-depth=1 as a heuristic.

Bug: 379111283
Change-Id: I977bf8cd521b11e37eba7ebc9f62120f2bbaf760
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/533802
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-08 12:18:48 -08:00
Kaushik Lingarkar 50c6226075 Prevent leftover bare gitdirs after failed sync attempts
The gitdir for a project may be left in a state with bare=true due
to a previous failed sync. In this state, during a subsequent sync
attempt, repo will skip initializing the gitdir (since the directory
already exists) and directly attempt to checkout the worktree, which
will fail because the project is bare. To reduce the chance of this
happening, initialize the gitdir in a temp directory and move it once
it is ready.

Bug: 457478027
Change-Id: I4767494a3a54e7734174eae3a0d939fa9d174288
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/524203
Tested-by: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Commit-Queue: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-12-05 10:35:46 -08:00
Peter Kjellerstedt 1e4b2887a7 project: Make the error message more logical when a linkfile fail
Due to the odd naming of the arguments to symlink(), the error when it
failed to create a symbolic link was misleading.

Change-Id: I1d0f30ade5970d80186f13e01c426b066cd1062f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/532541
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-03 08:48:11 -08:00
Peter Kjellerstedt 31b4b19387 info: Print a newline after printing the superproject's revision
Change-Id: Ib20233dad4e1f1fd54dbf5ca0324be22fe0e4db1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/528463
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-03 08:08:06 -08:00
Peter Kjellerstedt 2b6de52a36 Rename XmlManifest.GetGroupsStr() to XmlManifest.GetManifestGroupsStr()
This makes it more clear what kind of groups it refers to.

Change-Id: I47369050d1436efcc77f3a69d5b7c99a536b23bc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/528462
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-03 07:57:22 -08:00
Peter Kjellerstedt 91ec998598 manifest_xml, git_superproject: Rename an argument for XmlManifest.ToXml()
Rename the groups argument to filter_groups to make it more clear what
kind of groups it refers to.

Change-Id: I90e6e9aa74a7e3e697705dd4bf8676226055878b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/528461
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-12-02 11:44:24 -08:00
Mike Frysinger 08964a1658 docs: manifest-format: reformat spec to align the CDATA parts
Most of the file was doing this, but we've been inconsistent when
adding new entries.  Realign all of them.

Change-Id: I99ddb3a1e859235b249b6f08731bdadad8086d4e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/532461
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2025-12-02 10:43:56 -08:00
Peter Kjellerstedt 3073a90046 manifest: Propagate revision attribute through multiple levels of include
Make sure a revision attribute for an include element is propagated
through multiple levels of manifest includes.

Change-Id: If37d65b0cd47da673719976598175d0eb6b7cbbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525341
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-11-26 02:08:44 -08:00
Peter Kjellerstedt 75773b8b9d manifest, project: Store project groups as sets
This helps a lot when including common manifests with groups and they
use extend-project.

Change-Id: Ic574e7d6696139d0eb90d9915e8c7048d5e89c07
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525323
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
2025-11-26 02:08:07 -08:00
Peter Kjellerstedt 412367bfaf project: Use dicts to keep track of copyfiles and linkfiles
This avoids copying/linking the same file/link multiple times if a
copyfile/linkfile element with the same values has been specifed
multiple times. This can happen when including a common manifest that
uses an extend-project element that has a copyfile/linkfile element.

This uses dicts rather than sets to store the copyfiles and linkfiles to
make sure the order they are specified in the manifest is maintained.
For Python 3.7+, maintaining the order that keys are added to dicts is
guaranteed, and for Python 3.6 it happened to be true.

The _CopyFile class and the _LinkFile class are changed to inherit from
NamedTuple to be able to store them in dicts.

Change-Id: I9f5a80298b875251a81c5fe7d353e262d104fae4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525322
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
2025-11-26 02:07:35 -08:00
Peter Kjellerstedt 47c24b5c40 manifest: Make include groups propagate to extend-project elements
Any groups specified to an include element should propagate to any
extend-project elements and then on to the projects.

Change-Id: I62b95689cc13660858564ae569cbfd095961ecc7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525321
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-11-26 02:05:48 -08:00
Gavin Mak be33106ffc wipe: Add new repo wipe subcommand
This new command allows users to delete projects from the worktree
and from the `.repo` directory. It is a destructive operation.

It handles shared projects by refusing to wipe them unless the
`--force` flag is used. It also checks for uncommitted changes
before wiping.

Bug: 393383056
Change-Id: Ia30d8ffdc781a3f179af56310ce31c9dae331bbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/490801
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2025-11-21 10:48:42 -08:00
Mike Frysinger 5998c0b506 tests: manifest_xml: convert most path usage to pathlib
Should be functionally the same, but with pathlib APIs that we've
been slowly adopting in other places, especially unittests.

Change-Id: I81364117f8eaeaf138097cdfc484d4848b7ea5bd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525881
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2025-11-11 10:58:51 -08:00
Peter Kjellerstedt 877ef91be2 man: Regenerate after manifest update
Change-Id: I0e7ef5d4189eaaf6878be709b437ecfb57570e3f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/524921
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
2025-11-06 15:03:30 -08:00
Peter Kjellerstedt 4ab2284a94 manifest: Make extend-project support copyfile, linkfile and annotation
This allows an existing project to be extended by these elements.

Change-Id: I6826e518f39ca86485301491639101943b7e2ae0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/519781
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-10-27 11:38:07 -07:00
Gavin Mak 1afe96a7e9 sync: fix saving of fetch times and local state
Interleaved sync didn't save _fetch_times and _local_sync_state to disk.
Phased sync saved them, but incorrectly applied moving average smoothing
repeatedly when fetching submodules, and discarded historical data
during partial syncs.

Move .Save() calls to the end of main sync loops to ensure they run
once. Update _FetchTimes.Save() to merge new data with existing history,
preventing data loss.

Change-Id: I174f98a62ac86859f1eeea1daba65eb35c227852
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/519821
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-10-20 11:28:21 -07:00
Mike Frysinger 2719a8e203 run_tests: log each command run
This should make it clear to devs what commands are run and which fail
in the CI.

Change-Id: Ie863540cba6de7da933b4f32947ad09edee4aa45
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/519361
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-10-15 11:09:48 -07:00
Jeroen Dhollander e4872ac8ba sync: Use 'git rebase' during 'repo sync --rebase'
'repo sync --rebase' should do a rebase if it encounters local commits
during a 'repo sync'.
This was broken by
https://gerrit-review.git.corp.google.com/c/git-repo/+/437421,
which caused this to execute the '_doff' hook (which stands for
'do fast forward'), which is implemented using 'git merge --no-stat'.

This caused *multiple* actual editor windows to pop up (*) during
'repo sync --rebase', asking the user to enter a commit message for the
merge.

In this CL I explicitly make that code path do a 'git rebase'.

(*) and if you use a terminal editor like 'vim', this means you have 2+ concurrent vim windows rendered in the same terminal, while 'repo sync' keeps on printing other output lines, again in the same terminal. The result is .... not pretty to say the least :(

Bug: b:434565811
Test: Used it myself for over a week.
Change-Id: I0bf3ff181f15b9d5b2e3f85f7f84e302139fdab7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/518602
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jeroen Dhollander <jeroendh@google.com>
Commit-Queue: Jeroen Dhollander <jeroendh@google.com>
2025-10-15 08:32:00 -07:00
Kaushik Lingarkar 4623264809 Fix submodule initialization in interleaved sync mode
With the introduction of interleaved sync mode, the submodule activation
logic broke because the 'has_submodules' attribute was no longer being
populated when needed. With this change, each submodule is initialized
when it enters the Sync_LocalHalf stage, whereas previously all
submodules were initialized at once when the parent repository entered
the Sync_LocalHalf stage. The init is now retried if it fails, as
submodules may concurrently modify the parent’s git config, potentially
causing contention when attempting to obtain a lock on it.

This change makes the submodule activation logic more robust and less
prone to breakage.

Bug: 444366154
Change-Id: I25eca4ea2a6868219045cfa088988eb01ded47d2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/509041
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Reviewed-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Commit-Queue: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Reviewed-by: Scott Lee <ddoman@google.com>
2025-10-14 12:07:04 -07:00
Kaushik Lingarkar 67383bdba9 Follow up "Fix shallow clones when upstream attribute is present"
This reverts commit 38d2fe11b9.

Reason for revert: The issue described in I00acd4c61 remains unresolved.
The previous fix incorrectly accessed use_superproject from the Project
class, though it was only defined in ManifestProject. This change uses
it from the manifest attr available in the Project class.

Bug: b/427093249
Change-Id: Ife6d46cd85840f2989f60c2ca4d5a7dcf5d7477a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/508821
Reviewed-by: Xin Li <delphij@google.com>
Reviewed-by: Krzysztof Wesolowski <krzysztof.wesolowski@volvocars.com>
Commit-Queue: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
2025-09-22 12:40:22 -07:00
Mike Frysinger d30414bb53 forall: fix crash with no command
When callback= is used, optparse does not automatically initialize
The destination when a dest= is not specified.  Refine the test to
allow dest= options when callback= is used even when it seems like
it is otherwise redundant.

Bug: b/436611422
Change-Id: I5185f95cb857ca6d37357cac77fb117a83db9c0c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/509861
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-09-17 12:54:30 -07:00
Mike Frysinger 80d1a5ad3e run_tests: add file header checker for licensing blocks
Change-Id: Ic0bfa3b03e2ba46d565a5bc2c1b7a7463b7dca2c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/500103
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
2025-08-21 11:16:35 -07:00
Mike Frysinger c615c964fb man: regen after sync updates
Change-Id: I20937c365b3f0be76e278d17c05b76a0d5e59deb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/500101
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2025-08-21 11:11:38 -07:00
Mike Frysinger 5ed12ec81d standardize file header wrt licensing
We've been slightly inconsistent in the license header in files.
Standardize them so we can automate checking.

Change-Id: I3cdf85c9485d33cac2bb05c8080dfada3e5a5e8d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/500102
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2025-08-21 11:04:41 -07:00
Mike Frysinger 58a59fdfbc CONTRIBUTING: rename doc per Google OSS policies
Google OSS policies say to name this "CONTRIBUTING.md".

Change-Id: I037f52a443caacc89868b7c14af91dd3d1b681a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/499761
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2025-08-20 14:15:53 -07:00
Gavin Mak 38d2fe11b9 Revert "Fix shallow clones when upstream attribute is present"
This reverts commit d9cc0a1526.

Reason for revert: AttributeError: 'Project' object has no attribute 'use_superproject'

Bug: b/427093249
Change-Id: I57b285ab21f58b040e68ec14b85425f43f0abcca
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498641
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2025-08-14 16:35:26 -07:00
Gavin Mak 854fe440f2 git_superproject: fix AttributeError in Superproject logging
Ensure _git_event_log is initialized before use in _LogMessage. This
avoids crashes when _git_event_log is accessed before it's set, such as
during repo info.

Bug: 435317391
Change-Id: I3adc32d6a9377558e852bbb43f9cf82041fcf1bc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498521
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
2025-08-14 15:39:41 -07:00
Gavin Mak d534a5537f sync: Fix missing error details in interleaved summary
When checkout errors occurred in interleaved sync, they were wrapped in
a SyncError with no message, causing blank lines in the final summary.
Refactor _SyncResult to hold a list of exceptions, ensuring the original
error messages are propagated correctly.

Bug: 438178765
Change-Id: Ic25e515068959829cb6290cfd9e4c2d3963bbbea
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498342
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2025-08-14 09:54:15 -07:00
Gavin Mak a64149a7a7 sync: Record and propagate errors from deferred actions
Failures in deferred sync actions were not recorded because `_Later.Run`
discarded the `GitError` exception. Record the specific error using
`syncbuf.fail()` and propagate it for proper error aggregation and
reporting.

Bug: 438178765
Change-Id: Iad59e389f9677bd6b8d873ee1ea2aa6ce44c86fa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498141
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
2025-08-13 23:17:56 -07:00
Gavin Mak 3e6acf2778 progress: Fix race condition causing fileno crash
A race condition occurs when sync redirects sys.stderr to capture worker output, while a background progress thread simultaneously calls fileno() on it. This causes an io.UnsupportedOperation error. Fix by caching the original sys.stderr for all progress bar IO.

Change-Id: Idb1f45d707596d31238a19fd373cac3bf669c405
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498121
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
2025-08-13 23:16:55 -07:00
Gavin Mak a6e1a59ac1 sync: Avoid duplicate projects in error text
Keep track of finished projects, not just successful ones, when deciding
which projects still need to be synced. Also project errors are already
reported by sync workers so stall detection doesn't need to add failed
projects to the error list.

Bug: 438178765
Change-Id: Ibf15aad009ba7295e70c8df2ff158215085e9732
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498062
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-08-13 23:16:55 -07:00
Gavin Mak 380bf9546e sync: always show sync result stderr_text on error
_ProcessSyncInterleavedResults currently only shows stderr_text if
verbose. Show it if a sync worker fails, regardless of verbosity.

Bug: 438178765
Change-Id: If24dcb10fb5d6857386782d371e3f9c6844dece9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498061
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-08-13 23:16:55 -07:00
Krzysztof Wesolowski d9cc0a1526 Fix shallow clones when upstream attribute is present
The _CheckForImmutableRevision method was modified in commit 0e776a58 to
include upstream branch validation for superproject scenarios. However,
this change inadvertently broke shallow clones when both clone-depth and
upstream attributes are specified in regular (non-superproject)
manifests.

Issue: When upstream is present, _CheckForImmutableRevision performs two
additional checks: 1. git rev-list on the upstream reference 2. git
merge-base --is-ancestor between revision and upstream

In shallow clones, the upstream branch history may not be available
locally, causing these checks to fail. This triggers the retry mechanism
that removes depth limitations, effectively converting shallow clones to
full clones, resulting in excessive disk usage.

Fix: Make upstream validation conditional on superproject usage. This
preserves the original superproject fix while restoring the method's
original behavior for regular scenarios - checking only if the immutable
revision (SHA1/tag) exists locally.

Note: The SetRevisionId method from the same commit 0e776a58 is left
unchanged as it only stores upstream information (no git operations),
which is beneficial for preserving branch context for commands like
'repo start' without causing fetch-related issues.

The fix ensures that manifests with both clone-depth and upstream work
correctly in non-superproject scenarios, maintaining shallow clone
efficiency and reducing disk usage.

Bug: b/427093249
Change-Id: I00acd4c61b179cd2abf796c2fecb7a2f38016a18
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/493883
Tested-by: Krzysztof Wesolowski <krzysztof.wesolowski@volvocars.com>
Commit-Queue: Krzysztof Wesolowski <krzysztof.wesolowski@volvocars.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Kamaljeet Maini <kamaljeet@google.com>
Reviewed-by: Xin Li <delphij@google.com>
2025-08-05 08:28:37 -07:00
Gavin Mak 8c3585f367 project: fallback to reading HEAD when rev-parse fails
git rev-parse fails on invalid HEAD, e.g. after incomplete sync, causing
NoManifestException. Fall back to v2.56's direct file reading when
rev-parse fails.

Bug: 435045466
Change-Id: Ia14560335110c00d80408b2a93595a84446f8a57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/495181
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-08-04 12:17:44 -07:00
92 changed files with 8085 additions and 2998 deletions
+16
View File
@@ -0,0 +1,16 @@
# GitHub actions workflow.
# https://help.github.com/en/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions
# https://black.readthedocs.io/en/stable/integrations/github_actions.html
name: Format
on:
push:
branches: [main]
jobs:
format:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: psf/black@stable
+1 -1
View File
@@ -18,5 +18,5 @@ jobs:
Thanks for your contribution!
Unfortunately, we don't use GitHub pull requests to manage code
contributions to this repository.
Instead, please see [README.md](../blob/HEAD/SUBMITTING_PATCHES.md)
Instead, please see [README.md](../blob/HEAD/CONTRIBUTING.md)
which provides full instructions on how to get involved.
+3 -3
View File
@@ -27,6 +27,6 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install tox tox-gh-actions
- name: Test with tox
run: tox
python -m pip install pytest
- name: Run tests
run: python -m pytest
+1
View File
@@ -6,6 +6,7 @@ __pycache__
/dist
.repopickle_*
/repoc
/.cipd_bin
/.tox
/.venv
-41
View File
@@ -1,41 +0,0 @@
# Copyright 2023 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Config file for the isort python module.
# This is used to enforce import sorting standards.
#
# https://pycqa.github.io/isort/docs/configuration/options.html
[settings]
# Be compatible with `black` since it also matches what we want.
profile = black
line_length = 80
length_sort = false
force_single_line = true
lines_after_imports = 2
from_first = false
case_sensitive = false
force_sort_within_sections = true
order_by_type = false
# Ignore generated files.
extend_skip_glob = *_pb2.py
# Allow importing multiple classes on a single line from these modules.
# https://google.github.io/styleguide/pyguide#s2.2-imports
single_line_exclusions =
abc,
collections.abc,
typing,
-1
View File
@@ -5,6 +5,5 @@
<pydev_pathproperty name="org.python.pydev.PROJECT_SOURCE_PATH">
<path>/git-repo</path>
</pydev_pathproperty>
<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python 2.7</pydev_property>
<pydev_property name="org.python.pydev.PYTHON_PROJECT_INTERPRETER">Default</pydev_property>
</pydev_project>
+8 -17
View File
@@ -24,7 +24,7 @@ However there are some differences, so please review and familiarize
yourself with the following relevant bits.
## Make separate commits for logically separate changes.
## Make separate commits for logically separate changes
Unless your patch is really trivial, you should not be sending out a patch that
was generated between your working tree and your commit head.
@@ -43,17 +43,12 @@ probably need to split up your commit to finer grained pieces.
Lint any changes by running:
```sh
$ tox -e lint -- file.py
$ flake8
```
And format with:
```sh
$ tox -e format -- file.py
```
Or format everything:
```sh
$ tox -e format
$ black file.py
```
Repo uses [black](https://black.readthedocs.io/) with line length of 80 as its
@@ -73,15 +68,11 @@ the entire project in the included `.flake8` file.
[PEP 8]: https://www.python.org/dev/peps/pep-0008/
[flake8 documentation]: https://flake8.pycqa.org/en/3.1.1/user/ignoring-errors.html#in-line-ignoring-errors
## Running tests
We use [pytest](https://pytest.org/) and [tox](https://tox.readthedocs.io/) for
running tests. You should make sure to install those first.
To run the full suite against all supported Python versions, simply execute:
```sh
$ tox -p auto
```
We use [pytest](https://pytest.org/) for running tests. You should make sure to
install that first.
We have [`./run_tests`](./run_tests) which is a simple wrapper around `pytest`:
```sh
@@ -127,7 +118,7 @@ your patch. It is virtually impossible to remove a patch once it
has been applied and pushed out.
## Sending your patches.
## Sending your patches
Do not email your patches to anyone.
@@ -143,7 +134,7 @@ they have right to redistribute your work under the Apache License:
Ensure you have obtained an HTTP password to authenticate:
https://gerrit-review.googlesource.com/new-password
https://www.googlesource.com/new-password
Ensure that you have the local commit hook installed to automatically
add a ChangeId to your commits:
+38 -1
View File
@@ -14,7 +14,7 @@ that you can put anywhere in your path.
* Docs: <https://source.android.com/source/using-repo.html>
* [repo Manifest Format](./docs/manifest-format.md)
* [repo Hooks](./docs/repo-hooks.md)
* [Submitting patches](./SUBMITTING_PATCHES.md)
* [Contributing](./CONTRIBUTING.md)
* Running Repo in [Microsoft Windows](./docs/windows.md)
* GitHub mirror: <https://github.com/GerritCodeReview/git-repo>
* Postsubmit tests: <https://github.com/GerritCodeReview/git-repo/actions>
@@ -49,6 +49,43 @@ $ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/.bin/repo
$ chmod a+rx ~/.bin/repo
```
## Shell Completion
Repo includes completion scripts for Bash and Zsh.
### Bash
To enable completion in Bash, source `completion.bash` in your `~/.bashrc`:
```sh
source /path/to/git-repo/completion.bash
```
### Zsh
To enable completion in Zsh, you can either:
1. Copy or symlink `completion.zsh` to a file named `_repo` in a directory in your `$fpath`:
```sh
mkdir -p ~/.zsh/completion
# You can copy the file:
cp /path/to/git-repo/completion.zsh ~/.zsh/completion/_repo
# Or symlink it:
ln -s /path/to/git-repo/completion.zsh ~/.zsh/completion/_repo
```
Then add that directory to your `fpath` in `~/.zshrc` before `compinit`:
```zsh
fpath=(~/.zsh/completion $fpath)
autoload -Uz compinit
compinit
```
2. Or source the file directly and call `compdef` in your `~/.zshrc`:
```zsh
source /path/to/git-repo/completion.zsh
compdef _repo repo
```
[new-bug]: https://issues.gerritcodereview.com/issues/new?component=1370071
[issue tracker]: https://issues.gerritcodereview.com/issues?q=is:open%20componentid:1370071
+32
View File
@@ -0,0 +1,32 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file contains version pins of a few tools used by run_tests.
# Pin resolved versions in the repo, to reduce trust in the CIPD backend.
#
# Most of these tools are generated via builders at
# https://ci.chromium.org/p/infra/g/infra/console
#
# For these, the git revision is the one of
# https://chromium.googlesource.com/infra/infra.git.
#
# To regenerate them (after modifying this file):
# cipd ensure-file-resolve -ensure-file cipd_manifest.txt
$ResolvedVersions cipd_manifest.versions
# Supported platforms. Feel free to add more as long as the tools work.
$VerifiedPlatform linux-amd64 linux-arm64 mac-amd64 mac-arm64
infra/3pp/tools/help2man/${platform} version:3@1.49.3
+18
View File
@@ -0,0 +1,18 @@
# This file is auto-generated by 'cipd ensure-file-resolve'.
# Do not modify manually. All changes will be overwritten.
infra/3pp/tools/help2man/linux-amd64
version:3@1.49.3
82ySLC4--mvjnmuwj_j_2odHLj5zFaNFpWvfCyMunY0C
infra/3pp/tools/help2man/linux-arm64
version:3@1.49.3
szfWfjI4RyCT6gIB48CeUOGG6_Un4wfbIb0Vvs5M9MUC
infra/3pp/tools/help2man/mac-amd64
version:3@1.49.3
t7iyQx5TnGbuJfj6zMTvjPdBl-Cb_w5papUh80NI2S8C
infra/3pp/tools/help2man/mac-arm64
version:3@1.49.3
t5eD-eLKRpnSifZdbi072U8bnPfGye1pIMmQ07nxJ7kC
+4 -3
View File
@@ -14,6 +14,7 @@
import os
import sys
from typing import Optional
import pager
@@ -84,7 +85,7 @@ def _Color(fg=None, bg=None, attr=None):
DEFAULT = None
def SetDefaultColoring(state):
def SetDefaultColoring(state: Optional[str]) -> None:
"""Set coloring behavior to |state|.
This is useful for overriding config options via the command line.
@@ -97,9 +98,9 @@ def SetDefaultColoring(state):
state = state.lower()
if state in ("auto",):
DEFAULT = state
elif state in ("always", "yes", "true", True):
elif state in ("always", "yes", "true"):
DEFAULT = "always"
elif state in ("never", "no", "false", False):
elif state in ("never", "no", "false"):
DEFAULT = "never"
+24 -2
View File
@@ -101,6 +101,11 @@ class Command:
def WantPager(self, _opt):
return False
@staticmethod
def is_multiprocessing_active() -> bool:
"""Whether the current process is a worker in a pool."""
return multiprocessing.current_process().name != "MainProcess"
def ReadEnvironmentOptions(self, opts):
"""Set options from environment variables."""
@@ -399,7 +404,7 @@ class Command:
result = []
if not groups:
groups = manifest.GetGroupsStr()
groups = manifest.GetManifestGroupsStr()
groups = [x for x in re.split(r"[,\s]+", groups) if x]
if not args:
@@ -407,7 +412,8 @@ class Command:
for project in all_projects_list:
if submodules_ok or project.sync_s:
derived_projects.update(
(p.name, p) for p in project.GetDerivedSubprojects()
(p.RelPath(local=False), p)
for p in project.GetDerivedSubprojects()
)
all_projects_list.extend(derived_projects.values())
for project in all_projects_list:
@@ -511,6 +517,22 @@ class Command:
)
return result
def GetSmartSyncOverridePath(self, manifest=None) -> str:
"""Return the path where smart sync writes its override manifest."""
if manifest is None:
manifest = self.manifest
return os.path.join(
manifest.manifestProject.worktree, "smart_sync_override.xml"
)
def TryOverrideManifestWithSmartSync(self, manifest=None) -> None:
"""Override manifest with smart_sync_override.xml if it exists."""
if manifest is None:
manifest = self.manifest
smart_sync_manifest_path = self.GetSmartSyncOverridePath(manifest)
if os.path.isfile(smart_sync_manifest_path):
manifest.Override(smart_sync_manifest_path)
def ManifestList(self, opt):
"""Yields all of the manifests to traverse.
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright 2021 The Android Open Source Project
# Copyright (C) 2021 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
+449
View File
@@ -0,0 +1,449 @@
#compdef _repo repo
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
_repo() {
local context state line
typeset -A opt_args
local -a subcommands
subcommands=(
'abandon:Abandon a development branch'
'branches:View current topic branches'
'checkout:Checkout a branch'
'cherry-pick:Cherry-pick a change'
'diff:Show changes between commit and working tree'
'diffmanifests:Show differences between two manifests'
'download:Download and find a change'
'forall:Run a shell command in each project'
'gc:Garbage collect'
'grep:Print lines matching a pattern'
'help:Display help about a command'
'info:Get info about the manifest'
'init:Initialize a repo client'
'list:List projects and their names'
'manifest:Manifest management'
'overview:Display overview of topic branches'
'prune:Prune topic branches'
'rebase:Rebase local branches'
'selfupdate:Update repo'
'smartsync:Update working tree to latest known good revision'
'stage:Stage file(s) for commit'
'start:Start a new branch for development'
'status:Show the working tree status'
'sync:Update working tree'
'upload:Upload changes to the review server'
'version:Display version'
'wipe:Wipe uncommitted changes'
)
local -a global_opts
global_opts=(
'(-h --help)'{-h,--help}'[Show help message]'
'--help-all[Show help message for all commands]'
'(-p --paginate)'{-p,--paginate}'[Pipe all output into less]'
'--no-pager[Do not pipe output into a pager]'
'--color=[Control color output]:color:(always never auto)'
'--trace[Trace git command execution]'
'--trace-to-stderr[Trace git command execution to stderr]'
'--trace-python[Trace python execution]'
'--time[Time execute of command]'
'--version[Show version]'
'--show-toplevel[Show top level of workspace]'
'--event-log=[File to log events to]:file:_files'
'--git-trace2-event-log=[File to log git trace2 events to]:file:_files'
'--submanifest-path=[Path to submanifest]:path:_files'
)
_arguments -C \
${global_opts} \
'1: :->cmds' \
'*:: :->subcmds'
case ${state} in
cmds)
_describe -t commands 'repo command' subcommands
;;
subcmds)
local cmd=${words[1]}
curcontext="${curcontext%:*}-${cmd}:"
local -a common_opts
common_opts=(
'(-h --help)'{-h,--help}'[Show help message]'
'(-v --verbose)'{-v,--verbose}'[Show verbose messages]'
'(-q --quiet)'{-q,--quiet}'[Show only errors]'
'(-j --jobs)'{-j,--jobs=}'[Number of jobs to run in parallel]:jobs:'
'--outer-manifest[Use outer manifest]'
'--no-outer-manifest[Do not use outer manifest]'
'--this-manifest-only[Only use this manifest]'
'--no-this-manifest-only[Do not only use this manifest]'
'--all-manifests[Use all manifests]'
)
case ${cmd} in
abandon)
_arguments \
${common_opts} \
'--all[Abandon all branches]' \
'1: :->branch' \
'*: :->project'
;;
branches)
_arguments \
${common_opts} \
'*: :->project'
;;
checkout)
_arguments \
${common_opts} \
'1: :->branch' \
'*: :->project'
;;
cherry-pick)
_arguments \
${common_opts} \
'1: :_message "sha1"'
;;
diff)
_arguments \
${common_opts} \
'(-u --absolute)'{-u,--absolute}'[Show absolute paths]' \
'*: :->project'
;;
diffmanifests)
_arguments \
'--raw[Show raw diff]' \
'--no-color[Do not show color]' \
'--pretty-format=[Pretty format]:format:' \
'1: :_files -g "*.xml"' \
'2: :_files -g "*.xml"'
;;
download)
_arguments \
${common_opts} \
'(-b --branch)'{-b,--branch=}'[One or more branches to check]:branch:' \
'(-c --cherry-pick)'{-c,--cherry-pick}'[Cherry-pick the change]' \
'(-x --record-origin)'{-x,--record-origin}'[Record origin of cherry-pick]' \
'(-r --revert)'{-r,--revert}'[Revert the change]' \
'(-f --ff-only)'{-f,--ff-only}'[Force fast-forward]' \
'*: :_message "change[/patchset]"'
;;
forall)
_arguments \
${common_opts} \
'(-r --regex)'{-r,--regex}'[Execute command only on projects matching regex]' \
'(-i --inverse-regex)'{-i,--inverse-regex}'[Execute command only on projects not matching regex]' \
'(-g --groups)'{-g,--groups=}'[Execute command only on projects matching groups]:groups:' \
'(-c --command)'{-c,--command}'[Command to execute]' \
'(-e --abort-on-errors)'{-e,--abort-on-errors}'[Abort on errors]' \
'--ignore-missing[Ignore missing projects]' \
'--interactive[Run interactively]' \
'-p[Show project headers]' \
'*: :->project'
;;
gc)
_arguments \
${common_opts} \
'(-n --dry-run)'{-n,--dry-run}'[Dry run]' \
'(-y --yes)'{-y,--yes}'[Answer yes to all prompts]' \
'--repack[Repack objects]'
;;
grep)
_arguments \
${common_opts} \
'--cached[Search cached files]' \
'(-r --revision)'{-r,--revision=}'[Search in revision]:revision:' \
'-e[Pattern]' \
'(-i --ignore-case)'{-i,--ignore-case}'[Ignore case]' \
'(-a --text)'{-a,--text}'[Treat all files as text]' \
'-I[Do not match binary files]' \
'(-w --word-regexp)'{-w,--word-regexp}'[Match word boundaries]' \
'(-v --invert-match)'{-v,--invert-match}'[Invert match]' \
'(-G --basic-regexp)'{-G,--basic-regexp}'[Basic regexp]' \
'(-E --extended-regexp)'{-E,--extended-regexp}'[Extended regexp]' \
'(-F --fixed-strings)'{-F,--fixed-strings}'[Fixed strings]' \
'--all-match[All match]' \
'--and[And]' \
'--or[Or]' \
'--not[Not]' \
'-([Open paren]' \
'-)[Close paren]' \
'-n[Line number]' \
'-C[Context]:lines:' \
'-B[Before context]:lines:' \
'-A[After context]:lines:' \
'-l[Name only]' \
'--name-only[Name only]' \
'--files-with-matches[Files with matches]' \
'-L[Files without match]' \
'--files-without-match[Files without match]' \
'1: :->pattern' \
'*: :->project'
;;
help)
_arguments \
'(-a --all)'{-a,--all}'[Show all commands]' \
'--help-all[Show help message for all commands]' \
'1: :->help_cmds'
;;
info)
_arguments \
${common_opts} \
'(-d --diff)'{-d,--diff}'[Show diff]' \
'(-o --overview)'{-o,--overview}'[Show overview]' \
'(-c --current-branch)'{-c,--current-branch}'[Show current branch]' \
'--no-current-branch[Do not show current branch]' \
'(-l --local-only)'{-l,--local-only}'[Local only]' \
'*: :->project'
;;
init)
_arguments \
'(-u --manifest-url)'{-u,--manifest-url=}'[Manifest URL]:url:' \
'(-b --manifest-branch)'{-b,--manifest-branch=}'[Manifest branch]:branch:' \
'--manifest-upstream-branch=[Manifest upstream branch]:branch:' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'(-g --groups)'{-g,--groups=}'[Restrict manifest projects to groups]:groups:' \
'(-p --platform)'{-p,--platform=}'[Restrict manifest projects to platform]:platform:' \
'--submodules[Sync submodules]' \
'--standalone-manifest[Standalone manifest]' \
'--manifest-depth=[Manifest depth]:depth:' \
'(-c --current-branch)'{-c,--current-branch}'[Sync current branch only]' \
'--no-current-branch[Do not sync current branch only]' \
'--tags[Sync tags]' \
'--no-tags[Do not sync tags]' \
'--mirror[Mirror]' \
'--archive[Archive]' \
'--worktree[Worktree]' \
'--reference=[Reference repository]:repository:_files -/' \
'--dissociate[Dissociate from reference]' \
'--depth=[Depth]:depth:' \
'--partial-clone[Partial clone]' \
'--no-partial-clone[Do not partial clone]' \
'--partial-clone-exclude=[Exclude from partial clone]:projects:' \
'--clone-filter=[Clone filter]:filter:' \
'--use-superproject[Use superproject]' \
'--no-use-superproject[Do not use superproject]' \
'--clone-bundle[Use clone bundle]' \
'--no-clone-bundle[Do not use clone bundle]' \
'--git-lfs[Use git lfs]' \
'--no-git-lfs[Do not use git lfs]' \
'--repo-url=[Repo URL]:url:' \
'--repo-rev=[Repo revision]:revision:' \
'--no-repo-verify[Do not verify repo]' \
'--config-name[Use config name]'
;;
list)
_arguments \
${common_opts} \
'(-r --regex)'{-r,--regex}'[Filter by regex]' \
'(-g --groups)'{-g,--groups=}'[Filter by groups]:groups:' \
'(-a --all)'{-a,--all}'[Show all projects]' \
'(-n --name-only)'{-n,--name-only}'[Show only names]' \
'(-p --path-only)'{-p,--path-only}'[Show only paths]' \
'(-f --fullpath)'{-f,--fullpath}'[Show full paths]' \
'--relative-to=[Show paths relative to]:path:_files -/' \
'*: :->project'
;;
manifest)
_arguments \
'(-r --revision-as-HEAD)'{-r,--revision-as-HEAD}'[Save revisions as current HEAD]' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'--suppress-upstream-revision[Suppress upstream revision]' \
'--suppress-dest-branch[Suppress dest branch]' \
'--format=[Output format]:format:(xml json)' \
'--pretty[Pretty print]' \
'--no-local-manifests[Ignore local manifests]' \
'(-o --output-file)'{-o,--output-file=}'[Output file]:file:_files'
;;
overview)
_arguments \
${common_opts} \
'(-c --current-branch)'{-c,--current-branch}'[Show current branch]' \
'--no-current-branch[Do not show current branch]' \
'*: :->project'
;;
prune)
_arguments \
${common_opts} \
'*: :->project'
;;
rebase)
_arguments \
${common_opts} \
'--fail-fast[Fail fast]' \
'(-f --force-rebase)'{-f,--force-rebase}'[Force rebase]' \
'--no-ff[No fast forward]' \
'--autosquash[Autosquash]' \
'--whitespace=[Whitespace option]:option:' \
'--auto-stash[Auto stash]' \
'(-m --onto-manifest)'{-m,--onto-manifest}'[Rebase onto manifest]' \
'(-i --interactive)'{-i,--interactive}'[Interactive rebase]' \
'*: :->project'
;;
selfupdate)
_arguments \
'--no-repo-verify[Do not verify repo]'
;;
smartsync | sync)
_arguments \
${common_opts} \
'--jobs-network=[Number of network jobs]:jobs:' \
'--jobs-checkout=[Number of checkout jobs]:jobs:' \
'(-f --force-broken)'{-f,--force-broken}'[Continue sync even if a project fails]' \
'--fail-fast[Fail fast]' \
'--force-sync[Overwrite existing git directories]' \
'--force-checkout[Force checkout]' \
'--force-remove-dirty[Force remove dirty]' \
'--rebase[Rebase local branches]' \
'(-l --local-only)'{-l,--local-only}'[Only use local files]' \
'--no-manifest-update[Do not update manifest]' \
'--nmu[Do not update manifest]' \
'--interleaved[Interleave output]' \
'--no-interleaved[Do not interleave output]' \
'(-n --network-only)'{-n,--network-only}'[Only fetch]' \
'(-d --detach)'{-d,--detach}'[Detach HEAD]' \
'(-c --current-branch)'{-c,--current-branch}'[Fetch only current branch]' \
'--no-current-branch[Do not fetch only current branch]' \
'(-m --manifest-name)'{-m,--manifest-name=}'[Manifest name]:file:_files -g "*.xml"' \
'--clone-bundle[Use clone bundle]' \
'--no-clone-bundle[Do not use clone bundle]' \
'(-u --manifest-server-username)'{-u,--manifest-server-username=}'[Username for manifest server]:username:' \
'(-p --manifest-server-password)'{-p,--manifest-server-password=}'[Password for manifest server]:password:' \
'--fetch-submodules[Fetch submodules]' \
'--use-superproject[Use superproject]' \
'--no-use-superproject[Do not use superproject]' \
'--tags[Sync tags]' \
'--no-tags[Do not sync tags]' \
'--optimized-fetch[Optimized fetch]' \
'--retry-fetches=[Retry fetches]:count:' \
'--prune[Prune]' \
'--no-prune[Do not prune]' \
'--auto-gc[Run auto gc]' \
'--no-auto-gc[Do not run auto gc]' \
'(-s --smart-sync)'{-s,--smart-sync}'[Smart sync]' \
'(-t --smart-tag)'{-t,--smart-tag=}'[Smart tag]:tag:' \
'--no-repo-verify[Do not verify repo]' \
'--no-verify[Do not verify]' \
'--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \
'*: :->project'
;;
stage)
_arguments \
${common_opts} \
'(-i --interactive)'{-i,--interactive}'[Interactive staging]' \
'*: :->project'
;;
start)
_arguments \
${common_opts} \
'--all[Start branch in all projects]' \
'(-r --rev --revision)'{-r,--rev=,--revision=}'[Revision]:revision:' \
'--head[Head]' \
'--HEAD[Head]' \
'1: :->newbranch' \
'*: :->project'
;;
status)
_arguments \
${common_opts} \
'(-o --orphans)'{-o,--orphans}'[Show orphans]' \
'*: :->project'
;;
upload)
_arguments \
${common_opts} \
'(-t --topic-branch)'{-t,--topic-branch}'[Topic branch]' \
'--topic=[Topic]:topic:' \
'--hashtag=[Hashtag]:hashtag:' \
'--ht=[Hashtag]:hashtag:' \
'--hashtag-branch[Hashtag branch]' \
'--htb[Hashtag branch]' \
'(-l --label)'{-l,--label=}'[Label]:label:' \
'--pd=[Patchset description]:description:' \
'--patchset-description=[Patchset description]:description:' \
'--re=[Reviewers]:reviewers:' \
'--reviewers=[Reviewers]:reviewers:' \
'--cc=[CC]:cc:' \
'--br=[Branch]:branch:' \
'--branch=[Branch]:branch:' \
'(-c --current-branch)'{-c,--current-branch}'[Upload current branch]' \
'--no-current-branch[Do not upload current branch]' \
'--ne[No emails]' \
'--no-emails[No emails]' \
'(-p --private)'{-p,--private}'[Private]' \
'(-w --wip)'{-w,--wip}'[Work in progress]' \
'(-r --ready)'{-r,--ready}'[Ready]' \
'(-o --push-option)'{-o,--push-option=}'[Push option]:option:' \
'(-D --destination --dest)'{-D,--destination=,--dest=}'[Destination]:destination:' \
'(-n --dry-run)'{-n,--dry-run}'[Dry run]' \
'(-y --yes)'{-y,--yes}'[Answer yes to all prompts]' \
'--ignore-untracked-files[Ignore untracked files]' \
'--no-ignore-untracked-files[Do not ignore untracked files]' \
'--no-cert-checks[Do not check certificates]' \
'--no-verify[Do not verify]' \
'--verify[Verify]' \
'--ignore-hooks[Ignore hooks]' \
'*: :->project'
;;
version)
_arguments ${common_opts}
;;
wipe)
_arguments \
${common_opts} \
'(-f --force)'{-f,--force}'[Force wipe]' \
'--force-uncommitted[Force uncommitted]' \
'--force-shared[Force shared]' \
'*: :->project'
;;
esac
# Handle states for positional arguments.
case ${state} in
branch)
[[ ${PREFIX} != -* ]] && _repo_branches
;;
project)
[[ ${PREFIX} != -* ]] && _repo_projects
;;
pattern)
_message 'pattern'
;;
newbranch)
_message 'new branch name'
;;
help_cmds)
_describe -t commands 'repo command' subcommands
;;
esac
;;
esac
}
_repo_branches() {
local -a branches
branches=(${(f)"$(_call_program branches repo branches 2>/dev/null | sed -E 's/^.*[[:space:]]([^[:space:]]+)[[:space:]]+\|.*$/\1/' | awk '{print $1}')"})
_describe -t branches 'branch' branches
}
_repo_projects() {
local -a projects
projects=(${(f)"$(_call_program projects repo list -n 2>/dev/null)"})
_describe -t projects 'project' projects
}
_repo "$@"
+81 -41
View File
@@ -51,6 +51,7 @@ following DTD:
<!ATTLIST default dest-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync-j CDATA #IMPLIED>
<!ATTLIST default sync-j-max CDATA #IMPLIED>
<!ATTLIST default sync-c CDATA #IMPLIED>
<!ATTLIST default sync-s CDATA #IMPLIED>
<!ATTLIST default sync-tags CDATA #IMPLIED>
@@ -59,7 +60,7 @@ following DTD:
<!ATTLIST manifest-server url CDATA #REQUIRED>
<!ELEMENT submanifest EMPTY>
<!ATTLIST submanifest name ID #REQUIRED>
<!ATTLIST submanifest name ID #REQUIRED>
<!ATTLIST submanifest remote IDREF #IMPLIED>
<!ATTLIST submanifest project CDATA #IMPLIED>
<!ATTLIST submanifest manifest-name CDATA #IMPLIED>
@@ -72,18 +73,19 @@ following DTD:
project*,
copyfile*,
linkfile*)>
<!ATTLIST project name CDATA #REQUIRED>
<!ATTLIST project path CDATA #IMPLIED>
<!ATTLIST project remote IDREF #IMPLIED>
<!ATTLIST project revision CDATA #IMPLIED>
<!ATTLIST project dest-branch CDATA #IMPLIED>
<!ATTLIST project groups CDATA #IMPLIED>
<!ATTLIST project sync-c CDATA #IMPLIED>
<!ATTLIST project sync-s CDATA #IMPLIED>
<!ATTLIST project sync-tags CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project clone-depth CDATA #IMPLIED>
<!ATTLIST project force-path CDATA #IMPLIED>
<!ATTLIST project name CDATA #REQUIRED>
<!ATTLIST project path CDATA #IMPLIED>
<!ATTLIST project remote IDREF #IMPLIED>
<!ATTLIST project revision CDATA #IMPLIED>
<!ATTLIST project dest-branch CDATA #IMPLIED>
<!ATTLIST project groups CDATA #IMPLIED>
<!ATTLIST project sync-c CDATA #IMPLIED>
<!ATTLIST project sync-s CDATA #IMPLIED>
<!ATTLIST project sync-tags CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project clone-depth CDATA #IMPLIED>
<!ATTLIST project force-path CDATA #IMPLIED>
<!ATTLIST project sync-strategy CDATA #IMPLIED>
<!ELEMENT annotation EMPTY>
<!ATTLIST annotation name CDATA #REQUIRED>
@@ -95,19 +97,21 @@ following DTD:
<!ATTLIST copyfile dest CDATA #REQUIRED>
<!ELEMENT linkfile EMPTY>
<!ATTLIST linkfile src CDATA #REQUIRED>
<!ATTLIST linkfile src CDATA #REQUIRED>
<!ATTLIST linkfile dest CDATA #REQUIRED>
<!ELEMENT extend-project EMPTY>
<!ATTLIST extend-project name CDATA #REQUIRED>
<!ATTLIST extend-project path CDATA #IMPLIED>
<!ATTLIST extend-project dest-path CDATA #IMPLIED>
<!ATTLIST extend-project groups CDATA #IMPLIED>
<!ATTLIST extend-project revision CDATA #IMPLIED>
<!ATTLIST extend-project remote CDATA #IMPLIED>
<!ELEMENT extend-project (annotation*,
copyfile*,
linkfile*)>
<!ATTLIST extend-project name CDATA #REQUIRED>
<!ATTLIST extend-project path CDATA #IMPLIED>
<!ATTLIST extend-project dest-path CDATA #IMPLIED>
<!ATTLIST extend-project groups CDATA #IMPLIED>
<!ATTLIST extend-project revision CDATA #IMPLIED>
<!ATTLIST extend-project remote CDATA #IMPLIED>
<!ATTLIST extend-project dest-branch CDATA #IMPLIED>
<!ATTLIST extend-project upstream CDATA #IMPLIED>
<!ATTLIST extend-project base-rev CDATA #IMPLIED>
<!ATTLIST extend-project upstream CDATA #IMPLIED>
<!ATTLIST extend-project base-rev CDATA #IMPLIED>
<!ELEMENT remove-project EMPTY>
<!ATTLIST remove-project name CDATA #IMPLIED>
@@ -116,7 +120,7 @@ following DTD:
<!ATTLIST remove-project base-rev CDATA #IMPLIED>
<!ELEMENT repo-hooks EMPTY>
<!ATTLIST repo-hooks in-project CDATA #REQUIRED>
<!ATTLIST repo-hooks in-project CDATA #REQUIRED>
<!ATTLIST repo-hooks enabled-list CDATA #REQUIRED>
<!ELEMENT superproject EMPTY>
@@ -125,7 +129,7 @@ following DTD:
<!ATTLIST superproject revision CDATA #IMPLIED>
<!ELEMENT contactinfo EMPTY>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
<!ELEMENT include EMPTY>
<!ATTLIST include name CDATA #REQUIRED>
@@ -211,7 +215,9 @@ can be found. Used when syncing a revision locked manifest in
-c mode to avoid having to sync the entire ref space. Project elements
not setting their own `upstream` will inherit this value.
Attribute `sync-j`: Number of parallel jobs to use when synching.
Attribute `sync-j`: Number of parallel jobs to use when syncing.
Attribute `sync-j-max`: Maximum number of parallel jobs to use when syncing.
Attribute `sync-c`: Set to true to only sync the given Git
branch (specified in the `revision` attribute) rather than the
@@ -285,7 +291,7 @@ should be placed. If not supplied, `revision` is used.
`path` may not be an absolute path or use "." or ".." path components.
Attribute `groups`: List of additional groups to which all projects
Attribute `groups`: Set of additional groups to which all projects
in the included submanifest belong. This appends and recurses, meaning
all projects in submanifests carry all parent submanifest groups.
Same syntax as the corresponding element of `project`.
@@ -353,7 +359,7 @@ When using `repo upload`, changes will be submitted for code
review on this branch. If unspecified both here and in the
default element, `revision` is used instead.
Attribute `groups`: List of groups to which this project belongs,
Attribute `groups`: Set of groups to which this project belongs,
whitespace or comma separated. All projects belong to the group
"all", and each project automatically belongs to a group of
its name:`name` and path:`path`. E.g. for
@@ -384,6 +390,22 @@ rather than the `name` attribute. This attribute only applies to the
local mirrors syncing, it will be ignored when syncing the projects in a
client working directory.
Attribute `sync-strategy`: Set the sync strategy used when fetching this
project. Currently the only supported value is `stateless`. When set to
`stateless`, repo will run a reflog expiration and aggressive garbage collection
at the end of the sync process. This is useful for projects that contain
large binary files and use `clone-depth="1"`, where garbage can accumulate
as binaries are added, deleted, or modified across successive syncs.
During a stateless sync, repo checks the following before cleaning up:
1. The project does not share an object directory with other projects.
2. The working tree is clean (no uncommitted changes, no untracked files).
3. There are no unpushed local commits.
4. There is no Git stash.
If any of these conditions are not met, repo falls back to a standard
sync without garbage collection.
### Element extend-project
Modify the attributes of the named project.
@@ -393,6 +415,11 @@ attributes of an existing project without completely replacing the
existing project definition. This makes the local manifest more robust
against changes to the original manifest.
The `extend-project` element can also contain `annotation`, `copyfile`, and
`linkfile` child elements. These are added to the project's definition. A
`copyfile` or `linkfile` with a `dest` that already exists in the project
will overwrite the original.
Attribute `path`: If specified, limit the change to projects checked out
at the specified path, rather than all projects with the given name.
@@ -401,7 +428,7 @@ of the repo client where the Git working directory for this project
should be placed. This is used to move a project in the checkout by
overriding the existing `path` setting.
Attribute `groups`: List of additional groups to which this project
Attribute `groups`: Set of additional groups to which this project
belongs. Same syntax as the corresponding element of `project`.
Attribute `revision`: If specified, overrides the revision of the original
@@ -427,19 +454,20 @@ Same syntax as the corresponding element of `project`.
### Element annotation
Zero or more annotation elements may be specified as children of a
project or remote element. Each element describes a name-value pair.
For projects, this name-value pair will be exported into each project's
environment during a 'forall' command, prefixed with `REPO__`. In addition,
there is an optional attribute "keep" which accepts the case insensitive values
"true" (default) or "false". This attribute determines whether or not the
project element, an extend-project element, or a remote element. Each
element describes a name-value pair. For projects, this name-value pair
will be exported into each project's environment during a 'forall'
command, prefixed with `REPO__`. In addition, there is an optional
attribute "keep" which accepts the case insensitive values "true"
(default) or "false". This attribute determines whether or not the
annotation will be kept when exported with the manifest subcommand.
### Element copyfile
Zero or more copyfile elements may be specified as children of a
project element. Each element describes a src-dest pair of files;
the "src" file will be copied to the "dest" place during `repo sync`
command.
project element, or an extend-project element. Each element describes a
src-dest pair of files; the "src" file will be copied to the "dest"
place during `repo sync` command.
"src" is project relative, "dest" is relative to the top of the tree.
Copying from paths outside of the project or to paths outside of the repo
@@ -450,10 +478,14 @@ Intermediate paths must not be symlinks either.
Parent directories of "dest" will be automatically created if missing.
The files are copied in the order they are specified in the manifests.
If multiple elements specify the same source and destination, they will
only be applied as one, based on the first occurence. Files are copied
before any links specified via linkfile elements are created.
### Element linkfile
It's just like copyfile and runs at the same time as copyfile but
instead of copying it creates a symlink.
It's just like copyfile, but instead of copying it creates a symlink.
The symlink is created at "dest" (relative to the top of the tree) and
points to the path specified by "src" which is a path in the project.
@@ -463,6 +495,11 @@ Parent directories of "dest" will be automatically created if missing.
The symlink target may be a file or directory, but it may not point outside
of the repo client.
The links are created in the order they are specified in the manifests.
If multiple elements specify the same source and destination, they will
only be applied as one, based on the first occurence. Links are created
after any files specified via copyfile elements are copied.
### Element remove-project
Deletes a project from the internal manifest table, possibly
@@ -560,13 +597,16 @@ the manifest repository's root.
"name" may not be an absolute path or use "." or ".." path components.
These restrictions are not enforced for [Local Manifests].
Attribute `groups`: List of additional groups to which all projects
Attribute `groups`: Set of additional groups to which all projects
in the included manifest belong. This appends and recurses, meaning
all projects in included manifests carry all parent include groups.
This also applies to all extend-project elements in the included manifests.
Same syntax as the corresponding element of `project`.
Attribute `revision`: Name of a Git branch (e.g. `main` or `refs/heads/main`)
default to which all projects in the included manifest belong.
default to which all projects in the included manifest belong. This recurses,
meaning it will apply to all projects in all manifests included as a result of
this element.
## Local Manifests {#local-manifests}
+2 -1
View File
@@ -163,13 +163,14 @@ Example:
The `post-sync.py` file should be defined like:
```py
def main(repo_topdir=None, **kwargs):
def main(repo_topdir=None, sync_duration_seconds=None, **kwargs):
"""Main function invoked directly by repo.
We must use the name "main" as that is what repo requires.
Args:
repo_topdir: The absolute path to the top-level directory of the repo workspace.
sync_duration_seconds: The duration of the sync operation in seconds.
kwargs: Leave this here for forward-compatibility.
"""
```
+6 -3
View File
@@ -50,8 +50,11 @@ Git worktrees (see the previous section for more info).
Repo will use symlinks heavily internally.
On *NIX platforms, this isn't an issue, but Windows makes it a bit difficult.
There are some documents out there for how to do this, but usually the easiest
answer is to run your shell as an Administrator and invoke repo/git in that.
The easiest method to allow users to create symlinks is by enabling
[Windows Developer Mode](https://learn.microsoft.com/en-us/windows/advanced-settings/developer-mode).
The next easiest answer is to run your shell as an Administrator and invoke
repo/git in that.
This isn't a great solution, but Windows doesn't make this easy, so here we are.
@@ -142,7 +145,7 @@ You will have to specify this flag every time you upload.
[mintty]: https://github.com/mintty/mintty/issues/56
### repohooks always fail with an close_fds error.
### repohooks always fail with an close_fds error
When using the [reference repohooks project][repohooks] included in AOSP,
you might see errors like this when running `repo upload`:
+4 -3
View File
@@ -34,7 +34,7 @@ def fetch_file(url, verbose=False):
"""
scheme = urlparse(url).scheme
if scheme == "gs":
cmd = ["gsutil", "cat", url]
cmd = ["gcloud", "storage", "cat", url]
errors = []
try:
result = subprocess.run(
@@ -42,7 +42,7 @@ def fetch_file(url, verbose=False):
)
if result.stderr and verbose:
print(
'warning: non-fatal error running "gsutil": %s'
'warning: non-fatal error running "gcloud storage": %s'
% result.stderr,
file=sys.stderr,
)
@@ -50,7 +50,8 @@ def fetch_file(url, verbose=False):
except subprocess.CalledProcessError as e:
errors.append(e)
print(
'fatal: error running "gsutil": %s' % e.stderr, file=sys.stderr
'fatal: error running "gcloud storage": %s' % e.stderr,
file=sys.stderr,
)
raise FetchFileError(aggregate_errors=errors)
with urlopen(url) as f:
+4 -5
View File
@@ -22,7 +22,6 @@ from typing import Any, Optional
from error import GitError
from error import RepoExitError
from git_refs import HEAD
from git_trace2_event_log_base import BaseEventLog
import platform_utils
from repo_logging import RepoLogger
@@ -48,7 +47,7 @@ logger = RepoLogger(__file__)
class _GitCall:
@functools.lru_cache(maxsize=None)
@functools.lru_cache(maxsize=None) # noqa: B019
def version_tuple(self):
ret = Wrapper().ParseGitVersion()
if ret is None:
@@ -83,7 +82,7 @@ def RepoSourceVersion():
proj = os.path.dirname(os.path.abspath(__file__))
env[GIT_DIR] = os.path.join(proj, ".git")
result = subprocess.run(
[GIT, "describe", HEAD],
[GIT, "describe", "HEAD"],
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
encoding="utf-8",
@@ -96,7 +95,7 @@ def RepoSourceVersion():
ver = ver[1:]
else:
ver = "unknown"
setattr(RepoSourceVersion, "version", ver)
RepoSourceVersion.version = ver
return ver
@@ -612,7 +611,7 @@ class GitCommandError(GitError):
self.git_stderr = git_stderr
@property
@functools.lru_cache(maxsize=None)
@functools.lru_cache(maxsize=None) # noqa: B019
def suggestion(self):
"""Returns helpful next steps for the given stderr."""
if not self.git_stderr:
+13 -4
View File
@@ -42,7 +42,7 @@ SYNC_STATE_PREFIX = "repo.syncstate."
ID_RE = re.compile(r"^[0-9a-f]{40}$")
REVIEW_CACHE = dict()
REVIEW_CACHE = {}
def IsChange(rev):
@@ -111,7 +111,7 @@ class GitConfig:
return cls(configfile=os.path.join(gitdir, "config"), defaults=defaults)
def __init__(self, configfile, defaults=None, jsonFile=None):
self.file = configfile
self.file = str(configfile)
self.defaults = defaults
self._cache_dict = None
self._section_dict = None
@@ -222,6 +222,12 @@ class GitConfig:
value = "true" if value else "false"
self.SetString(name, value)
def SetInt(self, name: str, value: int) -> None:
"""Set an integer value for a key."""
if value is not None:
value = str(value)
self.SetString(name, value)
def GetString(self, name: str, all_keys: bool = False) -> Union[str, None]:
"""Get the first value for a key, or None if it is not defined.
@@ -432,7 +438,7 @@ class GitConfig:
if p.Wait() == 0:
return p.stdout
else:
raise GitError(f"git config {str(args)}: {p.stderr}")
raise GitError(f"git {' '.join(command)}: {p.stderr}")
class RepoConfig(GitConfig):
@@ -718,7 +724,10 @@ class Remote:
def Save(self):
"""Save this remote to the configuration."""
self._Set("url", self.url)
if self.pushUrl is not None:
# projectname is initialized for projects listed in the manifest, but
# not for others (e.g. the manifest project). This class is used for
# all of them.
if self.pushUrl is not None and self.projectname is not None:
self._Set("pushurl", self.pushUrl + "/" + self.projectname)
else:
self._Set("pushurl", self.pushUrl)
+88 -57
View File
@@ -14,6 +14,7 @@
import os
from git_command import GitCommand
import platform_utils
from repo_trace import Trace
@@ -86,9 +87,8 @@ class GitRefs:
self._symref = {}
self._mtime = {}
self._ReadPackedRefs()
self._ReadLoose("refs/")
self._ReadLoose1(os.path.join(self._gitdir, HEAD), HEAD)
self._ReadRefs()
self._ReadSymbolicRef(HEAD)
scan = self._symref
attempts = 0
@@ -102,64 +102,95 @@ class GitRefs:
scan = scan_next
attempts += 1
def _ReadPackedRefs(self):
path = os.path.join(self._gitdir, "packed-refs")
self._TrackMtime(HEAD)
self._TrackMtime("config")
self._TrackMtime("packed-refs")
self._TrackTreeMtimes("refs")
self._TrackTreeMtimes("reftable")
@staticmethod
def _IsNullRef(ref_id: str) -> bool:
"""Check if a ref_id is a null object ID."""
return ref_id and all(ch == "0" for ch in ref_id)
def _ReadRefs(self) -> None:
"""Read all references using git for-each-ref."""
p = GitCommand(
None,
["for-each-ref", "--format=%(objectname)%00%(refname)%00%(symref)"],
capture_stdout=True,
capture_stderr=True,
bare=True,
gitdir=self._gitdir,
)
if p.Wait() != 0:
return
for line in p.stdout.splitlines():
ref_id, name, symref = line.split("\0")
if symref:
self._symref[name] = symref
elif ref_id and not self._IsNullRef(ref_id):
self._phyref[name] = ref_id
def _ReadSymbolicRef(self, name: str) -> None:
"""Read a symbolic reference."""
p = GitCommand(
None,
["symbolic-ref", "-q", name],
capture_stdout=True,
capture_stderr=True,
bare=True,
gitdir=self._gitdir,
)
if p.Wait() == 0:
ref = p.stdout.strip()
if ref:
self._symref[name] = ref
return
p = GitCommand(
None,
["rev-parse", "--verify", "-q", name],
capture_stdout=True,
capture_stderr=True,
bare=True,
gitdir=self._gitdir,
)
if p.Wait() == 0:
ref_id = p.stdout.strip()
if ref_id:
self._phyref[name] = ref_id
def _TrackMtime(self, name: str) -> None:
"""Track the modification time of a specific gitdir path."""
path = os.path.join(self._gitdir, name)
try:
fd = open(path)
mtime = os.path.getmtime(path)
self._mtime[name] = os.path.getmtime(path)
except OSError:
return
def _TrackTreeMtimes(self, root: str) -> None:
"""Recursively track modification times for a directory tree."""
root_path = os.path.join(self._gitdir, root)
try:
for line in fd:
line = str(line)
if line[0] == "#":
continue
if line[0] == "^":
continue
line = line[:-1]
p = line.split(" ")
ref_id = p[0]
name = p[1]
self._phyref[name] = ref_id
finally:
fd.close()
self._mtime["packed-refs"] = mtime
def _ReadLoose(self, prefix):
base = os.path.join(self._gitdir, prefix)
for name in platform_utils.listdir(base):
p = os.path.join(base, name)
# We don't implement the full ref validation algorithm, just the
# simple rules that would show up in local filesystems.
# https://git-scm.com/docs/git-check-ref-format
if name.startswith(".") or name.endswith(".lock"):
pass
elif platform_utils.isdir(p):
self._mtime[prefix] = os.path.getmtime(base)
self._ReadLoose(prefix + name + "/")
else:
self._ReadLoose1(p, prefix + name)
def _ReadLoose1(self, path, name):
try:
with open(path) as fd:
mtime = os.path.getmtime(path)
ref_id = fd.readline()
except (OSError, UnicodeError):
if not platform_utils.isdir(root_path):
return
except OSError:
return
try:
ref_id = ref_id.decode()
except AttributeError:
pass
if not ref_id:
return
ref_id = ref_id[:-1]
to_scan = [root]
while to_scan:
name = to_scan.pop()
self._TrackMtime(name)
path = os.path.join(self._gitdir, name)
if not platform_utils.isdir(path):
continue
if ref_id.startswith("ref: "):
self._symref[name] = ref_id[5:]
else:
self._phyref[name] = ref_id
self._mtime[name] = mtime
for child in platform_utils.listdir(path):
child_name = os.path.join(name, child)
child_path = os.path.join(self._gitdir, child_name)
if platform_utils.isdir(child_path):
to_scan.append(child_name)
else:
self._TrackMtime(child_name)
-1
View File
@@ -1,5 +1,4 @@
#!/bin/sh
#
# Copyright (C) 2009 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
+77 -24
View File
@@ -23,17 +23,21 @@ Examples:
"""
import functools
import glob
import hashlib
import os
import sys
import tempfile
import time
from typing import NamedTuple
import urllib.parse
from git_command import git_require
from git_command import GitCommand
from git_config import IsId
from git_config import RepoConfig
from git_refs import GitRefs
import platform_utils
_SUPERPROJECT_GIT_NAME = "superproject.git"
@@ -97,7 +101,7 @@ class Superproject:
self._manifest = manifest
self.name = name
self.remote = remote
self.revision = self._branch = revision
self.revision = revision
self._repodir = manifest.repodir
self._superproject_dir = superproject_dir
self._superproject_path = manifest.SubmanifestInfoDir(
@@ -129,6 +133,10 @@ class Superproject:
"""Set the _print_messages attribute."""
self._print_messages = value
def SetRevisionId(self, revision_id: str) -> None:
"""Set the revisionId of the superproject to sync to."""
self.revision = revision_id
@property
def commit_id(self):
"""Returns the commit ID of the superproject checkout."""
@@ -190,12 +198,14 @@ class Superproject:
message = f"{self._LogMessagePrefix()} {fmt.format(*inputs)}"
if self._print_messages:
print(message, file=sys.stderr)
self._git_event_log.ErrorEvent(message, fmt)
if self._git_event_log:
self._git_event_log.ErrorEvent(message, fmt)
def _LogMessagePrefix(self):
"""Returns the prefix string to be logged in each log message"""
return (
f"repo superproject branch: {self._branch} url: {self._remote_url}"
f"repo superproject revision: {self.revision} "
f"url: {self._remote_url}"
)
def _LogError(self, fmt, *inputs):
@@ -214,30 +224,63 @@ class Superproject:
"""
if not os.path.exists(self._superproject_path):
os.mkdir(self._superproject_path)
if not self._quiet and not os.path.exists(self._work_git):
if os.path.exists(self._work_git):
return True
if not self._quiet:
print(
"%s: Performing initial setup for superproject; this might "
"take several minutes." % self._work_git
)
cmd = ["init", "--bare", self._work_git_name]
p = GitCommand(
None,
cmd,
cwd=self._superproject_path,
capture_stdout=True,
capture_stderr=True,
tmp_gitdir_prefix = ".tmp-superproject-initgitdir-"
tmp_gitdir = tempfile.mkdtemp(
prefix=tmp_gitdir_prefix,
dir=self._superproject_path,
)
retval = p.Wait()
if retval:
self._LogWarning(
"git init call failed, command: git {}, "
"return code: {}, stderr: {}",
tmp_git_name = os.path.basename(tmp_gitdir)
try:
cmd = ["init", "--bare", tmp_git_name]
p = GitCommand(
None,
cmd,
retval,
p.stderr,
cwd=self._superproject_path,
capture_stdout=True,
capture_stderr=True,
)
return False
return True
retval = p.Wait()
if retval:
self._LogWarning(
"git init call failed, command: git {}, "
"return code: {}, stderr: {}",
cmd,
retval,
p.stderr,
)
return False
platform_utils.rename(tmp_gitdir, self._work_git)
tmp_gitdir = None
return True
finally:
# Clean up the temporary directory created during the process,
# as well as any stale ones left over from previous attempts.
if tmp_gitdir and os.path.exists(tmp_gitdir):
platform_utils.rmtree(tmp_gitdir)
age_threshold = 60 * 60 * 24 # 1 day in seconds
now = time.time()
for tmp_dir in glob.glob(
os.path.join(self._superproject_path, f"{tmp_gitdir_prefix}*")
):
try:
mtime = os.path.getmtime(tmp_dir)
if now - mtime > age_threshold:
platform_utils.rmtree(tmp_dir)
except OSError:
pass
def _Fetch(self):
"""Fetches a superproject for the manifest based on |_remote_url|.
@@ -275,8 +318,15 @@ class Superproject:
if rev_commit:
cmd.extend(["--negotiation-tip", rev_commit])
if self._branch:
cmd += [self._branch + ":" + self._branch]
if self.revision:
# If revision is a commit hash, fetch it directly to avoid
# creating a local branch of the same name.
refspec = (
self.revision
if IsId(self.revision)
else f"{self.revision}:{self.revision}"
)
cmd.append(refspec)
p = GitCommand(
None,
cmd,
@@ -311,7 +361,7 @@ class Superproject:
)
return None
data = None
branch = "HEAD" if not self._branch else self._branch
branch = "HEAD" if not self.revision else self.revision
cmd = ["ls-tree", "-z", "-r", branch]
p = GitCommand(
@@ -363,6 +413,8 @@ class Superproject:
if not self._Init():
return SyncResult(False, should_exit)
if IsId(self.revision) and self.commit_id:
return SyncResult(True, False)
if not self._Fetch():
return SyncResult(False, should_exit)
if not self._quiet:
@@ -421,7 +473,8 @@ class Superproject:
)
return None
manifest_str = self._manifest.ToXml(
groups=self._manifest.GetGroupsStr(), omit_local=True
filter_groups=self._manifest.GetManifestGroupsStr(),
omit_local=True,
).toxml()
manifest_path = self._manifest_path
try:
+16
View File
@@ -1,3 +1,19 @@
# Copyright (C) 2020 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Event logging in the git trace2 EVENT format."""
from git_command import GetEventTargetPath
from git_command import RepoSourceVersion
from git_trace2_event_log_base import BaseEventLog
+23 -17
View File
@@ -68,6 +68,7 @@ class BaseEventLog:
global p_init_count
p_init_count += 1
self._log = []
self.verbose = False
# Try to get session-id (sid) from environment (setup in repo launcher).
KEY = "GIT_TRACE2_PARENT_SID"
if env is None:
@@ -309,10 +310,12 @@ class BaseEventLog:
# ignore the attempt and continue to DGRAM below. Otherwise,
# issue a warning.
if err.errno != errno.EPROTOTYPE:
print(
f"repo: warning: git trace2 logging failed: {err}",
file=sys.stderr,
)
if self.verbose:
print(
"repo: warning: git trace2 logging failed:",
f"{err}",
file=sys.stderr,
)
return None
if socket_type == socket.SOCK_DGRAM or socket_type is None:
try:
@@ -322,18 +325,20 @@ class BaseEventLog:
self._WriteLog(lambda bs: sock.sendto(bs, path))
return f"af_unix:dgram:{path}"
except OSError as err:
print(
f"repo: warning: git trace2 logging failed: {err}",
file=sys.stderr,
)
if self.verbose:
print(
f"repo: warning: git trace2 logging failed: {err}",
file=sys.stderr,
)
return None
# Tried to open a socket but couldn't connect (SOCK_STREAM) or write
# (SOCK_DGRAM).
print(
"repo: warning: git trace2 logging failed: could not write to "
"socket",
file=sys.stderr,
)
if self.verbose:
print(
"repo: warning: git trace2 logging failed: could not"
"write to socket",
file=sys.stderr,
)
return None
# Path is an absolute path
@@ -348,9 +353,10 @@ class BaseEventLog:
self._WriteLog(f.write)
log_path = f.name
except FileExistsError as err:
print(
"repo: warning: git trace2 logging failed: %r" % err,
file=sys.stderr,
)
if self.verbose:
print(
"repo: warning: git trace2 logging failed: %r" % err,
file=sys.stderr,
)
return None
return log_path
+1 -1
View File
@@ -25,7 +25,7 @@ from git_refs import HEAD
# The API we've documented to hook authors. Keep in sync with repo-hooks.md.
_API_ARGS = {
"pre-upload": {"project_list", "worktree_list"},
"post-sync": {"repo_topdir"},
"post-sync": {"repo_topdir", "sync_duration_seconds"},
}
+116 -15
View File
@@ -1,5 +1,4 @@
#!/usr/bin/env python3
#
# Copyright (C) 2008 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
@@ -20,6 +19,7 @@ People shouldn't run this directly; instead, they should use the `repo` wrapper
which takes care of execing this entry point.
"""
import difflib
import getpass
import json
import netrc
@@ -30,6 +30,7 @@ import signal
import sys
import textwrap
import time
from typing import Optional
import urllib.request
from repo_logging import RepoLogger
@@ -293,6 +294,102 @@ class _Repo:
result = run()
return result
def _autocorrect_command_name(
self, name: str, config: RepoConfig
) -> Optional[str]:
"""Autocorrect command name based on user's git config."""
close_commands = difflib.get_close_matches(
name, self.commands.keys(), n=5, cutoff=0.7
)
if not close_commands:
logger.error(
"repo: '%s' is not a repo command. See 'repo help'.", name
)
return None
assumed = close_commands[0]
autocorrect = config.GetString("help.autocorrect")
# If there are multiple close matches, git won't automatically run one.
# We'll always prompt instead of guessing.
if len(close_commands) > 1:
autocorrect = "prompt"
# Handle git configuration boolean values:
# 0, "false", "off", "no", "show": show suggestion (default)
# 1, "true", "on", "yes", "immediate": run suggestion immediately
# "never": don't run or show any suggested command
# "prompt": show the suggestion and prompt for confirmation
# positive number > 1: run suggestion after specified deciseconds
if autocorrect is None:
autocorrect = "0"
autocorrect = autocorrect.lower()
if autocorrect in ("0", "false", "off", "no", "show"):
autocorrect = 0
elif autocorrect in ("true", "on", "yes", "immediate"):
autocorrect = -1 # immediate
elif autocorrect == "never":
return None
elif autocorrect == "prompt":
logger.warning(
"You called a repo command named "
"'%s', which does not exist.",
name,
)
try:
resp = input(f"Run '{assumed}' instead [y/N]? ")
if resp.lower().startswith("y"):
return assumed
except (KeyboardInterrupt, EOFError):
pass
return None
else:
try:
autocorrect = int(autocorrect)
except ValueError:
autocorrect = 0
if autocorrect != 0:
if autocorrect < 0:
logger.warning(
"You called a repo command named "
"'%s', which does not exist.\n"
"Continuing assuming that "
"you meant '%s'.",
name,
assumed,
)
else:
delay = autocorrect * 0.1
logger.warning(
"You called a repo command named "
"'%s', which does not exist.\n"
"Continuing in %.1f seconds, assuming "
"that you meant '%s'.",
name,
delay,
assumed,
)
try:
time.sleep(delay)
except KeyboardInterrupt:
return None
return assumed
logger.error(
"repo: '%s' is not a repo command. See 'repo help'.", name
)
logger.warning(
"The most similar command%s\n\t%s",
"s are" if len(close_commands) > 1 else " is",
"\n\t".join(close_commands),
)
return None
def _RunLong(self, name, gopts, argv, git_trace2_event_log):
"""Execute the (longer running) requested subcommand."""
result = 0
@@ -307,20 +404,22 @@ class _Repo:
outer_client=outer_client,
)
try:
cmd = self.commands[name](
repodir=self.repodir,
client=repo_client,
manifest=repo_client.manifest,
outer_client=outer_client,
outer_manifest=outer_client.manifest,
git_event_log=git_trace2_event_log,
if name not in self.commands:
corrected_name = self._autocorrect_command_name(
name, outer_client.globalConfig
)
except KeyError:
logger.error(
"repo: '%s' is not a repo command. See 'repo help'.", name
)
return 1
if not corrected_name:
return 1
name = corrected_name
cmd = self.commands[name](
repodir=self.repodir,
client=repo_client,
manifest=repo_client.manifest,
outer_client=outer_client,
outer_manifest=outer_client.manifest,
git_event_log=git_trace2_event_log,
)
Editor.globalConfig = cmd.client.globalConfig
@@ -338,6 +437,9 @@ class _Repo:
)
return 1
cmd.CommonValidateOptions(copts, cargs)
git_trace2_event_log.verbose = copts.verbose
if gopts.pager is not False and not isinstance(cmd, InteractiveCommand):
config = cmd.client.globalConfig
if gopts.pager:
@@ -360,7 +462,6 @@ class _Repo:
Execute the subcommand.
"""
nonlocal result
cmd.CommonValidateOptions(copts, cargs)
cmd.ValidateOptions(copts, cargs)
this_manifest_only = copts.this_manifest_only
+6 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo forall" "Repo Manual"
.TH REPO "1" "May 2026" "repo forall" "Repo Manual"
.SH NAME
repo \- repo forall - manual page for repo forall
.SH SYNOPSIS
@@ -120,6 +120,11 @@ git command, use REPO_LREV.
REPO_RREV is the name of the revision from the manifest, exactly as written in
the manifest.
.PP
REPO_UPSTREAM is the name of the upstream branch as specified in the manifest.
.PP
REPO_DEST_BRANCH is the name of the destination branch for code review, as
specified in the manifest.
.PP
REPO_COUNT is the total number of projects being iterated.
.PP
REPO_I is the current (1\-based) iteration count. Can be used in conjunction with
+21 -2
View File
@@ -1,10 +1,10 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2022" "repo info" "Repo Manual"
.TH REPO "1" "May 2026" "repo info" "Repo Manual"
.SH NAME
repo \- repo info - manual page for repo info
.SH SYNOPSIS
.B repo
\fI\,info \/\fR[\fI\,-dl\/\fR] [\fI\,-o \/\fR[\fI\,-c\/\fR]] [\fI\,<project>\/\fR...]
\fI\,info \/\fR[\fI\,-dl\/\fR] [\fI\,-o \/\fR[\fI\,-c\/\fR]] [\fI\,--format=<format>\/\fR] [\fI\,<project>\/\fR...]
.SH DESCRIPTION
Summary
.PP
@@ -14,6 +14,10 @@ Get info on the manifest branch, current branch or unmerged branches
\fB\-h\fR, \fB\-\-help\fR
show this help message and exit
.TP
\fB\-j\fR JOBS, \fB\-\-jobs\fR=\fI\,JOBS\/\fR
number of jobs to run in parallel (default: based on
number of CPU cores)
.TP
\fB\-d\fR, \fB\-\-diff\fR
show full info and commit diff including remote
branches
@@ -21,6 +25,18 @@ branches
\fB\-o\fR, \fB\-\-overview\fR
show overview of all local commits
.TP
\fB\-\-include\-summary\fR
include manifest summary (default: true)
.TP
\fB\-\-no\-include\-summary\fR
exclude manifest summary
.TP
\fB\-\-include\-projects\fR
include project details (default: true)
.TP
\fB\-\-no\-include\-projects\fR
exclude project details
.TP
\fB\-c\fR, \fB\-\-current\-branch\fR
consider only checked out branches
.TP
@@ -29,6 +45,9 @@ consider all local branches
.TP
\fB\-l\fR, \fB\-\-local\-only\fR
disable all remote operations
.TP
\fB\-\-format\fR=\fI\,FORMAT\/\fR
output format: text, json (default: text)
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
+110 -41
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "April 2025" "repo manifest" "Repo Manual"
.TH REPO "1" "April 2026" "repo manifest" "Repo Manual"
.SH NAME
repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS
@@ -131,6 +131,7 @@ include*)>
<!ATTLIST default dest\-branch CDATA #IMPLIED>
<!ATTLIST default upstream CDATA #IMPLIED>
<!ATTLIST default sync\-j CDATA #IMPLIED>
<!ATTLIST default sync\-j\-max CDATA #IMPLIED>
<!ATTLIST default sync\-c CDATA #IMPLIED>
<!ATTLIST default sync\-s CDATA #IMPLIED>
<!ATTLIST default sync\-tags CDATA #IMPLIED>
@@ -139,7 +140,7 @@ include*)>
<!ATTLIST manifest\-server url CDATA #REQUIRED>
.IP
<!ELEMENT submanifest EMPTY>
<!ATTLIST submanifest name ID #REQUIRED>
<!ATTLIST submanifest name ID #REQUIRED>
<!ATTLIST submanifest remote IDREF #IMPLIED>
<!ATTLIST submanifest project CDATA #IMPLIED>
<!ATTLIST submanifest manifest\-name CDATA #IMPLIED>
@@ -164,15 +165,32 @@ IDREF #IMPLIED>
.TP
<!ATTLIST project revision
CDATA #IMPLIED>
.TP
<!ATTLIST project dest\-branch
CDATA #IMPLIED>
.TP
<!ATTLIST project groups
CDATA #IMPLIED>
.TP
<!ATTLIST project sync\-c
CDATA #IMPLIED>
.TP
<!ATTLIST project sync\-s
CDATA #IMPLIED>
.TP
<!ATTLIST project sync\-tags
CDATA #IMPLIED>
.TP
<!ATTLIST project upstream
CDATA #IMPLIED>
.TP
<!ATTLIST project clone\-depth
CDATA #IMPLIED>
.TP
<!ATTLIST project force\-path
CDATA #IMPLIED>
.IP
<!ATTLIST project dest\-branch CDATA #IMPLIED>
<!ATTLIST project groups CDATA #IMPLIED>
<!ATTLIST project sync\-c CDATA #IMPLIED>
<!ATTLIST project sync\-s CDATA #IMPLIED>
<!ATTLIST project sync\-tags CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project clone\-depth CDATA #IMPLIED>
<!ATTLIST project force\-path CDATA #IMPLIED>
<!ATTLIST project sync\-strategy CDATA #IMPLIED>
.IP
<!ELEMENT annotation EMPTY>
<!ATTLIST annotation name CDATA #REQUIRED>
@@ -184,19 +202,34 @@ CDATA #IMPLIED>
<!ATTLIST copyfile dest CDATA #REQUIRED>
.IP
<!ELEMENT linkfile EMPTY>
<!ATTLIST linkfile src CDATA #REQUIRED>
<!ATTLIST linkfile src CDATA #REQUIRED>
<!ATTLIST linkfile dest CDATA #REQUIRED>
.TP
<!ELEMENT extend\-project (annotation*,
copyfile*,
linkfile*)>
.TP
<!ATTLIST extend\-project name
CDATA #REQUIRED>
.TP
<!ATTLIST extend\-project path
CDATA #IMPLIED>
.TP
<!ATTLIST extend\-project dest\-path
CDATA #IMPLIED>
.TP
<!ATTLIST extend\-project groups
CDATA #IMPLIED>
.TP
<!ATTLIST extend\-project revision
CDATA #IMPLIED>
.TP
<!ATTLIST extend\-project remote
CDATA #IMPLIED>
.IP
<!ELEMENT extend\-project EMPTY>
<!ATTLIST extend\-project name CDATA #REQUIRED>
<!ATTLIST extend\-project path CDATA #IMPLIED>
<!ATTLIST extend\-project dest\-path CDATA #IMPLIED>
<!ATTLIST extend\-project groups CDATA #IMPLIED>
<!ATTLIST extend\-project revision CDATA #IMPLIED>
<!ATTLIST extend\-project remote CDATA #IMPLIED>
<!ATTLIST extend\-project dest\-branch CDATA #IMPLIED>
<!ATTLIST extend\-project upstream CDATA #IMPLIED>
<!ATTLIST extend\-project base\-rev CDATA #IMPLIED>
<!ATTLIST extend\-project upstream CDATA #IMPLIED>
<!ATTLIST extend\-project base\-rev CDATA #IMPLIED>
.IP
<!ELEMENT remove\-project EMPTY>
<!ATTLIST remove\-project name CDATA #IMPLIED>
@@ -205,7 +238,7 @@ CDATA #IMPLIED>
<!ATTLIST remove\-project base\-rev CDATA #IMPLIED>
.IP
<!ELEMENT repo\-hooks EMPTY>
<!ATTLIST repo\-hooks in\-project CDATA #REQUIRED>
<!ATTLIST repo\-hooks in\-project CDATA #REQUIRED>
<!ATTLIST repo\-hooks enabled\-list CDATA #REQUIRED>
.IP
<!ELEMENT superproject EMPTY>
@@ -214,7 +247,7 @@ CDATA #IMPLIED>
<!ATTLIST superproject revision CDATA #IMPLIED>
.IP
<!ELEMENT contactinfo EMPTY>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
.IP
<!ELEMENT include EMPTY>
<!ATTLIST include name CDATA #REQUIRED>
@@ -294,7 +327,9 @@ when syncing a revision locked manifest in \fB\-c\fR mode to avoid having to syn
entire ref space. Project elements not setting their own `upstream` will inherit
this value.
.PP
Attribute `sync\-j`: Number of parallel jobs to use when synching.
Attribute `sync\-j`: Number of parallel jobs to use when syncing.
.PP
Attribute `sync\-j\-max`: Maximum number of parallel jobs to use when syncing.
.PP
Attribute `sync\-c`: Set to true to only sync the given Git branch (specified in
the `revision` attribute) rather than the whole ref space. Project elements
@@ -362,7 +397,7 @@ supplied, `revision` is used.
.PP
`path` may not be an absolute path or use "." or ".." path components.
.PP
Attribute `groups`: List of additional groups to which all projects in the
Attribute `groups`: Set of additional groups to which all projects in the
included submanifest belong. This appends and recurses, meaning all projects in
submanifests carry all parent submanifest groups. Same syntax as the
corresponding element of `project`.
@@ -424,7 +459,7 @@ Attribute `dest\-branch`: Name of a Git branch (e.g. `main`). When using `repo
upload`, changes will be submitted for code review on this branch. If
unspecified both here and in the default element, `revision` is used instead.
.PP
Attribute `groups`: List of groups to which this project belongs, whitespace or
Attribute `groups`: Set of groups to which this project belongs, whitespace or
comma separated. All projects belong to the group "all", and each project
automatically belongs to a group of its name:`name` and path:`path`. E.g. for
`<project name="monkeys" path="barrel\-of"/>`, that project definition is
@@ -451,6 +486,21 @@ mirror repository according to its `path` attribute (if supplied) rather than
the `name` attribute. This attribute only applies to the local mirrors syncing,
it will be ignored when syncing the projects in a client working directory.
.PP
Attribute `sync\-strategy`: Set the sync strategy used when fetching this
project. Currently the only supported value is `stateless`. When set to
`stateless`, repo will run a reflog expiration and aggressive garbage collection
at the end of the sync process. This is useful for projects that contain large
binary files and use `clone\-depth="1"`, where garbage can accumulate as binaries
are added, deleted, or modified across successive syncs.
.PP
During a stateless sync, repo checks the following before cleaning up: 1. The
project does not share an object directory with other projects. 2. The working
tree is clean (no uncommitted changes, no untracked files). 3. There are no
unpushed local commits. 4. There is no Git stash.
.PP
If any of these conditions are not met, repo falls back to a standard sync
without garbage collection.
.PP
Element extend\-project
.PP
Modify the attributes of the named project.
@@ -460,6 +510,11 @@ of an existing project without completely replacing the existing project
definition. This makes the local manifest more robust against changes to the
original manifest.
.PP
The `extend\-project` element can also contain `annotation`, `copyfile`, and
`linkfile` child elements. These are added to the project's definition. A
`copyfile` or `linkfile` with a `dest` that already exists in the project will
overwrite the original.
.PP
Attribute `path`: If specified, limit the change to projects checked out at the
specified path, rather than all projects with the given name.
.PP
@@ -468,8 +523,8 @@ repo client where the Git working directory for this project should be placed.
This is used to move a project in the checkout by overriding the existing `path`
setting.
.PP
Attribute `groups`: List of additional groups to which this project belongs.
Same syntax as the corresponding element of `project`.
Attribute `groups`: Set of additional groups to which this project belongs. Same
syntax as the corresponding element of `project`.
.PP
Attribute `revision`: If specified, overrides the revision of the original
project. Same syntax as the corresponding element of `project`.
@@ -493,19 +548,21 @@ element of `project`.
.PP
Element annotation
.PP
Zero or more annotation elements may be specified as children of a project or
remote element. Each element describes a name\-value pair. For projects, this
name\-value pair will be exported into each project's environment during a
\&'forall' command, prefixed with `REPO__`. In addition, there is an optional
attribute "keep" which accepts the case insensitive values "true" (default) or
"false". This attribute determines whether or not the annotation will be kept
when exported with the manifest subcommand.
Zero or more annotation elements may be specified as children of a project
element, an extend\-project element, or a remote element. Each element describes
a name\-value pair. For projects, this name\-value pair will be exported into each
project's environment during a 'forall' command, prefixed with `REPO__`. In
addition, there is an optional attribute "keep" which accepts the case
insensitive values "true" (default) or "false". This attribute determines
whether or not the annotation will be kept when exported with the manifest
subcommand.
.PP
Element copyfile
.PP
Zero or more copyfile elements may be specified as children of a project
element. Each element describes a src\-dest pair of files; the "src" file will be
copied to the "dest" place during `repo sync` command.
element, or an extend\-project element. Each element describes a src\-dest pair of
files; the "src" file will be copied to the "dest" place during `repo sync`
command.
.PP
"src" is project relative, "dest" is relative to the top of the tree. Copying
from paths outside of the project or to paths outside of the repo client is not
@@ -516,10 +573,14 @@ Intermediate paths must not be symlinks either.
.PP
Parent directories of "dest" will be automatically created if missing.
.PP
The files are copied in the order they are specified in the manifests. If
multiple elements specify the same source and destination, they will only be
applied as one, based on the first occurence. Files are copied before any links
specified via linkfile elements are created.
.PP
Element linkfile
.PP
It's just like copyfile and runs at the same time as copyfile but instead of
copying it creates a symlink.
It's just like copyfile, but instead of copying it creates a symlink.
.PP
The symlink is created at "dest" (relative to the top of the tree) and points to
the path specified by "src" which is a path in the project.
@@ -529,6 +590,11 @@ Parent directories of "dest" will be automatically created if missing.
The symlink target may be a file or directory, but it may not point outside of
the repo client.
.PP
The links are created in the order they are specified in the manifests. If
multiple elements specify the same source and destination, they will only be
applied as one, based on the first occurence. Links are created after any files
specified via copyfile elements are copied.
.PP
Element remove\-project
.PP
Deletes a project from the internal manifest table, possibly allowing a
@@ -620,13 +686,16 @@ repository's root.
"name" may not be an absolute path or use "." or ".." path components. These
restrictions are not enforced for [Local Manifests].
.PP
Attribute `groups`: List of additional groups to which all projects in the
Attribute `groups`: Set of additional groups to which all projects in the
included manifest belong. This appends and recurses, meaning all projects in
included manifests carry all parent include groups. Same syntax as the
included manifests carry all parent include groups. This also applies to all
extend\-project elements in the included manifests. Same syntax as the
corresponding element of `project`.
.PP
Attribute `revision`: Name of a Git branch (e.g. `main` or `refs/heads/main`)
default to which all projects in the included manifest belong.
default to which all projects in the included manifest belong. This recurses,
meaning it will apply to all projects in all manifests included as a result of
this element.
.PP
Local Manifests
.PP
+21 -5
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2025" "repo smartsync" "Repo Manual"
.TH REPO "1" "May 2026" "repo smartsync" "Repo Manual"
.SH NAME
repo \- repo smartsync - manual page for repo smartsync
.SH SYNOPSIS
@@ -20,12 +20,11 @@ number of CPU cores)
.TP
\fB\-\-jobs\-network\fR=\fI\,JOBS\/\fR
number of network jobs to run in parallel (defaults to
\fB\-\-jobs\fR or 1). Ignored when \fB\-\-interleaved\fR is set
\fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel
(defaults to \fB\-\-jobs\fR or 8). Ignored when \fB\-\-interleaved\fR
is set
(defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-nointerleaved\fR is set
.TP
\fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future)
@@ -60,7 +59,10 @@ use the existing manifest checkout as\-is. (do not
update to the latest revision)
.TP
\fB\-\-interleaved\fR
fetch and checkout projects in parallel (experimental)
fetch and checkout projects in parallel (default)
.TP
\fB\-\-no\-interleaved\fR
fetch and checkout projects in phases
.TP
\fB\-n\fR, \fB\-\-network\-only\fR
fetch only, don't update working tree
@@ -99,6 +101,10 @@ implies \fB\-c\fR
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-\-superproject\-revision\fR=\fI\,SUPERPROJECT_REVISION\/\fR
sync to superproject revision (applies to outer
manifest)
.TP
\fB\-\-tags\fR
fetch tags
.TP
@@ -149,6 +155,16 @@ operate on this manifest and its submanifests
.TP
\fB\-\-no\-repo\-verify\fR
do not verify repo source code
.SS post\-sync hooks:
.TP
\fB\-\-no\-verify\fR
Do not run the post\-sync hook.
.TP
\fB\-\-verify\fR
Run the post\-sync hook without prompting.
.TP
\fB\-\-ignore\-hooks\fR
Do not abort if post\-sync hooks fail.
.PP
Run `repo help smartsync` to view the detailed manual.
.SH DETAILS
+21 -5
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "June 2025" "repo sync" "Repo Manual"
.TH REPO "1" "May 2026" "repo sync" "Repo Manual"
.SH NAME
repo \- repo sync - manual page for repo sync
.SH SYNOPSIS
@@ -20,12 +20,11 @@ number of CPU cores)
.TP
\fB\-\-jobs\-network\fR=\fI\,JOBS\/\fR
number of network jobs to run in parallel (defaults to
\fB\-\-jobs\fR or 1). Ignored when \fB\-\-interleaved\fR is set
\fB\-\-jobs\fR or 1). Ignored unless \fB\-\-no\-interleaved\fR is set
.TP
\fB\-\-jobs\-checkout\fR=\fI\,JOBS\/\fR
number of local checkout jobs to run in parallel
(defaults to \fB\-\-jobs\fR or 8). Ignored when \fB\-\-interleaved\fR
is set
(defaults to \fB\-\-jobs\fR or 8). Ignored unless \fB\-\-nointerleaved\fR is set
.TP
\fB\-f\fR, \fB\-\-force\-broken\fR
obsolete option (to be deleted in the future)
@@ -60,7 +59,10 @@ use the existing manifest checkout as\-is. (do not
update to the latest revision)
.TP
\fB\-\-interleaved\fR
fetch and checkout projects in parallel (experimental)
fetch and checkout projects in parallel (default)
.TP
\fB\-\-no\-interleaved\fR
fetch and checkout projects in phases
.TP
\fB\-n\fR, \fB\-\-network\-only\fR
fetch only, don't update working tree
@@ -99,6 +101,10 @@ implies \fB\-c\fR
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
.TP
\fB\-\-superproject\-revision\fR=\fI\,SUPERPROJECT_REVISION\/\fR
sync to superproject revision (applies to outer
manifest)
.TP
\fB\-\-tags\fR
fetch tags
.TP
@@ -156,6 +162,16 @@ operate on this manifest and its submanifests
.TP
\fB\-\-no\-repo\-verify\fR
do not verify repo source code
.SS post\-sync hooks:
.TP
\fB\-\-no\-verify\fR
Do not run the post\-sync hook.
.TP
\fB\-\-verify\fR
Run the post\-sync hook without prompting.
.TP
\fB\-\-ignore\-hooks\fR
Do not abort if post\-sync hooks fail.
.PP
Run `repo help sync` to view the detailed manual.
.SH DETAILS
+61
View File
@@ -0,0 +1,61 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "November 2025" "repo wipe" "Repo Manual"
.SH NAME
repo \- repo wipe - manual page for repo wipe
.SH SYNOPSIS
.B repo
\fI\,wipe <project>\/\fR...
.SH DESCRIPTION
Summary
.PP
Wipe projects from the worktree
.SH OPTIONS
.TP
\fB\-h\fR, \fB\-\-help\fR
show this help message and exit
.TP
\fB\-f\fR, \fB\-\-force\fR
force wipe shared projects and uncommitted changes
.TP
\fB\-\-force\-uncommitted\fR
force wipe even if there are uncommitted changes
.TP
\fB\-\-force\-shared\fR
force wipe even if the project shares an object
directory
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
show all output
.TP
\fB\-q\fR, \fB\-\-quiet\fR
only show errors
.SS Multi\-manifest options:
.TP
\fB\-\-outer\-manifest\fR
operate starting at the outermost manifest
.TP
\fB\-\-no\-outer\-manifest\fR
do not operate on outer manifests
.TP
\fB\-\-this\-manifest\-only\fR
only operate on this (sub)manifest
.TP
\fB\-\-no\-this\-manifest\-only\fR, \fB\-\-all\-manifests\fR
operate on this manifest and its submanifests
.PP
Run `repo help wipe` to view the detailed manual.
.SH DETAILS
.PP
The 'repo wipe' command removes the specified projects from the worktree (the
checked out source code) and deletes the project's git data from `.repo`.
.PP
This is a destructive operation and cannot be undone.
.PP
Projects can be specified either by name, or by a relative or absolute path to
the project's local directory.
.SH EXAMPLES
.SS # Wipe the project "platform/build" by name:
$ repo wipe platform/build
.SS # Wipe the project at the path "build/make":
$ repo wipe build/make
+4 -1
View File
@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "April 2025" "repo" "Repo Manual"
.TH REPO "1" "November 2025" "repo" "Repo Manual"
.SH NAME
repo \- repository management tool built on top of git
.SH SYNOPSIS
@@ -132,6 +132,9 @@ Upload changes for code review
.TP
version
Display the version of repo
.TP
wipe
Wipe projects from the worktree
.PP
See 'repo help <command>' for more information on a specific command.
Bug reports: https://issues.gerritcodereview.com/issues/new?component=1370071
+105 -68
View File
@@ -155,6 +155,7 @@ class _Default:
upstreamExpr = None
remote = None
sync_j = None
sync_j_max = None
sync_c = False
sync_s = False
sync_tags = True
@@ -255,7 +256,7 @@ class _XmlSubmanifest:
project: a string, the name of the manifest project.
revision: a string, the commitish.
manifestName: a string, the submanifest file name.
groups: a list of strings, the groups to add to all projects in the
groups: a set of strings, the groups to add to all projects in the
submanifest.
default_groups: a list of strings, the default groups to sync.
path: a string, the relative path for the submanifest checkout.
@@ -281,7 +282,7 @@ class _XmlSubmanifest:
self.project = project
self.revision = revision
self.manifestName = manifestName
self.groups = groups
self.groups = groups or set()
self.default_groups = default_groups
self.path = path
self.parent = parent
@@ -304,7 +305,7 @@ class _XmlSubmanifest:
self.repo_client = RepoClient(
parent.repodir,
linkFile,
parent_groups=",".join(groups) or "",
parent_groups=groups,
submanifest_path=os.path.join(parent.path_prefix, self.relpath),
outer_client=outer_client,
default_groups=default_groups,
@@ -345,7 +346,7 @@ class _XmlSubmanifest:
manifestName = self.manifestName or "default.xml"
revision = self.revision or self.name
path = self.path or revision.split("/")[-1]
groups = self.groups or []
groups = self.groups
return SubmanifestSpec(
self.name, manifestUrl, manifestName, revision, path, groups
@@ -359,9 +360,7 @@ class _XmlSubmanifest:
def GetGroupsStr(self):
"""Returns the `groups` given for this submanifest."""
if self.groups:
return ",".join(self.groups)
return ""
return ",".join(sorted(self.groups))
def GetDefaultGroupsStr(self):
"""Returns the `default-groups` given for this submanifest."""
@@ -381,7 +380,7 @@ class SubmanifestSpec:
self.manifestName = manifestName
self.revision = revision
self.path = path
self.groups = groups or []
self.groups = groups
class XmlManifest:
@@ -393,7 +392,7 @@ class XmlManifest:
manifest_file,
local_manifests=None,
outer_client=None,
parent_groups="",
parent_groups=None,
submanifest_path="",
default_groups=None,
):
@@ -409,7 +408,8 @@ class XmlManifest:
manifests. This will usually be
|repodir|/|LOCAL_MANIFESTS_DIR_NAME|.
outer_client: RepoClient of the outer manifest.
parent_groups: a string, the groups to apply to this projects.
parent_groups: a set of strings, the groups to apply to this
manifest.
submanifest_path: The submanifest root relative to the repo root.
default_groups: a string, the default manifest groups to use.
"""
@@ -432,7 +432,7 @@ class XmlManifest:
self.manifestFileOverrides = {}
self.local_manifests = local_manifests
self._load_local_manifests = True
self.parent_groups = parent_groups
self.parent_groups = parent_groups or set()
self.default_groups = default_groups
if submanifest_path and not outer_client:
@@ -567,21 +567,29 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
"""
return [x for x in re.split(r"[,\s]+", field) if x]
def _ParseSet(self, field):
"""Parse fields that contain flattened sets.
These are whitespace & comma separated. Empty elements will be
discarded.
"""
return set(self._ParseList(field))
def ToXml(
self,
peg_rev=False,
peg_rev_upstream=True,
peg_rev_dest_branch=True,
groups=None,
filter_groups=None,
omit_local=False,
):
"""Return the current manifest XML."""
mp = self.manifestProject
if groups is None:
groups = mp.manifest_groups
if groups:
groups = self._ParseList(groups)
if filter_groups is None:
filter_groups = mp.manifest_groups
if filter_groups:
filter_groups = self._ParseList(filter_groups)
doc = xml.dom.minidom.Document()
root = doc.createElement("manifest")
@@ -624,6 +632,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if d.sync_j is not None:
have_default = True
e.setAttribute("sync-j", "%d" % d.sync_j)
if d.sync_j_max is not None:
have_default = True
e.setAttribute("sync-j-max", "%d" % d.sync_j_max)
if d.sync_c:
have_default = True
e.setAttribute("sync-c", "true")
@@ -654,7 +665,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
output_project(parent, parent_node, project)
def output_project(parent, parent_node, p):
if not p.MatchesGroups(groups):
if not p.MatchesGroups(filter_groups):
return
if omit_local and self.IsFromLocalManifest(p):
@@ -725,10 +736,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
le.setAttribute("dest", lf.dest)
e.appendChild(le)
default_groups = ["all", "name:%s" % p.name, "path:%s" % p.relpath]
egroups = [g for g in p.groups if g not in default_groups]
if egroups:
e.setAttribute("groups", ",".join(egroups))
groups = p.groups - {"all", f"name:{p.name}", f"path:{p.relpath}"}
if groups:
e.setAttribute("groups", ",".join(sorted(groups)))
for a in p.annotations:
if a.keep == "true":
@@ -749,14 +759,17 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if p.clone_depth:
e.setAttribute("clone-depth", str(p.clone_depth))
if p.sync_strategy:
e.setAttribute("sync-strategy", str(p.sync_strategy))
self._output_manifest_project_extras(p, e)
if p.subprojects:
subprojects = {subp.name for subp in p.subprojects}
output_projects(p, e, list(sorted(subprojects)))
output_projects(p, e, sorted(subprojects))
projects = {p.name for p in self._paths.values() if not p.parent}
output_projects(None, root, list(sorted(projects)))
output_projects(None, root, sorted(projects))
if self._repo_hooks_project:
root.appendChild(doc.createTextNode(""))
@@ -813,7 +826,6 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
"submanifest",
# These are children of 'project' nodes.
"annotation",
"project",
"copyfile",
"linkfile",
}
@@ -1116,7 +1128,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
groups += f",platform-{platform.system().lower()}"
return groups
def GetGroupsStr(self):
def GetManifestGroupsStr(self):
"""Returns the manifest group string that should be synced."""
return (
self.manifestProject.manifest_groups or self.GetDefaultGroupsStr()
@@ -1171,12 +1183,12 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
b = b[len(R_HEADS) :]
self.branch = b
parent_groups = self.parent_groups
parent_groups = self.parent_groups.copy()
if self.path_prefix:
parent_groups = (
parent_groups |= {
f"{SUBMANIFEST_GROUP_PREFIX}:path:"
f"{self.path_prefix},{parent_groups}"
)
f"{self.path_prefix}"
}
# The manifestFile was specified by the user which is why we
# allow include paths to point anywhere.
@@ -1202,16 +1214,16 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
# Since local manifests are entirely managed by
# the user, allow them to point anywhere the
# user wants.
local_group = (
local_group = {
f"{LOCAL_MANIFEST_GROUP_PREFIX}:"
f"{local_file[:-4]}"
)
}
nodes.append(
self._ParseManifestXml(
local,
self.subdir,
parent_groups=(
f"{local_group},{parent_groups}"
local_group | parent_groups
),
restrict_includes=False,
)
@@ -1262,7 +1274,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
self,
path,
include_root,
parent_groups="",
parent_groups=None,
restrict_includes=True,
parent_node=None,
):
@@ -1271,11 +1283,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
Args:
path: The XML file to read & parse.
include_root: The path to interpret include "name"s relative to.
parent_groups: The groups to apply to this projects.
parent_groups: The set of groups to apply to this manifest.
restrict_includes: Whether to constrain the "name" attribute of
includes.
parent_node: The parent include node, to apply attribute to this
projects.
parent_node: The parent include node, to apply attributes to this
manifest.
Returns:
List of XML nodes.
@@ -1299,6 +1311,14 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
nodes = []
for node in manifest.childNodes:
if (
parent_node
and node.nodeName in ("include", "project")
and not node.hasAttribute("revision")
):
node.setAttribute(
"revision", parent_node.getAttribute("revision")
)
if node.nodeName == "include":
name = self._reqatt(node, "name")
if restrict_includes:
@@ -1307,12 +1327,10 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
raise ManifestInvalidPathError(
f'<include> invalid "name": {name}: {msg}'
)
include_groups = ""
if parent_groups:
include_groups = parent_groups
include_groups = (parent_groups or set()).copy()
if node.hasAttribute("groups"):
include_groups = (
node.getAttribute("groups") + "," + include_groups
include_groups |= self._ParseSet(
node.getAttribute("groups")
)
fp = os.path.join(include_root, name)
if not os.path.isfile(fp):
@@ -1335,21 +1353,16 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
f"failed parsing included manifest {name}: {e}"
)
else:
if parent_groups and node.nodeName == "project":
nodeGroups = parent_groups
if node.hasAttribute("groups"):
nodeGroups = (
node.getAttribute("groups") + "," + nodeGroups
)
node.setAttribute("groups", nodeGroups)
if (
parent_node
and node.nodeName == "project"
and not node.hasAttribute("revision")
if parent_groups and node.nodeName in (
"project",
"extend-project",
):
node.setAttribute(
"revision", parent_node.getAttribute("revision")
)
nodeGroups = parent_groups.copy()
if node.hasAttribute("groups"):
nodeGroups |= self._ParseSet(
node.getAttribute("groups")
)
node.setAttribute("groups", ",".join(sorted(nodeGroups)))
nodes.append(node)
return nodes
@@ -1458,7 +1471,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
dest_path = node.getAttribute("dest-path")
groups = node.getAttribute("groups")
if groups:
groups = self._ParseList(groups)
groups = self._ParseSet(groups or "")
revision = node.getAttribute("revision")
remote_name = node.getAttribute("remote")
if not remote_name:
@@ -1479,7 +1492,15 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if path and p.relpath != path:
continue
if groups:
p.groups.extend(groups)
p.groups |= groups
# Drop local groups so we don't mistakenly omit this
# project from the superproject override manifest.
p.groups = {
g
for g in p.groups
if not g.startswith(LOCAL_MANIFEST_GROUP_PREFIX)
}
if revision:
if base_revision:
if p.revisionExpr != base_revision:
@@ -1509,6 +1530,14 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
p.UpdatePaths(relpath, worktree, gitdir, objdir)
self._paths[p.relpath] = p
for n in node.childNodes:
if n.nodeName == "copyfile":
self._ParseCopyFile(p, n)
elif n.nodeName == "linkfile":
self._ParseLinkFile(p, n)
elif n.nodeName == "annotation":
self._ParseAnnotation(p, n)
if node.nodeName == "repo-hooks":
# Only one project can be the hooks project
if repo_hooks_project is not None:
@@ -1740,6 +1769,13 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
% (self.manifestFile, d.sync_j)
)
d.sync_j_max = XmlInt(node, "sync-j-max", None)
if d.sync_j_max is not None and d.sync_j_max <= 0:
raise ManifestParseError(
'%s: sync-j-max must be greater than 0, not "%s"'
% (self.manifestFile, d.sync_j_max)
)
d.sync_c = XmlBool(node, "sync-c", False)
d.sync_s = XmlBool(node, "sync-s", False)
d.sync_tags = XmlBool(node, "sync-tags", True)
@@ -1802,7 +1838,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
groups = ""
if node.hasAttribute("groups"):
groups = node.getAttribute("groups")
groups = self._ParseList(groups)
groups = self._ParseSet(groups)
default_groups = self._ParseList(node.getAttribute("default-groups"))
path = node.getAttribute("path")
if path == "":
@@ -1905,17 +1941,14 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
% (self.manifestFile, clone_depth)
)
sync_strategy = node.getAttribute("sync-strategy") or None
dest_branch = (
node.getAttribute("dest-branch") or self._default.destBranchExpr
)
upstream = node.getAttribute("upstream") or self._default.upstreamExpr
groups = ""
if node.hasAttribute("groups"):
groups = node.getAttribute("groups")
groups = self._ParseList(groups)
if parent is None:
(
relpath,
@@ -1930,8 +1963,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
parent, name, path
)
default_groups = ["all", "name:%s" % name, "path:%s" % relpath]
groups.extend(set(default_groups).difference(groups))
groups = ""
if node.hasAttribute("groups"):
groups = node.getAttribute("groups")
groups = self._ParseSet(groups)
groups |= {"all", f"name:{name}", f"path:{relpath}"}
if self.IsMirror and node.hasAttribute("force-path"):
if XmlBool(node, "force-path", False):
@@ -1953,6 +1989,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
sync_s=sync_s,
sync_tags=sync_tags,
clone_depth=clone_depth,
sync_strategy=sync_strategy,
upstream=upstream,
parent=parent,
dest_branch=dest_branch,
@@ -1963,11 +2000,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
for n in node.childNodes:
if n.nodeName == "copyfile":
self._ParseCopyFile(project, n)
if n.nodeName == "linkfile":
elif n.nodeName == "linkfile":
self._ParseLinkFile(project, n)
if n.nodeName == "annotation":
elif n.nodeName == "annotation":
self._ParseAnnotation(project, n)
if n.nodeName == "project":
elif n.nodeName == "project":
project.subprojects.append(
self._ParseProject(n, parent=project)
)
+37
View File
@@ -222,6 +222,43 @@ def rmdir(path):
os.rmdir(_makelongpath(path))
def removedirs(path: str) -> None:
"""Remove a directory tree of symlinks and empty directories.
Walks |path| bottom-up without following symlinks. Removes symlinks
and empty directories. Stops at (and preserves) regular files and
non-empty directories. Silently succeeds if |path| does not exist.
Availability: Unix, Windows.
"""
if islink(path):
remove(path)
return
if not isdir(path):
return
for dirpath, dirnames, filenames in walk(path, topdown=False):
for name in dirnames:
entry = os.path.join(dirpath, name)
if islink(entry):
remove(entry)
else:
try:
rmdir(entry)
except OSError:
pass
for name in filenames:
entry = os.path.join(dirpath, name)
if islink(entry):
remove(entry)
try:
rmdir(path)
except OSError:
pass
def isdir(path):
"""os.path.isdir(path) wrapper with support for long paths on Windows.
+12 -7
View File
@@ -25,7 +25,10 @@ except ImportError:
from repo_trace import IsTraceToStderr
_TTY = sys.stderr.isatty()
# Capture the original stderr stream. We use this exclusively for progress
# updates to ensure we talk to the terminal even if stderr is redirected.
_STDERR = sys.stderr
_TTY = _STDERR.isatty()
# This will erase all content in the current line (wherever the cursor is).
# It does not move the cursor, so this is usually followed by \r to move to
@@ -133,11 +136,11 @@ class Progress:
def _write(self, s):
s = "\r" + s
if self._elide:
col = os.get_terminal_size(sys.stderr.fileno()).columns
col = os.get_terminal_size(_STDERR.fileno()).columns
if len(s) > col:
s = s[: col - 1] + ".."
sys.stderr.write(s)
sys.stderr.flush()
_STDERR.write(s)
_STDERR.flush()
def start(self, name):
self._active += 1
@@ -156,6 +159,8 @@ class Progress:
inc: The number of items completed.
msg: The message to display. If None, use the last message.
"""
if self._ended:
return
self._done += inc
if msg is None:
msg = self._last_msg
@@ -211,9 +216,9 @@ class Progress:
# Erase the current line, print the message with a newline,
# and then immediately redraw the progress bar on the new line.
sys.stderr.write("\r" + CSI_ERASE_LINE)
sys.stderr.write(msg + "\n")
sys.stderr.flush()
_STDERR.write("\r" + CSI_ERASE_LINE)
_STDERR.write(msg + "\n")
_STDERR.flush()
self.update(inc=0)
def end(self):
+546 -264
View File
File diff suppressed because it is too large Load Diff
+25 -2
View File
@@ -1,4 +1,4 @@
# Copyright 2023 The Android Open Source Project
# Copyright (C) 2023 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,9 +14,32 @@
[tool.black]
line-length = 80
# NB: Keep in sync with tox.ini.
target-version = ['py36', 'py37', 'py38', 'py39', 'py310', 'py311'] #, 'py312'
# Config file for the isort python module.
# This is used to enforce import sorting standards.
#
# https://pycqa.github.io/isort/docs/configuration/options.html
[tool.isort]
# Be compatible with `black` since it also matches what we want.
profile = 'black'
line_length = 80
length_sort = false
force_single_line = true
lines_after_imports = 2
from_first = false
case_sensitive = false
force_sort_within_sections = true
order_by_type = false
# Ignore generated files.
extend_skip_glob = '*_pb2.py'
# Allow importing multiple classes on a single line from these modules.
# https://google.github.io/styleguide/pyguide#s2.2-imports
single_line_exclusions = ['abc', 'collections.abc', 'typing']
[tool.pytest.ini_options]
markers = """
skip_cq: Skip tests in the CQ. Should be rarely used!
+155
View File
@@ -0,0 +1,155 @@
#!/usr/bin/env python3
# Copyright (C) 2025 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Helper tool to check various metadata (e.g. licensing) in source files."""
import argparse
from pathlib import Path
import re
import sys
import util
_FILE_HEADER_RE = re.compile(
r"""# Copyright \(C\) 20[0-9]{2} The Android Open Source Project
#
# Licensed under the Apache License, Version 2\.0 \(the "License"\);
# you may not use this file except in compliance with the License\.
# You may obtain a copy of the License at
#
# http://www\.apache\.org/licenses/LICENSE-2\.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied\.
# See the License for the specific language governing permissions and
# limitations under the License\.
"""
)
def check_license(path: Path, lines: list[str]) -> bool:
"""Check license header."""
# Enforce licensing on configs & scripts.
if not (
path.suffix in (".bash", ".cfg", ".ini", ".py", ".toml")
or lines[0] in ("#!/bin/bash", "#!/bin/sh", "#!/usr/bin/env python3")
):
return True
# Extract the file header.
header_lines = []
for line in lines:
if line.startswith("#"):
header_lines.append(line)
else:
break
if not header_lines:
print(
f"error: {path.relative_to(util.TOPDIR)}: "
"missing file header (copyright+licensing)",
file=sys.stderr,
)
return False
# Skip the shebang.
if header_lines[0].startswith("#!"):
header_lines.pop(0)
# If this file is imported into the tree, then leave it be.
if header_lines[0] == "# DO NOT EDIT THIS FILE":
return True
header = "".join(f"{x}\n" for x in header_lines)
if not _FILE_HEADER_RE.match(header):
print(
f"error: {path.relative_to(util.TOPDIR)}: "
"file header incorrectly formatted",
file=sys.stderr,
)
print(
"".join(f"> {x}\n" for x in header_lines), end="", file=sys.stderr
)
return False
return True
def check_path(opts: argparse.Namespace, path: Path) -> bool:
"""Check a single path."""
try:
data = path.read_text(encoding="utf-8")
except FileNotFoundError:
return True
lines = data.splitlines()
# NB: Use list comprehension and not a generator so we run all the checks.
return all(
[
check_license(path, lines),
]
)
def check_paths(opts: argparse.Namespace, paths: list[Path]) -> bool:
"""Check all the paths."""
# NB: Use list comprehension and not a generator so we check all paths.
return all([check_path(opts, x) for x in paths]) # noqa: C419
def find_files(opts: argparse.Namespace) -> list[Path]:
"""Find all the files in the source tree."""
result = util.run(
opts,
["git", "ls-tree", "-r", "-z", "--name-only", "HEAD"],
cwd=util.TOPDIR,
capture_output=True,
encoding="utf-8",
)
return [util.TOPDIR / x for x in result.stdout.split("\0")[:-1]]
def get_parser() -> argparse.ArgumentParser:
"""Get a CLI parser."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"-n",
"--dry-run",
dest="dryrun",
action="store_true",
help="show everything that would be done",
)
parser.add_argument(
"paths",
nargs="*",
help="the paths to scan",
)
return parser
def main(argv: list[str]) -> int:
"""The main func!"""
parser = get_parser()
opts = parser.parse_args(argv)
paths = opts.paths
if not opts.paths:
paths = find_files(opts)
return 0 if check_paths(opts, paths) else 1
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+10 -7
View File
@@ -27,6 +27,9 @@ import sys
import util
assert sys.version_info >= (3, 9), "Release framework requires Python 3.9+"
def sign(opts):
"""Sign the launcher!"""
output = ""
@@ -82,18 +85,18 @@ Repo launcher bucket:
gs://git-repo-downloads/
You should first upload it with a specific version:
gsutil cp -a public-read {opts.launcher} gs://git-repo-downloads/repo-{version}
gsutil cp -a public-read {opts.launcher}.asc gs://git-repo-downloads/repo-{version}.asc
gcloud storage cp --predefined-acl=publicRead {opts.launcher} gs://git-repo-downloads/repo-{version}
gcloud storage cp --predefined-acl=publicRead {opts.launcher}.asc gs://git-repo-downloads/repo-{version}.asc
Then to make it the public default:
gsutil cp -a public-read gs://git-repo-downloads/repo-{version} gs://git-repo-downloads/repo
gsutil cp -a public-read gs://git-repo-downloads/repo-{version}.asc gs://git-repo-downloads/repo.asc
gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo-{version} gs://git-repo-downloads/repo
gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo-{version}.asc gs://git-repo-downloads/repo.asc
NB: If a rollback is necessary, the GS bucket archives old versions, and may be
accessed by specifying their unique id number.
gsutil ls -la gs://git-repo-downloads/repo gs://git-repo-downloads/repo.asc
gsutil cp -a public-read gs://git-repo-downloads/repo#<unique id> gs://git-repo-downloads/repo
gsutil cp -a public-read gs://git-repo-downloads/repo.asc#<unique id> gs://git-repo-downloads/repo.asc
gcloud storage ls --long --all-versions gs://git-repo-downloads/repo gs://git-repo-downloads/repo.asc
gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo#<unique id> gs://git-repo-downloads/repo
gcloud storage cp --predefined-acl=publicRead gs://git-repo-downloads/repo.asc#<unique id> gs://git-repo-downloads/repo.asc
""" # noqa: E501
)
+3
View File
@@ -30,6 +30,9 @@ import sys
import util
assert sys.version_info >= (3, 9), "Release framework requires Python 3.9+"
# We currently sign with the old DSA key as it's been around the longest.
# We should transition to RSA by Jun 2020, and ECC by Jun 2021.
KEYID = util.KEYID_DSA
+1 -1
View File
@@ -24,7 +24,7 @@ from typing import List, Optional
import urllib.request
assert sys.version_info >= (3, 8), "Python 3.8+ required"
assert sys.version_info >= (3, 9), "Release framework requires Python 3.9+"
TOPDIR = Path(__file__).resolve().parent.parent
+10 -4
View File
@@ -30,6 +30,9 @@ import tempfile
from typing import List
assert sys.version_info >= (3, 9), "Release framework requires Python 3.9+"
THIS_FILE = Path(__file__).resolve()
TOPDIR = THIS_FILE.parent.parent
MANDIR = TOPDIR.joinpath("man")
@@ -62,7 +65,11 @@ def main(argv: List[str]) -> int:
parser = get_parser()
opts = parser.parse_args(argv)
if not shutil.which("help2man"):
help2man = ["help2man"]
cipd_help2man = TOPDIR / ".cipd_bin/bin/help2man"
if cipd_help2man.exists():
help2man = [cipd_help2man]
elif not shutil.which("help2man"):
sys.exit("Please install help2man to continue.")
# Let repo know we're generating man pages so it can avoid some dynamic
@@ -77,7 +84,7 @@ def main(argv: List[str]) -> int:
version = RepoSourceVersion()
cmdlist = [
[
"help2man",
*help2man,
"-N",
"-n",
f"repo {cmd} - manual page for repo {cmd}",
@@ -96,7 +103,7 @@ def main(argv: List[str]) -> int:
]
cmdlist.append(
[
"help2man",
*help2man,
"-N",
"-n",
"repository management tool built on top of git",
@@ -174,7 +181,6 @@ def replace_regex(data):
"""
regex = (
(r"(It was generated by help2man) [0-9.]+", r"\g<1>."),
(r"^\033\[[0-9;]*m([^\033]*)\033\[m", r"\g<1>"),
(r"^\.IP\n(.*:)\n", r".SS \g<1>\n"),
(r"^\.PP\nDescription", r".SH DETAILS"),
)
+5 -4
View File
@@ -14,7 +14,7 @@
"""Random utility code for release tools."""
import os
from pathlib import Path
import re
import shlex
import subprocess
@@ -24,8 +24,9 @@ import sys
assert sys.version_info >= (3, 6), "This module requires Python 3.6+"
TOPDIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
HOMEDIR = os.path.expanduser("~")
THIS_FILE = Path(__file__).resolve()
TOPDIR = THIS_FILE.parent.parent
HOMEDIR = Path("~").expanduser()
# These are the release keys we sign with.
@@ -54,7 +55,7 @@ def run(opts, cmd, check=True, **kwargs):
def import_release_key(opts):
"""Import the public key of the official release repo signing key."""
# Extract the key from our repo launcher.
launcher = getattr(opts, "launcher", os.path.join(TOPDIR, "repo"))
launcher = getattr(opts, "launcher", TOPDIR / "repo")
print(f'Importing keys from "{launcher}" launcher script')
with open(launcher, encoding="utf-8") as fp:
data = fp.read()
-1
View File
@@ -1,5 +1,4 @@
#!/usr/bin/env python3
#
# Copyright (C) 2008 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
+80 -13
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright 2019 The Android Open Source Project
# Copyright (C) 2019 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -17,26 +17,37 @@
import functools
import os
import shlex
import shutil
import subprocess
import sys
from typing import List
# NB: While tests/* support Python >=3.6 to match requirements.json for `repo`,
# the higher level runner logic does not need to be held back.
assert sys.version_info >= (3, 9), "Test/release framework requires Python 3.9+"
ROOT_DIR = os.path.dirname(os.path.realpath(__file__))
def log_cmd(cmd: str, argv: list[str]) -> None:
"""Log a debug message to make history easier to track."""
print("+", cmd, shlex.join(argv), file=sys.stderr)
@functools.lru_cache()
def is_ci() -> bool:
"""Whether we're running in our CI system."""
return os.getenv("LUCI_CQ") == "yes"
def run_pytest(argv: List[str]) -> int:
def run_pytest(argv: list[str]) -> int:
"""Returns the exit code from pytest."""
if is_ci():
argv = ["-m", "not skip_cq"] + argv
log_cmd("pytest", argv)
return subprocess.run(
[sys.executable, "-m", "pytest"] + argv,
check=False,
@@ -44,11 +55,12 @@ def run_pytest(argv: List[str]) -> int:
).returncode
def run_pytest_py38(argv: List[str]) -> int:
def run_pytest_py38(argv: list[str]) -> int:
"""Returns the exit code from pytest under Python 3.8."""
if is_ci():
argv = ["-m", "not skip_cq"] + argv
log_cmd("[vpython 3.8] pytest", argv)
try:
return subprocess.run(
[
@@ -69,6 +81,14 @@ def run_pytest_py38(argv: List[str]) -> int:
def run_black():
"""Returns the exit code from black."""
argv = ["--version"]
log_cmd("black", argv)
subprocess.run(
[sys.executable, "-m", "black"] + argv,
check=True,
cwd=ROOT_DIR,
)
# Black by default only matches .py files. We have to list standalone
# scripts manually.
extra_programs = [
@@ -77,8 +97,10 @@ def run_black():
"release/update-hooks",
"release/update-manpages",
]
argv = ["--diff", "--check", ROOT_DIR] + extra_programs
log_cmd("black", argv)
return subprocess.run(
[sys.executable, "-m", "black", "--check", ROOT_DIR] + extra_programs,
[sys.executable, "-m", "black"] + argv,
check=False,
cwd=ROOT_DIR,
).returncode
@@ -86,8 +108,18 @@ def run_black():
def run_flake8():
"""Returns the exit code from flake8."""
argv = ["--version"]
log_cmd("flake8", argv)
subprocess.run(
[sys.executable, "-m", "flake8"] + argv,
check=True,
cwd=ROOT_DIR,
)
argv = [ROOT_DIR]
log_cmd("flake8", argv)
return subprocess.run(
[sys.executable, "-m", "flake8", ROOT_DIR],
[sys.executable, "-m", "flake8"] + argv,
check=False,
cwd=ROOT_DIR,
).returncode
@@ -95,8 +127,29 @@ def run_flake8():
def run_isort():
"""Returns the exit code from isort."""
argv = ["--version-number"]
log_cmd("isort", argv)
subprocess.run(
[sys.executable, "-m", "isort"] + argv,
check=True,
cwd=ROOT_DIR,
)
argv = ["--check", ROOT_DIR]
log_cmd("isort", argv)
return subprocess.run(
[sys.executable, "-m", "isort", "--check", ROOT_DIR],
[sys.executable, "-m", "isort"] + argv,
check=False,
cwd=ROOT_DIR,
).returncode
def run_check_metadata():
"""Returns the exit code from check-metadata."""
argv = []
log_cmd("release/check-metadata.py", argv)
return subprocess.run(
[sys.executable, "release/check-metadata.py"] + argv,
check=False,
cwd=ROOT_DIR,
).returncode
@@ -104,26 +157,40 @@ def run_isort():
def run_update_manpages() -> int:
"""Returns the exit code from release/update-manpages."""
# Allow this to fail on CI, but not local devs.
if is_ci() and not shutil.which("help2man"):
print("update-manpages: help2man not found; skipping test")
return 0
argv = ["--check"]
log_cmd("release/update-manpages", argv)
return subprocess.run(
[sys.executable, "release/update-manpages", "--check"],
[sys.executable, "release/update-manpages"] + argv,
check=False,
cwd=ROOT_DIR,
).returncode
def cipd_bootstrap() -> None:
"""Install packages via cipd if available."""
argv = ["ensure", "-root", ".cipd_bin", "-ensure-file", "cipd_manifest.txt"]
log_cmd("cipd", argv)
try:
subprocess.run(
["cipd"] + argv,
check=True,
cwd=ROOT_DIR,
)
except FileNotFoundError:
# Skip if the user doesn't have cipd from depot_tools.
return
def main(argv):
"""The main entry."""
cipd_bootstrap()
checks = (
functools.partial(run_pytest, argv),
functools.partial(run_pytest_py38, argv),
run_black,
run_flake8,
run_isort,
run_check_metadata,
run_update_manpages,
)
# Run all the tests all the time to get full feedback. Don't exit on the
+13 -3
View File
@@ -48,10 +48,10 @@ wheel: <
version: "version:3.0.7"
>
# Required by pytest==8.3.4
# Required by pytest==8.3.4 and flake8-bugbear==24.12.12
wheel: <
name: "infra/python/wheels/attrs-py2_py3"
version: "version:21.4.0"
name: "infra/python/wheels/attrs-py3"
version: "version:24.2.0"
>
# NB: Keep in sync with constraints.txt.
@@ -119,6 +119,16 @@ wheel: <
version: "version:2.10.0"
>
wheel: <
name: "infra/python/wheels/flake8-bugbear-py3"
version: "version:24.12.12"
>
wheel: <
name: "infra/python/wheels/flake8-comprehensions-py3"
version: "version:3.16.0"
>
wheel: <
name: "infra/python/wheels/isort-py3"
version: "version:5.10.1"
+2 -2
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# Copyright 2019 The Android Open Source Project
# Copyright (C) 2019 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the 'License");
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
+20 -5
View File
@@ -52,12 +52,12 @@ def _parse_ssh_version(ver_str=None):
@functools.lru_cache(maxsize=None)
def version():
"""return ssh version as a tuple"""
"""Return ssh version as a tuple.
If ssh is not available, a FileNotFoundError will be raised.
"""
try:
return _parse_ssh_version()
except FileNotFoundError:
print("fatal: ssh not installed", file=sys.stderr)
sys.exit(1)
except subprocess.CalledProcessError as e:
print(
"fatal: unable to detect ssh version"
@@ -102,9 +102,18 @@ class ProxyManager:
self._clients = manager.list()
# Path to directory for holding master sockets.
self._sock_path = None
# See if ssh is usable.
self._ssh_installed = False
def __enter__(self):
"""Enter a new context."""
# Check which version of ssh is available.
try:
version()
self._ssh_installed = True
except FileNotFoundError:
self._ssh_installed = False
return self
def __exit__(self, exc_type, exc_value, traceback):
@@ -140,7 +149,7 @@ class ProxyManager:
while True:
try:
procs.pop(0)
except: # noqa: E722
except IndexError:
break
def close(self):
@@ -282,6 +291,9 @@ class ProxyManager:
def preconnect(self, url):
"""If |uri| will create a ssh connection, setup the ssh master for it.""" # noqa: E501
if not self._ssh_installed:
return False
m = URI_ALL.match(url)
if m:
scheme = m.group(1)
@@ -306,6 +318,9 @@ class ProxyManager:
This has all the master sockets so clients can talk to them.
"""
if not self._ssh_installed:
return None
if self._sock_path is None:
if not create:
return None
+1
View File
@@ -94,6 +94,7 @@ It is equivalent to "git branch -D <branchname>".
def Execute(self, opt, args):
nb = args[0].split()
self.TryOverrideManifestWithSmartSync()
err = collections.defaultdict(list)
success = collections.defaultdict(list)
aggregate_errors = []
+9 -8
View File
@@ -102,6 +102,12 @@ revision to a locally executed git command, use REPO_LREV.
REPO_RREV is the name of the revision from the manifest, exactly
as written in the manifest.
REPO_UPSTREAM is the name of the upstream branch as specified in the
manifest.
REPO_DEST_BRANCH is the name of the destination branch for code review,
as specified in the manifest.
REPO_COUNT is the total number of projects being iterated.
REPO_I is the current (1-based) iteration count. Can be used in
@@ -133,7 +139,7 @@ without iterating through the remaining projects.
@staticmethod
def _cmd_option(option, _opt_str, _value, parser):
setattr(parser.values, option.dest or "command", list(parser.rargs))
setattr(parser.values, option.dest, list(parser.rargs))
while parser.rargs:
del parser.rargs[0]
@@ -161,6 +167,7 @@ without iterating through the remaining projects.
p.add_option(
"-c",
"--command",
dest="command",
help="command (and arguments) to execute",
action="callback",
callback=self._cmd_option,
@@ -235,13 +242,7 @@ without iterating through the remaining projects.
mirror = self.manifest.IsMirror
smart_sync_manifest_name = "smart_sync_override.xml"
smart_sync_manifest_path = os.path.join(
self.manifest.manifestProject.worktree, smart_sync_manifest_name
)
if os.path.isfile(smart_sync_manifest_path):
self.manifest.Override(smart_sync_manifest_path)
self.TryOverrideManifestWithSmartSync()
if opt.regex:
projects = self.FindProjects(args, all_manifests=all_trees)
+23 -5
View File
@@ -16,6 +16,7 @@ import os
from typing import List, Set
from command import Command
from git_command import git_require
from git_command import GitCommand
import platform_utils
from progress import Progress
@@ -204,6 +205,7 @@ class Gc(Command):
[
"rev-list",
"--objects",
"--missing=allow-promisor",
f"--remotes={project.remote.name}",
"--filter=blob:none",
"--tags",
@@ -215,7 +217,12 @@ class Gc(Command):
# Get all local objects and pack them.
local_head_objects_cmd = GitCommand(
project,
["rev-list", "--objects", "HEAD^{tree}"],
[
"rev-list",
"--objects",
"--missing=allow-promisor",
"HEAD^{tree}",
],
capture_stdout=True,
verify_command=True,
)
@@ -224,6 +231,7 @@ class Gc(Command):
[
"rev-list",
"--objects",
"--missing=allow-promisor",
"--all",
"--reflog",
"--indexed-objects",
@@ -284,11 +292,21 @@ class Gc(Command):
args, all_manifests=not opt.this_manifest_only
)
ret = self.delete_unused_projects(projects, opt)
# If the user specified projects, fetch the global list separately
# to avoid deleting untargeted projects.
if args:
all_projects = self.GetProjects(
[], all_manifests=not opt.this_manifest_only
)
else:
all_projects = projects
ret = self.delete_unused_projects(all_projects, opt)
if ret != 0:
return ret
if not opt.repack:
return
if opt.repack:
git_require((2, 17, 0), fail=True, msg="--repack")
ret = self.repack_projects(projects, opt)
return self.repack_projects(projects, opt)
return ret
+1 -1
View File
@@ -93,7 +93,7 @@ contain a line that matches both expressions:
pt = getattr(parser.values, "cmd_argv", None)
if pt is None:
pt = []
setattr(parser.values, "cmd_argv", pt)
parser.values.cmd_argv = pt
if opt_str == "-(":
pt.append("(")
+3 -5
View File
@@ -59,7 +59,7 @@ Displays detailed usage information about a command.
def PrintAllCommandsBody(self):
print("The complete list of recognized repo commands is:")
commandNames = list(sorted(all_commands))
commandNames = sorted(all_commands)
self._PrintCommands(commandNames)
print(
"See 'repo help <command>' for more information on a "
@@ -74,10 +74,8 @@ Displays detailed usage information about a command.
def PrintCommonCommandsBody(self):
print("The most commonly used repo commands are:")
commandNames = list(
sorted(
name for name, command in all_commands.items() if command.COMMON
)
commandNames = sorted(
name for name, command in all_commands.items() if command.COMMON
)
self._PrintCommands(commandNames)
+322 -123
View File
@@ -12,14 +12,41 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import enum
import functools
import io
import json
import optparse
import sys
from typing import Any, Dict, List, NamedTuple
from color import Coloring
from command import DEFAULT_LOCAL_JOBS
from command import PagedCommand
from git_refs import R_HEADS
from git_refs import R_M
class BranchInfo(NamedTuple):
"""Holds information about a branch in a project."""
relpath: str
name: str
commits: Any
date: str
is_current: bool
class OutputFormat(enum.Enum):
"""Type for the requested output format."""
# Human-readable text output.
TEXT = enum.auto()
# Machine-readable JSON output.
JSON = enum.auto()
class _Coloring(Coloring):
def __init__(self, config):
Coloring.__init__(self, config, "status")
@@ -27,10 +54,11 @@ class _Coloring(Coloring):
class Info(PagedCommand):
COMMON = True
PARALLEL_JOBS = DEFAULT_LOCAL_JOBS
helpSummary = (
"Get info on the manifest branch, current branch or unmerged branches"
)
helpUsage = "%prog [-dl] [-o [-c]] [<project>...]"
helpUsage = "%prog [-dl] [-o [-c]] [--format=<format>] [<project>...]"
def _Options(self, p):
p.add_option(
@@ -46,6 +74,30 @@ class Info(PagedCommand):
action="store_true",
help="show overview of all local commits",
)
p.add_option(
"--include-summary",
action="store_true",
default=True,
help="include manifest summary (default: true)",
)
p.add_option(
"--no-include-summary",
dest="include_summary",
action="store_false",
help="exclude manifest summary",
)
p.add_option(
"--include-projects",
action="store_true",
default=True,
help="include project details (default: true)",
)
p.add_option(
"--no-include-projects",
dest="include_projects",
action="store_false",
help="exclude project details",
)
p.add_option(
"-c",
"--current-branch",
@@ -72,8 +124,39 @@ class Info(PagedCommand):
action="store_true",
help="disable all remote operations",
)
formats = tuple(x.lower() for x in OutputFormat.__members__.keys())
p.add_option(
"--format",
default=OutputFormat.TEXT.name.lower(),
choices=formats,
help=f"output format: {', '.join(formats)} (default: %default)",
)
def WantPager(self, opt):
return OutputFormat[opt.format.upper()] == OutputFormat.TEXT
def ValidateOptions(self, opt, args):
output_format = OutputFormat[opt.format.upper()]
if output_format == OutputFormat.JSON:
if opt.all:
self.OptionParser.error("--diff is not supported with JSON")
if opt.overview:
self.OptionParser.error("--overview is not supported with JSON")
def Execute(self, opt, args):
if not opt.this_manifest_only:
self.manifest = self.manifest.outer_client
self.TryOverrideManifestWithSmartSync()
output_format = OutputFormat[opt.format.upper()]
if output_format == OutputFormat.JSON:
self._ExecuteJson(opt, args)
else:
self._ExecuteText(opt, args)
def _ExecuteText(self, opt, args) -> None:
"""Output info as human-readable text."""
self.out = _Coloring(self.client.globalConfig)
self.heading = self.out.printer("heading", attr="bold")
self.headtext = self.out.nofmt_printer("headtext", fg="yellow")
@@ -84,156 +167,272 @@ class Info(PagedCommand):
self.opt = opt
if not opt.this_manifest_only:
self.manifest = self.manifest.outer_client
manifestConfig = self.manifest.manifestProject.config
mergeBranch = manifestConfig.GetBranch("default").merge
manifestGroups = self.manifest.GetGroupsStr()
if opt.include_summary:
self._printSummary()
self.heading("Manifest branch: ")
if self.manifest.default.revisionExpr:
self.headtext(self.manifest.default.revisionExpr)
self.out.nl()
self.heading("Manifest merge branch: ")
# The manifest might not have a merge branch if it isn't in a git repo,
# e.g. if `repo init --standalone-manifest` is used.
self.headtext(mergeBranch or "")
self.out.nl()
self.heading("Manifest groups: ")
self.headtext(manifestGroups)
self.out.nl()
sp = self.manifest.superproject
srev = sp.commit_id if sp and sp.commit_id else "None"
self.heading("Superproject revision: ")
self.headtext(srev)
self.printSeparator()
if not opt.overview:
if not opt.include_projects:
return
elif not opt.overview:
self._printDiffInfo(opt, args)
else:
self._printCommitOverview(opt, args)
def _getSummaryData(self) -> Dict[str, Any]:
"""Gather manifest summary data as a dict."""
manifestConfig = self.manifest.manifestProject.config
mergeBranch = manifestConfig.GetBranch("default").merge
manifestGroups = self.manifest.GetManifestGroupsStr()
sp = self.manifest.superproject
srev = sp.commit_id if sp and sp.commit_id else None
return {
"manifest_branch": self.manifest.default.revisionExpr or "",
"manifest_merge_branch": mergeBranch or "",
"manifest_groups": manifestGroups,
"superproject_revision": srev,
}
def _getProjectData(self, project) -> Dict[str, Any]:
"""Gather project data as a dict."""
data = {
"name": project.name,
"mount_path": project.worktree,
"current_revision": project.GetRevisionId(),
"manifest_revision": project.revisionExpr,
"local_branches": list(project.GetBranches()),
}
currentBranch = project.CurrentBranch
if currentBranch:
data["current_branch"] = currentBranch
return data
def _ExecuteJson(self, opt, args) -> None:
"""Output info as JSON."""
result = {}
if opt.include_summary:
result["summary"] = self._getSummaryData()
if opt.include_projects:
projs = self.GetProjects(
args, all_manifests=not opt.this_manifest_only
)
result["projects"] = [self._getProjectData(p) for p in projs]
json_settings = {
# JSON style guide says Unicode characters are fully allowed.
"ensure_ascii": False,
# We use 2 space indent to match JSON style guide.
"indent": 2,
"separators": (",", ": "),
"sort_keys": True,
}
sys.stdout.write(json.dumps(result, **json_settings) + "\n")
def _printSummary(self) -> None:
"""Print manifest summary in text format."""
data = self._getSummaryData()
self.heading("Manifest branch: ")
self.headtext(data["manifest_branch"])
self.out.nl()
self.heading("Manifest merge branch: ")
self.headtext(data["manifest_merge_branch"])
self.out.nl()
self.heading("Manifest groups: ")
self.headtext(data["manifest_groups"])
self.out.nl()
self.heading("Superproject revision: ")
self.headtext(data["superproject_revision"] or "None")
self.out.nl()
self.printSeparator()
def printSeparator(self):
self.text("----------------------------")
self.out.nl()
@classmethod
def _DiffHelper(cls, project_idx: int, opt: Any) -> str:
"""Helper for ParallelContext to get diff info for a project."""
buf = io.StringIO()
project = cls.get_parallel_context()["projects"][project_idx]
config = cls.get_parallel_context()["config"]
out = _Coloring(config)
out.redirect(buf)
heading = out.printer("heading", attr="bold")
headtext = out.nofmt_printer("headtext", fg="yellow")
redtext = out.printer("redtext", fg="red")
sha = out.printer("sha", fg="yellow")
text = out.nofmt_printer("text")
dimtext = out.printer("dimtext", attr="dim")
heading("Project: ")
headtext(project.name)
out.nl()
heading("Mount path: ")
headtext(project.worktree)
out.nl()
heading("Current revision: ")
headtext(project.GetRevisionId())
out.nl()
currentBranch = project.CurrentBranch
if currentBranch:
heading("Current branch: ")
headtext(currentBranch)
out.nl()
heading("Manifest revision: ")
headtext(project.revisionExpr)
out.nl()
localBranches = list(project.GetBranches().keys())
heading("Local Branches: ")
redtext(str(len(localBranches)))
if localBranches:
text(" [")
text(", ".join(localBranches))
text("]")
out.nl()
if opt.all:
if not opt.local:
project.Sync_NetworkHalf(quiet=True, current_branch_only=True)
branch = project.manifest.manifestProject.config.GetBranch(
"default"
).merge
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS) :]
logTarget = R_M + branch
bareTmp = project.bare_git._bare
project.bare_git._bare = False
localCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
logTarget + "..",
"--",
)
originCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
".." + logTarget,
"--",
)
project.bare_git._bare = bareTmp
heading("Local Commits: ")
redtext(str(len(localCommits)))
dimtext(" (on current branch)")
out.nl()
for c in localCommits:
split = c.split()
sha(split[0] + " ")
text(" ".join(split[1:]))
out.nl()
text("----------------------------")
out.nl()
heading("Remote Commits: ")
redtext(str(len(originCommits)))
out.nl()
for c in originCommits:
split = c.split()
sha(split[0] + " ")
text(" ".join(split[1:]))
out.nl()
text("----------------------------")
out.nl()
return buf.getvalue()
def _printDiffInfo(self, opt, args):
# We let exceptions bubble up to main as they'll be well structured.
projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only)
for p in projs:
self.heading("Project: ")
self.headtext(p.name)
self.out.nl()
def _ProcessResults(_pool, _output, results):
for output in results:
if output:
print(output, end="")
self.heading("Mount path: ")
self.headtext(p.worktree)
self.out.nl()
with self.ParallelContext():
self.get_parallel_context()["projects"] = projs
self.get_parallel_context()[
"config"
] = self.manifest.manifestProject.config
self.heading("Current revision: ")
self.headtext(p.GetRevisionId())
self.out.nl()
self.ExecuteInParallel(
opt.jobs,
functools.partial(self._DiffHelper, opt=opt),
range(len(projs)),
callback=_ProcessResults,
ordered=True,
chunksize=1,
)
currentBranch = p.CurrentBranch
if currentBranch:
self.heading("Current branch: ")
self.headtext(currentBranch)
self.out.nl()
@classmethod
def _OverviewHelper(cls, project_idx: int, opt: Any) -> List[BranchInfo]:
"""Helper to get overview of uploadable branches."""
project = cls.get_parallel_context()["projects"][project_idx]
self.heading("Manifest revision: ")
self.headtext(p.revisionExpr)
self.out.nl()
branches = []
br = [project.GetUploadableBranch(x) for x in project.GetBranches()]
br = [x for x in br if x]
if opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch]
localBranches = list(p.GetBranches().keys())
self.heading("Local Branches: ")
self.redtext(str(len(localBranches)))
if localBranches:
self.text(" [")
self.text(", ".join(localBranches))
self.text("]")
self.out.nl()
if self.opt.all:
self.findRemoteLocalDiff(p)
self.printSeparator()
def findRemoteLocalDiff(self, project):
# Fetch all the latest commits.
if not self.opt.local:
project.Sync_NetworkHalf(quiet=True, current_branch_only=True)
branch = self.manifest.manifestProject.config.GetBranch("default").merge
if branch.startswith(R_HEADS):
branch = branch[len(R_HEADS) :]
logTarget = R_M + branch
bareTmp = project.bare_git._bare
project.bare_git._bare = False
localCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
logTarget + "..",
"--",
)
originCommits = project.bare_git.rev_list(
"--abbrev=8",
"--abbrev-commit",
"--pretty=oneline",
".." + logTarget,
"--",
)
project.bare_git._bare = bareTmp
self.heading("Local Commits: ")
self.redtext(str(len(localCommits)))
self.dimtext(" (on current branch)")
self.out.nl()
for c in localCommits:
split = c.split()
self.sha(split[0] + " ")
self.text(" ".join(split[1:]))
self.out.nl()
self.printSeparator()
self.heading("Remote Commits: ")
self.redtext(str(len(originCommits)))
self.out.nl()
for c in originCommits:
split = c.split()
self.sha(split[0] + " ")
self.text(" ".join(split[1:]))
self.out.nl()
for b in br:
branches.append(
BranchInfo(
relpath=project.RelPath(local=opt.this_manifest_only),
name=b.name,
commits=b.commits,
date=b.date,
is_current=b.name == project.CurrentBranch,
)
)
return branches
def _printCommitOverview(self, opt, args):
projs = self.GetProjects(args, all_manifests=not opt.this_manifest_only)
all_branches = []
for project in self.GetProjects(
args, all_manifests=not opt.this_manifest_only
):
br = [project.GetUploadableBranch(x) for x in project.GetBranches()]
br = [x for x in br if x]
if self.opt.current_branch:
br = [x for x in br if x.name == project.CurrentBranch]
all_branches.extend(br)
def _ProcessResults(_pool, _output, results):
for branches in results:
all_branches.extend(branches)
with self.ParallelContext():
self.get_parallel_context()["projects"] = projs
self.ExecuteInParallel(
opt.jobs,
functools.partial(self._OverviewHelper, opt=opt),
range(len(projs)),
callback=_ProcessResults,
ordered=True,
chunksize=1,
)
if not all_branches:
return
self.out.nl()
self.heading("Projects Overview")
project = None
current_relpath = None
for branch in all_branches:
if project != branch.project:
project = branch.project
if current_relpath != branch.relpath:
current_relpath = branch.relpath
self.out.nl()
self.headtext(project.RelPath(local=opt.this_manifest_only))
self.headtext(current_relpath)
self.out.nl()
commits = branch.commits
@@ -241,7 +440,7 @@ class Info(PagedCommand):
self.text(
"%s %-33s (%2d commit%s, %s)"
% (
branch.name == project.CurrentBranch and "*" or " ",
branch.is_current and "*" or " ",
branch.name,
len(commits),
len(commits) != 1 and "s" or "",
+1
View File
@@ -104,6 +104,7 @@ revision specified in the manifest.
def Execute(self, opt, args):
nb = args[0]
self.TryOverrideManifestWithSmartSync()
err_projects = []
err = []
projects = []
+486 -221
View File
File diff suppressed because it is too large Load Diff
+6 -4
View File
@@ -27,6 +27,7 @@ from error import SilentRepoExitError
from error import UploadError
from git_command import GitCommand
from git_refs import R_HEADS
import git_superproject
from hooks import RepoHook
from project import ReviewableBranch
from repo_logging import RepoLogger
@@ -627,7 +628,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
# If using superproject, add the root repo as a push option.
manifest = branch.project.manifest
push_options = list(opt.push_options)
if manifest.manifestProject.use_superproject:
if git_superproject.UseSuperproject(None, manifest):
sp = manifest.superproject
if sp:
r_id = sp.repo_id
@@ -802,9 +803,10 @@ Gerrit Code Review: https://www.gerritcodereview.com/
project_list=pending_proj_names, worktree_list=pending_worktrees
):
if LocalSyncState(manifest).IsPartiallySynced():
logger.error(
"Partially synced tree detected. Syncing all projects "
"may resolve issues you're seeing."
logger.info(
"Tip: A partially synced tree was detected. "
"If this failure involves cross-project dependencies, "
"a full `repo sync` might help."
)
ret = 1
if ret:
+184
View File
@@ -0,0 +1,184 @@
# Copyright (C) 2025 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
from typing import List
from command import Command
from error import GitError
from error import RepoExitError
import platform_utils
from project import DeleteWorktreeError
class Error(RepoExitError):
"""Exit error when wipe command fails."""
class Wipe(Command):
"""Delete projects from the worktree and .repo"""
COMMON = True
helpSummary = "Wipe projects from the worktree"
helpUsage = """
%prog <project>...
"""
helpDescription = """
The '%prog' command removes the specified projects from the worktree
(the checked out source code) and deletes the project's git data from `.repo`.
This is a destructive operation and cannot be undone.
Projects can be specified either by name, or by a relative or absolute path
to the project's local directory.
Examples:
# Wipe the project "platform/build" by name:
$ repo wipe platform/build
# Wipe the project at the path "build/make":
$ repo wipe build/make
"""
def _Options(self, p):
# TODO(crbug.com/gerrit/393383056): Add --broken option to scan and
# wipe broken projects.
p.add_option(
"-f",
"--force",
action="store_true",
help="force wipe shared projects and uncommitted changes",
)
p.add_option(
"--force-uncommitted",
action="store_true",
help="force wipe even if there are uncommitted changes",
)
p.add_option(
"--force-shared",
action="store_true",
help="force wipe even if the project shares an object directory",
)
def ValidateOptions(self, opt, args: List[str]):
if not args:
self.Usage()
def Execute(self, opt, args: List[str]):
# Get all projects to handle shared object directories.
all_projects = self.GetProjects(None, all_manifests=True, groups="all")
projects_to_wipe = self.GetProjects(args, all_manifests=True)
relpaths_to_wipe = {p.relpath for p in projects_to_wipe}
# Build a map from objdir to the relpaths of projects that use it.
objdir_map = {}
for p in all_projects:
objdir_map.setdefault(p.objdir, set()).add(p.relpath)
uncommitted_projects = []
shared_objdirs = {}
objdirs_to_delete = set()
for project in projects_to_wipe:
if project == self.manifest.manifestProject:
raise Error(
f"error: cannot wipe the manifest project: {project.name}"
)
try:
if project.HasChanges():
uncommitted_projects.append(project.name)
except GitError:
uncommitted_projects.append(f"{project.name} (corrupted)")
users = objdir_map.get(project.objdir, {project.relpath})
is_shared = not users.issubset(relpaths_to_wipe)
if is_shared:
shared_objdirs.setdefault(project.objdir, set()).update(users)
else:
objdirs_to_delete.add(project.objdir)
block_uncommitted = uncommitted_projects and not (
opt.force or opt.force_uncommitted
)
block_shared = shared_objdirs and not (opt.force or opt.force_shared)
if block_uncommitted or block_shared:
error_messages = []
if block_uncommitted:
error_messages.append(
"The following projects have uncommitted changes or are "
"corrupted:\n"
+ "\n".join(f" - {p}" for p in sorted(uncommitted_projects))
)
if block_shared:
shared_dir_messages = []
for objdir, users in sorted(shared_objdirs.items()):
other_users = users - relpaths_to_wipe
projects_to_wipe_in_dir = users & relpaths_to_wipe
message = f"""Object directory {objdir} is shared by:
Projects to be wiped: {', '.join(sorted(projects_to_wipe_in_dir))}
Projects not to be wiped: {', '.join(sorted(other_users))}"""
shared_dir_messages.append(message)
error_messages.append(
"The following projects have shared object directories:\n"
+ "\n".join(sorted(shared_dir_messages))
)
if block_uncommitted and block_shared:
error_messages.append(
"Use --force to wipe anyway, or --force-uncommitted and "
"--force-shared to specify."
)
elif block_uncommitted:
error_messages.append("Use --force-uncommitted to wipe anyway.")
else:
error_messages.append("Use --force-shared to wipe anyway.")
raise Error("\n\n".join(error_messages))
# If we are here, either there were no issues, or --force was used.
# Proceed with wiping.
successful_wipes = set()
for project in projects_to_wipe:
try:
# Force the delete here since we've already performed our
# own safety checks above.
project.DeleteWorktree(force=True, verbose=opt.verbose)
successful_wipes.add(project.relpath)
except DeleteWorktreeError as e:
print(
f"error: failed to wipe {project.name}: {e}",
file=sys.stderr,
)
# Clean up object directories only if all projects using them were
# successfully wiped.
for objdir in objdirs_to_delete:
users = objdir_map.get(objdir, set())
# Check if every project that uses this objdir has been
# successfully processed. If a project failed to be wiped, don't
# delete the object directory, or we'll corrupt the remaining
# project.
if users.issubset(successful_wipes):
if os.path.exists(objdir):
if opt.verbose:
print(
f"Deleting objects directory: {objdir}",
file=sys.stderr,
)
platform_utils.rmtree(objdir)
+20
View File
@@ -0,0 +1,20 @@
# Repo Tests
There is a mixture of [pytest] & [Python unittest] in here. We adopted [pytest]
later on but didn't migrate existing tests (since they still work). New tests
should be written using [pytest] only.
## File layout
* `test_xxx.py`: Unittests for the `xxx` module in the main repo codebase.
Modules that are in subdirs normalize the `/` into `_`.
For example, [test_error.py](./test_error.py) is for the
[error.py](../error.py) module, and
[test_subcmds_forall.py](./test_subcmds_forall.py) is for the
[subcmds/forall.py](../subcmds/forall.py) module.
* [conftest.py](./conftest.py): Custom pytest fixtures for sharing.
* [utils_for_test.py](./utils_for_test.py): Helpers for sharing in tests.
[pytest]: https://pytest.org/
[Python unittest]: https://docs.python.org/3/library/unittest.html#unittest.TestCase
+12 -1
View File
@@ -1,4 +1,4 @@
# Copyright 2022 The Android Open Source Project
# Copyright (C) 2022 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -18,6 +18,7 @@ import pathlib
import pytest
import color
import platform_utils
import repo_trace
@@ -28,6 +29,16 @@ def disable_repo_trace(tmp_path):
repo_trace._TRACE_FILE = str(tmp_path / "TRACE_FILE_from_test")
@pytest.fixture(autouse=True)
def restore_default_coloring() -> None:
"""Force disable color for reproducible test behavior.
The few tests that cover color behavior can still reset/change the color as
needed.
"""
color.SetDefaultColoring("never")
# adapted from pytest-home 0.5.1
def _set_home(monkeypatch, path: pathlib.Path):
"""
+43 -43
View File
@@ -14,61 +14,61 @@
"""Unittests for the color.py module."""
import os
import unittest
import pytest
import utils_for_test
import color
import git_config
def fixture(*paths):
"""Return a path relative to test/fixtures."""
return os.path.join(os.path.dirname(__file__), "fixtures", *paths)
@pytest.fixture
def coloring() -> color.Coloring:
"""Create a Coloring object for testing."""
config_fixture = utils_for_test.FIXTURES_DIR / "test.gitconfig"
config = git_config.GitConfig(config_fixture)
color.SetDefaultColoring("true")
return color.Coloring(config, "status")
class ColoringTests(unittest.TestCase):
"""tests of the Coloring class."""
def test_Color_Parse_all_params_none(coloring: color.Coloring) -> None:
"""all params are None"""
val = coloring._parse(None, None, None, None)
assert val == ""
def setUp(self):
"""Create a GitConfig object using the test.gitconfig fixture."""
config_fixture = fixture("test.gitconfig")
self.config = git_config.GitConfig(config_fixture)
color.SetDefaultColoring("true")
self.color = color.Coloring(self.config, "status")
def test_Color_Parse_all_params_none(self):
"""all params are None"""
val = self.color._parse(None, None, None, None)
self.assertEqual("", val)
def test_Color_Parse_first_parameter_none(coloring: color.Coloring) -> None:
"""check fg & bg & attr"""
val = coloring._parse(None, "black", "red", "ul")
assert val == "\x1b[4;30;41m"
def test_Color_Parse_first_parameter_none(self):
"""check fg & bg & attr"""
val = self.color._parse(None, "black", "red", "ul")
self.assertEqual("\x1b[4;30;41m", val)
def test_Color_Parse_one_entry(self):
"""check fg"""
val = self.color._parse("one", None, None, None)
self.assertEqual("\033[33m", val)
def test_Color_Parse_one_entry(coloring: color.Coloring) -> None:
"""check fg"""
val = coloring._parse("one", None, None, None)
assert val == "\033[33m"
def test_Color_Parse_two_entry(self):
"""check fg & bg"""
val = self.color._parse("two", None, None, None)
self.assertEqual("\033[35;46m", val)
def test_Color_Parse_three_entry(self):
"""check fg & bg & attr"""
val = self.color._parse("three", None, None, None)
self.assertEqual("\033[4;30;41m", val)
def test_Color_Parse_two_entry(coloring: color.Coloring) -> None:
"""check fg & bg"""
val = coloring._parse("two", None, None, None)
assert val == "\033[35;46m"
def test_Color_Parse_reset_entry(self):
"""check reset entry"""
val = self.color._parse("reset", None, None, None)
self.assertEqual("\033[m", val)
def test_Color_Parse_empty_entry(self):
"""check empty entry"""
val = self.color._parse("none", "blue", "white", "dim")
self.assertEqual("\033[2;34;47m", val)
val = self.color._parse("empty", "green", "white", "bold")
self.assertEqual("\033[1;32;47m", val)
def test_Color_Parse_three_entry(coloring: color.Coloring) -> None:
"""check fg & bg & attr"""
val = coloring._parse("three", None, None, None)
assert val == "\033[4;30;41m"
def test_Color_Parse_reset_entry(coloring: color.Coloring) -> None:
"""check reset entry"""
val = coloring._parse("reset", None, None, None)
assert val == "\033[m"
def test_Color_Parse_empty_entry(coloring: color.Coloring) -> None:
"""check empty entry"""
val = coloring._parse("none", "blue", "white", "dim")
assert val == "\033[2;34;47m"
val = coloring._parse("empty", "green", "white", "bold")
assert val == "\033[1;32;47m"
+88
View File
@@ -0,0 +1,88 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the command.py module."""
from command import Command
class FakeProject:
"""Minimal project double for Command.GetProjects tests."""
def __init__(
self,
name,
relpath,
*,
gitdir=None,
derived_subprojects=None,
sync_s=False,
):
self.name = name
self.relpath = relpath
self.gitdir = gitdir or f"/git/{relpath}"
self.sync_s = sync_s
self.Exists = True
self._derived_subprojects = derived_subprojects or []
def GetDerivedSubprojects(self):
return list(self._derived_subprojects)
def MatchesGroups(self, _groups):
return True
def RelPath(self, local=True):
return self.relpath
class FakeManifest:
"""Minimal manifest double for Command.GetProjects tests."""
def __init__(self, projects):
self.projects = projects
def GetManifestGroupsStr(self):
return "default"
def test_get_projects_keeps_derived_subprojects_for_repeated_repo():
"""Derived subprojects are keyed by checkout path, not repo identity."""
submodule_a = FakeProject(
"submodule",
"src/one/submodule",
gitdir="/shared/modules/submodule.git",
)
submodule_b = FakeProject(
"submodule",
"src/two/submodule",
gitdir="/shared/modules/submodule.git",
)
project_a = FakeProject(
"project",
"src/one",
derived_subprojects=[submodule_a],
sync_s=True,
)
project_b = FakeProject(
"project",
"src/two",
derived_subprojects=[submodule_b],
sync_s=True,
)
manifest = FakeManifest([project_a, project_b])
cmd = Command(manifest=manifest)
projects = cmd.GetProjects([])
assert set(projects) == {project_a, project_b, submodule_a, submodule_b}
+18 -29
View File
@@ -14,43 +14,32 @@
"""Unittests for the editor.py module."""
import unittest
import pytest
from editor import Editor
class EditorTestCase(unittest.TestCase):
@pytest.fixture(autouse=True)
def reset_editor() -> None:
"""Take care of resetting Editor state across tests."""
def setUp(self):
self.setEditor(None)
def tearDown(self):
self.setEditor(None)
@staticmethod
def setEditor(editor):
Editor._editor = editor
Editor._editor = None
yield
Editor._editor = None
class GetEditor(EditorTestCase):
"""Check GetEditor behavior."""
def test_basic(self):
"""Basic checking of _GetEditor."""
self.setEditor(":")
self.assertEqual(":", Editor._GetEditor())
def test_basic() -> None:
"""Basic checking of _GetEditor."""
Editor._editor = ":"
assert Editor._GetEditor() == ":"
class EditString(EditorTestCase):
"""Check EditString behavior."""
def test_no_editor() -> None:
"""Check behavior when no editor is available."""
Editor._editor = ":"
assert Editor.EditString("foo") == "foo"
def test_no_editor(self):
"""Check behavior when no editor is available."""
self.setEditor(":")
self.assertEqual("foo", Editor.EditString("foo"))
def test_cat_editor(self):
"""Check behavior when editor is `cat`."""
self.setEditor("cat")
self.assertEqual("foo", Editor.EditString("foo"))
def test_cat_editor() -> None:
"""Check behavior when editor is `cat`."""
Editor._editor = "cat"
assert Editor.EditString("foo") == "foo"
+34 -33
View File
@@ -1,4 +1,4 @@
# Copyright 2021 The Android Open Source Project
# Copyright (C) 2021 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -16,7 +16,9 @@
import inspect
import pickle
import unittest
from typing import Iterator, Type
import pytest
import command
import error
@@ -26,7 +28,7 @@ import project
from subcmds import all_modules
imports = all_modules + [
_IMPORTS = all_modules + [
error,
project,
git_command,
@@ -35,36 +37,35 @@ imports = all_modules + [
]
class PickleTests(unittest.TestCase):
"""Make sure all our custom exceptions can be pickled."""
def get_exceptions() -> Iterator[Type[Exception]]:
"""Return all our custom exceptions."""
for entry in _IMPORTS:
for name in dir(entry):
cls = getattr(entry, name)
if isinstance(cls, type) and issubclass(cls, Exception):
yield cls
def getExceptions(self):
"""Return all our custom exceptions."""
for entry in imports:
for name in dir(entry):
cls = getattr(entry, name)
if isinstance(cls, type) and issubclass(cls, Exception):
yield cls
def testExceptionLookup(self):
"""Make sure our introspection logic works."""
classes = list(self.getExceptions())
self.assertIn(error.HookError, classes)
# Don't assert the exact number to avoid being a change-detector test.
self.assertGreater(len(classes), 10)
def test_exception_lookup() -> None:
"""Make sure our introspection logic works."""
classes = list(get_exceptions())
assert error.HookError in classes
# Don't assert the exact number to avoid being a change-detector test.
assert len(classes) > 10
def testPickle(self):
"""Try to pickle all the exceptions."""
for cls in self.getExceptions():
args = inspect.getfullargspec(cls.__init__).args[1:]
obj = cls(*args)
p = pickle.dumps(obj)
try:
newobj = pickle.loads(p)
except Exception as e: # pylint: disable=broad-except
self.fail(
"Class %s is unable to be pickled: %s\n"
"Incomplete super().__init__(...) call?" % (cls, e)
)
self.assertIsInstance(newobj, cls)
self.assertEqual(str(obj), str(newobj))
@pytest.mark.parametrize("cls", get_exceptions())
def test_pickle(cls: Type[Exception]) -> None:
"""Try to pickle all the exceptions."""
args = inspect.getfullargspec(cls.__init__).args[1:]
obj = cls(*args)
p = pickle.dumps(obj)
try:
newobj = pickle.loads(p)
except Exception as e:
pytest.fail(
f"Class {cls} is unable to be pickled: {e}\n"
"Incomplete super().__init__(...) call?"
)
assert isinstance(newobj, cls)
assert str(obj) == str(newobj)
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright 2019 The Android Open Source Project
# Copyright (C) 2019 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
+208 -152
View File
@@ -14,177 +14,233 @@
"""Unittests for the git_config.py module."""
import os
import tempfile
import unittest
from pathlib import Path
from typing import Any
import pytest
import utils_for_test
import git_config
def fixture(*paths):
"""Return a path relative to test/fixtures."""
return os.path.join(os.path.dirname(__file__), "fixtures", *paths)
@pytest.fixture
def readonly_config() -> git_config.GitConfig:
"""Create a GitConfig object using the test.gitconfig fixture."""
config_fixture = utils_for_test.FIXTURES_DIR / "test.gitconfig"
return git_config.GitConfig(config_fixture)
class GitConfigReadOnlyTests(unittest.TestCase):
"""Read-only tests of the GitConfig class."""
def test_get_string_with_empty_config_values(
readonly_config: git_config.GitConfig,
) -> None:
"""Test config entries with no value.
def setUp(self):
"""Create a GitConfig object using the test.gitconfig fixture."""
config_fixture = fixture("test.gitconfig")
self.config = git_config.GitConfig(config_fixture)
[section]
empty
def test_GetString_with_empty_config_values(self):
"""
Test config entries with no value.
[section]
empty
"""
val = self.config.GetString("section.empty")
self.assertEqual(val, None)
def test_GetString_with_true_value(self):
"""
Test config entries with a string value.
[section]
nonempty = true
"""
val = self.config.GetString("section.nonempty")
self.assertEqual(val, "true")
def test_GetString_from_missing_file(self):
"""
Test missing config file
"""
config_fixture = fixture("not.present.gitconfig")
config = git_config.GitConfig(config_fixture)
val = config.GetString("empty")
self.assertEqual(val, None)
def test_GetBoolean_undefined(self):
"""Test GetBoolean on key that doesn't exist."""
self.assertIsNone(self.config.GetBoolean("section.missing"))
def test_GetBoolean_invalid(self):
"""Test GetBoolean on invalid boolean value."""
self.assertIsNone(self.config.GetBoolean("section.boolinvalid"))
def test_GetBoolean_true(self):
"""Test GetBoolean on valid true boolean."""
self.assertTrue(self.config.GetBoolean("section.booltrue"))
def test_GetBoolean_false(self):
"""Test GetBoolean on valid false boolean."""
self.assertFalse(self.config.GetBoolean("section.boolfalse"))
def test_GetInt_undefined(self):
"""Test GetInt on key that doesn't exist."""
self.assertIsNone(self.config.GetInt("section.missing"))
def test_GetInt_invalid(self):
"""Test GetInt on invalid integer value."""
self.assertIsNone(self.config.GetBoolean("section.intinvalid"))
def test_GetInt_valid(self):
"""Test GetInt on valid integers."""
TESTS = (
("inthex", 16),
("inthexk", 16384),
("int", 10),
("intk", 10240),
("intm", 10485760),
("intg", 10737418240),
)
for key, value in TESTS:
self.assertEqual(value, self.config.GetInt(f"section.{key}"))
"""
val = readonly_config.GetString("section.empty")
assert val is None
class GitConfigReadWriteTests(unittest.TestCase):
"""Read/write tests of the GitConfig class."""
def test_get_string_with_true_value(
readonly_config: git_config.GitConfig,
) -> None:
"""Test config entries with a string value.
def setUp(self):
self.tmpfile = tempfile.NamedTemporaryFile()
self.config = self.get_config()
[section]
nonempty = true
def get_config(self):
"""Get a new GitConfig instance."""
return git_config.GitConfig(self.tmpfile.name)
"""
val = readonly_config.GetString("section.nonempty")
assert val == "true"
def test_SetString(self):
"""Test SetString behavior."""
# Set a value.
self.assertIsNone(self.config.GetString("foo.bar"))
self.config.SetString("foo.bar", "val")
self.assertEqual("val", self.config.GetString("foo.bar"))
# Make sure the value was actually written out.
config = self.get_config()
self.assertEqual("val", config.GetString("foo.bar"))
def test_get_string_from_missing_file() -> None:
"""Test missing config file."""
config_fixture = utils_for_test.FIXTURES_DIR / "not.present.gitconfig"
config = git_config.GitConfig(config_fixture)
val = config.GetString("empty")
assert val is None
# Update the value.
self.config.SetString("foo.bar", "valll")
self.assertEqual("valll", self.config.GetString("foo.bar"))
config = self.get_config()
self.assertEqual("valll", config.GetString("foo.bar"))
# Delete the value.
self.config.SetString("foo.bar", None)
self.assertIsNone(self.config.GetString("foo.bar"))
config = self.get_config()
self.assertIsNone(config.GetString("foo.bar"))
def test_get_boolean_undefined(readonly_config: git_config.GitConfig) -> None:
"""Test GetBoolean on key that doesn't exist."""
assert readonly_config.GetBoolean("section.missing") is None
def test_SetBoolean(self):
"""Test SetBoolean behavior."""
# Set a true value.
self.assertIsNone(self.config.GetBoolean("foo.bar"))
for val in (True, 1):
self.config.SetBoolean("foo.bar", val)
self.assertTrue(self.config.GetBoolean("foo.bar"))
# Make sure the value was actually written out.
config = self.get_config()
self.assertTrue(config.GetBoolean("foo.bar"))
self.assertEqual("true", config.GetString("foo.bar"))
def test_get_boolean_invalid(readonly_config: git_config.GitConfig) -> None:
"""Test GetBoolean on invalid boolean value."""
assert readonly_config.GetBoolean("section.boolinvalid") is None
# Set a false value.
for val in (False, 0):
self.config.SetBoolean("foo.bar", val)
self.assertFalse(self.config.GetBoolean("foo.bar"))
# Make sure the value was actually written out.
config = self.get_config()
self.assertFalse(config.GetBoolean("foo.bar"))
self.assertEqual("false", config.GetString("foo.bar"))
def test_get_boolean_true(readonly_config: git_config.GitConfig) -> None:
"""Test GetBoolean on valid true boolean."""
assert readonly_config.GetBoolean("section.booltrue") is True
# Delete the value.
self.config.SetBoolean("foo.bar", None)
self.assertIsNone(self.config.GetBoolean("foo.bar"))
config = self.get_config()
self.assertIsNone(config.GetBoolean("foo.bar"))
def test_GetSyncAnalysisStateData(self):
"""Test config entries with a sync state analysis data."""
superproject_logging_data = {}
superproject_logging_data["test"] = False
options = type("options", (object,), {})()
options.verbose = "true"
options.mp_update = "false"
TESTS = (
("superproject.test", "false"),
("options.verbose", "true"),
("options.mpupdate", "false"),
("main.version", "1"),
)
self.config.UpdateSyncAnalysisState(options, superproject_logging_data)
sync_data = self.config.GetSyncAnalysisStateData()
for key, value in TESTS:
self.assertEqual(
sync_data[f"{git_config.SYNC_STATE_PREFIX}{key}"], value
)
self.assertTrue(
sync_data[f"{git_config.SYNC_STATE_PREFIX}main.synctime"]
)
def test_get_boolean_false(readonly_config: git_config.GitConfig) -> None:
"""Test GetBoolean on valid false boolean."""
assert readonly_config.GetBoolean("section.boolfalse") is False
def test_get_int_undefined(readonly_config: git_config.GitConfig) -> None:
"""Test GetInt on key that doesn't exist."""
assert readonly_config.GetInt("section.missing") is None
def test_get_int_invalid(readonly_config: git_config.GitConfig) -> None:
"""Test GetInt on invalid integer value."""
assert readonly_config.GetInt("section.intinvalid") is None
@pytest.mark.parametrize(
"key, expected",
(
("inthex", 16),
("inthexk", 16384),
("int", 10),
("intk", 10240),
("intm", 10485760),
("intg", 10737418240),
),
)
def test_get_int_valid(
readonly_config: git_config.GitConfig, key: str, expected: int
) -> None:
"""Test GetInt on valid integers."""
assert readonly_config.GetInt(f"section.{key}") == expected
@pytest.fixture
def rw_config_file(tmp_path: Path) -> Path:
"""Return a path to a temporary config file."""
return tmp_path / "config"
def test_set_string(rw_config_file: Path) -> None:
"""Test SetString behavior."""
config = git_config.GitConfig(str(rw_config_file))
# Set a value.
assert config.GetString("foo.bar") is None
config.SetString("foo.bar", "val")
assert config.GetString("foo.bar") == "val"
# Make sure the value was actually written out.
config2 = git_config.GitConfig(str(rw_config_file))
assert config2.GetString("foo.bar") == "val"
# Update the value.
config.SetString("foo.bar", "valll")
assert config.GetString("foo.bar") == "valll"
config3 = git_config.GitConfig(str(rw_config_file))
assert config3.GetString("foo.bar") == "valll"
# Delete the value.
config.SetString("foo.bar", None)
assert config.GetString("foo.bar") is None
config4 = git_config.GitConfig(str(rw_config_file))
assert config4.GetString("foo.bar") is None
def test_set_boolean(rw_config_file: Path) -> None:
"""Test SetBoolean behavior."""
config = git_config.GitConfig(str(rw_config_file))
# Set a true value.
assert config.GetBoolean("foo.bar") is None
for val in (True, 1):
config.SetBoolean("foo.bar", val)
assert config.GetBoolean("foo.bar") is True
# Make sure the value was actually written out.
config2 = git_config.GitConfig(str(rw_config_file))
assert config2.GetBoolean("foo.bar") is True
assert config2.GetString("foo.bar") == "true"
# Set a false value.
for val in (False, 0):
config.SetBoolean("foo.bar", val)
assert config.GetBoolean("foo.bar") is False
# Make sure the value was actually written out.
config3 = git_config.GitConfig(str(rw_config_file))
assert config3.GetBoolean("foo.bar") is False
assert config3.GetString("foo.bar") == "false"
# Delete the value.
config.SetBoolean("foo.bar", None)
assert config.GetBoolean("foo.bar") is None
config4 = git_config.GitConfig(str(rw_config_file))
assert config4.GetBoolean("foo.bar") is None
def test_set_int(rw_config_file: Path) -> None:
"""Test SetInt behavior."""
config = git_config.GitConfig(str(rw_config_file))
# Set a value.
assert config.GetInt("foo.bar") is None
config.SetInt("foo.bar", 10)
assert config.GetInt("foo.bar") == 10
# Make sure the value was actually written out.
config2 = git_config.GitConfig(str(rw_config_file))
assert config2.GetInt("foo.bar") == 10
assert config2.GetString("foo.bar") == "10"
# Update the value.
config.SetInt("foo.bar", 20)
assert config.GetInt("foo.bar") == 20
config3 = git_config.GitConfig(str(rw_config_file))
assert config3.GetInt("foo.bar") == 20
# Delete the value.
config.SetInt("foo.bar", None)
assert config.GetInt("foo.bar") is None
config4 = git_config.GitConfig(str(rw_config_file))
assert config4.GetInt("foo.bar") is None
def test_get_sync_analysis_state_data(rw_config_file: Path) -> None:
"""Test config entries with a sync state analysis data."""
config = git_config.GitConfig(str(rw_config_file))
superproject_logging_data: dict[str, Any] = {"test": False}
class Options:
"""Container for testing."""
options = Options()
options.verbose = "true"
options.mp_update = "false"
TESTS = (
("superproject.test", "false"),
("options.verbose", "true"),
("options.mpupdate", "false"),
("main.version", "1"),
)
config.UpdateSyncAnalysisState(options, superproject_logging_data)
sync_data = config.GetSyncAnalysisStateData()
for key, value in TESTS:
assert sync_data[f"{git_config.SYNC_STATE_PREFIX}{key}"] == value
assert sync_data[f"{git_config.SYNC_STATE_PREFIX}main.synctime"]
def test_remote_save_with_push_url_without_projectname(
rw_config_file: Path,
) -> None:
"""Test saving a remote pushUrl when projectname is unset."""
config = git_config.GitConfig(str(rw_config_file))
remote = config.GetRemote("origin")
remote.url = "https://example.com/repo"
remote.pushUrl = "ssh://example.com"
remote.projectname = None
remote.Save()
written_config = git_config.GitConfig(str(rw_config_file))
assert (
written_config.GetString("remote.origin.pushurl") == "ssh://example.com"
)
+99
View File
@@ -0,0 +1,99 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the git_refs.py module."""
import os
from pathlib import Path
import subprocess
import pytest
import utils_for_test
import git_refs
def _run(repo, *args):
return subprocess.run(
["git", "-C", repo, *args],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
encoding="utf-8",
check=True,
).stdout.strip()
def _init_repo(tmp_path, reftable=False):
repo = os.path.join(tmp_path, "repo")
ref_format = "reftable" if reftable else "files"
utils_for_test.init_git_tree(repo, ref_format=ref_format)
Path(os.path.join(repo, "a")).write_text("1")
_run(repo, "add", "a")
_run(repo, "commit", "-q", "-m", "init")
return repo
@pytest.mark.parametrize("reftable", [False, True])
def test_reads_refs(tmp_path, reftable):
if reftable and not utils_for_test.supports_reftable():
pytest.skip("reftable not supported")
repo = _init_repo(tmp_path, reftable=reftable)
gitdir = os.path.join(repo, ".git")
refs = git_refs.GitRefs(gitdir)
branch = _run(repo, "symbolic-ref", "--short", "HEAD")
head = _run(repo, "rev-parse", "HEAD")
assert refs.symref("HEAD") == f"refs/heads/{branch}"
assert refs.get("HEAD") == head
assert refs.get(f"refs/heads/{branch}") == head
@pytest.mark.parametrize("reftable", [False, True])
def test_updates_when_refs_change(tmp_path, reftable):
if reftable and not utils_for_test.supports_reftable():
pytest.skip("reftable not supported")
repo = _init_repo(tmp_path, reftable=reftable)
gitdir = os.path.join(repo, ".git")
refs = git_refs.GitRefs(gitdir)
head = _run(repo, "rev-parse", "HEAD")
assert refs.get("refs/heads/topic") == ""
_run(repo, "branch", "topic")
assert refs.get("refs/heads/topic") == head
_run(repo, "branch", "-D", "topic")
assert refs.get("refs/heads/topic") == ""
@pytest.mark.skipif(
not utils_for_test.supports_refs_migrate(),
reason="git refs migrate reftable support is required for this test",
)
def test_updates_when_storage_backend_toggles(tmp_path):
repo = _init_repo(tmp_path, reftable=False)
gitdir = os.path.join(repo, ".git")
refs = git_refs.GitRefs(gitdir)
head = _run(repo, "rev-parse", "HEAD")
assert refs.get("refs/heads/reftable-branch") == ""
_run(repo, "refs", "migrate", "--ref-format=reftable")
_run(repo, "branch", "reftable-branch")
assert refs.get("refs/heads/reftable-branch") == head
assert refs.get("refs/heads/files-branch") == ""
_run(repo, "refs", "migrate", "--ref-format=files")
_run(repo, "branch", "files-branch")
assert refs.get("refs/heads/files-branch") == head
+57 -1
View File
@@ -190,7 +190,7 @@ class SuperprojectTestCase(unittest.TestCase):
),
revision="refs/heads/main",
)
with mock.patch.object(self._superproject, "_branch", "junk"):
with mock.patch.object(self._superproject, "revision", "junk"):
sync_result = self._superproject.Sync(self.git_event_log)
self.assertFalse(sync_result.success)
self.assertTrue(sync_result.fatal)
@@ -461,6 +461,62 @@ class SuperprojectTestCase(unittest.TestCase):
"</manifest>",
)
def test_Init_success(self):
"""Test _Init succeeds and creates the work git dir."""
self.assertFalse(os.path.exists(self._superproject._work_git))
with mock.patch(
"git_superproject.GitCommand", autospec=True
) as mock_git_command:
instance = mock_git_command.return_value
instance.Wait.return_value = 0
self.assertTrue(self._superproject._Init())
mock_git_command.assert_called_once()
args, kwargs = mock_git_command.call_args
self.assertEqual(args[1][:2], ["init", "--bare"])
tmp_git_name = args[1][2]
self.assertTrue(
tmp_git_name.startswith(".tmp-superproject-initgitdir-")
)
self.assertTrue(os.path.exists(self._superproject._work_git))
tmp_git_path = os.path.join(
self._superproject._superproject_path, tmp_git_name
)
self.assertFalse(os.path.exists(tmp_git_path))
def test_Init_already_exists(self):
"""Test _Init returns early if the work git dir already exists."""
os.mkdir(self._superproject._superproject_path)
os.mkdir(self._superproject._work_git)
with mock.patch(
"git_superproject.GitCommand", autospec=True
) as mock_git_command:
self.assertTrue(self._superproject._Init())
mock_git_command.assert_not_called()
def test_Init_failure_cleans_up(self):
"""Test _Init cleans up the temporary directory if 'git init' fails."""
with mock.patch(
"git_superproject.GitCommand", autospec=True
) as mock_git_command:
instance = mock_git_command.return_value
instance.Wait.return_value = 1
instance.stderr = "mock git init failure"
self.assertFalse(self._superproject._Init())
mock_git_command.assert_called_once()
args, kwargs = mock_git_command.call_args
tmp_git_name = args[1][2]
tmp_git_path = os.path.join(
self._superproject._superproject_path, tmp_git_name
)
self.assertFalse(os.path.exists(tmp_git_path))
self.assertFalse(os.path.exists(self._superproject._work_git))
def test_Fetch(self):
manifest = self.getXmlManifest(
"""
+416 -334
View File
@@ -14,19 +14,28 @@
"""Unittests for the git_trace2_event_log.py module."""
import contextlib
import io
import json
import os
import re
import socket
import tempfile
import threading
import unittest
from typing import Any, Dict, List, Optional
from unittest import mock
import pytest
import git_trace2_event_log
import platform_utils
def serverLoggingThread(socket_path, server_ready, received_traces):
def server_logging_thread(
socket_path: str,
server_ready: threading.Condition,
received_traces: List[str],
) -> None:
"""Helper function to receive logs over a Unix domain socket.
Appends received messages on the provided socket and appends to
@@ -55,352 +64,425 @@ def serverLoggingThread(socket_path, server_ready, received_traces):
received_traces.extend(data.decode("utf-8").splitlines())
class EventLogTestCase(unittest.TestCase):
"""TestCase for the EventLog module."""
PARENT_SID_KEY = "GIT_TRACE2_PARENT_SID"
PARENT_SID_VALUE = "parent_sid"
SELF_SID_REGEX = r"repo-\d+T\d+Z-.*"
FULL_SID_REGEX = rf"^{PARENT_SID_VALUE}/{SELF_SID_REGEX}"
PARENT_SID_KEY = "GIT_TRACE2_PARENT_SID"
PARENT_SID_VALUE = "parent_sid"
SELF_SID_REGEX = r"repo-\d+T\d+Z-.*"
FULL_SID_REGEX = rf"^{PARENT_SID_VALUE}/{SELF_SID_REGEX}"
def setUp(self):
"""Load the event_log module every time."""
self._event_log_module = None
# By default we initialize with the expected case where
# repo launches us (so GIT_TRACE2_PARENT_SID is set).
env = {
self.PARENT_SID_KEY: self.PARENT_SID_VALUE,
}
self._event_log_module = git_trace2_event_log.EventLog(env=env)
self._log_data = None
@pytest.fixture
def event_log() -> git_trace2_event_log.EventLog:
"""Fixture for the EventLog module."""
# By default we initialize with the expected case where
# repo launches us (so GIT_TRACE2_PARENT_SID is set).
env = {PARENT_SID_KEY: PARENT_SID_VALUE}
return git_trace2_event_log.EventLog(env=env)
def verifyCommonKeys(
self, log_entry, expected_event_name=None, full_sid=True
def verify_common_keys(
log_entry: Dict[str, Any],
expected_event_name: Optional[str] = None,
full_sid: bool = True,
) -> None:
"""Helper function to verify common event log keys."""
assert "event" in log_entry
assert "sid" in log_entry
assert "thread" in log_entry
assert "time" in log_entry
# Do basic data format validation.
if expected_event_name:
assert expected_event_name == log_entry["event"]
if full_sid:
assert re.match(FULL_SID_REGEX, log_entry["sid"])
else:
assert re.match(SELF_SID_REGEX, log_entry["sid"])
assert re.match(r"^\d+-\d+-\d+T\d+:\d+:\d+\.\d+\+00:00$", log_entry["time"])
def read_log(log_path: str) -> List[Dict[str, Any]]:
"""Helper function to read log data into a list."""
log_data = []
with open(log_path, mode="rb") as f:
for line in f:
log_data.append(json.loads(line))
return log_data
def remove_prefix(s: str, prefix: str) -> str:
"""Return a copy string after removing |prefix| from |s|, if present or
the original string."""
if s.startswith(prefix):
return s[len(prefix) :]
else:
return s
def test_initial_state_with_parent_sid(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is set by parent."""
assert re.match(FULL_SID_REGEX, event_log.full_sid)
def test_initial_state_no_parent_sid() -> None:
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is not set."""
# Setup an empty environment dict (no parent sid).
event_log = git_trace2_event_log.EventLog(env={})
assert re.match(SELF_SID_REGEX, event_log.full_sid)
def test_version_event(event_log: git_trace2_event_log.EventLog) -> None:
"""Test 'version' event data is valid.
Verify that the 'version' event is written even when no other
events are added.
Expected event log:
<version event>
"""
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
# A log with no added events should only have the version entry.
assert len(log_data) == 1
version_event = log_data[0]
verify_common_keys(version_event, expected_event_name="version")
# Check for 'version' event specific fields.
assert "evt" in version_event
assert "exe" in version_event
# Verify "evt" version field is a string.
assert isinstance(version_event["evt"], str)
def test_start_event(event_log: git_trace2_event_log.EventLog) -> None:
"""Test and validate 'start' event data is valid.
Expected event log:
<version event>
<start event>
"""
event_log.StartEvent([])
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
start_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(start_event, expected_event_name="start")
# Check for 'start' event specific fields.
assert "argv" in start_event
assert isinstance(start_event["argv"], list)
def test_exit_event_result_none(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test 'exit' event data is valid when result is None.
We expect None result to be converted to 0 in the exit event data.
Expected event log:
<version event>
<exit event>
"""
event_log.ExitEvent(None)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
exit_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(exit_event, expected_event_name="exit")
# Check for 'exit' event specific fields.
assert "code" in exit_event
# 'None' result should convert to 0 (successful) return code.
assert exit_event["code"] == 0
def test_exit_event_result_integer(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test 'exit' event data is valid when result is an integer.
Expected event log:
<version event>
<exit event>
"""
event_log.ExitEvent(2)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
exit_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(exit_event, expected_event_name="exit")
# Check for 'exit' event specific fields.
assert "code" in exit_event
assert exit_event["code"] == 2
def test_command_event(event_log: git_trace2_event_log.EventLog) -> None:
"""Test and validate 'command' event data is valid.
Expected event log:
<version event>
<command event>
"""
event_log.CommandEvent(name="repo", subcommands=["init", "this"])
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
command_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(command_event, expected_event_name="cmd_name")
# Check for 'command' event specific fields.
assert "name" in command_event
assert command_event["name"] == "repo-init-this"
def test_def_params_event_repo_config(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test 'def_params' event data outputs only repo config keys.
Expected event log:
<version event>
<def_param event>
<def_param event>
"""
config = {
"git.foo": "bar",
"repo.partialclone": "true",
"repo.partialclonefilter": "blob:none",
}
event_log.DefParamRepoEvents(config)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 3
def_param_events = log_data[1:]
verify_common_keys(log_data[0], expected_event_name="version")
for event in def_param_events:
verify_common_keys(event, expected_event_name="def_param")
# Check for 'def_param' event specific fields.
assert "param" in event
assert "value" in event
assert event["param"].startswith("repo.")
def test_def_params_event_no_repo_config(
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test 'def_params' event data won't output non-repo config keys.
Expected event log:
<version event>
"""
config = {
"git.foo": "bar",
"git.core.foo2": "baz",
}
event_log.DefParamRepoEvents(config)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 1
verify_common_keys(log_data[0], expected_event_name="version")
def test_data_event_config(event_log: git_trace2_event_log.EventLog) -> None:
"""Test 'data' event data outputs all config keys.
Expected event log:
<version event>
<data event>
<data event>
"""
config = {
"git.foo": "bar",
"repo.partialclone": "false",
"repo.syncstate.superproject.hassuperprojecttag": "true",
"repo.syncstate.superproject.sys.argv": ["--", "sync", "protobuf"],
}
prefix_value = "prefix"
event_log.LogDataConfigEvents(config, prefix_value)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 5
data_events = log_data[1:]
verify_common_keys(log_data[0], expected_event_name="version")
for event in data_events:
verify_common_keys(event)
# Check for 'data' event specific fields.
assert "key" in event
assert "value" in event
key = event["key"]
key = remove_prefix(key, f"{prefix_value}/")
value = event["value"]
assert event_log.GetDataEventName(value) == event["event"]
assert key in config
assert value == config[key]
def test_error_event(event_log: git_trace2_event_log.EventLog) -> None:
"""Test and validate 'error' event data is valid.
Expected event log:
<version event>
<error event>
"""
msg = "invalid option: --cahced"
fmt = "invalid option: %s"
event_log.ErrorEvent(msg, fmt)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = event_log.Write(path=tempdir)
log_data = read_log(log_path)
assert len(log_data) == 2
error_event = log_data[1]
verify_common_keys(log_data[0], expected_event_name="version")
verify_common_keys(error_event, expected_event_name="error")
# Check for 'error' event specific fields.
assert "msg" in error_event
assert "fmt" in error_event
assert error_event["msg"] == f"RepoErrorEvent:{msg}"
assert error_event["fmt"] == f"RepoErrorEvent:{fmt}"
def test_write_with_filename(event_log: git_trace2_event_log.EventLog) -> None:
"""Test Write() with a path to a file exits with None."""
assert event_log.Write(path="path/to/file") is None
def test_write_with_git_config(
tmp_path,
event_log: git_trace2_event_log.EventLog,
) -> None:
"""Test Write() uses the git config path when 'git config' call succeeds."""
with mock.patch.object(
event_log,
"_GetEventTargetPath",
return_value=str(tmp_path),
):
"""Helper function to verify common event log keys."""
self.assertIn("event", log_entry)
self.assertIn("sid", log_entry)
self.assertIn("thread", log_entry)
self.assertIn("time", log_entry)
assert os.path.dirname(event_log.Write()) == str(tmp_path)
# Do basic data format validation.
if expected_event_name:
self.assertEqual(expected_event_name, log_entry["event"])
if full_sid:
self.assertRegex(log_entry["sid"], self.FULL_SID_REGEX)
else:
self.assertRegex(log_entry["sid"], self.SELF_SID_REGEX)
self.assertRegex(
log_entry["time"], r"^\d+-\d+-\d+T\d+:\d+:\d+\.\d+\+00:00$"
def test_write_no_git_config(event_log: git_trace2_event_log.EventLog) -> None:
"""Test Write() with no git config variable present exits with None."""
with mock.patch.object(event_log, "_GetEventTargetPath", return_value=None):
assert event_log.Write() is None
def test_write_non_string(event_log: git_trace2_event_log.EventLog) -> None:
"""Test Write() with non-string type for |path| throws TypeError."""
with pytest.raises(TypeError):
event_log.Write(path=1234)
@pytest.mark.skipif(
not hasattr(socket, "AF_UNIX"), reason="Requires AF_UNIX sockets"
)
def test_write_socket(event_log: git_trace2_event_log.EventLog) -> None:
"""Test Write() with Unix domain socket and validate received traces."""
received_traces: List[str] = []
with tempfile.TemporaryDirectory(prefix="test_server_sockets") as tempdir:
socket_path = os.path.join(tempdir, "server.sock")
server_ready = threading.Condition()
# Start "server" listening on Unix domain socket at socket_path.
server_thread = threading.Thread(
target=server_logging_thread,
args=(socket_path, server_ready, received_traces),
)
try:
server_thread.start()
def readLog(self, log_path):
"""Helper function to read log data into a list."""
log_data = []
with open(log_path, mode="rb") as f:
for line in f:
log_data.append(json.loads(line))
return log_data
with server_ready:
server_ready.wait(timeout=120)
def remove_prefix(self, s, prefix):
"""Return a copy string after removing |prefix| from |s|, if present or
the original string."""
if s.startswith(prefix):
return s[len(prefix) :]
else:
return s
event_log.StartEvent([])
path = event_log.Write(path=f"af_unix:{socket_path}")
finally:
server_thread.join(timeout=5)
def test_initial_state_with_parent_sid(self):
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is set by parent."""
self.assertRegex(self._event_log_module.full_sid, self.FULL_SID_REGEX)
assert path == f"af_unix:stream:{socket_path}"
assert len(received_traces) == 2
version_event = json.loads(received_traces[0])
start_event = json.loads(received_traces[1])
verify_common_keys(version_event, expected_event_name="version")
verify_common_keys(start_event, expected_event_name="start")
# Check for 'start' event specific fields.
assert "argv" in start_event
assert isinstance(start_event["argv"], list)
def test_initial_state_no_parent_sid(self):
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is not set."""
# Setup an empty environment dict (no parent sid).
self._event_log_module = git_trace2_event_log.EventLog(env={})
self.assertRegex(self._event_log_module.full_sid, self.SELF_SID_REGEX)
def test_version_event(self):
"""Test 'version' event data is valid.
class TestEventLogVerbose:
"""TestCase for the EventLog module verbose logging."""
Verify that the 'version' event is written even when no other
events are addded.
def test_write_socket_error_no_verbose(self) -> None:
"""Test Write() suppression of socket errors when not verbose."""
event_log = git_trace2_event_log.EventLog(env={})
event_log.verbose = False
with contextlib.redirect_stderr(
io.StringIO()
) as mock_stderr, mock.patch("socket.socket", side_effect=OSError):
event_log.Write(path="af_unix:stream:/tmp/test_sock")
assert mock_stderr.getvalue() == ""
Expected event log:
<version event>
"""
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
# A log with no added events should only have the version entry.
self.assertEqual(len(self._log_data), 1)
version_event = self._log_data[0]
self.verifyCommonKeys(version_event, expected_event_name="version")
# Check for 'version' event specific fields.
self.assertIn("evt", version_event)
self.assertIn("exe", version_event)
# Verify "evt" version field is a string.
self.assertIsInstance(version_event["evt"], str)
def test_start_event(self):
"""Test and validate 'start' event data is valid.
Expected event log:
<version event>
<start event>
"""
self._event_log_module.StartEvent([])
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
start_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(start_event, expected_event_name="start")
# Check for 'start' event specific fields.
self.assertIn("argv", start_event)
self.assertTrue(isinstance(start_event["argv"], list))
def test_exit_event_result_none(self):
"""Test 'exit' event data is valid when result is None.
We expect None result to be converted to 0 in the exit event data.
Expected event log:
<version event>
<exit event>
"""
self._event_log_module.ExitEvent(None)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
exit_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(exit_event, expected_event_name="exit")
# Check for 'exit' event specific fields.
self.assertIn("code", exit_event)
# 'None' result should convert to 0 (successful) return code.
self.assertEqual(exit_event["code"], 0)
def test_exit_event_result_integer(self):
"""Test 'exit' event data is valid when result is an integer.
Expected event log:
<version event>
<exit event>
"""
self._event_log_module.ExitEvent(2)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
exit_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(exit_event, expected_event_name="exit")
# Check for 'exit' event specific fields.
self.assertIn("code", exit_event)
self.assertEqual(exit_event["code"], 2)
def test_command_event(self):
"""Test and validate 'command' event data is valid.
Expected event log:
<version event>
<command event>
"""
self._event_log_module.CommandEvent(
name="repo", subcommands=["init", "this"]
)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
command_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(command_event, expected_event_name="cmd_name")
# Check for 'command' event specific fields.
self.assertIn("name", command_event)
self.assertEqual(command_event["name"], "repo-init-this")
def test_def_params_event_repo_config(self):
"""Test 'def_params' event data outputs only repo config keys.
Expected event log:
<version event>
<def_param event>
<def_param event>
"""
config = {
"git.foo": "bar",
"repo.partialclone": "true",
"repo.partialclonefilter": "blob:none",
}
self._event_log_module.DefParamRepoEvents(config)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 3)
def_param_events = self._log_data[1:]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
for event in def_param_events:
self.verifyCommonKeys(event, expected_event_name="def_param")
# Check for 'def_param' event specific fields.
self.assertIn("param", event)
self.assertIn("value", event)
self.assertTrue(event["param"].startswith("repo."))
def test_def_params_event_no_repo_config(self):
"""Test 'def_params' event data won't output non-repo config keys.
Expected event log:
<version event>
"""
config = {
"git.foo": "bar",
"git.core.foo2": "baz",
}
self._event_log_module.DefParamRepoEvents(config)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 1)
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
def test_data_event_config(self):
"""Test 'data' event data outputs all config keys.
Expected event log:
<version event>
<data event>
<data event>
"""
config = {
"git.foo": "bar",
"repo.partialclone": "false",
"repo.syncstate.superproject.hassuperprojecttag": "true",
"repo.syncstate.superproject.sys.argv": ["--", "sync", "protobuf"],
}
prefix_value = "prefix"
self._event_log_module.LogDataConfigEvents(config, prefix_value)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 5)
data_events = self._log_data[1:]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
for event in data_events:
self.verifyCommonKeys(event)
# Check for 'data' event specific fields.
self.assertIn("key", event)
self.assertIn("value", event)
key = event["key"]
key = self.remove_prefix(key, f"{prefix_value}/")
value = event["value"]
self.assertEqual(
self._event_log_module.GetDataEventName(value), event["event"]
)
self.assertTrue(key in config and value == config[key])
def test_error_event(self):
"""Test and validate 'error' event data is valid.
Expected event log:
<version event>
<error event>
"""
msg = "invalid option: --cahced"
fmt = "invalid option: %s"
self._event_log_module.ErrorEvent(msg, fmt)
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 2)
error_event = self._log_data[1]
self.verifyCommonKeys(self._log_data[0], expected_event_name="version")
self.verifyCommonKeys(error_event, expected_event_name="error")
# Check for 'error' event specific fields.
self.assertIn("msg", error_event)
self.assertIn("fmt", error_event)
self.assertEqual(error_event["msg"], f"RepoErrorEvent:{msg}")
self.assertEqual(error_event["fmt"], f"RepoErrorEvent:{fmt}")
def test_write_with_filename(self):
"""Test Write() with a path to a file exits with None."""
self.assertIsNone(self._event_log_module.Write(path="path/to/file"))
def test_write_with_git_config(self):
"""Test Write() uses the git config path when 'git config' call
succeeds."""
with tempfile.TemporaryDirectory(prefix="event_log_tests") as tempdir:
with mock.patch.object(
self._event_log_module,
"_GetEventTargetPath",
return_value=tempdir,
):
self.assertEqual(
os.path.dirname(self._event_log_module.Write()), tempdir
)
def test_write_no_git_config(self):
"""Test Write() with no git config variable present exits with None."""
with mock.patch.object(
self._event_log_module, "_GetEventTargetPath", return_value=None
def test_write_socket_error_verbose(self) -> None:
"""Test Write() printing of socket errors when verbose."""
event_log = git_trace2_event_log.EventLog(env={})
event_log.verbose = True
with contextlib.redirect_stderr(
io.StringIO()
) as mock_stderr, mock.patch(
"socket.socket", side_effect=OSError("Mock error")
):
self.assertIsNone(self._event_log_module.Write())
def test_write_non_string(self):
"""Test Write() with non-string type for |path| throws TypeError."""
with self.assertRaises(TypeError):
self._event_log_module.Write(path=1234)
def test_write_socket(self):
"""Test Write() with Unix domain socket for |path| and validate received
traces."""
received_traces = []
with tempfile.TemporaryDirectory(
prefix="test_server_sockets"
) as tempdir:
socket_path = os.path.join(tempdir, "server.sock")
server_ready = threading.Condition()
# Start "server" listening on Unix domain socket at socket_path.
server_thread = threading.Thread(
target=serverLoggingThread,
args=(socket_path, server_ready, received_traces),
event_log.Write(path="af_unix:stream:/tmp/test_sock")
assert (
"git trace2 logging failed: Mock error"
in mock_stderr.getvalue()
)
try:
server_thread.start()
with server_ready:
server_ready.wait(timeout=120)
def test_write_file_error_no_verbose(self) -> None:
"""Test Write() suppression of file errors when not verbose."""
event_log = git_trace2_event_log.EventLog(env={})
event_log.verbose = False
with contextlib.redirect_stderr(
io.StringIO()
) as mock_stderr, mock.patch(
"tempfile.NamedTemporaryFile", side_effect=FileExistsError
):
event_log.Write(path="/tmp")
assert mock_stderr.getvalue() == ""
self._event_log_module.StartEvent([])
path = self._event_log_module.Write(
path=f"af_unix:{socket_path}"
)
finally:
server_thread.join(timeout=5)
self.assertEqual(path, f"af_unix:stream:{socket_path}")
self.assertEqual(len(received_traces), 2)
version_event = json.loads(received_traces[0])
start_event = json.loads(received_traces[1])
self.verifyCommonKeys(version_event, expected_event_name="version")
self.verifyCommonKeys(start_event, expected_event_name="start")
# Check for 'start' event specific fields.
self.assertIn("argv", start_event)
self.assertIsInstance(start_event["argv"], list)
def test_write_file_error_verbose(self) -> None:
"""Test Write() printing of file errors when verbose."""
event_log = git_trace2_event_log.EventLog(env={})
event_log.verbose = True
with contextlib.redirect_stderr(
io.StringIO()
) as mock_stderr, mock.patch(
"tempfile.NamedTemporaryFile",
side_effect=FileExistsError("Mock error"),
):
event_log.Write(path="/tmp")
assert (
"git trace2 logging failed: FileExistsError"
in mock_stderr.getvalue()
)
+84 -32
View File
@@ -14,42 +14,94 @@
"""Unittests for the hooks.py module."""
import unittest
from io import StringIO
import sys
import pytest
import hooks
class RepoHookShebang(unittest.TestCase):
"""Check shebang parsing in RepoHook."""
@pytest.mark.parametrize(
"data",
(
"",
"#\n# foo\n",
"# Bad shebang in script\n#!/foo\n",
),
)
def test_no_shebang(data: str) -> None:
"""Lines w/out shebangs should be rejected."""
assert hooks.RepoHook._ExtractInterpFromShebang(data) is None
def test_no_shebang(self):
"""Lines w/out shebangs should be rejected."""
DATA = ("", "#\n# foo\n", "# Bad shebang in script\n#!/foo\n")
for data in DATA:
self.assertIsNone(hooks.RepoHook._ExtractInterpFromShebang(data))
def test_direct_interp(self):
"""Lines whose shebang points directly to the interpreter."""
DATA = (
("#!/foo", "/foo"),
("#! /foo", "/foo"),
("#!/bin/foo ", "/bin/foo"),
("#! /usr/foo ", "/usr/foo"),
("#! /usr/foo -args", "/usr/foo"),
)
for shebang, interp in DATA:
self.assertEqual(
hooks.RepoHook._ExtractInterpFromShebang(shebang), interp
)
@pytest.mark.parametrize(
"shebang, interp",
(
("#!/foo", "/foo"),
("#! /foo", "/foo"),
("#!/bin/foo ", "/bin/foo"),
("#! /usr/foo ", "/usr/foo"),
("#! /usr/foo -args", "/usr/foo"),
),
)
def test_direct_interp(shebang: str, interp: str) -> None:
"""Lines whose shebang points directly to the interpreter."""
assert hooks.RepoHook._ExtractInterpFromShebang(shebang) == interp
def test_env_interp(self):
"""Lines whose shebang launches through `env`."""
DATA = (
("#!/usr/bin/env foo", "foo"),
("#!/bin/env foo", "foo"),
("#! /bin/env /bin/foo ", "/bin/foo"),
)
for shebang, interp in DATA:
self.assertEqual(
hooks.RepoHook._ExtractInterpFromShebang(shebang), interp
)
@pytest.mark.parametrize(
"shebang, interp",
(
("#!/usr/bin/env foo", "foo"),
("#!/bin/env foo", "foo"),
("#! /bin/env /bin/foo ", "/bin/foo"),
),
)
def test_env_interp(shebang: str, interp: str) -> None:
"""Lines whose shebang launches through `env`."""
assert hooks.RepoHook._ExtractInterpFromShebang(shebang) == interp
def test_post_sync_argument_validation() -> None:
"""Test that post-sync hook requires exact API arguments."""
class FakeProject:
def __init__(self):
self.worktree = "/some/path"
self.enabled_repo_hooks = ["post-sync"]
hook = hooks.RepoHook(
hook_type="post-sync",
hooks_project=FakeProject(),
repo_topdir="/topdir",
manifest_url="https://gerrit",
allow_all_hooks=True,
)
old_stderr = sys.stderr
sys.stderr = StringIO()
try:
# Call with missing arg `sync_duration_seconds`
res = hook.Run(repo_topdir="/topdir")
assert res is False
assert "hook 'post-sync' called incorrectly" in sys.stderr.getvalue()
# Mock _CheckHook and _ExecuteHook to test success path
hook._CheckHook = lambda: None
executed_kwargs = {}
def fake_execute(**kw):
executed_kwargs.update(kw)
hook._ExecuteHook = fake_execute
res = hook.Run(repo_topdir="/topdir", sync_duration_seconds=12.345)
assert res is True
assert executed_kwargs.get("sync_duration_seconds") == 12.345
finally:
sys.stderr = old_stderr
+166
View File
@@ -0,0 +1,166 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests for the main repo script and subcommand routing."""
from unittest import mock
import pytest
from main import _Repo
@pytest.fixture(name="repo")
def fixture_repo():
repo = _Repo("repodir")
# Overriding the command list here ensures that we are only testing
# against a fixed set of commands, reducing fragility to new
# subcommands being added to the main repo tool.
repo.commands = {"start": None, "sync": None, "smart": None}
return repo
@pytest.fixture(name="mock_config")
def fixture_mock_config():
return mock.MagicMock()
@mock.patch("time.sleep")
def test_autocorrect_delay(mock_sleep, repo, mock_config):
"""Test autocorrect with positive delay."""
mock_config.GetString.return_value = "10"
res = repo._autocorrect_command_name("tart", mock_config)
mock_config.GetString.assert_called_with("help.autocorrect")
mock_sleep.assert_called_with(1.0)
assert res == "start"
@mock.patch("time.sleep")
def test_autocorrect_delay_one(mock_sleep, repo, mock_config):
"""Test autocorrect with '1' (0.1s delay, not immediate)."""
mock_config.GetString.return_value = "1"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_called_with(0.1)
assert res == "start"
@mock.patch("time.sleep", side_effect=KeyboardInterrupt())
def test_autocorrect_delay_interrupt(mock_sleep, repo, mock_config):
"""Test autocorrect handles KeyboardInterrupt during delay."""
mock_config.GetString.return_value = "10"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_called_with(1.0)
assert res is None
@mock.patch("time.sleep")
def test_autocorrect_immediate(mock_sleep, repo, mock_config):
"""Test autocorrect with immediate/negative delay."""
# Test numeric negative.
mock_config.GetString.return_value = "-1"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_not_called()
assert res == "start"
# Test string boolean "true".
mock_config.GetString.return_value = "true"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_not_called()
assert res == "start"
# Test string boolean "yes".
mock_config.GetString.return_value = "YES"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_not_called()
assert res == "start"
# Test string boolean "immediate".
mock_config.GetString.return_value = "Immediate"
res = repo._autocorrect_command_name("tart", mock_config)
mock_sleep.assert_not_called()
assert res == "start"
def test_autocorrect_zero_or_show(repo, mock_config):
"""Test autocorrect with zero delay (suggestions only)."""
# Test numeric zero.
mock_config.GetString.return_value = "0"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
# Test string boolean "false".
mock_config.GetString.return_value = "False"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
# Test string boolean "show".
mock_config.GetString.return_value = "show"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
def test_autocorrect_never(repo, mock_config):
"""Test autocorrect with 'never'."""
mock_config.GetString.return_value = "never"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
@mock.patch("builtins.input", return_value="y")
def test_autocorrect_prompt_yes(mock_input, repo, mock_config):
"""Test autocorrect with prompt and user answers yes."""
mock_config.GetString.return_value = "prompt"
res = repo._autocorrect_command_name("tart", mock_config)
assert res == "start"
@mock.patch("builtins.input", return_value="n")
def test_autocorrect_prompt_no(mock_input, repo, mock_config):
"""Test autocorrect with prompt and user answers no."""
mock_config.GetString.return_value = "prompt"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
@mock.patch("builtins.input", return_value="y")
def test_autocorrect_multiple_candidates(mock_input, repo, mock_config):
"""Test autocorrect with multiple matches forces a prompt."""
mock_config.GetString.return_value = "10" # Normally just delay
# 'snart' matches both 'start' and 'smart' with > 0.7 ratio
res = repo._autocorrect_command_name("snart", mock_config)
# Because there are multiple candidates, it should prompt
mock_input.assert_called_once()
assert res == "start"
@mock.patch("builtins.input", side_effect=KeyboardInterrupt())
def test_autocorrect_prompt_interrupt(mock_input, repo, mock_config):
"""Test autocorrect with prompt and user interrupts."""
mock_config.GetString.return_value = "prompt"
res = repo._autocorrect_command_name("tart", mock_config)
assert res is None
+821 -454
View File
File diff suppressed because it is too large Load Diff
+91 -27
View File
@@ -1,4 +1,4 @@
# Copyright 2021 The Android Open Source Project
# Copyright (C) 2021 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -14,39 +14,103 @@
"""Unittests for the platform_utils.py module."""
import os
import tempfile
import unittest
from pathlib import Path
import pytest
import platform_utils
class RemoveTests(unittest.TestCase):
"""Check remove() helper."""
def test_remove_missing_ok(tmp_path: Path) -> None:
"""Check missing_ok handling."""
path = tmp_path / "test"
def testMissingOk(self):
"""Check missing_ok handling."""
with tempfile.TemporaryDirectory() as tmpdir:
path = os.path.join(tmpdir, "test")
# Should not fail.
platform_utils.remove(path, missing_ok=True)
# Should not fail.
platform_utils.remove(path, missing_ok=True)
# Should fail.
with pytest.raises(OSError):
platform_utils.remove(path)
with pytest.raises(OSError):
platform_utils.remove(path, missing_ok=False)
# Should fail.
self.assertRaises(OSError, platform_utils.remove, path)
self.assertRaises(
OSError, platform_utils.remove, path, missing_ok=False
)
# Should not fail if it exists.
path.touch()
platform_utils.remove(path, missing_ok=True)
assert not path.exists()
# Should not fail if it exists.
open(path, "w").close()
platform_utils.remove(path, missing_ok=True)
self.assertFalse(os.path.exists(path))
path.touch()
platform_utils.remove(path)
assert not path.exists()
open(path, "w").close()
platform_utils.remove(path)
self.assertFalse(os.path.exists(path))
path.touch()
platform_utils.remove(path, missing_ok=False)
assert not path.exists()
open(path, "w").close()
platform_utils.remove(path, missing_ok=False)
self.assertFalse(os.path.exists(path))
def test_removedirs_nonexistent(tmp_path: Path) -> None:
"""removedirs should silently succeed on nonexistent paths."""
platform_utils.removedirs(tmp_path / "does-not-exist")
def test_removedirs_symlink(tmp_path: Path) -> None:
"""removedirs should remove a symlink."""
link = tmp_path / "link"
link.symlink_to("target")
platform_utils.removedirs(link)
assert not link.exists()
def test_removedirs_empty_dir(tmp_path: Path) -> None:
"""removedirs should remove an empty directory."""
d = tmp_path / "empty"
d.mkdir()
platform_utils.removedirs(d)
assert not d.exists()
def test_removedirs_nested_empty_dirs(tmp_path: Path) -> None:
"""removedirs should remove nested empty directories."""
d = tmp_path / "a" / "b" / "c"
d.mkdir(parents=True)
platform_utils.removedirs(tmp_path / "a")
assert not (tmp_path / "a").exists()
def test_removedirs_symlinks_inside_dir(tmp_path: Path) -> None:
"""removedirs should remove symlinks inside a directory."""
d = tmp_path / "dir"
d.mkdir()
(d / "link1").symlink_to("target1")
(d / "link2").symlink_to("target2")
platform_utils.removedirs(d)
assert not d.exists()
def test_removedirs_preserves_user_files(tmp_path: Path) -> None:
"""removedirs should not delete regular files or their parent dirs."""
d = tmp_path / "dir"
d.mkdir()
(d / "link").symlink_to("target")
(d / "user-file.txt").write_text("keep me")
platform_utils.removedirs(d)
assert d.exists()
assert not (d / "link").exists()
assert (d / "user-file.txt").read_text() == "keep me"
def test_removedirs_deep_nested_with_symlinks(tmp_path: Path) -> None:
"""removedirs should handle deep nesting: sub/dir/target."""
d = tmp_path / "sub" / "dir"
d.mkdir(parents=True)
(d / "link").symlink_to("target")
platform_utils.removedirs(tmp_path / "sub")
assert not (tmp_path / "sub").exists()
def test_removedirs_regular_file_noop(tmp_path: Path) -> None:
"""removedirs should not delete a regular file."""
f = tmp_path / "file.txt"
f.write_text("data")
platform_utils.removedirs(f)
assert f.exists()
+435 -22
View File
@@ -17,37 +17,22 @@
import contextlib
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
from typing import Optional
import unittest
from unittest import mock
import utils_for_test
import error
import git_command
import git_config
import manifest_xml
import platform_utils
import project
@contextlib.contextmanager
def TempGitTree():
"""Create a new empty git checkout for testing."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
# Tests need to assume, that main is default branch at init,
# which is not supported in config until 2.28.
cmd = ["git", "init"]
if git_command.git_require((2, 28, 0)):
cmd += ["--initial-branch=main"]
else:
# Use template dir for init.
templatedir = tempfile.mkdtemp(prefix=".test-template")
with open(os.path.join(templatedir, "HEAD"), "w") as fp:
fp.write("ref: refs/heads/main\n")
cmd += ["--template", templatedir]
subprocess.check_call(cmd, cwd=tempdir)
yield tempdir
class FakeProject:
"""A fake for Project for basic functionality."""
@@ -63,13 +48,16 @@ class FakeProject:
)
self.config = git_config.GitConfig.ForRepository(gitdir=self.gitdir)
def RelPath(self, local: Optional[bool] = None) -> str:
return self.name
class ReviewableBranchTests(unittest.TestCase):
"""Check ReviewableBranch behavior."""
def test_smoke(self):
"""A quick run through everything."""
with TempGitTree() as tempdir:
with utils_for_test.TempGitTree() as tempdir:
fakeproj = FakeProject(tempdir)
# Generate some commits.
@@ -116,6 +104,111 @@ class ProjectTests(unittest.TestCase):
"abcd00%21%21_%2b",
)
@unittest.skipUnless(
utils_for_test.supports_reftable(),
"git reftable support is required for this test",
)
def test_get_head_unborn_reftable(self):
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
subprocess.check_call(
[
"git",
"-c",
"init.defaultRefFormat=reftable",
"init",
"-q",
tempdir,
]
)
fakeproj = FakeProject(tempdir)
expected = subprocess.check_output(
["git", "-C", tempdir, "symbolic-ref", "-q", "HEAD"],
encoding="utf-8",
).strip()
self.assertEqual(expected, fakeproj.work_git.GetHead())
def _get_derived_subproject_url(self, submodule_url):
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
class FakeManifest:
def __init__(self, topdir):
self.topdir = topdir
self.globalConfig = None
self.is_multimanifest = False
self.path_prefix = ""
self.paths = {}
def GetSubprojectName(self, parent, path):
return path
def GetSubprojectPaths(self, parent, name, path):
relpath = path
worktree = os.path.join(self.topdir, path)
gitdir = os.path.join(self.topdir, f"{path}.git")
objdir = os.path.join(self.topdir, f"{path}.obj")
os.makedirs(worktree, exist_ok=True)
os.makedirs(gitdir, exist_ok=True)
os.makedirs(objdir, exist_ok=True)
return relpath, worktree, gitdir, objdir
manifest = FakeManifest(tempdir)
worktree = os.path.join(tempdir, "parent")
gitdir = os.path.join(tempdir, "parent.git")
objdir = os.path.join(tempdir, "parent.obj")
os.makedirs(worktree)
os.makedirs(gitdir)
os.makedirs(objdir)
parent = project.Project(
manifest=manifest,
name="parent",
remote=project.RemoteSpec(
"origin", url="https://example.com/platform/superproject"
),
gitdir=gitdir,
objdir=objdir,
worktree=worktree,
relpath="parent",
revisionExpr="refs/heads/main",
revisionId=None,
)
def fake_get_submodules(current):
if current is parent:
return [("subrev", "child", submodule_url, "false")]
return []
with mock.patch.object(
project.Project, "_GetSubmodules", autospec=True
) as get_submodules:
get_submodules.side_effect = fake_get_submodules
result = parent.GetDerivedSubprojects()
self.assertEqual(1, len(result))
return result[0].remote.url
def test_derived_subproject_joins_only_git_relative_urls(self):
tests = (
(
"./submodule",
"https://example.com/platform/superproject/submodule",
),
("../sibling", "https://example.com/platform/sibling"),
)
for submodule_url, expected in tests:
with self.subTest(submodule_url=submodule_url):
self.assertEqual(
expected, self._get_derived_subproject_url(submodule_url)
)
def test_derived_subproject_leaves_dot_prefixed_names_unchanged(self):
for submodule_url in (".foo", "..bar"):
with self.subTest(submodule_url=submodule_url):
self.assertEqual(
submodule_url,
self._get_derived_subproject_url(submodule_url),
)
class CopyLinkTestCase(unittest.TestCase):
"""TestCase for stub repo client checkouts.
@@ -354,11 +447,53 @@ class LinkFile(CopyLinkTestCase):
os.path.join("git-project", "foo.txt"), os.readlink(dest)
)
def test_replace_empty_dir_with_symlink(self):
"""A linkfile should replace an empty real directory at the dest path.
This is the common case: the old linkfiles inside the directory were
already cleaned up by UpdateCopyLinkfileList, leaving an empty parent
directory behind.
"""
src_dir = os.path.join(self.worktree, "dot-llms")
os.makedirs(src_dir)
dest = os.path.join(self.topdir, "mydir")
os.makedirs(dest)
lf = self.LinkFile("dot-llms", "mydir")
lf._Link()
self.assertTrue(os.path.islink(dest))
self.assertEqual(
os.path.join("git-project", "dot-llms"), os.readlink(dest)
)
def test_nonempty_dir_not_clobbered(self):
"""A linkfile must not delete a non-empty directory.
If the user created files in a directory that a new linkfile wants
to replace, __linkIt should fail safely rather than deleting content.
"""
src_dir = os.path.join(self.worktree, "dot-llms")
os.makedirs(src_dir)
dest = os.path.join(self.topdir, "mydir")
os.makedirs(dest)
user_file = os.path.join(dest, "user-notes.txt")
self.touch(user_file)
lf = self.LinkFile("dot-llms", "mydir")
lf._Link()
# The directory should NOT be replaced — user content is preserved.
self.assertFalse(os.path.islink(dest))
self.assertTrue(os.path.isdir(dest))
self.assertTrue(os.path.exists(user_file))
class MigrateWorkTreeTests(unittest.TestCase):
"""Check _MigrateOldWorkTreeGitDir handling."""
_SYMLINKS = {
# go/keep-sorted start
"config",
"description",
"hooks",
@@ -367,9 +502,11 @@ class MigrateWorkTreeTests(unittest.TestCase):
"objects",
"packed-refs",
"refs",
"reftable",
"rr-cache",
"shallow",
"svn",
# go/keep-sorted end
}
_FILES = {
"COMMIT_EDITMSG",
@@ -448,6 +585,25 @@ class MigrateWorkTreeTests(unittest.TestCase):
for name in self._SYMLINKS:
self.assertTrue((dotgit / name).is_symlink())
def test_reftable_anchor_with_refs_dir(self):
"""Migrate when reftable/ and refs/ are directories."""
with self._simple_layout() as tempdir:
dotgit = tempdir / "src/test/.git"
(dotgit / "refs").unlink()
(dotgit / "refs").mkdir()
(dotgit / "refs" / "heads").write_text("dummy")
(dotgit / "reftable").unlink()
(dotgit / "reftable").mkdir()
(dotgit / "reftable" / "tables.list").write_text("dummy")
project.Project._MigrateOldWorkTreeGitDir(str(dotgit))
self.assertTrue(dotgit.is_symlink())
self.assertEqual(
os.readlink(dotgit),
os.path.normpath("../../.repo/projects/src/test.git"),
)
class ManifestPropertiesFetchedCorrectly(unittest.TestCase):
"""Ensure properties are fetched properly."""
@@ -467,7 +623,7 @@ class ManifestPropertiesFetchedCorrectly(unittest.TestCase):
def test_manifest_config_properties(self):
"""Test we are fetching the manifest config properties correctly."""
with TempGitTree() as tempdir:
with utils_for_test.TempGitTree() as tempdir:
fakeproj = self.setUpManifest(tempdir)
# Set property using the expected Set method, then ensure
@@ -534,3 +690,260 @@ class ManifestPropertiesFetchedCorrectly(unittest.TestCase):
fakeproj.config.SetString("manifest.platform", "auto")
self.assertEqual(fakeproj.manifest_platform, "auto")
class StatelessSyncTests(unittest.TestCase):
"""Tests for stateless sync strategy."""
def _get_project(self, tempdir):
manifest = mock.MagicMock()
manifest.manifestProject.depth = None
manifest.manifestProject.dissociate = False
manifest.manifestProject.clone_filter = None
manifest.is_multimanifest = False
manifest.manifestProject.config.GetBoolean.return_value = False
remote = mock.MagicMock()
remote.name = "origin"
remote.url = "http://"
proj = project.Project(
manifest=manifest,
name="test-project",
remote=remote,
gitdir=os.path.join(tempdir, ".git"),
objdir=os.path.join(tempdir, ".git"),
worktree=tempdir,
relpath="test-project",
revisionExpr="1234abcd",
revisionId=None,
sync_strategy="stateless",
)
proj._CheckForImmutableRevision = mock.MagicMock(return_value=False)
proj._LsRemote = mock.MagicMock(
return_value="1234abcd\trefs/heads/main\n"
)
proj.bare_git = mock.MagicMock()
proj.bare_git.rev_parse.return_value = "5678abcd"
proj.bare_git.rev_list.return_value = ["0"]
proj.IsDirty = mock.MagicMock(return_value=False)
proj.GetBranches = mock.MagicMock(return_value=[])
proj.DeleteWorktree = mock.MagicMock()
proj._InitGitDir = mock.MagicMock()
proj._RemoteFetch = mock.MagicMock(return_value=True)
proj._InitRemote = mock.MagicMock()
proj._InitMRef = mock.MagicMock()
return proj
def test_sync_network_half_stateless_prune_needed(self):
"""Test stateless sync queues prune when needed."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
res = proj.Sync_NetworkHalf()
self.assertTrue(res.success)
proj.DeleteWorktree.assert_not_called()
self.assertTrue(proj.stateless_prune_needed)
proj._RemoteFetch.assert_called_once()
def test_sync_local_half_stateless_prune(self):
"""Test stateless GC pruning is queued in Sync_LocalHalf."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.stateless_prune_needed = True
proj._Checkout = mock.MagicMock()
proj._InitWorkTree = mock.MagicMock()
proj.IsRebaseInProgress = mock.MagicMock(return_value=False)
proj.IsCherryPickInProgress = mock.MagicMock(return_value=False)
proj.bare_ref = mock.MagicMock()
proj.bare_ref.all = {}
proj.GetRevisionId = mock.MagicMock(return_value="1234abcd")
proj._CopyAndLinkFiles = mock.MagicMock()
proj.work_git = mock.MagicMock()
proj.work_git.GetHead.return_value = "5678abcd"
syncbuf = project.SyncBuffer(proj.config)
with mock.patch("project.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
proj.Sync_LocalHalf(syncbuf)
syncbuf.Finish()
self.assertEqual(mock_git_cmd.call_count, 2)
mock_git_cmd.assert_any_call(
proj, ["reflog", "expire", "--expire=all", "--all"], bare=True
)
mock_git_cmd.assert_any_call(
proj,
["gc", "--prune=now"],
bare=True,
capture_stdout=True,
capture_stderr=True,
)
def test_sync_local_half_no_upstream_propagates_force_checkout(self):
"""Test Sync_LocalHalf forwards force_checkout when detaching."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj._InitWorkTree = mock.MagicMock()
proj.CleanPublishedCache = mock.MagicMock()
proj.GetRevisionId = mock.MagicMock(return_value="1234abcd")
proj._Checkout = mock.MagicMock()
proj._CopyAndLinkFiles = mock.MagicMock()
proj.work_git = mock.MagicMock()
proj.work_git.GetHead.return_value = "refs/heads/topic"
proj.bare_ref = mock.MagicMock()
proj.bare_ref.all = {"refs/heads/topic": "5678abcd"}
branch = mock.MagicMock()
branch.name = "topic"
branch.LocalMerge = False
proj.GetBranch = mock.MagicMock(return_value=branch)
syncbuf = project.SyncBuffer(proj.config)
proj.Sync_LocalHalf(syncbuf, force_checkout=True)
proj._Checkout.assert_called_once_with(
"1234abcd", force_checkout=True, quiet=True
)
proj._CopyAndLinkFiles.assert_called_once_with()
def test_sync_network_half_stateless_skips_if_stash(self):
"""Test stateless sync skips if stash exists."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.HasStash = mock.MagicMock(return_value=True)
res = proj.Sync_NetworkHalf()
self.assertTrue(res.success)
self.assertFalse(getattr(proj, "stateless_prune_needed", False))
def test_sync_network_half_stateless_skips_if_local_commits(self):
"""Test stateless sync skips if there are local-only commits."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir)
proj.bare_git.rev_list.return_value = ["1"]
res = proj.Sync_NetworkHalf()
self.assertTrue(res.success)
self.assertFalse(getattr(proj, "stateless_prune_needed", False))
class SyncOptimizationTests(unittest.TestCase):
"""Tests for sync optimization logic involving shallow clones."""
def _get_project(self, tempdir, depth=None):
manifest = mock.MagicMock()
manifest.manifestProject.depth = depth
manifest.manifestProject.dissociate = False
manifest.manifestProject.clone_filter = None
manifest.is_multimanifest = False
manifest.manifestProject.config.GetBoolean.return_value = False
manifest.IsMirror = False
remote = mock.MagicMock()
remote.name = "origin"
remote.url = "http://"
proj = project.Project(
manifest=manifest,
name="test-project",
remote=remote,
gitdir=os.path.join(tempdir, "gitdir"),
objdir=os.path.join(tempdir, "objdir"),
worktree=tempdir,
relpath="test-project",
revisionExpr="0123456789abcdef0123456789abcdef01234567",
revisionId=None,
)
proj._CheckForImmutableRevision = mock.MagicMock(return_value=True)
proj.DeleteWorktree = mock.MagicMock()
proj._InitGitDir = mock.MagicMock()
proj._InitRemote = mock.MagicMock()
proj._InitMRef = mock.MagicMock()
return proj
def test_sync_network_half_shallow_missing_fetches(self):
"""Test Sync_NetworkHalf fetches if shallow file is missing."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
# Ensure gitdir does not exist to simulate new project
if os.path.exists(proj.gitdir):
shutil.rmtree(proj.gitdir)
shallow_path = os.path.join(proj.gitdir, "shallow")
if os.path.exists(shallow_path):
os.unlink(shallow_path)
proj._RemoteFetch = mock.MagicMock(return_value=True)
res = proj.Sync_NetworkHalf(optimized_fetch=True)
self.assertTrue(res.success)
proj._RemoteFetch.assert_called_once()
def test_sync_network_half_shallow_exists_skips(self):
"""Test Sync_NetworkHalf skips fetch if shallow file exists."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
with open(os.path.join(proj.gitdir, "shallow"), "w") as f:
f.write("")
proj._RemoteFetch = mock.MagicMock()
res = proj.Sync_NetworkHalf(optimized_fetch=True)
self.assertTrue(res.success)
proj._RemoteFetch.assert_not_called()
def test_remote_fetch_shallow_missing_fetches(self):
"""Test _RemoteFetch fetches if shallow file is missing."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
shallow_path = os.path.join(proj.gitdir, "shallow")
if os.path.exists(shallow_path):
os.unlink(shallow_path)
with mock.patch("project.GitCommand") as mock_git_cmd:
mock_cmd_instance = mock.MagicMock()
mock_cmd_instance.Wait.return_value = 0
mock_git_cmd.return_value = mock_cmd_instance
res = proj._RemoteFetch(
current_branch_only=True,
depth=1,
use_superproject=False,
)
self.assertTrue(res)
mock_git_cmd.assert_called()
def test_remote_fetch_shallow_exists_skips(self):
"""Test _RemoteFetch skips fetch if shallow file exists."""
with utils_for_test.TempGitTree() as tempdir:
proj = self._get_project(tempdir, depth=1)
os.makedirs(proj.gitdir, exist_ok=True)
os.makedirs(proj.objdir, exist_ok=True)
with open(os.path.join(proj.gitdir, "shallow"), "w") as f:
f.write("")
with mock.patch("project.GitCommand") as mock_git_cmd:
res = proj._RemoteFetch(
current_branch_only=True,
depth=1,
use_superproject=False,
)
self.assertTrue(res)
mock_git_cmd.assert_not_called()
+77 -71
View File
@@ -12,90 +12,96 @@
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unit test for repo_logging module."""
"""Unittests for the repo_logging.py module."""
import contextlib
import io
import logging
import unittest
import re
from unittest import mock
import pytest
from color import SetDefaultColoring
from error import RepoExitError
from repo_logging import RepoLogger
class TestRepoLogger(unittest.TestCase):
@mock.patch.object(RepoLogger, "error")
def test_log_aggregated_errors_logs_aggregated_errors(self, mock_error):
"""Test if log_aggregated_errors logs a list of aggregated errors."""
logger = RepoLogger(__name__)
logger.log_aggregated_errors(
RepoExitError(
aggregate_errors=[
Exception("foo"),
Exception("bar"),
Exception("baz"),
Exception("hello"),
Exception("world"),
Exception("test"),
]
)
)
mock_error.assert_has_calls(
[
mock.call("=" * 80),
mock.call(
"Repo command failed due to the following `%s` errors:",
"RepoExitError",
),
mock.call("foo\nbar\nbaz\nhello\nworld"),
mock.call("+%d additional errors...", 1),
@mock.patch.object(RepoLogger, "error")
def test_log_aggregated_errors_logs_aggregated_errors(mock_error) -> None:
"""Test if log_aggregated_errors logs a list of aggregated errors."""
logger = RepoLogger(__name__)
logger.log_aggregated_errors(
RepoExitError(
aggregate_errors=[
Exception("foo"),
Exception("bar"),
Exception("baz"),
Exception("hello"),
Exception("world"),
Exception("test"),
]
)
)
@mock.patch.object(RepoLogger, "error")
def test_log_aggregated_errors_logs_single_error(self, mock_error):
"""Test if log_aggregated_errors logs empty aggregated_errors."""
mock_error.assert_has_calls(
[
mock.call("=" * 80),
mock.call(
"Repo command failed due to the following `%s` errors:",
"RepoExitError",
),
mock.call("foo\nbar\nbaz\nhello\nworld"),
mock.call("+%d additional errors...", 1),
]
)
@mock.patch.object(RepoLogger, "error")
def test_log_aggregated_errors_logs_single_error(mock_error) -> None:
"""Test if log_aggregated_errors logs empty aggregated_errors."""
logger = RepoLogger(__name__)
logger.log_aggregated_errors(RepoExitError())
mock_error.assert_has_calls(
[
mock.call("=" * 80),
mock.call("Repo command failed: %s", "RepoExitError"),
]
)
@pytest.mark.parametrize(
"level",
(
logging.INFO,
logging.WARN,
logging.ERROR,
),
)
def test_log_with_format_string(level: int) -> None:
"""Test different log levels with format strings."""
name = logging.getLevelName(level)
# Set color output to "always" for consistent test results.
# This ensures the logger's behavior is uniform across different
# environments and git configurations.
SetDefaultColoring("always")
# Regex pattern to match optional ANSI color codes.
# \033 - Escape character
# \[ - Opening square bracket
# [0-9;]* - Zero or more digits or semicolons
# m - Ending 'm' character
# ? - Makes the entire group optional
opt_color = r"(\033\[[0-9;]*m)?"
output = io.StringIO()
with contextlib.redirect_stderr(output):
logger = RepoLogger(__name__)
logger.log_aggregated_errors(RepoExitError())
logger.log(level, "%s", "100% pass")
mock_error.assert_has_calls(
[
mock.call("=" * 80),
mock.call("Repo command failed: %s", "RepoExitError"),
]
)
def test_log_with_format_string(self):
"""Test different log levels with format strings."""
# Set color output to "always" for consistent test results.
# This ensures the logger's behavior is uniform across different
# environments and git configurations.
SetDefaultColoring("always")
# Regex pattern to match optional ANSI color codes.
# \033 - Escape character
# \[ - Opening square bracket
# [0-9;]* - Zero or more digits or semicolons
# m - Ending 'm' character
# ? - Makes the entire group optional
opt_color = r"(\033\[[0-9;]*m)?"
for level in (logging.INFO, logging.WARN, logging.ERROR):
name = logging.getLevelName(level)
with self.subTest(level=level, name=name):
output = io.StringIO()
with contextlib.redirect_stderr(output):
logger = RepoLogger(__name__)
logger.log(level, "%s", "100% pass")
self.assertRegex(
output.getvalue().strip(),
f"^{opt_color}100% pass{opt_color}$",
f"failed for level {name}",
)
assert re.search(
f"^{opt_color}100% pass{opt_color}$", output.getvalue().strip()
), f"failed for level {name}"
+25 -34
View File
@@ -1,4 +1,4 @@
# Copyright 2022 The Android Open Source Project
# Copyright (C) 2022 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -15,46 +15,37 @@
"""Unittests for the repo_trace.py module."""
import os
import unittest
from unittest import mock
import pytest
import repo_trace
class TraceTests(unittest.TestCase):
def test_trace_max_size_enforced(monkeypatch: pytest.MonkeyPatch) -> None:
"""Check Trace behavior."""
content = "git chicken"
def testTrace_MaxSizeEnforced(self):
content = "git chicken"
with repo_trace.Trace(content, first_trace=True):
pass
first_trace_size = os.path.getsize(repo_trace._TRACE_FILE)
with repo_trace.Trace(content, first_trace=True):
pass
first_trace_size = os.path.getsize(repo_trace._TRACE_FILE)
with repo_trace.Trace(content):
pass
assert os.path.getsize(repo_trace._TRACE_FILE) > first_trace_size
with repo_trace.Trace(content):
pass
self.assertGreater(
os.path.getsize(repo_trace._TRACE_FILE), first_trace_size
)
# Check we clear everything if the last chunk is larger than _MAX_SIZE.
monkeypatch.setattr(repo_trace, "_MAX_SIZE", 0)
with repo_trace.Trace(content, first_trace=True):
pass
assert os.path.getsize(repo_trace._TRACE_FILE) == first_trace_size
# Check we clear everything is the last chunk is larger than _MAX_SIZE.
with mock.patch("repo_trace._MAX_SIZE", 0):
with repo_trace.Trace(content, first_trace=True):
pass
self.assertEqual(
first_trace_size, os.path.getsize(repo_trace._TRACE_FILE)
)
# Check we only clear the chunks we need to.
new_max = (first_trace_size + 1) / (1024 * 1024)
monkeypatch.setattr(repo_trace, "_MAX_SIZE", new_max)
with repo_trace.Trace(content, first_trace=True):
pass
assert os.path.getsize(repo_trace._TRACE_FILE) == first_trace_size * 2
# Check we only clear the chunks we need to.
repo_trace._MAX_SIZE = (first_trace_size + 1) / (1024 * 1024)
with repo_trace.Trace(content, first_trace=True):
pass
self.assertEqual(
first_trace_size * 2, os.path.getsize(repo_trace._TRACE_FILE)
)
with repo_trace.Trace(content, first_trace=True):
pass
self.assertEqual(
first_trace_size * 2, os.path.getsize(repo_trace._TRACE_FILE)
)
with repo_trace.Trace(content, first_trace=True):
pass
assert os.path.getsize(repo_trace._TRACE_FILE) == first_trace_size * 2
+65 -46
View File
@@ -1,4 +1,4 @@
# Copyright 2019 The Android Open Source Project
# Copyright (C) 2019 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -16,65 +16,84 @@
import multiprocessing
import subprocess
import unittest
from typing import Tuple
from unittest import mock
import pytest
import ssh
class SshTests(unittest.TestCase):
"""Tests the ssh functions."""
@pytest.fixture(autouse=True)
def clear_ssh_version_cache() -> None:
"""Clear the ssh version cache before each test."""
ssh.version.cache_clear()
def test_parse_ssh_version(self):
"""Check _parse_ssh_version() handling."""
ver = ssh._parse_ssh_version("Unknown\n")
self.assertEqual(ver, ())
ver = ssh._parse_ssh_version("OpenSSH_1.0\n")
self.assertEqual(ver, (1, 0))
ver = ssh._parse_ssh_version(
"OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13, OpenSSL 1.0.1f 6 Jan 2014\n"
)
self.assertEqual(ver, (6, 6, 1))
ver = ssh._parse_ssh_version(
"OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\n"
)
self.assertEqual(ver, (7, 6))
ver = ssh._parse_ssh_version("OpenSSH_9.0p1, LibreSSL 3.3.6\n")
self.assertEqual(ver, (9, 0))
def test_version(self):
"""Check version() handling."""
with mock.patch("ssh._run_ssh_version", return_value="OpenSSH_1.2\n"):
self.assertEqual(ssh.version(), (1, 2))
@pytest.mark.parametrize(
"input_str, expected",
(
("Unknown\n", ()),
("OpenSSH_1.0\n", (1, 0)),
(
"OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13, OpenSSL 1.0.1f 6 Jan 2014\n",
(6, 6, 1),
),
(
"OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\n",
(7, 6),
),
("OpenSSH_9.0p1, LibreSSL 3.3.6\n", (9, 0)),
),
)
def test_parse_ssh_version(input_str: str, expected: Tuple[int, ...]) -> None:
"""Check _parse_ssh_version() handling."""
assert ssh._parse_ssh_version(input_str) == expected
def test_context_manager_empty(self):
"""Verify context manager with no clients works correctly."""
with multiprocessing.Manager() as manager:
with ssh.ProxyManager(manager):
pass
def test_context_manager_child_cleanup(self):
"""Verify orphaned clients & masters get cleaned up."""
with multiprocessing.Manager() as manager:
def test_version() -> None:
"""Check version() handling."""
with mock.patch("ssh._run_ssh_version", return_value="OpenSSH_1.2\n"):
assert ssh.version() == (1, 2)
def test_context_manager_empty() -> None:
"""Verify context manager with no clients works correctly."""
with multiprocessing.Manager() as manager:
with ssh.ProxyManager(manager):
pass
def test_context_manager_child_cleanup() -> None:
"""Verify orphaned clients & masters get cleaned up."""
with multiprocessing.Manager() as manager:
with mock.patch("ssh.version", return_value=(1, 2)):
with ssh.ProxyManager(manager) as ssh_proxy:
client = subprocess.Popen(["sleep", "964853320"])
ssh_proxy.add_client(client)
master = subprocess.Popen(["sleep", "964853321"])
ssh_proxy.add_master(master)
# If the process still exists, these will throw timeout errors.
client.wait(0)
master.wait(0)
# If the process still exists, these will throw timeout errors.
client.wait(0)
master.wait(0)
def test_ssh_sock(self):
"""Check sock() function."""
manager = multiprocessing.Manager()
def test_ssh_sock(monkeypatch: pytest.MonkeyPatch) -> None:
"""Check sock() function."""
with multiprocessing.Manager() as manager:
proxy = ssh.ProxyManager(manager)
with mock.patch("tempfile.mkdtemp", return_value="/tmp/foo"):
# Old ssh version uses port.
with mock.patch("ssh.version", return_value=(6, 6)):
self.assertTrue(proxy.sock().endswith("%p"))
monkeypatch.setattr(
"tempfile.mkdtemp", lambda *args, **kwargs: "/tmp/foo"
)
proxy._sock_path = None
# New ssh version uses hash.
with mock.patch("ssh.version", return_value=(6, 7)):
self.assertTrue(proxy.sock().endswith("%C"))
# Old ssh version uses port.
with mock.patch("ssh.version", return_value=(6, 6)):
with proxy as ssh_proxy:
assert ssh_proxy.sock().endswith("%p")
proxy._sock_path = None
# New ssh version uses hash.
with mock.patch("ssh.version", return_value=(6, 7)):
with proxy as ssh_proxy:
assert ssh_proxy.sock().endswith("%C")
proxy._sock_path = None
+128 -82
View File
@@ -15,118 +15,164 @@
"""Unittests for the subcmds module (mostly __init__.py than subcommands)."""
import optparse
import unittest
from typing import Type
import pytest
from command import Command
import subcmds
class AllCommands(unittest.TestCase):
"""Check registered all_commands."""
# NB: We don't test all subcommands as we want to avoid "change detection"
# tests, so we just look for the most common/important ones here that are
# unlikely to ever change.
@pytest.mark.parametrize(
"cmd", ("cherry-pick", "help", "init", "start", "sync", "upload")
)
def test_required_basic(cmd: str) -> None:
"""Basic checking of registered commands."""
assert cmd in subcmds.all_commands
def test_required_basic(self):
"""Basic checking of registered commands."""
# NB: We don't test all subcommands as we want to avoid "change
# detection" tests, so we just look for the most common/important ones
# here that are unlikely to ever change.
for cmd in {"cherry-pick", "help", "init", "start", "sync", "upload"}:
self.assertIn(cmd, subcmds.all_commands)
def test_naming(self):
"""Verify we don't add things that we shouldn't."""
for cmd in subcmds.all_commands:
# Reject filename suffixes like "help.py".
self.assertNotIn(".", cmd)
@pytest.mark.parametrize("name", subcmds.all_commands.keys())
def test_naming(name: str) -> None:
"""Verify we don't add things that we shouldn't."""
# Reject filename suffixes like "help.py".
assert "." not in name
# Make sure all '_' were converted to '-'.
self.assertNotIn("_", cmd)
# Make sure all '_' were converted to '-'.
assert "_" not in name
# Reject internal python paths like "__init__".
self.assertFalse(cmd.startswith("__"))
# Reject internal python paths like "__init__".
assert not name.startswith("__")
def test_help_desc_style(self):
"""Force some consistency in option descriptions.
Python's optparse & argparse has a few default options like --help.
Their option description text uses lowercase sentence fragments, so
enforce our options follow the same style so UI is consistent.
@pytest.mark.parametrize("name, cls", subcmds.all_commands.items())
def test_help_desc_style(name: str, cls: Type[Command]) -> None:
"""Force some consistency in option descriptions.
We enforce:
* Text starts with lowercase.
* Text doesn't end with period.
"""
for name, cls in subcmds.all_commands.items():
cmd = cls()
parser = cmd.OptionParser
for option in parser.option_list:
if option.help == optparse.SUPPRESS_HELP:
continue
Python's optparse & argparse has a few default options like --help.
Their option description text uses lowercase sentence fragments, so
enforce our options follow the same style so UI is consistent.
c = option.help[0]
self.assertEqual(
c.lower(),
c,
msg=f"subcmds/{name}.py: {option.get_opt_string()}: "
f'help text should start with lowercase: "{option.help}"',
)
We enforce:
* Text starts with lowercase.
* Text doesn't end with period.
"""
cmd = cls()
parser = cmd.OptionParser
for option in parser.option_list:
if option.help == optparse.SUPPRESS_HELP or not option.help:
continue
self.assertNotEqual(
option.help[-1],
".",
msg=f"subcmds/{name}.py: {option.get_opt_string()}: "
f'help text should not end in a period: "{option.help}"',
)
c = option.help[0]
assert c.lower() == c, (
f"subcmds/{name}.py: {option.get_opt_string()}: "
f'help text should start with lowercase: "{option.help}"'
)
def test_cli_option_style(self):
"""Force some consistency in option flags."""
for name, cls in subcmds.all_commands.items():
cmd = cls()
parser = cmd.OptionParser
for option in parser.option_list:
for opt in option._long_opts:
self.assertNotIn(
"_",
opt,
msg=f"subcmds/{name}.py: {opt}: only use dashes in "
"options, not underscores",
)
assert option.help[-1] != ".", (
f"subcmds/{name}.py: {option.get_opt_string()}: "
f'help text should not end in a period: "{option.help}"'
)
def test_cli_option_dest(self):
"""Block redundant dest= arguments."""
def _check_dest(opt):
if opt.dest is None or not opt._long_opts:
return
@pytest.mark.parametrize("name, cls", subcmds.all_commands.items())
def test_cli_option_style(name: str, cls: Type[Command]) -> None:
"""Force some consistency in option flags."""
cmd = cls()
parser = cmd.OptionParser
for option in parser.option_list:
for opt in option._long_opts:
assert "_" not in opt, (
f"subcmds/{name}.py: {opt}: only use dashes in "
"options, not underscores"
)
long = opt._long_opts[0]
assert long.startswith("--")
# This matches optparse's behavior.
implicit_dest = long[2:].replace("-", "_")
if implicit_dest == opt.dest:
bad_opts.append((str(opt), opt.dest))
# Hook the option check list.
optparse.Option.CHECK_METHODS.insert(0, _check_dest)
def test_cli_option_dest() -> None:
"""Block redundant dest= arguments."""
bad_opts: list[tuple[str, str]] = []
def _check_dest(opt: optparse.Option) -> None:
"""Check the dest= setting."""
# If the destination is not set, nothing to check.
# If long options are not set, then there's no implicit destination.
# If callback is used, then a destination might be needed because
# optparse cannot assume a value is always stored.
if opt.dest is None or not opt._long_opts or opt.callback:
return
long = opt._long_opts[0]
assert long.startswith("--")
# This matches optparse's behavior.
implicit_dest = long[2:].replace("-", "_")
if implicit_dest == opt.dest:
bad_opts.append((str(opt), opt.dest))
# Hook the option check list.
optparse.Option.CHECK_METHODS.insert(0, _check_dest)
try:
# Gather all the bad options up front so people can see all bad options
# instead of failing at the first one.
all_bad_opts = {}
all_bad_opts: dict[str, list[tuple[str, str]]] = {}
for name, cls in subcmds.all_commands.items():
bad_opts = all_bad_opts[name] = []
bad_opts = []
cmd = cls()
# Trigger construction of parser.
cmd.OptionParser
_ = cmd.OptionParser
all_bad_opts[name] = bad_opts
errmsg = None
for name, bad_opts in sorted(all_bad_opts.items()):
if bad_opts:
errmsg = ""
for name, bad_opts_list in sorted(all_bad_opts.items()):
if bad_opts_list:
if not errmsg:
errmsg = "Omit redundant dest= when defining options.\n"
errmsg += f"\nSubcommand {name} (subcmds/{name}.py):\n"
errmsg += "".join(
f" {opt}: dest='{dest}'\n" for opt, dest in bad_opts
f" {opt}: dest='{dest}'\n" for opt, dest in bad_opts_list
)
if errmsg:
self.fail(errmsg)
pytest.fail(errmsg)
finally:
# Make sure we aren't popping the wrong stuff.
assert optparse.Option.CHECK_METHODS.pop(0) is _check_dest
@pytest.mark.parametrize("name, cls", subcmds.all_commands.items())
def test_common_validate_options(name: str, cls: Type[Command]) -> None:
"""Verify CommonValidateOptions sets up expected fields."""
cmd = cls()
opts, args = cmd.OptionParser.parse_args([])
# Verify the fields don't exist yet.
assert not hasattr(
opts, "verbose"
), f"{name}: has verbose before validation"
assert not hasattr(opts, "quiet"), f"{name}: has quiet before validation"
cmd.CommonValidateOptions(opts, args)
# Verify the fields exist now.
assert hasattr(opts, "verbose"), f"{name}: missing verbose after validation"
assert hasattr(opts, "quiet"), f"{name}: missing quiet after validation"
assert hasattr(
opts, "outer_manifest"
), f"{name}: missing outer_manifest after validation"
def test_attribute_error_repro() -> None:
"""Confirm that accessing verbose before CommonValidateOptions fails."""
from subcmds.sync import Sync
cmd = Sync()
opts, args = cmd.OptionParser.parse_args([])
# This confirms that without the fix in main.py, an AttributeError
# would be raised because CommonValidateOptions hasn't been called yet.
with pytest.raises(AttributeError):
_ = opts.verbose
cmd.CommonValidateOptions(opts, args)
assert hasattr(opts, "verbose")
+67 -117
View File
@@ -14,143 +14,93 @@
"""Unittests for the forall subcmd."""
from io import StringIO
import os
from shutil import rmtree
import subprocess
import tempfile
import unittest
import contextlib
import io
from pathlib import Path
from unittest import mock
import git_command
import utils_for_test
import manifest_xml
import project
import subcmds
class AllCommands(unittest.TestCase):
"""Check registered all_commands."""
def _create_manifest_with_8_projects(
topdir: Path,
) -> manifest_xml.XmlManifest:
"""Create a setup of 8 projects to execute forall."""
repodir = topdir / ".repo"
manifest_dir = repodir / "manifests"
manifest_file = repodir / manifest_xml.MANIFEST_FILE_NAME
def setUp(self):
"""Common setup."""
self.tempdirobj = tempfile.TemporaryDirectory(prefix="forall_tests")
self.tempdir = self.tempdirobj.name
self.repodir = os.path.join(self.tempdir, ".repo")
self.manifest_dir = os.path.join(self.repodir, "manifests")
self.manifest_file = os.path.join(
self.repodir, manifest_xml.MANIFEST_FILE_NAME
)
self.local_manifest_dir = os.path.join(
self.repodir, manifest_xml.LOCAL_MANIFESTS_DIR_NAME
)
os.mkdir(self.repodir)
os.mkdir(self.manifest_dir)
repodir.mkdir()
manifest_dir.mkdir()
def tearDown(self):
"""Common teardown."""
rmtree(self.tempdir, ignore_errors=True)
# Set up a manifest git dir for parsing to work.
gitdir = repodir / "manifests.git"
gitdir.mkdir()
(gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
verbose = false
"""
)
def initTempGitTree(self, git_dir):
"""Create a new empty git checkout for testing."""
# Add the manifest data.
manifest_file.write_text(
"""
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="project1" path="tests/path1" />
<project name="project2" path="tests/path2" />
<project name="project3" path="tests/path3" />
<project name="project4" path="tests/path4" />
<project name="project5" path="tests/path5" />
<project name="project6" path="tests/path6" />
<project name="project7" path="tests/path7" />
<project name="project8" path="tests/path8" />
</manifest>
""",
encoding="utf-8",
)
# Tests need to assume, that main is default branch at init,
# which is not supported in config until 2.28.
cmd = ["git", "init", "-q"]
if git_command.git_require((2, 28, 0)):
cmd += ["--initial-branch=main"]
else:
# Use template dir for init
templatedir = os.path.join(self.tempdirobj.name, ".test-template")
os.makedirs(templatedir)
with open(os.path.join(templatedir, "HEAD"), "w") as fp:
fp.write("ref: refs/heads/main\n")
cmd += ["--template", templatedir]
cmd += [git_dir]
subprocess.check_call(cmd)
# Set up 8 empty projects to match the manifest.
for x in range(1, 9):
(repodir / "projects" / "tests" / f"path{x}.git").mkdir(parents=True)
(repodir / "project-objects" / f"project{x}.git").mkdir(parents=True)
git_path = topdir / "tests" / f"path{x}"
utils_for_test.init_git_tree(git_path)
def getXmlManifestWith8Projects(self):
"""Create and return a setup of 8 projects with enough dummy
files and setup to execute forall."""
return manifest_xml.XmlManifest(str(repodir), str(manifest_file))
# Set up a manifest git dir for parsing to work
gitdir = os.path.join(self.repodir, "manifests.git")
os.mkdir(gitdir)
with open(os.path.join(gitdir, "config"), "w") as fp:
fp.write(
"""[remote "origin"]
url = https://localhost:0/manifest
verbose = false
"""
)
# Add the manifest data
manifest_data = """
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="project1" path="tests/path1" />
<project name="project2" path="tests/path2" />
<project name="project3" path="tests/path3" />
<project name="project4" path="tests/path4" />
<project name="project5" path="tests/path5" />
<project name="project6" path="tests/path6" />
<project name="project7" path="tests/path7" />
<project name="project8" path="tests/path8" />
</manifest>
"""
with open(self.manifest_file, "w", encoding="utf-8") as fp:
fp.write(manifest_data)
def test_forall_all_projects_called_once(tmp_path: Path) -> None:
"""Test that all projects get a command run once each."""
manifest = _create_manifest_with_8_projects(tmp_path)
# Set up 8 empty projects to match the manifest
for x in range(1, 9):
os.makedirs(
os.path.join(
self.repodir, "projects/tests/path" + str(x) + ".git"
)
)
os.makedirs(
os.path.join(
self.repodir, "project-objects/project" + str(x) + ".git"
)
)
git_path = os.path.join(self.tempdir, "tests/path" + str(x))
self.initTempGitTree(git_path)
cmd = subcmds.forall.Forall()
cmd.manifest = manifest
return manifest_xml.XmlManifest(self.repodir, self.manifest_file)
# Use echo project names as the test of forall.
opts, args = cmd.OptionParser.parse_args(["-c", "echo $REPO_PROJECT"])
opts.verbose = False
# Use mock to capture stdout from the forall run
@unittest.mock.patch("sys.stdout", new_callable=StringIO)
def test_forall_all_projects_called_once(self, mock_stdout):
"""Test that all projects get a command run once each."""
manifest_with_8_projects = self.getXmlManifestWith8Projects()
cmd = subcmds.forall.Forall()
cmd.manifest = manifest_with_8_projects
# Use echo project names as the test of forall
opts, args = cmd.OptionParser.parse_args(["-c", "echo $REPO_PROJECT"])
opts.verbose = False
# Mock to not have the Execute fail on remote check
with contextlib.redirect_stdout(io.StringIO()) as stdout:
# Mock to not have the Execute fail on remote check.
with mock.patch.object(
project.Project, "GetRevisionId", return_value="refs/heads/main"
):
# Run the forall command
# Run the forall command.
cmd.Execute(opts, args)
# Verify that we got every project name in the prints
for x in range(1, 9):
self.assertIn("project" + str(x), mock_stdout.getvalue())
output = stdout.getvalue()
# Verify that we got every project name in the output.
for x in range(1, 9):
assert f"project{x}" in output
# Split the captured output into lines to count them
line_count = 0
for line in mock_stdout.getvalue().split("\n"):
# A commented out print to stderr as a reminder
# that stdout is mocked, include sys and uncomment if needed
# print(line, file=sys.stderr)
if len(line) > 0:
line_count += 1
# Verify that we didn't get more lines than expected
assert line_count == 8
# Split the captured output into lines to count them.
line_count = sum(1 for x in output.splitlines() if x)
# Verify that we didn't get more lines than expected.
assert line_count == 8
+82
View File
@@ -0,0 +1,82 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the subcmds/gc.py module."""
import unittest
from unittest import mock
from subcmds import gc
class GcCommand(unittest.TestCase):
"""Tests for gc command."""
def setUp(self):
self.cmd = gc.Gc()
self.opt, self.args = self.cmd.OptionParser.parse_args([])
self.opt.this_manifest_only = False
self.opt.repack = False
self.mock_get_projects = mock.patch.object(
self.cmd, "GetProjects"
).start()
self.mock_delete = mock.patch.object(
self.cmd, "delete_unused_projects", return_value=0
).start()
self.mock_repack = mock.patch.object(
self.cmd, "repack_projects", return_value=0
).start()
def tearDown(self):
mock.patch.stopall()
def test_gc_no_args(self):
"""Test gc without specific projects."""
self.mock_get_projects.return_value = ["all_projects"]
self.cmd.Execute(self.opt, [])
self.mock_get_projects.assert_called_once_with([], all_manifests=True)
self.mock_delete.assert_called_once_with(["all_projects"], self.opt)
self.mock_repack.assert_not_called()
def test_gc_with_args(self):
"""Test gc with specific projects uses all_projects for delete."""
self.mock_get_projects.side_effect = [["projA"], ["all_projects"]]
self.opt.repack = True
self.cmd.Execute(self.opt, ["projA"])
self.mock_get_projects.assert_has_calls(
[
mock.call(["projA"], all_manifests=True),
mock.call([], all_manifests=True),
]
)
self.mock_delete.assert_called_once_with(["all_projects"], self.opt)
self.mock_repack.assert_called_once_with(["projA"], self.opt)
def test_gc_exit_on_delete_failure(self):
"""Test gc exits if delete_unused_projects fails."""
self.mock_get_projects.return_value = ["all_projects"]
self.mock_delete.return_value = 1
self.opt.repack = True
ret = self.cmd.Execute(self.opt, [])
self.assertEqual(ret, 1)
self.mock_repack.assert_not_called()
+201
View File
@@ -0,0 +1,201 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the subcmds/info.py module."""
import json
from unittest import mock
import pytest
from subcmds import info
def _get_cmd() -> info.Info:
"""Build a mock-backed Info command for testing."""
manifest = mock.MagicMock()
manifest.default.revisionExpr = "refs/heads/main"
manifest.manifestProject.config.GetBranch.return_value.merge = (
"refs/heads/main"
)
manifest.GetManifestGroupsStr.return_value = "all"
manifest.superproject = None
manifest.outer_client = manifest
client = mock.MagicMock()
git_event_log = mock.MagicMock()
return info.Info(
manifest=manifest,
client=client,
git_event_log=git_event_log,
)
def test_include_options_default_true() -> None:
"""Both include options should default to True."""
opts, _ = _get_cmd().OptionParser.parse_args([])
assert opts.include_summary
assert opts.include_projects
def test_no_include_summary_parses() -> None:
"""--no-include-summary should set include_summary to False."""
opts, _ = _get_cmd().OptionParser.parse_args(["--no-include-summary"])
assert not opts.include_summary
def test_no_include_projects_parses() -> None:
"""--no-include-projects should set include_projects to False."""
opts, _ = _get_cmd().OptionParser.parse_args(["--no-include-projects"])
assert not opts.include_projects
def test_format_default_text() -> None:
"""Default format should be text."""
opts, _ = _get_cmd().OptionParser.parse_args([])
assert opts.format == "text"
def test_format_json_parses() -> None:
"""--format=json should be accepted."""
opts, _ = _get_cmd().OptionParser.parse_args(["--format=json"])
assert opts.format == "json"
def test_no_include_projects_skips_projects() -> None:
"""--no-include-projects should skip project iteration."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--no-include-projects"])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_not_called()
def test_no_include_summary_skips_summary() -> None:
"""--no-include-summary should not query or print manifest metadata."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--no-include-summary"])
with mock.patch.object(cmd, "_printDiffInfo"):
cmd.Execute(opts, args)
cmd.manifest.GetManifestGroupsStr.assert_not_called()
def test_default_calls_diff_info() -> None:
"""Default options should call _printDiffInfo."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args([])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_called_once_with(opts, args)
mock_overview.assert_not_called()
def test_overview_calls_commit_overview() -> None:
"""--overview should call _printCommitOverview, not _printDiffInfo."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--overview"])
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_called_once_with(opts, args)
def test_no_include_projects_with_overview() -> None:
"""--no-include-projects should take priority over --overview."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--no-include-projects", "--overview"]
)
with mock.patch.object(
cmd, "_printDiffInfo"
) as mock_diff, mock.patch.object(
cmd, "_printCommitOverview"
) as mock_overview:
cmd.Execute(opts, args)
mock_diff.assert_not_called()
mock_overview.assert_not_called()
def test_json_summary_only(capsys) -> None:
"""--format=json --no-include-projects should emit only summary."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--format=json", "--no-include-projects"]
)
cmd.Execute(opts, args)
data = json.loads(capsys.readouterr().out)
assert "summary" in data
assert "projects" not in data
assert data["summary"]["manifest_branch"] == "refs/heads/main"
assert data["summary"]["manifest_groups"] == "all"
def test_json_no_summary(capsys) -> None:
"""--format=json --no-include-summary should omit summary."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(
["--format=json", "--no-include-summary", "--no-include-projects"]
)
cmd.Execute(opts, args)
data = json.loads(capsys.readouterr().out)
assert "summary" not in data
def test_json_rejects_diff() -> None:
"""--format=json --diff should be rejected."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--format=json", "--diff"])
with pytest.raises(SystemExit):
cmd.ValidateOptions(opts, args)
def test_json_rejects_overview() -> None:
"""--format=json --overview should be rejected."""
cmd = _get_cmd()
opts, args = cmd.OptionParser.parse_args(["--format=json", "--overview"])
with pytest.raises(SystemExit):
cmd.ValidateOptions(opts, args)
def test_json_disables_pager() -> None:
"""--format=json should disable the pager."""
cmd = _get_cmd()
opts, _ = cmd.OptionParser.parse_args(["--format=json"])
assert not cmd.WantPager(opts)
def test_text_enables_pager() -> None:
"""Default text format should enable the pager."""
cmd = _get_cmd()
opts, _ = cmd.OptionParser.parse_args([])
assert cmd.WantPager(opts)
+27 -24
View File
@@ -14,33 +14,36 @@
"""Unittests for the subcmds/init.py module."""
import unittest
from typing import List
import pytest
from subcmds import init
class InitCommand(unittest.TestCase):
"""Check registered all_commands."""
@pytest.mark.parametrize(
"argv",
([],),
)
def test_cli_parser_good(argv: List[str]) -> None:
"""Check valid command line options."""
cmd = init.Init()
opts, args = cmd.OptionParser.parse_args(argv)
cmd.ValidateOptions(opts, args)
def setUp(self):
self.cmd = init.Init()
def test_cli_parser_good(self):
"""Check valid command line options."""
ARGV = ([],)
for argv in ARGV:
opts, args = self.cmd.OptionParser.parse_args(argv)
self.cmd.ValidateOptions(opts, args)
def test_cli_parser_bad(self):
"""Check invalid command line options."""
ARGV = (
# Too many arguments.
["url", "asdf"],
# Conflicting options.
["--mirror", "--archive"],
)
for argv in ARGV:
opts, args = self.cmd.OptionParser.parse_args(argv)
with self.assertRaises(SystemExit):
self.cmd.ValidateOptions(opts, args)
@pytest.mark.parametrize(
"argv",
(
# Too many arguments.
["url", "asdf"],
# Conflicting options.
["--mirror", "--archive"],
),
)
def test_cli_parser_bad(argv: List[str]) -> None:
"""Check invalid command line options."""
cmd = init.Init()
opts, args = cmd.OptionParser.parse_args(argv)
with pytest.raises(SystemExit):
cmd.ValidateOptions(opts, args)
+220
View File
@@ -0,0 +1,220 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the status subcmd."""
import contextlib
import io
import os
from pathlib import Path
import subprocess
from typing import List, Tuple
from unittest import mock
import pytest
import utils_for_test
import manifest_xml
import subcmds
@pytest.fixture
def repo_client_checkout(
tmp_path: Path,
) -> Tuple[Path, manifest_xml.XmlManifest]:
"""Create a basic repo client checkout for status tests."""
# Create in a subdir to avoid noise (like the repo_trace file).
topdir = tmp_path / "client_checkout"
repodir = topdir / ".repo"
manifest_dir = repodir / "manifests"
manifest_file = repodir / manifest_xml.MANIFEST_FILE_NAME
repodir.mkdir(parents=True)
manifest_dir.mkdir()
gitdir = repodir / "manifests.git"
gitdir.mkdir()
(gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
verbose = false
"""
)
_init_temp_git_tree(manifest_dir)
manifest_file.write_text(
"""
<manifest>
<remote name="origin" fetch="http://localhost" />
<default remote="origin" revision="refs/heads/main" />
<project name="proj" path="src/proj" />
</manifest>
""",
encoding="utf-8",
)
(repodir / "projects" / "src" / "proj.git").mkdir(parents=True)
(repodir / "project-objects" / "proj.git").mkdir(parents=True)
worktree = topdir / "src" / "proj"
worktree.parent.mkdir(parents=True, exist_ok=True)
_init_temp_git_tree(worktree)
manifest = manifest_xml.XmlManifest(str(repodir), str(manifest_file))
return topdir, manifest
def _init_temp_git_tree(git_dir: Path) -> None:
"""Create a new git checkout with an initial commit for testing."""
utils_for_test.init_git_tree(git_dir)
(git_dir / "README").write_text("init")
subprocess.check_call(["git", "add", "README"], cwd=git_dir)
subprocess.check_call(["git", "commit", "-q", "-m", "init"], cwd=git_dir)
def _run_status(manifest: manifest_xml.XmlManifest, argv: List[str]) -> None:
"""Run the status subcommand with parsed options against a test manifest."""
cmd = subcmds.status.Status()
cmd.manifest = manifest
cmd.client = mock.MagicMock(globalConfig=manifest.globalConfig)
opts, args = cmd.OptionParser.parse_args(argv + ["--jobs=1"])
cmd.CommonValidateOptions(opts, args)
cmd.Execute(opts, args)
def _status_lines(output: str) -> List[str]:
"""Normalize path separators and split command output into lines."""
return output.replace(os.sep, "/").splitlines()
def _assert_project_header(line: str, project_path: str, branch: str) -> None:
"""Assert a status project header line for a project and branch."""
expected = f"project {(project_path + '/ '):<40}branch {branch}"
assert line == expected
def _assert_orphan_block(lines: List[str], expected: List[str]) -> None:
"""Assert orphan block header and entries, independent of entry ordering."""
assert lines
assert lines[0] == ("Objects not within a project (orphans)")
orphan_lines = lines[1:]
assert len(orphan_lines) == len(expected)
assert sorted(orphan_lines) == sorted(expected)
def test_orphans_basic(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify -o output includes project header and orphan block."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / "src" / "orphan_dir").mkdir(parents=True)
(topdir / "orphan.txt").write_text("data")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, ["-o"])
lines = _status_lines(stdout.getvalue())
_assert_project_header(lines[0], project_path, "main")
_assert_orphan_block(
lines[1:],
[
" --\torphan.txt",
" --\tsrc/orphan_dir/",
],
)
def test_empty_status_without_orphans(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify clean status without -o prints only the project header line."""
_, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, "main")
def test_status_without_orphans(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify modified tracked file appears in status output without -o."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / project_path / "README").write_text("updated")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 2
_assert_project_header(lines[0], project_path, "main")
assert lines[1] == " -m\tREADME"
def test_status_with_orphans_and_modified_file(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify modified-file status plus orphan block."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
(topdir / project_path / "README").write_text("updated")
(topdir / "src" / "orphan_dir").mkdir(parents=True)
(topdir / "orphan.txt").write_text("data")
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, ["-o"])
lines = _status_lines(stdout.getvalue())
_assert_project_header(lines[0], project_path, "main")
assert lines[1] == " -m\tREADME"
_assert_orphan_block(
lines[2:],
[
" --\torphan.txt",
" --\tsrc/orphan_dir/",
],
)
def test_empty_status_after_start_shows_started_branch(
repo_client_checkout: Tuple[Path, manifest_xml.XmlManifest],
) -> None:
"""Verify status shows the started branch name when the tree is clean."""
topdir, manifest = repo_client_checkout
project_path = next(iter(manifest.paths.keys()))
project_worktree = topdir / project_path
started_branch = "topic/test-status-branch"
subprocess.check_call(
["git", "checkout", "-q", "-b", started_branch], cwd=project_worktree
)
with contextlib.redirect_stdout(io.StringIO()) as stdout:
_run_status(manifest, [])
lines = _status_lines(stdout.getvalue())
assert len(lines) == 1
_assert_project_header(lines[0], project_path, started_branch)
+525 -7
View File
@@ -13,6 +13,7 @@
# limitations under the License.
"""Unittests for the subcmds/sync.py module."""
import json
import os
import shutil
import tempfile
@@ -97,6 +98,35 @@ def test_cli_jobs(argv, jobs_manifest, jobs, jobs_net, jobs_check):
"""Tests --jobs option behavior."""
mp = mock.MagicMock()
mp.manifest.default.sync_j = jobs_manifest
mp.manifest.default.sync_j_max = None
cmd = sync.Sync()
opts, args = cmd.OptionParser.parse_args(argv)
cmd.ValidateOptions(opts, args)
with mock.patch.object(sync, "_rlimit_nofile", return_value=(256, 256)):
with mock.patch.object(os, "cpu_count", return_value=OS_CPU_COUNT):
cmd._ValidateOptionsWithManifest(opts, mp)
assert opts.jobs == jobs
assert opts.jobs_network == jobs_net
assert opts.jobs_checkout == jobs_check
@pytest.mark.parametrize(
"argv, jobs_manifest, jobs_manifest_max, jobs, jobs_net, jobs_check",
[
(["--jobs=10"], None, 5, 5, 5, 5),
(["--jobs=10", "--jobs-network=10"], None, 5, 5, 5, 5),
(["--jobs=10", "--jobs-checkout=10"], None, 5, 5, 5, 5),
],
)
def test_cli_jobs_sync_j_max(
argv, jobs_manifest, jobs_manifest_max, jobs, jobs_net, jobs_check
):
"""Tests --jobs option behavior with sync-j-max."""
mp = mock.MagicMock()
mp.manifest.default.sync_j = jobs_manifest
mp.manifest.default.sync_j_max = jobs_manifest_max
cmd = sync.Sync()
opts, args = cmd.OptionParser.parse_args(argv)
@@ -448,6 +478,181 @@ class GetPreciousObjectsState(unittest.TestCase):
)
class KeyboardInterruptTest(unittest.TestCase):
"""Tests for KeyboardInterrupt handling in Sync operations."""
def setUp(self):
self.project = mock.MagicMock(name="project")
self.project.name = "project"
self.project.relpath = "proj"
self.project.manifest.IsArchive = False
self.opt = mock.Mock()
self.opt.quiet = True
self.opt.verbose = False
self.opt.tags = False
self.sync_dict = {}
self.get_parallel_context_mock = {
"projects": [self.project],
"sync_dict": self.sync_dict,
"ssh_proxy": None,
}
@mock.patch("subcmds.sync.Sync.is_multiprocessing_active")
def test_fetch_one_keyboard_interrupt_main_process(self, mock_is_active):
"""Test that _FetchOne re-raises KeyboardInterrupt if not worker."""
mock_is_active.return_value = False
self.project.Sync_NetworkHalf.side_effect = KeyboardInterrupt()
with mock.patch.object(
sync.Sync,
"get_parallel_context",
return_value=self.get_parallel_context_mock,
):
with self.assertRaises(KeyboardInterrupt):
sync.Sync._FetchOne(self.opt, 0)
@mock.patch("subcmds.sync.Sync.is_multiprocessing_active")
def test_fetch_one_keyboard_interrupt_worker_process(self, mock_is_active):
"""Test that _FetchOne suppresses KeyboardInterrupt in workers."""
mock_is_active.return_value = True
self.project.Sync_NetworkHalf.side_effect = KeyboardInterrupt()
with mock.patch.object(
sync.Sync,
"get_parallel_context",
return_value=self.get_parallel_context_mock,
):
result = sync.Sync._FetchOne(self.opt, 0)
self.assertFalse(result.success)
class CheckForBloatedProjects(unittest.TestCase):
"""Tests for Sync._CheckForBloatedProjects."""
def setUp(self):
self.cmd = sync.Sync()
self.opt = mock.Mock()
self.opt.quiet = True
self.opt.jobs = 1
self.project = mock.MagicMock(clone_depth="1")
self.project.name = "project"
self.project.Exists = True
self.project.worktree = "worktree"
self.project.stateless_prune_needed = False
self.cmd.git_event_log = mock.MagicMock()
self.cmd._bloated_projects = []
@mock.patch("subcmds.sync.git_require")
def test_git_version_unsupported(self, mock_git_require):
"""Test that it returns early if git version is unsupported."""
mock_git_require.return_value = False
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertFalse(self.cmd.git_event_log.ErrorEvent.called)
@mock.patch("subcmds.sync.git_require")
def test_no_projects(self, mock_git_require):
"""Test that it returns early if no projects have clone_depth."""
mock_git_require.return_value = True
self.project.clone_depth = None
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertFalse(self.cmd.git_event_log.ErrorEvent.called)
@mock.patch("subcmds.sync.git_require")
@mock.patch("subcmds.sync.Progress")
def test_bloated_project_found(self, mock_progress, mock_git_require):
"""Test that it adds project to _bloated_projects."""
mock_git_require.return_value = True
self.cmd.get_parallel_context = mock.Mock(
return_value={"projects": [self.project]}
)
def mock_execute_in_parallel(
jobs, func, work_items, callback, **kwargs
):
callback(None, mock.Mock(), ["project"])
return True
self.cmd.ExecuteInParallel = mock_execute_in_parallel
with mock.patch.object(self.cmd, "ParallelContext"):
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertEqual(self.cmd._bloated_projects, ["project"])
@mock.patch("subcmds.sync.git_require")
@mock.patch("subcmds.sync.Progress")
def test_stateless_prune_excluded(self, mock_progress, mock_git_require):
"""Test that projects pruned for stateless sync are excluded."""
mock_git_require.return_value = True
self.project.stateless_prune_needed = True
self.cmd.ExecuteInParallel = mock.Mock()
with mock.patch.object(self.cmd, "ParallelContext"):
self.cmd._CheckForBloatedProjects([self.project], self.opt)
self.assertFalse(self.cmd.ExecuteInParallel.called)
self.assertEqual(self.cmd._bloated_projects, [])
class GCProjectsTest(unittest.TestCase):
"""Tests for Sync._GCProjects."""
def setUp(self):
self.cmd = sync.Sync()
self.opt = mock.Mock()
self.opt.quiet = True
self.opt.auto_gc = True
self.opt.jobs = 1
self.project = mock.MagicMock()
self.project.name = "project"
self.project.objdir = "objdir"
self.project.gitdir = "gitdir"
self.project.bare_git = mock.MagicMock()
self.project.bare_git._project = self.project
self.cmd.git_event_log = mock.MagicMock()
@mock.patch("subcmds.sync.Progress")
def test_GCProjects_skip_gc(self, mock_progress):
"""Test that it skips GC if opt.auto_gc is False."""
self.opt.auto_gc = False
with mock.patch.object(
sync.Sync, "_SetPreciousObjectsState"
) as mock_set_state:
self.cmd._GCProjects([self.project], self.opt, None)
mock_set_state.assert_called_once_with(self.project, self.opt)
self.assertFalse(self.project.bare_git.gc.called)
@mock.patch("subcmds.sync.Progress")
def test_GCProjects_sequential(self, mock_progress):
"""Test sequential GC (jobs < 2)."""
with mock.patch.object(sync.Sync, "_SetPreciousObjectsState"):
self.cmd._GCProjects([self.project], self.opt, None)
self.project.bare_git.gc.assert_called_once_with(
"--auto", config={"gc.autoDetach": "false"}
)
# Verify that gc.autoDetach was not permanently set in config.
for call in self.project.config.SetString.call_args_list:
self.assertNotEqual(call.args[0], "gc.autoDetach")
@mock.patch("subcmds.sync.Progress")
def test_GCProjects_parallel(self, mock_progress):
"""Test parallel GC (jobs >= 2)."""
self.opt.jobs = 2
with mock.patch.object(sync.Sync, "_SetPreciousObjectsState"):
with mock.patch("subcmds.sync._threading.Thread") as mock_thread:
mock_t = mock.MagicMock()
mock_thread.return_value = mock_t
err_event = mock.Mock()
err_event.is_set.return_value = False
self.cmd._GCProjects([self.project], self.opt, err_event)
self.assertTrue(mock_thread.called)
class SyncCommand(unittest.TestCase):
"""Tests for cmd.Execute."""
@@ -681,6 +886,9 @@ class InterleavedSyncTest(unittest.TestCase):
# Mock _GetCurrentBranchOnly for worker tests.
mock.patch.object(sync.Sync, "_GetCurrentBranchOnly").start()
self.cmd._fetch_times = mock.Mock()
self.cmd._local_sync_state = mock.Mock()
def tearDown(self):
"""Clean up resources."""
shutil.rmtree(self.repodir)
@@ -801,6 +1009,7 @@ class InterleavedSyncTest(unittest.TestCase):
with mock.patch("subcmds.sync.SyncBuffer") as mock_sync_buffer:
mock_sync_buf_instance = mock.MagicMock()
mock_sync_buf_instance.Finish.return_value = True
mock_sync_buf_instance.errors = []
mock_sync_buffer.return_value = mock_sync_buf_instance
result_obj = self.cmd._SyncProjectList(opt, [0])
@@ -809,8 +1018,8 @@ class InterleavedSyncTest(unittest.TestCase):
result = result_obj.results[0]
self.assertTrue(result.fetch_success)
self.assertTrue(result.checkout_success)
self.assertIsNone(result.fetch_error)
self.assertIsNone(result.checkout_error)
self.assertEqual(result.fetch_errors, [])
self.assertEqual(result.checkout_errors, [])
project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_called_once()
@@ -832,8 +1041,8 @@ class InterleavedSyncTest(unittest.TestCase):
self.assertFalse(result.fetch_success)
self.assertFalse(result.checkout_success)
self.assertEqual(result.fetch_error, fetch_error)
self.assertIsNone(result.checkout_error)
self.assertEqual(result.fetch_errors, [fetch_error])
self.assertEqual(result.checkout_errors, [])
project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_not_called()
@@ -870,7 +1079,7 @@ class InterleavedSyncTest(unittest.TestCase):
self.assertFalse(result.fetch_success)
self.assertFalse(result.checkout_success)
self.assertEqual(result.fetch_error, fetch_error)
self.assertEqual(result.fetch_errors, [fetch_error])
project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_not_called()
@@ -892,8 +1101,8 @@ class InterleavedSyncTest(unittest.TestCase):
self.assertTrue(result.fetch_success)
self.assertFalse(result.checkout_success)
self.assertIsNone(result.fetch_error)
self.assertEqual(result.checkout_error, checkout_error)
self.assertEqual(result.fetch_errors, [])
self.assertEqual(result.checkout_errors, [checkout_error])
project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_called_once()
@@ -909,6 +1118,7 @@ class InterleavedSyncTest(unittest.TestCase):
with mock.patch("subcmds.sync.SyncBuffer") as mock_sync_buffer:
mock_sync_buf_instance = mock.MagicMock()
mock_sync_buf_instance.Finish.return_value = True
mock_sync_buf_instance.errors = []
mock_sync_buffer.return_value = mock_sync_buf_instance
result_obj = self.cmd._SyncProjectList(opt, [0])
@@ -936,3 +1146,311 @@ class InterleavedSyncTest(unittest.TestCase):
self.assertTrue(result.checkout_success)
project.Sync_NetworkHalf.assert_called_once()
project.Sync_LocalHalf.assert_not_called()
class UpdateCopyLinkfileListTest(unittest.TestCase):
"""Tests for Sync.UpdateCopyLinkfileList."""
def setUp(self):
self.tempdirobj = tempfile.TemporaryDirectory(prefix="repo_tests")
self.topdir = self.tempdirobj.name
self.repodir = os.path.join(self.topdir, ".repo")
os.makedirs(self.repodir)
manifest = mock.MagicMock()
manifest.subdir = self.repodir
self.manifest = manifest
git_event_log = mock.MagicMock(ErrorEvent=mock.Mock(return_value=None))
self.cmd = sync.Sync(
manifest=manifest,
outer_client=mock.MagicMock(),
git_event_log=git_event_log,
)
self.cmd.client = mock.MagicMock(topdir=self.topdir)
def tearDown(self):
self.tempdirobj.cleanup()
def _write_copylinkfile_json(self, data: dict) -> None:
path = os.path.join(self.repodir, "copy-link-files.json")
with open(path, "w") as f:
json.dump(data, f)
def _setup_projects(self, linkfile_dests: list) -> None:
project = mock.MagicMock()
project.linkfiles = [mock.MagicMock(dest=d) for d in linkfile_dests]
project.copyfiles = []
mock.patch.object(
self.cmd, "GetProjects", return_value=[project]
).start()
def test_removes_old_symlink_dest(self):
"""Old linkfile dests that are symlinks should be removed."""
old_dest = os.path.join(self.topdir, "old-link")
os.symlink("target", old_dest)
self._write_copylinkfile_json(
{"linkfile": ["old-link"], "copyfile": []}
)
self._setup_projects([])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(old_dest))
def test_does_not_delete_through_new_symlink(self):
"""Old dests that resolve through a new symlink must not delete files.
When the manifest changes from individual linkfiles inside a directory
to a single directory linkfile, and _CopyAndLinkFiles has already
created the symlink (interleaved mode), cleanup must not follow the
symlink and delete real project files.
"""
project_dir = os.path.join(self.topdir, "vendor", "tools", "llms")
os.makedirs(os.path.join(project_dir, "dot-llms", "rules"))
os.makedirs(os.path.join(project_dir, "dot-llms", "skills"))
with open(
os.path.join(project_dir, "dot-llms", "rules", "basics.md"), "w"
) as f:
f.write("# basics")
with open(
os.path.join(project_dir, "dot-llms", "skills", "repo.md"), "w"
) as f:
f.write("# repo")
# Simulate interleaved mode: .llms -> vendor/tools/llms/dot-llms.
llms_link = os.path.join(self.topdir, ".llms")
os.symlink("vendor/tools/llms/dot-llms", llms_link)
self._write_copylinkfile_json(
{"linkfile": [".llms/rules", ".llms/skills"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
# Real project files must still exist.
self.assertTrue(
os.path.exists(
os.path.join(project_dir, "dot-llms", "rules", "basics.md")
)
)
self.assertTrue(
os.path.exists(
os.path.join(project_dir, "dot-llms", "skills", "repo.md")
),
)
self.assertTrue(os.path.islink(llms_link))
def test_cleans_up_empty_parent_dirs(self):
"""After removing old dests, empty parent directories are removed."""
llms_dir = os.path.join(self.topdir, ".llms")
os.makedirs(llms_dir)
os.symlink(
"../vendor/tools/llms/rules", os.path.join(llms_dir, "rules")
)
os.symlink(
"../vendor/tools/llms/skills",
os.path.join(llms_dir, "skills"),
)
self._write_copylinkfile_json(
{"linkfile": [".llms/rules", ".llms/skills"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "rules")))
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "skills")))
# Parent directory should be removed since it's now empty.
self.assertFalse(os.path.exists(llms_dir))
def test_preserves_nonempty_parent_dirs(self):
"""Non-empty parent directories are preserved after old dest removal."""
llms_dir = os.path.join(self.topdir, ".llms")
os.makedirs(llms_dir)
os.symlink(
"../vendor/tools/llms/rules", os.path.join(llms_dir, "rules")
)
with open(os.path.join(llms_dir, "my-notes.txt"), "w") as f:
f.write("user content")
self._write_copylinkfile_json(
{"linkfile": [".llms/rules"], "copyfile": []}
)
self._setup_projects([".llms"])
self.cmd.UpdateCopyLinkfileList(self.manifest)
self.assertFalse(os.path.lexists(os.path.join(llms_dir, "rules")))
self.assertTrue(os.path.exists(os.path.join(llms_dir, "my-notes.txt")))
self.assertTrue(os.path.isdir(llms_dir))
class SyncToSuperprojectRevTests(unittest.TestCase):
"""Tests for Sync._SyncToSuperprojectRev."""
def setUp(self):
self.repodir = tempfile.mkdtemp(".repo")
self.manifest = mock.MagicMock(repodir=self.repodir)
self.manifest.superproject = mock.MagicMock()
self.manifest.path_prefix = ""
self.mp = mock.MagicMock()
self.cmd = sync.Sync(manifest=self.manifest)
self.cmd.outer_manifest = self.manifest
self.opt = mock.Mock()
self.opt.verbose = False
self.opt.superproject_revision = "deadbeef"
self.opt.mp_update = True
self.errors = []
def tearDown(self):
shutil.rmtree(self.repodir)
@mock.patch("subcmds.sync.GitCommand")
def test_successful_sync(self, mock_git_command):
"""Test successful sync to superproject rev."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 0
mock_git.stdout = "proj branch manifest_commit_hash\n"
mock_git_command.return_value = mock_git
with mock.patch.object(
self.cmd, "_UpdateManifestProject"
) as mock_update:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
mock_superproject.SetRevisionId.assert_called_with("deadbeef")
mock_superproject.Sync.assert_called_once()
mock_git_command.assert_called_once()
self.mp.SetRevision.assert_called_with("manifest_commit_hash")
mock_update.assert_called_once()
self.assertEqual(self.errors, [])
@mock.patch("subcmds.sync.GitCommand")
def test_parse_error(self, mock_git_command):
"""Test error when .supermanifest cannot be parsed."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 0
# Invalid format (not 3 parts)
mock_git.stdout = "invalid_content\n"
mock_git_command.return_value = mock_git
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("could not parse .supermanifest", str(e.exception))
@mock.patch("subcmds.sync.GitCommand")
def test_read_error(self, mock_git_command):
"""Test error when reading .supermanifest fails."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=True)
mock_git = mock.Mock()
mock_git.Wait.return_value = 1
mock_git.stderr = "git error"
mock_git_command.return_value = mock_git
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("failed to read .supermanifest", str(e.exception))
def test_no_superproject(self):
"""Test error when superproject is not defined."""
self.manifest.superproject = None
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("superproject not defined", str(e.exception))
@mock.patch("subcmds.sync.GitCommand")
def test_sync_failure(self, mock_git_command):
"""Test error when superproject sync fails."""
mock_superproject = self.manifest.superproject
mock_superproject.Sync.return_value = mock.Mock(success=False)
with self.assertRaises(sync.SyncError) as e:
self.cmd._SyncToSuperprojectRev(
self.opt, self.manifest, self.mp, "name", self.errors
)
self.assertIn("failed to sync superproject", str(e.exception))
class UpdateAllManifestProjectsTests(unittest.TestCase):
"""Tests for Sync._UpdateAllManifestProjects."""
def setUp(self):
self.repodir = tempfile.mkdtemp(".repo")
self.manifest = mock.MagicMock(repodir=self.repodir)
self.manifest.superproject = mock.MagicMock()
self.manifest.path_prefix = ""
self.manifest.standalone_manifest_url = None
self.manifest.submanifests = {}
self.mp = mock.MagicMock()
self.mp.manifest = self.manifest
self.mp.standalone_manifest_url = None
self.cmd = sync.Sync(manifest=self.manifest)
self.cmd.outer_manifest = self.manifest
self.opt = mock.Mock()
self.opt.verbose = False
self.opt.superproject_revision = None
self.opt.mp_update = True
self.errors = []
def tearDown(self):
shutil.rmtree(self.repodir)
def test_superproject_revision_outer_manifest(self):
"""Test that _SyncToSuperprojectRev is called for outer manifest."""
self.opt.superproject_revision = "deadbeef"
with mock.patch.object(
self.cmd, "_SyncToSuperprojectRev"
) as mock_sync_to_rev:
self.cmd._UpdateAllManifestProjects(
self.opt, self.mp, "name", self.errors
)
mock_sync_to_rev.assert_called_once_with(
self.opt, self.manifest, self.mp, "name", self.errors
)
def test_superproject_revision_submanifest(self):
"""Test that _SyncToSuperprojectRev is NOT called for submanifest."""
self.opt.superproject_revision = "deadbeef"
submanifest = mock.MagicMock()
submanifest.path_prefix = "sub/"
submanifest.standalone_manifest_url = None
self.mp.manifest = submanifest
with mock.patch.object(
self.cmd, "_SyncToSuperprojectRev"
) as mock_sync_to_rev:
with mock.patch.object(
self.cmd, "_UpdateManifestProject"
) as mock_update_manifest:
self.cmd._UpdateAllManifestProjects(
self.opt, self.mp, "name", self.errors
)
mock_sync_to_rev.assert_not_called()
mock_update_manifest.assert_called_once()
+33 -38
View File
@@ -14,9 +14,10 @@
"""Unittests for the subcmds/upload.py module."""
import unittest
from unittest import mock
import pytest
from error import GitError
from error import UploadError
from subcmds import upload
@@ -26,45 +27,39 @@ class UnexpectedError(Exception):
"""An exception not expected by upload command."""
class UploadCommand(unittest.TestCase):
"""Check registered all_commands."""
# A stub people list (reviewers, cc).
_STUB_PEOPLE = ([], [])
def setUp(self):
self.cmd = upload.Upload()
self.branch = mock.MagicMock()
self.people = mock.MagicMock()
self.opt, _ = self.cmd.OptionParser.parse_args([])
mock.patch.object(
self.cmd, "_AppendAutoList", return_value=None
).start()
mock.patch.object(self.cmd, "git_event_log").start()
def tearDown(self):
mock.patch.stopall()
@pytest.fixture
def cmd() -> upload.Upload:
"""Fixture to provide an Upload command instance with mocked methods."""
cmd = upload.Upload()
with mock.patch.object(
cmd, "_AppendAutoList", return_value=None
), mock.patch.object(cmd, "git_event_log"):
yield cmd
def test_UploadAndReport_UploadError(self):
"""Check UploadExitError raised when UploadError encountered."""
side_effect = UploadError("upload error")
with mock.patch.object(
self.cmd, "_UploadBranch", side_effect=side_effect
):
with self.assertRaises(upload.UploadExitError):
self.cmd._UploadAndReport(self.opt, [self.branch], self.people)
def test_UploadAndReport_GitError(self):
"""Check UploadExitError raised when GitError encountered."""
side_effect = GitError("some git error")
with mock.patch.object(
self.cmd, "_UploadBranch", side_effect=side_effect
):
with self.assertRaises(upload.UploadExitError):
self.cmd._UploadAndReport(self.opt, [self.branch], self.people)
def test_UploadAndReport_UploadError(cmd: upload.Upload) -> None:
"""Check UploadExitError raised when UploadError encountered."""
opt, _ = cmd.OptionParser.parse_args([])
with mock.patch.object(cmd, "_UploadBranch", side_effect=UploadError("")):
with pytest.raises(upload.UploadExitError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
def test_UploadAndReport_UnhandledError(self):
"""Check UnexpectedError passed through."""
side_effect = UnexpectedError("some os error")
with mock.patch.object(
self.cmd, "_UploadBranch", side_effect=side_effect
):
with self.assertRaises(type(side_effect)):
self.cmd._UploadAndReport(self.opt, [self.branch], self.people)
def test_UploadAndReport_GitError(cmd: upload.Upload) -> None:
"""Check UploadExitError raised when GitError encountered."""
opt, _ = cmd.OptionParser.parse_args([])
with mock.patch.object(cmd, "_UploadBranch", side_effect=GitError("")):
with pytest.raises(upload.UploadExitError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
def test_UploadAndReport_UnhandledError(cmd: upload.Upload) -> None:
"""Check UnexpectedError passed through."""
opt, _ = cmd.OptionParser.parse_args([])
with mock.patch.object(cmd, "_UploadBranch", side_effect=UnexpectedError):
with pytest.raises(UnexpectedError):
cmd._UploadAndReport(opt, [mock.MagicMock()], _STUB_PEOPLE)
+263
View File
@@ -0,0 +1,263 @@
# Copyright (C) 2025 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import shutil
from unittest import mock
import pytest
import project
from subcmds import wipe
def _create_mock_project(tempdir, name, objdir_path=None, has_changes=False):
"""Creates a mock project with necessary attributes and directories."""
worktree = os.path.join(tempdir, name)
gitdir = os.path.join(tempdir, ".repo/projects", f"{name}.git")
if objdir_path:
objdir = objdir_path
else:
objdir = os.path.join(tempdir, ".repo/project-objects", f"{name}.git")
os.makedirs(worktree, exist_ok=True)
os.makedirs(gitdir, exist_ok=True)
os.makedirs(objdir, exist_ok=True)
proj = project.Project(
manifest=mock.MagicMock(),
name=name,
remote=mock.MagicMock(),
gitdir=gitdir,
objdir=objdir,
worktree=worktree,
relpath=name,
revisionExpr="main",
revisionId="abcd",
)
proj.HasChanges = mock.MagicMock(return_value=has_changes)
def side_effect_delete_worktree(force=False, verbose=False):
if os.path.exists(proj.worktree):
shutil.rmtree(proj.worktree)
if os.path.exists(proj.gitdir):
shutil.rmtree(proj.gitdir)
return True
proj.DeleteWorktree = mock.MagicMock(
side_effect=side_effect_delete_worktree
)
return proj
def _run_wipe(all_projects, projects_to_wipe_names, options=None):
"""Helper to run the Wipe command with mocked projects."""
cmd = wipe.Wipe()
cmd.manifest = mock.MagicMock()
def get_projects_mock(projects, all_manifests=False, **kwargs):
if projects is None:
return all_projects
names_to_find = set(projects)
return [p for p in all_projects if p.name in names_to_find]
cmd.GetProjects = mock.MagicMock(side_effect=get_projects_mock)
if options is None:
options = []
opts = cmd.OptionParser.parse_args(options + projects_to_wipe_names)[0]
cmd.CommonValidateOptions(opts, projects_to_wipe_names)
cmd.ValidateOptions(opts, projects_to_wipe_names)
cmd.Execute(opts, projects_to_wipe_names)
def test_wipe_single_unshared_project(tmp_path):
"""Test wiping a single project that is not shared."""
p1 = _create_mock_project(str(tmp_path), "project/one")
_run_wipe([p1], ["project/one"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p1.objdir)
def test_wipe_multiple_unshared_projects(tmp_path):
"""Test wiping multiple projects that are not shared."""
p1 = _create_mock_project(str(tmp_path), "project/one")
p2 = _create_mock_project(str(tmp_path), "project/two")
_run_wipe([p1, p2], ["project/one", "project/two"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p1.objdir)
assert not os.path.exists(p2.worktree)
assert not os.path.exists(p2.gitdir)
assert not os.path.exists(p2.objdir)
def test_wipe_shared_project_no_force_raises_error(tmp_path):
"""Test that wiping a shared project without --force raises an error."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path), "project/one", objdir_path=shared_objdir
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
with pytest.raises(wipe.Error) as e:
_run_wipe([p1, p2], ["project/one"])
assert "shared object directories" in str(e.value)
assert "project/one" in str(e.value)
assert "project/two" in str(e.value)
assert os.path.exists(p1.worktree)
assert os.path.exists(p1.gitdir)
assert os.path.exists(p2.worktree)
assert os.path.exists(p2.gitdir)
assert os.path.exists(shared_objdir)
def test_wipe_shared_project_with_force(tmp_path):
"""Test wiping a shared project with --force."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path), "project/one", objdir_path=shared_objdir
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
_run_wipe([p1, p2], ["project/one"], options=["--force"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert os.path.exists(shared_objdir)
assert os.path.exists(p2.worktree)
assert os.path.exists(p2.gitdir)
def test_wipe_all_sharing_projects(tmp_path):
"""Test wiping all projects that share an object directory."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path), "project/one", objdir_path=shared_objdir
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
_run_wipe([p1, p2], ["project/one", "project/two"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p2.worktree)
assert not os.path.exists(p2.gitdir)
assert not os.path.exists(shared_objdir)
def test_wipe_with_uncommitted_changes_raises_error(tmp_path):
"""Test wiping a project with uncommitted changes raises an error."""
p1 = _create_mock_project(str(tmp_path), "project/one", has_changes=True)
with pytest.raises(wipe.Error) as e:
_run_wipe([p1], ["project/one"])
assert "uncommitted changes" in str(e.value)
assert "project/one" in str(e.value)
assert os.path.exists(p1.worktree)
assert os.path.exists(p1.gitdir)
assert os.path.exists(p1.objdir)
def test_wipe_with_uncommitted_changes_with_force(tmp_path):
"""Test wiping a project with uncommitted changes with --force."""
p1 = _create_mock_project(str(tmp_path), "project/one", has_changes=True)
_run_wipe([p1], ["project/one"], options=["--force"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p1.objdir)
def test_wipe_uncommitted_and_shared_raises_combined_error(tmp_path):
"""Test that uncommitted and shared projects raise a combined error."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path),
"project/one",
objdir_path=shared_objdir,
has_changes=True,
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
with pytest.raises(wipe.Error) as e:
_run_wipe([p1, p2], ["project/one"])
assert "uncommitted changes" in str(e.value)
assert "shared object directories" in str(e.value)
assert "project/one" in str(e.value)
assert "project/two" in str(e.value)
assert os.path.exists(p1.worktree)
assert os.path.exists(p1.gitdir)
assert os.path.exists(p2.worktree)
assert os.path.exists(p2.gitdir)
assert os.path.exists(shared_objdir)
def test_wipe_shared_project_with_force_shared(tmp_path):
"""Test wiping a shared project with --force-shared."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path), "project/one", objdir_path=shared_objdir
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
_run_wipe([p1, p2], ["project/one"], options=["--force-shared"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert os.path.exists(shared_objdir)
assert os.path.exists(p2.worktree)
assert os.path.exists(p2.gitdir)
def test_wipe_with_uncommitted_changes_with_force_uncommitted(tmp_path):
"""Test wiping uncommitted changes with --force-uncommitted."""
p1 = _create_mock_project(str(tmp_path), "project/one", has_changes=True)
_run_wipe([p1], ["project/one"], options=["--force-uncommitted"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p1.objdir)
-28
View File
@@ -1,28 +0,0 @@
# Copyright 2022 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the update_manpages module."""
import unittest
from release import update_manpages
class UpdateManpagesTest(unittest.TestCase):
"""Tests the update-manpages code."""
def test_replace_regex(self):
"""Check that replace_regex works."""
data = "\n\033[1mSummary\033[m\n"
self.assertEqual(update_manpages.replace_regex(data), "\nSummary\n")
+345 -314
View File
@@ -19,266 +19,303 @@ import os
import re
import subprocess
import sys
import tempfile
import unittest
from unittest import mock
import git_command
import pytest
import utils_for_test
import main
import wrapper
def fixture(*paths):
"""Return a path relative to tests/fixtures."""
return os.path.join(os.path.dirname(__file__), "fixtures", *paths)
@pytest.fixture(autouse=True)
def reset_wrapper() -> None:
"""Reset the wrapper module every time."""
wrapper.Wrapper.cache_clear()
class RepoWrapperTestCase(unittest.TestCase):
"""TestCase for the wrapper module."""
def setUp(self):
"""Load the wrapper module every time."""
wrapper.Wrapper.cache_clear()
self.wrapper = wrapper.Wrapper()
@pytest.fixture
def repo_wrapper() -> wrapper.Wrapper:
"""Fixture for the wrapper module."""
return wrapper.Wrapper()
class RepoWrapperUnitTest(RepoWrapperTestCase):
class GitCheckout:
"""Class to hold git checkout info for tests."""
def __init__(self, git_dir, rev_list):
self.git_dir = git_dir
self.rev_list = rev_list
@pytest.fixture(scope="module")
def git_checkout(tmp_path_factory) -> GitCheckout:
"""Fixture for tests that use a real/small git checkout.
Create a repo to operate on, but do it once per-test-run.
"""
tempdir = tmp_path_factory.mktemp("repo-rev-tests")
run_git = wrapper.Wrapper().run_git
remote = os.path.join(tempdir, "remote")
os.mkdir(remote)
utils_for_test.init_git_tree(remote)
run_git("commit", "--allow-empty", "-minit", cwd=remote)
run_git("branch", "stable", cwd=remote)
run_git("tag", "v1.0", cwd=remote)
run_git("commit", "--allow-empty", "-m2nd commit", cwd=remote)
rev_list = run_git("rev-list", "HEAD", cwd=remote).stdout.splitlines()
run_git("init", cwd=tempdir)
run_git(
"fetch",
remote,
"+refs/heads/*:refs/remotes/origin/*",
cwd=tempdir,
)
yield GitCheckout(tempdir, rev_list)
class TestRepoWrapper:
"""Tests helper functions in the repo wrapper"""
def test_version(self):
def test_version(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Make sure _Version works."""
with self.assertRaises(SystemExit) as e:
with pytest.raises(SystemExit) as e:
with mock.patch("sys.stdout", new_callable=io.StringIO) as stdout:
with mock.patch(
"sys.stderr", new_callable=io.StringIO
) as stderr:
self.wrapper._Version()
self.assertEqual(0, e.exception.code)
self.assertEqual("", stderr.getvalue())
self.assertIn("repo launcher version", stdout.getvalue())
repo_wrapper._Version()
assert e.value.code == 0
assert stderr.getvalue() == ""
assert "repo launcher version" in stdout.getvalue()
def test_python_constraints(self):
def test_python_constraints(self, repo_wrapper: wrapper.Wrapper) -> None:
"""The launcher should never require newer than main.py."""
self.assertGreaterEqual(
main.MIN_PYTHON_VERSION_HARD, self.wrapper.MIN_PYTHON_VERSION_HARD
assert (
main.MIN_PYTHON_VERSION_HARD >= repo_wrapper.MIN_PYTHON_VERSION_HARD
)
self.assertGreaterEqual(
main.MIN_PYTHON_VERSION_SOFT, self.wrapper.MIN_PYTHON_VERSION_SOFT
assert (
main.MIN_PYTHON_VERSION_SOFT >= repo_wrapper.MIN_PYTHON_VERSION_SOFT
)
# Make sure the versions are themselves in sync.
self.assertGreaterEqual(
self.wrapper.MIN_PYTHON_VERSION_SOFT,
self.wrapper.MIN_PYTHON_VERSION_HARD,
assert (
repo_wrapper.MIN_PYTHON_VERSION_SOFT
>= repo_wrapper.MIN_PYTHON_VERSION_HARD
)
def test_init_parser(self):
def test_repo_script_is_executable(self) -> None:
"""The repo launcher script should be executable."""
repo_path = utils_for_test.THIS_DIR.parent / "repo"
assert os.access(repo_path, os.X_OK), f"{repo_path} is not executable"
def test_init_parser(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Make sure 'init' GetParser works."""
parser = self.wrapper.GetParser()
parser = repo_wrapper.GetParser()
opts, args = parser.parse_args([])
self.assertEqual([], args)
self.assertIsNone(opts.manifest_url)
assert args == []
assert opts.manifest_url is None
class SetGitTrace2ParentSid(RepoWrapperTestCase):
class TestSetGitTrace2ParentSid:
"""Check SetGitTrace2ParentSid behavior."""
KEY = "GIT_TRACE2_PARENT_SID"
VALID_FORMAT = re.compile(r"^repo-[0-9]{8}T[0-9]{6}Z-P[0-9a-f]{8}$")
def test_first_set(self):
def test_first_set(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Test env var not yet set."""
env = {}
self.wrapper.SetGitTrace2ParentSid(env)
self.assertIn(self.KEY, env)
repo_wrapper.SetGitTrace2ParentSid(env)
assert self.KEY in env
value = env[self.KEY]
self.assertRegex(value, self.VALID_FORMAT)
assert self.VALID_FORMAT.match(value)
def test_append(self):
def test_append(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Test env var is appended."""
env = {self.KEY: "pfx"}
self.wrapper.SetGitTrace2ParentSid(env)
self.assertIn(self.KEY, env)
repo_wrapper.SetGitTrace2ParentSid(env)
assert self.KEY in env
value = env[self.KEY]
self.assertTrue(value.startswith("pfx/"))
self.assertRegex(value[4:], self.VALID_FORMAT)
assert value.startswith("pfx/")
assert self.VALID_FORMAT.match(value[4:])
def test_global_context(self):
def test_global_context(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check os.environ gets updated by default."""
os.environ.pop(self.KEY, None)
self.wrapper.SetGitTrace2ParentSid()
self.assertIn(self.KEY, os.environ)
repo_wrapper.SetGitTrace2ParentSid()
assert self.KEY in os.environ
value = os.environ[self.KEY]
self.assertRegex(value, self.VALID_FORMAT)
assert self.VALID_FORMAT.match(value)
class RunCommand(RepoWrapperTestCase):
class TestRunCommand:
"""Check run_command behavior."""
def test_capture(self):
def test_capture(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check capture_output handling."""
ret = self.wrapper.run_command(["echo", "hi"], capture_output=True)
ret = repo_wrapper.run_command(["echo", "hi"], capture_output=True)
# echo command appends OS specific linesep, but on Windows + Git Bash
# we get UNIX ending, so we allow both.
self.assertIn(ret.stdout, ["hi" + os.linesep, "hi\n"])
assert ret.stdout in ["hi" + os.linesep, "hi\n"]
def test_check(self):
def test_check(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check check handling."""
self.wrapper.run_command(["true"], check=False)
self.wrapper.run_command(["true"], check=True)
self.wrapper.run_command(["false"], check=False)
with self.assertRaises(subprocess.CalledProcessError):
self.wrapper.run_command(["false"], check=True)
repo_wrapper.run_command(["true"], check=False)
repo_wrapper.run_command(["true"], check=True)
repo_wrapper.run_command(["false"], check=False)
with pytest.raises(subprocess.CalledProcessError):
repo_wrapper.run_command(["false"], check=True)
class RunGit(RepoWrapperTestCase):
class TestRunGit:
"""Check run_git behavior."""
def test_capture(self):
def test_capture(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check capture_output handling."""
ret = self.wrapper.run_git("--version")
self.assertIn("git", ret.stdout)
ret = repo_wrapper.run_git("--version")
assert "git" in ret.stdout
def test_check(self):
def test_check(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check check handling."""
with self.assertRaises(self.wrapper.CloneFailure):
self.wrapper.run_git("--version-asdfasdf")
self.wrapper.run_git("--version-asdfasdf", check=False)
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper.run_git("--version-asdfasdf")
repo_wrapper.run_git("--version-asdfasdf", check=False)
class ParseGitVersion(RepoWrapperTestCase):
class TestParseGitVersion:
"""Check ParseGitVersion behavior."""
def test_autoload(self):
def test_autoload(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check we can load the version from the live git."""
ret = self.wrapper.ParseGitVersion()
self.assertIsNotNone(ret)
assert repo_wrapper.ParseGitVersion() is not None
def test_bad_ver(self):
def test_bad_ver(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check handling of bad git versions."""
ret = self.wrapper.ParseGitVersion(ver_str="asdf")
self.assertIsNone(ret)
assert repo_wrapper.ParseGitVersion(ver_str="asdf") is None
def test_normal_ver(self):
def test_normal_ver(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check handling of normal git versions."""
ret = self.wrapper.ParseGitVersion(ver_str="git version 2.25.1")
self.assertEqual(2, ret.major)
self.assertEqual(25, ret.minor)
self.assertEqual(1, ret.micro)
self.assertEqual("2.25.1", ret.full)
ret = repo_wrapper.ParseGitVersion(ver_str="git version 2.25.1")
assert ret.major == 2
assert ret.minor == 25
assert ret.micro == 1
assert ret.full == "2.25.1"
def test_extended_ver(self):
def test_extended_ver(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check handling of extended distro git versions."""
ret = self.wrapper.ParseGitVersion(
ret = repo_wrapper.ParseGitVersion(
ver_str="git version 1.30.50.696.g5e7596f4ac-goog"
)
self.assertEqual(1, ret.major)
self.assertEqual(30, ret.minor)
self.assertEqual(50, ret.micro)
self.assertEqual("1.30.50.696.g5e7596f4ac-goog", ret.full)
assert ret.major == 1
assert ret.minor == 30
assert ret.micro == 50
assert ret.full == "1.30.50.696.g5e7596f4ac-goog"
class CheckGitVersion(RepoWrapperTestCase):
class TestCheckGitVersion:
"""Check _CheckGitVersion behavior."""
def test_unknown(self):
def test_unknown(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Unknown versions should abort."""
with mock.patch.object(
self.wrapper, "ParseGitVersion", return_value=None
repo_wrapper, "ParseGitVersion", return_value=None
):
with self.assertRaises(self.wrapper.CloneFailure):
self.wrapper._CheckGitVersion()
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper._CheckGitVersion()
def test_old(self):
def test_old(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Old versions should abort."""
with mock.patch.object(
self.wrapper,
repo_wrapper,
"ParseGitVersion",
return_value=self.wrapper.GitVersion(1, 0, 0, "1.0.0"),
return_value=repo_wrapper.GitVersion(1, 0, 0, "1.0.0"),
):
with self.assertRaises(self.wrapper.CloneFailure):
self.wrapper._CheckGitVersion()
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper._CheckGitVersion()
def test_new(self):
def test_new(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Newer versions should run fine."""
with mock.patch.object(
self.wrapper,
repo_wrapper,
"ParseGitVersion",
return_value=self.wrapper.GitVersion(100, 0, 0, "100.0.0"),
return_value=repo_wrapper.GitVersion(100, 0, 0, "100.0.0"),
):
self.wrapper._CheckGitVersion()
repo_wrapper._CheckGitVersion()
class Requirements(RepoWrapperTestCase):
class TestRequirements:
"""Check Requirements handling."""
def test_missing_file(self):
def test_missing_file(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Don't crash if the file is missing (old version)."""
testdir = os.path.dirname(os.path.realpath(__file__))
self.assertIsNone(self.wrapper.Requirements.from_dir(testdir))
self.assertIsNone(
self.wrapper.Requirements.from_file(
os.path.join(testdir, "xxxxxxxxxxxxxxxxxxxxxxxx")
assert (
repo_wrapper.Requirements.from_dir(utils_for_test.THIS_DIR) is None
)
assert (
repo_wrapper.Requirements.from_file(
utils_for_test.THIS_DIR / "xxxxxxxxxxxxxxxxxxxxxxxx"
)
is None
)
def test_corrupt_data(self):
def test_corrupt_data(self, repo_wrapper: wrapper.Wrapper) -> None:
"""If the file can't be parsed, don't blow up."""
self.assertIsNone(self.wrapper.Requirements.from_file(__file__))
self.assertIsNone(self.wrapper.Requirements.from_data(b"x"))
assert repo_wrapper.Requirements.from_file(__file__) is None
assert repo_wrapper.Requirements.from_data(b"x") is None
def test_valid_data(self):
def test_valid_data(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Make sure we can parse the file we ship."""
self.assertIsNotNone(self.wrapper.Requirements.from_data(b"{}"))
rootdir = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
self.assertIsNotNone(self.wrapper.Requirements.from_dir(rootdir))
self.assertIsNotNone(
self.wrapper.Requirements.from_file(
os.path.join(rootdir, "requirements.json")
)
assert repo_wrapper.Requirements.from_data(b"{}") is not None
rootdir = utils_for_test.THIS_DIR.parent
assert repo_wrapper.Requirements.from_dir(rootdir) is not None
assert (
repo_wrapper.Requirements.from_file(rootdir / "requirements.json")
is not None
)
def test_format_ver(self):
def test_format_ver(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check format_ver can format."""
self.assertEqual(
"1.2.3", self.wrapper.Requirements._format_ver((1, 2, 3))
)
self.assertEqual("1", self.wrapper.Requirements._format_ver([1]))
assert repo_wrapper.Requirements._format_ver((1, 2, 3)) == "1.2.3"
assert repo_wrapper.Requirements._format_ver([1]) == "1"
def test_assert_all_unknown(self):
def test_assert_all_unknown(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all works with incompatible file."""
reqs = self.wrapper.Requirements({})
reqs = repo_wrapper.Requirements({})
reqs.assert_all()
def test_assert_all_new_repo(self):
def test_assert_all_new_repo(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all accepts new enough repo."""
reqs = self.wrapper.Requirements({"repo": {"hard": [1, 0]}})
reqs = repo_wrapper.Requirements({"repo": {"hard": [1, 0]}})
reqs.assert_all()
def test_assert_all_old_repo(self):
def test_assert_all_old_repo(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all rejects old repo."""
reqs = self.wrapper.Requirements({"repo": {"hard": [99999, 0]}})
with self.assertRaises(SystemExit):
reqs = repo_wrapper.Requirements({"repo": {"hard": [99999, 0]}})
with pytest.raises(SystemExit):
reqs.assert_all()
def test_assert_all_new_python(self):
def test_assert_all_new_python(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all accepts new enough python."""
reqs = self.wrapper.Requirements({"python": {"hard": sys.version_info}})
reqs = repo_wrapper.Requirements({"python": {"hard": sys.version_info}})
reqs.assert_all()
def test_assert_all_old_python(self):
def test_assert_all_old_python(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_all rejects old python."""
reqs = self.wrapper.Requirements({"python": {"hard": [99999, 0]}})
with self.assertRaises(SystemExit):
reqs = repo_wrapper.Requirements({"python": {"hard": [99999, 0]}})
with pytest.raises(SystemExit):
reqs.assert_all()
def test_assert_ver_unknown(self):
def test_assert_ver_unknown(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_ver works with incompatible file."""
reqs = self.wrapper.Requirements({})
reqs = repo_wrapper.Requirements({})
reqs.assert_ver("xxx", (1, 0))
def test_assert_ver_new(self):
def test_assert_ver_new(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_ver allows new enough versions."""
reqs = self.wrapper.Requirements(
reqs = repo_wrapper.Requirements(
{"git": {"hard": [1, 0], "soft": [2, 0]}}
)
reqs.assert_ver("git", (1, 0))
@@ -286,285 +323,279 @@ class Requirements(RepoWrapperTestCase):
reqs.assert_ver("git", (2, 0))
reqs.assert_ver("git", (2, 5))
def test_assert_ver_old(self):
def test_assert_ver_old(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check assert_ver rejects old versions."""
reqs = self.wrapper.Requirements(
reqs = repo_wrapper.Requirements(
{"git": {"hard": [1, 0], "soft": [2, 0]}}
)
with self.assertRaises(SystemExit):
with pytest.raises(SystemExit):
reqs.assert_ver("git", (0, 5))
class NeedSetupGnuPG(RepoWrapperTestCase):
class TestNeedSetupGnuPG:
"""Check NeedSetupGnuPG behavior."""
def test_missing_dir(self):
def test_missing_dir(self, tmp_path, repo_wrapper: wrapper.Wrapper) -> None:
"""The ~/.repoconfig tree doesn't exist yet."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
self.wrapper.home_dot_repo = os.path.join(tempdir, "foo")
self.assertTrue(self.wrapper.NeedSetupGnuPG())
repo_wrapper.home_dot_repo = str(tmp_path / "foo")
assert repo_wrapper.NeedSetupGnuPG()
def test_missing_keyring(self):
def test_missing_keyring(
self, tmp_path, repo_wrapper: wrapper.Wrapper
) -> None:
"""The keyring-version file doesn't exist yet."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
self.wrapper.home_dot_repo = tempdir
self.assertTrue(self.wrapper.NeedSetupGnuPG())
repo_wrapper.home_dot_repo = str(tmp_path)
assert repo_wrapper.NeedSetupGnuPG()
def test_empty_keyring(self):
def test_empty_keyring(
self, tmp_path, repo_wrapper: wrapper.Wrapper
) -> None:
"""The keyring-version file exists, but is empty."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
self.wrapper.home_dot_repo = tempdir
with open(os.path.join(tempdir, "keyring-version"), "w"):
pass
self.assertTrue(self.wrapper.NeedSetupGnuPG())
repo_wrapper.home_dot_repo = str(tmp_path)
(tmp_path / "keyring-version").write_text("")
assert repo_wrapper.NeedSetupGnuPG()
def test_old_keyring(self):
def test_old_keyring(self, tmp_path, repo_wrapper: wrapper.Wrapper) -> None:
"""The keyring-version file exists, but it's old."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
self.wrapper.home_dot_repo = tempdir
with open(os.path.join(tempdir, "keyring-version"), "w") as fp:
fp.write("1.0\n")
self.assertTrue(self.wrapper.NeedSetupGnuPG())
repo_wrapper.home_dot_repo = str(tmp_path)
(tmp_path / "keyring-version").write_text("1.0\n")
assert repo_wrapper.NeedSetupGnuPG()
def test_new_keyring(self):
def test_new_keyring(self, tmp_path, repo_wrapper: wrapper.Wrapper) -> None:
"""The keyring-version file exists, and is up-to-date."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
self.wrapper.home_dot_repo = tempdir
with open(os.path.join(tempdir, "keyring-version"), "w") as fp:
fp.write("1000.0\n")
self.assertFalse(self.wrapper.NeedSetupGnuPG())
repo_wrapper.home_dot_repo = str(tmp_path)
(tmp_path / "keyring-version").write_text("1000.0\n")
assert not repo_wrapper.NeedSetupGnuPG()
class SetupGnuPG(RepoWrapperTestCase):
class TestSetupGnuPG:
"""Check SetupGnuPG behavior."""
def test_full(self):
def test_full(self, tmp_path, repo_wrapper: wrapper.Wrapper) -> None:
"""Make sure it works completely."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
self.wrapper.home_dot_repo = tempdir
self.wrapper.gpg_dir = os.path.join(
self.wrapper.home_dot_repo, "gnupg"
)
self.assertTrue(self.wrapper.SetupGnuPG(True))
with open(os.path.join(tempdir, "keyring-version")) as fp:
data = fp.read()
self.assertEqual(
".".join(str(x) for x in self.wrapper.KEYRING_VERSION),
data.strip(),
)
repo_wrapper.home_dot_repo = str(tmp_path)
repo_wrapper.gpg_dir = str(tmp_path / "gnupg")
assert repo_wrapper.SetupGnuPG(True)
data = (tmp_path / "keyring-version").read_text()
assert (
".".join(str(x) for x in repo_wrapper.KEYRING_VERSION)
== data.strip()
)
class VerifyRev(RepoWrapperTestCase):
class TestVerifyRev:
"""Check verify_rev behavior."""
def test_verify_passes(self):
def test_verify_passes(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check when we have a valid signed tag."""
desc_result = subprocess.CompletedProcess([], 0, "v1.0\n", "")
gpg_result = subprocess.CompletedProcess([], 0, "", "")
with mock.patch.object(
self.wrapper, "run_git", side_effect=(desc_result, gpg_result)
repo_wrapper, "run_git", side_effect=(desc_result, gpg_result)
):
ret = self.wrapper.verify_rev(
ret = repo_wrapper.verify_rev(
"/", "refs/heads/stable", "1234", True
)
self.assertEqual("v1.0^0", ret)
assert ret == "v1.0^0"
def test_unsigned_commit(self):
def test_unsigned_commit(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check we fall back to signed tag when we have an unsigned commit."""
desc_result = subprocess.CompletedProcess([], 0, "v1.0-10-g1234\n", "")
gpg_result = subprocess.CompletedProcess([], 0, "", "")
with mock.patch.object(
self.wrapper, "run_git", side_effect=(desc_result, gpg_result)
repo_wrapper, "run_git", side_effect=(desc_result, gpg_result)
):
ret = self.wrapper.verify_rev(
ret = repo_wrapper.verify_rev(
"/", "refs/heads/stable", "1234", True
)
self.assertEqual("v1.0^0", ret)
assert ret == "v1.0^0"
def test_verify_fails(self):
def test_verify_fails(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Check we fall back to signed tag when we have an unsigned commit."""
desc_result = subprocess.CompletedProcess([], 0, "v1.0-10-g1234\n", "")
gpg_result = Exception
gpg_result = RuntimeError
with mock.patch.object(
self.wrapper, "run_git", side_effect=(desc_result, gpg_result)
repo_wrapper, "run_git", side_effect=(desc_result, gpg_result)
):
with self.assertRaises(Exception):
self.wrapper.verify_rev("/", "refs/heads/stable", "1234", True)
with pytest.raises(RuntimeError):
repo_wrapper.verify_rev("/", "refs/heads/stable", "1234", True)
class GitCheckoutTestCase(RepoWrapperTestCase):
"""Tests that use a real/small git checkout."""
GIT_DIR = None
REV_LIST = None
@classmethod
def setUpClass(cls):
# Create a repo to operate on, but do it once per-class.
cls.tempdirobj = tempfile.TemporaryDirectory(prefix="repo-rev-tests")
cls.GIT_DIR = cls.tempdirobj.name
run_git = wrapper.Wrapper().run_git
remote = os.path.join(cls.GIT_DIR, "remote")
os.mkdir(remote)
# Tests need to assume, that main is default branch at init,
# which is not supported in config until 2.28.
if git_command.git_require((2, 28, 0)):
initstr = "--initial-branch=main"
else:
# Use template dir for init.
templatedir = tempfile.mkdtemp(prefix=".test-template")
with open(os.path.join(templatedir, "HEAD"), "w") as fp:
fp.write("ref: refs/heads/main\n")
initstr = "--template=" + templatedir
run_git("init", initstr, cwd=remote)
run_git("commit", "--allow-empty", "-minit", cwd=remote)
run_git("branch", "stable", cwd=remote)
run_git("tag", "v1.0", cwd=remote)
run_git("commit", "--allow-empty", "-m2nd commit", cwd=remote)
cls.REV_LIST = run_git(
"rev-list", "HEAD", cwd=remote
).stdout.splitlines()
run_git("init", cwd=cls.GIT_DIR)
run_git(
"fetch",
remote,
"+refs/heads/*:refs/remotes/origin/*",
cwd=cls.GIT_DIR,
)
@classmethod
def tearDownClass(cls):
if not cls.tempdirobj:
return
cls.tempdirobj.cleanup()
class ResolveRepoRev(GitCheckoutTestCase):
class TestResolveRepoRev:
"""Check resolve_repo_rev behavior."""
def test_explicit_branch(self):
def test_explicit_branch(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check refs/heads/branch argument."""
rrev, lrev = self.wrapper.resolve_repo_rev(
self.GIT_DIR, "refs/heads/stable"
rrev, lrev = repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, "refs/heads/stable"
)
self.assertEqual("refs/heads/stable", rrev)
self.assertEqual(self.REV_LIST[1], lrev)
assert rrev == "refs/heads/stable"
assert lrev == git_checkout.rev_list[1]
with self.assertRaises(self.wrapper.CloneFailure):
self.wrapper.resolve_repo_rev(self.GIT_DIR, "refs/heads/unknown")
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, "refs/heads/unknown"
)
def test_explicit_tag(self):
def test_explicit_tag(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check refs/tags/tag argument."""
rrev, lrev = self.wrapper.resolve_repo_rev(
self.GIT_DIR, "refs/tags/v1.0"
rrev, lrev = repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, "refs/tags/v1.0"
)
self.assertEqual("refs/tags/v1.0", rrev)
self.assertEqual(self.REV_LIST[1], lrev)
assert rrev == "refs/tags/v1.0"
assert lrev == git_checkout.rev_list[1]
with self.assertRaises(self.wrapper.CloneFailure):
self.wrapper.resolve_repo_rev(self.GIT_DIR, "refs/tags/unknown")
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, "refs/tags/unknown"
)
def test_branch_name(self):
def test_branch_name(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check branch argument."""
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, "stable")
self.assertEqual("refs/heads/stable", rrev)
self.assertEqual(self.REV_LIST[1], lrev)
rrev, lrev = repo_wrapper.resolve_repo_rev(
git_checkout.git_dir, "stable"
)
assert rrev == "refs/heads/stable"
assert lrev == git_checkout.rev_list[1]
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, "main")
self.assertEqual("refs/heads/main", rrev)
self.assertEqual(self.REV_LIST[0], lrev)
rrev, lrev = repo_wrapper.resolve_repo_rev(git_checkout.git_dir, "main")
assert rrev == "refs/heads/main"
assert lrev == git_checkout.rev_list[0]
def test_tag_name(self):
def test_tag_name(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check tag argument."""
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, "v1.0")
self.assertEqual("refs/tags/v1.0", rrev)
self.assertEqual(self.REV_LIST[1], lrev)
rrev, lrev = repo_wrapper.resolve_repo_rev(git_checkout.git_dir, "v1.0")
assert rrev == "refs/tags/v1.0"
assert lrev == git_checkout.rev_list[1]
def test_full_commit(self):
def test_full_commit(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check specific commit argument."""
commit = self.REV_LIST[0]
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, commit)
self.assertEqual(commit, rrev)
self.assertEqual(commit, lrev)
commit = git_checkout.rev_list[0]
rrev, lrev = repo_wrapper.resolve_repo_rev(git_checkout.git_dir, commit)
assert rrev == commit
assert lrev == commit
def test_partial_commit(self):
def test_partial_commit(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check specific (partial) commit argument."""
commit = self.REV_LIST[0][0:20]
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, commit)
self.assertEqual(self.REV_LIST[0], rrev)
self.assertEqual(self.REV_LIST[0], lrev)
commit = git_checkout.rev_list[0][0:20]
rrev, lrev = repo_wrapper.resolve_repo_rev(git_checkout.git_dir, commit)
assert rrev == git_checkout.rev_list[0]
assert lrev == git_checkout.rev_list[0]
def test_unknown(self):
def test_unknown(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Check unknown ref/commit argument."""
with self.assertRaises(self.wrapper.CloneFailure):
self.wrapper.resolve_repo_rev(self.GIT_DIR, "boooooooya")
with pytest.raises(repo_wrapper.CloneFailure):
repo_wrapper.resolve_repo_rev(git_checkout.git_dir, "boooooooya")
class CheckRepoVerify(RepoWrapperTestCase):
class TestCheckRepoVerify:
"""Check check_repo_verify behavior."""
def test_no_verify(self):
def test_no_verify(self, repo_wrapper: wrapper.Wrapper) -> None:
"""Always fail with --no-repo-verify."""
self.assertFalse(self.wrapper.check_repo_verify(False))
assert not repo_wrapper.check_repo_verify(False)
def test_gpg_initialized(self):
def test_gpg_initialized(
self,
repo_wrapper: wrapper.Wrapper,
) -> None:
"""Should pass if gpg is setup already."""
with mock.patch.object(
self.wrapper, "NeedSetupGnuPG", return_value=False
repo_wrapper, "NeedSetupGnuPG", return_value=False
):
self.assertTrue(self.wrapper.check_repo_verify(True))
assert repo_wrapper.check_repo_verify(True)
def test_need_gpg_setup(self):
def test_need_gpg_setup(
self,
repo_wrapper: wrapper.Wrapper,
) -> None:
"""Should pass/fail based on gpg setup."""
with mock.patch.object(
self.wrapper, "NeedSetupGnuPG", return_value=True
repo_wrapper, "NeedSetupGnuPG", return_value=True
):
with mock.patch.object(self.wrapper, "SetupGnuPG") as m:
with mock.patch.object(repo_wrapper, "SetupGnuPG") as m:
m.return_value = True
self.assertTrue(self.wrapper.check_repo_verify(True))
assert repo_wrapper.check_repo_verify(True)
m.return_value = False
self.assertFalse(self.wrapper.check_repo_verify(True))
assert not repo_wrapper.check_repo_verify(True)
class CheckRepoRev(GitCheckoutTestCase):
class TestCheckRepoRev:
"""Check check_repo_rev behavior."""
def test_verify_works(self):
def test_verify_works(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Should pass when verification passes."""
with mock.patch.object(
self.wrapper, "check_repo_verify", return_value=True
repo_wrapper, "check_repo_verify", return_value=True
):
with mock.patch.object(
self.wrapper, "verify_rev", return_value="12345"
repo_wrapper, "verify_rev", return_value="12345"
):
rrev, lrev = self.wrapper.check_repo_rev(self.GIT_DIR, "stable")
self.assertEqual("refs/heads/stable", rrev)
self.assertEqual("12345", lrev)
rrev, lrev = repo_wrapper.check_repo_rev(
git_checkout.git_dir, "stable"
)
assert rrev == "refs/heads/stable"
assert lrev == "12345"
def test_verify_fails(self):
def test_verify_fails(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Should fail when verification fails."""
with mock.patch.object(
self.wrapper, "check_repo_verify", return_value=True
repo_wrapper, "check_repo_verify", return_value=True
):
with mock.patch.object(
self.wrapper, "verify_rev", side_effect=Exception
repo_wrapper, "verify_rev", side_effect=RuntimeError
):
with self.assertRaises(Exception):
self.wrapper.check_repo_rev(self.GIT_DIR, "stable")
with pytest.raises(RuntimeError):
repo_wrapper.check_repo_rev(git_checkout.git_dir, "stable")
def test_verify_ignore(self):
def test_verify_ignore(
self,
repo_wrapper: wrapper.Wrapper,
git_checkout: GitCheckout,
) -> None:
"""Should pass when verification is disabled."""
with mock.patch.object(
self.wrapper, "verify_rev", side_effect=Exception
repo_wrapper, "verify_rev", side_effect=RuntimeError
):
rrev, lrev = self.wrapper.check_repo_rev(
self.GIT_DIR, "stable", repo_verify=False
rrev, lrev = repo_wrapper.check_repo_rev(
git_checkout.git_dir, "stable", repo_verify=False
)
self.assertEqual("refs/heads/stable", rrev)
self.assertEqual(self.REV_LIST[1], lrev)
assert rrev == "refs/heads/stable"
assert lrev == git_checkout.rev_list[1]
+104
View File
@@ -0,0 +1,104 @@
# Copyright (C) 2026 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Various utility code used by tests.
If you want to write a per-test fixture, see conftest.py instead.
"""
import contextlib
import functools
from pathlib import Path
import subprocess
import tempfile
from typing import Optional, Union
import git_command
THIS_FILE = Path(__file__).resolve()
THIS_DIR = THIS_FILE.parent
FIXTURES_DIR = THIS_DIR / "fixtures"
def init_git_tree(
path: Union[str, Path],
ref_format: Optional[str] = None,
) -> None:
"""Initialize `path` as a new git repo."""
with contextlib.ExitStack() as stack:
# Tests need to assume, that main is default branch at init,
# which is not supported in config until 2.28.
cmd = ["git"]
if ref_format:
cmd += ["-c", f"init.defaultRefFormat={ref_format}"]
cmd += ["init"]
if git_command.git_require((2, 28, 0)):
cmd += ["--initial-branch=main"]
else:
# Use template dir for init.
templatedir = stack.enter_context(
tempfile.mkdtemp(prefix="git-template")
)
(Path(templatedir) / "HEAD").write_text("ref: refs/heads/main\n")
cmd += ["--template", templatedir]
cmd += [path]
subprocess.run(cmd, check=True)
@contextlib.contextmanager
def TempGitTree():
"""Create a new empty git checkout for testing."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
init_git_tree(tempdir)
yield tempdir
@functools.lru_cache(maxsize=None)
def supports_reftable() -> bool:
"""Check if git supports reftable."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
proc = subprocess.run(
["git", "-c", "init.defaultRefFormat=reftable", "init"],
cwd=tempdir,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
return proc.returncode == 0
@functools.lru_cache(maxsize=None)
def supports_refs_migrate() -> bool:
"""Check if git supports refs migrate."""
with tempfile.TemporaryDirectory(prefix="repo-tests") as tempdir:
subprocess.check_call(
["git", "-c", "init.defaultRefFormat=files", "init"],
cwd=tempdir,
)
proc = subprocess.run(
[
"git",
"refs",
"migrate",
"--ref-format=reftable",
"--dry-run",
],
cwd=tempdir,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
return proc.returncode == 0
-63
View File
@@ -1,63 +0,0 @@
# Copyright 2019 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# https://tox.readthedocs.io/
[tox]
envlist = lint, py36, py37, py38, py39, py310, py311, py312
requires = virtualenv<20.22.0
[gh-actions]
python =
3.6: py36
3.7: py37
3.8: py38
3.9: py39
3.10: py310
3.11: py311
3.12: py312
[testenv]
deps =
-c constraints.txt
black
flake8
isort
pytest
pytest-timeout
commands = {envpython} run_tests {posargs}
setenv =
GIT_AUTHOR_NAME = Repo test author
GIT_COMMITTER_NAME = Repo test committer
EMAIL = repo@gerrit.nodomain
[testenv:lint]
skip_install = true
deps =
-c constraints.txt
black
flake8
commands =
black --check {posargs:. repo run_tests release/update-hooks release/update-manpages}
flake8
[testenv:format]
skip_install = true
deps =
-c constraints.txt
black
flake8
commands =
black {posargs:. repo run_tests release/update-hooks release/update-manpages}
flake8