When running sync -j1, worker functions run directly in the main
process. Swallowing KeyboardInterrupt causes the loop to continue to the
next project instead of aborting.
Re-raise KeyboardInterrupt if running in the MainProcess, while
maintaining the suppression of stack traces in worker processes.
Bug: 468170157
Change-Id: I156d66bc209a265f7fa25eea0eb88737d1b51a34
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581342
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
str() inside an f-string is redundant. Actually join the array together
as a string to show a proper error message, and include the full command
line, not the extra options passed in.
Before:
error.GitError: git config ('--null', '--list'): fatal: unable to read config file 'xxx': No such file or directory
After:
error.GitError: git config --system --includes --null --list: fatal: unable to read config file 'xxx': No such file or directory
Change-Id: I8983389aa2e0de7808991e73e636b77810f04c4b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582341
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
For a large checkout like chromiumos or android, `repo info` takes a
really long time! On my machine it took ~6 minutes. On a randomly
selected ChromiumOS cq-orchestrator build it took 4.1 minutes:
https://ci.chromium.org/b/8682060180498819729. This adds up to a lot of
wasted runtime for both humans and bots.
The problem is that `repo info` was single-threaded, which causes poor
performance when the checkout has 1000+ projects. We already have a
pattern for parallelization; let's use it.
BUG=None
TEST=Manually run, ensure no diff
Change-Id: I6b82b9495eb2a0e602a142dd3a16f09217871e1b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581921
Tested-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
The warning suggests running "repo sync --auto-gc" which runs "git gc
--auto". Git may decide not to repack if its internal thresholds are not
met, leaving the warning active even after running the suggested
command.
"git repack -a -d" forces all reachable objects into a single pack and
deletes redundant packs, reducing the pack count to 1. This guarantees
that the warning (which triggers when pack count exceeds 10) goes away.
Bug: 505755299
Change-Id: I10163ef8efb7f3b7c5055378ad95051974d11b88
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580821
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
This PR adds `completion.zsh` to allow for relatively intelligent
completion of `repo` command-line arguments for all commands of the
`repo` CLI.
The `completion.zsh` can be added to your zsh completions by copying
(and renaming) the completion script to a directory that you have added
to your zsh completion path.
```
cp completion.zsh $ZSH_COMPLETION_DIRECTORY/_repo
```
You must name the file `_repo` for it to register as completion for
`repo`.
You can add a directory to your zsh completion by updating your `fpath`
variable before calling `compinit` as follows:
```
fpath=( /path/to/zsh-completions/dir $fpath )
```
Future work: Generate this file using the subcommand classes and python
reflection.
Change-Id: I6a4e785c1efcf9076bd693976ac03578836b691b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579561
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Kamal Sacranie <sacranie@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Add --format={text,json} to produce machine-readable output, and
boolean options to control which sections are displayed:
--include-summary / --no-include-summary (default: on)
--include-projects / --no-include-projects (default: on)
The JSON output respects the include flags, so callers can request
only the fields they need (e.g. `repo info --format=json
--no-include-projects` for manifest metadata only).
Change-Id: I9641bc4023b630d9c61c5170eb86e5f3b787236f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569203
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
The force_checkout parameter was not propagated in all calls to
Checkout in Sync_LocalHalf.
Without this, repo sync --force-checkout can still fail for projects
currently on a local branch with no upstream/tracking configuration,
because the detach-to-manifest checkout was executed without -f,
leaving local modifications or untracked files able to block sync.
Change-Id: I58551388e2f906c4db96e220707a369057a71c24
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579181
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Build out status subcommand unit coverage using a minimal fake repo
checkout wired through XmlManifest.
The new tests verify:
- clean status output prints the expected project header
- modified tracked files appear with the expected status marker
- `-o` output includes the orphan section and orphan entries
- branch names shown in status reflect a started non-default branch
Change-Id: Ia7c22593d0bbdc4aed81faeb168b846f3e4016ab
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558501
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Measure the duration of the sync operation in the Execute method of the
Sync command and pass it to post-sync hooks as a standard keyword
argument (`sync_duration_seconds`).
Updates based on code review:
- Update _API_ARGS in hooks.py to allow sync_duration_seconds for post-sync hooks.
- Do not cast sync_duration_seconds to int for better granularity.
- Update docs/repo-hooks.md to document sync_duration_seconds.
- Add unit test for argument validation in test_hooks.py.
Test: Ran run_tests using venv python, all 554 tests passed.
Bug: TBD
Change-Id: Ie29e002a5d283460d993ad96c224dbf4b6d7985c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/575021
Tested-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Extract _RunOneGC to handle GC on a single project. This refactoring
makes it easier to invoke GC from parallel worker tasks.
Also, avoid modifying the passed-in config dictionary in _RunOneGC by
creating a local copy, preventing unintended side effects on other
commands sharing the same config.
Bug: 498290329
Change-Id: I7b77ed6629b14b5ee3322870b9c6c8ce2bfd6ea2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574923
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Demote the partial sync warning from error to info and rephrase it to
be a tip rather than an error. This prevents users from thinking that
a partial sync is the cause of their hook failures when it is often a linting failure.
The message now suggests that a full sync might help if there are
cross-project dependencies, instead of implying it will fix any issue.
Change-Id: I5d8c52b53ac315aa9f145ed069798bf201fa0815
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574262
Tested-by: Marty Heavey <mheavey@google.com>
Commit-Queue: Marty Heavey <mheavey@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
When optimizing fetches for projects with immutable revisions, the fetch
should not be skipped if the project is configured for a shallow clone
(depth > 0) but the .git/shallow file is missing. The absence of the
.git/shallow file means the repository is not a shallow clone, or the
shallow clone is incomplete, so a fetch is necessary to ensure the
revision is present.
Bug: 503081454
Change-Id: Ic3549612bcd69050a926652ee4e522c79ad8124c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/573821
Tested-by: Becky Siegel <beckysiegel@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Becky Siegel <beckysiegel@google.com>
repo calls `git update-ref --stdin` when updating multiple refs during
repo init and repo sync. Historically, `--no-deref` was also passed.
Older Git 2.17 which we still support rejects the combination of
`--stdin` and `--no-deref`, emitting a usage error even when the stdin
input is valid.
The `--no-deref` option is only meaningful when updating symbolic refs
such as HEAD. The stdin-based update-ref path only operates on explicit
refs (tags, remote refs, alternates) and never symbolic refs.
Remove the unnecessary option to restore compatibility with Git 2.17
while preserving identical behavior on newer Git versions.
Tested with:
- Git 2.17.1
- Git 2.34.1
Change-Id: I22001de03800f5699b26a40bc1fb1fec002ed048
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/571721
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Enei <miyako.enei@alpsalpine.com>
Tested-by: Enei <miyako.enei@alpsalpine.com>
Implement in-situ shallow re-fetching and garbage collection logic.
Enables repositories with sync-strategy="stateless" to reclaim disk
space by running reflog expire and git gc --prune=now if the working
tree is clean and has no local commits.
Bug: 498730431
Change-Id: I940bdc9b74da29d3f7b13566667dcddea769ebd3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/568463
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
The only supported sync-strategy is "stateless". The intent is to keep
the local workspace as small as possible by not keeping history during
syncs. This prevents disk space waste for projects with large binaries
where we only care about the current version.
A follow up change will implement the logic.
Bug: 498730431
Change-Id: I84a436a9ca2492893163c6cfda6c28dc62a568f0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/568462
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Rewrite tests/test_subcmds_forall.py from unittest.TestCase to pytest
function-style tests to match the surrounding test suite conventions.
Replace setUp/tearDown and class-based helpers with tmp_path-based
setup, switch stdout capture to contextlib.redirect_stdout, and keep the
existing behavior checks intact (all eight projects are invoked exactly
once).
Change-Id: I9243f3461aa6850f867bdb864f4a34c442f817f6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569821
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Add `--missing=allow-promisor` to `git rev-list` calls in
`repack_projects`. This prevents Git from auto-fetching missing objects
from the promisor remote, which can cause stalls due to sequential
network requests.
Also add a Git version check to ensure Git is at least 2.17.0 before
running `--repack`, as `--missing=allow-promisor` was introduced in that
version.
Bug: 500133631
Change-Id: I2dcf9b46fac4c6a53a3c2a46f06f61d6aec40f2f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/570361
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Similar to `git`, when a user types an unknown command like `repo tart`,
we now use `difflib.get_close_matches` to suggest similar commands.
If `help.autocorrect` is set in the git config, it will optionally
prompt the user to automatically run the assumed command, or wait
for a configured delay before executing it.
Verification Steps:
1. Created a dummy repo project locally.
2. Verified `help.autocorrect=0|false|off|no|show` suggests
command and exits.
3. Verified `help.autocorrect=1|true|on|yes|immediate`
automatically runs suggestion.
4. Verified `help.autocorrect=<number>` runs after
`<number>*0.1` seconds.
5. Verified `help.autocorrect=never` exits immediately without
suggestions.
6. Verified `help.autocorrect=prompt` asks user to accept [y/n]
and handles correctly.
BUG: b/489753302
Change-Id: I6dcd63229cbd7badf5404459b48690c68f5b4857
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558021
Tested-by: Sam Saccone <samccone@google.com>
Commit-Queue: Sam Saccone <samccone@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
If a user has manually unshallowed a repo (e.g. via
`git fetch --unshallow`), the absence of the `shallow` file in the
gitdir indicates a full clone. Re-applying depth during a subsequent
sync would undo the user's intent. Skip re-shallowing in this case
by clearing depth when the project is not new and no shallow file
is present.
Change-Id: I4ee0e78018de9078fe1bd77a9615613ef0c40d33
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558743
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Lucas Alves <ltanure@gmail.com>
Tested-by: Lucas Alves <ltanure@gmail.com>
Reviewed-by: Lucas Alves <ltanure@gmail.com>
Currently somewhere use git_superproject.UseSuperproject(), which checks
both the manifest config and user's config, and otherwhere use
manifest.manifestProject.use_superproject, which only checks the
manifest config. This causes Inconsistent behaviors for users who do not
set --use-superproject when doing repo init but have
repo.superprojectChoice in their git config.
Replace where using manifest.manifestProject.use_superproject with
git_superproject.UseSuperproject() to respect user's config and avoid
inconsistency.
Bug: 454514213
Change-Id: I1f734235cdd67b8a6915f1d05967d1aaa4d03f2a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/561801
Commit-Queue: Jacky Liu <qsliu@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jacky Liu <qsliu@google.com>
Running sync with reftable on a files-backed workspace fails to re-init
the superproject dir with:
```
fatal: could not open
'.../.repo/exp-superproject/<hash>-superproject.git/refs/heads' for writing:
Is a directory
```
Bug: 476209856
Change-Id: Ie8473d66069aafefa5661bd3ea8e73b2b27c6a38
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550981
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
When running on a machine without the `ssh` command, repo sync would fail even if no ssh or ssh proxy was required. Use exception handling inside ssh.ProxyManager to more gracefully handle the case where ssh is not installed.
Bug: 467714011
Change-Id: I602a0819638ead4d02de88b750839bc3d70549ce
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/535141
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Jordan Esh <esh.jordan@gmail.com>
Commit-Queue: Jordan Esh <esh.jordan@gmail.com>
`delete_unused_projects` needs a full list of active projects to figure
out which orphaned .git dirs need to be deleted. Otherwise it thinks
that only the projects specified in args are active.
Bug: 447626164
Change-Id: I02beebf6a01c77742a8db78221452d71cd78ea73
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550061
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Gavin Mak
Mike Frysinger
Greg Edelston
Kamal Sacranie
Carlos Fernandez
Josef Malmström
Nasser Grainawi
Ram Peri
Marty Heavey
Becky Siegel
Miyako.Enei
Sam Saccone
Lucas Tanure
Jacky Liu
Jeffery Miller
Jordan Esh