847 Commits
Author SHA1 Message Date
Gavin Mak 0039e39000 project: carry current state from the ref snapshot
Attach the current marker computed by GetBranches to uploadable and
prunable branches. Overview and prune reporting can then reuse that
result instead of resolving HEAD once per displayed branch.

Bug: 553599402
Change-Id: Ie3e23d08d0caf66c47cc12d7295ec4b6a834560c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623184
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-08-31 13:26:22 -07:00
Gavin Mak d27034bf62 project: centralize safe branch and commit resolution
Resolve commit-ish values through one typed helper using rev-parse
--verify --quiet and, on Git 2.30+, --end-of-options. Reject option-like
revisions on older clients, reuse the helper for project and manifest
resolution, and validate user branch names with check-ref-format
--branch.

Bug: 553599402
Change-Id: I2dda49ff3e781cec14d3b84263a8dace06b4073f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623182
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-31 12:34:53 -07:00
James Hawkins e5bbb5c9e6 upload: prune ref scans for untouched projects and specific branches
Benchmark (100 untouched projects):
* GetUploadableBranches: 0.985s -> 0.0004s (2,200x faster)
* repo upload (3,045 projects): saves ~2.5s of redundant ref scans

During `repo upload`, `GetUploadableBranches()` currently scans
`self._allrefs` across all projects in the workspace. In large
manifests (e.g. Android with 3,000+ projects), `self._allrefs`
executes `git for-each-ref`, `git symbolic-ref`, and full filesystem
mtime traversals on thousands of projects where no local branch was
ever started or modified.

Any branch that can be uploaded for review must have upstream tracking
configured via `[branch "..."]` sections in `.git/config`
(otherwise `branch.LocalMerge` is None and `GetUploadableBranch`
returns None).

This patch introduces early-pruning fast paths:
1. When no branch subsections exist in `.git/config`
   (or only 2-part keys like `branch.autosetupmerge`), return []
   immediately without touching `_allrefs`.
2. When a `selected_branch` is specified (e.g. `repo upload --br=...`),
   query `branch.LocalMerge`, `refs/heads/<branch>`, and
   `refs/published/<branch>` directly, skipping the whole-tree
   ref scan.
3. In `subcmds/upload.py`, optimize `_GetMergeBranch` to resolve the
   merge branch in-memory via `project.CurrentBranch` and
   `project.GetBranch()` instead of spawning two `GitCommand`
   processes.

Test: ./run_tests tests/test_project.py tests/test_subcmds_upload.py
Change-Id: I3c2c9d63a68a393d478e1fb50f548ec8b7d44639
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623021
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: James Hawkins <jhawkins@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
2026-08-28 16:07:01 -07:00
kimhappy d7299422ae sync: skip submodules removed from their parent
The submodules to sync are derived from their parent before it is
fetched, so one that the parent's new revision no longer holds a gitlink
for is still synced. Interleaved sync then fails to check it out,
because `git submodule init` no longer knows that path:

  error: Cannot checkout a/c
  error: pathspec '.../a/c' did not match any file(s) known to git

Report such submodules while the gitlinks are read again, and leave them
out of the fetch and of the checkout, so the usual project list update
removes them from the working tree. Phased sync already left them out of
the checkout, but still fetched them.

Nothing is reported as removed while the gitlinks of a parent cannot be
read, since a missing gitlink cannot be told apart from a fetch that did
not happen.

Bug: 550074864
Change-Id: Ia429b35c12758a68e4df73665e5512a201af658a
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621681
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:54 -07:00
kimhappy 3f087a8dd9 sync: fetch submodules after their parent repository
Phased sync fetches every project at once, so a submodule is fetched at
the gitlink that was read before its parent was fetched. With -c that
stale revision is already present, the fetch is skipped as an immutable
revision, and the checkout then fails on the revision the reloaded
manifest resolved:

  error: Cannot checkout a/b
  fatal: bad object 5556dad99da5df7e53303c5732aacc8782493e03

Fetch in batches, holding a submodule back until the project holding its
gitlink is fetched, and read the gitlinks again between batches. A
manifest without submodules keeps its single batch.

Bug: 550074864
Change-Id: I04c2eb1ca81ab25b4df2c03c5dc048d2e4ecdcb8
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621663
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:49 -07:00
kimhappy 3a6e25af75 sync: check out submodules at the parent's new revision
Interleaved sync processes projects in hierarchical levels, so a
submodule is only handled after the project holding its gitlink has been
fetched and checked out. Its revision, however, is still the one read
before that parent was fetched, so the submodule is synced to the
revision of the previous sync and only catches up on the next one.

Read the gitlinks again right before a level is dispatched. The parent
is done by then, so its submodules are fetched and checked out at the
revision the parent now points at. Sibling submodules are spread over
several levels, so the gitlinks read for one level are carried over to
the next ones and every project is only read once per pass.

Bug: 550074864
Change-Id: I30395b8a16a9154f60972e1f408a066af64ba77c
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621662
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
2026-08-27 16:22:43 -07:00
Rahul Yadav e6ad708009 hooks: add --fix option to auto-apply hook fixes
Pass the --fix flag as a keyword argument "fix" to the hook main
function. This allows hooks (such as git-repohooks) to decouple
automated fix application from the -y/--yes flag so that -y can
answer yes to upload confirmation prompts without triggering file
mutations.

Companion change in git-repohooks:
https://gerrit-review.googlesource.com/c/git-repohooks/+/621761

Bug: 546510319
Test: python3 -m pytest tests/test_hooks.py

Change-Id: If0288d4791fc0a2aba6e88854e3aa81b0923b664
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/619281
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-08-27 03:28:58 -07:00
Ram Peri 09914bcab7 Add detailed upload context to Trace2 telemetry
This patch intercepts the output during a successful upload
execution to capture the generated CL URLs. It logs a dynamically
constructed "repo.uploadstate" data event to the active trace2
log containing:
- Uploaded CL URLs
- Target remote name
- Source branch
- Modified files

Test:
1. ./run_tests
2. pytest tests/test_project.py
3. Manual verification:
- Created a dummy branch `test_upload_branch` in a project with local file changes.
- Invoked repo upload passing an explicit trace output directory:
  `repo --git-trace2-event-log=/tmp/trace2out upload --dry-run --no-verify art`
- Verified the injected payload successfully appeared on disk within the generated log:
  ```json
  {"event":"data",..."key":"repo.uploadstate/cls","value":""}
  {"event":"data",..."key":"repo.uploadstate/remote","value":"goog"}
  {"event":"data",..."key":"repo.uploadstate/branch","value":"test_upload_branch"}
  {"event":"data",..."key":"repo.uploadstate/files","value":"dummy_file.txt"}
  ```
  (cls is correctly blank on --dry-run but populates on real HTTP pushes)

Bug: 543953499
Change-Id: I9c402a32d01d156d42cf24eaa2e60e22710b5e6f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/616581
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Ram Peri <ramperi@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
2026-08-26 10:19:42 -07:00
Gavin Mak 4bec297eb6 command: Respect smart sync override declaratively by default
Introduce a `RESPECT_SMART_SYNC_OVERRIDE` class attribute to the base
`Command` class, defaulting to `True`. This allows subcommands to
automatically respect the smart sync override manifest if it exists.

The override is applied in `CommonValidateOptions` before any
subcommand-specific validation or execution occurs. The `sync` and
`init` commands explicitly opt out.

This ensures all workspace-aware subcommands consistently align with the
active smart sync override manifest. It also fixes a bug in
multi-manifest setups where running a command from a submanifest would
not apply the override to the outer manifest, causing inconsistency when
resolving projects across all manifests.

Bug: 279204331
Change-Id: I9426e90a13a77ce6bd94b4a82efda4d485cbe116
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/585081
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-08-04 11:38:03 -07:00
Andrew Chant 0b82311632 sync: allow syncing groups with repo sync -g group
Similar to how repo init -g can restrict repo syncs to
a subset of the manifest globally, allow "repo sync -g" to only sync a
subset of projects from the manifest when running that specific
sync command.

Change-Id: I4929aad109de05c73a7db42bb27fd8d47eea32fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/609481
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Andrew Chant <achant@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Andrew Chant <achant@google.com>
2026-07-21 13:42:38 -07:00
Josef Malmström dd1130352b sync: Deprecate fetch-submodules flag names
The names for flags --fetch-submodules / --no-fetch-submodules are
misleading, since they impact the full sync operation (fetch and
checkout), not just the fetching.

Introduce new flags --recurse-submodules / --no-recurse-submodules
and treat the old ones as deprecated aliases.

Change-Id: I78339a3e0496a855c222c1869b27b578507886a7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/608881
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
2026-07-21 00:19:21 -07:00
Josef Malmström 978adb7ea5 sync: Add CLI flag for globally disabling submodule fetch
A global setting for disabling fetching of submodules is useful
since this can currently otherwise only be done by modifying
the manifest, or by explicitly providing projects on command line.

Add this setting as --no-fetch-submodules to mirror the existing
--fetch-submodules.

Change-Id: Ic727c54f11a594aa52315751284b87138cf246bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/607641
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-07-16 01:19:51 -07:00
Gavin Mak 3bb4871c44 rebase: Resolve revisionExpr to tracking branch for --onto-manifest
When running `repo rebase -m` (`--onto-manifest`), the command uses the
raw `revisionExpr` from the manifest (e.g. `main`) directly as the
`--onto` target. This can fail or behave incorrectly if it should
reference the local tracking branch (e.g. `refs/remotes/<remote>/main`).

Resolve `project.revisionExpr` to its local tracking branch using
`project.GetRemote().ToLocal()`. Fall back to using the raw
`revisionExpr` value if the resolution fails (raising a `GitError`).

Bug: 532028666
Change-Id: I4c1bca1374a5842688be227f6aa2afffcdad5397
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/604941
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-07-07 13:40:52 -07:00
Brian Gan e7cac4bca6 status: Show ahead/behind info for local branches
When viewing `repo status`, it is difficult to distinguish between branches that have active unpushed changes and stale branches that are fully synced. Previously, developers had to run commands like `repo forall -c "git status"` to see their ahead/behind counts.

This change updates Project.PrintWorkTreeStatus to automatically calculate and display the number of commits a branch is ahead and/or behind its upstream tracking branch. We use `git rev-list --left-right --count` to fetch this information natively and efficiently.

If the branch is completely synced with upstream, no extra text is shown.

Added tests for ahead-only, behind-only, diverged, no-tracking, and fully-synced branch states.

Bug: 319412954
Change-Id: I23879b2d472c7a7e11d01b565428a84b1b4f09c1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602423
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
2026-06-30 11:13:14 -07:00
Gavin Mak 91986011b0 info: Parallelize project data gathering for JSON output
https://gerrit-review.googlesource.com/c/git-repo/+/581921 parallelized
`repo info` for text output. This commit does the same for JSON output
format.

Benchmarked `repo info --format=json` on an Android workspace with ~3k
projects (N=3):
- Before (sequential): 1m 30s average
- After (parallelized): 46s average (~2x speedup)

Verified that the JSON output is identical before and after.

Bug: 526685287
Change-Id: If573223aba584f8b932f87d29e34ed565c5c930a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601861
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-30 09:19:35 -07:00
Gavin Mak 88a7e88e54 info: Report actual checked-out HEAD revision
When `repo info` runs, the reported "Current revision" is resolved using
the manifest's target branch tracking ref (e.g. refs/remotes/goog/main).

Introduce Project.GetHeadRevisionId(), which gets the checked-out HEAD
commit in the worktree, and use it in `repo info` with a fallback to the
old behavior if the project is not checked out.

Bug: 526685287
Change-Id: I72280ce27daa210cada27d722a94e365644f06e0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/599481
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-06-29 10:50:02 -07:00
Rahul Yadav 39c0b60900 sync: Support pluggable remote helpers for smart sync manifest server.
Introduce support for pluggable remote helpers (declared via the
optional 'helper' attribute in <manifest-server>) to dynamically resolve
proxy addresses. Route the XML-RPC manifest server connection through
the resolved proxy.

Bug: b/517477903
Change-Id: I3b6b8ea2640bb077521df4b4a9e8a34a8c6ecdad
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591642
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-18 09:48:49 -07:00
Gavin Mak cd307a6089 project: Add REPO_PROJECT_FETCH_URL environment variable
Add REPO_PROJECT_FETCH_URL to Project.GetEnvVars(), which resolves to
the remote fetch URL of the project. This is useful for exposing the
URL to custom fetch commands or other external scripts.

Bug: 513329573
Change-Id: Ic2b0a83493934d16bb1152366ee4e1a2c35ea2dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/596121
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-06-16 10:30:47 -07:00
Josef Malmström 4b462634e0 sync: do not init sibling submodules in parallel
Initializing a submodule requires locking <parent>/.git/config.
As the implementation was currently doing this in parallel for
sibling submodules, it lead to a race condition causing
intermittent errors like:

error: could not lock config file .git/config: File exists

This commit enforces that sibling submodules are initialized
sequentially, eliminating the race condition.

Change-Id: I5ffb3de90276ba43e262d0e279a3d34324220b63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591241
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-06-16 00:21:27 -07:00
Gavin Mak e0bd39c691 project: Extract project envvar generation to GetEnvVars
Move project environment variable setup from subcmds/forall.py to a
reusable Project.GetEnvVars() helper method.

Bug: 513329573
Change-Id: I3b4b113aa5a086e5fa5eaf4461c7ce517d928610
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583881
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-27 12:58:02 -07:00
Gavin Mak b8531133de init: Add --use-local-gitdirs for standard Git layouts
Introduce --use-local-gitdirs to bypass repo's symlink-based layouts in
favor of standard local .git directories.

Bug: 513329573
Bug: 508146070
Change-Id: I53d1602e61be0b86964529bcbea3dc801471f9c9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569001
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Dan Willemsen <dwillemsen@google.com>
2026-05-26 10:26:25 -07:00
Gavin Mak 2d54384a5e sync: Add --superproject-rev flag to sync to specific revision
Allow syncing the outer manifest to a state defined by a specific
superproject revision. It updates the superproject, reads the manifest
commit from .supermanifest, and checks out the outer manifest project
to that commit.

Submanifests are then processed normally, allowing them to be updated
to the revisions specified in the new outer manifest state.

Bug: 416589884
Change-Id: I304c37a2b8794f9b74cb7e5e209a8a93762bdb52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/576321
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-20 12:28:55 -07:00
Gavin Mak 51021fb209 abandon/start/info: Make them respect smart sync override
Call TryOverrideManifestWithSmartSync in start, abandon, and info
commands.

This ensures they pick up the pinned revisions from the smart sync
override manifest if it exists, rather than falling back to ToT from
the default manifest.

Bug: 279204331
Change-Id: I637f054a77773805daf0bf9cf5a712d82a5959b4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582503
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-14 13:19:55 -07:00
Gavin Mak d453273f06 command: Move smart sync override logic to Command base class
Deduplicate the logic for loading `smart_sync_override.xml` by
moving it from `forall.py` to the `Command` base class. This allows
other commands to reuse it to respect smart tags.

Bug: 279204331
Change-Id: I6f2f03995266c2a68c3225cacb92b2f580a89178
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582502
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-14 13:19:26 -07:00
Carlos Fernandez 5534f164d6 linkfile: Handle directory-to-symlink transitions safely
When a manifest changes from individual linkfiles inside a directory
(e.g. dest=".llms/rules", dest=".llms/skills") to a single linkfile
for the whole directory (e.g. dest=".llms", src="dot-llms"), two
things need to happen:

1. __linkIt must replace a real directory with a symlink.  Use
   os.rmdir() instead of platform_utils.remove() for real directories.
   rmdir only removes empty directories, so user-created content is
   never deleted.

2. UpdateCopyLinkfileList must handle the cleanup correctly:
   - Use os.rmdir() for directories (safe for non-empty)
   - Remove empty parent directories after cleaning old dests
   - Retry _CopyAndLinkFiles for all projects, since in interleaved
     sync mode _CopyAndLinkFiles runs before cleanup and may have
     failed because the directory was not yet empty

Change-Id: I0437b80beab98bce064cea81c11c47d699be91aa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569243
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
2026-05-12 11:10:43 -07:00
Gavin Mak d5037230e9 sync: Re-raise KeyboardInterrupt in main process
When running sync -j1, worker functions run directly in the main
process. Swallowing KeyboardInterrupt causes the loop to continue to the
next project instead of aborting.

Re-raise KeyboardInterrupt if running in the MainProcess, while
maintaining the suppression of stack traces in worker processes.

Bug: 468170157
Change-Id: I156d66bc209a265f7fa25eea0eb88737d1b51a34
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581342
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-05-12 10:41:47 -07:00
Greg Edelston 12ad396f67 info: Parallelize repo info to improve performance
For a large checkout like chromiumos or android, `repo info` takes a
really long time! On my machine it took ~6 minutes. On a randomly
selected ChromiumOS cq-orchestrator build it took 4.1 minutes:
https://ci.chromium.org/b/8682060180498819729. This adds up to a lot of
wasted runtime for both humans and bots.

The problem is that `repo info` was single-threaded, which causes poor
performance when the checkout has 1000+ projects. We already have a
pattern for parallelization; let's use it.

BUG=None
TEST=Manually run, ensure no diff

Change-Id: I6b82b9495eb2a0e602a142dd3a16f09217871e1b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581921
Tested-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-12 09:59:07 -07:00
Gavin Mak a6bc1b7cf0 sync: Exclude stateless sync pruned projects from bloat check
If a project has been pruned for stateless sync, it has already
undergone aggressive garbage collection. There should be no bloat
to check for.

Change-Id: I9233a4611e05b7a0b7c097827d6f408144f7380d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581841
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-05-11 19:36:59 -07:00
Gavin Mak 5d8585012f sync: Suggest "git repack -a -d" in bloat warning
The warning suggests running "repo sync --auto-gc" which runs "git gc
--auto". Git may decide not to repack if its internal thresholds are not
met, leaving the warning active even after running the suggested
command.

"git repack -a -d" forces all reachable objects into a single pack and
deletes redundant packs, reducing the pack count to 1. This guarantees
that the warning (which triggers when pack count exceeds 10) goes away.

Bug: 505755299
Change-Id: I10163ef8efb7f3b7c5055378ad95051974d11b88
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580821
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-05-08 14:12:22 -07:00
Gavin Mak 11428ae984 forall: Document REPO_UPSTREAM and REPO_DEST_BRANCH envvars
Change-Id: I74365295152f8828587c6b4ed93029efc6000881
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/580761
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2026-05-07 20:27:08 -07:00
Carlos Fernandez 27d2232eb3 info: add --format and --include-summary/--include-projects options
Add --format={text,json} to produce machine-readable output, and
boolean options to control which sections are displayed:
  --include-summary / --no-include-summary (default: on)
  --include-projects / --no-include-projects (default: on)

The JSON output respects the include flags, so callers can request
only the fields they need (e.g. `repo info --format=json
--no-include-projects` for manifest metadata only).

Change-Id: I9641bc4023b630d9c61c5170eb86e5f3b787236f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569203
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
2026-05-07 19:23:24 -07:00
Ram Peri 5af71ce907 Add timing keyword argument for hooks
Measure the duration of the sync operation in the Execute method of the
Sync command and pass it to post-sync hooks as a standard keyword
argument (`sync_duration_seconds`).

Updates based on code review:
- Update _API_ARGS in hooks.py to allow sync_duration_seconds for post-sync hooks.
- Do not cast sync_duration_seconds to int for better granularity.
- Update docs/repo-hooks.md to document sync_duration_seconds.
- Add unit test for argument validation in test_hooks.py.

Test: Ran run_tests using venv python, all 554 tests passed.
Bug: TBD
Change-Id: Ie29e002a5d283460d993ad96c224dbf4b6d7985c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/575021
Tested-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-21 17:10:07 -07:00
Gavin Mak baa281d99e sync: Refactor to use _RunOneGC and fix config leakage
Extract _RunOneGC to handle GC on a single project. This refactoring
makes it easier to invoke GC from parallel worker tasks.

Also, avoid modifying the passed-in config dictionary in _RunOneGC by
creating a local copy, preventing unintended side effects on other
commands sharing the same config.

Bug: 498290329
Change-Id: I7b77ed6629b14b5ee3322870b9c6c8ce2bfd6ea2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574923
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-20 15:34:07 -07:00
Gavin Mak 7e9079b7cf sync: Switch to using self._bloated_projects
Store bloated projects in self._bloated_projects and print warnings at
the end of execution. This sets up for moving the check to workers.

Bug: 498290329
Change-Id: I993f1fd741db2994d480994861588eb18f6c5503
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574922
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-04-20 14:07:14 -07:00
Marty Heavey 32e7327ca6 upload: Clarify partial sync message on hook failure
Demote the partial sync warning from error to info and rephrase it to
be a tip rather than an error. This prevents users from thinking that
a partial sync is the cause of their hook failures when it is often a linting failure.

The message now suggests that a full sync might help if there are
cross-project dependencies, instead of implying it will fix any issue.

Change-Id: I5d8c52b53ac315aa9f145ed069798bf201fa0815
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574262
Tested-by: Marty Heavey <mheavey@google.com>
Commit-Queue: Marty Heavey <mheavey@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-20 01:52:25 -07:00
Gavin Mak 4f707ff91e sync: Provide feedback during post-sync operations
After the main sync progress bar finishes, there's a pause while some
post-sync operations run. Print something to provide feedback so the
user doesn't think repo has hung.

Bug: 503869525
Change-Id: I695fd560e60dcb394e6844a56c8a336ca1f71c74
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574425
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2026-04-17 15:47:15 -07:00
Gavin Mak 951666fb23 gc: Fix hang during repack in partial clones
Add `--missing=allow-promisor` to `git rev-list` calls in
`repack_projects`. This prevents Git from auto-fetching missing objects
from the promisor remote, which can cause stalls due to sequential
network requests.

Also add a Git version check to ensure Git is at least 2.17.0 before
running `--repack`, as `--missing=allow-promisor` was introduced in that
version.

Bug: 500133631
Change-Id: I2dcf9b46fac4c6a53a3c2a46f06f61d6aec40f2f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/570361
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
2026-04-08 12:54:39 -07:00
Carlos Fernandez 573983948a Fix all flake8 warnings from newer flake8-bugbear and flake8-comprehensions
Address warnings introduced by flake8-bugbear 24.12.12 and
flake8-comprehensions 3.16.0:

- C408: Replace dict()/list() calls with literal {} and []
- C413: Remove unnecessary list() around sorted()
- C414: Remove unnecessary list() inside sorted()
- C419: Suppress intentional list comprehension in all() (noqa)
- B001: Replace bare except with except Exception
- B006: Replace mutable default arguments with None
- B010: Replace setattr() with direct attribute assignment
- B017: Use RuntimeError instead of Exception in tests
- B019: Suppress lru_cache on methods for long-lived objects (noqa)
- B033: Remove duplicate item in set literal

Change-Id: If4693d3e946200bbc22f689f7b94da604addcb80
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/566321
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2026-04-03 07:50:52 -07:00
Jacky Liu 0176586544 Use git_superproject.UseSuperproject() everywhere
Currently somewhere use git_superproject.UseSuperproject(), which checks
both the manifest config and user's config, and otherwhere use
manifest.manifestProject.use_superproject, which only checks the
manifest config. This causes Inconsistent behaviors for users who do not
set --use-superproject when doing repo init but have
repo.superprojectChoice in their git config.

Replace where using manifest.manifestProject.use_superproject with
git_superproject.UseSuperproject() to respect user's config and avoid
inconsistency.

Bug: 454514213
Change-Id: I1f734235cdd67b8a6915f1d05967d1aaa4d03f2a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/561801
Commit-Queue: Jacky Liu <qsliu@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jacky Liu <qsliu@google.com>
2026-03-18 21:03:07 -07:00
Gavin Mak 5cb0251248 gc: fix untargeted projects being deleted
`delete_unused_projects` needs a full list of active projects to figure
out which orphaned .git dirs need to be deleted. Otherwise it thinks
that only the projects specified in args are active.

Bug: 447626164
Change-Id: I02beebf6a01c77742a8db78221452d71cd78ea73
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550061
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-02-09 08:34:42 -08:00
Gavin Mak a214fd31bd manifest: Introduce sync-j-max attribute to cap sync jobs
Add a way for manifest owners to limit how many sync jobs run in
parallel.

Bug: 481100878
Change-Id: Ia6cbe02cbc83c9e414b53b8d14fe5e7e1b802505
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/548963
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2026-02-06 13:32:28 -08:00
Gavin Mak 871e4c7ed1 sync: skip bloat check if fresh sync
Initial syncs won't have accumulated any garbage.

Bug: 379111283
Change-Id: I04b2ecde3e33f1f055038861a2705ab6aabb36d1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/536083
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-15 15:24:45 -08:00
Gavin Mak 5b0b5513d6 project: only use --no-auto-gc for git 2.23.0+
The flag for git fetch was introduced in git 2.23.0. Also skip the bloat
check after sync if using an older version.

Bug: 468589976
Bug: 379111283
Change-Id: Ib53e5494350c71a83906e5219d3a8c2b654e531f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/536082
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-15 11:32:49 -08:00
Gavin Mak b5991d7128 sync: Add heuristic warning for bloated shallow repositories
For clone-depth="1" repositories that are dirty or have local commits,
add a check at the end of sync to detect excessive git object
accumulation.

This prevents silent performance degradation and disk exhaustion in
large prebuilts repos where automatic GC is typically disabled from
https://gerrit.googlesource.com/git-repo/+/7f87c54043ce9a35a5bb60a09ee846f9d7070352

Bug: 379111283
Change-Id: I376f38e1555cc6e906d852f6e63dc1c8f6331b4f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/534701
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-10 11:34:40 -08:00
Peter Kjellerstedt 31b4b19387 info: Print a newline after printing the superproject's revision
Change-Id: Ib20233dad4e1f1fd54dbf5ca0324be22fe0e4db1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/528463
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-03 08:08:06 -08:00
Peter Kjellerstedt 2b6de52a36 Rename XmlManifest.GetGroupsStr() to XmlManifest.GetManifestGroupsStr()
This makes it more clear what kind of groups it refers to.

Change-Id: I47369050d1436efcc77f3a69d5b7c99a536b23bc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/528462
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-03 07:57:22 -08:00
Gavin Mak be33106ffc wipe: Add new repo wipe subcommand
This new command allows users to delete projects from the worktree
and from the `.repo` directory. It is a destructive operation.

It handles shared projects by refusing to wipe them unless the
`--force` flag is used. It also checks for uncommitted changes
before wiping.

Bug: 393383056
Change-Id: Ia30d8ffdc781a3f179af56310ce31c9dae331bbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/490801
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2025-11-21 10:48:42 -08:00
Gavin Mak 1afe96a7e9 sync: fix saving of fetch times and local state
Interleaved sync didn't save _fetch_times and _local_sync_state to disk.
Phased sync saved them, but incorrectly applied moving average smoothing
repeatedly when fetching submodules, and discarded historical data
during partial syncs.

Move .Save() calls to the end of main sync loops to ensure they run
once. Update _FetchTimes.Save() to merge new data with existing history,
preventing data loss.

Change-Id: I174f98a62ac86859f1eeea1daba65eb35c227852
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/519821
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-10-20 11:28:21 -07:00
Mike Frysinger d30414bb53 forall: fix crash with no command
When callback= is used, optparse does not automatically initialize
The destination when a dest= is not specified.  Refine the test to
allow dest= options when callback= is used even when it seems like
it is otherwise redundant.

Bug: b/436611422
Change-Id: I5185f95cb857ca6d37357cac77fb117a83db9c0c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/509861
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-09-17 12:54:30 -07:00
Gavin Mak d534a5537f sync: Fix missing error details in interleaved summary
When checkout errors occurred in interleaved sync, they were wrapped in
a SyncError with no message, causing blank lines in the final summary.
Refactor _SyncResult to hold a list of exceptions, ensuring the original
error messages are propagated correctly.

Bug: 438178765
Change-Id: Ic25e515068959829cb6290cfd9e4c2d3963bbbea
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498342
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2025-08-14 09:54:15 -07:00