Benchmark (100 untouched projects):
* GetUploadableBranches: 0.985s -> 0.0004s (2,200x faster)
* repo upload (3,045 projects): saves ~2.5s of redundant ref scans
During `repo upload`, `GetUploadableBranches()` currently scans
`self._allrefs` across all projects in the workspace. In large
manifests (e.g. Android with 3,000+ projects), `self._allrefs`
executes `git for-each-ref`, `git symbolic-ref`, and full filesystem
mtime traversals on thousands of projects where no local branch was
ever started or modified.
Any branch that can be uploaded for review must have upstream tracking
configured via `[branch "..."]` sections in `.git/config`
(otherwise `branch.LocalMerge` is None and `GetUploadableBranch`
returns None).
This patch introduces early-pruning fast paths:
1. When no branch subsections exist in `.git/config`
(or only 2-part keys like `branch.autosetupmerge`), return []
immediately without touching `_allrefs`.
2. When a `selected_branch` is specified (e.g. `repo upload --br=...`),
query `branch.LocalMerge`, `refs/heads/<branch>`, and
`refs/published/<branch>` directly, skipping the whole-tree
ref scan.
3. In `subcmds/upload.py`, optimize `_GetMergeBranch` to resolve the
merge branch in-memory via `project.CurrentBranch` and
`project.GetBranch()` instead of spawning two `GitCommand`
processes.
Test: ./run_tests tests/test_project.py tests/test_subcmds_upload.py
Change-Id: I3c2c9d63a68a393d478e1fb50f548ec8b7d44639
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623021
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: James Hawkins <jhawkins@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
Benchmark:
* repo upload frameworks/base: 2.84s -> 0.50s (-82.3%, 5.6x faster)
* repo upload (3,045 projects): 7.66s -> 5.24s (-31.6%, 2.42s saved)
From repo's inception through v2.56, GetHead() directly read the
`.git/HEAD` file in Python. In commit 52bab0ba ("project: Use git
rev-parse to read HEAD"), this was replaced with git subprocess calls
on the premise that git provides a dedicated command. However, in
large multi-project workspaces (such as Android with 3,000+ projects),
spawning thousands of git processes introduced severe latency
regressions during `repo upload` and `repo status`.
Furthermore, switching to subprocesses broke detached HEADs and
unborn branches (fixed in commits 7f7d70ef and 8c3585f3 by re-adding
the v2.56 file-reading logic as an error recovery fallback).
This patch restores fast in-memory reading as the primary path, while
adding modern defensive safeguards:
* Symbolic refs (`ref: refs/heads/...`): Strips whitespace and tabs
and returns the ref directly in memory.
* Detached HEAD: Validates 40-char SHA-1 and 64-char SHA-256 commit
hashes via git_config.IsId(), normalizing to lowercase.
* Symlinks: Detects filesystem symlinks via os.path.islink() and
safely falls back to git symbolic-ref.
* Fallback: Catches (OSError, AssertionError) and transparently falls
back to native git commands for reftables, unexpected layouts, or
filesystem errors. Unifies recovery fallback parsing with the fast
path (CRLF/tabs, lowercase hashes, and consistent RelPath errors).
In addition, this change substantially expands test coverage in
tests/test_project.py, adding comprehensive unit tests for symbolic
refs, whitespace/tabs, CRLF line endings, SHA-1, SHA-256, uppercase
hash normalization, symlinks, corrupted worktrees, and fallback
robustness.
Test: ./run_tests tests/test_project.py
Change-Id: Ib5c2530117c6939e4b9293feda81aa745c003c6a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/623001
Commit-Queue: James Hawkins <jhawkins@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: James Hawkins <jhawkins@google.com>
The submodules to sync are derived from their parent before it is
fetched, so one that the parent's new revision no longer holds a gitlink
for is still synced. Interleaved sync then fails to check it out,
because `git submodule init` no longer knows that path:
error: Cannot checkout a/c
error: pathspec '.../a/c' did not match any file(s) known to git
Report such submodules while the gitlinks are read again, and leave them
out of the fetch and of the checkout, so the usual project list update
removes them from the working tree. Phased sync already left them out of
the checkout, but still fetched them.
Nothing is reported as removed while the gitlinks of a parent cannot be
read, since a missing gitlink cannot be told apart from a fetch that did
not happen.
Bug: 550074864
Change-Id: Ia429b35c12758a68e4df73665e5512a201af658a
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621681
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Phased sync fetches every project at once, so a submodule is fetched at
the gitlink that was read before its parent was fetched. With -c that
stale revision is already present, the fetch is skipped as an immutable
revision, and the checkout then fails on the revision the reloaded
manifest resolved:
error: Cannot checkout a/b
fatal: bad object 5556dad99da5df7e53303c5732aacc8782493e03
Fetch in batches, holding a submodule back until the project holding its
gitlink is fetched, and read the gitlinks again between batches. A
manifest without submodules keeps its single batch.
Bug: 550074864
Change-Id: I04c2eb1ca81ab25b4df2c03c5dc048d2e4ecdcb8
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621663
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
Interleaved sync processes projects in hierarchical levels, so a
submodule is only handled after the project holding its gitlink has been
fetched and checked out. Its revision, however, is still the one read
before that parent was fetched, so the submodule is synced to the
revision of the previous sync and only catches up on the next one.
Read the gitlinks again right before a level is dispatched. The parent
is done by then, so its submodules are fetched and checked out at the
revision the parent now points at. Sibling submodules are spread over
several levels, so the gitlinks read for one level are carried over to
the next ones and every project is only read once per pass.
Bug: 550074864
Change-Id: I30395b8a16a9154f60972e1f408a066af64ba77c
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621662
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
https://gerrit-review.googlesource.com/614762 resolves a fallback
upstream for a SHA-1-pinned project that has no explicit upstream, so
--current-branch can narrow the fetch to one branch instead of fetching
all heads.
It drew that fallback from dest-branch and from the manifest <default>
upstream, dest-branch, and revision. dest-branch and tags are not valid
fallback candidates:
* dest-branch is the review destination for upload, not a fetch
source.
* Tags are meant to be immutable snapshots of a single tagged commit,
and fetching it cannot retrieve an arbitrary pinned SHA-1.
Restrict the fallback to branch heads resolved from the manifest
upstream and revision defaults, and drop dest-branch entirely.
Bug: 541240657
Bug: 544041102
Change-Id: I89ae86891ee166f4c340553ac5a9068efcc18461
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/622541
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Brian Gan <brgan@google.com>
This patch intercepts the output during a successful upload
execution to capture the generated CL URLs. It logs a dynamically
constructed "repo.uploadstate" data event to the active trace2
log containing:
- Uploaded CL URLs
- Target remote name
- Source branch
- Modified files
Test:
1. ./run_tests
2. pytest tests/test_project.py
3. Manual verification:
- Created a dummy branch `test_upload_branch` in a project with local file changes.
- Invoked repo upload passing an explicit trace output directory:
`repo --git-trace2-event-log=/tmp/trace2out upload --dry-run --no-verify art`
- Verified the injected payload successfully appeared on disk within the generated log:
```json
{"event":"data",..."key":"repo.uploadstate/cls","value":""}
{"event":"data",..."key":"repo.uploadstate/remote","value":"goog"}
{"event":"data",..."key":"repo.uploadstate/branch","value":"test_upload_branch"}
{"event":"data",..."key":"repo.uploadstate/files","value":"dummy_file.txt"}
```
(cls is correctly blank on --dry-run but populates on real HTTP pushes)
Bug: 543953499
Change-Id: I9c402a32d01d156d42cf24eaa2e60e22710b5e6f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/616581
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Ram Peri <ramperi@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
A discovered submodule becomes a project whose revision is the gitlink
read from its parent at manifest load time, which is before the parent
has been fetched. Remember which gitlink a submodule was derived from,
and add a way to read those gitlinks again, so that callers can resolve
a submodule revision once its parent is up-to-date.
A revision that is set does not have to be fetched, so verify its
objects are really there. Without them a submodule that was removed
cannot be told apart from one that was never fetched, which the caller
has to know about.
Bug: 550074864
Change-Id: I4f35d53607126508eec6f02158a3fb604917a199
Signed-off-by: kimhappy <hwanhee.kim@laplacian.cc>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/621661
Reviewed-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Avoid disabling --current-branch when syncing a SHA-1 revision without
an explicit project upstream (e.g., smart tags). Resolve a fallback
upstream from dest-branch or manifest defaults so -c only fetches the
target branch.
Bug: 541240657
Change-Id: Ib44b6a732131210e1ec3a3136747d1a19bc5aa18
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/614762
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Introduce a `RESPECT_SMART_SYNC_OVERRIDE` class attribute to the base
`Command` class, defaulting to `True`. This allows subcommands to
automatically respect the smart sync override manifest if it exists.
The override is applied in `CommonValidateOptions` before any
subcommand-specific validation or execution occurs. The `sync` and
`init` commands explicitly opt out.
This ensures all workspace-aware subcommands consistently align with the
active smart sync override manifest. It also fixes a bug in
multi-manifest setups where running a command from a submanifest would
not apply the override to the outer manifest, causing inconsistency when
resolving projects across all manifests.
Bug: 279204331
Change-Id: I9426e90a13a77ce6bd94b4a82efda4d485cbe116
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/585081
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
When the default branch cannot be determined dynamically, repo fell back
to a hardcoded "refs/heads/master". This can point at a branch that does
not exist on the server, since projects increasingly default to "main".
Rather than swap one hardcoded name for another, ask git itself what it
would use.
Bug: 483758905
Change-Id: Ic66712acea98f8e548a2d6d8211865ee5c416b4d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/612845
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Similar to how repo init -g can restrict repo syncs to
a subset of the manifest globally, allow "repo sync -g" to only sync a
subset of projects from the manifest when running that specific
sync command.
Change-Id: I4929aad109de05c73a7db42bb27fd8d47eea32fc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/609481
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Andrew Chant <achant@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Andrew Chant <achant@google.com>
The names for flags --fetch-submodules / --no-fetch-submodules are
misleading, since they impact the full sync operation (fetch and
checkout), not just the fetching.
Introduce new flags --recurse-submodules / --no-recurse-submodules
and treat the old ones as deprecated aliases.
Change-Id: I78339a3e0496a855c222c1869b27b578507886a7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/608881
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Pass the -y flag as a keyword argument "yes" to the hook main function.
This allows upload hooks (such as auto-fixers) to automatically
apply fixes when the -y flag is passed, rather than prompting the user.
Bug: 498893733
Change-Id: I12096029aa3af471ba9175314d749deb0ebd1007
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/605261
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
The superproject-gated upstream check in _CheckForImmutableRevision
only applies to user projects listed in the manifest. MetaProjects
(ManifestProject and RepoProject) never participate in a superproject
relationship, so evaluating git_superproject.UseSuperproject(...,
self.manifest) for them serves no purpose and, worse, calls the
manifest.superproject property which forces a manifest load.
During repo init, ManifestProject._ConfigureDependencies calls
self.Sync_NetworkHalf before manifest.xml has been linked into
.repo/. That reaches _CheckForImmutableRevision, which triggered the
manifest load and failed with:
ManifestParseError: .../.repo/manifest.xml: [Errno 2] No such file
or directory
breaking fresh repo init with SHA-based --manifest-branch combined
with --manifest-upstream-branch.
Factor the "should we consult the superproject for upstream?"
decision into a small overridable hook, _UseSuperprojectForUpstream.
Project's default delegates to git_superproject.UseSuperproject;
MetaProject overrides it to return False, localizing the
MetaProject-specific behavior to MetaProject.
Test verifies that calling _CheckForImmutableRevision on a
ManifestProject whose manifest.xml is not yet on disk returns False
without raising and does not create the file.
Change-Id: I22059109243d914036c06c6fe0081a5aba05da89
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574201
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Ajay Gupta <ajagup@qti.qualcomm.com>
Tested-by: Ajay Gupta <ajagup@qti.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
A global setting for disabling fetching of submodules is useful
since this can currently otherwise only be done by modifying
the manifest, or by explicitly providing projects on command line.
Add this setting as --no-fetch-submodules to mirror the existing
--fetch-submodules.
Change-Id: Ic727c54f11a594aa52315751284b87138cf246bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/607641
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
When running `repo rebase -m` (`--onto-manifest`), the command uses the
raw `revisionExpr` from the manifest (e.g. `main`) directly as the
`--onto` target. This can fail or behave incorrectly if it should
reference the local tracking branch (e.g. `refs/remotes/<remote>/main`).
Resolve `project.revisionExpr` to its local tracking branch using
`project.GetRemote().ToLocal()`. Fall back to using the raw
`revisionExpr` value if the resolution fails (raising a `GitError`).
Bug: 532028666
Change-Id: I4c1bca1374a5842688be227f6aa2afffcdad5397
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/604941
Reviewed-by: Brian Gan <brgan@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Per the git documentation for color.ui [1], setting color.ui to "true"
(or "yes") should behave identically to "auto", enabling color only
when output is written to a terminal or an active pager. Previously,
repo was equating "true" and "yes" with "always", which caused color
escape codes to be emitted unconditionally, even when output was piped
or redirected.
Replace the duplicated string-matching logic in SetDefaultColoring and
Coloring.__init__ with a single CONFIG_TO_COLOR_SETTING dict that maps
all git color config values to their behavior. This makes the mapping
easy to verify against the git docs and impossible to get out of sync
between the two call sites.
Added tests for SetDefaultColoring and Coloring.__init__ covering
all color mode values (auto, true, yes, always, never, no, false),
case insensitivity, TTY vs pipe behavior, active pager detection,
and unrecognised input.
[1] https://git-scm.com/docs/git-config#Documentation/git-config.txt-colorui
Bug: 295841573
Change-Id: I8a04b9c7e4154de37ed7518c010233039e0afdc9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602981
Tested-by: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Python 3.14 changed the default multiprocessing start method on Linux
from "fork" to "forkserver". The test_forall_all_projects_called_once
test uses mock.patch.object on Project.GetRevisionId, but class-level
mock patches do not survive into forkserver worker processes because
they start from a clean Python interpreter rather than inheriting the
parent's memory.
Replace the mock with setting revisionId directly on each Project
instance so GetRevisionId() short-circuits without touching git.
This works with any multiprocessing start method since the string
attribute is part of the Project objects stored in _parallel_context,
which is properly serialized to workers via initargs.
Bug: 425319437
Change-Id: Icd3bbd010921d7652bb2425fad85974df9198367
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602941
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Introduce support for `repo.fetchcmd` configuration, allowing users to
specify a custom command to fetch project data instead of
`_RemoteFetch`.
When `repo.fetchcmd` is specified, `repo` will execute it instead of
`_RemoteFetch` during the network half of sync. Note that
`repo.fetchcmd` requires `repo.uselocalgitdirs` to be enabled.
The command is executed in a subshell with project-context environment
variables, including the new `REPO_TREV` (target revision resolved to a
commit hash) and `REPO_PROJECT_FETCH_URL`.
After execution, `repo` verifies that the target commit is available and
that the tracking ref and FETCH_HEAD are correctly updated.
Tested with:
```
> ~/git-repo/repo init -u https://android-review.googlesource.com/platform/manifest \
--repo-url file:///usr/local/google/home/gavinmak/git-repo \
--groups developers \
--no-repo-verify \
--use-local-gitdirs
...
> git config --file .repo/manifests.git/config repo.fetchcmd 'mkdir -p $REPO_PATH && cd $REPO_PATH && if [ ! -d .git ]; then git init && git remote add aosp $REPO_PROJECT_FETCH_URL; fi && git fetch aosp $REPO_TREV && git reset --hard $REPO_TREV && mkdir -p .git/refs/remotes/aosp && echo $REPO_TREV > .git/refs/remotes/aosp/main && echo $REPO_TREV > .git/FETCH_HEAD'
> ~/git-repo/repo sync -j32
warning: repo is not tracking a remote branch, so it will not receive updates; run `repo init --repo-rev=stable` to fix.
You are currently enrolled in Git submodules experiment (go/android-submodules-quickstart). Use --no-use-superproject to override.
Syncing: 100% (4/4), done in 1m15.686s
Finalizing sync state...
repo sync has finished successfully.
```
Bug: 513329573
Change-Id: I754d3f3c78e86fdeee1a72115297a75b571bc497
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/583883
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
When viewing `repo status`, it is difficult to distinguish between branches that have active unpushed changes and stale branches that are fully synced. Previously, developers had to run commands like `repo forall -c "git status"` to see their ahead/behind counts.
This change updates Project.PrintWorkTreeStatus to automatically calculate and display the number of commits a branch is ahead and/or behind its upstream tracking branch. We use `git rev-list --left-right --count` to fetch this information natively and efficiently.
If the branch is completely synced with upstream, no extra text is shown.
Added tests for ahead-only, behind-only, diverged, no-tracking, and fully-synced branch states.
Bug: 319412954
Change-Id: I23879b2d472c7a7e11d01b565428a84b1b4f09c1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602423
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Brian Gan <brgan@google.com>
Commit-Queue: Brian Gan <brgan@google.com>
In git_trace2_event_log_base.py's GetDataEventName method, it parses
value to identify if it represents a JSON list. When a config key has an
empty string value, GetDataEventName evaluates value[0], which raises
IndexError: string index out of range.
This change fixes the crash by checking if the value is a string and using
startswith/endswith to check for JSON lists instead of direct indexing.
Test: PYTHONPATH=. pytest tests/test_git_trace2_event_log.py
Bug: 512518342
TAG=agy
CONV=ff5d70d7-e5b3-42b3-8f16-23b9e3070754
Change-Id: Ic40a8c6a22df57d0e97f268f6e1bc8a14a5024a4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/602201
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
When `repo info` runs, the reported "Current revision" is resolved using
the manifest's target branch tracking ref (e.g. refs/remotes/goog/main).
Introduce Project.GetHeadRevisionId(), which gets the checked-out HEAD
commit in the worktree, and use it in `repo info` with a fallback to the
old behavior if the project is not checked out.
Bug: 526685287
Change-Id: I72280ce27daa210cada27d722a94e365644f06e0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/599481
Reviewed-by: Brian Gan <brgan@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Repo sync fails when the following conditions are met:
* There are several checkouts of the same project
in different paths.
* The checkouts are using git hashes as revisions
(not branches).
* There is a clone-depth set on these projects.
* sync-c="true" is set in the manifest.
* The revision specified in the manifest
has moved forward since the first repo init.
The sync fails because only the first gitdir gets the "shallow" file,
and subsequent dirs can't be synced.
Do not optimize away the fetch when the conditions above happen.
Simplified the boolean check in Sync_NetworkHalf and _RemoteFetch
using has_shallow, renamed loop variable to avoid shadowing.
Test: create a manifest matching conditions above, repo init,
forward the hash, and repo sync.
Test: added tests in test_project.py
Bug: 505072873
Originally-by: Elvira Khabirova <elvira.khabirova@volvocars.com>
Change-Id: I37c533c382e34fc5ddab489c5593b9e5d3875be2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/601441
Tested-by: Sainath Varanasi <varanasisai@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Sainath Varanasi <varanasisai@google.com>
Introduce support for pluggable remote helpers (declared via the
optional 'helper' attribute in <manifest-server>) to dynamically resolve
proxy addresses. Route the XML-RPC manifest server connection through
the resolved proxy.
Bug: b/517477903
Change-Id: I3b6b8ea2640bb077521df4b4a9e8a34a8c6ecdad
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591642
Tested-by: Rahul Yadav <yadavrah@google.com>
Commit-Queue: Rahul Yadav <yadavrah@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Add REPO_PROJECT_FETCH_URL to Project.GetEnvVars(), which resolves to
the remote fetch URL of the project. This is useful for exposing the
URL to custom fetch commands or other external scripts.
Bug: 513329573
Change-Id: Ic2b0a83493934d16bb1152366ee4e1a2c35ea2dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/596121
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Initializing a submodule requires locking <parent>/.git/config.
As the implementation was currently doing this in parallel for
sibling submodules, it lead to a race condition causing
intermittent errors like:
error: could not lock config file .git/config: File exists
This commit enforces that sibling submodules are initialized
sequentially, eliminating the race condition.
Change-Id: I5ffb3de90276ba43e262d0e279a3d34324220b63
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/591241
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Allow syncing the outer manifest to a state defined by a specific
superproject revision. It updates the superproject, reads the manifest
commit from .supermanifest, and checks out the outer manifest project
to that commit.
Submanifests are then processed normally, allowing them to be updated
to the revisions specified in the new outer manifest state.
Bug: 416589884
Change-Id: I304c37a2b8794f9b74cb7e5e209a8a93762bdb52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/576321
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
This code was causing an exception in cases where `pushUrl` was set
but `projectname` was not.
This can happen when `pushUrl` is set for the manifest repo which does
not have a `projectname`. For example:
repo init --manifest-url ssh://url.to/my/manifest
cd .repo/manifests
git config remote.origin.pushurl ssh://url.to/my/manifest
repo init --manifest-url ssh://url.to/my/manifest --repo-rev main
The last `repo init` invocation causes an error.
Change-Id: Ibb68c8446880cfbac22feee595d1fd1b678c7ade
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579162
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
When working with relative submodule paths, The "./" needs special
handling similar to "../".
See information on:
https://git-scm.com/docs/git-submodule
Which currently states:
"<repository> is the URL of the new submodule’s origin repository.
This may be either an absolute URL, or (if it begins with ./ or ../),
the location relative to the superproject’s default remote
repository (Please note that to specify a repository foo.git which is
located right next to a superproject bar.git, you’ll have to use
../foo.git instead of ./foo.git - as one might expect when following
the rules for relative URLs - because the evaluation of relative URLs
in Git is identical to that of relative directories)."
The implementation also was not handling file/directory names
starting with "." or "..". Explicitly look for "./" and "../"
instead.
Change-Id: I8ae68d61fb0cbb1624183b175236e98a36e4afdb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579182
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
We pull a pinned help2man from cipd now, so we should get stable
behavior between developers. That means the color filtering should
not be necessary anymore, and we can drop the logic & unittest.
Change-Id: Ib53e1ce7f8d610d7f624c9a019c79dc5f438ac0d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/582402
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
When a manifest changes from individual linkfiles inside a directory
(e.g. dest=".llms/rules", dest=".llms/skills") to a single linkfile
for the whole directory (e.g. dest=".llms", src="dot-llms"), two
things need to happen:
1. __linkIt must replace a real directory with a symlink. Use
os.rmdir() instead of platform_utils.remove() for real directories.
rmdir only removes empty directories, so user-created content is
never deleted.
2. UpdateCopyLinkfileList must handle the cleanup correctly:
- Use os.rmdir() for directories (safe for non-empty)
- Remove empty parent directories after cleaning old dests
- Retry _CopyAndLinkFiles for all projects, since in interleaved
sync mode _CopyAndLinkFiles runs before cleanup and may have
failed because the directory was not yet empty
Change-Id: I0437b80beab98bce064cea81c11c47d699be91aa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569243
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
When running sync -j1, worker functions run directly in the main
process. Swallowing KeyboardInterrupt causes the loop to continue to the
next project instead of aborting.
Re-raise KeyboardInterrupt if running in the MainProcess, while
maintaining the suppression of stack traces in worker processes.
Bug: 468170157
Change-Id: I156d66bc209a265f7fa25eea0eb88737d1b51a34
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/581342
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Add --format={text,json} to produce machine-readable output, and
boolean options to control which sections are displayed:
--include-summary / --no-include-summary (default: on)
--include-projects / --no-include-projects (default: on)
The JSON output respects the include flags, so callers can request
only the fields they need (e.g. `repo info --format=json
--no-include-projects` for manifest metadata only).
Change-Id: I9641bc4023b630d9c61c5170eb86e5f3b787236f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569203
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
The force_checkout parameter was not propagated in all calls to
Checkout in Sync_LocalHalf.
Without this, repo sync --force-checkout can still fail for projects
currently on a local branch with no upstream/tracking configuration,
because the detach-to-manifest checkout was executed without -f,
leaving local modifications or untracked files able to block sync.
Change-Id: I58551388e2f906c4db96e220707a369057a71c24
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/579181
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josef Malmstrom <Josef.Malmstrom@arm.com>
Tested-by: Josef Malmstrom <Josef.Malmstrom@arm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Build out status subcommand unit coverage using a minimal fake repo
checkout wired through XmlManifest.
The new tests verify:
- clean status output prints the expected project header
- modified tracked files appear with the expected status marker
- `-o` output includes the orphan section and orphan entries
- branch names shown in status reflect a started non-default branch
Change-Id: Ia7c22593d0bbdc4aed81faeb168b846f3e4016ab
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/558501
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Commit-Queue: Nasser Grainawi <nasser.grainawi@oss.qualcomm.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Measure the duration of the sync operation in the Execute method of the
Sync command and pass it to post-sync hooks as a standard keyword
argument (`sync_duration_seconds`).
Updates based on code review:
- Update _API_ARGS in hooks.py to allow sync_duration_seconds for post-sync hooks.
- Do not cast sync_duration_seconds to int for better granularity.
- Update docs/repo-hooks.md to document sync_duration_seconds.
- Add unit test for argument validation in test_hooks.py.
Test: Ran run_tests using venv python, all 554 tests passed.
Bug: TBD
Change-Id: Ie29e002a5d283460d993ad96c224dbf4b6d7985c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/575021
Tested-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Extract _RunOneGC to handle GC on a single project. This refactoring
makes it easier to invoke GC from parallel worker tasks.
Also, avoid modifying the passed-in config dictionary in _RunOneGC by
creating a local copy, preventing unintended side effects on other
commands sharing the same config.
Bug: 498290329
Change-Id: I7b77ed6629b14b5ee3322870b9c6c8ce2bfd6ea2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574923
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
James Hawkins
kimhappy
Gavin Mak
Rahul Yadav
Ram Peri
Brian Gan
Andrew Chant
Josef Malmström
Ajay Gupta
Sainath Varanasi
Mike Frysinger
Carlos Fernandez
Nasser Grainawi