Add --format={text,json} to produce machine-readable output, and
boolean options to control which sections are displayed:
--include-summary / --no-include-summary (default: on)
--include-projects / --no-include-projects (default: on)
The JSON output respects the include flags, so callers can request
only the fields they need (e.g. `repo info --format=json
--no-include-projects` for manifest metadata only).
Change-Id: I9641bc4023b630d9c61c5170eb86e5f3b787236f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/569203
Commit-Queue: Carlos Fernandez <carlosfsanz@meta.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Carlos Fernandez <carlosfsanz@meta.com>
Tested-by: Carlos Fernandez <carlosfsanz@meta.com>
Measure the duration of the sync operation in the Execute method of the
Sync command and pass it to post-sync hooks as a standard keyword
argument (`sync_duration_seconds`).
Updates based on code review:
- Update _API_ARGS in hooks.py to allow sync_duration_seconds for post-sync hooks.
- Do not cast sync_duration_seconds to int for better granularity.
- Update docs/repo-hooks.md to document sync_duration_seconds.
- Add unit test for argument validation in test_hooks.py.
Test: Ran run_tests using venv python, all 554 tests passed.
Bug: TBD
Change-Id: Ie29e002a5d283460d993ad96c224dbf4b6d7985c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/575021
Tested-by: Arif Kasim <arifkasim@google.com>
Commit-Queue: Ram Peri <ramperi@google.com>
Reviewed-by: Arif Kasim <arifkasim@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Extract _RunOneGC to handle GC on a single project. This refactoring
makes it easier to invoke GC from parallel worker tasks.
Also, avoid modifying the passed-in config dictionary in _RunOneGC by
creating a local copy, preventing unintended side effects on other
commands sharing the same config.
Bug: 498290329
Change-Id: I7b77ed6629b14b5ee3322870b9c6c8ce2bfd6ea2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574923
Reviewed-by: Becky Siegel <beckysiegel@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Demote the partial sync warning from error to info and rephrase it to
be a tip rather than an error. This prevents users from thinking that
a partial sync is the cause of their hook failures when it is often a linting failure.
The message now suggests that a full sync might help if there are
cross-project dependencies, instead of implying it will fix any issue.
Change-Id: I5d8c52b53ac315aa9f145ed069798bf201fa0815
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/574262
Tested-by: Marty Heavey <mheavey@google.com>
Commit-Queue: Marty Heavey <mheavey@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Add `--missing=allow-promisor` to `git rev-list` calls in
`repack_projects`. This prevents Git from auto-fetching missing objects
from the promisor remote, which can cause stalls due to sequential
network requests.
Also add a Git version check to ensure Git is at least 2.17.0 before
running `--repack`, as `--missing=allow-promisor` was introduced in that
version.
Bug: 500133631
Change-Id: I2dcf9b46fac4c6a53a3c2a46f06f61d6aec40f2f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/570361
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Currently somewhere use git_superproject.UseSuperproject(), which checks
both the manifest config and user's config, and otherwhere use
manifest.manifestProject.use_superproject, which only checks the
manifest config. This causes Inconsistent behaviors for users who do not
set --use-superproject when doing repo init but have
repo.superprojectChoice in their git config.
Replace where using manifest.manifestProject.use_superproject with
git_superproject.UseSuperproject() to respect user's config and avoid
inconsistency.
Bug: 454514213
Change-Id: I1f734235cdd67b8a6915f1d05967d1aaa4d03f2a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/561801
Commit-Queue: Jacky Liu <qsliu@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jacky Liu <qsliu@google.com>
`delete_unused_projects` needs a full list of active projects to figure
out which orphaned .git dirs need to be deleted. Otherwise it thinks
that only the projects specified in args are active.
Bug: 447626164
Change-Id: I02beebf6a01c77742a8db78221452d71cd78ea73
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/550061
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This new command allows users to delete projects from the worktree
and from the `.repo` directory. It is a destructive operation.
It handles shared projects by refusing to wipe them unless the
`--force` flag is used. It also checks for uncommitted changes
before wiping.
Bug: 393383056
Change-Id: Ia30d8ffdc781a3f179af56310ce31c9dae331bbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/490801
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Interleaved sync didn't save _fetch_times and _local_sync_state to disk.
Phased sync saved them, but incorrectly applied moving average smoothing
repeatedly when fetching submodules, and discarded historical data
during partial syncs.
Move .Save() calls to the end of main sync loops to ensure they run
once. Update _FetchTimes.Save() to merge new data with existing history,
preventing data loss.
Change-Id: I174f98a62ac86859f1eeea1daba65eb35c227852
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/519821
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
When callback= is used, optparse does not automatically initialize
The destination when a dest= is not specified. Refine the test to
allow dest= options when callback= is used even when it seems like
it is otherwise redundant.
Bug: b/436611422
Change-Id: I5185f95cb857ca6d37357cac77fb117a83db9c0c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/509861
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
When checkout errors occurred in interleaved sync, they were wrapped in
a SyncError with no message, causing blank lines in the final summary.
Refactor _SyncResult to hold a list of exceptions, ensuring the original
error messages are propagated correctly.
Bug: 438178765
Change-Id: Ic25e515068959829cb6290cfd9e4c2d3963bbbea
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498342
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Failures in deferred sync actions were not recorded because `_Later.Run`
discarded the `GitError` exception. Record the specific error using
`syncbuf.fail()` and propagate it for proper error aggregation and
reporting.
Bug: 438178765
Change-Id: Iad59e389f9677bd6b8d873ee1ea2aa6ce44c86fa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498141
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Keep track of finished projects, not just successful ones, when deciding
which projects still need to be synced. Also project errors are already
reported by sync workers so stall detection doesn't need to add failed
projects to the error list.
Bug: 438178765
Change-Id: Ibf15aad009ba7295e70c8df2ff158215085e9732
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/498062
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
This fixes two issues:
1. the progress bar could show a count greater than the total if new projects were discovered mid-sync. Update the progress bar total dynamically
2. Make "Stall detected" error message more actionable
Bug: 432206932
Change-Id: Ie2a4ada5b1770cae0302fb06590641c522cbb7e7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/491941
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Add support for a new hook type "post-sync" declared in the manifest using
<repo-hooks>. This allows executing a script automatically after a successful
`repo sync`.
This is useful for initializing developer environments, installing project-wide
Git hooks, generating configs, and other post-sync automation tasks.
Example manifest usage:
<project name="myorg/repo-hooks" path="hooks" revision="main" />
<repo-hooks in-project="myorg/repo-hooks" enabled-list="post-sync">
<hook name="post-sync" />
</repo-hooks>
The hook script must be named `post-sync.py` and located at the root of the
hook project.
The post-sync hook does not block `repo sync`; if the script fails, the sync
still completes successfully with a warning.
Test: Added `post-sync.py` in hook project and verified it runs after `repo sync`
Bug: b/421694721
Change-Id: I69f3158f0fc319d73a85028d6e90fea02c1dc8c8
Signed-off-by: Kenny Cheng <chao.shun.cheng.tw@gmail.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/480581
Reviewed-by: Scott Lee <ddoman@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Convenient way to always enable or disable git-lfs without having to
remember to put on the command line.
Useful if you want to ALWAYS have git-lfs enabled on your system when
you 'init' a new project.
Also useful if you are using the Jenkins repo plugin as it doesn't
provide an option for enabling git-lfs in its UI.
Change-Id: Ieb1bbe83de9c21523ab69b30fc5047c257d02731
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/437661
Commit-Queue: Scott Lee <ddoman@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Fatahillah Wk <fatahillahwkwk@gmail.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Matt Moeller <moeller.matt@gmail.com>
Reviewed-by: Yingchun Li <sword.l.dragon@gmail.com>
Dedupe error reporting logic for phased and interleaved sync modes by
extracting it into _ReportErrors.
Error reporting will now distinguish between network and local failures
and lists the specific repos that failed in each phase.
Bug: 421935613
Change-Id: I4604a83943dbbd71d979158d7a1c4b8c243347d2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/484541
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
The logic for checking for repo self-updates lives in _FetchMain, which
is part of the "phased" sync path.
Extract this logic into a new _UpdateRepoProject helper method. Call
this common helper from _ExecuteHelper before either sync mode begins,
so the repo self-update check is always performed.
Bug: 421935613
Change-Id: I9a804f43fbf6239c4146be446040be531f12fc8a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/484041
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
For each assigned project, the worker sequentially calls
Sync_NetworkHalf and Sync_LocalHalf, respecting --local-only and
--network-only flags. To prevent scrambled progress bars, all stderr
output from the checkout phase is captured (shown with --verbose).
Result objects now carry status and timing information from the worker
for state updates.
Bug: 421935613
Change-Id: I398602e08a375e974a8914e5fa48ffae673dda9b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/483301
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Introduce the parallel orchestration framework for `repo sync
--interleaved`.
The new logic respects project dependencies by processing them in
hierarchical levels. Projects sharing a git object directory are grouped
and processed serially. Also reuse the familiar fetch progress bar UX.
Bug: 421935613
Change-Id: Ia388a231fa96b3220e343f952f07021bc9817d19
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/483281
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@google.com>
Upload gets a FileNotFoundError if not using superproject because it
tries to access the superproject's repo_id before checking if
superproject was actually enabled.
Reorder the logic to check use_superproject first.
Change-Id: I65cd2adab481e799dd7bb75e1a83553ad6e34d8d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/484401
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Prepare for an interleaved fetch and checkout mode for `repo sync`. The
goal of the new mode is to significantly speed up syncs by running fetch
and checkout operations in parallel for different projects, rather than
waiting for all fetches to complete before starting any checkouts.
Bug: 421935613
Change-Id: I8c66d1e790c7bba6280e409b95238c5e4e61a9c8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/482821
Reviewed-by: Scott Lee <ddoman@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
This will make it easier to add more formats without exploding the
common --xxx space and checking a large set of boolean flags.
Also fill out the test coverage while we're here.
Bug: b/412725063
Change-Id: I754013dc6cb3445f8a0979cefec599d55dafdcff
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/471941
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Warn users if the effective job count specified via `-j`,
`--jobs-network`, or `--jobs-checkout` exceeds a threshold
(currently 100). This encourages users to use more reasonable
values.
Bug: 406868778
Bug: 254914814
Change-Id: I116e2bbaf3dc824c04d1b2fbe52cf9ca5be77b9a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/466801
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
If the repo index is stale, reset --keep will refuse to reset workspace.
An index can be stale if there are any modifications to file node,
including mtime, atime, ownership changes, etc.
Bug: b/375423099
Change-Id: Ibef03d9d8d2babbb107041707281687342ab7a77
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/460022
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Tested-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Scott Lee <ddoman@google.com>
If tags are omitted from the remote pack list, they must be present in
local pack. However, local packs don't have promisor objects, meaning
that all blobs must be available locally, and therefore all missing
blobs will be downloaded during rev-list phase. Git downloads those
sequentially, by invokving fetch operation (rev-list/fetch).
Instead of downloading tags' blobs, instruct Git to include all tags in
remote rev-list operation. This change was tested with `git fsck --all`.
R=yiwzhang@google.com
Bug: b/392732561
Change-Id: Id94a40aebbe4f084c952329583d559d296db1a11
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/451422
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Tested-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
When a repository is partially cloned, no longer needed blobs are never
removed. To reclaim some of disk space, allow user to pass --repack
which affects only repositories with filter=blob:none and if projects
are not shared.
Change-Id: I0608172c9eff82fb8a6b6ef703eb109fedb7a6cc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/447722
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Tested-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Scott Lee <ddoman@google.com>
Gavin Mak
Carlos Fernandez
Ram Peri
Marty Heavey
Jacky Liu
Peter Kjellerstedt
Mike Frysinger
Kuang-che Wu
Kenny Cheng
Matt Moeller
Scott Lee
Josip Sokcevic
Josip Sokcevic