Compare commits

..

18 Commits

Author SHA1 Message Date
Gavin Mak
871e4c7ed1 sync: skip bloat check if fresh sync
Initial syncs won't have accumulated any garbage.

Bug: 379111283
Change-Id: I04b2ecde3e33f1f055038861a2705ab6aabb36d1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/536083
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-15 15:24:45 -08:00
Gavin Mak
5b0b5513d6 project: only use --no-auto-gc for git 2.23.0+
The flag for git fetch was introduced in git 2.23.0. Also skip the bloat
check after sync if using an older version.

Bug: 468589976
Bug: 379111283
Change-Id: Ib53e5494350c71a83906e5219d3a8c2b654e531f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/536082
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-15 11:32:49 -08:00
Gavin Mak
b5991d7128 sync: Add heuristic warning for bloated shallow repositories
For clone-depth="1" repositories that are dirty or have local commits,
add a check at the end of sync to detect excessive git object
accumulation.

This prevents silent performance degradation and disk exhaustion in
large prebuilts repos where automatic GC is typically disabled from
https://gerrit.googlesource.com/git-repo/+/7f87c54043ce9a35a5bb60a09ee846f9d7070352

Bug: 379111283
Change-Id: I376f38e1555cc6e906d852f6e63dc1c8f6331b4f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/534701
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Gavin Mak <gavinmak@google.com>
2025-12-10 11:34:40 -08:00
Gavin Mak
7f87c54043 project: disable auto-gc on fetch for projects with clone-depth=1
This prevents GC hangs on repos with large binaries by skipping implicit
GC during network fetch, using clone-depth=1 as a heuristic.

Bug: 379111283
Change-Id: I977bf8cd521b11e37eba7ebc9f62120f2bbaf760
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/533802
Tested-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-08 12:18:48 -08:00
Kaushik Lingarkar
50c6226075 Prevent leftover bare gitdirs after failed sync attempts
The gitdir for a project may be left in a state with bare=true due
to a previous failed sync. In this state, during a subsequent sync
attempt, repo will skip initializing the gitdir (since the directory
already exists) and directly attempt to checkout the worktree, which
will fail because the project is bare. To reduce the chance of this
happening, initialize the gitdir in a temp directory and move it once
it is ready.

Bug: 457478027
Change-Id: I4767494a3a54e7734174eae3a0d939fa9d174288
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/524203
Tested-by: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Commit-Queue: Kaushik Lingarkar <kaushikl@qti.qualcomm.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-12-05 10:35:46 -08:00
Peter Kjellerstedt
1e4b2887a7 project: Make the error message more logical when a linkfile fail
Due to the odd naming of the arguments to symlink(), the error when it
failed to create a symbolic link was misleading.

Change-Id: I1d0f30ade5970d80186f13e01c426b066cd1062f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/532541
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-03 08:48:11 -08:00
Peter Kjellerstedt
31b4b19387 info: Print a newline after printing the superproject's revision
Change-Id: Ib20233dad4e1f1fd54dbf5ca0324be22fe0e4db1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/528463
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-03 08:08:06 -08:00
Peter Kjellerstedt
2b6de52a36 Rename XmlManifest.GetGroupsStr() to XmlManifest.GetManifestGroupsStr()
This makes it more clear what kind of groups it refers to.

Change-Id: I47369050d1436efcc77f3a69d5b7c99a536b23bc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/528462
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-12-03 07:57:22 -08:00
Peter Kjellerstedt
91ec998598 manifest_xml, git_superproject: Rename an argument for XmlManifest.ToXml()
Rename the groups argument to filter_groups to make it more clear what
kind of groups it refers to.

Change-Id: I90e6e9aa74a7e3e697705dd4bf8676226055878b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/528461
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-12-02 11:44:24 -08:00
Mike Frysinger
08964a1658 docs: manifest-format: reformat spec to align the CDATA parts
Most of the file was doing this, but we've been inconsistent when
adding new entries.  Realign all of them.

Change-Id: I99ddb3a1e859235b249b6f08731bdadad8086d4e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/532461
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2025-12-02 10:43:56 -08:00
Peter Kjellerstedt
3073a90046 manifest: Propagate revision attribute through multiple levels of include
Make sure a revision attribute for an include element is propagated
through multiple levels of manifest includes.

Change-Id: If37d65b0cd47da673719976598175d0eb6b7cbbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525341
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-11-26 02:08:44 -08:00
Peter Kjellerstedt
75773b8b9d manifest, project: Store project groups as sets
This helps a lot when including common manifests with groups and they
use extend-project.

Change-Id: Ic574e7d6696139d0eb90d9915e8c7048d5e89c07
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525323
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
2025-11-26 02:08:07 -08:00
Peter Kjellerstedt
412367bfaf project: Use dicts to keep track of copyfiles and linkfiles
This avoids copying/linking the same file/link multiple times if a
copyfile/linkfile element with the same values has been specifed
multiple times. This can happen when including a common manifest that
uses an extend-project element that has a copyfile/linkfile element.

This uses dicts rather than sets to store the copyfiles and linkfiles to
make sure the order they are specified in the manifest is maintained.
For Python 3.7+, maintaining the order that keys are added to dicts is
guaranteed, and for Python 3.6 it happened to be true.

The _CopyFile class and the _LinkFile class are changed to inherit from
NamedTuple to be able to store them in dicts.

Change-Id: I9f5a80298b875251a81c5fe7d353e262d104fae4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525322
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
2025-11-26 02:07:35 -08:00
Peter Kjellerstedt
47c24b5c40 manifest: Make include groups propagate to extend-project elements
Any groups specified to an include element should propagate to any
extend-project elements and then on to the projects.

Change-Id: I62b95689cc13660858564ae569cbfd095961ecc7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525321
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2025-11-26 02:05:48 -08:00
Gavin Mak
be33106ffc wipe: Add new repo wipe subcommand
This new command allows users to delete projects from the worktree
and from the `.repo` directory. It is a destructive operation.

It handles shared projects by refusing to wipe them unless the
`--force` flag is used. It also checks for uncommitted changes
before wiping.

Bug: 393383056
Change-Id: Ia30d8ffdc781a3f179af56310ce31c9dae331bbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/490801
Tested-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2025-11-21 10:48:42 -08:00
Mike Frysinger
5998c0b506 tests: manifest_xml: convert most path usage to pathlib
Should be functionally the same, but with pathlib APIs that we've
been slowly adopting in other places, especially unittests.

Change-Id: I81364117f8eaeaf138097cdfc484d4848b7ea5bd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/525881
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2025-11-11 10:58:51 -08:00
Peter Kjellerstedt
877ef91be2 man: Regenerate after manifest update
Change-Id: I0e7ef5d4189eaaf6878be709b437ecfb57570e3f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/524921
Commit-Queue: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
2025-11-06 15:03:30 -08:00
Peter Kjellerstedt
4ab2284a94 manifest: Make extend-project support copyfile, linkfile and annotation
This allows an existing project to be extended by these elements.

Change-Id: I6826e518f39ca86485301491639101943b7e2ae0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/519781
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2025-10-27 11:38:07 -07:00
13 changed files with 1180 additions and 251 deletions

View File

@@ -399,7 +399,7 @@ class Command:
result = []
if not groups:
groups = manifest.GetGroupsStr()
groups = manifest.GetManifestGroupsStr()
groups = [x for x in re.split(r"[,\s]+", groups) if x]
if not args:

View File

@@ -59,7 +59,7 @@ following DTD:
<!ATTLIST manifest-server url CDATA #REQUIRED>
<!ELEMENT submanifest EMPTY>
<!ATTLIST submanifest name ID #REQUIRED>
<!ATTLIST submanifest name ID #REQUIRED>
<!ATTLIST submanifest remote IDREF #IMPLIED>
<!ATTLIST submanifest project CDATA #IMPLIED>
<!ATTLIST submanifest manifest-name CDATA #IMPLIED>
@@ -81,9 +81,9 @@ following DTD:
<!ATTLIST project sync-c CDATA #IMPLIED>
<!ATTLIST project sync-s CDATA #IMPLIED>
<!ATTLIST project sync-tags CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project clone-depth CDATA #IMPLIED>
<!ATTLIST project force-path CDATA #IMPLIED>
<!ATTLIST project force-path CDATA #IMPLIED>
<!ELEMENT annotation EMPTY>
<!ATTLIST annotation name CDATA #REQUIRED>
@@ -95,19 +95,21 @@ following DTD:
<!ATTLIST copyfile dest CDATA #REQUIRED>
<!ELEMENT linkfile EMPTY>
<!ATTLIST linkfile src CDATA #REQUIRED>
<!ATTLIST linkfile src CDATA #REQUIRED>
<!ATTLIST linkfile dest CDATA #REQUIRED>
<!ELEMENT extend-project EMPTY>
<!ATTLIST extend-project name CDATA #REQUIRED>
<!ATTLIST extend-project path CDATA #IMPLIED>
<!ATTLIST extend-project dest-path CDATA #IMPLIED>
<!ATTLIST extend-project groups CDATA #IMPLIED>
<!ATTLIST extend-project revision CDATA #IMPLIED>
<!ATTLIST extend-project remote CDATA #IMPLIED>
<!ELEMENT extend-project (annotation*,
copyfile*,
linkfile*)>
<!ATTLIST extend-project name CDATA #REQUIRED>
<!ATTLIST extend-project path CDATA #IMPLIED>
<!ATTLIST extend-project dest-path CDATA #IMPLIED>
<!ATTLIST extend-project groups CDATA #IMPLIED>
<!ATTLIST extend-project revision CDATA #IMPLIED>
<!ATTLIST extend-project remote CDATA #IMPLIED>
<!ATTLIST extend-project dest-branch CDATA #IMPLIED>
<!ATTLIST extend-project upstream CDATA #IMPLIED>
<!ATTLIST extend-project base-rev CDATA #IMPLIED>
<!ATTLIST extend-project upstream CDATA #IMPLIED>
<!ATTLIST extend-project base-rev CDATA #IMPLIED>
<!ELEMENT remove-project EMPTY>
<!ATTLIST remove-project name CDATA #IMPLIED>
@@ -116,7 +118,7 @@ following DTD:
<!ATTLIST remove-project base-rev CDATA #IMPLIED>
<!ELEMENT repo-hooks EMPTY>
<!ATTLIST repo-hooks in-project CDATA #REQUIRED>
<!ATTLIST repo-hooks in-project CDATA #REQUIRED>
<!ATTLIST repo-hooks enabled-list CDATA #REQUIRED>
<!ELEMENT superproject EMPTY>
@@ -125,7 +127,7 @@ following DTD:
<!ATTLIST superproject revision CDATA #IMPLIED>
<!ELEMENT contactinfo EMPTY>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
<!ELEMENT include EMPTY>
<!ATTLIST include name CDATA #REQUIRED>
@@ -285,7 +287,7 @@ should be placed. If not supplied, `revision` is used.
`path` may not be an absolute path or use "." or ".." path components.
Attribute `groups`: List of additional groups to which all projects
Attribute `groups`: Set of additional groups to which all projects
in the included submanifest belong. This appends and recurses, meaning
all projects in submanifests carry all parent submanifest groups.
Same syntax as the corresponding element of `project`.
@@ -353,7 +355,7 @@ When using `repo upload`, changes will be submitted for code
review on this branch. If unspecified both here and in the
default element, `revision` is used instead.
Attribute `groups`: List of groups to which this project belongs,
Attribute `groups`: Set of groups to which this project belongs,
whitespace or comma separated. All projects belong to the group
"all", and each project automatically belongs to a group of
its name:`name` and path:`path`. E.g. for
@@ -401,7 +403,7 @@ of the repo client where the Git working directory for this project
should be placed. This is used to move a project in the checkout by
overriding the existing `path` setting.
Attribute `groups`: List of additional groups to which this project
Attribute `groups`: Set of additional groups to which this project
belongs. Same syntax as the corresponding element of `project`.
Attribute `revision`: If specified, overrides the revision of the original
@@ -427,19 +429,20 @@ Same syntax as the corresponding element of `project`.
### Element annotation
Zero or more annotation elements may be specified as children of a
project or remote element. Each element describes a name-value pair.
For projects, this name-value pair will be exported into each project's
environment during a 'forall' command, prefixed with `REPO__`. In addition,
there is an optional attribute "keep" which accepts the case insensitive values
"true" (default) or "false". This attribute determines whether or not the
project element, an extend-project element, or a remote element. Each
element describes a name-value pair. For projects, this name-value pair
will be exported into each project's environment during a 'forall'
command, prefixed with `REPO__`. In addition, there is an optional
attribute "keep" which accepts the case insensitive values "true"
(default) or "false". This attribute determines whether or not the
annotation will be kept when exported with the manifest subcommand.
### Element copyfile
Zero or more copyfile elements may be specified as children of a
project element. Each element describes a src-dest pair of files;
the "src" file will be copied to the "dest" place during `repo sync`
command.
project element, or an extend-project element. Each element describes a
src-dest pair of files; the "src" file will be copied to the "dest"
place during `repo sync` command.
"src" is project relative, "dest" is relative to the top of the tree.
Copying from paths outside of the project or to paths outside of the repo
@@ -450,10 +453,14 @@ Intermediate paths must not be symlinks either.
Parent directories of "dest" will be automatically created if missing.
The files are copied in the order they are specified in the manifests.
If multiple elements specify the same source and destination, they will
only be applied as one, based on the first occurence. Files are copied
before any links specified via linkfile elements are created.
### Element linkfile
It's just like copyfile and runs at the same time as copyfile but
instead of copying it creates a symlink.
It's just like copyfile, but instead of copying it creates a symlink.
The symlink is created at "dest" (relative to the top of the tree) and
points to the path specified by "src" which is a path in the project.
@@ -463,6 +470,11 @@ Parent directories of "dest" will be automatically created if missing.
The symlink target may be a file or directory, but it may not point outside
of the repo client.
The links are created in the order they are specified in the manifests.
If multiple elements specify the same source and destination, they will
only be applied as one, based on the first occurence. Links are created
after any files specified via copyfile elements are copied.
### Element remove-project
Deletes a project from the internal manifest table, possibly
@@ -560,13 +572,16 @@ the manifest repository's root.
"name" may not be an absolute path or use "." or ".." path components.
These restrictions are not enforced for [Local Manifests].
Attribute `groups`: List of additional groups to which all projects
Attribute `groups`: Set of additional groups to which all projects
in the included manifest belong. This appends and recurses, meaning
all projects in included manifests carry all parent include groups.
This also applies to all extend-project elements in the included manifests.
Same syntax as the corresponding element of `project`.
Attribute `revision`: Name of a Git branch (e.g. `main` or `refs/heads/main`)
default to which all projects in the included manifest belong.
default to which all projects in the included manifest belong. This recurses,
meaning it will apply to all projects in all manifests included as a result of
this element.
## Local Manifests {#local-manifests}

View File

@@ -422,7 +422,8 @@ class Superproject:
)
return None
manifest_str = self._manifest.ToXml(
groups=self._manifest.GetGroupsStr(), omit_local=True
filter_groups=self._manifest.GetManifestGroupsStr(),
omit_local=True,
).toxml()
manifest_path = self._manifest_path
try:

View File

@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "April 2025" "repo manifest" "Repo Manual"
.TH REPO "1" "December 2025" "repo manifest" "Repo Manual"
.SH NAME
repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS
@@ -139,7 +139,7 @@ include*)>
<!ATTLIST manifest\-server url CDATA #REQUIRED>
.IP
<!ELEMENT submanifest EMPTY>
<!ATTLIST submanifest name ID #REQUIRED>
<!ATTLIST submanifest name ID #REQUIRED>
<!ATTLIST submanifest remote IDREF #IMPLIED>
<!ATTLIST submanifest project CDATA #IMPLIED>
<!ATTLIST submanifest manifest\-name CDATA #IMPLIED>
@@ -170,9 +170,9 @@ CDATA #IMPLIED>
<!ATTLIST project sync\-c CDATA #IMPLIED>
<!ATTLIST project sync\-s CDATA #IMPLIED>
<!ATTLIST project sync\-tags CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project upstream CDATA #IMPLIED>
<!ATTLIST project clone\-depth CDATA #IMPLIED>
<!ATTLIST project force\-path CDATA #IMPLIED>
<!ATTLIST project force\-path CDATA #IMPLIED>
.IP
<!ELEMENT annotation EMPTY>
<!ATTLIST annotation name CDATA #REQUIRED>
@@ -184,19 +184,34 @@ CDATA #IMPLIED>
<!ATTLIST copyfile dest CDATA #REQUIRED>
.IP
<!ELEMENT linkfile EMPTY>
<!ATTLIST linkfile src CDATA #REQUIRED>
<!ATTLIST linkfile src CDATA #REQUIRED>
<!ATTLIST linkfile dest CDATA #REQUIRED>
.TP
<!ELEMENT extend\-project (annotation*,
copyfile*,
linkfile*)>
.TP
<!ATTLIST extend\-project name
CDATA #REQUIRED>
.TP
<!ATTLIST extend\-project path
CDATA #IMPLIED>
.TP
<!ATTLIST extend\-project dest\-path
CDATA #IMPLIED>
.TP
<!ATTLIST extend\-project groups
CDATA #IMPLIED>
.TP
<!ATTLIST extend\-project revision
CDATA #IMPLIED>
.TP
<!ATTLIST extend\-project remote
CDATA #IMPLIED>
.IP
<!ELEMENT extend\-project EMPTY>
<!ATTLIST extend\-project name CDATA #REQUIRED>
<!ATTLIST extend\-project path CDATA #IMPLIED>
<!ATTLIST extend\-project dest\-path CDATA #IMPLIED>
<!ATTLIST extend\-project groups CDATA #IMPLIED>
<!ATTLIST extend\-project revision CDATA #IMPLIED>
<!ATTLIST extend\-project remote CDATA #IMPLIED>
<!ATTLIST extend\-project dest\-branch CDATA #IMPLIED>
<!ATTLIST extend\-project upstream CDATA #IMPLIED>
<!ATTLIST extend\-project base\-rev CDATA #IMPLIED>
<!ATTLIST extend\-project upstream CDATA #IMPLIED>
<!ATTLIST extend\-project base\-rev CDATA #IMPLIED>
.IP
<!ELEMENT remove\-project EMPTY>
<!ATTLIST remove\-project name CDATA #IMPLIED>
@@ -205,7 +220,7 @@ CDATA #IMPLIED>
<!ATTLIST remove\-project base\-rev CDATA #IMPLIED>
.IP
<!ELEMENT repo\-hooks EMPTY>
<!ATTLIST repo\-hooks in\-project CDATA #REQUIRED>
<!ATTLIST repo\-hooks in\-project CDATA #REQUIRED>
<!ATTLIST repo\-hooks enabled\-list CDATA #REQUIRED>
.IP
<!ELEMENT superproject EMPTY>
@@ -214,7 +229,7 @@ CDATA #IMPLIED>
<!ATTLIST superproject revision CDATA #IMPLIED>
.IP
<!ELEMENT contactinfo EMPTY>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
.IP
<!ELEMENT include EMPTY>
<!ATTLIST include name CDATA #REQUIRED>
@@ -362,7 +377,7 @@ supplied, `revision` is used.
.PP
`path` may not be an absolute path or use "." or ".." path components.
.PP
Attribute `groups`: List of additional groups to which all projects in the
Attribute `groups`: Set of additional groups to which all projects in the
included submanifest belong. This appends and recurses, meaning all projects in
submanifests carry all parent submanifest groups. Same syntax as the
corresponding element of `project`.
@@ -424,7 +439,7 @@ Attribute `dest\-branch`: Name of a Git branch (e.g. `main`). When using `repo
upload`, changes will be submitted for code review on this branch. If
unspecified both here and in the default element, `revision` is used instead.
.PP
Attribute `groups`: List of groups to which this project belongs, whitespace or
Attribute `groups`: Set of groups to which this project belongs, whitespace or
comma separated. All projects belong to the group "all", and each project
automatically belongs to a group of its name:`name` and path:`path`. E.g. for
`<project name="monkeys" path="barrel\-of"/>`, that project definition is
@@ -468,8 +483,8 @@ repo client where the Git working directory for this project should be placed.
This is used to move a project in the checkout by overriding the existing `path`
setting.
.PP
Attribute `groups`: List of additional groups to which this project belongs.
Same syntax as the corresponding element of `project`.
Attribute `groups`: Set of additional groups to which this project belongs. Same
syntax as the corresponding element of `project`.
.PP
Attribute `revision`: If specified, overrides the revision of the original
project. Same syntax as the corresponding element of `project`.
@@ -493,19 +508,21 @@ element of `project`.
.PP
Element annotation
.PP
Zero or more annotation elements may be specified as children of a project or
remote element. Each element describes a name\-value pair. For projects, this
name\-value pair will be exported into each project's environment during a
\&'forall' command, prefixed with `REPO__`. In addition, there is an optional
attribute "keep" which accepts the case insensitive values "true" (default) or
"false". This attribute determines whether or not the annotation will be kept
when exported with the manifest subcommand.
Zero or more annotation elements may be specified as children of a project
element, an extend\-project element, or a remote element. Each element describes
a name\-value pair. For projects, this name\-value pair will be exported into each
project's environment during a 'forall' command, prefixed with `REPO__`. In
addition, there is an optional attribute "keep" which accepts the case
insensitive values "true" (default) or "false". This attribute determines
whether or not the annotation will be kept when exported with the manifest
subcommand.
.PP
Element copyfile
.PP
Zero or more copyfile elements may be specified as children of a project
element. Each element describes a src\-dest pair of files; the "src" file will be
copied to the "dest" place during `repo sync` command.
element, or an extend\-project element. Each element describes a src\-dest pair of
files; the "src" file will be copied to the "dest" place during `repo sync`
command.
.PP
"src" is project relative, "dest" is relative to the top of the tree. Copying
from paths outside of the project or to paths outside of the repo client is not
@@ -516,10 +533,14 @@ Intermediate paths must not be symlinks either.
.PP
Parent directories of "dest" will be automatically created if missing.
.PP
The files are copied in the order they are specified in the manifests. If
multiple elements specify the same source and destination, they will only be
applied as one, based on the first occurence. Files are copied before any links
specified via linkfile elements are created.
.PP
Element linkfile
.PP
It's just like copyfile and runs at the same time as copyfile but instead of
copying it creates a symlink.
It's just like copyfile, but instead of copying it creates a symlink.
.PP
The symlink is created at "dest" (relative to the top of the tree) and points to
the path specified by "src" which is a path in the project.
@@ -529,6 +550,11 @@ Parent directories of "dest" will be automatically created if missing.
The symlink target may be a file or directory, but it may not point outside of
the repo client.
.PP
The links are created in the order they are specified in the manifests. If
multiple elements specify the same source and destination, they will only be
applied as one, based on the first occurence. Links are created after any files
specified via copyfile elements are copied.
.PP
Element remove\-project
.PP
Deletes a project from the internal manifest table, possibly allowing a
@@ -620,13 +646,16 @@ repository's root.
"name" may not be an absolute path or use "." or ".." path components. These
restrictions are not enforced for [Local Manifests].
.PP
Attribute `groups`: List of additional groups to which all projects in the
Attribute `groups`: Set of additional groups to which all projects in the
included manifest belong. This appends and recurses, meaning all projects in
included manifests carry all parent include groups. Same syntax as the
included manifests carry all parent include groups. This also applies to all
extend\-project elements in the included manifests. Same syntax as the
corresponding element of `project`.
.PP
Attribute `revision`: Name of a Git branch (e.g. `main` or `refs/heads/main`)
default to which all projects in the included manifest belong.
default to which all projects in the included manifest belong. This recurses,
meaning it will apply to all projects in all manifests included as a result of
this element.
.PP
Local Manifests
.PP

61
man/repo-wipe.1 Normal file
View File

@@ -0,0 +1,61 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "November 2025" "repo wipe" "Repo Manual"
.SH NAME
repo \- repo wipe - manual page for repo wipe
.SH SYNOPSIS
.B repo
\fI\,wipe <project>\/\fR...
.SH DESCRIPTION
Summary
.PP
Wipe projects from the worktree
.SH OPTIONS
.TP
\fB\-h\fR, \fB\-\-help\fR
show this help message and exit
.TP
\fB\-f\fR, \fB\-\-force\fR
force wipe shared projects and uncommitted changes
.TP
\fB\-\-force\-uncommitted\fR
force wipe even if there are uncommitted changes
.TP
\fB\-\-force\-shared\fR
force wipe even if the project shares an object
directory
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR
show all output
.TP
\fB\-q\fR, \fB\-\-quiet\fR
only show errors
.SS Multi\-manifest options:
.TP
\fB\-\-outer\-manifest\fR
operate starting at the outermost manifest
.TP
\fB\-\-no\-outer\-manifest\fR
do not operate on outer manifests
.TP
\fB\-\-this\-manifest\-only\fR
only operate on this (sub)manifest
.TP
\fB\-\-no\-this\-manifest\-only\fR, \fB\-\-all\-manifests\fR
operate on this manifest and its submanifests
.PP
Run `repo help wipe` to view the detailed manual.
.SH DETAILS
.PP
The 'repo wipe' command removes the specified projects from the worktree (the
checked out source code) and deletes the project's git data from `.repo`.
.PP
This is a destructive operation and cannot be undone.
.PP
Projects can be specified either by name, or by a relative or absolute path to
the project's local directory.
.SH EXAMPLES
.SS # Wipe the project "platform/build" by name:
$ repo wipe platform/build
.SS # Wipe the project at the path "build/make":
$ repo wipe build/make

View File

@@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "April 2025" "repo" "Repo Manual"
.TH REPO "1" "November 2025" "repo" "Repo Manual"
.SH NAME
repo \- repository management tool built on top of git
.SH SYNOPSIS
@@ -132,6 +132,9 @@ Upload changes for code review
.TP
version
Display the version of repo
.TP
wipe
Wipe projects from the worktree
.PP
See 'repo help <command>' for more information on a specific command.
Bug reports: https://issues.gerritcodereview.com/issues/new?component=1370071

View File

@@ -255,7 +255,7 @@ class _XmlSubmanifest:
project: a string, the name of the manifest project.
revision: a string, the commitish.
manifestName: a string, the submanifest file name.
groups: a list of strings, the groups to add to all projects in the
groups: a set of strings, the groups to add to all projects in the
submanifest.
default_groups: a list of strings, the default groups to sync.
path: a string, the relative path for the submanifest checkout.
@@ -281,7 +281,7 @@ class _XmlSubmanifest:
self.project = project
self.revision = revision
self.manifestName = manifestName
self.groups = groups
self.groups = groups or set()
self.default_groups = default_groups
self.path = path
self.parent = parent
@@ -304,7 +304,7 @@ class _XmlSubmanifest:
self.repo_client = RepoClient(
parent.repodir,
linkFile,
parent_groups=",".join(groups) or "",
parent_groups=groups,
submanifest_path=os.path.join(parent.path_prefix, self.relpath),
outer_client=outer_client,
default_groups=default_groups,
@@ -345,7 +345,7 @@ class _XmlSubmanifest:
manifestName = self.manifestName or "default.xml"
revision = self.revision or self.name
path = self.path or revision.split("/")[-1]
groups = self.groups or []
groups = self.groups
return SubmanifestSpec(
self.name, manifestUrl, manifestName, revision, path, groups
@@ -359,9 +359,7 @@ class _XmlSubmanifest:
def GetGroupsStr(self):
"""Returns the `groups` given for this submanifest."""
if self.groups:
return ",".join(self.groups)
return ""
return ",".join(sorted(self.groups))
def GetDefaultGroupsStr(self):
"""Returns the `default-groups` given for this submanifest."""
@@ -381,7 +379,7 @@ class SubmanifestSpec:
self.manifestName = manifestName
self.revision = revision
self.path = path
self.groups = groups or []
self.groups = groups
class XmlManifest:
@@ -393,7 +391,7 @@ class XmlManifest:
manifest_file,
local_manifests=None,
outer_client=None,
parent_groups="",
parent_groups=None,
submanifest_path="",
default_groups=None,
):
@@ -409,7 +407,8 @@ class XmlManifest:
manifests. This will usually be
|repodir|/|LOCAL_MANIFESTS_DIR_NAME|.
outer_client: RepoClient of the outer manifest.
parent_groups: a string, the groups to apply to this projects.
parent_groups: a set of strings, the groups to apply to this
manifest.
submanifest_path: The submanifest root relative to the repo root.
default_groups: a string, the default manifest groups to use.
"""
@@ -432,7 +431,7 @@ class XmlManifest:
self.manifestFileOverrides = {}
self.local_manifests = local_manifests
self._load_local_manifests = True
self.parent_groups = parent_groups
self.parent_groups = parent_groups or set()
self.default_groups = default_groups
if submanifest_path and not outer_client:
@@ -567,21 +566,29 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
"""
return [x for x in re.split(r"[,\s]+", field) if x]
def _ParseSet(self, field):
"""Parse fields that contain flattened sets.
These are whitespace & comma separated. Empty elements will be
discarded.
"""
return set(self._ParseList(field))
def ToXml(
self,
peg_rev=False,
peg_rev_upstream=True,
peg_rev_dest_branch=True,
groups=None,
filter_groups=None,
omit_local=False,
):
"""Return the current manifest XML."""
mp = self.manifestProject
if groups is None:
groups = mp.manifest_groups
if groups:
groups = self._ParseList(groups)
if filter_groups is None:
filter_groups = mp.manifest_groups
if filter_groups:
filter_groups = self._ParseList(filter_groups)
doc = xml.dom.minidom.Document()
root = doc.createElement("manifest")
@@ -654,7 +661,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
output_project(parent, parent_node, project)
def output_project(parent, parent_node, p):
if not p.MatchesGroups(groups):
if not p.MatchesGroups(filter_groups):
return
if omit_local and self.IsFromLocalManifest(p):
@@ -725,10 +732,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
le.setAttribute("dest", lf.dest)
e.appendChild(le)
default_groups = ["all", "name:%s" % p.name, "path:%s" % p.relpath]
egroups = [g for g in p.groups if g not in default_groups]
if egroups:
e.setAttribute("groups", ",".join(egroups))
groups = p.groups - {"all", f"name:{p.name}", f"path:{p.relpath}"}
if groups:
e.setAttribute("groups", ",".join(sorted(groups)))
for a in p.annotations:
if a.keep == "true":
@@ -1116,7 +1122,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
groups += f",platform-{platform.system().lower()}"
return groups
def GetGroupsStr(self):
def GetManifestGroupsStr(self):
"""Returns the manifest group string that should be synced."""
return (
self.manifestProject.manifest_groups or self.GetDefaultGroupsStr()
@@ -1171,12 +1177,12 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
b = b[len(R_HEADS) :]
self.branch = b
parent_groups = self.parent_groups
parent_groups = self.parent_groups.copy()
if self.path_prefix:
parent_groups = (
parent_groups |= {
f"{SUBMANIFEST_GROUP_PREFIX}:path:"
f"{self.path_prefix},{parent_groups}"
)
f"{self.path_prefix}"
}
# The manifestFile was specified by the user which is why we
# allow include paths to point anywhere.
@@ -1202,16 +1208,16 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
# Since local manifests are entirely managed by
# the user, allow them to point anywhere the
# user wants.
local_group = (
local_group = {
f"{LOCAL_MANIFEST_GROUP_PREFIX}:"
f"{local_file[:-4]}"
)
}
nodes.append(
self._ParseManifestXml(
local,
self.subdir,
parent_groups=(
f"{local_group},{parent_groups}"
local_group | parent_groups
),
restrict_includes=False,
)
@@ -1262,7 +1268,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
self,
path,
include_root,
parent_groups="",
parent_groups=None,
restrict_includes=True,
parent_node=None,
):
@@ -1271,11 +1277,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
Args:
path: The XML file to read & parse.
include_root: The path to interpret include "name"s relative to.
parent_groups: The groups to apply to this projects.
parent_groups: The set of groups to apply to this manifest.
restrict_includes: Whether to constrain the "name" attribute of
includes.
parent_node: The parent include node, to apply attribute to this
projects.
parent_node: The parent include node, to apply attributes to this
manifest.
Returns:
List of XML nodes.
@@ -1299,6 +1305,14 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
nodes = []
for node in manifest.childNodes:
if (
parent_node
and node.nodeName in ("include", "project")
and not node.hasAttribute("revision")
):
node.setAttribute(
"revision", parent_node.getAttribute("revision")
)
if node.nodeName == "include":
name = self._reqatt(node, "name")
if restrict_includes:
@@ -1307,12 +1321,10 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
raise ManifestInvalidPathError(
f'<include> invalid "name": {name}: {msg}'
)
include_groups = ""
if parent_groups:
include_groups = parent_groups
include_groups = (parent_groups or set()).copy()
if node.hasAttribute("groups"):
include_groups = (
node.getAttribute("groups") + "," + include_groups
include_groups |= self._ParseSet(
node.getAttribute("groups")
)
fp = os.path.join(include_root, name)
if not os.path.isfile(fp):
@@ -1335,21 +1347,16 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
f"failed parsing included manifest {name}: {e}"
)
else:
if parent_groups and node.nodeName == "project":
nodeGroups = parent_groups
if node.hasAttribute("groups"):
nodeGroups = (
node.getAttribute("groups") + "," + nodeGroups
)
node.setAttribute("groups", nodeGroups)
if (
parent_node
and node.nodeName == "project"
and not node.hasAttribute("revision")
if parent_groups and node.nodeName in (
"project",
"extend-project",
):
node.setAttribute(
"revision", parent_node.getAttribute("revision")
)
nodeGroups = parent_groups.copy()
if node.hasAttribute("groups"):
nodeGroups |= self._ParseSet(
node.getAttribute("groups")
)
node.setAttribute("groups", ",".join(sorted(nodeGroups)))
nodes.append(node)
return nodes
@@ -1458,7 +1465,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
dest_path = node.getAttribute("dest-path")
groups = node.getAttribute("groups")
if groups:
groups = self._ParseList(groups)
groups = self._ParseSet(groups or "")
revision = node.getAttribute("revision")
remote_name = node.getAttribute("remote")
if not remote_name:
@@ -1479,7 +1486,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if path and p.relpath != path:
continue
if groups:
p.groups.extend(groups)
p.groups |= groups
if revision:
if base_revision:
if p.revisionExpr != base_revision:
@@ -1509,6 +1516,14 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
p.UpdatePaths(relpath, worktree, gitdir, objdir)
self._paths[p.relpath] = p
for n in node.childNodes:
if n.nodeName == "copyfile":
self._ParseCopyFile(p, n)
elif n.nodeName == "linkfile":
self._ParseLinkFile(p, n)
elif n.nodeName == "annotation":
self._ParseAnnotation(p, n)
if node.nodeName == "repo-hooks":
# Only one project can be the hooks project
if repo_hooks_project is not None:
@@ -1802,7 +1817,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
groups = ""
if node.hasAttribute("groups"):
groups = node.getAttribute("groups")
groups = self._ParseList(groups)
groups = self._ParseSet(groups)
default_groups = self._ParseList(node.getAttribute("default-groups"))
path = node.getAttribute("path")
if path == "":
@@ -1911,11 +1926,6 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
upstream = node.getAttribute("upstream") or self._default.upstreamExpr
groups = ""
if node.hasAttribute("groups"):
groups = node.getAttribute("groups")
groups = self._ParseList(groups)
if parent is None:
(
relpath,
@@ -1930,8 +1940,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
parent, name, path
)
default_groups = ["all", "name:%s" % name, "path:%s" % relpath]
groups.extend(set(default_groups).difference(groups))
groups = ""
if node.hasAttribute("groups"):
groups = node.getAttribute("groups")
groups = self._ParseSet(groups)
groups |= {"all", f"name:{name}", f"path:{relpath}"}
if self.IsMirror and node.hasAttribute("force-path"):
if XmlBool(node, "force-path", False):
@@ -1963,11 +1976,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
for n in node.childNodes:
if n.nodeName == "copyfile":
self._ParseCopyFile(project, n)
if n.nodeName == "linkfile":
elif n.nodeName == "linkfile":
self._ParseLinkFile(project, n)
if n.nodeName == "annotation":
elif n.nodeName == "annotation":
self._ParseAnnotation(project, n)
if n.nodeName == "project":
elif n.nodeName == "project":
project.subprojects.append(
self._ParseProject(n, parent=project)
)

View File

@@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime
import errno
import filecmp
import glob
@@ -390,22 +391,17 @@ def _SafeExpandPath(base, subpath, skipfinal=False):
return path
class _CopyFile:
class _CopyFile(NamedTuple):
"""Container for <copyfile> manifest element."""
def __init__(self, git_worktree, src, topdir, dest):
"""Register a <copyfile> request.
Args:
git_worktree: Absolute path to the git project checkout.
src: Relative path under |git_worktree| of file to read.
topdir: Absolute path to the top of the repo client checkout.
dest: Relative path under |topdir| of file to write.
"""
self.git_worktree = git_worktree
self.topdir = topdir
self.src = src
self.dest = dest
# Absolute path to the git project checkout.
git_worktree: str
# Relative path under |git_worktree| of file to read.
src: str
# Absolute path to the top of the repo client checkout.
topdir: str
# Relative path under |topdir| of file to write.
dest: str
def _Copy(self):
src = _SafeExpandPath(self.git_worktree, self.src)
@@ -439,22 +435,17 @@ class _CopyFile:
logger.error("error: Cannot copy file %s to %s", src, dest)
class _LinkFile:
class _LinkFile(NamedTuple):
"""Container for <linkfile> manifest element."""
def __init__(self, git_worktree, src, topdir, dest):
"""Register a <linkfile> request.
Args:
git_worktree: Absolute path to the git project checkout.
src: Target of symlink relative to path under |git_worktree|.
topdir: Absolute path to the top of the repo client checkout.
dest: Relative path under |topdir| of symlink to create.
"""
self.git_worktree = git_worktree
self.topdir = topdir
self.src = src
self.dest = dest
# Absolute path to the git project checkout.
git_worktree: str
# Target of symlink relative to path under |git_worktree|.
src: str
# Absolute path to the top of the repo client checkout.
topdir: str
# Relative path under |topdir| of symlink to create.
dest: str
def __linkIt(self, relSrc, absDest):
# Link file if it does not exist or is out of date.
@@ -471,9 +462,7 @@ class _LinkFile:
os.makedirs(dest_dir)
platform_utils.symlink(relSrc, absDest)
except OSError:
logger.error(
"error: Cannot link file %s to %s", relSrc, absDest
)
logger.error("error: Cannot symlink %s to %s", absDest, relSrc)
def _Link(self):
"""Link the self.src & self.dest paths.
@@ -564,7 +553,7 @@ class Project:
revisionExpr,
revisionId,
rebase=True,
groups=None,
groups=set(),
sync_c=False,
sync_s=False,
sync_tags=True,
@@ -633,8 +622,9 @@ class Project:
self.subprojects = []
self.snapshots = {}
self.copyfiles = []
self.linkfiles = []
# Use dicts to dedupe while maintaining declared order.
self.copyfiles = {}
self.linkfiles = {}
self.annotations = []
self.dest_branch = dest_branch
@@ -848,9 +838,9 @@ class Project:
"""
default_groups = self.manifest.default_groups or ["default"]
expanded_manifest_groups = manifest_groups or default_groups
expanded_project_groups = ["all"] + (self.groups or [])
expanded_project_groups = {"all"} | self.groups
if "notdefault" not in expanded_project_groups:
expanded_project_groups += ["default"]
expanded_project_groups |= {"default"}
matched = False
for group in expanded_manifest_groups:
@@ -1794,7 +1784,7 @@ class Project:
Paths should have basic validation run on them before being queued.
Further checking will be handled when the actual copy happens.
"""
self.copyfiles.append(_CopyFile(self.worktree, src, topdir, dest))
self.copyfiles[_CopyFile(self.worktree, src, topdir, dest)] = True
def AddLinkFile(self, src, dest, topdir):
"""Mark |dest| to create a symlink (relative to |topdir|) pointing to
@@ -1805,7 +1795,7 @@ class Project:
Paths should have basic validation run on them before being queued.
Further checking will be handled when the actual link happens.
"""
self.linkfiles.append(_LinkFile(self.worktree, src, topdir, dest))
self.linkfiles[_LinkFile(self.worktree, src, topdir, dest)] = True
def AddAnnotation(self, name, value, keep):
self.annotations.append(Annotation(name, value, keep))
@@ -2582,6 +2572,16 @@ class Project:
if os.path.exists(os.path.join(self.gitdir, "shallow")):
cmd.append("--depth=2147483647")
# Use clone-depth="1" as a heuristic for repositories containing
# large binaries and disable auto GC to prevent potential hangs.
# Check the configured depth because the `depth` argument might be None
# if REPO_ALLOW_SHALLOW=0 converted it to a partial clone.
effective_depth = (
self.clone_depth or self.manifest.manifestProject.depth
)
if effective_depth == 1 and git_require((2, 23, 0)):
cmd.append("--no-auto-gc")
if not verbose:
cmd.append("--quiet")
if not quiet and sys.stdout.isatty():
@@ -3084,8 +3084,13 @@ class Project:
raise GitError(f"{self.name} merge {head} ", project=self.name)
def _InitGitDir(self, mirror_git=None, force_sync=False, quiet=False):
# Prefix for temporary directories created during gitdir initialization.
TMP_GITDIR_PREFIX = ".tmp-project-initgitdir-"
init_git_dir = not os.path.exists(self.gitdir)
init_obj_dir = not os.path.exists(self.objdir)
tmp_gitdir = None
curr_gitdir = self.gitdir
curr_config = self.config
try:
# Initialize the bare repository, which contains all of the objects.
if init_obj_dir:
@@ -3105,27 +3110,33 @@ class Project:
# well.
if self.objdir != self.gitdir:
if init_git_dir:
os.makedirs(self.gitdir)
os.makedirs(os.path.dirname(self.gitdir), exist_ok=True)
tmp_gitdir = tempfile.mkdtemp(
prefix=TMP_GITDIR_PREFIX,
dir=os.path.dirname(self.gitdir),
)
curr_config = GitConfig.ForRepository(
gitdir=tmp_gitdir, defaults=self.manifest.globalConfig
)
curr_gitdir = tmp_gitdir
if init_obj_dir or init_git_dir:
self._ReferenceGitDir(
self.objdir, self.gitdir, copy_all=True
self.objdir, curr_gitdir, copy_all=True
)
try:
self._CheckDirReference(self.objdir, self.gitdir)
self._CheckDirReference(self.objdir, curr_gitdir)
except GitError as e:
if force_sync:
logger.error(
"Retrying clone after deleting %s", self.gitdir
)
try:
platform_utils.rmtree(os.path.realpath(self.gitdir))
if self.worktree and os.path.exists(
os.path.realpath(self.worktree)
):
platform_utils.rmtree(
os.path.realpath(self.worktree)
)
rm_dirs = (
tmp_gitdir,
self.gitdir,
self.worktree,
)
for d in rm_dirs:
if d and os.path.exists(d):
platform_utils.rmtree(os.path.realpath(d))
return self._InitGitDir(
mirror_git=mirror_git,
force_sync=False,
@@ -3176,18 +3187,21 @@ class Project:
m = self.manifest.manifestProject.config
for key in ["user.name", "user.email"]:
if m.Has(key, include_defaults=False):
self.config.SetString(key, m.GetString(key))
curr_config.SetString(key, m.GetString(key))
if not self.manifest.EnableGitLfs:
self.config.SetString(
curr_config.SetString(
"filter.lfs.smudge", "git-lfs smudge --skip -- %f"
)
self.config.SetString(
curr_config.SetString(
"filter.lfs.process", "git-lfs filter-process --skip"
)
self.config.SetBoolean(
curr_config.SetBoolean(
"core.bare", True if self.manifest.IsMirror else None
)
if tmp_gitdir:
platform_utils.rename(tmp_gitdir, self.gitdir)
tmp_gitdir = None
if not init_obj_dir:
# The project might be shared (obj_dir already initialized), but
# such information is not available here. Instead of passing it,
@@ -3204,6 +3218,27 @@ class Project:
if init_git_dir and os.path.exists(self.gitdir):
platform_utils.rmtree(self.gitdir)
raise
finally:
# Clean up the temporary directory created during the process,
# as well as any stale ones left over from previous attempts.
if tmp_gitdir and os.path.exists(tmp_gitdir):
platform_utils.rmtree(tmp_gitdir)
age_threshold = datetime.timedelta(days=1)
now = datetime.datetime.now()
for tmp_dir in glob.glob(
os.path.join(
os.path.dirname(self.gitdir), f"{TMP_GITDIR_PREFIX}*"
)
):
try:
mtime = datetime.datetime.fromtimestamp(
os.path.getmtime(tmp_dir)
)
if now - mtime > age_threshold:
platform_utils.rmtree(tmp_dir)
except OSError:
pass
def _UpdateHooks(self, quiet=False):
if os.path.exists(self.objdir):

View File

@@ -88,7 +88,7 @@ class Info(PagedCommand):
self.manifest = self.manifest.outer_client
manifestConfig = self.manifest.manifestProject.config
mergeBranch = manifestConfig.GetBranch("default").merge
manifestGroups = self.manifest.GetGroupsStr()
manifestGroups = self.manifest.GetManifestGroupsStr()
self.heading("Manifest branch: ")
if self.manifest.default.revisionExpr:
@@ -106,6 +106,7 @@ class Info(PagedCommand):
srev = sp.commit_id if sp and sp.commit_id else "None"
self.heading("Superproject revision: ")
self.headtext(srev)
self.out.nl()
self.printSeparator()

View File

@@ -87,6 +87,10 @@ _ONE_DAY_S = 24 * 60 * 60
_REPO_ALLOW_SHALLOW = os.environ.get("REPO_ALLOW_SHALLOW")
_BLOAT_PACK_COUNT_THRESHOLD = 10
_BLOAT_SIZE_PACK_THRESHOLD_KB = 10 * 1024 * 1024 # 10 GiB in KiB
_BLOAT_SIZE_GARBAGE_THRESHOLD_KB = 1 * 1024 * 1024 # 1 GiB in KiB
logger = RepoLogger(__file__)
@@ -1371,6 +1375,110 @@ later is required to fix a server side protocol bug.
t.join()
pm.end()
@classmethod
def _CheckOneBloatedProject(cls, project_index: int) -> Optional[str]:
"""Checks if a single project is bloated.
Args:
project_index: The index of the project in the parallel context.
Returns:
The name of the project if it is bloated, else None.
"""
project = cls.get_parallel_context()["projects"][project_index]
if not project.Exists or not project.worktree:
return None
# Only check dirty or locally modified projects. These can't be
# freshly cloned and will accumulate garbage.
try:
is_dirty = project.IsDirty(consider_untracked=True)
manifest_rev = project.GetRevisionId(project.bare_ref.all)
head_rev = project.work_git.rev_parse(HEAD)
has_local_commits = manifest_rev != head_rev
if not (is_dirty or has_local_commits):
return None
output = project.bare_git.count_objects("-v")
except Exception:
return None
stats = {}
for line in output.splitlines():
try:
key, value = line.split(": ", 1)
stats[key.strip()] = int(value.strip())
except ValueError:
pass
pack_count = stats.get("packs", 0)
size_pack_kb = stats.get("size-pack", 0)
size_garbage_kb = stats.get("size-garbage", 0)
is_fragmented = (
pack_count > _BLOAT_PACK_COUNT_THRESHOLD
and size_pack_kb > _BLOAT_SIZE_PACK_THRESHOLD_KB
)
has_excessive_garbage = (
size_garbage_kb > _BLOAT_SIZE_GARBAGE_THRESHOLD_KB
)
if is_fragmented or has_excessive_garbage:
return project.name
return None
def _CheckForBloatedProjects(self, projects, opt):
"""Check for shallow projects that are accumulating unoptimized data.
For projects with clone-depth="1" that are dirty (have local changes),
run 'git count-objects -v' and warn if the repository is accumulating
excessive pack files or garbage.
"""
# We only care about bloated projects if we have a git version that
# supports --no-auto-gc (2.23.0+) since what we use to disable auto-gc
# in Project._RemoteFetch.
if not git_require((2, 23, 0)):
return
projects = [p for p in projects if p.clone_depth]
if not projects:
return
bloated_projects = []
pm = Progress(
"Checking for bloat", len(projects), delay=False, quiet=opt.quiet
)
def _ProcessResults(pool, pm, results):
for result in results:
if result:
bloated_projects.append(result)
pm.update(msg="")
with self.ParallelContext():
self.get_parallel_context()["projects"] = projects
self.ExecuteInParallel(
opt.jobs,
self._CheckOneBloatedProject,
range(len(projects)),
callback=_ProcessResults,
output=pm,
chunksize=1,
)
pm.end()
for project_name in bloated_projects:
warn_msg = (
f'warning: Project "{project_name}" is accumulating '
'unoptimized data. Please run "repo sync --auto-gc" or '
'"repo gc --repack" to clean up.'
)
self.git_event_log.ErrorEvent(warn_msg)
logger.warning(warn_msg)
def _UpdateRepoProject(self, opt, manifest, errors):
"""Fetch the repo project and check for updates."""
if opt.local_only:
@@ -2002,6 +2110,9 @@ later is required to fix a server side protocol bug.
"experience, sync the entire tree."
)
if existing:
self._CheckForBloatedProjects(all_projects, opt)
if not opt.quiet:
print("repo sync has finished successfully.")

184
subcmds/wipe.py Normal file
View File

@@ -0,0 +1,184 @@
# Copyright (C) 2025 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
from typing import List
from command import Command
from error import GitError
from error import RepoExitError
import platform_utils
from project import DeleteWorktreeError
class Error(RepoExitError):
"""Exit error when wipe command fails."""
class Wipe(Command):
"""Delete projects from the worktree and .repo"""
COMMON = True
helpSummary = "Wipe projects from the worktree"
helpUsage = """
%prog <project>...
"""
helpDescription = """
The '%prog' command removes the specified projects from the worktree
(the checked out source code) and deletes the project's git data from `.repo`.
This is a destructive operation and cannot be undone.
Projects can be specified either by name, or by a relative or absolute path
to the project's local directory.
Examples:
# Wipe the project "platform/build" by name:
$ repo wipe platform/build
# Wipe the project at the path "build/make":
$ repo wipe build/make
"""
def _Options(self, p):
# TODO(crbug.com/gerrit/393383056): Add --broken option to scan and
# wipe broken projects.
p.add_option(
"-f",
"--force",
action="store_true",
help="force wipe shared projects and uncommitted changes",
)
p.add_option(
"--force-uncommitted",
action="store_true",
help="force wipe even if there are uncommitted changes",
)
p.add_option(
"--force-shared",
action="store_true",
help="force wipe even if the project shares an object directory",
)
def ValidateOptions(self, opt, args: List[str]):
if not args:
self.Usage()
def Execute(self, opt, args: List[str]):
# Get all projects to handle shared object directories.
all_projects = self.GetProjects(None, all_manifests=True, groups="all")
projects_to_wipe = self.GetProjects(args, all_manifests=True)
relpaths_to_wipe = {p.relpath for p in projects_to_wipe}
# Build a map from objdir to the relpaths of projects that use it.
objdir_map = {}
for p in all_projects:
objdir_map.setdefault(p.objdir, set()).add(p.relpath)
uncommitted_projects = []
shared_objdirs = {}
objdirs_to_delete = set()
for project in projects_to_wipe:
if project == self.manifest.manifestProject:
raise Error(
f"error: cannot wipe the manifest project: {project.name}"
)
try:
if project.HasChanges():
uncommitted_projects.append(project.name)
except GitError:
uncommitted_projects.append(f"{project.name} (corrupted)")
users = objdir_map.get(project.objdir, {project.relpath})
is_shared = not users.issubset(relpaths_to_wipe)
if is_shared:
shared_objdirs.setdefault(project.objdir, set()).update(users)
else:
objdirs_to_delete.add(project.objdir)
block_uncommitted = uncommitted_projects and not (
opt.force or opt.force_uncommitted
)
block_shared = shared_objdirs and not (opt.force or opt.force_shared)
if block_uncommitted or block_shared:
error_messages = []
if block_uncommitted:
error_messages.append(
"The following projects have uncommitted changes or are "
"corrupted:\n"
+ "\n".join(f" - {p}" for p in sorted(uncommitted_projects))
)
if block_shared:
shared_dir_messages = []
for objdir, users in sorted(shared_objdirs.items()):
other_users = users - relpaths_to_wipe
projects_to_wipe_in_dir = users & relpaths_to_wipe
message = f"""Object directory {objdir} is shared by:
Projects to be wiped: {', '.join(sorted(projects_to_wipe_in_dir))}
Projects not to be wiped: {', '.join(sorted(other_users))}"""
shared_dir_messages.append(message)
error_messages.append(
"The following projects have shared object directories:\n"
+ "\n".join(sorted(shared_dir_messages))
)
if block_uncommitted and block_shared:
error_messages.append(
"Use --force to wipe anyway, or --force-uncommitted and "
"--force-shared to specify."
)
elif block_uncommitted:
error_messages.append("Use --force-uncommitted to wipe anyway.")
else:
error_messages.append("Use --force-shared to wipe anyway.")
raise Error("\n\n".join(error_messages))
# If we are here, either there were no issues, or --force was used.
# Proceed with wiping.
successful_wipes = set()
for project in projects_to_wipe:
try:
# Force the delete here since we've already performed our
# own safety checks above.
project.DeleteWorktree(force=True, verbose=opt.verbose)
successful_wipes.add(project.relpath)
except DeleteWorktreeError as e:
print(
f"error: failed to wipe {project.name}: {e}",
file=sys.stderr,
)
# Clean up object directories only if all projects using them were
# successfully wiped.
for objdir in objdirs_to_delete:
users = objdir_map.get(objdir, set())
# Check if every project that uses this objdir has been
# successfully processed. If a project failed to be wiped, don't
# delete the object directory, or we'll corrupt the remaining
# project.
if users.issubset(successful_wipes):
if os.path.exists(objdir):
if opt.verbose:
print(
f"Deleting objects directory: {objdir}",
file=sys.stderr,
)
platform_utils.rmtree(objdir)

View File

@@ -15,6 +15,7 @@
"""Unittests for the manifest_xml.py module."""
import os
from pathlib import Path
import platform
import re
import tempfile
@@ -97,36 +98,34 @@ class ManifestParseTestCase(unittest.TestCase):
def setUp(self):
self.tempdirobj = tempfile.TemporaryDirectory(prefix="repo_tests")
self.tempdir = self.tempdirobj.name
self.repodir = os.path.join(self.tempdir, ".repo")
self.manifest_dir = os.path.join(self.repodir, "manifests")
self.manifest_file = os.path.join(
self.repodir, manifest_xml.MANIFEST_FILE_NAME
self.tempdir = Path(self.tempdirobj.name)
self.repodir = self.tempdir / ".repo"
self.manifest_dir = self.repodir / "manifests"
self.manifest_file = self.repodir / manifest_xml.MANIFEST_FILE_NAME
self.local_manifest_dir = (
self.repodir / manifest_xml.LOCAL_MANIFESTS_DIR_NAME
)
self.local_manifest_dir = os.path.join(
self.repodir, manifest_xml.LOCAL_MANIFESTS_DIR_NAME
)
os.mkdir(self.repodir)
os.mkdir(self.manifest_dir)
self.repodir.mkdir()
self.manifest_dir.mkdir()
# The manifest parsing really wants a git repo currently.
gitdir = os.path.join(self.repodir, "manifests.git")
os.mkdir(gitdir)
with open(os.path.join(gitdir, "config"), "w") as fp:
fp.write(
"""[remote "origin"]
gitdir = self.repodir / "manifests.git"
gitdir.mkdir()
(gitdir / "config").write_text(
"""[remote "origin"]
url = https://localhost:0/manifest
"""
)
)
def tearDown(self):
self.tempdirobj.cleanup()
def getXmlManifest(self, data):
"""Helper to initialize a manifest for testing."""
with open(self.manifest_file, "w", encoding="utf-8") as fp:
fp.write(data)
return manifest_xml.XmlManifest(self.repodir, self.manifest_file)
self.manifest_file.write_text(data, encoding="utf-8")
return manifest_xml.XmlManifest(
str(self.repodir), str(self.manifest_file)
)
@staticmethod
def encodeXmlAttr(attr):
@@ -243,12 +242,14 @@ class XmlManifestTests(ManifestParseTestCase):
def test_link(self):
"""Verify Link handling with new names."""
manifest = manifest_xml.XmlManifest(self.repodir, self.manifest_file)
with open(os.path.join(self.manifest_dir, "foo.xml"), "w") as fp:
fp.write("<manifest></manifest>")
manifest = manifest_xml.XmlManifest(
str(self.repodir), str(self.manifest_file)
)
(self.manifest_dir / "foo.xml").write_text("<manifest></manifest>")
manifest.Link("foo.xml")
with open(self.manifest_file) as fp:
self.assertIn('<include name="foo.xml" />', fp.read())
self.assertIn(
'<include name="foo.xml" />', self.manifest_file.read_text()
)
def test_toxml_empty(self):
"""Verify the ToXml() helper."""
@@ -406,10 +407,9 @@ class IncludeElementTests(ManifestParseTestCase):
def test_revision_default(self):
"""Check handling of revision attribute."""
root_m = os.path.join(self.manifest_dir, "root.xml")
with open(root_m, "w") as fp:
fp.write(
"""
root_m = self.manifest_dir / "root.xml"
root_m.write_text(
"""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
@@ -418,17 +418,34 @@ class IncludeElementTests(ManifestParseTestCase):
<project name="root-name2" path="root-path2" />
</manifest>
"""
)
with open(os.path.join(self.manifest_dir, "stable.xml"), "w") as fp:
fp.write(
"""
)
(self.manifest_dir / "stable.xml").write_text(
"""
<manifest>
<include name="man1.xml" />
<include name="man2.xml" revision="stable-branch2" />
<project name="stable-name1" path="stable-path1" />
<project name="stable-name2" path="stable-path2" revision="stable-branch2" />
</manifest>
"""
)
include_m = manifest_xml.XmlManifest(self.repodir, root_m)
)
(self.manifest_dir / "man1.xml").write_text(
"""
<manifest>
<project name="man1-name1" />
<project name="man1-name2" revision="stable-branch3" />
</manifest>
"""
)
(self.manifest_dir / "man2.xml").write_text(
"""
<manifest>
<project name="man2-name1" />
<project name="man2-name2" revision="stable-branch3" />
</manifest>
"""
)
include_m = manifest_xml.XmlManifest(str(self.repodir), str(root_m))
for proj in include_m.projects:
if proj.name == "root-name1":
# Check include revision not set on root level proj.
@@ -442,12 +459,19 @@ class IncludeElementTests(ManifestParseTestCase):
if proj.name == "stable-name2":
# Check stable proj revision can override include node.
self.assertEqual("stable-branch2", proj.revisionExpr)
if proj.name == "man1-name1":
self.assertEqual("stable-branch", proj.revisionExpr)
if proj.name == "man1-name2":
self.assertEqual("stable-branch3", proj.revisionExpr)
if proj.name == "man2-name1":
self.assertEqual("stable-branch2", proj.revisionExpr)
if proj.name == "man2-name2":
self.assertEqual("stable-branch3", proj.revisionExpr)
def test_group_levels(self):
root_m = os.path.join(self.manifest_dir, "root.xml")
with open(root_m, "w") as fp:
fp.write(
"""
root_m = self.manifest_dir / "root.xml"
root_m.write_text(
"""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
@@ -456,25 +480,23 @@ class IncludeElementTests(ManifestParseTestCase):
<project name="root-name2" path="root-path2" groups="r2g1,r2g2" />
</manifest>
"""
)
with open(os.path.join(self.manifest_dir, "level1.xml"), "w") as fp:
fp.write(
"""
)
(self.manifest_dir / "level1.xml").write_text(
"""
<manifest>
<include name="level2.xml" groups="level2-group" />
<project name="level1-name1" path="level1-path1" />
</manifest>
"""
)
with open(os.path.join(self.manifest_dir, "level2.xml"), "w") as fp:
fp.write(
"""
)
(self.manifest_dir / "level2.xml").write_text(
"""
<manifest>
<project name="level2-name1" path="level2-path1" groups="l2g1,l2g2" />
</manifest>
"""
)
include_m = manifest_xml.XmlManifest(self.repodir, root_m)
)
include_m = manifest_xml.XmlManifest(str(self.repodir), str(root_m))
for proj in include_m.projects:
if proj.name == "root-name1":
# Check include group not set on root level proj.
@@ -492,6 +514,41 @@ class IncludeElementTests(ManifestParseTestCase):
# Check level2 proj group not removed.
self.assertIn("l2g1", proj.groups)
def test_group_levels_with_extend_project(self):
root_m = self.manifest_dir / "root.xml"
root_m.write_text(
"""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
<include name="man1.xml" groups="top-group1" />
<include name="man2.xml" groups="top-group2" />
</manifest>
"""
)
(self.manifest_dir / "man1.xml").write_text(
"""
<manifest>
<project name="project1" path="project1" />
</manifest>
"""
)
(self.manifest_dir / "man2.xml").write_text(
"""
<manifest>
<extend-project name="project1" groups="eg1" />
</manifest>
"""
)
include_m = manifest_xml.XmlManifest(str(self.repodir), str(root_m))
proj = include_m.projects[0]
# Check project has inherited group via project element.
self.assertIn("top-group1", proj.groups)
# Check project has inherited group via extend-project element.
self.assertIn("top-group2", proj.groups)
# Check project has set group via extend-project element.
self.assertIn("eg1", proj.groups)
def test_allow_bad_name_from_user(self):
"""Check handling of bad name attribute from the user's input."""
@@ -510,9 +567,8 @@ class IncludeElementTests(ManifestParseTestCase):
manifest.ToXml()
# Setup target of the include.
target = os.path.join(self.tempdir, "target.xml")
with open(target, "w") as fp:
fp.write("<manifest></manifest>")
target = self.tempdir / "target.xml"
target.write_text("<manifest></manifest>")
# Include with absolute path.
parse(os.path.abspath(target))
@@ -526,12 +582,9 @@ class IncludeElementTests(ManifestParseTestCase):
def parse(name):
name = self.encodeXmlAttr(name)
# Setup target of the include.
with open(
os.path.join(self.manifest_dir, "target.xml"),
"w",
encoding="utf-8",
) as fp:
fp.write(f'<manifest><include name="{name}"/></manifest>')
(self.manifest_dir / "target.xml").write_text(
f'<manifest><include name="{name}"/></manifest>'
)
manifest = self.getXmlManifest(
"""
@@ -578,18 +631,18 @@ class ProjectElementTests(ManifestParseTestCase):
manifest.projects[0].name: manifest.projects[0].groups,
manifest.projects[1].name: manifest.projects[1].groups,
}
self.assertCountEqual(
result["test-name"], ["name:test-name", "all", "path:test-path"]
self.assertEqual(
result["test-name"], {"name:test-name", "all", "path:test-path"}
)
self.assertCountEqual(
self.assertEqual(
result["extras"],
["g1", "g2", "g1", "name:extras", "all", "path:path"],
{"g1", "g2", "name:extras", "all", "path:path"},
)
groupstr = "default,platform-" + platform.system().lower()
self.assertEqual(groupstr, manifest.GetGroupsStr())
self.assertEqual(groupstr, manifest.GetManifestGroupsStr())
groupstr = "g1,g2,g1"
manifest.manifestProject.config.SetString("manifest.groups", groupstr)
self.assertEqual(groupstr, manifest.GetGroupsStr())
self.assertEqual(groupstr, manifest.GetManifestGroupsStr())
def test_set_revision_id(self):
"""Check setting of project's revisionId."""
@@ -1214,6 +1267,166 @@ class ExtendProjectElementTests(ManifestParseTestCase):
self.assertEqual(len(manifest.projects), 1)
self.assertEqual(manifest.projects[0].upstream, "bar")
def test_extend_project_copyfiles(self):
manifest = self.getXmlManifest(
"""
<manifest>
<remote name="default-remote" fetch="http://localhost" />
<default remote="default-remote" revision="refs/heads/main" />
<project name="myproject" />
<extend-project name="myproject">
<copyfile src="foo" dest="bar" />
</extend-project>
</manifest>
"""
)
self.assertEqual(list(manifest.projects[0].copyfiles)[0].src, "foo")
self.assertEqual(list(manifest.projects[0].copyfiles)[0].dest, "bar")
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
'<remote fetch="http://localhost" name="default-remote"/>'
'<default remote="default-remote" revision="refs/heads/main"/>'
'<project name="myproject">'
'<copyfile dest="bar" src="foo"/>'
"</project>"
"</manifest>",
)
def test_extend_project_duplicate_copyfiles(self):
root_m = self.manifest_dir / "root.xml"
root_m.write_text(
"""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
<project name="myproject" />
<include name="man1.xml" />
<include name="man2.xml" />
</manifest>
"""
)
(self.manifest_dir / "man1.xml").write_text(
"""
<manifest>
<include name="common.xml" />
</manifest>
"""
)
(self.manifest_dir / "man2.xml").write_text(
"""
<manifest>
<include name="common.xml" />
</manifest>
"""
)
(self.manifest_dir / "common.xml").write_text(
"""
<manifest>
<extend-project name="myproject">
<copyfile dest="bar" src="foo"/>
</extend-project>
</manifest>
"""
)
manifest = manifest_xml.XmlManifest(str(self.repodir), str(root_m))
self.assertEqual(len(manifest.projects[0].copyfiles), 1)
self.assertEqual(list(manifest.projects[0].copyfiles)[0].src, "foo")
self.assertEqual(list(manifest.projects[0].copyfiles)[0].dest, "bar")
def test_extend_project_linkfiles(self):
manifest = self.getXmlManifest(
"""
<manifest>
<remote name="default-remote" fetch="http://localhost" />
<default remote="default-remote" revision="refs/heads/main" />
<project name="myproject" />
<extend-project name="myproject">
<linkfile src="foo" dest="bar" />
</extend-project>
</manifest>
"""
)
self.assertEqual(list(manifest.projects[0].linkfiles)[0].src, "foo")
self.assertEqual(list(manifest.projects[0].linkfiles)[0].dest, "bar")
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
'<remote fetch="http://localhost" name="default-remote"/>'
'<default remote="default-remote" revision="refs/heads/main"/>'
'<project name="myproject">'
'<linkfile dest="bar" src="foo"/>'
"</project>"
"</manifest>",
)
def test_extend_project_duplicate_linkfiles(self):
root_m = self.manifest_dir / "root.xml"
root_m.write_text(
"""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
<project name="myproject" />
<include name="man1.xml" />
<include name="man2.xml" />
</manifest>
"""
)
(self.manifest_dir / "man1.xml").write_text(
"""
<manifest>
<include name="common.xml" />
</manifest>
"""
)
(self.manifest_dir / "man2.xml").write_text(
"""
<manifest>
<include name="common.xml" />
</manifest>
"""
)
(self.manifest_dir / "common.xml").write_text(
"""
<manifest>
<extend-project name="myproject">
<linkfile dest="bar" src="foo"/>
</extend-project>
</manifest>
"""
)
manifest = manifest_xml.XmlManifest(str(self.repodir), str(root_m))
self.assertEqual(len(manifest.projects[0].linkfiles), 1)
self.assertEqual(list(manifest.projects[0].linkfiles)[0].src, "foo")
self.assertEqual(list(manifest.projects[0].linkfiles)[0].dest, "bar")
def test_extend_project_annotations(self):
manifest = self.getXmlManifest(
"""
<manifest>
<remote name="default-remote" fetch="http://localhost" />
<default remote="default-remote" revision="refs/heads/main" />
<project name="myproject" />
<extend-project name="myproject">
<annotation name="foo" value="bar" />
</extend-project>
</manifest>
"""
)
self.assertEqual(manifest.projects[0].annotations[0].name, "foo")
self.assertEqual(manifest.projects[0].annotations[0].value, "bar")
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
'<remote fetch="http://localhost" name="default-remote"/>'
'<default remote="default-remote" revision="refs/heads/main"/>'
'<project name="myproject">'
'<annotation name="foo" value="bar"/>'
"</project>"
"</manifest>",
)
class NormalizeUrlTests(ManifestParseTestCase):
"""Tests for normalize_url() in manifest_xml.py"""

263
tests/test_subcmds_wipe.py Normal file
View File

@@ -0,0 +1,263 @@
# Copyright (C) 2025 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import shutil
from unittest import mock
import pytest
import project
from subcmds import wipe
def _create_mock_project(tempdir, name, objdir_path=None, has_changes=False):
"""Creates a mock project with necessary attributes and directories."""
worktree = os.path.join(tempdir, name)
gitdir = os.path.join(tempdir, ".repo/projects", f"{name}.git")
if objdir_path:
objdir = objdir_path
else:
objdir = os.path.join(tempdir, ".repo/project-objects", f"{name}.git")
os.makedirs(worktree, exist_ok=True)
os.makedirs(gitdir, exist_ok=True)
os.makedirs(objdir, exist_ok=True)
proj = project.Project(
manifest=mock.MagicMock(),
name=name,
remote=mock.MagicMock(),
gitdir=gitdir,
objdir=objdir,
worktree=worktree,
relpath=name,
revisionExpr="main",
revisionId="abcd",
)
proj.HasChanges = mock.MagicMock(return_value=has_changes)
def side_effect_delete_worktree(force=False, verbose=False):
if os.path.exists(proj.worktree):
shutil.rmtree(proj.worktree)
if os.path.exists(proj.gitdir):
shutil.rmtree(proj.gitdir)
return True
proj.DeleteWorktree = mock.MagicMock(
side_effect=side_effect_delete_worktree
)
return proj
def _run_wipe(all_projects, projects_to_wipe_names, options=None):
"""Helper to run the Wipe command with mocked projects."""
cmd = wipe.Wipe()
cmd.manifest = mock.MagicMock()
def get_projects_mock(projects, all_manifests=False, **kwargs):
if projects is None:
return all_projects
names_to_find = set(projects)
return [p for p in all_projects if p.name in names_to_find]
cmd.GetProjects = mock.MagicMock(side_effect=get_projects_mock)
if options is None:
options = []
opts = cmd.OptionParser.parse_args(options + projects_to_wipe_names)[0]
cmd.CommonValidateOptions(opts, projects_to_wipe_names)
cmd.ValidateOptions(opts, projects_to_wipe_names)
cmd.Execute(opts, projects_to_wipe_names)
def test_wipe_single_unshared_project(tmp_path):
"""Test wiping a single project that is not shared."""
p1 = _create_mock_project(str(tmp_path), "project/one")
_run_wipe([p1], ["project/one"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p1.objdir)
def test_wipe_multiple_unshared_projects(tmp_path):
"""Test wiping multiple projects that are not shared."""
p1 = _create_mock_project(str(tmp_path), "project/one")
p2 = _create_mock_project(str(tmp_path), "project/two")
_run_wipe([p1, p2], ["project/one", "project/two"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p1.objdir)
assert not os.path.exists(p2.worktree)
assert not os.path.exists(p2.gitdir)
assert not os.path.exists(p2.objdir)
def test_wipe_shared_project_no_force_raises_error(tmp_path):
"""Test that wiping a shared project without --force raises an error."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path), "project/one", objdir_path=shared_objdir
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
with pytest.raises(wipe.Error) as e:
_run_wipe([p1, p2], ["project/one"])
assert "shared object directories" in str(e.value)
assert "project/one" in str(e.value)
assert "project/two" in str(e.value)
assert os.path.exists(p1.worktree)
assert os.path.exists(p1.gitdir)
assert os.path.exists(p2.worktree)
assert os.path.exists(p2.gitdir)
assert os.path.exists(shared_objdir)
def test_wipe_shared_project_with_force(tmp_path):
"""Test wiping a shared project with --force."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path), "project/one", objdir_path=shared_objdir
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
_run_wipe([p1, p2], ["project/one"], options=["--force"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert os.path.exists(shared_objdir)
assert os.path.exists(p2.worktree)
assert os.path.exists(p2.gitdir)
def test_wipe_all_sharing_projects(tmp_path):
"""Test wiping all projects that share an object directory."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path), "project/one", objdir_path=shared_objdir
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
_run_wipe([p1, p2], ["project/one", "project/two"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p2.worktree)
assert not os.path.exists(p2.gitdir)
assert not os.path.exists(shared_objdir)
def test_wipe_with_uncommitted_changes_raises_error(tmp_path):
"""Test wiping a project with uncommitted changes raises an error."""
p1 = _create_mock_project(str(tmp_path), "project/one", has_changes=True)
with pytest.raises(wipe.Error) as e:
_run_wipe([p1], ["project/one"])
assert "uncommitted changes" in str(e.value)
assert "project/one" in str(e.value)
assert os.path.exists(p1.worktree)
assert os.path.exists(p1.gitdir)
assert os.path.exists(p1.objdir)
def test_wipe_with_uncommitted_changes_with_force(tmp_path):
"""Test wiping a project with uncommitted changes with --force."""
p1 = _create_mock_project(str(tmp_path), "project/one", has_changes=True)
_run_wipe([p1], ["project/one"], options=["--force"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p1.objdir)
def test_wipe_uncommitted_and_shared_raises_combined_error(tmp_path):
"""Test that uncommitted and shared projects raise a combined error."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path),
"project/one",
objdir_path=shared_objdir,
has_changes=True,
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
with pytest.raises(wipe.Error) as e:
_run_wipe([p1, p2], ["project/one"])
assert "uncommitted changes" in str(e.value)
assert "shared object directories" in str(e.value)
assert "project/one" in str(e.value)
assert "project/two" in str(e.value)
assert os.path.exists(p1.worktree)
assert os.path.exists(p1.gitdir)
assert os.path.exists(p2.worktree)
assert os.path.exists(p2.gitdir)
assert os.path.exists(shared_objdir)
def test_wipe_shared_project_with_force_shared(tmp_path):
"""Test wiping a shared project with --force-shared."""
shared_objdir = os.path.join(
str(tmp_path), ".repo/project-objects", "shared.git"
)
p1 = _create_mock_project(
str(tmp_path), "project/one", objdir_path=shared_objdir
)
p2 = _create_mock_project(
str(tmp_path), "project/two", objdir_path=shared_objdir
)
_run_wipe([p1, p2], ["project/one"], options=["--force-shared"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert os.path.exists(shared_objdir)
assert os.path.exists(p2.worktree)
assert os.path.exists(p2.gitdir)
def test_wipe_with_uncommitted_changes_with_force_uncommitted(tmp_path):
"""Test wiping uncommitted changes with --force-uncommitted."""
p1 = _create_mock_project(str(tmp_path), "project/one", has_changes=True)
_run_wipe([p1], ["project/one"], options=["--force-uncommitted"])
assert not os.path.exists(p1.worktree)
assert not os.path.exists(p1.gitdir)
assert not os.path.exists(p1.objdir)