Make sure pathCache is properly locked for concurrent access.
Add RWMutex to the PublishedStorage struct:
- Cache initialization
Read-lock to test for nil, then write-lock with a second nil check before populating
- Cache reads
RLock/RUnlock, allowing concurrent readers
- Cache writes / deletes
Lock/Unlock
* show resources in task details
* fix task state locking
* return task object consistently
Race condition iexisted where task State, err, and processReturnValue fields
were written by consumer goroutine and read by concurrent accessors without
proper synchronization, causing torn reads and data races.
* load data inside background tasks
Perform collection.LoadComplete inside maybeRunTaskInBackground
Have tasks use a fresh copy of taskCollectionFactory, taskCollection
* perform collection.LoadComplete inside maybeRunTaskInBackground
* have tasks use a fresh copy of taskCollectionFactory, taskCollection
* fix locking for snapshots of snapshots by locking SourceSnapshots
* use uuids, since names can be renamed
* load data inside background tasks
Perform collection.LoadComplete inside maybeRunTaskInBackground
Have tasks use a fresh copy of taskCollectionFactory, taskCollection
* use uuids, since names can be renamed
* remove useless resource lock
Resource locks need to be before the background task. creating same publish endpoint at the same time is unlikely...
* load data inside background tasks
This fixes a flaw in async apis, which loaded the published repo from the DB and mutated it outside the task closure, before the task lock was acquired.
Perform collection.LoadComplete inside maybeRunTaskInBackground and have tasks use a fresh copy of taskCollectionFactory, taskCollection
* lock source repos/snapshots for publish operations
Concurrent tasks were not properly locking their resources, leading to inconsistent published indexes:
SourceLocalRepo: iterate published.Sources (component -> source UUID), look up each local repo via localRepoCollection.ByUUID and append string(repo.Key()) to resources
SourceSnapshot: iterate b.Snapshots,look up each snapshot via snapshotCollection.ByName and append string(snapshot.ResourceKey()) to resources.
* lock pool on non MultiDist publish
* revert mutex on LinkFromPool
* use uuids, since names can be renamed
* add test for MultiDist change
Implement support for the SOURCE_DATE_EPOCH environment variable as
specified by reproducible-builds.org. When set, this variable overrides
the current timestamp in the Release file's Date and Valid-Until fields,
enabling reproducible filesystem publishes.
- Read SOURCE_DATE_EPOCH environment variable in Publish()
- Use the epoch timestamp for both Date and Valid-Until fields
- Gracefully fallback to current time if unset or invalid
- Add comprehensive tests for valid and invalid SOURCE_DATE_EPOCH values
Initially found by automated repository health checks used by Termux
in https://github.com/termux/termux-packages/issues/27472
The root problem was 4.3.5a comparing less than 4.3.5-rc1-1 by aptly
According to debian "4.3.5a" > "4.3.5-rc1-1"
This is because dpkg splits hyphen for revision at the first hyphen,
whereas aptly was splitting at the last hyphen which is different from
dpkg's behaviour.
dpkg behaviour: https://git.dpkg.org/cgit/dpkg/dpkg.git/tree/lib/dpkg/parsehelp.c#n242
Perhaps this wasn't detected as there was broken tests in the repository
since the initial commit of aptly. This also fixes those tests
Enabling coverage near-doubles the incremental build time and adds
overhead to individual tests on the order of **5-10x** or more. It's not
essential to have this for quick local system-test runs, so add an option
to disable it.
When using rootless podman, the *current user* gets mapped to uid 0,
which results in the aptly user being unable to write to the build
directory. We can instead map the current user to the corresponding uid
in the container via `PODMAN_USERNS=keep-id`, which matches up with what
docker-wrapper wants...but then that will *enter the container as the
current uid*, which messes with the ability to set permissions on
`/var/lib/aptly`. That can be fixed by explicitly passing `--user 0:0`,
which should be a no-op on docker (since the container's default user is
already root).
Additionally, this adds `--security-opt label=disable` to avoid
permission errors when running on systems with SELinux enforcing.
André Roth
Catalin Muresan
Russell Greene
Tim Foerster
Ales Bregar
Brian Witt
Yaksh Bariya
Tobias Assarsson
Ryan Gonzalez
chesseed
JupiterRider
Alejandro Guijarro Monerris