* load data inside background tasks
Perform collection.LoadComplete inside maybeRunTaskInBackground
Have tasks use a fresh copy of taskCollectionFactory, taskCollection
* use uuids, since names can be renamed
* remove useless resource lock
Resource locks need to be before the background task. creating same publish endpoint at the same time is unlikely...
* load data inside background tasks
This fixes a flaw in async apis, which loaded the published repo from the DB and mutated it outside the task closure, before the task lock was acquired.
Perform collection.LoadComplete inside maybeRunTaskInBackground and have tasks use a fresh copy of taskCollectionFactory, taskCollection
* lock source repos/snapshots for publish operations
Concurrent tasks were not properly locking their resources, leading to inconsistent published indexes:
SourceLocalRepo: iterate published.Sources (component -> source UUID), look up each local repo via localRepoCollection.ByUUID and append string(repo.Key()) to resources
SourceSnapshot: iterate b.Snapshots,look up each snapshot via snapshotCollection.ByName and append string(snapshot.ResourceKey()) to resources.
* lock pool on non MultiDist publish
* revert mutex on LinkFromPool
* use uuids, since names can be renamed
* add test for MultiDist change
Implement support for the SOURCE_DATE_EPOCH environment variable as
specified by reproducible-builds.org. When set, this variable overrides
the current timestamp in the Release file's Date and Valid-Until fields,
enabling reproducible filesystem publishes.
- Read SOURCE_DATE_EPOCH environment variable in Publish()
- Use the epoch timestamp for both Date and Valid-Until fields
- Gracefully fallback to current time if unset or invalid
- Add comprehensive tests for valid and invalid SOURCE_DATE_EPOCH values
Enabling coverage near-doubles the incremental build time and adds
overhead to individual tests on the order of **5-10x** or more. It's not
essential to have this for quick local system-test runs, so add an option
to disable it.
When aptly crashes it is possible to get a corrupt database with a dangling key reference.
This results in an error with 'key not found', eg:
ERROR: unable to load package Pall example-package 1.2.3 778cf6f877bf6e2d: key not found
This change makes `db recover` fix this situation by removing the dangling references.
sort both aptly output and gold file. output original output for
debugging on failure.
* Makefile: enable CAPTURE=1 env variable for capturing gold files
* docker-system-test: use AWS env vars for S3 tests
* fix system tests timing issue with order of gpg logs in publish tests
The fix of the -with-filter flag causes the following previously
missing source files to be downloaded, so I updated the test file.
```
rkward_0.7.5-1~bullseyecran.0.debian.tar.xz
rkward_0.7.5-1~bullseyecran.0.dsc
rkward_0.7.5.orig.tar.gz
rpy2_3.5.12-1~bullseyecran.0.debian.tar.xz
rpy2_3.5.12-1~bullseyecran.0.dsc
rpy2_3.5.12.orig.tar.gz
```
André Roth
Tim Foerster
Ales Bregar
Ryan Gonzalez
Itay Porezky
Silke Hofstra
Gordian Schoenherr
Mauro Regli