For any action which is multi-step (requires updating more than 1 DB
key), use transaction to make update atomic.
Also pack big chunks of updates (importing packages for importing and
mirror updates) into single transaction to improve aptly performance and
get some isolation.
Note that still layers up (Collections) provide some level of isolation,
so this is going to shine with the future PRs to remove collection
locks.
Spin-off of #459
- new publish calls can now enable AcquireByHash by right away (previously
one would have had to create a new publishing endpoint and then
explicitly switch it to AcquireByHash)
- all json marshals of PublishedRepo now contain AcquireByHash (allows
inspecting if a given endpoint has AcquireByHash enabled already; also
enables verification that a switch/update actually applied a
potential AcquireByHash change
- update all tests to reflect that default state of AcquireByHash
- update creation and switch testing to explicitly toggle AcquireByHash to
make sure state mutation works as expected
LoadComplete() modifies object, so it would cause issues if it runs
concurrently with other methods. Uprage mutex locks to write
locks when LoadComplete() is being used.
The added "aptly publish repo" option "-access-by-hash" publishes
the index files (Packages*, Sources*) also as hardlinked hashes.
Example:
/dists/yakkety/main/binary-amd64/by-hash/SHA512/31833ec39acc...
The Release files indicate this with the option "Acquire-By-Hash: yes"
This is used by apt >= 1.2.0 and prevents the "Hash sum mismatch" race
condition between a server side "aptly publish repo" and "apt-get update"
on a client.
See: http://www.chiark.greenend.org.uk/~cjwatson/blog/no-more-hash-sum-mismatch-errors.html
This implementation uses symlinks in the by-hash/*/ directory for keeping
only two versions of the index files and deleting older files
automatically.
Note: this only works with aptly.FileSystemPublishedStorage
Closes: #536
Signed-off-by: André Roth <neolynx@gmail.com>
Allow skipping unreferenced files cleanup on publish switch/update/drop
via the -skip-cleanup command line option.
Also support API SkipCleanup parameter.
Fixes#570.
Allow database to be initialized without opening, unify all the
open paths to retry on failure.
In API router make sure open requests are matched with acks in explicit
way.
This also enables re-open attempts in all the aptly commands, so it
should make running aptly CLI much easier now hopefully.
Fix up system tests for oldoldstable ;)
When DB fails to be open, aptly was skipping "close" phase, so that next
request considered database to be still open (while it's closed) leading
to panic.
Fixes: #431
