The added "aptly publish repo" option "-access-by-hash" publishes
the index files (Packages*, Sources*) also as hardlinked hashes.
Example:
/dists/yakkety/main/binary-amd64/by-hash/SHA512/31833ec39acc...
The Release files indicate this with the option "Acquire-By-Hash: yes"
This is used by apt >= 1.2.0 and prevents the "Hash sum mismatch" race
condition between a server side "aptly publish repo" and "apt-get update"
on a client.
See: http://www.chiark.greenend.org.uk/~cjwatson/blog/no-more-hash-sum-mismatch-errors.html
This implementation uses symlinks in the by-hash/*/ directory for keeping
only two versions of the index files and deleting older files
automatically.
Note: this only works with aptly.FileSystemPublishedStorage
Closes: #536
Signed-off-by: André Roth <neolynx@gmail.com>
There are two fixes here:
1. Abort package download immediately as ^C is pressed.
2. Import all the already downloaded files into package pool,
so that next time mirror is updated, aptly won't download them
once again.
Allow skipping unreferenced files cleanup on publish switch/update/drop
via the -skip-cleanup command line option.
Also support API SkipCleanup parameter.
Fixes#570.
Allow database to be initialized without opening, unify all the
open paths to retry on failure.
In API router make sure open requests are matched with acks in explicit
way.
This also enables re-open attempts in all the aptly commands, so it
should make running aptly CLI much easier now hopefully.
Fix up system tests for oldoldstable ;)
This requires splitting up import file phase as separate step in then end,
it should be pretty fast, as it only does file move (hardlink) and
DB update for new checksums.
`PackageDownloadTask` is just a reference to file now. Whole process
was rewritten to follow pattern: download to temp location inside the pool,
verify/update checksums, import into pool as final step.
This removes a lot of edge cases when aptly internal state might be broken
if updating from rogue mirror.
Also this changes whole memory model: package list/files are kept in memory
now during the duration of `mirror update` command and saved to disk
only in the end.
This is related to #506
As a first step, don't pass MD5 explicitly, pass checksum info object,
so that as a next step we can choose which hash to use.
There should be no functional changes so far.
Next step: stop returning explicit paths from public package pool.
systemd has a feature called socket activation where initially systemd
manages and listens on ports/uds and only invokes a service when traffic
appears. to then hand over the involved sockets, systemd will pass the
relevant FDs into the invoked process and defines them in the environment.
use coreos/go-systemd to grab the active listeners passed by systemd and
use them to serve the api routes. only one listener may be specified right
now as we also only support one -listen argument for the binary.
this allows admins to craft a systemd socket and service file for aptly
where systemd manages the socket, its permission and its live time, and
lazy start aptly when needed.
`unix://$PATH` as listen argument will bind aptly to a unix domain socket
rather than TCP.
This allows binding the API to a UDS rather than a port.
Since aptly has no concept of authentication or any amount of high level
API hardening one needs to bottle it up in some other manner. Binding
to a localhost port is often a step in the right direction, ultimately is
still a scary insecure setup as any user on that host getting compromised
would mean that the entire archive is compromised as well.
UDS on the other hand are basically files and have their access managed
by regular file permission. As such, binding to a socket is in fact
the least insecure way to listen as you'd have to explicitly open up the
socket permissions to an access qualified group. In the most conservative
scenario that means no one but the aptly user can talk to the API, in a
more practical setup apache might get access as well and proxy the UDS
with authentication or limited to GET operations.
Using UDS allows reducing the attack surface of the API server while
preserving all the flexibility.
Oliver Sauder
Andrey Smirnov
André Roth
Stefan
Stefan Felkel
Ringo De Smet
Ludovico Cavedon
Clemens Rabe
Harald Sitter
jola5
jola5