Acquire-By-Hash entries are written for MD5Sum, SHA1, SHA256 and SHA512,
hardcoded in two places in deb/index_files.go. Every algorithm is a full
set of index copies on each publish, and on S3-like storage each of those
is a network round trip rather than a link.
Measured on an R2-backed archive of 25 packages across three suites, one
`aptly publish update` per suite:
Acquire-By-Hash off 38 S3 requests
Acquire-By-Hash on 222 S3 requests (88 HeadObject, 83 CopyObject,
35 DeleteObject, 13 PutObject)
Modern apt verifies with SHA256; MD5Sum and SHA1 are deprecated and
Debian has been phasing them out of Release files for years. Publishers
who do not need them are paying three quarters of that for nothing, with
no way to say so.
acquireByHashAlgorithms in aptly.conf now selects the set. An empty or
absent value keeps all four, so existing configurations are unaffected.
The order entries are written in comes from the code rather than the
config, so the directories created in one loop and the entries written in
another cannot disagree. An unrecognised name is an error rather than
something to skip: skipping a typo would produce "no by-hash entries for
that algorithm", which looks exactly like having configured it away.
Set per publish invocation from the config, the way SkipBz2 already is,
rather than persisted on the publish point.
Pre-existing test failures on this tree are unchanged by this commit:
deb goes 173->177 passed with the same 11 failures, utils keeps its one
(TestCompress), s3 stays green.
https://wiki.debian.org/DebianRepository/Format#Signed-By says:
> **Signed-By**
> An optional field containing a comma separated list of
> OpenPGP key fingerprints to be used for validating
> the next Release file. The fingerprints must consist
> only of hex digits and may not contain spaces.
> The fingerprint specifies either the key the Release file
> must be signed with or the key the signature key must be
> a subkey of. The later match can be disabled by appending
> an exclamation mark to the fingerprint.
>
> If the field is present, a client should only accept future updates
> to the repository that are signed with keys listed in the field.
> The field should be ignored if the Valid-Until field is not present
> or if it is expired.
For both the CLI tools and JSON, the field is taken as a string verbatim.
When specified, we must also provide `Valid-Until` field,
and i'm not sure there is an 'infinity' value for it,
so 100 years will have to do?
Fixes https://github.com/aptly-dev/aptly/issues/1497
When aptly crashes it is possible to get a corrupt database with a dangling key reference.
This results in an error with 'key not found', eg:
ERROR: unable to load package Pall example-package 1.2.3 778cf6f877bf6e2d: key not found
This change makes `db recover` fix this situation by removing the dangling references.
This commit allows to add, remove and update components of published repositories without the need to recreate them.
Signed-off-by: Christoph Fiehe <c.fiehe@eurodata.de>
Hansjoerg Maurer
Rascalov
Martin Simon
Luan Dang
André Roth
Philip Cramer
Zhang Xiao
Ales Bregar
Roman Lebedev
Juan Calderon-Perez
Silke Hofstra
Gordian Schoenherr
Christof Warlich
Mauro Regli
iofq
Christoph Fiehe