This patch adds validation to the version pasing logic (parseVersion) to
not allow packages to be registered which apt wouldn't accept anyway.
It then propogates validation errors through CompareVersions, the way
in which parseVersion gets called, so that appropriate decisions can be
made.
New packages with invalid versions are not accepted.
For any pre-exisiting packages with invalid versions, as much care is
taken as possible to log warnings, and not use those packages.
This PR is loosely bnased on one by by Aaron Foster
<afoster@cloudflare.com> though it has largely been written at this
point.
Signed off with internal and external emails for clarity, but commit
is under personal email where we typically do open source contributions
from.
Signed-off-by: Phil Dibowitz <pdibowitz@cloudflare.com>
Signed-off-by: Phil Dibowitz <pdibowitz@ipom.com>
Debian co-signs each suite's InRelease with more than one archive key, so
gpgv reports a good signature from the key you trust and NO_PUBKEY for the
others, exiting 2. Both verifiers treat that as failure, which means
mirroring Debian requires importing every key Debian happens to sign with.
apt does not require this: it is fatal only when there is no good signature
or a BADSIG.
Adopt that rule for unknown signers. When gpgv exits non-zero the status
stream is parsed per NEWSIG block and the result is a success only when the
process exited, at least one block is a good signature from a held key, at
least one block is an unknown issuer (ERRSIG reason 9 with NO_PUBKEY), and
nothing else appears. BADSIG, EXPSIG, EXPKEYSIG and REVKEYSIG from a held
key still fail, as does any unexplained failure. The exit-zero path is
unchanged.
KEYEXPIRED and KEYREVOKED are deliberately not disqualifiers: doc/DETAILS
notes KEYEXPIRED is emitted for an expired subkey even when that subkey was
not used, so treating it as a verdict rejects valid signatures.
checkDetachedSignature gets the equivalent change for the internal verifier.
Downloads set req.URL.Opaque to keep "+" percent-encoded in the path.
URL.String() prints "scheme:opaque" and drops the host, so every error
from the http client showed a URL such as "https:/repos/debian/...",
lacking a host name. The request itself was correct, but the message
sent me to look for a malformed URL.
Set req.URL.RawPath instead. The escaping still reaches the wire through
EscapedPath(), and String() keeps the scheme and the host.
The proxy special case is no longer necessary. Requests through a proxy
now get the same "+" escaping as direct requests.
codecov/patch flagged the change at 75% of diff hit. The uncovered lines
were the two the existing test could not reach: the fallthrough when no
metadata key matches, and the HeadObject error return.
Both are worth pinning regardless of the coverage gate.
packageIndexByHash calls ReadLink on a path it has only just confirmed
exists and treats an error as "leave it alone", so "this is not a link"
and "I could not find out" have to stay distinguishable: the first is an
empty string with a nil error, the second is an error.
ReadLink goes to 100%.
S3 lowercases user metadata keys. SymLink writes "SymLink" and ReadLink
looks up output.Metadata["SymLink"], but the value comes back under
"symlink", so the lookup misses and ReadLink returns an empty string with
a nil error. To the caller that reads as "this object is not a link"
rather than as a failure.
deb.packageIndexByHash uses ReadLink to find the physical file its .old
pointer refers to, and removes it before rotating:
linkTarget, err = publishedStorage.ReadLink(oldIndexPath)
if err == nil {
_ = publishedStorage.Remove(linkTarget)
}
With "" returned the Remove is a no-op and its error is discarded, so the
superseded by-hash index file is never deleted. The pointers rotate
correctly and the physical files accumulate: one per index file per hash
algorithm per publish, indefinitely. Filesystem backends are unaffected
because they have real symlinks.
Measured on a live R2-backed archive publishing three suites a few times
a day: dists/ held 711 objects, of which 672 were by-hash and only 39
were current indexes, growing by about 48 objects per publish since
Acquire-By-Hash was enabled.
The existing TestSymLink cannot catch this, because it ends in
c.Skip("copy not available in s3test") and never runs. The regression
test added here writes the metadata with PutObject instead, so it needs
no CopyObject support, and fails before this change with
obtained string = "".
André Roth
James Munson
Twaik Yont
Phil Dibowitz
Daniel Pinder
dependabot[bot]
Filipe Brandenburger
adam magued
Andreas Vögele
mmayabi
Martin Simon
Eleni Michalaki