Downloads set req.URL.Opaque to keep "+" percent-encoded in the path.
URL.String() prints "scheme:opaque" and drops the host, so every error
from the http client showed a URL such as "https:/repos/debian/...",
lacking a host name. The request itself was correct, but the message
sent me to look for a malformed URL.
Set req.URL.RawPath instead. The escaping still reaches the wire through
EscapedPath(), and String() keeps the scheme and the host.
The proxy special case is no longer necessary. Requests through a proxy
now get the same "+" escaping as direct requests.
codecov/patch flagged the change at 75% of diff hit. The uncovered lines
were the two the existing test could not reach: the fallthrough when no
metadata key matches, and the HeadObject error return.
Both are worth pinning regardless of the coverage gate.
packageIndexByHash calls ReadLink on a path it has only just confirmed
exists and treats an error as "leave it alone", so "this is not a link"
and "I could not find out" have to stay distinguishable: the first is an
empty string with a nil error, the second is an error.
ReadLink goes to 100%.
S3 lowercases user metadata keys. SymLink writes "SymLink" and ReadLink
looks up output.Metadata["SymLink"], but the value comes back under
"symlink", so the lookup misses and ReadLink returns an empty string with
a nil error. To the caller that reads as "this object is not a link"
rather than as a failure.
deb.packageIndexByHash uses ReadLink to find the physical file its .old
pointer refers to, and removes it before rotating:
linkTarget, err = publishedStorage.ReadLink(oldIndexPath)
if err == nil {
_ = publishedStorage.Remove(linkTarget)
}
With "" returned the Remove is a no-op and its error is discarded, so the
superseded by-hash index file is never deleted. The pointers rotate
correctly and the physical files accumulate: one per index file per hash
algorithm per publish, indefinitely. Filesystem backends are unaffected
because they have real symlinks.
Measured on a live R2-backed archive publishing three suites a few times
a day: dists/ held 711 objects, of which 672 were by-hash and only 39
were current indexes, growing by about 48 objects per publish since
Acquire-By-Hash was enabled.
The existing TestSymLink cannot catch this, because it ends in
c.Skip("copy not available in s3test") and never runs. The regression
test added here writes the metadata with PutObject instead, so it needs
no CopyObject support, and fails before this change with
obtained string = "".
Regression test for the duplicated skel payload: publishing two
components used to append each skeleton file once per component. Fails
against the previous code with doubled file contents, mirroring the
PublishRepo40Test system test at unit level.
Publish moved the skel and AppStream emission out of the per-component
loop, and codecov flags the relocated error branches as uncovered. Cover
them with deterministic failures:
- a regular file in the middle of the skel path makes the walk fail
with ENOTDIR, which is not swallowed as a not-exist error
- nested skel/AppStream names each within NAME_MAX flatten into a
temporary index file name that exceeds it, failing index creation
- an unreadable skel file (mode 0) fails the open (non-root only)
- a ~1MB filesystem as TMPDIR runs out of space mid-write, reusing the
/smallfs convention from DiskFullSuite
Also add a unit-level happy path test publishing nested skel files and
checking they land in the published tree and the Release checksums.
SkelIndex was the only index constructor not propagating acquireByHash,
so with -acquire-by-hash the Release file advertises Acquire-By-Hash: yes
while no by-hash/ entries exist for these files and apt gets a 404.
The skeleton and AppStream loops were nested inside the per-component
loop while iterating over all components themselves, so every file was
written once per component. Publishing main + contrib appended each
payload twice. Release checksums are computed after the fact, so the
duplicated content is signed and apt accepts it silently.
in the API, snapshotPull never finished reading the target repository
within the async block. So the target was treated as an empty
snapshot, and a pull would only carry over the content from the sources
Make sure pathCache is properly locked for concurrent access.
Add RWMutex to the PublishedStorage struct:
- Cache initialization
Read-lock to test for nil, then write-lock with a second nil check before populating
- Cache reads
RLock/RUnlock, allowing concurrent readers
- Cache writes / deletes
Lock/Unlock
André Roth
dependabot[bot]
Filipe Brandenburger
adam magued
Andreas Vögele
mmayabi
Martin Simon
Eleni Michalaki
KereMath
Thomas Berger
C0deSentin3l