Downloads set req.URL.Opaque to keep "+" percent-encoded in the path.
URL.String() prints "scheme:opaque" and drops the host, so every error
from the http client showed a URL such as "https:/repos/debian/...",
lacking a host name. The request itself was correct, but the message
sent me to look for a malformed URL.
Set req.URL.RawPath instead. The escaping still reaches the wire through
EscapedPath(), and String() keeps the scheme and the host.
The proxy special case is no longer necessary. Requests through a proxy
now get the same "+" escaping as direct requests.
- use s3 mirror instead of internet download
- reduce download verbosity
- do not use venv in docker-system-tests
- be more verbose on test output
- do not run golangci-lint in system-tests
Use CDN-backed Debian mirror to make tests run faster hopefully for
everyone. Redirects might be important to know what exactly is going on
when items are being downloaded.
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.
Unwrap `errors.Wrap` which is used in downloader.
Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
There are two fixes here:
1. Abort package download immediately as ^C is pressed.
2. Import all the already downloaded files into package pool,
so that next time mirror is updated, aptly won't download them
once again.
* Drop multi-threaded downloader. It doesn't really belong here -
some places require it, some do not, but it's definitely not the
right place to handle it, as it's being used only when updating
mirrors
* Pass expectedChecksums as pointer, so it's easy to drive `nil` value,
and also downloader can fill back checksums (not implemented right now).
* Break down downloader and tests into more files
* Use pkg/errors instead of fmt
Amazon S3 replies with a 403 when accessing files that don't exist,
while this should be fixed at Amazon, we can workaround it in aptly
for the moment.
Filipe Brandenburger
Ato Araki
JupiterRider
André Roth
iofq
Mauro Regli
Markus Muellner
Michael Stürmer
Lorenzo Bolla
Oliver Sauder
Andrey Smirnov
Raphael Medaer
Benoit Foucher
Szymon Sobik
Rohan Garg