Ato Araki
af483d1165
feat(http): add GCP authentication for ar+https scheme
2025-12-17 12:57:17 +09:00
JupiterRider
b49a631e0b
ran "gofmt -s -w ." to format the code
2025-08-20 19:41:26 +02:00
André Roth
f7057a9517
go1.24: fix lint, unit and system tests
...
- development env: base on debian trixie with go1.24
- lint: run with default config
- fix lint errors
- fix unit tests
- fix system test
2025-04-26 13:29:50 +02:00
iofq
8436001d5b
Make HTTP server wait for tasks before shutdown
2024-11-08 14:06:23 -06:00
iofq
372ce3c4bc
Avoid nil panic when downloadSpeedLimit is set in api mode
2024-08-16 10:04:46 +02:00
iofq
ff8a02959c
fix throttled downloader
2024-08-11 10:20:37 +02:00
André Roth
9deb031c44
fix system tests
...
- use s3 mirror instead of internet download
- reduce download verbosity
- do not use venv in docker-system-tests
- be more verbose on test output
- do not run golangci-lint in system-tests
2024-04-11 10:15:02 +02:00
Mauro Regli
ae61706a34
Fix: Implement golangci-lint suggestions
2023-09-21 11:25:18 +02:00
Markus Muellner
352f4e8772
update golangci-lint and replace deprecated calls to io/ioutil
2022-12-12 10:21:39 +01:00
Michael Stürmer
4b2efeec7a
Cope with zero-length http downloads
2022-06-20 09:47:41 +02:00
Lorenzo Bolla
787cc8e3ee
Fix system tests
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
fd83c1a5bf
Cap delay to sleep to avoid overflow
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
de2be9b8ae
Sleep between retries to download from http
...
Fix #1
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
19f7b0fe8d
mirror: increase logging for easier debugging
2022-01-27 09:30:14 +01:00
André Roth
c0528888f4
log download retries
2022-01-27 09:30:14 +01:00
Oliver Sauder
6ab5e60833
Add task api and resource locking ability
2022-01-27 09:30:14 +01:00
Oliver Sauder
25d7d7c037
Solving progress not safe issue for api
...
Progress is not safe so for api its always nil and
code needs to take care of this
2022-01-27 09:30:14 +01:00
Andrey Smirnov
769e984ef4
Fix issues with progress == nil causing panics
...
Part of PR #459
This prepares for more methods to be exposed via the API.
2019-09-03 20:28:28 +04:00
Andrey Smirnov
26098f6c8d
Print redirects being followed, drop mirror.yandex.ru.
...
Use CDN-backed Debian mirror to make tests run faster hopefully for
everyone. Redirects might be important to know what exactly is going on
when items are being downloaded.
2019-08-07 21:10:04 +03:00
Andrey Smirnov
f0a370db24
Rework HTTP downloader retry logic
...
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.
Unwrap `errors.Wrap` which is used in downloader.
Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
2019-08-07 20:23:05 +03:00
Andrey Smirnov
3b5840e248
Fix linter list and fix errors discovered by new staticcheck
2019-01-20 00:01:17 +03:00
Oliver Sauder
108b0ea226
Add support to mirror non package installer files
2018-07-06 15:02:37 +02:00
Andrey Smirnov
b8c5303fdb
Fix paths after repository transfer to aptly-dev
2018-04-18 21:19:43 +03:00
Andrey Smirnov
15618c8ea8
Use Go context to abort gracefully mirror updates
...
There are two fixes here:
1. Abort package download immediately as ^C is pressed.
2. Import all the already downloaded files into package pool,
so that next time mirror is updated, aptly won't download them
once again.
2017-11-30 00:49:37 +03:00
Andrey Smirnov
1f3cb2db5d
When downloading/importing packages, enforce all checksums
2017-04-26 23:17:04 +03:00
Andrey Smirnov
94b49818a1
Refactor HTTP downloader package
...
* Drop multi-threaded downloader. It doesn't really belong here -
some places require it, some do not, but it's definitely not the
right place to handle it, as it's being used only when updating
mirrors
* Pass expectedChecksums as pointer, so it's easy to drive `nil` value,
and also downloader can fill back checksums (not implemented right now).
* Break down downloader and tests into more files
* Use pkg/errors instead of fmt
2017-04-26 23:17:03 +03:00
Andrey Smirnov
11d828b3b1
Add govet/golint into Travis CI build
...
Fix current issues
2017-03-22 21:49:16 +03:00
Andrey Smirnov
acd8d4a6ea
Go 1.6 compatibility
2017-03-17 00:17:07 +03:00
Andrey Smirnov
c9768416ed
Fix up tests
2017-03-16 23:06:41 +03:00
Raphael Medaer
db8595711b
Added '.xz' reader in HTTP download. This fixed issue when you try to mirror a distribution with only Packages.xz (example 'debian/experimental').
2017-03-16 22:41:25 +03:00
Oliver Sauder
af71b9541c
Fixing typo
2016-11-29 09:00:16 +01:00
Oliver Sauder
6becd5a3aa
Added max-tries flag for mirror update
2016-11-28 17:02:24 +01:00
Benoit Foucher
7f5a7323a6
Fixed mkdir mode from 755 to 777
2016-03-18 09:37:18 +00:00
Andrey Smirnov
77be7b9e3b
Support for SHA-512 hashes on publishing/downloads.
2016-02-18 12:01:51 +03:00
Andrey Smirnov
7bb052ac37
Fix unit-tests. #324
2015-12-24 14:08:37 +03:00
Andrey Smirnov
631fe44c6b
Security: don't download files we don't have checksums for. #324
2015-12-22 13:52:53 +03:00
Szymon Sobik
4c51350517
fix EOF during mirror update
...
see http://stackoverflow.com/a/19006050
2015-06-19 12:36:28 +02:00
Andrey Smirnov
1845c493f4
Update mxk/flowcontrol package from Google Code to mxk/flowrate from GitHub.
2015-03-15 14:00:04 +03:00
Andrey Smirnov
1b0eb9d45a
Attempt to fix #189 and #130 : disable Amazon workaround when using proxy.
2015-02-03 21:49:55 +03:00
Andrey Smirnov
552b11e28d
Workaround for '+' escaping in Amazon S3. #130
2014-11-05 02:46:01 +03:00
Andrey Smirnov
4c1d6d1463
Fix operator precedence. #131
2014-10-26 19:53:49 +03:00
Rohan Garg
140a11c04a
Continue even when a server replies with 403
...
Amazon S3 replies with a 403 when accessing files that don't exist,
while this should be fixed at Amazon, we can workaround it in aptly
for the moment.
2014-10-25 21:41:57 +02:00
Andrey Smirnov
7be2ef8b85
Don't fallback between compression methods available unless we get strictly HTTP 404. #129 #125
...
Prior to that, some real errors could have been masked away by that fallback.
2014-10-24 08:51:04 +04:00
Andrey Smirnov
1042894123
Abort downloader on shutdown, don't wait for downloads to finish. #45 #114
2014-10-03 01:31:38 +04:00
Andrey Smirnov
91c753ad2f
Add URL to all download errrors, so that they're easier to understand. #26
2014-10-02 14:12:43 +04:00
Andrey Smirnov
7ae785f5a3
Implementation of ftp:// support for downloading. #48
2014-08-29 19:37:10 +04:00
Andrey Smirnov
94114f2c3d
Use idiomatic chan struct{} when there's nothing to send.
2014-07-24 01:25:59 +04:00
Andrey Smirnov
2906369a3b
Reuse default HTTP transport options.
2014-07-24 01:17:58 +04:00
Andrey Smirnov
83ceee1e3f
Remove debug output. #62
2014-07-13 16:10:53 +04:00
Andrey Smirnov
fb1e28b91b
Setting for downloader to limit download speed to specified level. #62
2014-07-12 23:56:32 +04:00
