Commit Graph

91 Commits

Author SHA1 Message Date
André Roth f8d2d3cb8d fix lint errors 2026-04-26 17:41:12 +02:00
Ato Araki af483d1165 feat(http): add GCP authentication for ar+https scheme 2025-12-17 12:57:17 +09:00
JupiterRider b49a631e0b ran "gofmt -s -w ." to format the code 2025-08-20 19:41:26 +02:00
André Roth f7057a9517 go1.24: fix lint, unit and system tests
- development env: base on debian trixie with go1.24
- lint: run with default config
- fix lint errors
- fix unit tests
- fix system test
2025-04-26 13:29:50 +02:00
iofq 840b76228a add shutdown context unit test 2024-11-09 15:34:35 -06:00
iofq 8436001d5b Make HTTP server wait for tasks before shutdown 2024-11-08 14:06:23 -06:00
André Roth f0ad0f9496 improve and test grab downloader 2024-10-04 13:37:56 +02:00
iofq 372ce3c4bc Avoid nil panic when downloadSpeedLimit is set in api mode 2024-08-16 10:04:46 +02:00
iofq ff8a02959c fix throttled downloader 2024-08-11 10:20:37 +02:00
Jens Reinsberger 4bc2180eed fix failing build on hosts with wildcard dns
on hosts which have wildcard dns domains in their local domain search
list, builds failed because "nosuch.host" could actually be resolved.

Since ".host" isn't a recommended TLD by RFC2606, we use ".invalid" now.
And since this is not enough to fix the problem, we use now absoulte
domain names (having a '.' at the end)
2024-05-15 16:42:37 +02:00
André Roth 9deb031c44 fix system tests
- use s3 mirror instead of internet download
- reduce download verbosity
- do not use venv in docker-system-tests
- be more verbose on test output
- do not run golangci-lint in system-tests
2024-04-11 10:15:02 +02:00
Mauro Regli ae61706a34 Fix: Implement golangci-lint suggestions 2023-09-21 11:25:18 +02:00
Markus Muellner 8e62195eb5 implement structured logging 2023-02-20 13:42:50 +01:00
Markus Muellner 352f4e8772 update golangci-lint and replace deprecated calls to io/ioutil 2022-12-12 10:21:39 +01:00
Michael Stürmer 4b2efeec7a Cope with zero-length http downloads 2022-06-20 09:47:41 +02:00
Markus Muellner 6539e1b856 Add metrics endpoint with http metrics using Prometheus client lib 2022-04-12 14:39:16 +02:00
Lorenzo Bolla 551a370c13 Basic tests for Grab downloader 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 1afcd68e01 Make downloader type configurable 2022-01-31 10:32:54 +01:00
Lorenzo Bolla cc30ef3ee2 Remove vendor directory
We use go modules now.
2022-01-31 10:32:54 +01:00
Lorenzo Bolla 235e35a2f3 Rate limit 0 effectively disables rate limiting 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 4c54f967b7 Fix error checking 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 8925949be3 Support rate limiting in grab downloader 2022-01-31 10:32:54 +01:00
Lorenzo Bolla e96372c999 Implement ignore checksum mismatch
Also, update "pkg/errors" library.
2022-01-31 10:32:54 +01:00
Lorenzo Bolla e5d9d27069 Wrap errors with more context 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 853c990b6e Handle checksums 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 86c1ffab2a Add logs for checksum 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 952287a787 Reenable checksums 2022-01-31 10:32:54 +01:00
Lorenzo Bolla b5d90b7b13 Add more logging 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 3e06af8515 Disable checksum for now 2022-01-31 10:32:54 +01:00
Lorenzo Bolla eff2e56d0d Specify output filename instead of directory
"temp" downloader uses its own naming for downloaded files.
2022-01-31 10:32:54 +01:00
Lorenzo Bolla eaac04ccf6 Improve logging in grab downloader 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 894192851e Grab downloader 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 3775d69a60 Fix linting errors 2022-01-27 09:30:14 +01:00
Lorenzo Bolla 6826efc723 Fix pure-go unittests
So they can run on e.g. LXC containers as root, or other conceivable setups.
2022-01-27 09:30:14 +01:00
Lorenzo Bolla 787cc8e3ee Fix system tests 2022-01-27 09:30:14 +01:00
Lorenzo Bolla fd83c1a5bf Cap delay to sleep to avoid overflow 2022-01-27 09:30:14 +01:00
Lorenzo Bolla de2be9b8ae Sleep between retries to download from http
Fix #1
2022-01-27 09:30:14 +01:00
Lorenzo Bolla 19f7b0fe8d mirror: increase logging for easier debugging 2022-01-27 09:30:14 +01:00
André Roth c0528888f4 log download retries 2022-01-27 09:30:14 +01:00
Oliver Sauder f09a273ad7 Add publish output progress counting remaining number of packages 2022-01-27 09:30:14 +01:00
Oliver Sauder 6ab5e60833 Add task api and resource locking ability 2022-01-27 09:30:14 +01:00
Oliver Sauder 25d7d7c037 Solving progress not safe issue for api
Progress is not safe so for api its always nil and
code needs to take care of this
2022-01-27 09:30:14 +01:00
Lorenzo Bolla f4dc87fa44 Use a hostname more likely to be non-existent than localhost
Otherwise, it's possible that certain network configuration defining
*.localhost cause the tests to fail.
2021-02-08 10:52:27 +01:00
Andrey Smirnov 769e984ef4 Fix issues with progress == nil causing panics
Part of PR #459

This prepares for more methods to be exposed via the API.
2019-09-03 20:28:28 +04:00
Andrey Smirnov 26098f6c8d Print redirects being followed, drop mirror.yandex.ru.
Use CDN-backed Debian mirror to make tests run faster hopefully for
everyone. Redirects might be important to know what exactly is going on
when items are being downloaded.
2019-08-07 21:10:04 +03:00
Andrey Smirnov f0a370db24 Rework HTTP downloader retry logic
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.

Unwrap `errors.Wrap` which is used in downloader.

Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
2019-08-07 20:23:05 +03:00
Andrey Smirnov 3b5840e248 Fix linter list and fix errors discovered by new staticcheck 2019-01-20 00:01:17 +03:00
Oliver Sauder b1a2523ef0 Add unit test for remote and http 2018-07-06 15:02:37 +02:00
Oliver Sauder 108b0ea226 Add support to mirror non package installer files 2018-07-06 15:02:37 +02:00
Andrey Smirnov b8c5303fdb Fix paths after repository transfer to aptly-dev 2018-04-18 21:19:43 +03:00