Lorenzo Bolla
b5d90b7b13
Add more logging
2022-01-31 10:32:54 +01:00
Lorenzo Bolla
3e06af8515
Disable checksum for now
2022-01-31 10:32:54 +01:00
Lorenzo Bolla
eff2e56d0d
Specify output filename instead of directory
...
"temp" downloader uses its own naming for downloaded files.
2022-01-31 10:32:54 +01:00
Lorenzo Bolla
eaac04ccf6
Improve logging in grab downloader
2022-01-31 10:32:54 +01:00
Lorenzo Bolla
894192851e
Grab downloader
2022-01-31 10:32:54 +01:00
Lorenzo Bolla
3775d69a60
Fix linting errors
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
6826efc723
Fix pure-go unittests
...
So they can run on e.g. LXC containers as root, or other conceivable setups.
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
787cc8e3ee
Fix system tests
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
fd83c1a5bf
Cap delay to sleep to avoid overflow
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
de2be9b8ae
Sleep between retries to download from http
...
Fix #1
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
19f7b0fe8d
mirror: increase logging for easier debugging
2022-01-27 09:30:14 +01:00
André Roth
c0528888f4
log download retries
2022-01-27 09:30:14 +01:00
Oliver Sauder
f09a273ad7
Add publish output progress counting remaining number of packages
2022-01-27 09:30:14 +01:00
Oliver Sauder
6ab5e60833
Add task api and resource locking ability
2022-01-27 09:30:14 +01:00
Oliver Sauder
25d7d7c037
Solving progress not safe issue for api
...
Progress is not safe so for api its always nil and
code needs to take care of this
2022-01-27 09:30:14 +01:00
Lorenzo Bolla
f4dc87fa44
Use a hostname more likely to be non-existent than localhost
...
Otherwise, it's possible that certain network configuration defining
*.localhost cause the tests to fail.
2021-02-08 10:52:27 +01:00
Andrey Smirnov
769e984ef4
Fix issues with progress == nil causing panics
...
Part of PR #459
This prepares for more methods to be exposed via the API.
2019-09-03 20:28:28 +04:00
Andrey Smirnov
26098f6c8d
Print redirects being followed, drop mirror.yandex.ru.
...
Use CDN-backed Debian mirror to make tests run faster hopefully for
everyone. Redirects might be important to know what exactly is going on
when items are being downloaded.
2019-08-07 21:10:04 +03:00
Andrey Smirnov
f0a370db24
Rework HTTP downloader retry logic
...
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.
Unwrap `errors.Wrap` which is used in downloader.
Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
2019-08-07 20:23:05 +03:00
Andrey Smirnov
3b5840e248
Fix linter list and fix errors discovered by new staticcheck
2019-01-20 00:01:17 +03:00
Oliver Sauder
b1a2523ef0
Add unit test for remote and http
2018-07-06 15:02:37 +02:00
Oliver Sauder
108b0ea226
Add support to mirror non package installer files
2018-07-06 15:02:37 +02:00
Andrey Smirnov
b8c5303fdb
Fix paths after repository transfer to aptly-dev
2018-04-18 21:19:43 +03:00
Andrey Smirnov
15618c8ea8
Use Go context to abort gracefully mirror updates
...
There are two fixes here:
1. Abort package download immediately as ^C is pressed.
2. Import all the already downloaded files into package pool,
so that next time mirror is updated, aptly won't download them
once again.
2017-11-30 00:49:37 +03:00
Andrey Smirnov
cafb89f30f
Re-work the way checksum matching works against Release file
...
Break up URL into base part and relative path. Match checksum against relative path
and never against full URL.
This might be fixing security issue if aptly was incorrectly matching against
wrong part of Release file.
2017-05-23 03:00:15 +03:00
Andrey Smirnov
f0360cf2d3
Use longest suffix match to pick up checksum
2017-05-23 03:00:15 +03:00
Andrey Smirnov
51213899b7
More Go linters enabled, issues fixed
...
Ref: #528
Enables "staticcheck", "varcheck", "structcheck", "aligncheck"
2017-05-03 18:23:14 +03:00
Andrey Smirnov
1f3cb2db5d
When downloading/importing packages, enforce all checksums
2017-04-26 23:17:04 +03:00
Andrey Smirnov
94b49818a1
Refactor HTTP downloader package
...
* Drop multi-threaded downloader. It doesn't really belong here -
some places require it, some do not, but it's definitely not the
right place to handle it, as it's being used only when updating
mirrors
* Pass expectedChecksums as pointer, so it's easy to drive `nil` value,
and also downloader can fill back checksums (not implemented right now).
* Break down downloader and tests into more files
* Use pkg/errors instead of fmt
2017-04-26 23:17:03 +03:00
Andrey Smirnov
516dd7b044
Switch to gometalinter
...
Only small amount of required checks is enabled,
plan is to enable more linters as issues are fixed in the code.
2017-03-23 01:51:08 +03:00
Andrey Smirnov
11d828b3b1
Add govet/golint into Travis CI build
...
Fix current issues
2017-03-22 21:49:16 +03:00
Andrey Smirnov
acd8d4a6ea
Go 1.6 compatibility
2017-03-17 00:17:07 +03:00
Andrey Smirnov
c9768416ed
Fix up tests
2017-03-16 23:06:41 +03:00
Raphael Medaer
9cfe1307e3
Added download tests for xz compression.
2017-03-16 22:41:25 +03:00
Raphael Medaer
db8595711b
Added '.xz' reader in HTTP download. This fixed issue when you try to mirror a distribution with only Packages.xz (example 'debian/experimental').
2017-03-16 22:41:25 +03:00
Oliver Sauder
af71b9541c
Fixing typo
2016-11-29 09:00:16 +01:00
Oliver Sauder
f31b5ec3f8
Adjusted test with new maxTries param for download
2016-11-28 17:02:24 +01:00
Oliver Sauder
6becd5a3aa
Added max-tries flag for mirror update
2016-11-28 17:02:24 +01:00
Benoit Foucher
7f5a7323a6
Fixed mkdir mode from 755 to 777
2016-03-18 09:37:18 +00:00
Andrey Smirnov
77be7b9e3b
Support for SHA-512 hashes on publishing/downloads.
2016-02-18 12:01:51 +03:00
Andrey Smirnov
7bb052ac37
Fix unit-tests. #324
2015-12-24 14:08:37 +03:00
Andrey Smirnov
631fe44c6b
Security: don't download files we don't have checksums for. #324
2015-12-22 13:52:53 +03:00
Szymon Sobik
4c51350517
fix EOF during mirror update
...
see http://stackoverflow.com/a/19006050
2015-06-19 12:36:28 +02:00
Andrey Smirnov
1845c493f4
Update mxk/flowcontrol package from Google Code to mxk/flowrate from GitHub.
2015-03-15 14:00:04 +03:00
Andrey Smirnov
903d4cefba
gofmt -s
2015-02-22 14:29:09 +03:00
Andrey Smirnov
1b0eb9d45a
Attempt to fix #189 and #130 : disable Amazon workaround when using proxy.
2015-02-03 21:49:55 +03:00
Chris Read
daf887e54f
Upgrade gocheck
2014-11-05 13:27:15 -06:00
Andrey Smirnov
552b11e28d
Workaround for '+' escaping in Amazon S3. #130
2014-11-05 02:46:01 +03:00
Andrey Smirnov
4c1d6d1463
Fix operator precedence. #131
2014-10-26 19:53:49 +03:00
Rohan Garg
140a11c04a
Continue even when a server replies with 403
...
Amazon S3 replies with a 403 when accessing files that don't exist,
while this should be fixed at Amazon, we can workaround it in aptly
for the moment.
2014-10-25 21:41:57 +02:00
