f9325fbc91
Add support for Azure package pools
...
This adds support for storing packages directly on Azure, with no truly
"local" (on-disk) repo used. The existing Azure PublishedStorage
implementation was refactored to move the shared code to a separate
context struct, which can then be re-used by the new PackagePool. In
addition, the files package's mockChecksumStorage was made public so
that it could be used in the Azure PackagePool tests as well.
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com >
2024-06-17 11:51:18 +02:00
8e37813129
Add support for custom package pool locations
...
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com >
2024-06-17 11:51:18 +02:00
0fdba29d51
make serving published repos in api mode configurable
2023-03-22 17:22:54 +01:00
8e62195eb5
implement structured logging
2023-02-20 13:42:50 +01:00
268c39ea8c
add forceVirtualHostedStyle for stores which only support virtual hosted style
2022-09-09 09:02:52 +02:00
f61514edaf
Allow disabling bzip2 compression for index files
...
Using bzip2 generates smaller index files (roughly 20% smaller Packages
files) but it comes with a big performance penalty. When publishing a
debian mirror snapshot (amd64, arm64, armhf, source) without contents
skipping bzip speeds things up around 1.8 times.
```
$ hyperfine -w 1 -L skip-bz2 true,false -m 3 -p "aptly -config aptly.conf publish drop bullseye || true" "aptly -config aptly.conf publish snapshot --skip-bz2={skip-bz2} --skip-contents --skip-signing bullseye"
Benchmark 1: aptly -config aptly.conf publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
Time (mean ± σ): 35.567 s ± 0.307 s [User: 39.366 s, System: 10.075 s]
Range (min … max): 35.311 s … 35.907 s 3 runs
Benchmark 2: aptly -config aptly.conf publish snapshot --skip-bz2=false --skip-contents --skip-signing bullseye
Time (mean ± σ): 64.740 s ± 0.135 s [User: 68.565 s, System: 10.129 s]
Range (min … max): 64.596 s … 64.862 s 3 runs
Summary
'aptly -config aptly.conf publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye' ran
1.82 ± 0.02 times faster than 'aptly -config aptly.conf publish snapshot --skip-bz2=false --skip-contents --skip-signing bullseye'
```
Allow skipping bz2 creation for setups where faster publishing is more
important then Package file size.
Signed-off-by: Sjoerd Simons <sjoerd@collabora.com >
2022-06-22 11:25:45 +02:00
152538ccc1
Support custom Azure publish endpoint
2022-04-25 11:41:04 +02:00
6539e1b856
Add metrics endpoint with http metrics using Prometheus client lib
2022-04-12 14:39:16 +02:00
1afcd68e01
Make downloader type configurable
2022-01-31 10:32:54 +01:00
9bf1a44f75
Fix test after merge
2022-01-27 09:30:14 +01:00
9b28d8984f
Configurable background task execution
2022-01-27 09:30:14 +01:00
bb42a2158d
Add support for Azure storage as a publishing backend
...
This adds a new configuration setting: AzurePublishEndpoints, similar
to the existing S3PublishEndpoints and SwiftPublishEndpoints.
For each endpoint, the following has to be defined:
- accountName
- accountKey
- container
- prefix
Azure tests require the following environment variables to be set:
- AZURE_STORAGE_ACCOUNT
- AZURE_STORAGE_ACCESS_KEY
With either of these not set, Azure-specific tests are skipped.
2022-01-21 11:46:36 +01:00
98e75f6d97
Make database open attempts configurable also via config file
2019-09-03 00:52:24 +03:00
f0a370db24
Rework HTTP downloader retry logic
...
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.
Unwrap `errors.Wrap` which is used in downloader.
Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
2019-08-07 20:23:05 +03:00
499ab35012
Implement flag/config falue for GPG provider
2017-07-21 01:01:58 +03:00
186bb2dff0
Add flag to disable/enable support for legacy pool paths
...
Legacy pool paths are enabled by default, but for new aptly installations
(when aptly config is first generated), it would be disabled explicitly.
2017-04-26 23:37:31 +03:00
25f9c29f00
Implemented filesystem endpoint with support for hardlinks, symlinks and copy.
2017-04-13 20:25:40 +02:00
85b4a8b1ae
Add new option for detailed logging on dependency resolving
...
This adds command-line arg and config option, with option enabled
aptly is more verbose on internal depeendency resolving cycles:
```
Missing dependencies: file-rc (>= 0.8.16) [amd64], python:any (>= 2.7.1-0ubuntu2) [amd64], python3:any (>= 3.3.2-2~) [amd64], file-rc [amd64], perl (<< 5.17) [amd64], iptables-router (>= 1.2.3) [amd64], systemd [amd64], sgml-base (>= 1.26+nmu2) [amd64], sed (>= 4.1.2-8) [amd64]
Unsatisfied dependency: file-rc (>= 0.8.16) [amd64]
Unsatisfied dependency: python:any (>= 2.7.1-0ubuntu2) [amd64]
Unsatisfied dependency: python3:any (>= 3.3.2-2~) [amd64]
Unsatisfied dependency: file-rc [amd64]
Unsatisfied dependency: perl (<< 5.17) [amd64]
Unsatisfied dependency: iptables-router (>= 1.2.3) [amd64]
Unsatisfied dependency: systemd [amd64]
Injecting package: sgml-base_1.26+nmu4ubuntu1_all
Injecting package: sed_4.2.2-4ubuntu1_amd64
```
2017-03-28 22:58:07 +03:00
8e62620880
Fixing tests after adding v3 swift auth
2016-09-14 20:53:01 +02:00
75b860e0b1
Support SigV2 and S3 debug for publishing.
2016-03-20 20:11:19 +03:00
f50e008763
Make 'SkipContents' configurable in API. #345
...
Also add global configuration to disable 'skipContents' by
default for all new published repos/snapshots.
2016-02-14 14:49:16 +03:00
02b937ad17
Fix unit-tests.
2016-02-08 14:42:30 +03:00
8cc7d1345b
Support for new S3 configuration options: endpoint & multi del disabling. #218
2015-04-17 01:18:34 +03:00
903d4cefba
gofmt -s
2015-02-22 14:29:09 +03:00
0f22dc590a
Fix config tests and update man page
...
Fix ConfigSuite.TestSaveConfig, ConfigShowTest and CreateConfigTest
tests
2015-02-05 21:27:09 +01:00
daf887e54f
Upgrade gocheck
2014-11-05 13:27:15 -06:00
aeae6009c4
Introduce plusWorkaround: generate copy of file with '+' -> ' ' to workaround S3/apt bug. #98
2014-10-01 21:32:56 +04:00
8aa1954ba7
Support for custom storage class and encryption method. #105
2014-10-01 19:16:15 +04:00
2c470c1535
Rename config option to endpoint. #15
2014-07-28 15:01:51 +04:00
7a7bb56557
Config options for S3 storage. #15
2014-07-18 17:37:08 +04:00
a54a366c95
New config setting: downloadSpeedLimit to limit download speed. #62
2014-07-13 15:47:44 +04:00
bed0ac475e
Short syntax for aptly mirror create to support ppa:user/project mirror URLs.
2014-03-06 21:06:50 +04:00
21123ac6a4
Add config option to follow dependency to source packages.
2014-02-17 15:08:21 +04:00
36446e46a1
Add config option 'downloadSourcePackages'.
2014-02-14 23:44:46 +04:00
09648044db
Test for new config options.
2014-02-06 21:58:40 +04:00
d684c87fd1
Add support for dependency & architectures as common options.
2014-01-12 23:55:58 +04:00
a7fb971905
Config file reading/writing.
2013-12-27 19:31:32 +04:00
Ryan Gonzalez
Markus Muellner
boxjan
Sjoerd Simons
Chuan Liu
Lorenzo Bolla
Frank Steinborn
Andrey Smirnov
Clemens Rabe
Einar Forselv
Sebastien Badia
Chris Read