Commit Graph

55 Commits

Author SHA1 Message Date
Ales Bregar 9a90038dd2 - #309 adding gpgKeys config key, accepting array of keyRef, cli args has precedence
- #691 adding handling of multiple keyRefs when signing with gpg
2026-01-24 10:55:15 +01:00
André Roth f7057a9517 go1.24: fix lint, unit and system tests
- development env: base on debian trixie with go1.24
- lint: run with default config
- fix lint errors
- fix unit tests
- fix system test
2025-04-26 13:29:50 +02:00
André Roth e50a5e175f update documentation and man page 2024-12-11 12:02:52 +01:00
André Roth a7d6782176 add test and improve config error messages 2024-12-11 12:02:52 +01:00
André Roth a15d8a3f7b add test 2024-12-11 12:02:52 +01:00
André Roth 4e566b4692 fix tests and lint 2024-12-11 12:02:52 +01:00
André Roth 87a36633e9 group and order config items
and fix lint
2024-12-11 12:02:52 +01:00
André Roth 0e0189f0eb use yaml config file 2024-12-11 12:02:52 +01:00
André Roth a880a88fc0 yaml config 2024-12-11 12:02:52 +01:00
André Roth ea80f6d49c write commented json default config 2024-12-11 10:40:44 +01:00
André Roth cc1fc7ccfe allow comments in config file 2024-12-11 10:40:43 +01:00
André Roth 1d1bd41bb8 add swagger support
- install swaggo
- add swagger config option
2024-10-01 01:07:09 +02:00
André Roth 32a3943821 support ~ in rootDir as home directory 2024-09-24 10:14:39 +02:00
André Roth f10acb3df8 etcd: fix db config and test
fix unit test for adapted config file
2024-07-31 22:16:00 +02:00
hudeng 59bf4501e8 feat: Use databaseBackend config repace databaseEtcd
databaseBackend config contains type and url sub config, It can facilitate the expansion of other types of databases in the future.
2024-07-31 22:16:00 +02:00
hudeng 78172d11d7 feat: Add etcd database support
improve concurrent access and high availability of aptly with the help of the characteristics of etcd
2024-07-31 22:16:00 +02:00
Ryan Gonzalez f9325fbc91 Add support for Azure package pools
This adds support for storing packages directly on Azure, with no truly
"local" (on-disk) repo used. The existing Azure PublishedStorage
implementation was refactored to move the shared code to a separate
context struct, which can then be re-used by the new PackagePool. In
addition, the files package's mockChecksumStorage was made public so
that it could be used in the Azure PackagePool tests as well.

Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
2024-06-17 11:51:18 +02:00
Ryan Gonzalez 8e37813129 Add support for custom package pool locations
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
2024-06-17 11:51:18 +02:00
Markus Muellner 0fdba29d51 make serving published repos in api mode configurable 2023-03-22 17:22:54 +01:00
Markus Muellner 8e62195eb5 implement structured logging 2023-02-20 13:42:50 +01:00
boxjan 268c39ea8c add forceVirtualHostedStyle for stores which only support virtual hosted style 2022-09-09 09:02:52 +02:00
Sjoerd Simons f61514edaf Allow disabling bzip2 compression for index files
Using bzip2 generates smaller index files (roughly 20% smaller Packages
files) but it comes with a big performance penalty.  When publishing a
debian mirror snapshot (amd64, arm64, armhf, source) without contents
skipping bzip speeds things up around 1.8 times.

```
$ hyperfine -w 1 -L skip-bz2 true,false  -m 3 -p "aptly -config aptly.conf publish drop bullseye || true" "aptly -config aptly.conf  publish snapshot  --skip-bz2={skip-bz2} --skip-contents --skip-signing bullseye"
Benchmark 1: aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     35.567 s ±  0.307 s    [User: 39.366 s, System: 10.075 s]
  Range (min … max):   35.311 s … 35.907 s    3 runs

Benchmark 2: aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye
  Time (mean ± σ):     64.740 s ±  0.135 s    [User: 68.565 s, System: 10.129 s]
  Range (min … max):   64.596 s … 64.862 s    3 runs

Summary
  'aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye' ran
    1.82 ± 0.02 times faster than 'aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye'
```

Allow skipping bz2 creation for setups where faster publishing is more
important then Package file size.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-22 11:25:45 +02:00
Chuan Liu 152538ccc1 Support custom Azure publish endpoint 2022-04-25 11:41:04 +02:00
Markus Muellner 6539e1b856 Add metrics endpoint with http metrics using Prometheus client lib 2022-04-12 14:39:16 +02:00
Lorenzo Bolla 1afcd68e01 Make downloader type configurable 2022-01-31 10:32:54 +01:00
Lorenzo Bolla 9b28d8984f Configurable background task execution 2022-01-27 09:30:14 +01:00
chuan bb42a2158d Add support for Azure storage as a publishing backend
This adds a new configuration setting: AzurePublishEndpoints, similar
to the existing S3PublishEndpoints and SwiftPublishEndpoints.

For each endpoint, the following has to be defined:
 - accountName
 - accountKey
 - container
 - prefix

Azure tests require the following environment variables to be set:
 - AZURE_STORAGE_ACCOUNT
 - AZURE_STORAGE_ACCESS_KEY

With either of these not set, Azure-specific tests are skipped.
2022-01-21 11:46:36 +01:00
Frank Steinborn 98e75f6d97 Make database open attempts configurable also via config file 2019-09-03 00:52:24 +03:00
Andrey Smirnov f0a370db24 Rework HTTP downloader retry logic
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.

Unwrap `errors.Wrap` which is used in downloader.

Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
2019-08-07 20:23:05 +03:00
Harald Sitter 46c2182ade fix linting by using new maligned linter instead of aligncheck
upstream switched the alignment check backend and in doing so fails to run
if the old backend is defined in the config.

also skip alignment linting on a struct we use for byte decoding as we have
no choice in its member order.
2017-10-31 12:24:31 +01:00
Andrey Smirnov 499ab35012 Implement flag/config falue for GPG provider 2017-07-21 01:01:58 +03:00
Andrey Smirnov 51213899b7 More Go linters enabled, issues fixed
Ref: #528

Enables "staticcheck", "varcheck", "structcheck", "aligncheck"
2017-05-03 18:23:14 +03:00
Andrey Smirnov 186bb2dff0 Add flag to disable/enable support for legacy pool paths
Legacy pool paths are enabled by default, but for new aptly installations
(when aptly config is first generated), it would be disabled explicitly.
2017-04-26 23:37:31 +03:00
Clemens Rabe 25f9c29f00 Implemented filesystem endpoint with support for hardlinks, symlinks and copy. 2017-04-13 20:25:40 +02:00
Andrey Smirnov 85b4a8b1ae Add new option for detailed logging on dependency resolving
This adds command-line arg and config option, with option enabled
aptly is more verbose on internal depeendency resolving cycles:

```
Missing dependencies: file-rc (>= 0.8.16) [amd64], python:any (>= 2.7.1-0ubuntu2) [amd64], python3:any (>= 3.3.2-2~) [amd64], file-rc [amd64], perl (<< 5.17) [amd64], iptables-router (>= 1.2.3) [amd64], systemd [amd64], sgml-base (>= 1.26+nmu2) [amd64], sed (>= 4.1.2-8) [amd64]
Unsatisfied dependency: file-rc (>= 0.8.16) [amd64]
Unsatisfied dependency: python:any (>= 2.7.1-0ubuntu2) [amd64]
Unsatisfied dependency: python3:any (>= 3.3.2-2~) [amd64]
Unsatisfied dependency: file-rc [amd64]
Unsatisfied dependency: perl (<< 5.17) [amd64]
Unsatisfied dependency: iptables-router (>= 1.2.3) [amd64]
Unsatisfied dependency: systemd [amd64]
Injecting package: sgml-base_1.26+nmu4ubuntu1_all
Injecting package: sed_4.2.2-4ubuntu1_amd64
```
2017-03-28 22:58:07 +03:00
Einar Forselv 511fb439ac Identity v3 support for Swift
ncw/swift was bumped to latest version
2016-09-14 15:29:11 +02:00
Andrey Smirnov 75b860e0b1 Support SigV2 and S3 debug for publishing. 2016-03-20 20:11:19 +03:00
Andrey Smirnov f50e008763 Make 'SkipContents' configurable in API. #345
Also add global configuration to disable 'skipContents' by
default for all new published repos/snapshots.
2016-02-14 14:49:16 +03:00
Phil Frost 640bd2b530 Use official AWS SDK; support STS credentials
Now that there's an official Go AWS SDK from Amazon, use that instead of
goamz. goamz isn't getting much love these days.

Implement support for STS credentials, as in assumed roles and EC2
instance profiles. The configuration is extended to support a session
token, though I'm not sure why anyone would put temporary credentials in
a configuration file. More likely, no credentials will be explicitly
configured at all, and they will be discovered through the standard SDK
mechanisms described at
<https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs>.

Resolves #342.
2016-02-03 15:13:01 -05:00
Andrey Smirnov 8cc7d1345b Support for new S3 configuration options: endpoint & multi del disabling. #218 2015-04-17 01:18:34 +03:00
Andrey Smirnov 48b8311150 Style fixes. #191 2015-02-20 23:58:00 +03:00
Sebastien Badia 7f038be1cb Add swift backend for repository publishing 2015-02-05 17:54:09 +01:00
Andrey Smirnov aeae6009c4 Introduce plusWorkaround: generate copy of file with '+' -> ' ' to workaround S3/apt bug. #98 2014-10-01 21:32:56 +04:00
Andrey Smirnov 8aa1954ba7 Support for custom storage class and encryption method. #105 2014-10-01 19:16:15 +04:00
Andrey Smirnov 2c470c1535 Rename config option to endpoint. #15 2014-07-28 15:01:51 +04:00
Andrey Smirnov de0954732a Style fix. 2014-07-21 17:43:35 +04:00
Andrey Smirnov 7a7bb56557 Config options for S3 storage. #15 2014-07-18 17:37:08 +04:00
Andrey Smirnov a54a366c95 New config setting: downloadSpeedLimit to limit download speed. #62 2014-07-13 15:47:44 +04:00
Andrey Smirnov bed0ac475e Short syntax for aptly mirror create to support ppa:user/project mirror URLs. 2014-03-06 21:06:50 +04:00
Andrey Smirnov 21123ac6a4 Add config option to follow dependency to source packages. 2014-02-17 15:08:21 +04:00