vincent/aptly
1
0
Fork 0
mirror of https://github.com/aptly-dev/aptly.git synced 2026-01-12 03:21:33 +00:00
Code Issues Projects Releases Wiki Activity
2,555 Commits 13 Branches 27 Tags
build/debian
Commit Graph

152 Commits

Author SHA1 Message Date
André Roth
c07bf2b108 s3: add debug logs for commands 
* initialize zerolog for commands
* Change default log format: remote colors and timestamp
 2025-04-24 12:13:38 +02:00
André Roth
e50a5e175f update documentation and man page 2024-12-11 12:02:52 +01:00
André Roth
a7d6782176 add test and improve config error messages 2024-12-11 12:02:52 +01:00
André Roth
a15d8a3f7b add test 2024-12-11 12:02:52 +01:00
André Roth
4e566b4692 fix tests and lint 2024-12-11 12:02:52 +01:00
André Roth
87a36633e9 group and order config items 
and fix lint
 2024-12-11 12:02:52 +01:00
André Roth
0e0189f0eb use yaml config file 2024-12-11 12:02:52 +01:00
André Roth
a880a88fc0 yaml config 2024-12-11 12:02:52 +01:00
André Roth
ea80f6d49c write commented json default config 2024-12-11 10:40:44 +01:00
André Roth
cc1fc7ccfe allow comments in config file 2024-12-11 10:40:43 +01:00
André Roth
5ddb718eab support ~ in rootDir 2024-11-17 14:09:37 +01:00
André Roth
cefc09a41b more sanitize 2024-10-11 14:11:09 +02:00
André Roth
57639c4adf Sanitize path api params 
- fix path traversal complains by CodeQL
 2024-10-11 12:56:08 +02:00
André Roth
1d1bd41bb8 add swagger support 
- install swaggo
- add swagger config option
 2024-10-01 01:07:09 +02:00
André Roth
32a3943821 support ~ in rootDir as home directory 2024-09-24 10:14:39 +02:00
André Roth
f10acb3df8 etcd: fix db config and test 
fix unit test for adapted config file
 2024-07-31 22:16:00 +02:00
hudeng
59bf4501e8 feat: Use databaseBackend config repace databaseEtcd 
databaseBackend config contains type and url sub config, It can facilitate the expansion of other types of databases in the future.
 2024-07-31 22:16:00 +02:00
hudeng
78172d11d7 feat: Add etcd database support 
improve concurrent access and high availability of aptly with the help of the characteristics of etcd
 2024-07-31 22:16:00 +02:00
André Roth
deae90485a fix DirIsAccessible 
perms 0000 need to be checked explicitly
 2024-07-24 21:19:47 +02:00
André Roth
9f1860dff7 fix unit test 2024-07-24 21:19:47 +02:00
Ryan Gonzalez
f9325fbc91 Add support for Azure package pools 
This adds support for storing packages directly on Azure, with no truly
"local" (on-disk) repo used. The existing Azure PublishedStorage
implementation was refactored to move the shared code to a separate
context struct, which can then be re-used by the new PackagePool. In
addition, the files package's mockChecksumStorage was made public so
that it could be used in the Azure PackagePool tests as well.

Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
 2024-06-17 11:51:18 +02:00
Ryan Gonzalez
8e37813129 Add support for custom package pool locations 
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
 2024-06-17 11:51:18 +02:00
Mauro Regli
ae61706a34 Fix: Implement golangci-lint suggestions 2023-09-21 11:25:18 +02:00
Markus Muellner
0fdba29d51 make serving published repos in api mode configurable 2023-03-22 17:22:54 +01:00
Markus Muellner
f74217ed9c implement system tests for serving api and published repos simultaneously 2023-03-22 17:22:54 +01:00
Markus Muellner
8e62195eb5 implement structured logging 2023-02-20 13:42:50 +01:00
Markus Muellner
352f4e8772 update golangci-lint and replace deprecated calls to io/ioutil 2022-12-12 10:21:39 +01:00
boxjan
268c39ea8c add forceVirtualHostedStyle for stores which only support virtual hosted style 2022-09-09 09:02:52 +02:00
Benj Fassbind
bfc86d3b30 Test copyfile 2022-07-13 08:33:48 +02:00
Benj Fassbind
3ce27743ae Test utils 2022-07-13 08:33:48 +02:00
Sjoerd Simons
f61514edaf Allow disabling bzip2 compression for index files 
Using bzip2 generates smaller index files (roughly 20% smaller Packages
files) but it comes with a big performance penalty.  When publishing a
debian mirror snapshot (amd64, arm64, armhf, source) without contents
skipping bzip speeds things up around 1.8 times.

```
$ hyperfine -w 1 -L skip-bz2 true,false  -m 3 -p "aptly -config aptly.conf publish drop bullseye || true" "aptly -config aptly.conf  publish snapshot  --skip-bz2={skip-bz2} --skip-contents --skip-signing bullseye"
Benchmark 1: aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     35.567 s ±  0.307 s    [User: 39.366 s, System: 10.075 s]
  Range (min … max):   35.311 s … 35.907 s    3 runs

Benchmark 2: aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye
  Time (mean ± σ):     64.740 s ±  0.135 s    [User: 68.565 s, System: 10.129 s]
  Range (min … max):   64.596 s … 64.862 s    3 runs

Summary
  'aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye' ran
    1.82 ± 0.02 times faster than 'aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye'
```

Allow skipping bz2 creation for setups where faster publishing is more
important then Package file size.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
 2022-06-22 11:25:45 +02:00
Sjoerd Simons
2aca913e92 Use parallel gzip instead of gzip for compression 
golangs compress/gzip isn't a parallel implementation, so it's quite a
bit slower on most modern servers then pgzip. The below benchmark
run shows that publishing a debian bullseye mirror snapshot (amd64, arm64,
armhf, source) shows a gain of about 35% in publishing time (when skipping
bz2 using MR #1081)

```
 hyperfine -w 1 -m 3 -L aptly aptly-nobz2,aptly-nobz2-pgzip -p "{aptly} -config aptly.conf publish drop bullseye || true" "{aptly} -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye"
Benchmark 1: aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     35.548 s ±  0.378 s    [User: 39.465 s, System: 10.046 s]
  Range (min … max):   35.149 s … 35.902 s    3 runs

Benchmark 2: aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     26.592 s ±  0.069 s    [User: 42.207 s, System: 9.676 s]
  Range (min … max):   26.521 s … 26.660 s    3 runs

Summary
  'aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye' ran
    1.34 ± 0.01 times faster than 'aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye'
```

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
 2022-06-21 15:43:58 +02:00
Chuan Liu
152538ccc1 Support custom Azure publish endpoint 2022-04-25 11:41:04 +02:00
Markus Muellner
6539e1b856 Add metrics endpoint with http metrics using Prometheus client lib 2022-04-12 14:39:16 +02:00
Lorenzo Bolla
1afcd68e01 Make downloader type configurable 2022-01-31 10:32:54 +01:00
Lorenzo Bolla
9bf1a44f75 Fix test after merge 2022-01-27 09:30:14 +01:00
Lorenzo Bolla
9b28d8984f Configurable background task execution 2022-01-27 09:30:14 +01:00
chuan
bb42a2158d Add support for Azure storage as a publishing backend 
This adds a new configuration setting: AzurePublishEndpoints, similar
to the existing S3PublishEndpoints and SwiftPublishEndpoints.

For each endpoint, the following has to be defined:
 - accountName
 - accountKey
 - container
 - prefix

Azure tests require the following environment variables to be set:
 - AZURE_STORAGE_ACCOUNT
 - AZURE_STORAGE_ACCESS_KEY

With either of these not set, Azure-specific tests are skipped.
 2022-01-21 11:46:36 +01:00
Frank Steinborn
98e75f6d97 Make database open attempts configurable also via config file 2019-09-03 00:52:24 +03:00
Andrey Smirnov
f0a370db24 Rework HTTP downloader retry logic 
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.

Unwrap `errors.Wrap` which is used in downloader.

Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
 2019-08-07 20:23:05 +03:00
Andrey Smirnov
3b5840e248 Fix linter list and fix errors discovered by new staticcheck 2019-01-20 00:01:17 +03:00
Harald Sitter
46c2182ade fix linting by using new maligned linter instead of aligncheck 
upstream switched the alignment check backend and in doing so fails to run
if the old backend is defined in the config.

also skip alignment linting on a struct we use for byte decoding as we have
no choice in its member order.
 2017-10-31 12:24:31 +01:00
Andrey Smirnov
499ab35012 Implement flag/config falue for GPG provider 2017-07-21 01:01:58 +03:00
Andrey Smirnov
1be8d39105 Refactor GPG signer/verifier 
Goal is to make it easier to plug in another implementation.
 2017-05-23 02:54:56 +03:00
Andrey Smirnov
51213899b7 More Go linters enabled, issues fixed 
Ref: #528

Enables "staticcheck", "varcheck", "structcheck", "aligncheck"
 2017-05-03 18:23:14 +03:00
Andrey Smirnov
bae3f949b4 Enable gosimple and ineffasign linters 2017-04-27 18:34:30 +03:00
Andrey Smirnov
186bb2dff0 Add flag to disable/enable support for legacy pool paths 
Legacy pool paths are enabled by default, but for new aptly installations
(when aptly config is first generated), it would be disabled explicitly.
 2017-04-26 23:37:31 +03:00
Andrey Smirnov
5566111a7b New ChecksumStorage and new PackagePool interfaces 2017-04-26 23:17:04 +03:00
Clemens Rabe
25f9c29f00 Implemented filesystem endpoint with support for hardlinks, symlinks and copy. 2017-04-13 20:25:40 +02:00
Andrey Smirnov
85b4a8b1ae Add new option for detailed logging on dependency resolving 
This adds command-line arg and config option, with option enabled
aptly is more verbose on internal depeendency resolving cycles:

```
Missing dependencies: file-rc (>= 0.8.16) [amd64], python:any (>= 2.7.1-0ubuntu2) [amd64], python3:any (>= 3.3.2-2~) [amd64], file-rc [amd64], perl (<< 5.17) [amd64], iptables-router (>= 1.2.3) [amd64], systemd [amd64], sgml-base (>= 1.26+nmu2) [amd64], sed (>= 4.1.2-8) [amd64]
Unsatisfied dependency: file-rc (>= 0.8.16) [amd64]
Unsatisfied dependency: python:any (>= 2.7.1-0ubuntu2) [amd64]
Unsatisfied dependency: python3:any (>= 3.3.2-2~) [amd64]
Unsatisfied dependency: file-rc [amd64]
Unsatisfied dependency: perl (<< 5.17) [amd64]
Unsatisfied dependency: iptables-router (>= 1.2.3) [amd64]
Unsatisfied dependency: systemd [amd64]
Injecting package: sgml-base_1.26+nmu4ubuntu1_all
Injecting package: sed_4.2.2-4ubuntu1_amd64
```
 2017-03-28 22:58:07 +03:00