Commit Graph

2228 Commits

Author SHA1 Message Date
Markus Muellner 2020ca9971 add ready and healthy probe endpoints 2022-12-12 13:39:07 +01:00
Markus Muellner 352f4e8772 update golangci-lint and replace deprecated calls to io/ioutil 2022-12-12 10:21:39 +01:00
Benj Fassbind 71fd730598 Return an empty array if no tasks are available
All other api endpoints also send empty arrays instead of nil.
Closes #1123
2022-11-17 10:44:35 +01:00
boxjan e90ac6767f Update AUTHORS 2022-09-09 09:02:52 +02:00
boxjan 268c39ea8c add forceVirtualHostedStyle for stores which only support virtual hosted style 2022-09-09 09:02:52 +02:00
Josh Bayfield b3d9055059 Fix system tests for custom codenames 2022-08-29 15:54:29 +02:00
Steven Stone 904265120b Fix PublishSnapshot39Test_release_i386 system test 2022-08-29 15:54:29 +02:00
Steven Stone 47930a4214 Fix system test 2022-08-29 15:54:29 +02:00
Steven Stone a59cad6f20 Enable the ability to pass in a custom codename
While testing out Aptly, the `apt-get` client complains with the following error, since the `codename` was switched from the InRelease files that are baked out by Aptly:

```
E: Repository 'http://debianrepo.example.com/bionic testing InRelease' changed its 'Codename' value from '' to 'testing'
```
2022-08-29 15:54:29 +02:00
Sjoerd Simons 393d1a6888 api: Allow querying the packages endpoint
The ".../packages" endpoints for mirror, local repos and snapshots all
share the same syntax for querying. However the "/api/packages" endpoint
doesn't match this. Adjust that to allow for a bit more consistency and
allow querying the full package database.

The current endpoint functionality "/packages/:name" is kept intact and
can be used the same as now

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-08-29 10:28:44 +02:00
Benj Fassbind 42cfee2c09 Fix mirror test 2022-08-16 09:04:16 +02:00
Benj Fassbind afdc10b919 Fix golangci-lint 2022-08-16 09:04:16 +02:00
Benj Fassbind af899149c7 Fix wrong nil check for SkipBz2 2022-08-16 09:04:16 +02:00
Adam Bambuch abf8abb59b upgrade go-xz go module 2022-08-04 10:48:20 +02:00
Benj Fassbind f0a85b2b6e Fix release build 2022-07-13 08:33:48 +02:00
Benj Fassbind 515e5532c8 Fix temp dir on ci 2022-07-13 08:33:48 +02:00
Benj Fassbind ff3bf4b180 Improve error messages 2022-07-13 08:33:48 +02:00
Benj Fassbind 1d4e6183be Capture coverage of integration tests
To capture the coverage also for the integration tests,
a test only executing the cmd.Run function is used.

The test always exits with code 0 and prints the
real exit code to stdout. Otherwise no coverage
report is generated.

Those changes enable a more accurate coverage report
for future contributions.
2022-07-13 08:33:48 +02:00
Benj Fassbind 69d473ea6f Fix failing mirror test
Add the https redirect to the gold ouptut of the test
as this was changed for the jenkins debian repos
and the tests were failing after this change.
2022-07-13 08:33:48 +02:00
Benj Fassbind bfc86d3b30 Test copyfile 2022-07-13 08:33:48 +02:00
Benj Fassbind 3ce27743ae Test utils 2022-07-13 08:33:48 +02:00
Wade Simmons c9f5763a70 S3: support disabling ACL with none value
This change lets you disable ACL when using S3 by using a configuration
value of `none`. This way we maintain backward compatibility with the
default setting being `private`.

Fixes: #1067
v1.5.0
2022-06-22 11:26:13 +02:00
Sjoerd Simons f61514edaf Allow disabling bzip2 compression for index files
Using bzip2 generates smaller index files (roughly 20% smaller Packages
files) but it comes with a big performance penalty.  When publishing a
debian mirror snapshot (amd64, arm64, armhf, source) without contents
skipping bzip speeds things up around 1.8 times.

```
$ hyperfine -w 1 -L skip-bz2 true,false  -m 3 -p "aptly -config aptly.conf publish drop bullseye || true" "aptly -config aptly.conf  publish snapshot  --skip-bz2={skip-bz2} --skip-contents --skip-signing bullseye"
Benchmark 1: aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     35.567 s ±  0.307 s    [User: 39.366 s, System: 10.075 s]
  Range (min … max):   35.311 s … 35.907 s    3 runs

Benchmark 2: aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye
  Time (mean ± σ):     64.740 s ±  0.135 s    [User: 68.565 s, System: 10.129 s]
  Range (min … max):   64.596 s … 64.862 s    3 runs

Summary
  'aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye' ran
    1.82 ± 0.02 times faster than 'aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye'
```

Allow skipping bz2 creation for setups where faster publishing is more
important then Package file size.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-22 11:25:45 +02:00
Sjoerd Simons 2aca913e92 Use parallel gzip instead of gzip for compression
golangs compress/gzip isn't a parallel implementation, so it's quite a
bit slower on most modern servers then pgzip. The below benchmark
run shows that publishing a debian bullseye mirror snapshot (amd64, arm64,
armhf, source) shows a gain of about 35% in publishing time (when skipping
bz2 using MR #1081)

```
 hyperfine -w 1 -m 3 -L aptly aptly-nobz2,aptly-nobz2-pgzip -p "{aptly} -config aptly.conf publish drop bullseye || true" "{aptly} -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye"
Benchmark 1: aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     35.548 s ±  0.378 s    [User: 39.465 s, System: 10.046 s]
  Range (min … max):   35.149 s … 35.902 s    3 runs

Benchmark 2: aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     26.592 s ±  0.069 s    [User: 42.207 s, System: 9.676 s]
  Range (min … max):   26.521 s … 26.660 s    3 runs

Summary
  'aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye' ran
    1.34 ± 0.01 times faster than 'aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye'
```

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-21 15:43:58 +02:00
Sjoerd Simons 26254a0ad8 Run go mod tidy
Seems go.mod had some modules that are no longer used since the last
version bumps? Running `make modules` or really `go mod tidy`
automagically cleans those up.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-21 15:43:58 +02:00
Benj Fassbind 6f130e1583 Add codecov configuration 2022-06-20 13:23:28 +02:00
Benj Fassbind 35ad6cacc8 Upload code coverage 2022-06-20 13:23:28 +02:00
Benj Fassbind f519ecded7 Update azure dependency 2022-06-20 12:50:24 +02:00
Michael Stürmer 4b2efeec7a Cope with zero-length http downloads 2022-06-20 09:47:41 +02:00
Sjoerd Simons a687df2f4f Use python3 for system tests
Most modern distribution use python3 for python (3). Default to that to
make it a bit simpler to run systems tests on Debian

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-20 09:39:23 +02:00
Sjoerd Simons 29deae6fe0 api: allow parameters with urlencoded names
Aptly allows create e.g. repos with a / to use those with the REST api
the router needs to allow urlencoded parameters in various places to
represent this. A specific example of this is the /api/repos/:name/packages path

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-15 17:21:15 +02:00
Chuan Liu f9f1c8ee75 Update azurite dir 2022-06-09 10:45:13 +02:00
myml a0544dc2b5 fix: typo in the comments 2022-06-06 13:13:27 +02:00
Chuan Liu 0a1798869a Enable Azure publish unit tests in Github actions 2022-04-29 21:23:41 +02:00
Russell Greene 751fd2f9ba add myself to authors 2022-04-27 13:50:14 +02:00
Russell Greene 954b222fb6 Use proper version comparisions for querys 2022-04-27 13:50:14 +02:00
Samuel Mutel 4c04e77489 enh: Give info when unable to load list of repos 2022-04-25 12:58:06 +02:00
Chuan Liu 152538ccc1 Support custom Azure publish endpoint 2022-04-25 11:41:04 +02:00
Benj Fassbind d955b06f03 Fix artifacts publishing 2022-04-13 09:27:50 +02:00
Markus Muellner db19a56458 Add functional test for metrics endpoint 2022-04-12 14:39:16 +02:00
Markus Muellner 6539e1b856 Add metrics endpoint with http metrics using Prometheus client lib 2022-04-12 14:39:16 +02:00
Benj Fassbind 8046fb1eb9 Fix failing checks 2022-04-05 11:41:14 +02:00
Benj Fassbind 0302e39d57 Update gin and jwt-go dependencies 2022-04-05 11:41:14 +02:00
Benj Fassbind c29ccaadbc Fix typo in ci config 2022-04-05 11:41:14 +02:00
Benj Fassbind d2d168f363 Fix system test env setup 2022-04-05 09:58:02 +02:00
Benj Fassbind cf98718a79 Fix default branch name in ci 2022-04-05 09:58:02 +02:00
Benj Fassbind 47bda055e0 Publish releases and nightly builds from ci 2022-04-04 20:07:08 +02:00
Reinhold Gschweicher c1e577c1ac Add unittest for zstd compression support 2022-04-04 17:51:21 +02:00
Ubuntu 5b98039291 Remove 1.14 from CI 2022-04-04 17:51:21 +02:00
Matt Bearup 5a23f71a7f Add support for zst compression 2022-04-04 17:51:21 +02:00