Commit Graph

21 Commits

Author SHA1 Message Date
Mauro Regli
972bf6b3cf Update Golang to 1.21 and dependencies 2023-09-21 11:25:18 +02:00
Sjoerd Simons
1df8cff842 Update go-xz to 0.1.0
Older versions go-xz didn't wait for child processes meaning for exery
unpack action a defunct xz would stick around. This got fixed in 0.1.0

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2023-05-19 19:49:54 +02:00
Markus Muellner
8e62195eb5 implement structured logging 2023-02-20 13:42:50 +01:00
dependabot[bot]
0c749922c9 Bump github.com/aws/aws-sdk-go from 1.33.0 to 1.34.0
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.33.0 to 1.34.0.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.34.0/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.33.0...v1.34.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-20 13:07:27 +01:00
dependabot[bot]
81582bffd2 Bump github.com/aws/aws-sdk-go from 1.25.0 to 1.33.0
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.25.0 to 1.33.0.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.33.0/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.25.0...v1.33.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 16:36:03 +01:00
Markus Muellner
352f4e8772 update golangci-lint and replace deprecated calls to io/ioutil 2022-12-12 10:21:39 +01:00
Adam Bambuch
abf8abb59b upgrade go-xz go module 2022-08-04 10:48:20 +02:00
Sjoerd Simons
2aca913e92 Use parallel gzip instead of gzip for compression
golangs compress/gzip isn't a parallel implementation, so it's quite a
bit slower on most modern servers then pgzip. The below benchmark
run shows that publishing a debian bullseye mirror snapshot (amd64, arm64,
armhf, source) shows a gain of about 35% in publishing time (when skipping
bz2 using MR #1081)

```
 hyperfine -w 1 -m 3 -L aptly aptly-nobz2,aptly-nobz2-pgzip -p "{aptly} -config aptly.conf publish drop bullseye || true" "{aptly} -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye"
Benchmark 1: aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     35.548 s ±  0.378 s    [User: 39.465 s, System: 10.046 s]
  Range (min … max):   35.149 s … 35.902 s    3 runs

Benchmark 2: aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     26.592 s ±  0.069 s    [User: 42.207 s, System: 9.676 s]
  Range (min … max):   26.521 s … 26.660 s    3 runs

Summary
  'aptly-nobz2-pgzip -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye' ran
    1.34 ± 0.01 times faster than 'aptly-nobz2 -config aptly.conf  publish snapshot --skip-bz2=true --skip-contents --skip-signing bullseye'
```

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-21 15:43:58 +02:00
Sjoerd Simons
26254a0ad8 Run go mod tidy
Seems go.mod had some modules that are no longer used since the last
version bumps? Running `make modules` or really `go mod tidy`
automagically cleans those up.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-21 15:43:58 +02:00
Benj Fassbind
f519ecded7 Update azure dependency 2022-06-20 12:50:24 +02:00
Markus Muellner
db19a56458 Add functional test for metrics endpoint 2022-04-12 14:39:16 +02:00
Markus Muellner
6539e1b856 Add metrics endpoint with http metrics using Prometheus client lib 2022-04-12 14:39:16 +02:00
Benj Fassbind
0302e39d57 Update gin and jwt-go dependencies 2022-04-05 11:41:14 +02:00
Matt Bearup
5a23f71a7f Add support for zst compression 2022-04-04 17:51:21 +02:00
Lorenzo Bolla
1afcd68e01 Make downloader type configurable 2022-01-31 10:32:54 +01:00
Lorenzo Bolla
cc30ef3ee2 Remove vendor directory
We use go modules now.
2022-01-31 10:32:54 +01:00
Lorenzo Bolla
e96372c999 Implement ignore checksum mismatch
Also, update "pkg/errors" library.
2022-01-31 10:32:54 +01:00
chuan
bb42a2158d Add support for Azure storage as a publishing backend
This adds a new configuration setting: AzurePublishEndpoints, similar
to the existing S3PublishEndpoints and SwiftPublishEndpoints.

For each endpoint, the following has to be defined:
 - accountName
 - accountKey
 - container
 - prefix

Azure tests require the following environment variables to be set:
 - AZURE_STORAGE_ACCOUNT
 - AZURE_STORAGE_ACCESS_KEY

With either of these not set, Azure-specific tests are skipped.
2022-01-21 11:46:36 +01:00
Andrey Smirnov
0438a7c76b Upgrade AWS SDK to the latest version 2019-09-27 15:39:48 +03:00
Andrey Smirnov
c86c3a803f Really upgrade goleveldb to the latest master version
PR #876 actually upgraded goleveldb to 1.0.0, not to the latest master.

Recent changes to goleveldb should improve performance
https://github.com/syndtr/goleveldb/issues/226#issuecomment-477568827
2019-09-27 14:19:39 +03:00
Andrey Smirnov
19db62d74f Add new Go modules stuff 2019-09-27 13:59:19 +03:00