Commit Graph

2655 Commits

Author SHA1 Message Date
Benj Fassbind ecc055180c Fix publishing race condition
A race condition for publishing packages and
mirrors at the same time was introduced in
commit 77d7c38.

The problem is that when opening a leveldb transaction
and performing another 'put' to the db
the system freezes.
2023-05-31 15:48:42 +02:00
Alexander Zubarev 1501a4e531 Add strike to AUTHORS 2023-05-26 17:20:16 +02:00
Alexander Zubarev 8f53e01749 Show storage of publish on graph 2023-05-26 17:20:16 +02:00
Sjoerd Simons 1df8cff842 Update go-xz to 0.1.0
Older versions go-xz didn't wait for child processes meaning for exery
unpack action a defunct xz would stick around. This got fixed in 0.1.0

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2023-05-19 19:49:54 +02:00
Mauro Regli 76744ead86 Fix Release failing 'Cannot find goxc' 2023-05-15 11:15:48 +02:00
Mauro Regli f6a7030654 Try to fix UnixSocketAPITest by upgrading dependencies
Updated urllib, requests and requests_unixsocket
2023-05-15 11:15:48 +02:00
Mauro Regli 7c8dd7362d Fix: Missing newline makes tests fail 2023-05-15 11:15:48 +02:00
Mauro Regli 0ae9884836 Fix: Tests with jenkins repo not finding public key. 2023-05-15 11:15:48 +02:00
Mauro Regli 95ca6fb376 Fix: Replace security.debian.org with archive 2023-05-15 11:15:48 +02:00
Mauro Regli c9b1782d62 Fix CreateMirror9Test by removing Acquire-By-Hash 2023-05-15 11:15:48 +02:00
Mauro Regli d1102e2e9c Fix: Pipeline dependency on deb.debian.org, replace with archive
This should fix some tests, as a lot of them are dependent on
deb.debian.org which no longer supports Debian 9 "Stretch".
Instead we use archive.debian.org which will continue to contain
"Stretch" packages for a long time.
2023-05-15 11:15:48 +02:00
Markus Muellner 9c6f896666 add endpoint for listing repos while serving in api mode and add more metrics 2023-03-22 17:22:54 +01:00
Markus Muellner 0fdba29d51 make serving published repos in api mode configurable 2023-03-22 17:22:54 +01:00
Markus Muellner f74217ed9c implement system tests for serving api and published repos simultaneously 2023-03-22 17:22:54 +01:00
Андрей Лухнов e25ade8af3 Serve api and published repos simultaneously
refs #1017 #975
2023-03-22 17:22:54 +01:00
Markus Muellner bece12ad4d update golangci-lint to v1.51.2 2023-03-22 17:22:54 +01:00
Mauro Regli 77e02bf7a3 Feature: Add Merge Snapshot API
Is part of Issue #176
2023-03-14 08:38:55 +01:00
Mauro Regli 90932cdac5 Improvement: Remove Magic Numbers in Tests with Tasks
Replaced 2 with TASK_SUCCEEDED, 3 with TASK_FAILED.

fixes: #1158
2023-03-13 13:17:17 +01:00
Mauro Regli 5b5307cc15 Fix CodeCov Config has two targets and thresholds
fixes: #1160
2023-03-13 08:20:18 +01:00
Mauro Regli aaa622288c Fix: Make CodeCov Pipeline more lenient
The Pipeline will only fail if the code coverage has fallen more than 2
Percent.

fixes: #1154
2023-03-07 17:05:16 +01:00
Mauro Regli dbf1ac7867 Fix: Drop Publish returned wrong status code if not found
Deleting a publish that does not exist now results in a status code 404
instead of 500.

Fixes: #1006
2023-03-07 13:46:57 +01:00
Mauro Regli c187b0d52c Fix: Switch gin mode depending on aptly.EnableDebug
If aptly.EnableDebug is active, we use Debug, otherwise we use
gin.ReleaseMode to remove the annoying nuding messages when running the
api.

fixes: #1103
2023-03-07 13:04:12 +01:00
Markus Muellner 8e62195eb5 implement structured logging 2023-02-20 13:42:50 +01:00
dependabot[bot] 0c749922c9 Bump github.com/aws/aws-sdk-go from 1.33.0 to 1.34.0
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.33.0 to 1.34.0.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.34.0/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.33.0...v1.34.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-20 13:07:27 +01:00
Markus Muellner ecc41f0c0f replace AbortWithError calls by custom function that sets the content type correctly 2023-01-23 10:42:57 +01:00
dependabot[bot] 81582bffd2 Bump github.com/aws/aws-sdk-go from 1.25.0 to 1.33.0
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.25.0 to 1.33.0.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.33.0/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.25.0...v1.33.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 16:36:03 +01:00
Samuel Bachmann ced5ac7876 return the snapshot in apiSnapshotsCreate
In v1.4.0 it [returned the snapshot](https://github.com/aptly-dev/aptly/blob/v1.4.0/api/snapshot.go#L168), but this was removed (by accident) in v1.5.0. This adds it back.
2022-12-22 15:17:12 +01:00
Markus Muellner 2020ca9971 add ready and healthy probe endpoints 2022-12-12 13:39:07 +01:00
Markus Muellner 352f4e8772 update golangci-lint and replace deprecated calls to io/ioutil 2022-12-12 10:21:39 +01:00
Benj Fassbind 71fd730598 Return an empty array if no tasks are available
All other api endpoints also send empty arrays instead of nil.
Closes #1123
2022-11-17 10:44:35 +01:00
boxjan e90ac6767f Update AUTHORS 2022-09-09 09:02:52 +02:00
boxjan 268c39ea8c add forceVirtualHostedStyle for stores which only support virtual hosted style 2022-09-09 09:02:52 +02:00
Josh Bayfield b3d9055059 Fix system tests for custom codenames 2022-08-29 15:54:29 +02:00
Steven Stone 904265120b Fix PublishSnapshot39Test_release_i386 system test 2022-08-29 15:54:29 +02:00
Steven Stone 47930a4214 Fix system test 2022-08-29 15:54:29 +02:00
Steven Stone a59cad6f20 Enable the ability to pass in a custom codename
While testing out Aptly, the `apt-get` client complains with the following error, since the `codename` was switched from the InRelease files that are baked out by Aptly:

```
E: Repository 'http://debianrepo.example.com/bionic testing InRelease' changed its 'Codename' value from '' to 'testing'
```
2022-08-29 15:54:29 +02:00
Sjoerd Simons 393d1a6888 api: Allow querying the packages endpoint
The ".../packages" endpoints for mirror, local repos and snapshots all
share the same syntax for querying. However the "/api/packages" endpoint
doesn't match this. Adjust that to allow for a bit more consistency and
allow querying the full package database.

The current endpoint functionality "/packages/:name" is kept intact and
can be used the same as now

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-08-29 10:28:44 +02:00
Benj Fassbind 42cfee2c09 Fix mirror test 2022-08-16 09:04:16 +02:00
Benj Fassbind afdc10b919 Fix golangci-lint 2022-08-16 09:04:16 +02:00
Benj Fassbind af899149c7 Fix wrong nil check for SkipBz2 2022-08-16 09:04:16 +02:00
Adam Bambuch abf8abb59b upgrade go-xz go module 2022-08-04 10:48:20 +02:00
Benj Fassbind f0a85b2b6e Fix release build 2022-07-13 08:33:48 +02:00
Benj Fassbind 515e5532c8 Fix temp dir on ci 2022-07-13 08:33:48 +02:00
Benj Fassbind ff3bf4b180 Improve error messages 2022-07-13 08:33:48 +02:00
Benj Fassbind 1d4e6183be Capture coverage of integration tests
To capture the coverage also for the integration tests,
a test only executing the cmd.Run function is used.

The test always exits with code 0 and prints the
real exit code to stdout. Otherwise no coverage
report is generated.

Those changes enable a more accurate coverage report
for future contributions.
2022-07-13 08:33:48 +02:00
Benj Fassbind 69d473ea6f Fix failing mirror test
Add the https redirect to the gold ouptut of the test
as this was changed for the jenkins debian repos
and the tests were failing after this change.
2022-07-13 08:33:48 +02:00
Benj Fassbind bfc86d3b30 Test copyfile 2022-07-13 08:33:48 +02:00
Benj Fassbind 3ce27743ae Test utils 2022-07-13 08:33:48 +02:00
Wade Simmons c9f5763a70 S3: support disabling ACL with none value
This change lets you disable ACL when using S3 by using a configuration
value of `none`. This way we maintain backward compatibility with the
default setting being `private`.

Fixes: #1067
v1.5.0
2022-06-22 11:26:13 +02:00
Sjoerd Simons f61514edaf Allow disabling bzip2 compression for index files
Using bzip2 generates smaller index files (roughly 20% smaller Packages
files) but it comes with a big performance penalty.  When publishing a
debian mirror snapshot (amd64, arm64, armhf, source) without contents
skipping bzip speeds things up around 1.8 times.

```
$ hyperfine -w 1 -L skip-bz2 true,false  -m 3 -p "aptly -config aptly.conf publish drop bullseye || true" "aptly -config aptly.conf  publish snapshot  --skip-bz2={skip-bz2} --skip-contents --skip-signing bullseye"
Benchmark 1: aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     35.567 s ±  0.307 s    [User: 39.366 s, System: 10.075 s]
  Range (min … max):   35.311 s … 35.907 s    3 runs

Benchmark 2: aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye
  Time (mean ± σ):     64.740 s ±  0.135 s    [User: 68.565 s, System: 10.129 s]
  Range (min … max):   64.596 s … 64.862 s    3 runs

Summary
  'aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye' ran
    1.82 ± 0.02 times faster than 'aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye'
```

Allow skipping bz2 creation for setups where faster publishing is more
important then Package file size.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-22 11:25:45 +02:00