Commit Graph

136 Commits

Author SHA1 Message Date
André Roth 4a0bdcbb64 improve system tests
- log import errors for test modules
- log output only on test failure
- improve docker system test container
- use go 1.19 in docker system tests
- download go dependencies in docker container
- system tests: color failues output
- imrpove test result output
- do not install golangci-lint in system tests
2024-07-24 21:19:47 +02:00
Ryan Gonzalez 91574b53d9 Add functional tests for Azure publishing
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
2024-06-17 11:51:18 +02:00
André Roth 2a2e35c096 add test for publishing distribution with slash (/) 2024-04-11 19:37:51 +02:00
André Roth d7cc9b89d1 system tests: use repository mirrors on S3 for reproducibility
no direct internet download from apt repositories,
which over time will change or cease to exist.

also migrate to gpg2 on newer ubuntu.
2024-02-05 13:04:45 +01:00
Mauro Regli d1102e2e9c Fix: Pipeline dependency on deb.debian.org, replace with archive
This should fix some tests, as a lot of them are dependent on
deb.debian.org which no longer supports Debian 9 "Stretch".
Instead we use archive.debian.org which will continue to contain
"Stretch" packages for a long time.
2023-05-15 11:15:48 +02:00
Josh Bayfield b3d9055059 Fix system tests for custom codenames 2022-08-29 15:54:29 +02:00
Steven Stone 904265120b Fix PublishSnapshot39Test_release_i386 system test 2022-08-29 15:54:29 +02:00
Steven Stone 47930a4214 Fix system test 2022-08-29 15:54:29 +02:00
Steven Stone a59cad6f20 Enable the ability to pass in a custom codename
While testing out Aptly, the `apt-get` client complains with the following error, since the `codename` was switched from the InRelease files that are baked out by Aptly:

```
E: Repository 'http://debianrepo.example.com/bionic testing InRelease' changed its 'Codename' value from '' to 'testing'
```
2022-08-29 15:54:29 +02:00
Sjoerd Simons f61514edaf Allow disabling bzip2 compression for index files
Using bzip2 generates smaller index files (roughly 20% smaller Packages
files) but it comes with a big performance penalty.  When publishing a
debian mirror snapshot (amd64, arm64, armhf, source) without contents
skipping bzip speeds things up around 1.8 times.

```
$ hyperfine -w 1 -L skip-bz2 true,false  -m 3 -p "aptly -config aptly.conf publish drop bullseye || true" "aptly -config aptly.conf  publish snapshot  --skip-bz2={skip-bz2} --skip-contents --skip-signing bullseye"
Benchmark 1: aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye
  Time (mean ± σ):     35.567 s ±  0.307 s    [User: 39.366 s, System: 10.075 s]
  Range (min … max):   35.311 s … 35.907 s    3 runs

Benchmark 2: aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye
  Time (mean ± σ):     64.740 s ±  0.135 s    [User: 68.565 s, System: 10.129 s]
  Range (min … max):   64.596 s … 64.862 s    3 runs

Summary
  'aptly -config aptly.conf  publish snapshot  --skip-bz2=true --skip-contents --skip-signing bullseye' ran
    1.82 ± 0.02 times faster than 'aptly -config aptly.conf  publish snapshot  --skip-bz2=false --skip-contents --skip-signing bullseye'
```

Allow skipping bz2 creation for setups where faster publishing is more
important then Package file size.

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
2022-06-22 11:25:45 +02:00
Lorenzo Bolla 1361bf20dd Revive skipped tests 2022-02-08 11:18:50 +01:00
Lorenzo Bolla 035d5314b0 Convert tests to Python 3
Fix #938
2022-01-27 15:06:33 +01:00
Lorenzo Bolla 8ecd01be1f Temporarily skip test failing on CI 2022-01-27 09:30:14 +01:00
Lorenzo Bolla 4933e3caf4 Try to fix test failing on CI
PublishRepo26Test fails to run because something in the CI environment forces
gpg to ask for the user's password. Try to require gpg1 for the test, which
seems to run fine in other environments.
2022-01-27 09:30:14 +01:00
Lorenzo Bolla 787cc8e3ee Fix system tests 2022-01-27 09:30:14 +01:00
Joshua Colson 899ed92ebc Add -json flag to publish list|show
Signed-off-by: Joshua Colson <joshua.colson@gmail.com>
2021-09-24 10:29:33 +02:00
Lorenzo Bolla 3201244d9b Fix tests and fixtures relying on expired pgp keys
PGP tests relied on expired gpg keys: upgrade with newer Debian keys from
https://ftp-master.debian.org/keys.html.
Download new fixtures files from http://ftp.debian.org/debian/dists/buster/
2021-02-08 10:52:27 +01:00
Andrey Smirnov c75ef8546e Fix system tests for Debian Stretch 9.11 2019-09-18 01:23:58 +03:00
Andrey Smirnov d80c2b6104 Fix system tests 2019-09-06 23:42:56 +03:00
Raúl Benencia 7a5ac3dbc2 Tests for custom and default suite 2019-09-06 23:42:56 +03:00
Andrey Smirnov 26098f6c8d Print redirects being followed, drop mirror.yandex.ru.
Use CDN-backed Debian mirror to make tests run faster hopefully for
everyone. Redirects might be important to know what exactly is going on
when items are being downloaded.
2019-08-07 21:10:04 +03:00
Andrey Smirnov bb1def2910 Try Travis on xenial workers 2019-07-11 00:16:20 +03:00
Andrey Smirnov 2c91bcdc30 Bump Go versions for Travis, fix tests
Replace gometalinter with golangci-lint.

Fix system tests (wheezy is gone, replace with stretch).

Fix linter warnings.
2019-07-04 00:16:12 +03:00
Andrey Smirnov 61e00b5fbd Test updates for Travis CI
Travis is running Trusty with GPG 2.0.x, which is
much different from 2.1.x.

Add tests for default key signing.

Add test for gpg1/2 in functional.
2018-10-10 01:34:58 +03:00
Oliver Sauder e23e30eb44 Merge branch 'master' into with_installer 2018-09-21 13:26:15 +02:00
Andrey Smirnov 747b9752ce Keep checksum of not compressed index file even if it's not uploaded
Fixes: #756
2018-07-14 00:17:36 +03:00
Oliver Sauder b7323db31b Add detached signature to installer hashsum file 2018-07-06 15:02:37 +02:00
Oliver Sauder 108b0ea226 Add support to mirror non package installer files 2018-07-06 15:02:37 +02:00
Andrey Smirnov 5b85522400 Implement 'legacy' Contents indexes to match Ubuntu <=16.04
Another index is created which unifies data for all the components.

This certainly requires more resources as we have to build yet another
index.
2018-04-11 00:57:15 +03:00
Oliver Sauder b4f3573d11 Add acquire by hash when updating publish 2017-11-30 09:46:02 +01:00
Oliver Sauder 2bd0b786ea Extend publish snapshot test with acquire by hash 2017-11-30 09:46:02 +01:00
Andrey Smirnov 2d66a4ca0a Enforce SSE/StorageClass in PUT Object Copy
"RenameFile" is implemented in S3 using `PUT Object Copy`, which
should enforce SSE/StorageClass same way as regular `PUT Object`.

Fixes: #647
2017-09-28 18:30:50 +03:00
Ludovico Cavedon d6a3917141 Add -skip-cleanup option for publish commands.
Allow skipping unreferenced files cleanup on publish switch/update/drop
via the -skip-cleanup command line option.
Also support API SkipCleanup parameter.

Fixes #570.
2017-08-15 19:08:17 -07:00
Andrey Smirnov a584b2e058 Fix bug with PoolPath field being overwritten on mirror update
While updating mirror, if package file is already in pool path,
field `PoolPath` was left as empty which results in package file
being unavailable later on while publishing.
2017-08-11 20:05:55 +03:00
Andrey Smirnov 84ef963d7d Trim slashes while parsing publish prefix
Fixes: #607
2017-08-09 01:26:47 +03:00
Andrey Smirnov cbb576cbcc Fix up system tests 2017-07-28 22:37:20 +03:00
Andrey Smirnov 68da8a674a Improve internal PGP provider
1. Print additional details about keys being used for signing
2. Skip expired keys
3. Add `\n` to logged messages
2017-07-28 00:53:50 +03:00
Andrey Smirnov ab20c2d329 Add publishing & repo include tests 2017-07-26 00:31:27 +03:00
Andrey Smirnov 7d179dd405 Small fixups for upcoming 1.1.0 release
Fix system tests, add -db-open-attempts to bash completion, small nits
for man page.
2017-07-07 00:14:03 +03:00
Oliver Sauder e3f1880ad4 Added support for NotAutomatic, ButAutomaticUpgrades and Origin fields 2017-07-05 15:08:02 +02:00
Andrey Smirnov c798db8056 Add system's requirements.txt, enforce flake8 linter
Fix style issues in functional tests.
2017-04-28 00:05:11 +03:00
Andrey Smirnov 7dcc0d597d Fix S3/Swift tests 2017-04-26 23:17:04 +03:00
Andrey Smirnov b7b9f12c88 Update system tests for SHA512 checksums being generated 2017-04-26 23:17:04 +03:00
Andrey Smirnov 1f3cb2db5d When downloading/importing packages, enforce all checksums 2017-04-26 23:17:04 +03:00
Andrey Smirnov 4171a73995 Fix up system test 2017-04-26 23:17:03 +03:00
Andrey Smirnov 45d187bc14 Fix up system test 2017-04-26 23:17:03 +03:00
Clemens Rabe 25f9c29f00 Implemented filesystem endpoint with support for hardlinks, symlinks and copy. 2017-04-13 20:25:40 +02:00
Andrey Smirnov 7a5be6736d When contents generation fails, don't bail out
This replaces `panic` which aborts aptly execution with warning
message on console. So aptly continues publishing actions, but
`Contents` indexes might be incomplete.

Error will be printed every time contents generation is triggered.
2017-03-31 00:57:18 +03:00
Oliver Sauder 0fdee9cbf6 Added publish show command 2017-01-10 10:59:07 +01:00
Andrey Smirnov 60cca0245b Fix system tests. 2016-03-20 22:01:41 +03:00