Commit Graph

2251 Commits

Author SHA1 Message Date
André Roth
556d7fa4b8 apply PR feedback 2024-08-03 00:14:26 +02:00
André Roth
5718f3f2f5 improve Makefile help 2024-08-03 00:14:26 +02:00
André Roth
0251fddae4 improve Makefile documentation 2024-08-03 00:14:26 +02:00
André Roth
0215925608 add graphviz to enable system tests
aptly uses dot in the aptly graph command and API. if it is not available, the test is skipped...
2024-08-03 00:14:26 +02:00
André Roth
696b78f207 docker: update dev env and documentation 2024-08-03 00:14:26 +02:00
André Roth
674f4f784b s3: use new Endpoint API
lint: s3/public.go#L136
SA1019: config.WithEndpointResolverWithOptions is deprecated: The global endpoint resolution interface is deprecated. See deprecation docs on [WithEndpointResolver]. (staticcheck)
lint: s3/public.go#L137
SA1019: aws.Endpoint is deprecated: This structure was used with the global [EndpointResolver] interface, which has been deprecated in favor of service-specific endpoint resolution. See the deprecation docs on that interface for more information.  (staticcheck)
lint: s3/public.go#L138
SA1019: aws.Endpoint is deprecated: This structure was used with the global [EndpointResolver] interface, which has been deprecated in favor of service-specific endpoint resolution. See the deprecation docs on that interface for more information.  (staticcheck)
2024-08-03 00:14:26 +02:00
André Roth
83a05a1900 golangci-lint: download and build before lint 2024-08-03 00:14:26 +02:00
André Roth
48a0bca35e allow s3 test in docker
read AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY from ./aws.creds when running:

  make docker-system-tests
2024-08-03 00:14:26 +02:00
dependabot[bot]
a7690c375e Bump google.golang.org/grpc from 1.38.0 to 1.56.3
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.38.0 to 1.56.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.38.0...v1.56.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-01 22:15:17 +02:00
André Roth
a0bd32a39c system tests: fix expired ppa mirror key 2024-07-31 22:16:00 +02:00
André Roth
0b3dd2709b apply PR feedback 2024-07-31 22:16:00 +02:00
André Roth
ff1557afee update go.mod 2024-07-31 22:16:00 +02:00
André Roth
67771795ca etcd: implement transactions
- use temporary db for lookups in transactions
- use batch implementation to commit transaction
2024-07-31 22:16:00 +02:00
André Roth
7a01c9c62d etcd: implement batch operations
- cache the operations internally in a list
- Write() applies the list to etcd
2024-07-31 22:16:00 +02:00
André Roth
9768ecef22 etcd: implement temporary db support
- temporary db support is implemented with a unique key prefix
- prevent closing etcd connection when closing temporary db
2024-07-31 22:16:00 +02:00
André Roth
640c202ee5 etcd: implement separate system tests
- add t13_etcd test directory
- etcd will be started for the unit tests and each system test
- etcd will load fixture DB export if requested by the test
- existing tests are reused for etcd testing
2024-07-31 22:16:00 +02:00
André Roth
f10acb3df8 etcd: fix db config and test
fix unit test for adapted config file
2024-07-31 22:16:00 +02:00
André Roth
5b74f82edb etcd: fix int overflow
goxc fails with:

Error: database/etcddb/database.go:17:25: cannot use 2048 * 1024 * 1024 (untyped int constant 2147483648) as int value in struct literal (overflows)
2024-07-31 22:16:00 +02:00
hudeng
59bf4501e8 feat: Use databaseBackend config repace databaseEtcd
databaseBackend config contains type and url sub config, It can facilitate the expansion of other types of databases in the future.
2024-07-31 22:16:00 +02:00
hudeng
f29449db14 feat: Add system test for etcd 2024-07-31 22:16:00 +02:00
hudeng
78172d11d7 feat: Add etcd database support
improve concurrent access and high availability of aptly with the help of the characteristics of etcd
2024-07-31 22:16:00 +02:00
NeroBurner
f42ff697d4 CONTRIBUTING: Python3 is supportet
Explicitly state that Python3 is supported and required.

Since aptly `v1.5.1` (with commit 035d5314b0)
the tests are ported to Python3.

With a687df2f4f the system
tests are run with `python3` per default.

With f4a152ab22 (after `v1.5.0` aptly tag)
the tests run against Python 3.11.
2024-07-25 10:10:05 +02:00
André Roth
57e2c5c670 api tests: use check_task_fail 2024-07-24 21:19:47 +02:00
André Roth
8b41ec48c8 use consistent golangci-lint version 2024-07-24 21:19:47 +02:00
André Roth
49ff832f94 reenable lost tests 2024-07-24 21:19:47 +02:00
André Roth
09a44ba409 fix empty mirror check 2024-07-24 21:19:47 +02:00
André Roth
deae90485a fix DirIsAccessible
perms 0000 need to be checked explicitly
2024-07-24 21:19:47 +02:00
André Roth
4a0bdcbb64 improve system tests
- log import errors for test modules
- log output only on test failure
- improve docker system test container
- use go 1.19 in docker system tests
- download go dependencies in docker container
- system tests: color failues output
- imrpove test result output
- do not install golangci-lint in system tests
2024-07-24 21:19:47 +02:00
André Roth
9f1860dff7 fix unit test 2024-07-24 21:19:47 +02:00
André Roth
fe25414b45 api: repo copy handle package not found
and add tests for error proper handling.
2024-07-24 21:19:47 +02:00
André Roth
49184c9163 fix apiReposCopyPackage getting corrupt file name
it seems c.Params.ByName("file") should not be used
inside maybeRunTaskInBackground, as the content may be corrupted sometimes.
2024-07-24 21:19:47 +02:00
André Roth
440c3debdc improve api tests and error output
show only relevant aptly logs if a test fails.
for async tasks, show task output, as it contains the error message.
2024-07-24 21:19:47 +02:00
André Roth
8029305d32 dependencies: remove duplicates / missing deps from test 2024-07-11 18:25:49 +02:00
5hir0kur0
02bdb7c76a Deduplicate missing dependency list 2024-07-11 18:25:49 +02:00
5hir0kur0
8d537b4e3e Fix bug in dependency resolution 2024-07-11 18:25:49 +02:00
Sylvain Nieuwlandt
11401ca472 [api/copy] create system tests for new copy api endpoint 2024-07-10 16:43:03 +02:00
Valentin BRICE
66429bff45 [api/repos] Add copy API 2024-07-10 16:43:03 +02:00
Sylvain Nieuwlandt
8114786179 Declare the Copy API 2024-07-10 16:43:03 +02:00
André Roth
d8c1e432c6 add test 2024-07-03 18:08:58 +02:00
André Roth
3a286ae07f fix unit tests 2024-07-03 18:08:58 +02:00
André Roth
a93ccd4100 fix tests 2024-07-03 18:08:58 +02:00
André Roth
c1f7e5fe96 handle GpgDisableVerify and ignore-signatures consistently
and be less verbose
2024-07-03 18:08:58 +02:00
André Roth
d16110068c allow not signed mirrors without InRelease file 2024-07-03 18:08:58 +02:00
André Roth
4661913265 add system test for maximumVersion filter 2024-06-24 17:44:40 +02:00
hudeng
ecc88e7a40 feat: repo and snapshots packages filter api add 'maximumVersion' query parameter support
example: `curl http://localhost:8080/api/repos/test/packages\?maximumVersion\=1`

Change-Id: Ie9ffd36146bf017bbb353737f32360f7b73d6b0a
2024-06-24 17:44:40 +02:00
André Roth
5cf8c54cb2 fix test 2024-06-20 23:40:46 +02:00
André Roth
b758033ccb fix compilation 2024-06-20 23:40:46 +02:00
Kevin Martin
13f4bb441d Check if S3 bucket is encrypted by default.
Adds check to see if the S3 bucket is encrypted by default. If so this
uses the existing workaround for object etags not matching file MD5s.
2024-06-20 23:40:46 +02:00
Kevin Martin
1af09069f7 Check both MD5 locations for S3 KMS support.
If the S3 bucket used to house a repo has KMS encryption enabled then
the etag of an object may not match the MD5 of the file. This may
cause an incorrect error to be reported stating the file already
exists and is different.

A mechanism exists to work around this issue by using the MD5 stored
in object metadata. This check doesn't always cover the case where KMS
is enabled as the fallback is only used if the etag is not 32
characters long.

This commit changes the fallback mechanism so that it is used in any
case where the object's etag does not match the source MD5. This will
incur a performance penalty of an extra head request for each object
with a mismatch.
2024-06-20 23:40:46 +02:00
Ryan Gonzalez
b5bf2cbcda Fix functional tests' '--capture' on Python 3
None of the commands' output is ever treated as binary, so we can just
always decode it as text.

Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
2024-06-17 11:51:18 +02:00