Compare commits

..

202 Commits

Author SHA1 Message Date
André Roth 48907fc915 Merge pull request #1594 from aptly-dev/fix/s3-race
s3: fix race conditions
2026-06-19 13:31:12 +02:00
André Roth 6f92c51134 Merge pull request #1593 from aptly-dev/fix/repo-edit-name-optional
Fix/repo edit name optional
2026-06-19 13:01:14 +02:00
André Roth c63324756b s3: fix pathCache race condition
Make sure pathCache is properly locked for concurrent access.

Add RWMutex to the PublishedStorage struct:
- Cache initialization
  Read-lock to test for nil, then write-lock with a second nil check before populating
- Cache reads
  RLock/RUnlock, allowing concurrent readers
- Cache writes / deletes
  Lock/Unlock
2026-06-19 12:53:44 +02:00
André Roth cfa79a0a32 api: make updating name optional in repo edit
and path escape the new name param
2026-06-19 11:47:07 +02:00
André Roth 1b177bbf68 tests: remove temporary folders 2026-06-19 11:47:07 +02:00
André Roth c0c4bc3e35 Merge pull request #1584 from aptly-dev/dependabot/go_modules/go.opentelemetry.io/otel/sdk-1.43.0
build(deps): bump go.opentelemetry.io/otel/sdk from 1.39.0 to 1.43.0
2026-06-19 10:07:31 +02:00
dependabot[bot] 64e298e737 build(deps): bump go.opentelemetry.io/otel/sdk from 1.39.0 to 1.43.0
Bumps [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) from 1.39.0 to 1.43.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.39.0...v1.43.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-version: 1.43.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-19 00:14:12 +00:00
André Roth 423fb6e813 Merge pull request #1590 from aptly-dev/dependabot/go_modules/github.com/go-git/go-git/v5-5.19.1
build(deps): bump github.com/go-git/go-git/v5 from 5.14.0 to 5.19.1
2026-06-19 02:12:40 +02:00
dependabot[bot] 6c4d7a75f9 build(deps): bump github.com/go-git/go-git/v5 from 5.14.0 to 5.19.1
Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.14.0 to 5.19.1.
- [Release notes](https://github.com/go-git/go-git/releases)
- [Changelog](https://github.com/go-git/go-git/blob/main/HISTORY.md)
- [Commits](https://github.com/go-git/go-git/compare/v5.14.0...v5.19.1)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
  dependency-version: 5.19.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-18 23:26:52 +00:00
André Roth a743f03242 go: use version 1.25.0 2026-06-19 01:23:28 +02:00
André Roth d9bedfa478 Merge pull request #1585 from aptly-dev/dependabot/go_modules/github.com/go-jose/go-jose/v4-4.1.4
build(deps): bump github.com/go-jose/go-jose/v4 from 4.1.3 to 4.1.4
2026-06-19 00:12:38 +02:00
dependabot[bot] e8e3f944ea build(deps): bump github.com/go-jose/go-jose/v4 from 4.1.3 to 4.1.4
Bumps [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Commits](https://github.com/go-jose/go-jose/compare/v4.1.3...v4.1.4)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.1.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-18 21:10:07 +00:00
André Roth 3c98aa237a Merge pull request #1588 from aptly-dev/fix/source-package-list
fix empty line in Source PackageList
2026-06-18 23:07:37 +02:00
André Roth be68b15b74 Source files: fix empty line in Package-List 2026-06-18 22:28:43 +02:00
André Roth 4a19238df8 Merge pull request #1587 from aptly-dev/fix/check-storage-exists
publish: check if storage exists
2026-06-18 22:14:25 +02:00
André Roth dd85493b1a publish: check if storage exists 2026-06-18 21:34:55 +02:00
André Roth 29e643cdf6 Merge pull request #1586 from aptly-dev/dependabot/go_modules/go.opentelemetry.io/otel-1.41.0
build(deps): bump go.opentelemetry.io/otel from 1.39.0 to 1.41.0
2026-06-18 20:22:46 +02:00
dependabot[bot] 9bb765d048 build(deps): bump go.opentelemetry.io/otel from 1.39.0 to 1.41.0
Bumps [go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go) from 1.39.0 to 1.41.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.39.0...v1.41.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel
  dependency-version: 1.41.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-18 17:24:49 +00:00
André Roth b9d4aed7d8 Merge pull request #1553 from linuxdataflow/feat/pls/jfrog-support
Add support for JFrog artifactory.
2026-06-18 19:05:07 +02:00
iofq 160d16d96c docs: Update JFrog Artifactory documentation in config 2026-06-18 18:18:31 +02:00
iofq 4655f10048 feat: Allow Jfrog Artifactory authentication to be configured via env var 2026-06-18 18:18:31 +02:00
Pierig Le Saux 3333a643cb fix(jfrog): fix LinkFromPool source path handling and restore YAML config tests 2026-06-18 18:18:31 +02:00
Pierig Le Saux acc79d2cf6 fix: upgrade ulikunitz/xz to v0.5.15 to fix 32-bit build failures
v0.5.14 introduced a `1 << 31` constant assigned to an `int` variable
in lzma/reader.go, which overflows on 32-bit architectures (i386, armhf).
v0.5.15 fixes this by using int64.
2026-06-18 18:18:31 +02:00
Pierig Le Saux 9502f3833f test(jfrog): cover JFrog storage init error path and fix branch lint/coverage hygiene 2026-06-18 18:18:31 +02:00
Pierig Le Saux 7a2a82c60e more code coverage 2026-06-18 18:18:31 +02:00
Pierig Le Saux 7c47b8662f update 2026-06-18 18:18:31 +02:00
Pierig Le Saux ce070ec010 fix: attempt to fox code coverage 2026-06-18 18:18:31 +02:00
Pierig Le Saux fbad25e2b5 fix: add jfrog parameters to default config generation test 2026-06-18 18:18:31 +02:00
Pierig Le Saux 041eeff67d test: update ConfigShowTest_gold with JFrogPublishEndpoints 2026-06-18 18:18:31 +02:00
Pierig Le Saux 6244747912 chore: fix flake8 linting errors in jfrog test wrappers 2026-06-18 18:18:31 +02:00
André Roth b28daa8417 initial commit for JFrog support
# Conflicts:
#	api/api_test.go
#	api/router.go
#	go.mod
#	go.sum
#	utils/config_test.go
2026-06-18 18:18:31 +02:00
Pierig Le Saux 214c151194 fix: resolve golangci-lint failures in jfrog storage implementation
- Remove unused pathCache field from PublishedStorage struct
- Remove unused md5s accumulation in Filelist()
- Fix unchecked error return on expectedOut.Write in config_test.go
- Suppress unused linter on configFileYAML/configFileYAMLError constants
  (retained to avoid merge conflicts with feat/pls/gcs-support)
- Add --timeout=10m to golangci-lint workflow
- add back removed tests
2026-06-18 18:18:31 +02:00
Pierig Le Saux 4b8f0c42ac initial commit for JFrog support 2026-06-18 13:38:03 +02:00
André Roth 8ebe80f066 Merge pull request #1550 from linuxdataflow/feat/pls/gcs-support
Add support for GCS.
2026-06-18 09:33:48 +02:00
Pierig Le Saux b0c65112cb another lint fix + timeout 2026-06-17 23:39:27 +02:00
Pierig Le Saux 46e9ac65fa missed some files for tests 2026-06-17 23:39:27 +02:00
Pierig Le Saux a2fb925a94 fix golangci ? not too sure what's going on here 2026-06-17 23:39:27 +02:00
Pierig Le Saux b4a171b3ea gcs: pin fake-gcs-server to v1.53.1, restore go 1.24 directive 2026-06-17 23:39:27 +02:00
Pierig Le Saux 83b1c1b6cb gcs: add fake-gcs-server test suite, raise coverage 2026-06-17 23:39:27 +02:00
André Roth 7dd5c34254 go: mod tidy 2026-06-17 23:39:27 +02:00
Pierig Le Saux e04df59065 Sync GCS config goldens with publish endpoint changes 2026-06-17 23:39:27 +02:00
Pierig Le Saux c02c469fa6 feat(gcs): implement GCS publish endpoint support and update config/show/system tests 2026-06-17 23:39:27 +02:00
Pierig Le Saux d5fbf0f795 feat: add GCS publish backend with config, API, docs, and tests 2026-06-17 23:39:27 +02:00
Pierig Le Saux ed4af9a0f6 add support for gcs 2026-06-17 23:39:26 +02:00
André Roth 9bb8de68d0 Merge pull request #1436 from benallard/dput_api
Add API endpoint suitable for dput.
2026-06-17 23:31:04 +02:00
André Roth 00e75ebf3c test(dput): add system test for PUT /api/files/:dir/:file via dput 2026-06-17 21:29:06 +02:00
André Roth c2b0379e91 docs(dput): add missing @Router swagger annotation to apiFilesUploadOne 2026-06-17 21:29:06 +02:00
André Roth 84a5c20abe fix(dput): call syncFile to surface ENOSPC on upload 2026-06-17 21:29:06 +02:00
André Roth d3a613c335 fix(dput): replace manual read loop with io.Copy 2026-06-17 21:29:06 +02:00
André Roth 4e4ca0f38e fix(dput): validate :file path param to prevent directory traversal 2026-06-17 21:29:06 +02:00
Benoît Allard 12390f102e Add API endpoint suitable for dput. 2026-06-17 21:29:06 +02:00
André Roth 1224708283 Merge pull request #1583 from aptly-dev/docs/integrations/jenny
Fix mistake translation documentation integration
2026-06-17 21:14:26 +02:00
André Roth 1c1de6564b Merge pull request #1574 from aptly-dev/fix/task-race
Fix concurrent API race conditions
2026-06-17 18:53:16 +02:00
Clément 9c121c8111 english 2026-06-16 10:47:31 +02:00
André Roth 2605dd24f5 Merge pull request #1579 from aptly-dev/docs/integrations/jenny
Add Integration links : Jenny, an APT repository manager based on aptly
2026-06-16 09:24:52 +02:00
Clément 781eaff397 french language 2026-06-15 13:36:24 +02:00
Clément 33c06b1b4a Add description 2026-06-15 13:34:01 +02:00
André Roth caed9c234d publish: support MultiDist toggle 2026-06-13 14:48:01 +02:00
André Roth af1df410a2 tasks: fix race conditions
* show resources in task details
* fix task state locking
* return task object consistently

Race condition iexisted where task State, err, and processReturnValue fields
were written by consumer goroutine and read by concurrent accessors without
proper synchronization, causing torn reads and data races.
2026-06-13 14:47:58 +02:00
André Roth dfa34157a0 mirror: fix race conditions
* load data inside background tasks
  Perform collection.LoadComplete inside maybeRunTaskInBackground
  Have tasks use a fresh copy of taskCollectionFactory, taskCollection
2026-06-13 14:47:58 +02:00
André Roth 722064363c snapshot: fix race conditions
* perform collection.LoadComplete inside maybeRunTaskInBackground
 * have tasks use a fresh copy of taskCollectionFactory, taskCollection
 * fix locking for snapshots of snapshots by locking SourceSnapshots
 * use uuids, since names can be renamed
2026-06-13 14:47:58 +02:00
André Roth 9771f228bc repos: fix race conditions
* load data inside background tasks
  Perform collection.LoadComplete inside maybeRunTaskInBackground
  Have tasks use a fresh copy of taskCollectionFactory, taskCollection
* use uuids, since names can be renamed
2026-06-13 14:47:58 +02:00
André Roth 6ee51b6454 publish: fix race conditions
* remove useless resource lock
  Resource locks need to be before the background task. creating same publish endpoint at the same time is unlikely...
* load data inside background tasks
  This fixes a flaw in async apis, which loaded the published repo from the DB and mutated it outside the task closure, before the task lock was acquired.
  Perform collection.LoadComplete inside maybeRunTaskInBackground and have tasks use a fresh copy of taskCollectionFactory, taskCollection
* lock source repos/snapshots for publish operations
  Concurrent tasks were not properly locking their resources, leading to inconsistent published indexes:
  SourceLocalRepo: iterate published.Sources (component -> source UUID), look up each local repo via localRepoCollection.ByUUID and append string(repo.Key()) to resources
  SourceSnapshot: iterate b.Snapshots,look up each snapshot via snapshotCollection.ByName and append string(snapshot.ResourceKey()) to resources.
* lock pool on non MultiDist publish
* revert mutex on LinkFromPool
* use uuids, since names can be renamed
* add test for MultiDist change
2026-06-13 14:47:40 +02:00
Clément ab150890cb Add jenny by group edf links 2026-06-09 09:53:21 +02:00
André Roth 2974558aa7 cleanup 2026-06-07 23:46:43 +02:00
André Roth 00773f9840 ci: update codecov-action to 7.0.0 2026-06-07 22:53:09 +02:00
André Roth 8dc61cf362 ci: use correct ubuntu 26.04 codename 2026-05-17 10:16:01 +02:00
André Roth 4a9ddbdc34 Merge pull request #1565 from muresan/fix/aptly-crash-db-recover
Crash in aptly db recover
2026-05-15 16:51:51 +02:00
André Roth c316ea9b73 Merge pull request #1567 from aptly-dev/fix/doc-typos
docs: fix typos
2026-05-15 16:49:14 +02:00
André Roth d027a251ba Merge pull request #1571 from aptly-dev/feature/ubu26.04
ci: build for ubuntu 26.04
2026-05-15 16:48:55 +02:00
André Roth 16b6348710 ci: build for ubuntu 26.04 2026-05-15 00:05:35 +02:00
Catalin Muresan 1c1abe6b10 Added tests to please codeconv 2026-05-14 23:33:27 +02:00
Catalin Muresan c4bfbe52ca Fix crash in aptly db recover 2026-05-14 23:33:27 +02:00
André Roth c723fea807 docs: fix typos 2026-05-04 11:35:55 +02:00
André Roth 0d31298f37 Merge pull request #1568 from aptly-dev/fix/launchpad-test-dependency
Fix/launchpad test dependency
2026-05-04 11:30:56 +02:00
André Roth bba6bd7db5 system tests: do not depend on launchpad.net 2026-05-04 11:05:04 +02:00
André Roth faeaad0378 config: allow setting PPA Base URL 2026-05-04 11:05:04 +02:00
André Roth a20eb6866a document prometheus API
* enable in dev and test env
* fix api/repos doc
2026-04-26 23:56:05 +02:00
André Roth 809ab47042 Merge pull request #1559 from linuxdataflow/feat/pls/api-package-count
feat(api): add NumPackages to mirrors/repos/snapshots list responses
2026-04-26 18:39:24 +02:00
André Roth 0b84009b4a tests: add new arguments 2026-04-26 18:37:36 +02:00
Pierig Le Saux 92d7561d49 test(api): add coverage for NumPackages list handlers and error paths 2026-04-26 18:37:36 +02:00
Pierig Le Saux e908531bef feat(api): add NumPackages to mirrors/repos/snapshots list responses
add API response wrappers with NumPackages derived from RefList length; keep show endpoint payloads unchanged for backward compatibility; add API tests for list endpoint NumPackages; update swagger response schemas for list endpoints
2026-04-26 18:37:36 +02:00
André Roth f8620d10b2 Merge pull request #1558 from linuxdataflow/feat/pls/gpg-list-and-delete
list and delete gpg keys
2026-04-26 18:36:30 +02:00
Pierig Le Saux 8be72b48a1 update 2026-04-26 17:44:25 +02:00
Pierig Le Saux 5655480e00 add codecoverage 2026-04-26 17:44:25 +02:00
Pierig Le Saux 3c8defa304 update 2026-04-26 17:44:25 +02:00
Pierig Le Saux 1ed50697ec fix: delete is interactive 2026-04-26 17:44:25 +02:00
Pierig Le Saux 3b432d42b5 documentation 2026-04-26 17:44:25 +02:00
Pierig Le Saux 89e3bdfa07 delete a gpg key 2026-04-26 17:44:25 +02:00
André Roth f8d2d3cb8d fix lint errors 2026-04-26 17:41:12 +02:00
André Roth 01004e19c0 Merge pull request #1546 from aptly-dev/dependabot/go_modules/google.golang.org/grpc-1.79.3
build(deps): bump google.golang.org/grpc from 1.64.1 to 1.79.3
2026-04-26 17:11:45 +02:00
dependabot[bot] 92bb28149c build(deps): bump google.golang.org/grpc from 1.64.1 to 1.79.3
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.64.1 to 1.79.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.64.1...v1.79.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-26 14:43:41 +00:00
André Roth 652210acfa Merge pull request #1554 from aptly-dev/dependabot/go_modules/github.com/aws/aws-sdk-go-v2/service/s3-1.97.3
build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.67.1 to 1.97.3
2026-04-26 16:38:23 +02:00
André Roth 45f3da256b Merge pull request #1543 from PhilipCramer/feat/appstream-mirror-support
Add appstream support
2026-04-26 15:36:07 +02:00
dependabot[bot] 3c5e83366a build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.67.1 to 1.97.3.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.67.1...service/s3/v1.97.3)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.97.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-26 13:12:54 +00:00
Philip Cramer a7a4bb7001 test: improve test coverage for AppStream feature 2026-04-26 15:04:38 +02:00
Philip Cramer 2f7f726d4c fix: reject AppStream flag for flat repos instead of silently skipping 2026-04-26 15:04:38 +02:00
Philip Cramer 43d7284657 docs: update man page and AUTHORS for AppStream support 2026-04-26 15:04:37 +02:00
Philip Cramer 02423af931 fix: prevent db cleanup from deleting AppStream pool files 2026-04-26 15:04:17 +02:00
Philip Cramer 2a228625e2 test: add system test for AppStream publish pass-through 2026-04-26 15:04:17 +02:00
Philip Cramer 16e0302f30 test: update snapshot golden files for AppStream field 2026-04-26 15:04:17 +02:00
Philip Cramer 6ecbc9ba90 test: add system tests for AppStream mirror create, edit, and update 2026-04-26 15:04:17 +02:00
Philip Cramer 7276b9621f feat: add --with-appstream to bash/zsh shell completions 2026-04-26 15:04:17 +02:00
Philip Cramer fb7734b5b0 test: add unit tests for AppStream pass-through feature 2026-04-26 15:04:17 +02:00
Philip Cramer 29c37293b9 feat: wire AppStream support through CLI, API, and publish 2026-04-26 15:04:17 +02:00
Philip Cramer f25ba2e6b0 feat: propagate AppStreamFiles through snapshots 2026-04-26 15:04:17 +02:00
Philip Cramer 6a5b9ddacf feat: add AppStream (DEP-11) download support to RemoteRepo 2026-04-26 15:04:17 +02:00
André Roth 48355f65ed Merge pull request #1542 from tonobo/reproducible-publish
Add SOURCE_DATE_EPOCH support for GPG signers
2026-04-26 15:01:27 +02:00
Tim Foerster d616977904 Add SOURCE_DATE_EPOCH support for GPG signers
Both the external GPG signer (--faked-system-time) and internal Go
OpenPGP signer (signerConfig.Time) now honor SOURCE_DATE_EPOCH,
producing reproducible signatures alongside the plain Release file dates.

Adds system tests for both signer backends verifying byte-identical
Release, Release.gpg and InRelease across repeated publishes.

The signer tests (PublishRepo3[78]Test) are using an ed25519 key because
ed25519 signatures are deterministic by design. The Go openpgp library
uses a random nonce for DSA/ECDSA (see signature.go Sign calls using
config.Random() link below) so those signatures vary across runs
even with a fixed timestamp, making byte-identical verification impossible.

In addition to 49f342878a
Ref: https://github.com/aptly-dev/aptly/pull/1537
Ref: https://github.com/ProtonMail/go-crypto/blob/v1.4.0/openpgp/packet/signature.go#L945-L979
2026-04-26 14:32:23 +02:00
André Roth 3c068febde Merge pull request #1541 from aptly-dev/dependabot/go_modules/github.com/cloudflare/circl-1.6.3
build(deps): bump github.com/cloudflare/circl from 1.6.1 to 1.6.3
2026-04-26 14:31:08 +02:00
dependabot[bot] 76adbe49e0 build(deps): bump github.com/cloudflare/circl from 1.6.1 to 1.6.3
Bumps [github.com/cloudflare/circl](https://github.com/cloudflare/circl) from 1.6.1 to 1.6.3.
- [Release notes](https://github.com/cloudflare/circl/releases)
- [Commits](https://github.com/cloudflare/circl/compare/v1.6.1...v1.6.3)

---
updated-dependencies:
- dependency-name: github.com/cloudflare/circl
  dependency-version: 1.6.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-26 12:00:02 +00:00
André Roth f6221a2413 Merge pull request #1535 from lecafard/push-qxtqtunqqqnu
Add edit mirror API endpoint
2026-04-26 13:57:49 +02:00
André Roth 4f46cb04f5 Merge pull request #1561 from russelltg/docs
fix docs for Serve in API mode
2026-04-25 20:07:40 +02:00
Russell Greene 66e814c086 fix docs for Serve in API mode 2026-04-13 10:59:47 -06:00
tom b3f5d96490 Add edit mirror API endpoint 2026-04-12 22:26:58 +02:00
André Roth 144265122a Merge pull request #1533 from xzhang1/add-version
`Release` file: support `Version` field
2026-04-12 20:09:39 +02:00
André Roth 4f95d75c37 Merge pull request #1547 from aptly-dev/dependabot/pip/system/requests-2.33.0
build(deps): bump requests from 2.32.4 to 2.33.0 in /system
2026-04-12 20:06:48 +02:00
dependabot[bot] 8db1d2e7f1 build(deps): bump requests from 2.32.4 to 2.33.0 in /system
Bumps [requests](https://github.com/psf/requests) from 2.32.4 to 2.33.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.4...v2.33.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.33.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-12 17:26:36 +00:00
André Roth 4088a811cd ci: do not upload coverage for dependabot 2026-04-12 18:55:54 +02:00
André Roth 2ac4c75fad ci: fix coverage 2026-03-28 15:17:47 +01:00
Zhang Xiao e2ebcbb02a Release file: support Version field
https://wiki.debian.org/DebianRepository/Format#Version

The Version field, if specified, shall be the version of the release.
On the other hand, if not set or set to an empty value, the Version
field will not be included in the Release file.

Signed-off-by: Zhang Xiao <xiao.zhang@windriver.com>
2026-03-03 07:38:44 +00:00
André Roth 9defe70190 Merge pull request #1537 from tonobo/reproducible-publish
Add SOURCE_DATE_EPOCH support for reproducible builds
2026-02-28 12:46:57 +01:00
André Roth 23943d47e9 Merge pull request #1531 from xzhang1/fix-testcase
fix `t06_publish/PublishSnapshot43Test_gold` gold error
2026-02-28 11:44:32 +01:00
Tim Foerster 49f342878a Add SOURCE_DATE_EPOCH support for reproducible builds
Implement support for the SOURCE_DATE_EPOCH environment variable as
specified by reproducible-builds.org. When set, this variable overrides
the current timestamp in the Release file's Date and Valid-Until fields,
enabling reproducible filesystem publishes.

- Read SOURCE_DATE_EPOCH environment variable in Publish()
- Use the epoch timestamp for both Date and Valid-Until fields
- Gracefully fallback to current time if unset or invalid
- Add comprehensive tests for valid and invalid SOURCE_DATE_EPOCH values
2026-02-20 07:24:52 +01:00
Zhang Xiao 1f29c65a95 fix t06_publish/PublishSnapshot43Test_gold gold error
Signed-off-by: Zhang Xiao <xiao.zhang@windriver.com>
2026-02-03 07:12:04 +00:00
André Roth a65f79eb79 Merge pull request #1479 from abregar/issues-309-691
Add config key for 'gpgKeys' and allow multiple keyRefs when signing with gpg, fixing Issues #309 and #691
2026-01-26 11:58:03 +01:00
André Roth c6a9f82358 multi sign: add test 2026-01-26 11:26:40 +01:00
Ales Bregar 1702537979 clearer REST api docs, put whitespace to docs to show that keyId strings are trimmed 2026-01-24 10:55:15 +01:00
Ales Bregar 12604b9379 updating REST api with multiple gpg keys support, due backwards compatibility introducing CSV under same key (gpg-key) 2026-01-24 10:55:15 +01:00
Ales Bregar 9b523e6bd5 review fix 2026-01-24 10:55:15 +01:00
Ales Bregar aa030e3f36 system test t12_api sends empty keyRef string, making gpg fail 2026-01-24 10:55:15 +01:00
Ales Bregar 8e739524b0 system test unexpected string fix (would be helpful, but not changing the test just for this) 2026-01-24 10:55:15 +01:00
Ales Bregar 4ba4c0cba6 system test configuration fix 2026-01-24 10:55:15 +01:00
Ales Bregar 48d02918c1 documentation updated 2026-01-24 10:55:15 +01:00
Ales Bregar d672bfa317 white space revert to minimize change 2026-01-24 10:55:15 +01:00
Ales Bregar 9a90038dd2 - #309 adding gpgKeys config key, accepting array of keyRef, cli args has precedence
- #691 adding handling of multiple keyRefs when signing with gpg
2026-01-24 10:55:15 +01:00
André Roth 7d23196f76 Merge pull request #1529 from aptly-dev/fix/deadlocks
tasklist: fix deadlocks
2026-01-20 21:28:00 +01:00
André Roth 2c6812934e tasklist: fix deadlocks
* lock correct resources
* unlock list before queueing
2026-01-18 19:31:26 +01:00
André Roth 60f3eb151b Merge pull request #1484 from xzhang1/update_label
Support updating label and origin domain of publish
2026-01-17 11:44:32 +01:00
André Roth 0db9797c4e add test 2026-01-16 14:50:09 +01:00
Zhang Xiao a2ffffedc1 Support updating label and origin domain of publish
Signed-off-by: Zhang Xiao <xiao.zhang@windriver.com>
2026-01-16 14:50:09 +01:00
André Roth 19b98c62c1 Merge pull request #1504 from bwitt/error_outofspace
Return error on out of space
2026-01-16 14:49:20 +01:00
André Roth 06fea598e1 ci: fail on failed coverage upload 2026-01-16 13:15:43 +01:00
André Roth 9a6f06d23e unit-test: use /smallfs when non-root 2026-01-11 21:19:24 +01:00
André Roth 67f6a0e458 ci: provide 1MB /smallfs to docker 2026-01-11 21:19:24 +01:00
André Roth a57e2aecd7 ci: run unit tests in docker
- run separate unit-test job
- build docker
- allow make docker-unit-tests in ci
2026-01-11 21:19:13 +01:00
Brian Witt 1e7c15b69b error on out of space 2026-01-11 14:26:56 +01:00
André Roth a70f572efd docs: update PR tempalte 2026-01-11 14:02:24 +01:00
André Roth 8c183b45c6 Merge pull request #1524 from jay7x/patch-1
Update Puppet module references in README
2026-01-11 13:06:46 +01:00
Yury Bushmelev a8f7f58dab Update Puppet module references in README
Removed outdated Puppet module references and added an actively
maintained one.
2026-01-07 13:52:12 +08:00
André Roth 31fe26de5e Merge pull request #1461 from aptly-dev/dependabot/go_modules/github.com/cloudflare/circl-1.6.1
build(deps): bump github.com/cloudflare/circl from 1.4.0 to 1.6.1
2026-01-04 18:14:13 +01:00
André Roth eb1b770dc2 Merge pull request #1506 from aptly-dev/dependabot/go_modules/golang.org/x/crypto-0.45.0
build(deps): bump golang.org/x/crypto from 0.36.0 to 0.45.0
2026-01-04 17:56:40 +01:00
dependabot[bot] abb2ad635f build(deps): bump golang.org/x/crypto from 0.36.0 to 0.45.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.36.0 to 0.45.0.
- [Commits](https://github.com/golang/crypto/compare/v0.36.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-04 16:30:54 +00:00
dependabot[bot] a75df0a697 build(deps): bump github.com/cloudflare/circl from 1.4.0 to 1.6.1
Bumps [github.com/cloudflare/circl](https://github.com/cloudflare/circl) from 1.4.0 to 1.6.1.
- [Release notes](https://github.com/cloudflare/circl/releases)
- [Commits](https://github.com/cloudflare/circl/compare/v1.4.0...v1.6.1)

---
updated-dependencies:
- dependency-name: github.com/cloudflare/circl
  dependency-version: 1.6.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-04 16:29:56 +00:00
André Roth ea797f8ebe Merge pull request #1518 from LebedevRI/inrelease-signedby
`InRelease` file: support `Signed-By` field
2026-01-04 12:46:35 +01:00
Roman Lebedev a4cc9211d6 InRelease file: support Signed-By field
https://wiki.debian.org/DebianRepository/Format#Signed-By says:
> **Signed-By**
> An optional field containing a comma separated list of
> OpenPGP key fingerprints to be used for validating
> the next Release file. The fingerprints must consist
> only of hex digits and may not contain spaces.
> The fingerprint specifies either the key the Release file
> must be signed with or the key the signature key must be
> a subkey of. The later match can be disabled by appending
> an exclamation mark to the fingerprint.
>
> If the field is present, a client should only accept future updates
> to the repository that are signed with keys listed in the field.
> The field should be ignored if the Valid-Until field is not present
> or if it is expired.

For both the CLI tools and JSON, the field is taken as a string verbatim.

When specified, we must also provide `Valid-Until` field,
and i'm not sure there is an 'infinity' value for it,
so 100 years will have to do?

Fixes https://github.com/aptly-dev/aptly/issues/1497
2025-12-30 06:06:48 +03:00
André Roth 836d9f3b8b Merge pull request #1509 from thunder-coding/dpkg-compliant-version-compare
make version comparision more similar to that of dpkg
2025-12-26 16:56:11 +01:00
André Roth 61650e5b3b Merge branch 'master' into dpkg-compliant-version-compare 2025-12-26 16:55:52 +01:00
André Roth 4e457aa570 Merge pull request #1513 from gaby/add-latest-flag-to-makecmdmirrorupdate
feat: Support for only downloading the latest version of packages
2025-12-26 16:26:43 +01:00
André Roth fe70da9c08 Merge branch 'master' into add-latest-flag-to-makecmdmirrorupdate 2025-12-26 16:26:01 +01:00
André Roth 4b57e65658 Merge pull request #1505 from atotto/feature/mirror-from-google-artifact-registry
feat(http): add Google Artifact Registry authentication for ar+https scheme
2025-12-26 16:13:01 +01:00
Ato Araki bcd81eeae4 update AUTHORS 2025-12-17 12:57:17 +09:00
Ato Araki af483d1165 feat(http): add GCP authentication for ar+https scheme 2025-12-17 12:57:17 +09:00
André Roth 6b8651fda2 Merge pull request #1510 from LeiCraft/fix-swagger-pascalcase
Fix swagger property casing
2025-12-15 10:23:41 +01:00
Juan Calderon-Perez de699aebe5 Update AUTHORS list 2025-12-11 09:42:15 -05:00
Juan Calderon-Perez 0021cf876b Harden latest-only filtering 2025-12-11 07:20:37 -05:00
Linus Fischer 32b601bde6 Fix swagger property casing 2025-12-02 11:03:28 +00:00
Yaksh Bariya b464e7f80b give myself some credit as well
Cause I'm nice :)
2025-12-01 00:33:04 +05:30
Yaksh Bariya e0f282aca9 make version comparision more similar to that of dpkg
Initially found by automated repository health checks used by Termux
in https://github.com/termux/termux-packages/issues/27472

The root problem was 4.3.5a comparing less than 4.3.5-rc1-1 by aptly
According to debian "4.3.5a" > "4.3.5-rc1-1"

This is because dpkg splits hyphen for revision at the first hyphen,
whereas aptly was splitting at the last hyphen which is different from
dpkg's behaviour.

dpkg behaviour: https://git.dpkg.org/cgit/dpkg/dpkg.git/tree/lib/dpkg/parsehelp.c#n242

Perhaps this wasn't detected as there was broken tests in the repository
since the initial commit of aptly. This also fixes those tests
2025-12-01 00:28:37 +05:30
André Roth ba65daf6cb Merge pull request #1480 from alguimodd/s3-reupload-fix
feat(s3): add publishedPrefix to pathCache to avoid reupload of files
2025-11-11 19:00:31 +01:00
Christoph Fiehe b8455f6de9 Merge branch 'master' into s3-reupload-fix 2025-11-10 20:58:43 +01:00
André Roth 132c923f25 Merge pull request #1493 from Daedaluz/fix/update-repo-api
Editing repo via api 404
2025-11-08 13:10:48 +01:00
André Roth b6d83a4f61 Merge pull request #1492 from refi64/docker-test-fixes
Various fixes for system-test and docker integration
2025-10-19 20:33:34 +02:00
André Roth 4526d6d831 Merge pull request #1498 from chesseed/fix/swagger-errors
Fix Swagger errors
2025-10-19 20:30:42 +02:00
chesseed 02d2ba255c fix comment 2025-10-09 21:33:20 +02:00
chesseed d94792dd65 fix swagger errors 2025-10-09 21:26:30 +02:00
Tobias Assarsson 66eb75f492 fix repo edit api. 2025-09-24 16:40:23 +02:00
Ryan Gonzalez 33a2f70d07 system-test: Allow skipping coverage
Enabling coverage near-doubles the incremental build time and adds
overhead to individual tests on the order of **5-10x** or more. It's not
essential to have this for quick local system-test runs, so add an option
to disable it.
2025-09-23 16:25:46 -05:00
Ryan Gonzalez 10f942c8e0 system-test: Forward CAPTURE to docker
The code was only forwarding TEST, but CAPTURE is useful too.
2025-09-23 16:25:46 -05:00
Ryan Gonzalez 568a9ce4d5 docker: Preserve the go build cache
Otherwise, every `make docker-...` invocation will need to rebuild
everything from scratch.
2025-09-23 16:25:46 -05:00
Ryan Gonzalez ddf415a359 docker: Fix usage with rootless podman and SELinux
When using rootless podman, the *current user* gets mapped to uid 0,
which results in the aptly user being unable to write to the build
directory. We can instead map the current user to the corresponding uid
in the container via `PODMAN_USERNS=keep-id`, which matches up with what
docker-wrapper wants...but then that will *enter the container as the
current uid*, which messes with the ability to set permissions on
`/var/lib/aptly`. That can be fixed by explicitly passing `--user 0:0`,
which should be a no-op on docker (since the container's default user is
already root).

Additionally, this adds `--security-opt label=disable` to avoid
permission errors when running on systems with SELinux enforcing.
2025-09-23 16:25:46 -05:00
Ryan Gonzalez 29ac9c1919 system-test: Fix crash when a comparison with a non-string value fails
`orig` isn't necessarily a string, so the string concatenation here can
raise a TypeError.
2025-09-23 16:25:46 -05:00
André Roth d3bed7830c Merge pull request #1482 from JupiterRider/tautological_condition
remove useless nil check
2025-08-30 19:22:52 +02:00
JupiterRider c2d5f47643 Merge branch 'master' into tautological_condition 2025-08-30 18:51:27 +02:00
André Roth 731e92c8e4 Merge pull request #1483 from JupiterRider/gofmt
format the Go code with gofmt
2025-08-30 18:04:29 +02:00
André Roth 94a600c0c1 README: remove buster 2025-08-29 10:35:02 +02:00
André Roth e1d8ae8a35 Merge pull request #1472 from Yye847/master
Update README.rst
2025-08-29 10:34:33 +02:00
André Roth d3b7186dea Merge pull request #1481 from agustinhenze/master
Add mutex on LinkFromPool to fix #1449
2025-08-29 09:55:15 +02:00
Agustin Henze 3608c137a0 Add mutex on LinkFromPool to fix #1449
This fixes the race condition that happens when you call publish
concurrently. It adds a valuable test that reproduces the error almost
deterministically, it's hard to say always but I have run this in loop
100 times and it reproduces the error consistently without the patch and
after the patch it works consistently.
2025-08-28 10:40:53 +02:00
JupiterRider 15a3efe758 add JupiterRider to AUTHORS file 2025-08-20 19:48:29 +02:00
JupiterRider 4b73ae462f remove tautological (unnecessary) nil condition 2025-08-20 19:45:53 +02:00
JupiterRider b49a631e0b ran "gofmt -s -w ." to format the code 2025-08-20 19:41:26 +02:00
Alejandro Guijarro Monerris 12b6b04055 chore: add name to AUTHORS 2025-08-13 16:38:44 +02:00
Alejandro Guijarro Monerris a1f659bea0 feat(s3): add publishedPrefix to pathCache to avoid reupload of files 2025-08-13 16:35:34 +02:00
André Roth 8ca4cb8dcb ci: remove EOL debian/buster 2025-08-12 14:31:44 +02:00
André Roth 8ce8f250d5 update Releasing.md 2025-08-12 14:00:57 +02:00
Yye847 3672f6f92f Update README.rst
add trixie in list of available dists also in CLI part of README
2025-07-25 12:29:59 +02:00
Yye847 888a6b2caa Update README.rst
add trixie in list of available dists
2025-07-25 12:23:04 +02:00
André Roth 231039e86c Merge pull request #1466 from itayporezky/itay-api
Fix non related API params from mirror update
2025-07-20 23:32:41 +02:00
Itay Porezky dc884e6052 Removing non related actions from mirror update 2025-06-29 21:55:27 +03:00
André Roth 4675589cf6 Merge pull request #1460 from aptly-dev/dependabot/pip/system/requests-2.32.4
build(deps): bump requests from 2.28.2 to 2.32.4 in /system
2025-06-21 15:40:30 +02:00
dependabot[bot] 32f03bfd62 build(deps): bump requests from 2.28.2 to 2.32.4 in /system
Bumps [requests](https://github.com/psf/requests) from 2.28.2 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.28.2...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-10 03:23:26 +00:00
186 changed files with 7492 additions and 2427 deletions
+5
View File
@@ -4,6 +4,10 @@ Fixes #
All new code should be covered with tests, documentation should be updated. CI should pass.
Also, to speed up things, if you could kindly "Allow edits and access to secrets by maintainers" in the
PR settings, as this allows us to rebase the PR on master, fix conflicts, run coverage and help with
implementing code and tests.
## Description of the Change
<!--
@@ -14,6 +18,7 @@ Why this change is important?
## Checklist
- [ ] allow Maintainers to edit PR (rebase, run coverage, help with tests, ...)
- [ ] unit-test added (if change is algorithm)
- [ ] functional test added/updated (if change is functional)
- [ ] man page updated (if applicable)
+56 -20
View File
@@ -1,3 +1,4 @@
---
name: CI
on:
@@ -10,7 +11,6 @@ on:
defaults:
run:
# see: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#using-a-specific-shell
shell: bash --noprofile --norc -eo pipefail {0}
env:
@@ -18,7 +18,7 @@ env:
jobs:
unit-test:
name: "Unit Tests (Debian 13)"
name: "Unit Tests"
runs-on: ubuntu-22.04
continue-on-error: false
timeout-minutes: 30
@@ -31,17 +31,18 @@ jobs:
- name: "Docker Image"
run: |
make docker-image
- name: "Unit Test"
- name: "Unit Tests"
run: |
make docker-unit-test
- name: "Upload Code Coverage"
uses: codecov/codecov-action@v2
mkdir -p out/coverage
mv unit.out out/coverage/
- uses: actions/upload-artifact@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: unit.out
name: unit-tests-coverage
path: out/
test:
name: "Test (Ubuntu 22.04)"
name: "System Test"
runs-on: ubuntu-22.04
continue-on-error: false
timeout-minutes: 30
@@ -56,7 +57,7 @@ jobs:
- name: "Install Test Packages"
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends graphviz gnupg2 gpgv2 git gcc make devscripts python3 python3-requests-unixsocket python3-termcolor python3-swiftclient python3-boto python3-azure-storage python3-etcd3 python3-plyvel flake8 faketime
sudo apt-get install -y --no-install-recommends graphviz gnupg2 gpgv2 git gcc make devscripts python3 python3-requests-unixsocket python3-termcolor python3-swiftclient python3-boto python3-azure-storage python3-etcd3 python3-plyvel flake8 faketime dput-ng
- name: "Checkout Repository"
uses: actions/checkout@v4
@@ -88,7 +89,8 @@ jobs:
- name: "Run Benchmark"
run: |
COVERAGE_DIR=${{ runner.temp }} make bench
mkdir -p out/coverage
COVERAGE_DIR=$PWD/out/coverage make bench
- name: "Run System Tests"
env:
@@ -98,9 +100,40 @@ jobs:
AZURE_STORAGE_ACCESS_KEY: "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
JFROG_URL: ${{ secrets.JFROG_URL }}
JFROG_USERNAME: ${{ secrets.JFROG_USERNAME }}
JFROG_PASSWORD: ${{ secrets.JFROG_PASSWORD }}
run: |
sudo mkdir -p /srv ; sudo chown runner /srv
COVERAGE_DIR=${{ runner.temp }} make system-test
mkdir -p out/coverage
COVERAGE_DIR=$PWD/out/coverage make system-test
- uses: actions/upload-artifact@v4
with:
name: system-tests-coverage
path: out/
coverage:
name: "Upload Coverage"
runs-on: ubuntu-22.04
continue-on-error: false
timeout-minutes: 30
needs:
- unit-test
- test
steps:
- name: "Checkout Repository"
uses: actions/checkout@v4
- name: "Download Unit Test Coverage"
uses: actions/download-artifact@v4
with:
name: unit-tests-coverage
- name: "Download System Test Coverage"
uses: actions/download-artifact@v4
with:
name: system-tests-coverage
- name: "Merge Code Coverage"
run: |
@@ -109,23 +142,26 @@ jobs:
awk 'FNR==1 && NR!=1 {next} {print}' coverage/*.out > coverage.txt
- name: "Upload Code Coverage"
uses: codecov/codecov-action@v2
if: github.actor != 'dependabot[bot]'
uses: codecov/codecov-action@v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.txt
fail_ci_if_error: true
ci-debian-build:
name: "Build"
needs:
- test
- coverage
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
name: ["Debian 13/testing", "Debian 12/bookworm", "Debian 11/bullseye", "Debian 10/buster", "Ubuntu 24.04", "Ubuntu 22.04", "Ubuntu 20.04"]
name: ["Debian 13/trixie", "Debian 12/bookworm", "Debian 11/bullseye", "Ubuntu 26.04", "Ubuntu 24.04", "Ubuntu 22.04", "Ubuntu 20.04"]
arch: ["amd64", "i386" , "arm64" , "armhf"]
include:
- name: "Debian 13/testing"
- name: "Debian 13/trixie"
suite: trixie
image: debian:trixie-slim
- name: "Debian 12/bookworm"
@@ -134,9 +170,9 @@ jobs:
- name: "Debian 11/bullseye"
suite: bullseye
image: debian:bullseye-slim
- name: "Debian 10/buster"
suite: buster
image: debian:buster-slim
- name: "Ubuntu 26.04"
suite: resolute
image: ubuntu:26.04
- name: "Ubuntu 24.04"
suite: noble
image: ubuntu:24.04
@@ -148,6 +184,7 @@ jobs:
image: ubuntu:20.04
container:
image: ${{ matrix.image }}
options: --user root
env:
APT_LISTCHANGES_FRONTEND: none
DEBIAN_FRONTEND: noninteractive
@@ -240,8 +277,7 @@ jobs:
ci-binary-build:
name: "Build"
needs:
- unit-test
- test
- coverage
runs-on: ubuntu-latest
strategy:
matrix:
+4 -3
View File
@@ -35,16 +35,17 @@ jobs:
- name: Install and initialize swagger
run: |
go install github.com/swaggo/swag/cmd/swag@latest
swag init -q --markdownFiles docs
swag init -q --propertyStrategy pascalcase --markdownFiles docs
shell: sh
- name: golangci-lint
uses: golangci/golangci-lint-action@v4
uses: golangci/golangci-lint-action@v9
with:
# Require: The version of golangci-lint to use.
# When `install-mode` is `binary` (default) the value can be v1.2 or v1.2.3 or `latest` to use the latest version.
# When `install-mode` is `goinstall` the value can be v1.2.3, `latest`, or the hash of a commit.
version: v1.64.5
version: v2.12.2
args: --timeout=10m
# Optional: working directory, useful for monorepos
# working-directory: somedir
+2
View File
@@ -26,6 +26,8 @@ _testmain.go
*.test
coverage.txt
coverage.out
coverage.html
*.pyc
+2
View File
@@ -1,4 +1,6 @@
version: "2"
run:
timeout: 5m
linters:
settings:
staticcheck:
+8
View File
@@ -70,9 +70,17 @@ List of contributors, in chronological order:
* Gordian Schoenherr (https://github.com/schoenherrg)
* Silke Hofstra (https://github.com/silkeh)
* Itay Porezky (https://github.com/itayporezky)
* Alejandro Guijarro Monerris (https://github.com/alguimodd)
* JupiterRider (https://github.com/JupiterRider)
* Agustin Henze (https://github.com/agustinhenze)
* Tobias Assarsson (https://github.com/daedaluz)
* Yaksh Bariya (https://github.com/thunder-coding)
* Juan Calderon-Perez (https://github.com/gaby)
* Ato Araki (https://github.com/atotto)
* Roman Lebedev (https://github.com/LebedevRI)
* Brian Witt (https://github.com/bwitt)
* Ales Bregar (https://github.com/abregar)
* Tim Foerster (https://github.com/tonobo)
* Zhang Xiao (https://github.com/xzhang1)
* Tom Nguyen (https://github.com/lecafard)
* Philip Cramer (https://github.com/PhilipCramer)
+2 -2
View File
@@ -130,14 +130,14 @@ aptly version: 1.5.0+189+g0fc90dff
In order to run aptly unit tests, enter the following:
```
make docker-unit-tests
make docker-unit-test
```
#### Running system tests
In order to run aptly system tests, enter the following:
```
make docker-system-tests
make docker-system-test
```
#### Running golangci-lint
+5 -5
View File
@@ -2,7 +2,7 @@ GOPATH=$(shell go env GOPATH)
VERSION=$(shell make -s version)
PYTHON?=python3
BINPATH?=$(GOPATH)/bin
GOLANGCI_LINT_VERSION=v2.0.2 # version supporting go 1.24
GOLANGCI_LINT_VERSION=v2.12.2 # version supporting go 1.25
COVERAGE_DIR?=$(shell mktemp -d)
GOOS=$(shell go env GOHOSTOS)
GOARCH=$(shell go env GOHOSTARCH)
@@ -75,9 +75,9 @@ azurite-start:
azurite-stop:
@kill `cat ~/.azurite.pid`
swagger: #swagger-install
swagger: swagger-install
# Generate swagger docs
#@PATH=$(BINPATH)/:$(PATH) swag init --parseDependency --parseInternal --markdownFiles docs --generalInfo docs/swagger.conf
@PATH=$(BINPATH)/:$(PATH) swag init --propertyStrategy pascalcase --parseDependency --parseInternal --markdownFiles docs --generalInfo docs/swagger.conf
etcd-install:
# Install etcd
@@ -132,7 +132,7 @@ serve: prepare swagger-install ## Run development server (auto recompiling)
cp debian/aptly.conf ~/.aptly.conf
sed -i /enable_swagger_endpoint/s/false/true/ ~/.aptly.conf
sed -i /enable_metrics_endpoint/s/false/true/ ~/.aptly.conf
PATH=$(BINPATH):$$PATH air -build.pre_cmd 'swag init -q --markdownFiles docs --generalInfo docs/swagger.conf' -build.exclude_dir docs,system,debian,pgp/keyrings,pgp/test-bins,completion.d,man,deb/testdata,console,_man,systemd,obj-x86_64-linux-gnu -- api serve -listen 0.0.0.0:3142
PATH=$(BINPATH):$$PATH air -build.pre_cmd 'swag init -q --propertyStrategy pascalcase --markdownFiles docs --generalInfo docs/swagger.conf' -build.exclude_dir docs,system,debian,pgp/keyrings,pgp/test-bins,completion.d,man,deb/testdata,console,_man,systemd,obj-x86_64-linux-gnu -- api serve -listen 0.0.0.0:3142
dpkg: prepare swagger ## Build debian packages
@test -n "$(DEBARCH)" || (echo "please define DEBARCH"; exit 1)
@@ -241,4 +241,4 @@ clean: ## remove local build and module cache
rm -f unit.out aptly.test VERSION docs/docs.go docs/swagger.json docs/swagger.yaml docs/swagger.conf
find system/ -type d -name __pycache__ -exec rm -rf {} \; 2>/dev/null || true
.PHONY: help man prepare swagger version binaries build docker-release docker-system-tests docker-unit-test docker-lint docker-build docker-image docker-man docker-shell docker-serve clean releasetype dpkg serve flake8
.PHONY: help man prepare swagger version binaries build docker-release docker-system-test docker-unit-test docker-lint docker-build docker-image docker-man docker-shell docker-serve clean releasetype dpkg serve flake8
+10 -6
View File
@@ -63,7 +63,7 @@ Define Release APT sources in ``/etc/apt/sources.list.d/aptly.list``::
deb [signed-by=/etc/apt/keyrings/aptly.asc] http://repo.aptly.info/release DIST main
Where DIST is one of: ``buster``, ``bullseye``, ``bookworm``, ``focal``, ``jammy``, ``noble``
Where DIST is one of: ``bullseye``, ``bookworm``, ``trixie``, ``focal``, ``jammy``, ``noble``
Install aptly packages::
@@ -80,7 +80,7 @@ Define CI APT sources in ``/etc/apt/sources.list.d/aptly-ci.list``::
deb [signed-by=/etc/apt/keyrings/aptly.asc] http://repo.aptly.info/ci DIST main
Where DIST is one of: ``buster``, ``bullseye``, ``bookworm``, ``focal``, ``jammy``, ``noble``
Where DIST is one of: ``bullseye``, ``bookworm``, ``trixie``, ``focal``, ``jammy``, ``noble``
Note: same gpg key is used as for the Upstream Debian Packages.
@@ -111,10 +111,8 @@ With configuration management systems:
- `Chef cookbook <https://github.com/hw-cookbooks/aptly>`_ by Aaron Baer
(Heavy Water Operations, LLC)
- `Puppet module <https://github.com/alphagov/puppet-aptly>`_ by
Government Digital Services
- `Puppet module <https://github.com/tubemogul/puppet-aptly>`_ by
TubeMogul
- `Puppet module <https://github.com/voxpupuli/puppet-aptly>`_ by
Vox Pupuli
- `SaltStack Formula <https://github.com/saltstack-formulas/aptly-formula>`_ by
Forrest Alvarez and Brian Jackson
- `Ansible role <https://github.com/aioue/ansible-role-aptly>`_ by Tom Paine
@@ -135,3 +133,9 @@ Scala sbt:
Molior:
- `Molior Debian Build System <https://github.com/molior-dbs/molior>`_ by André Roth
Jenny:
- `Jenny, an APT repository manager, a tool to manage Debian package repositories for a Debian-like distribution <https://github.com/groupe-edf/Jenny>`_ by EDF Group
It handles incoming packages either built locally or mirrored from external sources, follows them through phases of development (called "environments"), and publishes them as repositories usable by apt and related tools.
This tool is currently only in French language.
+1
View File
@@ -13,5 +13,6 @@ git push origin v$version master
- run swagger locally (`make docker-serve`)
- copy generated docs/swagger.json to https://github.com/aptly-dev/www.aptly.info/tree/master/static/swagger/aptly_1.x.y.json
- add new version to select tag in content/doc/api/swagger.md line 48
- update version in content/download.md
- push commit to master
- create release announcement on https://github.com/aptly-dev/aptly/discussions
+53
View File
@@ -8,6 +8,7 @@ import (
"net/http"
"net/http/httptest"
"os"
"sort"
"strings"
"testing"
@@ -41,6 +42,22 @@ func createTestConfig() *os.File {
jsonString, err := json.Marshal(gin.H{
"architectures": []string{},
"enableMetricsEndpoint": true,
"S3PublishEndpoints": map[string]map[string]string{
"test-s3": {
"region": "us-east-1",
"bucket": "bucket-s3",
},
},
"GcsPublishEndpoints": map[string]map[string]string{
"test-gcs": {
"bucket": "bucket-gcs",
},
},
"JFrogPublishEndpoints": map[string]map[string]string{
"test-jfrog": {
"url": "http://jfrog.example.com",
},
},
})
if err != nil {
return nil
@@ -173,3 +190,39 @@ func (s *APISuite) TestTruthy(c *C) {
c.Check(truthy(-1), Equals, true)
c.Check(truthy(gin.H{}), Equals, true)
}
func (s *APISuite) TestGetJFrogEndpoints(c *C) {
response, err := s.HTTPRequest("GET", "/api/jfrog", nil)
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 200)
var endpoints []string
err = json.Unmarshal(response.Body.Bytes(), &endpoints)
c.Assert(err, IsNil)
sort.Strings(endpoints)
c.Check(endpoints, DeepEquals, []string{"test-jfrog"})
}
func (s *APISuite) TestGetS3Endpoints(c *C) {
response, err := s.HTTPRequest("GET", "/api/s3", nil)
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 200)
var endpoints []string
err = json.Unmarshal(response.Body.Bytes(), &endpoints)
c.Assert(err, IsNil)
sort.Strings(endpoints)
c.Check(endpoints, DeepEquals, []string{"test-s3"})
}
func (s *APISuite) TestGetGCSEndpoints(c *C) {
response, err := s.HTTPRequest("GET", "/api/gcs", nil)
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 200)
var endpoints []string
err = json.Unmarshal(response.Body.Bytes(), &endpoints)
c.Assert(err, IsNil)
sort.Strings(endpoints)
c.Check(endpoints, DeepEquals, []string{"test-gcs"})
}
+63
View File
@@ -185,6 +185,69 @@ func apiFilesUpload(c *gin.Context) {
c.JSON(200, stored)
}
// @Summary Upload One File
// @Description **Upload one file to a directory**
// @Description
// @Description - file is uploaded
// @Description - existing uploaded are overwritten
// @Description
// @Description **Example:**
// @Description ```
// @Description $ dput aptly aptly_0.9~dev+217+ge5d646c_i386.changes
// @Description ```
// @Tags Files
// @Param dir path string true "Directory to upload files to. Created if does not exist"
// @Param file path string true "File to upload"
// @Produce json
// @Success 200 {array} string "Name of uploaded file"
// @Failure 400 {object} Error "Bad Request"
// @Failure 404 {object} Error "Not Found"
// @Failure 500 {object} Error "Internal Server Error"
// @Router /api/files/{dir}/{file} [put]
func apiFilesUploadOne(c *gin.Context) {
if !verifyDir(c) {
return
}
fileName := c.Params.ByName("file")
if !verifyPath(fileName) {
AbortWithJSONError(c, 400, fmt.Errorf("wrong file"))
return
}
path := filepath.Join(context.UploadPath(), utils.SanitizePath(c.Params.ByName("dir")))
err := os.MkdirAll(path, 0777)
if err != nil {
AbortWithJSONError(c, 500, err)
return
}
stored := []string{}
destPath := filepath.Join(path, fileName)
dst, err := os.Create(destPath)
if err != nil {
AbortWithJSONError(c, 500, err)
return
}
defer func() { _ = dst.Close() }()
if _, err = io.Copy(dst, c.Request.Body); err != nil {
AbortWithJSONError(c, 500, err)
return
}
if err = syncFile(dst); err != nil {
AbortWithJSONError(c, 500, fmt.Errorf("error syncing file %s: %s", fileName, err))
return
}
stored = append(stored, filepath.Join(c.Params.ByName("dir"), fileName))
apiFilesUploadedCounter.WithLabelValues(c.Params.ByName("dir")).Inc()
c.JSON(200, stored)
}
// @Summary List Files
// @Description **Show uploaded files in upload directory**
// @Description
+21
View File
@@ -0,0 +1,21 @@
package api
import (
"github.com/gin-gonic/gin"
)
// @Summary GCS buckets
// @Description **Get list of GCS buckets**
// @Description
// @Description List configured GCS buckets.
// @Tags Status
// @Produce json
// @Success 200 {array} string "List of GCS buckets"
// @Router /api/gcs [get]
func apiGCSList(c *gin.Context) {
keys := []string{}
for k := range context.Config().GCSPublishRoots {
keys = append(keys, k)
}
c.JSON(200, keys)
}
+201
View File
@@ -1,6 +1,7 @@
package api
import (
"bufio"
"fmt"
"os"
"os/exec"
@@ -12,6 +13,23 @@ import (
"github.com/gin-gonic/gin"
)
type gpgKeyInfo struct {
// 16-character key ID (short form)
KeyID string `json:"KeyID" example:"8B48AD6246925553"`
// Full fingerprint
Fingerprint string `json:"Fingerprint" example:"D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0"`
// Key validity (u=unknown, f=fulltrust, m=marginal, n=never)
Validity string `json:"Validity" example:"u"`
// User ID(s) associated with this key
UserIDs []string `json:"UserIDs" example:"John Doe <john@example.com>"`
// Creation date (Unix timestamp format from gpg)
CreatedAt string `json:"CreatedAt" example:"2023-01-15"`
}
type gpgKeyListResponse struct {
Keys []gpgKeyInfo `json:"Keys"`
}
type gpgAddKeyParams struct {
// Keyring for adding the keys (default: trustedkeys.gpg)
Keyring string `json:"Keyring" example:"trustedkeys.gpg"`
@@ -25,6 +43,14 @@ type gpgAddKeyParams struct {
GpgKeyID string `json:"GpgKeyID" example:"EF0F382A1A7B6500 8B48AD6246925553"`
}
type gpgDeleteKeyParams struct {
// Keyring to delete keys from (default: trustedkeys.gpg)
Keyring string `json:"Keyring" example:"trustedkeys.gpg"`
// Key ID or fingerprint to delete
GpgKeyID string `json:"GpgKeyID" example:"8B48AD6246925553"`
}
// @Summary Add GPG Keys
// @Description **Adds GPG keys to aptly keyring**
// @Description
@@ -108,3 +134,178 @@ func apiGPGAddKey(c *gin.Context) {
c.JSON(200, string(out))
}
// @Summary List GPG Keys
// @Description **Lists all GPG keys in aptly keyring**
// @Description
// @Description Returns all public keys currently installed in the aptly GPG keyring.
// @Description
// @Tags Mirrors
// @Param keyring query string false "Keyring file to list keys from (default: trustedkeys.gpg)" example(trustedkeys.gpg)
// @Produce json
// @Success 200 {object} gpgKeyListResponse "OK"
// @Failure 400 {object} Error "Bad Request"
// @Router /api/gpg/keys [get]
func apiGPGListKeys(c *gin.Context) {
keyring := c.DefaultQuery("keyring", "trustedkeys.gpg")
keyring = utils.SanitizePath(keyring)
finder := pgp.GPGDefaultFinder()
gpg, _, err := finder.FindGPG()
if err != nil {
AbortWithJSONError(c, 400, err)
return
}
args := []string{
"--no-default-keyring",
"--with-colons",
"--keyring", keyring,
"--list-keys",
}
cmd := exec.Command(gpg, args...)
out, err := cmd.CombinedOutput()
if err != nil {
AbortWithJSONError(c, 400, fmt.Errorf("failed to list keys: %s", string(out)))
return
}
keys := parseGPGOutput(string(out))
c.JSON(200, gpgKeyListResponse{Keys: keys})
}
// @Summary Delete GPG Key
// @Description **Deletes a GPG key from aptly keyring**
// @Description
// @Description Removes a public key from the aptly GPG keyring. This is useful for removing
// @Description compromised keys or cleaning up obsolete keys.
// @Description
// @Tags Mirrors
// @Consume json
// @Param request body gpgDeleteKeyParams true "Parameters"
// @Produce json
// @Success 200 {object} string "OK"
// @Failure 400 {object} Error "Bad Request"
// @Router /api/gpg/key [delete]
func apiGPGDeleteKey(c *gin.Context) {
b := gpgDeleteKeyParams{}
if c.Bind(&b) != nil {
AbortWithJSONError(c, 400, fmt.Errorf("invalid request body"))
return
}
if len(strings.TrimSpace(b.GpgKeyID)) == 0 {
AbortWithJSONError(c, 400, fmt.Errorf("GpgKeyID is required"))
return
}
b.GpgKeyID = utils.SanitizePath(b.GpgKeyID)
// b.Keyring can be an absolute path
finder := pgp.GPGDefaultFinder()
gpg, _, err := finder.FindGPG()
if err != nil {
AbortWithJSONError(c, 400, err)
return
}
args := []string{
"--no-default-keyring",
"--allow-non-selfsigned-uid",
"--batch",
"--yes",
}
keyring := "trustedkeys.gpg"
if len(b.Keyring) > 0 {
keyring = b.Keyring
}
args = append(args, "--keyring", keyring)
args = append(args, "--delete-keys", b.GpgKeyID)
cmd := exec.Command(gpg, args...)
fmt.Printf("running %s %s\n", gpg, strings.Join(args, " "))
out, err := cmd.CombinedOutput()
if err != nil {
AbortWithJSONError(c, 400, fmt.Errorf("failed to delete key: %s", string(out)))
return
}
c.JSON(200, string(out))
}
// parseGPGOutput parses the output of `gpg --with-colons --list-keys`
// and returns a structured list of keys
func parseGPGOutput(output string) []gpgKeyInfo {
var keys []gpgKeyInfo
var currentKey *gpgKeyInfo
scanner := bufio.NewScanner(strings.NewReader(output))
for scanner.Scan() {
line := scanner.Text()
if line == "" {
continue
}
parts := strings.Split(line, ":")
if len(parts) < 10 {
continue
}
recordType := parts[0]
// pub: public key record
if recordType == "pub" {
// Save previous key if it exists
if currentKey != nil && currentKey.KeyID != "" {
keys = append(keys, *currentKey)
}
// Create new key entry
// Format: pub:trust:length:algo:keyid:created:expires:uidhash:...
keyID := parts[4]
if len(keyID) >= 16 {
keyID = keyID[len(keyID)-16:] // Last 16 chars = short key ID
}
validity := parts[1]
createdAt := parts[5]
currentKey = &gpgKeyInfo{
KeyID: keyID,
Validity: validity,
CreatedAt: createdAt,
UserIDs: []string{},
Fingerprint: "",
}
}
// uid: user ID record
if recordType == "uid" && currentKey != nil {
// Format: uid:trust:created:expires:keyid:uidhash:uidtype:validity:userID:...
if len(parts) >= 10 {
userID := parts[9]
if userID != "" {
currentKey.UserIDs = append(currentKey.UserIDs, userID)
}
}
}
// fpr: fingerprint record
if recordType == "fpr" && currentKey != nil {
// Format: fpr:::::::::fingerprint:
if len(parts) >= 10 {
fingerprint := parts[9]
currentKey.Fingerprint = fingerprint
}
}
}
// Don't forget the last key
if currentKey != nil && currentKey.KeyID != "" {
keys = append(keys, *currentKey)
}
return keys
}
+451
View File
@@ -0,0 +1,451 @@
package api
import (
"bytes"
"encoding/json"
"os"
"path/filepath"
"strings"
. "gopkg.in/check.v1"
)
type GPGSuite struct {
APISuite
}
var _ = Suite(&GPGSuite{})
func (s *GPGSuite) withFakeGPG(c *C, scriptBody string, test func(scriptPath string)) {
tempDir, err := os.MkdirTemp("", "aptly-fake-gpg")
c.Assert(err, IsNil)
defer func() { _ = os.RemoveAll(tempDir) }()
scriptPath := filepath.Join(tempDir, "gpg")
err = os.WriteFile(scriptPath, []byte(scriptBody), 0o755)
c.Assert(err, IsNil)
oldPath := os.Getenv("PATH")
err = os.Setenv("PATH", tempDir+string(os.PathListSeparator)+oldPath)
c.Assert(err, IsNil)
defer func() { _ = os.Setenv("PATH", oldPath) }()
test(scriptPath)
}
func (s *GPGSuite) fakeGPGScript(c *C, listOutput string, deleteOutput string, deleteError string) string {
return "#!/bin/sh\n" +
"if [ \"$1\" = \"--version\" ]; then\n" +
" echo 'gpg (GnuPG) 2.2.27'\n" +
" exit 0\n" +
"fi\n" +
"args=\"$*\"\n" +
"if printf '%s' \"$args\" | grep -q -- '--list-keys'; then\n" +
" cat <<'EOF'\n" + listOutput + "\nEOF\n" +
" exit 0\n" +
"fi\n" +
"if printf '%s' \"$args\" | grep -q -- '--delete-keys'; then\n" +
" if [ -n \"" + strings.ReplaceAll(deleteError, "\n", "") + "\" ]; then\n" +
" echo '" + strings.ReplaceAll(deleteError, "'", "'\\''") + "'\n" +
" exit 1\n" +
" fi\n" +
" cat <<'EOF'\n" + deleteOutput + "\nEOF\n" +
" exit 0\n" +
"fi\n" +
"echo 'unexpected invocation' >&2\n" +
"exit 1\n"
}
// TestParseGPGOutputEmpty tests parsing of empty GPG output
func (s *GPGSuite) TestParseGPGOutputEmpty(c *C) {
output := ""
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 0)
}
// TestParseGPGOutputSingleKeyMinimal tests parsing a single key with minimal fields
func (s *GPGSuite) TestParseGPGOutputSingleKeyMinimal(c *C) {
// Minimal valid GPG output with one key
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
key := keys[0]
c.Check(key.KeyID, Equals, "8B48AD6246925553")
c.Check(key.Validity, Equals, "u")
c.Check(key.CreatedAt, Equals, "1611864000")
c.Check(key.Fingerprint, Equals, "D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0")
c.Check(key.UserIDs, DeepEquals, []string{"John Doe <john@example.com>"})
}
// TestParseGPGOutputMultipleKeys tests parsing multiple keys
func (s *GPGSuite) TestParseGPGOutputMultipleKeys(c *C) {
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:
pub:f:2048:1:A1B2C3D4E5F67890:1580592000:1612128000:uidhash:::scESC:::::::23::0:
uid:f::::1580592000::0987654321::Jane Smith <jane@example.com>::::::::::0:
fpr:::::::::E9F0A1B2C3D4E5F6A7B8C9D0E1F2A3B4:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 2)
// First key
c.Check(keys[0].KeyID, Equals, "8B48AD6246925553")
c.Check(keys[0].Validity, Equals, "u")
c.Check(keys[0].UserIDs, DeepEquals, []string{"John Doe <john@example.com>"})
// Second key
c.Check(keys[1].KeyID, Equals, "A1B2C3D4E5F67890")
c.Check(keys[1].Validity, Equals, "f")
c.Check(keys[1].UserIDs, DeepEquals, []string{"Jane Smith <jane@example.com>"})
}
// TestParseGPGOutputMultipleUIDs tests a key with multiple user IDs
func (s *GPGSuite) TestParseGPGOutputMultipleUIDs(c *C) {
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
uid:u::::1611864000::1234567891::John Doe <john.doe@company.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
key := keys[0]
c.Check(key.UserIDs, HasLen, 2)
c.Check(key.UserIDs, DeepEquals, []string{
"John Doe <john@example.com>",
"John Doe <john.doe@company.com>",
})
}
// TestParseGPGOutputMalformedLines tests that malformed lines are skipped
func (s *GPGSuite) TestParseGPGOutputMalformedLines(c *C) {
// Mix of valid and invalid lines (too few fields)
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
invalid:line:with:only:three:fields
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
c.Check(keys[0].KeyID, Equals, "8B48AD6246925553")
}
// TestParseGPGOutputEmptyLines tests that empty lines are skipped
func (s *GPGSuite) TestParseGPGOutputEmptyLines(c *C) {
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
c.Check(keys[0].KeyID, Equals, "8B48AD6246925553")
}
// TestParseGPGOutputKeyWithoutUID tests a public key without user ID
func (s *GPGSuite) TestParseGPGOutputKeyWithoutUID(c *C) {
// Key without uid record (should still be included)
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
key := keys[0]
c.Check(key.KeyID, Equals, "8B48AD6246925553")
c.Check(key.UserIDs, HasLen, 0)
c.Check(key.Fingerprint, Equals, "D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0")
}
// TestParseGPGOutputVariousValidity tests different validity values
func (s *GPGSuite) TestParseGPGOutputVariousValidity(c *C) {
output := `pub:u:4096:1:KEY1111111111111:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890::Key1::::::::::0:
fpr:::::::::1111111111111111111111111111111111111111:
pub:f:4096:1:KEY2222222222222:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:f::::1611864000::1234567891::Key2::::::::::0:
fpr:::::::::2222222222222222222222222222222222222222:
pub:m:4096:1:KEY3333333333333:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:m::::1611864000::1234567892::Key3::::::::::0:
fpr:::::::::3333333333333333333333333333333333333333:
pub:n:4096:1:KEY4444444444444:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:n::::1611864000::1234567893::Key4::::::::::0:
fpr:::::::::4444444444444444444444444444444444444444:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 4)
validities := []string{"u", "f", "m", "n"}
for i, validity := range validities {
c.Check(keys[i].Validity, Equals, validity)
}
}
// TestParseGPGOutputShortKeyID tests that key IDs are shortened to 16 chars
func (s *GPGSuite) TestParseGPGOutputShortKeyID(c *C) {
// 40-character key ID that should be shortened to last 16 chars
longKeyID := "0123456789ABCDEF0123456789ABCDEF8B48AD62"
output := `pub:u:4096:1:` + longKeyID + `:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
// Should extract the last 16 characters: 89ABCDEF8B48AD62
c.Check(keys[0].KeyID, Equals, "89ABCDEF8B48AD62")
}
// TestParseGPGOutputSpecialCharactersInUID tests user IDs with special characters
func (s *GPGSuite) TestParseGPGOutputSpecialCharactersInUID(c *C) {
// UID with Unicode characters and special formatting
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890::J\xc3\xb6hn D\xc3\xb6\xc3\xa9 (D\xc3\xbcss) <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
// Should preserve the encoded special characters
c.Check(keys[0].UserIDs, HasLen, 1)
}
// TestAPIGPGListKeysDefaultKeyring tests the HTTP endpoint with default keyring
func (s *GPGSuite) TestAPIGPGListKeysDefaultKeyring(c *C) {
s.withFakeGPG(c, s.fakeGPGScript(c, `pub:u:4096:1:8B48AD6246925553:1611864000:::::
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`, "", ""), func(_ string) {
response, err := s.HTTPRequest("GET", "/api/gpg/keys", nil)
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 200)
var result gpgKeyListResponse
err = json.NewDecoder(response.Body).Decode(&result)
c.Assert(err, IsNil)
c.Check(result.Keys, HasLen, 1)
c.Check(result.Keys[0].KeyID, Equals, "8B48AD6246925553")
})
}
// TestAPIGPGListKeysWithKeyringParam tests the HTTP endpoint with custom keyring parameter
func (s *GPGSuite) TestAPIGPGListKeysWithKeyringParam(c *C) {
argFile, err := os.CreateTemp("", "aptly-gpg-args")
c.Assert(err, IsNil)
_ = argFile.Close()
defer func() { _ = os.Remove(argFile.Name()) }()
script := "#!/bin/sh\n" +
"if [ \"$1\" = \"--version\" ]; then echo 'gpg (GnuPG) 2.2.27'; exit 0; fi\n" +
"printf '%s\n' \"$@\" > '" + argFile.Name() + "'\n" +
"if printf '%s' \"$*\" | grep -q -- '--list-keys'; then\n" +
"cat <<'EOF'\n" +
"pub:u:4096:1:8B48AD6246925553:1611864000:::::\n" +
"fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:\n" +
"EOF\n" +
"exit 0\n" +
"fi\n" +
"exit 1\n"
s.withFakeGPG(c, script, func(_ string) {
response, reqErr := s.HTTPRequest("GET", "/api/gpg/keys?keyring=/custom.gpg", nil)
c.Assert(reqErr, IsNil)
c.Check(response.Code, Equals, 200)
argBytes, readErr := os.ReadFile(argFile.Name())
c.Assert(readErr, IsNil)
c.Check(string(argBytes), Matches, `(?s).*--keyring\ncustom\.gpg\n.*`)
})
}
// TestAPIGPGListKeysResponseFormat tests that the response has the correct structure
func (s *GPGSuite) TestAPIGPGListKeysResponseFormat(c *C) {
s.withFakeGPG(c, s.fakeGPGScript(c, `pub:f:4096:1:A1B2C3D4E5F67890:1611864000:::::
uid:f::::1611864000::1234567890::Jane Smith <jane@example.com>::::::::::0:
fpr:::::::::E9F0A1B2C3D4E5F6A7B8C9D0E1F2A3B4:`, "", ""), func(_ string) {
response, err := s.HTTPRequest("GET", "/api/gpg/keys", nil)
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 200)
var result gpgKeyListResponse
err = json.NewDecoder(response.Body).Decode(&result)
c.Assert(err, IsNil)
c.Assert(result.Keys, HasLen, 1)
c.Check(result.Keys[0].KeyID, Equals, "A1B2C3D4E5F67890")
c.Check(result.Keys[0].Validity, Equals, "f")
c.Check(result.Keys[0].CreatedAt, Equals, "1611864000")
})
}
// TestParseGPGOutputEdgeCaseUIDWithoutFields tests UID record with missing fields
func (s *GPGSuite) TestParseGPGOutputEdgeCaseUIDWithoutFields(c *C) {
// UID record with fewer than 10 fields
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
// Should not have user ID since it's in field 9 and this record is too short
c.Check(keys[0].UserIDs, HasLen, 0)
}
// TestParseGPGOutputFingerprintWithoutCurrentKey tests FPR record appearing before any PUB
func (s *GPGSuite) TestParseGPGOutputFingerprintWithoutCurrentKey(c *C) {
// FPR record without a preceding PUB (should be ignored)
output := `fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:
pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
// Should only have one key with the correct fingerprint
c.Check(keys[0].Fingerprint, Equals, "D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0")
}
// TestParseGPGOutputComplexRealWorldExample tests real-world-like GPG output
func (s *GPGSuite) TestParseGPGOutputComplexRealWorldExample(c *C) {
// Real-world GPG output with multiple keys, UIDs, and other record types (sig, sub)
// Note: sub and sig records are skipped as we only care about pub/uid/fpr
realWorldOutput := `tru::1:1611864000:0:3:1:5
pub:u:4096:1:8B48AD6246925553:1611864000:2023-01-15T00:00:00:::::scESC:::::::23::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:
uid:u::::1611864000::1234567890::John Doe <john@example.com>::::::::::0:
uid:u::::1611864100::1234567891::John Doe <john@work.com>::::::::::0:
pub:f:2048:1:1234567890123456:1580592000:2022-12-31T00:00:00::u:::scESC:::::::23::0:
fpr:::::::::F4E3D2C1B0A9F8E7D6C5B4A3F2E1D0C9:
uid:f::::1580592000::0987654321::Maintainer Key <maint@example.com>::::::::::0:`
keys := parseGPGOutput(realWorldOutput)
c.Check(keys, HasLen, 2)
// First key should have 2 UIDs
c.Check(keys[0].KeyID, Equals, "8B48AD6246925553")
c.Check(keys[0].UserIDs, HasLen, 2)
c.Check(keys[0].Fingerprint, Equals, "D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0")
// Second key should have 1 UID
c.Check(keys[1].KeyID, Equals, "1234567890123456")
c.Check(keys[1].UserIDs, HasLen, 1)
c.Check(keys[1].Fingerprint, Equals, "F4E3D2C1B0A9F8E7D6C5B4A3F2E1D0C9")
}
// TestParseGPGOutputConsecutiveEmptyUIDs tests handling of consecutive empty user ID fields
func (s *GPGSuite) TestParseGPGOutputConsecutiveEmptyUIDs(c *C) {
output := `pub:u:4096:1:8B48AD6246925553:1611864000:1643400000:uidhash:::scESC:::::::23::0:
uid:u::::1611864000::1234567890:::::::::::0:
uid:u::::1611864000::1234567891::John Doe <john@example.com>::::::::::0:
fpr:::::::::D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0:`
keys := parseGPGOutput(output)
c.Check(keys, HasLen, 1)
// Should skip empty UID but include the non-empty one
c.Check(keys[0].UserIDs, HasLen, 1)
c.Check(keys[0].UserIDs[0], Equals, "John Doe <john@example.com>")
}
// TestGPGDeleteKeyParamsValidation tests gpgDeleteKeyParams validation
func (s *GPGSuite) TestGPGDeleteKeyParamsValidation(c *C) {
// This is a unit test that validates parameter structure (no HTTP needed)
params := gpgDeleteKeyParams{
Keyring: "custom.gpg",
GpgKeyID: "8B48AD6246925553",
}
c.Check(params.Keyring, Equals, "custom.gpg")
c.Check(params.GpgKeyID, Equals, "8B48AD6246925553")
}
// TestAPIGPGDeleteKeyMissingKeyID tests delete with missing key ID parameter
func (s *GPGSuite) TestAPIGPGDeleteKeyMissingKeyID(c *C) {
body, err := json.Marshal(map[string]string{
"Keyring": "trustedkeys.gpg",
// GpgKeyID is missing
})
c.Assert(err, IsNil)
response, err := s.HTTPRequest("DELETE", "/api/gpg/key", bytes.NewReader(body))
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 400)
}
// TestAPIGPGDeleteKeyInvalidJSON tests delete with invalid JSON request
func (s *GPGSuite) TestAPIGPGDeleteKeyInvalidJSON(c *C) {
response, err := s.HTTPRequest("DELETE", "/api/gpg/key", bytes.NewReader([]byte("invalid json")))
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 400)
}
// TestAPIGPGDeleteKeySuccess tests successful key deletion
func (s *GPGSuite) TestAPIGPGDeleteKeySuccess(c *C) {
argFile, err := os.CreateTemp("", "aptly-gpg-delete-args")
c.Assert(err, IsNil)
_ = argFile.Close()
defer func() { _ = os.Remove(argFile.Name()) }()
script := "#!/bin/sh\n" +
"if [ \"$1\" = \"--version\" ]; then echo 'gpg (GnuPG) 2.2.27'; exit 0; fi\n" +
"printf '%s\n' \"$@\" > '" + argFile.Name() + "'\n" +
"if printf '%s' \"$*\" | grep -q -- '--delete-keys'; then\n" +
"echo 'deleted'\n" +
"exit 0\n" +
"fi\n" +
"exit 1\n"
s.withFakeGPG(c, script, func(_ string) {
body, marshalErr := json.Marshal(gpgDeleteKeyParams{
Keyring: "/trustedkeys.gpg",
GpgKeyID: "8B48AD6246925553",
})
c.Assert(marshalErr, IsNil)
response, reqErr := s.HTTPRequest("DELETE", "/api/gpg/key", bytes.NewReader(body))
c.Assert(reqErr, IsNil)
c.Check(response.Code, Equals, 200)
c.Check(response.Body.String(), Matches, `"deleted\\n"`)
argBytes, readErr := os.ReadFile(argFile.Name())
c.Assert(readErr, IsNil)
argText := string(argBytes)
c.Check(argText, Matches, `(?s).*--batch\n--yes\n.*`)
c.Check(argText, Matches, `(?s).*--keyring\n/trustedkeys\.gpg\n.*`)
c.Check(argText, Matches, `(?s).*--delete-keys\n8B48AD6246925553\n.*`)
})
}
// TestAPIGPGListKeysCommandFailure tests list error propagation from gpg
func (s *GPGSuite) TestAPIGPGListKeysCommandFailure(c *C) {
script := "#!/bin/sh\n" +
"if [ \"$1\" = \"--version\" ]; then echo 'gpg (GnuPG) 2.2.27'; exit 0; fi\n" +
"if printf '%s' \"$*\" | grep -q -- '--list-keys'; then\n" +
"echo 'keyring missing'\n" +
"exit 1\n" +
"fi\n" +
"exit 1\n"
s.withFakeGPG(c, script, func(_ string) {
response, err := s.HTTPRequest("GET", "/api/gpg/keys", nil)
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 400)
c.Check(response.Body.String(), Matches, `(?s).*failed to list keys: keyring missing.*`)
})
}
// TestAPIGPGDeleteKeyCommandFailure tests delete error propagation from gpg
func (s *GPGSuite) TestAPIGPGDeleteKeyCommandFailure(c *C) {
s.withFakeGPG(c, s.fakeGPGScript(c, "", "", "delete failed"), func(_ string) {
body, err := json.Marshal(gpgDeleteKeyParams{
Keyring: "trustedkeys.gpg",
GpgKeyID: "8B48AD6246925553",
})
c.Assert(err, IsNil)
response, reqErr := s.HTTPRequest("DELETE", "/api/gpg/key", bytes.NewReader(body))
c.Assert(reqErr, IsNil)
c.Check(response.Code, Equals, 400)
c.Check(response.Body.String(), Matches, `(?s).*failed to delete key: delete failed.*`)
})
}
+21
View File
@@ -0,0 +1,21 @@
package api
import (
"github.com/gin-gonic/gin"
)
// @Summary JFrog repositories
// @Description **Get list of JFrog repositories**
// @Description
// @Description List configured JFrog publish endpoints.
// @Tags Status
// @Produce json
// @Success 200 {array} string "List of JFrog publish endpoints"
// @Router /api/jfrog [get]
func apiJFrogList(c *gin.Context) {
keys := []string{}
for k := range context.Config().JFrogPublishRoots {
keys = append(keys, k)
}
c.JSON(200, keys)
}
+179 -6
View File
@@ -31,22 +31,61 @@ func getVerifier(keyRings []string) (pgp.Verifier, error) {
return verifier, nil
}
// stringSlicesEqual compares two string slices for equality (order matters)
func stringSlicesEqual(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
// uniqueStrings returns a new slice with only unique strings from the input, sorted
func uniqueStrings(input []string) []string {
if len(input) == 0 {
return input
}
seen := make(map[string]struct{}, len(input))
result := make([]string, 0, len(input))
for _, s := range input {
if _, ok := seen[s]; !ok {
seen[s] = struct{}{}
result = append(result, s)
}
}
sort.Strings(result)
return result
}
// @Summary List Mirrors
// @Description **Show list of currently available mirrors**
// @Description Each mirror is returned as in “show” API.
// @Tags Mirrors
// @Produce json
// @Success 200 {array} deb.RemoteRepo
// @Success 200 {array} remoteRepoResponse
// @Router /api/mirrors [get]
func apiMirrorsList(c *gin.Context) {
collectionFactory := context.NewCollectionFactory()
collection := collectionFactory.RemoteRepoCollection()
result := []*deb.RemoteRepo{}
_ = collection.ForEach(func(repo *deb.RemoteRepo) error {
result = append(result, repo)
result := []remoteRepoResponse{}
err := collection.ForEach(func(repo *deb.RemoteRepo) error {
err := collection.LoadComplete(repo)
if err != nil {
return err
}
result = append(result, newRemoteRepoResponse(repo))
return nil
})
if err != nil {
AbortWithJSONError(c, 500, fmt.Errorf("unable to show: %s", err))
return
}
c.JSON(200, result)
}
@@ -72,6 +111,8 @@ type mirrorCreateParams struct {
DownloadUdebs bool ` json:"DownloadUdebs"`
// Set "true" to mirror installer files
DownloadInstaller bool ` json:"DownloadInstaller"`
// Set "true" to mirror AppStream (DEP-11) metadata
DownloadAppStream bool ` json:"DownloadAppStream"`
// Set "true" to include dependencies of matching packages when filtering
FilterWithDeps bool ` json:"FilterWithDeps"`
// Set "true" to skip if the given components are in the Release file
@@ -123,7 +164,7 @@ func apiMirrorsCreate(c *gin.Context) {
}
repo, err := deb.NewRemoteRepo(b.Name, b.ArchiveURL, b.Distribution, b.Components, b.Architectures,
b.DownloadSources, b.DownloadUdebs, b.DownloadInstaller)
b.DownloadSources, b.DownloadUdebs, b.DownloadInstaller, b.DownloadAppStream)
if err != nil {
AbortWithJSONError(c, 400, fmt.Errorf("unable to create mirror: %s", err))
@@ -344,6 +385,128 @@ func apiMirrorsPackages(c *gin.Context) {
}
}
type mirrorEditParams struct {
// Package query that is applied to mirror packages
Filter *string ` json:"Filter" example:"xserver-xorg"`
// Set "true" to include dependencies of matching packages when filtering
FilterWithDeps *bool ` json:"FilterWithDeps"`
// Set "true" to mirror installer files
DownloadInstaller *bool `json:"DownloadInstaller"`
// Set "true" to mirror source packages
DownloadSources *bool ` json:"DownloadSources"`
// Set "true" to mirror udeb files
DownloadUdebs *bool ` json:"DownloadUdebs"`
// URL of the archive to mirror
ArchiveURL *string ` json:"ArchiveURL" example:"http://deb.debian.org/debian"`
// Comma separated list of architectures
Architectures *[]string `json:"Architectures" example:"amd64"`
// Gpg keyring(s) for verifying Release file if a mirror update is required.
Keyrings []string ` json:"Keyrings" example:"trustedkeys.gpg"`
// Set "true" to skip the verification of Release file signatures
IgnoreSignatures *bool ` json:"IgnoreSignatures"`
}
// @Summary Edit Mirror
// @Description **Edit mirror config**
// @Tags Mirrors
// @Param name path string true "mirror name to edit"
// @Consume json
// @Param request body mirrorEditParams true "Parameters"
// @Produce json
// @Success 200 {object} deb.RemoteRepo "Mirror was edited successfully"
// @Failure 400 {object} Error "Bad Request"
// @Failure 404 {object} Error "Mirror not found"
// @Failure 409 {object} Error "Aptly db locked"
// @Failure 500 {object} Error "Internal Error"
// @Router /api/mirrors/{name} [post]
func apiMirrorsEdit(c *gin.Context) {
var (
err error
b mirrorEditParams
repo *deb.RemoteRepo
)
collectionFactory := context.NewCollectionFactory()
collection := collectionFactory.RemoteRepoCollection()
name := c.Params.ByName("name")
repo, err = collection.ByName(name)
if err != nil {
AbortWithJSONError(c, 404, fmt.Errorf("unable to edit: %s", err))
return
}
err = repo.CheckLock()
if err != nil {
AbortWithJSONError(c, 409, fmt.Errorf("unable to edit: %s", err))
return
}
if c.Bind(&b) != nil {
return
}
fetchMirror := false
ignoreSignatures := context.Config().GpgDisableVerify
if b.Filter != nil {
repo.Filter = *b.Filter
}
if b.FilterWithDeps != nil {
repo.FilterWithDeps = *b.FilterWithDeps
}
if b.DownloadInstaller != nil {
repo.DownloadInstaller = *b.DownloadInstaller
}
if b.DownloadSources != nil {
repo.DownloadSources = *b.DownloadSources
}
if b.DownloadUdebs != nil {
repo.DownloadUdebs = *b.DownloadUdebs
}
if b.ArchiveURL != nil && *b.ArchiveURL != repo.ArchiveRoot {
repo.SetArchiveRoot(*b.ArchiveURL)
fetchMirror = true
}
if b.Architectures != nil {
uniqueArchitectures := uniqueStrings(*b.Architectures)
if !stringSlicesEqual(uniqueArchitectures, uniqueStrings(repo.Architectures)) {
repo.Architectures = uniqueArchitectures
fetchMirror = true
}
}
if b.IgnoreSignatures != nil {
ignoreSignatures = *b.IgnoreSignatures
}
if repo.IsFlat() && repo.DownloadUdebs {
AbortWithJSONError(c, 400, fmt.Errorf("unable to edit: flat mirrors don't support udebs"))
return
}
if fetchMirror {
verifier, err := getVerifier(b.Keyrings)
if err != nil {
AbortWithJSONError(c, 500, fmt.Errorf("unable to initialize GPG verifier: %s", err))
return
}
err = repo.Fetch(context.Downloader(), verifier, ignoreSignatures)
if err != nil {
AbortWithJSONError(c, 500, fmt.Errorf("unable to edit: %s", err))
return
}
}
err = collection.Update(repo)
if err != nil {
AbortWithJSONError(c, 500, fmt.Errorf("unable to edit: %s", err))
return
}
c.JSON(200, repo)
}
type mirrorUpdateParams struct {
// Change mirror name to `Name`
Name string ` json:"Name" example:"mirror1"`
@@ -357,6 +520,8 @@ type mirrorUpdateParams struct {
ForceUpdate bool ` json:"ForceUpdate"`
// Set "true" to skip downloading already downloaded packages
SkipExistingPackages bool ` json:"SkipExistingPackages"`
// Set "true" to download only the latest version per package/architecture
LatestOnly bool ` json:"LatestOnly"`
}
// @Summary Update Mirror
@@ -452,6 +617,14 @@ func apiMirrorsUpdate(c *gin.Context) {
return &task.ProcessReturnValue{Code: http.StatusInternalServerError, Value: nil}, fmt.Errorf("unable to update: %s", err)
}
if remote.DownloadAppStream && !remote.IsFlat() {
err = remote.DownloadAppStreamFiles(out, downloader,
context.PackagePool(), taskCollectionFactory.ChecksumCollection(nil), b.IgnoreChecksums)
if err != nil {
return &task.ProcessReturnValue{Code: http.StatusInternalServerError, Value: nil}, fmt.Errorf("unable to update: %s", err)
}
}
if remote.Filter != "" {
var filterQuery deb.PackageQuery
@@ -467,7 +640,7 @@ func apiMirrorsUpdate(c *gin.Context) {
}
queue, downloadSize, err := remote.BuildDownloadQueue(context.PackagePool(), taskCollectionFactory.PackageCollection(),
taskCollectionFactory.ChecksumCollection(nil), b.SkipExistingPackages)
taskCollectionFactory.ChecksumCollection(nil), b.SkipExistingPackages, b.LatestOnly)
if err != nil {
return &task.ProcessReturnValue{Code: http.StatusInternalServerError, Value: nil}, fmt.Errorf("unable to update: %s", err)
}
+66 -1
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"encoding/json"
"github.com/aptly-dev/aptly/deb"
"github.com/gin-gonic/gin"
. "gopkg.in/check.v1"
)
@@ -17,7 +18,10 @@ var _ = Suite(&MirrorSuite{})
func (s *MirrorSuite) TestGetMirrors(c *C) {
response, _ := s.HTTPRequest("GET", "/api/mirrors", nil)
c.Check(response.Code, Equals, 200)
c.Check(response.Body.String(), Equals, "[]")
var mirrors []map[string]interface{}
err := json.Unmarshal(response.Body.Bytes(), &mirrors)
c.Assert(err, IsNil)
}
func (s *MirrorSuite) TestDeleteMirrorNonExisting(c *C) {
@@ -26,6 +30,21 @@ func (s *MirrorSuite) TestDeleteMirrorNonExisting(c *C) {
c.Check(response.Body.String(), Equals, "{\"error\":\"unable to drop: mirror with name does-not-exist not found\"}")
}
func (s *MirrorSuite) TestCreateMirrorFlatWithAppStream(c *C) {
body, err := json.Marshal(gin.H{
"Name": "test-flat-appstream",
"ArchiveURL": "http://example.com/repo/",
"Distribution": "./",
"DownloadAppStream": true,
})
c.Assert(err, IsNil)
response, err := s.HTTPRequest("POST", "/api/mirrors", bytes.NewReader(body))
c.Assert(err, IsNil)
c.Check(response.Code, Equals, 400)
c.Check(response.Body.String(), Matches, ".*AppStream.*flat.*")
}
func (s *MirrorSuite) TestCreateMirror(c *C) {
c.ExpectFailure("Need to mock downloads")
body, err := json.Marshal(gin.H{
@@ -38,3 +57,49 @@ func (s *MirrorSuite) TestCreateMirror(c *C) {
c.Check(response.Code, Equals, 400)
c.Check(response.Body.String(), Equals, "")
}
func (s *MirrorSuite) TestGetMirrorsIncludesNumPackages(c *C) {
collection := s.context.NewCollectionFactory().RemoteRepoCollection()
repo, err := deb.NewRemoteRepo("count-mirror", "http://example.com/debian", "stable", []string{"main"}, []string{}, false, false, false, false)
c.Assert(err, IsNil)
err = collection.Add(repo)
c.Assert(err, IsNil)
putRawDBValue(c, &s.APISuite, repo.RefKey(), makePackageRefList(c).Encode())
response, err := s.HTTPRequest("GET", "/api/mirrors", nil)
c.Assert(err, IsNil)
c.Assert(response.Code, Equals, 200)
var mirrors []map[string]interface{}
err = json.Unmarshal(response.Body.Bytes(), &mirrors)
c.Assert(err, IsNil)
found := false
for _, mirror := range mirrors {
if mirror["Name"] == "count-mirror" {
found = true
value, ok := mirror["NumPackages"]
c.Assert(ok, Equals, true)
c.Assert(value, Equals, float64(2))
break
}
}
c.Assert(found, Equals, true)
}
func (s *MirrorSuite) TestGetMirrorsReturns500OnCorruptRefList(c *C) {
collection := s.context.NewCollectionFactory().RemoteRepoCollection()
repo, err := deb.NewRemoteRepo("broken-mirror", "http://example.com/debian", "stable", []string{"main"}, []string{}, false, false, false, false)
c.Assert(err, IsNil)
c.Assert(collection.Add(repo), IsNil)
putRawDBValue(c, &s.APISuite, repo.RefKey(), []byte("not-msgpack"))
response, err := s.HTTPRequest("GET", "/api/mirrors", nil)
c.Assert(err, IsNil)
c.Assert(response.Code, Equals, 500)
c.Assert(response.Body.String(), Matches, ".*unable to show:.*")
}
+30
View File
@@ -0,0 +1,30 @@
package api
import "github.com/aptly-dev/aptly/deb"
type remoteRepoResponse struct {
*deb.RemoteRepo
NumPackages int `json:"NumPackages"`
}
type localRepoResponse struct {
*deb.LocalRepo
NumPackages int `json:"NumPackages"`
}
type snapshotResponse struct {
*deb.Snapshot
NumPackages int `json:"NumPackages"`
}
func newRemoteRepoResponse(repo *deb.RemoteRepo) remoteRepoResponse {
return remoteRepoResponse{RemoteRepo: repo, NumPackages: repo.NumPackages()}
}
func newLocalRepoResponse(repo *deb.LocalRepo) localRepoResponse {
return localRepoResponse{LocalRepo: repo, NumPackages: repo.NumPackages()}
}
func newSnapshotResponse(snapshot *deb.Snapshot) snapshotResponse {
return snapshotResponse{Snapshot: snapshot, NumPackages: snapshot.NumPackages()}
}
+19
View File
@@ -0,0 +1,19 @@
package api
import (
"github.com/aptly-dev/aptly/deb"
. "gopkg.in/check.v1"
)
func makePackageRefList(c *C) *deb.PackageRefList {
list := deb.NewPackageList()
c.Assert(list.Add(&deb.Package{Name: "libcount", Version: "1.0", Architecture: "amd64"}), IsNil)
c.Assert(list.Add(&deb.Package{Name: "appcount", Version: "2.0", Architecture: "all"}), IsNil)
return deb.NewPackageRefListFromPackageList(list)
}
func putRawDBValue(c *C, s *APISuite, key []byte, value []byte) {
db, err := s.context.Database()
c.Assert(err, IsNil)
c.Assert(db.Put(key, value), IsNil)
}
+61 -1
View File
@@ -160,6 +160,10 @@ type publishedRepoCreateParams struct {
Sources []sourceParams `binding:"required" json:"Sources"`
// Distribution name, if missing Aptly would try to guess from sources
Distribution string ` json:"Distribution" example:"bookworm"`
// Value of Label: field in published repository stanza
Label string ` json:"Label" example:""`
// Value of Origin: field in published repository stanza
Origin string ` json:"Origin" example:""`
// when publishing, overwrite files in pool/ directory without notice
ForceOverwrite bool ` json:"ForceOverwrite" example:"false"`
// Override list of published architectures
@@ -178,8 +182,12 @@ type publishedRepoCreateParams struct {
SkipBz2 *bool ` json:"SkipBz2" example:"false"`
// Provide index files by hash
AcquireByHash *bool ` json:"AcquireByHash" example:"false"`
// An optional field containing a comma separated list of OpenPGP key fingerprints to be used for validating the next Release file.
SignedBy *string ` json:"SignedBy" example:""`
// Enable multiple packages with the same filename in different distributions
MultiDist *bool ` json:"MultiDist" example:"false"`
// Version of the release
Version string ` json:"Version" example:""`
}
// @Summary Create Published Repository
@@ -192,7 +200,7 @@ type publishedRepoCreateParams struct {
// @Description **Example:**
// @Description ```
// @Description $ curl -X POST -H 'Content-Type: application/json' --data '{"Distribution": "wheezy", "Sources": [{"Name": "aptly-repo"}]}' http://localhost:8080/api/publish//repos
// @Description {"Architectures":["i386"],"Distribution":"wheezy","Prefix":".","SourceKind":"local","Sources":[{"Component":"main","Name":"aptly-repo"}],"Storage":""}
// @Description {"Architectures":["i386"],"Distribution":"wheezy","Label":"","Origin":"","Prefix":".","SourceKind":"local","Sources":[{"Component":"main","Name":"aptly-repo"}],"Storage":""}
// @Description ```
// @Description
// @Description See also: `aptly publish create`
@@ -337,12 +345,16 @@ func apiPublishRepoOrSnapshot(c *gin.Context) {
return &task.ProcessReturnValue{Code: http.StatusInternalServerError, Value: nil}, fmt.Errorf("unable to publish: %s", err)
}
if b.Origin != "" {
published.Origin = b.Origin
}
if b.NotAutomatic != "" {
published.NotAutomatic = b.NotAutomatic
}
if b.ButAutomaticUpgrades != "" {
published.ButAutomaticUpgrades = b.ButAutomaticUpgrades
}
published.Label = b.Label
published.SkipContents = context.Config().SkipContentsPublishing
if b.SkipContents != nil {
@@ -358,6 +370,14 @@ func apiPublishRepoOrSnapshot(c *gin.Context) {
published.AcquireByHash = *b.AcquireByHash
}
if b.SignedBy != nil {
published.SignedBy = *b.SignedBy
}
if b.Version != "" {
published.Version = b.Version
}
duplicate := taskCollection.CheckDuplicate(published)
if duplicate != nil {
_ = taskCollectionFactory.PublishedRepoCollection().LoadComplete(duplicate, taskCollectionFactory)
@@ -393,8 +413,16 @@ type publishedRepoUpdateSwitchParams struct {
Snapshots []sourceParams ` json:"Snapshots"`
// Provide index files by hash
AcquireByHash *bool ` json:"AcquireByHash" example:"false"`
// An optional field containing a comma separated list of OpenPGP key fingerprints to be used for validating the next Release file
SignedBy *string ` json:"SignedBy" example:""`
// Enable multiple packages with the same filename in different distributions
MultiDist *bool ` json:"MultiDist" example:"false"`
// Value of Label: field in published repository stanza
Label *string ` json:"Label" example:"Debian"`
// Value of Origin: field in published repository stanza
Origin *string ` json:"Origin" example:"Debian"`
// Version of the release: Optional
Version *string ` json:"Version" example:"13.3"`
}
// @Summary Update Published Repository
@@ -515,9 +543,21 @@ func apiPublishUpdateSwitch(c *gin.Context) {
if b.AcquireByHash != nil {
published.AcquireByHash = *b.AcquireByHash
}
if b.SignedBy != nil {
published.SignedBy = *b.SignedBy
}
if b.MultiDist != nil {
published.MultiDist = *b.MultiDist
}
if b.Label != nil {
published.Label = *b.Label
}
if b.Origin != nil {
published.Origin = *b.Origin
}
if b.Version != nil {
published.Version = *b.Version
}
revision := published.ObtainRevision()
sources := revision.Sources
@@ -1060,8 +1100,16 @@ type publishedRepoUpdateParams struct {
SkipCleanup *bool ` json:"SkipCleanup" example:"false"`
// Provide index files by hash
AcquireByHash *bool ` json:"AcquireByHash" example:"false"`
// An optional field containing a comma separated list of OpenPGP key fingerprints to be used for validating the next Release file
SignedBy *string ` json:"SignedBy" example:""`
// Enable multiple packages with the same filename in different distributions
MultiDist *bool ` json:"MultiDist" example:"false"`
// Value of Label: field in published repository stanza
Label *string ` json:"Label" example:"Debian"`
// Value of Origin: field in published repository stanza
Origin *string ` json:"Origin" example:"Debian"`
// Version of the release: Optional
Version *string ` json:"Version" example:"13.3"`
}
// @Summary Update Published Repository
@@ -1173,9 +1221,21 @@ func apiPublishUpdate(c *gin.Context) {
if b.AcquireByHash != nil {
published.AcquireByHash = *b.AcquireByHash
}
if b.SignedBy != nil {
published.SignedBy = *b.SignedBy
}
if b.MultiDist != nil {
published.MultiDist = *b.MultiDist
}
if b.Label != nil {
published.Label = *b.Label
}
if b.Origin != nil {
published.Origin = *b.Origin
}
if b.Version != nil {
published.Version = *b.Version
}
result, err := published.Update(taskCollectionFactory, out)
if err != nil {
+13 -4
View File
@@ -74,17 +74,26 @@ func reposServeInAPIMode(c *gin.Context) {
// @Description Each repo is returned as in “show” API.
// @Tags Repos
// @Produce json
// @Success 200 {array} deb.LocalRepo
// @Success 200 {array} localRepoResponse
// @Router /api/repos [get]
func apiReposList(c *gin.Context) {
result := []*deb.LocalRepo{}
result := []localRepoResponse{}
collectionFactory := context.NewCollectionFactory()
collection := collectionFactory.LocalRepoCollection()
_ = collection.ForEach(func(r *deb.LocalRepo) error {
result = append(result, r)
err := collection.ForEach(func(r *deb.LocalRepo) error {
err := collection.LoadComplete(r)
if err != nil {
return err
}
result = append(result, newLocalRepoResponse(r))
return nil
})
if err != nil {
AbortWithJSONError(c, 500, err)
return
}
c.JSON(200, result)
}
+63
View File
@@ -0,0 +1,63 @@
package api
import (
"bytes"
"encoding/json"
"github.com/aptly-dev/aptly/deb"
"github.com/gin-gonic/gin"
. "gopkg.in/check.v1"
)
type ReposSuite struct {
APISuite
}
var _ = Suite(&ReposSuite{})
func (s *ReposSuite) TestGetReposIncludesNumPackages(c *C) {
collection := s.context.NewCollectionFactory().LocalRepoCollection()
repo := deb.NewLocalRepo("count-repo-list", "")
repo.UpdateRefList(makePackageRefList(c))
c.Assert(collection.Add(repo), IsNil)
response, err := s.HTTPRequest("GET", "/api/repos", nil)
c.Assert(err, IsNil)
c.Assert(response.Code, Equals, 200)
var repos []map[string]interface{}
err = json.Unmarshal(response.Body.Bytes(), &repos)
c.Assert(err, IsNil)
found := false
for _, repo := range repos {
if repo["Name"] == "count-repo-list" {
found = true
value, ok := repo["NumPackages"]
c.Assert(ok, Equals, true)
c.Assert(value, Equals, float64(2))
break
}
}
c.Assert(found, Equals, true)
}
func (s *ReposSuite) TestGetReposReturns500OnCorruptRefList(c *C) {
body, err := json.Marshal(gin.H{"Name": "broken-repo-list"})
c.Assert(err, IsNil)
response, err := s.HTTPRequest("POST", "/api/repos", bytes.NewReader(body))
c.Assert(err, IsNil)
c.Assert(response.Code, Equals, 201)
collection := s.context.NewCollectionFactory().LocalRepoCollection()
repo, err := collection.ByName("broken-repo-list")
c.Assert(err, IsNil)
putRawDBValue(c, &s.APISuite, repo.RefKey(), []byte("not-msgpack"))
response, err = s.HTTPRequest("GET", "/api/repos", nil)
c.Assert(err, IsNil)
c.Assert(response.Code, Equals, 500)
c.Assert(response.Body.String(), Matches, ".*msgpack.*|.*decode.*")
}
+32 -26
View File
@@ -11,9 +11,9 @@ import (
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/rs/zerolog/log"
// _ "github.com/aptly-dev/aptly/docs" // import docs
// swaggerFiles "github.com/swaggo/files"
// ginSwagger "github.com/swaggo/gin-swagger"
"github.com/aptly-dev/aptly/docs"
swaggerFiles "github.com/swaggo/files"
ginSwagger "github.com/swaggo/gin-swagger"
)
var context *ctx.AptlyContext
@@ -31,21 +31,21 @@ func apiMetricsGet() gin.HandlerFunc {
}
}
// func redirectSwagger(c *gin.Context) {
// if c.Request.URL.Path == "/docs/index.html" {
// c.Redirect(http.StatusMovedPermanently, "/docs.html")
// return
// }
// if c.Request.URL.Path == "/docs/" {
// c.Redirect(http.StatusMovedPermanently, "/docs.html")
// return
// }
// if c.Request.URL.Path == "/docs" {
// c.Redirect(http.StatusMovedPermanently, "/docs.html")
// return
// }
// c.Next()
// }
func redirectSwagger(c *gin.Context) {
if c.Request.URL.Path == "/docs/index.html" {
c.Redirect(http.StatusMovedPermanently, "/docs.html")
return
}
if c.Request.URL.Path == "/docs/" {
c.Redirect(http.StatusMovedPermanently, "/docs.html")
return
}
if c.Request.URL.Path == "/docs" {
c.Redirect(http.StatusMovedPermanently, "/docs.html")
return
}
c.Next()
}
// Router returns prebuilt with routes http.Handler
func Router(c *ctx.AptlyContext) http.Handler {
@@ -69,14 +69,14 @@ func Router(c *ctx.AptlyContext) http.Handler {
router.Use(gin.Recovery(), gin.ErrorLogger())
// if c.Config().EnableSwaggerEndpoint {
// router.GET("docs.html", func(c *gin.Context) {
// c.Data(http.StatusOK, "text/html; charset=utf-8", docs.DocsHTML)
// })
// router.Use(redirectSwagger)
// url := ginSwagger.URL("/docs/doc.json")
// router.GET("/docs/*any", ginSwagger.WrapHandler(swaggerFiles.Handler, url))
// }
if c.Config().EnableSwaggerEndpoint {
router.GET("docs.html", func(c *gin.Context) {
c.Data(http.StatusOK, "text/html; charset=utf-8", docs.DocsHTML)
})
router.Use(redirectSwagger)
url := ginSwagger.URL("/docs/doc.json")
router.GET("/docs/*any", ginSwagger.WrapHandler(swaggerFiles.Handler, url))
}
if c.Config().EnableMetricsEndpoint {
MetricsCollectorRegistrar.Register(router)
@@ -164,21 +164,27 @@ func Router(c *ctx.AptlyContext) http.Handler {
api.GET("/mirrors/:name", apiMirrorsShow)
api.GET("/mirrors/:name/packages", apiMirrorsPackages)
api.POST("/mirrors", apiMirrorsCreate)
api.POST("/mirrors/:name", apiMirrorsEdit)
api.PUT("/mirrors/:name", apiMirrorsUpdate)
api.DELETE("/mirrors/:name", apiMirrorsDrop)
}
{
api.GET("/gpg/keys", apiGPGListKeys)
api.POST("/gpg/key", apiGPGAddKey)
api.DELETE("/gpg/key", apiGPGDeleteKey)
}
{
api.GET("/s3", apiS3List)
api.GET("/gcs", apiGCSList)
api.GET("/jfrog", apiJFrogList)
}
{
api.GET("/files", apiFilesListDirs)
api.POST("/files/:dir", apiFilesUpload)
api.PUT("/files/:dir/:file", apiFilesUploadOne)
api.GET("/files/:dir", apiFilesListFiles)
api.DELETE("/files/:dir", apiFilesDeleteDir)
api.DELETE("/files/:dir/:name", apiFilesDeleteFile)
+13 -4
View File
@@ -20,7 +20,7 @@ import (
// @Description Each snapshot is returned as in “show” API.
// @Tags Snapshots
// @Produce json
// @Success 200 {array} deb.Snapshot
// @Success 200 {array} snapshotResponse
// @Router /api/snapshots [get]
func apiSnapshotsList(c *gin.Context) {
SortMethodString := c.Request.URL.Query().Get("sort")
@@ -32,11 +32,20 @@ func apiSnapshotsList(c *gin.Context) {
SortMethodString = "name"
}
result := []*deb.Snapshot{}
_ = collection.ForEachSorted(SortMethodString, func(snapshot *deb.Snapshot) error {
result = append(result, snapshot)
result := []snapshotResponse{}
err := collection.ForEachSorted(SortMethodString, func(snapshot *deb.Snapshot) error {
err := collection.LoadComplete(snapshot)
if err != nil {
return err
}
result = append(result, newSnapshotResponse(snapshot))
return nil
})
if err != nil {
AbortWithJSONError(c, 500, err)
return
}
c.JSON(200, result)
}
+53
View File
@@ -0,0 +1,53 @@
package api
import (
"encoding/json"
"github.com/aptly-dev/aptly/deb"
. "gopkg.in/check.v1"
)
type SnapshotsSuite struct {
APISuite
}
var _ = Suite(&SnapshotsSuite{})
func (s *SnapshotsSuite) TestGetSnapshotsIncludesNumPackages(c *C) {
collection := s.context.NewCollectionFactory().SnapshotCollection()
snapshot := deb.NewSnapshotFromRefList("count-snapshot-list", nil, makePackageRefList(c), "")
c.Assert(collection.Add(snapshot), IsNil)
response, err := s.HTTPRequest("GET", "/api/snapshots", nil)
c.Assert(err, IsNil)
c.Assert(response.Code, Equals, 200)
var snapshots []map[string]interface{}
err = json.Unmarshal(response.Body.Bytes(), &snapshots)
c.Assert(err, IsNil)
found := false
for _, snapshot := range snapshots {
if snapshot["Name"] == "count-snapshot-list" {
found = true
value, ok := snapshot["NumPackages"]
c.Assert(ok, Equals, true)
c.Assert(value, Equals, float64(2))
break
}
}
c.Assert(found, Equals, true)
}
func (s *SnapshotsSuite) TestGetSnapshotsReturns500OnCorruptRefList(c *C) {
collection := s.context.NewCollectionFactory().SnapshotCollection()
snapshot := deb.NewSnapshotFromRefList("broken-snapshot-list", nil, makePackageRefList(c), "")
c.Assert(collection.Add(snapshot), IsNil)
putRawDBValue(c, &s.APISuite, snapshot.RefKey(), []byte("not-msgpack"))
response, err := s.HTTPRequest("GET", "/api/snapshots", nil)
c.Assert(err, IsNil)
c.Assert(response.Code, Equals, 500)
c.Assert(response.Body.String(), Matches, ".*msgpack.*|.*decode.*")
}
+46 -39
View File
@@ -5,28 +5,35 @@ package azure
import (
"context"
"encoding/hex"
"errors"
"fmt"
"io"
"net/url"
"os"
"path/filepath"
"time"
"github.com/Azure/azure-storage-blob-go/azblob"
"github.com/Azure/azure-sdk-for-go/sdk/azcore"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob"
"github.com/aptly-dev/aptly/aptly"
)
func isBlobNotFound(err error) bool {
storageError, ok := err.(azblob.StorageError)
return ok && storageError.ServiceCode() == azblob.ServiceCodeBlobNotFound
var respErr *azcore.ResponseError
if errors.As(err, &respErr) {
return respErr.StatusCode == 404 // BlobNotFound
}
return false
}
type azContext struct {
container azblob.ContainerURL
client *azblob.Client
container string
prefix string
}
func newAzContext(accountName, accountKey, container, prefix, endpoint string) (*azContext, error) {
credential, err := azblob.NewSharedKeyCredential(accountName, accountKey)
cred, err := azblob.NewSharedKeyCredential(accountName, accountKey)
if err != nil {
return nil, err
}
@@ -35,15 +42,14 @@ func newAzContext(accountName, accountKey, container, prefix, endpoint string) (
endpoint = fmt.Sprintf("https://%s.blob.core.windows.net", accountName)
}
url, err := url.Parse(fmt.Sprintf("%s/%s", endpoint, container))
serviceClient, err := azblob.NewClientWithSharedKeyCredential(endpoint, cred, nil)
if err != nil {
return nil, err
}
containerURL := azblob.NewContainerURL(*url, azblob.NewPipeline(credential, azblob.PipelineOptions{}))
result := &azContext{
container: containerURL,
client: serviceClient,
container: container,
prefix: prefix,
}
@@ -54,10 +60,6 @@ func (az *azContext) blobPath(path string) string {
return filepath.Join(az.prefix, path)
}
func (az *azContext) blobURL(path string) azblob.BlobURL {
return az.container.NewBlobURL(az.blobPath(path))
}
func (az *azContext) internalFilelist(prefix string, progress aptly.Progress) (paths []string, md5s []string, err error) {
const delimiter = "/"
paths = make([]string, 0, 1024)
@@ -67,27 +69,33 @@ func (az *azContext) internalFilelist(prefix string, progress aptly.Progress) (p
prefix += delimiter
}
for marker := (azblob.Marker{}); marker.NotDone(); {
listBlob, err := az.container.ListBlobsFlatSegment(
context.Background(), marker, azblob.ListBlobsSegmentOptions{
Prefix: prefix,
MaxResults: 1,
Details: azblob.BlobListingDetails{Metadata: true}})
ctx := context.Background()
maxResults := int32(1)
pager := az.client.NewListBlobsFlatPager(az.container, &azblob.ListBlobsFlatOptions{
Prefix: &prefix,
MaxResults: &maxResults,
Include: azblob.ListBlobsInclude{Metadata: true},
})
// Iterate over each page
for pager.More() {
page, err := pager.NextPage(ctx)
if err != nil {
return nil, nil, fmt.Errorf("error listing under prefix %s in %s: %s", prefix, az, err)
}
marker = listBlob.NextMarker
for _, blob := range listBlob.Segment.BlobItems {
for _, blob := range page.Segment.BlobItems {
if prefix == "" {
paths = append(paths, blob.Name)
paths = append(paths, *blob.Name)
} else {
paths = append(paths, blob.Name[len(prefix):])
name := *blob.Name
paths = append(paths, name[len(prefix):])
}
md5s = append(md5s, fmt.Sprintf("%x", blob.Properties.ContentMD5))
}
b := *blob
md5 := b.Properties.ContentMD5
md5s = append(md5s, fmt.Sprintf("%x", md5))
}
if progress != nil {
time.Sleep(time.Duration(500) * time.Millisecond)
progress.AddBar(1)
@@ -97,28 +105,27 @@ func (az *azContext) internalFilelist(prefix string, progress aptly.Progress) (p
return paths, md5s, nil
}
func (az *azContext) putFile(blob azblob.BlobURL, source io.Reader, sourceMD5 string) error {
uploadOptions := azblob.UploadStreamToBlockBlobOptions{
BufferSize: 4 * 1024 * 1024,
MaxBuffers: 8,
func (az *azContext) putFile(blobName string, source io.Reader, sourceMD5 string) error {
uploadOptions := &azblob.UploadFileOptions{
BlockSize: 4 * 1024 * 1024,
Concurrency: 8,
}
path := az.blobPath(blobName)
if len(sourceMD5) > 0 {
decodedMD5, err := hex.DecodeString(sourceMD5)
if err != nil {
return err
}
uploadOptions.BlobHTTPHeaders = azblob.BlobHTTPHeaders{
ContentMD5: decodedMD5,
uploadOptions.HTTPHeaders = &blob.HTTPHeaders{
BlobContentMD5: decodedMD5,
}
}
_, err := azblob.UploadStreamToBlockBlob(
context.Background(),
source,
blob.ToBlockBlobURL(),
uploadOptions,
)
var err error
if file, ok := source.(*os.File); ok {
_, err = az.client.UploadFile(context.TODO(), az.container, path, file, uploadOptions)
}
return err
}
+21 -23
View File
@@ -5,7 +5,6 @@ import (
"os"
"path/filepath"
"github.com/Azure/azure-storage-blob-go/azblob"
"github.com/aptly-dev/aptly/aptly"
"github.com/aptly-dev/aptly/utils"
"github.com/pkg/errors"
@@ -41,10 +40,7 @@ func (pool *PackagePool) buildPoolPath(filename string, checksums *utils.Checksu
return filepath.Join(hash[0:2], hash[2:4], hash[4:32]+"_"+filename)
}
func (pool *PackagePool) ensureChecksums(
poolPath string,
checksumStorage aptly.ChecksumStorage,
) (*utils.ChecksumInfo, error) {
func (pool *PackagePool) ensureChecksums(poolPath string, checksumStorage aptly.ChecksumStorage) (*utils.ChecksumInfo, error) {
targetChecksums, err := checksumStorage.Get(poolPath)
if err != nil {
return nil, err
@@ -52,8 +48,7 @@ func (pool *PackagePool) ensureChecksums(
if targetChecksums == nil {
// we don't have checksums stored yet for this file
blob := pool.az.blobURL(poolPath)
download, err := blob.Download(context.Background(), 0, 0, azblob.BlobAccessConditions{}, false, azblob.ClientProvidedKeyOptions{})
download, err := pool.az.client.DownloadStream(context.Background(), pool.az.container, poolPath, nil)
if err != nil {
if isBlobNotFound(err) {
return nil, nil
@@ -63,7 +58,7 @@ func (pool *PackagePool) ensureChecksums(
}
targetChecksums = &utils.ChecksumInfo{}
*targetChecksums, err = utils.ChecksumsForReader(download.Body(azblob.RetryReaderOptions{}))
*targetChecksums, err = utils.ChecksumsForReader(download.Body)
if err != nil {
return nil, errors.Wrapf(err, "error checksumming blob at %s", poolPath)
}
@@ -92,45 +87,49 @@ func (pool *PackagePool) LegacyPath(_ string, _ *utils.ChecksumInfo) (string, er
}
func (pool *PackagePool) Size(path string) (int64, error) {
blob := pool.az.blobURL(path)
props, err := blob.GetProperties(context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
serviceClient := pool.az.client.ServiceClient()
containerClient := serviceClient.NewContainerClient(pool.az.container)
blobClient := containerClient.NewBlobClient(path)
props, err := blobClient.GetProperties(context.TODO(), nil)
if err != nil {
return 0, errors.Wrapf(err, "error examining %s from %s", path, pool)
}
return props.ContentLength(), nil
return *props.ContentLength, nil
}
func (pool *PackagePool) Open(path string) (aptly.ReadSeekerCloser, error) {
blob := pool.az.blobURL(path)
temp, err := os.CreateTemp("", "blob-download")
if err != nil {
return nil, errors.Wrap(err, "error creating temporary file for blob download")
return nil, errors.Wrapf(err, "error creating tempfile for %s", path)
}
defer func() { _ = os.Remove(temp.Name()) }()
err = azblob.DownloadBlobToFile(context.Background(), blob, 0, 0, temp, azblob.DownloadFromBlobOptions{})
_, err = pool.az.client.DownloadFile(context.TODO(), pool.az.container, path, temp, nil)
if err != nil {
return nil, errors.Wrapf(err, "error downloading blob at %s", path)
return nil, errors.Wrapf(err, "error downloading blob %s", path)
}
return temp, nil
}
func (pool *PackagePool) Remove(path string) (int64, error) {
blob := pool.az.blobURL(path)
props, err := blob.GetProperties(context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
serviceClient := pool.az.client.ServiceClient()
containerClient := serviceClient.NewContainerClient(pool.az.container)
blobClient := containerClient.NewBlobClient(path)
props, err := blobClient.GetProperties(context.TODO(), nil)
if err != nil {
return 0, errors.Wrapf(err, "error getting props of %s from %s", path, pool)
return 0, errors.Wrapf(err, "error examining %s from %s", path, pool)
}
_, err = blob.Delete(context.Background(), azblob.DeleteSnapshotsOptionNone, azblob.BlobAccessConditions{})
_, err = pool.az.client.DeleteBlob(context.Background(), pool.az.container, path, nil)
if err != nil {
return 0, errors.Wrapf(err, "error deleting %s from %s", path, pool)
}
return props.ContentLength(), nil
return *props.ContentLength, nil
}
func (pool *PackagePool) Import(srcPath, basename string, checksums *utils.ChecksumInfo, _ bool, checksumStorage aptly.ChecksumStorage) (string, error) {
@@ -144,7 +143,6 @@ func (pool *PackagePool) Import(srcPath, basename string, checksums *utils.Check
}
path := pool.buildPoolPath(basename, checksums)
blob := pool.az.blobURL(path)
targetChecksums, err := pool.ensureChecksums(path, checksumStorage)
if err != nil {
return "", err
@@ -160,7 +158,7 @@ func (pool *PackagePool) Import(srcPath, basename string, checksums *utils.Check
}
defer func() { _ = source.Close() }()
err = pool.az.putFile(blob, source, checksums.MD5)
err = pool.az.putFile(path, source, checksums.MD5)
if err != nil {
return "", err
}
+5 -3
View File
@@ -7,7 +7,7 @@ import (
"path/filepath"
"runtime"
"github.com/Azure/azure-storage-blob-go/azblob"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
"github.com/aptly-dev/aptly/aptly"
"github.com/aptly-dev/aptly/files"
"github.com/aptly-dev/aptly/utils"
@@ -50,8 +50,10 @@ func (s *PackagePoolSuite) SetUpTest(c *C) {
s.pool, err = NewPackagePool(s.accountName, s.accountKey, container, "", s.endpoint)
c.Assert(err, IsNil)
cnt := s.pool.az.container
_, err = cnt.Create(context.Background(), azblob.Metadata{}, azblob.PublicAccessContainer)
publicAccessType := azblob.PublicAccessTypeContainer
_, err = s.pool.az.client.CreateContainer(context.TODO(), s.pool.az.container, &azblob.CreateContainerOptions{
Access: &publicAccessType,
})
c.Assert(err, IsNil)
s.prefixedPool, err = NewPackagePool(s.accountName, s.accountKey, container, prefix, s.endpoint)
+70 -57
View File
@@ -3,19 +3,22 @@ package azure
import (
"context"
"fmt"
"net/http"
"os"
"path/filepath"
"time"
"github.com/Azure/azure-storage-blob-go/azblob"
"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/lease"
"github.com/aptly-dev/aptly/aptly"
"github.com/aptly-dev/aptly/utils"
"github.com/google/uuid"
"github.com/pkg/errors"
)
// PublishedStorage abstract file system with published files (actually hosted on Azure)
type PublishedStorage struct {
// FIXME: unused ???? prefix string
az *azContext
pathCache map[string]map[string]string
}
@@ -64,7 +67,7 @@ func (storage *PublishedStorage) PutFile(path string, sourceFilename string) err
}
defer func() { _ = source.Close() }()
err = storage.az.putFile(storage.az.blobURL(path), source, sourceMD5)
err = storage.az.putFile(path, source, sourceMD5)
if err != nil {
err = errors.Wrap(err, fmt.Sprintf("error uploading %s to %s", sourceFilename, storage))
}
@@ -74,14 +77,15 @@ func (storage *PublishedStorage) PutFile(path string, sourceFilename string) err
// RemoveDirs removes directory structure under public path
func (storage *PublishedStorage) RemoveDirs(path string, _ aptly.Progress) error {
path = storage.az.blobPath(path)
filelist, err := storage.Filelist(path)
if err != nil {
return err
}
for _, filename := range filelist {
blob := storage.az.blobURL(filepath.Join(path, filename))
_, err := blob.Delete(context.Background(), azblob.DeleteSnapshotsOptionNone, azblob.BlobAccessConditions{})
blob := filepath.Join(path, filename)
_, err := storage.az.client.DeleteBlob(context.Background(), storage.az.container, blob, nil)
if err != nil {
return fmt.Errorf("error deleting path %s from %s: %s", filename, storage, err)
}
@@ -92,8 +96,8 @@ func (storage *PublishedStorage) RemoveDirs(path string, _ aptly.Progress) error
// Remove removes single file under public path
func (storage *PublishedStorage) Remove(path string) error {
blob := storage.az.blobURL(path)
_, err := blob.Delete(context.Background(), azblob.DeleteSnapshotsOptionNone, azblob.BlobAccessConditions{})
path = storage.az.blobPath(path)
_, err := storage.az.client.DeleteBlob(context.Background(), storage.az.container, path, nil)
if err != nil {
err = errors.Wrap(err, fmt.Sprintf("error deleting %s from %s: %s", path, storage, err))
}
@@ -112,9 +116,8 @@ func (storage *PublishedStorage) LinkFromPool(publishedPrefix, publishedRelPath,
sourcePath string, sourceChecksums utils.ChecksumInfo, force bool) error {
relFilePath := filepath.Join(publishedRelPath, fileName)
// prefixRelFilePath := filepath.Join(publishedPrefix, relFilePath)
// FIXME: check how to integrate publishedPrefix:
poolPath := storage.az.blobPath(fileName)
prefixRelFilePath := filepath.Join(publishedPrefix, relFilePath)
poolPath := storage.az.blobPath(prefixRelFilePath)
if storage.pathCache == nil {
storage.pathCache = make(map[string]map[string]string)
@@ -157,7 +160,7 @@ func (storage *PublishedStorage) LinkFromPool(publishedPrefix, publishedRelPath,
}
defer func() { _ = source.Close() }()
err = storage.az.putFile(storage.az.blobURL(relFilePath), source, sourceMD5)
err = storage.az.putFile(relFilePath, source, sourceMD5)
if err == nil {
pathCache[relFilePath] = sourceMD5
} else {
@@ -174,57 +177,60 @@ func (storage *PublishedStorage) Filelist(prefix string) ([]string, error) {
}
// Internal copy or move implementation
func (storage *PublishedStorage) internalCopyOrMoveBlob(src, dst string, metadata azblob.Metadata, move bool) error {
func (storage *PublishedStorage) internalCopyOrMoveBlob(src, dst string, metadata map[string]*string, move bool) error {
const leaseDuration = 30
leaseID := uuid.NewString()
dstBlobURL := storage.az.blobURL(dst)
srcBlobURL := storage.az.blobURL(src)
leaseResp, err := srcBlobURL.AcquireLease(context.Background(), "", leaseDuration, azblob.ModifiedAccessConditions{})
if err != nil || leaseResp.StatusCode() != http.StatusCreated {
return fmt.Errorf("error acquiring lease on source blob %s", srcBlobURL)
serviceClient := storage.az.client.ServiceClient()
containerClient := serviceClient.NewContainerClient(storage.az.container)
srcBlobClient := containerClient.NewBlobClient(src)
blobLeaseClient, err := lease.NewBlobClient(srcBlobClient, &lease.BlobClientOptions{LeaseID: to.Ptr(leaseID)})
if err != nil {
return fmt.Errorf("error acquiring lease on source blob %s", src)
}
defer func() { _, _ = srcBlobURL.BreakLease(context.Background(), azblob.LeaseBreakNaturally, azblob.ModifiedAccessConditions{}) }()
srcBlobLeaseID := leaseResp.LeaseID()
copyResp, err := dstBlobURL.StartCopyFromURL(
context.Background(),
srcBlobURL.URL(),
metadata,
azblob.ModifiedAccessConditions{},
azblob.BlobAccessConditions{},
azblob.DefaultAccessTier,
nil)
_, err = blobLeaseClient.AcquireLease(context.Background(), leaseDuration, nil)
if err != nil {
return fmt.Errorf("error acquiring lease on source blob %s", src)
}
defer func() {
_, _ = blobLeaseClient.BreakLease(context.Background(), &lease.BlobBreakOptions{BreakPeriod: to.Ptr(int32(60))})
}()
dstBlobClient := containerClient.NewBlobClient(dst)
copyResp, err := dstBlobClient.StartCopyFromURL(context.Background(), srcBlobClient.URL(), &blob.StartCopyFromURLOptions{
Metadata: metadata,
})
if err != nil {
return fmt.Errorf("error copying %s -> %s in %s: %s", src, dst, storage, err)
}
copyStatus := copyResp.CopyStatus()
copyStatus := *copyResp.CopyStatus
for {
if copyStatus == azblob.CopyStatusSuccess {
if copyStatus == blob.CopyStatusTypeSuccess {
if move {
_, err = srcBlobURL.Delete(
context.Background(),
azblob.DeleteSnapshotsOptionNone,
azblob.BlobAccessConditions{
LeaseAccessConditions: azblob.LeaseAccessConditions{LeaseID: srcBlobLeaseID},
})
_, err := storage.az.client.DeleteBlob(context.Background(), storage.az.container, src, &blob.DeleteOptions{
AccessConditions: &blob.AccessConditions{
LeaseAccessConditions: &blob.LeaseAccessConditions{
LeaseID: &leaseID,
},
},
})
return err
}
return nil
} else if copyStatus == azblob.CopyStatusPending {
} else if copyStatus == blob.CopyStatusTypePending {
time.Sleep(1 * time.Second)
blobPropsResp, err := dstBlobURL.GetProperties(
context.Background(),
azblob.BlobAccessConditions{LeaseAccessConditions: azblob.LeaseAccessConditions{LeaseID: srcBlobLeaseID}},
azblob.ClientProvidedKeyOptions{})
getMetadata, err := dstBlobClient.GetProperties(context.TODO(), nil)
if err != nil {
return fmt.Errorf("error getting destination blob properties %s", dstBlobURL)
return fmt.Errorf("error getting copy progress %s", dst)
}
copyStatus = blobPropsResp.CopyStatus()
copyStatus = *getMetadata.CopyStatus
_, err = srcBlobURL.RenewLease(context.Background(), srcBlobLeaseID, azblob.ModifiedAccessConditions{})
_, err = blobLeaseClient.RenewLease(context.Background(), nil)
if err != nil {
return fmt.Errorf("error renewing source blob lease %s", srcBlobURL)
return fmt.Errorf("error renewing source blob lease %s", src)
}
} else {
return fmt.Errorf("error copying %s -> %s in %s: %s", dst, src, storage, copyStatus)
@@ -239,7 +245,9 @@ func (storage *PublishedStorage) RenameFile(oldName, newName string) error {
// SymLink creates a copy of src file and adds link information as meta data
func (storage *PublishedStorage) SymLink(src string, dst string) error {
return storage.internalCopyOrMoveBlob(src, dst, azblob.Metadata{"SymLink": src}, false /* move */)
metadata := make(map[string]*string)
metadata["SymLink"] = &src
return storage.internalCopyOrMoveBlob(src, dst, metadata, false /* do not remove src */)
}
// HardLink using symlink functionality as hard links do not exist
@@ -249,28 +257,33 @@ func (storage *PublishedStorage) HardLink(src string, dst string) error {
// FileExists returns true if path exists
func (storage *PublishedStorage) FileExists(path string) (bool, error) {
blob := storage.az.blobURL(path)
resp, err := blob.GetProperties(context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
serviceClient := storage.az.client.ServiceClient()
containerClient := serviceClient.NewContainerClient(storage.az.container)
blobClient := containerClient.NewBlobClient(path)
_, err := blobClient.GetProperties(context.Background(), nil)
if err != nil {
if isBlobNotFound(err) {
return false, nil
}
return false, err
} else if resp.StatusCode() == http.StatusOK {
return true, nil
return false, fmt.Errorf("error checking if blob %s exists: %v", path, err)
}
return false, fmt.Errorf("error checking if blob %s exists %d", blob, resp.StatusCode())
return true, nil
}
// ReadLink returns the symbolic link pointed to by path.
// This simply reads text file created with SymLink
func (storage *PublishedStorage) ReadLink(path string) (string, error) {
blob := storage.az.blobURL(path)
resp, err := blob.GetProperties(context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
serviceClient := storage.az.client.ServiceClient()
containerClient := serviceClient.NewContainerClient(storage.az.container)
blobClient := containerClient.NewBlobClient(path)
props, err := blobClient.GetProperties(context.Background(), nil)
if err != nil {
return "", err
} else if resp.StatusCode() != http.StatusOK {
return "", fmt.Errorf("error checking if blob %s exists %d", blob, resp.StatusCode())
return "", fmt.Errorf("failed to get blob properties: %v", err)
}
return resp.NewMetadata()["SymLink"], nil
metadata := props.Metadata
if originalBlob, exists := metadata["original_blob"]; exists {
return *originalBlob, nil
}
return "", fmt.Errorf("error reading link %s: %v", path, err)
}
+26 -23
View File
@@ -1,6 +1,7 @@
package azure
import (
"bytes"
"context"
"crypto/md5"
"crypto/rand"
@@ -8,7 +9,9 @@ import (
"os"
"path/filepath"
"github.com/Azure/azure-storage-blob-go/azblob"
"github.com/Azure/azure-sdk-for-go/sdk/azcore"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob"
"github.com/aptly-dev/aptly/files"
"github.com/aptly-dev/aptly/utils"
. "gopkg.in/check.v1"
@@ -66,8 +69,10 @@ func (s *PublishedStorageSuite) SetUpTest(c *C) {
s.storage, err = NewPublishedStorage(s.accountName, s.accountKey, container, "", s.endpoint)
c.Assert(err, IsNil)
cnt := s.storage.az.container
_, err = cnt.Create(context.Background(), azblob.Metadata{}, azblob.PublicAccessContainer)
publicAccessType := azblob.PublicAccessTypeContainer
_, err = s.storage.az.client.CreateContainer(context.Background(), s.storage.az.container, &azblob.CreateContainerOptions{
Access: &publicAccessType,
})
c.Assert(err, IsNil)
s.prefixedStorage, err = NewPublishedStorage(s.accountName, s.accountKey, container, prefix, s.endpoint)
@@ -75,41 +80,39 @@ func (s *PublishedStorageSuite) SetUpTest(c *C) {
}
func (s *PublishedStorageSuite) TearDownTest(c *C) {
cnt := s.storage.az.container
_, err := cnt.Delete(context.Background(), azblob.ContainerAccessConditions{})
_, err := s.storage.az.client.DeleteContainer(context.Background(), s.storage.az.container, nil)
c.Assert(err, IsNil)
}
func (s *PublishedStorageSuite) GetFile(c *C, path string) []byte {
blob := s.storage.az.container.NewBlobURL(path)
resp, err := blob.Download(context.Background(), 0, azblob.CountToEnd, azblob.BlobAccessConditions{}, false, azblob.ClientProvidedKeyOptions{})
resp, err := s.storage.az.client.DownloadStream(context.Background(), s.storage.az.container, path, nil)
c.Assert(err, IsNil)
body := resp.Body(azblob.RetryReaderOptions{MaxRetryRequests: 3})
data, err := io.ReadAll(body)
data, err := io.ReadAll(resp.Body)
c.Assert(err, IsNil)
return data
}
func (s *PublishedStorageSuite) AssertNoFile(c *C, path string) {
_, err := s.storage.az.container.NewBlobURL(path).GetProperties(
context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
serviceClient := s.storage.az.client.ServiceClient()
containerClient := serviceClient.NewContainerClient(s.storage.az.container)
blobClient := containerClient.NewBlobClient(path)
_, err := blobClient.GetProperties(context.Background(), nil)
c.Assert(err, NotNil)
storageError, ok := err.(azblob.StorageError)
storageError, ok := err.(*azcore.ResponseError)
c.Assert(ok, Equals, true)
c.Assert(string(storageError.ServiceCode()), Equals, string(string(azblob.StorageErrorCodeBlobNotFound)))
c.Assert(storageError.StatusCode, Equals, 404)
}
func (s *PublishedStorageSuite) PutFile(c *C, path string, data []byte) {
hash := md5.Sum(data)
_, err := azblob.UploadBufferToBlockBlob(
context.Background(),
data,
s.storage.az.container.NewBlockBlobURL(path),
azblob.UploadToBlockBlobOptions{
BlobHTTPHeaders: azblob.BlobHTTPHeaders{
ContentMD5: hash[:],
},
})
uploadOptions := &azblob.UploadStreamOptions{
HTTPHeaders: &blob.HTTPHeaders{
BlobContentMD5: hash[:],
},
}
reader := bytes.NewReader(data)
_, err := s.storage.az.client.UploadStream(context.Background(), s.storage.az.container, path, reader, uploadOptions)
c.Assert(err, IsNil)
}
@@ -330,7 +333,7 @@ func (s *PublishedStorageSuite) TestLinkFromPool(c *C) {
// 2nd link from pool, providing wrong path for source file
//
// this test should check that file already exists in S3 and skip upload (which would fail if not skipped)
// this test should check that file already exists in Azure and skip upload (which would fail if not skipped)
s.prefixedStorage.pathCache = nil
err = s.prefixedStorage.LinkFromPool("", filepath.Join("pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, "wrong-looks-like-pathcache-doesnt-work", cksum1, false)
c.Check(err, IsNil)
+11
View File
@@ -26,6 +26,7 @@ func aptlyDBCleanup(cmd *commander.Command, args []string) error {
// collect information about references packages...
existingPackageRefs := deb.NewPackageRefList()
referencedAppStreamFiles := []string{}
// used only in verbose mode to report package use source
packageRefSources := map[string][]string{}
@@ -55,6 +56,10 @@ func aptlyDBCleanup(cmd *commander.Command, args []string) error {
}
}
for _, poolPath := range repo.AppStreamFiles {
referencedAppStreamFiles = append(referencedAppStreamFiles, poolPath)
}
return nil
})
if err != nil {
@@ -118,6 +123,11 @@ func aptlyDBCleanup(cmd *commander.Command, args []string) error {
return nil
})
}
for _, poolPath := range snapshot.AppStreamFiles {
referencedAppStreamFiles = append(referencedAppStreamFiles, poolPath)
}
return nil
})
if err != nil {
@@ -236,6 +246,7 @@ func aptlyDBCleanup(cmd *commander.Command, args []string) error {
return err
}
referencedFiles = append(referencedFiles, referencedAppStreamFiles...)
sort.Strings(referencedFiles)
context.Progress().ShutdownBar()
+3 -1
View File
@@ -20,6 +20,7 @@ func aptlyMirrorCreate(cmd *commander.Command, args []string) error {
downloadSources := LookupOption(context.Config().DownloadSourcePackages, context.Flags(), "with-sources")
downloadUdebs := context.Flags().Lookup("with-udebs").Value.Get().(bool)
downloadInstaller := context.Flags().Lookup("with-installer").Value.Get().(bool)
downloadAppStream := context.Flags().Lookup("with-appstream").Value.Get().(bool)
ignoreSignatures := context.Config().GpgDisableVerify
if context.Flags().IsSet("ignore-signatures") {
ignoreSignatures = context.Flags().Lookup("ignore-signatures").Value.Get().(bool)
@@ -41,7 +42,7 @@ func aptlyMirrorCreate(cmd *commander.Command, args []string) error {
}
repo, err := deb.NewRemoteRepo(mirrorName, archiveURL, distribution, components, context.ArchitecturesList(),
downloadSources, downloadUdebs, downloadInstaller)
downloadSources, downloadUdebs, downloadInstaller, downloadAppStream)
if err != nil {
return fmt.Errorf("unable to create mirror: %s", err)
}
@@ -100,6 +101,7 @@ Example:
}
cmd.Flag.Bool("ignore-signatures", false, "disable verification of Release file signatures")
cmd.Flag.Bool("with-appstream", false, "download AppStream (DEP-11) metadata")
cmd.Flag.Bool("with-installer", false, "download additional not packaged installer files")
cmd.Flag.Bool("with-sources", false, "download source packages in addition to binary packages")
cmd.Flag.Bool("with-udebs", false, "download .udeb packages (Debian installer support)")
+7
View File
@@ -35,6 +35,8 @@ func aptlyMirrorEdit(cmd *commander.Command, args []string) error {
repo.Filter = flag.Value.String() // allows file/stdin with @
case "filter-with-deps":
repo.FilterWithDeps = flag.Value.Get().(bool)
case "with-appstream":
repo.DownloadAppStream = flag.Value.Get().(bool)
case "with-installer":
repo.DownloadInstaller = flag.Value.Get().(bool)
case "with-sources":
@@ -53,6 +55,10 @@ func aptlyMirrorEdit(cmd *commander.Command, args []string) error {
return fmt.Errorf("unable to edit: flat mirrors don't support udebs")
}
if repo.IsFlat() && repo.DownloadAppStream {
return fmt.Errorf("unable to edit: flat mirrors don't support AppStream (DEP-11) metadata")
}
if repo.Filter != "" {
_, err = query.Parse(repo.Filter)
if err != nil {
@@ -107,6 +113,7 @@ Example:
AddStringOrFileFlag(&cmd.Flag, "filter", "", "filter packages in mirror, use '@file' to read filter from file or '@-' for stdin")
cmd.Flag.Bool("filter-with-deps", false, "when filtering, include dependencies of matching packages as well")
cmd.Flag.Bool("ignore-signatures", false, "disable verification of Release file signatures")
cmd.Flag.Bool("with-appstream", false, "download AppStream (DEP-11) metadata")
cmd.Flag.Bool("with-installer", false, "download additional not packaged installer files")
cmd.Flag.Bool("with-sources", false, "download source packages in addition to binary packages")
cmd.Flag.Bool("with-udebs", false, "download .udeb packages (Debian installer support)")
+5
View File
@@ -61,6 +61,11 @@ func aptlyMirrorShowTxt(_ *commander.Command, args []string) error {
downloadUdebs = Yes
}
fmt.Printf("Download .udebs: %s\n", downloadUdebs)
downloadAppStream := No
if repo.DownloadAppStream {
downloadAppStream = Yes
}
fmt.Printf("Download AppStream: %s\n", downloadAppStream)
if repo.Filter != "" {
fmt.Printf("Filter: %s\n", repo.Filter)
filterWithDeps := No
+12 -1
View File
@@ -64,6 +64,15 @@ func aptlyMirrorUpdate(cmd *commander.Command, args []string) error {
return fmt.Errorf("unable to update: %s", err)
}
if repo.DownloadAppStream && !repo.IsFlat() {
context.Progress().Printf("Downloading AppStream metadata...\n")
err = repo.DownloadAppStreamFiles(context.Progress(), context.Downloader(),
context.PackagePool(), collectionFactory.ChecksumCollection(nil), ignoreChecksums)
if err != nil {
return fmt.Errorf("unable to update: %s", err)
}
}
if repo.Filter != "" {
context.Progress().Printf("Applying filter...\n")
var filterQuery deb.PackageQuery
@@ -87,10 +96,11 @@ func aptlyMirrorUpdate(cmd *commander.Command, args []string) error {
)
skipExistingPackages := context.Flags().Lookup("skip-existing-packages").Value.Get().(bool)
latestOnly := context.Flags().Lookup("latest").Value.Get().(bool)
context.Progress().Printf("Building download queue...\n")
queue, downloadSize, err = repo.BuildDownloadQueue(context.PackagePool(), collectionFactory.PackageCollection(),
collectionFactory.ChecksumCollection(nil), skipExistingPackages)
collectionFactory.ChecksumCollection(nil), skipExistingPackages, latestOnly)
if err != nil {
return fmt.Errorf("unable to update: %s", err)
@@ -292,6 +302,7 @@ Example:
cmd.Flag.Bool("ignore-checksums", false, "ignore checksum mismatches while downloading package files and metadata")
cmd.Flag.Bool("ignore-signatures", false, "disable verification of Release file signatures")
cmd.Flag.Bool("skip-existing-packages", false, "do not check file existence for packages listed in the internal database of the mirror")
cmd.Flag.Bool("latest", false, "download only latest version of each package (per architecture)")
cmd.Flag.Int64("download-limit", 0, "limit download speed (kbytes/sec)")
cmd.Flag.String("downloader", "default", "downloader to use (e.g. grab)")
cmd.Flag.Int("max-tries", 1, "max download tries till process fails with download error")
+2
View File
@@ -51,7 +51,9 @@ Example:
cmd.Flag.String("codename", "", "codename to publish (defaults to distribution)")
cmd.Flag.Bool("force-overwrite", false, "overwrite files in package pool in case of mismatch")
cmd.Flag.Bool("acquire-by-hash", false, "provide index files by hash")
cmd.Flag.String("signed-by", "", "an optional field containing a comma separated list of OpenPGP key fingerprints to be used for validating the next Release file")
cmd.Flag.Bool("multi-dist", false, "enable multiple packages with the same filename in different distributions")
cmd.Flag.String("version", "", "version of the release")
return cmd
}
+10
View File
@@ -150,10 +150,18 @@ func aptlyPublishSnapshotOrRepo(cmd *commander.Command, args []string) error {
published.AcquireByHash = context.Flags().Lookup("acquire-by-hash").Value.Get().(bool)
}
if context.Flags().IsSet("signed-by") {
published.SignedBy = context.Flags().Lookup("signed-by").Value.String()
}
if context.Flags().IsSet("multi-dist") {
published.MultiDist = context.Flags().Lookup("multi-dist").Value.Get().(bool)
}
if context.Flags().IsSet("version") {
published.Version = context.Flags().Lookup("version").Value.String()
}
duplicate := collectionFactory.PublishedRepoCollection().CheckDuplicate(published)
if duplicate != nil {
_ = collectionFactory.PublishedRepoCollection().LoadComplete(duplicate, collectionFactory)
@@ -249,6 +257,8 @@ Example:
cmd.Flag.String("codename", "", "codename to publish (defaults to distribution)")
cmd.Flag.Bool("force-overwrite", false, "overwrite files in package pool in case of mismatch")
cmd.Flag.Bool("acquire-by-hash", false, "provide index files by hash")
cmd.Flag.String("signed-by", "", "an optional field containing a comma separated list of OpenPGP key fingerprints to be used for validating the next Release file")
cmd.Flag.String("version", "", "version of the release")
cmd.Flag.Bool("multi-dist", false, "enable multiple packages with the same filename in different distributions")
return cmd
+10
View File
@@ -99,6 +99,14 @@ func aptlyPublishSwitch(cmd *commander.Command, args []string) error {
published.SkipBz2 = context.Flags().Lookup("skip-bz2").Value.Get().(bool)
}
if context.Flags().IsSet("signed-by") {
published.SignedBy = context.Flags().Lookup("signed-by").Value.String()
}
if context.Flags().IsSet("version") {
published.Version = context.Flags().Lookup("version").Value.String()
}
if context.Flags().IsSet("multi-dist") {
published.MultiDist = context.Flags().Lookup("multi-dist").Value.Get().(bool)
}
@@ -162,6 +170,8 @@ This command would switch published repository (with one component) named ppa/wh
cmd.Flag.Bool("skip-bz2", false, "don't generate bzipped indexes")
cmd.Flag.String("component", "", "component names to update (for multi-component publishing, separate components with commas)")
cmd.Flag.Bool("force-overwrite", false, "overwrite files in package pool in case of mismatch")
cmd.Flag.String("signed-by", "", "an optional field containing a comma separated list of OpenPGP key fingerprints to be used for validating the next Release file")
cmd.Flag.String("version", "", "version of the release")
cmd.Flag.Bool("skip-cleanup", false, "don't remove unreferenced files in prefix/component")
cmd.Flag.Bool("multi-dist", false, "enable multiple packages with the same filename in different distributions")
+20
View File
@@ -60,10 +60,26 @@ func aptlyPublishUpdate(cmd *commander.Command, args []string) error {
published.SkipBz2 = context.Flags().Lookup("skip-bz2").Value.Get().(bool)
}
if context.Flags().IsSet("signed-by") {
published.SignedBy = context.Flags().Lookup("signed-by").Value.String()
}
if context.Flags().IsSet("origin") {
published.Origin = context.Flags().Lookup("origin").Value.String()
}
if context.Flags().IsSet("label") {
published.Label = context.Flags().Lookup("label").Value.String()
}
if context.Flags().IsSet("multi-dist") {
published.MultiDist = context.Flags().Lookup("multi-dist").Value.Get().(bool)
}
if context.Flags().IsSet("version") {
published.Version = context.Flags().Lookup("version").Value.String()
}
err = published.Publish(context.PackagePool(), context, collectionFactory, signer, context.Progress(), forceOverwrite, context.SkelPath())
if err != nil {
return fmt.Errorf("unable to publish: %s", err)
@@ -125,8 +141,12 @@ Example:
cmd.Flag.Bool("skip-contents", false, "don't generate Contents indexes")
cmd.Flag.Bool("skip-bz2", false, "don't generate bzipped indexes")
cmd.Flag.Bool("force-overwrite", false, "overwrite files in package pool in case of mismatch")
cmd.Flag.String("signed-by", "", "an optional field containing a comma separated list of OpenPGP key fingerprints to be used for validating the next Release file")
cmd.Flag.Bool("skip-cleanup", false, "don't remove unreferenced files in prefix/component")
cmd.Flag.Bool("multi-dist", false, "enable multiple packages with the same filename in different distributions")
cmd.Flag.String("origin", "", "overwrite origin name to publish")
cmd.Flag.String("label", "", "overwrite label to publish")
cmd.Flag.String("version", "", "version of the release")
return cmd
}
+3
View File
@@ -185,6 +185,7 @@ local keyring="*-keyring=[gpg keyring to use when verifying Release file (could
$keyring \
"-with-sources=[download source packages in addition to binary packages]:$bool" \
"-with-udebs=[download .udeb packages (Debian installer support)]:$bool" \
"-with-appstream=[download AppStream (DEP-11) metadata]:$bool" \
"(-)2:new mirror name: " ":archive url:_urls" ":distribution:($dists)" "*:components:_values -s ' ' components $components"
;;
list)
@@ -211,6 +212,7 @@ local keyring="*-keyring=[gpg keyring to use when verifying Release file (could
$keyring \
"-max-tries=[max download tries till process fails with download error]:number: " \
"-skip-existing-packages=[do not check file existence for packages listed in the internal database of the mirror]:$bool" \
"-latest=[download only latest version of each package (per architecture)]:$bool" \
"(-)2:mirror name:$mirrors"
;;
rename)
@@ -223,6 +225,7 @@ local keyring="*-keyring=[gpg keyring to use when verifying Release file (could
"-filter-with-deps=[when filtering, include dependencies of matching packages as well]:$bool" \
"-with-sources=[download source packages in addition to binary packages]:$bool" \
"-with-udebs=[download .udeb packages (Debian installer support)]:$bool" \
"-with-appstream=[download AppStream (DEP-11) metadata]:$bool" \
"(-)2:mirror name:$mirrors"
;;
search)
+3 -3
View File
@@ -203,7 +203,7 @@ _aptly()
"create")
if [[ $numargs -eq 0 ]]; then
if [[ "$cur" == -* ]]; then
COMPREPLY=($(compgen -W "-filter= -filter-with-deps -force-components -ignore-signatures -keyring= -with-installer -with-sources -with-udebs" -- ${cur}))
COMPREPLY=($(compgen -W "-filter= -filter-with-deps -force-components -ignore-signatures -keyring= -with-appstream -with-installer -with-sources -with-udebs" -- ${cur}))
return 0
fi
fi
@@ -211,7 +211,7 @@ _aptly()
"edit")
if [[ $numargs -eq 0 ]]; then
if [[ "$cur" == -* ]]; then
COMPREPLY=($(compgen -W "-archive-url= -filter= -filter-with-deps -ignore-signatures -keyring= -with-installer -with-sources -with-udebs" -- ${cur}))
COMPREPLY=($(compgen -W "-archive-url= -filter= -filter-with-deps -ignore-signatures -keyring= -with-appstream -with-installer -with-sources -with-udebs" -- ${cur}))
else
COMPREPLY=($(compgen -W "$(__aptly_mirror_list)" -- ${cur}))
fi
@@ -263,7 +263,7 @@ _aptly()
"update")
if [[ $numargs -eq 0 ]]; then
if [[ "$cur" == -* ]]; then
COMPREPLY=($(compgen -W "-force -download-limit= -downloader= -ignore-checksums -ignore-signatures -keyring= -skip-existing-packages" -- ${cur}))
COMPREPLY=($(compgen -W "-force -download-limit= -downloader= -ignore-checksums -ignore-signatures -keyring= -skip-existing-packages -latest" -- ${cur}))
else
COMPREPLY=($(compgen -W "$(__aptly_mirror_list)" -- ${cur}))
fi
+28 -1
View File
@@ -23,7 +23,9 @@ import (
"github.com/aptly-dev/aptly/database/goleveldb"
"github.com/aptly-dev/aptly/deb"
"github.com/aptly-dev/aptly/files"
"github.com/aptly-dev/aptly/gcs"
"github.com/aptly-dev/aptly/http"
"github.com/aptly-dev/aptly/jfrog"
"github.com/aptly-dev/aptly/pgp"
"github.com/aptly-dev/aptly/s3"
"github.com/aptly-dev/aptly/swift"
@@ -100,7 +102,6 @@ func (context *AptlyContext) config() *utils.ConfigStructure {
configLocations := []string{homeLocation, "/usr/local/etc/aptly.conf", "/etc/aptly.conf"}
for _, configLocation := range configLocations {
// FIXME: check if exists, check if readable
err = utils.LoadConfig(configLocation, &utils.Config)
if os.IsPermission(err) || os.IsNotExist(err) {
continue
@@ -443,6 +444,20 @@ func (context *AptlyContext) GetPublishedStorage(name string) (aptly.PublishedSt
if err != nil {
return nil, err
}
} else if strings.HasPrefix(name, "gcs:") {
params, ok := context.config().GCSPublishRoots[name[4:]]
if !ok {
return nil, fmt.Errorf("published GCS storage %v not configured", name[4:])
}
var err error
publishedStorage, err = gcs.NewPublishedStorage(
params.Bucket, params.Prefix, params.CredentialsFile, params.ServiceAccountJSON,
params.Project, params.Endpoint, params.ACL, params.StorageClass, params.EncryptionKey,
params.DisableMultiDel, params.Debug)
if err != nil {
return nil, err
}
} else if strings.HasPrefix(name, "swift:") {
params, ok := context.config().SwiftPublishRoots[name[6:]]
if !ok {
@@ -467,6 +482,18 @@ func (context *AptlyContext) GetPublishedStorage(name string) (aptly.PublishedSt
if err != nil {
return nil, err
}
} else if strings.HasPrefix(name, "jfrog:") {
params, ok := context.config().JFrogPublishRoots[name[6:]]
if !ok {
return nil, fmt.Errorf("published JFrog storage %v not configured", name[6:])
}
var err error
publishedStorage, err = jfrog.NewPublishedStorage(
name[6:], params)
if err != nil {
return nil, fmt.Errorf("error creating jfrog manager: %w", err)
}
} else {
return nil, fmt.Errorf("unknown published storage format: %v", name)
}
+57
View File
@@ -1,9 +1,11 @@
package context
import (
"fmt"
"reflect"
"testing"
"github.com/aptly-dev/aptly/utils"
"github.com/smira/flag"
. "gopkg.in/check.v1"
@@ -84,3 +86,58 @@ func (s *AptlyContextSuite) TestGetPublishedStorageBadFS(c *C) {
_, err := s.context.GetPublishedStorage("filesystem:fuji")
c.Assert(err, NotNil)
}
func (s *AptlyContextSuite) TestGetPublishedStorageJFrogConfigured(c *C) {
prevConfig := utils.Config
defer func() { utils.Config = prevConfig }()
s.context.configLoaded = true
utils.Config.RootDir = c.MkDir()
utils.Config.JFrogPublishRoots = map[string]utils.JFrogPublishRoot{
"test": {
Repository: "aptly-repo",
URL: "https://example.jfrog.local/artifactory",
AccessToken: "token",
Prefix: "public",
},
}
storage, err := s.context.GetPublishedStorage("jfrog:test")
c.Assert(err, IsNil)
c.Assert(storage, NotNil)
c.Assert(fmt.Sprintf("%v", storage), Equals, "jfrog:aptly-repo:public")
// Ensure we get the cached object on repeated lookups.
storageAgain, err := s.context.GetPublishedStorage("jfrog:test")
c.Assert(err, IsNil)
c.Assert(storageAgain, Equals, storage)
}
func (s *AptlyContextSuite) TestGetPublishedStorageJFrogMissing(c *C) {
prevConfig := utils.Config
defer func() { utils.Config = prevConfig }()
s.context.configLoaded = true
utils.Config.JFrogPublishRoots = map[string]utils.JFrogPublishRoot{}
_, err := s.context.GetPublishedStorage("jfrog:missing")
c.Assert(err, NotNil)
c.Check(err.Error(), Equals, "published JFrog storage missing not configured")
}
func (s *AptlyContextSuite) TestGetPublishedStorageJFrogInitError(c *C) {
prevConfig := utils.Config
defer func() { utils.Config = prevConfig }()
s.context.configLoaded = true
utils.Config.JFrogPublishRoots = map[string]utils.JFrogPublishRoot{
"broken": {
Repository: "aptly-repo",
URL: "ssh://example.local/artifactory",
},
}
_, err := s.context.GetPublishedStorage("jfrog:broken")
c.Assert(err, NotNil)
c.Check(err.Error(), Matches, `error creating jfrog manager: .*`)
}
+3
View File
@@ -26,8 +26,11 @@ var (
"Version",
"Codename",
"Date",
"Valid-Until",
"NotAutomatic",
"ButAutomaticUpgrades",
"Acquire-By-Hash",
"Signed-By",
"Architectures",
"Architecture",
"Components",
+33 -1
View File
@@ -172,6 +172,39 @@ func (l *PackageList) ForEach(handler func(*Package) error) error {
return err
}
// FilterLatest creates a copy of the package list containing only the
// latest version for each package name/architecture pair.
func (l *PackageList) FilterLatest() (*PackageList, error) {
if l == nil {
return nil, fmt.Errorf("package list is nil")
}
filtered := make(map[string]*Package, l.Len())
err := l.ForEach(func(p *Package) error {
key := p.Architecture + "|" + p.Name
if existing, found := filtered[key]; !found || CompareVersions(p.Version, existing.Version) > 0 {
filtered[key] = p
}
return nil
})
if err != nil {
return nil, err
}
result := NewPackageListWithDuplicates(l.duplicatesAllowed, len(filtered))
for _, pkg := range filtered {
if err = result.Add(pkg); err != nil {
return nil, err
}
}
return result, nil
}
// ForEachIndexed calls handler for each package in list in indexed order
func (l *PackageList) ForEachIndexed(handler func(*Package) error) error {
if !l.indexed {
@@ -598,7 +631,6 @@ func (l *PackageList) Filter(options FilterOptions) (*PackageList, error) {
//
// when follow-all-variants is enabled, we need to try to expand anyway,
// as even if dependency is satisfied now, there might be other ways to satisfy dependency
// FIXME: do not search twice
if result.Search(dep, false, true) != nil {
if options.DependencyOptions&DepVerboseResolve == DepVerboseResolve && options.Progress != nil {
options.Progress.ColoredPrintf("@{y}Already satisfied dependency@|: %s with %s", &dep, result.Search(dep, true, true))
+49
View File
@@ -503,3 +503,52 @@ func (s *PackageListSuite) TestArchitectures(c *C) {
sort.Strings(archs)
c.Check(archs, DeepEquals, []string{"amd64", "arm", "i386", "s390"})
}
func (s *PackageListSuite) TestFilterLatest(c *C) {
list := NewPackageList()
older := packageStanza.Copy()
older["Version"] = "1.0"
olderPkg := NewPackageFromControlFile(older)
_ = list.Add(olderPkg)
newer := packageStanza.Copy()
newer["Version"] = "2.0"
newerPkg := NewPackageFromControlFile(newer)
_ = list.Add(newerPkg)
shared := packageStanza.Copy()
shared["Architecture"] = ArchitectureAll
shared["Version"] = "3.0"
shared["Package"] = "shared"
sharedPkg := NewPackageFromControlFile(shared)
_ = list.Add(sharedPkg)
filtered, err := list.FilterLatest()
c.Assert(err, IsNil)
c.Assert(filtered.Len(), Equals, 2)
c.Check(filtered.Has(newerPkg), Equals, true)
c.Check(filtered.Has(sharedPkg), Equals, true)
}
func (s *PackageListSuite) TestFilterLatestPreservesDuplicatesFlag(c *C) {
list := NewPackageListWithDuplicates(true, 2)
_ = list.Add(NewPackageFromControlFile(packageStanza.Copy()))
another := packageStanza.Copy()
another["Version"] = "7.41-1"
_ = list.Add(NewPackageFromControlFile(another))
filtered, err := list.FilterLatest()
c.Assert(err, IsNil)
c.Assert(filtered.duplicatesAllowed, Equals, true)
}
func (s *PackageListSuite) TestFilterLatestNil(c *C) {
var list *PackageList
filtered, err := list.FilterLatest()
c.Assert(err, ErrorMatches, "package list is nil")
c.Assert(filtered, IsNil)
}
+1 -1
View File
@@ -59,7 +59,7 @@ func (s *PackageSuite) TestNewUdebFromPara(c *C) {
}
func (s *PackageSuite) TestNewInstallerFromPara(c *C) {
repo, _ := NewRemoteRepo("yandex", "http://example.com/debian", "squeeze", []string{"main"}, []string{}, false, false, false)
repo, _ := NewRemoteRepo("yandex", "http://example.com/debian", "squeeze", []string{"main"}, []string{}, false, false, false, false)
downloader := http.NewFakeDownloader()
downloader.ExpectResponse("http://example.com/debian/dists/squeeze/main/installer-i386/current/images/MANIFEST.udebs", "MANIFEST.udebs")
downloader.ExpectResponse("http://example.com/debian/dists/squeeze/main/installer-i386/current/images/udeb.list", "udeb.list")
+60 -2
View File
@@ -55,6 +55,7 @@ type PublishedRepo struct {
Label string
Suite string
Codename string
Version string
// Architectures is a list of all architectures published
Architectures []string
// SourceKind is "local"/"repo"
@@ -82,6 +83,11 @@ type PublishedRepo struct {
// Provide index files per hash also
AcquireByHash bool
// An optional field containing a comma separated list
// of OpenPGP key fingerprints to be used
// for validating the next Release file
SignedBy string
// Support multiple distributions
MultiDist bool
@@ -521,6 +527,7 @@ func (p *PublishedRepo) MarshalJSON() ([]byte, error) {
"Origin": p.Origin,
"Suite": p.Suite,
"Codename": p.Codename,
"Version": p.Version,
"NotAutomatic": p.NotAutomatic,
"ButAutomaticUpgrades": p.ButAutomaticUpgrades,
"Prefix": p.Prefix,
@@ -530,6 +537,7 @@ func (p *PublishedRepo) MarshalJSON() ([]byte, error) {
"Storage": p.Storage,
"SkipContents": p.SkipContents,
"AcquireByHash": p.AcquireByHash,
"SignedBy": p.SignedBy,
"MultiDist": p.MultiDist,
})
}
@@ -1059,6 +1067,38 @@ func (p *PublishedRepo) Publish(packagePool aptly.PackagePool, publishedStorageP
}
}
// Pass-through AppStream (DEP-11) files from snapshots
for component, item := range p.sourceItems {
if item.snapshot == nil || len(item.snapshot.AppStreamFiles) == 0 {
continue
}
prefix := component + "/"
for relPath, poolPath := range item.snapshot.AppStreamFiles {
if !strings.HasPrefix(relPath, prefix) {
continue
}
withinComponent := strings.TrimPrefix(relPath, prefix)
poolFile, err := packagePool.Open(poolPath)
if err != nil {
return fmt.Errorf("unable to open AppStream file from pool: %v", err)
}
bufWriter, err := indexes.SkelIndex(component, withinComponent).BufWriter()
if err != nil {
_ = poolFile.Close()
return fmt.Errorf("unable to generate AppStream index: %v", err)
}
_, err = bufio.NewReader(poolFile).WriteTo(bufWriter)
_ = poolFile.Close()
if err != nil {
return fmt.Errorf("unable to write AppStream file: %v", err)
}
}
}
udebs := []bool{false}
if hadUdebs {
udebs = append(udebs, true)
@@ -1083,6 +1123,12 @@ func (p *PublishedRepo) Publish(packagePool aptly.PackagePool, publishedStorageP
if p.AcquireByHash {
release["Acquire-By-Hash"] = "yes"
}
if p.SignedBy != "" {
release["Signed-By"] = p.SignedBy
}
if p.Version != "" {
release["Version"] = p.Version
}
var bufWriter *bufio.Writer
bufWriter, err = indexes.ReleaseIndex(component, arch, udeb).BufWriter()
@@ -1139,7 +1185,7 @@ func (p *PublishedRepo) Publish(packagePool aptly.PackagePool, publishedStorageP
release["Label"] = p.GetLabel()
release["Suite"] = p.GetSuite()
release["Codename"] = p.GetCodename()
datetimeFormat := "Mon, 2 Jan 2006 15:04:05 MST"
datetimeformat := "Mon, 2 Jan 2006 15:04:05 MST"
publishDate := time.Now().UTC()
if epoch := os.Getenv("SOURCE_DATE_EPOCH"); epoch != "" {
@@ -1147,11 +1193,23 @@ func (p *PublishedRepo) Publish(packagePool aptly.PackagePool, publishedStorageP
publishDate = time.Unix(sec, 0).UTC()
}
}
release["Date"] = publishDate.Format(datetimeFormat)
release["Date"] = publishDate.Format(datetimeformat)
release["Architectures"] = strings.Join(utils.StrSlicesSubstract(p.Architectures, []string{ArchitectureSource}), " ")
if p.AcquireByHash {
release["Acquire-By-Hash"] = "yes"
}
if p.SignedBy != "" {
// "If the field is present, a client should only accept future updates
// to the repository that are signed with keys listed in the field.
// The field should be ignored if the Valid-Until field
// is not present or if it is expired."
release["Signed-By"] = p.SignedBy
// Let's use a century as a "forever" value.
release["Valid-Until"] = publishDate.AddDate(100, 0, 0).Format(datetimeformat)
}
if p.Version != "" {
release["Version"] = p.Version
}
release["Description"] = " Generated by aptly\n"
release["MD5Sum"] = ""
release["SHA1"] = ""
+77 -1
View File
@@ -13,6 +13,7 @@ import (
"github.com/aptly-dev/aptly/database"
"github.com/aptly-dev/aptly/database/goleveldb"
"github.com/aptly-dev/aptly/files"
"github.com/aptly-dev/aptly/utils"
"github.com/ugorji/go/codec"
. "gopkg.in/check.v1"
@@ -115,7 +116,7 @@ func (s *PublishedRepoSuite) SetUpTest(c *C) {
s.reflist = NewPackageRefListFromPackageList(s.list)
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false)
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false, false)
repo.packageRefs = s.reflist
_ = s.factory.RemoteRepoCollection().Add(repo)
@@ -425,6 +426,81 @@ func (s *PublishedRepoSuite) TestPublish(c *C) {
c.Assert(err, IsNil)
}
func (s *PublishedRepoSuite) TestPublishAppStream(c *C) {
// Components + icons
content1 := []byte("DEP-11 test content for Components-amd64.yml.gz")
tmpFile1 := filepath.Join(c.MkDir(), "Components-amd64.yml.gz")
c.Assert(os.WriteFile(tmpFile1, content1, 0644), IsNil)
checksums1 := utils.ChecksumInfo{Size: int64(len(content1))}
poolPath1, err := s.packagePool.Import(tmpFile1, "Components-amd64.yml.gz", &checksums1, false, s.cs)
c.Assert(err, IsNil)
content2 := []byte("DEP-11 icons tar data")
tmpFile2 := filepath.Join(c.MkDir(), "icons-48x48.tar.gz")
c.Assert(os.WriteFile(tmpFile2, content2, 0644), IsNil)
checksums2 := utils.ChecksumInfo{Size: int64(len(content2))}
poolPath2, err := s.packagePool.Import(tmpFile2, "icons-48x48.tar.gz", &checksums2, false, s.cs)
c.Assert(err, IsNil)
// Include contrib file that should be skipped
contribContent := []byte("DEP-11 contrib content")
tmpFile3 := filepath.Join(c.MkDir(), "Components-contrib.yml.gz")
c.Assert(os.WriteFile(tmpFile3, contribContent, 0644), IsNil)
checksums3 := utils.ChecksumInfo{Size: int64(len(contribContent))}
poolPath3, err := s.packagePool.Import(tmpFile3, "Components-contrib.yml.gz", &checksums3, false, s.cs)
c.Assert(err, IsNil)
s.snapshot.AppStreamFiles = map[string]string{
"main/dep11/Components-amd64.yml.gz": poolPath1,
"main/dep11/icons-48x48.tar.gz": poolPath2,
"contrib/dep11/Components-amd64.yml.gz": poolPath3,
}
err = s.repo.Publish(s.packagePool, s.provider, s.factory, &NullSigner{}, nil, false, "")
c.Assert(err, IsNil)
// Both main files should exist
appstreamPath1 := filepath.Join(s.publishedStorage.PublicPath(), "ppa/dists/squeeze/main/dep11/Components-amd64.yml.gz")
c.Check(appstreamPath1, PathExists)
actual1, err := os.ReadFile(appstreamPath1)
c.Assert(err, IsNil)
c.Check(actual1, DeepEquals, content1)
appstreamPath2 := filepath.Join(s.publishedStorage.PublicPath(), "ppa/dists/squeeze/main/dep11/icons-48x48.tar.gz")
c.Check(appstreamPath2, PathExists)
actual2, err := os.ReadFile(appstreamPath2)
c.Assert(err, IsNil)
c.Check(actual2, DeepEquals, content2)
// Contrib file should not appear
contribPath := filepath.Join(s.publishedStorage.PublicPath(), "ppa/dists/squeeze/contrib/dep11/Components-amd64.yml.gz")
_, statErr := os.Stat(contribPath)
c.Check(os.IsNotExist(statErr), Equals, true)
// Release file should reference AppStream files
rf, err := os.Open(filepath.Join(s.publishedStorage.PublicPath(), "ppa/dists/squeeze/Release"))
c.Assert(err, IsNil)
defer func() { _ = rf.Close() }()
cfr := NewControlFileReader(rf, true, false)
st, err := cfr.ReadStanza()
c.Assert(err, IsNil)
c.Check(st["MD5Sum"], Matches, "(?s).*main/dep11/Components-amd64\\.yml\\.gz.*")
c.Check(st["SHA256"], Matches, "(?s).*main/dep11/Components-amd64\\.yml\\.gz.*")
// Pool open error
s.snapshot.AppStreamFiles = map[string]string{
"main/dep11/Components-amd64.yml.gz": "nonexistent/pool/path",
}
err = s.repo.Publish(s.packagePool, s.provider, s.factory, &NullSigner{}, nil, false, "")
c.Assert(err, ErrorMatches, "unable to open AppStream file from pool.*")
}
func (s *PublishedRepoSuite) TestPublishNoSigner(c *C) {
err := s.repo.Publish(s.packagePool, s.provider, s.factory, nil, nil, false, "")
c.Assert(err, IsNil)
+101 -2
View File
@@ -70,6 +70,10 @@ type RemoteRepo struct {
DownloadUdebs bool
// Should we download installer files?
DownloadInstaller bool
// Should we download AppStream (DEP-11) metadata?
DownloadAppStream bool
// AppStream files: relative path (e.g. "main/dep11/Components-amd64.yml.gz") → pool path
AppStreamFiles map[string]string `codec:"AppStreamFiles" json:"-"`
// Packages for json output
Packages []string `codec:"-" json:",omitempty"`
// "Snapshot" of current list of packages
@@ -82,7 +86,7 @@ type RemoteRepo struct {
// NewRemoteRepo creates new instance of Debian remote repository with specified params
func NewRemoteRepo(name string, archiveRoot string, distribution string, components []string,
architectures []string, downloadSources bool, downloadUdebs bool, downloadInstaller bool) (*RemoteRepo, error) {
architectures []string, downloadSources bool, downloadUdebs bool, downloadInstaller bool, downloadAppStream bool) (*RemoteRepo, error) {
result := &RemoteRepo{
UUID: uuid.NewString(),
Name: name,
@@ -93,6 +97,7 @@ func NewRemoteRepo(name string, archiveRoot string, distribution string, compone
DownloadSources: downloadSources,
DownloadUdebs: downloadUdebs,
DownloadInstaller: downloadInstaller,
DownloadAppStream: downloadAppStream,
}
err := result.prepare()
@@ -111,6 +116,9 @@ func NewRemoteRepo(name string, archiveRoot string, distribution string, compone
if result.DownloadUdebs {
return nil, fmt.Errorf("debian-installer udebs aren't supported for flat repos")
}
if result.DownloadAppStream {
return nil, fmt.Errorf("AppStream (DEP-11) metadata isn't supported for flat repos")
}
result.Components = nil
}
@@ -147,6 +155,9 @@ func (repo *RemoteRepo) String() string {
if repo.DownloadInstaller {
srcFlag += " [installer]"
}
if repo.DownloadAppStream {
srcFlag += " [appstream]"
}
distribution := repo.Distribution
if distribution == "" {
distribution = "./"
@@ -264,6 +275,82 @@ func (repo *RemoteRepo) InstallerPath(component string, architecture string) str
return fmt.Sprintf("%s/installer-%s/current/images/SHA256SUMS", component, architecture)
}
// AppStreamPaths returns dep11 file paths from ReleaseFiles for a given component
func (repo *RemoteRepo) AppStreamPaths(component string) []string {
prefix := component + "/dep11/"
var paths []string
for path := range repo.ReleaseFiles {
if strings.HasPrefix(path, prefix) {
paths = append(paths, path)
}
}
sort.Strings(paths)
return paths
}
// DownloadAppStreamFiles downloads AppStream (DEP-11) metadata files and imports them into the pool
func (repo *RemoteRepo) DownloadAppStreamFiles(progress aptly.Progress, d aptly.Downloader,
packagePool aptly.PackagePool, checksumStorage aptly.ChecksumStorage, ignoreChecksums bool) error {
repo.AppStreamFiles = make(map[string]string)
for _, component := range repo.Components {
paths := repo.AppStreamPaths(component)
if len(paths) == 0 {
continue
}
for _, relativePath := range paths {
info, ok := repo.ReleaseFiles[relativePath]
if !ok {
continue
}
url := repo.IndexesRootURL().ResolveReference(&url.URL{Path: relativePath}).String()
if progress != nil {
progress.Printf("Downloading AppStream file %s...\n", relativePath)
}
tempDir, err := os.MkdirTemp("", "aptly-appstream-*")
if err != nil {
return fmt.Errorf("unable to create temp dir for AppStream file %s: %s", relativePath, err)
}
tempPath := path.Join(tempDir, path.Base(relativePath))
var expected *utils.ChecksumInfo
if !ignoreChecksums {
expected = &info
}
err = d.DownloadWithChecksum(gocontext.TODO(), url, tempPath, expected, ignoreChecksums)
if err != nil {
_ = os.RemoveAll(tempDir)
// Skip files that are not found (some repos list dep11 files but don't serve them)
if herr, ok := err.(*http.Error); ok && (herr.Code == 404 || herr.Code == 403) {
if progress != nil {
progress.ColoredPrintf("@y[!]@| @!skipping AppStream file %s: not found@|", relativePath)
}
continue
}
return fmt.Errorf("unable to download AppStream file %s: %s", relativePath, err)
}
basename := path.Base(relativePath)
poolPath, err := packagePool.Import(tempPath, basename, &info, true, checksumStorage)
_ = os.RemoveAll(tempDir)
if err != nil {
return fmt.Errorf("unable to import AppStream file %s: %s", relativePath, err)
}
repo.AppStreamFiles[relativePath] = poolPath
}
}
return nil
}
// PackageURL returns URL of package file relative to repository root
// architecture
func (repo *RemoteRepo) PackageURL(filename string) *url.URL {
@@ -612,7 +699,19 @@ func (repo *RemoteRepo) ApplyFilter(dependencyOptions int, filterQuery PackageQu
}
// BuildDownloadQueue builds queue, discards current PackageList
func (repo *RemoteRepo) BuildDownloadQueue(packagePool aptly.PackagePool, packageCollection *PackageCollection, checksumStorage aptly.ChecksumStorage, skipExistingPackages bool) (queue []PackageDownloadTask, downloadSize int64, err error) {
func (repo *RemoteRepo) BuildDownloadQueue(packagePool aptly.PackagePool, packageCollection *PackageCollection, checksumStorage aptly.ChecksumStorage, skipExistingPackages, latestOnly bool) (queue []PackageDownloadTask, downloadSize int64, err error) {
if repo.packageList == nil {
err = fmt.Errorf("package list is empty, please (re)download package indexes")
return
}
if latestOnly {
repo.packageList, err = repo.packageList.FilterLatest()
if err != nil {
return
}
}
queue = make([]PackageDownloadTask, 0, repo.packageList.Len())
seen := make(map[string]int, repo.packageList.Len())
+164 -26
View File
@@ -2,6 +2,7 @@ package deb
import (
"errors"
"fmt"
"io"
"os"
"sort"
@@ -90,8 +91,8 @@ type RemoteRepoSuite struct {
var _ = Suite(&RemoteRepoSuite{})
func (s *RemoteRepoSuite) SetUpTest(c *C) {
s.repo, _ = NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian", "squeeze", []string{"main"}, []string{}, false, false, false)
s.flat, _ = NewRemoteRepo("exp42", "http://repos.express42.com/virool/precise/", "./", []string{}, []string{}, false, false, false)
s.repo, _ = NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian", "squeeze", []string{"main"}, []string{}, false, false, false, false)
s.flat, _ = NewRemoteRepo("exp42", "http://repos.express42.com/virool/precise/", "./", []string{}, []string{}, false, false, false, false)
s.downloader = http.NewFakeDownloader().ExpectResponse("http://mirror.yandex.ru/debian/dists/squeeze/Release", exampleReleaseFile)
s.progress = console.NewProgress(false)
s.db, _ = goleveldb.NewOpenDB(c.MkDir())
@@ -108,7 +109,7 @@ func (s *RemoteRepoSuite) TearDownTest(c *C) {
}
func (s *RemoteRepoSuite) TestInvalidURL(c *C) {
_, err := NewRemoteRepo("s", "http://lolo%2", "squeeze", []string{"main"}, []string{}, false, false, false)
_, err := NewRemoteRepo("s", "http://lolo%2", "squeeze", []string{"main"}, []string{}, false, false, false, false)
c.Assert(err, ErrorMatches, ".*(hexadecimal escape in host|percent-encoded characters in host|invalid URL escape).*")
}
@@ -117,12 +118,15 @@ func (s *RemoteRepoSuite) TestFlatCreation(c *C) {
c.Check(s.flat.Distribution, Equals, "./")
c.Check(s.flat.Components, IsNil)
flat2, _ := NewRemoteRepo("flat2", "http://pkg.jenkins-ci.org/debian-stable", "binary/", []string{}, []string{}, false, false, false)
flat2, _ := NewRemoteRepo("flat2", "http://pkg.jenkins-ci.org/debian-stable", "binary/", []string{}, []string{}, false, false, false, false)
c.Check(flat2.IsFlat(), Equals, true)
c.Check(flat2.Distribution, Equals, "./binary/")
_, err := NewRemoteRepo("fl", "http://some.repo/", "./", []string{"main"}, []string{}, false, false, false)
_, err := NewRemoteRepo("fl", "http://some.repo/", "./", []string{"main"}, []string{}, false, false, false, false)
c.Check(err, ErrorMatches, "components aren't supported for flat repos")
_, err = NewRemoteRepo("fl", "http://some.repo/", "./", []string{}, []string{}, false, false, false, true)
c.Check(err, ErrorMatches, "AppStream \\(DEP-11\\) metadata isn't supported for flat repos")
}
func (s *RemoteRepoSuite) TestString(c *C) {
@@ -135,6 +139,43 @@ func (s *RemoteRepoSuite) TestString(c *C) {
s.flat.DownloadSources = true
c.Check(s.repo.String(), Equals, "[yandex]: http://mirror.yandex.ru/debian/ squeeze [src] [udeb] [installer]")
c.Check(s.flat.String(), Equals, "[exp42]: http://repos.express42.com/virool/precise/ ./ [src]")
s.repo.DownloadAppStream = true
c.Check(s.repo.String(), Equals, "[yandex]: http://mirror.yandex.ru/debian/ squeeze [src] [udeb] [installer] [appstream]")
// AppStream is not supported for flat repos, so no flat test here
}
func (s *RemoteRepoSuite) TestAppStreamPaths(c *C) {
s.repo.ReleaseFiles = nil
c.Check(s.repo.AppStreamPaths("main"), DeepEquals, []string(nil))
s.repo.ReleaseFiles = map[string]utils.ChecksumInfo{
"main/binary-amd64/Packages": {Size: 100},
"main/dep11/Components-amd64.yml.gz": {Size: 200},
"main/dep11/Components-i386.yml.gz": {Size: 300},
"main/dep11/icons-48x48.tar.gz": {Size: 400},
"contrib/dep11/Components-amd64.yml.gz": {Size: 500},
"main/source/Sources": {Size: 600},
}
paths := s.repo.AppStreamPaths("main")
c.Check(paths, DeepEquals, []string{
"main/dep11/Components-amd64.yml.gz",
"main/dep11/Components-i386.yml.gz",
"main/dep11/icons-48x48.tar.gz",
})
paths = s.repo.AppStreamPaths("contrib")
c.Check(paths, DeepEquals, []string{
"contrib/dep11/Components-amd64.yml.gz",
})
paths = s.repo.AppStreamPaths("non-free")
c.Check(paths, DeepEquals, []string(nil))
s.repo.ReleaseFiles = map[string]utils.ChecksumInfo{}
c.Check(s.repo.AppStreamPaths("main"), DeepEquals, []string(nil))
}
func (s *RemoteRepoSuite) TestNumPackages(c *C) {
@@ -236,13 +277,13 @@ func (s *RemoteRepoSuite) TestFetchNullVerifier2(c *C) {
}
func (s *RemoteRepoSuite) TestFetchWrongArchitecture(c *C) {
s.repo, _ = NewRemoteRepo("s", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{"xyz"}, false, false, false)
s.repo, _ = NewRemoteRepo("s", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{"xyz"}, false, false, false, false)
err := s.repo.Fetch(s.downloader, nil, true)
c.Assert(err, ErrorMatches, "architecture xyz not available in repo.*")
}
func (s *RemoteRepoSuite) TestFetchWrongComponent(c *C) {
s.repo, _ = NewRemoteRepo("s", "http://mirror.yandex.ru/debian/", "squeeze", []string{"xyz"}, []string{"i386"}, false, false, false)
s.repo, _ = NewRemoteRepo("s", "http://mirror.yandex.ru/debian/", "squeeze", []string{"xyz"}, []string{"i386"}, false, false, false, false)
err := s.repo.Fetch(s.downloader, nil, true)
c.Assert(err, ErrorMatches, "component xyz not available in repo.*")
}
@@ -281,7 +322,7 @@ func (s *RemoteRepoSuite) TestDownload(c *C) {
c.Assert(err, IsNil)
c.Assert(s.downloader.Empty(), Equals, true)
queue, size, err := s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err := s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(3))
c.Check(queue, HasLen, 1)
@@ -308,7 +349,7 @@ func (s *RemoteRepoSuite) TestDownload(c *C) {
c.Assert(err, IsNil)
c.Assert(s.downloader.Empty(), Equals, true)
queue, size, err = s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, true)
queue, size, err = s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, true, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(0))
c.Check(queue, HasLen, 0)
@@ -329,7 +370,7 @@ func (s *RemoteRepoSuite) TestDownload(c *C) {
c.Assert(err, IsNil)
c.Assert(s.downloader.Empty(), Equals, true)
queue, size, err = s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err = s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(3))
c.Check(queue, HasLen, 1)
@@ -356,7 +397,7 @@ func (s *RemoteRepoSuite) TestDownloadWithInstaller(c *C) {
c.Assert(err, IsNil)
c.Assert(s.downloader.Empty(), Equals, true)
queue, size, err := s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err := s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(3)+int64(len(exampleInstallerManifestFile)))
c.Check(queue, HasLen, 2)
@@ -382,6 +423,35 @@ func (s *RemoteRepoSuite) TestDownloadWithInstaller(c *C) {
c.Check(pkg.Name, Equals, "installer")
}
func (s *RemoteRepoSuite) TestBuildDownloadQueueLatestOnly(c *C) {
s.repo.Architectures = []string{"i386"}
err := s.repo.Fetch(s.downloader, nil, true)
c.Assert(err, IsNil)
s.downloader.ExpectError("http://mirror.yandex.ru/debian/dists/squeeze/main/binary-i386/Packages.bz2", &http.Error{Code: 404})
s.downloader.ExpectError("http://mirror.yandex.ru/debian/dists/squeeze/main/binary-i386/Packages.gz", &http.Error{Code: 404})
s.downloader.ExpectResponse("http://mirror.yandex.ru/debian/dists/squeeze/main/binary-i386/Packages", examplePackagesFile)
err = s.repo.DownloadPackageIndexes(s.progress, s.downloader, nil, s.collectionFactory, true, false)
c.Assert(err, IsNil)
c.Assert(s.downloader.Empty(), Equals, true)
stanza := packageStanza.Copy()
stanza["Package"] = "amanda-client"
stanza["Version"] = "1:3.4.0-1"
stanza["Filename"] = "pool/main/a/amanda/amanda-client_3.4.0-1_i386.deb"
newest := NewPackageFromControlFile(stanza)
_ = s.repo.packageList.Add(newest)
queue, size, err := s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, true)
c.Assert(err, IsNil)
c.Check(queue, HasLen, 1)
c.Check(queue[0].File.DownloadURL(), Equals, "pool/main/a/amanda/amanda-client_3.4.0-1_i386.deb")
c.Check(size, Equals, int64(187518))
}
func (s *RemoteRepoSuite) TestDownloadWithSources(c *C) {
s.repo.Architectures = []string{"i386"}
s.repo.DownloadSources = true
@@ -400,7 +470,7 @@ func (s *RemoteRepoSuite) TestDownloadWithSources(c *C) {
c.Assert(err, IsNil)
c.Assert(s.downloader.Empty(), Equals, true)
queue, size, err := s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err := s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(15))
c.Check(queue, HasLen, 4)
@@ -444,7 +514,7 @@ func (s *RemoteRepoSuite) TestDownloadWithSources(c *C) {
c.Assert(err, IsNil)
c.Assert(s.downloader.Empty(), Equals, true)
queue, size, err = s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, true)
queue, size, err = s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, true, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(0))
c.Check(queue, HasLen, 0)
@@ -469,7 +539,7 @@ func (s *RemoteRepoSuite) TestDownloadWithSources(c *C) {
c.Assert(err, IsNil)
c.Assert(s.downloader.Empty(), Equals, true)
queue, size, err = s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err = s.repo.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(15))
c.Check(queue, HasLen, 4)
@@ -493,7 +563,7 @@ func (s *RemoteRepoSuite) TestDownloadFlat(c *C) {
c.Assert(err, IsNil)
c.Assert(downloader.Empty(), Equals, true)
queue, size, err := s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err := s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(3))
c.Check(queue, HasLen, 1)
@@ -521,7 +591,7 @@ func (s *RemoteRepoSuite) TestDownloadFlat(c *C) {
c.Assert(err, IsNil)
c.Assert(downloader.Empty(), Equals, true)
queue, size, err = s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, true)
queue, size, err = s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, true, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(0))
c.Check(queue, HasLen, 0)
@@ -543,7 +613,7 @@ func (s *RemoteRepoSuite) TestDownloadFlat(c *C) {
c.Assert(err, IsNil)
c.Assert(downloader.Empty(), Equals, true)
queue, size, err = s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err = s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(3))
c.Check(queue, HasLen, 1)
@@ -574,7 +644,7 @@ func (s *RemoteRepoSuite) TestDownloadWithSourcesFlat(c *C) {
c.Assert(err, IsNil)
c.Assert(downloader.Empty(), Equals, true)
queue, size, err := s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err := s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(15))
c.Check(queue, HasLen, 4)
@@ -620,7 +690,7 @@ func (s *RemoteRepoSuite) TestDownloadWithSourcesFlat(c *C) {
c.Assert(err, IsNil)
c.Assert(downloader.Empty(), Equals, true)
queue, size, err = s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, true)
queue, size, err = s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, true, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(0))
c.Check(queue, HasLen, 0)
@@ -646,7 +716,7 @@ func (s *RemoteRepoSuite) TestDownloadWithSourcesFlat(c *C) {
c.Assert(err, IsNil)
c.Assert(downloader.Empty(), Equals, true)
queue, size, err = s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false)
queue, size, err = s.flat.BuildDownloadQueue(s.packagePool, s.collectionFactory.PackageCollection(), s.cs, false, false)
c.Assert(err, IsNil)
c.Check(size, Equals, int64(15))
c.Check(queue, HasLen, 4)
@@ -655,6 +725,74 @@ func (s *RemoteRepoSuite) TestDownloadWithSourcesFlat(c *C) {
c.Assert(s.flat.packageRefs, NotNil)
}
func (s *RemoteRepoSuite) TestDownloadAppStreamFiles(c *C) {
// No dep11 entries
s.repo.Components = []string{"main"}
s.repo.ReleaseFiles = map[string]utils.ChecksumInfo{
"main/binary-amd64/Packages": {Size: 100},
"main/source/Sources": {Size: 200},
}
err := s.repo.DownloadAppStreamFiles(s.progress, s.downloader, s.packagePool, s.cs, false)
c.Assert(err, IsNil)
c.Check(s.repo.AppStreamFiles, HasLen, 0)
s.repo.ReleaseFiles = map[string]utils.ChecksumInfo{
"main/dep11/Components-amd64.yml.gz": {Size: 16},
"main/dep11/icons-48x48.tar.gz": {Size: 16},
"main/binary-amd64/Packages": {Size: 100},
}
downloader := http.NewFakeDownloader()
downloader.AnyExpectResponse("http://mirror.yandex.ru/debian/dists/squeeze/main/dep11/Components-amd64.yml.gz", "dep11-components")
downloader.AnyExpectResponse("http://mirror.yandex.ru/debian/dists/squeeze/main/dep11/icons-48x48.tar.gz", "dep11-icons-data")
err = s.repo.DownloadAppStreamFiles(s.progress, downloader, s.packagePool, s.cs, false)
c.Assert(err, IsNil)
c.Check(s.repo.AppStreamFiles, HasLen, 2)
c.Check(s.repo.AppStreamFiles["main/dep11/Components-amd64.yml.gz"], Not(Equals), "")
c.Check(s.repo.AppStreamFiles["main/dep11/icons-48x48.tar.gz"], Not(Equals), "")
// 404 skipped
s.repo.ReleaseFiles = map[string]utils.ChecksumInfo{
"main/dep11/Components-amd64.yml.gz": {Size: 16},
"main/dep11/icons-48x48.tar.gz": {Size: 15},
}
downloader = http.NewFakeDownloader()
downloader.AnyExpectResponse("http://mirror.yandex.ru/debian/dists/squeeze/main/dep11/Components-amd64.yml.gz", "dep11-components")
downloader.ExpectError("http://mirror.yandex.ru/debian/dists/squeeze/main/dep11/icons-48x48.tar.gz", &http.Error{Code: 404, URL: "http://mirror.yandex.ru/debian/dists/squeeze/main/dep11/icons-48x48.tar.gz"})
err = s.repo.DownloadAppStreamFiles(s.progress, downloader, s.packagePool, s.cs, false)
c.Assert(err, IsNil)
c.Check(s.repo.AppStreamFiles, HasLen, 1)
c.Check(s.repo.AppStreamFiles["main/dep11/Components-amd64.yml.gz"], Not(Equals), "")
// Generic download error propagated
s.repo.ReleaseFiles = map[string]utils.ChecksumInfo{
"main/dep11/Components-amd64.yml.gz": {Size: 18},
}
downloader = http.NewFakeDownloader()
downloader.ExpectError("http://mirror.yandex.ru/debian/dists/squeeze/main/dep11/Components-amd64.yml.gz", fmt.Errorf("connection refused"))
err = s.repo.DownloadAppStreamFiles(s.progress, downloader, s.packagePool, s.cs, false)
c.Assert(err, ErrorMatches, "unable to download AppStream file.*connection refused")
// Bypass checksum validation
s.repo.ReleaseFiles = map[string]utils.ChecksumInfo{
"main/dep11/Components-amd64.yml.gz": {Size: 999, MD5: "bad", SHA256: "bad"},
}
downloader = http.NewFakeDownloader()
downloader.AnyExpectResponse("http://mirror.yandex.ru/debian/dists/squeeze/main/dep11/Components-amd64.yml.gz", "dep11-components")
err = s.repo.DownloadAppStreamFiles(s.progress, downloader, s.packagePool, s.cs, true)
c.Assert(err, IsNil)
c.Check(s.repo.AppStreamFiles, HasLen, 1)
c.Check(s.repo.AppStreamFiles["main/dep11/Components-amd64.yml.gz"], Not(Equals), "")
}
type RemoteRepoCollectionSuite struct {
PackageListMixinSuite
db database.Storage
@@ -677,7 +815,7 @@ func (s *RemoteRepoCollectionSuite) TestAddByName(c *C) {
_, err := s.collection.ByName("yandex")
c.Assert(err, ErrorMatches, "*.not found")
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false)
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false, false)
c.Assert(s.collection.Add(repo), IsNil)
c.Assert(s.collection.Add(repo), ErrorMatches, ".*already exists")
@@ -695,7 +833,7 @@ func (s *RemoteRepoCollectionSuite) TestByUUID(c *C) {
_, err := s.collection.ByUUID("some-uuid")
c.Assert(err, ErrorMatches, "*.not found")
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false)
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false, false)
c.Assert(s.collection.Add(repo), IsNil)
r, err := s.collection.ByUUID(repo.UUID)
@@ -709,7 +847,7 @@ func (s *RemoteRepoCollectionSuite) TestByUUID(c *C) {
}
func (s *RemoteRepoCollectionSuite) TestUpdateLoadComplete(c *C) {
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false)
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false, false)
c.Assert(s.collection.Update(repo), IsNil)
collection := NewRemoteRepoCollection(s.db)
@@ -730,7 +868,7 @@ func (s *RemoteRepoCollectionSuite) TestUpdateLoadComplete(c *C) {
}
func (s *RemoteRepoCollectionSuite) TestForEachAndLen(c *C) {
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false)
repo, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false, false)
_ = s.collection.Add(repo)
count := 0
@@ -752,10 +890,10 @@ func (s *RemoteRepoCollectionSuite) TestForEachAndLen(c *C) {
}
func (s *RemoteRepoCollectionSuite) TestDrop(c *C) {
repo1, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false)
repo1, _ := NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false, false)
_ = s.collection.Add(repo1)
repo2, _ := NewRemoteRepo("tyndex", "http://mirror.yandex.ru/debian/", "wheezy", []string{"main"}, []string{}, false, false, false)
repo2, _ := NewRemoteRepo("tyndex", "http://mirror.yandex.ru/debian/", "wheezy", []string{"main"}, []string{}, false, false, false, false)
_ = s.collection.Add(repo2)
r1, _ := s.collection.ByUUID(repo1.UUID)
+25 -7
View File
@@ -40,6 +40,9 @@ type Snapshot struct {
NotAutomatic string
ButAutomaticUpgrades string
// AppStream files: relative path → pool path (pass-through from mirror)
AppStreamFiles map[string]string `json:",omitempty"`
packageRefs *PackageRefList
}
@@ -59,6 +62,7 @@ func NewSnapshotFromRepository(name string, repo *RemoteRepo) (*Snapshot, error)
Origin: repo.Meta["Origin"],
NotAutomatic: repo.Meta["NotAutomatic"],
ButAutomaticUpgrades: repo.Meta["ButAutomaticUpgrades"],
AppStreamFiles: repo.AppStreamFiles,
packageRefs: repo.packageRefs,
}, nil
}
@@ -94,14 +98,28 @@ func NewSnapshotFromRefList(name string, sources []*Snapshot, list *PackageRefLi
sourceUUIDs[i] = sources[i].UUID
}
// Merge AppStreamFiles from all source snapshots
var mergedAppStream map[string]string
for _, source := range sources {
if len(source.AppStreamFiles) > 0 {
if mergedAppStream == nil {
mergedAppStream = make(map[string]string)
}
for k, v := range source.AppStreamFiles {
mergedAppStream[k] = v
}
}
}
return &Snapshot{
UUID: uuid.NewString(),
Name: name,
CreatedAt: time.Now(),
SourceKind: "snapshot",
SourceIDs: sourceUUIDs,
Description: description,
packageRefs: list,
UUID: uuid.NewString(),
Name: name,
CreatedAt: time.Now(),
SourceKind: "snapshot",
SourceIDs: sourceUUIDs,
Description: description,
AppStreamFiles: mergedAppStream,
packageRefs: list,
}
}
+60 -4
View File
@@ -19,7 +19,7 @@ var _ = Suite(&SnapshotSuite{})
func (s *SnapshotSuite) SetUpTest(c *C) {
s.SetUpPackages()
s.repo, _ = NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false)
s.repo, _ = NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false, false)
s.repo.packageRefs = s.reflist
}
@@ -101,6 +101,62 @@ func (s *SnapshotSuite) TestEncodeDecode(c *C) {
c.Assert(snapshot2.packageRefs, IsNil)
}
func (s *SnapshotSuite) TestSnapshotFromRepositoryAppStream(c *C) {
s.repo.AppStreamFiles = map[string]string{
"main/dep11/Components-amd64.yml.gz": "ab/cd/Components-amd64.yml.gz",
"main/dep11/icons-48x48.tar.gz": "ef/gh/icons-48x48.tar.gz",
}
snapshot, err := NewSnapshotFromRepository("snap-as", s.repo)
c.Assert(err, IsNil)
c.Check(snapshot.AppStreamFiles, DeepEquals, s.repo.AppStreamFiles)
s.repo.AppStreamFiles = nil
snapshot2, err := NewSnapshotFromRepository("snap-no-as", s.repo)
c.Assert(err, IsNil)
c.Check(snapshot2.AppStreamFiles, IsNil)
}
func (s *SnapshotSuite) TestSnapshotFromRefListAppStreamMerge(c *C) {
snap1, _ := NewSnapshotFromRepository("snap1", s.repo)
snap1.AppStreamFiles = map[string]string{
"main/dep11/Components-amd64.yml.gz": "aa/bb/Components-amd64.yml.gz",
"main/dep11/icons-48x48.tar.gz": "cc/dd/icons-48x48.tar.gz",
}
snap2, _ := NewSnapshotFromRepository("snap2", s.repo)
snap2.AppStreamFiles = map[string]string{
"contrib/dep11/Components-amd64.yml.gz": "ee/ff/Components-amd64.yml.gz",
"main/dep11/Components-amd64.yml.gz": "xx/yy/Components-amd64.yml.gz",
}
merged := NewSnapshotFromRefList("merged", []*Snapshot{snap1, snap2}, s.reflist, "Merged")
c.Check(len(merged.AppStreamFiles), Equals, 3)
c.Check(merged.AppStreamFiles["main/dep11/icons-48x48.tar.gz"], Equals, "cc/dd/icons-48x48.tar.gz")
c.Check(merged.AppStreamFiles["contrib/dep11/Components-amd64.yml.gz"], Equals, "ee/ff/Components-amd64.yml.gz")
c.Check(merged.AppStreamFiles["main/dep11/Components-amd64.yml.gz"], Equals, "xx/yy/Components-amd64.yml.gz")
snap3, _ := NewSnapshotFromRepository("snap3", s.repo)
snap3.AppStreamFiles = nil
snap4, _ := NewSnapshotFromRepository("snap4", s.repo)
snap4.AppStreamFiles = nil
merged2 := NewSnapshotFromRefList("merged2", []*Snapshot{snap3, snap4}, s.reflist, "Merged2")
c.Check(merged2.AppStreamFiles, IsNil)
}
func (s *SnapshotSuite) TestEncodeDecodeAppStream(c *C) {
snapshot, _ := NewSnapshotFromRepository("snap-as-enc", s.repo)
snapshot.AppStreamFiles = map[string]string{
"main/dep11/Components-amd64.yml.gz": "ab/cd/Components-amd64.yml.gz",
"main/dep11/icons-48x48.tar.gz": "ef/gh/icons-48x48.tar.gz",
}
decoded := &Snapshot{}
c.Assert(decoded.Decode(snapshot.Encode()), IsNil)
c.Check(decoded.Name, Equals, snapshot.Name)
c.Check(decoded.AppStreamFiles, DeepEquals, snapshot.AppStreamFiles)
}
type SnapshotCollectionSuite struct {
PackageListMixinSuite
db database.Storage
@@ -118,11 +174,11 @@ func (s *SnapshotCollectionSuite) SetUpTest(c *C) {
s.collection = NewSnapshotCollection(s.db)
s.SetUpPackages()
s.repo1, _ = NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false)
s.repo1, _ = NewRemoteRepo("yandex", "http://mirror.yandex.ru/debian/", "squeeze", []string{"main"}, []string{}, false, false, false, false)
s.repo1.packageRefs = s.reflist
s.snapshot1, _ = NewSnapshotFromRepository("snap1", s.repo1)
s.repo2, _ = NewRemoteRepo("android", "http://mirror.yandex.ru/debian/", "lenny", []string{"main"}, []string{}, false, false, false)
s.repo2, _ = NewRemoteRepo("android", "http://mirror.yandex.ru/debian/", "lenny", []string{"main"}, []string{}, false, false, false, false)
s.repo2.packageRefs = s.reflist
s.snapshot2, _ = NewSnapshotFromRepository("snap2", s.repo2)
@@ -223,7 +279,7 @@ func (s *SnapshotCollectionSuite) TestFindByRemoteRepoSource(c *C) {
c.Check(s.collection.ByRemoteRepoSource(s.repo1), DeepEquals, []*Snapshot{s.snapshot1})
c.Check(s.collection.ByRemoteRepoSource(s.repo2), DeepEquals, []*Snapshot{s.snapshot2})
repo3, _ := NewRemoteRepo("other", "http://mirror.yandex.ru/debian/", "lenny", []string{"main"}, []string{}, false, false, false)
repo3, _ := NewRemoteRepo("other", "http://mirror.yandex.ru/debian/", "lenny", []string{"main"}, []string{}, false, false, false, false)
c.Check(s.collection.ByRemoteRepoSource(repo3), DeepEquals, []*Snapshot(nil))
}
Vendored
-33
View File
@@ -1,33 +0,0 @@
aptly (1.6.0+ds1-1) unstable; urgency=medium
- aptly-api: configuration file is now /etc/aptly.conf, and `rootDir`
defaults to `~/.aptly`
- aptly-api: default port is 8080, as declared in
`/etc/default/aptly-api`
- aptly: swagger support is disabled, but will be re-enabled after the
corresponding golang packages make it to unstable
- aptly: default format is yaml, but the old format is still supported
for now
-- Sebastien Delafond <seb@debian.org> Sun, 29 Dec 2024 08:46:07 +0100
aptly (1.3.0+ds1-2.2) unstable; urgency=medium
This version tries to fix the database backwards compatibility,
so you don't need to rebuild the database if you upgrade from
aptly <= 1.3.0-6.
However some fields are missing, like created time of a snapshot.
-- Shengjing Zhu <zhsj@debian.org> Sat, 13 Apr 2019 23:26:39 +0800
aptly (1.3.0+ds1-2) unstable; urgency=medium
* The database created by aptly <= 1.3.0-6 is not compatible
with greater versions. Users must create a new database.
The `aptly db recover` will not fix the issue.
-- Alexandre Viau <aviau@debian.org> Fri, 26 Oct 2018 13:20:53 -0400
+82 -2
View File
@@ -83,9 +83,8 @@ serve_in_api_mode: false
# Enable metrics for Prometheus client
enable_metrics_endpoint: false
# Not implemented in this version.
# Enable API documentation on /docs
#enable_swagger_endpoint: false
enable_swagger_endpoint: false
# OBSOLETE: use via url param ?_async=true
async_api: false
@@ -197,6 +196,35 @@ filesystem_publish_endpoints:
#
# `aptly publish snapshot wheezy-main s3:test:`
#
# JFrog Artifactory Endpoint Support
#
# aptly can be configured to publish repositories directly to JFrog Artifactory. First,
# publishing endpoints should be described in the aptly configuration file.
#
# The destination Artifactory repo should be of the "generic" type, not "debian".
#
# In order to publish to JFrog, specify endpoint as `jfrog:endpoint-name:` before
# publishing prefix on the command line, e.g.:
#
# `aptly publish snapshot wheezy-main jfrog:test:`
#
jfrog_publish_endpoints:
# # Endpoint Name
# test:
# # JFrog URL
# url: "https://artifactory.example.com/artifactory/"
# # Repository
# repository: apt-local
# # Jfrog credentials to authenticate to Artifactory. If not supplied, the
# # environment variables `JFROG_USERNAME`, `JFROG_PASSWORD`, `JFROG_APIKEY`,
# # and `JFROG_ACCESSTOKEN` can be used
# # Authentication requires one of: user+pass, api key, or access token
# username: admin
# password: password
# api_key: api_key
# access_token: access_token
s3_publish_endpoints:
# # Endpoint Name
# test:
@@ -257,6 +285,58 @@ s3_publish_endpoints:
# # Enables detailed request/response dump for each S3 operation
# debug: false
# GCS Endpoint Support
#
# aptly can be configured to publish repositories directly to Google Cloud
# Storage. First, publishing endpoints should be described in the aptly
# configuration file. Each endpoint has a name and associated settings.
#
# In order to publish to GCS, specify endpoint as `gcs:endpoint-name:` before
# publishing prefix on the command line, e.g.:
#
# `aptly publish snapshot wheezy-main gcs:test:`
#
gcs_publish_endpoints:
# # Endpoint Name
# test:
# # Bucket name
# bucket: test-bucket
# # Prefix (optional)
# # publishing under specified prefix in the bucket, defaults to
# # no prefix (bucket root)
# prefix: ""
# # Credentials File (optional)
# # Path to a service account credentials JSON file
# credentials_file: ""
# # Service Account JSON (optional)
# # Inline service account credentials JSON payload
# service_account_json: ""
# # Project (optional)
# # Quota project used for GCS requests
# project: ""
# # Endpoint (optional)
# # Override the GCS endpoint (e.g. for staging or a fake server);
# # leave empty to use the default GCS endpoint
# endpoint: ""
# # Default ACLs (optional)
# # assign ACL to published files:
# # * private (default)
# # * public-read (public repository)
# # * none (don't set ACL)
# acl: private
# # Storage Class (optional)
# # GCS storage class, e.g. `STANDARD`
# storage_class: STANDARD
# # Encryption Key (optional)
# # Customer-supplied encryption key (32-byte AES-256 key)
# encryption_key: ""
# # Disable MultiDel (optional)
# # Kept for parity with S3 settings; GCS deletes are one-by-one
# disable_multidel: false
# # Debug (optional)
# # Enables detailed logs for each GCS operation
# debug: false
# Swift Endpoint Support
#
# aptly can publish a repository directly to OpenStack Swift.
+51 -522
View File
@@ -1,522 +1,51 @@
aptly (1.6.2-3) unstable; urgency=medium
[ Sébastien Delafond ]
* tests: disable t04_mirror/create/CreateMirror18Test (Closes: #1135740)
* tests: disable t12_api/gpg/GPGAPITestAddKey (Closes: #1135672)
* d/control: bump-up Standards-Version
-- Sebastien Delafond <seb@debian.org> Tue, 05 May 2026 18:14:44 +0200
aptly (1.6.2-2) unstable; urgency=medium
[ Sébastien Delafond ]
* Remove Built-Using
* Mark patches as "Forwarded: not-needed"
-- Sebastien Delafond <seb@debian.org> Fri, 21 Nov 2025 15:46:51 +0100
aptly (1.6.2-1) unstable; urgency=medium
[ Sébastien Delafond ]
* d/watch: v5
* Bump up Standards-Version
* Remove +ds suffix
* Add Static-Built-Using
* New upstream version 1.6.2
-- Sebastien Delafond <seb@debian.org> Wed, 24 Sep 2025 06:19:54 +0200
aptly (1.6.1+ds1-3) unstable; urgency=medium
[ Sébastien Delafond ]
* tests: declare needs-internet for system-test
* tests: disable unit test TestVerifyClearsigned & system test
CreateMirror31Test (Closes: #1108828)
-- Sebastien Delafond <seb@debian.org> Tue, 08 Jul 2025 14:12:52 +0200
aptly (1.6.1+ds1-2) unstable; urgency=medium
[ Sébastien Delafond ]
* Do not re-publish unchanged files to S3 every single time (Closes: #1104299)
-- Sebastien Delafond <seb@debian.org> Mon, 28 Apr 2025 15:38:43 +0200
aptly (1.6.1+ds1-1) unstable; urgency=medium
[ Sébastien Delafond ]
* New upstream version 1.6.1
-- Sebastien Delafond <seb@debian.org> Mon, 24 Feb 2025 09:04:35 +0100
aptly (1.6.0+ds1-8) unstable; urgency=medium
[ Sébastien Delafond ]
* tests: disable riscv64-flaky EditRepo4Test
-- Sebastien Delafond <seb@debian.org> Fri, 21 Feb 2025 06:54:18 +0100
aptly (1.6.0+ds1-7) unstable; urgency=medium
[ Sébastien Delafond ]
* tests: clean way to disable single tests, disable s390-flaky CreateMirror35Test
-- Sebastien Delafond <seb@debian.org> Thu, 20 Feb 2025 07:26:41 +0100
aptly (1.6.0+ds1-6) unstable; urgency=medium
[ Sébastien Delafond ]
* tests: disable system-test's etcd tests as the corresponding fixture is also arch-specific
-- Sebastien Delafond <seb@debian.org> Wed, 19 Feb 2025 14:02:19 +0100
aptly (1.6.0+ds1-5) unstable; urgency=medium
[ Sébastien Delafond ]
* tests: do not use upstream's etcd installer
-- Sebastien Delafond <seb@debian.org> Wed, 19 Feb 2025 07:20:44 +0100
aptly (1.6.0+ds1-4) unstable; urgency=medium
[ Sébastien Delafond ]
* postrm: remove aptly-api user and home directory on purge
* tests: use extensive coverage from make's test and system-test
-- Sebastien Delafond <seb@debian.org> Tue, 18 Feb 2025 10:36:13 +0100
aptly (1.6.0+ds1-3) unstable; urgency=medium
[ Sébastien Delafond ]
* aptly.conf: fix s3 example
-- Sebastien Delafond <seb@debian.org> Mon, 13 Jan 2025 14:51:29 +0100
aptly (1.6.0+ds1-2) unstable; urgency=medium
[ Sébastien Delafond ]
* Document disabled swagger in default config file
* Rediff patches: swagger references in man page
-- Sebastien Delafond <seb@debian.org> Mon, 30 Dec 2024 11:11:07 +0100
aptly (1.6.0+ds1-1) unstable; urgency=medium
[ Sébastien Delafond ]
* Official 1.6 release
-- Sebastien Delafond <seb@debian.org> Fri, 27 Dec 2024 14:23:29 +0100
aptly (1.6.0+ds1~beta3-1) experimental; urgency=medium
[ Sébastien Delafond ]
* Latest upstream version
-- Sebastien Delafond <seb@debian.org> Wed, 11 Dec 2024 18:16:19 +0100
aptly (1.6.0+ds1~beta2-1) experimental; urgency=medium
[ Sébastien Delafond ]
* Latest upstream version
[ André Roth ]
* set systemd service file limit to 32768
-- Sebastien Delafond <seb@debian.org> Thu, 21 Nov 2024 16:08:05 +0100
aptly (1.6.0+ds1~beta1-1) experimental; urgency=medium
[ Sébastien Delafond ]
* Latest upstream version
* Disable "use new azure-sdk"
-- Sebastien Delafond <seb@debian.org> Sun, 17 Nov 2024 18:52:06 +0100
aptly (1.6.0+ds1~alpha5-1) experimental; urgency=medium
* Sort out dependencies on gpg*
* Reduce diff with upstream some more
-- Sebastien Delafond <seb@debian.org> Sun, 17 Nov 2024 15:59:23 +0100
aptly (1.6.0+ds1~alpha4-1) experimental; urgency=medium
[ Sébastien Delafond ]
* Remove build-dep on git
* Adjust systemd unit for aptly-api
-- Sebastien Delafond <seb@debian.org> Sun, 17 Nov 2024 14:51:49 +0100
aptly (1.6.0+ds1~alpha3-1) experimental; urgency=medium
[ Sébastien Delafond ]
* Latest upstream version
* Remove useless maintainer scripts
-- Sebastien Delafond <seb@debian.org> Sun, 17 Nov 2024 07:50:06 +0100
aptly (1.6.0+ds1~alpha2-1) experimental; urgency=medium
[ Sébastien Delafond ]
* Latest upstream version
-- Sebastien Delafond <seb@debian.org> Sat, 16 Nov 2024 14:42:10 +0100
aptly (1.6.0+ds1~alpha1-2) experimental; urgency=medium
[ Sébastien Delafond ]
* Add zsh completion
* aptly-api: revert arch:all, so mv_conffile does the right thing
-- Sebastien Delafond <seb@debian.org> Wed, 16 Oct 2024 09:10:05 +0200
aptly (1.6.0+ds1~alpha1-1) experimental; urgency=medium
[ Sébastien Delafond ]
* d/patches: remove old patch, and disable swagger support
* d/control: bump up Standards-Version
* Adjust packaging for 1.6.0~alpha1
-- Sebastien Delafond <seb@debian.org> Tue, 15 Oct 2024 14:40:57 +0200
aptly (1.5.0+ds1-2) unstable; urgency=medium
[ Shengjing Zhu ]
* Replace golang-gopkg-cheggaaa-pb.v1-dev with
golang-github-cheggaaa-pb-dev (Closes: #1050900)
-- Sebastien Delafond <seb@debian.org> Mon, 04 Sep 2023 08:49:36 +0200
aptly (1.5.0+ds1-1) unstable; urgency=medium
* Team upload.
* New upstream release (Closes: #1022721), including fix for "Order of
fields in Packages/Sources is unpredictable" (Closes: #907121).
-- Roland Mas <lolando@debian.org> Tue, 31 Jan 2023 14:47:04 +0100
aptly (1.4.0+ds1-7) unstable; urgency=medium
* Team upload.
* Add support for zstd compression (Closes: #1010465)
-- Anton Gladky <gladk@debian.org> Tue, 17 May 2022 22:42:29 +0200
aptly (1.4.0+ds1-6) unstable; urgency=medium
* Conflict on gpgv1 (Closes: #990821)
-- Sebastien Delafond <seb@debian.org> Thu, 04 Nov 2021 10:24:53 +0100
aptly (1.4.0+ds1-5) unstable; urgency=medium
* Conflict on gnupg1 (Closes: #990821)
-- Sebastien Delafond <seb@debian.org> Thu, 14 Oct 2021 18:43:04 +0200
aptly (1.4.0+ds1-4) unstable; urgency=medium
* Install correct bash completion snippet (Closes: #984979)
-- Sebastien Delafond <seb@debian.org> Thu, 11 Mar 2021 15:20:57 +0100
aptly (1.4.0+ds1-3) unstable; urgency=medium
* Fix s3 etag issue (Closes: #983877)
* Bump-up d/watch version
* Bump-up Standards-Version
-- Sebastien Delafond <seb@debian.org> Wed, 03 Mar 2021 10:50:51 +0100
aptly (1.4.0+ds1-2) unstable; urgency=medium
* Use pipeline from salsa-ci-team
* Allow reprotest failure
* Pass version from d/rules (Closes: #968585)
-- Sebastien Delafond <seb@debian.org> Fri, 21 Aug 2020 10:13:44 +0200
aptly (1.4.0+ds1-1) unstable; urgency=medium
* New upstream version 1.4.0+ds1
* Rediff patches
* Depend on gnupg 2
-- Sebastien Delafond <seb@debian.org> Sun, 22 Dec 2019 15:16:25 +0100
aptly (1.3.0+ds1-4) unstable; urgency=medium
[ Debian Janitor ]
* Rename obsolete path debian/tests/control.autodep8 to debian/tests/control.
* Use secure URI in Homepage field.
* Bump debhelper from old 11 to 12.
* Set debhelper-compat version in Build-Depends.
[ Sébastien Delafond ]
* Bump up Standards-Version
-- Sebastien Delafond <seb@debian.org> Sun, 22 Dec 2019 14:10:19 +0100
aptly (1.3.0+ds1-3) unstable; urgency=medium
* Build-Depend on golang-golang-x-tools-dev instead of golang-go.tools (Closes: #945884)
* Lintian fix
-- Sebastien Delafond <seb@debian.org> Sat, 21 Dec 2019 10:29:09 +0100
aptly (1.3.0+ds1-2.3) unstable; urgency=medium
* Non-maintainer upload.
* Remove myself from uploaders.
-- Alexandre Viau <aviau@debian.org> Sun, 15 Sep 2019 19:27:47 -0400
aptly (1.3.0+ds1-2.2) unstable; urgency=medium
* Non-maintainer upload.
* Add patch to fix DB backwards compatibility (Closes: #911924)
* Fix struct field tag typo
* Update debian/NEWS about DB compatibility
-- Shengjing Zhu <zhsj@debian.org> Tue, 16 Apr 2019 00:18:23 +0800
aptly (1.3.0+ds1-2.1) unstable; urgency=medium
[ Shengjing Zhu ]
* Non-maintainer upload.
* Add patch to fix UUID struct field not encoded in msgpack (Closes: #923866)
[ Tobias Frost ]
* Prepare upload.
-- Tobias Frost <tobi@debian.org> Fri, 05 Apr 2019 17:19:14 +0200
aptly (1.3.0+ds1-2) unstable; urgency=medium
* Add NEWS to warn about database compatibility.
-- Alexandre Viau <aviau@debian.org> Fri, 26 Oct 2018 13:22:38 -0400
aptly (1.3.0+ds1-1) unstable; urgency=medium
[ Ondřej Nový ]
* d/changelog: Remove trailing whitespaces
[ Alexandre Viau ]
* d/watch: Append +ds suffix.
* d/copyright: ignore vendor/*. (Closes: #902128)
* d/copyright: remove vendor/* sections.
* d/copyright: MIT -> Expat.
* d/control: add vendor/* build dependencies.
* Patch: Use Debian's uuid package.
* Patch: Use Debian's lzma package.
* d/rules: remove trailing whitespace.
-- Alexandre Viau <aviau@debian.org> Mon, 15 Oct 2018 11:54:03 -0400
aptly (1.3.0-6) unstable; urgency=medium
* Combine autodep8 and autopkgtest.
-- Alexandre Viau <aviau@debian.org> Fri, 29 Jun 2018 18:11:41 -0400
aptly (1.3.0-5) unstable; urgency=medium
* Fix syntax-error-in-dep5-copyright.
* Fix unnecessary-testsuite-autopkgtest-field.
* Fix broken autopkgtest.
* Depend on gpgv1.
-- Alexandre Viau <aviau@debian.org> Tue, 26 Jun 2018 23:01:36 -0400
aptly (1.3.0-4) unstable; urgency=medium
* Document #902128 in debian/copyright
* Point debian/watch to new git repo on GitHub
* Add simple autopkgtest
* Add debian/.gitlab-ci.yml
* Depend on gnupg1 (Closes: #902419)
-- Sebastien Delafond <seb@debian.org> Tue, 26 Jun 2018 14:24:34 +0200
aptly (1.3.0-3) unstable; urgency=medium
* Create aptly-api package. (Closes: #902032)
-- Alexandre Viau <aviau@debian.org> Fri, 22 Jun 2018 13:51:50 -0400
aptly (1.3.0-2) unstable; urgency=medium
* Team upload, many thanks to Alexandre Viau for this work
* Switch to dh-golang. (Closes: #902038)
* Fix vcs-field-not-canonical.
* Fix insecure-copyright-format-uri.
-- Sebastien Delafond <seb@debian.org> Fri, 22 Jun 2018 13:53:11 +0200
aptly (1.3.0-1) unstable; urgency=medium
* Fix copyright
* New upstream version 1.3.0
-- Sebastien Delafond <seb@debian.org> Thu, 21 Jun 2018 15:14:57 +0200
aptly (1.2.0-4) unstable; urgency=medium
* Enable Built-Using in debian/control (thanks M. Staperberg)
-- Sebastien Delafond <seb@debian.org> Tue, 06 Mar 2018 10:10:42 +0100
aptly (1.2.0-3) unstable; urgency=medium
* Switch to DEP 14
* Update Vcs-* to point to salsa.d.o
-- Sebastien Delafond <seb@debian.org> Fri, 16 Feb 2018 17:02:25 +0100
aptly (1.2.0-2) unstable; urgency=medium
* Team upload.
* Build-Depend on golang-any, not golang
* Set XS-Go-Import-Path
-- Michael Stapelberg <stapelberg@debian.org> Sat, 10 Feb 2018 18:41:37 +0100
aptly (1.2.0-1) unstable; urgency=medium
* New upstream version 1.2.0
-- Sebastien Delafond <seb@debian.org> Wed, 03 Jan 2018 13:43:30 +0100
aptly (1.1.1-2) unstable; urgency=medium
* Support both uncompressed control.tar, and xz-compressed
control.tar.xz. Thanks to Boyuan Yang for the patch (Closes: 879718)
-- Sebastien Delafond <seb@debian.org> Thu, 02 Nov 2017 13:23:29 +0100
aptly (1.1.1-1) unstable; urgency=medium
* New upstream version 1.1.1
* Use bash-completion snippet from upstream
* Bump up Standards-Version
* Update debian/copyright
-- Sebastien Delafond <seb@debian.org> Thu, 02 Nov 2017 09:03:23 +0100
aptly (1.0.1-1) unstable; urgency=medium
* Imported Upstream version 1.0.1
* Update debian/copyright
* Adapt debian/rules
* Update Vcs-Git
* Bump-up Standards-Version
-- Sebastien Delafond <seb@debian.org> Tue, 18 Jul 2017 11:53:29 +0200
aptly (0.9.7-1) unstable; urgency=medium
* Imported new upstream version 0.9.7
* Add new licenses and copyrights info
-- Sebastien Delafond <seb@debian.org> Tue, 24 May 2016 09:17:08 +0200
aptly (0.9.6-1) unstable; urgency=medium
* Import new upstream version 0.9.6
* Add dependency on xz-utils
* Licenses and copyrights
-- Sebastien Delafond <seb@debian.org> Wed, 10 Feb 2016 18:28:51 +0100
aptly (0.9.5-2) unstable; urgency=medium
* Remove empty component in GOPATH (Closes: #793838)
[ Sebastien Delafond ]
-- Sebastien Delafond <seb@debian.org> Sun, 16 Aug 2015 18:42:22 +0200
aptly (0.9.5-1) unstable; urgency=medium
* Imported Upstream version 0.9.5
* Up-to-date bash completion
-- Sebastien Delafond <seb@debian.org> Fri, 15 May 2015 10:46:51 +0200
aptly (0.9.1-1) unstable; urgency=medium
* Imported Upstream version 0.9.1
* Document licenses and copyrights for new dependencies
* Suggest graphviz
* Proper name for bash-completion file
* Bump-up Standards-Version
-- Sebastien Delafond <seb@debian.org> Sat, 02 May 2015 12:57:16 +0200
aptly (0.8-3) unstable; urgency=medium
* Correct Vcs-Browser entry (Closes: #764622)
* Correct dependency from "gpg" to "gnupg" (Closes: #764619)
-- Sebastien Delafond <seb@debian.org> Thu, 09 Oct 2014 18:41:38 +0200
aptly (0.8-2) unstable; urgency=medium
* Add missing dependencies on bzip2, gpg and gpgv
-- Sebastien Delafond <seb@debian.org> Thu, 09 Oct 2014 11:15:21 +0200
aptly (0.8-1) unstable; urgency=medium
* Imported Upstream version 0.8
* Document new copyrights and licenses
* Add bash completion snippet
* Reference full paths in debian/copyright, and remove unused
references, so lintian does not complain
-- Sebastien Delafond <seb@debian.org> Sat, 04 Oct 2014 17:46:24 +0200
aptly (0.7.1-1) unstable; urgency=medium
* Imported Upstream version 0.7.1
* Add copyright information for new libraries
-- Sebastien Delafond <seb@debian.org> Wed, 10 Sep 2014 10:03:27 +0200
aptly (0.5-5) unstable; urgency=medium
* Turn off verbose mode
* Correct Vcs-* information
* gocov is licensed MIT, not BSD-3
* Add missing MIT license text
-- Sebastien Delafond <seb@debian.org> Tue, 09 Sep 2014 09:20:40 +0200
aptly (0.5-4) unstable; urgency=medium
* Collect licenses by going over files one by one
* Use packaged golang-go.tools
-- Sebastien Delafond <seb@debian.org> Thu, 10 Jul 2014 00:49:09 +0200
aptly (0.5-3) unstable; urgency=low
* Licensing:
- goleveldb is BSD-2, not BSD-3
- _vendor/src/code.google.com/p/gographviz/scanner/scanner is BSD-3,
copyright 2009 The Go Authors
-- Sebastien Delafond <seb@debian.org> Wed, 28 May 2014 10:04:01 +0200
aptly (0.5-2) unstable; urgency=low
* Go interpreter not needed at runtime
-- Sebastien Delafond <seb@debian.org> Fri, 02 May 2014 12:04:02 +0200
aptly (0.5-1) unstable; urgency=low
* Initial release (Closes: #746343)
-- Sebastien Delafond <seb@debian.org> Tue, 29 Apr 2014 15:47:31 +0200
aptly (1.6.2) stable; urgency=medium
* doc: add swagger doc for /api/gpg/key (https://github.com/aptly-dev/aptly/pull/1456)
* bash-completion: include global options in aptly command completions (https://github.com/aptly-dev/aptly/pull/1452)
* Bump golang.org/x/net from 0.33.0 to 0.38.0 (https://github.com/aptly-dev/aptly/pull/1443)
* Bump golang.org/x/crypto from 0.31.0 to 0.35.0 (https://github.com/aptly-dev/aptly/pull/1441)
* Remove corrupt package references in `db recover` (https://github.com/aptly-dev/aptly/pull/1445)
* Fix upload of unchanged packages in S3 (https://github.com/aptly-dev/aptly/pull/1440)
* use go 1.24 (https://github.com/aptly-dev/aptly/pull/1439)
-- André Roth <neolynx@gmail.com> Mon, 09 Jun 2025 13:45:15 +0200
aptly (1.6.1) stable; urgency=medium
* update golang-github-syndtr-goleveldb-dev dependency (v1.0.1-0.20220721030215-126854af5e6d) to fix segfault on arm64
(bug in golang-github-golang-snappy-dev)
* allow snapshotting empty mirrors again (regression)
* debian compliance: add postrm (note: `apt purge aptly-api` will remove all data in ~aptly-api/)
* update other dependencies (x/net 0.33.0, gin-gonic/gin 1.9.1)
-- André Roth <neolynx@gmail.com> Sat, 15 Feb 2025 13:03:16 +0100
aptly (1.6.0) stable; urgency=medium
* support reading filters from file or stdin
* fix mirroring source packages
* support yaml config per default
* provide swagger API documentation
* provide API for querying aptly storage usage
* provide snapshot pull via API
* support creating repos from snapshots
* fix mirroring flat remote repos
* support skeleton files for publishing
* use new azure sdk
* support updating the components of a published repo
* support publishing multiple distributions (-multi-dist)
* support etcd database
* allow slash (/) in distribution names
* support for storing the "local" pool on Azure
* provide copy package API
* fix publish concurrency and improve performance
* improved mirroring
* fix download throttling
* fix resuming package downloads
* fix ignoring signatures
* fix packages dependency resolution (Virtual Packages, version numbers in Provides)
* improved S3 support and performance
* fix race condition with goleveldb
* use go 1.22
-- André Roth <neolynx@gmail.com> Tue, 24 Dec 2024 17:44:35 +0100
+16 -7
View File
@@ -1,7 +1,7 @@
Source: aptly
Section: utils
Priority: optional
Maintainer: Sebastien Delafond <seb@debian.org>
Maintainer: André Roth <neolynx@gmail.com>
Build-Depends: bash-completion,
debhelper-compat (= 13),
dh-golang,
@@ -77,18 +77,20 @@ Build-Depends: bash-completion,
golang-go.uber-zap-dev,
golang-etcd-server-dev (>= 3.5.15-7),
golang-gopkg-yaml.v3-dev,
Standards-Version: 4.7.4
git
Standards-Version: 4.7.0
Homepage: https://www.aptly.info
Vcs-Git: https://salsa.debian.org/debian/aptly.git
Vcs-Browser: https://salsa.debian.org/debian/aptly
Vcs-Git: https://github.com/aptly-dev/aptly.git
Vcs-Browser: https://github.com/aptly-dev/aptly
XS-Go-Import-Path: github.com/aptly-dev/aptly
Testsuite: autopkgtest-pkg-go
Package: aptly
Architecture: any
Depends: ${misc:Depends}, ${shlibs:Depends}, bzip2, xz-utils, gpgv, gpg
Static-Built-Using: ${misc:Static-Built-Using}
Conflicts: gnupg1, gpgv1
Suggests: graphviz
Conflicts: gnupg1, gpgv1
Built-Using: ${misc:Static-Built-Using}, ${misc:Built-Using}
Description: Swiss army knife for Debian repository management - main package
It offers several features making it easy to manage Debian package
repositories:
@@ -105,7 +107,7 @@ Description: Swiss army knife for Debian repository management - main package
This is the main package, it contains the aptly command-line utility.
Package: aptly-api
Architecture: all
Architecture: any
Depends: ${misc:Depends}, aptly
Description: Swiss army knife for Debian repository management - API
It offers several features making it easy to manage Debian package
@@ -121,3 +123,10 @@ Description: Swiss army knife for Debian repository management - API
- merge two or more snapshots into one
.
This package contains the aptly-api service.
Package: aptly-dbg
Architecture: any
Depends: ${misc:Depends}
Built-Using: ${misc:Static-Built-Using}, ${misc:Built-Using}
Description: Debian repository management tool (debug files)
Debug symbols for aptly
-3
View File
@@ -1,3 +0,0 @@
[DEFAULT]
debian-branch = debian/master
upstream-branch = upstream/latest
-1
View File
@@ -1 +0,0 @@
usr/bin/files
-123
View File
@@ -1,123 +0,0 @@
From: =?utf-8?q?Andr=C3=A9_Roth?= <neolynx@gmail.com>
Date: Tue, 15 Oct 2024 12:09:33 +0200
Subject: Disable swagger
This can be enabled once the following modules make it into Debian:
- github.com/swaggo/files v1.0.1
- github.com/swaggo/gin-swagger v1.6.0
- github.com/swaggo/swag v1.16.3
Forwarded: not-needed
---
api/router.go | 22 +++++++++++-----------
docs/index.go | 10 ----------
docs/index.go.disabled | 10 ++++++++++
man/aptly.1 | 3 ++-
man/aptly.1.ronn.tmpl | 3 ++-
5 files changed, 25 insertions(+), 23 deletions(-)
delete mode 100644 docs/index.go
create mode 100644 docs/index.go.disabled
diff --git a/api/router.go b/api/router.go
index 3cd7d42..cf53cd2 100644
--- a/api/router.go
+++ b/api/router.go
@@ -11,9 +11,9 @@
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/rs/zerolog/log"
- "github.com/aptly-dev/aptly/docs"
- swaggerFiles "github.com/swaggo/files"
- ginSwagger "github.com/swaggo/gin-swagger"
+ // _ "github.com/aptly-dev/aptly/docs" // import docs
+ // swaggerFiles "github.com/swaggo/files"
+ // ginSwagger "github.com/swaggo/gin-swagger"
)
var context *ctx.AptlyContext
@@ -63,14 +63,14 @@ func Router(c *ctx.AptlyContext) http.Handler {
router.Use(gin.Recovery(), gin.ErrorLogger())
- if c.Config().EnableSwaggerEndpoint {
- router.GET("docs.html", func(c *gin.Context) {
- c.Data(http.StatusOK, "text/html; charset=utf-8", docs.DocsHTML)
- })
- router.Use(redirectSwagger)
- url := ginSwagger.URL("/docs/doc.json")
- router.GET("/docs/*any", ginSwagger.WrapHandler(swaggerFiles.Handler, url))
- }
+ // if c.Config().EnableSwaggerEndpoint {
+ // router.GET("docs.html", func(c *gin.Context) {
+ // c.Data(http.StatusOK, "text/html; charset=utf-8", docs.DocsHTML)
+ // })
+ // router.Use(redirectSwagger)
+ // url := ginSwagger.URL("/docs/doc.json")
+ // router.GET("/docs/*any", ginSwagger.WrapHandler(swaggerFiles.Handler, url))
+ // }
if c.Config().EnableMetricsEndpoint {
MetricsCollectorRegistrar.Register(router)
diff --git a/docs/index.go b/docs/index.go
deleted file mode 100644
index ca4c914..0000000
--- a/docs/index.go
+++ /dev/null
@@ -1,10 +0,0 @@
-package docs
-
-import (
- _ "embed" // embed html below
-
- _ "github.com/swaggo/swag" // make sure swag is in go.mod
-)
-
-//go:embed docs.html
-var DocsHTML []byte
diff --git a/docs/index.go.disabled b/docs/index.go.disabled
new file mode 100644
index 0000000..ca4c914
--- /dev/null
+++ b/docs/index.go.disabled
@@ -0,0 +1,10 @@
+package docs
+
+import (
+ _ "embed" // embed html below
+
+ _ "github.com/swaggo/swag" // make sure swag is in go.mod
+)
+
+//go:embed docs.html
+var DocsHTML []byte
diff --git a/man/aptly.1 b/man/aptly.1
index bd6ad22..11ed990 100644
--- a/man/aptly.1
+++ b/man/aptly.1
@@ -111,8 +111,9 @@ The legacy json configuration is still supported (and also supports comments):
// Enable metrics for Prometheus client
"enableMetricsEndpoint": false,
+ // Not implemented in this version\.
// Enable API documentation on /docs
- "enableSwaggerEndpoint": false,
+ //"enableSwaggerEndpoint": false,
// OBSOLETE: use via url param ?_async=true
"AsyncAPI": false,
diff --git a/man/aptly.1.ronn.tmpl b/man/aptly.1.ronn.tmpl
index 203cc7f..ed2c87c 100644
--- a/man/aptly.1.ronn.tmpl
+++ b/man/aptly.1.ronn.tmpl
@@ -100,8 +100,9 @@ The legacy json configuration is still supported (and also supports comments):
// Enable metrics for Prometheus client
"enableMetricsEndpoint": false,
+ // Not implemented in this version.
// Enable API documentation on /docs
- "enableSwaggerEndpoint": false,
+ //"enableSwaggerEndpoint": false,
// OBSOLETE: use via url param ?_async=true
"AsyncAPI": false,
-880
View File
@@ -1,880 +0,0 @@
From: =?utf-8?q?Andr=C3=A9_Roth?= <neolynx@gmail.com>
Date: Sun, 17 Nov 2024 17:58:04 +0100
Subject: Disable new azure-sdk
This reverts commit e2cbd637b82a153a6756f2af0519e8fe769ee9ab.
We can enable this once the golang-github-azure-azure-sdk-for-go-dev
packages are upgraded in Debian:
- github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0
- github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0
- github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1
Forwarded: not-needed
---
azure/azure.go | 89 +++++++++++++++-----------------
azure/package_pool.go | 45 ++++++++--------
azure/package_pool_test.go | 8 ++-
azure/public.go | 125 +++++++++++++++++++++------------------------
azure/public_test.go | 49 +++++++++---------
context/context.go | 1 +
deb/list.go | 1 +
go.mod | 8 +--
go.sum | 52 +++++++++++--------
9 files changed, 185 insertions(+), 193 deletions(-)
diff --git a/azure/azure.go b/azure/azure.go
index 3f12678..b313f90 100644
--- a/azure/azure.go
+++ b/azure/azure.go
@@ -5,35 +5,28 @@
import (
"context"
"encoding/hex"
- "errors"
"fmt"
"io"
- "os"
+ "net/url"
"path/filepath"
"time"
- "github.com/Azure/azure-sdk-for-go/sdk/azcore"
- "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
- "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob"
+ "github.com/Azure/azure-storage-blob-go/azblob"
"github.com/aptly-dev/aptly/aptly"
)
func isBlobNotFound(err error) bool {
- var respErr *azcore.ResponseError
- if errors.As(err, &respErr) {
- return respErr.StatusCode == 404 // BlobNotFound
- }
- return false
+ storageError, ok := err.(azblob.StorageError)
+ return ok && storageError.ServiceCode() == azblob.ServiceCodeBlobNotFound
}
type azContext struct {
- client *azblob.Client
- container string
+ container azblob.ContainerURL
prefix string
}
func newAzContext(accountName, accountKey, container, prefix, endpoint string) (*azContext, error) {
- cred, err := azblob.NewSharedKeyCredential(accountName, accountKey)
+ credential, err := azblob.NewSharedKeyCredential(accountName, accountKey)
if err != nil {
return nil, err
}
@@ -42,14 +35,15 @@ func newAzContext(accountName, accountKey, container, prefix, endpoint string) (
endpoint = fmt.Sprintf("https://%s.blob.core.windows.net", accountName)
}
- serviceClient, err := azblob.NewClientWithSharedKeyCredential(endpoint, cred, nil)
+ url, err := url.Parse(fmt.Sprintf("%s/%s", endpoint, container))
if err != nil {
return nil, err
}
+ containerURL := azblob.NewContainerURL(*url, azblob.NewPipeline(credential, azblob.PipelineOptions{}))
+
result := &azContext{
- client: serviceClient,
- container: container,
+ container: containerURL,
prefix: prefix,
}
@@ -60,6 +54,10 @@ func (az *azContext) blobPath(path string) string {
return filepath.Join(az.prefix, path)
}
+func (az *azContext) blobURL(path string) azblob.BlobURL {
+ return az.container.NewBlobURL(az.blobPath(path))
+}
+
func (az *azContext) internalFilelist(prefix string, progress aptly.Progress) (paths []string, md5s []string, err error) {
const delimiter = "/"
paths = make([]string, 0, 1024)
@@ -69,33 +67,27 @@ func (az *azContext) internalFilelist(prefix string, progress aptly.Progress) (p
prefix += delimiter
}
- ctx := context.Background()
- maxResults := int32(1)
- pager := az.client.NewListBlobsFlatPager(az.container, &azblob.ListBlobsFlatOptions{
- Prefix: &prefix,
- MaxResults: &maxResults,
- Include: azblob.ListBlobsInclude{Metadata: true},
- })
-
- // Iterate over each page
- for pager.More() {
- page, err := pager.NextPage(ctx)
+ for marker := (azblob.Marker{}); marker.NotDone(); {
+ listBlob, err := az.container.ListBlobsFlatSegment(
+ context.Background(), marker, azblob.ListBlobsSegmentOptions{
+ Prefix: prefix,
+ MaxResults: 1,
+ Details: azblob.BlobListingDetails{Metadata: true}})
if err != nil {
return nil, nil, fmt.Errorf("error listing under prefix %s in %s: %s", prefix, az, err)
}
- for _, blob := range page.Segment.BlobItems {
- if prefix == "" {
- paths = append(paths, *blob.Name)
- } else {
- name := *blob.Name
- paths = append(paths, name[len(prefix):])
- }
- b := *blob
- md5 := b.Properties.ContentMD5
- md5s = append(md5s, fmt.Sprintf("%x", md5))
+ marker = listBlob.NextMarker
+ for _, blob := range listBlob.Segment.BlobItems {
+ if prefix == "" {
+ paths = append(paths, blob.Name)
+ } else {
+ paths = append(paths, blob.Name[len(prefix):])
+ }
+ md5s = append(md5s, fmt.Sprintf("%x", blob.Properties.ContentMD5))
}
+
if progress != nil {
time.Sleep(time.Duration(500) * time.Millisecond)
progress.AddBar(1)
@@ -105,27 +97,28 @@ func (az *azContext) internalFilelist(prefix string, progress aptly.Progress) (p
return paths, md5s, nil
}
-func (az *azContext) putFile(blobName string, source io.Reader, sourceMD5 string) error {
- uploadOptions := &azblob.UploadFileOptions{
- BlockSize: 4 * 1024 * 1024,
- Concurrency: 8,
+func (az *azContext) putFile(blob azblob.BlobURL, source io.Reader, sourceMD5 string) error {
+ uploadOptions := azblob.UploadStreamToBlockBlobOptions{
+ BufferSize: 4 * 1024 * 1024,
+ MaxBuffers: 8,
}
- path := az.blobPath(blobName)
if len(sourceMD5) > 0 {
decodedMD5, err := hex.DecodeString(sourceMD5)
if err != nil {
return err
}
- uploadOptions.HTTPHeaders = &blob.HTTPHeaders{
- BlobContentMD5: decodedMD5,
+ uploadOptions.BlobHTTPHeaders = azblob.BlobHTTPHeaders{
+ ContentMD5: decodedMD5,
}
}
- var err error
- if file, ok := source.(*os.File); ok {
- _, err = az.client.UploadFile(context.TODO(), az.container, path, file, uploadOptions)
- }
+ _, err := azblob.UploadStreamToBlockBlob(
+ context.Background(),
+ source,
+ blob.ToBlockBlobURL(),
+ uploadOptions,
+ )
return err
}
diff --git a/azure/package_pool.go b/azure/package_pool.go
index 97be8e6..6d7af1a 100644
--- a/azure/package_pool.go
+++ b/azure/package_pool.go
@@ -5,6 +5,7 @@
"os"
"path/filepath"
+ "github.com/Azure/azure-storage-blob-go/azblob"
"github.com/aptly-dev/aptly/aptly"
"github.com/aptly-dev/aptly/utils"
"github.com/pkg/errors"
@@ -40,7 +41,10 @@ func (pool *PackagePool) buildPoolPath(filename string, checksums *utils.Checksu
return filepath.Join(hash[0:2], hash[2:4], hash[4:32]+"_"+filename)
}
-func (pool *PackagePool) ensureChecksums(poolPath string, checksumStorage aptly.ChecksumStorage) (*utils.ChecksumInfo, error) {
+func (pool *PackagePool) ensureChecksums(
+ poolPath string,
+ checksumStorage aptly.ChecksumStorage,
+) (*utils.ChecksumInfo, error) {
targetChecksums, err := checksumStorage.Get(poolPath)
if err != nil {
return nil, err
@@ -48,7 +52,8 @@ func (pool *PackagePool) ensureChecksums(poolPath string, checksumStorage aptly.
if targetChecksums == nil {
// we don't have checksums stored yet for this file
- download, err := pool.az.client.DownloadStream(context.Background(), pool.az.container, poolPath, nil)
+ blob := pool.az.blobURL(poolPath)
+ download, err := blob.Download(context.Background(), 0, 0, azblob.BlobAccessConditions{}, false, azblob.ClientProvidedKeyOptions{})
if err != nil {
if isBlobNotFound(err) {
return nil, nil
@@ -58,7 +63,7 @@ func (pool *PackagePool) ensureChecksums(poolPath string, checksumStorage aptly.
}
targetChecksums = &utils.ChecksumInfo{}
- *targetChecksums, err = utils.ChecksumsForReader(download.Body)
+ *targetChecksums, err = utils.ChecksumsForReader(download.Body(azblob.RetryReaderOptions{}))
if err != nil {
return nil, errors.Wrapf(err, "error checksumming blob at %s", poolPath)
}
@@ -87,49 +92,46 @@ func (pool *PackagePool) LegacyPath(_ string, _ *utils.ChecksumInfo) (string, er
}
func (pool *PackagePool) Size(path string) (int64, error) {
- serviceClient := pool.az.client.ServiceClient()
- containerClient := serviceClient.NewContainerClient(pool.az.container)
- blobClient := containerClient.NewBlobClient(path)
-
- props, err := blobClient.GetProperties(context.TODO(), nil)
+ blob := pool.az.blobURL(path)
+ props, err := blob.GetProperties(context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
if err != nil {
return 0, errors.Wrapf(err, "error examining %s from %s", path, pool)
}
- return *props.ContentLength, nil
+ return props.ContentLength(), nil
}
func (pool *PackagePool) Open(path string) (aptly.ReadSeekerCloser, error) {
+ blob := pool.az.blobURL(path)
+
temp, err := os.CreateTemp("", "blob-download")
if err != nil {
- return nil, errors.Wrapf(err, "error creating tempfile for %s", path)
+ return nil, errors.Wrap(err, "error creating temporary file for blob download")
}
+
defer func () { _ = os.Remove(temp.Name()) }()
- _, err = pool.az.client.DownloadFile(context.TODO(), pool.az.container, path, temp, nil)
+ err = azblob.DownloadBlobToFile(context.Background(), blob, 0, 0, temp, azblob.DownloadFromBlobOptions{})
if err != nil {
- return nil, errors.Wrapf(err, "error downloading blob %s", path)
+ return nil, errors.Wrapf(err, "error downloading blob at %s", path)
}
return temp, nil
}
func (pool *PackagePool) Remove(path string) (int64, error) {
- serviceClient := pool.az.client.ServiceClient()
- containerClient := serviceClient.NewContainerClient(pool.az.container)
- blobClient := containerClient.NewBlobClient(path)
-
- props, err := blobClient.GetProperties(context.TODO(), nil)
+ blob := pool.az.blobURL(path)
+ props, err := blob.GetProperties(context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
if err != nil {
- return 0, errors.Wrapf(err, "error examining %s from %s", path, pool)
+ return 0, errors.Wrapf(err, "error getting props of %s from %s", path, pool)
}
- _, err = pool.az.client.DeleteBlob(context.Background(), pool.az.container, path, nil)
+ _, err = blob.Delete(context.Background(), azblob.DeleteSnapshotsOptionNone, azblob.BlobAccessConditions{})
if err != nil {
return 0, errors.Wrapf(err, "error deleting %s from %s", path, pool)
}
- return *props.ContentLength, nil
+ return props.ContentLength(), nil
}
func (pool *PackagePool) Import(srcPath, basename string, checksums *utils.ChecksumInfo, _ bool, checksumStorage aptly.ChecksumStorage) (string, error) {
@@ -143,6 +145,7 @@ func (pool *PackagePool) Import(srcPath, basename string, checksums *utils.Check
}
path := pool.buildPoolPath(basename, checksums)
+ blob := pool.az.blobURL(path)
targetChecksums, err := pool.ensureChecksums(path, checksumStorage)
if err != nil {
return "", err
@@ -158,7 +161,7 @@ func (pool *PackagePool) Import(srcPath, basename string, checksums *utils.Check
}
defer func() { _ = source.Close() }()
- err = pool.az.putFile(path, source, checksums.MD5)
+ err = pool.az.putFile(blob, source, checksums.MD5)
if err != nil {
return "", err
}
diff --git a/azure/package_pool_test.go b/azure/package_pool_test.go
index ef562cb..6b1341d 100644
--- a/azure/package_pool_test.go
+++ b/azure/package_pool_test.go
@@ -7,7 +7,7 @@
"path/filepath"
"runtime"
- "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
+ "github.com/Azure/azure-storage-blob-go/azblob"
"github.com/aptly-dev/aptly/aptly"
"github.com/aptly-dev/aptly/files"
"github.com/aptly-dev/aptly/utils"
@@ -50,10 +50,8 @@ func (s *PackagePoolSuite) SetUpTest(c *C) {
s.pool, err = NewPackagePool(s.accountName, s.accountKey, container, "", s.endpoint)
c.Assert(err, IsNil)
- publicAccessType := azblob.PublicAccessTypeContainer
- _, err = s.pool.az.client.CreateContainer(context.TODO(), s.pool.az.container, &azblob.CreateContainerOptions{
- Access: &publicAccessType,
- })
+ cnt := s.pool.az.container
+ _, err = cnt.Create(context.Background(), azblob.Metadata{}, azblob.PublicAccessContainer)
c.Assert(err, IsNil)
s.prefixedPool, err = NewPackagePool(s.accountName, s.accountKey, container, prefix, s.endpoint)
diff --git a/azure/public.go b/azure/public.go
index 6775e14..efd2e7a 100644
--- a/azure/public.go
+++ b/azure/public.go
@@ -3,22 +3,21 @@
import (
"context"
"fmt"
+ "net/http"
"os"
"path/filepath"
"time"
- "github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
- "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob"
- "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/lease"
+ "github.com/Azure/azure-storage-blob-go/azblob"
"github.com/aptly-dev/aptly/aptly"
"github.com/aptly-dev/aptly/utils"
- "github.com/google/uuid"
"github.com/pkg/errors"
)
// PublishedStorage abstract file system with published files (actually hosted on Azure)
type PublishedStorage struct {
// FIXME: unused ???? prefix string
+ container azblob.ContainerURL
az *azContext
pathCache map[string]map[string]string
}
@@ -67,7 +66,7 @@ func (storage *PublishedStorage) PutFile(path string, sourceFilename string) err
}
defer func() { _ = source.Close() }()
- err = storage.az.putFile(path, source, sourceMD5)
+ err = storage.az.putFile(storage.az.blobURL(path), source, sourceMD5)
if err != nil {
err = errors.Wrap(err, fmt.Sprintf("error uploading %s to %s", sourceFilename, storage))
}
@@ -77,15 +76,14 @@ func (storage *PublishedStorage) PutFile(path string, sourceFilename string) err
// RemoveDirs removes directory structure under public path
func (storage *PublishedStorage) RemoveDirs(path string, _ aptly.Progress) error {
- path = storage.az.blobPath(path)
filelist, err := storage.Filelist(path)
if err != nil {
return err
}
for _, filename := range filelist {
- blob := filepath.Join(path, filename)
- _, err := storage.az.client.DeleteBlob(context.Background(), storage.az.container, blob, nil)
+ blob := storage.az.blobURL(filepath.Join(path, filename))
+ _, err := blob.Delete(context.Background(), azblob.DeleteSnapshotsOptionNone, azblob.BlobAccessConditions{})
if err != nil {
return fmt.Errorf("error deleting path %s from %s: %s", filename, storage, err)
}
@@ -96,8 +94,8 @@ func (storage *PublishedStorage) RemoveDirs(path string, _ aptly.Progress) error
// Remove removes single file under public path
func (storage *PublishedStorage) Remove(path string) error {
- path = storage.az.blobPath(path)
- _, err := storage.az.client.DeleteBlob(context.Background(), storage.az.container, path, nil)
+ blob := storage.az.blobURL(path)
+ _, err := blob.Delete(context.Background(), azblob.DeleteSnapshotsOptionNone, azblob.BlobAccessConditions{})
if err != nil {
err = errors.Wrap(err, fmt.Sprintf("error deleting %s from %s: %s", path, storage, err))
}
@@ -116,8 +114,9 @@ func (storage *PublishedStorage) LinkFromPool(publishedPrefix, publishedRelPath,
sourcePath string, sourceChecksums utils.ChecksumInfo, force bool) error {
relFilePath := filepath.Join(publishedRelPath, fileName)
- prefixRelFilePath := filepath.Join(publishedPrefix, relFilePath)
- poolPath := storage.az.blobPath(prefixRelFilePath)
+ // prefixRelFilePath := filepath.Join(publishedPrefix, relFilePath)
+ // FIXME: check how to integrate publishedPrefix:
+ poolPath := storage.az.blobPath(fileName)
if storage.pathCache == nil {
storage.pathCache = make(map[string]map[string]string)
@@ -160,7 +159,7 @@ func (storage *PublishedStorage) LinkFromPool(publishedPrefix, publishedRelPath,
}
defer func() { _ = source.Close() }()
- err = storage.az.putFile(relFilePath, source, sourceMD5)
+ err = storage.az.putFile(storage.az.blobURL(relFilePath), source, sourceMD5)
if err == nil {
pathCache[relFilePath] = sourceMD5
} else {
@@ -177,60 +176,59 @@ func (storage *PublishedStorage) Filelist(prefix string) ([]string, error) {
}
// Internal copy or move implementation
-func (storage *PublishedStorage) internalCopyOrMoveBlob(src, dst string, metadata map[string]*string, move bool) error {
+func (storage *PublishedStorage) internalCopyOrMoveBlob(src, dst string, metadata azblob.Metadata, move bool) error {
const leaseDuration = 30
- leaseID := uuid.NewString()
- serviceClient := storage.az.client.ServiceClient()
- containerClient := serviceClient.NewContainerClient(storage.az.container)
- srcBlobClient := containerClient.NewBlobClient(src)
- blobLeaseClient, err := lease.NewBlobClient(srcBlobClient, &lease.BlobClientOptions{LeaseID: to.Ptr(leaseID)})
- if err != nil {
- return fmt.Errorf("error acquiring lease on source blob %s", src)
- }
-
- _, err = blobLeaseClient.AcquireLease(context.Background(), leaseDuration, nil)
- if err != nil {
- return fmt.Errorf("error acquiring lease on source blob %s", src)
+ dstBlobURL := storage.az.blobURL(dst)
+ srcBlobURL := storage.az.blobURL(src)
+ leaseResp, err := srcBlobURL.AcquireLease(context.Background(), "", leaseDuration, azblob.ModifiedAccessConditions{})
+ if err != nil || leaseResp.StatusCode() != http.StatusCreated {
+ return fmt.Errorf("error acquiring lease on source blob %s", srcBlobURL)
}
defer func() {
- _, _ = blobLeaseClient.BreakLease(context.Background(), &lease.BlobBreakOptions{BreakPeriod: to.Ptr(int32(60))})
+ _, _ = srcBlobURL.BreakLease(context.Background(), azblob.LeaseBreakNaturally, azblob.ModifiedAccessConditions{})
}()
+ srcBlobLeaseID := leaseResp.LeaseID()
- dstBlobClient := containerClient.NewBlobClient(dst)
- copyResp, err := dstBlobClient.StartCopyFromURL(context.Background(), srcBlobClient.URL(), &blob.StartCopyFromURLOptions{
- Metadata: metadata,
- })
-
+ copyResp, err := dstBlobURL.StartCopyFromURL(
+ context.Background(),
+ srcBlobURL.URL(),
+ metadata,
+ azblob.ModifiedAccessConditions{},
+ azblob.BlobAccessConditions{},
+ azblob.DefaultAccessTier,
+ nil)
if err != nil {
return fmt.Errorf("error copying %s -> %s in %s: %s", src, dst, storage, err)
}
- copyStatus := *copyResp.CopyStatus
+ copyStatus := copyResp.CopyStatus()
for {
- if copyStatus == blob.CopyStatusTypeSuccess {
+ if copyStatus == azblob.CopyStatusSuccess {
if move {
- _, err := storage.az.client.DeleteBlob(context.Background(), storage.az.container, src, &blob.DeleteOptions{
- AccessConditions: &blob.AccessConditions{
- LeaseAccessConditions: &blob.LeaseAccessConditions{
- LeaseID: &leaseID,
- },
- },
- })
+ _, err = srcBlobURL.Delete(
+ context.Background(),
+ azblob.DeleteSnapshotsOptionNone,
+ azblob.BlobAccessConditions{
+ LeaseAccessConditions: azblob.LeaseAccessConditions{LeaseID: srcBlobLeaseID},
+ })
return err
}
return nil
- } else if copyStatus == blob.CopyStatusTypePending {
+ } else if copyStatus == azblob.CopyStatusPending {
time.Sleep(1 * time.Second)
- getMetadata, err := dstBlobClient.GetProperties(context.TODO(), nil)
+ blobPropsResp, err := dstBlobURL.GetProperties(
+ context.Background(),
+ azblob.BlobAccessConditions{LeaseAccessConditions: azblob.LeaseAccessConditions{LeaseID: srcBlobLeaseID}},
+ azblob.ClientProvidedKeyOptions{})
if err != nil {
- return fmt.Errorf("error getting copy progress %s", dst)
+ return fmt.Errorf("error getting destination blob properties %s", dstBlobURL)
}
- copyStatus = *getMetadata.CopyStatus
+ copyStatus = blobPropsResp.CopyStatus()
- _, err = blobLeaseClient.RenewLease(context.Background(), nil)
+ _, err = srcBlobURL.RenewLease(context.Background(), srcBlobLeaseID, azblob.ModifiedAccessConditions{})
if err != nil {
- return fmt.Errorf("error renewing source blob lease %s", src)
+ return fmt.Errorf("error renewing source blob lease %s", srcBlobURL)
}
} else {
return fmt.Errorf("error copying %s -> %s in %s: %s", dst, src, storage, copyStatus)
@@ -245,9 +243,7 @@ func (storage *PublishedStorage) RenameFile(oldName, newName string) error {
// SymLink creates a copy of src file and adds link information as meta data
func (storage *PublishedStorage) SymLink(src string, dst string) error {
- metadata := make(map[string]*string)
- metadata["SymLink"] = &src
- return storage.internalCopyOrMoveBlob(src, dst, metadata, false /* do not remove src */)
+ return storage.internalCopyOrMoveBlob(src, dst, azblob.Metadata{"SymLink": src}, false /* move */)
}
// HardLink using symlink functionality as hard links do not exist
@@ -257,33 +253,28 @@ func (storage *PublishedStorage) HardLink(src string, dst string) error {
// FileExists returns true if path exists
func (storage *PublishedStorage) FileExists(path string) (bool, error) {
- serviceClient := storage.az.client.ServiceClient()
- containerClient := serviceClient.NewContainerClient(storage.az.container)
- blobClient := containerClient.NewBlobClient(path)
- _, err := blobClient.GetProperties(context.Background(), nil)
+ blob := storage.az.blobURL(path)
+ resp, err := blob.GetProperties(context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
if err != nil {
if isBlobNotFound(err) {
return false, nil
}
- return false, fmt.Errorf("error checking if blob %s exists: %v", path, err)
+ return false, err
+ } else if resp.StatusCode() == http.StatusOK {
+ return true, nil
}
- return true, nil
+ return false, fmt.Errorf("error checking if blob %s exists %d", blob, resp.StatusCode())
}
// ReadLink returns the symbolic link pointed to by path.
// This simply reads text file created with SymLink
func (storage *PublishedStorage) ReadLink(path string) (string, error) {
- serviceClient := storage.az.client.ServiceClient()
- containerClient := serviceClient.NewContainerClient(storage.az.container)
- blobClient := containerClient.NewBlobClient(path)
- props, err := blobClient.GetProperties(context.Background(), nil)
+ blob := storage.az.blobURL(path)
+ resp, err := blob.GetProperties(context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
if err != nil {
- return "", fmt.Errorf("failed to get blob properties: %v", err)
+ return "", err
+ } else if resp.StatusCode() != http.StatusOK {
+ return "", fmt.Errorf("error checking if blob %s exists %d", blob, resp.StatusCode())
}
-
- metadata := props.Metadata
- if originalBlob, exists := metadata["original_blob"]; exists {
- return *originalBlob, nil
- }
- return "", fmt.Errorf("error reading link %s: %v", path, err)
+ return resp.NewMetadata()["SymLink"], nil
}
diff --git a/azure/public_test.go b/azure/public_test.go
index 5c912c5..f58ad51 100644
--- a/azure/public_test.go
+++ b/azure/public_test.go
@@ -7,11 +7,8 @@
"io"
"os"
"path/filepath"
- "bytes"
- "github.com/Azure/azure-sdk-for-go/sdk/azcore"
- "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
- "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob"
+ "github.com/Azure/azure-storage-blob-go/azblob"
"github.com/aptly-dev/aptly/files"
"github.com/aptly-dev/aptly/utils"
. "gopkg.in/check.v1"
@@ -69,10 +66,8 @@ func (s *PublishedStorageSuite) SetUpTest(c *C) {
s.storage, err = NewPublishedStorage(s.accountName, s.accountKey, container, "", s.endpoint)
c.Assert(err, IsNil)
- publicAccessType := azblob.PublicAccessTypeContainer
- _, err = s.storage.az.client.CreateContainer(context.Background(), s.storage.az.container, &azblob.CreateContainerOptions{
- Access: &publicAccessType,
- })
+ cnt := s.storage.az.container
+ _, err = cnt.Create(context.Background(), azblob.Metadata{}, azblob.PublicAccessContainer)
c.Assert(err, IsNil)
s.prefixedStorage, err = NewPublishedStorage(s.accountName, s.accountKey, container, prefix, s.endpoint)
@@ -80,39 +75,41 @@ func (s *PublishedStorageSuite) SetUpTest(c *C) {
}
func (s *PublishedStorageSuite) TearDownTest(c *C) {
- _, err := s.storage.az.client.DeleteContainer(context.Background(), s.storage.az.container, nil)
+ cnt := s.storage.az.container
+ _, err := cnt.Delete(context.Background(), azblob.ContainerAccessConditions{})
c.Assert(err, IsNil)
}
func (s *PublishedStorageSuite) GetFile(c *C, path string) []byte {
- resp, err := s.storage.az.client.DownloadStream(context.Background(), s.storage.az.container, path, nil)
+ blob := s.storage.az.container.NewBlobURL(path)
+ resp, err := blob.Download(context.Background(), 0, azblob.CountToEnd, azblob.BlobAccessConditions{}, false, azblob.ClientProvidedKeyOptions{})
c.Assert(err, IsNil)
- data, err := io.ReadAll(resp.Body)
+ body := resp.Body(azblob.RetryReaderOptions{MaxRetryRequests: 3})
+ data, err := io.ReadAll(body)
c.Assert(err, IsNil)
return data
}
func (s *PublishedStorageSuite) AssertNoFile(c *C, path string) {
- serviceClient := s.storage.az.client.ServiceClient()
- containerClient := serviceClient.NewContainerClient(s.storage.az.container)
- blobClient := containerClient.NewBlobClient(path)
- _, err := blobClient.GetProperties(context.Background(), nil)
+ _, err := s.storage.az.container.NewBlobURL(path).GetProperties(
+ context.Background(), azblob.BlobAccessConditions{}, azblob.ClientProvidedKeyOptions{})
c.Assert(err, NotNil)
-
- storageError, ok := err.(*azcore.ResponseError)
+ storageError, ok := err.(azblob.StorageError)
c.Assert(ok, Equals, true)
- c.Assert(storageError.StatusCode, Equals, 404)
+ c.Assert(string(storageError.ServiceCode()), Equals, string(string(azblob.StorageErrorCodeBlobNotFound)))
}
func (s *PublishedStorageSuite) PutFile(c *C, path string, data []byte) {
hash := md5.Sum(data)
- uploadOptions := &azblob.UploadStreamOptions{
- HTTPHeaders: &blob.HTTPHeaders{
- BlobContentMD5: hash[:],
- },
- }
- reader := bytes.NewReader(data)
- _, err := s.storage.az.client.UploadStream(context.Background(), s.storage.az.container, path, reader, uploadOptions)
+ _, err := azblob.UploadBufferToBlockBlob(
+ context.Background(),
+ data,
+ s.storage.az.container.NewBlockBlobURL(path),
+ azblob.UploadToBlockBlobOptions{
+ BlobHTTPHeaders: azblob.BlobHTTPHeaders{
+ ContentMD5: hash[:],
+ },
+ })
c.Assert(err, IsNil)
}
@@ -333,7 +330,7 @@ func (s *PublishedStorageSuite) TestLinkFromPool(c *C) {
// 2nd link from pool, providing wrong path for source file
//
- // this test should check that file already exists in Azure and skip upload (which would fail if not skipped)
+ // this test should check that file already exists in S3 and skip upload (which would fail if not skipped)
s.prefixedStorage.pathCache = nil
err = s.prefixedStorage.LinkFromPool("", filepath.Join("pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, "wrong-looks-like-pathcache-doesnt-work", cksum1, false)
c.Check(err, IsNil)
diff --git a/context/context.go b/context/context.go
index 0ffc3f7..503cad2 100644
--- a/context/context.go
+++ b/context/context.go
@@ -100,6 +100,7 @@ func (context *AptlyContext) config() *utils.ConfigStructure {
configLocations := []string{homeLocation, "/usr/local/etc/aptly.conf", "/etc/aptly.conf"}
for _, configLocation := range configLocations {
+ // FIXME: check if exists, check if readable
err = utils.LoadConfig(configLocation, &utils.Config)
if os.IsPermission(err) || os.IsNotExist(err) {
continue
diff --git a/deb/list.go b/deb/list.go
index 25a2d28..9eda528 100644
--- a/deb/list.go
+++ b/deb/list.go
@@ -598,6 +598,7 @@ func (l *PackageList) Filter(options FilterOptions) (*PackageList, error) {
//
// when follow-all-variants is enabled, we need to try to expand anyway,
// as even if dependency is satisfied now, there might be other ways to satisfy dependency
+ // FIXME: do not search twice
if result.Search(dep, false, true) != nil {
if options.DependencyOptions&DepVerboseResolve == DepVerboseResolve && options.Progress != nil {
options.Progress.ColoredPrintf("@{y}Already satisfied dependency@|: %s with %s", &dep, result.Search(dep, true, true))
diff --git a/go.mod b/go.mod
index 53c5e78..d7f145a 100644
--- a/go.mod
+++ b/go.mod
@@ -4,6 +4,7 @@ go 1.24
require (
github.com/AlekSi/pointer v1.1.0
+ github.com/Azure/azure-storage-blob-go v0.15.0
github.com/DisposaBoy/JsonConfigReader v0.0.0-20171218180944-5ea4d0ddac55
github.com/awalterschulze/gographviz v2.0.1+incompatible
github.com/cavaliergopher/grab/v3 v3.0.1
@@ -41,7 +42,7 @@ require (
)
require (
- github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
+ github.com/Azure/azure-pipeline-go v0.2.3 // indirect
github.com/KyleBanks/depth v1.2.1 // indirect
github.com/PuerkitoBio/purell v1.1.1 // indirect
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 // indirect
@@ -87,6 +88,7 @@ require (
github.com/kr/text v0.2.0 // indirect
github.com/leodido/go-urn v1.2.4 // indirect
github.com/mailru/easyjson v0.7.6 // indirect
+ github.com/mattn/go-ieproxy v0.0.9 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
@@ -96,7 +98,7 @@ require (
github.com/prometheus/common v0.59.1 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
- github.com/rogpeppe/go-internal v1.12.0 // indirect
+ github.com/rogpeppe/go-internal v1.10.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
go.etcd.io/etcd/api/v3 v3.5.15 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.15 // indirect
@@ -115,8 +117,6 @@ require (
)
require (
- github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0
- github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1
github.com/ProtonMail/go-crypto v1.0.0
github.com/aws/aws-sdk-go-v2 v1.32.5
github.com/aws/aws-sdk-go-v2/config v1.28.5
diff --git a/go.sum b/go.sum
index 502f4b2..453a288 100644
--- a/go.sum
+++ b/go.sum
@@ -1,17 +1,20 @@
github.com/AlekSi/pointer v1.1.0 h1:SSDMPcXD9jSl8FPy9cRzoRaMJtm9g9ggGTxecRUbQoI=
github.com/AlekSi/pointer v1.1.0/go.mod h1:y7BvfRI3wXPWKXEBhU71nbnIEEZX0QTSB2Bj48UJIZE=
-github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0 h1:nyQWyZvwGTvunIMxi1Y9uXkcyr+I7TeNrr/foo4Kpk8=
-github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0/go.mod h1:l38EPgmsp71HHLq9j7De57JcKOWPyhrsW1Awm1JS6K0=
-github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0 h1:tfLQ34V6F7tVSwoTf/4lH5sE0o6eCJuNDTmH09nDpbc=
-github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0/go.mod h1:9kIvujWAA58nmPmWB1m23fyWic1kYZMxD9CxaWn4Qpg=
-github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY=
-github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY=
-github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.6.0 h1:PiSrjRPpkQNjrM8H0WwKMnZUdu1RGMtd/LdGKUrOo+c=
-github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.6.0/go.mod h1:oDrbWx4ewMylP7xHivfgixbfGBT6APAwsSoHRKotnIc=
-github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1 h1:cf+OIKbkmMHBaC3u78AXomweqM0oxQSgBXRZf3WH4yM=
-github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1/go.mod h1:ap1dmS6vQKJxSMNiGJcq4QuUQkOynyD93gLw6MDF7ek=
-github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 h1:XHOnouVk1mxXfQidrMEnLlPk9UMeRtyBTnEFtxkV0kU=
-github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI=
+github.com/Azure/azure-pipeline-go v0.2.3 h1:7U9HBg1JFK3jHl5qmo4CTZKFTVgMwdFHMVtCdfBE21U=
+github.com/Azure/azure-pipeline-go v0.2.3/go.mod h1:x841ezTBIMG6O3lAcl8ATHnsOPVl2bqk7S3ta6S6u4k=
+github.com/Azure/azure-storage-blob-go v0.15.0 h1:rXtgp8tN1p29GvpGgfJetavIG0V7OgcSXPpwp3tx6qk=
+github.com/Azure/azure-storage-blob-go v0.15.0/go.mod h1:vbjsVbX0dlxnRc4FFMPsS9BsJWPcne7GB7onqlPvz58=
+github.com/Azure/go-autorest v14.2.0+incompatible h1:V5VMDjClD3GiElqLWO7mz2MxNAK/vTfRHdAubSIPRgs=
+github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
+github.com/Azure/go-autorest/autorest/adal v0.9.13 h1:Mp5hbtOePIzM8pJVRa3YLrWWmZtoxRXqUEzCfJt3+/Q=
+github.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M=
+github.com/Azure/go-autorest/autorest/date v0.3.0 h1:7gUk1U5M/CQbp9WoqinNzJar+8KY+LPI6wiWrP/myHw=
+github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74=
+github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k=
+github.com/Azure/go-autorest/logger v0.2.1 h1:IG7i4p/mDa2Ce4TRyAO8IHnVhAVF3RFU+ZtXWSmf4Tg=
+github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8=
+github.com/Azure/go-autorest/tracing v0.6.0 h1:TYi4+3m5t6K48TGI9AUdb+IzbnSxvnvUMfuitfgcfuo=
+github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=
github.com/DisposaBoy/JsonConfigReader v0.0.0-20171218180944-5ea4d0ddac55 h1:jbGlDKdzAZ92NzK65hUP98ri0/r50vVVvmZsFP/nIqo=
github.com/DisposaBoy/JsonConfigReader v0.0.0-20171218180944-5ea4d0ddac55/go.mod h1:GCzqZQHydohgVLSIqRKZeTt8IGb1Y4NaFfim3H40uUI=
github.com/KyleBanks/depth v1.2.1 h1:5h8fQADFrWtarTdtDudMmGsC7GPbOAu6RVB3ffsVFHc=
@@ -91,6 +94,8 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fatih/color v1.17.0 h1:GlRw1BRJxkpqUCBKzKOw098ed57fEsKeNjpTe3cSjK4=
github.com/fatih/color v1.17.0/go.mod h1:YZ7TlrGPkiz6ku9fK3TLD/pl3CpsiFyu8N92HLgmosI=
+github.com/form3tech-oss/jwt-go v3.2.2+incompatible h1:TcekIExNqud5crz4xD2pavyTgWiPvpYe4Xau31I0PRk=
+github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI=
@@ -127,8 +132,6 @@ github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MG
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
-github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk=
-github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
@@ -149,7 +152,8 @@ github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
-github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/h2non/filetype v1.1.3 h1:FKkx9QbD7HR/zjK1Ia5XiBsq9zdLi5Kf3zGyFTAFkGg=
@@ -197,6 +201,9 @@ github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJ
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
+github.com/mattn/go-ieproxy v0.0.1/go.mod h1:pYabZ6IHcRpFh7vIaLfK7rdcWgFEb3SFJ6/gNWuh88E=
+github.com/mattn/go-ieproxy v0.0.9 h1:RvVbLiMv/Hbjf1gRaC2AQyzwbdVhdId7D2vPnXIml4k=
+github.com/mattn/go-ieproxy v0.0.9/go.mod h1:eF30/rfdQUO9EnzNIZQr0r9HiLMlZNCpJkHbmMuOAE0=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
@@ -235,8 +242,6 @@ github.com/onsi/gomega v1.19.0 h1:4ieX6qQjPP/BfC3mpsAtIGGlxTWPeA3Inl/7DtXw1tw=
github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro=
github.com/pelletier/go-toml/v2 v2.1.0 h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4=
github.com/pelletier/go-toml/v2 v2.1.0/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc=
-github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ=
-github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -254,8 +259,8 @@ github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJ
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
-github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
-github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
+github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
+github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/rs/xid v1.4.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
github.com/rs/zerolog v1.29.1 h1:cO+d60CHkknCbvzEWxP0S9K6KqyTjrCNUy1LdQLCGPc=
github.com/rs/zerolog v1.29.1/go.mod h1:Le6ESbR7hc+DP6Lt1THiV8CQSdkkNrd3R0XbEgp3ZBU=
@@ -319,6 +324,8 @@ golang.org/x/arch v0.3.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
@@ -333,14 +340,14 @@ golang.org/x/mod v0.23.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20191112182307-2180aed22343/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210421230115-4e50805a0758/go.mod h1:72T/g9IO56b78aLF+1Kcs5dz7/ng1VjMUvfKvpfy+jM=
-golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
-golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
-golang.org/x/net v0.0.0-20220607020251-c690dde0001d/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
+golang.org/x/net v0.0.0-20210610132358-84b48f89b13b/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20220630215102-69896b714898/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
@@ -362,6 +369,7 @@ golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191112214154-59a1497f0cea/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -1,35 +0,0 @@
From: =?utf-8?q?S=C3=A9bastien_Delafond?= <seb@debian.org>
Date: Mon, 17 Feb 2025 10:11:55 +0100
Subject: tests: no upstream's etcd install as it's arch-specific,
and no swagger-related or modules tasks
Forwarded: not-needed
---
Makefile | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/Makefile b/Makefile
index ffe2e8a..91f96a8 100644
--- a/Makefile
+++ b/Makefile
@@ -89,17 +89,12 @@ install:
# go install -v
@out=`mktemp`; if ! go install -v > $$out 2>&1; then cat $$out; rm -f $$out; echo "\nBuild failed\n"; exit 1; else rm -f $$out; fi
-test: prepare swagger etcd-install ## Run unit tests (add TEST=regex to specify which tests to run)
- @echo "\e[33m\e[1mStarting etcd ...\e[0m"
- @mkdir -p /tmp/aptly-etcd-data; system/t13_etcd/start-etcd.sh > /tmp/aptly-etcd-data/etcd.log 2>&1 &
+test: ## Run unit tests
@echo "\e[33m\e[1mRunning go test ...\e[0m"
- faketime "$(TEST_FAKETIME)" go test -v ./... -gocheck.v=true -check.f "$(TEST)" -coverprofile=unit.out; echo $$? > .unit-test.ret
- @echo "\e[33m\e[1mStopping etcd ...\e[0m"
- @pid=`cat /tmp/etcd.pid`; kill $$pid
- @rm -f /tmp/aptly-etcd-data/etcd.log
+ go test -v ./... -gocheck.v=true -coverprofile=unit.out; echo $$? > .unit-test.ret
@ret=`cat .unit-test.ret`; if [ "$$ret" = "0" ]; then echo "\n\e[32m\e[1mUnit Tests SUCCESSFUL\e[0m"; else echo "\n\e[31m\e[1mUnit Tests FAILED\e[0m"; fi; rm -f .unit-test.ret; exit $$ret
-system-test: prepare swagger etcd-install ## Run system tests
+system-test: ## Run system tests
# build coverage binary
go test -v -coverpkg="./..." -c -tags testruncli
# Download fixture-db, fixture-pool, etcd.db
@@ -1,40 +0,0 @@
From: =?utf-8?q?S=C3=A9bastien_Delafond?= <seb@debian.org>
Date: Wed, 24 Sep 2025 07:23:24 +0200
Subject: Revert "system-tests: abort on failure"
We'd rather have the test suite always run completely, and report
every failed test at the end.
Forwarded: not-needed
---
system/run.py | 8 --------
1 file changed, 8 deletions(-)
diff --git a/system/run.py b/system/run.py
index 4e73fb2..599afe5 100755
--- a/system/run.py
+++ b/system/run.py
@@ -50,7 +50,6 @@ def run(include_long_tests=False, capture_results=False, tests=None, filters=Non
if not coverage_dir:
coverage_dir = mkdtemp(suffix="aptly-coverage")
- failed = False
for test in tests:
orig_stdout = sys.stdout
orig_stderr = sys.stderr
@@ -158,15 +157,8 @@ def run(include_long_tests=False, capture_results=False, tests=None, filters=Non
t.shutdown()
- if failed:
- break
- if failed:
- break
-
sys.stdout = orig_stdout
sys.stderr = orig_stderr
- if failed:
- break
if lastBase is not None:
lastBase.shutdown_class()
-4
View File
@@ -1,4 +0,0 @@
0001-disable-swagger.patch
0002-disable-new-azure-sdk.patch
0003-tests-no-upstream-s-etcd-install-as-it-s-arch-specif.patch
0004-Revert-system-tests-abort-on-failure.patch
+34 -4
View File
@@ -2,17 +2,47 @@
include /usr/share/dpkg/pkg-info.mk
export GOPATH=$(shell pwd)/.go
export DEB_BUILD_OPTIONS=crossbuildcanrunhostbinaries
export GOARCH := $(shell if [ $(DEB_TARGET_ARCH) = "i386" ]; then echo "386"; elif [ $(DEB_TARGET_ARCH) = "armhf" ]; then echo "arm"; else echo $(DEB_TARGET_ARCH); fi)
export CGO_ENABLED=1
ifneq ($(DEB_HOST_GNU_TYPE), $(DEB_BUILD_GNU_TYPE))
export CC=$(DEB_HOST_GNU_TYPE)-gcc
endif
%:
dh $@ --buildsystem=golang --with=golang,bash-completion
override_dh_auto_clean:
rm -rf build/
rm -rf obj-$(DEB_TARGET_GNU_TYPE)/
dh_auto_clean
override_dh_auto_test:
# run during autopkgtests
override_dh_auto_install:
dh_auto_install -- --no-source
override_dh_strip:
dh_strip --dbg-package=aptly-dbg
override_dh_golang: # fails on non native debian build
# override_dh_makeshlibs: # fails with cross compiling on non native debian build
override_dh_dwz: # somehow dwz works only with certain newer debhelper versions
dhver=`dpkg-query -f '$${Version}' -W debhelper`; (dpkg --compare-versions "$$dhver" lt 13 || test "$$dhver" = "13.3.4" || test "$$dhver" = "13.6ubuntu1") || dh_dwz
override_dh_shlibdeps:
ifneq ($(DEB_HOST_GNU_TYPE), $(DEB_BUILD_GNU_TYPE))
LD_LIBRARY_PATH=/usr/$(DEB_HOST_GNU_TYPE)/lib:$$LD_LIBRARY_PATH dh_shlibdeps
else
dh_shlibdeps
endif
override_dh_auto_build:
echo $(DEB_VERSION) > obj-$(DEB_TARGET_GNU_TYPE)/src/github.com/aptly-dev/aptly/VERSION
mkdir -p obj-$(DEB_TARGET_GNU_TYPE)/src/github.com/aptly-dev/aptly/debian
cp debian/aptly.conf obj-$(DEB_TARGET_GNU_TYPE)/src/github.com/aptly-dev/aptly/debian/
dh_auto_build
echo $(DEB_VERSION) > VERSION
go build -buildmode=pie -o usr/bin/aptly
+1 -1
View File
@@ -1 +1 @@
3.0 (quilt)
3.0 (git)
-32
View File
@@ -1,32 +0,0 @@
#!/bin/sh
# run upstream's unit tests with Debian-supplied dependencies
# FIXME: right now this fails hard because many prerequisites are only
# handled in `make test`
set -e
set -u
set -x
BUILD_DIR="${PWD}/_build"
DH_OPTIONS="-O--buildsystem=golang -O--builddirectory=$BUILD_DIR"
APTLY_DIR="${BUILD_DIR}/src/github.com/aptly-dev/aptly"
dpkg-source --before-build .
dh_update_autotools_config $DH_OPTIONS
dh_autoreconf $DH_OPTIONS
dh_auto_configure $DH_OPTIONS
dpkg-parsechangelog --show-field Version | perl -pe 's/\n//' >| ${APTLY_DIR}/VERSION
find . -name VERSION
mkdir -p ${APTLY_DIR}/debian
cp debian/aptly.conf ${APTLY_DIR}/debian/
dh_auto_build $DH_OPTIONS
export PATH=${BUILD_DIR}/bin:$PATH
dh_auto_test $DH_OPTIONS --no-parallel
+3 -6
View File
@@ -1,7 +1,4 @@
Tests: unit-test
Depends: @, @builddeps@, ca-certificates, curl, git, gpg, gpg-agent
Restrictions: allow-stderr
# This file is an addition to the autodep8 tests.
Tests: system-test
Depends: @, @builddeps@, ca-certificates, curl, dirmngr, git, gpg, gpg-agent, gpgconf, graphviz, procps, python3, python3-requests-unixsocket, python3-termcolor, python3-swiftclient, python3-boto3, python3-azure-storage, python3-etcd3, python3-plyvel, sudo, zip
Restrictions: allow-stderr, needs-internet
Test-Command: HOME=/tmp aptly repo create autopkgtest
Restrictions: allow-stderr
-20
View File
@@ -1,20 +0,0 @@
#!/bin/sh
set -eux
TMP_DIR="$(mktemp -d)"
APTLY_SRC_DIR="${TMP_DIR}/src/github.com/aptly-dev/aptly"
# apply patches
dpkg-source --before-build .
# copy source to GOPATH-compatible dir
mkdir -p $(dirname $APTLY_SRC_DIR)
cp -a . $APTLY_SRC_DIR
# not in a git tree, so we need to generate the version ourselves
dpkg-parsechangelog --show-field Version | perl -pe 's/\n//' > ${APTLY_SRC_DIR}/VERSION
# use only apt-supplied go libraries
export GOPATH="${TMP_DIR}:/usr/share/gocode"
export GO111MODULE=off
-41
View File
@@ -1,41 +0,0 @@
#!/bin/sh
# run upstream's integration tests
set -eux
. debian/tests/setup
## env
TESTS_DIR="${APTLY_SRC_DIR}/system"
## functions
disable_test() {
local file=${1}.py
local name=$2
local reason=$3
echo "${name}.skipTest = 'Debian autopkgtest: $reason'" >> ${TESTS_DIR}/${file}
}
## main
export USER=root # for t07/RootDirInaccessible
disable_test t01_version/version VersionTest "version"
disable_test t02_config/config CreateConfigTest "different conf"
disable_test t04_mirror/create CreateMirror31Test "public key not found"
disable_test t04_mirror/create CreateMirror35Test "flaky on s390"
disable_test t07_serve/serve Serve1Test "minor html diff"
disable_test t09_repo/edit EditRepo4Test "flaky on riscv64"
disable_test t10_task/run RunTask1Test "version"
disable_test t12_api/docs TaskAPITestSwaggerDocs "no recent swag"
disable_test t12_api/gpg GPGAPITestAddKey "flaky on s390"
disable_test t12_api/unix_socket UnixSocketAPITest "type mismatch"
disable_test t12_api/version VersionAPITest "type mismatch"
disable_test t14_graph/graph CreateGraphTest "no viewer"
disable_test t14_graph/graph CreateGraphOutputTest "no viewer"
# etcd fixture is entirely arch-specific
rm -fr ${TESTS_DIR}/t13_etcd
make -C $APTLY_SRC_DIR system-test
-20
View File
@@ -1,20 +0,0 @@
#!/bin/sh
# run upstream's unit tests with their full etcd fixtures, etc
set -eux
. debian/tests/setup
# FIXME: errors with non-constant format string in call to
# github.com/aptly-dev/aptly/s3.fatalError
rm ${APTLY_SRC_DIR}/s3/server_test.go
rm ${APTLY_SRC_DIR}/s3/public_test.go
# upstream's etcd fixture is arch-specific
rm ${APTLY_SRC_DIR}/database/etcddb/database_test.go
# TestVerifyClearsigned fails because of an extra signature
perl -i -pe 's/(TestVerifyClearsigned)/No$1/' ${APTLY_SRC_DIR}/pgp/verify_test.go
make -C $APTLY_SRC_DIR test
+5 -6
View File
@@ -1,6 +1,5 @@
Version: 5
Template: Github
Owner: aptly-dev
Project: aptly
Dversion-Mangle: s/\+ds\d*$//
Repacksuffix: +ds1
version=4
opts=\
repacksuffix=+ds1,\
dversionmangle=s/\+ds\d*$// \
https://github.com/aptly-dev/aptly/tags .*/v(\d[\d\.]*)\.tar\.gz debian uupdate
+209
View File
@@ -0,0 +1,209 @@
# GPG Keys Management
GPG keys are used by aptly to verify the authenticity of remote repository Release files when creating mirrors. This document describes the API endpoints for managing GPG keys in the aptly keyring.
## Overview
Aptly uses GNU Privacy Guard (GPG) to verify signed repository metadata. You must add the repository's GPG public key to aptly's keyring before creating mirrors that verify signatures.
Keys are stored in the aptly keyring (default: `trustedkeys.gpg`). You can have multiple keyrings and specify which one to use via the `Keyring` parameter.
## API Endpoints
### List GPG Keys
**GET /api/gpg/keys**
Lists all public GPG keys currently installed in the aptly keyring.
**Parameters:**
- `keyring` (query, optional): Keyring file to list keys from. Default: `trustedkeys.gpg`
**Response:**
```json
{
"Keys": [
{
"KeyID": "8B48AD6246925553",
"Fingerprint": "D8E8F5A516E7A2C4F3E4B5A6C7D8E9F0",
"Validity": "f",
"UserIDs": ["John Doe <john@example.com>"],
"CreatedAt": "1611864000"
}
]
}
```
**Status Codes:**
- `200 OK`: Keys successfully retrieved
- `400 Bad Request`: GPG execution failed or invalid parameters
**Example:**
```bash
curl http://localhost:8080/api/gpg/keys
curl "http://localhost:8080/api/gpg/keys?keyring=custom.gpg"
```
---
### Add GPG Key
**POST /api/gpg/key**
Adds a GPG public key to the aptly keyring. Keys can be added in two ways:
1. Provide the ASCII-armored key directly
2. Provide a key server and key ID(s) to download from
**Request Body:**
```json
{
"Keyring": "trustedkeys.gpg",
"GpgKeyArmor": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n...",
"Keyserver": "hkp://keyserver.ubuntu.com:80",
"GpgKeyID": "8B48AD6246925553"
}
```
**Parameters:**
- `Keyring` (optional): Keyring file to add keys to. Default: `trustedkeys.gpg`
- `GpgKeyArmor` (optional): ASCII-armored GPG public key
- `Keyserver` (optional): Keyserver URL (e.g., `hkp://keyserver.ubuntu.com:80`)
- `GpgKeyID` (optional): Space-separated key IDs to download from keyserver
**Status Codes:**
- `200 OK`: Key successfully added
- `400 Bad Request`: Invalid parameters or GPG execution failed
**Example - From ASCII Key:**
```bash
curl -X POST http://localhost:8080/api/gpg/key \
-H "Content-Type: application/json" \
-d '{
"GpgKeyArmor": "-----BEGIN PGP PUBLIC KEY BLOCK-----\nVersion: GnuPG v2\n...\n-----END PGP PUBLIC KEY BLOCK-----"
}'
```
**Example - From Keyserver:**
```bash
curl -X POST http://localhost:8080/api/gpg/key \
-H "Content-Type: application/json" \
-d '{
"Keyserver": "hkp://keyserver.ubuntu.com:80",
"GpgKeyID": "8B48AD6246925553 A1B2C3D4E5F67890"
}'
```
---
### Delete GPG Key
**DELETE /api/gpg/key**
Removes a GPG key from the aptly keyring.
**Request Body:**
```json
{
"Keyring": "trustedkeys.gpg",
"GpgKeyID": "8B48AD6246925553"
}
```
**Parameters:**
- `Keyring` (optional): Keyring file to delete from. Default: `trustedkeys.gpg`
- `GpgKeyID` (required): Key ID or fingerprint to delete
**Status Codes:**
- `200 OK`: Key successfully deleted
- `400 Bad Request`: Invalid parameters or GPG execution failed
**Example:**
```bash
curl -X DELETE http://localhost:8080/api/gpg/key \
-H "Content-Type: application/json" \
-d '{
"GpgKeyID": "8B48AD6246925553"
}'
```
---
## Use Cases
### 1. Verify Downloaded Repository Metadata
Before creating a mirror from a signed repository, add the repository's GPG key:
```bash
# Add the key from a keyserver
curl -X POST http://localhost:8080/api/gpg/key \
-H "Content-Type: application/json" \
-d '{
"Keyserver": "hkp://keyserver.ubuntu.com:80",
"GpgKeyID": "EB9B46B91F2D3B7E"
}'
# Now create a mirror with signature verification
# (signature verification configured in mirror settings)
```
### 2. Manage Multiple Keyrings
Aptly supports using different keyrings for different purposes. For example, one for Debian repositories and another for custom internal repositories:
```bash
# Add key to Debian keyring
curl -X POST http://localhost:8080/api/gpg/key \
-H "Content-Type: application/json" \
-d '{
"Keyring": "debian-keys.gpg",
"Keyserver": "hkp://keyserver.ubuntu.com:80",
"GpgKeyID": "EB9B46B91F2D3B7E"
}'
# List keys in Debian keyring
curl "http://localhost:8080/api/gpg/keys?keyring=debian-keys.gpg"
```
### 3. Remove Compromised Keys
If a GPG key is compromised, remove it from the keyring immediately:
```bash
curl -X DELETE http://localhost:8080/api/gpg/key \
-H "Content-Type: application/json" \
-d '{
"GpgKeyID": "COMPROMISED_KEY_ID"
}'
```
---
## Key Validity Values
Keys retrieved from `GET /api/gpg/keys` have a `Validity` field with the following possible values:
- `u` — Unknown validity
- `f` — Full trust
- `m` — Marginal trust
- `n` — Never trust
- `-` — Trust not set
The trust level is typically managed in your GPG configuration and does not affect aptly's ability to verify signatures.
---
## Troubleshooting
**"failed to list keys"**
- Check that the keyring file exists and is readable
- Verify GPG is installed and configured
**"unable to delete key: no public key"**
- The key might not exist in the keyring
- Verify the key ID is correct by listing keys first
**"invalid request body"**
- Ensure the JSON is properly formatted
- For POST requests, provide either `GpgKeyArmor` or (`Keyserver` + `GpgKeyID`)
- For DELETE requests, `GpgKeyID` is required
+1 -1
View File
@@ -5,7 +5,7 @@ Publish snapshot or local repo as Debian repository to be used as APT source on
The published repository is signed with the user's GnuPG key.
Repositories can be published to local directories, Amazon S3 buckets, Azure or Swift Storage.
Repositories can be published to local directories, Amazon S3 buckets, Azure, Swift, or JFrog Artifactory Storage.
#### GPG Keys
+2
View File
@@ -0,0 +1,2 @@
// Package gcs handles publishing to Google Cloud Storage.
package gcs
+12
View File
@@ -0,0 +1,12 @@
package gcs
import (
"testing"
. "gopkg.in/check.v1"
)
// Launch gocheck tests.
func Test(t *testing.T) {
TestingT(t)
}
+422
View File
@@ -0,0 +1,422 @@
package gcs
import (
"context"
"encoding/hex"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"cloud.google.com/go/storage"
"github.com/aptly-dev/aptly/aptly"
"github.com/aptly-dev/aptly/utils"
"github.com/pkg/errors"
"github.com/rs/zerolog/log"
"google.golang.org/api/googleapi"
"google.golang.org/api/iterator"
"google.golang.org/api/option"
)
// PublishedStorage abstracts published files hosted on GCS.
type PublishedStorage struct {
client *storage.Client
bucket *storage.BucketHandle
bucketName string
prefix string
acl string
storageClass string
encryptionKey string
disableMultiDel bool
debug bool
pathCache map[string]string
}
var (
_ aptly.PublishedStorage = (*PublishedStorage)(nil)
)
// NewPublishedStorage creates a GCS-backed published storage.
func NewPublishedStorage(bucket, prefix, credentialsFile, serviceAccountJSON,
project, endpoint, defaultACL, storageClass, encryptionKey string,
disableMultiDel, debug bool) (*PublishedStorage, error) {
ctx := context.TODO()
opts := make([]option.ClientOption, 0, 4)
if endpoint != "" {
opts = append(opts, option.WithEndpoint(endpoint), option.WithoutAuthentication())
} else if credentialsFile != "" {
opts = append(opts, option.WithAuthCredentialsFile(option.ServiceAccount, credentialsFile))
} else if serviceAccountJSON != "" {
opts = append(opts, option.WithAuthCredentialsJSON(option.ServiceAccount, []byte(serviceAccountJSON)))
}
if project != "" {
opts = append(opts, option.WithQuotaProject(project))
}
// When pointing at a non-production endpoint (an explicit override or the
// STORAGE_EMULATOR_HOST hook the GCS Go client honours natively), force
// JSON reads. The default XML download API uses virtual-host-style URLs
// (https://<bucket>.storage.googleapis.com/...) that emulators and most
// dev/staging endpoints can't serve under a single listener; the JSON API
// uses path-based URLs that work the same against real GCS or an emulator.
if endpoint != "" || os.Getenv("STORAGE_EMULATOR_HOST") != "" {
opts = append(opts, storage.WithJSONReads())
}
client, err := storage.NewClient(ctx, opts...)
if err != nil {
return nil, err
}
result := &PublishedStorage{
client: client,
bucket: client.Bucket(bucket),
bucketName: bucket,
prefix: prefix,
acl: defaultACL,
storageClass: storageClass,
encryptionKey: encryptionKey,
disableMultiDel: disableMultiDel,
debug: debug,
}
return result, nil
}
func (g *PublishedStorage) String() string {
return fmt.Sprintf("GCS: %s/%s", g.bucketName, g.prefix)
}
// MkDir creates directory recursively under public path.
func (g *PublishedStorage) MkDir(_ string) error {
// no-op for GCS
return nil
}
func (g *PublishedStorage) objectPath(path string) string {
return filepath.Join(g.prefix, path)
}
func (g *PublishedStorage) objectHandle(path string) *storage.ObjectHandle {
obj := g.bucket.Object(g.objectPath(path))
if g.encryptionKey != "" {
obj = obj.Key([]byte(g.encryptionKey))
}
return obj
}
// PutFile puts file into published storage at specified path.
func (g *PublishedStorage) PutFile(path string, sourceFilename string) error {
source, err := os.Open(sourceFilename)
if err != nil {
return err
}
defer func() { _ = source.Close() }()
if g.debug {
log.Debug().Msgf("GCS: PutFile '%s'", path)
}
err = g.putFile(path, source, "")
if err != nil {
return errors.Wrap(err, fmt.Sprintf("error uploading %s to %s", sourceFilename, g))
}
return nil
}
func (g *PublishedStorage) applyACL(obj *storage.ObjectHandle) error {
switch g.acl {
case "", "none", "private":
return nil
case "public-read":
return obj.ACL().Set(context.TODO(), storage.AllUsers, storage.RoleReader)
default:
return fmt.Errorf("unsupported GCS ACL value: %s", g.acl)
}
}
func (g *PublishedStorage) putFile(path string, source io.Reader, sourceMD5 string) error {
obj := g.objectHandle(path)
writer := obj.NewWriter(context.TODO())
if g.storageClass != "" {
writer.StorageClass = g.storageClass
}
if sourceMD5 != "" {
writer.Metadata = map[string]string{"Md5": sourceMD5}
}
if _, err := io.Copy(writer, source); err != nil {
_ = writer.Close()
return err
}
if err := writer.Close(); err != nil {
return err
}
return g.applyACL(obj)
}
func (g *PublishedStorage) getMD5(path string) (string, error) {
attrs, err := g.objectHandle(path).Attrs(context.TODO())
if err != nil {
return "", err
}
if attrs.Metadata != nil {
if md5, ok := attrs.Metadata["Md5"]; ok && md5 != "" {
return strings.ToLower(md5), nil
}
}
return strings.ToLower(hex.EncodeToString(attrs.MD5)), nil
}
// Remove removes single file under public path.
func (g *PublishedStorage) Remove(path string) error {
if g.debug {
log.Debug().Msgf("GCS: Remove '%s'", path)
}
err := g.objectHandle(path).Delete(context.TODO())
if err != nil {
if err == storage.ErrObjectNotExist {
return nil
}
var apiErr *googleapi.Error
if errors.As(err, &apiErr) && apiErr.Code == 404 {
return nil
}
return errors.Wrap(err, fmt.Sprintf("error deleting %s from %s", path, g))
}
delete(g.pathCache, path)
return nil
}
// RemoveDirs removes directory structure under public path.
func (g *PublishedStorage) RemoveDirs(path string, _ aptly.Progress) error {
filelist, _, err := g.internalFilelist(path)
if err != nil {
return err
}
if g.debug {
log.Debug().Msgf("GCS: RemoveDirs '%s'", path)
}
for _, file := range filelist {
objPath := filepath.Join(path, file)
if err := g.Remove(objPath); err != nil {
return err
}
}
_ = g.disableMultiDel
return nil
}
// LinkFromPool links package file from pool to dist's pool location.
func (g *PublishedStorage) LinkFromPool(publishedPrefix, publishedRelPath, fileName string, sourcePool aptly.PackagePool,
sourcePath string, sourceChecksums utils.ChecksumInfo, force bool) error {
publishedDirectory := filepath.Join(publishedPrefix, publishedRelPath)
relPath := filepath.Join(publishedDirectory, fileName)
poolPath := filepath.Join(g.prefix, relPath)
if g.pathCache == nil {
paths, md5s, err := g.internalFilelist(filepath.Join(publishedPrefix, "pool"))
if err != nil {
return errors.Wrap(err, "error caching paths under prefix")
}
g.pathCache = make(map[string]string, len(paths))
for i := range paths {
g.pathCache[filepath.Join(publishedPrefix, "pool", paths[i])] = md5s[i]
}
}
destinationMD5, exists := g.pathCache[relPath]
sourceMD5 := strings.ToLower(sourceChecksums.MD5)
if exists {
if sourceMD5 == "" {
return fmt.Errorf("unable to compare object, MD5 checksum missing")
}
if len(destinationMD5) != 32 {
var err error
destinationMD5, err = g.getMD5(relPath)
if err != nil {
return errors.Wrap(err, fmt.Sprintf("error verifying MD5 for %s: %s", g, poolPath))
}
g.pathCache[relPath] = destinationMD5
}
if destinationMD5 == sourceMD5 {
return nil
}
if !force {
return fmt.Errorf("error putting file to %s: file already exists and is different: %s", poolPath, g)
}
}
source, err := sourcePool.Open(sourcePath)
if err != nil {
return err
}
defer func() { _ = source.Close() }()
if g.debug {
log.Debug().Msgf("GCS: LinkFromPool '%s'", relPath)
}
err = g.putFile(relPath, source, sourceMD5)
if err != nil {
return errors.Wrap(err, fmt.Sprintf("error uploading %s to %s: %s", sourcePath, g, poolPath))
}
g.pathCache[relPath] = sourceMD5
return nil
}
// Filelist returns list of files under prefix.
func (g *PublishedStorage) Filelist(prefix string) ([]string, error) {
paths, _, err := g.internalFilelist(prefix)
return paths, err
}
func (g *PublishedStorage) internalFilelist(prefix string) ([]string, []string, error) {
paths := make([]string, 0, 1024)
md5s := make([]string, 0, 1024)
fullPrefix := filepath.Join(g.prefix, prefix)
if fullPrefix != "" {
fullPrefix += "/"
}
it := g.bucket.Objects(context.TODO(), &storage.Query{Prefix: fullPrefix})
for {
attrs, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return nil, nil, errors.WithMessagef(err, "error listing under prefix %s in %s", fullPrefix, g)
}
path := attrs.Name
if fullPrefix != "" {
path = strings.TrimPrefix(path, fullPrefix)
}
paths = append(paths, path)
if attrs.Metadata != nil {
if md5, ok := attrs.Metadata["Md5"]; ok && md5 != "" {
md5s = append(md5s, strings.ToLower(md5))
continue
}
}
md5s = append(md5s, strings.ToLower(hex.EncodeToString(attrs.MD5)))
}
return paths, md5s, nil
}
// RenameFile renames (moves) file.
func (g *PublishedStorage) RenameFile(oldName, newName string) error {
src := g.objectHandle(oldName)
dst := g.objectHandle(newName)
if g.debug {
log.Debug().Msgf("GCS: RenameFile %s -> %s", oldName, newName)
}
_, err := dst.CopierFrom(src).Run(context.TODO())
if err != nil {
return fmt.Errorf("error copying %s -> %s in %s: %s", oldName, newName, g, err)
}
err = g.applyACL(dst)
if err != nil {
return err
}
return g.Remove(oldName)
}
// SymLink creates a copy of src file and stores link information in metadata.
func (g *PublishedStorage) SymLink(src string, dst string) error {
source := g.objectHandle(src)
dest := g.objectHandle(dst)
if g.debug {
log.Debug().Msgf("GCS: SymLink %s -> %s", src, dst)
}
_, err := dest.CopierFrom(source).Run(context.TODO())
if err != nil {
return fmt.Errorf("error symlinking %s -> %s in %s: %s", src, dst, g, err)
}
_, err = dest.Update(context.TODO(), storage.ObjectAttrsToUpdate{
Metadata: map[string]string{"SymLink": src},
})
if err != nil {
return fmt.Errorf("error updating symlink metadata %s -> %s in %s: %s", src, dst, g, err)
}
return g.applyACL(dest)
}
// HardLink uses symlink functionality as hard links do not exist on object stores.
func (g *PublishedStorage) HardLink(src string, dst string) error {
if g.debug {
log.Debug().Msgf("GCS: HardLink %s -> %s", src, dst)
}
return g.SymLink(src, dst)
}
// FileExists returns true if path exists.
func (g *PublishedStorage) FileExists(path string) (bool, error) {
_, err := g.objectHandle(path).Attrs(context.TODO())
if err != nil {
if err == storage.ErrObjectNotExist {
return false, nil
}
var apiErr *googleapi.Error
if errors.As(err, &apiErr) && apiErr.Code == 404 {
return false, nil
}
return false, err
}
return true, nil
}
// ReadLink returns symbolic link target from metadata.
func (g *PublishedStorage) ReadLink(path string) (string, error) {
attrs, err := g.objectHandle(path).Attrs(context.TODO())
if err != nil {
return "", err
}
return attrs.Metadata["SymLink"], nil
}
+530
View File
@@ -0,0 +1,530 @@
package gcs
import (
"context"
"errors"
"io"
"os"
"path/filepath"
"sort"
"cloud.google.com/go/storage"
"github.com/fsouza/fake-gcs-server/fakestorage"
. "gopkg.in/check.v1"
"github.com/aptly-dev/aptly/files"
"github.com/aptly-dev/aptly/utils"
)
type PublishedStorageSuite struct {
srv *fakestorage.Server
prevEmulatorHost string
prevEmulatorHostSet bool
storage, prefixedStorage *PublishedStorage
noSuchBucketStorage *PublishedStorage
}
var _ = Suite(&PublishedStorageSuite{})
func (s *PublishedStorageSuite) SetUpTest(c *C) {
var err error
s.srv, err = fakestorage.NewServerWithOptions(fakestorage.Options{
Scheme: "http",
Host: "127.0.0.1",
})
c.Assert(err, IsNil)
c.Assert(s.srv, NotNil)
s.srv.CreateBucketWithOpts(fakestorage.CreateBucketOpts{Name: "test"})
// The cloud.google.com/go/storage client honors STORAGE_EMULATOR_HOST and
// will route all requests (including media uploads) to the fake server.
s.prevEmulatorHost, s.prevEmulatorHostSet = os.LookupEnv("STORAGE_EMULATOR_HOST")
c.Assert(os.Setenv("STORAGE_EMULATOR_HOST", s.srv.URL()), IsNil)
s.storage, err = NewPublishedStorage("test", "", "", "", "", "", "", "", "", false, false)
c.Assert(err, IsNil)
s.prefixedStorage, err = NewPublishedStorage("test", "lala", "", "", "", "", "", "", "", false, false)
c.Assert(err, IsNil)
s.noSuchBucketStorage, err = NewPublishedStorage("no-bucket", "", "", "", "", "", "", "", "", false, false)
c.Assert(err, IsNil)
}
func (s *PublishedStorageSuite) TearDownTest(c *C) {
if s.prevEmulatorHostSet {
_ = os.Setenv("STORAGE_EMULATOR_HOST", s.prevEmulatorHost)
} else {
_ = os.Unsetenv("STORAGE_EMULATOR_HOST")
}
s.srv.Stop()
}
func (s *PublishedStorageSuite) GetFile(c *C, path string) []byte {
r, err := s.storage.bucket.Object(path).NewReader(context.TODO())
c.Assert(err, IsNil)
defer func() { _ = r.Close() }()
body, err := io.ReadAll(r)
c.Assert(err, IsNil)
return body
}
func (s *PublishedStorageSuite) AssertNoFile(c *C, path string) {
_, err := s.storage.bucket.Object(path).Attrs(context.TODO())
c.Assert(errors.Is(err, storage.ErrObjectNotExist), Equals, true)
}
func (s *PublishedStorageSuite) PutFile(c *C, path string, data []byte) {
w := s.storage.bucket.Object(path).NewWriter(context.TODO())
_, err := w.Write(data)
c.Assert(err, IsNil)
c.Assert(w.Close(), IsNil)
}
func (s *PublishedStorageSuite) TestString(c *C) {
c.Check(s.storage.String(), Equals, "GCS: test/")
c.Check(s.prefixedStorage.String(), Equals, "GCS: test/lala")
}
func (s *PublishedStorageSuite) TestMkDir(c *C) {
c.Check(s.storage.MkDir("anything"), IsNil)
}
func (s *PublishedStorageSuite) TestApplyACLNoOpModes(c *C) {
for _, acl := range []string{"", "none", "private"} {
st := &PublishedStorage{acl: acl}
c.Check(st.applyACL(nil), IsNil)
}
}
func (s *PublishedStorageSuite) TestApplyACLUnsupported(c *C) {
st := &PublishedStorage{acl: "bucket-owner-full-control"}
err := st.applyACL(nil)
c.Assert(err, NotNil)
c.Check(err, ErrorMatches, "unsupported GCS ACL value: bucket-owner-full-control")
}
func (s *PublishedStorageSuite) TestPutFile(c *C) {
dir := c.MkDir()
err := os.WriteFile(filepath.Join(dir, "a"), []byte("welcome to gcs!"), 0644)
c.Assert(err, IsNil)
err = s.storage.PutFile("a/b.txt", filepath.Join(dir, "a"))
c.Check(err, IsNil)
c.Check(s.GetFile(c, "a/b.txt"), DeepEquals, []byte("welcome to gcs!"))
err = s.prefixedStorage.PutFile("a/b.txt", filepath.Join(dir, "a"))
c.Check(err, IsNil)
c.Check(s.GetFile(c, "lala/a/b.txt"), DeepEquals, []byte("welcome to gcs!"))
}
func (s *PublishedStorageSuite) TestPutFileMissingSource(c *C) {
err := s.storage.PutFile("a/b.txt", filepath.Join(c.MkDir(), "does-not-exist"))
c.Check(err, ErrorMatches, ".*no such file or directory.*")
}
func (s *PublishedStorageSuite) TestFilelist(c *C) {
paths := []string{"a", "b", "c", "testa", "test/a", "test/b", "lala/a", "lala/b", "lala/c"}
for _, path := range paths {
s.PutFile(c, path, []byte("test"))
}
list, err := s.storage.Filelist("")
c.Check(err, IsNil)
sort.Strings(list)
c.Check(list, DeepEquals, []string{"a", "b", "c", "lala/a", "lala/b", "lala/c", "test/a", "test/b", "testa"})
list, err = s.storage.Filelist("test")
c.Check(err, IsNil)
sort.Strings(list)
c.Check(list, DeepEquals, []string{"a", "b"})
list, err = s.storage.Filelist("test2")
c.Check(err, IsNil)
c.Check(list, DeepEquals, []string{})
list, err = s.prefixedStorage.Filelist("")
c.Check(err, IsNil)
sort.Strings(list)
c.Check(list, DeepEquals, []string{"a", "b", "c"})
}
func (s *PublishedStorageSuite) TestRemove(c *C) {
s.PutFile(c, "a/b", []byte("test"))
err := s.storage.Remove("a/b")
c.Check(err, IsNil)
s.AssertNoFile(c, "a/b")
s.PutFile(c, "lala/xyz", []byte("test"))
err = s.prefixedStorage.Remove("xyz")
c.Check(err, IsNil)
s.AssertNoFile(c, "lala/xyz")
}
func (s *PublishedStorageSuite) TestRemoveMissing(c *C) {
c.Check(s.storage.Remove("does/not/exist"), IsNil)
}
func (s *PublishedStorageSuite) TestRemoveDirs(c *C) {
paths := []string{"a", "b", "c", "testa", "test/a", "test/b", "lala/a", "lala/b", "lala/c"}
for _, path := range paths {
s.PutFile(c, path, []byte("test"))
}
err := s.storage.RemoveDirs("test", nil)
c.Check(err, IsNil)
list, err := s.storage.Filelist("")
c.Check(err, IsNil)
sort.Strings(list)
c.Check(list, DeepEquals, []string{"a", "b", "c", "lala/a", "lala/b", "lala/c", "testa"})
}
func (s *PublishedStorageSuite) TestRenameFile(c *C) {
s.PutFile(c, "src", []byte("payload"))
err := s.storage.RenameFile("src", "dst")
c.Check(err, IsNil)
c.Check(s.GetFile(c, "dst"), DeepEquals, []byte("payload"))
s.AssertNoFile(c, "src")
}
func (s *PublishedStorageSuite) TestSymLink(c *C) {
s.PutFile(c, "a/b", []byte("test"))
err := s.storage.SymLink("a/b", "a/b.link")
c.Check(err, IsNil)
link, err := s.storage.ReadLink("a/b.link")
c.Check(err, IsNil)
c.Check(link, Equals, "a/b")
c.Check(s.GetFile(c, "a/b.link"), DeepEquals, []byte("test"))
}
func (s *PublishedStorageSuite) TestHardLink(c *C) {
s.PutFile(c, "a/b", []byte("test"))
err := s.storage.HardLink("a/b", "a/b.hard")
c.Check(err, IsNil)
link, err := s.storage.ReadLink("a/b.hard")
c.Check(err, IsNil)
c.Check(link, Equals, "a/b")
}
func (s *PublishedStorageSuite) TestFileExists(c *C) {
s.PutFile(c, "a/b", []byte("test"))
exists, err := s.storage.FileExists("a/b")
c.Check(err, IsNil)
c.Check(exists, Equals, true)
exists, err = s.storage.FileExists("a/b.invalid")
c.Check(err, IsNil)
c.Check(exists, Equals, false)
}
func (s *PublishedStorageSuite) TestObjectPath(c *C) {
st := &PublishedStorage{prefix: "root"}
c.Check(st.objectPath("dists/stable/Release"), Equals, filepath.Join("root", "dists/stable/Release"))
}
func (s *PublishedStorageSuite) TestLinkFromPool(c *C) {
root := c.MkDir()
pool := files.NewPackagePool(root, false)
cs := files.NewMockChecksumStorage()
tmpFile1 := filepath.Join(c.MkDir(), "mars-invaders_1.03.deb")
c.Assert(os.WriteFile(tmpFile1, []byte("Contents"), 0644), IsNil)
cksum1 := utils.ChecksumInfo{MD5: "c1df1da7a1ce305a3b60af9d5733ac1d"}
tmpFile2 := filepath.Join(c.MkDir(), "mars-invaders_1.03.deb")
c.Assert(os.WriteFile(tmpFile2, []byte("Spam"), 0644), IsNil)
cksum2 := utils.ChecksumInfo{MD5: "e9dfd31cc505d51fc26975250750deab"}
tmpFile3 := filepath.Join(c.MkDir(), "netboot/boot.img.gz")
_ = os.MkdirAll(filepath.Dir(tmpFile3), 0777)
c.Assert(os.WriteFile(tmpFile3, []byte("Contents"), 0644), IsNil)
cksum3 := utils.ChecksumInfo{MD5: "c1df1da7a1ce305a3b60af9d5733ac1d"}
src1, err := pool.Import(tmpFile1, "mars-invaders_1.03.deb", &cksum1, true, cs)
c.Assert(err, IsNil)
src2, err := pool.Import(tmpFile2, "mars-invaders_1.03.deb", &cksum2, true, cs)
c.Assert(err, IsNil)
src3, err := pool.Import(tmpFile3, "netboot/boot.img.gz", &cksum3, true, cs)
c.Assert(err, IsNil)
// first link from pool
err = s.storage.LinkFromPool("", filepath.Join("pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, src1, cksum1, false)
c.Check(err, IsNil)
c.Check(s.GetFile(c, "pool/main/m/mars-invaders/mars-invaders_1.03.deb"), DeepEquals, []byte("Contents"))
// duplicate link from pool (same MD5 → no-op)
err = s.storage.LinkFromPool("", filepath.Join("pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, src1, cksum1, false)
c.Check(err, IsNil)
c.Check(s.GetFile(c, "pool/main/m/mars-invaders/mars-invaders_1.03.deb"), DeepEquals, []byte("Contents"))
// link from pool with conflict, no force
err = s.storage.LinkFromPool("", filepath.Join("pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, src2, cksum2, false)
c.Check(err, ErrorMatches, ".*file already exists and is different.*")
c.Check(s.GetFile(c, "pool/main/m/mars-invaders/mars-invaders_1.03.deb"), DeepEquals, []byte("Contents"))
// link from pool with conflict and force
err = s.storage.LinkFromPool("", filepath.Join("pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, src2, cksum2, true)
c.Check(err, IsNil)
c.Check(s.GetFile(c, "pool/main/m/mars-invaders/mars-invaders_1.03.deb"), DeepEquals, []byte("Spam"))
// for prefixed storage:
err = s.prefixedStorage.LinkFromPool("", filepath.Join("pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, src1, cksum1, false)
c.Check(err, IsNil)
// 2nd link from pool, providing wrong path for source file:
// should hit the path cache and skip upload (which would otherwise fail).
err = s.prefixedStorage.LinkFromPool("", filepath.Join("pool", "main", "m/mars-invaders"), "mars-invaders_1.03.deb", pool, "wrong-looks-like-pathcache-doesnt-work", cksum1, false)
c.Check(err, IsNil)
c.Check(s.GetFile(c, "lala/pool/main/m/mars-invaders/mars-invaders_1.03.deb"), DeepEquals, []byte("Contents"))
// link from pool with nested file name
err = s.storage.LinkFromPool("", "dists/jessie/non-free/installer-i386/current/images", "netboot/boot.img.gz", pool, src3, cksum3, false)
c.Check(err, IsNil)
c.Check(s.GetFile(c, "dists/jessie/non-free/installer-i386/current/images/netboot/boot.img.gz"), DeepEquals, []byte("Contents"))
}
func (s *PublishedStorageSuite) TestLinkFromPoolMissingMD5(c *C) {
publishedPrefix := "repo"
publishedRelPath := "pool/main/a/aptly"
fileName := "pkg.deb"
relPath := filepath.Join(filepath.Join(publishedPrefix, publishedRelPath), fileName)
st := &PublishedStorage{pathCache: map[string]string{relPath: "0123456789abcdef0123456789abcdef"}}
err := st.LinkFromPool(publishedPrefix, publishedRelPath, fileName, nil, "", utils.ChecksumInfo{}, false)
c.Assert(err, NotNil)
c.Check(err, ErrorMatches, "unable to compare object, MD5 checksum missing")
}
func (s *PublishedStorageSuite) TestLinkFromPoolDifferentMD5NoForce(c *C) {
publishedPrefix := "repo"
publishedRelPath := "pool/main/a/aptly"
fileName := "pkg.deb"
relPath := filepath.Join(filepath.Join(publishedPrefix, publishedRelPath), fileName)
st := &PublishedStorage{pathCache: map[string]string{relPath: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}}
err := st.LinkFromPool(publishedPrefix, publishedRelPath, fileName, nil, "", utils.ChecksumInfo{MD5: "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}, false)
c.Assert(err, NotNil)
c.Check(err, ErrorMatches, ".*file already exists and is different.*")
}
func (s *PublishedStorageSuite) TestLinkFromPoolSameMD5NoUpload(c *C) {
publishedPrefix := "repo"
publishedRelPath := "pool/main/a/aptly"
fileName := "pkg.deb"
relPath := filepath.Join(filepath.Join(publishedPrefix, publishedRelPath), fileName)
st := &PublishedStorage{pathCache: map[string]string{relPath: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}}
err := st.LinkFromPool(publishedPrefix, publishedRelPath, fileName, nil, "", utils.ChecksumInfo{MD5: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}, false)
c.Check(err, IsNil)
}
// putWithMetadata uploads an object with arbitrary metadata directly via the
// storage client, bypassing the production putFile path. Used to seed objects
// that exercise getMD5 / metadata-handling branches.
func (s *PublishedStorageSuite) putWithMetadata(c *C, path string, data []byte, metadata map[string]string) {
w := s.storage.bucket.Object(path).NewWriter(context.TODO())
w.Metadata = metadata
_, err := w.Write(data)
c.Assert(err, IsNil)
c.Assert(w.Close(), IsNil)
}
// TestLinkFromPoolShortCachedMD5 exercises the LinkFromPool branch where the
// path cache holds a non-32-char checksum (so getMD5 must be called to fetch
// the real MD5 from object attrs), and along the way covers getMD5 plus the
// Md5-metadata branch in internalFilelist.
func (s *PublishedStorageSuite) TestLinkFromPoolShortCachedMD5(c *C) {
root := c.MkDir()
pool := files.NewPackagePool(root, false)
cs := files.NewMockChecksumStorage()
tmpFile := filepath.Join(c.MkDir(), "mars-invaders_1.03.deb")
c.Assert(os.WriteFile(tmpFile, []byte("Contents"), 0644), IsNil)
cksum := utils.ChecksumInfo{MD5: "c1df1da7a1ce305a3b60af9d5733ac1d"}
src, err := pool.Import(tmpFile, "mars-invaders_1.03.deb", &cksum, true, cs)
c.Assert(err, IsNil)
// Seed the bucket with a short Md5 metadata value so internalFilelist
// returns it (covering the metadata branch) and LinkFromPool then has to
// re-fetch via getMD5 because len != 32.
relPath := filepath.Join("pool/main/m/mars-invaders", "mars-invaders_1.03.deb")
s.putWithMetadata(c, relPath, []byte("Contents"), map[string]string{"Md5": "short"})
// force=true so the conflict (the seeded "short" md5 will never match
// sourceMD5 once getMD5 normalises) resolves into a fresh upload rather
// than an error — what matters here is that we traverse the getMD5 +
// short-cache + internalFilelist md5-metadata branches.
err = s.storage.LinkFromPool("", "pool/main/m/mars-invaders", "mars-invaders_1.03.deb", pool, src, cksum, true)
c.Check(err, IsNil)
}
// TestLinkFromPoolMissingSource covers the source-pool open error path.
func (s *PublishedStorageSuite) TestLinkFromPoolMissingSource(c *C) {
pool := files.NewPackagePool(c.MkDir(), false)
err := s.storage.LinkFromPool("", "pool/x", "y.deb", pool, "non-existent-pool-key", utils.ChecksumInfo{MD5: "33333333333333333333333333333333"}, false)
c.Check(err, ErrorMatches, ".*no such file or directory.*")
}
// TestPutFilePublicReadACL covers the applyACL public-read branch and the
// ACL().Set call against the fake server.
func (s *PublishedStorageSuite) TestPutFilePublicReadACL(c *C) {
st, err := NewPublishedStorage("test", "", "", "", "", "", "public-read", "", "", false, false)
c.Assert(err, IsNil)
dir := c.MkDir()
src := filepath.Join(dir, "f")
c.Assert(os.WriteFile(src, []byte("hello"), 0644), IsNil)
c.Check(st.PutFile("a/b.txt", src), IsNil)
c.Check(s.GetFile(c, "a/b.txt"), DeepEquals, []byte("hello"))
}
// TestPutFileUnsupportedACL covers the default (error) branch of applyACL
// when invoked from the production putFile flow.
func (s *PublishedStorageSuite) TestPutFileUnsupportedACL(c *C) {
st, err := NewPublishedStorage("test", "", "", "", "", "", "bucket-owner-full-control", "", "", false, false)
c.Assert(err, IsNil)
dir := c.MkDir()
src := filepath.Join(dir, "f")
c.Assert(os.WriteFile(src, []byte("hello"), 0644), IsNil)
err = st.PutFile("a/b.txt", src)
c.Assert(err, NotNil)
c.Check(err, ErrorMatches, ".*unsupported GCS ACL value.*")
}
// TestPutFileWithStorageClass covers the storageClass branch in putFile.
func (s *PublishedStorageSuite) TestPutFileWithStorageClass(c *C) {
st, err := NewPublishedStorage("test", "", "", "", "", "", "", "NEARLINE", "", false, false)
c.Assert(err, IsNil)
dir := c.MkDir()
src := filepath.Join(dir, "f")
c.Assert(os.WriteFile(src, []byte("hi"), 0644), IsNil)
c.Check(st.PutFile("a/b.txt", src), IsNil)
attrs, err := s.storage.bucket.Object("a/b.txt").Attrs(context.TODO())
c.Assert(err, IsNil)
c.Check(attrs.StorageClass, Equals, "NEARLINE")
}
// TestRemoveDirsNoSuchBucket covers the internalFilelist error path inside
// RemoveDirs (and the iterator error branch in internalFilelist itself).
func (s *PublishedStorageSuite) TestRemoveDirsNoSuchBucket(c *C) {
err := s.noSuchBucketStorage.RemoveDirs("a/b", nil)
c.Check(err, ErrorMatches, ".*error listing under prefix.*")
}
// TestFilelistNoSuchBucket also covers the iterator error path.
func (s *PublishedStorageSuite) TestFilelistNoSuchBucket(c *C) {
_, err := s.noSuchBucketStorage.Filelist("")
c.Check(err, ErrorMatches, ".*error listing under prefix.*")
}
// TestRemoveCacheEviction verifies that a successful Remove evicts the entry
// from pathCache (covers the delete(g.pathCache, ...) line).
func (s *PublishedStorageSuite) TestRemoveCacheEviction(c *C) {
s.PutFile(c, "a/b", []byte("test"))
s.storage.pathCache = map[string]string{"a/b": "deadbeefdeadbeefdeadbeefdeadbeef"}
c.Check(s.storage.Remove("a/b"), IsNil)
_, present := s.storage.pathCache["a/b"]
c.Check(present, Equals, false)
}
// TestDebugMode exercises the if-debug log branches across the main verbs.
func (s *PublishedStorageSuite) TestDebugMode(c *C) {
st, err := NewPublishedStorage("test", "", "", "", "", "", "", "", "", false, true)
c.Assert(err, IsNil)
dir := c.MkDir()
src := filepath.Join(dir, "f")
c.Assert(os.WriteFile(src, []byte("dbg"), 0644), IsNil)
c.Check(st.PutFile("d/a", src), IsNil)
c.Check(st.RenameFile("d/a", "d/b"), IsNil)
c.Check(st.SymLink("d/b", "d/b.link"), IsNil)
c.Check(st.HardLink("d/b", "d/b.hard"), IsNil)
c.Check(st.Remove("d/b"), IsNil)
c.Check(st.RemoveDirs("d", nil), IsNil)
pool := files.NewPackagePool(c.MkDir(), false)
cs := files.NewMockChecksumStorage()
tmp := filepath.Join(c.MkDir(), "x.deb")
c.Assert(os.WriteFile(tmp, []byte("Contents"), 0644), IsNil)
cksum := utils.ChecksumInfo{MD5: "c1df1da7a1ce305a3b60af9d5733ac1d"}
srcKey, err := pool.Import(tmp, "x.deb", &cksum, true, cs)
c.Assert(err, IsNil)
c.Check(st.LinkFromPool("", "pool/x", "x.deb", pool, srcKey, cksum, false), IsNil)
}
// TestObjectHandleWithEncryptionKey covers the encryptionKey branch in
// objectHandle. fsouza doesn't enforce CSEK headers but we just need to walk
// the code path.
func (s *PublishedStorageSuite) TestObjectHandleWithEncryptionKey(c *C) {
st := &PublishedStorage{
bucket: s.storage.bucket,
bucketName: "test",
encryptionKey: "0123456789abcdef0123456789abcdef",
}
c.Check(st.objectHandle("a/b"), NotNil)
}
// TestReadLinkMissing covers the Attrs-error return in ReadLink.
func (s *PublishedStorageSuite) TestReadLinkMissing(c *C) {
_, err := s.storage.ReadLink("does/not/exist")
c.Check(err, ErrorMatches, ".*object doesn't exist.*")
}
// TestFileExistsNoSuchBucket exercises FileExists' wrapped-googleapi-404 path.
func (s *PublishedStorageSuite) TestFileExistsNoSuchBucket(c *C) {
exists, err := s.noSuchBucketStorage.FileExists("a/b")
c.Check(err, IsNil)
c.Check(exists, Equals, false)
}
// TestNewPublishedStorageWithEndpoint exercises the endpoint-injection branch
// in NewPublishedStorage (the production knob, separate from the env-var path).
func (s *PublishedStorageSuite) TestNewPublishedStorageWithEndpoint(c *C) {
saved := os.Getenv("STORAGE_EMULATOR_HOST")
c.Assert(os.Unsetenv("STORAGE_EMULATOR_HOST"), IsNil)
defer func() { _ = os.Setenv("STORAGE_EMULATOR_HOST", saved) }()
st, err := NewPublishedStorage("test", "", "", "", "", s.srv.URL()+"/storage/v1/", "", "", "", false, false)
c.Assert(err, IsNil)
_, err = st.Filelist("")
c.Check(err, IsNil)
}
// TestNewPublishedStorageWithProject covers the project!="" → WithQuotaProject
// branch. WithQuotaProject is incompatible with WithEndpoint, so this test
// relies on the STORAGE_EMULATOR_HOST env var (still set from SetUpTest) for
// fake-server routing.
func (s *PublishedStorageSuite) TestNewPublishedStorageWithProject(c *C) {
st, err := NewPublishedStorage("test", "", "", "", "fake-project", "", "", "", "", false, false)
c.Assert(err, IsNil)
_, err = st.Filelist("")
c.Check(err, IsNil)
}
+110 -34
View File
@@ -1,6 +1,6 @@
module github.com/aptly-dev/aptly
go 1.24
go 1.25.0
require (
github.com/AlekSi/pointer v1.1.0
@@ -13,8 +13,8 @@ require (
github.com/h2non/filetype v1.1.3
github.com/jlaffaye/ftp v0.2.0 // indirect
github.com/kjk/lzma v0.0.0-20120628231508-2a7c55cad4a2
github.com/klauspost/compress v1.17.9
github.com/klauspost/pgzip v1.2.5
github.com/klauspost/compress v1.18.2
github.com/klauspost/pgzip v1.2.6
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/mattn/go-shellwords v1.0.12
@@ -32,29 +32,44 @@ require (
github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d
github.com/ugorji/go/codec v1.2.11
github.com/wsxiaoys/terminal v0.0.0-20160513160801-0940f3fc43a0
golang.org/x/crypto v0.36.0 // indirect
golang.org/x/sys v0.31.0
golang.org/x/term v0.30.0
golang.org/x/time v0.5.0
google.golang.org/protobuf v1.34.2 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/sys v0.43.0
golang.org/x/term v0.42.0
golang.org/x/time v0.14.0
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
)
require (
github.com/Azure/azure-pipeline-go v0.2.3 // indirect
cel.dev/expr v0.25.1 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.18.1 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/iam v1.5.3 // indirect
cloud.google.com/go/monitoring v1.24.3 // indirect
cloud.google.com/go/pubsub/v2 v2.4.0 // indirect
dario.cat/mergo v1.0.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
github.com/CycloneDX/cyclonedx-go v0.9.2 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.55.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.55.0 // indirect
github.com/KyleBanks/depth v1.2.1 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/PuerkitoBio/purell v1.1.1 // indirect
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7 // indirect
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.20 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.24 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.24 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.24 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.5 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.22 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.13 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.21 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.21 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.6 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.5 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.33.1 // indirect
@@ -62,12 +77,26 @@ require (
github.com/bytedance/sonic v1.9.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 // indirect
github.com/cloudflare/circl v1.4.0 // indirect
github.com/cloudflare/circl v1.6.3 // indirect
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 // indirect
github.com/coreos/go-semver v0.3.0 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/cyphar/filepath-securejoin v0.6.1 // indirect
github.com/dsnet/compress v0.0.1 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/envoyproxy/go-control-plane/envoy v1.36.0 // indirect
github.com/envoyproxy/protoc-gen-validate v1.3.0 // indirect
github.com/fatih/color v1.17.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/forPelevin/gomoji v1.3.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.2 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.9.0 // indirect
github.com/go-git/go-git/v5 v5.19.1 // indirect
github.com/go-jose/go-jose/v4 v4.1.4 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.19.6 // indirect
github.com/go-openapi/spec v0.20.4 // indirect
@@ -76,55 +105,102 @@ require (
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/renameio/v2 v2.0.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.11 // indirect
github.com/googleapis/gax-go/v2 v2.17.0 // indirect
github.com/gookit/color v1.5.4 // indirect
github.com/gorilla/handlers v1.5.2 // indirect
github.com/gorilla/mux v1.8.1 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/jfrog/archiver/v3 v3.6.1 // indirect
github.com/jfrog/build-info-go v1.11.0 // indirect
github.com/jfrog/gofrog v1.7.6 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.4 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/leodido/go-urn v1.2.4 // indirect
github.com/mailru/easyjson v0.7.6 // indirect
github.com/mattn/go-ieproxy v0.0.1 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/minio/sha256-simd v1.0.1 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/nwaples/rardecode v1.1.3 // indirect
github.com/pelletier/go-toml/v2 v2.1.0 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect
github.com/pkg/xattr v0.4.12 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.59.1 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/rogpeppe/go-internal v1.12.0 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/spiffe/go-spiffe/v2 v2.6.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
go.etcd.io/etcd/api/v3 v3.5.15 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.5.15 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.39.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/sdk v1.43.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.uber.org/multierr v1.10.0 // indirect
go.uber.org/zap v1.26.0 // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/net v0.38.0 // indirect
golang.org/x/sync v0.12.0 // indirect
golang.org/x/text v0.23.0 // indirect
golang.org/x/tools v0.30.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240318140521-94a12d6c2237 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237 // indirect
google.golang.org/grpc v1.64.1 // indirect
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
golang.org/x/net v0.53.0 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/text v0.36.0 // indirect
golang.org/x/tools v0.44.0 // indirect
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260203192932-546029d2fa20 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 // indirect
google.golang.org/grpc v1.79.3 // indirect
gopkg.in/cheggaaa/pb.v1 v1.0.28 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
)
require (
github.com/Azure/azure-storage-blob-go v0.15.0
github.com/ProtonMail/go-crypto v1.0.0
github.com/aws/aws-sdk-go-v2 v1.32.5
cloud.google.com/go/storage v1.60.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1
github.com/ProtonMail/go-crypto v1.4.0
github.com/aws/aws-sdk-go-v2 v1.41.5
github.com/aws/aws-sdk-go-v2/config v1.28.5
github.com/aws/aws-sdk-go-v2/credentials v1.17.46
github.com/aws/aws-sdk-go-v2/service/s3 v1.67.1
github.com/aws/smithy-go v1.22.1
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.3
github.com/aws/smithy-go v1.24.2
github.com/fsouza/fake-gcs-server v1.53.1
github.com/google/uuid v1.6.0
github.com/jfrog/jfrog-client-go v1.55.0
github.com/swaggo/files v1.0.1
github.com/swaggo/gin-swagger v1.6.0
github.com/swaggo/swag v1.16.3
go.etcd.io/etcd/client/v3 v3.5.15
golang.org/x/oauth2 v0.35.0
google.golang.org/api v0.266.0
gopkg.in/yaml.v3 v3.0.1
)
+374 -114
View File
@@ -1,76 +1,122 @@
cel.dev/expr v0.25.1 h1:1KrZg61W6TWSxuNZ37Xy49ps13NUovb66QLprthtwi4=
cel.dev/expr v0.25.1/go.mod h1:hrXvqGP6G6gyx8UAHSHJ5RGk//1Oj5nXQ2NI02Nrsg4=
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.18.1 h1:IwTEx92GFUo2pJ6Qea0EU3zYvKnTAeRCODxfA/G5UWs=
cloud.google.com/go/auth v0.18.1/go.mod h1:GfTYoS9G3CWpRA3Va9doKN9mjPGRS+v41jmZAhBzbrA=
cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=
cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
cloud.google.com/go/iam v1.5.3 h1:+vMINPiDF2ognBJ97ABAYYwRgsaqxPbQDlMnbHMjolc=
cloud.google.com/go/iam v1.5.3/go.mod h1:MR3v9oLkZCTlaqljW6Eb2d3HGDGK5/bDv93jhfISFvU=
cloud.google.com/go/logging v1.13.1 h1:O7LvmO0kGLaHY/gq8cV7T0dyp6zJhYAOtZPX4TF3QtY=
cloud.google.com/go/logging v1.13.1/go.mod h1:XAQkfkMBxQRjQek96WLPNze7vsOmay9H5PqfsNYDqvw=
cloud.google.com/go/longrunning v0.8.0 h1:LiKK77J3bx5gDLi4SMViHixjD2ohlkwBi+mKA7EhfW8=
cloud.google.com/go/longrunning v0.8.0/go.mod h1:UmErU2Onzi+fKDg2gR7dusz11Pe26aknR4kHmJJqIfk=
cloud.google.com/go/monitoring v1.24.3 h1:dde+gMNc0UhPZD1Azu6at2e79bfdztVDS5lvhOdsgaE=
cloud.google.com/go/monitoring v1.24.3/go.mod h1:nYP6W0tm3N9H/bOw8am7t62YTzZY+zUeQ+Bi6+2eonI=
cloud.google.com/go/pubsub/v2 v2.4.0 h1:oMKNiBQpXImRWnHYla9uSU66ZzByZwBSCJOEs/pTKVg=
cloud.google.com/go/pubsub/v2 v2.4.0/go.mod h1:2lS/XQKq5qtOMs6kHBK+WX1ytUC36kLl2ig3zqsGUx8=
cloud.google.com/go/storage v1.60.0 h1:oBfZrSOCimggVNz9Y/bXY35uUcts7OViubeddTTVzQ8=
cloud.google.com/go/storage v1.60.0/go.mod h1:q+5196hXfejkctrnx+VYU8RKQr/L3c0cBIlrjmiAKE0=
cloud.google.com/go/trace v1.11.7 h1:kDNDX8JkaAG3R2nq1lIdkb7FCSi1rCmsEtKVsty7p+U=
cloud.google.com/go/trace v1.11.7/go.mod h1:TNn9d5V3fQVf6s4SCveVMIBS2LJUqo73GACmq/Tky0s=
dario.cat/mergo v1.0.1 h1:Ra4+bf83h2ztPIQYNP99R6m+Y7KfnARDfID+a+vLl4s=
dario.cat/mergo v1.0.1/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
github.com/AlekSi/pointer v1.1.0 h1:SSDMPcXD9jSl8FPy9cRzoRaMJtm9g9ggGTxecRUbQoI=
github.com/AlekSi/pointer v1.1.0/go.mod h1:y7BvfRI3wXPWKXEBhU71nbnIEEZX0QTSB2Bj48UJIZE=
github.com/Azure/azure-pipeline-go v0.2.3 h1:7U9HBg1JFK3jHl5qmo4CTZKFTVgMwdFHMVtCdfBE21U=
github.com/Azure/azure-pipeline-go v0.2.3/go.mod h1:x841ezTBIMG6O3lAcl8ATHnsOPVl2bqk7S3ta6S6u4k=
github.com/Azure/azure-storage-blob-go v0.15.0 h1:rXtgp8tN1p29GvpGgfJetavIG0V7OgcSXPpwp3tx6qk=
github.com/Azure/azure-storage-blob-go v0.15.0/go.mod h1:vbjsVbX0dlxnRc4FFMPsS9BsJWPcne7GB7onqlPvz58=
github.com/Azure/go-autorest v14.2.0+incompatible h1:V5VMDjClD3GiElqLWO7mz2MxNAK/vTfRHdAubSIPRgs=
github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
github.com/Azure/go-autorest/autorest/adal v0.9.13 h1:Mp5hbtOePIzM8pJVRa3YLrWWmZtoxRXqUEzCfJt3+/Q=
github.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M=
github.com/Azure/go-autorest/autorest/date v0.3.0 h1:7gUk1U5M/CQbp9WoqinNzJar+8KY+LPI6wiWrP/myHw=
github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74=
github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k=
github.com/Azure/go-autorest/logger v0.2.1 h1:IG7i4p/mDa2Ce4TRyAO8IHnVhAVF3RFU+ZtXWSmf4Tg=
github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8=
github.com/Azure/go-autorest/tracing v0.6.0 h1:TYi4+3m5t6K48TGI9AUdb+IzbnSxvnvUMfuitfgcfuo=
github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0 h1:nyQWyZvwGTvunIMxi1Y9uXkcyr+I7TeNrr/foo4Kpk8=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0/go.mod h1:l38EPgmsp71HHLq9j7De57JcKOWPyhrsW1Awm1JS6K0=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0 h1:tfLQ34V6F7tVSwoTf/4lH5sE0o6eCJuNDTmH09nDpbc=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0/go.mod h1:9kIvujWAA58nmPmWB1m23fyWic1kYZMxD9CxaWn4Qpg=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.6.0 h1:PiSrjRPpkQNjrM8H0WwKMnZUdu1RGMtd/LdGKUrOo+c=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.6.0/go.mod h1:oDrbWx4ewMylP7xHivfgixbfGBT6APAwsSoHRKotnIc=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1 h1:cf+OIKbkmMHBaC3u78AXomweqM0oxQSgBXRZf3WH4yM=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.1/go.mod h1:ap1dmS6vQKJxSMNiGJcq4QuUQkOynyD93gLw6MDF7ek=
github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 h1:XHOnouVk1mxXfQidrMEnLlPk9UMeRtyBTnEFtxkV0kU=
github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/CycloneDX/cyclonedx-go v0.9.2 h1:688QHn2X/5nRezKe2ueIVCt+NRqf7fl3AVQk+vaFcIo=
github.com/CycloneDX/cyclonedx-go v0.9.2/go.mod h1:vcK6pKgO1WanCdd61qx4bFnSsDJQ6SbM2ZuMIgq86Jg=
github.com/DisposaBoy/JsonConfigReader v0.0.0-20171218180944-5ea4d0ddac55 h1:jbGlDKdzAZ92NzK65hUP98ri0/r50vVVvmZsFP/nIqo=
github.com/DisposaBoy/JsonConfigReader v0.0.0-20171218180944-5ea4d0ddac55/go.mod h1:GCzqZQHydohgVLSIqRKZeTt8IGb1Y4NaFfim3H40uUI=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0 h1:sBEjpZlNHzK1voKq9695PJSX2o5NEXl7/OL3coiIY0c=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0/go.mod h1:P4WPRUkOhJC13W//jWpyfJNDAIpvRbAUIYLX/4jtlE0=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.55.0 h1:UnDZ/zFfG1JhH/DqxIZYU/1CUAlTUScoXD/LcM2Ykk8=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.55.0/go.mod h1:IA1C1U7jO/ENqm/vhi7V9YYpBsp+IMyqNrEN94N7tVc=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.55.0 h1:7t/qx5Ost0s0wbA/VDrByOooURhp+ikYwv20i9Y07TQ=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.55.0/go.mod h1:vB2GH9GAYYJTO3mEn8oYwzEdhlayZIdQz6zdzgUIRvA=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.55.0 h1:0s6TxfCu2KHkkZPnBfsQ2y5qia0jl3MMrmBhu3nCOYk=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.55.0/go.mod h1:Mf6O40IAyB9zR/1J8nGDDPirZQQPbYJni8Yisy7NTMc=
github.com/KyleBanks/depth v1.2.1 h1:5h8fQADFrWtarTdtDudMmGsC7GPbOAu6RVB3ffsVFHc=
github.com/KyleBanks/depth v1.2.1/go.mod h1:jzSb9d0L43HxTQfT+oSA1EEp2q+ne2uh6XgeJcm8brE=
github.com/ProtonMail/go-crypto v1.0.0 h1:LRuvITjQWX+WIfr930YHG2HNfjR1uOfyf5vE0kC2U78=
github.com/ProtonMail/go-crypto v1.0.0/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/ProtonMail/go-crypto v1.4.0 h1:Zq/pbM3F5DFgJiMouxEdSVY44MVoQNEKp5d5QxIQceQ=
github.com/ProtonMail/go-crypto v1.4.0/go.mod h1:e1OaTyu5SYVrO9gKOEhTc+5UcXtTUa+P3uLudwcgPqo=
github.com/PuerkitoBio/purell v1.1.1 h1:WEQqlqaGbrPkxLJWfBwQmfEAE1Z7ONdDLqrN38tNFfI=
github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 h1:d+Bc7a5rLufV/sSk/8dngufqelfh6jnri85riMAaF/M=
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/awalterschulze/gographviz v2.0.1+incompatible h1:XIECBRq9VPEQqkQL5pw2OtjCAdrtIgFKoJU8eT98AS8=
github.com/awalterschulze/gographviz v2.0.1+incompatible/go.mod h1:GEV5wmg4YquNw7v1kkyoX9etIk8yVmXj+AkDHuuETHs=
github.com/aws/aws-sdk-go-v2 v1.32.5 h1:U8vdWJuY7ruAkzaOdD7guwJjD06YSKmnKCJs7s3IkIo=
github.com/aws/aws-sdk-go-v2 v1.32.5/go.mod h1:P5WJBrYqqbWVaOxgH0X/FYYD47/nooaPOZPlQdmiN2U=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7 h1:lL7IfaFzngfx0ZwUGOZdsFFnQ5uLvR0hWqqhyE7Q9M8=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.7/go.mod h1:QraP0UcVlQJsmHfioCrveWOC1nbiWUl3ej08h4mXWoc=
github.com/aws/aws-sdk-go-v2 v1.41.5 h1:dj5kopbwUsVUVFgO4Fi5BIT3t4WyqIDjGKCangnV/yY=
github.com/aws/aws-sdk-go-v2 v1.41.5/go.mod h1:mwsPRE8ceUUpiTgF7QmQIJ7lgsKUPQOUl3o72QBrE1o=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 h1:eBMB84YGghSocM7PsjmmPffTa+1FBUeNvGvFou6V/4o=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8/go.mod h1:lyw7GFp3qENLh7kwzf7iMzAxDn+NzjXEAGjKS2UOKqI=
github.com/aws/aws-sdk-go-v2/config v1.28.5 h1:Za41twdCXbuyyWv9LndXxZZv3QhTG1DinqlFsSuvtI0=
github.com/aws/aws-sdk-go-v2/config v1.28.5/go.mod h1:4VsPbHP8JdcdUDmbTVgNL/8w9SqOkM5jyY8ljIxLO3o=
github.com/aws/aws-sdk-go-v2/credentials v1.17.46 h1:AU7RcriIo2lXjUfHFnFKYsLCwgbz1E7Mm95ieIRDNUg=
github.com/aws/aws-sdk-go-v2/credentials v1.17.46/go.mod h1:1FmYyLGL08KQXQ6mcTlifyFXfJVCNJTVGuQP4m0d/UA=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.20 h1:sDSXIrlsFSFJtWKLQS4PUWRvrT580rrnuLydJrCQ/yA=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.20/go.mod h1:WZ/c+w0ofps+/OUqMwWgnfrgzZH1DZO1RIkktICsqnY=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.24 h1:4usbeaes3yJnCFC7kfeyhkdkPtoRYPa/hTmCqMpKpLI=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.24/go.mod h1:5CI1JemjVwde8m2WG3cz23qHKPOxbpkq0HaoreEgLIY=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.24 h1:N1zsICrQglfzaBnrfM0Ys00860C+QFwu6u/5+LomP+o=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.24/go.mod h1:dCn9HbJ8+K31i8IQ8EWmWj0EiIk0+vKiHNMxTTYveAg=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.21 h1:Rgg6wvjjtX8bNHcvi9OnXWwcE0a2vGpbwmtICOsvcf4=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.21/go.mod h1:A/kJFst/nm//cyqonihbdpQZwiUhhzpqTsdbhDdRF9c=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.21 h1:PEgGVtPoB6NTpPrBgqSE5hE/o47Ij9qk/SEZFbUOe9A=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.21/go.mod h1:p+hz+PRAYlY3zcpJhPwXlLC4C+kqn70WIHwnzAfs6ps=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 h1:VaRN3TlFdd6KxX1x3ILT5ynH6HvKgqdiXoTxAF4HQcQ=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1/go.mod h1:FbtygfRFze9usAadmnGJNc8KsP346kEe+y2/oyhGAGc=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.24 h1:JX70yGKLj25+lMC5Yyh8wBtvB01GDilyRuJvXJ4piD0=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.24/go.mod h1:+Ln60j9SUTD0LEwnhEB0Xhg61DHqplBrbZpLgyjoEHg=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 h1:iXtILhvDxB6kPvEXgsDhGaZCSC6LQET5ZHSdJozeI0Y=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1/go.mod h1:9nu0fVANtYiAePIBh2/pFUSwtJ402hLnp854CNoDOeE=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.5 h1:gvZOjQKPxFXy1ft3QnEyXmT+IqneM9QAUWlM3r0mfqw=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.5/go.mod h1:DLWnfvIcm9IET/mmjdxeXbBKmTCm0ZB8p1za9BVteM8=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.5 h1:wtpJ4zcwrSbwhECWQoI/g6WM9zqCcSpHDJIWSbMLOu4=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.5/go.mod h1:qu/W9HXQbbQ4+1+JcZp0ZNPV31ym537ZJN+fiS7Ti8E=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.5 h1:P1doBzv5VEg1ONxnJss1Kh5ZG/ewoIE4MQtKKc6Crgg=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.5/go.mod h1:NOP+euMW7W3Ukt28tAxPuoWao4rhhqJD3QEBk7oCg7w=
github.com/aws/aws-sdk-go-v2/service/s3 v1.67.1 h1:LXLnDfjT/P6SPIaCE86xCOjJROPn4FNB2EdN68vMK5c=
github.com/aws/aws-sdk-go-v2/service/s3 v1.67.1/go.mod h1:ralv4XawHjEMaHOWnTFushl0WRqim/gQWesAMF6hTow=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.22 h1:rWyie/PxDRIdhNf4DzRk0lvjVOqFJuNnO8WwaIRVxzQ=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.22/go.mod h1:zd/JsJ4P7oGfUhXn1VyLqaRZwPmZwg44Jf2dS84Dm3Y=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 h1:5EniKhLZe4xzL7a+fU3C2tfUN4nWIqlLesfrjkuPFTY=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7/go.mod h1:x0nZssQ3qZSnIcePWLvcoFisRXJzcTVvYpAAdYX8+GI=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.13 h1:JRaIgADQS/U6uXDqlPiefP32yXTda7Kqfx+LgspooZM=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.13/go.mod h1:CEuVn5WqOMilYl+tbccq8+N2ieCy0gVn3OtRb0vBNNM=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.21 h1:c31//R3xgIJMSC8S6hEVq+38DcvUlgFY0FM6mSI5oto=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.21/go.mod h1:r6+pf23ouCB718FUxaqzZdbpYFyDtehyZcmP5KL9FkA=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.21 h1:ZlvrNcHSFFWURB8avufQq9gFsheUgjVD9536obIknfM=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.21/go.mod h1:cv3TNhVrssKR0O/xxLJVRfd2oazSnZnkUeTf6ctUwfQ=
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.3 h1:HwxWTbTrIHm5qY+CAEur0s/figc3qwvLWsNkF4RPToo=
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.3/go.mod h1:uoA43SdFwacedBfSgfFSjjCvYe8aYBS7EnU5GZ/YKMM=
github.com/aws/aws-sdk-go-v2/service/sso v1.24.6 h1:3zu537oLmsPfDMyjnUS2g+F2vITgy5pB74tHI+JBNoM=
github.com/aws/aws-sdk-go-v2/service/sso v1.24.6/go.mod h1:WJSZH2ZvepM6t6jwu4w/Z45Eoi75lPN7DcydSRtJg6Y=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.5 h1:K0OQAsDywb0ltlFrZm0JHPY3yZp/S9OaoLU33S7vPS8=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.5/go.mod h1:ORITg+fyuMoeiQFiVGoqB3OydVTLkClw/ljbblMq6Cc=
github.com/aws/aws-sdk-go-v2/service/sts v1.33.1 h1:6SZUVRQNvExYlMLbHdlKB48x0fLbc2iVROyaNEwBHbU=
github.com/aws/aws-sdk-go-v2/service/sts v1.33.1/go.mod h1:GqWyYCwLXnlUB1lOAXQyNSPqPLQJvmo8J0DWBzp9mtg=
github.com/aws/smithy-go v1.22.1 h1:/HPHZQ0g7f4eUeK6HKglFz8uwVfZKgoI25rb/J+dnro=
github.com/aws/smithy-go v1.22.1/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
github.com/aws/smithy-go v1.24.2 h1:FzA3bu/nt/vDvmnkg+R8Xl46gmzEDam6mZ1hzmwXFng=
github.com/aws/smithy-go v1.24.2/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/bradleyjkemp/cupaloy/v2 v2.8.0 h1:any4BmKE+jGIaMpnU8YgH/I2LPiLBufr6oMMlVBbn9M=
github.com/bradleyjkemp/cupaloy/v2 v2.8.0/go.mod h1:bm7JXdkRd4BHJk9HpwqAI8BoAY1lps46Enkdqw6aRX0=
github.com/bytedance/sonic v1.5.0/go.mod h1:ED5hyg4y6t3/9Ku1R6dU/4KyJ48DZ4jPhfY1O2AihPM=
github.com/bytedance/sonic v1.9.1 h1:6iJ6NqdoxCDr6mbY8h18oSO+cShGSMRGCEo7F2h0x8s=
github.com/bytedance/sonic v1.9.1/go.mod h1:i736AoUSYt75HyZLoJW9ERYxcy6eaN6h4BZXU064P/U=
github.com/cavaliergopher/grab/v3 v3.0.1 h1:4z7TkBfmPjmLAAmkkAZNX/6QJ1nNFdv3SdIHXju0Fr4=
github.com/cavaliergopher/grab/v3 v3.0.1/go.mod h1:1U/KNnD+Ft6JJiYoYBAimKH2XrYptb8Kl3DFGmsjpq4=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cheggaaa/pb v1.0.25 h1:tFpebHTkI7QZx1q1rWGOKhbunhZ3fMaxTvHDWn1bH/4=
@@ -81,31 +127,83 @@ github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311/go.mod h1:b583j
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA=
github.com/cloudflare/circl v1.4.0 h1:BV7h5MgrktNzytKmWjpOtdYrf0lkkbF8YMlBGPhJQrY=
github.com/cloudflare/circl v1.4.0/go.mod h1:PDRU+oXvdD7KCtgKxW95M5Z8BpSCJXQORiZFnBQS5QU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/circl v1.6.3 h1:9GPOhQGF9MCYUeXyMYlqTR6a5gTrgR/fBLXvUgtVcg8=
github.com/cloudflare/circl v1.6.3/go.mod h1:2eXP6Qfat4O/Yhh8BznvKnJ+uzEoTQ6jVKJRn81BiS4=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 h1:6xNmx7iTtyBRev0+D/Tv1FZd4SCg8axKApyNyRsAt/w=
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5/go.mod h1:KdCmV+x/BuvyMxRnYBlmVaq4OLiKW6iRQfvC62cvdkI=
github.com/coreos/go-semver v0.3.0 h1:wkHLiw0WNATZnSG7epLsujiMCgPAc9xhjJ4tgnAxmfM=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/cyphar/filepath-securejoin v0.6.1 h1:5CeZ1jPXEiYt3+Z6zqprSAgSWiggmpVyciv8syjIpVE=
github.com/cyphar/filepath-securejoin v0.6.1/go.mod h1:A8hd4EnAeyujCJRrICiOWqjS1AX0a9kM5XL+NwKoYSc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dsnet/compress v0.0.1 h1:PlZu0n3Tuv04TzpfPbrnI0HW/YwodEXDS+oPKahKF0Q=
github.com/dsnet/compress v0.0.1/go.mod h1:Aw8dCMJ7RioblQeTqt88akK31OvO8Dhf5JflhBbQEHo=
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/elazarl/goproxy v1.7.2 h1:Y2o6urb7Eule09PjlhQRGNsqRfPmYI3KKQLFpCAV3+o=
github.com/elazarl/goproxy v1.7.2/go.mod h1:82vkLNir0ALaW14Rc399OTTjyNREgmdL2cVoIbS6XaE=
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.14.0 h1:hbG2kr4RuFj222B6+7T83thSPqLjwBIfQawTkC++2HA=
github.com/envoyproxy/go-control-plane v0.14.0/go.mod h1:NcS5X47pLl/hfqxU70yPwL9ZMkUlwlKxtAohpi2wBEU=
github.com/envoyproxy/go-control-plane/envoy v1.36.0 h1:yg/JjO5E7ubRyKX3m07GF3reDNEnfOboJ0QySbH736g=
github.com/envoyproxy/go-control-plane/envoy v1.36.0/go.mod h1:ty89S1YCCVruQAm9OtKeEkQLTb+Lkz0k8v9W0Oxsv98=
github.com/envoyproxy/go-control-plane/ratelimit v0.1.0 h1:/G9QYbddjL25KvtKTv3an9lx6VBE2cnb8wp1vEGNYGI=
github.com/envoyproxy/go-control-plane/ratelimit v0.1.0/go.mod h1:Wk+tMFAFbCXaJPzVVHnPgRKdUdwW/KdbRt94AzgRee4=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/envoyproxy/protoc-gen-validate v1.3.0 h1:TvGH1wof4H33rezVKWSpqKz5NXWg5VPuZ0uONDT6eb4=
github.com/envoyproxy/protoc-gen-validate v1.3.0/go.mod h1:HvYl7zwPa5mffgyeTUHA9zHIH36nmrm7oCbo4YKoSWA=
github.com/fatih/color v1.17.0 h1:GlRw1BRJxkpqUCBKzKOw098ed57fEsKeNjpTe3cSjK4=
github.com/fatih/color v1.17.0/go.mod h1:YZ7TlrGPkiz6ku9fK3TLD/pl3CpsiFyu8N92HLgmosI=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible h1:TcekIExNqud5crz4xD2pavyTgWiPvpYe4Xau31I0PRk=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/forPelevin/gomoji v1.3.0 h1:WPIOLWB1bvRYlKZnSSEevLt3IfKlLs+tK+YA9fFYlkE=
github.com/forPelevin/gomoji v1.3.0/go.mod h1:mM6GtmCgpoQP2usDArc6GjbXrti5+FffolyQfGgPboQ=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI=
github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU=
github.com/fsouza/fake-gcs-server v1.53.1 h1:/gjEYut23/MMhe4daYJ5yIBGPUmLAYupgITuoWG3+jI=
github.com/fsouza/fake-gcs-server v1.53.1/go.mod h1:kF+DadfinC7mlc1/2d/ZDHS9VyUk1hTcXJ6VwLSlzfM=
github.com/gabriel-vasile/mimetype v1.4.2 h1:w5qFW6JKBz9Y393Y4q372O9A7cUSequkh1Q7OhCmWKU=
github.com/gabriel-vasile/mimetype v1.4.2/go.mod h1:zApsH/mKG4w07erKIaJPFiX0Tsq9BFQgN3qGY5GnNgA=
github.com/gin-contrib/gzip v0.0.6 h1:NjcunTcGAj5CO1gn4N8jHOSIeRFHIbn51z6K+xaN4d4=
github.com/gin-contrib/gzip v0.0.6/go.mod h1:QOJlmV2xmayAjkNS2Y8NQsMneuRShOU/kjovCXNuzzk=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
github.com/gin-gonic/gin v1.9.1 h1:4idEAncQnU5cB7BeOkPtxjfCSye0AAm1R0RVIqJ+Jmg=
github.com/gin-gonic/gin v1.9.1/go.mod h1:hPrL7YrpYKXt5YId3A/Tnip5kqbEAP+KLuI3SUcPTeU=
github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
github.com/go-git/go-billy/v5 v5.9.0 h1:jItGXszUDRtR/AlferWPTMN4j38BQ88XnXKbilmmBPA=
github.com/go-git/go-billy/v5 v5.9.0/go.mod h1:jCnQMLj9eUgGU7+ludSTYoZL/GGmii14RxKFj7ROgHw=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII=
github.com/go-git/go-git/v5 v5.19.1 h1:nX27AnaU43/K5bKktKwgBmR9lawoYVe1Ckg0rgzzN00=
github.com/go-git/go-git/v5 v5.19.1/go.mod h1:Pb1v0c7/g8aGQJwx9Us09W85yGoyvSwuhEGMH7zjDKQ=
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=
github.com/go-jose/go-jose/v4 v4.1.4/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY=
github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
@@ -130,30 +228,61 @@ github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MG
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v4 v4.5.2 h1:YtQM7lnr8iZ+j5q71MGKkNw9Mn7AjHM68uc9g5fXeUI=
github.com/golang-jwt/jwt/v4 v4.5.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk=
github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 h1:f+oWsMOmNPc8JmEHVZIycC7hBoQxHH9pNKQORJNozsQ=
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8/go.mod h1:wcDNUvekVysuuOpQKo3191zZyTpiI6se1N1ULghS0sw=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian/v3 v3.3.3 h1:DIhPTQrbPkgs2yJYdXU/eNACCG5DVQjySNRNlflZ9Fc=
github.com/google/martian/v3 v3.3.3/go.mod h1:iEPrYcgCF7jA9OtScMFQyAlZZ4YXTKEtJ1E6RWzmBA0=
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/renameio/v2 v2.0.0 h1:UifI23ZTGY8Tt29JbYFiuyIU3eX+RNFtUwefq9qAhxg=
github.com/google/renameio/v2 v2.0.0/go.mod h1:BtmJXm5YlszgC+TD4HOEEUFgkJP3nLxehU6hfe7jRt4=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.11 h1:vAe81Msw+8tKUxi2Dqh/NZMz7475yUvmRIkXr4oN2ao=
github.com/googleapis/enterprise-certificate-proxy v0.3.11/go.mod h1:RFV7MUdlb7AgEq2v7FmMCfeSMCllAzWxFgRdusoGks8=
github.com/googleapis/gax-go/v2 v2.17.0 h1:RksgfBpxqff0EZkDWYuz9q/uWsTVz+kf43LsZ1J6SMc=
github.com/googleapis/gax-go/v2 v2.17.0/go.mod h1:mzaqghpQp4JDh3HvADwrat+6M3MOIDp5YKHhb9PAgDY=
github.com/gookit/color v1.5.4 h1:FZmqs7XOyGgCAxmWyPslpiok1k05wmY3SJTytgvYFs0=
github.com/gookit/color v1.5.4/go.mod h1:pZJOeOS8DM43rXbp4AZo1n9zCU2qjpcRko0b6/QJi9w=
github.com/gorilla/handlers v1.5.2 h1:cLTUSsNkgcwhgRqvCNmdbRWG0A3N4F+M2nWKdScwyEE=
github.com/gorilla/handlers v1.5.2/go.mod h1:dX+xVpaxdSw+q0Qek8SSsl3dfMk3jNddUkMzo0GtH0w=
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
github.com/h2non/filetype v1.1.3 h1:FKkx9QbD7HR/zjK1Ia5XiBsq9zdLi5Kf3zGyFTAFkGg=
github.com/h2non/filetype v1.1.3/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
@@ -163,23 +292,39 @@ github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+l
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
github.com/jfrog/archiver/v3 v3.6.1 h1:LOxnkw9pOn45DzCbZNFV6K0+6dCsQ0L8mR3ZcujO5eI=
github.com/jfrog/archiver/v3 v3.6.1/go.mod h1:VgR+3WZS4N+i9FaDwLZbq+jeU4B4zctXL+gL4EMzfLw=
github.com/jfrog/build-info-go v1.11.0 h1:qEONCgaHKlW3e2y0zIwTZVbgS/ERZrPlBWEbOYJbaSU=
github.com/jfrog/build-info-go v1.11.0/go.mod h1:szdz9+WzB7+7PGnILLUgyY+OF5qD5geBT7UGNIxibyw=
github.com/jfrog/gofrog v1.7.6 h1:QmfAiRzVyaI7JYGsB7cxfAJePAZTzFz0gRWZSE27c6s=
github.com/jfrog/gofrog v1.7.6/go.mod h1:ntr1txqNOZtHplmaNd7rS4f8jpA5Apx8em70oYEe7+4=
github.com/jfrog/jfrog-client-go v1.55.0 h1:dZq7sLjUJMps8X1I5coVUChprtR7xklp7oSfmZnI48w=
github.com/jfrog/jfrog-client-go v1.55.0/go.mod h1:/e2kaF1oZTmSRgMIk7wYna5xMtNY7Xk8ahpSNZQ2d3s=
github.com/jlaffaye/ftp v0.2.0 h1:lXNvW7cBu7R/68bknOX3MrRIIqZ61zELs1P2RAiA3lg=
github.com/jlaffaye/ftp v0.2.0/go.mod h1:is2Ds5qkhceAPy2xD6RLI6hmp/qysSoymZ+Z2uTnspI=
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kjk/lzma v0.0.0-20120628231508-2a7c55cad4a2 h1:TVZQgMi+I83S3rCuE65HnmDO6+wFPRi3n2LOzr+tr68=
github.com/kjk/lzma v0.0.0-20120628231508-2a7c55cad4a2/go.mod h1:phT/jsRPBAEqjAibu1BurrabCBNTYiVI+zbmyCZJY6Q=
github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA=
github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.2.4 h1:acbojRNwl3o09bUq+yDCtZFc1aiwaAAxtcn8YkZXnvk=
github.com/klauspost/cpuid/v2 v2.2.4/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
github.com/klauspost/pgzip v1.2.5 h1:qnWYvvKqedOF2ulHpMG72XQol4ILEJ8k2wwRl/Km8oE=
github.com/klauspost/pgzip v1.2.5/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/crc32 v1.3.0 h1:sSmTt3gUt81RP655XGZPElI0PelVTZ6YwCRnPSupoFM=
github.com/klauspost/crc32 v1.3.0/go.mod h1:D7kQaZhnkX/Y0tstFGf8VUzv2UofNGqCjnC3zdHB0Hw=
github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU=
github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
@@ -199,8 +344,6 @@ github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJ
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-ieproxy v0.0.1 h1:qiyop7gCflfhwCzGyeT0gro3sF9AIg9HU98JORTkqfI=
github.com/mattn/go-ieproxy v0.0.1/go.mod h1:pYabZ6IHcRpFh7vIaLfK7rdcWgFEb3SFJ6/gNWuh88E=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
@@ -209,6 +352,14 @@ github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZ
github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=
github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/minio/crc64nvme v1.1.1 h1:8dwx/Pz49suywbO+auHCBpCtlW1OfpcLN7wYgVR6wAI=
github.com/minio/crc64nvme v1.1.1/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
github.com/minio/minio-go/v7 v7.0.98 h1:MeAVKjLVz+XJ28zFcuYyImNSAh8Mq725uNW4beRisi0=
github.com/minio/minio-go/v7 v7.0.98/go.mod h1:cY0Y+W7yozf0mdIclrttzo1Iiu7mEf9y7nk2uXqMOvM=
github.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM=
github.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8=
github.com/mkrautz/goar v0.0.0-20150919110319-282caa8bd9da h1:Iu5QFXIMK/YrHJ0NgUnK0rqYTTyb0ldt/rqNenAj39U=
github.com/mkrautz/goar v0.0.0-20150919110319-282caa8bd9da/go.mod h1:NfnmoBY0gGkr3/NmI+DP/UXbZvOCurCUYAzOdYJjlOc=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -223,6 +374,8 @@ github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+
github.com/ncw/swift v1.0.53 h1:luHjjTNtekIEvHg5KdAFIBaH7bWfNkefwFnpDffSIks=
github.com/ncw/swift v1.0.53/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/nwaples/rardecode v1.1.3 h1:cWCaZwfM5H7nAD6PyEdcVnczzV8i/JtotnyW/dD9lEc=
github.com/nwaples/rardecode v1.1.3/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=
github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
@@ -235,19 +388,34 @@ github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3
github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=
github.com/onsi/gomega v1.19.0 h1:4ieX6qQjPP/BfC3mpsAtIGGlxTWPeA3Inl/7DtXw1tw=
github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro=
github.com/onsi/gomega v1.34.1 h1:EUMJIKUjM8sKjYbtxQI9A4z2o+rruxnzNvpknOXie6k=
github.com/onsi/gomega v1.34.1/go.mod h1:kU1QgUvBDLXBJq618Xvm2LUX6rSAfRaFRTcdOeDLwwY=
github.com/pelletier/go-toml/v2 v2.1.0 h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4=
github.com/pelletier/go-toml/v2 v2.1.0/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc=
github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=
github.com/philhofer/fwd v1.2.0/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
github.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU=
github.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/pjbgf/sha1cd v0.6.0 h1:3WJ8Wz8gvDz29quX1OcEmkAlUg9diU4GxJHqs0/XiwU=
github.com/pjbgf/sha1cd v0.6.0/go.mod h1:lhpGlyHLpQZoxMv8HcgXvZEhcGs0PG/vsZnEJ7H0iCM=
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ=
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pkg/xattr v0.4.12 h1:rRTkSyFNTRElv6pkA3zpjHpQ90p/OdHQC1GmGh1aTjM=
github.com/pkg/xattr v0.4.12/go.mod h1:di8WF84zAKk8jzR1UBTEWh9AUlIZZ7M/JNt8e9B6ktU=
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 h1:GFCKgmp0tecUJ0sJuv4pzYCqS9+RGSn52M3FUwPs+uo=
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.20.0 h1:jBzTZ7B099Rg24tny+qngoynol8LtVYlA2bqx3vEloI=
github.com/prometheus/client_golang v1.20.0/go.mod h1:PIEt8X02hGcP8JWbeHyeZ53Y/jReSnHgO035n//V5WE=
github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E=
github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.59.1 h1:LXb1quJHWm1P6wq/U824uxYi4Sg0oGvNeUm1z5dJoX0=
github.com/prometheus/common v0.59.1/go.mod h1:GpWM7dewqmVYcd7SmRaiWVe9SSqjf0UrwnYnpEZNuT0=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
@@ -256,13 +424,20 @@ github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJ
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/rs/xid v1.4.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/rs/zerolog v1.29.1 h1:cO+d60CHkknCbvzEWxP0S9K6KqyTjrCNUy1LdQLCGPc=
github.com/rs/zerolog v1.29.1/go.mod h1:Le6ESbR7hc+DP6Lt1THiV8CQSdkkNrd3R0XbEgp3ZBU=
github.com/saracen/walker v0.1.2 h1:/o1TxP82n8thLvmL4GpJXduYaRmJ7qXp8u9dSlV0zmo=
github.com/saracen/walker v0.1.2/go.mod h1:0oKYMsKVhSJ+ful4p/XbjvXbMgLEkLITZaxozsl4CGE=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/skeema/knownhosts v1.3.1 h1:X2osQ+RAjK76shCbvhHHHVl3ZlgDm8apHEHFqRjnBY8=
github.com/skeema/knownhosts v1.3.1/go.mod h1:r7KTdC8l4uxWRyK2TpQZ/1o5HaSzh06ePQNxPwTcfiY=
github.com/smira/commander v0.0.0-20140515201010-f408b00e68d5 h1:jLFwP6SDEUHmb6QSu5n2FHseWzMio1ou1FV9p7W6p7I=
github.com/smira/commander v0.0.0-20140515201010-f408b00e68d5/go.mod h1:XTQy55hw5s3pxmC42m7X0/b+9naXQ1rGN9Of6BGIZmU=
github.com/smira/flag v0.0.0-20170926215700-695ea5e84e76 h1:OM075OkN4x9IB1mbzkzaKaJjFxx8Mfss8Z3E1LHwawQ=
@@ -271,10 +446,14 @@ github.com/smira/go-ftp-protocol v0.0.0-20140829150050-066b75c2b70d h1:rvtR4+9N2
github.com/smira/go-ftp-protocol v0.0.0-20140829150050-066b75c2b70d/go.mod h1:Jm7yHrROA5tC42gyJ5EwiR8EWp0PUy0qOc4sE7Y8Uzo=
github.com/smira/go-xz v0.1.0 h1:1zVLT1sITUKcWNysfHMLZWJ2Yh7yJfhREsgmUdK4zb0=
github.com/smira/go-xz v0.1.0/go.mod h1:OmdEWnIIkuLzRLHGF4YtjDzF9VFUevEcP6YxDPRqVrs=
github.com/spiffe/go-spiffe/v2 v2.6.0 h1:l+DolpxNWYgruGQVV0xsfeya3CsC7m8iBzDnMpsbLuo=
github.com/spiffe/go-spiffe/v2 v2.6.0/go.mod h1:gm2SeUoMZEtpnzPNs2Csc0D/gX33k1xIx7lEzqblHEs=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
@@ -284,160 +463,237 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/swaggo/files v1.0.1 h1:J1bVJ4XHZNq0I46UU90611i9/YzdrF7x92oX1ig5IdE=
github.com/swaggo/files v1.0.1/go.mod h1:0qXmMNH6sXNf+73t65aKeB+ApmgxdnkQzVTAj2uaMUg=
github.com/swaggo/gin-swagger v1.6.0 h1:y8sxvQ3E20/RCyrXeFfg60r6H0Z+SwpTjMYsMm+zy8M=
github.com/swaggo/gin-swagger v1.6.0/go.mod h1:BG00cCEy294xtVpyIAHG6+e2Qzj/xKlRdOqDkvq0uzo=
github.com/swaggo/swag v1.16.3 h1:PnCYjPCah8FK4I26l2F/KQ4yz3sILcVUN3cTlBFA9Pg=
github.com/swaggo/swag v1.16.3/go.mod h1:DImHIuOFXKpMFAQjcC7FG4m3Dg4+QuUgUzJmKjI/gRk=
github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d h1:vfofYNRScrDdvS342BElfbETmL1Aiz3i2t0zfRj16Hs=
github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d/go.mod h1:RRCYJbIwD5jmqPI9XoAFR0OcDxqUctll6zUj/+B4S48=
github.com/terminalstatic/go-xsd-validate v0.1.6 h1:TenYeQ3eY631qNi1/cTmLH/s2slHPRKTTHT+XSHkepo=
github.com/terminalstatic/go-xsd-validate v0.1.6/go.mod h1:18lsvYFofBflqCrvo1umpABZ99+GneNTw2kEEc8UPJw=
github.com/tinylib/msgp v1.6.1 h1:ESRv8eL3u+DNHUoSAAQRE50Hm162zqAnBoGv9PzScPY=
github.com/tinylib/msgp v1.6.1/go.mod h1:RSp0LW9oSxFut3KzESt5Voq4GVWyS+PSulT77roAqEA=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
github.com/ulikunitz/xz v0.5.6/go.mod h1:2bypXElzHzzJZwzH67Y6wb67pO62Rzfn7BSiF4ABRW8=
github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/wsxiaoys/terminal v0.0.0-20160513160801-0940f3fc43a0 h1:3UeQBvD0TFrlVjOeLOBz+CPAI8dnbqNSVwUwRrkp7vQ=
github.com/wsxiaoys/terminal v0.0.0-20160513160801-0940f3fc43a0/go.mod h1:IXCdmsXIht47RaVFLEdVnh1t+pgYtTAhQGj73kz+2DM=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f h1:J9EGpcZtP0E/raorCMxlFGSTBrsSlaDGf3jU/qvAE2c=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHovont7NscjpAxXsDA8S8BMYve8Y5+7cuRE7R0=
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74=
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.einride.tech/aip v0.79.0 h1:19zdPlZzlUvxOA8syAFw4LkdJdXepzyTl6gt9XEeqdU=
go.einride.tech/aip v0.79.0/go.mod h1:E8+wdTApA70odnpFzJgsGogHozC2JCIhFJBKPr8bVig=
go.etcd.io/etcd/api/v3 v3.5.15 h1:3KpLJir1ZEBrYuV2v+Twaa/e2MdDCEZ/70H+lzEiwsk=
go.etcd.io/etcd/api/v3 v3.5.15/go.mod h1:N9EhGzXq58WuMllgH9ZvnEr7SI9pS0k0+DHZezGp7jM=
go.etcd.io/etcd/client/pkg/v3 v3.5.15 h1:fo0HpWz/KlHGMCC+YejpiCmyWDEuIpnTDzpJLB5fWlA=
go.etcd.io/etcd/client/pkg/v3 v3.5.15/go.mod h1:mXDI4NAOwEiszrHCb0aqfAYNCrZP4e9hRca3d1YK8EU=
go.etcd.io/etcd/client/v3 v3.5.15 h1:23M0eY4Fd/inNv1ZfU3AxrbbOdW79r9V9Rl62Nm6ip4=
go.etcd.io/etcd/client/v3 v3.5.15/go.mod h1:CLSJxrYjvLtHsrPKsy7LmZEE+DK2ktfd2bN4RhBMwlU=
go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk=
go.uber.org/goleak v1.2.0/go.mod h1:XJYK+MuIchqpmGmUSAzotztawfKvYLUIgg7guXrwVUo=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/detectors/gcp v1.39.0 h1:kWRNZMsfBHZ+uHjiH4y7Etn2FK26LAGkNFw7RHv1DhE=
go.opentelemetry.io/contrib/detectors/gcp v1.39.0/go.mod h1:t/OGqzHBa5v6RHZwrDBJ2OirWc+4q/w2fTbLZwAKjTk=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 h1:YH4g8lQroajqUwWbq/tr2QX1JFmEXaDLgG+ew9bLMWo=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0/go.mod h1:fvPi2qXDqFs8M4B4fmJhE92TyQs9Ydjlg3RvfUp+NbQ=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.39.0 h1:5gn2urDL/FBnK8OkCfD1j3/ER79rUuTYmCvlXBKeYL8=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.39.0/go.mod h1:0fBG6ZJxhqByfFZDwSwpZGzJU671HkwpWaNe2t4VUPI=
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ=
go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/arch v0.3.0 h1:02VY4/ZcO/gBOH6PUaoiptASxtXU10jazRCP865E97k=
golang.org/x/arch v0.3.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.23.0 h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM=
golang.org/x/mod v0.23.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191112182307-2180aed22343/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210421230115-4e50805a0758/go.mod h1:72T/g9IO56b78aLF+1Kcs5dz7/ng1VjMUvfKvpfy+jM=
golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
golang.org/x/net v0.0.0-20210610132358-84b48f89b13b/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220607020251-c690dde0001d/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8=
golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.35.0 h1:Mv2mzuHuZuY2+bkyWXIHMfhNdJAdwW3FuWeCPYN5GVQ=
golang.org/x/oauth2 v0.35.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.12.0 h1:MHc5BpPuC30uJk597Ri8TV3CNZcTLu6B6z4lJy+g6Jw=
golang.org/x/sync v0.12.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191112214154-59a1497f0cea/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210420072515-93ed5bcd2bfe/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220408201424-a24fb2fb8a0f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
golang.org/x/term v0.30.0 h1:PQ39fJZ+mfadBm0y5WlL4vlM7Sx1Hgf13sMIY2+QS9Y=
golang.org/x/term v0.30.0/go.mod h1:NYYFdzHoI5wRh/h5tDMdMqCqPJZEuNqVR5xJLd/n67g=
golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY=
golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.30.0 h1:BgcpHewrV5AUp2G9MebG4XPFI1E2W41zU1SaqVA9vJY=
golang.org/x/tools v0.30.0/go.mod h1:c347cR/OJfw5TI+GfX7RUPNMdDRRbjvYTS0jPyvsVtY=
golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c=
golang.org/x/tools v0.44.0/go.mod h1:KA0AfVErSdxRZIsOVipbv3rQhVXTnlU6UhKxHd1seDI=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20220517211312-f3a8303e98df/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8=
google.golang.org/genproto/googleapis/api v0.0.0-20240318140521-94a12d6c2237 h1:RFiFrvy37/mpSpdySBDrUdipW/dHwsRwh3J3+A9VgT4=
google.golang.org/genproto/googleapis/api v0.0.0-20240318140521-94a12d6c2237/go.mod h1:Z5Iiy3jtmioajWHDGFk7CeugTyHtPvMHA4UTmUkyalE=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237 h1:NnYq6UN9ReLM9/Y01KWNOWyI5xQ9kbIms5GGJVwS/Yc=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237/go.mod h1:WtryC6hu0hhx87FDGxWCDptyssuo68sk10vYjF+T9fY=
google.golang.org/grpc v1.64.1 h1:LKtvyfbX3UGVPFcGqJ9ItpVWW6oN/2XqTxfAnwRRXiA=
google.golang.org/grpc v1.64.1/go.mod h1:hiQF4LFZelK2WKaP6W0L92zGHtiQdZxk8CrSdvyjeP0=
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
google.golang.org/api v0.266.0 h1:hco+oNCf9y7DmLeAtHJi/uBAY7n/7XC9mZPxu1ROiyk=
google.golang.org/api v0.266.0/go.mod h1:Jzc0+ZfLnyvXma3UtaTl023TdhZu6OMBP9tJ+0EmFD0=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409 h1:VQZ/yAbAtjkHgH80teYd2em3xtIkkHd7ZhqfH2N9CsM=
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409/go.mod h1:rxKD3IEILWEu3P44seeNOAwZN4SaoKaQ/2eTg4mM6EM=
google.golang.org/genproto/googleapis/api v0.0.0-20260203192932-546029d2fa20 h1:7ei4lp52gK1uSejlA8AZl5AJjeLUOHBQscRQZUgAcu0=
google.golang.org/genproto/googleapis/api v0.0.0-20260203192932-546029d2fa20/go.mod h1:ZdbssH/1SOVnjnDlXzxDHK2MCidiqXtbYccJNzNYPEE=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 h1:Jr5R2J6F6qWyzINc+4AM8t5pfUz6beZpHp678GNrMbE=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20/go.mod h1:j9x/tPzZkyxcgEFkiKEEGxfvyumM01BEtsW8xzOahRQ=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=
google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
@@ -446,6 +702,8 @@ gopkg.in/cheggaaa/pb.v1 v1.0.28/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qS
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
@@ -455,4 +713,6 @@ gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=
+1
View File
@@ -44,6 +44,7 @@ func NewDownloader(downLimit int64, maxTries int, progress aptly.Progress) aptly
transport.DisableCompression = true
initTransport(&transport)
transport.RegisterProtocol("ftp", &protocol.FTPRoundTripper{})
transport.RegisterProtocol("ar+https", NewGCPRoundTripper(&transport))
downloader := &downloaderImpl{
progress: progress,
+64
View File
@@ -0,0 +1,64 @@
package http
import (
"context"
"fmt"
"net/http"
"strings"
"sync"
"golang.org/x/oauth2"
"golang.org/x/oauth2/google"
)
// gcpRoundTripper wraps http.RoundTripper to add Google Cloud authentication.
// It delays GCP authentication initialization until the first actual request is made.
// This avoids unnecessary credential loading when ar+https protocol is not actually used.
//
// It uses Application Default Credentials (ADC) which checks:
// 1. GOOGLE_APPLICATION_CREDENTIALS environment variable
// 2. gcloud auth application-default credentials
// 3. GCE/GKE metadata server
// See https://cloud.google.com/docs/authentication/application-default-credentials for usage details.
type gcpRoundTripper struct {
base http.RoundTripper
initOnce sync.Once
tokenSrc oauth2.TokenSource
initErr error
}
func (t *gcpRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
// Lazy initialization: only initialize GCP credentials on first request
t.initOnce.Do(func() {
creds, err := google.FindDefaultCredentials(context.Background(),
"https://www.googleapis.com/auth/cloud-platform")
if err != nil {
t.initErr = fmt.Errorf("failed to find default credentials: %w", err)
return
}
t.tokenSrc = creds.TokenSource
})
reqCopy := req.Clone(req.Context())
reqCopy.URL.Scheme = strings.TrimPrefix(reqCopy.URL.Scheme, "ar+")
// Fall back to base transport if GCP auth initialization failed
if t.initErr != nil {
return t.base.RoundTrip(reqCopy)
}
token, err := t.tokenSrc.Token()
if err != nil {
return nil, fmt.Errorf("failed to get OAuth2 token: %w", err)
}
token.SetAuthHeader(reqCopy)
return t.base.RoundTrip(reqCopy)
}
// NewGCPRoundTripper creates a new RoundTripper that handles GCP authentication for ar+https protocol.
func NewGCPRoundTripper(base http.RoundTripper) http.RoundTripper {
return &gcpRoundTripper{
base: base,
}
}
+110
View File
@@ -0,0 +1,110 @@
package http
import (
"net/http"
"net/http/httptest"
"os"
"testing"
"golang.org/x/oauth2"
)
func TestGCPAuthTransport_RoundTrip(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
auth := r.Header.Get("Authorization")
if auth == "" {
t.Error("Expected Authorization header, got none")
}
w.WriteHeader(http.StatusOK)
}))
defer ts.Close()
transport := NewGCPRoundTripper(http.DefaultTransport)
if os.Getenv("GOOGLE_APPLICATION_CREDENTIALS") == "" {
t.Skip("Skipping test: GOOGLE_APPLICATION_CREDENTIALS not set")
}
client := &http.Client{Transport: transport}
resp, err := client.Get(ts.URL)
if err != nil {
t.Fatalf("Failed to make request: %v", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
t.Errorf("Expected status 200, got %d", resp.StatusCode)
}
}
func TestGCPAuthTransport_RoundTrip_with_dummy_tokenSource(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
auth := r.Header.Get("Authorization")
if auth != "Bearer dummy-token" {
t.Errorf("Expected Authorization header 'Bearer dummy-token', got '%s'", auth)
}
w.WriteHeader(http.StatusOK)
}))
defer ts.Close()
// Use a dummy token source for testing
transport := &gcpRoundTripper{
base: http.DefaultTransport,
tokenSrc: oauth2.StaticTokenSource(&oauth2.Token{
AccessToken: "dummy-token",
}),
}
transport.initOnce.Do(func() {}) // Mark as initialized for testing
client := &http.Client{Transport: transport}
resp, err := client.Get(ts.URL)
if err != nil {
t.Fatalf("Failed to make request: %v", err)
}
defer func(){ _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
t.Errorf("Expected status 200, got %d", resp.StatusCode)
}
}
func TestGCPAuthTransport_RoundTrip_with_InvalidCredentials(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusForbidden)
}))
defer ts.Close()
// Create a temporary invalid credentials file
tmpFile, err := os.CreateTemp("", "invalid_credentials.json")
if err != nil {
t.Fatalf("Failed to create temp file: %s", err)
}
defer func() { _ = os.Remove(tmpFile.Name()) }()
if _, err := tmpFile.WriteString(`{"invalid": "data"}`); err != nil {
t.Fatalf("Failed to write to temp file: %s", err)
}
_ = tmpFile.Close()
defaultEnv := os.Getenv("GOOGLE_APPLICATION_CREDENTIALS")
_ = os.Setenv("GOOGLE_APPLICATION_CREDENTIALS", tmpFile.Name())
defer func() { _ = os.Setenv("GOOGLE_APPLICATION_CREDENTIALS", defaultEnv) }()
transport := &gcpRoundTripper{
base: http.DefaultTransport,
}
client := &http.Client{Transport: transport}
resp, err := client.Get(ts.URL)
if err != nil {
t.Fatalf("Failed to make request: %s", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusForbidden {
t.Errorf("Expected status 403, got %d", resp.StatusCode)
}
if transport.initErr == nil {
t.Error("Expected init error due to invalid credentials, got none")
}
}
+2
View File
@@ -0,0 +1,2 @@
// Package jfrog handles publishing to JFrog Artifactory
package jfrog
+292
View File
@@ -0,0 +1,292 @@
package jfrog
import (
"fmt"
"io"
"os"
"path/filepath"
"strings"
"github.com/aptly-dev/aptly/aptly"
aptly_utils "github.com/aptly-dev/aptly/utils"
"github.com/jfrog/jfrog-client-go/artifactory"
"github.com/jfrog/jfrog-client-go/artifactory/auth"
"github.com/jfrog/jfrog-client-go/artifactory/services"
"github.com/jfrog/jfrog-client-go/artifactory/services/utils"
"github.com/jfrog/jfrog-client-go/config"
"github.com/pkg/errors"
)
// PublishedStorage represents published repository on JFrog Artifactory
type PublishedStorage struct {
manager artifactory.ArtifactoryServicesManager
repository string
prefix string
plusWorkaround bool
}
// Check interface
var (
_ aptly.PublishedStorage = (*PublishedStorage)(nil)
)
func createPublishedStorageConfig(url, user, password, apiKey, accessToken string) (config.Config, error) {
artDetails := auth.NewArtifactoryDetails()
artDetails.SetUrl(url)
if user == "" {
user = os.Getenv("JFROG_USERNAME");
}
if password == "" {
password = os.Getenv("JFROG_PASSWORD");
}
if apiKey == "" {
apiKey = os.Getenv("JFROG_APIKEY");
}
if accessToken == "" {
accessToken = os.Getenv("JFROG_ACCESSTOKEN");
}
if user != "" && password != "" {
artDetails.SetUser(user)
artDetails.SetPassword(password)
} else if apiKey != "" {
artDetails.SetApiKey(apiKey)
} else if accessToken != "" {
artDetails.SetAccessToken(accessToken)
}
return config.NewConfigBuilder().
SetServiceDetails(artDetails).
SetDryRun(false).
Build()
}
// NewPublishedStorageRaw creates jfrog PublishedStorage from raw connection specs
func NewPublishedStorageRaw(
repository, url, user, password, apiKey, accessToken, prefix string,
plusWorkaround, debug bool,
) (*PublishedStorage, error) {
serviceConfig, err := createPublishedStorageConfig(url, user, password, apiKey, accessToken)
if err != nil {
return nil, errors.Wrap(err, "error building jfrog client config")
}
manager, err := artifactory.New(serviceConfig)
if err != nil {
return nil, errors.Wrap(err, "error creating jfrog manager")
}
return &PublishedStorage{
manager: manager,
repository: repository,
prefix: prefix,
plusWorkaround: plusWorkaround,
}, nil
}
// NewPublishedStorage creates published storage from aptly configuration struct
func NewPublishedStorage(
account string, root aptly_utils.JFrogPublishRoot,
) (*PublishedStorage, error) {
return NewPublishedStorageRaw(
root.Repository, root.URL, root.User, root.Password, root.APIKey, root.AccessToken,
root.Prefix, root.PlusWorkaround, root.Debug)
}
func (storage *PublishedStorage) String() string {
return fmt.Sprintf("jfrog:%s:%s", storage.repository, storage.prefix)
}
func (storage *PublishedStorage) MkDir(path string) error {
return nil
}
func (storage *PublishedStorage) PutFile(path string, sourceFilename string) error {
targetPath := filepath.Join(storage.repository, storage.prefix, path)
if storage.plusWorkaround {
targetPath = strings.Replace(targetPath, "+", "%2B", -1)
}
params := services.NewUploadParams()
params.Pattern = sourceFilename
params.Target = targetPath
params.Flat = true
_, _, err := storage.manager.UploadFiles(artifactory.UploadServiceOptions{}, params)
return err
}
func (storage *PublishedStorage) Remove(path string) error {
targetPath := filepath.Join(storage.repository, storage.prefix, path)
if storage.plusWorkaround {
targetPath = strings.Replace(targetPath, "+", "%2B", -1)
}
deleteParams := services.NewDeleteParams()
deleteParams.SetPattern(targetPath)
res, err := storage.manager.GetPathsToDelete(deleteParams)
if err != nil {
return err
}
defer func() { _ = res.Close() }()
_, err = storage.manager.DeleteFiles(res)
return err
}
func (storage *PublishedStorage) RemoveDirs(path string, progress aptly.Progress) error {
return storage.Remove(path)
}
func (storage *PublishedStorage) LinkFromPool(publishedPrefix, publishedRelPath, fileName string, sourcePool aptly.PackagePool, sourcePath string, sourceMD5 aptly_utils.ChecksumInfo, force bool) error {
sourceFilename := sourcePath
cleanup := func() {}
if sourcePool != nil {
if localPool, ok := sourcePool.(aptly.LocalPackagePool); ok {
sourceFilename = localPool.FullPath(sourcePath)
} else {
src, err := sourcePool.Open(sourcePath)
if err != nil {
return err
}
defer func() { _ = src.Close() }()
tmpFile, err := os.CreateTemp("", "aptly-jfrog-pool-*")
if err != nil {
return err
}
if _, err := io.Copy(tmpFile, src); err != nil {
_ = tmpFile.Close()
_ = os.Remove(tmpFile.Name())
return err
}
if err := tmpFile.Close(); err != nil {
_ = os.Remove(tmpFile.Name())
return err
}
sourceFilename = tmpFile.Name()
cleanup = func() {
_ = os.Remove(sourceFilename)
}
}
}
defer cleanup()
return storage.PutFile(filepath.Join(publishedPrefix, publishedRelPath, fileName), sourceFilename)
}
func (storage *PublishedStorage) Filelist(prefix string) ([]string, error) {
searchPattern := filepath.Join(storage.repository, storage.prefix, prefix, "*")
params := services.NewSearchParams()
params.Pattern = searchPattern
reader, err := storage.manager.SearchFiles(params)
if err != nil {
return nil, err
}
defer func() { _ = reader.Close() }()
var paths []string
for element := new(utils.ResultItem); reader.NextRecord(element) == nil; element = new(utils.ResultItem) {
path := element.Path + "/" + element.Name
relPath := strings.TrimPrefix(path, storage.repository+"/"+storage.prefix+"/")
if storage.plusWorkaround {
relPath = strings.Replace(relPath, "%2B", "+", -1)
}
paths = append(paths, relPath)
}
return paths, nil
}
func (storage *PublishedStorage) RenameFile(oldName, newName string) error {
oldTarget := filepath.Join(storage.repository, storage.prefix, oldName)
newTarget := filepath.Join(storage.repository, storage.prefix, newName)
if storage.plusWorkaround {
oldTarget = strings.Replace(oldTarget, "+", "%2B", -1)
newTarget = strings.Replace(newTarget, "+", "%2B", -1)
}
params := services.NewMoveCopyParams()
params.Pattern = oldTarget
params.Target = newTarget
params.Flat = true
_, _, err := storage.manager.Move(params)
return err
}
func (storage *PublishedStorage) SymLink(src string, dst string) error {
oldTarget := filepath.Join(storage.repository, storage.prefix, src)
newTarget := filepath.Join(storage.repository, storage.prefix, dst)
if storage.plusWorkaround {
oldTarget = strings.Replace(oldTarget, "+", "%2B", -1)
newTarget = strings.Replace(newTarget, "+", "%2B", -1)
}
params := services.NewMoveCopyParams()
params.Pattern = oldTarget
params.Target = newTarget
params.Flat = true
props := utils.NewProperties()
props.AddProperty("SymLink", src)
params.SetTargetProps(props)
_, _, err := storage.manager.Copy(params)
return err
}
func (storage *PublishedStorage) HardLink(src string, dst string) error {
return storage.SymLink(src, dst)
}
func (storage *PublishedStorage) FileExists(path string) (bool, error) {
targetPath := filepath.Join(storage.repository, storage.prefix, path)
if storage.plusWorkaround {
targetPath = strings.Replace(targetPath, "+", "%2B", -1)
}
params := services.NewSearchParams()
params.Pattern = targetPath
reader, err := storage.manager.SearchFiles(params)
if err != nil {
return false, err
}
defer func() { _ = reader.Close() }()
length, err := reader.Length()
isEmpty := length == 0
return !isEmpty, err
}
func (storage *PublishedStorage) ReadLink(path string) (string, error) {
targetPath := filepath.Join(storage.repository, storage.prefix, path)
if storage.plusWorkaround {
targetPath = strings.Replace(targetPath, "+", "%2B", -1)
}
props, err := storage.manager.GetItemProps(targetPath)
if err != nil {
return "", nil
}
for k, v := range props.Properties {
if k == "SymLink" && len(v) > 0 {
return v[0], nil
}
}
return "", nil
}
+511
View File
@@ -0,0 +1,511 @@
package jfrog
import (
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/aptly-dev/aptly/aptly"
aptly_utils "github.com/aptly-dev/aptly/utils"
"github.com/jfrog/jfrog-client-go/artifactory"
"github.com/jfrog/jfrog-client-go/artifactory/services"
jfrogutils "github.com/jfrog/jfrog-client-go/artifactory/services/utils"
"github.com/jfrog/jfrog-client-go/utils/io/content"
. "gopkg.in/check.v1"
)
func Test(t *testing.T) {
t.Setenv("JFROG_USERNAME", "userfromenv")
TestingT(t)
}
type fakeJFrogManager struct {
artifactory.EmptyArtifactoryServicesManager
uploadParams []services.UploadParams
uploadErr error
deleteParams []services.DeleteParams
getPathsToDelete *content.ContentReader
getPathsDeleteErr error
deleteErr error
deleteCalled bool
searchParams []services.SearchParams
searchReader *content.ContentReader
searchErr error
moveParams []services.MoveCopyParams
moveErr error
copyParams []services.MoveCopyParams
copyErr error
itemProps *jfrogutils.ItemProperties
itemPropsErr error
}
func (m *fakeJFrogManager) UploadFiles(_ artifactory.UploadServiceOptions, params ...services.UploadParams) (int, int, error) {
m.uploadParams = append(m.uploadParams, params...)
return len(params), 0, m.uploadErr
}
func (m *fakeJFrogManager) GetPathsToDelete(params services.DeleteParams) (*content.ContentReader, error) {
m.deleteParams = append(m.deleteParams, params)
if m.getPathsDeleteErr != nil {
return nil, m.getPathsDeleteErr
}
if m.getPathsToDelete != nil {
return m.getPathsToDelete, nil
}
return content.NewEmptyContentReader("results"), nil
}
func (m *fakeJFrogManager) DeleteFiles(_ *content.ContentReader) (int, error) {
m.deleteCalled = true
return 1, m.deleteErr
}
func (m *fakeJFrogManager) SearchFiles(params services.SearchParams) (*content.ContentReader, error) {
m.searchParams = append(m.searchParams, params)
if m.searchErr != nil {
return nil, m.searchErr
}
if m.searchReader != nil {
return m.searchReader, nil
}
return content.NewEmptyContentReader("results"), nil
}
func (m *fakeJFrogManager) Move(params ...services.MoveCopyParams) (int, int, error) {
m.moveParams = append(m.moveParams, params...)
return len(params), 0, m.moveErr
}
func (m *fakeJFrogManager) Copy(params ...services.MoveCopyParams) (int, int, error) {
m.copyParams = append(m.copyParams, params...)
return len(params), 0, m.copyErr
}
func (m *fakeJFrogManager) GetItemProps(_ string) (*jfrogutils.ItemProperties, error) {
if m.itemPropsErr != nil {
return nil, m.itemPropsErr
}
if m.itemProps != nil {
return m.itemProps, nil
}
return &jfrogutils.ItemProperties{}, nil
}
type resultFixture struct {
Results []jfrogutils.ResultItem `json:"results"`
}
type fakeLocalPool struct{}
func (p *fakeLocalPool) Verify(string, string, *aptly_utils.ChecksumInfo, aptly.ChecksumStorage) (string, bool, error) {
return "", false, nil
}
func (p *fakeLocalPool) Import(string, string, *aptly_utils.ChecksumInfo, bool, aptly.ChecksumStorage) (string, error) {
return "", nil
}
func (p *fakeLocalPool) LegacyPath(string, *aptly_utils.ChecksumInfo) (string, error) {
return "", nil
}
func (p *fakeLocalPool) Size(string) (int64, error) {
return 0, nil
}
func (p *fakeLocalPool) Open(string) (aptly.ReadSeekerCloser, error) {
return nil, errors.New("not implemented")
}
func (p *fakeLocalPool) FilepathList(aptly.Progress) ([]string, error) {
return nil, nil
}
func (p *fakeLocalPool) Remove(string) (int64, error) {
return 0, nil
}
func (p *fakeLocalPool) Stat(string) (os.FileInfo, error) {
return nil, errors.New("not implemented")
}
func (p *fakeLocalPool) GenerateTempPath(string) (string, error) {
return "", nil
}
func (p *fakeLocalPool) Link(string, string) error {
return nil
}
func (p *fakeLocalPool) Symlink(string, string) error {
return nil
}
func (p *fakeLocalPool) FullPath(path string) string {
return filepath.Join("/var/lib/aptly/pool", path)
}
type fakeRemotePool struct {
openPath string
openErr error
}
func (p *fakeRemotePool) Verify(string, string, *aptly_utils.ChecksumInfo, aptly.ChecksumStorage) (string, bool, error) {
return "", false, nil
}
func (p *fakeRemotePool) Import(string, string, *aptly_utils.ChecksumInfo, bool, aptly.ChecksumStorage) (string, error) {
return "", nil
}
func (p *fakeRemotePool) LegacyPath(string, *aptly_utils.ChecksumInfo) (string, error) {
return "", nil
}
func (p *fakeRemotePool) Size(string) (int64, error) {
return 0, nil
}
func (p *fakeRemotePool) Open(string) (aptly.ReadSeekerCloser, error) {
if p.openErr != nil {
return nil, p.openErr
}
return os.Open(p.openPath)
}
func (p *fakeRemotePool) FilepathList(aptly.Progress) ([]string, error) {
return nil, nil
}
func (p *fakeRemotePool) Remove(string) (int64, error) {
return 0, nil
}
func createReader(c *C, results []jfrogutils.ResultItem) *content.ContentReader {
filePath := filepath.Join(c.MkDir(), "results.json")
data, err := json.Marshal(resultFixture{Results: results})
c.Assert(err, IsNil)
c.Assert(os.WriteFile(filePath, data, 0o644), IsNil)
return content.NewContentReader(filePath, "results")
}
type PublishedStorageSuite struct {
manager *fakeJFrogManager
storage *PublishedStorage
}
var _ = Suite(&PublishedStorageSuite{})
func (s *PublishedStorageSuite) SetUpTest(c *C) {
s.manager = &fakeJFrogManager{}
s.storage = &PublishedStorage{
manager: s.manager,
repository: "repo",
prefix: "prefix",
}
}
func (s *PublishedStorageSuite) TestStringAndMkDir(c *C) {
c.Assert(s.storage.String(), Equals, "jfrog:repo:prefix")
c.Assert(s.storage.MkDir("anything"), IsNil)
}
func (s *PublishedStorageSuite) TestPutFile(c *C) {
err := s.storage.PutFile("pool/main/a+b.deb", "/tmp/source.deb")
c.Assert(err, IsNil)
c.Assert(len(s.manager.uploadParams), Equals, 1)
c.Assert(s.manager.uploadParams[0].Pattern, Equals, "/tmp/source.deb")
c.Assert(s.manager.uploadParams[0].Target, Equals, filepath.Join("repo", "prefix", "pool/main/a+b.deb"))
c.Assert(s.manager.uploadParams[0].Flat, Equals, true)
}
func (s *PublishedStorageSuite) TestPutFilePlusWorkaroundAndError(c *C) {
s.storage.plusWorkaround = true
s.manager.uploadErr = errors.New("upload failed")
err := s.storage.PutFile("pool/main/a+b.deb", "/tmp/source.deb")
c.Assert(err, ErrorMatches, "upload failed")
c.Assert(s.manager.uploadParams[0].Target, Equals, filepath.Join("repo", "prefix", "pool/main/a%2Bb.deb"))
}
func (s *PublishedStorageSuite) TestRemove(c *C) {
s.manager.getPathsToDelete = createReader(c, []jfrogutils.ResultItem{})
err := s.storage.Remove("dists/stable+main")
c.Assert(err, IsNil)
c.Assert(len(s.manager.deleteParams), Equals, 1)
c.Assert(s.manager.deleteParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "dists/stable+main"))
c.Assert(s.manager.deleteCalled, Equals, true)
}
func (s *PublishedStorageSuite) TestRemovePlusWorkaround(c *C) {
s.storage.plusWorkaround = true
s.manager.getPathsToDelete = createReader(c, []jfrogutils.ResultItem{})
err := s.storage.Remove("pool/a+b.deb")
c.Assert(err, IsNil)
c.Assert(s.manager.deleteParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "pool/a%2Bb.deb"))
}
func (s *PublishedStorageSuite) TestRemoveErrors(c *C) {
s.manager.getPathsDeleteErr = errors.New("search delete failed")
err := s.storage.Remove("x")
c.Assert(err, ErrorMatches, "search delete failed")
s.manager.getPathsDeleteErr = nil
s.manager.getPathsToDelete = createReader(c, []jfrogutils.ResultItem{})
s.manager.deleteErr = errors.New("delete failed")
err = s.storage.Remove("x")
c.Assert(err, ErrorMatches, "delete failed")
}
func (s *PublishedStorageSuite) TestRemoveDirsDelegatesToRemove(c *C) {
s.manager.getPathsToDelete = createReader(c, []jfrogutils.ResultItem{})
c.Assert(s.storage.RemoveDirs("x", nil), IsNil)
c.Assert(len(s.manager.deleteParams), Equals, 1)
}
func (s *PublishedStorageSuite) TestLinkFromPoolDelegatesToPutFile(c *C) {
err := s.storage.LinkFromPool("", "pool/main/p", "pkg.deb", nil, "/tmp/source.deb", aptly_utils.ChecksumInfo{}, false)
c.Assert(err, IsNil)
c.Assert(s.manager.uploadParams[0].Target, Equals, filepath.Join("repo", "prefix", "pool/main/p", "pkg.deb"))
}
func (s *PublishedStorageSuite) TestLinkFromPoolUsesLocalPoolFullPath(c *C) {
pool := &fakeLocalPool{}
poolPath := "e3/48/84d71bb98002bf0c775479aa31ee_accountsservice_0.6.55-0ubuntu11_amd64.deb"
err := s.storage.LinkFromPool("", "pool/main/p", "pkg.deb", pool, poolPath, aptly_utils.ChecksumInfo{}, false)
c.Assert(err, IsNil)
c.Assert(s.manager.uploadParams[0].Pattern, Equals, filepath.Join("/var/lib/aptly/pool", poolPath))
}
func (s *PublishedStorageSuite) TestLinkFromPoolCopiesFromRemotePool(c *C) {
tmpFile := filepath.Join(c.MkDir(), "source.deb")
c.Assert(os.WriteFile(tmpFile, []byte("package-bytes"), 0o644), IsNil)
pool := &fakeRemotePool{openPath: tmpFile}
err := s.storage.LinkFromPool("", "pool/main/p", "pkg.deb", pool, "hash/path/pkg.deb", aptly_utils.ChecksumInfo{}, false)
c.Assert(err, IsNil)
uploadPath := s.manager.uploadParams[0].Pattern
c.Assert(uploadPath, Not(Equals), "hash/path/pkg.deb")
_, statErr := os.Stat(uploadPath)
c.Assert(os.IsNotExist(statErr), Equals, true)
}
func (s *PublishedStorageSuite) TestFilelist(c *C) {
s.manager.searchReader = createReader(c, []jfrogutils.ResultItem{
{Path: "repo/prefix/pool/main/a", Name: "a.deb", Actual_Md5: "m1"},
{Path: "repo/prefix/pool/main/b", Name: "b.deb", Actual_Md5: "m2"},
})
list, err := s.storage.Filelist("pool/main")
c.Assert(err, IsNil)
c.Assert(list, DeepEquals, []string{"pool/main/a/a.deb", "pool/main/b/b.deb"})
c.Assert(s.manager.searchParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "pool/main", "*"))
}
func (s *PublishedStorageSuite) TestFilelistPlusWorkaroundAndSearchError(c *C) {
s.storage.plusWorkaround = true
s.manager.searchReader = createReader(c, []jfrogutils.ResultItem{
{Path: "repo/prefix/pool/main", Name: "a%2Bb.deb", Actual_Md5: "m1"},
})
list, err := s.storage.Filelist("pool/main")
c.Assert(err, IsNil)
c.Assert(list, DeepEquals, []string{"pool/main/a+b.deb"})
s.manager.searchErr = errors.New("search failed")
_, err = s.storage.Filelist("pool/main")
c.Assert(err, ErrorMatches, "search failed")
}
func (s *PublishedStorageSuite) TestRenameFile(c *C) {
err := s.storage.RenameFile("old+name", "new+name")
c.Assert(err, IsNil)
c.Assert(len(s.manager.moveParams), Equals, 1)
c.Assert(s.manager.moveParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "old+name"))
c.Assert(s.manager.moveParams[0].Target, Equals, filepath.Join("repo", "prefix", "new+name"))
c.Assert(s.manager.moveParams[0].Flat, Equals, true)
}
func (s *PublishedStorageSuite) TestRenameFilePlusWorkaroundAndError(c *C) {
s.storage.plusWorkaround = true
s.manager.moveErr = errors.New("move failed")
err := s.storage.RenameFile("old+name", "new+name")
c.Assert(err, ErrorMatches, "move failed")
c.Assert(s.manager.moveParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "old%2Bname"))
c.Assert(s.manager.moveParams[0].Target, Equals, filepath.Join("repo", "prefix", "new%2Bname"))
}
func (s *PublishedStorageSuite) TestSymLinkAndHardLink(c *C) {
err := s.storage.SymLink("src+name", "dst+name")
c.Assert(err, IsNil)
c.Assert(len(s.manager.copyParams), Equals, 1)
c.Assert(s.manager.copyParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "src+name"))
c.Assert(s.manager.copyParams[0].Target, Equals, filepath.Join("repo", "prefix", "dst+name"))
c.Assert(s.manager.copyParams[0].Flat, Equals, true)
targetProps := s.manager.copyParams[0].TargetProps.ToMap()
c.Assert(targetProps["SymLink"], DeepEquals, []string{"src+name"})
err = s.storage.HardLink("a", "b")
c.Assert(err, IsNil)
c.Assert(len(s.manager.copyParams), Equals, 2)
}
func (s *PublishedStorageSuite) TestSymLinkPlusWorkaroundAndError(c *C) {
s.storage.plusWorkaround = true
s.manager.copyErr = errors.New("copy failed")
err := s.storage.SymLink("src+name", "dst+name")
c.Assert(err, ErrorMatches, "copy failed")
c.Assert(s.manager.copyParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "src%2Bname"))
c.Assert(s.manager.copyParams[0].Target, Equals, filepath.Join("repo", "prefix", "dst%2Bname"))
}
func (s *PublishedStorageSuite) TestFileExists(c *C) {
s.manager.searchReader = createReader(c, []jfrogutils.ResultItem{{Path: "repo/prefix/pool", Name: "x"}})
ok, err := s.storage.FileExists("pool/x")
c.Assert(err, IsNil)
c.Assert(ok, Equals, true)
c.Assert(s.manager.searchParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "pool/x"))
s.manager.searchReader = content.NewEmptyContentReader("results")
ok, err = s.storage.FileExists("pool/y")
c.Assert(err, IsNil)
c.Assert(ok, Equals, false)
}
func (s *PublishedStorageSuite) TestFileExistsSearchErrorAndPlusWorkaround(c *C) {
s.storage.plusWorkaround = true
s.manager.searchErr = errors.New("search failed")
ok, err := s.storage.FileExists("pool/a+b")
c.Assert(ok, Equals, false)
c.Assert(err, ErrorMatches, "search failed")
c.Assert(s.manager.searchParams[0].Pattern, Equals, filepath.Join("repo", "prefix", "pool/a%2Bb"))
}
func (s *PublishedStorageSuite) TestReadLink(c *C) {
s.manager.itemProps = &jfrogutils.ItemProperties{
Properties: map[string][]string{
"SymLink": {"src/file"},
},
}
link, err := s.storage.ReadLink("path/to/link")
c.Assert(err, IsNil)
c.Assert(link, Equals, "src/file")
}
func (s *PublishedStorageSuite) TestReadLinkNoPropertyAndErrors(c *C) {
s.manager.itemProps = &jfrogutils.ItemProperties{Properties: map[string][]string{"Other": {"value"}}}
link, err := s.storage.ReadLink("path/to/link")
c.Assert(err, IsNil)
c.Assert(link, Equals, "")
s.manager.itemPropsErr = errors.New("props failed")
link, err = s.storage.ReadLink("path/to/link")
c.Assert(err, IsNil)
c.Assert(link, Equals, "")
}
func (s *PublishedStorageSuite) TestReadLinkPlusWorkaround(c *C) {
s.storage.plusWorkaround = true
s.manager.itemProps = &jfrogutils.ItemProperties{}
_, _ = s.storage.ReadLink("a+b")
// Ensure the method runs with plusWorkaround path conversion.
c.Assert(s.manager.itemPropsErr, IsNil)
}
func (s *PublishedStorageSuite) TestCreatePublishedStorageConfig(c *C) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}))
defer server.Close()
withUserPassword, err := createPublishedStorageConfig(server.URL, "user", "password", "", "")
c.Assert(err, IsNil)
withUserPasswordDetails := withUserPassword.GetServiceDetails()
c.Assert(withUserPasswordDetails, NotNil)
c.Assert(withUserPasswordDetails.GetUser(), Equals, "user")
withAPIKey, err := createPublishedStorageConfig(server.URL, "", "", "api-123", "")
c.Assert(err, IsNil)
withAPIKeyDetails := withAPIKey.GetServiceDetails()
c.Assert(withAPIKeyDetails, NotNil)
c.Assert(withAPIKeyDetails.GetApiKey(), Equals, "api-123")
withAccessToken, err := createPublishedStorageConfig(server.URL, "", "", "", "token")
c.Assert(err, IsNil)
withAccessTokenDetails := withAccessToken.GetServiceDetails()
c.Assert(withAccessTokenDetails, NotNil)
c.Assert(withAccessTokenDetails.GetAccessToken(), Equals, "token")
withUserPasswordFromEnv, err := createPublishedStorageConfig(server.URL, "", "password", "", "")
c.Assert(err, IsNil)
withUserPasswordFromEnvDetails := withUserPasswordFromEnv.GetServiceDetails()
c.Assert(withUserPasswordFromEnvDetails, NotNil)
c.Assert(withUserPasswordFromEnvDetails.GetUser(), Equals, "userfromenv")
}
func (s *PublishedStorageSuite) TestNewPublishedStorageRaw(c *C) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}))
defer server.Close()
withUserPassword, err := NewPublishedStorageRaw("repo", server.URL, "user", "password", "", "", "prefix", true, false)
c.Assert(err, IsNil)
c.Assert(withUserPassword, NotNil)
c.Assert(withUserPassword.String(), Equals, "jfrog:repo:prefix")
withAPIKey, err := NewPublishedStorageRaw("repo", server.URL, "", "", "api-key", "", "prefix", false, false)
c.Assert(err, IsNil)
c.Assert(withAPIKey, NotNil)
withToken, err := NewPublishedStorageRaw("repo", server.URL, "", "", "", "token", "prefix", false, false)
c.Assert(err, IsNil)
c.Assert(withToken, NotNil)
}
func (s *PublishedStorageSuite) TestNewPublishedStorageRawManagerError(c *C) {
// An SSH URL causes artifactory.New() to fail (no SSH key configured),
// exercising the error return on lines 59-61.
_, err := NewPublishedStorageRaw("repo", "ssh://example.local/artifactory", "", "", "", "", "", false, false)
c.Assert(err, ErrorMatches, "error creating jfrog manager: .*")
}
func (s *PublishedStorageSuite) TestNewPublishedStorage(c *C) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}))
defer server.Close()
storage, err := NewPublishedStorage("test", aptly_utils.JFrogPublishRoot{
Repository: "repo",
URL: server.URL,
AccessToken: "token",
Prefix: "pref",
PlusWorkaround: true,
})
c.Assert(err, IsNil)
c.Assert(storage, NotNil)
c.Assert(storage.String(), Equals, "jfrog:repo:pref")
}
var _ aptly.PublishedStorage = (*PublishedStorage)(nil)
+101 -2
View File
@@ -111,9 +111,8 @@ The legacy json configuration is still supported (and also supports comments):
// Enable metrics for Prometheus client
"enableMetricsEndpoint": false,
// Not implemented in this version\.
// Enable API documentation on /docs
//"enableSwaggerEndpoint": false,
"enableSwaggerEndpoint": false,
// OBSOLETE: use via url param ?_async=true
"AsyncAPI": false,
@@ -308,6 +307,66 @@ The legacy json configuration is still supported (and also supports comments):
// }
},
// GCS Endpoint Support
//
// aptly can be configured to publish repositories directly to Google Cloud
// Storage\. First, publishing endpoints should be described in the aptly
// configuration file\. Each endpoint has a name and associated settings\.
//
// In order to publish to GCS, specify endpoint as `gcs:endpoint\-name:` before
// publishing prefix on the command line, e\.g\.:
//
// `aptly publish snapshot wheezy\-main gcs:test:`
//
"GcsPublishEndpoints": {
// // Endpoint Name
// "test": {
// // Bucket name
// "bucket": "test\-bucket",
// // Prefix (optional)
// // publishing under specified prefix in the bucket, defaults to
// // no prefix (bucket root)
// "prefix": "",
// // Credentials file (optional)
// // Path to a service account credentials JSON file\. If omitted,
// // Application Default Credentials are used\.
// "credentialsFile": "",
// // Service Account JSON (optional)
// // Inline service account credentials JSON content\.
// "serviceAccountJSON": "",
// // Project (optional)
// // Quota project to bill requests to\.
// "project": "",
// // Default ACLs (optional)
// // assign ACL to published files\. Useful values: `private` (default),
// // `public\-read` (public repository), or `none` (don't set ACL)\.
// "acl": "private",
// // Storage Class (optional)
// // GCS storage class, e\.g\. `STANDARD`\.
// "storageClass": "STANDARD",
// // Encryption Key (optional)
// // Customer-supplied encryption key (32-byte AES-256 key) for object operations\.
// "encryptionKey": "",
// // Disable MultiDel (optional)
// // Kept for parity with S3 settings\.
// // GCS deletes are currently performed one object at a time\.
// "disableMultiDel": false,
// // Debug (optional)
// // Enables detailed GCS operation logs
// "debug": false
// }
},
// Swift Endpoint Support
//
// aptly could be configured to publish repository directly to OpenStack Swift\. First,
@@ -616,6 +675,10 @@ gpg keyring to use when verifying Release file (could be specified multiple time
max download tries till process fails with download error
.
.TP
\-\fBwith\-appstream\fR
download AppStream (DEP\-11) metadata
.
.TP
\-\fBwith\-installer\fR
download additional not packaged installer files
.
@@ -739,6 +802,10 @@ max download tries till process fails with download error
\-\fBskip\-existing\-packages\fR
do not check file existence for packages listed in the internal database of the mirror
.
.TP
\-\fBlatest\fR
download only latest version of each package (per architecture)
.
.SH "RENAMES MIRROR"
\fBaptly\fR \fBmirror\fR \fBrename\fR \fIold\-name\fR \fInew\-name\fR
.
@@ -787,6 +854,10 @@ disable verification of Release file signatures
gpg keyring to use when verifying Release file (could be specified multiple times)
.
.TP
\-\fBwith\-appstream\fR
download AppStream (DEP\-11) metadata
.
.TP
\-\fBwith\-installer\fR
download additional not packaged installer files
.
@@ -1562,6 +1633,14 @@ $ aptly publish repo testing
Options:
.
.TP
\-\fBsigned\-by\fR
set value for Signed-By field
.
.TP
\-\fBversion\fR
version of the release
.
.TP
\-\fBacquire\-by\-hash\fR
provide index files by hash
.
@@ -1703,6 +1782,14 @@ $ aptly publish snapshot wheezy\-main
Options:
.
.TP
\-\fBsigned\-by\fR
set value for Signed-By field
.
.TP
\-\fBversion\fR
version of the release
.
.TP
\-\fBacquire\-by\-hash\fR
provide index files by hash
.
@@ -2062,6 +2149,10 @@ This command would switch published repository (with one component) named ppa/wh
Options:
.
.TP
\-\fBsigned\-by\fR
set value for Signed-By field
.
.TP
\-\fBbatch\fR
run GPG with detached tty
.
@@ -2168,6 +2259,14 @@ $ aptly publish update wheezy ppa
Options:
.
.TP
\-\fBsigned\-by\fR
set value for Signed-By field
.
.TP
\-\fBversion\fR
version of the release
.
.TP
\-\fBbatch\fR
run GPG with detached tty
.
+84 -2
View File
@@ -100,9 +100,8 @@ The legacy json configuration is still supported (and also supports comments):
// Enable metrics for Prometheus client
"enableMetricsEndpoint": false,
// Not implemented in this version.
// Enable API documentation on /docs
//"enableSwaggerEndpoint": false,
"enableSwaggerEndpoint": false,
// OBSOLETE: use via url param ?_async=true
"AsyncAPI": false,
@@ -297,6 +296,89 @@ The legacy json configuration is still supported (and also supports comments):
// }
},
// GCS Endpoint Support
//
// aptly can be configured to publish repositories directly to Google Cloud
// Storage. First, publishing endpoints should be described in the aptly
// configuration file. Each endpoint has a name and associated settings.
//
// In order to publish to GCS, specify endpoint as `gcs:endpoint-name:` before
// publishing prefix on the command line, e.g.:
//
// `aptly publish snapshot wheezy-main gcs:test:`
//
"GcsPublishEndpoints": {
// // Endpoint Name
// "test": {
// // Bucket name
// "bucket": "test-bucket",
// // Prefix (optional)
// // publishing under specified prefix in the bucket, defaults to
// // no prefix (bucket root)
// "prefix": "",
// // Credentials file (optional)
// // Path to a service account credentials JSON file. If omitted,
// // Application Default Credentials are used.
// "credentialsFile": "",
// // Service Account JSON (optional)
// // Inline service account credentials JSON content.
// "serviceAccountJSON": "",
// // Project (optional)
// // Quota project to bill requests to.
// "project": "",
// // Default ACLs (optional)
// // assign ACL to published files. Useful values: `private` (default),
// // `public-read` (public repository), or `none` (don't set ACL).
// "acl": "private",
// // Storage Class (optional)
// // GCS storage class, e.g. `STANDARD`.
// "storageClass": "STANDARD",
// // Encryption Key (optional)
// // Customer-supplied encryption key (32-byte AES-256 key) for object operations.
// "encryptionKey": "",
// // Disable MultiDel (optional)
// // Kept for parity with S3 settings.
// // GCS deletes are currently performed one object at a time.
// "disableMultiDel": false,
// // Debug (optional)
// // Enables detailed GCS operation logs
// "debug": false
// }
},
// JFrog Artifactory Endpoint Support
// aptly could be configured to publish repository directly to JFrog Artifactory. First,
// endpoints should be described in aptly.conf:
//
// The destination Artifactory repo should be of the "generic" type, not "debian".
// Authentication requires one of: user+pass, api key, or access token
//
// In order to publish to JFrog, specify endpoint as `jfrog:endpoint-name:` before
// publishing prefix on the command line, e.g.:
//
// `aptly publish snapshot wheezy-main jfrog:test:`
//
"JFrogPublishEndpoints": {
"test": {
"url": "https://artifactory.example.com/artifactory/",
"repository": "apt-local",
"username": "admin",
"password": "password",
"api_key": "api_key",
"access_token": "access_token"
}
}
// Swift Endpoint Support
//
// aptly could be configured to publish repository directly to OpenStack Swift. First,
+7
View File
@@ -9,6 +9,7 @@ import (
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
)
@@ -89,6 +90,12 @@ func (g *GpgSigner) gpgArgs() []string {
}
}
if epoch := os.Getenv("SOURCE_DATE_EPOCH"); epoch != "" {
if _, err := strconv.ParseInt(epoch, 10, 64); err == nil {
args = append(args, "--faked-system-time", epoch)
}
}
return args
}
+9
View File
@@ -7,6 +7,7 @@ import (
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"syscall"
"time"
@@ -74,6 +75,14 @@ func (g *GoSigner) Init() error {
},
}
if epoch := os.Getenv("SOURCE_DATE_EPOCH"); epoch != "" {
if sec, err := strconv.ParseInt(epoch, 10, 64); err == nil {
t := time.Unix(sec, 0).UTC()
g.signerConfig.Time = func() time.Time { return t }
g.signerConfig.NonDeterministicSignaturesViaNotation = packet.BoolPointer(false)
}
}
if g.passphraseFile != "" {
passF, err := os.Open(g.passphraseFile)
if err != nil {
+7 -4
View File
@@ -1,14 +1,17 @@
FROM debian:trixie-slim
RUN apt-get update -y && apt-get install -y --no-install-recommends curl gnupg bzip2 xz-utils ca-certificates vim procps \
golang golang-go golang-doc golang-src \
RUN echo 'deb http://deb.debian.org/debian trixie-backports main' > /etc/apt/sources.list.d/backports.list && \
apt-get update -y && apt-get install -y --no-install-recommends curl gnupg bzip2 xz-utils ca-certificates vim procps \
golang-1.25 golang-1.25-go golang-1.25-src \
make git python3 python3-requests-unixsocket python3-termcolor python3-swiftclient python3-boto3 python3-azure-storage \
g++ python3-etcd3 python3-plyvel graphviz devscripts sudo dh-golang binutils-i686-linux-gnu binutils-aarch64-linux-gnu \
binutils-arm-linux-gnueabihf bash-completion zip ruby-dev lintian npm \
libc6-dev-i386-cross libc6-dev-armhf-cross libc6-dev-arm64-cross \
gcc-i686-linux-gnu gcc-arm-linux-gnueabihf gcc-aarch64-linux-gnu \
faketime && \
apt-get clean && rm -rf /var/lib/apt/lists/*
faketime dput-ng && \
apt-get clean && rm -rf /var/lib/apt/lists/* && \
ln -sf /usr/lib/go-1.25/bin/go /usr/local/bin/go && \
ln -sf /usr/lib/go-1.25/bin/gofmt /usr/local/bin/gofmt
RUN useradd -m --shell /bin/bash --home-dir /var/lib/aptly aptly
RUN sed -i 's/#force_color_prompt=yes/force_color_prompt=yes/' /var/lib/aptly/.bashrc
Binary file not shown.
Binary file not shown.
+101
View File
@@ -0,0 +1,101 @@
from lib import BaseTest
import os
import uuid
try:
from google.cloud import storage
gcs_project = os.environ.get('GCS_PROJECT')
if gcs_project:
gcs_client = storage.Client(project=gcs_project)
else:
print('GCS tests disabled: GCS_PROJECT is not set')
gcs_client = None
except ImportError as e:
print("GCS tests disabled: can't import google.cloud.storage", e)
gcs_client = None
except Exception as e:
print('GCS tests disabled: unable to initialize GCS client', e)
gcs_client = None
class GCSTest(BaseTest):
"""
BaseTest + support for GCS
"""
gcsOverrides = {}
def __init__(self) -> None:
super(GCSTest, self).__init__()
self.bucket_name = None
self.bucket = None
self.bucket_contents = None
def fixture_available(self):
return super(GCSTest, self).fixture_available() and gcs_client is not None
def prepare(self):
# GCS bucket names must be globally unique and lower-case.
self.bucket_name = 'aptly-sys-test-' + str(uuid.uuid4()).replace('_', '-').lower()
self.bucket = gcs_client.create_bucket(self.bucket_name)
self.configOverride = {
'GcsPublishEndpoints': {
'test1': {
'bucket': self.bucket_name,
'project': gcs_project,
},
},
}
self.configOverride['GcsPublishEndpoints']['test1'].update(**self.gcsOverrides)
super(GCSTest, self).prepare()
def shutdown(self):
if self.bucket is not None:
for blob in self.bucket.list_blobs():
blob.delete()
self.bucket.delete(force=True)
super(GCSTest, self).shutdown()
def _normalize_path(self, path):
if path.startswith('public/'):
return path[7:]
return path
def check_path(self, path):
if self.bucket_contents is None:
self.bucket_contents = [blob.name for blob in self.bucket.list_blobs()]
path = self._normalize_path(path)
if path in self.bucket_contents:
return True
if not path.endswith('/'):
path = path + '/'
for item in self.bucket_contents:
if item.startswith(path):
return True
return False
def check_exists(self, path):
if not self.check_path(path):
raise Exception("path %s doesn't exist" % (path, ))
def check_not_exists(self, path):
if self.check_path(path):
raise Exception("path %s exists" % (path, ))
def read_file(self, path, mode=''):
assert not mode
path = self._normalize_path(path)
blob = self.bucket.blob(path)
return blob.download_as_text()
+97
View File
@@ -0,0 +1,97 @@
from lib import BaseTest
import uuid
import os
try:
import requests
if 'JFROG_URL' in os.environ and 'JFROG_USERNAME' in os.environ and \
os.environ['JFROG_URL'] != "" and os.environ['JFROG_USERNAME'] != "":
jfrog_ready = True
else:
print("JFrog tests disabled: JFrog creds not found in the environment (JFROG_URL, JFROG_USERNAME, JFROG_PASSWORD)")
jfrog_ready = False
except ImportError as e:
print("JFrog tests disabled: can't import requests", e)
jfrog_ready = False
class JFrogTest(BaseTest):
"""
BaseTest + support for JFrog
"""
jfrogOverrides = {}
def fixture_available(self):
return super(JFrogTest, self).fixture_available() and jfrog_ready
def prepare(self):
self.repository = "aptly-sys-test-" + str(uuid.uuid1())
self.jfrog_url = os.environ["JFROG_URL"]
self.jfrog_username = os.environ["JFROG_USERNAME"]
self.jfrog_password = os.environ["JFROG_PASSWORD"]
# Create repository via REST API
auth = (self.jfrog_username, self.jfrog_password)
create_url = f"{self.jfrog_url}/api/repositories/{self.repository}"
payload = {
"key": self.repository,
"rclass": "local",
"packageType": "generic"
}
res = requests.put(create_url, json=payload, auth=auth)
if res.status_code >= 400:
raise Exception(f"Failed to create JFrog repository: {res.text}")
self.configOverride = {"JFrogPublishEndpoints": {
"test1": {
"url": self.jfrog_url,
"repository": self.repository,
"username": self.jfrog_username,
"password": self.jfrog_password
}
}}
self.configOverride["JFrogPublishEndpoints"]["test1"].update(**self.jfrogOverrides)
super(JFrogTest, self).prepare()
def shutdown(self):
if hasattr(self, "repository"):
auth = (self.jfrog_username, self.jfrog_password)
delete_url = f"{self.jfrog_url}/api/repositories/{self.repository}"
requests.delete(delete_url, auth=auth)
super(JFrogTest, self).shutdown()
def check_path(self, path):
if path.startswith("public/"):
path = path[7:]
# Check against JFrog Artifactory API
auth = (self.jfrog_username, self.jfrog_password)
check_url = f"{self.jfrog_url}/api/storage/{self.repository}/{path}"
res = requests.head(check_url, auth=auth)
if res.status_code == 200:
return True
return False
def check_exists(self, path):
if not self.check_path(path):
raise Exception("path %s doesn't exist" % (path, ))
def check_not_exists(self, path):
if self.check_path(path):
raise Exception("path %s exists" % (path, ))
def read_file(self, path, mode=''):
assert not mode
if path.startswith("public/"):
path = path[7:]
auth = (self.jfrog_username, self.jfrog_password)
get_url = f"{self.jfrog_url}/{self.repository}/{path}"
res = requests.get(get_url, auth=auth)
res.raise_for_status()
return res.text
+2 -1
View File
@@ -1,6 +1,7 @@
azure-storage-blob
google-cloud-storage
boto
requests==2.28.2
requests==2.33.0
requests-unixsocket
python-swiftclient
flake8
+2
View File
@@ -34,7 +34,9 @@
"skipContentsPublishing": false,
"skipBz2Publishing": false,
"FileSystemPublishEndpoints": {},
"JFrogPublishEndpoints": null,
"S3PublishEndpoints": {},
"GcsPublishEndpoints": {},
"SwiftPublishEndpoints": {},
"AzurePublishEndpoints": {},
"packagePoolStorage": {}
@@ -32,7 +32,9 @@ gpg_keys: []
skip_contents_publishing: false
skip_bz2_publishing: false
filesystem_publish_endpoints: {}
jfrog_publish_endpoints: {}
s3_publish_endpoints: {}
gcs_publish_endpoints: {}
swift_publish_endpoints: {}
azure_publish_endpoints: {}
packagepool_storage: {}

Some files were not shown because too many files have changed in this diff Show More