Commit Graph

29 Commits

Author SHA1 Message Date
André Roth 5cf8c54cb2 fix test 2024-06-20 23:40:46 +02:00
Ludovico Cavedon eeb5bd79d0 s3: fix test 2024-02-06 20:49:35 +01:00
Ludovico Cavedon fad660450c Cache bucket content by prefix
When a publishing uses a publish prefix, instead of listing the contents
of the whole bucket under the storage prefix, only list the contents of
the bucket under the storage prefix and publish prefix, and cache it by
publish prefix.
This speeds up publish operations under a prefix.
2024-02-06 20:49:35 +01:00
Sylvain Baubeau 3aaf0a8c44 Switch to aws-sdk-go-v2 2023-10-24 15:30:52 +02:00
boxjan 268c39ea8c add forceVirtualHostedStyle for stores which only support virtual hosted style 2022-09-09 09:02:52 +02:00
Andrey Smirnov f104e53fd4 Ignore 'NoSuchBucket' error when deleting S3 objects
Also ignore any removal errors when `-force-drop` is used.
2019-01-23 18:17:08 +03:00
Oliver Sauder 2e52692ba6 Test LinkFromPool with nested filenames 2018-07-06 15:02:37 +02:00
Andrey Smirnov b8c5303fdb Fix paths after repository transfer to aptly-dev 2018-04-18 21:19:43 +03:00
Oliver Sauder b2bf4f7884 Adjust FileExists to differentiate between error and actual file existence 2017-11-30 09:46:02 +01:00
Oliver Sauder d6b4b795a5 Fix linting errors 2017-11-30 09:46:02 +01:00
Oliver Sauder 7498fd8fc8 Extend s3 storage with link and file exists methods 2017-11-30 09:46:02 +01:00
Moritz Bechler 308ea83cc0 S3 backend: include path prefix in removal requests.
DELETE requests, both for temporary files and no longer referenced
packages, lacked the configured path prefix and therefor were not
removed if a prefix is configured.
2017-11-13 14:48:25 +01:00
Andrey Smirnov 790d85881b Fix S3 path caching double-prefix
Original PR: #621
Fixes: #619

I've added unit-test to Martyn's PR.

Without this fix, if `prefix` is set on S3 publish endpoint,
aptly would incorrectly build path cache and re-upload every object
on publish.
2017-08-16 23:57:41 +03:00
Andrey Smirnov 186bb2dff0 Add flag to disable/enable support for legacy pool paths
Legacy pool paths are enabled by default, but for new aptly installations
(when aptly config is first generated), it would be disabled explicitly.
2017-04-26 23:37:31 +03:00
Andrey Smirnov 3ce8227122 Add baseName to LinkFromPool as explicit argument 2017-04-26 23:17:04 +03:00
Andrey Smirnov 10c096fbb6 Update all other pieces for the CheckumStorage and Verify 2017-04-26 23:17:04 +03:00
Andrey Smirnov f4ff8d957f Fix S3 published storage to use new PackagePool interface
Change PackagePool to return Seeker interface from Open call.
2017-04-26 23:17:03 +03:00
Andrey Smirnov 50cf2b49bd Refactoring: use checksums instead of MD5 for pool/published
This is related to #506

As a first step, don't pass MD5 explicitly, pass checksum info object,
so that as a next step we can choose which hash to use.

There should be no functional changes so far.

Next step: stop returning explicit paths from public package pool.
2017-04-01 00:12:31 +03:00
Andrey Smirnov 75b860e0b1 Support SigV2 and S3 debug for publishing. 2016-03-20 20:11:19 +03:00
Andrey Smirnov 02b937ad17 Fix unit-tests. 2016-02-08 14:42:30 +03:00
Andrey Smirnov 38dfe3435a For plusWorkaround, correctly handle cleanup, deletions. #239 2015-05-29 02:13:59 +03:00
Andrey Smirnov 0791c88a02 Support for custom endpoints and multi del disabling. #218 2015-04-17 01:16:50 +03:00
Andrey Smirnov 903d4cefba gofmt -s 2015-02-22 14:29:09 +03:00
Chris Read daf887e54f Upgrade gocheck 2014-11-05 13:27:15 -06:00
Andrey Smirnov aeae6009c4 Introduce plusWorkaround: generate copy of file with '+' -> ' ' to workaround S3/apt bug. #98 2014-10-01 21:32:56 +04:00
Andrey Smirnov 8aa1954ba7 Support for custom storage class and encryption method. #105 2014-10-01 19:16:15 +04:00
Andrey Smirnov da6d5b7cf8 Add 'force' to LinkFromPool method: overwrite file even if exists and different content. #90 2014-08-05 14:50:06 +04:00
Andrey Smirnov 076ecd586f Fix style issues. #15 2014-07-17 18:09:13 +04:00
Andrey Smirnov c54406e29f First version of PublishedStorage for S3. #15 2014-07-17 18:05:38 +04:00