Oliver Sauder
e504fdcd54
Build src path on basis of storage prefix when symlinking
2017-11-30 09:46:02 +01:00
Oliver Sauder
d6b4b795a5
Fix linting errors
2017-11-30 09:46:02 +01:00
Oliver Sauder
7498fd8fc8
Extend s3 storage with link and file exists methods
2017-11-30 09:46:02 +01:00
André Roth
e07912770e
Extend PublishedStorage interface for Acquire-By-Hash
...
Signed-off-by: André Roth <neolynx@gmail.com >
2017-11-30 09:46:02 +01:00
Andrey Smirnov
340d1fdd7c
Fix formatting
2017-11-19 19:53:24 +03:00
Moritz Bechler
308ea83cc0
S3 backend: include path prefix in removal requests.
...
DELETE requests, both for temporary files and no longer referenced
packages, lacked the configured path prefix and therefor were not
removed if a prefix is configured.
2017-11-13 14:48:25 +01:00
Andrey Smirnov
2d66a4ca0a
Enforce SSE/StorageClass in PUT Object Copy
...
"RenameFile" is implemented in S3 using `PUT Object Copy`, which
should enforce SSE/StorageClass same way as regular `PUT Object`.
Fixes : #647
2017-09-28 18:30:50 +03:00
Andrey Smirnov
790d85881b
Fix S3 path caching double-prefix
...
Original PR: #621
Fixes : #619
I've added unit-test to Martyn's PR.
Without this fix, if `prefix` is set on S3 publish endpoint,
aptly would incorrectly build path cache and re-upload every object
on publish.
2017-08-16 23:57:41 +03:00
Andrey Smirnov
51213899b7
More Go linters enabled, issues fixed
...
Ref: #528
Enables "staticcheck", "varcheck", "structcheck", "aligncheck"
2017-05-03 18:23:14 +03:00
Andrey Smirnov
bae3f949b4
Enable gosimple and ineffasign linters
2017-04-27 18:34:30 +03:00
Andrey Smirnov
186bb2dff0
Add flag to disable/enable support for legacy pool paths
...
Legacy pool paths are enabled by default, but for new aptly installations
(when aptly config is first generated), it would be disabled explicitly.
2017-04-26 23:37:31 +03:00
Andrey Smirnov
3ce8227122
Add baseName to LinkFromPool as explicit argument
2017-04-26 23:17:04 +03:00
Andrey Smirnov
10c096fbb6
Update all other pieces for the CheckumStorage and Verify
2017-04-26 23:17:04 +03:00
Andrey Smirnov
2535367c3c
Update Swift published storage to work with new package pool
2017-04-26 23:17:03 +03:00
Andrey Smirnov
f4ff8d957f
Fix S3 published storage to use new PackagePool interface
...
Change PackagePool to return Seeker interface from Open call.
2017-04-26 23:17:03 +03:00
Andrey Smirnov
50cf2b49bd
Refactoring: use checksums instead of MD5 for pool/published
...
This is related to #506
As a first step, don't pass MD5 explicitly, pass checksum info object,
so that as a next step we can choose which hash to use.
There should be no functional changes so far.
Next step: stop returning explicit paths from public package pool.
2017-04-01 00:12:31 +03:00
Andrey Smirnov
516dd7b044
Switch to gometalinter
...
Only small amount of required checks is enabled,
plan is to enable more linters as issues are fixed in the code.
2017-03-23 01:51:08 +03:00
Andrey Smirnov
11d828b3b1
Add govet/golint into Travis CI build
...
Fix current issues
2017-03-22 21:49:16 +03:00
Andrey Smirnov
2fe8cfdc12
Allow credentials for S3 SigV2 to be specified in config once again. #356
2016-03-28 12:52:50 +03:00
Andrey Smirnov
d9a1299f6b
Replace object listing with SDK-standard iteration.
2016-03-24 13:04:16 +03:00
Andrey Smirnov
75b860e0b1
Support SigV2 and S3 debug for publishing.
2016-03-20 20:11:19 +03:00
Andrey Smirnov
505da096e6
Remove S3 retrying client which is leftover from goamz times.
...
Also workaround go vet warnings in s3/sever_test.go
2016-02-18 12:03:04 +03:00
Andrey Smirnov
8c9cc41099
Fix nil pointer dereference on S3 publishing. #338
2016-02-14 14:52:49 +03:00
Andrey Smirnov
02b937ad17
Fix unit-tests.
2016-02-08 14:42:30 +03:00
Phil Frost
640bd2b530
Use official AWS SDK; support STS credentials
...
Now that there's an official Go AWS SDK from Amazon, use that instead of
goamz. goamz isn't getting much love these days.
Implement support for STS credentials, as in assumed roles and EC2
instance profiles. The configuration is extended to support a session
token, though I'm not sure why anyone would put temporary credentials in
a configuration file. More likely, no credentials will be explicitly
configured at all, and they will be discovered through the standard SDK
mechanisms described at
<https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs >.
Resolves #342 .
2016-02-03 15:13:01 -05:00
Andrey Smirnov
a294a91685
Cache filepath list in s3.LinkFromPool instead of doing Get checks #297
...
This speeds up publishing with many files already present in the pool
2015-10-01 14:02:32 +03:00
Andrey Smirnov
8fdc222196
Fix retry policy. #297
2015-09-24 13:21:13 +03:00
Andrey Smirnov
1e4d825d36
Disable keep alives, fix return on last retry. #297
2015-09-24 12:21:31 +03:00
Andrey Smirnov
38dfe3435a
For plusWorkaround, correctly handle cleanup, deletions. #239
2015-05-29 02:13:59 +03:00
Andrey Smirnov
313c71dff6
Rework s3 retry policy by copying sources from goamz :( #255
2015-05-29 01:47:02 +03:00
Andrey Smirnov
9d298dee51
Remove deadline timeout. #255
2015-05-28 12:52:29 +03:00
Andrey Smirnov
2f1df39204
Use S3 retrying transport. #255
2015-05-28 11:45:37 +03:00
Andrey Smirnov
0791c88a02
Support for custom endpoints and multi del disabling. #218
2015-04-17 01:16:50 +03:00
Andrey Smirnov
903d4cefba
gofmt -s
2015-02-22 14:29:09 +03:00
Chris Read
daf887e54f
Upgrade gocheck
2014-11-05 13:27:15 -06:00
Andrey Smirnov
aeae6009c4
Introduce plusWorkaround: generate copy of file with '+' -> ' ' to workaround S3/apt bug. #98
2014-10-01 21:32:56 +04:00
Andrey Smirnov
8aa1954ba7
Support for custom storage class and encryption method. #105
2014-10-01 19:16:15 +04:00
Andrey Smirnov
da6d5b7cf8
Add 'force' to LinkFromPool method: overwrite file even if exists and different content. #90
2014-08-05 14:50:06 +04:00
Andrey Smirnov
77d6a10984
Implementation of Rename method for S3 PublishedStorage. #15
2014-07-26 01:11:23 +04:00
Andrey Smirnov
71d90947c9
Remove debugging output. #15
2014-07-22 00:27:38 +04:00
Andrey Smirnov
237d25fe5b
Fix issue with ETag/MD5 comparison, add extra info in error messages. #15
2014-07-21 17:43:42 +04:00
Andrey Smirnov
96e878a2e0
Separate out LocalPublishedStorage interface. #15
2014-07-18 17:44:54 +04:00
Andrey Smirnov
076ecd586f
Fix style issues. #15
2014-07-17 18:09:13 +04:00
Andrey Smirnov
c54406e29f
First version of PublishedStorage for S3. #15
2014-07-17 18:05:38 +04:00
