This adds support for storing packages directly on Azure, with no truly
"local" (on-disk) repo used. The existing Azure PublishedStorage
implementation was refactored to move the shared code to a separate
context struct, which can then be re-used by the new PackagePool. In
addition, the files package's mockChecksumStorage was made public so
that it could be used in the Azure PackagePool tests as well.
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
This adds a new configuration setting: AzurePublishEndpoints, similar
to the existing S3PublishEndpoints and SwiftPublishEndpoints.
For each endpoint, the following has to be defined:
- accountName
- accountKey
- container
- prefix
Azure tests require the following environment variables to be set:
- AZURE_STORAGE_ACCOUNT
- AZURE_STORAGE_ACCESS_KEY
With either of these not set, Azure-specific tests are skipped.
This is spin-off of changes from #459.
Transactions are not being used yet, but batches are updated to work
with the new API.
`database/` package was refactored to split abstract interfaces and
implementation via goleveldb. This should make it easier to implement
new database types.
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.
Unwrap `errors.Wrap` which is used in downloader.
Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
* aptly can sign and verify without issues with GnuPG 1.x and 2.x
* aptly auto-detects GnuPG version and adapts accordingly
* aptly automatically finds suitable GnuPG version
Majority of the work was to get unit-tests which can work with GnuPG 1.x & 2.x.
Locally I've verified that aptly supports GnuPG 1.4.x & 2.2.x. Travis CI
environment is based on trusty, so it runs gpg2 tests with GnuPG 2.0.x.
Configuration parameter gpgProvider now supports three values for GnuPG:
* gpg (same as before, default): use GnuPG 1.x if available (checks gpg, gpg1),
otherwise uses GnuPG 2.x; for aptly users who already have GnuPG 1.x
environment (as it was the only supported version) nothing should change; new
users might start with GnuPG 2.x if that's their installed version
* gpg1 looks for GnuPG 1.x only, fails otherwise
* gpg2 looks for GnuPG 2.x only, fails otherwise
Init is actually never called and I have no clue why it is there if it is
not called.
Take this opportunity to introduce a New function which only does the
helper lookup and panics iff that fails. Panic may be a bit too aggressive,
but seems the most certain way to get out of not finding a suitable gpg1
binary.
There are two fixes here:
1. Abort package download immediately as ^C is pressed.
2. Import all the already downloaded files into package pool,
so that next time mirror is updated, aptly won't download them
once again.
Allow database to be initialized without opening, unify all the
open paths to retry on failure.
In API router make sure open requests are matched with acks in explicit
way.
This also enables re-open attempts in all the aptly commands, so it
should make running aptly CLI much easier now hopefully.
Fix up system tests for oldoldstable ;)
Now that there's an official Go AWS SDK from Amazon, use that instead of
goamz. goamz isn't getting much love these days.
Implement support for STS credentials, as in assumed roles and EC2
instance profiles. The configuration is extended to support a session
token, though I'm not sure why anyone would put temporary credentials in
a configuration file. More likely, no credentials will be explicitly
configured at all, and they will be discovered through the standard SDK
mechanisms described at
<https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs>.
Resolves#342.
