Commit Graph

1884 Commits

Author SHA1 Message Date
Andrey Smirnov
77d7c3871a Consistently use transactions to update database
For any action which is multi-step (requires updating more than 1 DB
key), use transaction to make update atomic.

Also pack big chunks of updates (importing packages for importing and
mirror updates) into single transaction to improve aptly performance and
get some isolation.

Note that still layers up (Collections) provide some level of isolation,
so this is going to shine with the future PRs to remove collection
locks.

Spin-off of #459
2019-08-11 00:11:53 +03:00
Andrey Smirnov
67e38955ae Refactor database code to support standalone batches, transactions.
This is spin-off of changes from #459.

Transactions are not being used yet, but batches are updated to work
with the new API.

`database/` package was refactored to split abstract interfaces and
implementation via goleveldb. This should make it easier to implement
new database types.
2019-08-09 00:46:40 +03:00
Andrey Smirnov
26098f6c8d Print redirects being followed, drop mirror.yandex.ru.
Use CDN-backed Debian mirror to make tests run faster hopefully for
everyone. Redirects might be important to know what exactly is going on
when items are being downloaded.
2019-08-07 21:10:04 +03:00
Andrey Smirnov
021b6f694b Fix flakey tests related to identity name ordering. 2019-08-07 20:47:52 +03:00
Andrey Smirnov
f0a370db24 Rework HTTP downloader retry logic
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.

Unwrap `errors.Wrap` which is used in downloader.

Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
2019-08-07 20:23:05 +03:00
Andrey Smirnov
2e7f624b34 Add test for publishing with non-empty Origin & Label
See also #848
2019-07-19 22:58:56 +03:00
Shengjing Zhu
b63c0c7dfc Update AUTHORS 2019-07-15 21:51:09 +03:00
Shengjing Zhu
906cbf1e6f Fix time.Time msgpack decoding backwards compatibility
See https://github.com/ugorji/go-codec/issues/269
2019-07-15 21:51:09 +03:00
Shengjing Zhu
5aefc741f2 Add codec tag to fields which are ignored in new codec package
github.com/ugorji/go/codec 1.1.4 ignores field with json:"-" tag
2019-07-15 21:51:09 +03:00
Shengjing Zhu
5c28ea3064 Update github.com/ugorji/go to v1.1.4 2019-07-15 21:51:09 +03:00
Andrey Smirnov
70cd11e30f Revert "Don't remove API file socket if it exists and it's usable"
See PR #807

Fixes: #849

This reverts commit 22848b010d.
2019-07-13 00:45:54 +03:00
Andrey Smirnov
94a72b23ff Update Go AWS SDK to the latest version 2019-07-13 00:19:00 +03:00
Andrey Smirnov
d08be990ef Skip uploading release versions of aptly to nightly repo
This breaks releases, as two versions of the package with same version
might end up in internal.aptly.info.
2019-07-11 01:52:00 +03:00
Andrey Smirnov
ca5b7758ce Print when test is skipped 2019-07-11 00:49:36 +03:00
Andrey Smirnov
bb1def2910 Try Travis on xenial workers 2019-07-11 00:16:20 +03:00
Andrey Smirnov
673abae1be Update system tests after Debian buster was released. 2019-07-10 22:27:11 +03:00
Andrey Smirnov
3b8c067e70 Merge pull request #850 from smira/no-bintray
Bintray no longer used for artifacts. [ci skip]
2019-07-10 19:54:09 +03:00
Andrey Smirnov
528459eeb2 Bintray no longer used for artifacts. [ci skip] 2019-07-10 19:53:07 +03:00
Andrey Smirnov
bc1ab4e55c Merge pull request #847 from smira/fix-repo-name
Fix repo name in release script
v1.4.0
2019-07-06 16:03:14 +03:00
Andrey Smirnov
5aefd0b393 Fix repo name in release script 2019-07-06 16:02:43 +03:00
Andrey Smirnov
7bc53a4253 Merge pull request #846 from smira/fix-releases-api-key
Fix releases API key
2019-07-06 00:05:25 +03:00
Andrey Smirnov
8b12dccd76 Fix releases API key 2019-07-06 00:04:46 +03:00
Andrey Smirnov
952afb6040 Merge pull request #845 from smira/upload-artifacts-fix
Fix upload artifacts script to fail, add release upload script
2019-07-05 22:02:59 +03:00
Andrey Smirnov
56ca5e9e62 Temporary disable test as linux.dell.com is NXDOMAIN 2019-07-05 21:32:41 +03:00
Andrey Smirnov
a834461752 Fix upload artifacts script to fail, add release upload script
This should improve reliability
2019-07-05 20:08:31 +03:00
Andrey Smirnov
37166af321 Merge pull request #842 from aptly-dev/bump-go-version
Bump Go versions for Travis, fix tests
2019-07-04 00:44:52 +03:00
Andrey Smirnov
2c91bcdc30 Bump Go versions for Travis, fix tests
Replace gometalinter with golangci-lint.

Fix system tests (wheezy is gone, replace with stretch).

Fix linter warnings.
2019-07-04 00:16:12 +03:00
Andrey Smirnov
e2d6a53de5 Merge pull request #803 from stb-tester/deterministic-stanza-WriteTo
Stanza.WriteTo: Sort extra fields alphabetically
2019-01-25 17:34:34 +03:00
Andrey Smirnov
89537b1521 Merge branch 'master' into deterministic-stanza-WriteTo 2019-01-25 01:27:31 +03:00
Oliver Sauder
152b3cae90 Merge pull request #808 from aptly-dev/797-no-such-bucket-s3
Ignore 'NoSuchBucket' error when deleting S3 objects
2019-01-24 08:09:29 +01:00
Andrey Smirnov
f104e53fd4 Ignore 'NoSuchBucket' error when deleting S3 objects
Also ignore any removal errors when `-force-drop` is used.
2019-01-23 18:17:08 +03:00
William Manley
fd99ae0e59 Merge branch 'master' into deterministic-stanza-WriteTo 2019-01-21 13:48:07 +00:00
Andrey Smirnov
4b6c159e3a Vendor update github.com/pkg/errors 2019-01-20 22:54:13 +03:00
Andrey Smirnov
50f8cfbc15 Merge pull request #807 from aptly-dev/806-file-socket
Don't remove API file socket if it exists and usable
2019-01-20 22:52:30 +03:00
Andrey Smirnov
22848b010d Don't remove API file socket if it exists and it's usable 2019-01-20 00:01:44 +03:00
Andrey Smirnov
3b5840e248 Fix linter list and fix errors discovered by new staticcheck 2019-01-20 00:01:17 +03:00
William Manley
f955707201 Add William Manley (@wmanley) to AUTHORS
My measly contribution hardly merits it but it's a requirement in
`CONTRIBUTING.md`.
2019-01-08 15:14:39 +00:00
William Manley
86dc10028f Stanza.WriteTo: Sort extra fields alphabetically
This makes the output deterministic.  This is important to me as I am
using `Packages` index files as a kind of lockfile and committing it
to my git repository.  Without this we get a lot of noise in the diff
whenever the file is regenerated because
[go randomises map iteration order][1].

[1]: https://nathanleclaire.com/blog/2014/04/27/a-surprising-feature-of-golang-that-colored-me-impressed/
2019-01-08 15:12:34 +00:00
Andrey Smirnov
a64807efda Merge pull request #779 from aptly-dev/pgp-finder
Compatibility with GnuPG 1.x and 2.x, auto-detect GnuPG version
2018-10-10 17:27:52 +03:00
Andrey Smirnov
61e00b5fbd Test updates for Travis CI
Travis is running Trusty with GPG 2.0.x, which is
much different from 2.1.x.

Add tests for default key signing.

Add test for gpg1/2 in functional.
2018-10-10 01:34:58 +03:00
Andrey Smirnov
1b2fccb615 Compatibility with GnuPG 1.x and 2.x, auto-detect GnuPG version
* aptly can sign and verify without issues with GnuPG 1.x and 2.x
* aptly auto-detects GnuPG version and adapts accordingly
* aptly automatically finds suitable GnuPG version

Majority of the work was to get unit-tests which can work with GnuPG 1.x & 2.x.
Locally I've verified that aptly supports GnuPG 1.4.x & 2.2.x. Travis CI
environment is based on trusty, so it runs gpg2 tests with GnuPG 2.0.x.

Configuration parameter gpgProvider now supports three values for GnuPG:

* gpg (same as before, default): use GnuPG 1.x if available (checks gpg, gpg1),
otherwise uses GnuPG 2.x; for aptly users who already have GnuPG 1.x
environment (as it was the only supported version) nothing should change; new
users might start with GnuPG 2.x if that's their installed version

* gpg1 looks for GnuPG 1.x only, fails otherwise

* gpg2 looks for GnuPG 2.x only, fails otherwise
2018-10-10 01:34:00 +03:00
Oliver Sauder
702c1ff217 Merge pull request #680 from sliverc/with_installer
Add support to mirror non package installer files
2018-09-27 09:50:37 +02:00
Oliver Sauder
d1b2814ec6 Merge branch 'master' into with_installer 2018-09-27 09:35:09 +02:00
Andrey Smirnov
ec57d1786c Merge pull request #780 from aptly-dev/773-non-armored-sig
Support for non-armored detached signatures
2018-09-26 16:37:42 +03:00
Andrey Smirnov
9f7c1f90ec Support for non-armored detached signatures 2018-09-26 01:36:52 +03:00
Oliver Sauder
e23e30eb44 Merge branch 'master' into with_installer 2018-09-21 13:26:15 +02:00
Andrey Smirnov
2b4a61b84c Merge pull request #778 from aptly-dev/go-1-11
Bump Go versions with Go 1.11 release
2018-09-20 20:15:52 +03:00
Andrey Smirnov
fbafde6e27 Bump Go versions with Go 1.11 release 2018-09-19 01:23:17 +03:00
Andrey Smirnov
14e5a75d35 Merge pull request #776 from urpylka/master
Replace Docker container w aptly & nginx in README
2018-09-14 23:49:58 +03:00
Artem Smirnov
ea32d8627e Update AUTHORS 2018-09-14 01:29:11 +03:00