Several sections of the code *required* a LocalPackagePool, but they
could still perform their operations with a standard PackagePool.
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
This change makes it possible to publish multiple distributions
with packages named the same but with different content by changing
structure of the generated pool hierarchy. The option not enabled
by default as this changes the structure of the output which could
break the expectations of other tools.
The output doesn't actually depend on the reflists, and loading them for
every published repo starts to take substantial time and memory.
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
This will properly close the db and, more particularly, flush out any
profile files being written. Otherwise, they can end up empty or
truncated.
Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
While testing out Aptly, the `apt-get` client complains with the following error, since the `codename` was switched from the InRelease files that are baked out by Aptly:
```
E: Repository 'http://debianrepo.example.com/bionic testing InRelease' changed its 'Codename' value from '' to 'testing'
```
For any action which is multi-step (requires updating more than 1 DB
key), use transaction to make update atomic.
Also pack big chunks of updates (importing packages for importing and
mirror updates) into single transaction to improve aptly performance and
get some isolation.
Note that still layers up (Collections) provide some level of isolation,
so this is going to shine with the future PRs to remove collection
locks.
Spin-off of #459
This is spin-off of changes from #459.
Transactions are not being used yet, but batches are updated to work
with the new API.
`database/` package was refactored to split abstract interfaces and
implementation via goleveldb. This should make it easier to implement
new database types.
Apply retries as global, config-level option `downloadRetries` so that
it can be applied to any aptly command which downloads objects.
Unwrap `errors.Wrap` which is used in downloader.
Unwrap `*url.Error` which should be the actual error returned from the
HTTP client, catch more cases, be more specific around failures.
* aptly can sign and verify without issues with GnuPG 1.x and 2.x
* aptly auto-detects GnuPG version and adapts accordingly
* aptly automatically finds suitable GnuPG version
Majority of the work was to get unit-tests which can work with GnuPG 1.x & 2.x.
Locally I've verified that aptly supports GnuPG 1.4.x & 2.2.x. Travis CI
environment is based on trusty, so it runs gpg2 tests with GnuPG 2.0.x.
Configuration parameter gpgProvider now supports three values for GnuPG:
* gpg (same as before, default): use GnuPG 1.x if available (checks gpg, gpg1),
otherwise uses GnuPG 2.x; for aptly users who already have GnuPG 1.x
environment (as it was the only supported version) nothing should change; new
users might start with GnuPG 2.x if that's their installed version
* gpg1 looks for GnuPG 1.x only, fails otherwise
* gpg2 looks for GnuPG 2.x only, fails otherwise
This is not a complete fix, but the easiest first step.
During `db cleanup`, aptly is loading every repo/mirror/... into memory,
and even though each object is processed only once, collection holds
a reference to all the loaded objects, so they won't be GC'd until
process exits.
CollectionFactory.Flush() releases pointers to collection objects,
making objects egligble for GC.
This is not a complete fix, as during iteration we could have tried
to release a link to every object being GCed and that would have
helped much more.
