See #761
aptly had a concept of loading small amount of info per each object
into memory once collection is accessed for the first time.
This might have simplified some operations, but it doesn't scale well
with huge aptly databases.
This is just intermediate step towards better memory management -
list of objects is not loaded unless some method is called.
`ForEach` method (mainly used in cleanup) is reimplemented to
iterate over database without ever loading all the objects into memory.
Memory was even worse with previous approach, as for each item usually
`LoadComplete()` is called, which pulls even more data into memory
and item stays in memory till the end of the iteration as it is referenced
from `collection.list`.
For the subsequent PR: reimplement `ByUUID()` and probably other methods
to avoid loading all the items into memory, at least for all the collecitons
except for published repos. When published repository is being loaded, it
might pull source local repo which in turn would trigger loading for all the
local repos which is not acceptable.
This is not a complete fix, but the easiest first step.
During `db cleanup`, aptly is loading every repo/mirror/... into memory,
and even though each object is processed only once, collection holds
a reference to all the loaded objects, so they won't be GC'd until
process exits.
CollectionFactory.Flush() releases pointers to collection objects,
making objects egligble for GC.
This is not a complete fix, as during iteration we could have tried
to release a link to every object being GCed and that would have
helped much more.
These unit-tests cover operations via both PGP providers:
built-in `openpgp` and external `gpg`.
Next step is to run these tests for gpg1 & gpg2
as separate entities.
This test has been failing very often because of changes in nvidia
repository. As this test is not related to filtering
remove number of filtered packages from output for a more robust test.
This updates previous work in #739 to build
Debian packages and zip files for other OS.
All the build artifacts are uploaded to S3
public bucket `aptly-nightly` so that there's
archive for all the builds.
All `.deb` packages are automatically uploaded
to repo.aptly.info/nightly on build.
- new phony target build: same as install but creating aptly-$version and
putting it into a build/ subdir
- env TRAVIS_TAG in the makefile now overrides the TAG lookup, this ensures
that the tag travis is working with is actually the one being used to
construct the version number
- subdir is gitignored
- travis runs new target - lists artifacts - deploys artifacts to github
all of the above only happens on builds that are a tag and DEPLOY_BINARIES
is set to yes (which is only the case for latest stable go version)
package count chainged again -.-
I am working on a fixture set for repositories so we can stop talking to
live repositories. that's quite the undertaking though, so let's fix the
output reference to unbreak the test in the meantime
Init is actually never called and I have no clue why it is there if it is
not called.
Take this opportunity to introduce a New function which only does the
helper lookup and panics iff that fails. Panic may be a bit too aggressive,
but seems the most certain way to get out of not finding a suitable gpg1
binary.
Newer versions of debian and ubuntu come with gpg pointing to gpg2.
We can currently only handle gpg1 CLIs though. Luckily the old gpg is still
available in the package gnupg1 (providing bin/gpg1).
As a bit of a stop-gap, until #657 can be resolved properly, we'll detect
the version of bin/gpg. If it is unsuitable we'll fall back and try
bin/gpg1. If neither is found to be suitable the signer/verifier will
not work.
Same applies to gpgv/gpgv1.
1. Don't run long steps for Go versions other than 1.9 & 1.10
according to Golang Release Policy (two latest versions).
2. Switch to codecov.io, collect coverage only on Go 1.10 which
has fixes for multi-module coverage & ./... ignoring vendor.
3. Simplify Makefile.
Andrey Smirnov
Oliver Sauder
aviau
Harald Sitter
Oliver Sauder
tomascassidy
Harald Sitter
Strajan Sebastian Ioan