Commit Graph

39 Commits

Author SHA1 Message Date
Andrey Smirnov 01512df853 Rework mirror update to support closing/reoping DB for the download duration
This requires splitting up import file phase as separate step in then end,
it should be pretty fast, as it only does file move (hardlink) and
DB update for new checksums.
2017-04-26 23:17:04 +03:00
Andrey Smirnov 3ce8227122 Add baseName to LinkFromPool as explicit argument 2017-04-26 23:17:04 +03:00
Andrey Smirnov 10c096fbb6 Update all other pieces for the CheckumStorage and Verify 2017-04-26 23:17:04 +03:00
Andrey Smirnov bc7903f86e Rework mirror update (download packages) implementation
`PackageDownloadTask` is just a reference to file now. Whole process
was rewritten to follow pattern: download to temp location inside the pool,
verify/update checksums, import into pool as final step.

This removes a lot of edge cases when aptly internal state might be broken
if updating from rogue mirror.

Also this changes whole memory model: package list/files are kept in memory
now during the duration of `mirror update` command and saved to disk
only in the end.
2017-04-26 23:17:03 +03:00
Andrey Smirnov 72d233b587 Final round of updates, everything except mirror download should be ready 2017-04-26 23:17:03 +03:00
Andrey Smirnov 50cf2b49bd Refactoring: use checksums instead of MD5 for pool/published
This is related to #506

As a first step, don't pass MD5 explicitly, pass checksum info object,
so that as a next step we can choose which hash to use.

There should be no functional changes so far.

Next step: stop returning explicit paths from public package pool.
2017-04-01 00:12:31 +03:00
Andrey Smirnov 7a5be6736d When contents generation fails, don't bail out
This replaces `panic` which aborts aptly execution with warning
message on console. So aptly continues publishing actions, but
`Contents` indexes might be incomplete.

Error will be printed every time contents generation is triggered.
2017-03-31 00:57:18 +03:00
Andrey Smirnov 516dd7b044 Switch to gometalinter
Only small amount of required checks is enabled,
plan is to enable more linters as issues are fixed in the code.
2017-03-23 01:51:08 +03:00
Andrey Smirnov 644d24d1cc Attempt to lower memory usage when publishing with contents. 2016-03-28 13:28:26 +03:00
Andrey Smirnov 77be7b9e3b Support for SHA-512 hashes on publishing/downloads. 2016-02-18 12:01:51 +03:00
Andrey Smirnov 8bda799545 Support for Go-style templating in format for aptly * search. #254 2015-07-02 12:19:41 +03:00
Andrey Smirnov d04f08c1cf Correctly handle multine fields in Release/non-Release files. #266 2015-06-26 03:07:33 +03:00
Andrey Smirnov cc8a87b448 Cached calculation of package contents. #142 2015-03-31 00:07:07 +03:00
Andrey Smirnov 4d622e467c Refactoring: make PackageQuery work on PackageLike objects (not necessarily Packages). #71 2015-03-18 23:25:20 +03:00
Andrey Smirnov 903e999cdc Refactor checksum parsing out of package parsing code. #71 2015-03-13 21:23:22 +03:00
Andrey Smirnov 69eff97b34 Relax .dsc checkshums parsing. #71 2015-03-13 20:53:53 +03:00
Andrey Smirnov 618d06678c Style fixes from go vet. 2015-02-22 14:36:14 +03:00
Andrey Smirnov 1f73a34a54 Don't put empty Source: fields into package stanza. #195 2015-02-09 19:23:41 +03:00
Andrey Smirnov fa2eef564c Enhance Package JSON representation with Key, ShortKey and FilesHash. #168 2015-02-05 01:47:10 +03:00
Andrey Smirnov 25d048fe49 Add Package serialization to JSON via stanza. #168 2015-02-05 01:26:55 +03:00
Andrey Smirnov 8e8ff8ba65 Revert "Make files hash a type for proper JSON serialization"
This reverts commit e138212593.
2015-02-05 00:27:14 +03:00
Sylvain Baubeau e138212593 Make files hash a type for proper JSON serialization 2014-12-18 11:14:56 +01:00
Andrey Smirnov f228ad811b Accept MD5 in package from 'MD5Sum' as well. #151 2014-11-29 16:18:23 +03:00
Andrey Smirnov 2fbf465fbf Support for .udeb in deb.Package. #108 2014-09-25 19:31:21 +04:00
Andrey Smirnov 38ea595c9a Add forceOverwrite on the path to LinkFromPool. #90 2014-08-05 15:47:23 +04:00
Andrey Smirnov da6d5b7cf8 Add 'force' to LinkFromPool method: overwrite file even if exists and different content. #90 2014-08-05 14:50:06 +04:00
Andrey Smirnov b260b0010a Refactoring: add MD5 to LinkFromPool. #15 2014-07-17 18:04:56 +04:00
Andrey Smirnov 56d777af0a Introduce regexp query matching. 2014-07-15 19:00:06 +04:00
Andrey Smirnov d523d2b415 Package.GetField implementation for querying. 2014-07-12 18:00:13 +04:00
Andrey Smirnov b42fd71acf MatchesDependency should check on Provides: as well. 2014-07-11 16:30:41 +04:00
Andrey Smirnov d262a131cc Preparation for query matching: introduce Regexp + PatternMatch. 2014-07-10 21:16:30 +04:00
Andrey Smirnov bdbb5acb11 Remove 'allMatches' on version equal. #70 2014-07-10 00:20:28 +04:00
Simon Aquino 3cf281965b Implementation of all-matches functionality + tests
When performing an *aptly snapshot pull*, users might list dependency
versions that can potentially match multiple packages in the source
snapshot. However, the current implementation of the 'snapshot pull'
command only allows one package to be pulled from a snapshot at a time
for a given dependency.

The newly implemented all-matches flag allows users to pull all the
matching packages from a source snapshot, provided that they satisfy the
version requirements indicated by the dependencies.

The all-matches flag defaults to false and only produces the described
behaviour when it is explicitly set to true.
2014-06-27 03:36:03 +01:00
Andrey Smirnov 90bba977d7 Introduce Package.ShortKey(): packages should have no conflict on that key in one list. #60 2014-05-29 18:00:14 +04:00
Andrey Smirnov f007465d18 Change the way package key works: now it includes FilesHash. #60
Now duplicate packages (the same name/version/arch) but with different set of files
would be handled as separate entities.
2014-05-29 16:49:05 +04:00
Andrey Smirnov 5301e8a341 Fix pool directory for packages with version in Source: field. #44 2014-05-10 17:25:44 +04:00
Andrey Smirnov 1bac201687 Refactoring: build pool path in Package. #8 2014-04-22 16:20:51 +04:00
Andrey Smirnov ff045f9a48 Fixups after renaming debian -> deb. #21 2014-04-07 21:22:58 +04:00
Andrey Smirnov fd662c9275 Rename debian -> deb. #21 2014-04-07 21:15:13 +04:00