Commit Graph

435 Commits

Author SHA1 Message Date
Oliver Sauder 5d301fb1b7 Prepare archive root when editing it 2017-11-27 11:08:31 +01:00
Stefan Felkel 8a4d866810 #679: added *.buildinfo file to processedFile list (will be removed, afterwards) 2017-11-24 14:23:26 +01:00
Felix Abecassis e682639b20 Handle SHA512 in Release files
Fix: #656
2017-11-08 11:54:19 +03:00
Harald Sitter 1885cbd6a2 make deb reader handle new control.tar options introduced in dpkg 1.17.6
newly supported is uncompressed control.tar and xz compressed
control.tar.xz. latter is used by ubuntu for dbgsym ddebs.

Fixes #655
2017-10-31 14:43:00 +01:00
Harald Sitter 46c2182ade fix linting by using new maligned linter instead of aligncheck
upstream switched the alignment check backend and in doing so fails to run
if the old backend is defined in the config.

also skip alignment linting on a struct we use for byte decoding as we have
no choice in its member order.
2017-10-31 12:24:31 +01:00
Andrey Smirnov 0d94f29c27 Allow using files from the pool while importing source packages
Sometimes source packages reference files already present in the pool.

Allow for those file to be omitted when importing packages either via
`repo add` or `repo include`. If file is missing, aptly would make
an attempt to look up file in the package pool (by checksum) and
use it.

Fixes: #278
2017-09-29 22:39:51 +03:00
Andrey Smirnov b4deedda01 Merge branch 'master' into skipCleanup 2017-09-27 00:14:24 +03:00
Andrey Smirnov e5198178a5 Fix incomplete dependencies with follow-all-variants
When `-dep-follow-all-variants` option is enabled, dependency resolving
process shouldn't stop even if dependency is already satisfied - there
mgiht be other ways to satisfy dependency.

Also fix issue with parsing multiarch specs like
`python:any`.
2017-09-26 00:09:15 +03:00
Andrey Smirnov 6d2f265980 Prefer exact match on package name over provides match
When searching for packages which might satisfy given dependency,
aptly was first returning packages which `Provides` mentioned
name. By default aptly is picking up only first match (unless
follow all variants options is enabled), so `Provides:` takes
precedence over exact package name match.

Invert this logic by searching first for package name match.
2017-09-25 18:24:45 +03:00
Andrey Smirnov 31f4af5722 Merge branch 'master' into skipCleanup 2017-09-20 17:53:24 +03:00
Andrey Smirnov 9ca81ff3bc Fix lint warning & add Go 1.9 to the mix 2017-09-15 22:54:39 +03:00
Andrey Smirnov a27b489ba2 Allow package queries to return duplicate entries on PackageCollection
Fixes #446
2017-08-17 00:40:34 +03:00
Ludovico Cavedon d6a3917141 Add -skip-cleanup option for publish commands.
Allow skipping unreferenced files cleanup on publish switch/update/drop
via the -skip-cleanup command line option.
Also support API SkipCleanup parameter.

Fixes #570.
2017-08-15 19:08:17 -07:00
Andrey Smirnov a584b2e058 Fix bug with PoolPath field being overwritten on mirror update
While updating mirror, if package file is already in pool path,
field `PoolPath` was left as empty which results in package file
being unavailable later on while publishing.
2017-08-11 20:05:55 +03:00
Andrey Smirnov 84ef963d7d Trim slashes while parsing publish prefix
Fixes: #607
2017-08-09 01:26:47 +03:00
Andrey Smirnov 0e9f966dd1 Fix up other code to support new GPG provider structure 2017-07-21 01:01:58 +03:00
Oliver Sauder e3f1880ad4 Added support for NotAutomatic, ButAutomaticUpgrades and Origin fields 2017-07-05 15:08:02 +02:00
Andrey Smirnov c13eb99925 Style fixups 2017-07-05 00:36:48 +03:00
Andrey Smirnov 211ac0501f Rework the way database is open/re-open in aptly
Allow database to be initialized without opening, unify all the
open paths to retry on failure.

In API router make sure open requests are matched with acks in explicit
way.

This also enables re-open attempts in all the aptly commands, so it
should make running aptly CLI much easier now hopefully.

Fix up system tests for oldoldstable ;)
2017-07-05 00:17:48 +03:00
Andrey Smirnov 12a6b0ceb8 Merge pull request #575 from smira/pgp-refactoring
Refactor GPG signer/verifier
2017-05-24 19:24:38 +03:00
Andrey Smirnov cafb89f30f Re-work the way checksum matching works against Release file
Break up URL into base part and relative path. Match checksum against relative path
and never against full URL.

This might be fixing security issue if aptly was incorrectly matching against
wrong part of Release file.
2017-05-23 03:00:15 +03:00
Andrey Smirnov 1be8d39105 Refactor GPG signer/verifier
Goal is to make it easier to plug in another implementation.
2017-05-23 02:54:56 +03:00
Andrey Smirnov 470165a419 Enable goconst & interfacer linters 2017-05-17 00:53:10 +03:00
Andrey Smirnov 5ce6bf8718 Enable vetshadow linter 2017-05-04 23:00:13 +03:00
Andrey Smirnov 51213899b7 More Go linters enabled, issues fixed
Ref: #528

Enables "staticcheck", "varcheck", "structcheck", "aligncheck"
2017-05-03 18:23:14 +03:00
Andrey Smirnov bae3f949b4 Enable gosimple and ineffasign linters 2017-04-27 18:34:30 +03:00
Andrey Smirnov 186bb2dff0 Add flag to disable/enable support for legacy pool paths
Legacy pool paths are enabled by default, but for new aptly installations
(when aptly config is first generated), it would be disabled explicitly.
2017-04-26 23:37:31 +03:00
Andrey Smirnov 01512df853 Rework mirror update to support closing/reoping DB for the download duration
This requires splitting up import file phase as separate step in then end,
it should be pretty fast, as it only does file move (hardlink) and
DB update for new checksums.
2017-04-26 23:17:04 +03:00
Andrey Smirnov b48e8425ec Fix bug with file not being updated properly 2017-04-26 23:17:04 +03:00
Andrey Smirnov 3ce8227122 Add baseName to LinkFromPool as explicit argument 2017-04-26 23:17:04 +03:00
Andrey Smirnov 5dd11a2ec2 Pull original packages when skipping existing packages 2017-04-26 23:17:04 +03:00
Andrey Smirnov 10c096fbb6 Update all other pieces for the CheckumStorage and Verify 2017-04-26 23:17:04 +03:00
Andrey Smirnov 6994e35119 ChecksumCollection implementation 2017-04-26 23:17:04 +03:00
Andrey Smirnov 4eedb62418 Fix misspelling 2017-04-26 23:17:04 +03:00
Andrey Smirnov 1f3cb2db5d When downloading/importing packages, enforce all checksums 2017-04-26 23:17:04 +03:00
Andrey Smirnov c40025a335 Add progress bar on package saving progress 2017-04-26 23:17:03 +03:00
Andrey Smirnov bc7903f86e Rework mirror update (download packages) implementation
`PackageDownloadTask` is just a reference to file now. Whole process
was rewritten to follow pattern: download to temp location inside the pool,
verify/update checksums, import into pool as final step.

This removes a lot of edge cases when aptly internal state might be broken
if updating from rogue mirror.

Also this changes whole memory model: package list/files are kept in memory
now during the duration of `mirror update` command and saved to disk
only in the end.
2017-04-26 23:17:03 +03:00
Andrey Smirnov 72d233b587 Final round of updates, everything except mirror download should be ready 2017-04-26 23:17:03 +03:00
Andrey Smirnov 8dc6a14766 Allow snapshot to be created from empty local repo
Fixes #288
2017-04-26 00:33:09 +03:00
Clemens Rabe 25f9c29f00 Implemented filesystem endpoint with support for hardlinks, symlinks and copy. 2017-04-13 20:25:40 +02:00
Andrey Smirnov 60800b5f25 Fix temporary contents DB being left behind after publishing
NB: Go `defer` order execution is reverse to the order `defer` statements
are executed.

So before the change, `Drop()` was called before `Close()`, which was no-op.

Change that to explicit order in single func, print errors if they happen.
2017-04-10 23:43:33 +03:00
Andrey Smirnov 50cf2b49bd Refactoring: use checksums instead of MD5 for pool/published
This is related to #506

As a first step, don't pass MD5 explicitly, pass checksum info object,
so that as a next step we can choose which hash to use.

There should be no functional changes so far.

Next step: stop returning explicit paths from public package pool.
2017-04-01 00:12:31 +03:00
Andrey Smirnov bc469eecfb Merge branch 'master' into dep-verbose-resolve 2017-03-31 19:17:57 +03:00
Andrey Smirnov 7a5be6736d When contents generation fails, don't bail out
This replaces `panic` which aborts aptly execution with warning
message on console. So aptly continues publishing actions, but
`Contents` indexes might be incomplete.

Error will be printed every time contents generation is triggered.
2017-03-31 00:57:18 +03:00
Andrey Smirnov 85b4a8b1ae Add new option for detailed logging on dependency resolving
This adds command-line arg and config option, with option enabled
aptly is more verbose on internal depeendency resolving cycles:

```
Missing dependencies: file-rc (>= 0.8.16) [amd64], python:any (>= 2.7.1-0ubuntu2) [amd64], python3:any (>= 3.3.2-2~) [amd64], file-rc [amd64], perl (<< 5.17) [amd64], iptables-router (>= 1.2.3) [amd64], systemd [amd64], sgml-base (>= 1.26+nmu2) [amd64], sed (>= 4.1.2-8) [amd64]
Unsatisfied dependency: file-rc (>= 0.8.16) [amd64]
Unsatisfied dependency: python:any (>= 2.7.1-0ubuntu2) [amd64]
Unsatisfied dependency: python3:any (>= 3.3.2-2~) [amd64]
Unsatisfied dependency: file-rc [amd64]
Unsatisfied dependency: perl (<< 5.17) [amd64]
Unsatisfied dependency: iptables-router (>= 1.2.3) [amd64]
Unsatisfied dependency: systemd [amd64]
Injecting package: sgml-base_1.26+nmu4ubuntu1_all
Injecting package: sed_4.2.2-4ubuntu1_amd64
```
2017-03-28 22:58:07 +03:00
Andrey Smirnov 800c5c1e06 Merge branch 'master' into feature_skip_existing_packages_latest 2017-03-28 21:26:28 +03:00
Andrey Smirnov 2943422d5d Automatic versioning for aptly
New version format:

* for releases, `x.y.z` (follows tag without leading `v`)
* for nightly builds, `x.y.z+N+hash` (previous version, not the upcoming one)

This means that each nightly build `aptly` would report
correct version now.

Version is now complied into the aptly binary, system tests
automatically check for current version, no need to update them
anymore.
2017-03-25 00:18:45 +03:00
Clemens Rabe aa16899c60 Adaption of tests. 2017-03-24 06:25:46 +01:00
Clemens Rabe 16a0d0d428 Added option --skip-existing-packages to speed up mirror update. 2017-03-23 22:01:11 +01:00
Clemens Rabe 66f51d2b17 Added option --skip-existing-packages to speed up mirror update. 2017-03-23 21:55:22 +01:00