Andrey Smirnov
5dd11a2ec2
Pull original packages when skipping existing packages
2017-04-26 23:17:04 +03:00
Andrey Smirnov
10c096fbb6
Update all other pieces for the CheckumStorage and Verify
2017-04-26 23:17:04 +03:00
Andrey Smirnov
6994e35119
ChecksumCollection implementation
2017-04-26 23:17:04 +03:00
Andrey Smirnov
4eedb62418
Fix misspelling
2017-04-26 23:17:04 +03:00
Andrey Smirnov
1f3cb2db5d
When downloading/importing packages, enforce all checksums
2017-04-26 23:17:04 +03:00
Andrey Smirnov
c40025a335
Add progress bar on package saving progress
2017-04-26 23:17:03 +03:00
Andrey Smirnov
bc7903f86e
Rework mirror update (download packages) implementation
...
`PackageDownloadTask` is just a reference to file now. Whole process
was rewritten to follow pattern: download to temp location inside the pool,
verify/update checksums, import into pool as final step.
This removes a lot of edge cases when aptly internal state might be broken
if updating from rogue mirror.
Also this changes whole memory model: package list/files are kept in memory
now during the duration of `mirror update` command and saved to disk
only in the end.
2017-04-26 23:17:03 +03:00
Andrey Smirnov
72d233b587
Final round of updates, everything except mirror download should be ready
2017-04-26 23:17:03 +03:00
Andrey Smirnov
8dc6a14766
Allow snapshot to be created from empty local repo
...
Fixes #288
2017-04-26 00:33:09 +03:00
Clemens Rabe
25f9c29f00
Implemented filesystem endpoint with support for hardlinks, symlinks and copy.
2017-04-13 20:25:40 +02:00
Andrey Smirnov
60800b5f25
Fix temporary contents DB being left behind after publishing
...
NB: Go `defer` order execution is reverse to the order `defer` statements
are executed.
So before the change, `Drop()` was called before `Close()`, which was no-op.
Change that to explicit order in single func, print errors if they happen.
2017-04-10 23:43:33 +03:00
Andrey Smirnov
50cf2b49bd
Refactoring: use checksums instead of MD5 for pool/published
...
This is related to #506
As a first step, don't pass MD5 explicitly, pass checksum info object,
so that as a next step we can choose which hash to use.
There should be no functional changes so far.
Next step: stop returning explicit paths from public package pool.
2017-04-01 00:12:31 +03:00
Andrey Smirnov
bc469eecfb
Merge branch 'master' into dep-verbose-resolve
2017-03-31 19:17:57 +03:00
Andrey Smirnov
7a5be6736d
When contents generation fails, don't bail out
...
This replaces `panic` which aborts aptly execution with warning
message on console. So aptly continues publishing actions, but
`Contents` indexes might be incomplete.
Error will be printed every time contents generation is triggered.
2017-03-31 00:57:18 +03:00
Andrey Smirnov
85b4a8b1ae
Add new option for detailed logging on dependency resolving
...
This adds command-line arg and config option, with option enabled
aptly is more verbose on internal depeendency resolving cycles:
```
Missing dependencies: file-rc (>= 0.8.16) [amd64], python:any (>= 2.7.1-0ubuntu2) [amd64], python3:any (>= 3.3.2-2~) [amd64], file-rc [amd64], perl (<< 5.17) [amd64], iptables-router (>= 1.2.3) [amd64], systemd [amd64], sgml-base (>= 1.26+nmu2) [amd64], sed (>= 4.1.2-8) [amd64]
Unsatisfied dependency: file-rc (>= 0.8.16) [amd64]
Unsatisfied dependency: python:any (>= 2.7.1-0ubuntu2) [amd64]
Unsatisfied dependency: python3:any (>= 3.3.2-2~) [amd64]
Unsatisfied dependency: file-rc [amd64]
Unsatisfied dependency: perl (<< 5.17) [amd64]
Unsatisfied dependency: iptables-router (>= 1.2.3) [amd64]
Unsatisfied dependency: systemd [amd64]
Injecting package: sgml-base_1.26+nmu4ubuntu1_all
Injecting package: sed_4.2.2-4ubuntu1_amd64
```
2017-03-28 22:58:07 +03:00
Andrey Smirnov
800c5c1e06
Merge branch 'master' into feature_skip_existing_packages_latest
2017-03-28 21:26:28 +03:00
Andrey Smirnov
2943422d5d
Automatic versioning for aptly
...
New version format:
* for releases, `x.y.z` (follows tag without leading `v`)
* for nightly builds, `x.y.z+N+hash` (previous version, not the upcoming one)
This means that each nightly build `aptly` would report
correct version now.
Version is now complied into the aptly binary, system tests
automatically check for current version, no need to update them
anymore.
2017-03-25 00:18:45 +03:00
Clemens Rabe
aa16899c60
Adaption of tests.
2017-03-24 06:25:46 +01:00
Clemens Rabe
16a0d0d428
Added option --skip-existing-packages to speed up mirror update.
2017-03-23 22:01:11 +01:00
Clemens Rabe
66f51d2b17
Added option --skip-existing-packages to speed up mirror update.
2017-03-23 21:55:22 +01:00
Andrey Smirnov
516dd7b044
Switch to gometalinter
...
Only small amount of required checks is enabled,
plan is to enable more linters as issues are fixed in the code.
2017-03-23 01:51:08 +03:00
Andrey Smirnov
11d828b3b1
Add govet/golint into Travis CI build
...
Fix current issues
2017-03-22 21:49:16 +03:00
Raphael Medaer
bfb9ffad1d
Added expected error on 'Packages.xz' for TestDownload[WithSources]Flat.
2017-03-16 22:41:25 +03:00
Szymon Sobik
b8e7ad9022
update changes unit test to account for dbgsym matching
2017-03-08 10:32:11 +01:00
Szymon Sobik
1b80d55ea4
since -dbgsym is for each binary package use that for PackageQuery
2017-03-08 10:31:44 +01:00
Szymon Sobik
a0832adfa5
include dbgsym packages
...
fixes #331
2017-03-07 17:06:59 +01:00
Andrey Smirnov
4c06e26d85
Throttle compaction on temporary DB
2017-02-23 01:01:17 +03:00
Andrey Smirnov
f2dc4eeec9
Generating contents indexes via temporary LevelDB
2017-02-21 19:09:51 +03:00
Andrey Smirnov
33d6cd8c0a
Allow filter to be empty for aptly * search commands
...
Empty filter implies "select all packages".
2017-02-10 23:07:06 +03:00
Andrey Smirnov
c75d4c749c
Add workaround for reading data.tar.gz as data.tar
...
It seems that in the wild there are .deb package which have
`data.tar.gz` which is actually `.tar` archive.
Add magic detection based on signature.
2017-01-24 19:30:53 +03:00
jolo
b0ab39e07f
Manually undo unintended changes
2017-01-20 02:19:44 +01:00
jola5
4bf27d1dae
Merge branch 'master' into master
2017-01-19 23:07:49 +01:00
Andrey Smirnov
d5df049630
Sort paths when generating checksums for Release/InRelease
2017-01-18 23:50:22 +03:00
jolo
91561b40f6
Change 'vertical' argument to a more generic 'layout', fix api
2017-01-16 22:13:13 +01:00
jolo
0e8ea6363a
Support vertical graph layouts
2017-01-14 02:18:56 +01:00
Oliver Sauder
f31b5ec3f8
Adjusted test with new maxTries param for download
2016-11-28 17:02:24 +01:00
Oliver Sauder
6becd5a3aa
Added max-tries flag for mirror update
2016-11-28 17:02:24 +01:00
Oliver Sauder
fcd4429370
only create bzip file if needed. #415
2016-10-13 13:48:28 +02:00
Daniel Stelter-Gliese
1c45c79cc1
Allow overriding architecture info from Release file
...
Adds a flag -force-architectures to ignore missing architectures from
mirrors. This flag can be used in cases where the mirrored repository
does not provide an "Architecture: " line.
Example Release file:
http://mitaka-jessie.pkgs.mirantis.com/debian/dists/jessie-mitaka-backports/Release
2016-05-16 03:25:00 +02:00
Andrey Smirnov
aa53b8da15
Go 1.6.
2016-04-18 12:47:00 +03:00
Andrey Smirnov
d7665119e4
Few more fixes.
2016-03-28 13:44:19 +03:00
Andrey Smirnov
587086beb4
Misc style and simple mistakes fixes.
2016-03-28 13:34:05 +03:00
Andrey Smirnov
644d24d1cc
Attempt to lower memory usage when publishing with contents.
2016-03-28 13:28:26 +03:00
Andrey Smirnov
ff52d2655a
Fix package search missing duplicate packages. #225
...
Implement package list with duplicate entries, use it when
initiating search from PackageCollection.
2016-03-22 12:23:13 +03:00
Harald Sitter
f1b6841757
add Checksums-Sha512 to isMultilineField
...
Otherwise line breaks are not properly handled and the output contains
excess newlines
Fixes #361
2016-03-10 14:40:10 +01:00
Andrey Smirnov
77be7b9e3b
Support for SHA-512 hashes on publishing/downloads.
2016-02-18 12:01:51 +03:00
Andrey Smirnov
f50e008763
Make 'SkipContents' configurable in API. #345
...
Also add global configuration to disable 'skipContents' by
default for all new published repos/snapshots.
2016-02-14 14:49:16 +03:00
Andrey Smirnov
7bb052ac37
Fix unit-tests. #324
2015-12-24 14:08:37 +03:00
Paul Krohn
9d973aeceb
shorten regex to match generated error only
2015-11-26 00:03:51 +00:00
Paul Krohn
7caeac7515
add diagnostic output
2015-11-25 23:48:47 +00:00
