39aa0fdbfe
Merge pull request #557 from smira/linters-1
...
Enable `gosimple` and `ineffasign` linters
2017-04-28 00:29:20 +03:00
c983810e2d
Manually create system/env, workaround Travis issues
2017-04-28 00:16:21 +03:00
c798db8056
Add system's requirements.txt, enforce flake8 linter
...
Fix style issues in functional tests.
2017-04-28 00:05:11 +03:00
1e4a80252e
Extend linter deadline to 1 minute
2017-04-27 22:32:35 +03:00
bae3f949b4
Enable gosimple and ineffasign linters
2017-04-27 18:34:30 +03:00
7a7b981d4f
Merge pull request #539 from smira/public-pool-paths
...
New package pool with configurable hashing base
2017-04-27 16:24:14 +03:00
2ffefeb1e0
Add man page for skipLegacyPool
2017-04-27 00:51:46 +03:00
1941418c10
Add system test for disabled legacy pool path support
2017-04-27 00:51:46 +03:00
186bb2dff0
Add flag to disable/enable support for legacy pool paths
...
Legacy pool paths are enabled by default, but for new aptly installations
(when aptly config is first generated), it would be disabled explicitly.
2017-04-26 23:37:31 +03:00
2308632683
Add system tests for legacy pool files
2017-04-26 23:17:04 +03:00
ee21b69402
Add test for mirror without MD5 checksums
2017-04-26 23:17:04 +03:00
01512df853
Rework mirror update to support closing/reoping DB for the download duration
...
This requires splitting up import file phase as separate step in then end,
it should be pretty fast, as it only does file move (hardlink) and
DB update for new checksums.
2017-04-26 23:17:04 +03:00
7dcc0d597d
Fix S3/Swift tests
2017-04-26 23:17:04 +03:00
154ef7fe65
Fix system tests for aptly repo include
2017-04-26 23:17:04 +03:00
4601f07349
Fix system tests for aptly repo add
2017-04-26 23:17:04 +03:00
b7b9f12c88
Update system tests for SHA512 checksums being generated
2017-04-26 23:17:04 +03:00
b48e8425ec
Fix bug with file not being updated properly
2017-04-26 23:17:04 +03:00
3ce8227122
Add baseName to LinkFromPool as explicit argument
2017-04-26 23:17:04 +03:00
0bc3f71d27
Use SHA256 as main checksum in the pool
2017-04-26 23:17:04 +03:00
c1d4c0fb88
Temporarily disable db close/open cycle (to be addressed later)
2017-04-26 23:17:04 +03:00
8078f3b588
We should remove file only when checksums are calculated
2017-04-26 23:17:04 +03:00
5dd11a2ec2
Pull original packages when skipping existing packages
2017-04-26 23:17:04 +03:00
cc34a021ce
Fix misspellings
2017-04-26 23:17:04 +03:00
10c096fbb6
Update all other pieces for the CheckumStorage and Verify
2017-04-26 23:17:04 +03:00
a85d8b6f90
Mock for ChecksumStorage
2017-04-26 23:17:04 +03:00
5566111a7b
New ChecksumStorage and new PackagePool interfaces
2017-04-26 23:17:04 +03:00
6994e35119
ChecksumCollection implementation
2017-04-26 23:17:04 +03:00
4eedb62418
Fix misspelling
2017-04-26 23:17:04 +03:00
1f3cb2db5d
When downloading/importing packages, enforce all checksums
2017-04-26 23:17:04 +03:00
c40025a335
Add progress bar on package saving progress
2017-04-26 23:17:03 +03:00
4171a73995
Fix up system test
2017-04-26 23:17:03 +03:00
29e5f4ca10
Vendor import github.com/pkg/errors
2017-04-26 23:17:03 +03:00
05f6c75743
Add PR #506 original author [ci skip]
2017-04-26 23:17:03 +03:00
45d187bc14
Fix up system test
2017-04-26 23:17:03 +03:00
bc7903f86e
Rework mirror update (download packages) implementation
...
`PackageDownloadTask` is just a reference to file now. Whole process
was rewritten to follow pattern: download to temp location inside the pool,
verify/update checksums, import into pool as final step.
This removes a lot of edge cases when aptly internal state might be broken
if updating from rogue mirror.
Also this changes whole memory model: package list/files are kept in memory
now during the duration of `mirror update` command and saved to disk
only in the end.
2017-04-26 23:17:03 +03:00
72d233b587
Final round of updates, everything except mirror download should be ready
2017-04-26 23:17:03 +03:00
2535367c3c
Update Swift published storage to work with new package pool
2017-04-26 23:17:03 +03:00
f4ff8d957f
Fix S3 published storage to use new PackagePool interface
...
Change PackagePool to return Seeker interface from Open call.
2017-04-26 23:17:03 +03:00
7bad358408
Local package pool and local publishing rewritten with new constraints
...
Local package pool now implements more generic package pool API.
The base idea is to never expose full paths to files, so that other
kinds of package pools (e.g. package pool in S3) could be used to implement
the same interface.
Files get into the pool only using `Import` method. `Import` method is
now more smart, it supports moving files into the pool, it can detect if
files reside on the same filesystem and use hardlinking instead of copying.
This will make direct mirror downloads still as fast as they were with previous
version which was performing download directly to package pool.
New package pool doesn't have two things implemented yet:
1. New file placement according to SHA256 or other configured hash
2. Calculate at least SHA256/MD5 for each imported files.
MD5 would be required for S3/Swift publishing
2017-04-26 23:17:03 +03:00
94b49818a1
Refactor HTTP downloader package
...
* Drop multi-threaded downloader. It doesn't really belong here -
some places require it, some do not, but it's definitely not the
right place to handle it, as it's being used only when updating
mirrors
* Pass expectedChecksums as pointer, so it's easy to drive `nil` value,
and also downloader can fill back checksums (not implemented right now).
* Break down downloader and tests into more files
* Use pkg/errors instead of fmt
2017-04-26 23:17:03 +03:00
a245b722a8
Merge pull request #555 from smira/288-empty-repo-snapshot
...
Allow snapshot to be created from empty local repo
2017-04-26 01:11:40 +03:00
8dc6a14766
Allow snapshot to be created from empty local repo
...
Fixes #288
2017-04-26 00:33:09 +03:00
d66185ca03
Merge pull request #554 from smira/flat-system-test
...
Fix system tests for flat repos
2017-04-24 23:45:58 +03:00
c3acabe303
Fix system tests for flat repos
...
Old mirror used for testing is gone, switch to CRAN.
2017-04-24 23:04:48 +03:00
4697d8eaf8
Merge pull request #550 from smira/549-fix-deps
...
Rewrite `dep` files into new format
2017-04-14 22:48:53 +03:00
8bf71a5561
Rewrite dep files into new format
...
Conversion to TOML plus some manual fixups.
2017-04-14 17:24:49 +03:00
898cbd2c83
Merge pull request #548 from smira/546-fix-depends
...
Update `Depends:` for homegrown packages
2017-04-14 16:22:03 +03:00
62762f1616
Merge branch 'master' into 546-fix-depends
2017-04-14 00:57:36 +03:00
4d38e0bc87
Merge pull request #521 from seeraven/feature_localfiles_publishstorage
...
Implemented local filesystem endpoint with support for hardlinks, symlinks and copy
2017-04-14 00:52:49 +03:00
25f9c29f00
Implemented filesystem endpoint with support for hardlinks, symlinks and copy.
2017-04-13 20:25:40 +02:00
Andrey Smirnov
Clemens Rabe