mirror of
https://github.com/aptly-dev/aptly.git
synced 2026-05-31 04:30:44 +00:00
André Roth
8477274bb0
Affected endpoints: apiReposDrop, apiReposPackagesAddDelete, apiReposPackageFromDir, apiReposCopyPackage, apiReposIncludePackageFromDir, apiReposEdit, apiReposCreate. All seven endpoints shared the same architectural flaw as the previously fixed publish endpoints: operations were performed outside the task lock, with stale DB state used inside the lock. Issues Fixed: 1. apiReposDrop - Collections created before task lock Problem: snapshotCollection, publishedCollection captured from pre-task factory. Concurrent snapshot/published modifications not detected. Fix: Create fresh taskCollectionFactory inside task, re-read repo after lock acquired, use fresh collections for checks. 2. apiReposPackagesAddDelete - Repo and factory stale before lock Problem: repo loaded outside task, collectionFactory created before lock. Concurrent add/delete operations both load same pre-task state, last write wins, packages lost. Fix: Create fresh taskCollectionFactory inside task, re-read repo after lock acquired, use fresh factory for all operations. 3. apiReposPackageFromDir - Repo and factory stale before lock Problem: repo loaded outside task, collectionFactory created before lock. Concurrent file imports both load same pre-task state, last write wins. Fix: Create fresh taskCollectionFactory inside task, re-read repo after lock acquired, use fresh factory for imports. 4. apiReposCopyPackage - Both repos and factory stale before lock Problem: dstRepo and srcRepo loaded outside task, collectionFactory created before lock. Concurrent copy operations race on stale state. Fix: Create fresh taskCollectionFactory inside task, re-read both repos after lock acquired, use fresh factory for all operations. 5. apiReposIncludePackageFromDir - Repo and factory stale before lock Problem: repo loaded outside task, collectionFactory created before lock. Concurrent .changes file processing races on stale state. Fix: Create fresh taskCollectionFactory inside task, use fresh factory for import operations. 6. apiReposEdit - No serialization, concurrent modification race Problem: Direct update without task locking. Two concurrent renames can both pass duplicate check, second overwrites first. Fix: Convert to async task. Duplicate check and update now atomic inside lock, after fresh load from DB. 7. apiReposCreate - No serialization, TOCTOU on duplicate check Problem: Duplicate check outside task lock, add outside lock. Two concurrent creates with same name both pass check, second overwrites first. Fix: Convert to async task. Duplicate check and add now atomic inside lock, after fresh load from DB. Root cause analysis: The fundamental issue is the split between pre-task work and task-protected work. Collections and objects were being loaded before lock acquisition, then stale copies used inside the lock. Correct pattern (now applied consistently across all 7 endpoints): 1. HTTP Handler (before task lock): - Shallow load for 404 check only - Extract resource keys - Submit task with resources 2. Task Closure (after lock acquired): - Create fresh collectionFactory - Fresh load of all objects - LoadComplete on fresh copies - All mutations on fresh state - All checks atomic inside lock - Save using fresh collections This ensures: - Concurrent operations are serialized by task queue - No stale DB state used for mutations - No lost updates from concurrent modifications - No TOCTOU races on duplicate checks - No DB handle issues from pre-task factory capture
.. image:: https://github.com/aptly-dev/aptly/actions/workflows/ci.yml/badge.svg
:target: https://github.com/aptly-dev/aptly/actions
.. image:: https://codecov.io/gh/aptly-dev/aptly/branch/master/graph/badge.svg
:target: https://codecov.io/gh/aptly-dev/aptly
.. image:: https://badges.gitter.im/Join Chat.svg
:target: https://matrix.to/#/#aptly:gitter.im
.. image:: https://goreportcard.com/badge/github.com/aptly-dev/aptly
:target: https://goreportcard.com/report/aptly-dev/aptly
aptly
=====
Aptly is a swiss army knife for Debian repository management.
.. image:: http://www.aptly.info/img/aptly_logo.png
:target: http://www.aptly.info/
Documentation is available at `http://www.aptly.info/ <http://www.aptly.info/>`_. For support please use
open `issues <https://github.com/aptly-dev/aptly/issues>`_ or `discussions <https://github.com/aptly-dev/aptly/discussions>`_.
Aptly features:
* make mirrors of remote Debian/Ubuntu repositories, limiting by components/architectures
* take snapshots of mirrors at any point in time, fixing state of repository at some moment of time
* publish snapshot as Debian repository, ready to be consumed by apt
* controlled update of one or more packages in snapshot from upstream mirror, tracking dependencies
* merge two or more snapshots into one
* filter repository by search query, pulling dependencies when required
* publish self-made packages as Debian repositories
* REST API for remote access
Any contributions are welcome! Please see `CONTRIBUTING.md <CONTRIBUTING.md>`_.
Installation
=============
Aptly can be installed on several operating systems.
Debian / Ubuntu
----------------
Aptly is provided in the following debian packages:
* **aptly**: Includes the main Aptly binary, man pages, and shell completions
* **aptly-api**: A systemd service for the REST API, using the global /etc/aptly.conf
* **aptly-dbg**: Debug symbols for troubleshooting
The packages can be installed on official `Debian <https://packages.debian.org/search?keywords=aptly>`_ and `Ubuntu <https://packages.ubuntu.com/search?keywords=aptly>`_ distributions.
Upstream Debian Packages
~~~~~~~~~~~~~~~~~~~~~~~~~
If a newer version (not available in Debian/Ubuntu) of aptly is required, upstream debian packages (built from git tags) can be installed as follows:
Install the following APT key (as root)::
wget -O /etc/apt/keyrings/aptly.asc https://www.aptly.info/pubkey.txt
Define Release APT sources in ``/etc/apt/sources.list.d/aptly.list``::
deb [signed-by=/etc/apt/keyrings/aptly.asc] http://repo.aptly.info/release DIST main
Where DIST is one of: ``bullseye``, ``bookworm``, ``trixie``, ``focal``, ``jammy``, ``noble``
Install aptly packages::
apt-get update
apt-get install aptly
apt-get install aptly-api # REST API systemd service
CI Builds
~~~~~~~~~~
For testing new features or bugfixes, recent builds are available as CI builds (built from master, may be unstable!) and can be installed as follows:
Define CI APT sources in ``/etc/apt/sources.list.d/aptly-ci.list``::
deb [signed-by=/etc/apt/keyrings/aptly.asc] http://repo.aptly.info/ci DIST main
Where DIST is one of: ``bullseye``, ``bookworm``, ``trixie``, ``focal``, ``jammy``, ``noble``
Note: same gpg key is used as for the Upstream Debian Packages.
Other Operating Systems
------------------------
Binary executables (depends almost only on libc) are available on `GitHub Releases <https://github.com/aptly-dev/aptly/releases>`_ for:
- macOS / darwin (amd64, arm64)
- FreeBSD (amd64, arm64, 386, arm)
- Generic Linux (amd64, arm64, 386, arm)
Integrations
=============
Vagrant:
- `Vagrant configuration <https://github.com/sepulworld/aptly-vagrant>`_ by
Zane Williamson, allowing to bring two virtual servers, one with aptly installed
and another one set up to install packages from repository published by aptly
Docker:
- `Docker container <https://github.com/mikepurvis/aptly-docker>`_ with aptly inside by Mike Purvis
- `Docker container <https://github.com/urpylka/docker-aptly>`_ with aptly and nginx by Artem Smirnov
With configuration management systems:
- `Chef cookbook <https://github.com/hw-cookbooks/aptly>`_ by Aaron Baer
(Heavy Water Operations, LLC)
- `Puppet module <https://github.com/voxpupuli/puppet-aptly>`_ by
Vox Pupuli
- `SaltStack Formula <https://github.com/saltstack-formulas/aptly-formula>`_ by
Forrest Alvarez and Brian Jackson
- `Ansible role <https://github.com/aioue/ansible-role-aptly>`_ by Tom Paine
CLI for aptly API:
- `Ruby aptly CLI/library <https://github.com/sepulworld/aptly_cli>`_ by Zane Williamson
- `Python aptly CLI (good for CI) <https://github.com/TimSusa/aptly_api_cli>`_ by Tim Susa
GUI for aptly API:
- `Python aptly GUI (via pyqt5) <https://github.com/chnyda/python-aptly-gui>`_ by Cedric Hnyda
Scala sbt:
- `sbt aptly plugin <https://github.com/amalakar/sbt-aptly>`_ by Arup Malakar
Molior:
- `Molior Debian Build System <https://github.com/molior-dbs/molior>`_ by André Roth