Move the logic of apply_src_policy and apply_srcarch_policy into PolicyEngine.
This fixes an issue with the excuses.yaml output introduced in commit
15e5228669: only the last verdict was added to the excuse info for that
policy.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
When a source has only arch: all binaries, the Build-Depends had no relevant
architectures, so the check was skipped. Instead check it on any architecture,
just like Build-Depends-Indep.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
The src_policy defines wether, for source items, the source policy should be
run (RUN_SRC, the default), the arch policy should be run on every arch
(RUN_ON_EVERY_ARCH_ONLY), or both (RUN_SRC_AND_EVERY_ARCH).
Signed-off-by: Ivo De Decker <ivodd@debian.org>
The recent code changes made use remove from the "binaries" field in
SourcePackages. Lists are not particularly optimized for this kind
of removal and we have a few source packages with a lot of binary
packages (e.g. libreoffice, gcc-X-cross{,-ports}) that might trip
poor performance.
Signed-off-by: Niels Thykier <niels@thykier.net>
When an item is migrated to the target suite, always create a new
SourcePackage object with a separate binaries list (independent from the list
in the source suite). That list is updated for every binary that is added,
updated or removed. This should ensure consistent source package information
in the target suite.
The undo code doesn't need to be updated for this, because the old
SourcePackage object is put back on undo, with the old binary list.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
When the item is a single binary cruft removal, the source name is different
from the package. compute_groups has this info anyway, so just pass it on as a
returm value.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
This test checks if every binary in the target suite is listed as a binary for
its source package and vice-versa.
This adds a config option check_consistency_level:
0: no checks
1: only check at the end
2: also check for every hint run (default)
3: check for every transaction (slow - only useful for the testsuite)
Signed-off-by: Ivo De Decker <ivodd@debian.org>
Regex compilation is often rather expensive and in this case, we can
do it once instad of once per migration item.
Signed-off-by: Niels Thykier <niels@thykier.net>
Note that this now also applies to binNMUs. A source block also blocks
binaries. Binaries on a specific architecture can be unblock with an
architecture-specific unblock hint.
This also means all binaries from non-primary suites
(testing-proposed-updates, etc) need approval.
Closes: #916209
Currently no policy implements this.
A policy can now implement apply_src_policy_impl or apply_srcarch_policy_impl
(or both), so apply_src_policy_impl is no longer an abstractmethod.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
apply_src_policy expects an excuse with a new source and binaries. It doesn't
apply to srcarch excuses, which only have new binaries for an existing source.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
This is a step towards making migration unit-testable. This step
reduces the need for global state (in the MigrationItem class as class
fields) and with another step we can remove the global state entirely
and enable unit tests to create migration items without having to
worry about other unit tests.
Signed-off-by: Niels Thykier <niels@thykier.net>
All types of dependencies between excuses (Depends, Build-Depends,
Build-Depends-Indep, ...) are handled by the same code. The DependencyType is
used to distinguish between the types where needed.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
The other fields related to invalidation are currently not deterministic and
depend on the order in which the excuses are invalidated.
The new invalidated-by-other-package field is just a boolean which shows the
excuse was invalidated by some other package for some reason (without
specifying the reason). It does not depend on the order of processing and
should be deterministic.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
In Debian, the same override is applied to both suites and they are
always consistent. It's not the case in Kali, we keep the value from
Debian when we import the package in britney's source distribution, but
if the same version is already present in the target distribution, it
keeps its original section (dating back to its initial import). In those
situations, the code will fail with an error like this one:
E: [2018-12-28T19:57:57+0000] - Mismatch found coinor-libdylp0 1.6.0-1.1 amd64 differs
I: [2018-12-28T19:57:57+0000] - ... section libs != science
[...]
ValueError: Inconsistent / Unsupported data set
Commit 7efa865a04 which was supposed to
move code around introduced the check on this field. Prior to this, the
section was not checked. Since the section only has an impact on which
packages take part to the smooth updates feature, the impact of such a
mismatch is negligible so I simply dropped that check.
This is useful to run tests with the data files from a specific point in time,
without changes due to ageing when the test runs later.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
Have the transaction code verify that there is at most one active
child at the time and no one is using the parent while child is
active. This is how the code is intended to be used and also the
code almost certainly does not work otherwise.
The new code does not cover commiting/rolling back a parent before a
child but that is already covered by the existing code (it will
trigger when child transaction is rolled back/committed or when
leaving the contextmanager from start_transaction).
This would have caught 7d758760d1
immediately with an assertion error.
Signed-off-by: Niels Thykier <niels@thykier.net>
Correct the return value of current_transaction that treated the
_transaction field incorrectly as a queue rather than a stack like
everything else. This completely broke the ability to commit and
rollback child transactions (correctly). Fortunately, it could only
trigger on a "hint"-hint.
Signed-off-by: Niels Thykier <niels@thykier.net>
When we try to compute_groups for a group which has a source or a binary with
a lower version than testing, throw an exception. In the cases where this can
happen, the exception is caught. In other cases, it is not and it serves as an
assert.
This can only happen when there are multiple candidates (from multiple suites)
changing the same source or binary.
This should fix the ordering issues tested in these tests:
- tpu-unstable-binnmu
- binnmu-tpu
- tpu-with-unstable-binnmu
With this change, it should be possible to accept binNMUs from *pu again.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
This isolates the undo handling in the new transaction object and in
doop_source, which currently generates the undo items. This commit
will be a stepping stone to rewriting the undo handling.
Signed-off-by: Niels Thykier <niels@thykier.net>
When calculating smooth updateable binaries, filter out cruft binaries from
unstable, because they will not be part of the set of packages that britney
will try to migrate to testing.
Signed-off-by: Ivo De Decker <ivodd@debian.org>
The parsing of migration items should also look for the suite name in the
architecture part. This fixes the parsing for migration items like
some-src/amd64_tpu and some-src/amd64_tpu/1.0-1
Signed-off-by: Ivo De Decker <ivodd@debian.org>
Niels Thykier
Paul Gevers
Ivo De Decker
Raphaël Hertzog