1423 Commits

Author SHA1 Message Date
Ivo De Decker
0709f8d96c Move block handling to BlockPolicy
Note that this now also applies to binNMUs. A source block also blocks
binaries. Binaries on a specific architecture can be unblock with an
architecture-specific unblock hint.

This also means all binaries from non-primary suites
(testing-proposed-updates, etc) need approval.

Closes: #916209
2019-01-04 17:24:38 +00:00
Ivo De Decker
49d9a38d25 Skip unimplemented policies in excuses
Return PolicyVerdict.NOT_APPLICABLE, which means no data is added to
policy_info in the excuse.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2019-01-02 00:34:49 +00:00
Ivo De Decker
b395c6f760 Only ignore hints for the same package if arch matches
Signed-off-by: Ivo De Decker <ivodd@debian.org>
2019-01-01 18:34:59 +00:00
Ivo De Decker
7090c23efc Expose architecture and suite property for hints
Also allow searching based on these properties.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2019-01-01 18:34:59 +00:00
Ivo De Decker
7e05c0e5ac Apply policy to srcarch excuses
Currently no policy implements this.

A policy can now implement apply_src_policy_impl or apply_srcarch_policy_impl
(or both), so apply_src_policy_impl is no longer an abstractmethod.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2019-01-01 18:33:02 +00:00
Ivo De Decker
4cf03bbdc5 Rearrange decision for should_upgrade_srcarch
This should not change the logic in any way. It should allow adding policy
checks in a future commit.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2019-01-01 18:28:27 +00:00
Ivo De Decker
d0b5cc2ce1 Rename apply_policy to apply_src_policy
apply_src_policy expects an excuse with a new source and binaries. It doesn't
apply to srcarch excuses, which only have new binaries for an existing source.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2019-01-01 18:28:27 +00:00
Ivo De Decker
0d80bad65c Move handling of policies to PolicyEngine
Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-31 19:15:29 +00:00
Niels Thykier
b16ae8046a
Add some '# pragma: no cover' for error cases, abstract methods, etc.
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-31 18:07:29 +00:00
Niels Thykier
5bcbed499e
Fix the doc string for compile_nuninst
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-31 12:16:09 +00:00
Niels Thykier
04318473f6
Simplify MigrationItem.parse_item by extracting a method
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-31 12:00:54 +00:00
Niels Thykier
542f030a96
Remove now obsolete code from MigrationItem
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-31 12:00:53 +00:00
Niels Thykier
61d3068de6
Move suite name auto-correction to MigrationItemFactory
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-31 12:00:52 +00:00
Niels Thykier
eec7b9b7ef
MigrationItem: Remove setter for "name"
It is no longer used and would prevent the removal of global state in
the MigrationItem class.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-31 12:00:51 +00:00
Niels Thykier
6174d2c3f9
Create a MigrationItemFactory and migrate most code to use it
This is a step towards making migration unit-testable.  This step
reduces the need for global state (in the MigrationItem class as class
fields) and with another step we can remove the global state entirely
and enable unit tests to create migration items without having to
worry about other unit tests.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-31 12:00:50 +00:00
Niels Thykier
584f75bbc0
Defer loading and registering of hints
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-31 12:00:48 +00:00
Ivo De Decker
e52d46a84d britney.conf: Bump autopkgtest regression deadline by 2 days
Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-31 11:47:03 +00:00
Ivo De Decker
18d951be25 Handle dependencies between excuses in a generic way
All types of dependencies between excuses (Depends, Build-Depends,
Build-Depends-Indep, ...) are handled by the same code. The DependencyType is
used to distinguish between the types where needed.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-31 11:46:10 +00:00
Ivo De Decker
847e6e41e1 Add invalidated-by-other-package field in excuses.yaml
The other fields related to invalidation are currently not deterministic and
depend on the order in which the excuses are invalidated.

The new invalidated-by-other-package field is just a boolean which shows the
excuse was invalidated by some other package for some reason (without
specifying the reason). It does not depend on the order of processing and
should be deterministic.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-30 17:27:50 +00:00
Niels Thykier
ba7b737fae
docs/contributing: Update to include optional amqplib dependency
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-30 16:40:54 +00:00
Niels Thykier
2bf4092947
Remove INSTALL as it is out of date and redundant 2018-12-30 16:40:15 +00:00
Niels Thykier
7502bc72f3
Avoid some hash lookups
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-29 14:15:55 +00:00
Niels Thykier
dd3dcf818f
Mark a regex pattern as a raw string
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-29 04:46:55 +00:00
Niels Thykier
7fa64043c9
Add libyaml-syck-perl dependency to .gitlab-ci + Dockerfile
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-29 04:30:43 +00:00
Niels Thykier
e13eb7d0cb
Pass correct source_data to find_smooth_updateable_binaries
This fixes a thinko in e0c19b3.  Thanks to ivodd for reporting it.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-29 04:21:18 +00:00
Raphaël Hertzog
6c1cf01898 Do not ensure consistency of "Section" field across unstable and testing
In Debian, the same override is applied to both suites and they are
always consistent. It's not the case in Kali, we keep the value from
Debian when we import the package in britney's source distribution, but
if the same version is already present in the target distribution, it
keeps its original section (dating back to its initial import). In those
situations, the code will fail with an error like this one:

E: [2018-12-28T19:57:57+0000] - Mismatch found coinor-libdylp0 1.6.0-1.1 amd64 differs
I: [2018-12-28T19:57:57+0000] -  ... section libs != science
[...]
ValueError: Inconsistent / Unsupported data set

Commit 7efa865a04892b94a0ef18b2e611f210a6963925 which was supposed to
move code around introduced the check on this field. Prior to this, the
section was not checked. Since the section only has an impact on which
packages take part to the smooth updates feature, the impact of such a
mismatch is negligible so I simply dropped that check.
2018-12-28 21:30:48 +01:00
Ivo De Decker
22c6be76ba Split calculating and running of auto-hinter hints
Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-27 20:16:59 +00:00
Ivo De Decker
12187421c8 Show newly uninstallables before validation
Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-27 20:16:10 +00:00
Ivo De Decker
348274947e Add option fake_runtime to override runtime in britney.conf
This is useful to run tests with the data files from a specific point in time,
without changes due to ageing when the test runs later.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-25 22:12:33 +00:00
Niels Thykier
179faeb23b
Update some comments to match a renamed method
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-22 21:14:33 +00:00
Niels Thykier
6fa06f8ac4
britney.conf: Bump autopkgtest regression deadline by 2 days 2018-12-22 21:13:15 +00:00
Niels Thykier
8cefd1a546
Have transactions verify/assert safe usage
Have the transaction code verify that there is at most one active
child at the time and no one is using the parent while child is
active.  This is how the code is intended to be used and also the
code almost certainly does not work otherwise.

The new code does not cover commiting/rolling back a parent before a
child but that is already covered by the existing code (it will
trigger when child transaction is rolled back/committed or when
leaving the contextmanager from start_transaction).

This would have caught 7d758760d1c08bcd3b364f7e6ec76b2c0eafe68a
immediately with an assertion error.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-22 21:12:01 +00:00
Niels Thykier
7d758760d1
MigrationManager: _transaction is a stack; not a queue
Correct the return value of current_transaction that treated the
_transaction field incorrectly as a queue rather than a stack like
everything else.  This completely broke the ability to commit and
rollback child transactions (correctly).  Fortunately, it could only
trigger on a "hint"-hint.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-22 20:50:06 +00:00
Niels Thykier
a5352d353b
Simplify undo code handling sources
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 20:29:02 +00:00
Niels Thykier
59ae7e3d68
Optimize out "nvirtual" member of undo items
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 20:29:01 +00:00
Niels Thykier
5642a90449
Rename _compute_groups to compute_groups
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 20:29:00 +00:00
Niels Thykier
33ba41c76e
Split out a _compute_removals from _compute_groups
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 20:28:59 +00:00
Niels Thykier
d3b391aed1
MigrationManager: Precompute a set of all architectures
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 19:41:46 +00:00
Niels Thykier
89fcccba13
MigrationManager: Create a is_nuninst_worse
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 19:41:45 +00:00
Niels Thykier
aa6dd4a0f3
Extract a compute eqv_set function
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 19:41:44 +00:00
Niels Thykier
a95565c0e8
Split migrate_item_to_target_suite into two
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-19 07:02:30 +00:00
Niels Thykier
57ded55674
Remove dead assignment
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 22:15:01 +00:00
Niels Thykier
c7dbd95c0b
Have MigrationManager keep track of active transaction
This leaves callers with only having to track the transaction they
need to care about (if any).

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 22:08:20 +00:00
Niels Thykier
4803065e1a
Move (and rename) try_migration to the MigrationManager
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:57:12 +00:00
Niels Thykier
28c7a052fa
Move to MM and rename doop_source to apply_item_to_target_suite
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:12 +00:00
Niels Thykier
4ae6a34a27
Move _compute_groups to a new MigrationManager class
Other migration methods are expected to follow in future commits.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:11 +00:00
Niels Thykier
d688c9753b
Split a loop to make the special-case more obvious
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:11 +00:00
Niels Thykier
ce6a5704fa
Make _compute_group take a migration item instead of 4 parameters
All callers can supply one and currently just unfold it at the
call-site.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:10 +00:00
Niels Thykier
e0c19b3adb
Inline a call to _compute_groups
The should_upgrade_srcarch method only needs _compute_groups for the
purpose of computing smooth update candidates.  In the concrete case,
it is rather simple to inline that part directly (and fold away
irrelevant code).

This also enables us to rewrite _compute_groups to accept an migration
item as all remaining callers simply unfold the migration item into
the arguments required by _compute_groups.  This change will come in a
future commit.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:09 +00:00
Niels Thykier
82ff8010e2
Rename unused variable/return value to "_"
This makes it more explicit that we are not using the value for
anything (and its name did not match it content due to a mistake
in 5d49a4120488e5beba0e1946613a1e8873bfefdd).

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-17 22:18:47 +00:00