1150 Commits

Author SHA1 Message Date
Niels Thykier
7fa64043c9
Add libyaml-syck-perl dependency to .gitlab-ci + Dockerfile
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-29 04:30:43 +00:00
Niels Thykier
e13eb7d0cb
Pass correct source_data to find_smooth_updateable_binaries
This fixes a thinko in e0c19b3.  Thanks to ivodd for reporting it.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-29 04:21:18 +00:00
Raphaël Hertzog
6c1cf01898 Do not ensure consistency of "Section" field across unstable and testing
In Debian, the same override is applied to both suites and they are
always consistent. It's not the case in Kali, we keep the value from
Debian when we import the package in britney's source distribution, but
if the same version is already present in the target distribution, it
keeps its original section (dating back to its initial import). In those
situations, the code will fail with an error like this one:

E: [2018-12-28T19:57:57+0000] - Mismatch found coinor-libdylp0 1.6.0-1.1 amd64 differs
I: [2018-12-28T19:57:57+0000] -  ... section libs != science
[...]
ValueError: Inconsistent / Unsupported data set

Commit 7efa865a04892b94a0ef18b2e611f210a6963925 which was supposed to
move code around introduced the check on this field. Prior to this, the
section was not checked. Since the section only has an impact on which
packages take part to the smooth updates feature, the impact of such a
mismatch is negligible so I simply dropped that check.
2018-12-28 21:30:48 +01:00
Ivo De Decker
22c6be76ba Split calculating and running of auto-hinter hints
Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-27 20:16:59 +00:00
Ivo De Decker
12187421c8 Show newly uninstallables before validation
Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-27 20:16:10 +00:00
Ivo De Decker
348274947e Add option fake_runtime to override runtime in britney.conf
This is useful to run tests with the data files from a specific point in time,
without changes due to ageing when the test runs later.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-25 22:12:33 +00:00
Niels Thykier
179faeb23b
Update some comments to match a renamed method
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-22 21:14:33 +00:00
Niels Thykier
6fa06f8ac4
britney.conf: Bump autopkgtest regression deadline by 2 days 2018-12-22 21:13:15 +00:00
Niels Thykier
8cefd1a546
Have transactions verify/assert safe usage
Have the transaction code verify that there is at most one active
child at the time and no one is using the parent while child is
active.  This is how the code is intended to be used and also the
code almost certainly does not work otherwise.

The new code does not cover commiting/rolling back a parent before a
child but that is already covered by the existing code (it will
trigger when child transaction is rolled back/committed or when
leaving the contextmanager from start_transaction).

This would have caught 7d758760d1c08bcd3b364f7e6ec76b2c0eafe68a
immediately with an assertion error.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-22 21:12:01 +00:00
Niels Thykier
7d758760d1
MigrationManager: _transaction is a stack; not a queue
Correct the return value of current_transaction that treated the
_transaction field incorrectly as a queue rather than a stack like
everything else.  This completely broke the ability to commit and
rollback child transactions (correctly).  Fortunately, it could only
trigger on a "hint"-hint.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-22 20:50:06 +00:00
Niels Thykier
a5352d353b
Simplify undo code handling sources
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 20:29:02 +00:00
Niels Thykier
59ae7e3d68
Optimize out "nvirtual" member of undo items
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 20:29:01 +00:00
Niels Thykier
5642a90449
Rename _compute_groups to compute_groups
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 20:29:00 +00:00
Niels Thykier
33ba41c76e
Split out a _compute_removals from _compute_groups
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 20:28:59 +00:00
Niels Thykier
d3b391aed1
MigrationManager: Precompute a set of all architectures
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 19:41:46 +00:00
Niels Thykier
89fcccba13
MigrationManager: Create a is_nuninst_worse
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 19:41:45 +00:00
Niels Thykier
aa6dd4a0f3
Extract a compute eqv_set function
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-20 19:41:44 +00:00
Niels Thykier
a95565c0e8
Split migrate_item_to_target_suite into two
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-19 07:02:30 +00:00
Niels Thykier
57ded55674
Remove dead assignment
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 22:15:01 +00:00
Niels Thykier
c7dbd95c0b
Have MigrationManager keep track of active transaction
This leaves callers with only having to track the transaction they
need to care about (if any).

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 22:08:20 +00:00
Niels Thykier
4803065e1a
Move (and rename) try_migration to the MigrationManager
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:57:12 +00:00
Niels Thykier
28c7a052fa
Move to MM and rename doop_source to apply_item_to_target_suite
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:12 +00:00
Niels Thykier
4ae6a34a27
Move _compute_groups to a new MigrationManager class
Other migration methods are expected to follow in future commits.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:11 +00:00
Niels Thykier
d688c9753b
Split a loop to make the special-case more obvious
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:11 +00:00
Niels Thykier
ce6a5704fa
Make _compute_group take a migration item instead of 4 parameters
All callers can supply one and currently just unfold it at the
call-site.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:10 +00:00
Niels Thykier
e0c19b3adb
Inline a call to _compute_groups
The should_upgrade_srcarch method only needs _compute_groups for the
purpose of computing smooth update candidates.  In the concrete case,
it is rather simple to inline that part directly (and fold away
irrelevant code).

This also enables us to rewrite _compute_groups to accept an migration
item as all remaining callers simply unfold the migration item into
the arguments required by _compute_groups.  This change will come in a
future commit.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-18 21:55:09 +00:00
Niels Thykier
82ff8010e2
Rename unused variable/return value to "_"
This makes it more explicit that we are not using the value for
anything (and its name did not match it content due to a mistake
in 5d49a4120488e5beba0e1946613a1e8873bfefdd).

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-17 22:18:47 +00:00
Ivo De Decker
4dc2cf7172
Try to remove old libs during hint run
Attempt the removal of the old libraries from smooth updates during every hint
run. In some cases, the removal needs to happen during the hint run for the
hint to succeed.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-17 21:56:35 +00:00
Niels Thykier
70d461ff11
Remove "skip" return value of _compute_groups
The "skip" value was a hack to support keeping cruft out of testing
while not breaking the undo code.  With new the undo code
(a05a249e04926034ea8f915abcd51b62acac41c5), britney can now handle
this cause without the "skip" value and it has therefore out-lived its
purpose.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-17 20:20:16 +00:00
Ivo De Decker
12debfc7c8 Check that version increases when adding packages to testing
When we try to compute_groups for a group which has a source or a binary with
a lower version than testing, throw an exception. In the cases where this can
happen, the exception is caught. In other cases, it is not and it serves as an
assert.

This can only happen when there are multiple candidates (from multiple suites)
changing the same source or binary.

This should fix the ordering issues tested in these tests:
- tpu-unstable-binnmu
- binnmu-tpu
- tpu-with-unstable-binnmu

With this change, it should be possible to accept binNMUs from *pu again.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-17 20:08:53 +00:00
Niels Thykier
db885b7184
Drop comment that no longer matches the context
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-17 18:50:48 +00:00
Niels Thykier
814926f5c3
try_migration: Remove implicit rollback
The issue with an implicit rollback is that the caller has no idea
whether it should invoke rollback or not if an exception occurs.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-17 18:30:12 +00:00
Niels Thykier
a05a249e04
Record exactly which binaries are updated in the transaction
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-16 20:03:19 +00:00
Niels Thykier
5d49a41204
Introduce a "Transaction" for changes to testing
This isolates the undo handling in the new transaction object and in
doop_source, which currently generates the undo items.  This commit
will be a stepping stone to rewriting the undo handling.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-16 19:58:02 +00:00
Niels Thykier
68fd0ba4b2
Remove --components/COMPONENTS as it is auto-detected
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-16 19:55:04 +00:00
Niels Thykier
7124313aa6
Make InstallabilityTester suite agnostic
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-16 11:00:02 +00:00
Niels Thykier
ced7b7b413
Refactor britney.py to use InstallabilityTester less
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-16 11:00:01 +00:00
Niels Thykier
0099c34d93
Refactor _read_binaries to reduce its complexity a bit
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-16 11:00:00 +00:00
Niels Thykier
f547fb20b1
suiteloader: Create a _setup_architecture method
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-16 10:59:59 +00:00
Niels Thykier
7efa865a04
Move Suite/Package loading into a separate class/module
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-15 21:58:37 +00:00
Niels Thykier
deb9482a2f
Move parse_provides to britney2.utils
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-15 21:58:36 +00:00
Niels Thykier
bb9712d7e9
Fix typo in the logging level mapping
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-15 21:58:35 +00:00
Niels Thykier
b322279735
britney.conf: Bump autopkgtest regression deadline by 2 days
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-15 19:40:37 +00:00
Ivo De Decker
3828a76053 Filter out cruft in find_smooth_updateable_binaries
When calculating smooth updateable binaries, filter out cruft binaries from
unstable, because they will not be part of the set of packages that britney
will try to migrate to testing.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-11 16:18:53 +00:00
Ivo De Decker
a56c9458c8 Disable binNMUs from *pu for now
Support for binNMUs from *pu is currently broken, so disable it for now.
see https://bugs.debian.org/916209 for more info.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-11 15:00:19 +00:00
Ivo De Decker
f0ddfc253f Fix parsing of migration item name for binNMU in tpu
The parsing of migration items should also look for the suite name in the
architecture part. This fixes the parsing for migration items like
some-src/amd64_tpu and some-src/amd64_tpu/1.0-1

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-11 14:37:55 +00:00
Ivo De Decker
a3b194c13e Don't calculate smoothbins for items from *pu
When building the excuses, don't calculate smoothbins for items not from
unstable.

This improves the change from commit db584d9fdc

Also try to make the code more readable.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-11 14:32:53 +00:00
Niels Thykier
b41a1a4123
britney.conf: Bump autopkgtest regression deadline by 2 days
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-09 18:03:26 +00:00
Ivo De Decker
db584d9fdc Don't calculate smoothbins for sources not in unstable
When building the excuses, don't calculate smoothbins if the source is not in
unstable. Doing so would result in a KeyError and a crash. This happens for
sources that are in testing and (testing-)proposed-updates, but not in
unstable.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2018-12-07 22:59:18 +00:00
Niels Thykier
12c1d70cb3
britney.conf: Bump autopkgtest regression deadline by 2 days
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-12-02 16:44:25 +00:00