749 Commits

Author SHA1 Message Date
Niels Thykier
855fe68009 Make tpu optional in the config file
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-06-03 15:34:24 +00:00
Niels Thykier
c224c55d88 Add a "block-all new-source" hint
This hint will block all "new" source migrations.  Source migrations
for packages already in testing will be affected by this.  As usual,
this hint can be overruled by an unblock hint.

Closes: GH#8

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-06-03 15:34:24 +00:00
Niels Thykier
481f7fde08 auto-hinter: Reduce size of inner loop
Reduce a "loop over all valid items" to a "loop over an item's
dependencies + reverse dependencies".  For sparse graphs, this
is much more efficient.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-06-03 15:34:23 +00:00
Niels Thykier
e517c3f315 auto-hinter: Remove some now redundant conditionals
Made possibly be the previous commit.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-06-03 15:34:23 +00:00
Niels Thykier
3294c99036 auto-hinter: Prune size of valid excuses and their deps
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-06-03 15:34:23 +00:00
Niels Thykier
43266ee091 auto hinter: Prune some duplicates earlier
The same item can re-appear multiple times in one iteration, so prune
duplicates in each iteration.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-06-03 15:34:23 +00:00
Niels Thykier
463d1a67d9 Prefer pkg_id.foo to pkg_id[X]
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-28 09:37:28 +00:00
Niels Thykier
429e8f1199 britney.py: Turn pkg_ids into namedtuples
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-28 09:37:28 +00:00
Niels Thykier
bee0a99528 Reuse pkg_ids instead of creating them ad-hoc
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-28 09:37:28 +00:00
Niels Thykier
71d181aa83 Add pkg_id field to BinaryPackage
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-28 09:37:28 +00:00
Niels Thykier
856bbba232 Add missing "None" guard for faux packages
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 17:48:19 +00:00
Niels Thykier
c14ee0410b Let hintdir specific the actual directory of hints files
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:49:09 +00:00
Niels Thykier
1ee1375d38 britney: Fix bug in reading d-i Packages
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:46:19 +00:00
Niels Thykier
f23a72dcdb britney2: Mirror layout - also read d-i Packages files
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:39:06 +00:00
Niels Thykier
3c0dcf0de3 britney: Permit comments in Package-List field of constraints
python-apt's "TagFile" seems to cut the field short if it meets a
comment.  Therefore allow comments in the actual field value to avoid
this nasty behaviour.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 17:13:22 +00:00
Niels Thykier
e50037a7da constraints parsing: Fix off-by-one in split
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 16:44:20 +00:00
Niels Thykier
3fde3ca7a0 Make Faux packages default to non-free
Packages in main should not need them.  Presuming we eventually make
Britney enforce separation between components, "non-free" seems like a
better default.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 13:39:11 +00:00
Niels Thykier
17a8fd6cdc Correct error message to use the real field name
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 13:38:52 +00:00
Niels Thykier
c9ed47ca6d britney.py: Remove unused import
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:28 +00:00
Niels Thykier
ccb473e19f britney.py: Support compressed Packages/Sources
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:28 +00:00
Niels Thykier
398c7a4959 Add support for loading faux packages
Part of GH#5.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:27 +00:00
Niels Thykier
58b704f3d0 britney.py: Extract a _parse_provides method
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:27 +00:00
Niels Thykier
7ccbfe2fdf Make "Keep-installable" constraints overrule nuninst counters
If there is a regression in "present-and-installable" constraints (on
non-break architectures), then discard the item even if the nuninst
counters have improved.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:27 +00:00
Niels Thykier
f3e37faf7e Add present-and-installable constraints support
Solves first half of GH#5.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:23 +00:00
Niels Thykier
ec37bd06aa Optimise checking of affected packages
For (non-hint) migrations, split the set of affected into two
parts; one for reverse dependencies and one "negative dependencies"
(e.g. Conflicts).

If there are only regressions in the nuninst after checking the first
set, then there is no reason to continue with the second set (as
"negative dependencies" can only make it worse at that point).

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
a4daee0866 britney.py: Optimise scheduling a bit
Avoid rescheduling the tail of packages if we can avoid it.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
ac66e3118d Reject some excuses with unsatisfiable depends
Ideally we would reject all items with known unsatisfiable
dependencies as they would not be installable in testing.  However,
there are a few known corner cases where we still want to migrate them
(notably when they are already broken in testing).

This commit is an attempt to weed out some of the "obviously" broken
items that will not successfully migrate.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
8991889fad Prefer pkg.foo to pkg[FOO] for binary packages
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
4eebb5180f Make binary packages a namedtuple
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
7230cd29c0 Re-order parsing of suites to avoid changing binary pkgs
In the next commit, the binary packages will be turned into
namedtuples and will become immutable objects.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
c4e7596837 read_pkgs: Parse Provides a bit earlier
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
946d705e2b read_pkgs: Refactor source+src_version parsing
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
eb2c6aad27 Remove redundant notes about arch:all packages
In an architecture only migration, we currently see two notes for
every arch:all packages:

 * Ignoring "new" arch:all package
 * Ignoring "removal" of arch:all package

But a closer look at the situation is that the arch:all packages is
generally the same in both testing and the source suite.

This commit removes these notes when the arch:all is the same in both
suites.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
5d31eee805 Add block/unblock-hint info to excuses.yaml
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
1028860c53 Move missing-builds/cruft to excuses
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
1400fa1384 Remove "Not considered" note from excuses.yaml
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
63b08f8278 Optimise a few hints.search calls
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
6d6a7ac529 Move RC bug handing into a policy
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
5ec3aea43a Move age-handling into a separate file
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
4ced8a3d41 britney.py: Fix crash with arch:all packages being shadowed
There was an implicit assumption in Britney that was violated recently
(presumbly in 12d9ae8).  This assumption is that a source package will
reference the *latest* version of an arch:all package; even if that
version is not built by that source (version).

Image the following packages:

  * p-all/1 and p-all/2 are arch:all packages.
    - p-all/1 is generally superseded by p-all/2 except it has not
      been removed yet (regardless of why that happens).
  * src-a/2 builds p-all/2.

When Britney loads the packages (assuming packages are sorted[1]):

 * p-all/1 is read and associated with src-a/2 (despite not being
   built from that source)
 * p-all/2 is read afterwards.  It would replace p-all/1 and be
   assoicated with src-a/2 as well.
   - Prior to 12d9ae8, the assoication would be "pkg, arch" and
     therefore the same.
   - In 12d9ae8a and after, the association would include version
     and both would be associated at the same time.
 * Since p-all/1 is discarded, it is never added to the
   installability tester (nor the solver).  It then promptly throws
   an exception when asked to deal with a package it believe does
   not exist.
   - This can trivially be solved by removing the association
     between p-all/1 and src-a/2 when p-all/2 is read.

So far so good.  Unfortunately, this adds another another problem!

When Britney migrates a binNMU, she currently removes *all* binary
packages on that architecture - *including* the arch:all packages.
This works out in the end because the arch:all packages be re-added
since they associated with the source package.

  At first glance, this appears to be a non-issue until you add
hijacking to the mix.  This is *exactly* the case behind #709460.
However, now we end up in a situation where the arch:all package is
removed during binNMU and not re-added (because the arch:all package
is not associated with *that* source any more).

This commit fixes all of the above by:

 1) Correctly disassociating superseded arch:all packages from their
    source packages
 2) Falsely associating the "new" arch:all package to the source that
    provided the superseded arch:all package (to avoid issues with
    binNMUs and hijacked packages).
    - This should probably be undone at some point when Britney does
      not need this "hack" any more.

This fix is currently relying on the packages being sorted.  But so
did the fix for #709460 (see [1] again).  It might be prudent to fix
that as well (or at least reject the packages file rather than produce
"random" results).

[1] Which is another assumption we have been relying and that is
currently satisfied.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-12 18:18:16 +00:00
Niels Thykier
8ab7fc0e89 Avoid some unnecessary effort in compuing "affected"
* A package being removed is *not* affected
   - It will just be filtered out later in "check_packages"
 * Since all transitive reverse dependencies will be added to
   "affected" at the end of the method, there is no reason to
   find the immediate set of reverse dependencies of a package
   if the package is added itself as well.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:28 +00:00
Niels Thykier
38a198e8b2 Replace some string concat+split with tuples
Replace some of the:

 p = binary + "/" + parch
 binary, parch = p.split("/")

with regular tuples.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00
Niels Thykier
1ddf63e14b britney.py: Remove uncommented printf debugging
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00
Niels Thykier
28f69fa505 britney.py: Log when old libs are present but not removed
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00
Niels Thykier
b5db57715b Move two installability testing functions to britney_util
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00
Niels Thykier
05de911090 Make excuses a dict rather than a list
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00
Niels Thykier
9b9d44db3a Move some of the hint parsing into hints.py
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00
Niels Thykier
5083311ec2 britney.py: Rename __log to log
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00
Niels Thykier
7b0d4023d0 britney.py: Remove redundant open before TagFile
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00
Niels Thykier
5bbca73391 Make components and --control-files mutually exclusive
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:27 +00:00