681 Commits

Author SHA1 Message Date
Niels Thykier
bee0a99528 Reuse pkg_ids instead of creating them ad-hoc
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-28 09:37:28 +00:00
Niels Thykier
71d181aa83 Add pkg_id field to BinaryPackage
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-28 09:37:28 +00:00
Niels Thykier
856bbba232 Add missing "None" guard for faux packages
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 17:48:19 +00:00
Niels Thykier
c14ee0410b Let hintdir specific the actual directory of hints files
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:49:09 +00:00
Niels Thykier
1ee1375d38 britney: Fix bug in reading d-i Packages
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:46:19 +00:00
Niels Thykier
dfcbae7403 britney.conf: Set HINTSDIR
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:41:45 +00:00
Niels Thykier
f23a72dcdb britney2: Mirror layout - also read d-i Packages files
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:39:06 +00:00
Niels Thykier
247b083e31 Revert "britney.conf: Add components for d-i"
Turns out it does not work (as d-i does not have a Sources file as
they would be redundant).

This reverts commit bb79d9333fbffb77673e6d2e6da0c8b5d0a5b965.
2016-05-26 16:37:55 +00:00
Niels Thykier
bb79d9333f britney.conf: Add components for d-i
Possibly, Britney should support this somewhat more gracefully.  But
start with this work around for now.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:32:50 +00:00
Niels Thykier
d2663d2631 Use paths valid for franck.d.o
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:30:31 +00:00
Niels Thykier
f5c037cbb7 britney.conf: Read packages from a local mirror
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-26 16:16:34 +00:00
Niels Thykier
93757b5650 Omit faux packages from the HeidiResult file
We do not need these in the HeidiResult file, so we might as well stop
emitting them.  Note that faux packages will still appear in other
output (like the "uninst" lists/caches).

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-22 05:48:51 +00:00
Niels Thykier
3c0dcf0de3 britney: Permit comments in Package-List field of constraints
python-apt's "TagFile" seems to cut the field short if it meets a
comment.  Therefore allow comments in the actual field value to avoid
this nasty behaviour.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 17:13:22 +00:00
Niels Thykier
e50037a7da constraints parsing: Fix off-by-one in split
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 16:44:20 +00:00
Niels Thykier
117a5c2166 Write a short incomplete README
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 15:32:25 +00:00
Niels Thykier
3fde3ca7a0 Make Faux packages default to non-free
Packages in main should not need them.  Presuming we eventually make
Britney enforce separation between components, "non-free" seems like a
better default.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 13:39:11 +00:00
Niels Thykier
17a8fd6cdc Correct error message to use the real field name
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 13:38:52 +00:00
Niels Thykier
cf4dd06e3c Update britney_nobreakall.conf
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:57:07 +00:00
Niels Thykier
c9ed47ca6d britney.py: Remove unused import
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:28 +00:00
Niels Thykier
ccb473e19f britney.py: Support compressed Packages/Sources
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:28 +00:00
Niels Thykier
398c7a4959 Add support for loading faux packages
Part of GH#5.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:27 +00:00
Niels Thykier
58b704f3d0 britney.py: Extract a _parse_provides method
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:27 +00:00
Niels Thykier
7ccbfe2fdf Make "Keep-installable" constraints overrule nuninst counters
If there is a regression in "present-and-installable" constraints (on
non-break architectures), then discard the item even if the nuninst
counters have improved.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:27 +00:00
Niels Thykier
f3e37faf7e Add present-and-installable constraints support
Solves first half of GH#5.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:47:23 +00:00
Niels Thykier
e92bbf386f britney_util: Replace broken if with an assert
Replace a "if" with an unconditionally false condition with a negative
assert of the "correct" condition.  The bug has been present for 2½
years with no known adverse effects.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
ec37bd06aa Optimise checking of affected packages
For (non-hint) migrations, split the set of affected into two
parts; one for reverse dependencies and one "negative dependencies"
(e.g. Conflicts).

If there are only regressions in the nuninst after checking the first
set, then there is no reason to continue with the second set (as
"negative dependencies" can only make it worse at that point).

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
a4daee0866 britney.py: Optimise scheduling a bit
Avoid rescheduling the tail of packages if we can avoid it.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
ac66e3118d Reject some excuses with unsatisfiable depends
Ideally we would reject all items with known unsatisfiable
dependencies as they would not be installable in testing.  However,
there are a few known corner cases where we still want to migrate them
(notably when they are already broken in testing).

This commit is an attempt to weed out some of the "obviously" broken
items that will not successfully migrate.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
8991889fad Prefer pkg.foo to pkg[FOO] for binary packages
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:38 +00:00
Niels Thykier
4eebb5180f Make binary packages a namedtuple
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
7230cd29c0 Re-order parsing of suites to avoid changing binary pkgs
In the next commit, the binary packages will be turned into
namedtuples and will become immutable objects.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
c4e7596837 read_pkgs: Parse Provides a bit earlier
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
946d705e2b read_pkgs: Refactor source+src_version parsing
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
71a29aa264 excuse: present dependency info in a machine parseable way
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
eb2c6aad27 Remove redundant notes about arch:all packages
In an architecture only migration, we currently see two notes for
every arch:all packages:

 * Ignoring "new" arch:all package
 * Ignoring "removal" of arch:all package

But a closer look at the situation is that the arch:all packages is
generally the same in both testing and the source suite.

This commit removes these notes when the arch:all is the same in both
suites.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
5d31eee805 Add block/unblock-hint info to excuses.yaml
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
1028860c53 Move missing-builds/cruft to excuses
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
7b8b9e8258 excuses.yaml: Distinguish between source and item names
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:37 +00:00
Niels Thykier
1400fa1384 Remove "Not considered" note from excuses.yaml
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
a7e493c64d excuses.yaml: Make maint + component info machine parsable
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
26c6f3eb87 Add policy-info to YAML excuses
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
63b08f8278 Optimise a few hints.search calls
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
ce5b893a19 Prefer bugs files from the state-dir
Closes GH#2.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
7e94e5567c Prefer Urgencies file from the state-dir
Partly solves GH#2

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
bcff800040 Move Dates into a new state-dir
Partly solves GH#2.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
6d6a7ac529 Move RC bug handing into a policy
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
5ec3aea43a Move age-handling into a separate file
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-05-21 12:43:36 +00:00
Niels Thykier
2e81e55c56 undo_items: Gracefully handle skipped cruft items in undo
In 203466b, we stopped migrating cruft items if they were not in
testing already.  However, if the migration failed we would still
attempt to undo it from testing, which (unsurprisingly) does not work
very well.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-27 17:11:34 +00:00
Niels Thykier
4ced8a3d41 britney.py: Fix crash with arch:all packages being shadowed
There was an implicit assumption in Britney that was violated recently
(presumbly in 12d9ae8).  This assumption is that a source package will
reference the *latest* version of an arch:all package; even if that
version is not built by that source (version).

Image the following packages:

  * p-all/1 and p-all/2 are arch:all packages.
    - p-all/1 is generally superseded by p-all/2 except it has not
      been removed yet (regardless of why that happens).
  * src-a/2 builds p-all/2.

When Britney loads the packages (assuming packages are sorted[1]):

 * p-all/1 is read and associated with src-a/2 (despite not being
   built from that source)
 * p-all/2 is read afterwards.  It would replace p-all/1 and be
   assoicated with src-a/2 as well.
   - Prior to 12d9ae8, the assoication would be "pkg, arch" and
     therefore the same.
   - In 12d9ae8a and after, the association would include version
     and both would be associated at the same time.
 * Since p-all/1 is discarded, it is never added to the
   installability tester (nor the solver).  It then promptly throws
   an exception when asked to deal with a package it believe does
   not exist.
   - This can trivially be solved by removing the association
     between p-all/1 and src-a/2 when p-all/2 is read.

So far so good.  Unfortunately, this adds another another problem!

When Britney migrates a binNMU, she currently removes *all* binary
packages on that architecture - *including* the arch:all packages.
This works out in the end because the arch:all packages be re-added
since they associated with the source package.

  At first glance, this appears to be a non-issue until you add
hijacking to the mix.  This is *exactly* the case behind #709460.
However, now we end up in a situation where the arch:all package is
removed during binNMU and not re-added (because the arch:all package
is not associated with *that* source any more).

This commit fixes all of the above by:

 1) Correctly disassociating superseded arch:all packages from their
    source packages
 2) Falsely associating the "new" arch:all package to the source that
    provided the superseded arch:all package (to avoid issues with
    binNMUs and hijacked packages).
    - This should probably be undone at some point when Britney does
      not need this "hack" any more.

This fix is currently relying on the packages being sorted.  But so
did the fix for #709460 (see [1] again).  It might be prudent to fix
that as well (or at least reject the packages file rather than produce
"random" results).

[1] Which is another assumption we have been relying and that is
currently satisfied.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-12 18:18:16 +00:00
Niels Thykier
8ab7fc0e89 Avoid some unnecessary effort in compuing "affected"
* A package being removed is *not* affected
   - It will just be filtered out later in "check_packages"
 * Since all transitive reverse dependencies will be added to
   "affected" at the end of the method, there is no reason to
   find the immediate set of reverse dependencies of a package
   if the package is added itself as well.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-04-11 16:27:28 +00:00