In the rare case that a hint removed an uninstallable binary, the
binary could still be included in the nuninst counter.
Regression introduced in a46dd88.
Signed-off-by: Niels Thykier <niels@thykier.net>
sort_actions() can be quite expensive and it is wasteful to resort
actions after each successful "easy"-hint.
Signed-off-by: Niels Thykier <niels@thykier.net>
There are no uses of "lundo" left for a non-hint recurse run (i.e.
the "main run"), so there is no point in building it.
The "lundo"-list is still used in the recurse run of a "hint"-hint.
Signed-off-by: Niels Thykier <niels@thykier.net>
The "do_all"-method now checks the architectures of all changes
applied. If they entirely consist of items from "break archs", then
"do_all" will disregard the current "break archs" setting when
comparing nuninst counters.
This change avoids unintended installability regressions on break
arches when a hint (manual or automatic) apply only to packages on
break arches.
Signed-off-by: Niels Thykier <niels@thykier.net>
Rename local variables and avoid repeated chained lookups. In
particular, avoid confusing cases like:
[...]
version = binaries[parch][0][binary][VERSION]
[...]
binaries[parch][0][binary] = self.binaries[item.suite][parch][0][binary]
version = binaries[parch][0][binary][VERSION]
Where "version" here will refer to two different versions. The former
the version from testing of a hijacked binary and the latter the
version from the source suite (despite the look up using the "testing"
table, due to the testing copy being updated).
Notable renamings:
* binaries => packages_t (a.k.a. self.binaries['testing'])
* binaries[parch][0] => binaries_t_a
* binaries[parch][1] => provides_t_a
* Similar naming used for "item.suite" instead of "testing"
The naming is based on the following logic:
* self.binaries from "packages" files
(by this logic, it ought to be "self.packages", but that is
for later)
* The "_X_a" is short for "[<suite>][<parch>]" look ups.
* binaries_X_a and provides_X_a are the specialised parts of
packages_X_a that deal with (real) binary packages and
provides (i.e. virtual packages) respectively.
Signed-off-by: Niels Thykier <niels@thykier.net>
Extract a specialised iter_packages_hint from iter_packages that only
deals with applying hints. This simplifies iter_packages AND avoids
having to re-compute the uninstallability counters after each single
item in the hint.
This means that a hint can now avoid triggering expontential runtime
provided only that the "post-hint" stage does not trigger expontential
runtime. Previously the hint had to be ordered such that none of the
items in the hint caused such behaviour (if at all possible).
Signed-off-by: Niels Thykier <niels@thykier.net>
Avoid creating two dependency clauses for dependencies emulating a
"version range" a la:
Depends: pkg-a (>= 2), pkg-a (<< 3~)
Previously this would create two clauses a la:
- (pkg-a, 2, arch), (pkg-a, 3, arch)
- (pkg-a, 1, arch), (pkg-a, 2, arch)
However, it is plain to see that only (pkg-a, 2, arch) is a valid
solution and the other options are just noise. This patch makes
Britney merge these two claues into a single clause containing exactly
(pkg-a, 2, arch).
Signed-off-by: Niels Thykier <niels@thykier.net>
The "new" auto hinter relies on partial ordering to determine, when
what can migrate (and what needs to migrate at the same time). At the
same time, it leverages on "_compute_groups" to allow it to include
"removals" in its hints.
Signed-off-by: Niels Thykier <niels@thykier.net>
Avoid smooth-updating libraries in hints, when all of their reverse
dependencies will certainly disappear in the same hint.
Note that in "hint"-hint, reverse dependencies removed in the
following "full run" will not cause the smooth-updated library to be
removed. Instead these will still be removed in the end as usual, but
in some cases that is too late.
Signed-off-by: Niels Thykier <niels@thykier.net>
Rename find_upgraded_binaries into _compute_groups. The new method
will also compute what binaries will be updated in or added to testing
after migration.
Signed-off-by: Niels Thykier <niels@thykier.net>
- A test for a package/version can be triggered by several distincts causes with different results. Modify the data structure to store this instead of only storing the last cause and result, leading to missing test results in excuses.
- Check if failures are regressions (there is at least one pass and migration is blocked) or if test has always been failing. In the latter case, migration is not blocked.
- Add colours to excuses.html to distinguishes the test results
- Point jenkins URL to lastBuild
In excuse_unsat_deps(), mark unsatisfiable dependencies not just in the HTML,
but also in the invalid_deps list. If we have any of those in
should_upgrade_src(), block the package and don't run the autopkgtest. This
avoid running tests for known-uninstallable packages, which just leads to
guaranteed failures, manual intervention of re-running tests after it becomes
installable, and spamming maintainers with a FAIL/PASS notification.
Based on Colin Watson's code to do the same from the "britney2-ubuntu"
repository[1] revision 306, 308 and 309.
Notable differences include:
* output include version of source package being removed
* output prefix removals with a "-" (otherwise it would be identical to
a upgrade/new source with the change above).
[1] http://bazaar.launchpad.net/~ubuntu-release/britney/britney2-ubuntu/revision/306
Signed-off-by: Niels Thykier <niels@thykier.net>
This introduces a new variable HINTSDIR, which overrides the location of the
Hints dir (normally it is read from the UNSTABLE dir).
Please note that this is the location of the dir that contains the Hints dir.
Signed-off-by: Niels Thykier <niels@thykier.net>
This introduces a new variable OUTPUTDIR, which overrides the location where
the new dates file is written. This allow to run britney against a read-only
copy of the data.
Signed-off-by: Niels Thykier <niels@thykier.net>
Set the default to maxint until we've read something.
Reported-by: Ivo De Decker <ivo.dedecker@ugent.be>
Signed-off-by: Julien Cristau <jcristau@debian.org>
The new Installability Tester (IT) module replaces the remaining
C-parts. Unlike C-implementation, it does not give up and emit an
"AIEEE" half-way through.
In order to determine installability, it uses two sets "musts" and
"never". As the names suggest, the sets represents the packages that
must be (co-)installable with the package being tested and those that
can never be co-installable. For a package to be installable, "musts"
and "never" have remain disjoint.
These sets are also used to reduce the number of alternatives that are
available to satisfy a given dependency. When these sets are unable
to remove the choice completely, the new IT defers the choice to later.
This occasionally reduces backtracking as a later package may conflict
or unconditionally depend on one of the remaining alternatives.
Signed-off-by: Niels Thykier <niels@thykier.net>
FAILED/SUCCESS lines would be separated by a whitespace from the list
of architectures, but not itself followed by whitespace. This is slightly
confusing, as one could interpret it as being a heading for the following
block of tested packages, rather that the final result of the previous
block.
We stopped populating the element with real data some time ago, it's
time to drop it entirely.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
Multiarch adds a Depends: foo:any syntax, permitted only if the
target of the dependency is "Multi-Arch: allowed". This has
been supported by dpkg and apt for some time and is now safe to
use in unstable.
[Adam D. Barratt: adjusted to use consts.py]
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
As HintItem is now redundant, also replace it with a new class -
UnversionnedMigrationItem - and migrate users of the classes to use
the new versions.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
Where possible, avoid creating a list only to discard immediately
afterwards. Example:
"""
for x in sorted([x for x in ...]):
...
"""
Creates a list, passes it to sorted, which generates a new list and
sorts that copy. Since sorted accepts an iterable, we can avoid the
"inner" list and just pass it a generator expression instead.
Signed-off-by: Niels Thykier <niels@thykier.net>
By moving the package loop inside register_reverses, it will be
invoked a lot less (reducing the overhead of invoking functions).
Signed-off-by: Niels Thykier <niels@thykier.net>
Beside some "minor differences" they were computing the same "tree"
(read: "graph"), so merge them into one (get_reverse_tree) and
properly document return value and special cases.
Signed-off-by: Niels Thykier <niels@thykier.net>
Rewrite the arguments of find_upgraded_binaries to not use an instance
of MigrationItem. We want to call it at a time where we have not
created MigrationItems yet.
Signed-off-by: Niels Thykier <niels@thykier.net>
A removal hint will generate both source and per-arch excuses if the
version of the source package differs between testing and unstable. If
the source versions are the same then only the per-arch excuses will
be generated.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
In rare cases with hints with overlapping virtual packages provided by
different sources, this can make a difference.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
If there are multiple versions of an arch:all package in unstable (due
to outdated or no longer built arch:any packages) then only one of them
should be recorded in the list of binary packages built from the source
package. Otherwise we may try and remove the binary package from various
lists multiple times, leading to crashes.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
Given a source which provides two packages and has different versions
in testing and unstable, binNMUs in unstable corresponding to the
older source version should not be considered as migration candidates.
For example:
testing
-------
source 1
bin 1 arch1
bin 1 arch2
unstable
--------
source 2
bin 2 arch1
bin 1+b1 arch2
The binary migration on arch2 should not be considered a candidate.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
Although this should never happen, rather than crashing if one of the
versions is none, simply indicate that they are unequal.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
Although this isn't an issue during normal runs, the excuses might be
built multiple times during a hint-tester run and should not accumulate
during the run.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
For those hints which don't cause an immediate run (i.e. other than
easy, hint and force-hint), re-build the excuses after adding the
hint so that the actions are accounted for in later hints.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
A binNMU does not rebuild architecture:all packages. For migrations via
unstable this is not a problem as the packages corresponding to the
source upload are still present. However, for *pu migrations, the set of
packages considered only includes architecture-specific packages. In
order to avoid installability issues with packages in testing which
depend on the arch:all packages, we leave the existing arch:all packages
in testing and only consider the arch-specific packages for migration.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
The test only needs to consider whether any binaries exist on a given
arch, not how many of them there are (or indeed which binaries they are)
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
When checking whether a tpu source has built on a particular arch, we
should only consider binaries produced by the latest version of the
source package in tpu.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
Originally when binNMUs for packages in testing were scheduled, the
binaries would be installed into tpu with no accompanying source. This
allowed the "removed binary" portions of should_upgrade_srcarch() to be
skipped (as britney had generated a faux source record).
dak now adds the source package to tpu in such cases which lead to the
"removed binary" checks being applied to binNMUs in tpu with potentially
destructive consequences. For example, if a package with amd64 and i386
binaries in testing were binNMUed on just amd64, britney would notice
that there were no i386 binaries in tpu and subsequently remove the i386
binaries from testing as well.
In order to resolve this, we skip the check for removed binaries when
building excuses for a binary-only migration via *pu.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
The primary difference between the parsing / output of excuses for *pu
and unstable unblocks is the messages displayed. We can therefore remove
some duplication by having the same code handle both, outputting the
appropriate message.
Where a *pu package is also the subject of a "block" (most likely during
a freeze) we only supply the "needs approval" or "approved" message;
previously both "needs approval" and "not touching due to block" were
output, which is redundant. We ensure that there is always a dummy
"block" hint for *pu packages to provide the "needs approval" behaviour.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
An "approve" hint is effectively an unblock for tpu packages and britney
is already quiite happy to parse "unblock $pkg/$tpuversion".
We allow the old name to be used for compatibility and replace it with
"unblock" internally.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
A dependency on an arch-specific package which is not a valid candidate
should lead to the depending package not being a candidate.
For now we ensure that the generated excuses output remains the same,
so that we don't have to wait for consumers to adapt to a new format.
Changing the output format should be revisited at a later point.
See Debian bug #693068.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
The code using the variables was refactored in 694d614b. As a result
they were still set in iter_packages() but never subsequently used.
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
Niels Thykier
Colin Watson
Adam D. Barratt
Jean-Baptiste Lallement
Martin Pitt
Ivo De Decker
Julien Cristau
Iain Lane
Mathieu Trudel-Lapierre
Colin Watson