Don't use kombu's SimpleQueue() as that always declares queues and we neither
want that (we want to know when we try to talk to a nonexisting queue), nor
might the RabbitMQ ACL allow us to do that. So use kombu.Producer without queue
declaration.
Until now, autopkgtests were triggered via an external "adt-britney" command
from lp:auto-package-testing. This duplicated a lot of effort (such as its own
chdist and reverse dependency calculation), maintains a lot of state files,
uses hardcoded absolute paths to these external tools, and is quite hard to
understand and maintain. We want to get rid of all this.
Add logic to AutoPackageTest.request() to use britney's existing reverse
dependency maps and figure out the set of tests to run for packages in
unstable. These are being tracked in "requested_tests".
Add logic to AutoPackageTest.submit() to send test requests to the AMQP server
specified in the new "ADT_AMQP" config key. For testing this can be a file://
URL, and if not set no test requests will be sent at all.
The set of tests which were requested in previous runs are tracked in
UNSTABLE/autopkgtest/pending.txt, so that we don't re-request tests in
subsequent runs.
There is no implementation for collect() and results() yet, these will be done
in a separate commit.
Add initial test cases.
Don't change britney.conf from the source tree and restore it back after the
tests; this is prone to leave a broken config behind if a test gets
interrupted. Instead, copy the file to our temp dir, and let tests hack on it
there.
(Introduced in lp:~canonical-ci-engineering/britney/boottesting-support)
Extend read_sources to store a new AUTOPKGTEST boolean flag, which is true if
the Testsuite: field exists and starts with "autopkgtest" (this covers autodep8
cases like autopkgtest-pkg-perl).
Extend TestData.add() to take a new testsuite argument which specifies the
source's Testsuite: field.
Move nuninst cloning out of the check loop and always populate the new
nuninst entirely.
This will allow some simplifications in other places.
Signed-off-by: Niels Thykier <niels@thykier.net>
Use a set to filter out seen items to avoid doing O(n^2)
de-duplication. For very large hints, this can take considerable
time.
Using "seen_items" to build the actual hints on the (unverified)
assumption that Python can do something "smart" to turn a set into a
frozenset faster than it can with a list.
Signed-off-by: Niels Thykier <niels@thykier.net>
Britney is now smart enough to produce the same result from hints
regardless of the order of the items in the hint. With this in mind,
we can have the original auto-hinter produce hints as sets and filter
out duplicates as we produce them.
Note that the hints are sorted to produce deterministic output (to
make it easier to compare the hints between runs and changes).
Signed-off-by: Niels Thykier <niels@thykier.net>
Avoid some cases of O(n^2) behaviour in sort_actions and reduce the
size of n for the remaining O(n^2)-ish behaviour by filtering out
removals early on.
Signed-off-by: Niels Thykier <niels@thykier.net>
It's more natural to say "check this package if the current arch is in
this list" than "do not check this package if the current arch is not
in this list"
Signed-off-by: Adam D. Barratt <adam@adam-barratt.org.uk>
This updates the doop_source and _compute_groups functions so
that binary packages that are built from a different source
aren't included as part of an update to the original source.
In the event that it's a binary-only update, also don't remove
the hijacked packages from testing.
This change also removes an obsolete comment regardarding pre-conditions
for the _compute_groups function.
The live-data tests rely on an inconsistency, since they were before
Britney started to record the Essential field.
Signed-off-by: Niels Thykier <niels@thykier.net>
britney assumes that a package build is uniquely described by its
name, version and architecture. Particularly when constructing
Packages files by hand for testing purposes this assumption can be
violated, leading to confusing behaviour. This change makes britney
look for such mismatches, and report if any are found.
In some scenarios, it was possible to trigger a bug in the
installability tester, where it would fail to update the "choices" set.
The requirements for triggering this seems to be something like:
* Obtain a choice that is possible to solve.
* Resolve the choice without recursing (with a backtrack point)
* Obtain a second choice that is impossible to solve.
After the two first steps, the installability tester would fail to
update the "choices" set (or, rather, changes would be invisible to
the "_pick_choice" function). Fortunately, most packages are either
trivially installable or trivially uninstallable, so the bug seems to
be rather rare if triggred at all.
Signed-off-by: Niels Thykier <niels@thykier.net>
Notably:
* Avoid repeated calls frozenset(X), where we can trivially do
without.
* Skip the inner loop, when "i" is in "to_skip".
* Use a set rather than a list for "to_skip" as we do more
membership tests.
Signed-off-by: Niels Thykier <niels@thykier.net>
The get_dependency_solvers method returns a (boolean, list)-tuple, but
the boolean can always be implied from the list (in boolean context).
Signed-off-by: Niels Thykier <niels@thykier.net>
Martin Pitt
Steve Langasek
Francis Ginther
Niels Thykier
Adam D. Barratt
Anthony Towns
Ivo De Decker
Julien Cristau