Add new "huge" AMQP queues
Whenever a package like glibc or perl gets uploaded, these trigger thousands of
tests that fill the queues for several days. This can't (solely) be addressed
by adding more capacity due to the sheer size. The main annoyance of this is
that this blocks propagation of every other package in -proposed during that
time.
AMQP has no concept of priorities within a queue (they are strictly FIFO), so
for packages that trigger a huge number of tests, send these to the new
debci-$release-huge-$arch queues instead.
LP: #1647948
Add new autopkgtest policy: it determines the autopkgtests for a
source package (its own, direct reverse binary dependencies, and
Testsuite-Triggers), requests tests via AMQP, fetches results from swift, and
keeps track of pending tests between run. This also caches the downloaded
results from swift, as re-dowloading them all is very expensive.
This introduces two new hints:
* force-badtest pkg/ver[/arch]: Failing results for that package will be
ignored. This is useful to deal with broken tests that get imported from
Debian or are from under-maintained packages, or broke due to some
infrastructure changes. These are long-lived usually.
* force-skiptest pkg/ver: Test results *triggered by* that package (i. e.
reverse dependencies) will be ignored. This is mostly useful for landing
packages that trigger a huge amount of tests (glibc, perl) where some tests
are just too flaky to get them all passing, and one just wants to land it
after the remaining failures have been checked. This should be used rarely
and the hints should be removed immediately again.
Add integration tests that call britney in various scenarios on constructed
fake archives, with mocked AMQP and Swift results.
Martin Pitt