We usually bump force-badtest versions for the devel series, which can cause
apparent regressions in stable tests, or when running tests against
devel-release when bumping the version for devel-proposed.
If a particular test is running on all architectures, the tested version is not
very useful and often even wrong, as we can't yet predict which version will
actually be used (sometimes from testing, sometimes from unstable depending on
apt pinning). So only show it once we have at least one result.
Revert commit 551/bug 1522893 again, as we still want to see test results even
for force-skip'ed packages in some cases.
In cases where we really don't want to run the tests at all, we need to flush
the AMQP queues manually and remove pending.json after the package got
accepted.
Dial back commit 570 a bit: We should tolerate 404's, something went wrong on
uploading the result. We still want to fail hard on any other error as these
would indicate infra problems.
This separates them from Ubuntu and upstream test requests, avoids that any of
those can completely starve the other two, and makes queues easier to manage.
Some tests are known-broken on a particular architecture only.
force-badtest'ing the entire version is overzealous as it hides regressions on
the other architectures that we expect to work. It's also hard to maintain as
the version has to be bumped constantly.
Support hints of the form "force-badtest srcpkg/architecture/all" (e. g.
"force-badtest chromium-browser/armhf/all"). The special version "all" will
match any version.
Add new state "IGNORE-FAIL" for regressions which have a 'force' or
'force-badtest' hint. In the HTML, show them as yellow "Ignored failure"
(without a retry link) instead of "Regression", and drop the separate
"Should wait for ..." reason, as that is hard to read for packages with a long
list of tests.
This also makes retry-autopkgtest-regressions more useful as this will now only
run the "real" regressions.
When using a shared results cache with PPAs (silos) we cannot rely on the
latest time stamp from the distro's results.cache. As soon as there is a new
run for a package in Ubuntu proper, that updated time stamp hides all previous
results for the PPA, and causes tests to be re-requested unnecessarily.
In the CI train we sometimes run into transient "HTTP Error 502: Proxy Error".
As we don't keep a results.cache there, this leads to retrying tests for which
we already have a result in swift, but can't download it. Treat this as a hard
failure now, to let the next britney run try again. This will also tell us if
we need to handle any other status code than 200, 204 (empty result), or 401
(container does not exist).
This is needed by the CI train, where we
(1) don't want to cache intermediate results for PPA runs, as they might
"accidentally" pass in between and fail again for the final silo,
(2) want to seed britney with the Ubuntu results.cache, to detect regressions
relative to Ubuntu.
Introduce ADT_SHARED_RESULTS_CACHE option which can point to a path to
results.cache. This will then not be updated by britney.
This was a giant copy&paste, was disabled four months ago, and the
infrastructure for this ceased to exist.
If this comes back, the AutoPackageTest class should be generalized to also
issue phone boot tests (exposed as new architectures, which should then be
called "platforms"), to avoid all this duplicated code.
Generate https://autopkgtest.ubuntu.com/retry.cgi links for re-running tests
that regressed.
Change Excuse.html() back to usual % string formatting to be consistent with
the rest of the code.
If we have a result, directly link to the log file on swift in excuses.html.
The architecture name still leads to the package history as before.
If result is still pending, link to the "running tests" page instead.
Don't clobber passed run IDs with newer failed results. This is potentially a
bit more expensive as we might re-fetch failed results at every run after a
PASS, but the IDs in our cache will be correct so that we can expose them in
the UI.
Martin Pitt
Iain Lane
Dimitri John Ledkov
Robert Bruce Park
Didier Roche