After using the logging from the previous commit, we figured out which
variable to use to get the architecture for the binary packages of a
source package. This commit amends the issue by checking said variable.
This commit also removes the excess logging implemented in the former
commit.
Still trying to figure out this new functionality. We're getting close
with more logging, so I have added a little bit more logging, and
stopped using what was presumably a lit of all binaries for i386, and
now am using what is hopefully just the list of binaries for the source
package
Since we can't seem to figure out where we're going wrong, I thought I'd
just add verbose logging here and hope it helps with debugging the new
feature, which aims to reduce the number of i386 tests queued.
The latest approach unfortunately didn't work, so I'm trying this
approach using the binaries_info variable.
I've also included the full traceback when the functionality doesn't
work, to enable easier debugging
Recent investigations indicated that approximately 85% of all of the
i386 tests run at autopkgtest.ubuntu.com are pointless.
These tests are pointless for the following reason:
for end users, the dependencies of an arch: all package on
non-arch: all packages are satisfied by the amd64 binaries, not i386
binaries.
This commit introduces a check in the `tests_for_source` function, which
is the function that generates a list of tests to be requested for a
src package on a specified architecture.
The check itself takes the src package name, gets the list of binaries
for that src package and checks to see if the architecture for all of
the binaries of said src package is "all". If all the binaries are
Architecture: "all", then the function returns an empty list and no
tests will be requested for that src package on i386.
Since it's quite hard to test britney code, the implementation is
wrapped in a try except block as to avoid tracebacks blocking britney
runs.
The try except block should be removed once the change is considered to
be stable.
autopkgtest-cloud will now serve:
autopkgtest.ubuntu.com/static/autopkgtest.db.sha256
Britney now calculates the sha256 of the newly downloaded db locally and
checks that it matches the sha256 file served by autopkgtest-cloud,
instead of checking that the content-length header matches the
size of the new downloaded database.
Since the most recent apache2 security update in focal [1], the
content-length header isn't served by default, and it seems that when
it is served it's not entirely accurate. This check has become
brittle, and so we have implemented this new mechanism.
[1] https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/2061816
We don't know why the autopkgtest webserver has stopped providing a
Content-Length header but the current code doesn't handle its absence, so
detect this rather than throwing an exception.
Should be reverted once the header is back, we don't want to be in the dark
about short reads of the db.
As seen in
https://ubuntu-archive-team.ubuntu.com/proposed-migration/log/noble/2024-04-01/12:52:27.log
Having an entire britney run bail because of a connection reset is a bad
outcome!
Instead, catch this exception and avoid adding the test in question to the
list of queued tests (we can pick it up on the next run).
Possibly we should do more clever handling of a ConnectionResetError such as
reconnecting, but this is a minimum fix that will stop britney from aborting.
This avoids endlessly requeuing the test if the test produces
an older result.
This will make tests "disappear" if the infrastructure returns
old results for newer triggers but avoids the problem right
now where we end up queuing the same tests every run.
Tim assures me the auth token in the name is read-only, and also exposed via
http redirects whenever one accesses the bucket via autopkgtest.u.c
frontend, so there is no security issue here; and accessing direct gets us
results even when the autopkgtest.db is out of date (which is the problem we
have right now that we want to route around).
Steve Langasek
Simon Quigley
Paride Legovini
Simon Quigley
Graham Inggs
Tim Andersson
Brian Murray
Julian Andres Klode
Michael Hudson-Doyle
Juerg Haefliger
Łukasz 'sil2100' Zemczak