220 Commits

Author SHA1 Message Date
Iain Lane
93c476a3db autopkgtest: Submit multiple triggers to amqp as a list
We currently concatenate all triggers together into a string, but the
AMQP consumer expects this to be a list.

When using AMQP, keep the triggers as a list. Ensure that the "real"
trigger (the package being tested) is kept first, as before.
2020-07-26 13:50:56 +01:00
Iain Lane
6f5d6bb703 autopkgtest: Don't output an info line for ignored failures
We already skip displaying most other types of results where no action
is required including ALWAYSFAIL, should do it for hints too
2020-07-26 13:50:56 +01:00
Iain Lane
96fcaa481a autopkgtest: Retry downloads
If we have a slightly flaky network / endpoint, retrying can help us to
keep going rather than crash & restart.
2020-07-26 13:50:56 +01:00
Iain Lane
de359a9ae1 excusefinder: Support not having arch-all buildds
On some distros (Ubuntu), arch:all packages are built along with one of
the architectures. We shouldn't be listing 'all' as its own arch in this
case. Instead we filter out the binaries except for on the
'all_buildarch'.
2020-07-26 13:50:56 +01:00
Iain Lane
e7bc681e9a autopkgtest: Save pending.json after every test request
We just had the autopkgtest queues DoSed because britney was crashing
after requesting each reverse dependency for a perl upload, but before
it had written pending.json out so it knew what not to request again.
This was 25,000 requests per arch...

Let's write pending.json straight after sending each request, so that
the next run - even after a crash - won't re-request the same things
again.
2020-07-26 13:50:56 +01:00
Iain Lane
c201c000ee BuildDependsPolicy: handle old python3-apt
Very old versions, such as that run on the Ubuntu archive machine, don't
have the arch parameter.

Handle that by setting APT::Architecture instead
2020-07-26 13:50:56 +01:00
Iain Lane
02492f070b autopkgtest: Support Ubuntu's run_id format
These have a hash appended.

We don't actually use the baseline retrying, which is where the ID
parsing is used, but we might as well handle this, not least so we don't
crash.
2020-07-26 13:50:56 +01:00
Iain Lane
36b606a6be autopkgtest: Allow adt_reference_max_age to be empty 2020-07-26 13:50:56 +01:00
Iain Lane
ed751fd1d8 autopkgtest: Add new force-reset-test hint type
This allows hinters to reset the baseline for a package/arch, so that
previous failures are ignore and not considered regressions
2020-07-26 13:50:56 +01:00
Iain Lane
cfd51ddc2f autopkgtest: Only request reference tests when we want them 2020-07-26 13:50:56 +01:00
Iain Lane
0f6b785d46 autopkgtest: Support autopkgtest.u.c in-progress URL
This is at /running. We use 'are we running on swift?' as a proxy for
deciding which URL format to use
2020-07-26 13:50:56 +01:00
Iain Lane
113e197efa autopkgtest: Handle 'blacklisted' version
The apt version comparison sorts 'blacklisted' greater than most version
numbers, which means that we accidentally apply force hints for version
'blacklisted' to all uploads. Since this is the only case of a hacked
version number, let's special case it so that 'blacklisted' hints only
match packages with 'blacklisted' version.
2020-07-26 13:50:56 +01:00
Iain Lane
e944f23834 autopkgtest: store the submission time when making requests
This is so that we can see more easily see if requests are stuck
2020-07-26 13:50:56 +01:00
Iain Lane
77ee66fcb3 Repair requests without testpkg-version
We're supposed to synthesise an "unknown" version for these, but a bug
in the worker meant we didn't do that in some cases and these leaked
into swift. Let's repair it client-side.
2020-07-26 13:50:56 +01:00
Iain Lane
ff6ba2553a autopkgtest: Publish AMQP messages persistently
We want to ask the broker to write them out to disk, hoepfully
mitigating against it crashing.
2020-07-26 13:50:56 +01:00
Iain Lane
ffbadc30cf autopkgtest: Trigger dkms when there's a dep on dkms
Most DKMS packages do not declare Testsuite: autopkgtest-pkg-dkms, but
we can detect this anyway, and this way we can enforce that the module
is buildable.
2020-07-26 13:50:56 +01:00
Łukasz 'sil2100' Zemczak
2559049b9d Add a new policy to message bugs on SRU regressions
It works like this. We wait until all tests have finished running. and
then grab their results. If there are any regressions, we mail each bug
with a link to pending-sru.html. There's a state file which records the
mails we've sent out, so that we don't mail the same bug multiple times.
2020-07-26 13:50:56 +01:00
Tiago Daitx
8c166cf5e1 Link to update-excuse bugs on update_excuses
LP: #1683749
2020-07-26 13:50:56 +01:00
Robert Bruce Park
35932c55c1 Email Policy, send emails when packages are rejected. 2020-07-26 13:50:56 +01:00
Robert Bruce Park
fea7d283a8 Reject packages if entire source ppa won't migrate 2020-07-26 13:50:56 +01:00
Iain Lane
8c3b9e5893 LinuxPolicy: Make linux* wait on corresponding -meta package
We want to treat linux-$flavor and linux-meta-$flavor as one set in
britney which goes in together or not at all. We never want to promote
linux-$flavor without the accompanying linux-meta-$flavor.

Add a new LinuxPolicy which runs after most of the other policies, which
invalidates linux-foo if linux-meta-foo is invalid.
2020-07-26 13:50:56 +01:00
Iain Lane
27905ccab0
autopkgtest: Hardcode some linux-meta tests
We should really fix this to use Testsuite-Triggers / hint-testsuite-triggers
2020-07-10 21:43:07 +01:00
Iain Lane
c436aafda7
Implement Ubuntu component relationship constraints (ogre model) 2020-07-10 21:43:07 +01:00
Colin Watson
dbb9883380
Allow blocking uploads by way of bugs with the block-proposed tag set 2020-06-29 18:25:41 +01:00
Colin Watson
843751c5cf
Change date format to seconds-since-epoch 2020-06-29 18:25:40 +01:00
Colin Watson
3b8daa98ec
Make sure that containing directories exist before writing output files 2020-06-11 13:00:02 +01:00
Martin Pitt
883f01740f
Skip piuparts policy if summary file is missing 2020-06-11 13:00:02 +01:00
Colin Watson
af7da4e764
Skip bug-based processing if BugsV is missing 2020-06-11 13:00:02 +01:00
Colin Watson
ba3fec61c3
Use default urgency for all packages if Urgencies is missing 2020-06-11 13:00:02 +01:00
Paul Gevers
dc4ea6ca2c
autopkgtest: remember old results instead of discarting them
For packages with lots of reverse dependencies, new versions of those reverse
dependencies may keep on showing up in testing. If migration is blocked until
the results for these new version, migration may take extremely long. If there
are results for the current trigger but for the previous version of the reverse
dependency, use those until the fresh resuts are available.

Similar for the reference runs.
2020-05-01 21:34:30 +02:00
Paul Gevers
6e6214efe8
autopkgtest: call filter_old_results after adding triggers to avoid code duplication 2020-05-01 21:28:50 +02:00
Ivo De Decker
920d82445d Add allow-smooth-update hint
Hint to allow smooth update, even if the section isn't allowed in the
configuration.

Note that this takes the source name and the source version IN TESTING
of the binaries that must be allowed to stay around to allow a smooth
update.
2020-04-01 16:37:56 +00:00
Ivo De Decker
2d34f8220e split off is_smooth_update_allowed() function 2020-04-01 15:49:14 +00:00
Ivo De Decker
e2b196dd4b Don't check implict depends when build is missing 2020-03-20 17:27:33 +00:00
Ivo De Decker
a08b7ebc48 Add implicit dependency policy
Based in part on patches by Niels Thykier <niels@thykier.net>

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2020-03-19 20:10:35 +01:00
Paul Gevers
23c79608ec
autopkgtest: make sure an item is not removed twice 2020-02-22 07:35:39 +01:00
Paul Gevers
ea6d65f657
Let britney reschedule migration-reference/0 runs when they are too old
Currently, britney only schedules reference runs when they don't exist. It does
strip out runs against older versions of the autopkgtest, but the current version
may exist for a while and the reference run can be old. So, add an option to
ignore old results.
2020-02-20 13:21:36 +01:00
Paul Gevers
aa8b84f4b8
Drop the special case that gcc has; it's just a package 2020-02-20 11:24:27 +01:00
Paul Gevers
56babea058
Don't run autopkgtest on architectures where the triggering package doesn't exist 2020-02-14 22:47:03 +01:00
Ivo De Decker
a567dfd58d Mark packages as uninstallable, even if this doesn't block migration
This allows the autopkgtest policy to skip tests for these packages.
2020-02-14 19:34:09 +00:00
Ivo De Decker
c308361fc5 autopkgtest: filter conflicting packages
We only want to add packages which conflict in testing, but don't
conflict in unstable. For those, the newer version (from unstable)
probably fixes the conflict.
2020-02-07 20:46:30 +01:00
Ivo De Decker
97c3b32f1f autpkgtest: ignore deps already satisfied in testing 2020-02-07 19:36:40 +00:00
Ivo De Decker
976accaf57 Track uninstallable arch:all packages on non-nobreakall archs
When an arch:all binary is uninstallable on a non-nobreakall arch, don't
block it, but still mark it as uninstallable, so the autopkgtest policy
knows not to schedule tests for it.
2020-01-30 18:41:39 +01:00
Paul Gevers
a78b26d55d Don't block autopkgtest on uninstallable archs
Currently when a package is uninstallable on an arch, no autopkgtests for that arch are triggered
and the autopkgtest policy blocks migration. However it's not the job of the autopkgtest policy
to judge uninstallability and packages that build an arch:all package that just isn't installable
on the autopkgtest arch should not be blocked for this.

Closes: #918620
2020-01-30 18:41:39 +01:00
Ivo De Decker
4e5926a41f Add depends policy 2020-01-30 15:09:12 +00:00
Ivo De Decker
8b6638c566 Specify dependencies between excuses based on packages
Rework the dependencies between excuses.

Dependencies are specified based on packages (source or binary). Based
on these packages, dependencies on other excuses (that have these
packages) are calculated.

As with package dependencies, dependencies between excuses can contain
alternatives.

Each alternative can satisfy the dependency, so the excuse only becomes
invalid if all of the alternatives of a specific dependency are
invalidated.

Tracking of the alternatives and their validity is moved to separate
objects.
2020-01-30 15:09:12 +00:00
Paul Gevers
177b65d674
autopkgtest: also test for autodep8 for migration runs 2019-11-21 11:40:48 +01:00
Paul Gevers
b380508635
Drop dkms auto-detection for autodep8 as it isn't opt-in
Because autopkgtest failure in a package is now a serious bug (RC), packages
that don't opt-in should not be tested. Due to the way autopkgtest and autodep8
work together, package that don't declare they have an autopkgtest can still be
tested. However, maintainers should not be force to say their package doesn't
work with autodep8 by introducing a fake test.
2019-11-18 20:48:49 +01:00
Ivo De Decker
628b93b791 Make adding a failing test a regression
When there is no test in testing, and a failing test in unstable, don't
allow the package to migrate. Doing so would make it instantly RC-buggy.

Signed-off-by: Ivo De Decker <ivodd@debian.org>
2019-11-14 11:12:38 +00:00
Ivo De Decker
4de569114a Fix excuses output for package with urgent hint
Signed-off-by: Ivo De Decker <ivodd@debian.org>
2019-09-04 19:51:34 +00:00