47 Commits

Author SHA1 Message Date
Iain Lane
15f431d37c autopkgtest: Add new force-reset-test hint type
This allows hinters to reset the baseline for a package/arch, so that
previous failures are ignore and not considered regressions
2020-11-20 17:37:44 +00:00
Iain Lane
e5b7f5f539 autopkgtest: Support autopkgtest.u.c in-progress URL
This is at /running. We use 'are we running on swift?' as a proxy for
deciding which URL format to use
2020-11-20 17:37:44 +00:00
Iain Lane
af085da46e autopkgtest: Handle 'blacklisted' version
The apt version comparison sorts 'blacklisted' greater than most version
numbers, which means that we accidentally apply force hints for version
'blacklisted' to all uploads. Since this is the only case of a hacked
version number, let's special case it so that 'blacklisted' hints only
match packages with 'blacklisted' version.
2020-11-20 17:37:44 +00:00
Iain Lane
34c8b05010 autopkgtest: store the submission time when making requests
This is so that we can see more easily see if requests are stuck
2020-11-20 17:37:44 +00:00
Iain Lane
2623053f82 test_autopkgtest: Re-enable test for package not built on arch
This is a test carried over from Ubuntu which ensures that a package
which builds no binaries on an arch doesn't have tests requested.

It was disabled. Enable it.
2020-11-20 17:37:44 +00:00
Iain Lane
af091a504c autopkgtest: Trigger dkms when there's a dep on dkms
Most DKMS packages do not declare Testsuite: autopkgtest-pkg-dkms, but
we can detect this anyway, and this way we can enforce that the module
is buildable.
2020-11-20 17:37:44 +00:00
Iain Lane
bae4fdbd45 LinuxPolicy: Make linux* wait on corresponding -meta package
We want to treat linux-$flavor and linux-meta-$flavor as one set in
britney which goes in together or not at all. We never want to promote
linux-$flavor without the accompanying linux-meta-$flavor.

Add a new LinuxPolicy which runs after most of the other policies, which
invalidates linux-foo if linux-meta-foo is invalid.
2020-11-20 17:37:44 +00:00
Robert Bruce Park
36e8b65c8b Email Policy, send emails when packages are rejected. 2020-11-20 17:37:44 +00:00
Robert Bruce Park
9c777eac1b Reject packages if entire source ppa won't migrate 2020-11-11 09:41:40 +00:00
Iain Lane
d2dbdadf1f autopkgtest: Hardcode some linux-meta tests
We should really fix this to use Testsuite-Triggers / hint-testsuite-triggers
2020-11-11 09:41:40 +00:00
Iain Lane
7ed335893e Implement Ubuntu component relationship constraints (ogre model) 2020-11-11 09:41:40 +00:00
Colin Watson
4f9a33ddab Allow blocking uploads by way of bugs with the block-proposed tag set 2020-11-11 09:41:40 +00:00
Iain Lane
1487096a15 test_autopkgtest: Split setup commands into a separate class
This lets other policies share this code for their integration tests
2020-10-20 14:57:51 +01:00
Paul Gevers
aa8b84f4b8
Drop the special case that gcc has; it's just a package 2020-02-20 11:24:27 +01:00
Ivo De Decker
237c5d7357 fix test_autopkgtest tests 2020-01-30 18:41:39 +01:00
Paul Gevers
455d962ebd Update test_autopkgtest.py for previous commit 2020-01-30 18:41:39 +01:00
Paul Gevers
b380508635
Drop dkms auto-detection for autodep8 as it isn't opt-in
Because autopkgtest failure in a package is now a serious bug (RC), packages
that don't opt-in should not be tested. Due to the way autopkgtest and autodep8
work together, package that don't declare they have an autopkgtest can still be
tested. However, maintainers should not be force to say their package doesn't
work with autodep8 by introducing a fake test.
2019-11-18 20:48:49 +01:00
Ivo De Decker
f202601b81 fix codestyle issue in test_autopkgtest.py
This issue only shows up in the bullseye test.
2019-09-04 12:02:52 +00:00
Paul Gevers
747c67854a
tests: use yaml.safe_load as yaml.load is deprecated 2019-08-25 20:47:14 +02:00
Paul Gevers
964ecaf045
tests: add the suite to debci results 2019-05-18 21:40:26 +02:00
Paul Gevers
e8b5f5e631
Fix loads of pycodestyle issues in test_autopkgtest 2019-02-07 19:17:16 +01:00
Paul Gevers
42ac9c34a1
autopkgtest: add timestamp to results 2019-01-31 15:37:13 +01:00
Niels Thykier
c9d523da93
Remove trailing whitespace
Signed-off-by: Niels Thykier <niels@thykier.net>
2019-01-19 12:03:34 +00:00
Niels Thykier
edfe40d69c
Fix and re-add a test
Thanks: Paul Gevers <elbrus@debian.org>
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-10-03 05:35:48 +00:00
Niels Thykier
712b618ee8
tests: Comment out test requiring "do_test" function
It does not exist in the Debian version (yet?).

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-10-01 19:48:07 +00:00
Iain Lane
c09a229dc9
autopkgtest: Run gcc-n's testuite, if it has one 2018-10-01 19:22:29 +00:00
Paul Gevers
18633e275c
autopkgtest: figure out which packages from unstable are needed
Currently autopkgtest tries to install our trigger from unstable and the rest
from testing. If that fails, than autopkgtest has a fall-back to allow all
packages from unstable to be installed. This has two severe issues:

1) the version of the test and the package it came from may be out-of-sync

2) too much from unstable may be installed, even stuff that should not/is not
allowed to migrate as it breaks stuff.

Make sure that test depends also get added to triggers if they are broken.
E.g. imagine the following scenario: trigger X changes (breaks) the output
generated by Y. Package Z has Y in the test dependencies and compares the
output in the autopkgtest. We want to have the opportunity that a new version
is automatically fixing the situation.

Two use cases are currently unsupported: needs-build (autopkgtest restriction)
and test dependencies generated by autodep8.
2018-09-05 21:37:23 +02:00
Niels Thykier
9a7394c89d britney: Rewrite should_* to be (almost) suite agnostic
The "almost" comes from the fact that there is a reference to
"pu"/"stable" which does not have an obvious fix at the moment.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-07-29 09:23:12 +00:00
Niels Thykier
37f02d611c Revert "Rename autopkgtest related hints"
This reverts commit a07fa9f9070fc9c328021db5cc3e4c4f3063798e.

The new names are just as unhelpful and confusing as the old ones.
Undo for now.
2018-07-25 18:17:36 +00:00
Niels Thykier
a07fa9f907 Rename autopkgtest related hints
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-07-25 16:51:03 +00:00
Paul Gevers
624b185ba6
autopkgtest: enable NEUTRAL state for the case where all tests were skipped (or none available)
Closes: #901847
2018-07-21 13:21:39 +02:00
Paul Gevers
7e8ec20bc1
Put autopkgtest state files in their own namespace 2018-04-16 20:52:24 +02:00
Paul Gevers
9a28ec184b
Revert "autopkgtest: Drop retry_url from the excuses as this works different in Debian"
This reverts commit 054830d03f0a04c0306e5e71752646c2f2de2626.
2018-04-11 11:17:00 +02:00
Paul Gevers
cb716e3186
Don't excuse.force() on skiptest hint 2018-04-11 11:16:59 +02:00
Paul Gevers
054830d03f
autopkgtest: Drop retry_url from the excuses as this works different in Debian 2018-03-15 11:38:47 +01:00
Paul Gevers
d6f713b8a1
tests: Update for recent changes 2018-03-05 21:09:59 +01:00
Paul Gevers
597eac6130
Adapt test_autopkgtest.py so nosetests3 picks it up correctly 2017-10-25 20:14:05 +02:00
Paul Gevers
a16e4e5a55
Enable autopkgtesting on built arches when not all have been built yet
- autopkgtest now honors break_arches option
- incomplete testing is now treated with penalty behavior
2017-10-24 20:46:21 +02:00
Paul Gevers
f2f20eb460
Limit accumulated bounties to configurable minum age 2017-10-19 21:50:55 +02:00
Paul Gevers
77ea1cc8d5
Enable no-penalties urgencies; to exempt urgency >= high from penalties 2017-10-19 21:50:43 +02:00
Paul Gevers
287a9d328c
os.path.join(options.unstable, 'autopkgtest') isn't writable in the Debian setup and we have options.state_dir already 2017-10-18 19:54:57 +02:00
Paul Gevers
eb60fc557e
Implement bounty/penalty system for autopkgtest 2017-10-15 21:42:51 +02:00
Paul Gevers
acec564443
Make autopkgtest web site URL an option 2017-10-14 20:15:47 +02:00
Paul Gevers
648feb71d3
Implement swift free usage of autopkgtest 2017-09-25 21:25:10 +02:00
Paul Gevers
278b3dcdd0
autopkgtest: switch from testing=series to testing=testing 2017-09-08 11:08:22 +02:00
Paul Gevers
5ae8ccbcc6
Get the autopkgtest test suite to run with Debian's britney
* test_autopkgtest: use --no-compute-migrations instead of removing UPGRADE_OUTPUT
* comment out some tests that I can't get to work in Debian, while all but three work if
  run inside Ubuntu's britney
2017-09-08 11:07:04 +02:00
Paul Gevers
2bce9e334e
Initial commit for autokpgtest
* copy of Ubuntu's version at 593acb2753ce167ed4d2dec0091702315e89e705
* merge tests/__init__.py with Ubuntu's (that only contained autopkgtest)
2017-09-08 10:25:16 +02:00