64 Commits

Author SHA1 Message Date
Paul Gevers
b4370ada0b tests: add autopkgtest policy unittest 2018-11-25 11:41:23 +00:00
Niels Thykier
07a407e810
Partly separate the solver from the inst_tester
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-11-11 09:38:14 +00:00
Niels Thykier
35b06ee007
tests: Separate inst_tester from universe
This commit updates the test suite to use the BinaryPackageUniverse
instead of the InstallabilityTester where that makes sense.  The rest
of Britney has yet to be updated except where absolutely necessary (as
that will come in a later commit).

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-11-11 09:38:11 +00:00
Paul Gevers
32ea0ccb70
tests: naming of variables in test_policy:apply_policy is highly confusing 2018-11-01 15:22:02 +01:00
Niels Thykier
edfe40d69c
Fix and re-add a test
Thanks: Paul Gevers <elbrus@debian.org>
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-10-03 05:35:48 +00:00
Niels Thykier
712b618ee8
tests: Comment out test requiring "do_test" function
It does not exist in the Debian version (yet?).

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-10-01 19:48:07 +00:00
Iain Lane
c09a229dc9
autopkgtest: Run gcc-n's testuite, if it has one 2018-10-01 19:22:29 +00:00
Paul Gevers
18633e275c
autopkgtest: figure out which packages from unstable are needed
Currently autopkgtest tries to install our trigger from unstable and the rest
from testing. If that fails, than autopkgtest has a fall-back to allow all
packages from unstable to be installed. This has two severe issues:

1) the version of the test and the package it came from may be out-of-sync

2) too much from unstable may be installed, even stuff that should not/is not
allowed to migrate as it breaks stuff.

Make sure that test depends also get added to triggers if they are broken.
E.g. imagine the following scenario: trigger X changes (breaks) the output
generated by Y. Package Z has Y in the test dependencies and compares the
output in the autopkgtest. We want to have the opportunity that a new version
is automatically fixing the situation.

Two use cases are currently unsupported: needs-build (autopkgtest restriction)
and test dependencies generated by autodep8.
2018-09-05 21:37:23 +02:00
Paul Gevers
713b98ad6b
inst_tester: add is_pkg_in_testing 2018-09-05 21:27:41 +02:00
Niels Thykier
0446ead9e8 inst-builder: Replace RelationBuilder with a set_relations method
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-08-12 13:06:48 +00:00
Niels Thykier
ab6b2ef953 inst-builder: Refactor to bulk adding dependency clauses
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-08-12 12:47:11 +00:00
Niels Thykier
bc1d3afa38 inst-builder: Make add_breaks a bulk call
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-08-12 12:33:09 +00:00
Niels Thykier
1623828a48 SourcePackage: Add field for Build-Depends-Indep
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-08-08 16:18:02 +00:00
Mathieu Trudel-Lapierre
8d2cabd914 Add 'unsatisifiable dependencies' to YAML excuses
Cherry-Pick: 80bf9060de6b7a7fe21f10da90f0dcc15b96ac1c
Cherry-Pick: f32907aceab7bdf37856ceef3122149c046f0ee1
Cherry-Pick: 9ef496177f86b18d9f910da1360dd773b82f1fb7
Cherry-Pick: b16530a37df93851fda35db016775056a6b0c218
Signed-off-by: Niels Thykier <niels@thykier.net>

Flatten the defaultdict(set) for unsat_deps into a standard dict for output
2018-08-07 18:55:57 +00:00
Niels Thykier
36608194f8 test_policy: Add additional tests of AgePolicy
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-08-04 20:09:03 +00:00
Niels Thykier
b0f0dde8f5 test_policy: Properly add the fake source package to the Suite.sources field
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-08-04 20:09:03 +00:00
Niels Thykier
9a7394c89d britney: Rewrite should_* to be (almost) suite agnostic
The "almost" comes from the fact that there is a reference to
"pu"/"stable" which does not have an obvious fix at the moment.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-07-29 09:23:12 +00:00
Niels Thykier
d1350606bb Use a proper Suite object for MigrationItem.suite
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-07-29 09:23:12 +00:00
Niels Thykier
8768e2a02a Define a SuiteClass to classify suites
Into 3 categories:

 * target suite ("testing")
 * primary source suite ("unstable")
 * additional source suites ("pu" and "tpu")

This will be useful for implementing logic working with suites without
basing it on the name of the suite.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-07-29 09:23:12 +00:00
Niels Thykier
b63ade583c Rename and turn SuiteInfo into a proper class
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-07-29 09:23:12 +00:00
Niels Thykier
e63aa05708 Add Suites to keep track of all suites
At the moment, it is just a glorified dict.  However, we will
eventually use it to get rid of the hardcoded references to "testing"
etc. all over the code.

Signed-off-by: Niels Thykier <niels@thykier.net>
2018-07-29 09:23:12 +00:00
Niels Thykier
37f02d611c Revert "Rename autopkgtest related hints"
This reverts commit a07fa9f9070fc9c328021db5cc3e4c4f3063798e.

The new names are just as unhelpful and confusing as the old ones.
Undo for now.
2018-07-25 18:17:36 +00:00
Niels Thykier
a07fa9f907 Rename autopkgtest related hints
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-07-25 16:51:03 +00:00
Paul Gevers
624b185ba6
autopkgtest: enable NEUTRAL state for the case where all tests were skipped (or none available)
Closes: #901847
2018-07-21 13:21:39 +02:00
Paul Gevers
7e8ec20bc1
Put autopkgtest state files in their own namespace 2018-04-16 20:52:24 +02:00
Paul Gevers
9a28ec184b
Revert "autopkgtest: Drop retry_url from the excuses as this works different in Debian"
This reverts commit 054830d03f0a04c0306e5e71752646c2f2de2626.
2018-04-11 11:17:00 +02:00
Paul Gevers
cb716e3186
Don't excuse.force() on skiptest hint 2018-04-11 11:16:59 +02:00
Paul Gevers
6252826fad
autopkgtest: add adt_baseline = reference option
- revert most of commit adbe6d5 as checking the version in testing doesn't work
  when other packages migrate and cause regressions

- Alternative way of determining if a package is regressing, by comparison to a
  reference set. The reference set is to be created by a holy trigger that
  doesn't take packages from the base suite, but instead tests in the testing
  suite. This reference needs a retry when a package causing regression
  migrates nevertheless, e.g. due to hints or to bounty/penalty policy.
2018-04-11 11:16:59 +02:00
Niels Thykier
f752ea4ac7 Merge branch 'master' into autopkgtest 2018-03-31 05:35:06 +00:00
Niels Thykier
ac69b3b5c7 Migrate hint parser to fully use logging
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-03-30 17:11:10 +00:00
Niels Thykier
0c1147e4e1 test: Fix typo
Signed-off-by: Niels Thykier <niels@thykier.net>
2018-03-24 06:38:21 +00:00
Paul Gevers
054830d03f
autopkgtest: Drop retry_url from the excuses as this works different in Debian 2018-03-15 11:38:47 +01:00
Paul Gevers
d6f713b8a1
tests: Update for recent changes 2018-03-05 21:09:59 +01:00
Paul Gevers
fd03083ea5
Merge branch 'master' into autopkgtest 2017-11-28 20:19:48 +01:00
Niels Thykier
b5319e33fa tests: Explicitly define encoding for excuses
Signed-off-by: Niels Thykier <niels@thykier.net>
2017-11-28 18:53:49 +00:00
Niels Thykier
ee27d7a67c Add BuildDependsPolicy to check Build-Depends(-Arch) availability
Add a new "BuildDependsPolicy" that will check the satisfiability of
the build-dependencies listed in the Build-Depends and
Build-Depends-Arch fields.  This enables gating of packages based on
missing / broken build-dependencies.

There are some limitations:

 * Build-Depends-Indep is ignored for now.  Missing or broken packages
   listed in Build-Depends-Indep will be continue to be silently
   ignored.

 * Being a policy check, it does not enforce "self-containedness" as
   a package can still migrate before a build-dependency.  However,
   this can only happen if the build-dependency is ready to migrate
   itself.  If the build-dependency is not ready (e.g. new RC bugs),
   then packages build-depending on it cannot migrate either (unless
   the version in testing satisfies there requirements).

Signed-off-by: Niels Thykier <niels@thykier.net>
2017-11-11 15:49:48 +00:00
Paul Gevers
597eac6130
Adapt test_autopkgtest.py so nosetests3 picks it up correctly 2017-10-25 20:14:05 +02:00
Paul Gevers
a16e4e5a55
Enable autopkgtesting on built arches when not all have been built yet
- autopkgtest now honors break_arches option
- incomplete testing is now treated with penalty behavior
2017-10-24 20:46:21 +02:00
Paul Gevers
d975b2fc39
Make the huge queue in autopkgtest optional 2017-10-19 21:50:56 +02:00
Paul Gevers
f2f20eb460
Limit accumulated bounties to configurable minum age 2017-10-19 21:50:55 +02:00
Paul Gevers
77ea1cc8d5
Enable no-penalties urgencies; to exempt urgency >= high from penalties 2017-10-19 21:50:43 +02:00
Paul Gevers
287a9d328c
os.path.join(options.unstable, 'autopkgtest') isn't writable in the Debian setup and we have options.state_dir already 2017-10-18 19:54:57 +02:00
Paul Gevers
eb60fc557e
Implement bounty/penalty system for autopkgtest 2017-10-15 21:42:51 +02:00
Paul Gevers
acec564443
Make autopkgtest web site URL an option 2017-10-14 20:15:47 +02:00
Paul Gevers
e5756dec9e
Fix existing tests for autopkgtest extention of SourcePackage class 2017-09-26 19:45:23 +02:00
Paul Gevers
648feb71d3
Implement swift free usage of autopkgtest 2017-09-25 21:25:10 +02:00
Paul Gevers
278b3dcdd0
autopkgtest: switch from testing=series to testing=testing 2017-09-08 11:08:22 +02:00
Paul Gevers
5ae8ccbcc6
Get the autopkgtest test suite to run with Debian's britney
* test_autopkgtest: use --no-compute-migrations instead of removing UPGRADE_OUTPUT
* comment out some tests that I can't get to work in Debian, while all but three work if
  run inside Ubuntu's britney
2017-09-08 11:07:04 +02:00
Paul Gevers
2bce9e334e
Initial commit for autokpgtest
* copy of Ubuntu's version at 593acb2753ce167ed4d2dec0091702315e89e705
* merge tests/__init__.py with Ubuntu's (that only contained autopkgtest)
2017-09-08 10:25:16 +02:00
Niels Thykier
6e022eed54 test: Add test for a bug I was about to introduce
When I rewrote the compute_scc function into an iterative variant, I
almost included a bug that could make it come up with components that
were not strongly-connected.

Signed-off-by: Niels Thykier <niels@thykier.net>
2017-07-29 08:05:52 +00:00