13 Commits

Author SHA1 Message Date
Martin Pitt
ce775eeb5d Add test results from swift
Until now, autopkgtest results were triggered via an external "adt-britney"
command from lp:auto-package-testing. This required a lot of state files and
duplicated effort, uses hardcoded absolute paths to these external tools, and
is quite hard to understand and maintain. We also want to move away from
Jenkins and rsyncing state files.

Directly retrieve autopkgtest results from a publicly readable and browsable
Swift container, with a debci-compatible layout
(https://wiki.debian.org/debci/DistributedSpec). This now tracks both requests
and results on a per-architecture granularity, so that we can track
per-architecture regressions/always-failed.

Introduce a new ADT_SWIFT_URL config option that sets the swift base URL. If
this key is not set, the behaviour does not change compared to previous
versions, and no results will be retrieved from the cloud.

This still keeps the old adt-britney requests/results as the authoritative
data and for now merely shows the swift results in addition. With that we can
compare the results and run the cloud testing in parallel to find/fix problems
until we switch over. Due to that, the code to britney.py is temporary, does
*not* use AutoPackageTest.results(), and instead just reads the internal
results map.
2015-07-10 06:21:46 +02:00
Martin Pitt
6e167a0343 Track architectures in requested/pending tests
This is necessary so that we can properly match requested to received results
when the latter arrive in different runs for different architectures.

This also opens up the possibility of per-arch blacklisting later.
2015-07-07 11:59:07 +02:00
Martin Pitt
bf470c6da0 Use current reverse dep version instead of None/-
This makes tracking test results easier and removes some special cases.
2015-07-07 11:11:44 +02:00
Martin Pitt
ded6912443 Add autopkgtest test case for a package pair 2015-07-07 09:10:58 +02:00
Martin Pitt
335073e901 Add requesting tests through AMQP
Until now, autopkgtests were triggered via an external "adt-britney" command
from lp:auto-package-testing. This duplicated a lot of effort (such as its own
chdist and reverse dependency calculation), maintains a lot of state files,
uses hardcoded absolute paths to these external tools, and is quite hard to
understand and maintain. We want to get rid of all this.

Add logic to AutoPackageTest.request() to use britney's existing reverse
dependency maps and figure out the set of tests to run for packages in
unstable. These are being tracked in "requested_tests".

Add logic to AutoPackageTest.submit() to send test requests to the AMQP server
specified in the new "ADT_AMQP" config key. For testing this can be a file://
URL, and if not set no test requests will be sent at all.

The set of tests which were requested in previous runs are tracked in
UNSTABLE/autopkgtest/pending.txt, so that we don't re-request tests in
subsequent runs.

There is no implementation for collect() and results() yet, these will be done
in a separate commit.

Add initial test cases.
2015-07-02 17:41:49 +02:00
Martin Pitt
65a1d73164 tests: Don't touch original britney.conf
Don't change britney.conf from the source tree and restore it back after the
tests; this is prone to leave a broken config behind if a test gets
interrupted. Instead, copy the file to our temp dir, and let tests hack on it
there.

(Introduced in lp:~canonical-ci-engineering/britney/boottesting-support)
2015-07-02 16:24:53 +02:00
Celso Providelo
72b01f8cc7 Using addCleanup() for restoring original configuration contents in tests. 2015-02-05 09:43:23 -05:00
Celso Providelo
6e8e2adc35 Fix autopkgtest tests, extending the current test setup to mangle the shared configuration file appropriately for each test context (autopkgtest & boottest). 2015-02-05 09:17:54 -05:00
Celso Providelo
46281510e3 Refactoring the existing test (autopkgtest), so its features can be re-used for other criterias (boottest) tests. 2015-01-19 17:16:46 -02:00
Colin Watson
5d0c4a6b1e Make test_autopkgtest pass again 2014-12-10 10:42:27 +00:00
Colin Watson
a6fbf59cbc merge lp:~jibel/britney/fix_missing_results 2014-05-12 15:40:45 +01:00
Jean-Baptiste Lallement
890b82aaef Merged trunk 2014-05-12 15:01:36 +02:00
Jean-Baptiste Lallement
1052dea946 * tests/test_autopkgtest.py:
- Added docstrings
- Replaced format by %
- Import ADT_EXCUSES_LABELS from autopkgtest.py instead of redefining it
- Renamed test to avoid name conflict with autopkgtest.py from britney
- Fixed some formatting
2014-05-12 11:12:24 +02:00