60 Commits

Author SHA1 Message Date
Martin Pitt
335073e901 Add requesting tests through AMQP
Until now, autopkgtests were triggered via an external "adt-britney" command
from lp:auto-package-testing. This duplicated a lot of effort (such as its own
chdist and reverse dependency calculation), maintains a lot of state files,
uses hardcoded absolute paths to these external tools, and is quite hard to
understand and maintain. We want to get rid of all this.

Add logic to AutoPackageTest.request() to use britney's existing reverse
dependency maps and figure out the set of tests to run for packages in
unstable. These are being tracked in "requested_tests".

Add logic to AutoPackageTest.submit() to send test requests to the AMQP server
specified in the new "ADT_AMQP" config key. For testing this can be a file://
URL, and if not set no test requests will be sent at all.

The set of tests which were requested in previous runs are tracked in
UNSTABLE/autopkgtest/pending.txt, so that we don't re-request tests in
subsequent runs.

There is no implementation for collect() and results() yet, these will be done
in a separate commit.

Add initial test cases.
2015-07-02 17:41:49 +02:00
Martin Pitt
65a1d73164 tests: Don't touch original britney.conf
Don't change britney.conf from the source tree and restore it back after the
tests; this is prone to leave a broken config behind if a test gets
interrupted. Instead, copy the file to our temp dir, and let tests hack on it
there.

(Introduced in lp:~canonical-ci-engineering/britney/boottesting-support)
2015-07-02 16:24:53 +02:00
Martin Pitt
add46d2904 Store autopkgtest flag in sources
Extend read_sources to store a new AUTOPKGTEST boolean flag, which is true if
the Testsuite: field exists and starts with "autopkgtest" (this covers autodep8
cases like autopkgtest-pkg-perl).

Extend TestData.add() to take a new testsuite argument which specifies the
source's Testsuite: field.
2015-07-01 15:49:06 +02:00
Francis Ginther
ca1fc41f9e Refactor the __fetch_manifest() code so that the a successful response is immediately handled and we don't end up with an undefined 'response' on an exception. Also update the tests to return two mocked reponses to match the size of the url_list. 2015-06-05 11:08:48 -05:00
Celso Providelo
ea6493ae71 Only enable BOOTTEST for testing, so britney.conf is safe on trunk. 2015-02-20 14:28:47 -02:00
Celso Providelo
c76874fda5 Test ADT & BootTest criteria running simultaneously. Fixed minor issue related to doubling-blocking promotions. 2015-02-20 14:12:51 -02:00
Celso Providelo
341ec4a3e0 Typo fixed 2015-02-11 10:41:43 -05:00
Celso Providelo
bf8056a061 Cope with missing/broken results due to outdated apt cache. Britney will not fail, but source promotion will be blocked. 2015-02-11 09:45:20 -05:00
Celso Providelo
496a2f3134 Suppress boottest SKIPPED notice on excuses, it's unnecessary noise. 2015-02-10 15:56:35 -05:00
Celso Providelo
b9890e1b7c Adding links to the boottest jobs (results) in the public and the private servers. 2015-02-10 14:51:44 -05:00
Celso Providelo
011aa3570e Stop announcing skipped boottest attempts. 2015-02-10 13:49:43 -05:00
Celso Providelo
a80a1e4f1c Boottest integration adjustments. 2015-02-06 11:43:38 -05:00
Celso Providelo
c3c9005d5f Restore boottest status lookup by source name & version. 2015-02-06 10:38:36 -05:00
Joe Talbott
6269f50abc Don't bother creating another mocked urllib.urlopen. 2015-02-05 17:27:39 -05:00
Celso Providelo
760d08c459 Fix BootTest.get_status to return status for the latest known version. Boottests results are reported for the current (published) version, not the proposed one. 2015-02-05 16:42:29 -05:00
Joe Talbott
f0a55936f5 boottest - Add exception message to warning print and fix typo 2015-02-05 11:20:28 -05:00
Joe Talbott
87ccbfd731 boottest - Add exception handling for manifest fetching 2015-02-05 11:08:53 -05:00
Celso Providelo
72b01f8cc7 Using addCleanup() for restoring original configuration contents in tests. 2015-02-05 09:43:23 -05:00
Celso Providelo
6e8e2adc35 Fix autopkgtest tests, extending the current test setup to mangle the shared configuration file appropriately for each test context (autopkgtest & boottest). 2015-02-05 09:17:54 -05:00
Celso Providelo
df62c7fcf7 solve '~' expansion for tests. 2015-02-05 08:15:07 -05:00
Celso Providelo
8ffc7049b5 Test fixing (proposal) 2015-02-05 08:04:39 -05:00
Vincent Ladeuil
01fdb16995 According to cjwatson, cdimage deals with projects, not distribution, fix TouchManifest accordingly.
Fix boottest-britney location to match production.
Fix TestTouchManifest test failures, now that we retry on manifest download errors, the tests should inhibit the retries when testing the failures.

5 out of the 8 TestBoottestEnd2End are still failing because the excuse says the tests are skipped instead of running/failing/being in progress, etc.
2015-02-05 10:42:12 +01:00
Celso Providelo
82b1c1cd83 Adjusting boottest-jenkins arguments to match reality. 2015-02-04 12:42:03 -05:00
Celso Providelo
436dadae0a Cleanup the boottest-britney testing script to reflect better its real behavior. 2015-02-04 11:15:08 -05:00
Celso Providelo
1e8dc398e9 Merge the boottest-jenkins script wrapper into BootTest class for simplicity and extend the API to match what we already for auto-package-testing (ADT). 2015-02-04 08:54:00 -05:00
Celso Providelo
06ea2ab941 Request boottests for sources, not binaries. Jenkins glue will drive adt properly. 2015-02-03 12:27:22 -05:00
Celso Providelo
49f0a5b3ff Wraps 'boottest-britney' script for checking status or submitting new jobs. 2015-02-02 17:04:24 -05:00
Celso Providelo
37bd9924f7 Introduce BOOTTEST_FETCH configuration option for enabling/disabling TouchManifest auto-fetching feature, for faster and isolated tests. 2015-02-02 13:10:53 -05:00
Joe Talbott
056a0f6d37 Handle boottest like autopkgtest and remove boottest config file 2015-01-30 17:51:44 -05:00
Joe Talbott
2e5084ee79 boottest - Fetch manifests dynamically but use cached manifests if the fetch fails 2015-01-30 16:41:20 -05:00
Celso Providelo
913539ffe9 Allow hints for ignoring boottest results (force and force-skiptest). 2015-01-23 10:17:10 -02:00
Celso Providelo
eeaf8e3deb Silently skipping boottesting for sources not yet built on the allowed architectures (all and armhf). 2015-01-22 21:30:03 -02:00
Celso Providelo
ed31f1129b Cosmetic changes for adding excuse labels (html) and more readable image manifest path. 2015-01-22 16:44:18 -02:00
Celso Providelo
cc9097e2ed Check phone-image presence for all binary files for the give excused sourcename. Tests improved. 2015-01-21 14:59:12 -02:00
Celso Providelo
599fda8505 Basic (and ugly) phone image manifest wrapper. 2015-01-21 12:37:43 -02:00
Celso Providelo
f7e1fa67c0 Implementing boottest entry points for checking phone-image presence and test current status. 2015-01-21 10:25:40 -02:00
Celso Providelo
fe5f4c09ac Very basic bootstack testing criteria support. 2015-01-19 17:59:19 -02:00
Celso Providelo
46281510e3 Refactoring the existing test (autopkgtest), so its features can be re-used for other criterias (boottest) tests. 2015-01-19 17:16:46 -02:00
Colin Watson
5d0c4a6b1e Make test_autopkgtest pass again 2014-12-10 10:42:27 +00:00
Colin Watson
a6fbf59cbc merge lp:~jibel/britney/fix_missing_results 2014-05-12 15:40:45 +01:00
Jean-Baptiste Lallement
890b82aaef Merged trunk 2014-05-12 15:01:36 +02:00
Martin Pitt
9b4eeafef2 Mark test_result_from_older_version as XFAIL 2014-05-12 14:04:55 +02:00
Jean-Baptiste Lallement
1052dea946 * tests/test_autopkgtest.py:
- Added docstrings
- Replaced format by %
- Import ADT_EXCUSES_LABELS from autopkgtest.py instead of redefining it
- Renamed test to avoid name conflict with autopkgtest.py from britney
- Fixed some formatting
2014-05-12 11:12:24 +02:00
Jean-Baptiste Lallement
4382d5c1e0 tests/autopkgtest.py: Removed debugging code 2014-05-06 16:55:57 +02:00
Jean-Baptiste Lallement
d490ca6a87 tests/autopkgtest.py: Added new tests when results already exist in history 2014-05-06 16:55:16 +02:00
Jean-Baptiste Lallement
1f09f02570 tests/autopkgtest.py: Use the right key for sorting history 2014-05-06 09:10:53 +02:00
Jean-Baptiste Lallement
a190cedd55 tests/autopkgtest.py: shell test with history 2014-05-05 18:19:48 +02:00
Jean-Baptiste Lallement
acd4de7868 tests/autopkgtest.py: Fixed testsuite and added tests to match new labels and rules for regression/always failing tests 2014-05-05 18:08:40 +02:00
Martin Pitt
0cd14d1394 fix source package name in cause of new_source tests 2014-03-17 09:35:20 +01:00
Martin Pitt
2b7e93515f add test for uninstallable binary built from new source package 2014-03-17 08:43:25 +01:00