63 Commits

Author SHA1 Message Date
Iain Lane
0329fb28e2
autopkgtest: When we're missing builds on non-adt arches only, go ahead and run the tests
Again the rest of britney will take care of invalidating the excuse if
it needs to.
2020-04-16 15:27:04 +01:00
Iain Lane
f63ec075f4
Drop ppc64el from results for uninstallable test, not needed 2020-04-16 15:27:03 +01:00
Iain Lane
865ac5d5f1
autopkgtest: If there are unsatisfied deps on non-adt arches, still request the test
We still want to be blocked by other parts of britney, but we can run
the tests to see their results now.
2020-04-16 15:27:03 +01:00
Iain Lane
eee029b353
tests: Allow the same package to be added on different arches
So we can make uninstallable packages on some arches only
2020-04-16 15:27:03 +01:00
Iain Lane
da67cc10b9
autopkgtest: Don't block packages in BREAK_ARCHES
We currently reject packages which have *any* unsatisfiable
dependencies, which is more strict than britney itself.
2020-04-16 15:27:03 +01:00
Iain Lane
45cc7b4366
force-reset-test tests: Add a test for reset-test with a different trigger 2020-02-06 11:35:16 +00:00
Iain Lane
70ede93f64
force-reset-test: Make the hint apply 'backwards' too
If we're force-reset-test foo/N, we should consider all versions up to
and including N as ALWAYSFAILED.
2020-02-06 11:35:16 +00:00
Iain Lane
9d541b4f5c
bikeshedding: call it force-reset-test 2020-02-06 11:35:16 +00:00
Julian Andres Klode
1f95e65860
Allow marking a version as accepted regression
LP: #1688516
2020-02-06 11:35:16 +00:00
Steve Langasek
2163d2be80 test case for not running tests on dropped arch 2020-01-20 21:37:51 -08:00
Iain Lane
177c7a69f7
test_autopkgtest: Strip submit-time
Testsuite regression from 4c27eadffa01bc1c084e947a7fa1e8a7509d1653.

submit-time is the time the request was submitted to amqp. For tests
it's the time the testcase was run. Many of our testcases test the exact
parameters that were passed to amqp, so they now fail because
submit-time is in there but it's not being checked for. It would be
possible to either verify the submit-time too, or switch to fuzzy
matching - but it's actually not very valuable to test submit-time,
since it is only for human readers of autopkgtest-cloud log files or
autopkgtest.ubuntu.com/running.

Instead strip the field from the amqp parameters before any tests see
them.
2020-01-16 10:13:41 +00:00
Łukasz 'sil2100' Zemczak
32467742bd SRU ADT regression: use the right policy_info to determine if tests have regressed. 2019-08-28 18:29:45 +02:00
Łukasz 'sil2100' Zemczak
e51cc58a2b Switch logic of detecting failures to looking for failed tests in policy_info. Explicitly list all failing tests in the bug comment. Change wording. 2019-07-19 15:43:54 +02:00
Łukasz 'sil2100' Zemczak
1cbc21d258 SRU ADT regression: Fix state checking as now it again needs to include the info for distro and series. Add unit test. 2019-05-24 16:18:34 +02:00
Łukasz 'sil2100' Zemczak
49e3d7e51c SRU ADT regression: Remove the no-longer-needed state-file backup functionality as currently there's basically low risk that we'd crash mid-state-save. Add a test for this. 2019-05-23 19:08:29 +02:00
Iain Lane
c8492d4cc7
SRU ADT regression: Need to decode the returned bytes from urlopen() 2019-05-23 10:47:59 +01:00
Iain Lane
73dabf2136
SRU ADT tests: Fix tests after the previous changes
This is the problem when implementing to a mock...
2019-05-23 10:38:58 +01:00
Iain Lane
bb032d98d8
SRU ADT: Be clear when printing log messages in dry-run mode 2019-05-22 17:47:29 +01:00
Iain Lane
c1f8ab2f9c
SRU ADT tests: Be more idiomatic
Use @patch decorator instead of ExitStack context manager. This is more
consistent with the rest of the testsuite we have in britney.

Use some more idiomatic assertions, e.g.
unittest.mock.assert_not_called().
2019-05-22 17:47:18 +01:00
Łukasz 'sil2100' Zemczak
1690624d11
Add a new policy to message bugs on SRU regressions
It works like this. We wait until all tests have finished running. and
then grab their results. If there are any regressions, we mail each bug
with a link to pending-sru.html. There's a state file which records the
mails we've sent out, so that we don't mail the same bug multiple times.
2019-05-22 17:47:17 +01:00
Iain Lane
c12d92efa4
autopkgtest: Handle 'blacklisted' version
The apt version comparison sorts 'blacklisted' greater than most version
numbers, which means that we accidentally apply force hints for version
'blacklisted' to all uploads. Since this is the only case of a hacked
version number, let's special case it so that 'blacklisted' hints only
match packages with 'blacklisted' version.
2019-01-24 13:08:03 +00:00
Steve Langasek
bf26a09948 Fix autopkgtests wrt the fact that excuses is no longer in the yaml 2018-09-20 13:34:18 +00:00
Iain Lane
f2456289ac autopkgtest: Run gcc-n's testuite, if it has one 2018-09-06 15:30:12 +01:00
Iain Lane
3e1ee5a2ef sourceppa: Add some more logging so we can hopefully see what is going on 2018-07-31 17:42:27 +01:00
Mathieu Trudel-Lapierre
88b4d023ad Use the CSafeLoader YAML loader to catch issues with complex objects 2018-03-08 13:30:00 +01:00
Mathieu Trudel-Lapierre
f32907acea Fix review issues. 2018-03-08 11:11:44 +01:00
Mathieu Trudel-Lapierre
80bf9060de Re-add 'unsatisfiable dependencies' information which dropped with the excuses key. 2018-03-06 17:32:13 +01:00
Iain Lane
400a6c57c4 email: When there are multiple SPPHs, use the last one
We were just seeing an interesting crash in britney. It was trying to
look up the gpg_keys of ~python-modules-team, which is a suspended
account - in LP API terms that's a HTTP error 410.

  https://api.launchpad.net/devel/~python-modules-team/gpg_keys

The direct error was fixed in commit 272f41c, but we actually should
*not* have been trying to look up this team's email address in the first
place. This upload was an auto-sync and so should not cause email to be
sent. The problem is that it was synced into universe and then promoted
into main. We were looking at the SPPH for after the promotion, which
has different values in the various signer/creator/sponsor/... fields,
and that made us think that it was a regular upload to email about.

Fix this by always looking at the oldest SPPH which should correspond to
the initial upload and not whatever happened to it afterwards.
2017-11-09 13:04:33 +00:00
Steve Langasek
132ffaf465 Refine the logic for recording the nominal sent age of the mail
In the previous iteration, if we were ever down/frozen/disabled long enough
to miss sending two mails in a row, we would see unintended "catch-up"
behavior where each subsequent run of britney would send a mail until the
right total number of mails had been sent.  Don't do this; instead, catch us
up in one go to the most recent mail that should have been sent, avoiding
bunching of notifications.

This changes one of the tests also to match.
2017-07-06 21:14:19 -07:00
Steve Langasek
f680bb3df8 add a couple more tests to show how the email frequency varies with valid/invalid candidates 2017-06-29 12:53:05 -07:00
Steve Langasek
6459977822 Make it possible to test the email frequency for a package that is sometimes a valid candidate and sometimes not 2017-06-29 12:53:05 -07:00
Steve Langasek
64d9bdda91 Adjust the email tests to the intervals we expect 2017-06-29 11:17:04 -07:00
Robert Bruce Park
9c76c762cc Tweak recurring email frequency. 2017-04-06 22:30:30 -07:00
Robert Bruce Park
16cca9c55d Change SourcePPAPolicy from a blacklist to a whitelist. 2017-03-24 11:46:52 -07:00
Iain Lane
858775a3c1 email: Make REJECTED_TEMPORARILY packages be not emailed for
These are rejections that we expect to clear out on their own or convert
to REJECTED_PERMANENTLY, at which point we will email.

https://bugs.launchpad.net/britney/+bug/1671468
2017-03-23 11:10:14 +00:00
Iain Lane
e00d86ad26 email: If a package is blocked by an explicit block, don't mail 2017-03-22 17:00:38 +00:00
Iain Lane
8350694348 email: Add tests that send email through a mocked SMTP server
I want to fix two bugs in interactions between other parts of britney
and the email policy. It's not currently easy to do so because we just
run the policy itself manually by creating some fake excuses.

Steal part of the machinery from the autopkgtest tests, and run a few
tests through britney completely. Use a fake SMTP server to record which
emails we sent.

(The port is hardcoded - that might not be so smart.)
2017-03-22 16:59:50 +00:00
Robert Bruce Park
63573fa24a Resend emails periodically. 2017-03-22 09:50:50 -07:00
Iain Lane
322108df73 email: beef up the tests very slightly
- add a day/days case for '1' exactly
  - make some other daysold be non ints
2017-03-14 18:17:03 +00:00
Robert Bruce Park
0579677457 Stop writing "1 days" in emails. 2017-03-14 11:05:10 -07:00
Steve Langasek
442993d3f0 Merge remote-tracking branch 'robru/fix-sourceppa' 2017-03-09 14:59:40 -08:00
Iain Lane
7cc72bf2d6 email: For direct sponsored uploads, email both the sponsor and sponsoree 2017-03-09 17:16:21 +00:00
Robert Bruce Park
df495582e2 Fix SourcePPA policy by inspecting build_link instead. (LP: #1648000) 2017-03-09 09:06:25 -08:00
Robert Bruce Park
2775a5435c Email Policy, send emails when packages are rejected. 2017-03-09 09:53:17 +00:00
Iain Lane
fe627aaa15 autopkgtest: Don't request tests for new packages that FTBFS
These are treated separately by britney ('no-binaries' vs.
'missing-builds').
2017-03-02 10:28:22 +00:00
Iain Lane
d366288a7b test_autopkgtest: Remove Testsuite-Triggers
This way we are testing the synthesisted test-dep explicitly
2017-02-24 16:07:39 +00:00
Andy Whitcroft
e16195ef46 tests/test_autopkgtest.py: fix test package
Signed-off-by: Andy Whitcroft <apw@ubuntu.com>
2017-02-24 15:32:44 +00:00
Andy Whitcroft
71c8773b7d linux-meta: add synthetic dependancy for snapd
Signed-off-by: Andy Whitcroft <apw@canonical.com>
2017-02-24 14:27:43 +00:00
Iain Lane
766ed38d40 sourceppa: Refactor exception handling
Use a for loop for retrying instead of recursion, so we don't get nested
exceptions being thrown. Add a test for this flakiness.
2017-02-08 13:17:16 +00:00
Iain Lane
9661362880 sourceppa: Retry 5 times on HTTP 503s
We get these quite frequently from the LP API - have a few attempts.
2017-02-07 17:26:03 +00:00