849 Commits

Author SHA1 Message Date
Steve Langasek
b80b8c533a Fix wrong log message
Britney2's Hint object has no 'days' attribute, so we ought not try to
print it.

Seen in the wild for several hours during the Artful Alpha 1 milestone
freeze, when a duplicate block hint caused britney to fail to run due to
the resulting exception.
2017-06-27 23:07:44 -07:00
Steve Langasek
41ac7a81f9 Drop transitional code 2017-04-06 22:30:18 -07:00
Robert Bruce Park
16cca9c55d Change SourcePPAPolicy from a blacklist to a whitelist. 2017-03-24 11:46:52 -07:00
Iain Lane
858775a3c1 email: Make REJECTED_TEMPORARILY packages be not emailed for
These are rejections that we expect to clear out on their own or convert
to REJECTED_PERMANENTLY, at which point we will email.

https://bugs.launchpad.net/britney/+bug/1671468
2017-03-23 11:10:14 +00:00
Iain Lane
e00d86ad26 email: If a package is blocked by an explicit block, don't mail 2017-03-22 17:00:38 +00:00
Iain Lane
8350694348 email: Add tests that send email through a mocked SMTP server
I want to fix two bugs in interactions between other parts of britney
and the email policy. It's not currently easy to do so because we just
run the policy itself manually by creating some fake excuses.

Steal part of the machinery from the autopkgtest tests, and run a few
tests through britney completely. Use a fake SMTP server to record which
emails we sent.

(The port is hardcoded - that might not be so smart.)
2017-03-22 16:59:50 +00:00
Robert Bruce Park
63573fa24a Resend emails periodically. 2017-03-22 09:50:50 -07:00
Iain Lane
322108df73 email: beef up the tests very slightly
- add a day/days case for '1' exactly
  - make some other daysold be non ints
2017-03-14 18:17:03 +00:00
Robert Bruce Park
0579677457 Stop writing "1 days" in emails. 2017-03-14 11:05:10 -07:00
Robert Bruce Park
1b4828bfb3 Write email cache after every email sent. 2017-03-10 09:48:37 -08:00
Steve Langasek
442993d3f0 Merge remote-tracking branch 'robru/fix-sourceppa' 2017-03-09 14:59:40 -08:00
Robert Bruce Park
389f8624dd Round age. 2017-03-09 11:12:20 -08:00
Robert Bruce Park
e7181465a8 Stop returning None. 2017-03-09 11:04:12 -08:00
Iain Lane
9a32ef781f email: Enable again 2017-03-09 18:02:59 +00:00
Robert Bruce Park
2405fcf8bc Backport EmailPolicy to Python 3.2 2017-03-09 18:00:22 +00:00
Iain Lane
1fe3e56cc7 Revert "enable the email policy"
We're getting

  Traceback (most recent call last):
    File "/srv/ubuntu-archive/proposed-migration/code/b2/britney2/policies/email.py", line 171, in apply_policy_impl
      with smtplib.SMTP('localhost') as smtp:
  AttributeError: __exit__

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/ubuntu-archive/proposed-migration/code/b2/britney.py", line 2892, in <module>
      Britney().main()
    File "/home/ubuntu-archive/proposed-migration/code/b2/britney.py", line 2860, in main
      self.write_excuses()
    File "/home/ubuntu-archive/proposed-migration/code/b2/britney.py", line 1680, in write_excuses
      if should_upgrade_src(pkg, 'unstable'):
    File "/home/ubuntu-archive/proposed-migration/code/b2/britney.py", line 1585, in should_upgrade_src
      v = policy.apply_policy(policy_info, suite, src, source_t, source_u, excuse)
    File "/srv/ubuntu-archive/proposed-migration/code/b2/britney2/policies/policy.py", line 103, in apply_policy
      return self.apply_policy_impl(pinfo, suite, source_name, source_data_tdist, source_data_srcdist, excuse)
    File "/srv/ubuntu-archive/proposed-migration/code/b2/britney2/policies/email.py", line 176, in apply_policy_impl
      except ConnectionRefusedError as err:
  NameError: global name 'ConnectionRefusedError' is not defined

on the production machine, which is currently running Python 3.2. It seems like smtplib.SMTP isn't a context manager in 3.2, and also ConnectionRefusedError doesn't exist there (it raises socket.error instead).

Robert's going to fix this, but for now let's go back to dry-run.

This reverts commit c05b6871851143efe5e2f2388b52544ecf3f7a32.
2017-03-09 17:35:55 +00:00
Iain Lane
7cc72bf2d6 email: For direct sponsored uploads, email both the sponsor and sponsoree 2017-03-09 17:16:21 +00:00
Robert Bruce Park
df495582e2 Fix SourcePPA policy by inspecting build_link instead. (LP: #1648000) 2017-03-09 09:06:25 -08:00
Iain Lane
5f4c2e735d email: Log a message when we send emails 2017-03-09 16:46:47 +00:00
Iain Lane
c05b687185 enable the email policy
weee
2017-03-09 16:43:40 +00:00
Iain Lane
d9887d320e email: be From the 'Ubuntu Release Team' 2017-03-09 16:41:34 +00:00
Robert Bruce Park
d66116d809 Update email MESSAGE_BODY to reflect dynamic max_age. 2017-03-09 08:33:09 -08:00
Iain Lane
b896fbf038 Enable email policy based on a config flag
So we can turn it off for the "notest" run and for the non-dev series.

This is a tristate

 - 'yes': send email as normal
 - 'dry-run': log what it would do, but send no email [nor update the
   cache, so each run is effectively a fresh run]
 - 'no': disable completely
2017-03-09 11:43:30 +00:00
Robert Bruce Park
2775a5435c Email Policy, send emails when packages are rejected. 2017-03-09 09:53:17 +00:00
Iain Lane
fe627aaa15 autopkgtest: Don't request tests for new packages that FTBFS
These are treated separately by britney ('no-binaries' vs.
'missing-builds').
2017-03-02 10:28:22 +00:00
Iain Lane
d366288a7b test_autopkgtest: Remove Testsuite-Triggers
This way we are testing the synthesisted test-dep explicitly
2017-02-24 16:07:39 +00:00
Andy Whitcroft
e16195ef46 tests/test_autopkgtest.py: fix test package
Signed-off-by: Andy Whitcroft <apw@ubuntu.com>
2017-02-24 15:32:44 +00:00
Andy Whitcroft
71c8773b7d linux-meta: add synthetic dependancy for snapd
Signed-off-by: Andy Whitcroft <apw@canonical.com>
2017-02-24 14:27:43 +00:00
Iain Lane
766ed38d40 sourceppa: Refactor exception handling
Use a for loop for retrying instead of recursion, so we don't get nested
exceptions being thrown. Add a test for this flakiness.
2017-02-08 13:17:16 +00:00
Iain Lane
9661362880 sourceppa: Retry 5 times on HTTP 503s
We get these quite frequently from the LP API - have a few attempts.
2017-02-07 17:26:03 +00:00
Iain Lane
41d51369f5 autopkgtest: Accept packages which have dropped their tests in unstable
Currently we re-trigger all reverse binary dependencies of a package,
including binary packages built from the same source. We already
explicity trigger the source's own tests if they still exist in unstable
- don't also consider the source when looking at reverse dependencies.
2017-02-07 15:16:51 +00:00
Steve Langasek
8310df678b Merge fix for LP: #1651537 from robru. 2017-01-31 23:42:30 -08:00
Robert Bruce Park
18ba74be11 Disable PIUParts (LP: #1651537) 2017-01-24 15:42:38 -08:00
Robert Bruce Park
820c4460ce Fix upstream policy tests. 2017-01-24 15:41:46 -08:00
Iain Lane
30b11ff4f9 test_autopkgtest: Test that blacklisted packages can be hinted over 2017-01-19 09:52:04 +00:00
Iain Lane
7946be64df test_autopkgtest: Make the version 'blacklisted', to be clearer 2017-01-19 09:51:40 +00:00
Iain Lane
6573a1a5af tests/test_autopkgtest: Make sure that blacklisted (code 99) packages are handled 2017-01-18 17:41:45 +00:00
Martin Pitt
1a53882811 Python 3.2 workaround: Replace FileNotFoundError with {IO,OS}Error
The FileNotFound class is not yet available in Python 3.2, but as long as
snakefruit still runs Ubuntu 12.04 we need to get along without it.
2016-12-12 11:41:31 +01:00
Martin Pitt
fadcea4995 Python 3.2 workaround: Drop usage of clear()
The clear() collection method is not yet available in Python 3.2, but as long
as snakefruit still runs Ubuntu 12.04 we need to get along without it.
2016-12-12 11:38:47 +01:00
Martin Pitt
7c6fd99417 Python 3.2 workaround: Drop usage of enum module
enum is not yet available in Python 3.2, but as long as snakefruit still runs
Ubuntu 12.04 we need to get along without it.
2016-12-12 11:38:47 +01:00
Robert Bruce Park
c5c9c6f979 Reject packages if entire source ppa won't migrate 2016-12-12 11:38:47 +01:00
Martin Pitt
a217ea6ade Add "run-autopkgtest" tool to re-run tests 2016-12-12 11:33:45 +01:00
Martin Pitt
96df4080b9 Add autopkgtest policy
Add new autopkgtest policy: it determines the autopkgtests for a
source package (its own, direct reverse binary dependencies, and
Testsuite-Triggers), requests tests via AMQP, fetches results from swift, and
keeps track of pending tests between run. This also caches the downloaded
results from swift, as re-dowloading them all is very expensive.

This introduces two new hints:

 * force-badtest pkg/ver[/arch]: Failing results for that package will be
   ignored. This is useful to deal with broken tests that get imported from
   Debian or are from under-maintained packages, or broke due to some
   infrastructure changes. These are long-lived usually.

 * force-skiptest pkg/ver: Test results *triggered by* that package (i. e.
   reverse dependencies) will be ignored. This is mostly useful for landing
   packages that trigger a huge amount of tests (glibc, perl) where some tests
   are just too flaky to get them all passing, and one just wants to land it
   after the remaining failures have been checked. This should be used rarely
   and the hints should be removed immediately again.

Add integration tests that call britney in various scenarios on constructed
fake archives, with mocked AMQP and Swift results.
2016-12-12 11:33:45 +01:00
Martin Pitt
65b85ca7e9 Make linux* wait on corresponding -meta package
We want to treat linux-$flavor and linux-meta-$flavor as one set in britney
which goes in together or not at all. We never want to promote linux-$flavor
without the accompanying linux-meta-$flavor.

Introduce a synthetic linux* → linux-meta* dependency to enforce this grouping.
2016-12-12 11:33:45 +01:00
Dimitri John Ledkov
d2cde7de2b Teach excuse_unsat_deps() about components ogre model 2016-12-12 11:33:45 +01:00
Colin Watson
158d7ca479 Change date format to seconds-since-epoch 2016-12-12 11:33:44 +01:00
Colin Watson
8b140b3900 Allow blocking uploads by way of bugs with the block-proposed tag set 2016-12-12 11:33:44 +01:00
Colin Watson
b874ff151d Link to wiki documentation 2016-12-12 11:33:44 +01:00
Colin Watson
39fb743f3b Update contact information for Ubuntu 2016-12-12 11:33:44 +01:00
Colin Watson
0fd5a5640d Show links to Launchpad package pages in excuses 2016-12-12 11:33:44 +01:00