67 Commits

Author SHA1 Message Date
Robert Bruce Park
16cca9c55d Change SourcePPAPolicy from a blacklist to a whitelist. 2017-03-24 11:46:52 -07:00
Iain Lane
858775a3c1 email: Make REJECTED_TEMPORARILY packages be not emailed for
These are rejections that we expect to clear out on their own or convert
to REJECTED_PERMANENTLY, at which point we will email.

https://bugs.launchpad.net/britney/+bug/1671468
2017-03-23 11:10:14 +00:00
Iain Lane
e00d86ad26 email: If a package is blocked by an explicit block, don't mail 2017-03-22 17:00:38 +00:00
Iain Lane
8350694348 email: Add tests that send email through a mocked SMTP server
I want to fix two bugs in interactions between other parts of britney
and the email policy. It's not currently easy to do so because we just
run the policy itself manually by creating some fake excuses.

Steal part of the machinery from the autopkgtest tests, and run a few
tests through britney completely. Use a fake SMTP server to record which
emails we sent.

(The port is hardcoded - that might not be so smart.)
2017-03-22 16:59:50 +00:00
Robert Bruce Park
63573fa24a Resend emails periodically. 2017-03-22 09:50:50 -07:00
Robert Bruce Park
0579677457 Stop writing "1 days" in emails. 2017-03-14 11:05:10 -07:00
Robert Bruce Park
1b4828bfb3 Write email cache after every email sent. 2017-03-10 09:48:37 -08:00
Steve Langasek
442993d3f0 Merge remote-tracking branch 'robru/fix-sourceppa' 2017-03-09 14:59:40 -08:00
Robert Bruce Park
389f8624dd Round age. 2017-03-09 11:12:20 -08:00
Robert Bruce Park
e7181465a8 Stop returning None. 2017-03-09 11:04:12 -08:00
Robert Bruce Park
2405fcf8bc Backport EmailPolicy to Python 3.2 2017-03-09 18:00:22 +00:00
Iain Lane
7cc72bf2d6 email: For direct sponsored uploads, email both the sponsor and sponsoree 2017-03-09 17:16:21 +00:00
Robert Bruce Park
df495582e2 Fix SourcePPA policy by inspecting build_link instead. (LP: #1648000) 2017-03-09 09:06:25 -08:00
Iain Lane
5f4c2e735d email: Log a message when we send emails 2017-03-09 16:46:47 +00:00
Iain Lane
d9887d320e email: be From the 'Ubuntu Release Team' 2017-03-09 16:41:34 +00:00
Robert Bruce Park
d66116d809 Update email MESSAGE_BODY to reflect dynamic max_age. 2017-03-09 08:33:09 -08:00
Iain Lane
b896fbf038 Enable email policy based on a config flag
So we can turn it off for the "notest" run and for the non-dev series.

This is a tristate

 - 'yes': send email as normal
 - 'dry-run': log what it would do, but send no email [nor update the
   cache, so each run is effectively a fresh run]
 - 'no': disable completely
2017-03-09 11:43:30 +00:00
Robert Bruce Park
2775a5435c Email Policy, send emails when packages are rejected. 2017-03-09 09:53:17 +00:00
Iain Lane
fe627aaa15 autopkgtest: Don't request tests for new packages that FTBFS
These are treated separately by britney ('no-binaries' vs.
'missing-builds').
2017-03-02 10:28:22 +00:00
Andy Whitcroft
71c8773b7d linux-meta: add synthetic dependancy for snapd
Signed-off-by: Andy Whitcroft <apw@canonical.com>
2017-02-24 14:27:43 +00:00
Iain Lane
766ed38d40 sourceppa: Refactor exception handling
Use a for loop for retrying instead of recursion, so we don't get nested
exceptions being thrown. Add a test for this flakiness.
2017-02-08 13:17:16 +00:00
Iain Lane
9661362880 sourceppa: Retry 5 times on HTTP 503s
We get these quite frequently from the LP API - have a few attempts.
2017-02-07 17:26:03 +00:00
Iain Lane
41d51369f5 autopkgtest: Accept packages which have dropped their tests in unstable
Currently we re-trigger all reverse binary dependencies of a package,
including binary packages built from the same source. We already
explicity trigger the source's own tests if they still exist in unstable
- don't also consider the source when looking at reverse dependencies.
2017-02-07 15:16:51 +00:00
Robert Bruce Park
820c4460ce Fix upstream policy tests. 2017-01-24 15:41:46 -08:00
Martin Pitt
1a53882811 Python 3.2 workaround: Replace FileNotFoundError with {IO,OS}Error
The FileNotFound class is not yet available in Python 3.2, but as long as
snakefruit still runs Ubuntu 12.04 we need to get along without it.
2016-12-12 11:41:31 +01:00
Martin Pitt
7c6fd99417 Python 3.2 workaround: Drop usage of enum module
enum is not yet available in Python 3.2, but as long as snakefruit still runs
Ubuntu 12.04 we need to get along without it.
2016-12-12 11:38:47 +01:00
Robert Bruce Park
c5c9c6f979 Reject packages if entire source ppa won't migrate 2016-12-12 11:38:47 +01:00
Martin Pitt
a217ea6ade Add "run-autopkgtest" tool to re-run tests 2016-12-12 11:33:45 +01:00
Martin Pitt
96df4080b9 Add autopkgtest policy
Add new autopkgtest policy: it determines the autopkgtests for a
source package (its own, direct reverse binary dependencies, and
Testsuite-Triggers), requests tests via AMQP, fetches results from swift, and
keeps track of pending tests between run. This also caches the downloaded
results from swift, as re-dowloading them all is very expensive.

This introduces two new hints:

 * force-badtest pkg/ver[/arch]: Failing results for that package will be
   ignored. This is useful to deal with broken tests that get imported from
   Debian or are from under-maintained packages, or broke due to some
   infrastructure changes. These are long-lived usually.

 * force-skiptest pkg/ver: Test results *triggered by* that package (i. e.
   reverse dependencies) will be ignored. This is mostly useful for landing
   packages that trigger a huge amount of tests (glibc, perl) where some tests
   are just too flaky to get them all passing, and one just wants to land it
   after the remaining failures have been checked. This should be used rarely
   and the hints should be removed immediately again.

Add integration tests that call britney in various scenarios on constructed
fake archives, with mocked AMQP and Swift results.
2016-12-12 11:33:45 +01:00
Dimitri John Ledkov
d2cde7de2b Teach excuse_unsat_deps() about components ogre model 2016-12-12 11:33:45 +01:00
Colin Watson
158d7ca479 Change date format to seconds-since-epoch 2016-12-12 11:33:44 +01:00
Colin Watson
8b140b3900 Allow blocking uploads by way of bugs with the block-proposed tag set 2016-12-12 11:33:44 +01:00
Colin Watson
b874ff151d Link to wiki documentation 2016-12-12 11:33:44 +01:00
Colin Watson
0fd5a5640d Show links to Launchpad package pages in excuses 2016-12-12 11:33:44 +01:00
Martin Pitt
15aab3348a Store Testsuite-Triggers: list in sources
Compute an inverse trigger → sources map, as that's usually the direction that
britney is interested in.
2016-12-12 11:33:44 +01:00
Martin Pitt
0d43a20471 Store Testsuite: list in sources
Extend read_sources to store the components of Testsuite: as a list.
2016-12-12 11:33:44 +01:00
Colin Watson
7eb3412828 Add a --distribution option 2016-12-12 11:33:44 +01:00
Colin Watson
a2dcb26901 Make sure that containing directories exist before writing output files
We don't use os.makedirs(dir, exist_ok=True) as that is too strict: it fails if
the directory already exists with different permissions (e. g. with 775). Thus
introduce a helper function ensuredir().
2016-12-12 11:33:44 +01:00
Martin Pitt
e93d8ae221 Skip piuparts policy if summary file is missing 2016-12-12 11:33:44 +01:00
Colin Watson
c26dfad9c2 Skip bug-based processing if BugsV is missing 2016-12-12 11:33:44 +01:00
Colin Watson
538d5c61ff Use default urgency for all packages if Urgencies is missing 2016-12-12 11:33:44 +01:00
Martin Pitt
e5f306c5f5 Consider packages with M-A qualifiers for reverse dependencies
Strip of Multi-Arch qualifiers ":any" and ":native" when building the
dependency fields, as they are not part of the package name.

This will fix cases like

  Package: ipython3
  Depends: python3:any (>= 3)

and include ipython3 in python3's reverse dependencies.

Closes: #794194
2016-12-12 11:33:44 +01:00
Niels Thykier
3e888a9fdf PiupartsPolicy: Improve html message
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-12-04 10:38:15 +00:00
Niels Thykier
bcdb2b39cd Add PiupartsPolicy to avoid piuparts regressions
Closes: Debian/britney2#16
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-26 17:11:40 +00:00
Niels Thykier
e6e221ad52 Extract a compile_nuninst and move it to utils
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:33:39 +00:00
Niels Thykier
eaf4660918 utils: Replace explicit loop index handling with enumerate
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:33:21 +00:00
Niels Thykier
06504f2ae7 Move invalidate_exuses to utils
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:22:22 +00:00
Niels Thykier
166a6533ba completer: Replace src[X] with src.X
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:03:40 +00:00
Niels Thykier
0cc7f7a8d2 Move build_installability_tester to builder.py
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:03:40 +00:00
Niels Thykier
7a63784876 Move get_dependency_solvers to utils
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:03:40 +00:00