580 Commits

Author SHA1 Message Date
Iain Lane
b896fbf038 Enable email policy based on a config flag
So we can turn it off for the "notest" run and for the non-dev series.

This is a tristate

 - 'yes': send email as normal
 - 'dry-run': log what it would do, but send no email [nor update the
   cache, so each run is effectively a fresh run]
 - 'no': disable completely
2017-03-09 11:43:30 +00:00
Robert Bruce Park
2775a5435c Email Policy, send emails when packages are rejected. 2017-03-09 09:53:17 +00:00
Robert Bruce Park
18ba74be11 Disable PIUParts (LP: #1651537) 2017-01-24 15:42:38 -08:00
Martin Pitt
1a53882811 Python 3.2 workaround: Replace FileNotFoundError with {IO,OS}Error
The FileNotFound class is not yet available in Python 3.2, but as long as
snakefruit still runs Ubuntu 12.04 we need to get along without it.
2016-12-12 11:41:31 +01:00
Martin Pitt
fadcea4995 Python 3.2 workaround: Drop usage of clear()
The clear() collection method is not yet available in Python 3.2, but as long
as snakefruit still runs Ubuntu 12.04 we need to get along without it.
2016-12-12 11:38:47 +01:00
Martin Pitt
7c6fd99417 Python 3.2 workaround: Drop usage of enum module
enum is not yet available in Python 3.2, but as long as snakefruit still runs
Ubuntu 12.04 we need to get along without it.
2016-12-12 11:38:47 +01:00
Robert Bruce Park
c5c9c6f979 Reject packages if entire source ppa won't migrate 2016-12-12 11:38:47 +01:00
Martin Pitt
96df4080b9 Add autopkgtest policy
Add new autopkgtest policy: it determines the autopkgtests for a
source package (its own, direct reverse binary dependencies, and
Testsuite-Triggers), requests tests via AMQP, fetches results from swift, and
keeps track of pending tests between run. This also caches the downloaded
results from swift, as re-dowloading them all is very expensive.

This introduces two new hints:

 * force-badtest pkg/ver[/arch]: Failing results for that package will be
   ignored. This is useful to deal with broken tests that get imported from
   Debian or are from under-maintained packages, or broke due to some
   infrastructure changes. These are long-lived usually.

 * force-skiptest pkg/ver: Test results *triggered by* that package (i. e.
   reverse dependencies) will be ignored. This is mostly useful for landing
   packages that trigger a huge amount of tests (glibc, perl) where some tests
   are just too flaky to get them all passing, and one just wants to land it
   after the remaining failures have been checked. This should be used rarely
   and the hints should be removed immediately again.

Add integration tests that call britney in various scenarios on constructed
fake archives, with mocked AMQP and Swift results.
2016-12-12 11:33:45 +01:00
Martin Pitt
65b85ca7e9 Make linux* wait on corresponding -meta package
We want to treat linux-$flavor and linux-meta-$flavor as one set in britney
which goes in together or not at all. We never want to promote linux-$flavor
without the accompanying linux-meta-$flavor.

Introduce a synthetic linux* → linux-meta* dependency to enforce this grouping.
2016-12-12 11:33:45 +01:00
Dimitri John Ledkov
d2cde7de2b Teach excuse_unsat_deps() about components ogre model 2016-12-12 11:33:45 +01:00
Colin Watson
8b140b3900 Allow blocking uploads by way of bugs with the block-proposed tag set 2016-12-12 11:33:44 +01:00
Colin Watson
39fb743f3b Update contact information for Ubuntu 2016-12-12 11:33:44 +01:00
Colin Watson
a3b5ebf22d buildd.debian.org -> launchpad.net 2016-12-12 11:33:44 +01:00
Martin Pitt
15aab3348a Store Testsuite-Triggers: list in sources
Compute an inverse trigger → sources map, as that's usually the direction that
britney is interested in.
2016-12-12 11:33:44 +01:00
Martin Pitt
0d43a20471 Store Testsuite: list in sources
Extend read_sources to store the components of Testsuite: as a list.
2016-12-12 11:33:44 +01:00
Colin Watson
7eb3412828 Add a --distribution option 2016-12-12 11:33:44 +01:00
Colin Watson
3db97e9741 Add new --series option
Replace %(SERIES) macro in config files with given value.
2016-12-12 11:33:44 +01:00
Colin Watson
8fb3c782aa Only issue "From wrong source" for binaries not at the same version in testing
Same-version could happen for merged binaries in partial-unstable mode.
2016-12-12 11:33:44 +01:00
Colin Watson
b144470940 Optionally merge packages from testing to unstable
To cope with a partial unstable suite like Ubuntu's -proposed pocket.
2016-12-12 11:33:44 +01:00
Colin Watson
a2dcb26901 Make sure that containing directories exist before writing output files
We don't use os.makedirs(dir, exist_ok=True) as that is too strict: it fails if
the directory already exists with different permissions (e. g. with 775). Thus
introduce a helper function ensuredir().
2016-12-12 11:33:44 +01:00
Robert Bruce Park
f32a8e37da Make Heidi optional
Setting HEIDI_OUTPUT to an empty value will skip the heidi generation, to speed
up britney in scenarios where heidi is not needed.
2016-12-12 11:33:44 +01:00
Martin Pitt
0c73ebfa98 Skip second-stage upgrade testing if UPGRADE_OUTPUT option is absent or empty
This is useful to speed up britney in dry run mode, if we are only interested
in builds and package tests.
2016-12-12 11:33:44 +01:00
Martin Pitt
e987fedd80 Save policy state before upgrade tester
We might not (want to) run the upgrade tester for some use cases, but we still
need to save the policy state.
2016-12-12 11:33:44 +01:00
Martin Pitt
e5f306c5f5 Consider packages with M-A qualifiers for reverse dependencies
Strip of Multi-Arch qualifiers ":any" and ":native" when building the
dependency fields, as they are not part of the package name.

This will fix cases like

  Package: ipython3
  Depends: python3:any (>= 3)

and include ipython3 in python3's reverse dependencies.

Closes: #794194
2016-12-12 11:33:44 +01:00
Iain Lane
8e0405d2e2 hint-tester: Don't lowercase the input
Doing this means that you can't use the hint tester for packages with
uppercase characters in the version, e.g.

  Version mismatch, dreamchess 0.2.1-rc2-2build1 != 0.2.1-RC2-2build1
2016-12-12 08:49:50 +01:00
Emilio Pozuelo Monfort
af7c96142d Add ignore-piuparts to the standard hint set
Signed-off-by: Emilio Pozuelo Monfort <pochu@debian.org>
2016-12-09 08:59:56 +01:00
Robert Bruce Park
40ccb4bdc6 Invalidate excuse after each policy run
This fixes commit 497edc to really allow policies to see if the excuse has
already been invalidated by previous policies.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-26 19:34:54 +00:00
Niels Thykier
bcdb2b39cd Add PiupartsPolicy to avoid piuparts regressions
Closes: Debian/britney2#16
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-26 17:11:40 +00:00
Niels Thykier
e6e221ad52 Extract a compile_nuninst and move it to utils
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:33:39 +00:00
Niels Thykier
06504f2ae7 Move invalidate_exuses to utils
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:22:22 +00:00
Niels Thykier
20f2e27873 Fix --print-nuninst + --nuninst-cache
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:21:06 +00:00
Niels Thykier
fd614062ae Minor optimisation of force handling
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:20:45 +00:00
Niels Thykier
56fa65f596 Replace a use of src[X] with src.x
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:08:37 +00:00
Niels Thykier
0cc7f7a8d2 Move build_installability_tester to builder.py
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:03:40 +00:00
Niels Thykier
7a63784876 Move get_dependency_solvers to utils
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:03:40 +00:00
Niels Thykier
c4be8436db Move read_sources_file to utils
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 21:03:40 +00:00
Niels Thykier
0f45b195a6 Refactor doop_source to simplify logic
Cleanly split doop_source into a (small) part about source packages
and a (longer) part about binary packages.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 20:21:13 +00:00
Niels Thykier
027142e12b Fix bug where compute_groups returned tuple instead of pkg_id
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 20:21:13 +00:00
Niels Thykier
a14ced6403 Drop now unused "from __future__" imports
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 20:21:06 +00:00
Niels Thykier
7af650d36d "selected" is never None in iter_packages
It used to be possible, but it is not any more.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 20:21:06 +00:00
Niels Thykier
ecf95229d3 Use asserts to guard single binary removal items
Coverage suggests that the conditions are always true.  If so, we can
replace the "elif" with a regular "if" - but for now, lets keep an
assert.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 20:21:06 +00:00
Niels Thykier
9ca30aac81 Rename a variable for clarity reasons
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-22 20:21:06 +00:00
Niels Thykier
9c563f1a96 Move data classes into britney2/__init__.py
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-13 17:33:45 +00:00
Niels Thykier
6360daf8c5 Assert that binaries have sensible arch values
Add a check to ensure we do not create broken test data (as happened
recently, where the architecture values were swapped for a single
binary).  A cold hard failure like this is easier to debug
compared to Britney playing "garbage in, garbage out".

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-13 12:49:39 +00:00
Niels Thykier
20c2b87833 Sprinkle some "no cover" pragmas over the code base
Add some "no cover" to some unrecoverable exceptions
(e.g. misconfiguration) or base-class methods that are not intended to
be invoked.

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-13 11:39:46 +00:00
Martin Pitt
15fdf960d7 Use Excuse.is_valid when iterating through checks and Policies
This gives Policies the opportunity to see if a previous check
(build/installability) or earlier policies already invalidated the update. This
allows writing policies that work on groups of packages, or skipping expensive
checks (such as triggering autopkgtests while the package is not built or
installable yet).

Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-13 11:18:46 +00:00
Niels Thykier
e13763d335 Refactor out some some hash lookup in _compute_groups
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-13 11:18:45 +00:00
Niels Thykier
636f349def Correct description to say "Release" instead of "InRelease" 2016-11-13 11:18:45 +00:00
Niels Thykier
0e30d5c170 Use suite-info to provide the excuses suffix
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-13 11:18:45 +00:00
Niels Thykier
50d4d45c59 Collect suite metadata in a new member hash
Signed-off-by: Niels Thykier <niels@thykier.net>
2016-11-13 11:18:45 +00:00