In the autopkgtest policy, cache the list of force-reset-test hints once

Due to the number of hints in standing use in Ubuntu, hints.search() is an
expensive operation, and we call it once for *every single test* referenced
from -proposed.  Since force-reset-test are a small proportion of the hints
in use, searching once for all the hints of this type and only searching
this subset for each autopkgtest improves performance (with 23000
autopkgtests referenced in -proposed, this saves roughly 1 minute of
runtime, or 11% on a 9-minute britney run; the number of packages in
-proposed is typically much higher at other points in the release cycle,
therefore the absolute improvement in performance is expected to be
greater.)

The force-reset-test hints are an Ubuntu delta so this is not expected to be
upstreamed; and it could eventually be dropped if and when baseline
retesting is implemented in Ubuntu and the number of hints required drops.

This could be implemented with a more generic, elegant solution in
HintsCollection, but again, the scalability problem of hints is hopefully
short-lived so I didn't consider it worth the investment here.
less-recipients
Steve Langasek 3 years ago
parent 82fad57aa9
commit 6cd47f352b

@ -1286,32 +1286,38 @@ class AutopkgtestPolicy(BasePolicy):
def find_max_lower_force_reset_test(self, src, ver, arch):
'''Find the maximum force-reset-test hint before/including ver'''
hints = self.hints.search('force-reset-test', package=src)
found_ver = None
if hints:
for hint in hints:
for mi in hint.packages:
if (mi.architecture in ['source', arch] and
mi.version != 'all' and
apt_pkg.version_compare(mi.version, ver) <= 0 and
(found_ver is None or apt_pkg.version_compare(found_ver, mi.version) < 0)):
found_ver = mi.version
if not hasattr(self, 'reset_hints'):
self.reset_hints = self.hints.search('force-reset-test')
for hint in self.reset_hints:
for mi in hint.packages:
if mi.package != src:
continue
if (mi.architecture in ['source', arch] and
mi.version != 'all' and
apt_pkg.version_compare(mi.version, ver) <= 0 and
(found_ver is None or apt_pkg.version_compare(found_ver, mi.version) < 0)):
found_ver = mi.version
return found_ver
def has_higher_force_reset_test(self, src, ver, arch):
'''Find if there is a minimum force-reset-test hint after/including ver'''
hints = self.hints.search('force-reset-test', package=src)
if hints:
self.logger.info('Checking hints for %s/%s/%s: %s' % (src, ver, arch, [str(h) for h in hints]))
for hint in hints:
for mi in hint.packages:
if (mi.architecture in ['source', arch] and
mi.version != 'all' and
apt_pkg.version_compare(mi.version, ver) >= 0):
return True
if not hasattr(self, 'reset_hints'):
self.reset_hints = self.hints.search('force-reset-test')
for hint in self.reset_hints:
self.logger.info('Checking hints for %s/%s/%s: %s' % (src, ver, arch, str(hint)))
for mi in hint.packages:
if mi.package != src:
continue
if (mi.architecture in ['source', arch] and
mi.version != 'all' and
apt_pkg.version_compare(mi.version, ver) >= 0):
return True
return False

Loading…
Cancel
Save