In the autopkgtest policy, cache the list of force-reset-test hints once

Due to the number of hints in standing use in Ubuntu, hints.search() is an
expensive operation, and we call it once for *every single test* referenced
from -proposed.  Since force-reset-test are a small proportion of the hints
in use, searching once for all the hints of this type and only searching
this subset for each autopkgtest improves performance (with 23000
autopkgtests referenced in -proposed, this saves roughly 1 minute of
runtime, or 11% on a 9-minute britney run; the number of packages in
-proposed is typically much higher at other points in the release cycle,
therefore the absolute improvement in performance is expected to be
greater.)

The force-reset-test hints are an Ubuntu delta so this is not expected to be
upstreamed; and it could eventually be dropped if and when baseline
retesting is implemented in Ubuntu and the number of hints required drops.

This could be implemented with a more generic, elegant solution in
HintsCollection, but again, the scalability problem of hints is hopefully
short-lived so I didn't consider it worth the investment here.
less-recipients
Steve Langasek 3 years ago
parent 82fad57aa9
commit 6cd47f352b

@ -1286,32 +1286,38 @@ class AutopkgtestPolicy(BasePolicy):
def find_max_lower_force_reset_test(self, src, ver, arch): def find_max_lower_force_reset_test(self, src, ver, arch):
'''Find the maximum force-reset-test hint before/including ver''' '''Find the maximum force-reset-test hint before/including ver'''
hints = self.hints.search('force-reset-test', package=src)
found_ver = None found_ver = None
if hints: if not hasattr(self, 'reset_hints'):
for hint in hints: self.reset_hints = self.hints.search('force-reset-test')
for mi in hint.packages:
if (mi.architecture in ['source', arch] and for hint in self.reset_hints:
mi.version != 'all' and for mi in hint.packages:
apt_pkg.version_compare(mi.version, ver) <= 0 and if mi.package != src:
(found_ver is None or apt_pkg.version_compare(found_ver, mi.version) < 0)): continue
found_ver = mi.version if (mi.architecture in ['source', arch] and
mi.version != 'all' and
apt_pkg.version_compare(mi.version, ver) <= 0 and
(found_ver is None or apt_pkg.version_compare(found_ver, mi.version) < 0)):
found_ver = mi.version
return found_ver return found_ver
def has_higher_force_reset_test(self, src, ver, arch): def has_higher_force_reset_test(self, src, ver, arch):
'''Find if there is a minimum force-reset-test hint after/including ver''' '''Find if there is a minimum force-reset-test hint after/including ver'''
hints = self.hints.search('force-reset-test', package=src)
if hints: if not hasattr(self, 'reset_hints'):
self.logger.info('Checking hints for %s/%s/%s: %s' % (src, ver, arch, [str(h) for h in hints])) self.reset_hints = self.hints.search('force-reset-test')
for hint in hints:
for mi in hint.packages: for hint in self.reset_hints:
if (mi.architecture in ['source', arch] and self.logger.info('Checking hints for %s/%s/%s: %s' % (src, ver, arch, str(hint)))
mi.version != 'all' and for mi in hint.packages:
apt_pkg.version_compare(mi.version, ver) >= 0): if mi.package != src:
return True continue
if (mi.architecture in ['source', arch] and
mi.version != 'all' and
apt_pkg.version_compare(mi.version, ver) >= 0):
return True
return False return False

Loading…
Cancel
Save