Compare commits

..

8 Commits

Author SHA1 Message Date
Steve Langasek
cb871ef819 changelog bump 2024-05-13 22:28:10 +02:00
Steve Langasek
2f6c057748 Runtime dep in ALL packages 2024-05-13 22:27:58 +02:00
Steve Langasek
94e63bd4f2 changelog bump 2024-05-10 18:10:42 +02:00
Steve Langasek
8b389d6f30 Also needs dropped as a runtime dep... 2024-05-10 18:10:29 +02:00
Steve Langasek
7192e99427 changelog bump 2024-05-08 11:33:37 +02:00
Steve Langasek
475533f542 Correct wrong backport of python3-launchpadlib-desktop change 2024-05-08 11:33:23 +02:00
Steve Langasek
823b790da3 releasing package ubuntu-dev-tools version 0.201ubuntu2~23.10.1 2024-03-12 17:47:28 -07:00
Steve Langasek
7cac9c9c3b Backport to mantic 2024-03-12 17:47:19 -07:00
35 changed files with 368 additions and 693 deletions

2
.gitignore vendored
View File

@ -1,2 +0,0 @@
__pycache__
*.egg-info

1
debian/.gitignore vendored
View File

@ -1 +0,0 @@
files

88
debian/changelog vendored
View File

@ -1,83 +1,10 @@
ubuntu-dev-tools (0.206) unstable; urgency=medium ubuntu-dev-tools (0.201ubuntu2~23.10.4) mantic; urgency=medium
[ Dan Bungert ] * Backport current ubuntu-dev-tools to mantic. LP: #2057716.
* mk-sbuild: enable pkgmaintainermangler
[ Shengjing Zhu ] -- Steve Langasek <steve.langasek@ubuntu.com> Tue, 12 Mar 2024 17:47:25 -0700
* import-bug-from-debian: package option is overridden and not used
[ Fernando Bravo Hernández ] ubuntu-dev-tools (0.201ubuntu2) noble; urgency=medium
* Parsing arch parameter to getBinaryPackage() (LP: #2081861)
[ Simon Quigley ]
* Read ~/.devscripts in a more robust way, to ideally pick up multi-line
variables (Closes: #725418).
* mk-sbuild: default to using UTC for schroots (LP: #2097159).
* syncpackage: s/syncblacklist/syncblocklist/g
* syncpackage: Cache the sync blocklist in-memory, so it's not fetched
multiple times when syncing more than one package.
* syncpackage: Catch exceptions cleanly, simply skipping to the next
package (erring on the side of caution) if there is an error doing the
download (LP: #1943286).
-- Simon Quigley <tsimonq2@debian.org> Tue, 04 Mar 2025 13:43:15 -0600
ubuntu-dev-tools (0.205) unstable; urgency=medium
* [syncpackage] When syncing multiple packages, if one of the packages is in
the sync blocklist, do not exit, simply continue.
* [syncpackage] Do not use exit(1) on an error or exception unless it
applies to all packages, instead return None so we can continue to the
next package.
* [syncpackage] Add support for -y or --yes, noted that it should be used
with care.
* Update Standards-Version to 4.7.2, no changes needed.
-- Simon Quigley <tsimonq2@debian.org> Sat, 01 Mar 2025 11:29:54 -0600
ubuntu-dev-tools (0.204) unstable; urgency=medium
[ Simon Quigley ]
* Update Standards-Version to 4.7.1, no changes needed.
* Add several Lintian overrides related to .pyc files.
* Add my name to the copyright file.
* Rename bitesize to lp-bitesize (Closes: #1076224).
* Add a manpage for running-autopkgtests.
* Add a large warning at the top of mk-sbuild encouraging the use of the
unshare backend. This is to provide ample warning to users.
* Remove mail line from default ~/.sbuildrc, to resolve the undeclared
dependency on sendmail (Closes: #1074632).
[ Julien Plissonneau Duquène ]
* Fix reverse-depends -b crash on packages that b-d on themselves
(Closes: #1087760).
-- Simon Quigley <tsimonq2@debian.org> Mon, 24 Feb 2025 19:54:39 -0600
ubuntu-dev-tools (0.203) unstable; urgency=medium
[ Steve Langasek ]
* ubuntu-build: handle TOCTOU issue with the "can be retried" value on
builds.
* Recommend sbuild over pbuilder. sbuild is the tool recommended by
Ubuntu developers whose behavior most closely approximates Launchpad
builds.
[ Florent 'Skia' Jacquet ]
* import-bug-from-debian: handle multipart message (Closes: #969510)
[ Benjamin Drung ]
* import-bug-from-debian: add type hints
* Bump Standards-Version to 4.7.0
* Bump year and add missing files to copyright
* setup.py: add pm-helper
* Format code with black and isort
* Address several issues pointed out by Pylint
* Depend on python3-yaml for pm-helper
-- Benjamin Drung <bdrung@debian.org> Sat, 02 Nov 2024 18:19:24 +0100
ubuntu-dev-tools (0.202) unstable; urgency=medium
[ Steve Langasek ] [ Steve Langasek ]
* ubuntu-build: support --batch with no package names to retry all * ubuntu-build: support --batch with no package names to retry all
@ -88,11 +15,14 @@ ubuntu-dev-tools (0.202) unstable; urgency=medium
* ubuntu-build: Handling of proposed vs release pocket default for ppas * ubuntu-build: Handling of proposed vs release pocket default for ppas
* ubuntu-build: update manpage * ubuntu-build: update manpage
[ Chris Peterson ] -- Steve Langasek <steve.langasek@ubuntu.com> Tue, 12 Mar 2024 17:03:43 -0700
ubuntu-dev-tools (0.201ubuntu1) noble; urgency=medium
* Replace Depends on python3-launchpadlib with Depends on * Replace Depends on python3-launchpadlib with Depends on
python3-launchpadlib-desktop (LP: #2049217) python3-launchpadlib-desktop (LP: #2049217)
-- Simon Quigley <tsimonq2@ubuntu.com> Fri, 12 Apr 2024 23:33:14 -0500 -- Chris Peterson <chris.peterson@canonical.com> Fri, 01 Mar 2024 14:08:07 -0800
ubuntu-dev-tools (0.201) unstable; urgency=medium ubuntu-dev-tools (0.201) unstable; urgency=medium

15
debian/control vendored
View File

@ -1,7 +1,8 @@
Source: ubuntu-dev-tools Source: ubuntu-dev-tools
Section: devel Section: devel
Priority: optional Priority: optional
Maintainer: Ubuntu Developers <ubuntu-dev-tools@packages.debian.org> Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
XSBC-Original-Maintainer: Ubuntu Developers <ubuntu-dev-tools@packages.debian.org>
Uploaders: Uploaders:
Benjamin Drung <bdrung@debian.org>, Benjamin Drung <bdrung@debian.org>,
Stefano Rivera <stefanor@debian.org>, Stefano Rivera <stefanor@debian.org>,
@ -26,12 +27,11 @@ Build-Depends:
python3-debianbts, python3-debianbts,
python3-distro-info, python3-distro-info,
python3-httplib2, python3-httplib2,
python3-launchpadlib-desktop, python3-launchpadlib,
python3-pytest, python3-pytest,
python3-requests <!nocheck>, python3-requests <!nocheck>,
python3-setuptools, python3-setuptools,
python3-yaml <!nocheck>, Standards-Version: 4.6.2
Standards-Version: 4.7.2
Rules-Requires-Root: no Rules-Requires-Root: no
Vcs-Git: https://git.launchpad.net/ubuntu-dev-tools Vcs-Git: https://git.launchpad.net/ubuntu-dev-tools
Vcs-Browser: https://git.launchpad.net/ubuntu-dev-tools Vcs-Browser: https://git.launchpad.net/ubuntu-dev-tools
@ -54,10 +54,9 @@ Depends:
python3-debianbts, python3-debianbts,
python3-distro-info, python3-distro-info,
python3-httplib2, python3-httplib2,
python3-launchpadlib-desktop, python3-launchpadlib,
python3-lazr.restfulclient, python3-lazr.restfulclient,
python3-ubuntutools (= ${binary:Version}), python3-ubuntutools (= ${binary:Version}),
python3-yaml,
sensible-utils, sensible-utils,
sudo, sudo,
tzdata, tzdata,
@ -72,7 +71,7 @@ Recommends:
genisoimage, genisoimage,
lintian, lintian,
patch, patch,
sbuild | pbuilder | cowbuilder, pbuilder | cowbuilder | sbuild,
python3-dns, python3-dns,
quilt, quilt,
reportbug (>= 3.39ubuntu1), reportbug (>= 3.39ubuntu1),
@ -142,7 +141,7 @@ Depends:
python3-debian, python3-debian,
python3-distro-info, python3-distro-info,
python3-httplib2, python3-httplib2,
python3-launchpadlib-desktop, python3-launchpadlib,
python3-lazr.restfulclient, python3-lazr.restfulclient,
python3-requests, python3-requests,
sensible-utils, sensible-utils,

20
debian/copyright vendored
View File

@ -11,7 +11,6 @@ Files: backportpackage
doc/check-symbols.1 doc/check-symbols.1
doc/requestsync.1 doc/requestsync.1
doc/ubuntu-iso.1 doc/ubuntu-iso.1
doc/running-autopkgtests.1
GPL-2 GPL-2
README.updates README.updates
requestsync requestsync
@ -20,13 +19,12 @@ Files: backportpackage
ubuntu-iso ubuntu-iso
ubuntutools/requestsync/*.py ubuntutools/requestsync/*.py
Copyright: 2007, Albert Damen <albrt@gmx.net> Copyright: 2007, Albert Damen <albrt@gmx.net>
2010-2024, Benjamin Drung <bdrung@ubuntu.com> 2010-2022, Benjamin Drung <bdrung@ubuntu.com>
2007-2023, Canonical Ltd. 2007-2023, Canonical Ltd.
2006-2007, Daniel Holbach <daniel.holbach@ubuntu.com> 2006-2007, Daniel Holbach <daniel.holbach@ubuntu.com>
2010, Evan Broder <evan@ebroder.net> 2010, Evan Broder <evan@ebroder.net>
2006-2007, Luke Yelavich <themuso@ubuntu.com> 2006-2007, Luke Yelavich <themuso@ubuntu.com>
2009-2010, Michael Bienia <geser@ubuntu.com> 2009-2010, Michael Bienia <geser@ubuntu.com>
2024-2025, Simon Quigley <tsimonq2@debian.org>
2010-2011, Stefano Rivera <stefanor@ubuntu.com> 2010-2011, Stefano Rivera <stefanor@ubuntu.com>
2008, Stephan Hermann <sh@sourcecode.de> 2008, Stephan Hermann <sh@sourcecode.de>
2007, Steve Kowalik <stevenk@ubuntu.com> 2007, Steve Kowalik <stevenk@ubuntu.com>
@ -74,28 +72,23 @@ License: GPL-2+
On Debian systems, the complete text of the GNU General Public License On Debian systems, the complete text of the GNU General Public License
version 2 can be found in the /usr/share/common-licenses/GPL-2 file. version 2 can be found in the /usr/share/common-licenses/GPL-2 file.
Files: doc/lp-bitesize.1 Files: doc/bitesize.1
doc/check-mir.1 doc/check-mir.1
doc/grab-merge.1 doc/grab-merge.1
doc/merge-changelog.1 doc/merge-changelog.1
doc/pm-helper.1
doc/setup-packaging-environment.1 doc/setup-packaging-environment.1
doc/syncpackage.1 doc/syncpackage.1
lp-bitesize bitesize
check-mir check-mir
GPL-3 GPL-3
grab-merge grab-merge
merge-changelog merge-changelog
pm-helper
pyproject.toml pyproject.toml
run-linters run-linters
running-autopkgtests
setup-packaging-environment setup-packaging-environment
syncpackage syncpackage
ubuntutools/running_autopkgtests.py Copyright: 2010, Benjamin Drung <bdrung@ubuntu.com>
ubuntutools/utils.py 2007-2023, Canonical Ltd.
Copyright: 2010-2024, Benjamin Drung <bdrung@ubuntu.com>
2007-2024, Canonical Ltd.
2008, Jonathan Patrick Davies <jpds@ubuntu.com> 2008, Jonathan Patrick Davies <jpds@ubuntu.com>
2008-2010, Martin Pitt <martin.pitt@canonical.com> 2008-2010, Martin Pitt <martin.pitt@canonical.com>
2009, Siegfried-Angel Gevatter Pujals <rainct@ubuntu.com> 2009, Siegfried-Angel Gevatter Pujals <rainct@ubuntu.com>
@ -184,12 +177,11 @@ Files: doc/pull-debian-debdiff.1
ubuntutools/version.py ubuntutools/version.py
update-maintainer update-maintainer
.pylintrc .pylintrc
Copyright: 2009-2024, Benjamin Drung <bdrung@ubuntu.com> Copyright: 2009-2023, Benjamin Drung <bdrung@ubuntu.com>
2010, Evan Broder <evan@ebroder.net> 2010, Evan Broder <evan@ebroder.net>
2008, Siegfried-Angel Gevatter Pujals <rainct@ubuntu.com> 2008, Siegfried-Angel Gevatter Pujals <rainct@ubuntu.com>
2010-2011, Stefano Rivera <stefanor@ubuntu.com> 2010-2011, Stefano Rivera <stefanor@ubuntu.com>
2017-2021, Dan Streetman <ddstreet@canonical.com> 2017-2021, Dan Streetman <ddstreet@canonical.com>
2024, Canonical Ltd.
License: ISC License: ISC
Permission to use, copy, modify, and/or distribute this software for any Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above purpose with or without fee is hereby granted, provided that the above

View File

@ -1,3 +0,0 @@
# pyc files are machine-generated; they're expected to have long lines and have unstated copyright
source: file-without-copyright-information *.pyc [debian/copyright]
source: very-long-line-length-in-source-file * > 512 [*.pyc:*]

View File

@ -1,21 +1,21 @@
.TH lp-bitesize "1" "May 9 2010" "ubuntu-dev-tools" .TH bitesize "1" "May 9 2010" "ubuntu-dev-tools"
.SH NAME .SH NAME
lp-bitesize \- Add \fBbitesize\fR tag to bugs and add a comment. bitesize \- Add \fBbitesize\fR tag to bugs and add a comment.
.SH SYNOPSIS .SH SYNOPSIS
.B lp-bitesize \fR<\fIbug number\fR> .B bitesize \fR<\fIbug number\fR>
.br .br
.B lp-bitesize \-\-help .B bitesize \-\-help
.SH DESCRIPTION .SH DESCRIPTION
\fBlp-bitesize\fR adds a bitesize tag to the bug, if it's not there yet. It \fBbitesize\fR adds a bitesize tag to the bug, if it's not there yet. It
also adds a comment to the bug indicating that you are willing to help with also adds a comment to the bug indicating that you are willing to help with
fixing it. fixing it.
It checks for permission to operate on a given bug first, It checks for permission to operate on a given bug first,
then perform required tasks on Launchpad. then perform required tasks on Launchpad.
.SH OPTIONS .SH OPTIONS
Listed below are the command line options for \fBlp-bitesize\fR: Listed below are the command line options for \fBbitesize\fR:
.TP .TP
.BR \-h ", " \-\-help .BR \-h ", " \-\-help
Display a help message and exit. Display a help message and exit.
@ -48,7 +48,7 @@ The default value for \fB--lpinstance\fR.
.BR ubuntu\-dev\-tools (5) .BR ubuntu\-dev\-tools (5)
.SH AUTHORS .SH AUTHORS
\fBlp-bitesize\fR and this manual page were written by Daniel Holbach \fBbitesize\fR and this manual page were written by Daniel Holbach
<daniel.holbach@canonical.com>. <daniel.holbach@canonical.com>.
.PP .PP
Both are released under the terms of the GNU General Public License, version 3. Both are released under the terms of the GNU General Public License, version 3.

View File

@ -20,7 +20,7 @@ like for example \fBpbuilder\-feisty\fP, \fBpbuilder\-sid\fP, \fBpbuilder\-gutsy
.PP .PP
The same applies to \fBcowbuilder\-dist\fP, which uses cowbuilder. The main The same applies to \fBcowbuilder\-dist\fP, which uses cowbuilder. The main
difference between both is that pbuilder compresses the created chroot as a difference between both is that pbuilder compresses the created chroot as a
tarball, thus using less disc space but needing to uncompress (and possibly a tarball, thus using less disc space but needing to uncompress (and possibly
compress) its contents again on each run, and cowbuilder doesn't do this. compress) its contents again on each run, and cowbuilder doesn't do this.
.SH USAGE .SH USAGE

View File

@ -1,15 +0,0 @@
.TH running\-autopkgtests "1" "18 January 2024" "ubuntu-dev-tools"
.SH NAME
running\-autopkgtests \- dumps a list of currently running autopkgtests
.SH SYNOPSIS
.B running\-autopkgtests
.SH DESCRIPTION
Dumps a list of currently running and queued tests in Autopkgtest.
Pass --running to only see running tests, or --queued to only see
queued tests. Passing both will print both, which is the default behavior.
.SH AUTHOR
.B running\-autopkgtests
was written by Chris Peterson <chris.peterson@canonical.com>.

View File

@ -58,7 +58,7 @@ Display more progress information.
\fB\-F\fR, \fB\-\-fakesync\fR \fB\-F\fR, \fB\-\-fakesync\fR
Perform a fakesync, to work around a tarball mismatch between Debian and Perform a fakesync, to work around a tarball mismatch between Debian and
Ubuntu. Ubuntu.
This option ignores blocklisting, and performs a local sync. This option ignores blacklisting, and performs a local sync.
It implies \fB\-\-no\-lp\fR, and will leave a signed \fB.changes\fR file It implies \fB\-\-no\-lp\fR, and will leave a signed \fB.changes\fR file
for you to upload. for you to upload.
.TP .TP

View File

@ -29,8 +29,6 @@ import logging
import re import re
import sys import sys
import webbrowser import webbrowser
from collections.abc import Iterable
from email.message import EmailMessage
import debianbts import debianbts
from launchpadlib.launchpad import Launchpad from launchpadlib.launchpad import Launchpad
@ -39,10 +37,11 @@ from ubuntutools import getLogger
from ubuntutools.config import UDTConfig from ubuntutools.config import UDTConfig
Logger = getLogger() Logger = getLogger()
ATTACHMENT_MAX_SIZE = 2000
def parse_args() -> argparse.Namespace: def main():
bug_re = re.compile(r"bug=(\d+)")
parser = argparse.ArgumentParser() parser = argparse.ArgumentParser()
parser.add_argument( parser.add_argument(
"-b", "-b",
@ -72,15 +71,28 @@ def parse_args() -> argparse.Namespace:
"--no-conf", action="store_true", help="Don't read config files or environment variables." "--no-conf", action="store_true", help="Don't read config files or environment variables."
) )
parser.add_argument("bugs", nargs="+", help="Bug number(s) or URL(s)") parser.add_argument("bugs", nargs="+", help="Bug number(s) or URL(s)")
return parser.parse_args() options = parser.parse_args()
config = UDTConfig(options.no_conf)
if options.lpinstance is None:
options.lpinstance = config.get_value("LPINSTANCE")
def get_bug_numbers(bug_list: Iterable[str]) -> list[int]: if options.dry_run:
bug_re = re.compile(r"bug=(\d+)") launchpad = Launchpad.login_anonymously("ubuntu-dev-tools")
options.verbose = True
else:
launchpad = Launchpad.login_with("ubuntu-dev-tools", options.lpinstance)
if options.verbose:
Logger.setLevel(logging.DEBUG)
debian = launchpad.distributions["debian"]
ubuntu = launchpad.distributions["ubuntu"]
lp_debbugs = launchpad.bug_trackers.getByName(name="debbugs")
bug_nums = [] bug_nums = []
for bug_num in bug_list: for bug_num in options.bugs:
if bug_num.startswith("http"): if bug_num.startswith("http"):
# bug URL # bug URL
match = bug_re.search(bug_num) match = bug_re.search(bug_num)
@ -89,81 +101,24 @@ def get_bug_numbers(bug_list: Iterable[str]) -> list[int]:
sys.exit(1) sys.exit(1)
bug_num = match.groups()[0] bug_num = match.groups()[0]
bug_num = bug_num.lstrip("#") bug_num = bug_num.lstrip("#")
bug_nums.append(int(bug_num)) bug_num = int(bug_num)
bug_nums.append(bug_num)
return bug_nums bugs = debianbts.get_status(bug_nums)
if not bugs:
def walk_multipart_message(message: EmailMessage) -> tuple[str, list[tuple[int, EmailMessage]]]: Logger.error("Cannot find any of the listed bugs")
summary = "" sys.exit(1)
attachments = []
i = 1
for part in message.walk():
content_type = part.get_content_type()
if content_type.startswith("multipart/"):
# we're already iterating on multipart items
# let's just skip the multipart extra metadata
continue
if content_type == "application/pgp-signature":
# we're not interested in importing pgp signatures
continue
if part.is_attachment():
attachments.append((i, part))
elif content_type.startswith("image/"):
# images here are not attachment, they are inline, but Launchpad can't handle that,
# so let's add them as attachments
summary += f"Message part #{i}\n"
summary += f"[inline image '{part.get_filename()}']\n\n"
attachments.append((i, part))
elif content_type.startswith("text/html"):
summary += f"Message part #{i}\n"
summary += "[inline html]\n\n"
attachments.append((i, part))
elif content_type == "text/plain":
summary += f"Message part #{i}\n"
summary += part.get_content() + "\n"
else:
raise RuntimeError(
f"""Unknown message part
Your Debian bug is too weird to be imported in Launchpad, sorry.
You can fix that by patching this script in ubuntu-dev-tools.
Faulty message part:
{part}"""
)
i += 1
return summary, attachments
def process_bugs(
bugs: Iterable[debianbts.Bugreport],
launchpad: Launchpad,
package: str,
dry_run: bool = True,
browserless: bool = False,
) -> bool:
debian = launchpad.distributions["debian"]
ubuntu = launchpad.distributions["ubuntu"]
lp_debbugs = launchpad.bug_trackers.getByName(name="debbugs")
err = False err = False
for bug in bugs: for bug in bugs:
ubupackage = bug.source ubupackage = package = bug.source
if package: if options.package:
ubupackage = package ubupackage = options.package
bug_num = bug.bug_num bug_num = bug.bug_num
subject = bug.subject subject = bug.subject
log = debianbts.get_bug_log(bug_num) log = debianbts.get_bug_log(bug_num)
message = log[0]["message"] summary = log[0]["message"].get_payload()
assert isinstance(message, EmailMessage)
attachments: list[tuple[int, EmailMessage]] = []
if message.is_multipart():
summary, attachments = walk_multipart_message(message)
else:
summary = str(message.get_payload())
target = ubuntu.getSourcePackage(name=ubupackage) target = ubuntu.getSourcePackage(name=ubupackage)
if target is None: if target is None:
Logger.error( Logger.error(
@ -182,73 +137,24 @@ def process_bugs(
Logger.debug("Subject: %s", subject) Logger.debug("Subject: %s", subject)
Logger.debug("Description: ") Logger.debug("Description: ")
Logger.debug(description) Logger.debug(description)
for i, attachment in attachments:
Logger.debug("Attachment #%s (%s)", i, attachment.get_filename() or "inline")
Logger.debug("Content:")
if attachment.get_content_type() == "text/plain":
content = attachment.get_content()
if len(content) > ATTACHMENT_MAX_SIZE:
content = (
content[:ATTACHMENT_MAX_SIZE]
+ f" [attachment cropped after {ATTACHMENT_MAX_SIZE} characters...]"
)
Logger.debug(content)
else:
Logger.debug("[data]")
if dry_run: if options.dry_run:
Logger.info("Dry-Run: not creating Ubuntu bug.") Logger.info("Dry-Run: not creating Ubuntu bug.")
continue continue
u_bug = launchpad.bugs.createBug(target=target, title=subject, description=description) u_bug = launchpad.bugs.createBug(target=target, title=subject, description=description)
for i, attachment in attachments:
name = f"#{i}-{attachment.get_filename() or "inline"}"
content = attachment.get_content()
if isinstance(content, str):
# Launchpad only wants bytes
content = content.encode()
u_bug.addAttachment(
filename=name,
data=content,
comment=f"Imported from Debian bug http://bugs.debian.org/{bug_num}",
)
d_sp = debian.getSourcePackage(name=package) d_sp = debian.getSourcePackage(name=package)
if d_sp is None and package: if d_sp is None and options.package:
d_sp = debian.getSourcePackage(name=package) d_sp = debian.getSourcePackage(name=options.package)
d_task = u_bug.addTask(target=d_sp) d_task = u_bug.addTask(target=d_sp)
d_watch = u_bug.addWatch(remote_bug=bug_num, bug_tracker=lp_debbugs) d_watch = u_bug.addWatch(remote_bug=bug_num, bug_tracker=lp_debbugs)
d_task.bug_watch = d_watch d_task.bug_watch = d_watch
d_task.lp_save() d_task.lp_save()
Logger.info("Opened %s", u_bug.web_link) Logger.info("Opened %s", u_bug.web_link)
if not browserless: if not options.browserless:
webbrowser.open(u_bug.web_link) webbrowser.open(u_bug.web_link)
return err if err:
def main() -> None:
options = parse_args()
config = UDTConfig(options.no_conf)
if options.lpinstance is None:
options.lpinstance = config.get_value("LPINSTANCE")
if options.dry_run:
launchpad = Launchpad.login_anonymously("ubuntu-dev-tools")
options.verbose = True
else:
launchpad = Launchpad.login_with("ubuntu-dev-tools", options.lpinstance)
if options.verbose:
Logger.setLevel(logging.DEBUG)
bugs = debianbts.get_status(get_bug_numbers(options.bugs))
if not bugs:
Logger.error("Cannot find any of the listed bugs")
sys.exit(1)
if process_bugs(bugs, launchpad, options.package, options.dry_run, options.browserless):
sys.exit(1) sys.exit(1)

View File

@ -155,7 +155,6 @@ proxy="_unset_"
DEBOOTSTRAP_NO_CHECK_GPG=0 DEBOOTSTRAP_NO_CHECK_GPG=0
EATMYDATA=1 EATMYDATA=1
CCACHE=0 CCACHE=0
USE_PKGBINARYMANGLER=0
while :; do while :; do
case "$1" in case "$1" in
@ -304,27 +303,11 @@ if [ ! -w /var/lib/sbuild ]; then
# Prepare a usable default .sbuildrc # Prepare a usable default .sbuildrc
if [ ! -e ~/.sbuildrc ]; then if [ ! -e ~/.sbuildrc ]; then
cat > ~/.sbuildrc <<EOM cat > ~/.sbuildrc <<EOM
# *** THIS COMMAND IS DEPRECATED ***
#
# In sbuild 0.87.0 and later, the unshare backend is available. This is
# expected to become the default in a future release.
#
# This is the new preferred way of building Debian packages, making the manual
# creation of schroots no longer necessary. To retain the default behavior,
# you may remove this comment block and continue.
#
# To test the unshare backend while retaining the default settings, run sbuild
# with --chroot-mode=unshare like this:
# $ sbuild --chroot-mode=unshare --dist=unstable hello
#
# To switch to the unshare backend by default (recommended), uncomment the
# following lines and delete the rest of the file (with the exception of the
# last two lines):
#\$chroot_mode = 'unshare';
#\$unshare_mmdebstrap_keep_tarball = 1;
# *** VERIFY AND UPDATE \$mailto and \$maintainer_name BELOW *** # *** VERIFY AND UPDATE \$mailto and \$maintainer_name BELOW ***
# Mail address where logs are sent to (mandatory, no default!)
\$mailto = '$USER';
# Name to use as override in .changes files for the Maintainer: field # Name to use as override in .changes files for the Maintainer: field
#\$maintainer_name='$USER <$USER@localhost>'; #\$maintainer_name='$USER <$USER@localhost>';
@ -668,7 +651,6 @@ ubuntu)
if ubuntu_dist_ge "$RELEASE" "edgy"; then if ubuntu_dist_ge "$RELEASE" "edgy"; then
# Add pkgbinarymangler (edgy and later) # Add pkgbinarymangler (edgy and later)
BUILD_PKGS="$BUILD_PKGS pkgbinarymangler" BUILD_PKGS="$BUILD_PKGS pkgbinarymangler"
USE_PKGBINARYMANGLER=1
# Disable recommends for a smaller chroot (gutsy and later only) # Disable recommends for a smaller chroot (gutsy and later only)
if ubuntu_dist_ge "$RELEASE" "gutsy"; then if ubuntu_dist_ge "$RELEASE" "gutsy"; then
BUILD_PKGS="--no-install-recommends $BUILD_PKGS" BUILD_PKGS="--no-install-recommends $BUILD_PKGS"
@ -928,8 +910,8 @@ if [ -n "$TEMP_PREFERENCES" ]; then
sudo mv "$TEMP_PREFERENCES" $MNT/etc/apt/preferences.d/proposed.pref sudo mv "$TEMP_PREFERENCES" $MNT/etc/apt/preferences.d/proposed.pref
fi fi
# Copy the timezone (uncomment this if you want to use your local time zone) # Copy the timezone (comment this out if you want to leave the chroot at UTC)
#sudo cp -P --remove-destination /etc/localtime /etc/timezone "$MNT"/etc/ sudo cp -P --remove-destination /etc/localtime /etc/timezone "$MNT"/etc/
# Create a schroot entry for this chroot # Create a schroot entry for this chroot
TEMP_SCHROOTCONF=`mktemp -t schrootconf-XXXXXX` TEMP_SCHROOTCONF=`mktemp -t schrootconf-XXXXXX`
TEMPLATE_SCHROOTCONF=~/.mk-sbuild.schroot.conf TEMPLATE_SCHROOTCONF=~/.mk-sbuild.schroot.conf
@ -1048,25 +1030,6 @@ EOF
EOM EOM
fi fi
if [ "$USE_PKGBINARYMANGLER" = 1 ]; then
sudo bash -c "cat >> $MNT/finish.sh" <<EOM
mkdir -p /etc/pkgbinarymangler/
cat > /etc/pkgbinarymangler/maintainermangler.conf <<EOF
# pkgmaintainermangler configuration file
# pkgmaintainermangler will do nothing unless enable is set to "true"
enable: true
# Configure what happens if /CurrentlyBuilding is present, but invalid
# (i. e. it does not contain a Package: field). If "ignore" (default),
# the file is ignored (i. e. the Maintainer field is mangled) and a
# warning is printed. If "fail" (or any other value), pkgmaintainermangler
# exits with an error, which causes a package build to fail.
invalid_currentlybuilding: ignore
EOF
EOM
fi
if [ -n "$TARGET_ARCH" ]; then if [ -n "$TARGET_ARCH" ]; then
sudo bash -c "cat >> $MNT/finish.sh" <<EOM sudo bash -c "cat >> $MNT/finish.sh" <<EOM
# Configure target architecture # Configure target architecture
@ -1085,7 +1048,7 @@ apt-get update || true
echo set debconf/frontend Noninteractive | debconf-communicate echo set debconf/frontend Noninteractive | debconf-communicate
echo set debconf/priority critical | debconf-communicate echo set debconf/priority critical | debconf-communicate
# Install basic build tool set, trying to match buildd # Install basic build tool set, trying to match buildd
apt-get -y --force-yes -o Dpkg::Options::="--force-confold" install $BUILD_PKGS apt-get -y --force-yes install $BUILD_PKGS
# Set up expected /dev entries # Set up expected /dev entries
if [ ! -r /dev/stdin ]; then ln -s /proc/self/fd/0 /dev/stdin; fi if [ ! -r /dev/stdin ]; then ln -s /proc/self/fd/0 /dev/stdin; fi
if [ ! -r /dev/stdout ]; then ln -s /proc/self/fd/1 /dev/stdout; fi if [ ! -r /dev/stdout ]; then ln -s /proc/self/fd/1 /dev/stdout; fi

View File

@ -294,9 +294,7 @@ class PbuilderDist:
if self.target_distro in self._debian_distros: if self.target_distro in self._debian_distros:
try: try:
codename = self.debian_distro_info.codename( codename = self.debian_distro_info.codename(self.target_distro, default=self.target_distro)
self.target_distro, default=self.target_distro
)
except DistroDataOutdated as error: except DistroDataOutdated as error:
Logger.warning(error) Logger.warning(error)
if codename in (self.debian_distro_info.devel(), "experimental"): if codename in (self.debian_distro_info.devel(), "experimental"):

View File

@ -15,51 +15,53 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import lzma import lzma
from argparse import ArgumentParser
import sys import sys
import webbrowser import webbrowser
from argparse import ArgumentParser
import yaml import yaml
from launchpadlib.launchpad import Launchpad from launchpadlib.launchpad import Launchpad
from ubuntutools.utils import get_url from ubuntutools.utils import get_url
# proposed-migration is only concerned with the devel series; unlike other # proposed-migration is only concerned with the devel series; unlike other
# tools, don't make this configurable # tools, don't make this configurable
excuses_url = "https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.yaml.xz" excuses_url = 'https://ubuntu-archive-team.ubuntu.com/proposed-migration/' \
+ 'update_excuses.yaml.xz'
def get_proposed_version(excuses, package): def get_proposed_version(excuses, package):
for k in excuses["sources"]: for k in excuses['sources']:
if k["source"] == package: if k['source'] == package:
return k.get("new-version") return k.get('new-version')
return None return None
def claim_excuses_bug(launchpad, bug, package): def claim_excuses_bug(launchpad, bug, package):
print(f"LP: #{bug.id}: {bug.title}") print("LP: #%d: %s" % (bug.id, bug.title))
ubuntu = launchpad.distributions["ubuntu"] ubuntu = launchpad.distributions['ubuntu']
series = ubuntu.current_series.fullseriesname series = ubuntu.current_series.fullseriesname
for task in bug.bug_tasks: for task in bug.bug_tasks:
# targeting to a series doesn't make the default task disappear, # targeting to a series doesn't make the default task disappear,
# it just makes it useless # it just makes it useless
if task.bug_target_name == f"{package} ({series})": if task.bug_target_name == "%s (%s)" % (package, series):
our_task = task our_task = task
break break
if task.bug_target_name == f"{package} (Ubuntu)": elif task.bug_target_name == "%s (Ubuntu)" % package:
our_task = task our_task = task
if our_task.assignee == launchpad.me: if our_task.assignee == launchpad.me:
print("Bug already assigned to you.") print("Bug already assigned to you.")
return True return True
if our_task.assignee: elif our_task.assignee:
print(f"Currently assigned to {our_task.assignee.name}") print("Currently assigned to %s" % our_task.assignee.name)
print("""Do you want to claim this bug? [yN] """, end="") print('''Do you want to claim this bug? [yN] ''', end="")
sys.stdout.flush() sys.stdout.flush()
response = sys.stdin.readline() response = sys.stdin.readline()
if response.strip().lower().startswith("y"): if response.strip().lower().startswith('y'):
our_task.assignee = launchpad.me our_task.assignee = launchpad.me
our_task.lp_save() our_task.lp_save()
return True return True
@ -70,37 +72,38 @@ def claim_excuses_bug(launchpad, bug, package):
def create_excuses_bug(launchpad, package, version): def create_excuses_bug(launchpad, package, version):
print("Will open a new bug") print("Will open a new bug")
bug = launchpad.bugs.createBug( bug = launchpad.bugs.createBug(
title=f"proposed-migration for {package} {version}", title = 'proposed-migration for %s %s' % (package, version),
tags=("update-excuse"), tags = ('update-excuse'),
target=f"https://api.launchpad.net/devel/ubuntu/+source/{package}", target = 'https://api.launchpad.net/devel/ubuntu/+source/%s' % package,
description=f"{package} {version} is stuck in -proposed.", description = '%s %s is stuck in -proposed.' % (package, version)
) )
task = bug.bug_tasks[0] task = bug.bug_tasks[0]
task.assignee = launchpad.me task.assignee = launchpad.me
task.lp_save() task.lp_save()
print(f"Opening {bug.web_link} in browser") print("Opening %s in browser" % bug.web_link)
webbrowser.open(bug.web_link) webbrowser.open(bug.web_link)
return bug return bug
def has_excuses_bugs(launchpad, package): def has_excuses_bugs(launchpad, package):
ubuntu = launchpad.distributions["ubuntu"] ubuntu = launchpad.distributions['ubuntu']
pkg = ubuntu.getSourcePackage(name=package) pkg = ubuntu.getSourcePackage(name=package)
if not pkg: if not pkg:
raise ValueError(f"No such source package: {package}") raise ValueError(f"No such source package: {package}")
tasks = pkg.searchTasks(tags=["update-excuse"], order_by=["id"]) tasks = pkg.searchTasks(tags=['update-excuse'], order_by=['id'])
bugs = [task.bug for task in tasks] bugs = [task.bug for task in tasks]
if not bugs: if not bugs:
return False return False
if len(bugs) == 1: if len(bugs) == 1:
print(f"There is 1 open update-excuse bug against {package}") print("There is 1 open update-excuse bug against %s" % package)
else: else:
print(f"There are {len(bugs)} open update-excuse bugs against {package}") print("There are %d open update-excuse bugs against %s" \
% (len(bugs), package))
for bug in bugs: for bug in bugs:
if claim_excuses_bug(launchpad, bug, package): if claim_excuses_bug(launchpad, bug, package):
@ -111,14 +114,17 @@ def has_excuses_bugs(launchpad, package):
def main(): def main():
parser = ArgumentParser() parser = ArgumentParser()
parser.add_argument("-l", "--launchpad", dest="launchpad_instance", default="production")
parser.add_argument( parser.add_argument(
"-v", "--verbose", default=False, action="store_true", help="be more verbose" "-l", "--launchpad", dest="launchpad_instance", default="production")
) parser.add_argument(
parser.add_argument("package", nargs="?", help="act on this package only") "-v", "--verbose", default=False, action="store_true",
help="be more verbose")
parser.add_argument(
'package', nargs='?', help="act on this package only")
args = parser.parse_args() args = parser.parse_args()
args.launchpad = Launchpad.login_with("pm-helper", args.launchpad_instance, version="devel") args.launchpad = Launchpad.login_with(
"pm-helper", args.launchpad_instance, version="devel")
f = get_url(excuses_url, False) f = get_url(excuses_url, False)
with lzma.open(f) as lzma_f: with lzma.open(f) as lzma_f:
@ -129,14 +135,15 @@ def main():
if not has_excuses_bugs(args.launchpad, args.package): if not has_excuses_bugs(args.launchpad, args.package):
proposed_version = get_proposed_version(excuses, args.package) proposed_version = get_proposed_version(excuses, args.package)
if not proposed_version: if not proposed_version:
print(f"Package {args.package} not found in -proposed.") print("Package %s not found in -proposed." % args.package)
sys.exit(1) sys.exit(1)
create_excuses_bug(args.launchpad, args.package, proposed_version) create_excuses_bug(args.launchpad, args.package,
proposed_version)
except ValueError as e: except ValueError as e:
sys.stderr.write(f"{e}\n") sys.stderr.write(f"{e}\n")
else: else:
pass # for now pass # for now
if __name__ == "__main__": if __name__ == '__main__':
sys.exit(main()) sys.exit(main())

View File

@ -183,7 +183,7 @@ def display_verbose(package, values):
Logger.info("No reverse dependencies found") Logger.info("No reverse dependencies found")
return return
def log_package(values, package, arch, dependency, visited, offset=0): def log_package(values, package, arch, dependency, offset=0):
line = f"{' ' * offset}* {package}" line = f"{' ' * offset}* {package}"
if all_archs and set(arch) != all_archs: if all_archs and set(arch) != all_archs:
line += f" [{' '.join(sorted(arch))}]" line += f" [{' '.join(sorted(arch))}]"
@ -192,9 +192,6 @@ def display_verbose(package, values):
line += " " * (30 - len(line)) line += " " * (30 - len(line))
line += f" (for {dependency})" line += f" (for {dependency})"
Logger.info(line) Logger.info(line)
if package in visited:
return
visited = visited.copy().add(package)
data = values.get(package) data = values.get(package)
if data: if data:
offset = offset + 1 offset = offset + 1
@ -205,7 +202,6 @@ def display_verbose(package, values):
rdep["Package"], rdep["Package"],
rdep.get("Architectures", all_archs), rdep.get("Architectures", all_archs),
rdep.get("Dependency"), rdep.get("Dependency"),
visited,
offset, offset,
) )
@ -227,7 +223,6 @@ def display_verbose(package, values):
rdep["Package"], rdep["Package"],
rdep.get("Architectures", all_archs), rdep.get("Architectures", all_archs),
rdep.get("Dependency"), rdep.get("Dependency"),
{package},
) )
Logger.info("") Logger.info("")

View File

@ -4,31 +4,13 @@
# Authors: # Authors:
# Andy P. Whitcroft # Andy P. Whitcroft
# Christian Ehrhardt # Christian Ehrhardt
# Chris Peterson <chris.peterson@canonical.com>
#
# Copyright (C) 2024 Canonical Ltd.
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 3, as published
# by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranties of
# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
"""Dumps a list of currently running tests in Autopkgtest""" """Dumps a list of currently running tests in Autopkgtest"""
__example__ = """ __example__ = """
Display first listed test running on amd64 hardware: Display first listed test running on amd64 hardware:
$ running-autopkgtests | grep amd64 | head -n1 $ running-autopkgtests | grep amd64 | head -n1
R 0:01:40 systemd-upstream - focal amd64\ R 0:01:40 systemd-upstream - focal amd64 upstream-systemd-ci/systemd-ci - ['CFLAGS=-O0', 'DEB_BUILD_PROFILES=noudeb', 'TEST_UPSTREAM=1', 'CONFFLAGS_UPSTREAM=--werror -Dslow-tests=true', 'UPSTREAM_PULL_REQUEST=23153', 'GITHUB_STATUSES_URL=https://api.github.com/repos/systemd/systemd/statuses/cfb0935923dff8050315b5dd22ce8ab06461ff0e']
upstream-systemd-ci/systemd-ci - ['CFLAGS=-O0', 'DEB_BUILD_PROFILES=noudeb',\
'TEST_UPSTREAM=1', 'CONFFLAGS_UPSTREAM=--werror -Dslow-tests=true',\
'UPSTREAM_PULL_REQUEST=23153',\
'GITHUB_STATUSES_URL=https://api.github.com/repos/\
systemd/systemd/statuses/cfb0935923dff8050315b5dd22ce8ab06461ff0e']
""" """
import sys import sys
@ -51,10 +33,16 @@ def parse_args():
formatter_class=RawDescriptionHelpFormatter, formatter_class=RawDescriptionHelpFormatter,
) )
parser.add_argument( parser.add_argument(
"-r", "--running", action="store_true", help="Print runnning autopkgtests (default: true)" "-r",
"--running",
action="store_true",
help="Print runnning autopkgtests (default: true)",
) )
parser.add_argument( parser.add_argument(
"-q", "--queued", action="store_true", help="Print queued autopkgtests (default: true)" "-q",
"--queued",
action="store_true",
help="Print queued autopkgtests (default: true)",
) )
options = parser.parse_args() options = parser.parse_args()

View File

@ -32,18 +32,17 @@ def make_pep440_compliant(version: str) -> str:
scripts = [ scripts = [
"backportpackage", "backportpackage",
"bitesize",
"check-mir", "check-mir",
"check-symbols", "check-symbols",
"dch-repeat", "dch-repeat",
"grab-merge", "grab-merge",
"grep-merges", "grep-merges",
"import-bug-from-debian", "import-bug-from-debian",
"lp-bitesize",
"merge-changelog", "merge-changelog",
"mk-sbuild", "mk-sbuild",
"pbuilder-dist", "pbuilder-dist",
"pbuilder-dist-simple", "pbuilder-dist-simple",
"pm-helper",
"pull-pkg", "pull-pkg",
"pull-debian-debdiff", "pull-debian-debdiff",
"pull-debian-source", "pull-debian-source",

View File

@ -49,7 +49,6 @@ from ubuntutools.requestsync.mail import get_debian_srcpkg as requestsync_mail_g
from ubuntutools.version import Version from ubuntutools.version import Version
Logger = getLogger() Logger = getLogger()
cached_sync_blocklist = None
def remove_signature(dscname): def remove_signature(dscname):
@ -144,7 +143,7 @@ def sync_dsc(
if ubuntu_ver.is_modified_in_ubuntu(): if ubuntu_ver.is_modified_in_ubuntu():
if not force: if not force:
Logger.error("--force is required to discard Ubuntu changes.") Logger.error("--force is required to discard Ubuntu changes.")
return None sys.exit(1)
Logger.warning( Logger.warning(
"Overwriting modified Ubuntu version %s, setting current version to %s", "Overwriting modified Ubuntu version %s, setting current version to %s",
@ -158,7 +157,7 @@ def sync_dsc(
src_pkg.pull() src_pkg.pull()
except DownloadError as e: except DownloadError as e:
Logger.error("Failed to download: %s", str(e)) Logger.error("Failed to download: %s", str(e))
return None sys.exit(1)
src_pkg.unpack() src_pkg.unpack()
needs_fakesync = not (need_orig or ubu_pkg.verify_orig()) needs_fakesync = not (need_orig or ubu_pkg.verify_orig())
@ -167,13 +166,13 @@ def sync_dsc(
Logger.warning("Performing a fakesync") Logger.warning("Performing a fakesync")
elif not needs_fakesync and fakesync: elif not needs_fakesync and fakesync:
Logger.error("Fakesync not required, aborting.") Logger.error("Fakesync not required, aborting.")
return None sys.exit(1)
elif needs_fakesync and not fakesync: elif needs_fakesync and not fakesync:
Logger.error( Logger.error(
"The checksums of the Debian and Ubuntu packages " "The checksums of the Debian and Ubuntu packages "
"mismatch. A fake sync using --fakesync is required." "mismatch. A fake sync using --fakesync is required."
) )
return None sys.exit(1)
if fakesync: if fakesync:
# Download Ubuntu files (override Debian source tarballs) # Download Ubuntu files (override Debian source tarballs)
@ -181,7 +180,7 @@ def sync_dsc(
ubu_pkg.pull() ubu_pkg.pull()
except DownloadError as e: except DownloadError as e:
Logger.error("Failed to download: %s", str(e)) Logger.error("Failed to download: %s", str(e))
return None sys.exit(1)
# change into package directory # change into package directory
directory = src_pkg.source + "-" + new_ver.upstream_version directory = src_pkg.source + "-" + new_ver.upstream_version
@ -266,7 +265,7 @@ def sync_dsc(
returncode = subprocess.call(cmd) returncode = subprocess.call(cmd)
if returncode != 0: if returncode != 0:
Logger.error("Source-only build with debuild failed. Please check build log above.") Logger.error("Source-only build with debuild failed. Please check build log above.")
return None sys.exit(1)
def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror): def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
@ -296,7 +295,7 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
udtexceptions.SeriesNotFoundException, udtexceptions.SeriesNotFoundException,
) as e: ) as e:
Logger.error(str(e)) Logger.error(str(e))
return None sys.exit(1)
if version is None: if version is None:
version = Version(debian_srcpkg.getVersion()) version = Version(debian_srcpkg.getVersion())
try: try:
@ -307,7 +306,7 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
ubuntu_version = Version("~") ubuntu_version = Version("~")
except udtexceptions.SeriesNotFoundException as e: except udtexceptions.SeriesNotFoundException as e:
Logger.error(str(e)) Logger.error(str(e))
return None sys.exit(1)
if ubuntu_version >= version: if ubuntu_version >= version:
# The LP importer is maybe out of date # The LP importer is maybe out of date
debian_srcpkg = requestsync_mail_get_debian_srcpkg(package, dist) debian_srcpkg = requestsync_mail_get_debian_srcpkg(package, dist)
@ -321,7 +320,7 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
ubuntu_version, ubuntu_version,
ubuntu_release, ubuntu_release,
) )
return None sys.exit(1)
if component is None: if component is None:
component = debian_srcpkg.getComponent() component = debian_srcpkg.getComponent()
@ -330,7 +329,7 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
return DebianSourcePackage(package, version.full_version, component, mirrors=mirrors) return DebianSourcePackage(package, version.full_version, component, mirrors=mirrors)
def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False, yes=False): def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False):
"""Copy a source package from Debian to Ubuntu using the Launchpad API.""" """Copy a source package from Debian to Ubuntu using the Launchpad API."""
ubuntu = Distribution("ubuntu") ubuntu = Distribution("ubuntu")
debian_archive = Distribution("debian").getArchive() debian_archive = Distribution("debian").getArchive()
@ -353,7 +352,7 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False, ye
"Debian version %s has not been picked up by LP yet. Please try again later.", "Debian version %s has not been picked up by LP yet. Please try again later.",
src_pkg.version, src_pkg.version,
) )
return None sys.exit(1)
try: try:
ubuntu_spph = get_ubuntu_srcpkg(src_pkg.source, ubuntu_series, ubuntu_pocket) ubuntu_spph = get_ubuntu_srcpkg(src_pkg.source, ubuntu_series, ubuntu_pocket)
@ -374,7 +373,7 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False, ye
base_version = ubuntu_version.get_related_debian_version() base_version = ubuntu_version.get_related_debian_version()
if not force and ubuntu_version.is_modified_in_ubuntu(): if not force and ubuntu_version.is_modified_in_ubuntu():
Logger.error("--force is required to discard Ubuntu changes.") Logger.error("--force is required to discard Ubuntu changes.")
return None sys.exit(1)
# Check whether a fakesync would be required. # Check whether a fakesync would be required.
if not src_pkg.dsc.compare_dsc(ubuntu_pkg.dsc): if not src_pkg.dsc.compare_dsc(ubuntu_pkg.dsc):
@ -382,7 +381,7 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False, ye
"The checksums of the Debian and Ubuntu packages " "The checksums of the Debian and Ubuntu packages "
"mismatch. A fake sync using --fakesync is required." "mismatch. A fake sync using --fakesync is required."
) )
return None sys.exit(1)
except udtexceptions.PackageNotFoundException: except udtexceptions.PackageNotFoundException:
base_version = Version("~") base_version = Version("~")
Logger.info( Logger.info(
@ -403,10 +402,9 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False, ye
if sponsoree: if sponsoree:
Logger.info("Sponsoring this sync for %s (%s)", sponsoree.display_name, sponsoree.name) Logger.info("Sponsoring this sync for %s (%s)", sponsoree.display_name, sponsoree.name)
if not yes: answer = YesNoQuestion().ask("Sync this package", "no")
answer = YesNoQuestion().ask("Sync this package", "no") if answer != "yes":
if answer != "yes": return
return
try: try:
ubuntu_archive.copyPackage( ubuntu_archive.copyPackage(
@ -421,29 +419,26 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False, ye
except HTTPError as error: except HTTPError as error:
Logger.error("HTTP Error %s: %s", error.response.status, error.response.reason) Logger.error("HTTP Error %s: %s", error.response.status, error.response.reason)
Logger.error(error.content) Logger.error(error.content)
return None sys.exit(1)
Logger.info("Request succeeded; you should get an e-mail once it is processed.") Logger.info("Request succeeded; you should get an e-mail once it is processed.")
bugs = sorted(set(bugs)) bugs = sorted(set(bugs))
if bugs: if bugs:
Logger.info("Launchpad bugs to be closed: %s", ", ".join(str(bug) for bug in bugs)) Logger.info("Launchpad bugs to be closed: %s", ", ".join(str(bug) for bug in bugs))
Logger.info("Please wait for the sync to be successful before closing bugs.") Logger.info("Please wait for the sync to be successful before closing bugs.")
if yes: answer = YesNoQuestion().ask("Close bugs", "yes")
if answer == "yes":
close_bugs(bugs, src_pkg.source, src_pkg.version.full_version, changes, sponsoree) close_bugs(bugs, src_pkg.source, src_pkg.version.full_version, changes, sponsoree)
else:
answer = YesNoQuestion().ask("Close bugs", "yes")
if answer == "yes":
close_bugs(bugs, src_pkg.source, src_pkg.version.full_version, changes, sponsoree)
def is_blocklisted(query): def is_blacklisted(query):
"""Determine if package "query" is in the sync blocklist """Determine if package "query" is in the sync blacklist
Returns tuple of (blocklisted, comments) Returns tuple of (blacklisted, comments)
blocklisted is one of False, 'CURRENT', 'ALWAYS' blacklisted is one of False, 'CURRENT', 'ALWAYS'
""" """
series = Launchpad.distributions["ubuntu"].current_series series = Launchpad.distributions["ubuntu"].current_series
lp_comments = series.getDifferenceComments(source_package_name=query) lp_comments = series.getDifferenceComments(source_package_name=query)
blocklisted = False blacklisted = False
comments = [ comments = [
f"{c.body_text}\n -- {c.comment_author.name}" f"{c.body_text}\n -- {c.comment_author.name}"
f" {c.comment_date.strftime('%a, %d %b %Y %H:%M:%S +0000')}" f" {c.comment_date.strftime('%a, %d %b %Y %H:%M:%S +0000')}"
@ -451,38 +446,32 @@ def is_blocklisted(query):
] ]
for diff in series.getDifferencesTo(source_package_name_filter=query): for diff in series.getDifferencesTo(source_package_name_filter=query):
if diff.status == "Blacklisted current version" and blocklisted != "ALWAYS": if diff.status == "Blacklisted current version" and blacklisted != "ALWAYS":
blocklisted = "CURRENT" blacklisted = "CURRENT"
if diff.status == "Blacklisted always": if diff.status == "Blacklisted always":
blocklisted = "ALWAYS" blacklisted = "ALWAYS"
global cached_sync_blocklist # Old blacklist:
if not cached_sync_blocklist: url = "https://ubuntu-archive-team.ubuntu.com/sync-blacklist.txt"
url = "https://ubuntu-archive-team.ubuntu.com/sync-blocklist.txt" with urllib.request.urlopen(url) as f:
try: applicable_lines = []
with urllib.request.urlopen(url) as f: for line in f:
cached_sync_blocklist = f.read().decode("utf-8") line = line.decode("utf-8")
except: if not line.strip():
print("WARNING: unable to download the sync blocklist. Erring on the side of caution.") applicable_lines = []
return ("ALWAYS", "INTERNAL ERROR: Unable to fetch sync blocklist") continue
applicable_lines.append(line)
try:
line = line[: line.index("#")]
except ValueError:
pass
source = line.strip()
if source and fnmatch.fnmatch(query, source):
comments += ["From sync-blacklist.txt:"] + applicable_lines
blacklisted = "ALWAYS"
break
applicable_lines = [] return (blacklisted, comments)
for line in cached_sync_blocklist.splitlines():
if not line.strip():
applicable_lines = []
continue
applicable_lines.append(line)
try:
line = line[:line.index("#")]
except ValueError:
pass
source = line.strip()
if source and fnmatch.fnmatch(query, source):
comments += ["From sync-blocklist.txt:"] + applicable_lines
blocklisted = "ALWAYS"
break
return (blocklisted, comments)
def close_bugs(bugs, package, version, changes, sponsoree): def close_bugs(bugs, package, version, changes, sponsoree):
@ -519,12 +508,6 @@ def parse():
epilog = f"See {os.path.basename(sys.argv[0])}(1) for more info." epilog = f"See {os.path.basename(sys.argv[0])}(1) for more info."
parser = argparse.ArgumentParser(usage=usage, epilog=epilog) parser = argparse.ArgumentParser(usage=usage, epilog=epilog)
parser.add_argument(
"-y",
"--yes",
action="store_true",
help="Automatically sync without prompting. Use with caution and care."
)
parser.add_argument("-d", "--distribution", help="Debian distribution to sync from.") parser.add_argument("-d", "--distribution", help="Debian distribution to sync from.")
parser.add_argument("-r", "--release", help="Specify target Ubuntu release.") parser.add_argument("-r", "--release", help="Specify target Ubuntu release.")
parser.add_argument("-V", "--debian-version", help="Specify the version to sync from.") parser.add_argument("-V", "--debian-version", help="Specify the version to sync from.")
@ -729,38 +712,36 @@ def main():
args.release, args.release,
args.debian_mirror, args.debian_mirror,
) )
if not src_pkg:
continue
blocklisted, comments = is_blocklisted(src_pkg.source) blacklisted, comments = is_blacklisted(src_pkg.source)
blocklist_fail = False blacklist_fail = False
if blocklisted: if blacklisted:
messages = [] messages = []
if blocklisted == "CURRENT": if blacklisted == "CURRENT":
Logger.debug( Logger.debug(
"Source package %s is temporarily blocklisted " "Source package %s is temporarily blacklisted "
"(blocklisted_current). " "(blacklisted_current). "
"Ubuntu ignores these for now. " "Ubuntu ignores these for now. "
"See also LP: #841372", "See also LP: #841372",
src_pkg.source, src_pkg.source,
) )
else: else:
if args.fakesync: if args.fakesync:
messages += ["Doing a fakesync, overriding blocklist."] messages += ["Doing a fakesync, overriding blacklist."]
else: else:
blocklist_fail = True blacklist_fail = True
messages += [ messages += [
"If this package needs a fakesync, use --fakesync", "If this package needs a fakesync, use --fakesync",
"If you think this package shouldn't be " "If you think this package shouldn't be "
"blocklisted, please file a bug explaining your " "blacklisted, please file a bug explaining your "
"reasoning and subscribe ~ubuntu-archive.", "reasoning and subscribe ~ubuntu-archive.",
] ]
if blocklist_fail: if blacklist_fail:
Logger.error("Source package %s is blocklisted.", src_pkg.source) Logger.error("Source package %s is blacklisted.", src_pkg.source)
elif blocklisted == "ALWAYS": elif blacklisted == "ALWAYS":
Logger.info("Source package %s is blocklisted.", src_pkg.source) Logger.info("Source package %s is blacklisted.", src_pkg.source)
if messages: if messages:
for message in messages: for message in messages:
for line in textwrap.wrap(message): for line in textwrap.wrap(message):
@ -772,15 +753,14 @@ def main():
for line in textwrap.wrap(comment): for line in textwrap.wrap(comment):
Logger.info(" %s", line) Logger.info(" %s", line)
if blocklist_fail: if blacklist_fail:
continue sys.exit(1)
if args.lp: if args.lp:
if not copy(src_pkg, args.release, args.bugs, sponsoree, args.simulate, args.force, args.yes): copy(src_pkg, args.release, args.bugs, sponsoree, args.simulate, args.force)
continue
else: else:
os.environ["DEB_VENDOR"] = "Ubuntu" os.environ["DEB_VENDOR"] = "Ubuntu"
if not sync_dsc( sync_dsc(
src_pkg, src_pkg,
args.distribution, args.distribution,
args.release, args.release,
@ -792,8 +772,7 @@ def main():
args.simulate, args.simulate,
args.force, args.force,
args.fakesync, args.fakesync,
): )
continue
if __name__ == "__main__": if __name__ == "__main__":

View File

@ -28,8 +28,9 @@
import argparse import argparse
import sys import sys
import lazr.restfulclient.errors from launchpadlib.credentials import TokenAuthorizationException
from launchpadlib.launchpad import Launchpad from launchpadlib.launchpad import Launchpad
import lazr.restfulclient.errors
from ubuntutools import getLogger from ubuntutools import getLogger
from ubuntutools.lp.udtexceptions import PocketDoesNotExistError from ubuntutools.lp.udtexceptions import PocketDoesNotExistError
@ -38,7 +39,7 @@ from ubuntutools.misc import split_release_pocket
Logger = getLogger() Logger = getLogger()
def get_build_states(pkg, archs): def getBuildStates(pkg, archs):
res = [] res = []
for build in pkg.getBuilds(): for build in pkg.getBuilds():
@ -47,8 +48,7 @@ def get_build_states(pkg, archs):
msg = "\n".join(res) msg = "\n".join(res)
return f"Build state(s) for '{pkg.source_package_name}':\n{msg}" return f"Build state(s) for '{pkg.source_package_name}':\n{msg}"
def rescoreBuilds(pkg, archs, score):
def rescore_builds(pkg, archs, score):
res = [] res = []
for build in pkg.getBuilds(): for build in pkg.getBuilds():
@ -61,19 +61,18 @@ def rescore_builds(pkg, archs, score):
res.append(f" {arch}: done") res.append(f" {arch}: done")
except lazr.restfulclient.errors.Unauthorized: except lazr.restfulclient.errors.Unauthorized:
Logger.error( Logger.error(
"You don't have the permissions to rescore builds." "You don't have the permissions to rescore builds. Ignoring your rescore request."
" Ignoring your rescore request."
) )
return None return None
except lazr.restfulclient.errors.BadRequest: except lazr.restfulclient.errors.BadRequest:
Logger.info("Cannot rescore build of %s on %s.", build.source_package_name, arch) Logger.info("Cannot rescore build of %s on %s.",
build.source_package_name, arch)
res.append(f" {arch}: failed") res.append(f" {arch}: failed")
msg = "\n".join(res) msg = "\n".join(res)
return f"Rescoring builds of '{pkg.source_package_name}' to {score}:\n{msg}" return f"Rescoring builds of '{pkg.source_package_name}' to {score}:\n{msg}"
def retryBuilds(pkg, archs):
def retry_builds(pkg, archs):
res = [] res = []
for build in pkg.getBuilds(): for build in pkg.getBuilds():
arch = build.arch_tag arch = build.arch_tag
@ -95,7 +94,16 @@ def main():
# Valid architectures. # Valid architectures.
valid_archs = set( valid_archs = set(
["armhf", "arm64", "amd64", "i386", "powerpc", "ppc64el", "riscv64", "s390x"] [
"armhf",
"arm64",
"amd64",
"i386",
"powerpc",
"ppc64el",
"riscv64",
"s390x",
]
) )
# Prepare our option parser. # Prepare our option parser.
@ -110,7 +118,8 @@ def main():
f"include: {', '.join(valid_archs)}.", f"include: {', '.join(valid_archs)}.",
) )
parser.add_argument("-A", "--archive", help="operate on ARCHIVE", default="ubuntu") parser.add_argument("-A", "--archive", help="operate on ARCHIVE",
default="ubuntu")
# Batch processing options # Batch processing options
batch_options = parser.add_argument_group( batch_options = parser.add_argument_group(
@ -139,9 +148,7 @@ def main():
help="Rescore builds to <priority>.", help="Rescore builds to <priority>.",
) )
batch_options.add_argument( batch_options.add_argument(
"--state", "--state", action="store", dest="state",
action="store",
dest="state",
help="Act on builds that are in the specified state", help="Act on builds that are in the specified state",
) )
@ -150,8 +157,11 @@ def main():
# Parse our options. # Parse our options.
args = parser.parse_args() args = parser.parse_args()
launchpad = Launchpad.login_with("ubuntu-dev-tools", "production", version="devel") launchpad = Launchpad.login_with("ubuntu-dev-tools", "production",
ubuntu = launchpad.distributions["ubuntu"] version="devel")
me = launchpad.me
ubuntu = launchpad.distributions['ubuntu']
if args.batch: if args.batch:
release = args.series release = args.series
@ -159,8 +169,8 @@ def main():
# ppas don't have a proposed pocket so just use the release pocket; # ppas don't have a proposed pocket so just use the release pocket;
# but for the main archive we default to -proposed # but for the main archive we default to -proposed
release = ubuntu.getDevelopmentSeries()[0].name release = ubuntu.getDevelopmentSeries()[0].name
if args.archive == "ubuntu": if args.archive == 'ubuntu':
release = f"{release}-proposed" release = release + "-proposed"
try: try:
(release, pocket) = split_release_pocket(release) (release, pocket) = split_release_pocket(release)
except PocketDoesNotExistError as error: except PocketDoesNotExistError as error:
@ -213,13 +223,12 @@ def main():
# Get list of published sources for package in question. # Get list of published sources for package in question.
try: try:
sources = archive.getPublishedSources( sources = archive.getPublishedSources(
distro_series=distroseries, distro_series=distroseries,
exact_match=True, exact_match=True,
pocket=pocket, pocket=pocket,
source_name=package, source_name=package,
status="Published", status='Published')[0]
)[0] except IndexError as error:
except IndexError:
Logger.error("No publication found for package %s", package) Logger.error("No publication found for package %s", package)
sys.exit(1) sys.exit(1)
# Get list of builds for that package. # Get list of builds for that package.
@ -234,20 +243,21 @@ def main():
# are in place. # are in place.
if operation == "retry": if operation == "retry":
necessary_privs = archive.checkUpload( necessary_privs = archive.checkUpload(
component=sources.getComponent(), component=sources.getComponent(),
distroseries=distroseries, distroseries=distroseries,
person=launchpad.me, person=launchpad.me,
pocket=pocket, pocket=pocket,
sourcepackagename=sources.getPackageName(), sourcepackagename=sources.getPackageName(),
) )
if not necessary_privs:
Logger.error( if operation == "retry" and not necessary_privs:
"You cannot perform the %s operation on a %s package as you" Logger.error(
" do not have the permissions to do this action.", "You cannot perform the %s operation on a %s package as you"
operation, " do not have the permissions to do this action.",
component, operation,
) component,
sys.exit(1) )
sys.exit(1)
# Output details. # Output details.
Logger.info( Logger.info(
@ -278,8 +288,7 @@ def main():
build.rescore(score=priority) build.rescore(score=priority)
except lazr.restfulclient.errors.Unauthorized: except lazr.restfulclient.errors.Unauthorized:
Logger.error( Logger.error(
"You don't have the permissions to rescore builds." "You don't have the permissions to rescore builds. Ignoring your rescore request."
" Ignoring your rescore request."
) )
break break
else: else:
@ -316,22 +325,24 @@ def main():
if not args.state: if not args.state:
if args.retry: if args.retry:
args.state = "Failed to build" args.state='Failed to build'
elif args.priority: elif args.priority:
args.state = "Needs building" args.state='Needs building'
# there is no equivalent to series.getBuildRecords() for a ppa. # there is no equivalent to series.getBuildRecords() for a ppa.
# however, we don't want to have to traverse all build records for # however, we don't want to have to traverse all build records for
# all series when working on the main archive, so we use # all series when working on the main archive, so we use
# series.getBuildRecords() for ubuntu and handle ppas separately # series.getBuildRecords() for ubuntu and handle ppas separately
series = ubuntu.getSeries(name_or_version=release) series = ubuntu.getSeries(name_or_version=release)
if args.archive == "ubuntu": if args.archive == 'ubuntu':
builds = series.getBuildRecords(build_state=args.state, pocket=pocket) builds = series.getBuildRecords(build_state=args.state,
pocket=pocket)
else: else:
builds = [] builds = []
for build in archive.getBuildRecords(build_state=args.state, pocket=pocket): for build in archive.getBuildRecords(build_state=args.state,
pocket=pocket):
if not build.current_source_publication: if not build.current_source_publication:
continue continue
if build.current_source_publication.distro_series == series: if build.current_source_publication.distro_series==series:
builds.append(build) builds.append(build)
for build in builds: for build in builds:
if build.arch_tag not in archs: if build.arch_tag not in archs:
@ -350,31 +361,24 @@ def main():
) )
if args.retry and not can_retry: if args.retry and not can_retry:
Logger.error( Logger.error(
"You don't have the permissions to retry the build of '%s', skipping.", "You don't have the permissions to retry the "
build.source_package_name, "build of '%s', skipping.",
build.source_package_name
) )
continue continue
Logger.info( Logger.info(
"The source version for '%s' in '%s' (%s) is: %s", "The source version for '%s' in '%s' (%s) is: %s",
build.source_package_name, build.source_package_name,
release, release,
pocket, pocket,
build.source_package_version, build.source_package_version
) )
if args.retry and build.can_be_retried: if args.retry and build.can_be_retried:
Logger.info( Logger.info("Retrying build of %s on %s...",
"Retrying build of %s on %s...", build.source_package_name, build.arch_tag build.source_package_name, build.arch_tag)
) build.retry()
try: retry_count += 1
build.retry()
retry_count += 1
except lazr.restfulclient.errors.BadRequest:
Logger.info(
"Failed to retry build of %s on %s",
build.source_package_name,
build.arch_tag,
)
if args.priority and can_rescore: if args.priority and can_rescore:
if build.can_be_rescored: if build.can_be_rescored:
@ -382,32 +386,28 @@ def main():
build.rescore(score=args.priority) build.rescore(score=args.priority)
except lazr.restfulclient.errors.Unauthorized: except lazr.restfulclient.errors.Unauthorized:
Logger.error( Logger.error(
"You don't have the permissions to rescore builds." "You don't have the permissions to rescore builds. Ignoring your rescore request."
" Ignoring your rescore request."
) )
can_rescore = False can_rescore = False
except lazr.restfulclient.errors.BadRequest: except lazr.restfulclient.errors.BadRequest:
Logger.info( Logger.info("Cannot rescore build of %s on %s.",
"Cannot rescore build of %s on %s.", build.source_package_name, build.arch_tag)
build.source_package_name,
build.arch_tag,
)
Logger.info("") Logger.info("")
if args.retry: if args.retry:
Logger.info("%d package builds retried", retry_count) Logger.info("%d package builds retried", retry_count)
sys.exit(0) sys.exit(0)
for pkg in args.packages: for pkg in args.packages:
try: try:
pkg = archive.getPublishedSources( pkg = archive.getPublishedSources(
distro_series=distroseries, distro_series=distroseries,
exact_match=True, exact_match=True,
pocket=pocket, pocket=pocket,
source_name=pkg, source_name=pkg,
status="Published", status='Published')[0]
)[0] except IndexError as error:
except IndexError:
Logger.error("No publication found for package %s", pkg) Logger.error("No publication found for package %s", pkg)
continue continue
@ -435,14 +435,15 @@ def main():
pkg.source_package_version, pkg.source_package_version,
) )
Logger.info(get_build_states(pkg, archs)) Logger.info(getBuildStates(pkg, archs))
if can_retry: if can_retry:
Logger.info(retry_builds(pkg, archs)) Logger.info(retryBuilds(pkg, archs))
if args.priority: if args.priority:
Logger.info(rescore_builds(pkg, archs, args.priority)) Logger.info(rescoreBuilds(pkg, archs, args.priority))
Logger.info("") Logger.info("")
if __name__ == "__main__": if __name__ == "__main__":
main() main()

View File

@ -165,7 +165,6 @@ class SourcePackage(ABC):
series = kwargs.get("series") series = kwargs.get("series")
pocket = kwargs.get("pocket") pocket = kwargs.get("pocket")
status = kwargs.get("status") status = kwargs.get("status")
arch = kwargs.get("arch")
verify_signature = kwargs.get("verify_signature", False) verify_signature = kwargs.get("verify_signature", False)
try_binary = kwargs.get("try_binary", True) try_binary = kwargs.get("try_binary", True)
@ -185,7 +184,6 @@ class SourcePackage(ABC):
self._series = series self._series = series
self._pocket = pocket self._pocket = pocket
self._status = status self._status = status
self._arch = arch
# dscfile can be either a path or an URL. misc.py's download() will # dscfile can be either a path or an URL. misc.py's download() will
# later fiture it out # later fiture it out
self._dsc_source = dscfile self._dsc_source = dscfile
@ -254,7 +252,6 @@ class SourcePackage(ABC):
) )
try: try:
params["archtag"] = self._arch
bpph = archive.getBinaryPackage(self.source, **params) bpph = archive.getBinaryPackage(self.source, **params)
except PackageNotFoundException as bpnfe: except PackageNotFoundException as bpnfe:
# log binary lookup failure, in case it provides hints # log binary lookup failure, in case it provides hints
@ -546,7 +543,7 @@ class SourcePackage(ABC):
Return the debdiff filename. Return the debdiff filename.
""" """
cmd = ["debdiff", self.dsc_name, newpkg.dsc_name] cmd = ["debdiff", self.dsc_name, newpkg.dsc_name]
difffn = f"{newpkg.dsc_name[:-3]}debdiff" difffn = newpkg.dsc_name[:-3] + "debdiff"
Logger.debug("%s > %s", " ".join(cmd), difffn) Logger.debug("%s > %s", " ".join(cmd), difffn)
with open(difffn, "w", encoding="utf-8") as f: with open(difffn, "w", encoding="utf-8") as f:
if subprocess.call(cmd, stdout=f, cwd=str(self.workdir)) > 2: if subprocess.call(cmd, stdout=f, cwd=str(self.workdir)) > 2:
@ -1345,7 +1342,7 @@ class SnapshotSPPH:
self.getComponent(), self.getComponent(),
subdir, subdir,
name, name,
f"{name}_{pkgversion}", name + "_" + pkgversion,
"changelog.txt", "changelog.txt",
) )
try: try:

View File

@ -71,8 +71,8 @@ class Pbuilder(Builder):
cmd = [ cmd = [
"sudo", "sudo",
"-E", "-E",
f"ARCH={self.architecture}", "ARCH=" + self.architecture,
f"DIST={dist}", "DIST=" + dist,
self.name, self.name,
"--build", "--build",
"--architecture", "--architecture",
@ -91,8 +91,8 @@ class Pbuilder(Builder):
cmd = [ cmd = [
"sudo", "sudo",
"-E", "-E",
f"ARCH={self.architecture}", "ARCH=" + self.architecture,
f"DIST={dist}", "DIST=" + dist,
self.name, self.name,
"--update", "--update",
"--architecture", "--architecture",
@ -140,7 +140,7 @@ class Sbuild(Builder):
workdir = os.getcwd() workdir = os.getcwd()
Logger.debug("cd %s", result_directory) Logger.debug("cd %s", result_directory)
os.chdir(result_directory) os.chdir(result_directory)
cmd = ["sbuild", "--arch-all", f"--dist={dist}", f"--arch={self.architecture}", dsc_file] cmd = ["sbuild", "--arch-all", "--dist=" + dist, "--arch=" + self.architecture, dsc_file]
Logger.debug(" ".join(cmd)) Logger.debug(" ".join(cmd))
returncode = subprocess.call(cmd) returncode = subprocess.call(cmd)
Logger.debug("cd %s", workdir) Logger.debug("cd %s", workdir)

View File

@ -68,19 +68,21 @@ class UDTConfig:
config = {} config = {}
for filename in ("/etc/devscripts.conf", "~/.devscripts"): for filename in ("/etc/devscripts.conf", "~/.devscripts"):
try: try:
with open(os.path.expanduser(filename), "r", encoding="utf-8") as f: f = open(os.path.expanduser(filename), "r", encoding="utf-8")
content = f.read()
except IOError: except IOError:
continue continue
try: for line in f:
tokens = shlex.split(content, comments=True) parsed = shlex.split(line, comments=True)
except ValueError as e: if len(parsed) > 1:
Logger.error("Error parsing %s: %s", filename, e) Logger.warning(
continue "Cannot parse variable assignment in %s: %s",
for token in tokens: getattr(f, "name", "<config>"),
if "=" in token: line,
key, value = token.split("=", 1) )
if len(parsed) >= 1 and "=" in parsed[0]:
key, value = parsed[0].split("=", 1)
config[key] = value config[key] = value
f.close()
return config return config
def get_value(self, key, default=None, boolean=False, compat_keys=()): def get_value(self, key, default=None, boolean=False, compat_keys=()):
@ -97,9 +99,9 @@ class UDTConfig:
if default is None and key in self.defaults: if default is None and key in self.defaults:
default = self.defaults[key] default = self.defaults[key]
keys = [f"{self.prefix}_{key}"] keys = [self.prefix + "_" + key]
if key in self.defaults: if key in self.defaults:
keys.append(f"UBUNTUTOOLS_{key}") keys.append("UBUNTUTOOLS_" + key)
keys += compat_keys keys += compat_keys
for k in keys: for k in keys:
@ -112,9 +114,9 @@ class UDTConfig:
else: else:
continue continue
if k in compat_keys: if k in compat_keys:
replacements = f"{self.prefix}_{key}" replacements = self.prefix + "_" + key
if key in self.defaults: if key in self.defaults:
replacements += f"or UBUNTUTOOLS_{key}" replacements += "or UBUNTUTOOLS_" + key
Logger.warning( Logger.warning(
"Using deprecated configuration variable %s. You should use %s.", "Using deprecated configuration variable %s. You should use %s.",
k, k,
@ -178,7 +180,7 @@ def ubu_email(name=None, email=None, export=True):
mailname = socket.getfqdn() mailname = socket.getfqdn()
if os.path.isfile("/etc/mailname"): if os.path.isfile("/etc/mailname"):
mailname = open("/etc/mailname", "r", encoding="utf-8").read().strip() mailname = open("/etc/mailname", "r", encoding="utf-8").read().strip()
email = f"{pwd.getpwuid(os.getuid()).pw_name}@{mailname}" email = pwd.getpwuid(os.getuid()).pw_name + "@" + mailname
if export: if export:
os.environ["DEBFULLNAME"] = name os.environ["DEBFULLNAME"] = name

View File

@ -883,7 +883,7 @@ class SourcePackagePublishingHistory(BaseWrapper):
""" """
release = self.getSeriesName() release = self.getSeriesName()
if self.pocket != "Release": if self.pocket != "Release":
release += f"-{self.pocket.lower()}" release += "-" + self.pocket.lower()
return release return release
def getArchive(self): def getArchive(self):

View File

@ -385,7 +385,7 @@ class _StderrProgressBar:
pctstr = f"{pct:>3}%" pctstr = f"{pct:>3}%"
barlen = self.width * pct // 100 barlen = self.width * pct // 100
barstr = "=" * barlen barstr = "=" * barlen
barstr = f"{barstr[:-1]}>" barstr = barstr[:-1] + ">"
barstr = barstr.ljust(self.width) barstr = barstr.ljust(self.width)
fullstr = f"\r[{barstr}]{pctstr}" fullstr = f"\r[{barstr}]{pctstr}"
sys.stderr.write(fullstr) sys.stderr.write(fullstr)

View File

@ -340,7 +340,6 @@ class PullPkg:
params = {} params = {}
params["package"] = options["package"] params["package"] = options["package"]
params["arch"] = options["arch"]
if options["release"]: if options["release"]:
(release, version, pocket) = self.parse_release_and_version( (release, version, pocket) = self.parse_release_and_version(
@ -454,7 +453,7 @@ class PullPkg:
if key.startswith("vcs-"): if key.startswith("vcs-"):
if key == "vcs-browser": if key == "vcs-browser":
continue continue
if key == "vcs-git": elif key == "vcs-git":
vcs = "Git" vcs = "Git"
elif key == "vcs-bzr": elif key == "vcs-bzr":
vcs = "Bazaar" vcs = "Bazaar"
@ -463,13 +462,9 @@ class PullPkg:
uri = srcpkg.dsc[original_key] uri = srcpkg.dsc[original_key]
Logger.warning( Logger.warning("\nNOTICE: '%s' packaging is maintained in "
"\nNOTICE: '%s' packaging is maintained in " "the '%s' version control system at:\n"
"the '%s' version control system at:\n %s\n", " %s\n" % (package, vcs, uri))
package,
vcs,
uri,
)
if vcs == "Bazaar": if vcs == "Bazaar":
vcscmd = " $ bzr branch " + uri vcscmd = " $ bzr branch " + uri
@ -477,11 +472,9 @@ class PullPkg:
vcscmd = " $ git clone " + uri vcscmd = " $ git clone " + uri
if vcscmd: if vcscmd:
Logger.info( Logger.info(f"Please use:\n{vcscmd}\n"
"Please use:\n%s\n" "to retrieve the latest (possibly unreleased) "
"to retrieve the latest (possibly unreleased) updates to the package.\n", "updates to the package.\n")
vcscmd,
)
if pull == PULL_LIST: if pull == PULL_LIST:
Logger.info("Source files:") Logger.info("Source files:")

View File

@ -31,9 +31,9 @@ class Question:
def get_options(self): def get_options(self):
if len(self.options) == 2: if len(self.options) == 2:
options = f"{self.options[0]} or {self.options[1]}" options = self.options[0] + " or " + self.options[1]
else: else:
options = f"{', '.join(self.options[:-1])}, or {self.options[-1]}" options = ", ".join(self.options[:-1]) + ", or " + self.options[-1]
return options return options
def ask(self, question, default=None): def ask(self, question, default=None):
@ -67,7 +67,7 @@ class Question:
if selected == option[0]: if selected == option[0]:
selected = option selected = option
if selected not in self.options: if selected not in self.options:
print(f"Please answer the question with {self.get_options()}.") print("Please answer the question with " + self.get_options() + ".")
return selected return selected
@ -170,7 +170,7 @@ class EditBugReport(EditFile):
split_re = re.compile(r"^Summary.*?:\s+(.*?)\s+Description:\s+(.*)$", re.DOTALL | re.UNICODE) split_re = re.compile(r"^Summary.*?:\s+(.*?)\s+Description:\s+(.*)$", re.DOTALL | re.UNICODE)
def __init__(self, subject, body, placeholders=None): def __init__(self, subject, body, placeholders=None):
prefix = f"{os.path.basename(sys.argv[0])}_" prefix = os.path.basename(sys.argv[0]) + "_"
tmpfile = tempfile.NamedTemporaryFile(prefix=prefix, suffix=".txt", delete=False) tmpfile = tempfile.NamedTemporaryFile(prefix=prefix, suffix=".txt", delete=False)
tmpfile.write((f"Summary (one line):\n{subject}\n\nDescription:\n{body}").encode("utf-8")) tmpfile.write((f"Summary (one line):\n{subject}\n\nDescription:\n{body}").encode("utf-8"))
tmpfile.close() tmpfile.close()

View File

@ -183,7 +183,7 @@ Content-Type: text/plain; charset=UTF-8
backup = tempfile.NamedTemporaryFile( backup = tempfile.NamedTemporaryFile(
mode="w", mode="w",
delete=False, delete=False,
prefix=f"requestsync-{re.sub('[^a-zA-Z0-9_-]', '', bugtitle.replace(' ', '_'))}", prefix="requestsync-" + re.sub(r"[^a-zA-Z0-9_-]", "", bugtitle.replace(" ", "_")),
) )
with backup: with backup:
backup.write(mail) backup.write(mail)

View File

@ -1,19 +1,18 @@
# Copyright (C) 2024 Canonical Ltd. # Copyright (C) 2024 Canonical Ltd.
# Author: Chris Peterson <chris.peterson@canonical.com> # Author: Chris Peterson <chris.peterson@canonical.com>
# Author: Andy P. Whitcroft
# Author: Christian Ehrhardt
# #
# This program is free software: you can redistribute it and/or modify it # This program is free software: you can redistribute it and/or modify
# under the terms of the GNU General Public License version 3, as published # it under the terms of the GNU Affero General Public License as
# by the Free Software Foundation. # published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
# #
# This program is distributed in the hope that it will be useful, but # This program is distributed in the hope that it will be useful,
# WITHOUT ANY WARRANTY; without even the implied warranties of # but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE. # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# See the GNU General Public License for more details. # GNU Affero General Public License for more details.
# #
# You should have received a copy of the GNU General Public License along # You should have received a copy of the GNU Affero General Public License
# with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import datetime import datetime
import json import json
@ -26,7 +25,10 @@ URL_QUEUED = "http://autopkgtest.ubuntu.com/queues.json"
def _get_jobs(url: str) -> dict: def _get_jobs(url: str) -> dict:
request = urllib.request.Request(url, headers={"Cache-Control": "max-age-0"}) request = urllib.request.Request(
url,
headers={"Cache-Control": "max-age-0"},
)
with urllib.request.urlopen(request) as response: with urllib.request.urlopen(request) as response:
data = response.read() data = response.read()
jobs = json.loads(data.decode("utf-8")) jobs = json.loads(data.decode("utf-8"))
@ -49,10 +51,7 @@ def get_running():
env = jobinfo[0].get("env", "-") env = jobinfo[0].get("env", "-")
time = str(datetime.timedelta(seconds=jobinfo[1])) time = str(datetime.timedelta(seconds=jobinfo[1]))
try: try:
line = ( line = f"R {time:6} {pkg:30} {'-':10} {series:8} {arch:8} {ppas:31} {triggers} {env}\n"
f"R {time:6} {pkg:30} {'-':10} {series:8} {arch:8}"
f" {ppas:31} {triggers} {env}\n"
)
running.append((jobinfo[1], line)) running.append((jobinfo[1], line))
except BrokenPipeError: except BrokenPipeError:
sys.exit(1) sys.exit(1)
@ -86,10 +85,7 @@ def get_queued():
n = n + 1 n = n + 1
try: try:
output += ( output += f"Q{n:04d} {'-:--':>6} {pkg:30} {origin:10} {series:8} {arch:8} {ppas:31} {triggers}\n"
f"Q{n:04d} {'-:--':>6} {pkg:30} {origin:10} {series:8} {arch:8}"
f" {ppas:31} {triggers}\n"
)
except BrokenPipeError: except BrokenPipeError:
sys.exit(1) sys.exit(1)
return output return output

View File

@ -255,7 +255,7 @@ class SourcePackage:
def _changes_file(self): def _changes_file(self):
"""Returns the file name of the .changes file.""" """Returns the file name of the .changes file."""
return os.path.join( return os.path.join(
self._workdir, f"{self._package}_{strip_epoch(self._version)}_source.changes" self._workdir, f"{self._package}_{ strip_epoch(self._version)}_source.changes"
) )
def check_target(self, upload, launchpad): def check_target(self, upload, launchpad):

View File

@ -39,7 +39,7 @@ def is_command_available(command, check_sbin=False):
"Is command in $PATH?" "Is command in $PATH?"
path = os.environ.get("PATH", "/usr/bin:/bin").split(":") path = os.environ.get("PATH", "/usr/bin:/bin").split(":")
if check_sbin: if check_sbin:
path += [f"{directory[:-3]}sbin" for directory in path if directory.endswith("/bin")] path += [directory[:-3] + "sbin" for directory in path if directory.endswith("/bin")]
return any(os.access(os.path.join(directory, command), os.X_OK) for directory in path) return any(os.access(os.path.join(directory, command), os.X_OK) for directory in path)
@ -303,7 +303,7 @@ def _download_and_change_into(task, dsc_file, patch, branch):
extract_source(dsc_file, Logger.isEnabledFor(logging.DEBUG)) extract_source(dsc_file, Logger.isEnabledFor(logging.DEBUG))
# change directory # change directory
directory = f"{task.package}-{task.get_version().upstream_version}" directory = task.package + "-" + task.get_version().upstream_version
Logger.debug("cd %s", directory) Logger.debug("cd %s", directory)
os.chdir(directory) os.chdir(directory)

View File

@ -1,33 +0,0 @@
# Copyright (C) 2024 Canonical Ltd.
# Author: Chris Peterson <chris.peterson@canonical.com>
#
# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
# AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
# OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
# PERFORMANCE OF THIS SOFTWARE.
import unittest
# Binary Tests
class BinaryTests(unittest.TestCase):
# The requestsync binary has the option of using the launchpad api
# to log in but requires python3-keyring in addition to
# python3-launchpadlib. Testing the integrated login functionality
# automatically isn't very feasbile, but we can at least write a smoke
# test to make sure the required packages are installed.
# See LP: #2049217
def test_keyring_installed(self):
"""Smoke test for required lp api dependencies"""
try:
import keyring # noqa: F401
except ModuleNotFoundError:
raise ModuleNotFoundError("package python3-keyring is not installed")

View File

@ -1,17 +1,18 @@
# Copyright (C) 2024 Canonical Ltd. # Copyright (C) 2024 Canonical Ltd.
# Author: Chris Peterson <chris.peterson@canonical.com> # Author: Chris Peterson <chris.peterson@canonical.com>
# #
# Permission to use, copy, modify, and/or distribute this software for any # This program is free software: you can redistribute it and/or modify
# purpose with or without fee is hereby granted, provided that the above # it under the terms of the GNU Affero General Public License as
# copyright notice and this permission notice appear in all copies. # published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
# #
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH # This program is distributed in the hope that it will be useful,
# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # but WITHOUT ANY WARRANTY; without even the implied warranty of
# AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # GNU Affero General Public License for more details.
# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR #
# OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # You should have received a copy of the GNU Affero General Public License
# PERFORMANCE OF THIS SOFTWARE. # along with this program. If not, see <http://www.gnu.org/licenses/>.
""" Tests for running_autopkgtests """ Tests for running_autopkgtests
Tests using cached data from autopkgtest servers. Tests using cached data from autopkgtest servers.
@ -32,17 +33,8 @@ from ubuntutools.running_autopkgtests import (
) )
# Cached binary response data from autopkgtest server # Cached binary response data from autopkgtest server
RUN_DATA = ( RUN_DATA = b'{"pyatem": { "submit-time_2024-01-19 19:37:36;triggers_[\'python3-defaults/3.12.1-0ubuntu1\'];": {"noble": {"arm64": [{"triggers": ["python3-defaults/3.12.1-0ubuntu1"], "submit-time": "2024-01-19 19:37:36"}, 380, "<omitted log>"]}}}}'
b'{"pyatem": {' QUEUED_DATA = b'{"ubuntu": {"noble": {"arm64": ["libobject-accessor-perl {\\"requester\\": \\"someone\\", \\"submit-time\\": \\"2024-01-18 01:08:55\\", \\"triggers\\": [\\"perl/5.38.2-3\\", \\"liblocale-gettext-perl/1.07-6build1\\"]}"]}}}'
b" \"submit-time_2024-01-19 19:37:36;triggers_['python3-defaults/3.12.1-0ubuntu1'];\":"
b' {"noble": {"arm64": [{"triggers": ["python3-defaults/3.12.1-0ubuntu1"],'
b' "submit-time": "2024-01-19 19:37:36"}, 380, "<omitted log>"]}}}}'
)
QUEUED_DATA = (
b'{"ubuntu": {"noble": {"arm64": ["libobject-accessor-perl {\\"requester\\": \\"someone\\",'
b' \\"submit-time\\": \\"2024-01-18 01:08:55\\",'
b' \\"triggers\\": [\\"perl/5.38.2-3\\", \\"liblocale-gettext-perl/1.07-6build1\\"]}"]}}}'
)
# Expected result(s) of parsing the above JSON data # Expected result(s) of parsing the above JSON data
RUNNING_JOB = { RUNNING_JOB = {
@ -66,9 +58,7 @@ QUEUED_JOB = {
"ubuntu": { "ubuntu": {
"noble": { "noble": {
"arm64": [ "arm64": [
'libobject-accessor-perl {"requester": "someone",' 'libobject-accessor-perl {"requester": "someone", "submit-time": "2024-01-18 01:08:55", "triggers": ["perl/5.38.2-3", "liblocale-gettext-perl/1.07-6build1"]}',
' "submit-time": "2024-01-18 01:08:55",'
' "triggers": ["perl/5.38.2-3", "liblocale-gettext-perl/1.07-6build1"]}'
] ]
} }
} }
@ -79,18 +69,9 @@ PRIVATE_JOB = {"ppa": {"noble": {"arm64": ["private job"]}}}
# Expected textual output of the program based on the above data # Expected textual output of the program based on the above data
RUNNING_OUTPUT = ( RUNNING_OUTPUT = "R 0:06:20 pyatem - noble arm64 - python3-defaults/3.12.1-0ubuntu1 -\n"
"R 0:06:20 pyatem - noble arm64" QUEUED_OUTPUT = "Q0001 -:-- libobject-accessor-perl ubuntu noble arm64 - perl/5.38.2-3,liblocale-gettext-perl/1.07-6build1\n"
" - python3-defaults/3.12.1-0ubuntu1 -\n" PRIVATE_OUTPUT = "Q0001 -:-- private job ppa noble arm64 private job private job\n"
)
QUEUED_OUTPUT = (
"Q0001 -:-- libobject-accessor-perl ubuntu noble arm64"
" - perl/5.38.2-3,liblocale-gettext-perl/1.07-6build1\n"
)
PRIVATE_OUTPUT = (
"Q0001 -:-- private job ppa noble arm64"
" private job private job\n"
)
class RunningAutopkgtestTestCase(unittest.TestCase): class RunningAutopkgtestTestCase(unittest.TestCase):

View File

@ -72,17 +72,17 @@ class Control:
def set_maintainer(self, maintainer): def set_maintainer(self, maintainer):
"""Sets the value of the Maintainer field.""" """Sets the value of the Maintainer field."""
pattern = re.compile("^Maintainer: ?.*$", re.MULTILINE) pattern = re.compile("^Maintainer: ?.*$", re.MULTILINE)
self._content = pattern.sub(f"Maintainer: {maintainer}", self._content) self._content = pattern.sub("Maintainer: " + maintainer, self._content)
def set_original_maintainer(self, original_maintainer): def set_original_maintainer(self, original_maintainer):
"""Sets the value of the XSBC-Original-Maintainer field.""" """Sets the value of the XSBC-Original-Maintainer field."""
original_maintainer = f"XSBC-Original-Maintainer: {original_maintainer}" original_maintainer = "XSBC-Original-Maintainer: " + original_maintainer
if self.get_original_maintainer(): if self.get_original_maintainer():
pattern = re.compile("^(?:[XSBC]*-)?Original-Maintainer:.*$", re.MULTILINE) pattern = re.compile("^(?:[XSBC]*-)?Original-Maintainer:.*$", re.MULTILINE)
self._content = pattern.sub(original_maintainer, self._content) self._content = pattern.sub(original_maintainer, self._content)
else: else:
pattern = re.compile("^(Maintainer:.*)$", re.MULTILINE) pattern = re.compile("^(Maintainer:.*)$", re.MULTILINE)
self._content = pattern.sub(f"\\1\\n{original_maintainer}", self._content) self._content = pattern.sub(r"\1\n" + original_maintainer, self._content)
def remove_original_maintainer(self): def remove_original_maintainer(self):
"""Strip out out the XSBC-Original-Maintainer line""" """Strip out out the XSBC-Original-Maintainer line"""

View File

@ -15,44 +15,47 @@
"""Portions of archive related code that is re-used by various tools.""" """Portions of archive related code that is re-used by various tools."""
from datetime import datetime
import os import os
import re import re
import urllib.request import urllib.request
from datetime import datetime
import dateutil.parser import dateutil.parser
from dateutil.tz import tzutc from dateutil.tz import tzutc
def get_cache_dir(): def get_cache_dir():
cache_dir = os.environ.get("XDG_CACHE_HOME", os.path.expanduser(os.path.join("~", ".cache"))) cache_dir = os.environ.get('XDG_CACHE_HOME',
uat_cache = os.path.join(cache_dir, "ubuntu-archive-tools") os.path.expanduser(os.path.join('~', '.cache')))
uat_cache = os.path.join(cache_dir, 'ubuntu-archive-tools')
os.makedirs(uat_cache, exist_ok=True) os.makedirs(uat_cache, exist_ok=True)
return uat_cache return uat_cache
def get_url(url, force_cached): def get_url(url, force_cached):
"""Return file to the URL, possibly caching it""" ''' Return file to the URL, possibly caching it
'''
cache_file = None cache_file = None
# ignore bileto urls wrt caching, they're usually too small to matter # ignore bileto urls wrt caching, they're usually too small to matter
# and we don't do proper cache expiry # and we don't do proper cache expiry
m = re.search("ubuntu-archive-team.ubuntu.com/proposed-migration/([^/]*)/([^/]*)", url) m = re.search('ubuntu-archive-team.ubuntu.com/proposed-migration/'
'([^/]*)/([^/]*)',
url)
if m: if m:
cache_dir = get_cache_dir() cache_dir = get_cache_dir()
cache_file = os.path.join(cache_dir, f"{m.group(1)}_{m.group(2)}") cache_file = os.path.join(cache_dir, '%s_%s' % (m.group(1), m.group(2)))
else: else:
# test logs can be cached, too # test logs can be cached, too
m = re.search( m = re.search(
"https://autopkgtest.ubuntu.com/results/autopkgtest-[^/]*/([^/]*)/([^/]*)" 'https://autopkgtest.ubuntu.com/results/autopkgtest-[^/]*/([^/]*)/([^/]*)'
"/[a-z0-9]*/([^/]*)/([_a-f0-9]*)@/log.gz", '/[a-z0-9]*/([^/]*)/([_a-f0-9]*)@/log.gz',
url, url)
)
if m: if m:
cache_dir = get_cache_dir() cache_dir = get_cache_dir()
cache_file = os.path.join( cache_file = os.path.join(
cache_dir, f"{m.group(1)}_{m.group(2)}_{m.group(3)}_{m.group(4)}.gz" cache_dir, '%s_%s_%s_%s.gz' % (
) m.group(1), m.group(2), m.group(3), m.group(4)))
if cache_file: if cache_file:
try: try:
@ -62,18 +65,18 @@ def get_url(url, force_cached):
prev_timestamp = datetime.fromtimestamp(prev_mtime, tz=tzutc()) prev_timestamp = datetime.fromtimestamp(prev_mtime, tz=tzutc())
new_timestamp = datetime.now(tz=tzutc()).timestamp() new_timestamp = datetime.now(tz=tzutc()).timestamp()
if force_cached: if force_cached:
return open(cache_file, "rb") return open(cache_file, 'rb')
f = urllib.request.urlopen(url) f = urllib.request.urlopen(url)
if cache_file: if cache_file:
remote_ts = dateutil.parser.parse(f.headers["last-modified"]) remote_ts = dateutil.parser.parse(f.headers['last-modified'])
if remote_ts > prev_timestamp: if remote_ts > prev_timestamp:
with open(f"{cache_file}.new", "wb") as new_cache: with open('%s.new' % cache_file, 'wb') as new_cache:
for line in f: for line in f:
new_cache.write(line) new_cache.write(line)
os.rename(f"{cache_file}.new", cache_file) os.rename('%s.new' % cache_file, cache_file)
os.utime(cache_file, times=(new_timestamp, new_timestamp)) os.utime(cache_file, times=(new_timestamp, new_timestamp))
f.close() f.close()
f = open(cache_file, "rb") f = open(cache_file, 'rb')
return f return f