Compare commits

...

32 Commits
0.207 ... main

Author SHA1 Message Date
Florent 'Skia' Jacquet
7f5e9c8680 pm-helper: make use of YesNoQuestion 2025-12-03 16:46:51 +01:00
Benjamin Drung
d35268b797 Release ubuntu-dev-tools 0.208
Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2025-12-03 16:33:51 +01:00
Benjamin Drung
bf9ead2204 requestsync: support pocket parameter in get_ubuntu_srcpkg
The command `requestsync --email -d sid <package> <target>` fails with
the following stacktrace:

```
Traceback (most recent call last):
  File "/usr/bin/requestsync", line 402, in <module>
    main()
  File "/usr/bin/requestsync", line 225, in main
    ubuntu_srcpkg = get_ubuntu_srcpkg(srcpkg, args.release, "Proposed")
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: get_ubuntu_srcpkg() takes 2 positional arguments but 3 were given
```

LP: #2115990
Fixes: 5eb960dd3fe57daa16d8cee8cefee035cebb8e5d
2025-12-03 16:25:28 +01:00
Benjamin Drung
38988ed183 Modernize SourcePackage._run_lintian() 2025-12-03 15:43:23 +01:00
Benjamin Drung
5e2f94cdb4 SourcePackage: introduce package_and_version
Introduce `package_and_version` to avoid code duplication.
2025-12-03 15:34:17 +01:00
Benjamin Drung
29914382cf ubuntu-build: introduce parse_args helper function
Move the argument parsing code into a separate `parse_args` function to
make the `main` function a little bit smaller.

The `IndexError` on accessing `args.packages` cannot happen.
2025-12-03 15:21:27 +01:00
Benjamin Drung
addeb4f7fb sponsor-patch: stop checking for bzr being present 2025-12-03 15:10:17 +01:00
Benjamin Drung
ee87f312bf run mypy during package build 2025-12-03 14:54:54 +01:00
Benjamin Drung
32530e356d run-linters: avoid searching scripts in build/
During package build the Python code is copied to the `build` directory.
Ignore this directory when searching for Python scripts.
2025-12-03 14:51:51 +01:00
Benjamin Drung
38ef3c506e Run wrap-and-sort -ast 2025-12-03 14:40:47 +01:00
Benjamin Drung
5fc7e15f96 Fix type annotation for resource_type
mypy complains:

```
ubuntutools/lp/lpapicache.py:143: error: Incompatible types in assignment (expression has type "None", variable has type "str")  [assignment]
ubuntutools/lp/lpapicache.py:1328: error: Incompatible types in assignment (expression has type "tuple[str, str]", base class "BaseWrapper" defined the type as "str")  [assignment]
```
2025-12-03 14:37:33 +01:00
Benjamin Drung
f0326592bd add type annotation for mypy
mypy comlains:

```
ubuntutools/config.py:53: error: Need type annotation for "config" (hint: "config: dict[<type>, <type>] = ...")  [var-annotated]
ubuntutools/lp/lpapicache.py:1504: error: Need type annotation for "_source_sets" (hint: "_source_sets: dict[<type>, <type>] = ...")  [var-annotated]
```
2025-12-03 14:36:49 +01:00
Benjamin Drung
aa439fec02 add missing return type annotation 2025-12-03 14:35:04 +01:00
Benjamin Drung
4bcfa0dd5a .pylintrc: ignore too-many-positional-arguments for now 2025-12-03 14:17:06 +01:00
Benjamin Drung
63b3d54264 Drop obsolete Rules-Requires-Root: no 2025-12-03 14:15:06 +01:00
Benjamin Drung
654af1a613 Drop Lintian overrides related to .pyc files
.pyc files should not be included in the source tarball.
2025-12-03 14:14:20 +01:00
Benjamin Drung
524f590af2 Run linters that can detect real errors on package build 2025-12-03 14:11:36 +01:00
Benjamin Drung
4a2f194860 run-linters: add --errors-only mode
Add `--errors-only` that runs only linters that can detect real errors
and all other ignoring the result.
2025-12-03 14:10:11 +01:00
Benjamin Drung
7b9aee4c0c run-linters: introduce helper functions 2025-12-03 14:07:56 +01:00
Benjamin Drung
45d317cc87 mark non-returning functions with typing.NoReturn
To help pylint, mark non-returning functions with `typing.NoReturn`.
2025-12-03 14:01:18 +01:00
Benjamin Drung
2e041ac1ff syncpackage: replace global by LRU cache
pylint complains:

```
syncpackage:459:4: W0603: Using the global statement (global-statement)
```

Replace the `global` statement by `functools.lru_cache`.
2025-12-03 13:59:06 +01:00
Benjamin Drung
c8fe724560 Fix pylint in ubuntutools/test/test_requestsync.py
pylint complains:

```
ubuntutools/test/test_requestsync.py:31:12: C0415: Import outside toplevel (keyring) (import-outside-toplevel)
ubuntutools/test/test_requestsync.py:33:12: W0707: Consider explicitly re-raising using 'except ModuleNotFoundError as exc' and 'raise ModuleNotFoundError('package python3-keyring is not installed') from exc' (raise-missing-from)
ubuntutools/test/test_requestsync.py:31:12: W0611: Unused import keyring (unused-import)
```
2025-12-03 13:51:50 +01:00
Benjamin Drung
768a517370 ubuntutools/archive.py: use 'yield from'
pylint complains:

```
ubuntutools/archive.py:343:8: R1737: Use 'yield from' directly instead of yielding each element one by one (use-yield-from)
ubuntutools/archive.py:346:8: R1737: Use 'yield from' directly instead of yielding each element one by one (use-yield-from)
ubuntutools/archive.py:638:8: R1737: Use 'yield from' directly instead of yielding each element one by one (use-yield-from)
```
2025-12-03 13:47:54 +01:00
Benjamin Drung
3d2ee5a1b7 ubuntutools/archive.py: do not raise general exception
pylint complains:

```
ubuntutools/archive.py:935:8: W0719: Raising too general exception: Exception (broad-exception-raised)
```
2025-12-03 13:45:46 +01:00
Benjamin Drung
1c81f0872d Use lazy % formatting in logging functions
pylint complains:

```
W1201: Use lazy % formatting in logging functions
```
2025-12-03 13:36:48 +01:00
Benjamin Drung
ef5e3d8066 requestsync: silence possibly-used-before-assignment
pylint complains:

```
requestsync:156:34: E0606: Possibly using variable 'bug_mail_domain' before assignment (possibly-used-before-assignment)
requestsync:217:27: E0606: Possibly using variable 'Distribution' before assignment (possibly-used-before-assignment)
requestsync:380:8: E0606: Possibly using variable 'post_bug' before assignment (possibly-used-before-assignment)
```

These errors are false positives. So silence them. The perfect solution
would be to restructure the code.
2025-12-03 13:32:33 +01:00
Benjamin Drung
816323ea5c sponsor_patch: make pylint happy
pylint complains:

```
ubuntutools/sponsor_patch/sponsor_patch.py:243:45: E0606: Possibly using variable 'task' before assignment (possibly-used-before-assignment)
```

Drop the `len(ubuntu_tasks) > 1` check and rely on being the else case.
2025-12-03 13:21:19 +01:00
Benjamin Drung
41e7d2d714 ubuntutools/pullpkg.py: initialize vcscmd
Fixes: 2f396fe54956
2025-12-03 13:11:47 +01:00
Sebastien Bacher
7e82344d57 ubuntu-build: fix non batch mode errors
The current version is not working

```
$ ubuntu-build librsync resolute retry

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/resource.py", line 354, in __getattr__
    return self.lp_get_parameter(attr)
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/resource.py", line 249, in lp_get_parameter
    raise KeyError("No such parameter: %s" % param_name)
KeyError: 'No such parameter: getComponent'
```

The way launchpadlib is used was changed in 010af53 but non batch mode
was not correctly updated.

Also tweak the way the release is computed for distroseries to be able
to retry a package in e.g "resolute-proposed".
2025-12-03 13:07:25 +01:00
Benjamin Drung
cf88f4b92f syncpackage: do not use bare except for urlopen()
flake8 complains:

```
./syncpackage:465:9: E722 do not use bare 'except'
```

The function `urllib.request.urlopen` might throw
`urllib.error.URLError`, `urllib.error.HTTPError`, `socket.gaierror`,
`ssl.SSLError` which are all subclasses of `OSError`.
2025-12-03 12:24:47 +01:00
Benjamin Drung
c6a4c10da2 Format code with black and isort
```
isort .
black -C . $(grep -l -r '^#! */usr/bin/python3$' .)
```
2025-12-03 12:01:23 +01:00
Gianfranco Costamagna
dff0b269d2 ubuntu-build: consider amd64v3 as valid architecture 2025-10-19 09:29:46 +02:00
29 changed files with 188 additions and 116 deletions

View File

@ -34,6 +34,7 @@ disable=fixme,locally-disabled,missing-docstring,useless-option-value,
duplicate-code, duplicate-code,
too-many-instance-attributes, too-many-instance-attributes,
too-many-nested-blocks, too-many-nested-blocks,
too-many-positional-arguments,
too-many-lines, too-many-lines,

View File

@ -25,6 +25,7 @@ import shutil
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
from typing import Any, NoReturn
from urllib.parse import quote from urllib.parse import quote
try: try:
@ -50,7 +51,7 @@ from ubuntutools.question import YesNoQuestion
Logger = getLogger() Logger = getLogger()
def error(msg, *args): def error(msg: str, *args: Any) -> NoReturn:
Logger.error(msg, *args) Logger.error(msg, *args)
sys.exit(1) sys.exit(1)

23
debian/changelog vendored
View File

@ -1,3 +1,26 @@
ubuntu-dev-tools (0.208) unstable; urgency=medium
[ Gianfranco Costamagna ]
* ubuntu-build: consider amd64v3 as valid architecture
[ Sebastien Bacher ]
* ubuntu-build: fix non batch mode errors.
[ Benjamin Drung ]
* Format code with black and isort
* ubuntutools/pullpkg.py: initialize vcscmd
* make pylint and mypy happy
* mark non-returning functions with typing.NoReturn
* run-linters: add --errors-only mode and run this during package build
* Drop Lintian overrides related to .pyc files
* Drop obsolete Rules-Requires-Root: no
* run mypy during package build
* sponsor-patch: stop checking for bzr being present
* Modernize SourcePackage._run_lintian()
* requestsync: support pocket parameter in get_ubuntu_srcpkg (LP: #2115990)
-- Benjamin Drung <bdrung@debian.org> Wed, 03 Dec 2025 16:33:47 +0100
ubuntu-dev-tools (0.207) unstable; urgency=medium ubuntu-dev-tools (0.207) unstable; urgency=medium
* Team upload. * Team upload.

13
debian/control vendored
View File

@ -8,16 +8,17 @@ Uploaders:
Mattia Rizzolo <mattia@debian.org>, Mattia Rizzolo <mattia@debian.org>,
Simon Quigley <tsimonq2@debian.org>, Simon Quigley <tsimonq2@debian.org>,
Build-Depends: Build-Depends:
black <!nocheck>,
dctrl-tools,
debhelper-compat (= 13), debhelper-compat (= 13),
devscripts (>= 2.11.0~),
dh-make, dh-make,
dh-python, dh-python,
black <!nocheck>,
dctrl-tools,
devscripts (>= 2.11.0~),
distro-info (>= 0.2~), distro-info (>= 0.2~),
flake8, flake8,
isort <!nocheck>, isort <!nocheck>,
lsb-release, lsb-release,
mypy <!nocheck>,
pylint <!nocheck>, pylint <!nocheck>,
python3-all, python3-all,
python3-apt, python3-apt,
@ -30,9 +31,9 @@ Build-Depends:
python3-pytest, python3-pytest,
python3-requests <!nocheck>, python3-requests <!nocheck>,
python3-setuptools, python3-setuptools,
python3-typeshed <!nocheck>,
python3-yaml <!nocheck>, python3-yaml <!nocheck>,
Standards-Version: 4.7.2 Standards-Version: 4.7.2
Rules-Requires-Root: no
Vcs-Git: https://git.launchpad.net/ubuntu-dev-tools Vcs-Git: https://git.launchpad.net/ubuntu-dev-tools
Vcs-Browser: https://git.launchpad.net/ubuntu-dev-tools Vcs-Browser: https://git.launchpad.net/ubuntu-dev-tools
Homepage: https://launchpad.net/ubuntu-dev-tools Homepage: https://launchpad.net/ubuntu-dev-tools
@ -40,12 +41,12 @@ Homepage: https://launchpad.net/ubuntu-dev-tools
Package: ubuntu-dev-tools Package: ubuntu-dev-tools
Architecture: all Architecture: all
Depends: Depends:
dpkg-dev,
binutils, binutils,
dctrl-tools, dctrl-tools,
devscripts (>= 2.11.0~), devscripts (>= 2.11.0~),
diffstat, diffstat,
distro-info (>= 0.2~), distro-info (>= 0.2~),
dpkg-dev,
dput, dput,
lsb-release, lsb-release,
python3, python3,
@ -72,10 +73,10 @@ Recommends:
genisoimage, genisoimage,
lintian, lintian,
patch, patch,
sbuild | pbuilder | cowbuilder,
python3-dns, python3-dns,
quilt, quilt,
reportbug (>= 3.39ubuntu1), reportbug (>= 3.39ubuntu1),
sbuild | pbuilder | cowbuilder,
ubuntu-keyring | ubuntu-archive-keyring, ubuntu-keyring | ubuntu-archive-keyring,
Suggests: Suggests:
bzr | brz, bzr | brz,

3
debian/rules vendored
View File

@ -3,10 +3,11 @@
override_dh_auto_clean: override_dh_auto_clean:
dh_auto_clean dh_auto_clean
rm -f .coverage rm -f .coverage
rm -rf .tox rm -rf .mypy_cache .tox
override_dh_auto_test: override_dh_auto_test:
ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS))) ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
./run-linters --errors-only
python3 -m pytest -v ubuntutools python3 -m pytest -v ubuntutools
endif endif

View File

@ -1,3 +0,0 @@
# pyc files are machine-generated; they're expected to have long lines and have unstated copyright
source: file-without-copyright-information *.pyc [debian/copyright]
source: very-long-line-length-in-source-file * > 512 [*.pyc:*]

View File

@ -4,4 +4,5 @@ Depends:
python3-pytest, python3-pytest,
python3-setuptools, python3-setuptools,
@, @,
Restrictions: allow-stderr Restrictions:
allow-stderr,

View File

@ -43,7 +43,7 @@ operations.
\fB\-a\fR ARCHITECTURE, \fB\-\-arch\fR=\fIARCHITECTURE\fR \fB\-a\fR ARCHITECTURE, \fB\-\-arch\fR=\fIARCHITECTURE\fR
Rebuild or rescore a specific architecture. Valid Rebuild or rescore a specific architecture. Valid
architectures are: architectures are:
armhf, arm64, amd64, i386, powerpc, ppc64el, riscv64, s390x. armhf, arm64, amd64, amd64v3, i386, powerpc, ppc64el, riscv64, s390x.
.TP .TP
Batch processing: Batch processing:
.IP .IP
@ -66,7 +66,7 @@ Rescore builds to <priority>.
\fB\-\-arch\fR=\fIARCHITECTURE\fR \fB\-\-arch\fR=\fIARCHITECTURE\fR
Affect only 'architecture' (can be used several Affect only 'architecture' (can be used several
times). Valid architectures are: times). Valid architectures are:
arm64, amd64, i386, powerpc, ppc64el, riscv64, s390x. armhf, arm64, amd64, amd64v3, i386, powerpc, ppc64el, riscv64, s390x.
.IP .IP
\fB\-A=\fIARCHIVE\fR \fB\-A=\fIARCHIVE\fR
Act on the named archive (ppa) instead of on the main Ubuntu archive. Act on the named archive (ppa) instead of on the main Ubuntu archive.

View File

@ -23,6 +23,7 @@
import argparse import argparse
import sys import sys
from typing import Any, NoReturn
from launchpadlib.errors import HTTPError from launchpadlib.errors import HTTPError
from launchpadlib.launchpad import Launchpad from launchpadlib.launchpad import Launchpad
@ -33,7 +34,7 @@ from ubuntutools.config import UDTConfig
Logger = getLogger() Logger = getLogger()
def error_out(msg, *args): def error_out(msg: str, *args: Any) -> NoReturn:
Logger.error(msg, *args) Logger.error(msg, *args)
sys.exit(1) sys.exit(1)

View File

@ -22,6 +22,7 @@
# pylint: enable=invalid-name # pylint: enable=invalid-name
import sys import sys
from typing import NoReturn
from debian.changelog import Changelog from debian.changelog import Changelog
@ -30,7 +31,7 @@ from ubuntutools import getLogger
Logger = getLogger() Logger = getLogger()
def usage(exit_code=1): def usage(exit_code: int = 1) -> NoReturn:
Logger.info( Logger.info(
"""Usage: merge-changelog <left changelog> <right changelog> """Usage: merge-changelog <left changelog> <right changelog>

View File

@ -38,6 +38,7 @@ import shutil
import subprocess import subprocess
import sys import sys
from contextlib import suppress from contextlib import suppress
from typing import NoReturn
import debian.deb822 import debian.deb822
from distro_info import DebianDistroInfo, DistroDataOutdated, UbuntuDistroInfo from distro_info import DebianDistroInfo, DistroDataOutdated, UbuntuDistroInfo
@ -411,7 +412,7 @@ class PbuilderDist:
] + arguments ] + arguments
def show_help(exit_code=0): def show_help(exit_code: int = 0) -> NoReturn:
"""help() -> None """help() -> None
Print a help message for pbuilder-dist, and exit with the given code. Print a help message for pbuilder-dist, and exit with the given code.

View File

@ -22,6 +22,7 @@ from argparse import ArgumentParser
import yaml import yaml
from launchpadlib.launchpad import Launchpad from launchpadlib.launchpad import Launchpad
from ubuntutools.question import YesNoQuestion
from ubuntutools.utils import get_url from ubuntutools.utils import get_url
# proposed-migration is only concerned with the devel series; unlike other # proposed-migration is only concerned with the devel series; unlike other
@ -56,10 +57,8 @@ def claim_excuses_bug(launchpad, bug, package):
if our_task.assignee: if our_task.assignee:
print(f"Currently assigned to {our_task.assignee.name}") print(f"Currently assigned to {our_task.assignee.name}")
print("""Do you want to claim this bug? [yN] """, end="") answer = YesNoQuestion().ask("Do you want to claim this bug?", "no")
sys.stdout.flush() if answer == "yes":
response = sys.stdin.readline()
if response.strip().lower().startswith("y"):
our_task.assignee = launchpad.me our_task.assignee = launchpad.me
our_task.lp_save() our_task.lp_save()
return True return True
@ -131,7 +130,9 @@ def main():
if not proposed_version: if not proposed_version:
print(f"Package {args.package} not found in -proposed.") print(f"Package {args.package} not found in -proposed.")
sys.exit(1) sys.exit(1)
create_excuses_bug(args.launchpad, args.package, proposed_version) answer = YesNoQuestion().ask("Do you want to create a bug?", "no")
if answer == "yes":
create_excuses_bug(args.launchpad, args.package, proposed_version)
except ValueError as e: except ValueError as e:
sys.stderr.write(f"{e}\n") sys.stderr.write(f"{e}\n")
else: else:

View File

@ -4,3 +4,7 @@ line-length = 99
[tool.isort] [tool.isort]
line_length = 99 line_length = 99
profile = "black" profile = "black"
[tool.mypy]
disallow_incomplete_defs = true
ignore_missing_imports = true

View File

@ -46,7 +46,7 @@ Logger = getLogger()
# #
def main(): def main() -> None:
# Our usage options. # Our usage options.
usage = "%(prog)s [options] <source package> [<target release> [base version]]" usage = "%(prog)s [options] <source package> [<target release> [base version]]"
parser = argparse.ArgumentParser(usage=usage) parser = argparse.ArgumentParser(usage=usage)
@ -153,6 +153,7 @@ def main():
import DNS # pylint: disable=import-outside-toplevel import DNS # pylint: disable=import-outside-toplevel
DNS.DiscoverNameServers() DNS.DiscoverNameServers()
# imported earlier, pylint: disable-next=possibly-used-before-assignment
mxlist = DNS.mxlookup(bug_mail_domain) mxlist = DNS.mxlookup(bug_mail_domain)
firstmx = mxlist[0] firstmx = mxlist[0]
mailserver_host = firstmx[1] mailserver_host = firstmx[1]
@ -214,6 +215,7 @@ def main():
if not args.release: if not args.release:
if lpapi: if lpapi:
# imported earlier, pylint: disable-next=possibly-used-before-assignment
args.release = Distribution("ubuntu").getDevelopmentSeries().name args.release = Distribution("ubuntu").getDevelopmentSeries().name
else: else:
ubu_info = UbuntuDistroInfo() ubu_info = UbuntuDistroInfo()
@ -377,6 +379,7 @@ def main():
# Map status to the values expected by LP API # Map status to the values expected by LP API
mapping = {"new": "New", "confirmed": "Confirmed"} mapping = {"new": "New", "confirmed": "Confirmed"}
# Post sync request using LP API # Post sync request using LP API
# imported earlier, pylint: disable-next=possibly-used-before-assignment
post_bug(srcpkg, subscribe, mapping[status], title, report) post_bug(srcpkg, subscribe, mapping[status], title, report)
else: else:
email_from = ubu_email(export=False)[1] email_from = ubu_email(export=False)[1]

View File

@ -4,16 +4,45 @@ set -eu
# Copyright 2023, Canonical Ltd. # Copyright 2023, Canonical Ltd.
# SPDX-License-Identifier: GPL-3.0 # SPDX-License-Identifier: GPL-3.0
PYTHON_SCRIPTS=$(grep -l -r '^#! */usr/bin/python3$' .) PYTHON_SCRIPTS=$(find . -maxdepth 1 -type f -exec grep -l '^#! */usr/bin/python3$' {} +)
echo "Running black..." run_black() {
black --check --diff . $PYTHON_SCRIPTS echo "Running black..."
black -C --check --diff . ${PYTHON_SCRIPTS}
}
echo "Running isort..." run_isort() {
isort --check-only --diff . echo "Running isort..."
isort --check-only --diff .
}
echo "Running flake8..." run_flake8() {
flake8 --max-line-length=99 --ignore=E203,W503 . $PYTHON_SCRIPTS echo "Running flake8..."
flake8 --max-line-length=99 --ignore=E203,W503 . $PYTHON_SCRIPTS
}
echo "Running pylint..." run_mypy() {
pylint $(find * -name '*.py') $PYTHON_SCRIPTS echo "Running mypy..."
mypy .
mypy --scripts-are-modules $PYTHON_SCRIPTS
}
run_pylint() {
echo "Running pylint..."
pylint "$@" $(find * -name '*.py') $PYTHON_SCRIPTS
}
if test "${1-}" = "--errors-only"; then
# Run only linters that can detect real errors (ignore formatting)
run_black || true
run_isort || true
run_flake8 || true
run_mypy
run_pylint --errors-only
else
run_black
run_isort
run_flake8
run_mypy
run_pylint
fi

View File

@ -22,6 +22,7 @@
import argparse import argparse
import fnmatch import fnmatch
import functools
import logging import logging
import os import os
import shutil import shutil
@ -49,7 +50,6 @@ from ubuntutools.requestsync.mail import get_debian_srcpkg as requestsync_mail_g
from ubuntutools.version import Version from ubuntutools.version import Version
Logger = getLogger() Logger = getLogger()
cached_sync_blocklist = None
def remove_signature(dscname): def remove_signature(dscname):
@ -436,6 +436,14 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False, ye
close_bugs(bugs, src_pkg.source, src_pkg.version.full_version, changes, sponsoree) close_bugs(bugs, src_pkg.source, src_pkg.version.full_version, changes, sponsoree)
@functools.lru_cache(maxsize=1)
def _fetch_sync_blocklist() -> str:
url = "https://ubuntu-archive-team.ubuntu.com/sync-blocklist.txt"
with urllib.request.urlopen(url) as f:
sync_blocklist = f.read().decode("utf-8")
return sync_blocklist
def is_blocklisted(query): def is_blocklisted(query):
"""Determine if package "query" is in the sync blocklist """Determine if package "query" is in the sync blocklist
Returns tuple of (blocklisted, comments) Returns tuple of (blocklisted, comments)
@ -456,24 +464,20 @@ def is_blocklisted(query):
if diff.status == "Blacklisted always": if diff.status == "Blacklisted always":
blocklisted = "ALWAYS" blocklisted = "ALWAYS"
global cached_sync_blocklist try:
if not cached_sync_blocklist: sync_blocklist = _fetch_sync_blocklist()
url = "https://ubuntu-archive-team.ubuntu.com/sync-blocklist.txt" except OSError:
try: print("WARNING: unable to download the sync blocklist. Erring on the side of caution.")
with urllib.request.urlopen(url) as f: return ("ALWAYS", "INTERNAL ERROR: Unable to fetch sync blocklist")
cached_sync_blocklist = f.read().decode("utf-8")
except:
print("WARNING: unable to download the sync blocklist. Erring on the side of caution.")
return ("ALWAYS", "INTERNAL ERROR: Unable to fetch sync blocklist")
applicable_lines = [] applicable_lines = []
for line in cached_sync_blocklist.splitlines(): for line in sync_blocklist.splitlines():
if not line.strip(): if not line.strip():
applicable_lines = [] applicable_lines = []
continue continue
applicable_lines.append(line) applicable_lines.append(line)
try: try:
line = line[:line.index("#")] line = line[: line.index("#")]
except ValueError: except ValueError:
pass pass
source = line.strip() source = line.strip()
@ -523,7 +527,7 @@ def parse():
"-y", "-y",
"--yes", "--yes",
action="store_true", action="store_true",
help="Automatically sync without prompting. Use with caution and care." help="Automatically sync without prompting. Use with caution and care.",
) )
parser.add_argument("-d", "--distribution", help="Debian distribution to sync from.") parser.add_argument("-d", "--distribution", help="Debian distribution to sync from.")
parser.add_argument("-r", "--release", help="Specify target Ubuntu release.") parser.add_argument("-r", "--release", help="Specify target Ubuntu release.")
@ -776,7 +780,9 @@ def main():
continue continue
if args.lp: if args.lp:
if not copy(src_pkg, args.release, args.bugs, sponsoree, args.simulate, args.force, args.yes): if not copy(
src_pkg, args.release, args.bugs, sponsoree, args.simulate, args.force, args.yes
):
continue continue
else: else:
os.environ["DEB_VENDOR"] = "Ubuntu" os.environ["DEB_VENDOR"] = "Ubuntu"

View File

@ -87,17 +87,13 @@ def retry_builds(pkg, archs):
return f"Retrying builds of '{pkg.source_package_name}':\n{msg}" return f"Retrying builds of '{pkg.source_package_name}':\n{msg}"
def main(): def parse_args(argv: list[str], valid_archs: set[str]) -> argparse.Namespace:
"""Parse command line arguments and return namespace."""
# Usage. # Usage.
usage = "%(prog)s <srcpackage> <release> <operation>\n\n" usage = "%(prog)s <srcpackage> <release> <operation>\n\n"
usage += "Where operation may be one of: rescore, retry, or status.\n" usage += "Where operation may be one of: rescore, retry, or status.\n"
usage += "Only Launchpad Buildd Admins may rescore package builds." usage += "Only Launchpad Buildd Admins may rescore package builds."
# Valid architectures.
valid_archs = set(
["armhf", "arm64", "amd64", "i386", "powerpc", "ppc64el", "riscv64", "s390x"]
)
# Prepare our option parser. # Prepare our option parser.
parser = argparse.ArgumentParser(usage=usage) parser = argparse.ArgumentParser(usage=usage)
@ -148,7 +144,23 @@ def main():
parser.add_argument("packages", metavar="package", nargs="*", help=argparse.SUPPRESS) parser.add_argument("packages", metavar="package", nargs="*", help=argparse.SUPPRESS)
# Parse our options. # Parse our options.
args = parser.parse_args() args = parser.parse_args(argv)
if not args.batch:
# Check we have the correct number of arguments.
if len(args.packages) < 3:
parser.error("Incorrect number of arguments.")
return args
def main():
# Valid architectures.
valid_archs = set(
["armhf", "arm64", "amd64", "amd64v3", "i386", "powerpc", "ppc64el", "riscv64", "s390x"]
)
args = parse_args(sys.argv[1:], valid_archs)
launchpad = Launchpad.login_with("ubuntu-dev-tools", "production", version="devel") launchpad = Launchpad.login_with("ubuntu-dev-tools", "production", version="devel")
ubuntu = launchpad.distributions["ubuntu"] ubuntu = launchpad.distributions["ubuntu"]
@ -167,21 +179,13 @@ def main():
Logger.error(error) Logger.error(error)
sys.exit(1) sys.exit(1)
else: else:
# Check we have the correct number of arguments. package = str(args.packages[0]).lower()
if len(args.packages) < 3: release = str(args.packages[1]).lower()
parser.error("Incorrect number of arguments.") operation = str(args.packages[2]).lower()
try:
package = str(args.packages[0]).lower()
release = str(args.packages[1]).lower()
operation = str(args.packages[2]).lower()
except IndexError:
parser.print_help()
sys.exit(1)
archive = launchpad.archives.getByReference(reference=args.archive) archive = launchpad.archives.getByReference(reference=args.archive)
try: try:
distroseries = ubuntu.getSeries(name_or_version=release) distroseries = ubuntu.getSeries(name_or_version=release.split("-")[0])
except lazr.restfulclient.errors.NotFound as error: except lazr.restfulclient.errors.NotFound as error:
Logger.error(error) Logger.error(error)
sys.exit(1) sys.exit(1)
@ -234,11 +238,11 @@ def main():
# are in place. # are in place.
if operation == "retry": if operation == "retry":
necessary_privs = archive.checkUpload( necessary_privs = archive.checkUpload(
component=sources.getComponent(), component=component,
distroseries=distroseries, distroseries=distroseries,
person=launchpad.me, person=launchpad.me,
pocket=pocket, pocket=pocket,
sourcepackagename=sources.getPackageName(), sourcepackagename=sources.source_package_name,
) )
if not necessary_privs: if not necessary_privs:
Logger.error( Logger.error(

View File

@ -65,7 +65,7 @@ def main():
err = True err = True
continue continue
Logger.info(prefix + version) Logger.info("%s%s", prefix, version)
if err: if err:
sys.exit(1) sys.exit(1)

View File

@ -340,11 +340,9 @@ class SourcePackage(ABC):
def _archive_servers(self): def _archive_servers(self):
"Generator for mirror and master servers" "Generator for mirror and master servers"
# Always provide the mirrors first # Always provide the mirrors first
for server in self.mirrors: yield from self.mirrors
yield server
# Don't repeat servers that are in both mirrors and masters # Don't repeat servers that are in both mirrors and masters
for server in set(self.masters) - set(self.mirrors): yield from set(self.masters) - set(self.mirrors)
yield server
def _source_urls(self, name): def _source_urls(self, name):
"Generator of sources for name" "Generator of sources for name"
@ -635,8 +633,7 @@ class DebianSourcePackage(SourcePackage):
def _source_urls(self, name): def _source_urls(self, name):
"Generator of sources for name" "Generator of sources for name"
for url in super()._source_urls(name): yield from super()._source_urls(name)
yield url
if name in self.snapshot_files: if name in self.snapshot_files:
yield self.snapshot_files[name] yield self.snapshot_files[name]
@ -734,6 +731,7 @@ class PersonalPackageArchiveSourcePackage(UbuntuSourcePackage):
class UbuntuCloudArchiveSourcePackage(PersonalPackageArchiveSourcePackage): class UbuntuCloudArchiveSourcePackage(PersonalPackageArchiveSourcePackage):
"Download / unpack an Ubuntu Cloud Archive source package" "Download / unpack an Ubuntu Cloud Archive source package"
TEAM = "ubuntu-cloud-archive" TEAM = "ubuntu-cloud-archive"
PROJECT = "cloud-archive" PROJECT = "cloud-archive"
VALID_POCKETS = ["updates", "proposed", "staging"] VALID_POCKETS = ["updates", "proposed", "staging"]
@ -930,8 +928,8 @@ class UbuntuCloudArchiveSourcePackage(PersonalPackageArchiveSourcePackage):
class _WebJSON: class _WebJSON:
def getHostUrl(self): # pylint: disable=no-self-use def getHostUrl(self):
raise Exception("Not implemented") raise NotImplementedError(f"{self.__class__.__name__}.getHostUrl() is not implemented")
def load(self, path=""): def load(self, path=""):
reader = codecs.getreader("utf-8") reader = codecs.getreader("utf-8")

View File

@ -50,7 +50,7 @@ class UDTConfig:
"KEYID": None, "KEYID": None,
} }
# Populated from the configuration files: # Populated from the configuration files:
config = {} config: dict[str, str] = {}
def __init__(self, no_conf=False, prefix=None): def __init__(self, no_conf=False, prefix=None):
self.no_conf = no_conf self.no_conf = no_conf
@ -61,7 +61,7 @@ class UDTConfig:
self.config = self.parse_devscripts_config() self.config = self.parse_devscripts_config()
@staticmethod @staticmethod
def parse_devscripts_config(): def parse_devscripts_config() -> dict[str, str]:
"""Read the devscripts configuration files, and return the values as a """Read the devscripts configuration files, and return the values as a
dictionary dictionary
""" """

View File

@ -26,6 +26,7 @@ import logging
import os import os
import re import re
from copy import copy from copy import copy
from typing import Any
from urllib.error import URLError from urllib.error import URLError
from urllib.parse import urlparse from urllib.parse import urlparse
@ -139,7 +140,7 @@ class BaseWrapper(metaclass=MetaWrapper):
A base class from which other wrapper classes are derived. A base class from which other wrapper classes are derived.
""" """
resource_type: str = None # it's a base class after all resource_type: str | tuple[str, str] = "" # it's a base class after all
def __new__(cls, data): def __new__(cls, data):
if isinstance(data, str) and data.startswith(str(Launchpad._root_uri)): if isinstance(data, str) and data.startswith(str(Launchpad._root_uri)):
@ -667,20 +668,19 @@ class Archive(BaseWrapper):
rversion = getattr(record, "binary_package_version", None) rversion = getattr(record, "binary_package_version", None)
else: else:
rversion = getattr(record, "source_package_version", None) rversion = getattr(record, "source_package_version", None)
skipmsg = f"Skipping version {rversion}: "
if record.pocket not in pockets: if record.pocket not in pockets:
err_msg = f"pocket {record.pocket} not in ({','.join(pockets)})" err_msg = f"pocket {record.pocket} not in ({','.join(pockets)})"
Logger.debug(skipmsg + err_msg) Logger.debug("Skipping version %s: %s", rversion, err_msg)
continue continue
if record.status not in statuses: if record.status not in statuses:
err_msg = f"status {record.status} not in ({','.join(statuses)})" err_msg = f"status {record.status} not in ({','.join(statuses)})"
Logger.debug(skipmsg + err_msg) Logger.debug("Skipping version %s: %s", rversion, err_msg)
continue continue
release = wrapper(record) release = wrapper(record)
if binary and archtag and archtag != release.arch: if binary and archtag and archtag != release.arch:
err_msg = f"arch {release.arch} does not match requested arch {archtag}" err_msg = f"arch {release.arch} does not match requested arch {archtag}"
Logger.debug(skipmsg + err_msg) Logger.debug("Skipping version %s: %s", rversion, err_msg)
continue continue
# results are ordered so first is latest # results are ordered so first is latest
cache[index] = release cache[index] = release
@ -1502,7 +1502,7 @@ class Packageset(BaseWrapper): # pylint: disable=too-few-public-methods
resource_type = "packageset" resource_type = "packageset"
_lp_packagesets = None _lp_packagesets = None
_source_sets = {} _source_sets: dict[tuple[str, str | None, bool], Any] = {}
@classmethod @classmethod
def setsIncludingSource(cls, sourcepackagename, distroseries=None, direct_inclusion=False): def setsIncludingSource(cls, sourcepackagename, distroseries=None, direct_inclusion=False):

View File

@ -471,6 +471,7 @@ class PullPkg:
uri, uri,
) )
vcscmd = ""
if vcs == "Bazaar": if vcs == "Bazaar":
vcscmd = " $ bzr branch " + uri vcscmd = " $ bzr branch " + uri
elif vcs == "Git": elif vcs == "Git":

View File

@ -62,8 +62,17 @@ def get_debian_srcpkg(name, release):
return DebianSourcePackage(package=name, series=release).lp_spph return DebianSourcePackage(package=name, series=release).lp_spph
def get_ubuntu_srcpkg(name, release): def get_ubuntu_srcpkg(name, release, pocket="Proposed"):
return UbuntuSourcePackage(package=name, series=release).lp_spph srcpkg = UbuntuSourcePackage(package=name, series=release, pocket=pocket)
try:
return srcpkg.lp_spph
except PackageNotFoundException:
if pocket != "Release":
parent_pocket = "Release"
if pocket == "Updates":
parent_pocket = "Proposed"
return get_ubuntu_srcpkg(name, release, parent_pocket)
raise
def need_sponsorship(name, component, release): def need_sponsorship(name, component, release):

View File

@ -16,6 +16,7 @@
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. # OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
import sys import sys
from typing import NoReturn
from ubuntutools.question import Question, YesNoQuestion from ubuntutools.question import Question, YesNoQuestion
@ -42,7 +43,7 @@ def ask_for_manual_fixing():
user_abort() user_abort()
def user_abort(): def user_abort() -> NoReturn:
"""Print abort and quit the program.""" """Print abort and quit the program."""
print("User abort.") print("User abort.")

View File

@ -17,6 +17,7 @@
import logging import logging
import os import os
import pathlib
import re import re
import subprocess import subprocess
import sys import sys
@ -407,22 +408,16 @@ class SourcePackage:
return True return True
def _run_lintian(self): def _run_lintian(self) -> str:
"""Runs lintian on either the source or binary changes file. """Runs lintian on either the source or binary changes file.
Returns the filename of the created lintian output file. Returns the filename of the created lintian output file.
""" """
# Determine whether to use the source or binary build for lintian # Determine whether to use the source or binary build for lintian
package_and_version = f"{self._package}_{strip_epoch(self._version)}"
if self._build_log: if self._build_log:
build_changes = ( build_changes = f"{package_and_version}_{self._builder.get_architecture()}.changes"
self._package
+ "_"
+ strip_epoch(self._version)
+ "_"
+ self._builder.get_architecture()
+ ".changes"
)
changes_for_lintian = os.path.join(self._buildresult, build_changes) changes_for_lintian = os.path.join(self._buildresult, build_changes)
else: else:
changes_for_lintian = self._changes_file changes_for_lintian = self._changes_file
@ -430,18 +425,12 @@ class SourcePackage:
# Check lintian # Check lintian
assert os.path.isfile(changes_for_lintian), f"{changes_for_lintian} does not exist." assert os.path.isfile(changes_for_lintian), f"{changes_for_lintian} does not exist."
cmd = ["lintian", "-IE", "--pedantic", "-q", "--profile", "ubuntu", changes_for_lintian] cmd = ["lintian", "-IE", "--pedantic", "-q", "--profile", "ubuntu", changes_for_lintian]
lintian_filename = os.path.join( lintian_file = pathlib.Path(self._workdir) / f"{package_and_version}.lintian"
self._workdir, self._package + "_" + strip_epoch(self._version) + ".lintian" Logger.debug("%s > %s", " ".join(cmd), lintian_file)
) with lintian_file.open("wb") as outfile:
Logger.debug("%s > %s", " ".join(cmd), lintian_filename) subprocess.run(cmd, stdout=outfile, check=True)
report = subprocess.check_output(cmd, encoding="utf-8")
# write lintian report file return str(lintian_file)
lintian_file = open(lintian_filename, "w", encoding="utf-8")
lintian_file.writelines(report)
lintian_file.close()
return lintian_filename
def sync(self, upload, series, bug_number, requester): def sync(self, upload, series, bug_number, requester):
"""Does a sync of the source package.""" """Does a sync of the source package."""

View File

@ -46,11 +46,9 @@ def is_command_available(command, check_sbin=False):
def check_dependencies(): def check_dependencies():
"Do we have all the commands we need for full functionality?" "Do we have all the commands we need for full functionality?"
missing = [] missing = []
for cmd in ("patch", "bzr", "quilt", "dput", "lintian"): for cmd in ("patch", "quilt", "dput", "lintian"):
if not is_command_available(cmd): if not is_command_available(cmd):
missing.append(cmd) missing.append(cmd)
if not is_command_available("bzr-buildpackage"):
missing.append("bzr-builddeb")
if not any( if not any(
is_command_available(cmd, check_sbin=True) for cmd in ("pbuilder", "sbuild", "cowbuilder") is_command_available(cmd, check_sbin=True) for cmd in ("pbuilder", "sbuild", "cowbuilder")
): ):
@ -212,14 +210,14 @@ def get_open_ubuntu_bug_task(launchpad, bug, branch=None):
sys.exit(1) sys.exit(1)
elif len(ubuntu_tasks) == 1: elif len(ubuntu_tasks) == 1:
task = ubuntu_tasks[0] task = ubuntu_tasks[0]
if len(ubuntu_tasks) > 1 and branch and branch[1] == "ubuntu": elif branch and branch[1] == "ubuntu":
tasks = [t for t in ubuntu_tasks if t.get_series() == branch[2] and t.package == branch[3]] tasks = [t for t in ubuntu_tasks if t.get_series() == branch[2] and t.package == branch[3]]
if len(tasks) > 1: if len(tasks) > 1:
# A bug targeted to the development series? # A bug targeted to the development series?
tasks = [t for t in tasks if t.series is not None] tasks = [t for t in tasks if t.series is not None]
assert len(tasks) == 1 assert len(tasks) == 1
task = tasks[0] task = tasks[0]
elif len(ubuntu_tasks) > 1: else:
task_list = [t.get_short_info() for t in ubuntu_tasks] task_list = [t.get_short_info() for t in ubuntu_tasks]
Logger.debug( Logger.debug(
"%i Ubuntu tasks exist for bug #%i.\n%s", "%i Ubuntu tasks exist for bug #%i.\n%s",

View File

@ -60,7 +60,7 @@ class ExamplePackage:
with tempfile.TemporaryDirectory() as tmpdir: with tempfile.TemporaryDirectory() as tmpdir:
self._create(Path(tmpdir)) self._create(Path(tmpdir))
def _create(self, directory: Path): def _create(self, directory: Path) -> None:
pkgdir = directory / self.dirname pkgdir = directory / self.dirname
pkgdir.mkdir() pkgdir.mkdir()
(pkgdir / self.content_filename).write_text(self.content_text) (pkgdir / self.content_filename).write_text(self.content_text)

View File

@ -28,6 +28,7 @@ class BinaryTests(unittest.TestCase):
def test_keyring_installed(self): def test_keyring_installed(self):
"""Smoke test for required lp api dependencies""" """Smoke test for required lp api dependencies"""
try: try:
# pylint: disable-next=import-outside-toplevel,unused-import
import keyring # noqa: F401 import keyring # noqa: F401
except ModuleNotFoundError: except ModuleNotFoundError as error:
raise ModuleNotFoundError("package python3-keyring is not installed") raise ModuleNotFoundError("package python3-keyring is not installed") from error

View File

@ -12,7 +12,7 @@
# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
# OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
# PERFORMANCE OF THIS SOFTWARE. # PERFORMANCE OF THIS SOFTWARE.
""" Tests for running_autopkgtests """Tests for running_autopkgtests
Tests using cached data from autopkgtest servers. Tests using cached data from autopkgtest servers.
These tests only ensure code changes don't change parsing behavior These tests only ensure code changes don't change parsing behavior