Compare commits

...

31 Commits
0.203 ... main

Author SHA1 Message Date
Simon Quigley
466e2784de Upload to Unstable 2025-03-04 13:43:32 -06:00
Simon Quigley
ba3f0511f9 syncpackage: Catch exceptions cleanly, simply skipping to the next package (erring on the side of caution) if there is an error doing the download (LP: #1943286). 2025-03-04 13:42:50 -06:00
Simon Quigley
2e550ceff2 syncpackage: Cache the sync blocklist in-memory, so it's not fetched multiple times when syncing more than one package. 2025-03-04 13:39:07 -06:00
Simon Quigley
6c8a5d74bd syncpackage: s/syncblacklist/syncblocklist/g 2025-03-04 13:29:02 -06:00
Simon Quigley
3d11516599 mk-sbuild: default to using UTC for schroots (LP: #2097159). 2025-03-04 13:22:40 -06:00
Simon Quigley
5a20308ab1 Read ~/.devscripts in a more robust way, to ideally pick up multi-line variables (Closes: #725418). 2025-03-04 13:17:30 -06:00
Simon Quigley
b551877651 Add a changelog entry 2025-03-04 13:10:04 -06:00
ferbraher
4a4c4e0a27 Parsing arch parameter to getBinaryPackage() 2025-03-04 13:08:59 -06:00
Simon Quigley
865c1c97bc Add a changelog entry 2025-03-04 13:07:42 -06:00
Shengjing Zhu
d09718e976 import-bug-from-debian: package option is overridden and not used 2025-03-04 13:07:11 -06:00
Simon Quigley
bff7baecc9 Add a changelog entry 2025-03-04 13:06:38 -06:00
Dan Bungert
45fbbb5bd1 mk-sbuild: enable pkgmaintainermangler
mk-sbuild installs pkgbinarymangler into the schroot.  Of of the
provided tools in pkgbinarymangler is pkgmaintainermangler.
pkgmaintainermangler is disabled by default, and enabled with
configuration.

A difference between launchpad builds of a synced package and an sbuild
is that the maintainer information will be different.

Enable pkgmaintainermangler to close this difference.
2025-03-04 13:05:57 -06:00
Simon Quigley
ca217c035e Add a new changelog entry 2025-03-04 13:04:49 -06:00
Simon Quigley
b5e117788b Upload to Unstable 2025-03-01 11:30:18 -06:00
Simon Quigley
ddba2d1e98 Update Standards-Version to 4.7.2, no changes needed. 2025-03-01 11:29:53 -06:00
Simon Quigley
02d65a5804 [syncpackage] Do not use exit(1) on an error or exception unless it applies to all packages, instead return None so we can continue to the next package. 2025-03-01 11:26:59 -06:00
Simon Quigley
bda85fa6a8 [syncpackage] Add support for -y or --yes, noted that it should be used with care. 2025-03-01 11:22:52 -06:00
Simon Quigley
86a83bf74d [syncpackage] Within fetch_source_pkg, do not exit(1) on an error or exception, simply return None so we can continue to the next package. 2025-03-01 11:17:02 -06:00
Simon Quigley
162e758671 [syncpackage] When syncing multiple packages, if one of the packages is in the sync blocklist, do not exit, simply continue. 2025-03-01 11:12:49 -06:00
Simon Quigley
049425adb7 Add debian/files to .gitignore 2025-03-01 11:11:34 -06:00
Simon Quigley
f6ca6cad92 Add a new changelog entry 2025-03-01 11:11:17 -06:00
Simon Quigley
3dc17934d6 Upload to Unstable 2025-02-24 19:55:03 -06:00
Simon Quigley
10a176567a Remove mail line from default ~/.sbuildrc, to resolve the undeclared dependency on sendmail (Closes: #1074632). 2025-02-24 19:52:59 -06:00
Simon Quigley
86b366c6c5 Add a large warning at the top of mk-sbuild encouraging the use of the unshare backend. This is to provide ample warning to users. 2025-02-24 19:15:55 -06:00
Simon Quigley
50b580b30e Add a manpage for running-autopkgtests. 2025-02-24 18:51:12 -06:00
Simon Quigley
6ba0641f63 Rename bitesize to lp-bitesize (Closes: #1076224). 2025-02-24 18:51:10 -06:00
Simon Quigley
1e815db9d2 Add my name to the copyright file. 2025-02-24 18:35:20 -06:00
Simon Quigley
e2f43318bd Add several Lintian overrides related to .pyc files. 2025-02-24 18:34:18 -06:00
Julien Plissonneau Duquène
cdd81232d9 Fix reverse-depends -b crash on packages that b-d on themselves (Closes: #1087760). 2025-02-24 18:31:33 -06:00
Simon Quigley
65044d84d9 Update Standards-Version to 4.7.1, no changes needed. 2025-02-24 18:26:59 -06:00
Mattia Rizzolo
19e40b49c2
Fix minor typo in pbuilder-dist(1)
LP: #2096956
Thanks: Rolf Leggewie for the patch
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2025-01-30 07:52:22 +01:00
18 changed files with 240 additions and 98 deletions

1
debian/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
files

56
debian/changelog vendored
View File

@ -1,3 +1,59 @@
ubuntu-dev-tools (0.206) unstable; urgency=medium
[ Dan Bungert ]
* mk-sbuild: enable pkgmaintainermangler
[ Shengjing Zhu ]
* import-bug-from-debian: package option is overridden and not used
[ Fernando Bravo Hernández ]
* Parsing arch parameter to getBinaryPackage() (LP: #2081861)
[ Simon Quigley ]
* Read ~/.devscripts in a more robust way, to ideally pick up multi-line
variables (Closes: #725418).
* mk-sbuild: default to using UTC for schroots (LP: #2097159).
* syncpackage: s/syncblacklist/syncblocklist/g
* syncpackage: Cache the sync blocklist in-memory, so it's not fetched
multiple times when syncing more than one package.
* syncpackage: Catch exceptions cleanly, simply skipping to the next
package (erring on the side of caution) if there is an error doing the
download (LP: #1943286).
-- Simon Quigley <tsimonq2@debian.org> Tue, 04 Mar 2025 13:43:15 -0600
ubuntu-dev-tools (0.205) unstable; urgency=medium
* [syncpackage] When syncing multiple packages, if one of the packages is in
the sync blocklist, do not exit, simply continue.
* [syncpackage] Do not use exit(1) on an error or exception unless it
applies to all packages, instead return None so we can continue to the
next package.
* [syncpackage] Add support for -y or --yes, noted that it should be used
with care.
* Update Standards-Version to 4.7.2, no changes needed.
-- Simon Quigley <tsimonq2@debian.org> Sat, 01 Mar 2025 11:29:54 -0600
ubuntu-dev-tools (0.204) unstable; urgency=medium
[ Simon Quigley ]
* Update Standards-Version to 4.7.1, no changes needed.
* Add several Lintian overrides related to .pyc files.
* Add my name to the copyright file.
* Rename bitesize to lp-bitesize (Closes: #1076224).
* Add a manpage for running-autopkgtests.
* Add a large warning at the top of mk-sbuild encouraging the use of the
unshare backend. This is to provide ample warning to users.
* Remove mail line from default ~/.sbuildrc, to resolve the undeclared
dependency on sendmail (Closes: #1074632).
[ Julien Plissonneau Duquène ]
* Fix reverse-depends -b crash on packages that b-d on themselves
(Closes: #1087760).
-- Simon Quigley <tsimonq2@debian.org> Mon, 24 Feb 2025 19:54:39 -0600
ubuntu-dev-tools (0.203) unstable; urgency=medium
[ Steve Langasek ]

2
debian/control vendored
View File

@ -31,7 +31,7 @@ Build-Depends:
python3-requests <!nocheck>,
python3-setuptools,
python3-yaml <!nocheck>,
Standards-Version: 4.7.0
Standards-Version: 4.7.2
Rules-Requires-Root: no
Vcs-Git: https://git.launchpad.net/ubuntu-dev-tools
Vcs-Browser: https://git.launchpad.net/ubuntu-dev-tools

6
debian/copyright vendored
View File

@ -11,6 +11,7 @@ Files: backportpackage
doc/check-symbols.1
doc/requestsync.1
doc/ubuntu-iso.1
doc/running-autopkgtests.1
GPL-2
README.updates
requestsync
@ -25,6 +26,7 @@ Copyright: 2007, Albert Damen <albrt@gmx.net>
2010, Evan Broder <evan@ebroder.net>
2006-2007, Luke Yelavich <themuso@ubuntu.com>
2009-2010, Michael Bienia <geser@ubuntu.com>
2024-2025, Simon Quigley <tsimonq2@debian.org>
2010-2011, Stefano Rivera <stefanor@ubuntu.com>
2008, Stephan Hermann <sh@sourcecode.de>
2007, Steve Kowalik <stevenk@ubuntu.com>
@ -72,14 +74,14 @@ License: GPL-2+
On Debian systems, the complete text of the GNU General Public License
version 2 can be found in the /usr/share/common-licenses/GPL-2 file.
Files: doc/bitesize.1
Files: doc/lp-bitesize.1
doc/check-mir.1
doc/grab-merge.1
doc/merge-changelog.1
doc/pm-helper.1
doc/setup-packaging-environment.1
doc/syncpackage.1
bitesize
lp-bitesize
check-mir
GPL-3
grab-merge

3
debian/source/lintian-overrides vendored Normal file
View File

@ -0,0 +1,3 @@
# pyc files are machine-generated; they're expected to have long lines and have unstated copyright
source: file-without-copyright-information *.pyc [debian/copyright]
source: very-long-line-length-in-source-file * > 512 [*.pyc:*]

View File

@ -1,21 +1,21 @@
.TH bitesize "1" "May 9 2010" "ubuntu-dev-tools"
.TH lp-bitesize "1" "May 9 2010" "ubuntu-dev-tools"
.SH NAME
bitesize \- Add \fBbitesize\fR tag to bugs and add a comment.
lp-bitesize \- Add \fBbitesize\fR tag to bugs and add a comment.
.SH SYNOPSIS
.B bitesize \fR<\fIbug number\fR>
.B lp-bitesize \fR<\fIbug number\fR>
.br
.B bitesize \-\-help
.B lp-bitesize \-\-help
.SH DESCRIPTION
\fBbitesize\fR adds a bitesize tag to the bug, if it's not there yet. It
\fBlp-bitesize\fR adds a bitesize tag to the bug, if it's not there yet. It
also adds a comment to the bug indicating that you are willing to help with
fixing it.
It checks for permission to operate on a given bug first,
then perform required tasks on Launchpad.
.SH OPTIONS
Listed below are the command line options for \fBbitesize\fR:
Listed below are the command line options for \fBlp-bitesize\fR:
.TP
.BR \-h ", " \-\-help
Display a help message and exit.
@ -48,7 +48,7 @@ The default value for \fB--lpinstance\fR.
.BR ubuntu\-dev\-tools (5)
.SH AUTHORS
\fBbitesize\fR and this manual page were written by Daniel Holbach
\fBlp-bitesize\fR and this manual page were written by Daniel Holbach
<daniel.holbach@canonical.com>.
.PP
Both are released under the terms of the GNU General Public License, version 3.

View File

@ -20,7 +20,7 @@ like for example \fBpbuilder\-feisty\fP, \fBpbuilder\-sid\fP, \fBpbuilder\-gutsy
.PP
The same applies to \fBcowbuilder\-dist\fP, which uses cowbuilder. The main
difference between both is that pbuilder compresses the created chroot as a
a tarball, thus using less disc space but needing to uncompress (and possibly
tarball, thus using less disc space but needing to uncompress (and possibly
compress) its contents again on each run, and cowbuilder doesn't do this.
.SH USAGE

View File

@ -0,0 +1,15 @@
.TH running\-autopkgtests "1" "18 January 2024" "ubuntu-dev-tools"
.SH NAME
running\-autopkgtests \- dumps a list of currently running autopkgtests
.SH SYNOPSIS
.B running\-autopkgtests
.SH DESCRIPTION
Dumps a list of currently running and queued tests in Autopkgtest.
Pass --running to only see running tests, or --queued to only see
queued tests. Passing both will print both, which is the default behavior.
.SH AUTHOR
.B running\-autopkgtests
was written by Chris Peterson <chris.peterson@canonical.com>.

View File

@ -58,7 +58,7 @@ Display more progress information.
\fB\-F\fR, \fB\-\-fakesync\fR
Perform a fakesync, to work around a tarball mismatch between Debian and
Ubuntu.
This option ignores blacklisting, and performs a local sync.
This option ignores blocklisting, and performs a local sync.
It implies \fB\-\-no\-lp\fR, and will leave a signed \fB.changes\fR file
for you to upload.
.TP

View File

@ -150,7 +150,7 @@ def process_bugs(
err = False
for bug in bugs:
ubupackage = package = bug.source
ubupackage = bug.source
if package:
ubupackage = package
bug_num = bug.bug_num

View File

@ -155,6 +155,7 @@ proxy="_unset_"
DEBOOTSTRAP_NO_CHECK_GPG=0
EATMYDATA=1
CCACHE=0
USE_PKGBINARYMANGLER=0
while :; do
case "$1" in
@ -303,10 +304,26 @@ if [ ! -w /var/lib/sbuild ]; then
# Prepare a usable default .sbuildrc
if [ ! -e ~/.sbuildrc ]; then
cat > ~/.sbuildrc <<EOM
# *** VERIFY AND UPDATE \$mailto and \$maintainer_name BELOW ***
# *** THIS COMMAND IS DEPRECATED ***
#
# In sbuild 0.87.0 and later, the unshare backend is available. This is
# expected to become the default in a future release.
#
# This is the new preferred way of building Debian packages, making the manual
# creation of schroots no longer necessary. To retain the default behavior,
# you may remove this comment block and continue.
#
# To test the unshare backend while retaining the default settings, run sbuild
# with --chroot-mode=unshare like this:
# $ sbuild --chroot-mode=unshare --dist=unstable hello
#
# To switch to the unshare backend by default (recommended), uncomment the
# following lines and delete the rest of the file (with the exception of the
# last two lines):
#\$chroot_mode = 'unshare';
#\$unshare_mmdebstrap_keep_tarball = 1;
# Mail address where logs are sent to (mandatory, no default!)
\$mailto = '$USER';
# *** VERIFY AND UPDATE \$mailto and \$maintainer_name BELOW ***
# Name to use as override in .changes files for the Maintainer: field
#\$maintainer_name='$USER <$USER@localhost>';
@ -651,6 +668,7 @@ ubuntu)
if ubuntu_dist_ge "$RELEASE" "edgy"; then
# Add pkgbinarymangler (edgy and later)
BUILD_PKGS="$BUILD_PKGS pkgbinarymangler"
USE_PKGBINARYMANGLER=1
# Disable recommends for a smaller chroot (gutsy and later only)
if ubuntu_dist_ge "$RELEASE" "gutsy"; then
BUILD_PKGS="--no-install-recommends $BUILD_PKGS"
@ -910,8 +928,8 @@ if [ -n "$TEMP_PREFERENCES" ]; then
sudo mv "$TEMP_PREFERENCES" $MNT/etc/apt/preferences.d/proposed.pref
fi
# Copy the timezone (comment this out if you want to leave the chroot at UTC)
sudo cp -P --remove-destination /etc/localtime /etc/timezone "$MNT"/etc/
# Copy the timezone (uncomment this if you want to use your local time zone)
#sudo cp -P --remove-destination /etc/localtime /etc/timezone "$MNT"/etc/
# Create a schroot entry for this chroot
TEMP_SCHROOTCONF=`mktemp -t schrootconf-XXXXXX`
TEMPLATE_SCHROOTCONF=~/.mk-sbuild.schroot.conf
@ -1030,6 +1048,25 @@ EOF
EOM
fi
if [ "$USE_PKGBINARYMANGLER" = 1 ]; then
sudo bash -c "cat >> $MNT/finish.sh" <<EOM
mkdir -p /etc/pkgbinarymangler/
cat > /etc/pkgbinarymangler/maintainermangler.conf <<EOF
# pkgmaintainermangler configuration file
# pkgmaintainermangler will do nothing unless enable is set to "true"
enable: true
# Configure what happens if /CurrentlyBuilding is present, but invalid
# (i. e. it does not contain a Package: field). If "ignore" (default),
# the file is ignored (i. e. the Maintainer field is mangled) and a
# warning is printed. If "fail" (or any other value), pkgmaintainermangler
# exits with an error, which causes a package build to fail.
invalid_currentlybuilding: ignore
EOF
EOM
fi
if [ -n "$TARGET_ARCH" ]; then
sudo bash -c "cat >> $MNT/finish.sh" <<EOM
# Configure target architecture
@ -1048,7 +1085,7 @@ apt-get update || true
echo set debconf/frontend Noninteractive | debconf-communicate
echo set debconf/priority critical | debconf-communicate
# Install basic build tool set, trying to match buildd
apt-get -y --force-yes install $BUILD_PKGS
apt-get -y --force-yes -o Dpkg::Options::="--force-confold" install $BUILD_PKGS
# Set up expected /dev entries
if [ ! -r /dev/stdin ]; then ln -s /proc/self/fd/0 /dev/stdin; fi
if [ ! -r /dev/stdout ]; then ln -s /proc/self/fd/1 /dev/stdout; fi

View File

@ -183,7 +183,7 @@ def display_verbose(package, values):
Logger.info("No reverse dependencies found")
return
def log_package(values, package, arch, dependency, offset=0):
def log_package(values, package, arch, dependency, visited, offset=0):
line = f"{' ' * offset}* {package}"
if all_archs and set(arch) != all_archs:
line += f" [{' '.join(sorted(arch))}]"
@ -192,6 +192,9 @@ def display_verbose(package, values):
line += " " * (30 - len(line))
line += f" (for {dependency})"
Logger.info(line)
if package in visited:
return
visited = visited.copy().add(package)
data = values.get(package)
if data:
offset = offset + 1
@ -202,6 +205,7 @@ def display_verbose(package, values):
rdep["Package"],
rdep.get("Architectures", all_archs),
rdep.get("Dependency"),
visited,
offset,
)
@ -223,6 +227,7 @@ def display_verbose(package, values):
rdep["Package"],
rdep.get("Architectures", all_archs),
rdep.get("Dependency"),
{package},
)
Logger.info("")

View File

@ -32,13 +32,13 @@ def make_pep440_compliant(version: str) -> str:
scripts = [
"backportpackage",
"bitesize",
"check-mir",
"check-symbols",
"dch-repeat",
"grab-merge",
"grep-merges",
"import-bug-from-debian",
"lp-bitesize",
"merge-changelog",
"mk-sbuild",
"pbuilder-dist",

View File

@ -49,6 +49,7 @@ from ubuntutools.requestsync.mail import get_debian_srcpkg as requestsync_mail_g
from ubuntutools.version import Version
Logger = getLogger()
cached_sync_blocklist = None
def remove_signature(dscname):
@ -143,7 +144,7 @@ def sync_dsc(
if ubuntu_ver.is_modified_in_ubuntu():
if not force:
Logger.error("--force is required to discard Ubuntu changes.")
sys.exit(1)
return None
Logger.warning(
"Overwriting modified Ubuntu version %s, setting current version to %s",
@ -157,7 +158,7 @@ def sync_dsc(
src_pkg.pull()
except DownloadError as e:
Logger.error("Failed to download: %s", str(e))
sys.exit(1)
return None
src_pkg.unpack()
needs_fakesync = not (need_orig or ubu_pkg.verify_orig())
@ -166,13 +167,13 @@ def sync_dsc(
Logger.warning("Performing a fakesync")
elif not needs_fakesync and fakesync:
Logger.error("Fakesync not required, aborting.")
sys.exit(1)
return None
elif needs_fakesync and not fakesync:
Logger.error(
"The checksums of the Debian and Ubuntu packages "
"mismatch. A fake sync using --fakesync is required."
)
sys.exit(1)
return None
if fakesync:
# Download Ubuntu files (override Debian source tarballs)
@ -180,7 +181,7 @@ def sync_dsc(
ubu_pkg.pull()
except DownloadError as e:
Logger.error("Failed to download: %s", str(e))
sys.exit(1)
return None
# change into package directory
directory = src_pkg.source + "-" + new_ver.upstream_version
@ -265,7 +266,7 @@ def sync_dsc(
returncode = subprocess.call(cmd)
if returncode != 0:
Logger.error("Source-only build with debuild failed. Please check build log above.")
sys.exit(1)
return None
def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
@ -295,7 +296,7 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
udtexceptions.SeriesNotFoundException,
) as e:
Logger.error(str(e))
sys.exit(1)
return None
if version is None:
version = Version(debian_srcpkg.getVersion())
try:
@ -306,7 +307,7 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
ubuntu_version = Version("~")
except udtexceptions.SeriesNotFoundException as e:
Logger.error(str(e))
sys.exit(1)
return None
if ubuntu_version >= version:
# The LP importer is maybe out of date
debian_srcpkg = requestsync_mail_get_debian_srcpkg(package, dist)
@ -320,7 +321,7 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
ubuntu_version,
ubuntu_release,
)
sys.exit(1)
return None
if component is None:
component = debian_srcpkg.getComponent()
@ -329,7 +330,7 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
return DebianSourcePackage(package, version.full_version, component, mirrors=mirrors)
def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False):
def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False, yes=False):
"""Copy a source package from Debian to Ubuntu using the Launchpad API."""
ubuntu = Distribution("ubuntu")
debian_archive = Distribution("debian").getArchive()
@ -352,7 +353,7 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False):
"Debian version %s has not been picked up by LP yet. Please try again later.",
src_pkg.version,
)
sys.exit(1)
return None
try:
ubuntu_spph = get_ubuntu_srcpkg(src_pkg.source, ubuntu_series, ubuntu_pocket)
@ -373,7 +374,7 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False):
base_version = ubuntu_version.get_related_debian_version()
if not force and ubuntu_version.is_modified_in_ubuntu():
Logger.error("--force is required to discard Ubuntu changes.")
sys.exit(1)
return None
# Check whether a fakesync would be required.
if not src_pkg.dsc.compare_dsc(ubuntu_pkg.dsc):
@ -381,7 +382,7 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False):
"The checksums of the Debian and Ubuntu packages "
"mismatch. A fake sync using --fakesync is required."
)
sys.exit(1)
return None
except udtexceptions.PackageNotFoundException:
base_version = Version("~")
Logger.info(
@ -402,9 +403,10 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False):
if sponsoree:
Logger.info("Sponsoring this sync for %s (%s)", sponsoree.display_name, sponsoree.name)
answer = YesNoQuestion().ask("Sync this package", "no")
if answer != "yes":
return
if not yes:
answer = YesNoQuestion().ask("Sync this package", "no")
if answer != "yes":
return
try:
ubuntu_archive.copyPackage(
@ -419,26 +421,29 @@ def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False):
except HTTPError as error:
Logger.error("HTTP Error %s: %s", error.response.status, error.response.reason)
Logger.error(error.content)
sys.exit(1)
return None
Logger.info("Request succeeded; you should get an e-mail once it is processed.")
bugs = sorted(set(bugs))
if bugs:
Logger.info("Launchpad bugs to be closed: %s", ", ".join(str(bug) for bug in bugs))
Logger.info("Please wait for the sync to be successful before closing bugs.")
answer = YesNoQuestion().ask("Close bugs", "yes")
if answer == "yes":
if yes:
close_bugs(bugs, src_pkg.source, src_pkg.version.full_version, changes, sponsoree)
else:
answer = YesNoQuestion().ask("Close bugs", "yes")
if answer == "yes":
close_bugs(bugs, src_pkg.source, src_pkg.version.full_version, changes, sponsoree)
def is_blacklisted(query):
"""Determine if package "query" is in the sync blacklist
Returns tuple of (blacklisted, comments)
blacklisted is one of False, 'CURRENT', 'ALWAYS'
def is_blocklisted(query):
"""Determine if package "query" is in the sync blocklist
Returns tuple of (blocklisted, comments)
blocklisted is one of False, 'CURRENT', 'ALWAYS'
"""
series = Launchpad.distributions["ubuntu"].current_series
lp_comments = series.getDifferenceComments(source_package_name=query)
blacklisted = False
blocklisted = False
comments = [
f"{c.body_text}\n -- {c.comment_author.name}"
f" {c.comment_date.strftime('%a, %d %b %Y %H:%M:%S +0000')}"
@ -446,32 +451,38 @@ def is_blacklisted(query):
]
for diff in series.getDifferencesTo(source_package_name_filter=query):
if diff.status == "Blacklisted current version" and blacklisted != "ALWAYS":
blacklisted = "CURRENT"
if diff.status == "Blacklisted current version" and blocklisted != "ALWAYS":
blocklisted = "CURRENT"
if diff.status == "Blacklisted always":
blacklisted = "ALWAYS"
blocklisted = "ALWAYS"
# Old blacklist:
url = "https://ubuntu-archive-team.ubuntu.com/sync-blacklist.txt"
with urllib.request.urlopen(url) as f:
applicable_lines = []
for line in f:
line = line.decode("utf-8")
if not line.strip():
applicable_lines = []
continue
applicable_lines.append(line)
try:
line = line[: line.index("#")]
except ValueError:
pass
source = line.strip()
if source and fnmatch.fnmatch(query, source):
comments += ["From sync-blacklist.txt:"] + applicable_lines
blacklisted = "ALWAYS"
break
global cached_sync_blocklist
if not cached_sync_blocklist:
url = "https://ubuntu-archive-team.ubuntu.com/sync-blocklist.txt"
try:
with urllib.request.urlopen(url) as f:
cached_sync_blocklist = f.read().decode("utf-8")
except:
print("WARNING: unable to download the sync blocklist. Erring on the side of caution.")
return ("ALWAYS", "INTERNAL ERROR: Unable to fetch sync blocklist")
return (blacklisted, comments)
applicable_lines = []
for line in cached_sync_blocklist.splitlines():
if not line.strip():
applicable_lines = []
continue
applicable_lines.append(line)
try:
line = line[:line.index("#")]
except ValueError:
pass
source = line.strip()
if source and fnmatch.fnmatch(query, source):
comments += ["From sync-blocklist.txt:"] + applicable_lines
blocklisted = "ALWAYS"
break
return (blocklisted, comments)
def close_bugs(bugs, package, version, changes, sponsoree):
@ -508,6 +519,12 @@ def parse():
epilog = f"See {os.path.basename(sys.argv[0])}(1) for more info."
parser = argparse.ArgumentParser(usage=usage, epilog=epilog)
parser.add_argument(
"-y",
"--yes",
action="store_true",
help="Automatically sync without prompting. Use with caution and care."
)
parser.add_argument("-d", "--distribution", help="Debian distribution to sync from.")
parser.add_argument("-r", "--release", help="Specify target Ubuntu release.")
parser.add_argument("-V", "--debian-version", help="Specify the version to sync from.")
@ -712,36 +729,38 @@ def main():
args.release,
args.debian_mirror,
)
if not src_pkg:
continue
blacklisted, comments = is_blacklisted(src_pkg.source)
blacklist_fail = False
if blacklisted:
blocklisted, comments = is_blocklisted(src_pkg.source)
blocklist_fail = False
if blocklisted:
messages = []
if blacklisted == "CURRENT":
if blocklisted == "CURRENT":
Logger.debug(
"Source package %s is temporarily blacklisted "
"(blacklisted_current). "
"Source package %s is temporarily blocklisted "
"(blocklisted_current). "
"Ubuntu ignores these for now. "
"See also LP: #841372",
src_pkg.source,
)
else:
if args.fakesync:
messages += ["Doing a fakesync, overriding blacklist."]
messages += ["Doing a fakesync, overriding blocklist."]
else:
blacklist_fail = True
blocklist_fail = True
messages += [
"If this package needs a fakesync, use --fakesync",
"If you think this package shouldn't be "
"blacklisted, please file a bug explaining your "
"blocklisted, please file a bug explaining your "
"reasoning and subscribe ~ubuntu-archive.",
]
if blacklist_fail:
Logger.error("Source package %s is blacklisted.", src_pkg.source)
elif blacklisted == "ALWAYS":
Logger.info("Source package %s is blacklisted.", src_pkg.source)
if blocklist_fail:
Logger.error("Source package %s is blocklisted.", src_pkg.source)
elif blocklisted == "ALWAYS":
Logger.info("Source package %s is blocklisted.", src_pkg.source)
if messages:
for message in messages:
for line in textwrap.wrap(message):
@ -753,14 +772,15 @@ def main():
for line in textwrap.wrap(comment):
Logger.info(" %s", line)
if blacklist_fail:
sys.exit(1)
if blocklist_fail:
continue
if args.lp:
copy(src_pkg, args.release, args.bugs, sponsoree, args.simulate, args.force)
if not copy(src_pkg, args.release, args.bugs, sponsoree, args.simulate, args.force, args.yes):
continue
else:
os.environ["DEB_VENDOR"] = "Ubuntu"
sync_dsc(
if not sync_dsc(
src_pkg,
args.distribution,
args.release,
@ -772,7 +792,8 @@ def main():
args.simulate,
args.force,
args.fakesync,
)
):
continue
if __name__ == "__main__":

View File

@ -165,6 +165,7 @@ class SourcePackage(ABC):
series = kwargs.get("series")
pocket = kwargs.get("pocket")
status = kwargs.get("status")
arch = kwargs.get("arch")
verify_signature = kwargs.get("verify_signature", False)
try_binary = kwargs.get("try_binary", True)
@ -184,6 +185,7 @@ class SourcePackage(ABC):
self._series = series
self._pocket = pocket
self._status = status
self._arch = arch
# dscfile can be either a path or an URL. misc.py's download() will
# later fiture it out
self._dsc_source = dscfile
@ -252,6 +254,7 @@ class SourcePackage(ABC):
)
try:
params["archtag"] = self._arch
bpph = archive.getBinaryPackage(self.source, **params)
except PackageNotFoundException as bpnfe:
# log binary lookup failure, in case it provides hints

View File

@ -68,21 +68,19 @@ class UDTConfig:
config = {}
for filename in ("/etc/devscripts.conf", "~/.devscripts"):
try:
f = open(os.path.expanduser(filename), "r", encoding="utf-8")
with open(os.path.expanduser(filename), "r", encoding="utf-8") as f:
content = f.read()
except IOError:
continue
for line in f:
parsed = shlex.split(line, comments=True)
if len(parsed) > 1:
Logger.warning(
"Cannot parse variable assignment in %s: %s",
getattr(f, "name", "<config>"),
line,
)
if len(parsed) >= 1 and "=" in parsed[0]:
key, value = parsed[0].split("=", 1)
try:
tokens = shlex.split(content, comments=True)
except ValueError as e:
Logger.error("Error parsing %s: %s", filename, e)
continue
for token in tokens:
if "=" in token:
key, value = token.split("=", 1)
config[key] = value
f.close()
return config
def get_value(self, key, default=None, boolean=False, compat_keys=()):

View File

@ -340,6 +340,7 @@ class PullPkg:
params = {}
params["package"] = options["package"]
params["arch"] = options["arch"]
if options["release"]:
(release, version, pocket) = self.parse_release_and_version(