syncpackage: Support sponsorship for native-syncs, now that LP does.

This commit is contained in:
Stefano Rivera 2011-12-09 10:55:18 +02:00
commit 919f091820
4 changed files with 181 additions and 95 deletions

1
debian/changelog vendored
View File

@ -26,6 +26,7 @@ ubuntu-dev-tools (0.137) UNRELEASED; urgency=low
- ubuntu-upload-permission: Query upload permissions.
- seeded-in-ubuntu: Query a package's seed status. Whether it is on
current daily images and/or part of the supported seed.
* syncpackage: Support sponsorship for native-syncs, now that LP does.
[ Andreas Moog ]
* sponsor-patch: Check permission to unsubscribe sponsors-team (LP: #896884)

View File

@ -1,9 +1,11 @@
.TH SYNCPACKAGE "1" "June 2010" "ubuntu-dev-tools"
.SH NAME
syncpackage \- copy source packages from Debian to Ubuntu
.\"
.SH SYNOPSIS
.B syncpackage
[\fIoptions\fR] \fI<.dsc URL/path or package name>\fR
.\"
.SH DESCRIPTION
\fBsyncpackage\fR causes a source package to be copied from Debian to
Ubuntu.
@ -14,16 +16,19 @@ this way you can preserve source files integrity between the two distributions.
.PP
\fBsyncpackage\fR will detect source tarballs with mismatching
checksums, and can perform fake syncs.
.\"
.SH WARNING
The use of \fBsyncpackage \-\-no\-lp\fR, which generates a changes file to
be directly uploaded to the Ubuntu primary archive or a PPA, is discouraged
by the Ubuntu Archive Administrators, as it introduces an unnecessary window
for error.
This only exists for backward compatibility, for unusual corner cases, and
for uploads to archives other than the Ubuntu primary archive.
This only exists for backward compatibility, for unusual corner cases
(such as fakesyncs), and for uploads to archives other than the Ubuntu
primary archive.
Omitting this option will cause Launchpad to perform the sync request
directly, which is the preferred method for uploads to the Ubuntu primary
archive.
.\"
.SH OPTIONS
.TP
\fB\-h\fR, \fB\-\-help\fR
@ -42,20 +47,45 @@ Specify the version to sync from.
\fB\-c\fI COMPONENT\fR, \fB\-\-component\fR=\fICOMPONENT\fR
Specify the component to sync from.
.TP
\fB\-b\fI BUG\fR, \fB\-\-bug\fR=\fIBUG\fR
Mark a Launchpad bug as being fixed by this upload.
.TP
\fB\-s\fI USERNAME\fR, \fB\-\-sponsor\fR=\fIUSERNAME\fR
Sponsor the sync for \fIUSERNAME\fR (a Launchpad username).
.TP
\fB\-v\fR, \fB\-\-verbose\fR
Display more progress information.
.TP
.B \-\-no\-lp
Construct sync locally rather than letting Launchpad copy the package
directly (not recommended).
.TP
\fB\-F\fR, \fB\-\-fakesync\fR
Perform a fakesync, to work around a tarball mismatch between Debian and
Ubuntu. This option ignores blacklisting, and performs a local sync.
Ubuntu.
This option ignores blacklisting, and performs a local sync.
It implies \fB\-\-no\-lp\fR, and will leave a signed \fB.changes\fR file
for you to upload.
.TP
\fB\-f\fR, \fB\-\-force\fR
Force sync over the top of Ubuntu changes.
.TP
.B \-\-no\-conf
Do not read any configuration files, or configuration from environment
variables.
.TP
\fB\-l\fI INSTANCE\fR, \fB\-\-lpinstance\fR=\fIINSTANCE\fR
Launchpad instance to connect to (default: production).
.TP
.B \-\-simulate
Show what would be done, but don't actually do it.
.\"
.SH LOCAL SYNC PREPARATION OPTIONS
.TP
Options that only apply when using \fB\-\-no\-lp\fR:
.TP
.B \-\-no\-lp
Construct sync locally, rather than letting Launchpad copy the package
directly.
It will leave a signed \fB.changes\fR file for you to upload.
See the \fBWARNING\fR above.
.TP
\fB\-n\fI UPLOADER_NAME\fR, \fB\-\-uploader\-name\fR=\fIUPLOADER_NAME\fR
Use UPLOADER_NAME as the name of the maintainer for this upload instead
of evaluating DEBFULLNAME and UBUMAIL.
@ -72,12 +102,6 @@ Specify the key ID to be used for signing.
\fB\-\-dont-sign\fR
Do not sign the upload.
.TP
\fB\-b\fI BUG\fR, \fB\-\-bug\fR=\fIBUG\fR
Mark a Launchpad bug as being fixed by this upload.
.TP
\fB\-f\fR, \fB\-\-force\fR
Force sync over the top of Ubuntu changes.
.TP
.B \-d \fIDEBIAN_MIRROR\fR, \fB\-\-debian\-mirror\fR=\fIDEBIAN_MIRROR\fR
Use the specified mirror.
Should be in the form \fBhttp://ftp.debian.org/debian\fR.
@ -89,13 +113,7 @@ Use the specified Debian security mirror.
Should be in the form \fBhttp://archive.ubuntu.com/ubuntu\fR.
If the package isn't found on this mirror, \fBsyncpackage\fR will fall
back to the default mirror.
.TP
.B \-\-no\-conf
Do not read any configuration files, or configuration from environment
variables.
.TP
.B \-\-simulate
Show what would be done, but don't actually do it.
.\"
.SH ENVIRONMENT
.TP
.BR DEBFULLNAME ", " DEBEMAIL ", " UBUMAIL
@ -108,6 +126,7 @@ All of the \fBCONFIGURATION VARIABLES\fR below are also supported as
environment variables.
Variables in the environment take precedence to those in configuration
files.
.\"
.SH CONFIGURATION VARIABLES
The following variables can be set in the environment or in
.BR ubuntu\-dev\-tools (5)
@ -120,9 +139,11 @@ The default value for \fB\-\-debian\-mirror\fR.
.TP
.BR SYNCPACKAGE_UBUNTU_MIRROR ", " UBUNTUTOOLS_DEBSEC_MIRROR
The default value for \fB\-\-ubuntu\-mirror\fR.
.\"
.SH SEE ALSO
.BR requestsync (1),
.BR ubuntu\-dev\-tools (5)
.\"
.SH AUTHOR
\fBsyncpackage\fR was written by Martin Pitt <martin.pitt@canonical.com> and Benjamin Drung <bdrung@ubuntu.com>.
.PP

View File

@ -37,11 +37,11 @@ from lazr.restfulclient.errors import HTTPError
from ubuntutools.archive import (DebianSourcePackage, UbuntuSourcePackage,
DownloadError)
from ubuntutools.config import UDTConfig, ubu_email
from ubuntutools.requestsync.mail import (get_debian_srcpkg
as requestsync_mail_get_debian_srcpkg)
from ubuntutools.requestsync.mail import (
get_debian_srcpkg as requestsync_mail_get_debian_srcpkg)
from ubuntutools.requestsync.lp import get_debian_srcpkg, get_ubuntu_srcpkg
from ubuntutools.lp import udtexceptions
from ubuntutools.lp.lpapicache import (Distribution, Launchpad,
from ubuntutools.lp.lpapicache import (Distribution, Launchpad, PersonTeam,
SourcePackagePublishingHistory)
from ubuntutools.misc import split_release_pocket
from ubuntutools.question import YesNoQuestion
@ -52,9 +52,9 @@ class Version(debian.debian_support.Version):
def strip_epoch(self):
'''Removes the epoch from a Debian version string.
strip_epoch(1:1.52-1) will return "1.52-1" and strip_epoch(1.1.3-1) will
return "1.1.3-1".'''
strip_epoch(1:1.52-1) will return "1.52-1" and strip_epoch(1.1.3-1)
will return "1.1.3-1".
'''
parts = self.full_version.split(':')
if len(parts) > 1:
del parts[0]
@ -62,6 +62,7 @@ class Version(debian.debian_support.Version):
return version_without_epoch
def get_related_debian_version(self):
'''Strip the ubuntu-specific bits off the version'''
related_debian_version = self.full_version
uidx = related_debian_version.find('ubuntu')
if uidx > 0:
@ -72,6 +73,7 @@ class Version(debian.debian_support.Version):
return Version(related_debian_version)
def is_modified_in_ubuntu(self):
'''Did Ubuntu modify this (and mark the version appropriately)?'''
return 'ubuntu' in self.full_version
@ -97,6 +99,7 @@ def remove_signature(dscname):
dsc_file.writelines(unsigned_file)
dsc_file.close()
def add_fixed_bugs(changes, bugs):
'''Add additional Launchpad bugs to the list of fixed bugs in changes
file.'''
@ -117,15 +120,22 @@ def add_fixed_bugs(changes, bugs):
return "\n".join(changes + [""])
def sync_dsc(src_pkg, debian_dist, release, name, email, bugs, ubuntu_mirror,
keyid=None, simulate=False, force=False, fakesync=False):
'''Local sync, trying to emulate sync-source.py
Grabs a source package, replaces the .orig.tar with the one from Ubuntu,
if necessary, writes a sync-appropriate .changes file, and signs it.
'''
uploader = name + " <" + email + ">"
src_pkg.pull_dsc()
new_ver = Version(src_pkg.dsc["Version"])
try:
ubuntu_source = get_ubuntu_srcpkg(src_pkg.source, release.split("-")[0])
series = release.split("-")[0]
ubuntu_source = get_ubuntu_srcpkg(src_pkg.source, series)
ubuntu_ver = Version(ubuntu_source.getVersion())
ubu_pkg = UbuntuSourcePackage(src_pkg.source, ubuntu_ver.full_version,
ubuntu_source.getComponent(),
@ -206,7 +216,8 @@ def sync_dsc(src_pkg, debian_dist, release, name, email, bugs, ubuntu_mirror,
if not Logger.verbose:
cmd += ["-q"]
Logger.command(cmd + ['>', '../' + changes_filename])
changes = subprocess.Popen(cmd, stdout=subprocess.PIPE).communicate()[0]
process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
changes = process.communicate()[0]
# Add additional bug numbers
if len(bugs) > 0:
@ -266,7 +277,9 @@ def sync_dsc(src_pkg, debian_dist, release, name, email, bugs, ubuntu_mirror,
'Please check build log above.')
sys.exit(1)
def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
def fetch_source_pkg(package, dist, version, component, ubuntu_release,
mirror):
"""Download the specified source package.
dist, version, component, mirror can all be None.
"""
@ -320,7 +333,8 @@ def fetch_source_pkg(package, dist, version, component, ubuntu_release, mirror):
return DebianSourcePackage(package, version.full_version, component,
mirrors=mirrors)
def copy(src_pkg, release, bugs, simulate=False, force=False):
def copy(src_pkg, release, bugs, sponsoree=None, simulate=False, force=False):
"""Copy a source package from Debian to Ubuntu using the Launchpad API."""
ubuntu = Distribution('ubuntu')
debian_archive = Distribution('debian').getArchive()
@ -382,6 +396,9 @@ def copy(src_pkg, release, bugs, simulate=False, force=False):
if simulate:
return
if sponsoree:
Logger.normal("Sponsoring this sync for %s (%s)",
sponsoree.display_name, sponsoree.name)
answer = YesNoQuestion().ask("Sync this package", "no")
if answer != "yes":
return
@ -393,7 +410,8 @@ def copy(src_pkg, release, bugs, simulate=False, force=False):
from_archive=debian_archive,
to_series=ubuntu_series,
to_pocket=ubuntu_pocket,
include_binaries=False)
include_binaries=False,
sponsored=sponsoree)
except HTTPError, error:
Logger.error("HTTP Error %s: %s", error.response.status,
error.response.reason)
@ -413,6 +431,7 @@ def copy(src_pkg, release, bugs, simulate=False, force=False):
close_bugs(bugs, src_pkg.source, src_pkg.version.full_version,
changes)
def is_blacklisted(query):
""""Determine if package "query" is in the sync blacklist
Returns tuple of (blacklisted, comments)
@ -422,7 +441,7 @@ def is_blacklisted(query):
lp_comments = series.getDifferenceComments(source_package_name=query)
blacklisted = False
comments = [u'%s\n -- %s %s'
%(c.body_text, c.comment_author.name,
% (c.body_text, c.comment_author.name,
c.comment_date.strftime('%a, %d %b %Y %H:%M:%S +0000'))
for c in lp_comments]
@ -451,6 +470,7 @@ def is_blacklisted(query):
return (blacklisted, comments)
def close_bugs(bugs, package, version, changes):
"""Close the correct task on all bugs, with changes"""
ubuntu = Launchpad.distributions['ubuntu']
@ -475,6 +495,7 @@ def close_bugs(bugs, package, version, changes):
else:
Logger.error(u"Cannot find any tasks on LP: #%i to close.", bug.id)
def parse():
"""Parse given command-line parameters."""
@ -483,71 +504,76 @@ def parse():
parser = optparse.OptionParser(usage=usage, epilog=epilog)
parser.add_option("-d", "--distribution",
dest="dist", default=None,
help="Debian distribution to sync from.")
parser.add_option("-r", "--release",
dest="release", default=None,
help="Specify target Ubuntu release.")
parser.add_option("-V", "--debian-version",
dest="debversion", default=None,
help="Specify the version to sync from.")
parser.add_option("-c", "--component",
dest="component", default=None,
help="Specify the Debian component to sync from.")
parser.add_option("-v", "--verbose",
dest="verbose", action="store_true", default=False,
help="Display more progress information.")
parser.add_option("-F", "--fakesync",
dest="fakesync", action="store_true", default=False,
help="Perform a fakesync (a sync where Debian and Ubuntu "
"have a .orig.tar mismatch).")
parser.add_option("--no-lp",
dest="lp", action="store_false", default=True,
help="Construct sync locally rather than letting "
"Launchpad copy the package directly (not "
"recommended).")
parser.add_option('-l', '--lpinstance', metavar='INSTANCE',
dest='lpinstance', default=None,
help='Launchpad instance to connect to '
'(default: production).')
parser.add_option("-n", "--uploader-name",
dest="uploader_name", default=None,
help="Use UPLOADER_NAME as the name of the maintainer "
"for this upload.")
parser.add_option("-e", "--uploader-email",
dest="uploader_email", default=None,
help="Use UPLOADER_EMAIL as email address of the "
"maintainer for this upload.")
parser.add_option("-k", "--key",
dest="keyid", default=None,
help="Specify the key ID to be used for signing.")
parser.add_option('--dont-sign',
dest='keyid', action='store_false',
help='Do not sign the upload.')
parser.add_option("-b", "--bug", metavar="BUG",
dest="bugs", action="append", default=list(),
help="Mark Launchpad bug BUG as being fixed by this "
"upload.")
parser.add_option("-s", "--sponsor", metavar="USERNAME",
dest="sponsoree", default=None,
help="Sponsor the sync for USERNAME (a Launchpad "
"username).")
parser.add_option("-v", "--verbose",
action="store_true", default=False,
help="Display more progress information.")
parser.add_option("-F", "--fakesync",
action="store_true", default=False,
help="Perform a fakesync (a sync where Debian and "
"Ubuntu have a .orig.tar mismatch). "
"This implies --no-lp and will leave a signed "
".changes file for you to upload.")
parser.add_option("-f", "--force",
dest="force", action="store_true", default=False,
action="store_true", default=False,
help="Force sync over the top of Ubuntu changes.")
parser.add_option('-D', '--debian-mirror', metavar='DEBIAN_MIRROR',
dest='debian_mirror',
parser.add_option('--no-conf',
default=False, action='store_true',
help="Don't read config files or environment variables.")
parser.add_option('-l', '--lpinstance', metavar='INSTANCE',
help='Launchpad instance to connect to '
'(default: production).')
parser.add_option('--simulate',
default=False, action='store_true',
help="Show what would be done, but don't actually do "
"it.")
no_lp = optparse.OptionGroup(parser, "Local sync preparation options",
"Options that only apply when using --no-lp. "
"WARNING: The use of --no-lp is not recommended for uploads "
"targeted at Ubuntu. "
"The archive-admins discourage its use, except for fakesyncs.")
no_lp.add_option("--no-lp",
dest="lp", action="store_false", default=True,
help="Construct sync locally, rather than letting "
"Launchpad copy the package directly. "
"It will leave a signed .changes file for you to "
"upload.")
no_lp.add_option("-n", "--uploader-name",
help="Use UPLOADER_NAME as the name of the maintainer "
"for this upload.")
no_lp.add_option("-e", "--uploader-email",
help="Use UPLOADER_EMAIL as email address of the "
"maintainer for this upload.")
no_lp.add_option("-k", "--key",
dest="keyid",
help="Specify the key ID to be used for signing.")
no_lp.add_option('--dont-sign',
dest='keyid', action='store_false',
help='Do not sign the upload.')
no_lp.add_option('-D', '--debian-mirror', metavar='DEBIAN_MIRROR',
help='Preferred Debian mirror '
'(default: %s)'
% UDTConfig.defaults['DEBIAN_MIRROR'])
parser.add_option('-U', '--ubuntu-mirror', metavar='UBUNTU_MIRROR',
dest='ubuntu_mirror',
no_lp.add_option('-U', '--ubuntu-mirror', metavar='UBUNTU_MIRROR',
help='Preferred Ubuntu mirror '
'(default: %s)'
% UDTConfig.defaults['UBUNTU_MIRROR'])
parser.add_option('--no-conf',
dest='no_conf', default=False, action='store_true',
help="Don't read config files or environment variables.")
parser.add_option('--simulate',
dest='simulate', default=False, action='store_true',
help="Show what would be done, but don't actually do "
"it.")
parser.add_option_group(no_lp)
(options, args) = parser.parse_args()
@ -586,6 +612,7 @@ def parse():
def main():
'''Handle parameters and get the ball rolling'''
(options, package) = parse()
Logger.verbose = options.verbose
@ -594,10 +621,6 @@ def main():
options.debian_mirror = config.get_value('DEBIAN_MIRROR')
if options.ubuntu_mirror is None:
options.ubuntu_mirror = config.get_value('UBUNTU_MIRROR')
if options.uploader_name is None:
options.uploader_name = ubu_email(export=False)[0]
if options.uploader_email is None:
options.uploader_email = ubu_email(export=False)[1]
if options.lpinstance is None:
options.lpinstance = config.get_value('LPINSTANCE')
@ -611,8 +634,42 @@ def main():
ubuntu = Launchpad.distributions["ubuntu"]
options.release = ubuntu.current_series.name
src_pkg = fetch_source_pkg(package, options.dist, options.debversion,
options.component, options.release.split("-")[0],
if not options.fakesync and not options.lp:
Logger.warn("The use of --no-lp is not recommended for uploads "
"targeted at Ubuntu. "
"The archive-admins discourage its use, except for "
"fakesyncs.")
sponsoree = None
if options.sponsoree:
try:
sponsoree = PersonTeam(options.sponsoree)
except KeyError:
Logger.error('Cannot find the username "%s" in Launchpad.',
options.sponsoree)
sys.exit(1)
if sponsoree and options.uploader_name is None:
options.uploader_name = sponsoree.display_name
elif options.uploader_name is None:
options.uploader_name = ubu_email(export=False)[0]
if sponsoree and options.uploader_email is None:
try:
options.uploader_email = sponsoree.preferred_email_address.email
except ValueError:
if not options.lp:
Logger.error("%s doesn't have a publicly visible e-mail "
"address in LP, please provide one "
"--uploader-email option", sponsoree.display_name)
sys.exit(1)
elif options.uploader_email is None:
options.uploader_email = ubu_email(export=False)[1]
src_pkg = fetch_source_pkg(package, options.distribution,
options.debian_version,
options.component,
options.release.split("-")[0],
options.debian_mirror)
blacklisted, comments = is_blacklisted(src_pkg.source)
@ -622,14 +679,16 @@ def main():
if blacklisted == 'CURRENT':
Logger.debug("Source package %s is temporarily blacklisted "
"(blacklisted_current). Ubuntu ignores these for now. "
"(blacklisted_current). "
"Ubuntu ignores these for now. "
"See also LP: #841372", src_pkg.source)
else:
if options.fakesync:
messages += ["Doing a fakesync, overriding blacklist."]
else:
blacklist_fail = True
messages += ["If this package needs a fakesync, use --fakesync",
messages += ["If this package needs a fakesync, "
"use --fakesync",
"If you think this package shouldn't be "
"blacklisted, please file a bug explaining your "
"reasoning and subscribe ~ubuntu-archive."]
@ -653,14 +712,15 @@ def main():
sys.exit(1)
if options.lp:
copy(src_pkg, options.release, options.bugs, options.simulate,
options.force)
copy(src_pkg, options.release, options.bugs, sponsoree,
options.simulate, options.force)
else:
os.environ['DEB_VENDOR'] = 'Ubuntu'
sync_dsc(src_pkg, options.dist, options.release, options.uploader_name,
options.uploader_email, options.bugs, options.ubuntu_mirror,
options.keyid, options.simulate, options.force,
options.fakesync)
sync_dsc(src_pkg, options.distribution, options.release,
options.uploader_name, options.uploader_email, options.bugs,
options.ubuntu_mirror, options.keyid, options.simulate,
options.force, options.fakesync)
if __name__ == "__main__":
try:

View File

@ -383,7 +383,7 @@ class Archive(BaseWrapper):
return cache[index]
def copyPackage(self, source_name, version, from_archive, to_pocket,
to_series=None, include_binaries=False):
to_series=None, sponsored=None, include_binaries=False):
'''Copy a single named source into this archive.
Asynchronously copy a specific version of a named source to the
@ -392,12 +392,16 @@ class Archive(BaseWrapper):
will happen sometime later with full checking.
'''
if isinstance(sponsored, PersonTeam):
sponsored = sponsored._lpobject
self._lpobject.copyPackage(
source_name=source_name,
version=version,
from_archive=from_archive._lpobject,
to_pocket=to_pocket,
to_series=to_series,
sponsored=sponsored,
include_binaries=include_binaries
)