596 Commits

Author SHA1 Message Date
Dan Streetman
90e8fe81e1 replace ubuntutools.logger with standard python logging 2019-11-25 10:10:06 -05:00
Dan Streetman
c9c7fed1f6 pull-uca-*: search in reverse order of UCA releases, if none specified
if only the version of a UCA package is specified, search each UCA
archive from latest backwards.  This avoids having to specify both
the package version *and* UCA release.
2019-11-25 10:10:06 -05:00
Dan Streetman
7c097b19ba ubuntutools: add --status param to pull-pkg
Allow specifying what package statuses should be searched for.

By default search only for Pending and Published, unless a specific
version number is being searched for.
2019-11-25 10:10:06 -05:00
Dan Streetman
0f61836b10 ubuntutools: update archive/lpapicache to optionally search all series
For PPA and UCA repos, the latest build is not necessarily in the
'development' release; so for those SourcePackage classes, search
all the 'active' series (starting with latest devel) for any
matches to the provided package name.  This allows not having to
specify the series name when looking in PPA/UCA repos for the 'latest'
version of a specific package.
2019-11-25 10:10:06 -05:00
Dan Streetman
b11b83f0e2 pull-pkg: change UCA source pkg class to subclass of PPA source pkg class
The UCA source pkg class was its own specific class, but with the
addition of the generic PPA source pkg class, the UCA class is just
a specific case of PPA.  Changing it to a subclass of the PPA class
simplifies code.
2019-11-25 10:10:06 -05:00
Dan Streetman
7084bfc8bc pull-pkg: add pull-ppa-* functionality
Add functionality, and frontend pull-ppa-* scripts, to be able to pull
from PPA archives.
2019-11-25 10:10:06 -05:00
Dan Streetman
3491b0cff9 pull-pkg: change pullpkg into class PullPkg
instead of pullpkg.py containing a simple method to call, change it
into a normal class PullPkg that callers can create and use.
2019-11-25 10:10:06 -05:00
Dan Streetman
fb750e38bb lpapicache: find binaries from lp
Also update getBinaries() to allow retreival by binary name

This significantly speeds up binary file lookup for non-published
package versions, since we can get the list of binary urls, but
then have to look up the BPPH for each of those urls, which is slow.
If the user only wants a specific binary package and/or arch, this
speeds up getting that, especially for packages with a lot of binary
files (like the kernel).
2019-11-25 10:10:06 -05:00
Dan Streetman
3a413760f3 ubuntutools: add pull-* --no-verify-signature option, don't fail if no pub key
Change dsc verification to fail only if the public key was available, but
signature verification failed.  If no public key is available for the dsc,
print warning only. (LP: #1700846)

Also add pull-* parameter --no-verify-signature to manually prevent failure
when signature verification fails.
2019-11-25 10:10:06 -05:00
Dan Streetman
cdaf5d0761 pull-pkg: change pull-pkg script code into modular code 2019-11-25 10:10:06 -05:00
Dan Streetman
79b705d235 pull-pkg: rename pull-pkg to ubuntutools/pullpkg.py module
rename only with no content changes, so next commit shows code
changes converting script pull-pkg to module pullpkg.py

This also changes the file mode from 755 to 644, since we're changing
an executable script into a module.
2019-11-25 10:09:26 -05:00
Dan Streetman
3dabf05370 pull-pkg: add debian and ubuntu ddebs mirror urls 2019-11-25 09:57:54 -05:00
Dan Streetman
f4f16f95f7 pull-uca: update UbuntuCloudArchiveSourcePackage
look in its PPA for its SPPHs
look up the development (latest) UCA release if not specified
use the 'series' param as the UCA release, instead of 'uca_release' param
2019-11-25 09:57:54 -05:00
Dan Streetman
b96885f05e ubuntutools/archive: convert external rmadison call to native implementation
instead of a function that calls the system program rmadison, use
a fully-functional class to interface with the madison api, as well
as the debian snapshot api
2019-11-25 09:57:54 -05:00
Dan Streetman
f944d3146a pull-pkg: find src pkg name from binary pkg name 2019-11-25 09:57:54 -05:00
Dan Streetman
9f58f5cb80 pull-pkg: find latest pkg in series in SourcePackage
Allow specifying series instead of version, to find the latest
version in that series; or only specify package name, to get the
latest version in the devel series
2019-11-25 09:57:54 -05:00
Dan Streetman
b0c22e1d57 pull-pkg: update archive.py SourcePackage
verify it can parse/load DSC when created
throw PackageNotFoundException from lp_spph if can't find package
update test case expected order of url processing
2019-11-25 09:57:54 -05:00
Dan Streetman
506e3db601 pull-pkg: add SourcePackage.pull_binaries() function 2019-11-25 09:57:54 -05:00
Dan Streetman
d3b8d7a1b7 ubuntutools/lp/lpapicache: expand coverage for LP api 2019-11-25 09:57:54 -05:00
Dan Streetman
d0aa64a51b ubuntutools/misc: define POCKETS and DEFAULT_POCKETS
POCKETS is all valid pockets (capitalized), DEFAULT_POCKETS is all in
POCKETS except 'Backports'.
2019-11-25 09:57:54 -05:00
Dan Streetman
b5ae0bdca3 simplify subprocess usage 2019-11-25 09:57:54 -05:00
Dan Streetman
0931098a2c tests/pylint.conf: use jobs=0 to speed up tests 2019-11-07 15:03:29 -05:00
Mattia Rizzolo
982ebe6a46
Trust the installed debian-keyring when checking validity of dsc signatures. 2019-10-28 14:35:01 +01:00
Mattia Rizzolo
21adb8f61f
fix flake8 warning
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2019-09-26 11:05:48 +02:00
Stefano Rivera
317b68bc08 Use a context manager for opening files 2019-09-11 13:38:42 -03:00
Stefano Rivera
7a6b779e77 ubuntutools.misc: Replace Popen() calls with check_output() 2019-09-11 13:38:42 -03:00
Mattia Rizzolo
68b41fc82f
Drop workaround for an old pylint bug
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2019-09-10 14:25:35 +02:00
Mattia Rizzolo
434ca8952e
Merge branch 'python3' of git+ssh://git.launchpad.net/ubuntu-dev-tools
MR: https://code.launchpad.net/~ubuntu-dev/ubuntu-dev-tools/+git/ubuntu-dev-tools/+merge/372305
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2019-09-10 14:19:07 +02:00
Dan Streetman
a7d166b52f pull-pkg: use ubuntutools/version/Version
instead of debian.debian_support.Version, use ubuntutools version,
which extends it and adds the strip_epoch() function which is
needed to convert full version string to version used in filenames
2019-09-09 12:25:21 -04:00
Stefano Rivera
06642ef9f7 Remove some more u'' strings 2019-09-05 16:10:26 -03:00
Stefano Rivera
7b2f1a713b Remove more Python 2 fallback imports 2019-09-05 09:49:41 -03:00
Stefano Rivera
301569e809 Pull out Python 2 support hacks 2019-09-04 19:24:44 -03:00
Stefano Rivera
bb765237db Port Python scripts to Python 3, remove python 2 modules. (Closes: #938740, LP: #1099537) 2019-09-04 19:24:44 -03:00
Stefano Rivera
7c0efe2914 Port requestbackport to Python 3 2019-09-04 19:24:44 -03:00
Stefano Rivera
a41af75643 Close FDs after tests 2019-09-04 19:24:44 -03:00
Stefano Rivera
23c7d67425 Replace simple Popen() calls with check_output() 2019-09-04 19:24:44 -03:00
Stefano Rivera
2d3765522e Use the Popen() encoding flag to decode to unicode
This should make behavior identical on Python 2 & 3.
2019-09-04 19:24:44 -03:00
Stefano Rivera
43ad610a66 Add encoding support to our Popen wrapper 2019-09-04 19:24:44 -03:00
Stefano Rivera
5769a70d02 Increase --help timeout, seeing a few of them 2019-09-03 21:16:59 -03:00
Stefano Rivera
e163ee0158 Test with the pylint command, we don't have py2+3 versions any more 2019-09-03 21:10:38 -03:00
Mattia Rizzolo
e0f6b6f97b
archive: Disable dsc signature verification for non-Debian.
Ubuntu doesn't have a unified keyring of developers like Debian has, so
it is not feasible to check for the dsc signatures.

Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2019-08-12 13:41:16 +02:00
Mattia Rizzolo
3fb7516d72
Default to checking signatures while pulling a .dsc.
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2019-08-05 13:50:33 +02:00
Mattia Rizzolo
3f87486de3
Merge branch 'pull-debian-source-edge-cases' of git+ssh://git.launchpad.net/~racb/ubuntu-dev-tools
MR: https://code.launchpad.net/~racb/ubuntu-dev-tools/+git/ubuntu-dev-tools/+merge/326608
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2019-07-20 15:10:35 +02:00
Benjamin Drung
190ad30a7b Fix invalid escape sequence '\(' or '\)' (found by flake8)
flake8 found issues:
ubuntutools/sponsor_patch/bugtask.py:46:11: W605 invalid escape sequence '\('
ubuntutools/sponsor_patch/bugtask.py:62:50: W605 invalid escape sequence '\('
ubuntutools/sponsor_patch/bugtask.py:62:58: W605 invalid escape sequence '\)'
setup.py:14:8: W605 invalid escape sequence '\('
setup.py:14:14: W605 invalid escape sequence '\)'

Fix these issues by marking these strings as raw strings.
2018-10-06 18:04:25 +02:00
Benjamin Drung
74df5b3869 Update pylint and flake8 unittests
Import improvements from https://github.com/bdrung/snippets
2018-10-06 17:42:59 +02:00
Benjamin Drung
468dbc7746 ubuntutools/test: Introduce get_source_files function
The flake8 and pylint unittest use the same logic to determine the source
files. Therefore put this logic in one function.
2018-10-06 17:16:12 +02:00
Mattia Rizzolo
607f5a712f
Fix FTBFS due to newest tar being picker about arguments order
Closes: #897478
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2018-05-10 10:38:25 +02:00
Mattia Rizzolo
edb49c652c
Fix test failures with newer flake8 and pylint.
Closes: #891721
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2018-03-08 12:56:36 +01:00
Robie Basak
a761ccfc72 Fix rmadison parsing for Python 3
In Python 3, it matters that the output of rmadison must be decoded
before it can be parsed.
2017-06-30 15:47:29 +01:00
Robie Basak
4c66fba4d9 pull-debian-source: --no-verify-signature option
Using pull-debian-source fails on some very old packages such as on
texinfo 4.8.dfsg.1-4. I have hand-verified that the signature is good
(though with no trust path), so presumably this is because the signature
has rotated out of debian-keyring.

Add a --no-verify-signature option so that developers can still make use
of the find-and-download functionality of this tool, albeit without
signture verification.
2017-06-30 15:47:29 +01:00