Dan Streetman
04ae04e17b
archive: simplify PPA source package class
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
ae3837be1d
lp: add Project and ProjectSeries classes
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
20036e6c36
ubuntutools/pullpkg: add --login parameter
...
Just in case we want to pull from a private ppa...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
e13a4eb45e
ubuntutools/archive: deprecate passing 'lp' object to SourcePackage()
...
There is no point to doing this, as the Launchpad object is a singleton;
the caller should just call Launchpad.login_with() directly and leave
the SourcePackage class out of it.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
cb865e3b86
ubuntutools/lp: change default login to anonymous
...
If not specified, default to logging in anonymously.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
61f0023c37
archive: use verify_file_checksums()
...
This reduces duplicate verification steps, and results in logging
error from the verification function if there is a size mismatch,
instead of the silent verification failure that was present in case
neither sha checksum was provided.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
e5a42b1ba1
misc: add verify_file_checksums() function
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
359cb18d8d
lpapicache: replace httplib2 use with call to download_text()
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
b89ad962f4
misc: add download_text() method
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
58ca95e719
pullpkg: provide dst to download()
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
a26a154bc9
misc: move logic into download() to handle plain file paths, and src == dst
...
Update download() function to handle src of plain path, by prepending
'file://' to it. Also handle the case of src and dst pointing to the same
file.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
fed562405d
archive: simplify handling of dsc file
...
There are only 2 options for getting the dsc file:
1) it's provided as 'dscfile' param
2) we look it up via lp_spph
For case #1 , update the constructor to immediately read the provided file
to create the self._dsc object, and set package and version from that.
For case #2 , simplify various functions to simply use the normal
_download_file_from_urls() function to get the dsc file from the
normal locations (mirrors, masters, directly from lp), using the same
process as any other file, including checking checksum(s).
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
248cf38d79
misc: update download() function
...
-require 'dst' parameter
-allow 'dst' parameter to be dir or dest file
-use contextlib.suppress instead of try/except/pass
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
3d0921ee54
archive: simplify _source_urls() and _binary_urls()
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
c29b660dbf
lpapicache: make sure source/binary file metadata includes all keys
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
b2f4ceee8e
archive: use _download_file_from_urls()
...
Both pull() and pull_binaries() are changed to use the common function.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
b6e0b5b388
archive: add _download_file_from_urls()
...
This will consolidate the download functionality from pull() and
pull_binaries()
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
494e0d6ddd
archive: change log level of some messages
...
Change some messages to the correct log level for their importanance.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
52739e44ad
test: simplify test_help with subtests
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
12c2e6bf42
test: remove unneeded stuff in test/__init__.py
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
ae74f71a1e
test: remove flake8 and pylint unittest files
...
unit tests aren't needed just to run flake8 or pylint.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Dan Streetman
48c7130008
test: remove dep on mock, use unittest.mock instead
...
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-02-02 06:25:12 -05:00
Stefano Rivera
90cb51becb
archive.py: Evaluate the filter() fixing syncpackage.
2021-01-29 22:18:22 -07:00
Dan Streetman
390be3f0b3
archive: remove DebianSourcePackage.pull_dsc()
...
No longer necessary to override this function as the superclass
now does everything the subclass function does.
Signed-off-by: Dan Streetman <ddstreet@canonical.com>
2021-01-22 16:19:49 -05:00
Dimitri John Ledkov
02c531dfb5
archive.py: use Regular, Ports, and Internal mirrors by default. Thus enabling pull-lp-debs to work with ports architectures, and inside launchpad builds too.
2021-01-19 10:43:48 +00:00
Dimitri John Ledkov
5a428b462d
config.py: add UBUNTU_INTERNAL_MIRROR option, for launchpad internal mirror.
2021-01-19 10:43:47 +00:00
Dimitri John Ledkov
6cc8040510
pullpkg.py: fix --mirror option parsing.
2021-01-19 10:43:46 +00:00
Dan Streetman
b30df2b227
pullpkg: also catch InvalidPullValueError
...
LP: #1908770
2020-12-21 11:28:08 -05:00
Dan Streetman
8aa2d602c5
pullpkg: add support for pulling from upload queue
2020-03-13 10:02:05 -04:00
Dan Streetman
8f42fb976f
lpapicache: add PackageUpload class
2020-03-13 10:02:05 -04:00
Dan Streetman
ea30b9f5bc
ubuntutools/misc: add download() function
...
Move the download functionality from the archive.py into the common
misc.py file.
2020-03-13 10:02:05 -04:00
Dan Streetman
fe4d2f08a5
test: remove test_dsc_missing test
...
This test class performs 'local source package' tests, but a missing
local dsc file won't perform any local test, it will query the LP server.
The test attempted to narrowly mock out specific network access to
emulate a 'missing' package by trying to return 404 when the url from
SourcePackage._lp_url() is opened, but now that _lp_url() is removed,
a missing local dsc file will result in full querying of the LP api,
which is not as easily mocked.
2020-03-06 13:01:37 +01:00
Dan Streetman
11429b213c
archive: remove _lp_url()
2020-03-06 13:01:37 +01:00
Dan Streetman
eaa07014e0
archive: verify and use existing already-downloaded files
2020-03-06 13:01:37 +01:00
Dan Streetman
8682c44957
archive: update pull_dsc to use _source_urls() and verify_file_checksum()
2020-03-06 13:01:37 +01:00
Dan Streetman
20dd65b281
archive: verify binary file checksums
2020-03-06 13:01:37 +01:00
Dan Streetman
45253b398c
archive: remove unneeded 'found' flag
...
This var isn't needed - just using for/else is better
2020-03-06 13:01:37 +01:00
Dan Streetman
e1f2cbf831
archive: remove _binary_files_info()
...
also use spph/bpph provided urls
2020-03-06 13:01:37 +01:00
Dan Streetman
8ebd086aac
ubuntutools/misc: add verify_file_checksum function
2020-03-06 13:01:37 +01:00
Dan Streetman
9f428e471b
ubuntutools/misc: convert print() to std logging
2020-03-06 13:01:37 +01:00
Dan Streetman
907061c15e
lpapicache: support 'include_meta' param in binaryFileUrls()
...
support the param for both native BPPH as well as Debian Snapshot
emulated BPPH
add sourceFileUrls() function
add helper functions to get file urls, checksums, and size
2020-03-06 13:01:37 +01:00
Dan Streetman
ee98423715
lpapicache: use urlparse instead of manual parsing
2020-03-06 13:01:37 +01:00
Dan Streetman
d98e16226b
archive: change _download_file verify param to dscverify
...
The param is specific to dsc-verifiable files, so name it accordingly.
2020-03-06 12:18:10 +01:00
Dan Streetman
53fa2336b8
archive: cache Debian srcpkg when pulling binaries
2020-03-06 12:18:10 +01:00
Dan Streetman
f5694d7cf6
archive: pass src pkg file size to _download_file
2020-03-06 12:18:10 +01:00
Dan Streetman
7900a09763
archive: fix DebianSourcePackage implementation of _source_urls()
2020-03-06 12:18:10 +01:00
Dan Streetman
36ffc92e00
pullpkg: for -p list, show bpph arch for files built for 'all' arch
2020-02-13 17:43:24 -05:00
Dan Streetman
3d7f542e49
archive: allow pull_binaries() to accept arch=None
2020-02-13 17:43:24 -05:00
Dan Streetman
ebe460aad1
lpapicache: allow getBinaries(arch=None) to get all archs
...
this was the previous behavior and some scripts still call with no args.
LP: #1862286
2020-02-13 17:43:22 -05:00
Dan Streetman
5553d98e47
lpapicache: remove fallback_arch from getBinaries
...
Even for 'all' arch binaries, there are separate BPPHs per arch,
so we should index them properly by actual arch.
2020-02-13 17:41:35 -05:00