2839 Commits

Author SHA1 Message Date
Dan Streetman
20261960f6 pullpkg: unpack downloaded src from upload queue 2021-11-19 08:11:31 -05:00
Dan Streetman
d05e023dfe ubuntutools: don't sys.exit if unpack fails, just log it 2021-11-19 08:11:31 -05:00
Dan Streetman
de295fe524 ubuntutools/misc: return dst Path object from download() 2021-11-19 08:11:31 -05:00
Dan Streetman
1e2036399e ubuntutools/misc: use iter_content instead of raw stream access
Reading the raw stream doesn't decode files that are content-encoded,
which is true for the 'changes' file from launchpad, so it's saved
to disk gzipped which isn't what is expected.

Using the python requests iter_content method instead uses the
built-in stream decoding that the requests library provides, and
saves the file uncompressed/unencoded.

Note that since the encoded Content-Length won't match the resulting
unencoded actual length of data we save to file, this also turns off
the progress bar for any files that have Content-Encoding.
2021-11-19 08:11:31 -05:00
Dan Streetman
85125e3c90 ubuntutools/misc: allow specifying blocksize to download methods 2021-11-19 08:11:15 -05:00
Dan Streetman
cfa45994d0 ubuntutools/misc: create helper class to display download progress bar 2021-11-19 08:09:08 -05:00
Mattia Rizzolo
a74a49fb81
backportpackage: Support backporting to Debian releases. LP: #974132
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-10-31 16:47:43 +01:00
Peter Pentchev
74867c90f4
mk-sbuild: Fix a check for TARGET_ARCH in a message.
Closes: #968316
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-10-31 16:34:00 +01:00
Mattia Rizzolo
b904993e05
Merge branch 'mk-sbuild-zfs' of git+ssh://git.launchpad.net/~paride/ubuntu-dev-tools
MR: https://code.launchpad.net/~paride/ubuntu-dev-tools/+git/ubuntu-dev-tools/+merge/409346
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-10-31 16:31:39 +01:00
Mattia Rizzolo
000d3c1c2d
open changelog
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-10-31 16:25:12 +01:00
Stefano Rivera
9a44175a17 Upload to unstable 0.186 2021-10-24 16:13:01 -07:00
Stefano Rivera
a9e2a2689d Replace nose with pytest (see: #997758). 2021-10-24 16:05:08 -07:00
Paride Legovini
fec7a72ef7 mk-sbuild: add support for zfs-snapshot schroots
Creating datasets with mountpoint=legacy is necessary because the
schroot helper scripts expect it. This will make zfs-snapshot schroots
behave more like lvm-snapshot schroots and less like btrfs-snapshot
schroots (e.g. the source schroot is not permanently mounted/visible).

Switching mount "style" requires changes in at least the
/etc/schroot/setup.d/05zfs and 10mount scripts.

LP: #1945349
2021-09-28 19:59:42 +02:00
Stefano Rivera
9360b17bcb Upload to unstable 0.185 2021-09-17 15:53:13 -07:00
Stefano Rivera
5eeb707142 Bump Standards-Version to 4.6.0, no changes needed. 2021-09-17 15:52:51 -07:00
Stefano Rivera
c1b1c106dc Update changelog 2021-09-17 15:52:51 -07:00
Dan Streetman
266085d587 misc: fix flake8 complaints 2021-09-17 07:25:47 -04:00
Dan Streetman
6ca12331d6 archive: use proper component
source packages sometimes have different component than their
bpphs, so use the correct component when downloading binaries

LP: #1943819
2021-09-16 21:01:19 -04:00
Dan Streetman
5fcc4b5b46 misc: handle ConnectionError as NotFoundError 2021-09-16 20:29:14 -04:00
Dan Streetman
a3ff68be5a misc: download to tmp file, to avoid leftover 0-size file on error 2021-09-16 19:24:00 -04:00
Krytarik Raido
a1b56ac31f
merge-changelog: Fix setting of newlines.
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-09-08 19:44:15 +02:00
Mattia Rizzolo
ceb020d0fa
lpapicache: fix sorting in Archive.getUploadersForPackage().
LP: #1862372
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-08-17 15:55:14 +02:00
Mattia Rizzolo
fbbcee9cc1
reflow code for the next commit
Gbp-Dch: Ignore
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-08-17 15:51:23 +02:00
Mattia Rizzolo
511aa3d80c
Merge branch 'fix-ppa-pull-lp-1938659' of git+ssh://git.launchpad.net/~alexmurray/ubuntu-dev-tools
MR: https://code.launchpad.net/~alexmurray/ubuntu-dev-tools/+git/ubuntu-dev-tools/+merge/406518
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-08-04 12:02:46 +02:00
Mattia Rizzolo
6c8109b6ae
init changelog
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-08-04 11:59:24 +02:00
Alex Murray
296e498fe9
archive: Fix PersonalPackageArchiveSourcePackage to yield URLs
Yielding the result of super()_source_urls() / _binary_urls() yields the
generator object itself which generates the list of URLs - not the URLs
which would be returned from this generator. Instead use yield from which
forwards the yield onto the generator object itself.

Fixes LP: #1938659

Signed-off-by: Alex Murray <alex.murray@canonical.com>
2021-08-03 10:39:55 +09:30
Mattia Rizzolo
3f0d63d5b6
Release 0.184
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
0.184
2021-07-17 17:57:22 +02:00
Mattia Rizzolo
49527c36e1
d/control: Bump debhelper compat level to 13.
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-07-17 17:56:30 +02:00
Mattia Rizzolo
14a8005d45
d/control: remove redudant Recommends that are already in Depends.
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-07-17 17:56:08 +02:00
Mattia Rizzolo
52032eb081
Merge branch 'ccache-support' of git+ssh://git.launchpad.net/~3v1n0/ubuntu-dev-tools
MR: https://code.launchpad.net/~3v1n0/ubuntu-dev-tools/+git/ubuntu-dev-tools/+merge/401817
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-07-17 17:54:04 +02:00
Mattia Rizzolo
4fc36efcf2
pbuilder-dist: use shutil.which instead of distutils.spawn.find_executable() to save a dependency
LP: #1936697
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-07-17 17:47:51 +02:00
Dan Streetman
7f227f6264 update changelog 2021-07-14 17:53:26 -04:00
Dan Streetman
d598900df7 d/control: remove no longer needed requirements
Nothing uses termcolor or yaml anymore in the code
2021-07-14 17:41:10 -04:00
Dan Streetman
b4ca04efaa test: fix archive tests
Assumptions were made about the implementation by mocking
that are no longer true, and the tests generally need to be
fixed to be more robust about testing
2021-07-14 17:41:10 -04:00
Dan Streetman
0eaf71737d test: fix example package class
this class can be much simpler and more robust and doesn't require
checking in the example package files to git
2021-07-14 17:41:10 -04:00
Dan Streetman
1b12d8b4e3 archive: convert to use pathlib instead of os.path
Mostly convert to using pathlib
2021-07-14 17:41:10 -04:00
Dan Streetman
be6e09a02b archive: don't use existing file if no verification methods provided
we shouldn't use an existing file if we aren't checking its checksums
2021-07-14 17:41:10 -04:00
Dan Streetman
86276665cd archive: Update PersonalPackageArchiveSourcePackage to handle private PPA
Unfortunately private PPAs require downloading files from the special server
private-ppa.launchpad.net, and all the usual urls provided by the LP api fail.
So add code to handle using those custom private URLs, and use authentication
when downloading.
2021-07-14 17:41:10 -04:00
Dan Streetman
3f2983c157 misc: Change download() method to use python requests and optional authentication
Since private PPAs require authentication, use python requests library instead
of urlopen(), since requests handles authentication easily
2021-07-14 17:41:10 -04:00
Dan Streetman
4f6d6bf2d8 misc: add extract_authentication method
This pulls the username:password out of a URL
2021-07-14 17:41:10 -04:00
Dan Streetman
df93a225a8 misc: replace os.path with Pathlib
also change some strings to use f-strings
2021-07-14 17:41:10 -04:00
Dan Streetman
d8df8cc869 misc: add download_bytes() and deprecate mode param for download_text()
Passing 'mode' assumes use of open(), but callers don't care about
implementation, just if the returned object is text or bytes
2021-07-14 17:41:04 -04:00
Dan Streetman
243a728d6c Move DownloadError into ubuntutools/misc 2021-07-13 14:08:59 -04:00
Dan Streetman
6e18d60de4 lpapicache: add Archive.getMySubscriptionURL()
Private PPA require using username/password to access their files,
so make this information available. This gets the currently logged in
user's "subscription URL", which includes the authentication data.
2021-07-13 07:55:02 -04:00
Dan Streetman
dbd453876e flake8 tests no longer need to specifically exclude ubuntu-archive-assistant
Now that ubuntu-archive-assistant is removed, we can simply test everything
with flake8
2021-07-12 12:45:27 -04:00
Dan Streetman
732ff00cac Completely remove ubuntu-archive-assistant code
This hasn't been updated even a single time since it was added without
review almost 3 years ago. It additionally has never been included in the
ubuntu-dev-tools package. It's unclear if anyone is using it for anything,
but in any case it certainly shouldn't be in the ubuntu-dev-tools repository.
2021-07-12 12:45:27 -04:00
Marco Trevisan (Treviño)
2ac69a89e3 mk-sbuild: Add support to configure ccache for each schroot
ccache can help greatly in speeding up recompilations in sbuild, but it
is a bit annoying to configure at each schroot creation.

So, add --cache option (and relative CCACHE config parameter) to
configure ccache for sbuild schroots.

By default we use a shared ccache directory, but each schroot can use
a customized one if needed (with local parameters) by using --cache-dir
(or CCACHE_DIR).

Default ccache max-size is 4G, but can be configured with --ccache-size
(or CCACHE_SIZE), the size value is applied to each ccache path, so can
be shared by multiple schroots or applied to a single one.
2021-06-15 02:24:16 +02:00
Marco Trevisan (Treviño)
4e5e6efdb1 mk-sbuild: Enable debugging in the finish.sh script if --debug is used 2021-06-15 02:23:05 +02:00
Marco Trevisan (Treviño)
e193c30695 mk-sbuild: Use a more maintainable OPTS list using a bash array 2021-06-15 02:23:05 +02:00
Stefano Rivera
6bf3640e8f Upload to unstable 0.183 2021-06-08 10:09:16 -04:00