ubuntutools/misc: use iter_content instead of raw stream access

Reading the raw stream doesn't decode files that are content-encoded,
which is true for the 'changes' file from launchpad, so it's saved
to disk gzipped which isn't what is expected.

Using the python requests iter_content method instead uses the
built-in stream decoding that the requests library provides, and
saves the file uncompressed/unencoded.

Note that since the encoded Content-Length won't match the resulting
unencoded actual length of data we save to file, this also turns off
the progress bar for any files that have Content-Encoding.
This commit is contained in:
Dan Streetman 2021-11-19 07:18:30 -05:00
parent 85125e3c90
commit 1e2036399e

View File

@ -397,9 +397,11 @@ def _download(fsrc, fdst, size, *, blocksize):
# logging INFO is suppressed
# stderr isn't a tty
# we don't know the total file size
# the file is content-encoded (i.e. compressed)
show_progress = all((Logger.isEnabledFor(logging.INFO),
sys.stderr.isatty(),
size > 0))
size > 0,
'Content-Encoding' not in fsrc.headers))
terminal_width = 0
if show_progress:
@ -411,10 +413,7 @@ def _download(fsrc, fdst, size, *, blocksize):
downloaded = 0
try:
while True:
block = fsrc.raw.read(blocksize)
if not block:
break
for block in fsrc.iter_content(blocksize):
fdst.write(block)
downloaded += len(block)
progress_bar.update(downloaded, size)