736 Commits

Author SHA1 Message Date
Simon Quigley
5a20308ab1 Read ~/.devscripts in a more robust way, to ideally pick up multi-line variables (Closes: #725418). 2025-03-04 13:17:30 -06:00
ferbraher
4a4c4e0a27 Parsing arch parameter to getBinaryPackage() 2025-03-04 13:08:59 -06:00
Benjamin Drung
e7ba650414 Avoid unnecessary "elif" after "continue"
Address pylint's no-else-continue.
2024-11-02 17:55:33 +01:00
Benjamin Drung
3bc802a209 Use lazy % formatting in logging functions 2024-11-02 17:55:20 +01:00
Benjamin Drung
d7362d9ed8 Use Python f-strings
```
flynt -ll 99 -tc -tj -a pbuilder-dist pm-helper running-autopkgtests ubuntu-build ubuntutools
```
2024-11-02 17:49:20 +01:00
Benjamin Drung
c7a855ff20 Format code with black and isort
```
isort pbuilder-dist pm-helper running-autopkgtests ubuntu-build ubuntutools
black -C pbuilder-dist pm-helper running-autopkgtests ubuntu-build ubuntutools
```
2024-11-02 17:21:30 +01:00
Chris Peterson
f01234e8a5 update debian/copyright
- Correctly add ISC licenses to new files in ubuntutools/tests/*
  as specified in debian/copyright
- Add GPL-3 licenses and correct attribution for:
    - running-autopkgtests
    - ubuntutools/running_autopkgtests.py
2024-03-13 09:21:30 -07:00
Chris Peterson
43891eda88 depends: python3-launchpadlib-desktop
Replace the dependency on python3-launchpadlib with
python3-launchpadlib-desktop. This package is the same as python3-launchpadlib
except that it also includes python3-keyring, which is a requirement for
some of the desktop-centric code-paths. In the case, requestsync has a
path for logging in via a web browser which also requires python3-keyring
to be installed. This had caused a ModuleNotFoundError when
python3-launchpadlib dropped python3-keyring from Recommends to Suggests
(LP: #2049217).
2024-03-13 09:17:49 -07:00
Gianfranco Costamagna
0ec53180f2 Merge remote-tracking branch 'vorlon/ubuntu-build-revamp' 2024-03-12 10:36:13 +01:00
Steve Langasek
010af53d7c Add a -A archive option to act on ppas as well.
This results in a major refactor of the code to use launchpadlib directly
instead of the ubuntutools.lp.lpapicache module in ubuntu-dev-tools which is
idiosyncratic and does not expose the full launchpad API.  Easier to rewrite
to use the standard library.
2024-03-10 14:35:47 -07:00
Chris Peterson
bce1ef88c5 running-autopkgtests: use f-strings 2024-02-14 15:19:48 -08:00
Chris Peterson
a9eb902b83 running-autopkgtests: Changelog entry, ArgumentParser, refactor, tests
Created a new changelog entry to include addition of the running-autopkgtests
script. This includes a refactor of the original script resulting in a new
module in ubuntutools, test cases, and the addition an argument parser to
allow printing just the queued tests, just the running tests, or both
(default).
2024-02-14 15:19:43 -08:00
Simon Quigley
fd885ec239 Merge remote-tracking branch 'vorlon/pm-helper' 2024-01-29 09:57:52 -06:00
Simon Quigley
2f396fe549 When using pull-*-source to grab a package which already has a defined Vcs- field, display the exact same warning message apt source does. 2023-10-03 14:01:19 -05:00
Simon Quigley
bed2dc470d Add support for the non-free-firmware components in all tools already referencing non-free. 2023-07-26 13:04:12 -05:00
Robie Basak
232a73de31 ubuntutools/misc: swap iter_content for raw stream
This is a partial revert of 1e20363.

When downloading a .diff.gz source package file, we do expect it to be
written to disk still compressed. If we were to uncompress it, then we
would get a size mismatch and even if we were to ignore that, we'd get a
hash mismatch.

On the other hand when downloading a changes file we need to make sure
that is written to disk uncompressed.

To make this work in both cases we can ask the HTTP server for no
special content encoding using "Accept-Encoding: identity". This is what
wget requests, for example. Then we can write the output to the file
without performing any decoding at our end by using the raw response
object again.

This fixes both cases.

LP: #2025748
2023-07-06 11:28:21 +01:00
Steve Langasek
ad014685ea Import utils.py from ubuntu-archive-tools 2023-06-13 13:47:15 -07:00
Steve Langasek
89e788bf48 Remove references to deprecated http://people.canonical.com/~ubuntu-archive. 2023-05-30 19:37:11 -07:00
Steve Langasek
83158d24d9 Merge staged changes 2023-05-30 10:00:57 -07:00
Steve Langasek
6e6e1f1e1a Excise all references to cdbs (including in test cases) 2023-05-30 10:00:17 -07:00
Steve Langasek
ac2f980e0f Remove references to ftpmaster.internal. When this name is resolvable but firewalled, syncpackage hangs; and these are tools for developers, not for running in an automated context in the DCs where ftpmaster.internal is reachable. 2023-04-12 17:59:40 -07:00
Benjamin Drung
c8a757eb07 Format Python code with black 23.1
Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-04-04 12:11:36 +02:00
Benjamin Drung
21784052ba test: Fix deprecated return value for test case
```
ubuntutools/test/test_archive.py::LocalSourcePackageTestCase::test_pull
  /usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method LocalSourcePackageTestCase.test_pull of <ubuntutools.test.test_archive.LocalSourcePackageTestCase testMethod=test_pull>>)
    return self.run(*args, **kwds)
```

`test_pull` does not need to be run directly. Make it private.

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 17:39:12 +01:00
Benjamin Drung
aa556af89d Use f-strings
pylint complains about C0209: Formatting a regular string which could be
a f-string (consider-using-f-string)

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 19:32:58 +01:00
Benjamin Drung
069a6926c0 Implement conventions found by pylint
Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 17:28:33 +01:00
Benjamin Drung
444b319c12 Implement refactorings found by pylint
Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 16:58:24 +01:00
Benjamin Drung
4449cf2437 Fix warnings found by pylint
Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 15:51:29 +01:00
Benjamin Drung
f6fde2e217 fix: Use lazy % formatting in logging functions
pylint complains about W1201: Use lazy % formatting in logging functions
(logging-not-lazy) and W1203: Use lazy % formatting in logging functions
(logging-fstring-interpolation).

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 11:13:07 +01:00
Benjamin Drung
17bed46ffb feat: Add some type hints
Add some type hints to satisfy mypy.

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 10:35:22 +01:00
Benjamin Drung
72add78e9d Fix errors found by pylint
Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 10:19:24 +01:00
Benjamin Drung
ab64467f33 Run pylint during package build again
Commit ae74f71a1e9d4be043162b19d23f2d44c964c771 removed the pylint unit
test saying that unit tests are not needed to just run flake8 or pylint.

Since pylint is useful, add it back, but this time call it directly and
not embed it into a unit test.

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-31 00:05:15 +01:00
Benjamin Drung
b1bc7e1cdc Address pylint complaints
Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-30 23:10:31 +01:00
Benjamin Drung
4e27045f49 style: Sort Python imports with isort
```
isort -l 99 --profile=black .
```

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-30 21:28:47 +01:00
Benjamin Drung
3354b526b5 style: Format Python code with black
```
PYTHON_SCRIPTS=$(grep -l -r '^#! */usr/bin/python3$' .)
black -C -l 99 . $PYTHON_SCRIPTS
```

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2023-01-30 19:45:36 +01:00
Benjamin Drung
096d5612e7 sponsor-patch: Use --skip-patches when extract source package
Use `--skip-patches` when extract source packages with `dpkg-source`.
`--no-preparation` is a source package build option and `--skip-patches`
is the correct extract option.

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2022-11-16 18:37:56 +01:00
Benjamin Drung
b510dbd91e sponsor-patch: Ignore exit code 1 of debdiff call
sponsor-patch calls `debdiff` which exits with 1 if there are
differences. So accept exit codes 0 and 1 as expected.

Signed-off-by: Benjamin Drung <benjamin.drung@canonical.com>
2022-11-15 16:43:27 +01:00
Mattia Rizzolo
dabe475067
ubuntutools/archive.py: fix crash in SourcePackage()._source_urls()
Fix operation of SourcePackage._source_urls() (as used, for example, in
SourcePackage.pull() called by backportpackage) to also work when the
class is instantiated with a URL as .dsc.

This is a regression caused by 1b12d8b4e3315de3bf417b40a3c66279f309d72c
(first in v0.184) that moved from os.path.join() to Pathlib, but
os.path.join() was also used to join URLs.

Thanks: Unit 193 for the initial patch.
Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2022-09-29 10:34:51 +02:00
Dan Streetman
562e6b13cd lpapicache: force lp access on login to workaround possibly invalid cached creds 2022-08-09 12:08:50 -04:00
Dan Streetman
9c1561ff26 lpapicache: remove try-except around login that only logs the error and then re-raises 2022-08-09 12:07:31 -04:00
Graham Inggs
0dde3262d1
ubuntutools/lp: Python 3.10 compatibility 2022-01-20 15:38:03 +01:00
Mattia Rizzolo
3a903ca628
archive.py: support python 3.6
this is needed for the backports to bionic

Signed-off-by: Mattia Rizzolo <mattia@debian.org>
2021-12-09 17:56:50 +01:00
Dan Streetman
8fe22fbbb6 Remove hugdaylist script
The last email announcing a 'hug day' appears to have been over
5 years ago, and the hugdaylist script doesn't seem to work
right anymore anyway.

This also removes ubuntutools/lp/libsupport, which has no functions
used by anything else.
2021-11-22 13:00:52 -05:00
Dan Streetman
4d72d184db pullpkg: change pull_upload_queue params into specific keyword-only params 2021-11-19 08:11:31 -05:00
Dan Streetman
20261960f6 pullpkg: unpack downloaded src from upload queue 2021-11-19 08:11:31 -05:00
Dan Streetman
d05e023dfe ubuntutools: don't sys.exit if unpack fails, just log it 2021-11-19 08:11:31 -05:00
Dan Streetman
de295fe524 ubuntutools/misc: return dst Path object from download() 2021-11-19 08:11:31 -05:00
Dan Streetman
1e2036399e ubuntutools/misc: use iter_content instead of raw stream access
Reading the raw stream doesn't decode files that are content-encoded,
which is true for the 'changes' file from launchpad, so it's saved
to disk gzipped which isn't what is expected.

Using the python requests iter_content method instead uses the
built-in stream decoding that the requests library provides, and
saves the file uncompressed/unencoded.

Note that since the encoded Content-Length won't match the resulting
unencoded actual length of data we save to file, this also turns off
the progress bar for any files that have Content-Encoding.
2021-11-19 08:11:31 -05:00
Dan Streetman
85125e3c90 ubuntutools/misc: allow specifying blocksize to download methods 2021-11-19 08:11:15 -05:00
Dan Streetman
cfa45994d0 ubuntutools/misc: create helper class to display download progress bar 2021-11-19 08:09:08 -05:00
Dan Streetman
266085d587 misc: fix flake8 complaints 2021-09-17 07:25:47 -04:00