Compare commits

..

19 Commits

Author SHA1 Message Date
Steve Langasek
0050471249 releasing package livecd-rootfs version 2.542.4 2019-05-21 17:20:56 -07:00
Robert C Jennings
ceacaad3b2
ubuntu-cpc: Update changelog with change bug number 2019-05-21 17:32:47 -05:00
Robert C Jennings
ecba14d40e
magic-proxy: Send headers on error 2019-05-21 17:32:47 -05:00
Robert C Jennings
3b1f2bba99
ubuntu-cpc: Ensure base disk image is the same between all build targets
The following targets have livecd.ubuntu-cpc.manifest (and
livecd.ubuntu-cpc.ext4) which differ in some way from the 'all'
target. They are all missing grub-efi and other modifications:

root-dir
squashfs
tarball

These targets do not depend on the 'disk-image' target. This means that
the ext4 produced will lack the uefi modifications (and any from the
disk-image target binary hooks).

Since the ext4 file is common to all builds there is a chance that a
parallel build from one of these targets could overwrite this artifact.
This patch ensures that all targets will produce consistent base output.
2019-05-21 17:32:47 -05:00
Robert C Jennings
1257763c98
ubuntu-cpc: Allow comments in series files
An upcoming addition of the disk-image target to several series files will
required some explanation.  Without comment support in series files that
is not possible.  This patch adds support for comments in series files.
2019-05-21 17:32:47 -05:00
Robert C Jennings
c373d81ee0
ubuntu-cpc: Fixup comment references to file names post-parallelization 2019-05-21 17:32:47 -05:00
Ubuntu
c46aad38fa
Drop snap-tool
The cohort API used in the snap-tool has been dropped and
a replacement is needed.  Until then the code will be unused.
2019-05-21 17:32:47 -05:00
Robert C Jennings
620e17d2b7
ubuntu-cpc: parallel builds
* Replace "snap download" with tool that uses snap store's coherence feature

    This is important for parallel image builds to ensure all pre-seeded snaps
    have the same versions across image variants.

* Inject a proxy into the build providing a snapshot view of the package repo.

    When the REPO_SNAPSHOT_STAMP variable is set, the auto/build script will attempt
    to launch a transparent HTTP proxy on port 8080, and insert an iptables rule to
    redirect all outgoing HTTP requests to this proxy.

    The proxy, contained in the `magic-proxy` Python script, examines each request
    and silently overrides those pointing to InRelease files or files that are
    listed in InRelease files. It will instead provide the contents of the requested
    file as it was at REPO_SNAPSHOT_STAMP, by downloading the corresponding asset
    "by hash".

* Use series files with dependency handling to generate hook symlinks dynamically

    This patch currently only applies to the "ubuntu-cpc" project.

    More and more logic has been going into the hook scripts to decide
    under which conditions they should run or not. As we are moving
    to parallelized builds of image sets, this will get even more
    complicated. Base hooks will have to know which image sets they
    belong to and modification of the dependency chain between scripts
    will become more complicated and prone to errors, as the number of
    image sets grows.

    This patch introduces explicit ordering and dependency handling for
    scripts through the use of `series` files and an explicit syntax
    for dependency specification.
2019-05-21 17:32:47 -05:00
Michael Hudson-Doyle
9579ee1539 releasing package livecd-rootfs version 2.542.3 2019-05-08 10:54:09 +12:00
Michael Hudson-Doyle
a7b8928928 Remove device nodes later for ubuntu-base:minimized (i.e. docker) builds. (LP: #1828118) 2019-05-08 10:50:23 +12:00
Julian Andres Klode
af525ff0b4 releasing package livecd-rootfs version 2.542.2 2019-04-26 10:36:48 +02:00
Julian Andres Klode
82870b1185 Remove crufty files after minimize-manual
minimize-manual would be creating those files again, and other
parts may be creating them, so best to do the cleanup at the
very end.

LP: #1826377
2019-04-25 11:51:08 +02:00
Steve Langasek
5566c26c63 releasing package livecd-rootfs version 2.542.1 2018-12-20 13:13:11 -06:00
Balint Reczey
6d3346f5cf debian/control: Update Vcs-*: URLs 2018-12-12 16:25:54 +01:00
Balint Reczey
3efca0cf7a Add .keep files to preserve empty directories 2018-12-12 15:36:14 +01:00
Steve Langasek
96eb05a4db Merge lp:~codyshepherd/livecd-rootfs/ubuntu-core-dev-snaps-manifest 2018-11-13 11:20:02 -08:00
Cody Shepherd
3f07ddd5c9 Fixing version number in changelog. 2018-11-13 10:54:27 -08:00
Cody Shepherd
f661d400a1 Fix snap-seed-parse to take filename argument instead of printing to
stdout
2018-11-13 10:48:25 -08:00
Steve Langasek
44f8ddaa25 Merge lp:~codyshepherd/livecd-rootfs/uefi-manifests-cosmic/ 2018-10-29 10:52:49 -07:00
575 changed files with 5200 additions and 11316 deletions

View File

@ -1,63 +0,0 @@
#!/usr/bin/python3
"""List the packages which could be automatically installed without becoming autoremovable
Finds all manually installed meta packages, and prints their dependencies
which could be marked as automatically installed.
"""
import sys
import apt
def is_root(pkg):
"""Check if the package is a root package (manually inst. meta)"""
section = pkg.candidate.section if pkg.candidate else ""
return (pkg.is_installed and
not pkg.is_auto_installed and
(section == "metapackages" or
section.endswith("/metapackages")))
def main():
"""Main function"""
cache = apt.Cache(rootdir=sys.argv[1] if len(sys.argv) > 1 else None)
roots = set(pkg for pkg in cache if is_root(pkg))
workset = set(roots)
seen = set()
ubiquity_depends = set()
with cache.actiongroup():
while True:
print("Iteration", file=sys.stderr)
to_proc = workset - seen
if not to_proc:
break
for pkg in sorted(to_proc):
print(" Visiting", pkg, file=sys.stderr)
if pkg not in roots and pkg not in ubiquity_depends:
if not pkg.is_auto_installed:
print(pkg.name)
for dep in (pkg.installed.dependencies +
pkg.installed.recommends):
for bdep in dep.or_dependencies:
for ver in bdep.target_versions:
if ver.package.is_installed:
if pkg.name == "ubiquity":
ubiquity_depends.add(ver.package)
if pkg.name != "ubiquity":
# Reprocess this package again, as we did not mark it when we visited it from ubiquity
try:
ubiquity_depends.remove(ver.package)
# This will raise the KeyError here if ubiquity did not depend on it
seen.remove(ver.package)
except KeyError:
pass
workset.add(ver.package)
seen.add(pkg)
if __name__ == '__main__':
main()

View File

@ -1,23 +0,0 @@
#!/bin/sh
set -eux
branch=$1
dir=$2
target="$(readlink -f "${3}")"
tmpdir="$(mktemp -d)"
cd "${tmpdir}"
cleanup () {
rm -rf "${tmpdir}"
}
trap cleanup EXIT
mkdir -p $target
git clone $branch checkout
for po in checkout/$dir/*.po; do
msgfmt "${po}" -o "${target}/$(basename "${po}" .po).mo"
done

1
debian/.gitignore vendored
View File

@ -1 +0,0 @@
files

4448
debian/changelog vendored

File diff suppressed because it is too large Load Diff

1
debian/compat vendored Normal file
View File

@ -0,0 +1 @@
7

34
debian/control vendored
View File

@ -1,54 +1,42 @@
Source: livecd-rootfs
Section: devel
Priority: optional
Build-Depends: debhelper-compat (= 13)
Build-Depends: debhelper (>= 7)
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Standards-Version: 4.7.0
Vcs-Git: https://git.launchpad.net/livecd-rootfs -b ubuntu/master
Vcs-Browser: https://git.launchpad.net/livecd-rootfs
Rules-Requires-Root: no
Standards-Version: 3.9.6
Vcs-Git: https://git.launchpad.net/livecd-rootfs -b ubuntu/cosmic
Package: livecd-rootfs
Architecture: any
Depends: ${misc:Depends},
apt-utils,
attr,
debootstrap,
devscripts,
distro-info,
dosfstools,
e2fsprogs,
gdisk,
genisoimage,
germinate (>= 1.25.1),
gettext,
git,
gnupg,
grep-dctrl,
kpartx,
live-build (>= 3.0~a57-1ubuntu31~),
lsb-release,
lzma,
make,
mount,
mtools [!i386],
parted,
procps,
python3,
python-minimal | python,
python3-apt,
python3-launchpadlib [!i386],
python3-yaml,
qemu-utils [!i386],
rsync [!i386],
snapd (>= 2.39) [!i386],
python3-software-properties,
qemu-utils,
rsync,
snapd,
squashfs-tools (>= 1:3.3-1),
sudo,
u-boot-tools [arm64 armhf],
python3-vmdkstream [amd64 i386],
xorriso [!i386],
ubuntu-image,
vmdk-stream-converter [amd64 i386],
xz-utils,
zerofree,
zstd
zerofree
Breaks: ubuntu-defaults-builder (<< 0.32)
Description: construction script for the livecd rootfs
livecd-rootfs provides the script used to create the root filesystem

2
debian/copyright vendored
View File

@ -11,7 +11,7 @@ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU General Public License with
your Ubuntu system, in /usr/share/common-licenses/GPL-2, or with the
your Ubuntu system, in /usr/share/common-licenses/GPL, or with the
livecd-rootfs source package as the file COPYING. If not, write to
the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
Boston, MA 02110-1301 USA.

4
debian/gbp.conf vendored
View File

@ -1,4 +0,0 @@
[DEFAULT]
debian-branch=ubuntu/master
debian-tag = %(version)s
dch-opt = --vendor=debian

6
debian/install vendored
View File

@ -1,7 +1,5 @@
auto-markable-pkgs usr/share/livecd-rootfs
live-build usr/share/livecd-rootfs
get-ppa-fingerprint usr/share/livecd-rootfs
minimize-manual usr/share/livecd-rootfs
checkout-translations-branch usr/share/livecd-rootfs
update-source-catalog usr/share/livecd-rootfs
sync-mtime usr/share/livecd-rootfs
magic-proxy usr/share/livecd-rootfs
lp-in-release usr/share/livecd-rootfs

View File

@ -1,10 +0,0 @@
# This is expected and okay
livecd-rootfs: service-file-is-not-a-file [usr/share/livecd-rootfs/*.service]
livecd-rootfs: uses-dpkg-database-directly [usr/share/livecd-rootfs/live-build/lb_chroot_layered]
livecd-rootfs: package-contains-no-arch-dependent-files
# Developer documentation in an already-expected spot
livecd-rootfs: package-contains-documentation-outside-usr-share-doc [usr/share/livecd-rootfs/live-build/ubuntu-cpc/README.cpc.md]
# This is going to occur quite a bit; we're making a livefs
livecd-rootfs: repeated-path-segment * [*]

View File

@ -1,5 +0,0 @@
# This is expected and okay
livecd-rootfs source: absolute-symbolic-link-target-in-source * [live-build/*]
# The description indicates that it's not required in debhelper >= 13, which we have
livecd-rootfs source: override_dh_auto_test-does-not-check-DEB_BUILD_OPTIONS [debian/rules:*]

View File

@ -1,7 +1,7 @@
Tests: default-bootstraps
Depends: @, lsb-release
Restrictions: needs-root isolation-machine allow-stderr
Restrictions: needs-root isolation-machine
Tests: minimized
Depends: @, lsb-release
Restrictions: needs-root isolation-machine allow-stderr
Restrictions: needs-root isolation-machine

View File

@ -13,41 +13,46 @@ fi
# Listed subprojects can be combined with other projects as well,
# but this list gives reasonable coverage.
ALL_TRIPLETS="
base::
edubuntu::
edubuntu-dvd::
kubuntu::
kubuntu-active::
kubuntu-dvd::
kubuntu-plasma5::
lubuntu::
ubuntu-unity::
mythbuntu::
ubuntu::
ubuntu-base::
ubuntu-base:buildd:
ubuntu-budgie::
ubuntu-budgie-desktop::
ubuntu-budgie-live::
ubuntucinnamon::
ubuntu-core:system-image:ubuntu-core
ubuntu-cpc::ubuntu-cpc
ubuntu-cpc:minimized:ubuntu-cpc
ubuntu-desktop-next:system-image:ubuntu-desktop-next
ubuntu-desktop-next::ubuntu-desktop-next
ubuntu-dvd::
ubuntu-gnome::
ubuntukylin::
ubuntu-mate::
ubuntu-mate-core::
ubuntu-mate-desktop::
ubuntu-mate-live::
ubuntu-server:live:ubuntu-server
xubuntu::
xubuntu:minimal:"
ARCH=$(dpkg --print-architecture)
ubuntu-netbook::
ubuntu-server::
ubuntu-server:ubuntu-rtm:
ubuntu-server:ubuntu-rtm/foo:
ubuntu-server:wubi:
ubuntu-touch-custom::ubuntu-touch-custom
ubuntu-touch::ubuntu-touch
xubuntu::"
if [ -z "$SELECTED_TRIPLETS" ]; then
SELECTED_TRIPLETS="
ubuntu-base::
ubuntu-cpc::ubuntu-cpc
"
if [ "$ARCH" = amd64 ]; then
SELECTED_TRIPLETS="$SELECTED_TRIPLETS
ubuntu::"
fi
fi
live_build_rootfs() {
@ -55,6 +60,7 @@ live_build_rootfs() {
local SUBPROJECT_TMP=${1%:*}
SUBPROJECT=${SUBPROJECT_TMP#*:}
TEMPLATE=${1##*:}
ARCH=$(dpkg --print-architecture)
echo "Building rootfs for project: '$PROJECT' subproject: '$SUBPROJECT' template: '$TEMPLATE' in $PWD"
cp -a /usr/share/livecd-rootfs/live-build/auto .
if [ -n "$TEMPLATE" ]; then
@ -64,22 +70,19 @@ live_build_rootfs() {
SUBPROJECT=$SUBPROJECT \
SUITE=$SUITE \
ARCH=$ARCH \
NOW=$(date '+%Y%m%d') \
lb config
mkdir chroot
# this part needs root rights, but right now the whole script ran as root by autopkgtest
env PROJECT=$PROJECT \
SUBPROJECT=$SUBPROJECT \
ARCH=$ARCH \
NOW=$(date '+%Y%m%d') \
lb build
echo "Build results for project: '$PROJECT' subproject: '$SUBPROJECT' template: '$TEMPLATE' in $PWD"
du -sh *
echo ""
}
[ -d /adt ] || mkdir /adt
WORKDIR=$(mktemp -d -p /adt)
WORKDIR=$(mktemp -d)
trap "RET=\$?; rm -rf $WORKDIR; exit \$RET" 0 INT QUIT ABRT PIPE TERM
cd $WORKDIR

View File

@ -2,14 +2,16 @@
from __future__ import print_function
from launchpadlib.launchpad import Launchpad
from optparse import OptionParser
from softwareproperties import ppa
parser = OptionParser(usage="%prog OWNER/NAME")
_, args = parser.parse_args()
if not args:
parser.error("must provide a PPA owner/name")
owner_name, ppa_name = args[0].split("/", 1)
lp = Launchpad.login_anonymously('livecd-rootfs', 'production', version='devel')
print(lp.people(owner_name).getPPAByName(name=ppa_name).signing_key_fingerprint)
owner_name = "~%s" % owner_name
ppa_info = ppa.get_ppa_info_from_lp(owner_name, ppa_name)
print(ppa_info["signing_key_fingerprint"])

View File

@ -1 +0,0 @@
0xffffff

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
chown dac_override dac_read_search fowner fsetid kill setgid setuid setpcap linux_immutable net_bind_service net_broadcast net_admin net_raw ipc_lock ipc_owner sys_module sys_rawio sys_chroot sys_ptrace sys_pacct sys_admin sys_boot sys_nice sys_resource sys_time sys_tty_config mknod lease audit_write audit_control setfcap mac_override mac_admin syslog wake_alarm block_suspend audit_read perfmon bpf checkpoint_restore

View File

@ -1 +0,0 @@
acquire send receive

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
1.2

View File

@ -1 +0,0 @@
create read write exec append mmap_exec link lock

View File

@ -1 +0,0 @@
sqpoll override_creds

View File

@ -1 +0,0 @@
create read write open delete setattr getattr

View File

@ -1 +0,0 @@
mount umount pivot_root

View File

@ -1 +0,0 @@
detached

View File

@ -1 +0,0 @@
userns_create

View File

@ -1 +0,0 @@
no

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
pciu&

View File

@ -1 +0,0 @@
unspec unix inet ax25 ipx appletalk netrom bridge atmpvc x25 inet6 rose netbeui security key netlink packet ash econet atmsvc rds sna irda pppox wanpipe llc ib mpls can tipc bluetooth iucv rxrpc isdn phonet ieee802154 caif alg nfc vsock kcm qipcrtr smc xdp mctp

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
unspec unix inet ax25 ipx appletalk netrom bridge atmpvc x25 inet6 rose netbeui security key netlink packet ash econet atmsvc rds sna irda pppox wanpipe llc ib mpls can tipc bluetooth iucv rxrpc isdn phonet ieee802154 caif alg nfc vsock kcm qipcrtr smc xdp mctp

View File

@ -1 +0,0 @@
file

View File

@ -1 +0,0 @@
0x000001

View File

@ -1 +0,0 @@
allow deny subtree cond kill complain prompt audit quiet hide xindex tag label

View File

@ -1 +0,0 @@
0x000003

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
0x000001

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
read trace

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
allow deny audit quiet

View File

@ -1 +0,0 @@
cpu fsize data stack core rss nproc nofile memlock as locks sigpending msgqueue nice rtprio rttime

View File

@ -1 +0,0 @@
hup int quit ill trap abrt bus fpe kill usr1 segv usr2 pipe alrm term stkflt chld cont stop stp ttin ttou urg xcpu xfsz vtalrm prof winch io pwr sys emt lost

View File

@ -1 +0,0 @@
0xffffff

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
chown dac_override dac_read_search fowner fsetid kill setgid setuid setpcap linux_immutable net_bind_service net_broadcast net_admin net_raw ipc_lock ipc_owner sys_module sys_rawio sys_chroot sys_ptrace sys_pacct sys_admin sys_boot sys_nice sys_resource sys_time sys_tty_config mknod lease audit_write audit_control setfcap mac_override mac_admin syslog wake_alarm block_suspend audit_read perfmon bpf checkpoint_restore

View File

@ -1 +0,0 @@
acquire send receive

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
1.2

View File

@ -1 +0,0 @@
create read write exec append mmap_exec link lock

View File

@ -1 +0,0 @@
sqpoll override_creds

View File

@ -1 +0,0 @@
create read write open delete setattr getattr

View File

@ -1 +0,0 @@
mount umount pivot_root

View File

@ -1 +0,0 @@
detached

View File

@ -1 +0,0 @@
userns_create

View File

@ -1 +0,0 @@
no

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
pciu&

View File

@ -1 +0,0 @@
unspec unix inet ax25 ipx appletalk netrom bridge atmpvc x25 inet6 rose netbeui security key netlink packet ash econet atmsvc rds sna irda pppox wanpipe llc ib mpls can tipc bluetooth iucv rxrpc isdn phonet ieee802154 caif alg nfc vsock kcm qipcrtr smc xdp mctp

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
yes

View File

@ -1 +0,0 @@
unspec unix inet ax25 ipx appletalk netrom bridge atmpvc x25 inet6 rose netbeui security key netlink packet ash econet atmsvc rds sna irda pppox wanpipe llc ib mpls can tipc bluetooth iucv rxrpc isdn phonet ieee802154 caif alg nfc vsock kcm qipcrtr smc xdp mctp

View File

@ -1 +0,0 @@
file

View File

@ -1 +0,0 @@
0x000001

View File

@ -1 +0,0 @@
allow deny subtree cond kill complain prompt audit quiet hide xindex tag label

View File

@ -1 +0,0 @@
0x000003

Some files were not shown because too many files have changed in this diff Show More