Update changelog

LP: #1827930

debian/changelog

@@ -1,3 +1,246 @@
+livecd-rootfs (2.525.24) bionic; urgency=medium
+
+  * Build WSL rootfs tarball (LP: #1827930)
+
+ -- Balint Reczey <rbalint@ubuntu.com>  Fri, 17 May 2019 15:11:42 +0200
+
+livecd-rootfs (2.525.23) bionic; urgency=medium
+
+  * Backport two minimizations for the docker images: remove apt lists that
+    are removed downstream anyway, and remove device nodes from the image.
+    (LP: #1828118)
+
+ -- Michael Hudson-Doyle <michael.hudson@ubuntu.com>  Wed, 08 May 2019 10:48:39 +1200
+
+livecd-rootfs (2.525.22) bionic; urgency=medium
+
+  * Subiquity specific changes SRU LP: #1827357
+    - subiquity: make subiquity_config.mount optional
+    - Make serial-subiquity@ use the same codepath as tty1 subiquity.
+    - Fix ubuntu-server-live images to generate initrd with casper UUID.
+
+ -- Dimitri John Ledkov <xnox@ubuntu.com>  Thu, 02 May 2019 11:34:34 +0100
+
+livecd-rootfs (2.525.21) bionic; urgency=medium
+
+  * Remove crufty files after minimize-manual (LP: #1826377)
+
+ -- Julian Andres Klode <juliank@ubuntu.com>  Fri, 26 Apr 2019 10:40:08 +0200
+
+livecd-rootfs (2.525.20) bionic; urgency=medium
+
+  [ Steve Langasek ]
+  * Drop /etc/update-motd.d/51-cloudguest from cloud images; this is not
+    consistent with current Ubuntu Advantage product language.  Any future
+    customizations to update-motd for cloud images should be done via a
+    package instead.
+
+  [ Julian Andres Klode ]
+  * Minimize the number of manually installed packages in images by marking
+    dependencies of metapackages as automatically installed. (LP: #1800610);
+    but do not mark direct dependencies of ubiquity as auto installed. This
+    caused cryptsetup to remain auto on the installed system in bionic (see
+    LP #1801629)
+
+ -- Steve Langasek <steve.langasek@ubuntu.com>  Thu, 14 Mar 2019 11:04:49 -0700
+
+livecd-rootfs (2.525.19) bionic; urgency=medium
+
+  * Make sure buildd images have a /usr/sbin/policy-rc.d symlink
+    (LP: #1815251).
+
+ -- Colin Watson <cjwatson@ubuntu.com>  Mon, 18 Feb 2019 16:03:46 +0000
+
+livecd-rootfs (2.525.18) bionic; urgency=medium
+
+  [ Michael Hudson-Doyle ]
+  * Changes to kernel handling for live-server: do not include kernel/initrd
+    in filesystem.squashfs but rather install it in a throwaway layer on top
+    of installer.squashfs and fish kernel, initrd and modules out of that with
+    an initrd hook that records kernel metapackage name in /run and mounts
+    /lib/modules from a squashfs on the ISO.
+  * Do no install openssh-server in the base filsystem for the live server
+    installer.
+  * A few simple tweaks to reduce size of live servers installer.squashfs:
+    - Do not run apt-get update (which can bring in package lists if we are
+      unlucky wrt publisher schedules).
+    - Run apt-get clean to clear out downloaded debs of curtin/casper and
+      dependencies.
+    - Do not install user-setup.
+    - Use the core snap from the base filesystem if present.
+  * Do not include curtin in the live-server installer.squashfs as the
+    version of subiquity that includes it in the snap has now been released to
+    stable.
+
+ -- Adam Conrad <adconrad@ubuntu.com>  Tue, 12 Feb 2019 11:35:48 -0700
+
+livecd-rootfs (2.525.17) bionic; urgency=medium
+
+  * Add a buildd subproject (LP: #1815251).
+  * Add a LXD image to builds for the buildd subproject.
+  * Move buildd image building to binary hooks.
+
+ -- Colin Watson <cjwatson@ubuntu.com>  Fri, 08 Feb 2019 22:57:59 +0000
+
+livecd-rootfs (2.525.16) bionic; urgency=medium
+
+  * lubuntu: Select lubuntu-gtk-core task as well, so we get the HWE filter
+    applied to the explicit dependencies it has on Xorg-recommended packages.
+  * lubuntu: Manually install some packages that fall out due to the above.
+
+ -- Adam Conrad <adconrad@ubuntu.com>  Mon, 04 Feb 2019 12:48:51 -0700
+
+livecd-rootfs (2.525.15) bionic; urgency=medium
+
+  * Forward-port HWE support from xenial, make it more generic, and enable
+    it for the current LTS flavours.
+
+ -- Adam Conrad <adconrad@ubuntu.com>  Fri, 01 Feb 2019 12:41:39 -0700
+
+livecd-rootfs (2.525.14) bionic; urgency=medium
+
+  * More changes for raspi3 build support (LP: #1805668):
+    - Fix 100-purge-grub-legacy-ec2-arm.chroot to not gate on ARCH as that's
+      not defined in .chroot hooks.
+    - Use the new linux-firmware-raspi2 for boot binary blobs as
+      raspi3-firmware includes upgrade hooks that conflict with our image
+      configuration.
+
+ -- Łukasz 'sil2100' Zemczak <lukasz.zemczak@ubuntu.com>  Fri, 18 Jan 2019 16:56:58 +0100
+
+livecd-rootfs (2.525.13) bionic; urgency=medium
+
+  [ Balint Reczey ]
+  * Update Vcs-* fields in debian/control to point to git.
+
+  [ Łukasz 'sil2100' Zemczak ]
+  * Another batch of cherry-picks for raspi3 support (LP: #1805668)
+    - Add wpasupplicant to the additional packages installed for the raspi2 and
+      raspi3 targets.
+    - Default to IMAGEFORMAT=ubuntu-image for raspi3 ubuntu-cpc builds.
+    - Link the resulting raspi3 image to a filename that cdimage expects from a
+      preinstalled image build.
+  * Add the 100-purge-grub-legacy-ec2-arm.chroot hook to remove
+    grub-legacy-ec2 from any ARM based ubuntu-cpc images we create. The package
+    is no longer in the server seed of newer series anyway.
+
+ -- Łukasz 'sil2100' Zemczak <lukasz.zemczak@ubuntu.com>  Fri, 11 Jan 2019 14:53:10 +0100
+
+livecd-rootfs (2.525.12) bionic; urgency=medium
+
+  * Key netplan delegation to NetworkManager on presence of
+    /usr/sbin/NetworkManager, not on /usr/lib/NetworkManager which may have
+    hooks from other packages (i.e., wpasupplicant).
+
+  [ Cody Shepherd ]
+  * Include snaps in image manifests (LP: #1805497)
+  * Change call to add grub efi packages using new create_manifests()
+    function.
+
+ -- Steve Langasek <steve.langasek@ubuntu.com>  Mon, 10 Dec 2018 12:46:46 -0800
+
+livecd-rootfs (2.525.11) bionic; urgency=medium
+
+  * Backport all the required changes to enable Raspberry Pi 3 armhf and arm64
+    preinstalled image builds. (LP: #1805668)
+    - Add support for raspi3 rootfs builds (based on Ryan Finnie's changes).
+    - For ubuntu-image consumption, export the kernel and initrd to
+      image/boot/uboot for raspi*.
+    - Avoid issues of hard-linking to a symbolic vmlinuz as this can lead to a
+      dangling symlink.
+    - Add raspi3 arm64 rootfs build support.
+    - Add git to the build dependencies for the gadget tree pull.
+    - Minor fixes to raspi3 builds: add arm64+raspi3 to the supported model
+      list, pass SUITE on to the build stage and use the git:// url for the
+      gadget tree.
+
+ -- Łukasz 'sil2100' Zemczak <lukasz.zemczak@ubuntu.com>  Thu, 29 Nov 2018 16:24:23 +0100
+
+livecd-rootfs (2.525.10) bionic; urgency=medium
+
+  [ Cody Shepherd ]
+  * Include grub efi packages in manifests for uefi images.
+    (LP: #1805190)
+
+  [ Robert C Jennings ]
+  * Disable checksum generation. (LP: #1799773)
+
+  [Steve Langasek]
+  * Fix Vcs-Bzr link.
+
+ -- Steve Langasek <steve.langasek@ubuntu.com>  Mon, 26 Nov 2018 12:55:11 -0800
+
+livecd-rootfs (2.525.9) bionic; urgency=medium
+
+  * Ensure /lib/modules exists in root tarballs and sqashfs.
+    (LP: #1792905)
+
+ -- Tobias Koch <tobias.koch@canonical.com>  Thu, 20 Sep 2018 09:30:34 +0200
+
+livecd-rootfs (2.525.8) bionic; urgency=medium
+
+  * Disentangle enabling universe in the final image a little from having
+    PREINSTALLED=true set and enable it for a live-server build.
+    (LP: #1783129)
+  * Fix live-server journald config snippet to actually disable journald rate
+    limiting.
+
+ -- Michael Hudson-Doyle <michael.hudson@ubuntu.com>  Tue, 28 Aug 2018 11:03:37 +1200
+
+livecd-rootfs (2.525.7) bionic; urgency=medium
+
+  * Disable journald rate limiting in the live-server live session. 
+    (LP: #1776891) 
+
+ -- Michael Hudson-Doyle <michael.hudson@ubuntu.com>  Tue, 14 Aug 2018 21:41:53 -0700
+
+livecd-rootfs (2.525.6) bionic; urgency=medium
+
+  [ Steve Langasek ]
+  * generate all tar files with --xattrs.  LP: #1302192.
+
+  [ Daniel Watkins ]
+  * ubuntu-cpc: Reintroduce the -root.tar.xz artifact (LP: #1585233).
+  * ubuntu-cpc: Generate the root image contents once, and use it for both the
+    -root.tar.xz and the .squashfs.
+  * ubuntu-cpc: Generate -root.tar.xz with --xattrs.
+
+ -- Steve Langasek <steve.langasek@ubuntu.com>  Mon, 06 Aug 2018 14:16:04 -0700
+
+livecd-rootfs (2.525.5) bionic; urgency=medium
+
+  * live-build/auto/config: Improve linux-tools filter to work for i386 too.
+
+ -- Adam Conrad <adconrad@ubuntu.com>  Tue, 24 Jul 2018 22:21:27 -0600
+
+livecd-rootfs (2.525.4) bionic; urgency=medium
+
+  * live-build/auto/config: Dirty hack to filter linux-tools-aws from budgie.
+
+ -- Adam Conrad <adconrad@ubuntu.com>  Tue, 24 Jul 2018 21:55:49 -0600
+
+livecd-rootfs (2.525.3) bionic; urgency=medium
+
+  * live-build/auto/config: Add nasty hack to order gtk-common-themese snap
+    first in snapd's seed.yaml to work around the snapd bug in LP: #1772844
+
+ -- Adam Conrad <adconrad@ubuntu.com>  Mon, 23 Jul 2018 10:02:04 -0600
+
+livecd-rootfs (2.525.2) bionic; urgency=medium
+
+  * Update unminimize script text and install ubuntu-standard when
+    unminimizing a minimal image (LP: #1778777)
+
+ -- Francis Ginther <francis.ginther@canonical.com>  Mon, 02 Jul 2018 13:27:15 -0500
+
+livecd-rootfs (2.525.1) bionic; urgency=medium
+
+  * Symlink systemd-networkd-wait-online to /bin/true in the live installer
+    live session to avoid depending on buggy systemd behaviour. (LP: #1773719)
+  * Remove "optional: true" from installer netplan config.
+
+ -- Michael Hudson-Doyle <michael.hudson@ubuntu.com>  Wed, 23 May 2018 14:27:01 +1200
+
 livecd-rootfs (2.525) bionic; urgency=medium
 
   * Don't fail to build CPC images if /lib/modules already exists in the

debian/control

@@ -4,12 +4,13 @@ Priority: optional
 Build-Depends: debhelper (>= 7)
 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
 Standards-Version: 3.9.6
-Vcs-Bzr: http://bazaar.launchpad.net/~ubuntu-core-dev/livecd-rootfs/trunk
+Vcs-Git: https://git.launchpad.net/livecd-rootfs -b ubuntu/bionic
 
 Package: livecd-rootfs
 Architecture: any
 Depends: ${misc:Depends},
          apt-utils,
+         attr,
          debootstrap,
          distro-info,
          dosfstools,
@@ -17,21 +18,25 @@ Depends: ${misc:Depends},
          gdisk,
          genisoimage,
          germinate (>= 1.25.1),
+         git,
          gnupg,
          grep-dctrl,
          kpartx,
          live-build (>= 3.0~a57-1ubuntu31~),
          lsb-release,
          lzma,
+         make,
          parted,
          procps,
          python-minimal | python,
+         python3-apt,
          python3-software-properties,
          qemu-utils,
          rsync,
          snapd,
          squashfs-tools (>= 1:3.3-1),
          sudo,
+         u-boot-tools [armhf arm64],
          ubuntu-image,
          vmdk-stream-converter [amd64 i386],
          xz-utils,

debian/install

@@ -1,2 +1,3 @@
 live-build usr/share/livecd-rootfs
 get-ppa-fingerprint usr/share/livecd-rootfs
+minimize-manual usr/share/livecd-rootfs

debian/tests/default-bootstraps

@@ -25,6 +25,7 @@ ALL_TRIPLETS="
  mythbuntu::
  ubuntu::
  ubuntu-base::
+ ubuntu-base:buildd:
  ubuntu-budgie::
  ubuntu-budgie-desktop::
  ubuntu-budgie-live::

live-build/auto/build

@@ -17,18 +17,39 @@ fi
 
 . config/functions
 
-# Link output files somewhere BuildLiveCD will be able to find them.
+# Link output files somewhere launchpad-buildd will be able to find them.
 PREFIX="livecd.$PROJECT${SUBARCH:+-$SUBARCH}"
 
 if [ "${IMAGEFORMAT:-}" = "ubuntu-image" ]; then
 	# Use ubuntu-image instead of live-build
 
-	CHANNEL="${CHANNEL:-edge}"
-	env SNAPPY_STORE_NO_CDN=1 \
-	ubuntu-image -c "$CHANNEL" $UBUNTU_IMAGE_ARGS \
-		-o "$PREFIX".img "$PREFIX".model-assertion
-	xz -0 -T4 "$PREFIX".img
-	mv seed.manifest "$PREFIX".manifest
+	if [ "$PROJECT" = "ubuntu-core"]; then
+		CHANNEL="${CHANNEL:-edge}"
+		env SNAPPY_STORE_NO_CDN=1 \
+		ubuntu-image snap -c "$CHANNEL" $UBUNTU_IMAGE_ARGS \
+			-O output "$PREFIX".model-assertion
+		# XXX: currently we only have one image generated, but really
+		#  we should be supporting more than one for models that
+		#  define those.
+		mv output/*.img "$PREFIX".img
+		xz -0 -T4 "$PREFIX".img
+		mv output/seed.manifest "$PREFIX".manifest
+	else
+		# First we need to build the gadget tree
+		make -C "config/$PREFIX-gadget" ARCH=$ARCH SERIES=$SUITE
+		ubuntu-image classic $UBUNTU_IMAGE_ARGS \
+			-s $SUITE -p $PROJECT -a $ARCH --subarch $SUBARCH \
+			-O output config/$PREFIX-gadget/install
+		# XXX: currently we only have one image generated, but really
+		#  we should be supporting more than one for models that
+		#  define those.
+		mv output/*.img "$PREFIX".img
+		xz -0 -T4 "$PREFIX".img
+		# Also link the output image to a filename that cdimage expects
+		ln "$PREFIX".img.xz livecd.ubuntu-cpc.disk1.img.xz
+		mv output/filesystem.manifest "$PREFIX".manifest
+	fi
+
 	exit 0
 fi
 
@@ -109,12 +130,16 @@ set -e
 echo "This system has been minimized by removing packages and content that are"
 echo "not required on a system that users do not log into."
 echo ""
-echo "This script restores the content and packages that are found on a default"
-echo "Ubuntu server system."
+echo "This script restores content and packages that are found on a default"
+echo "Ubuntu server system in order to make this system more suitable for"
+echo "interactive use."
 echo ""
 echo "Reinstallation of packages may fail due to changes to the system"
 echo "configuration, the presence of third-party packages, or for other"
 echo "reasons."
+echo ""
+echo "This operation may take some time."
+echo ""
 read -p "Would you like to continue? [y/N]" REPLY
 echo    # (optional) move to a new line
 if [ "$REPLY" != "y" ] && [ "$REPLY" != "Y" ]
@@ -155,7 +180,7 @@ fi
 
 if ! dpkg-query --show --showformat='${db:Status-Status}\n' ubuntu-minimal 2> /dev/null | grep -q '^installed$'; then
     echo "Installing ubuntu-minimal package to provide the familiar Ubuntu minimal system..."
-    DEBIAN_FRONTEND=noninteractive apt-get install -y ubuntu-minimal
+    DEBIAN_FRONTEND=noninteractive apt-get install -y ubuntu-minimal ubuntu-standard
 fi
 
 if dpkg-query --show --showformat='${db:Status-Status}\n' ubuntu-server 2> /dev/null | grep -q '^installed$' \
@@ -229,11 +254,7 @@ EOF
 			apt-get -y --purge autoremove"
 	fi
 
-	# remove crufty files that shouldn't be left in an image
-	rm -f chroot/var/cache/debconf/*-old chroot/var/lib/dpkg/*-old
-	Chroot chroot apt clean
-
-	if [ -f config/oem-config-preinstalled ]; then
+	if [ -f config/universe-enabled ]; then
 
 # This is cargo-culted almost verbatim (with some syntax changes for
 # preinstalled being slightly different in what it doesn't ask) from
@@ -392,7 +413,7 @@ EOF
 	# default. Installing NM on an existing system only manages wifi and wwan via
 	# /usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf. When setting
 	# the global backend to NM, netplan overrides that file.
-	if [ -d chroot/usr/lib/NetworkManager ]; then
+	if [ -e chroot/usr/sbin/NetworkManager ]; then
 		echo "===== Enabling all devices in NetworkManager ===="
 		mkdir -p chroot/etc/netplan
 		cat <<EOF > chroot/etc/netplan/01-network-manager-all.yaml
@@ -410,6 +431,23 @@ EOF
 	(cd chroot && find usr/share/doc -maxdepth 1 -type d | xargs du -s | sort -nr)
 	echo END docdirs
 
+	/usr/share/livecd-rootfs/minimize-manual chroot
+
+	# remove crufty files that shouldn't be left in an image
+	rm -f chroot/var/cache/debconf/*-old chroot/var/lib/dpkg/*-old
+	Chroot chroot apt clean
+	# For the docker images we remove even more stuff.
+	if [ "${PROJECT}:${SUBPROJECT:-}" = "ubuntu-base:minimized" ]; then
+		# Remove apt lists (that are currently removed downstream
+		# anyway)
+		rm -rf chroot/var/lib/apt/lists/*
+		# Having device nodes in the docker image can cause problems
+		# (https://github.com/tianon/docker-brew-ubuntu-core/issues/62)
+		# so remove them.  We only do this for docker out of an
+		# abundance of caution.
+		rm -rf chroot/dev/*
+	fi
+
 	lb binary "$@"
 	touch binary.success
 ) 2>&1 | tee binary.log
@@ -455,7 +493,7 @@ for ISO in binary.iso binary.hybrid.iso; do
 done
 
 if [ -e "binary/$INITFS/filesystem.dir" ]; then
-	(cd "binary/$INITFS/filesystem.dir/" && tar -c *) | \
+	(cd "binary/$INITFS/filesystem.dir/" && tar -c --xattrs *) | \
 		gzip -9 --rsyncable > "$PREFIX.rootfs.tar.gz"
 	chmod 644 "$PREFIX.rootfs.tar.gz"
 elif [ -e binary-tar.tar.gz ]; then
@@ -492,13 +530,14 @@ EOF
 fi
 
 if [ "$PROJECT" = "ubuntu-touch" ] || [ "$PROJECT" = "ubuntu-touch-custom" ]; then
-	(cd "binary/$INITFS/custom.dir/" && tar -c *) | \
+	(cd "binary/$INITFS/custom.dir/" && tar -c --xattrs *) | \
 		gzip -9 --rsyncable > "$PREFIX.custom.tar.gz"
 	chmod 644 "$PREFIX.custom.tar.gz"
 fi
 
 # '--initramfs none' produces different manifest names.
 if [ -e "binary/$INITFS/filesystem.packages" ]; then
+	./config/snap-seed-parse "chroot/" "binary/${INITFS}/filesystem.packages"
 	ln "binary/$INITFS/filesystem.packages" "$PREFIX.manifest"
 	chmod 644 "$PREFIX.manifest"
 fi
@@ -763,6 +802,8 @@ for FLAVOUR in $LB_LINUX_FLAVOURS; do
 	if [ -z "$LB_LINUX_FLAVOURS" ] || [ "$LB_LINUX_FLAVOURS" = "none" ]; then
 		continue
 	fi
+	# hwe-* kernels don't use the hwe suffix on the filesystem:
+	FLAVOUR=${FLAVOUR%%-hwe-*}
 	if [ "$FLAVOUR" = "virtual" ]; then
 		# The virtual kernel is named generic in /boot
 		FLAVOUR="generic"
@@ -796,50 +837,67 @@ done
 NUMFLAVOURS="$(set -- $LB_LINUX_FLAVOURS; echo $#)"
 if [ "$NUMFLAVOURS" = 1 ] && [ "$LB_LINUX_FLAVOURS" != "none" ]; then
 	# only one kernel flavour
-	if [ -e "binary/$INITFS/vmlinuz" ]; then
+	FLAVOUR=${LB_LINUX_FLAVOURS%%-hwe-*}
+	if [ -f "binary/$INITFS/vmlinuz" ] && ! [ -h "binary/$INITFS/vmlinuz" ]; then
 		ln "binary/$INITFS/vmlinuz" "$PREFIX.kernel"
 		chmod 644 "$PREFIX.kernel"
 	else
-		ln -sf "$PREFIX.kernel-$LB_LINUX_FLAVOURS" "$PREFIX.kernel"
+		ln -sf "$PREFIX.kernel-$FLAVOUR" "$PREFIX.kernel"
 	fi
-	if [ -e "binary/$INITFS/initrd.lz" ]; then
+	if [ -f "binary/$INITFS/initrd.lz" ] && ! [ -h "binary/$INITFS/initrd.lz" ]; then
 		ln "binary/$INITFS/initrd.lz" "$PREFIX.initrd"
 		chmod 644 "$PREFIX.initrd"
 	else
-		ln -sf "$PREFIX.initrd-$LB_LINUX_FLAVOURS" "$PREFIX.initrd"
+		ln -sf "$PREFIX.initrd-$FLAVOUR" "$PREFIX.initrd"
 	fi
 fi
 
-if [ "$SUBARCH" = "ac100" ]; then
-	# create the md5sum and size files for which we are actually doing all this
-	md5sum $PREFIX.rootfs.tar.gz >chroot/installer.md5
-	wc -c $PREFIX.rootfs.tar.gz >chroot/installer.size
+case $SUBARCH in
+	ac100)
+		# create the md5sum and size files for which we are actually doing all this
+		md5sum $PREFIX.rootfs.tar.gz >chroot/installer.md5
+		wc -c $PREFIX.rootfs.tar.gz >chroot/installer.size
+	
+		INFO_DESC="$(lsb_release -d -s)"
+		INFO_STAMP=$(date +20%y%m%d-%H:%M)
+	
+		echo "$INFO_DESC - $ARCH ($INFO_STAMP)" >chroot/media-info
+	
+		# make sure update-initramfs feels cosy and warm in the environment
+		lb chroot_proc install "$@"
+		lb chroot_sysfs install "$@"
+		lb chroot_devpts install "$@"
+	
+		# re-create initrd to contain the installer.md5 file
+		Chroot chroot "env FLASH_KERNEL_SKIP=1 update-initramfs -k all -t -u -v"
+	
+		# create boot.img
+		Chroot chroot "abootimg --create /boot/installer-${KVERS}.img -f /boot/bootimg.cfg-$SUBARCH -r /boot/initrd.img-${KVERS} -k /boot/vmlinuz-${KVERS}"
+	
+		# clean up
+		lb chroot_devpts remove "$@"
+		lb chroot_sysfs remove "$@"
+		lb chroot_proc remove "$@"
+	
+		cp "chroot/boot/installer-${KVERS}.img" "$PREFIX.bootimg-$FLAVOUR"
+		ln -sf "$PREFIX.bootimg-$FLAVOUR" "$PREFIX.bootimg"
+		;;
 
-	INFO_DESC="$(lsb_release -d -s)"
-	INFO_STAMP=$(date +20%y%m%d-%H:%M)
+	raspi2|raspi3)
+		# copy the kernel and initrd to a predictable directory for
+		# ubuntu-image consumption.  In some cases, like in pi2/3
+		# u-boot, the bootloader needs to contain the kernel and initrd,
+		# so during rootfs build we copy it over to a directory that
+		# ubuntu-image looks for and shoves into the bootloader
+		# partition.
+		UBOOT_BOOT="image/boot/uboot"
 
-	echo "$INFO_DESC - $ARCH ($INFO_STAMP)" >chroot/media-info
+		mkdir -p $UBOOT_BOOT
 
-	# make sure update-initramfs feels cosy and warm in the environment
-	lb chroot_proc install "$@"
-	lb chroot_sysfs install "$@"
-	lb chroot_devpts install "$@"
-
-	# re-create initrd to contain the installer.md5 file
-	Chroot chroot "env FLASH_KERNEL_SKIP=1 update-initramfs -k all -t -u -v"
-
-	# create boot.img
-	Chroot chroot "abootimg --create /boot/installer-${KVERS}.img -f /boot/bootimg.cfg-$SUBARCH -r /boot/initrd.img-${KVERS} -k /boot/vmlinuz-${KVERS}"
-
-	# clean up
-	lb chroot_devpts remove "$@"
-	lb chroot_sysfs remove "$@"
-	lb chroot_proc remove "$@"
-
-	cp "chroot/boot/installer-${KVERS}.img" "$PREFIX.bootimg-$FLAVOUR"
-	ln -sf "$PREFIX.bootimg-$FLAVOUR" "$PREFIX.bootimg"
-
-fi
+		cp $PREFIX.initrd $UBOOT_BOOT/initrd.img || true
+		cp $PREFIX.kernel $UBOOT_BOOT/vmlinuz || true
+		;;
+esac
 
 if [ "$PROJECT" = "ubuntu-touch" ] || [ "$PROJECT" = "ubuntu-touch-custom" ]; then
 	sourceslist="chroot/etc/apt/sources.list"

live-build/auto/config

@@ -4,6 +4,8 @@ set -e
 rm -rf config
 
 echo "Building on $(hostname --fqdn)"
+SUITENUM=$(distro-info --series="$SUITE" -r | awk '{ print $1 }')
+HWE_SUFFIX="hwe-${SUITENUM}"
 
 SEEDMIRROR=http://people.canonical.com/~ubuntu-archive/seeds/
 if [ -z "$MIRROR" ]; then
@@ -33,6 +35,7 @@ fi
 
 mkdir -p config
 cp -af /usr/share/livecd-rootfs/live-build/functions config/functions
+cp -af /usr/share/livecd-rootfs/live-build/snap-seed-parse.py config/snap-seed-parse
 
 mkdir -p config/package-lists
 
@@ -59,9 +62,19 @@ add_task ()
 	# failure.
 
 	for task; do
+		if [ "$task" = "ubuntu-budgie-desktop" ]; then
+			filter="| grep -v '^linux.*tools'"
+		else
+			filter=""
+		fi
+		if [ -n "$HWE" ]; then
+			# If HWE is set, we strip out drivers not available in the
+			# HWE stack, then sed the rest with their HWE suffixes:
+			filter="$filter | sed -e 's/xserver-xorg.*/&-${HWE_SUFFIX}/'"
+		fi
 		# We need a ridiculous number of backslashes to protect
 		# parentheses from eval.
-		echo "!chroot chroot apt-cache dumpavail | grep-dctrl -nsPackage \\\\\\( -XFArchitecture $ARCH -o -XFArchitecture all \\\\\\) -a -wFTask $task" >> "config/package-lists/livecd-rootfs.list.chroot_$pass"
+		echo "!chroot chroot apt-cache dumpavail | grep-dctrl -nsPackage \\\\\\( -XFArchitecture $ARCH -o -XFArchitecture all \\\\\\) -a -wFTask $task $filter" >> "config/package-lists/livecd-rootfs.list.chroot_$pass"
 	done
 }
 
@@ -108,7 +121,12 @@ add_binary_hook ()
 if [ -z "${IMAGEFORMAT:-}" ]; then
 	case $PROJECT:${SUBPROJECT:-} in
 		ubuntu-cpc:*)
-			IMAGEFORMAT=ext4
+			if [ "$SUBARCH" = "raspi3" ]; then
+				# For now only raspi3, but others are soon to follow
+				IMAGEFORMAT=ubuntu-image
+			else
+				IMAGEFORMAT=ext4
+			fi
 			;;
 		ubuntu-server:live)
 			IMAGEFORMAT=plain
@@ -144,19 +162,15 @@ case $IMAGEFORMAT in
 		;;
 
 	plain)
-		INITRAMFS_TYPE=none
 		case $PROJECT:${SUBPROJECT:-} in
 			ubuntu-server:live)
-				# Stop lb installing casper into filesystem.squashfs
-				#  by skipping lb_chroot_live-packages.
-				skip_lb_stage chroot_live-packages
-				INITRAMFS_TYPE=auto
+	                        touch config/universe-enabled
 				;;
 			*)
 				PREINSTALLED=true
 				;;
 		esac
-		OPTS="${OPTS:+$OPTS }--initramfs $INITRAMFS_TYPE --chroot-filesystem $IMAGEFORMAT"
+		OPTS="${OPTS:+$OPTS }--initramfs none --chroot-filesystem $IMAGEFORMAT"
 		;;
 
 	ubuntu-image)
@@ -169,7 +183,7 @@ case $IMAGEFORMAT in
 				MODEL=dragonboard ;;
 			armhf+raspi2)
 				MODEL=pi2 ;;
-			armhf+raspi3)
+			armhf+raspi3|arm64+raspi3)
 				MODEL=pi3 ;;
 			armhf+cm3)
 				MODEL=cm3 ;;
@@ -178,20 +192,39 @@ case $IMAGEFORMAT in
 				exit 1
 				;;
 		esac
-		case $MODEL in
-			pc-amd64|pc-i386)
-				UBUNTU_IMAGE_ARGS="--image-size 3700M" ;;
-			*)
-				UBUNTU_IMAGE_ARGS="" ;;
-		esac
 
-		echo "IMAGEFORMAT=$IMAGEFORMAT" >> config/common
-		echo "UBUNTU_IMAGE_ARGS=\"$UBUNTU_IMAGE_ARGS\"" >> config/common
-		# Store model assertion in top dir to get it picked up later as a build artifact
-		env SNAPPY_STORE_NO_CDN=1 snap known --remote model series=16 model="$MODEL" brand-id=canonical > "$PREFIX".model-assertion
-		echo "Configured ubuntu-image for the following model assertion:"
-		cat "$PREFIX".model-assertion
-		echo "----------------------------------------------------------"
+		if [ $PROJECT = "ubuntu-core" ]; then
+			# snap-based core images
+
+			case $MODEL in
+				pc-amd64|pc-i386)
+					UBUNTU_IMAGE_ARGS="--image-size 3700M" ;;
+				*)
+					UBUNTU_IMAGE_ARGS="" ;;
+			esac
+	
+			echo "IMAGEFORMAT=$IMAGEFORMAT" >> config/common
+			echo "UBUNTU_IMAGE_ARGS=\"$UBUNTU_IMAGE_ARGS\"" >> config/common
+			# Store model assertion in top dir to get it picked up later as a build artifact
+			env SNAPPY_STORE_NO_CDN=1 snap known --remote model series=16 model="$MODEL" brand-id=canonical > "$PREFIX".model-assertion
+			echo "Configured ubuntu-image for the following model assertion:"
+			cat "$PREFIX".model-assertion
+			echo "----------------------------------------------------------"
+		else
+			# classic images
+
+			UBUNTU_IMAGE_ARGS=""
+			UBUNTU_IMAGE_ARGS="$UBUNTU_IMAGE_ARGS${PROPOSED:+ --with-proposed}"
+			UBUNTU_IMAGE_ARGS="$UBUNTU_IMAGE_ARGS${EXTRA_PPAS:+ --extra-ppas \"$EXTRA_PPAS\"}"
+
+			git clone git://git.launchpad.net/~canonical-foundations/snap-$MODEL/+git/github-mirror -b classic config/$PREFIX-gadget
+
+			echo "IMAGEFORMAT=$IMAGEFORMAT" >> config/common
+			echo "SUITE=$SUITE" >> config/common
+			echo "UBUNTU_IMAGE_ARGS=\"$UBUNTU_IMAGE_ARGS\"" >> config/common
+
+			echo "Configured ubuntu-image for the following gadget model: $MODEL"
+		fi
 		# Fake finished configuration for lb build
 		mkdir -p .build
 		touch .build/config
@@ -213,9 +246,9 @@ case $IMAGEFORMAT in
 esac
 
 if [ "$PREINSTALLED" = "true" ]; then
-	# This is an oem-config preinstalled image, touch a random file that
-	# we can refer back to during build, cause that's wildly hackish
-	touch config/oem-config-preinstalled
+	# Touch a random file that we can refer back to during build,
+	# cause that's wildly hackish
+	touch config/universe-enabled
 	case $PROJECT in
 		kubuntu*)
 			add_package live oem-config-kde ubiquity-frontend-kde
@@ -254,8 +287,6 @@ case $BINARYFORMAT in
 		;;
 esac
 
-SIGNED_KERNEL_PACKAGE="linux-signed-generic"
-
 if [ "${SUBPROJECT:-}" = minimized ]; then
 	OPTS="${OPTS:+$OPTS }--bootstrap-flavour=minimal --linux-packages=linux-image"
 fi
@@ -330,10 +361,12 @@ esac
 
 case $PROJECT in
 	ubuntu|ubuntu-dvd)
+		HWE="yes"
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard ubuntu-desktop
 		LIVE_TASK='ubuntu-live'
 		case $ARCH in
-			amd64)	add_package live $SIGNED_KERNEL_PACKAGE ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		;;
 
@@ -341,9 +374,9 @@ case $PROJECT in
 		add_task install minimal standard ubuntu-desktop-next ubuntu-sdk-libs
 		COMPONENTS='main restricted universe'
 		LIVE_TASK='ubuntu-touch-live'
-		KERNEL_FLAVOURS=generic
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 
 		# system image snappy desktop next image
@@ -357,11 +390,13 @@ case $PROJECT in
 		;;
 
 	kubuntu|kubuntu-dvd)
+		HWE="yes"
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard
 		add_task install kubuntu-desktop
 		LIVE_TASK='kubuntu-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe'
 		add_chroot_hook remove-gnome-icon-cache
@@ -387,25 +422,25 @@ case $PROJECT in
 		;;
 
 	edubuntu|edubuntu-dvd)
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard ubuntu-desktop edubuntu-desktop-gnome
 		LIVE_TASK='edubuntu-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe'
 		;;
 
 	xubuntu)
+		HWE="yes"
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard xubuntu-desktop
 		add_package install xterm
 		LIVE_TASK='xubuntu-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe multiverse'
-		case $ARCH in
-			amd64|i386)	KERNEL_FLAVOURS=generic ;;
-		esac
 		;;
 
 	ubuntu-netbook)
@@ -414,24 +449,29 @@ case $PROJECT in
 		;;
 
 	mythbuntu)
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard mythbuntu-desktop
 		LIVE_TASK='mythbuntu-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe multiverse'
 		;;
 
 	lubuntu)
-		add_task install minimal standard lubuntu-desktop
+		HWE="yes"
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
+		add_task install minimal standard lubuntu-gtk-core lubuntu-desktop
+		# Installing the lubuntu-gtk-core task explicitly appears
+		# to make dbus-user-session not get installed anymore:
+		add_package install dbus-user-session
+		# These themes also get dropped, maybe worth looking closer:
+		add_package install humanity-icon-theme gnome-icon-theme adwaita-icon-theme
 		LIVE_TASK='lubuntu-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe multiverse'
-		case $ARCH in
-			amd64|i386)	KERNEL_FLAVOURS=generic ;;
-		esac
 
 		# The Lubuntu STRUCTURE file has "feature
 		# no-follow-recommends".  Mirror this.
@@ -439,15 +479,13 @@ case $PROJECT in
 		;;
 
 	lubuntu-next)
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard lubuntu-qt-desktop
 		LIVE_TASK='lubuntu-live-qt'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe multiverse'
-		case $ARCH in
-			amd64|i386)	KERNEL_FLAVOURS=generic ;;
-		esac
 
 		# The Lubuntu STRUCTURE file has "feature
 		# no-follow-recommends". Mirror this.
@@ -455,46 +493,51 @@ case $PROJECT in
 		;;
 
 	ubuntu-gnome)
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard ubuntu-gnome-desktop
 		LIVE_TASK='ubuntu-gnome-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe'
 		;;
 
 	ubuntu-budgie)
+		HWE="yes"
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard ubuntu-budgie-desktop
 		LIVE_TASK='ubuntu-budgie-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe'
 		;;
 
 	ubuntu-mate)
+		HWE="yes"
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard ubuntu-mate-core ubuntu-mate-desktop
 		LIVE_TASK='ubuntu-mate-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe multiverse'
 		;;
 
 	ubuntustudio-dvd)
+		KERNEL_FLAVOURS="lowlatency${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard ubuntustudio-desktop ubuntustudio-audio ubuntustudio-fonts ubuntustudio-graphics ubuntustudio-video ubuntustudio-publishing ubuntustudio-photography
 		COMPONENTS='main restricted universe multiverse'
-		case $ARCH in
-			amd64|i386)	KERNEL_FLAVOURS=lowlatency ;;
-		esac
 		;;
 
 	ubuntukylin)
+		HWE="yes"
+		KERNEL_FLAVOURS="generic${HWE:+-$HWE_SUFFIX}"
 		add_task install minimal standard ubuntukylin-desktop
 		add_package install ubuntukylin-default-settings
 		LIVE_TASK='ubuntukylin-live'
 		case $ARCH in
-			amd64)	add_package live linux-signed-generic ;;
+			amd64)	add_package live linux-signed-$KERNEL_FLAVOURS ;;
 		esac
 		COMPONENTS='main restricted universe'
 		;;
@@ -509,10 +552,7 @@ case $PROJECT in
 			live)
 				add_task install standard
 				add_task install server
-				LIVE_TASK='cloud-image'
-				case $ARCH in
-					amd64)	add_package live linux-signed-generic ;;
-				esac
+				add_package install cloud-init
 				;;
 		esac
 		COMPONENTS='main'
@@ -668,6 +708,37 @@ case $PROJECT in
 		;;
 esac
 
+case $SUBPROJECT in
+	buildd)
+		OPTS="${OPTS:+$OPTS }--archive-areas main"
+		COMPONENTS='main restricted universe multiverse'
+		OPTS="${OPTS:+$OPTS }--apt-recommends false"
+		OPTS="${OPTS:+$OPTS }--apt-secure false"
+		OPTS="${OPTS:+$OPTS }--parent-mirror-binary ${MIRROR}"
+		# XXX cjwatson 2018-04-27: We need to work out how to make
+		# this conditional so that we can do things like building
+		# buildd chroots with -updates.  This probably involves
+		# either extending the PROPOSED hack or fixing the strange
+		# way that SUITE is in fact a series; in either case it's
+		# likely to involve work both here and in launchpad-buildd.
+		OPTS="${OPTS:+$OPTS }--security false --volatile false"
+
+		add_package install adduser
+		add_package install policyrcd-script-zg2
+		add_package install pkgbinarymangler
+		add_package install ca-certificates
+		add_package install gpg
+		add_package install gpg-agent
+		add_package install tzdata
+		add_package install fakeroot
+		add_package install build-essential
+		# Needed for LXD-based builds.
+		add_package install init
+
+		cp -af /usr/share/livecd-rootfs/live-build/make-lxd-metadata.py config/make-lxd-metadata
+		;;
+esac
+
 # we'll expand the base seed given here according to the STRUCTURE file, and
 # then look in all of the seeds found to see which snaps are seeded
 case $PROJECT:${SUBPROJECT:-} in
@@ -700,7 +771,12 @@ if [ -n "${BASE_SEED}" ]; then
 		seed_snaps=$(sed -rn '1,/-----/d;/-----/,$d; s/(.*) \|.*/\1/; s, \(classic\),/classic,; p' "${file}")
 		for snap in ${seed_snaps}; do
 			echo "snap: found ${snap}"
-			ALL_SNAPS="${ALL_SNAPS:+${ALL_SNAPS} }${snap}"
+			# Reorder gtk-common-themes first due to LP: #1772844
+			if [ "${snap}" = "gtk-common-themes" ]; then
+				ALL_SNAPS="${snap}${ALL_SNAPS:+ ${ALL_SNAPS}}"
+			else
+				ALL_SNAPS="${ALL_SNAPS:+${ALL_SNAPS} }${snap}"
+			fi
 		done
 	done
 	if [ -n "${ALL_SNAPS}" ] || [ -n "${HOOK_SNAPS}" ]; then
@@ -740,7 +816,7 @@ case $PROJECT in
 esac
 
 case $ARCH in
-	armel|armhf)
+	armel|armhf|arm64)
 		KERNEL_FLAVOURS="${SUBARCH:-$KERNEL_FLAVOURS}"
 		case $SUBARCH in
 			dove)
@@ -775,16 +851,21 @@ case $ARCH in
 				;;
 			raspi2)
 				COMPONENTS='main restricted universe multiverse'
-				add_package install linux-firmware-raspi2 u-boot-rpi flash-kernel u-boot-tools
+				add_package install linux-firmware-raspi2 u-boot-rpi flash-kernel u-boot-tools wpasupplicant
 				BINARY_REMOVE_LINUX=false
 				;;
+			raspi3)
+				COMPONENTS='main restricted universe multiverse'
+				KERNEL_FLAVOURS=raspi2
+				add_package install linux-firmware-raspi2 u-boot-rpi flash-kernel u-boot-tools wpasupplicant
+				BINARY_REMOVE_LINUX=false
+				;;
+
 		esac
 		;;
 esac
 
 case $PROJECT:${SUBPROJECT:-} in
-	ubuntu-server:live)
-		;;
 	ubuntu-server:*|ubuntu-base:*|ubuntu-touch:*|ubuntu-touch-custom:*)
 		OPTS="${OPTS:+$OPTS }--linux-packages=none --initramfs=none"
 		KERNEL_FLAVOURS=none
@@ -828,6 +909,7 @@ lb config noauto \
 	--initsystem none \
 	--bootloader "$BOOTLOADER" \
 	${INITRAMFS_COMPRESSION:+--initramfs-compression "$INITRAMFS_COMPRESSION"} \
+	--checksums none \
 	--cache false \
 	${BOOTAPPEND_LIVE:+--bootappend-live "$BOOTAPPEND_LIVE"} \
 	$OPTS \
@@ -842,7 +924,7 @@ echo "SUBPROJECT=\"${SUBPROJECT:-}\"" >> config/binary
 echo "LB_DISTRIBUTION=\"$SUITE\"" >> config/binary
 
 case $ARCH+$SUBARCH in
-	armhf+raspi2)
+	armhf+raspi2|armhf+raspi3|arm64+raspi3)
 		cat > config/hooks/01-firmware-directory.chroot_early <<EOF
 #!/bin/sh -ex
 mkdir -p /boot/firmware
@@ -977,6 +1059,12 @@ EOF
 		;;
 esac
 
+case $SUBPROJECT in
+	buildd)
+		cp -af /usr/share/livecd-rootfs/live-build/buildd/* config/
+		;;
+esac
+
 if [ "$EXTRA_PPAS" ]; then
 	rm -f config/archives/extra-ppas.list.chroot \
 	      config/archives/extra-ppas.pref.chroot \

live-build/buildd/hooks/00-kernel-img.chroot

@@ -0,0 +1,5 @@
+#! /bin/sh
+set -e
+
+# At one point, kernel builds needed this.
+echo do_initrd = Yes >>/etc/kernel-img.conf

live-build/buildd/hooks/00-mirror.binary

@@ -0,0 +1,12 @@
+#! /bin/sh
+set -e
+
+. config/bootstrap
+
+# Use a public-facing mirror URL, for the benefit of
+# sbuild-launchpad-chroot.  We deliberately do this only after live-build
+# has run "apt-get update" for the last time, in order that
+# /var/lib/apt/lists/ has suitable cached Packages files; this speeds up
+# builds on buildds.
+sed -i "s,${LB_PARENT_MIRROR_BINARY},${LB_MIRROR_BINARY},g" \
+	binary/etc/apt/sources.list

live-build/buildd/hooks/01-pkgbinarymangler.chroot

@@ -0,0 +1,10 @@
+#! /bin/sh
+set -e
+
+# Configure pkgbinarymangler.
+sed -i /^enable/s/false/true/ \
+	/etc/pkgbinarymangler/maintainermangler.conf \
+	/etc/pkgbinarymangler/striptranslations.conf || true
+sed -i /^invalid_current/s/ignore/fail/ \
+	/etc/pkgbinarymangler/maintainermangler.conf \
+	/etc/pkgbinarymangler/striptranslations.conf || true

live-build/buildd/hooks/01-policy-rc-d.binary

@@ -0,0 +1,13 @@
+#! /bin/sh
+set -e
+
+# Put the /usr/sbin/policy-rc.d alternatives symlink in place.  Ordinarily
+# update-alternatives ought to create this when policyrcd-script-zg2 is
+# installed, but that doesn't work because live-build has already installed
+# a dummy one at that point.  The simplest approach is to repair the
+# situation by putting it in place here.
+if [ -L binary/etc/alternatives/policy-rc.d ] && \
+   [ ! -e binary/usr/sbin/policy-rc.d ] && \
+   [ ! -L binary/usr/sbin/policy-rc.d ]; then
+	ln -s /etc/alternatives/policy-rc.d binary/usr/sbin/policy-rc.d
+fi

live-build/buildd/hooks/02-user.chroot

@@ -0,0 +1,9 @@
+#! /bin/sh
+set -e
+
+# Create the buildd user and group.
+addgroup --gid 2501 buildd
+adduser --system --disabled-password --gecos 'Build Daemon user' \
+	--ingroup buildd --uid 2001 --shell /bin/bash buildd
+mkdir -p /build/buildd
+chown buildd:buildd /build/buildd

live-build/buildd/hooks/50-buildd-tar.binary

@@ -0,0 +1,10 @@
+#! /bin/sh
+# A few things (launchpad-buildd, sbuild-launchpad-chroot) rely on the
+# top-level directory being "chroot-autobuild", so we have to do this
+# ourselves.
+set -e
+
+# gzip was chosen for fastest decompression speed: it decompresses buildd
+# chroots about twice as fast as xz and about five times as fast as bzip2.
+tar --transform='s,^binary,chroot-autobuild,' --sort=name --numeric-owner \
+	-czf "livecd.$PROJECT.rootfs.tar.gz" binary

live-build/buildd/hooks/51-buildd-lxd.binary

@@ -0,0 +1,16 @@
+#! /bin/sh
+# Some build types prefer a LXD image over a traditional chroot tarball.
+set -e
+
+. config/bootstrap
+
+TMPDIR="$(mktemp -d)"
+config/make-lxd-metadata "${LB_DISTRIBUTION%-*}" "$ARCH" \
+	>"$TMPDIR/metadata.yaml"
+tar --numeric-owner -cf "livecd.$PROJECT.lxd.tar" -C "$TMPDIR" metadata.yaml
+rm -rf "$TMPDIR"
+# When using the combined metadata/rootfs form, the rootfs must be under
+# rootfs/ rather than under chroot-autobuild/.
+tar --transform='s,^binary,rootfs,' --sort=name --numeric-owner \
+	-rf "livecd.$PROJECT.lxd.tar" binary
+gzip -9 "livecd.$PROJECT.lxd.tar"

live-build/buildd/includes.chroot/etc/apt/apt.conf.d/99buildd

@@ -0,0 +1,2 @@
+DPkg::Options {"--force-unsafe-io";};
+DPkg::Use-Pty "false";

live-build/buildd/includes.chroot/etc/apt/preferences.d/backports

@@ -0,0 +1,3 @@
+Package: *
+Pin: release a=*-backports
+Pin-Priority: 500

live-build/buildd/includes.chroot/etc/fstab

@@ -0,0 +1 @@
+/dev/root / ext2 noatime,errors=remount-ro 0 1

live-build/buildd/includes.chroot/etc/hostname

@@ -0,0 +1 @@
+INVALID

live-build/buildd/includes.chroot/etc/hosts

@@ -0,0 +1,9 @@
+127.0.0.1	localhost.localdomain	localhost
+
+# The following lines are desirable for IPv6 capable hosts
+::1     ip6-localhost ip6-loopback
+fe00::0 ip6-localnet
+ff00::0 ip6-mcastprefix
+ff02::1 ip6-allnodes
+ff02::2 ip6-allrouters
+ff02::3 ip6-allhosts

live-build/buildd/includes.chroot/usr/local/sbin/policy-rc.d

@@ -0,0 +1,13 @@
+#!/bin/sh
+
+# policy-rc.d script for chroots.
+# Copyright (c) 2007 Peter Palfrader <peter@palfrader.org>
+# License: <weasel> MIT, if you want one.
+
+while true; do
+  case "$1" in
+    -*)         shift ;;
+    makedev)    exit 0;;
+    *) echo "Not running services in chroot."; exit 101 ;;
+  esac
+done

live-build/buildd/preseed/debconf.preseed

@@ -0,0 +1,2 @@
+# We never want debconf interaction.
+debconf debconf/frontend select Noninteractive

live-build/buildd/preseed/man-db.preseed

@@ -0,0 +1,3 @@
+# Avoid unnecessary manual page database builds (see
+# https://bugs.debian.org/554914).
+man-db man-db/auto-update boolean false

live-build/buildd/preseed/sun-java6.preseed

@@ -0,0 +1,3 @@
+# Pre-accept interactive EULA prompts.
+sun-java6-bin shared/accepted-sun-dlj-v1-1 boolean true
+sun-java6-jre shared/accepted-sun-dlj-v1-1 boolean true

live-build/functions

@@ -43,6 +43,17 @@ create_empty_disk_image() {
     dd if=/dev/zero of="$1" bs=1 count=0 seek="${imagesize}"
 }
 
+create_manifest() {
+    local chroot_root=${1}
+    local target_file=${2}
+    echo "create_manifest chroot_root: ${chroot_root}"
+    dpkg-query --show --admindir="${chroot_root}/var/lib/dpkg" > ${target_file}
+    echo "create_manifest call to dpkg-query finished."
+    ./config/snap-seed-parse "${chroot_root}" "${target_file}"
+    echo "create_manifest call to snap_seed_parse finished."
+    echo "create_manifest finished"
+}
+
 make_ext4_partition() {
     device="$1"
     label=${fs_label:+-L "${fs_label}"}

live-build/make-lxd-metadata.py

@@ -0,0 +1,49 @@
+#! /usr/bin/python3
+
+"""Make a metadata.yaml file for a LXD image."""
+
+import argparse
+import json
+import sys
+import time
+
+
+# Map dpkg architecture names to LXD architecture names.
+lxd_arches = {
+    "amd64": "x86_64",
+    "arm64": "aarch64",
+    "armhf": "armv7l",
+    "i386": "i686",
+    "powerpc": "ppc",
+    "ppc64el": "ppc64le",
+    "s390x": "s390x",
+    }
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("series", help="Ubuntu series name")
+    parser.add_argument("architecture", help="Ubuntu architecture name")
+    args = parser.parse_args()
+
+    metadata = {
+        "architecture": lxd_arches[args.architecture],
+        "creation_date": int(time.time()),
+        "properties": {
+            "os": "Ubuntu",
+            "series": args.series,
+            "architecture": args.architecture,
+            "description": "Ubuntu buildd %s %s" % (
+                args.series, args.architecture),
+            },
+        }
+
+    # Encoding this as JSON is good enough, and saves pulling in a YAML
+    # library dependency.
+    json.dump(
+        metadata, sys.stdout, sort_keys=True, indent=4, separators=(",", ": "),
+        ensure_ascii=False)
+
+
+if __name__ == "__main__":
+    main()

live-build/snap-seed-parse.py

@@ -0,0 +1,68 @@
+#!/usr/bin/python3
+ 
+"""
+Usage: snap-seed-parse [${chroot_dir}] <output file>
+ 
+This script looks for a seed.yaml path in the given root directory, parsing
+it and appending the parsed lines to the given output file.
+ 
+The $chroot_dir argument is optional and will default to the empty string.
+"""
+ 
+import argparse
+import os.path
+import re
+import yaml
+ 
+ 
+def log(msg):
+    print("snap-seed-parse: {}".format(msg))
+ 
+ 
+log("Parsing seed.yaml")
+ 
+parser = argparse.ArgumentParser()
+parser.add_argument('chroot', nargs='?', default='',
+                    help='root dir for the chroot from which to generate the '
+                         'manifest')
+parser.add_argument('file', help='Output manifest to this file')
+ 
+ARGS = parser.parse_args()
+CHROOT_ROOT = ARGS.chroot
+FNAME = ARGS.file
+ 
+# Trim any trailing slashes for correct appending
+log("CHROOT_ROOT: {}".format(CHROOT_ROOT))
+if len(CHROOT_ROOT) > 0 and CHROOT_ROOT[-1] == '/':
+    CHROOT_ROOT = CHROOT_ROOT[:-1]
+ 
+# This is where we expect to find the seed.yaml file
+YAML_PATH = CHROOT_ROOT + '/var/lib/snapd/seed/seed.yaml'
+ 
+# Snaps are prepended with this string in the manifest
+LINE_PREFIX = 'snap:'
+ 
+log("yaml path: {}".format(YAML_PATH))
+if not os.path.isfile(YAML_PATH):
+    log("WARNING: yaml path not found; no seeded snaps found.")
+    exit(0)
+else:
+    log("yaml path found.")
+ 
+with open(YAML_PATH, 'r') as fh:
+    yaml_lines = yaml.safe_load(fh)['snaps']
+ 
+log('Writing manifest to {}'.format(FNAME))
+ 
+with open(FNAME, 'a+') as fh:
+    for item in yaml_lines:
+        filestring = item['file']
+        # Pull the revision number off the file name
+        revision = filestring[filestring.rindex('_')+1:]
+        revision = re.sub(r'[^0-9]', '', revision)
+        fh.write("{}{}\t{}\t{}\n".format(LINE_PREFIX,
+                                         item['name'],
+                                         item['channel'],
+                                         revision,
+                                         ))
+log('Manifest output finished.')

live-build/ubuntu-cpc/hooks/031-0-create-root-dir.binary

@@ -0,0 +1,29 @@
+#!/bin/bash -ex
+# vi: ts=4 expandtab
+#
+# Generate the root directory/manifest for rootfs.tar.xz and squashfs
+
+if [ -n "$SUBARCH" ]; then
+    echo "Skipping rootfs build for subarch flavor build"
+    exit 0
+fi
+
+. config/functions
+
+rootfs_dir=rootfs.dir
+mkdir $rootfs_dir
+cp -a chroot/* $rootfs_dir
+
+setup_mountpoint $rootfs_dir
+
+env DEBIAN_FRONTEND=noninteractive chroot $rootfs_dir apt-get --purge remove --assume-yes '^linux-.*' 'linux-base+'
+env DEBIAN_FRONTEND=noninteractive chroot $rootfs_dir apt-get --purge remove --assume-yes '^grub-.*'
+env DEBIAN_FRONTEND=noninteractive chroot $rootfs_dir apt-get autoremove --purge --assume-yes
+rm -rf $rootfs_dir/boot/grub
+
+# Keep this as some derivatives mount a tempfs here
+mkdir -p $rootfs_dir/lib/modules
+
+teardown_mountpoint $rootfs_dir
+
+create_manifest "${rootfs_dir}" "${rootfs_dir}.manifest"

live-build/ubuntu-cpc/hooks/031-1-root-xz.binary

@@ -0,0 +1,15 @@
+#!/bin/bash -ex
+# vi: ts=4 expandtab
+#
+# Generate the rootfs.tar.xz and manifest
+
+if [ -n "$SUBARCH" ]; then
+    echo "Skipping rootfs build for subarch flavor build"
+    exit 0
+fi
+
+# This is the directory created by 031-0-create-root-dir.binary
+rootfs_dir=rootfs.dir
+
+cp $rootfs_dir.manifest livecd.ubuntu-cpc.rootfs.manifest
+(cd $rootfs_dir/ && tar -c --xattrs *) | xz > livecd.ubuntu-cpc.rootfs.tar.xz

live-build/ubuntu-cpc/hooks/031-2-root-squashfs.binary

@@ -0,0 +1,29 @@
+#!/bin/bash -ex
+# vi: ts=4 noexpandtab
+#
+# Generate a squashfs root and manifest
+
+case $IMAGE_TARGETS in
+	""|*squashfs*)
+		;;
+	*)
+		echo "Skipping squashfs build"
+		exit 0
+		;;
+esac
+
+if [ -n "$SUBARCH" ]; then
+	echo "Skipping rootfs build for subarch flavor build"
+	exit 0
+fi
+
+# This is the directory created by 031-0-create-root-dir.binary
+rootfs_dir=rootfs.dir
+
+squashfs_f="$PWD/livecd.ubuntu-cpc.squashfs"
+
+cp $rootfs_dir.manifest $squashfs_f.manifest
+
+(cd $rootfs_dir &&
+      mksquashfs . $squashfs_f \
+        -no-progress -xattrs -comp xz )

live-build/ubuntu-cpc/hooks/031-3-wsl-gz.binary

@@ -0,0 +1,56 @@
+#!/bin/bash -eux
+# vi: ts=4 expandtab
+#
+# Generate the compressed root directory for WSL
+
+case ${SUBPROJECT:-} in
+    minimized)
+        echo "Skipping minimized $0 build as WSL systems are designed to be interactive"
+        exit 0
+        ;;
+    *)
+        ;;
+esac
+
+case $ARCH in
+    amd64|arm64)
+        ;;
+    *)
+        echo "WSL root tarballs are not generated for $ARCH."
+        exit 0;;
+esac
+
+if [ -n "${SUBARCH:-}" ]; then
+    echo "Skipping rootfs build for subarch flavor build"
+    exit 0
+fi
+
+. config/functions
+
+rootfs_dir=wslroot.dir
+
+# This is the directory created by create-root-dir.binary
+cp -a rootfs.dir $rootfs_dir
+
+setup_mountpoint $rootfs_dir
+
+env DEBIAN_FRONTEND=noninteractive chroot $rootfs_dir apt-get -y -qq install ubuntu-wsl
+
+create_manifest $rootfs_dir livecd.ubuntu-cpc.wsl.rootfs.manifest
+teardown_mountpoint $rootfs_dir
+
+# remove attributes not supported by WSL's tar
+if [ -d $rootfs_dir/var/log/journal ]; then
+    setfattr -x system.posix_acl_access $rootfs_dir/var/log/journal
+    setfattr -x system.posix_acl_default $rootfs_dir/var/log/journal
+fi
+
+# The reason not using just tar .. -C $rootfs_dir . is that using '.' was found
+# not working once and checking if using the simpler command is safe needs
+# verification of the app installation on all Windows 10 builds we support
+# with WSL.
+cd $rootfs_dir
+tar --xattrs --sort=name -czf ../livecd.ubuntu-cpc.wsl.rootfs.tar.gz *
+cd ..
+
+rm -rf $rootfs_dir

live-build/ubuntu-cpc/hooks/032-root-squashfs.binary

@@ -1,49 +0,0 @@
-#!/bin/bash -ex
-# vi: ts=4 noexpandtab
-#
-# Generate a squashfs root and manifest
-
-case $IMAGE_TARGETS in
-	""|*squashfs*)
-		;;
-	*)
-		echo "Skipping squashfs build"
-		exit 0
-		;;
-esac
-
-if [ -n "$SUBARCH" ]; then
-	echo "Skipping rootfs build for subarch flavor build"
-	exit 0
-fi
-
-. config/functions
-
-mkdir binary/boot/squashfs.dir
-cp -a chroot/* binary/boot/squashfs.dir
-
-setup_mountpoint binary/boot/squashfs.dir
-
-chroot binary/boot/squashfs.dir dpkg-divert --local --rename /usr/sbin/grub-probe
-chroot binary/boot/squashfs.dir touch /usr/sbin/grub-probe
-chroot binary/boot/squashfs.dir chmod +x /usr/sbin/grub-probe
-
-env DEBIAN_FRONTEND=noninteractive chroot binary/boot/squashfs.dir apt-get --purge remove --assume-yes '^linux-.*' 'linux-base+'
-env DEBIAN_FRONTEND=noninteractive chroot binary/boot/squashfs.dir apt-get --purge remove --assume-yes '^grub-.*'
-env DEBIAN_FRONTEND=noninteractive chroot binary/boot/squashfs.dir apt-get autoremove --purge --assume-yes
-rm -rf binary/boot/squashfs.dir/boot/grub
-chroot binary/boot/squashfs.dir mkdir -p /lib/modules
-
-chroot binary/boot/squashfs.dir rm /usr/sbin/grub-probe
-chroot binary/boot/squashfs.dir dpkg-divert --remove --local --rename /usr/sbin/grub-probe
-
-teardown_mountpoint binary/boot/squashfs.dir
-
-squashfs_f="${PWD}/livecd.ubuntu-cpc.squashfs"
-squashfs_f_manifest="${squashfs_f}.manifest"
-
-dpkg-query --admindir=binary/boot/squashfs.dir/var/lib/dpkg -W > ${squashfs_f_manifest}
-
-(cd "binary/boot/squashfs.dir/" &&
-      mksquashfs . ${squashfs_f} \
-        -no-progress -xattrs -comp xz )

live-build/ubuntu-cpc/hooks/033-disk-image-uefi.binary

@@ -93,6 +93,12 @@ install_grub() {
             ;;
     esac
 
+    # This call to populate the package manifest is added here to capture
+    # grub-efi packages that otherwise would not make it into the base
+    # manifest. filesystem.packages is moved into place via symlinking to
+    # livecd.ubuntu-cpc.manifest by live-build/auto/build after lb_binary runs
+    create_manifest "mountpoint" "binary/boot/filesystem.packages"
+
     chroot mountpoint grub-install "${loop_device}" \
         --boot-directory=/boot \
         --efi-directory=/boot/efi \

live-build/ubuntu-cpc/hooks/100-purge-grub-legacy-ec2-arm.chroot

@@ -0,0 +1,13 @@
+#!/bin/sh -eux
+
+# Only execute the hack for ARM images
+# ARCH is not available in .chroot hooks so we need to get the architecture
+# manually.
+arch=$(dpkg --print-architecture)
+if [ "$arch" != "armhf" ] && [ "$arch" != "arm64" ]; then
+	exit 0
+fi
+
+export DEBIAN_FRONTEND=noninteractive
+# Why is grub-legacy-ec2 even on the image?
+apt-get remove  --yes --purge grub-legacy-ec2 || true

live-build/ubuntu-cpc/hooks/999-cpc-fixes.chroot

@@ -102,24 +102,6 @@ cat > /etc/fstab << EOM
 LABEL=cloudimg-rootfs	/	 ext4	defaults	0 0
 EOM
 
-## Make sure that the update-motd.d directory exists
-[ ! -e "${rootd}/etc/update-motd.d" ] &&
-   mkdir -p "${rootd}/etc/update-motd.d"
-
-## write a MOTD file advertising support for images
-cat > "${rootd}/etc/update-motd.d/51-cloudguest" << EOF
-#!/bin/sh
-#
-${CLOUD_IMG_STR}
-# This file is not managed by a package.  If you no longer want to
-# see this message you can safely remove the file.
-echo ""
-echo "  Get cloud support with Ubuntu Advantage Cloud Guest:"
-echo "    http://www.ubuntu.com/business/services/cloud"
-EOF
-
-chmod +x "${rootd}/etc/update-motd.d/51-cloudguest"
-
 # for quantal and newer, add /etc/overlayroot.local.conf
 # but do not overwrite anything that somehow got there
 if [ -f "${rootd}/etc/overlayroot.conf" ] &&

live-build/ubuntu-server/hooks/030-root-squashfs.binary

@@ -28,7 +28,7 @@ cp -a chroot/* binary/boot/squashfs.dir
 squashfs_f="${PWD}/livecd.${PROJECT}.squashfs"
 squashfs_f_manifest="${squashfs_f}.manifest"
 
-dpkg-query --admindir=binary/boot/squashfs.dir/var/lib/dpkg -W > ${squashfs_f_manifest}
+create_manifest "binary/boot/squashfs.dir" "${squashfs_f_manifest}"
 
 (cd "binary/boot/squashfs.dir/" &&
       mksquashfs . ${squashfs_f} \

live-build/ubuntu-server/hooks/032-installer-squashfs.binary

@@ -24,13 +24,14 @@ fi
 . config/functions
 . config/common
 
-SQUASH_ROOT=binary/boot/squashfs.dir
+FILESYSTEM_ROOT=binary/boot/squashfs.dir
+INSTALLER_ROOT=binary/boot/installer.squashfs.dir
 OVERLAY_ROOT=binary/overlay
 
-mkdir -p "$OVERLAY_ROOT"
+mkdir -p "$INSTALLER_ROOT" "$OVERLAY_ROOT"
 
 # Create an installer squashfs layer
-mount_overlay "$SQUASH_ROOT/" "$OVERLAY_ROOT/" "$SQUASH_ROOT/"
+mount_overlay "$FILESYSTEM_ROOT/" "$OVERLAY_ROOT/" "$INSTALLER_ROOT/"
 
 setup_mountpoint binary/boot/squashfs.dir
 
@@ -40,42 +41,41 @@ setup_mountpoint binary/boot/squashfs.dir
 # It would be better to have this in ../includes.binary/overlay but
 # you can't have backslashes in filenames in bzr branches!
 DEVICE_UNIT_NAME='dev-disk-by\x2duuid-00c629d6\x2d06ab\x2d4dfd\x2db21e\x2dc3186f34105d.device'
-mkdir -p "$SQUASH_ROOT/etc/systemd/system/$DEVICE_UNIT_NAME.d"
-cat > "$SQUASH_ROOT/etc/systemd/system/$DEVICE_UNIT_NAME.d/override.conf" <<EOF
+mkdir -p "$INSTALLER_ROOT/etc/systemd/system/$DEVICE_UNIT_NAME.d"
+cat > "$INSTALLER_ROOT/etc/systemd/system/$DEVICE_UNIT_NAME.d/override.conf" <<EOF
 [Unit]
 JobRunningTimeoutSec=0s
+Wants=subiquity_config.mount
 EOF
 
 # Prepare installer layer.
 
-# Install any requirements for the installer, for things we don't want
-# to see on the installed system
-chroot $SQUASH_ROOT apt-get update
-chroot $SQUASH_ROOT apt-get -y install user-setup curtin lupin-casper
+# Install casper for live session magic.
+chroot $INSTALLER_ROOT apt-get -y install lupin-casper
+chroot $INSTALLER_ROOT apt-get clean
 
 # For bug #1743643 "Install to dirty disk with swap fails" remove the
 # "helpful" casper script that mounts any swap partitions it finds.
-rm -f $SQUASH_ROOT/usr/share/initramfs-tools/scripts/casper-bottom/*swap
-
-# Installing casper means we need a new initramfs
-UPDATE_INITRAMFS_OPTIONS=CASPER_GENERATE_UUID=1 recreate_initramfs $SQUASH_ROOT
+rm -f $INSTALLER_ROOT/usr/share/initramfs-tools/scripts/casper-bottom/*swap
 
 # Don't let cloud-init run in the live session.
-touch $SQUASH_ROOT/etc/cloud/cloud-init.disabled
+touch $INSTALLER_ROOT/etc/cloud/cloud-init.disabled
 
 # Do the snap seeding dance.
-chroot $SQUASH_ROOT mkdir -p /var/lib/snapd/seed/snaps /var/lib/snapd/seed/assertions
-chroot $SQUASH_ROOT sh -c '
+chroot $INSTALLER_ROOT mkdir -p /var/lib/snapd/seed/snaps /var/lib/snapd/seed/assertions
+chroot $INSTALLER_ROOT sh -c '
 set -x;
-cd /var/lib/snapd/seed;
-sudo SNAPPY_STORE_NO_CDN=1 snap download core;
-sudo SNAPPY_STORE_NO_CDN=1 snap download subiquity;
+mkdir -p /var/lib/snapd/seed/snaps/
+cd /var/lib/snapd/seed/snaps/;
+if [ ! -e core_*.snap ]; then
+	SNAPPY_STORE_NO_CDN=1 snap download core
+fi
+SNAPPY_STORE_NO_CDN=1 snap download subiquity;
 
-CORE_SNAP=$(ls -1 core*.snap);
+CORE_SNAP=$(ls -1 core_*.snap);
 SUBIQUITY_SNAP=$(ls -1 subiquity*.snap);
 
 mv *.assert /var/lib/snapd/seed/assertions/;
-mv *.snap /var/lib/snapd/seed/snaps/;
 
 cat <<EOF > /var/lib/snapd/seed/seed.yaml
 snaps:
@@ -89,10 +89,7 @@ snaps:
 EOF
 '
 
-teardown_mountpoint "$SQUASH_ROOT"
-
-# Then unmount the overlay
-umount "$SQUASH_ROOT"
+teardown_mountpoint "$INSTALLER_ROOT"
 
 squashfs_f="${PWD}/livecd.${PROJECT}.installer.squashfs"
 

live-build/ubuntu-server/hooks/033-kernel-bits.binary

@@ -0,0 +1,90 @@
+#!/bin/bash -eux
+# vi: ts=4 noexpandtab
+#
+# Generate a squashfs root and manifest
+
+echo "033-kernel-bits.binary"
+
+case ${IMAGE_TARGETS-} in
+	""|*squashfs*)
+		;;
+	*)
+		echo "Skipping squashfs build"
+		exit 0
+		;;
+esac
+
+if [ -n "${SUBARCH-}" ]; then
+	echo "Skipping rootfs build for subarch flavor build"
+	exit 0
+fi
+
+. config/functions
+. config/common
+
+INSTALLER_ROOT=binary/boot/installer.squashfs.dir
+
+KERNEL_BITS_ROOT=binary/boot/kernel-bits.dir
+KERNEL_BITS_OVERLAY=binary/boot/overlay-kernel-bits
+
+variants='ga hwe'
+
+for variant in $variants; do
+	if [ "$variant" = "ga" ]; then
+		kernel_metapkg=linux-generic
+		flavor=generic
+	elif [ "$variant" = "hwe" ]; then
+		kernel_metapkg=linux-generic-hwe-$(lsb_release -sr)
+		flavor=generic-hwe
+	else
+		echo "bogus variant: $variant"
+		exit 1
+	fi
+
+	# Make preparations
+	mkdir -p $KERNEL_BITS_ROOT $KERNEL_BITS_OVERLAY
+	mount_overlay "$INSTALLER_ROOT/" "$KERNEL_BITS_OVERLAY/" "$KERNEL_BITS_ROOT/"
+	setup_mountpoint $KERNEL_BITS_ROOT
+
+	# Our initramfs hook implements a kind of extreme version of
+	# cloud-initramfs-copymods, so remove that and prevent duelling hooks
+	env DEBIAN_FRONTEND=noninteractive chroot $KERNEL_BITS_ROOT apt-get -y remove cloud-initramfs-copymods ubuntu-server || true
+
+	# Configure initramfs creation
+	mkdir -p "$KERNEL_BITS_ROOT"/etc/initramfs-tools/conf.d/
+	if [ -n "$LB_INITRAMFS_COMPRESSION" ]; then
+		echo "COMPRESS=$LB_INITRAMFS_COMPRESSION" > "$KERNEL_BITS_ROOT"/etc/initramfs-tools/conf.d/livecd-rootfs.conf
+	fi
+	echo "export CASPER_GENERATE_UUID=1" > "$KERNEL_BITS_ROOT"/etc/initramfs-tools/conf.d/casper.conf
+
+	# Add a hook to record which kernel was booted and mount the
+	# modules.squashfs created below.
+	cat <<EOF > "$KERNEL_BITS_ROOT"/etc/initramfs-tools/scripts/init-bottom/live-server
+#!/bin/sh
+case \$1 in
+prereqs) exit 0;;
+esac
+
+echo ${kernel_metapkg} > /run/kernel-meta-package
+mkdir -p \$rootmnt/lib/modules
+mount \$rootmnt/cdrom/casper/extras/modules.squashfs-$flavor \$rootmnt/lib/modules
+EOF
+	chmod +x "$KERNEL_BITS_ROOT"/etc/initramfs-tools/scripts/init-bottom/live-server
+
+	# Install the kernel!
+	env DEBIAN_FRONTEND=noninteractive chroot $KERNEL_BITS_ROOT apt-get -y install ${kernel_metapkg}
+
+	# Fish out generated kernel image and initrd
+	mv "$KERNEL_BITS_ROOT"/boot/initrd.img-* ${PWD}/livecd.${PROJECT}.initrd-$flavor
+	mv "$KERNEL_BITS_ROOT"/boot/vmlinu?-* ${PWD}/livecd.${PROJECT}.kernel-$flavor
+
+	# Create squashfs containing all the modules
+	modules_squashfs_path="${PWD}/livecd.${PROJECT}.modules.squashfs-$flavor"
+	(cd "$KERNEL_BITS_ROOT/lib/modules" &&
+		mksquashfs . $modules_squashfs_path -no-progress -xattrs -comp xz)
+
+	# And clean up
+	teardown_mountpoint $KERNEL_BITS_ROOT
+	umount $KERNEL_BITS_ROOT
+	rm -rf $KERNEL_BITS_ROOT $KERNEL_BITS_OVERLAY
+done

live-build/ubuntu-server/includes.binary/overlay/etc/netplan/00-installer-config.yaml

@@ -7,9 +7,7 @@ network:
             match:
                 name: "en*"
             dhcp4: true
-            optional: true
         all-eth:
             match:
                 name: "eth*"
             dhcp4: true
-            optional: true

live-build/ubuntu-server/includes.binary/overlay/etc/systemd/journald.conf.d/no-rate-limit.conf

@@ -0,0 +1,2 @@
+[Journal]
+RateLimitIntervalSec=0

live-build/ubuntu-server/includes.binary/overlay/lib/systemd/system/local-fs.target.wants/subiquity_config.mount

@@ -1 +0,0 @@
-../subiquity_config.mount

live-build/ubuntu-server/includes.binary/overlay/lib/systemd/system/serial-subiquity@.service

@@ -7,9 +7,9 @@ ConditionPathExists=!/run/subiquity/complete
 StartLimitInterval=0
 
 [Service]
-Environment=PYTHONPATH=/usr/share/subiquity
+Environment=SNAP_REEXEC=0
 ExecStartPre=/bin/systemctl stop serial-getty@%I
-ExecStart=/sbin/agetty -n --keep-baud -l /snap/bin/subiquity --login-options "--serial" 115200,38400,9600 %I $TERM
+ExecStart=/usr/bin/snap run subiquity.subiquity-service %I
 ExecStopPost=/bin/systemctl start serial-getty@%I
 Type=idle
 Restart=always

live-build/ubuntu-server/includes.binary/overlay/lib/systemd/systemd-networkd-wait-online

@@ -0,0 +1 @@
+/bin/true

minimize-manual

@@ -0,0 +1,63 @@
+#!/usr/bin/python3
+"""Minimize the number of manually installed packages in the image.
+
+Finds all manually installed meta packages, and marks their dependencies
+as automatically installed.
+"""
+import sys
+
+import apt
+
+
+def is_root(pkg):
+    """Check if the package is a root package (manually inst. meta)"""
+    return (pkg.is_installed and
+            not pkg.is_auto_installed and
+            (pkg.section == "metapackages" or
+             pkg.section.endswith("/metapackages")))
+
+
+def main():
+    """Main function"""
+    cache = apt.Cache(rootdir=sys.argv[1] if len(sys.argv) > 1 else None)
+    roots = set(pkg for pkg in cache if is_root(pkg))
+    workset = set(roots)
+    seen = set()
+    ubiquity_depends = set()
+
+    with cache.actiongroup():
+        while True:
+            print("Iteration", file=sys.stderr)
+            to_proc = workset - seen
+            if not to_proc:
+                break
+            for pkg in sorted(to_proc):
+                print("    Visiting", pkg, file=sys.stderr)
+
+                if pkg not in roots and pkg not in ubiquity_depends:
+                    pkg.mark_auto()
+
+                for dep in (pkg.installed.dependencies +
+                            pkg.installed.recommends):
+                    for bdep in dep.or_dependencies:
+                        for ver in bdep.target_versions:
+                            if ver.package.is_installed:
+                                if pkg.name == "ubiquity":
+                                    ubiquity_depends.add(ver.package)
+                                if pkg.name != "ubiquity":
+                                    # Reprocess this package again, as we did not mark it when we visited it from ubiquity
+                                    try:
+                                        ubiquity_depends.remove(ver.package)
+                                        # This will raise the KeyError here if ubiquity did not depend on it
+                                        seen.remove(ver.package)
+                                    except KeyError:
+                                        pass
+                                workset.add(ver.package)
+
+                seen.add(pkg)
+
+        cache.commit()
+
+
+if __name__ == '__main__':
+    main()