amd64: always install grub-efi-amd64-signed
shim-signed does not depend on grub-efi-amd64-signed in Xenial.
Historically, Xenial did not always ship with signatures. This is
different than LTSes after Xenial where this is the case. A future SRU
for grub should change this, but for now ensure to install the signed
package so that secure boot systems can actually boot.
MP: https://code.launchpad.net/~powersj/livecd-rootfs/+git/livecd-rootfs-1/+merge/394769
shim-signed does not depend on grub-efi-amd64-signed in Xenial.
Historically, Xenial did not always ship with signatures. This is
different than LTSes after Xenial where this is the case. A future SRU
for grub should change this, but for now ensure to install the signed
package so that secure boot systems can actually boot.
LP: #1901906
Backport vmtools version in vmdk (LP: #1893898)
Backport
LP: #1893898 describes missing vmtools version from the vmdk headers.
The version should be added as ddb.toolsVersion = "2147483647" however
the sed was no longer replacing a ddb.comment field with the tools
version. Rather than subbing ddb.comment with toolsVersion, this commit
deletes ddb.comment (which the comment mentions could cause errors),
and adds the correct value. There was no visibility into the descriptor
during hook creation, so debug statements were added. This allows us to
quickly verify in the logs that bad statements are removed (the possibly
offending comments), as well as ensuring that the toolsVersion is added
MP: https://code.launchpad.net/~jchittum/livecd-rootfs/+git/livecd-rootfs/+merge/394145
Make Ubuntu Vagrant box 40G. (LP: #1580596)
Vagrant images were previously put at 10G, but this was a regression
from Trusty, in which they were 40G. This made it a tough sell for
users to upgrade if they were using a Ubuntu desktop experience.
This change does not impact disk usage as Vagrant with the virtualbox
provider dynamically allocates space with the VMDK. On a test system,
the VMDK took up 1.1G of disk space according to df, and after
creating a 2G file in Vagrant, the VMDK grew to 3.1G.
Therefore, users who are running on a system with little free space will
not see adverse effects if they upgrade to a new vagrant image
MP: https://code.launchpad.net/~patviafore/livecd-rootfs/+git/livecd-rootfs/+merge/384636
Older version of vmdk-stream-converter has an incorrect header. The
original sed command replaced the incorrect "Description File" comment
with the correct "Disk DescriptorFile".
Backport
LP: #1893898 describes missing vmtools version from the vmdk headers.
The version should be added as ddb.toolsVersion = "2147483647" however
the sed was no longer replacing a ddb.comment field with the tools
version. Rather than subbing ddb.comment with toolsVersion, this commit
deletes ddb.comment (which the comment mentions could cause errors),
and adds the correct value. There was no visibility into the descriptor
during hook creation, so debug statements were added. This allows us to
quickly verify in the logs that bad statements are removed (the possibly
offending comments), as well as ensuring that the toolsVersion is added
shim-signed depends on grub-efi-amd64-signed, which in turn has
alternative depends on either `grub-efi-amd64 | grub-pc`. However to
support booting with either via shim&signed-grub and BIOS, the choice
must be made to install grub-pc, not grub-efi-amd64.
This makes images consistent with Ubuntu Deskop, Live Server, buildd
bootable images; all of which already do install grub-pc and
shim-signed.
Additionally, this will ensure that autoremove is run after installing
anything in the CPC build hooks. This is done to avoid shipping images
that include packages that are autoremovable. This will clean-up as
packages are installed and detect any breakage at build time.
LP: #1901906
In the buildd image chroot, /etc/resolv.conf is a symbolic link to
a configuration file in the /run directory. A call to truncate will
modify that file, which we should not do. Instead, we want to remove
the symbolic link and replace it with an empty file.
Instead of injecting an empty resolv.conf with an includes.chroot, we'll
inject it in late with a hook. The empty resolv.conf breaks DNS early in
the build, and causes some binary hooks to fail
Vagrant images were previously put at 10G, but this was a regression
from Trusty, in which they were 40G. This made it a tough sell for
users to upgrade if they were using a Ubuntu desktop experience.
This change does not impact disk usage as Vagrant with the virtualbox
provider dynamically allocates space with the VMDK. On a test system,
the VMDK took up 1.1G of disk space according to df, and after
creating a 2G file in Vagrant, the VMDK grew to 3.1G.
Therefore, users who are running on a system with little free space will
not see adverse effects if they upgrade to a new vagrant image
Xenial builds with core18-based snaps pre-seed core snap instead core18
and snapd. This patch is a backport of snap pre-seeding functionality
that is snap base aware.
When we encounter a failure in 'lb binary' the launchpad builders can
only surface the build output from stdout. If the binary hook failure
implicates the archive we can not determine fault without the apt
proxy log. This patch will dump the proxy log to stdout to aid in
debugging these failures.
Revert exclusion of makedev from buildd chroots, as it turned out not to
be the problem. Instead, fix up /dev/ptmx to be a character device node
rather than a symlink to /dev/pts/ptmx, in line with the discussion in
https://bugs.debian.org/817236; I think this is safer than
cherry-picking the fix to debootstrap at this point in a stable release
cycle.
LP: #1844504
The livecd.ubuntu-cpc.ext4 that is present in each build (plus kernel
and initrd) are not renamed from /build/binary/boot/filsystem.ext4
and friends until after the binary hooks are run, so this patch moves
from trying to perform this cleanup in a binary hook. Now the cleanup
will be run at the end of live-build/binary for the ubuntu-cpc project.
In parallel builds where a list of image targets are provided the build
may produce binaries that are not part of the named set of targets but
are created by series dependencies. These implicitly created binaries
may be generated by multiple builds but are unused as our convention for
the ubuntu-cpc project is to only consume binaries from the explicitly
named image targets; this avoid overwriting the same object by multiple
parallel builds.
This patch adds support for a 'provides' keyword for series files. It can
be specified multiple times per series file. The field is used by the
make-hooks script to generate a list of output files created explicitly by
the named image targets. The list is saved to the "explicit_provides"
file in the hooks output directory. In the case of the "all" target
this list would be empty. This list is consumed by the "final.binary"
hook file.
This patch adds support for optional final.binary hooks in hooks.d/base
and/or hooks.d/extra. These final.binary hooks are always included as
the last hook(s) if either exist with the hook in "extra" running last.
The base/final.binary hook includes logic to parse the "explicit_provides"
file generated by the make-hooks script and remove any binary output not
explicitly specified.
Some series files named unnecessary dependencies, specifically
disk-image, to keep output of implicit artifacts consistent between
parallel builds. These unnecessary dependencies are removed in this
patch.
The following targets have livecd.ubuntu-cpc.manifest (and
livecd.ubuntu-cpc.ext4) which differ in some way from the 'all'
target. They are all missing grub-efi and other modifications:
root-dir
squashfs
tarball
These targets do not depend on the 'disk-image' target. This means that
the ext4 produced will lack the uefi modifications (and any from the
disk-image target binary hooks).
Since the ext4 file is common to all builds there is a chance that a
parallel build from one of these targets could overwrite this artifact.
This patch ensures that all targets will produce consistent base output.
An upcoming addition of the disk-image target to several series files will
required some explanation. Without comment support in series files that
is not possible. This patch adds support for comments in series files.
* Replace "snap download" with tool that uses snap store's coherence feature
This is important for parallel image builds to ensure all pre-seeded snaps
have the same versions across image variants.
* Inject a proxy into the build providing a snapshot view of the package repo.
When the REPO_SNAPSHOT_STAMP variable is set, the auto/build script will attempt
to launch a transparent HTTP proxy on port 8080, and insert an iptables rule to
redirect all outgoing HTTP requests to this proxy.
The proxy, contained in the `magic-proxy` Python script, examines each request
and silently overrides those pointing to InRelease files or files that are
listed in InRelease files. It will instead provide the contents of the requested
file as it was at REPO_SNAPSHOT_STAMP, by downloading the corresponding asset
"by hash".
* Use series files with dependency handling to generate hook symlinks dynamically
This patch currently only applies to the "ubuntu-cpc" project.
More and more logic has been going into the hook scripts to decide
under which conditions they should run or not. As we are moving
to parallelized builds of image sets, this will get even more
complicated. Base hooks will have to know which image sets they
belong to and modification of the dependency chain between scripts
will become more complicated and prone to errors, as the number of
image sets grows.
This patch introduces explicit ordering and dependency handling for
scripts through the use of `series` files and an explicit syntax
for dependency specification.
Installing policyrcd-script-zg2 doesn't quite do this because of the way
that live-build installs its own temporary version of policy-rc.d. The
only remotely sensible way I can see to deal with this is to create the
symlink manually.
When storing the source in a git tree empty directories are dropped.
Adding the empty directories under includes.chroot to debian/dirs
fixes this up in binary packages built from git sources.
Test showed very little difference in file-size between -0 and -9 while
both compression and decompression need much less memory and compression
needs less CPU time for -0.
Supported xz versions before 5.2 also accept the -T4 option but ignore it.
Also depend on xz-utils instead of downloading pxz.
LP: #1701132
all our livefses.
* Pass --cache false to lb config; otherwise we copy around caches of
.debs that are never used properly, and which prevent us from emptying
/var/cache/apt in images.
* When building minimized cloud images, remove various packages that we
don't want installed by default. Some are tools that aren't needed for
non-interactive use; some are libraries whose reverse-dependencies
will have already been removed; and one, open-vm-tools, should only be
included in images that are targeted to VMWare (which is not the case
for any of the current minimal images), rather than being included
directly in the cloud-image seed.
(expanded changelog)
* Make non-x86 minimized images consistent with x86 by not explicitly
installing the server task.
* Clean up dangling /boot/initrd.img symlink left behind on minimized
builds.
instead of having bogus root=stuff arg generated in grub.cfg, it is
actually empty. Therefore update the sed command to make the arg in
the root= token optional. This should resolve non-booting livecd cpc
images.
Kernels which are able to boot without initramfs now dropped dependency
on initramfs-tools thus initramfs-tools can be removed from the image
instead of having to divert it to avoid initramfs generation.
unmount them, instead of working from a hard-coded list. This makes
the code resilient against other submounts being added later, including
downstream.
* Also nuke the sleep / udevadm settle calls in the process, which should
never be required and slow down the builds.
* live-build/ubuntu-cpc/functions: Add a function, teardown_mountpoint,
to reverse the work done in setup_mountpoint. Lack of this function
has forced users of setup_mountpoint to implement this separately
and the implementations have diverged. (LP: #1716992)
* live-build/ubuntu-cpc/functions: Remove umount_settle function.
The was only used where teardown_mountpoint was lacking.
/var/lib/apt, so we don't have to leave empty space in our derivative
images for packages that have been downloaded/installed/removed. This
normally isn't relevant for the installed system, since the root
filesystem will auto-expand in place on the target disk, but lets us
ship smaller images.
* Replace sources.list generated using COMPONENTS with the sources.list from
an Ubuntu Server installation (i.e. with all components enabled, and all
deb-src lines commented). LP: #1513529.
[ Chris Glass ]
* Fix the manifest generation in OVA files so that ovf files don't have
double extensions. (LP: #1627931)
* Fix the OVF's metadata to include Ubuntu specific identifiers and
descriptions instead of the generic Linux ones. (LP: #1656293)
[ Daniel Watkins ]
* Add replace_grub_root_with_label function thereby consolidating multiple
uses of the same calls to sed.
When update-grub is run, it will detect the disks in the build system.
Currently, we sed the grub configuration in places where this happens;
replace_grub_root_with_label moves that seddery in to a single callable
location to avoid ending up with multiple versions of it.
(It's worth noting that we also do this in 999-cpc-fixes.chroot, but as our
functions file isn't available within the chroot we don't use it there.)
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
