Due to how `disk-image` file is structured, it builds BIOS and UEFI
images at the same time. However, certain images (e.g., GCE images)
require only UEFI image to be built, BIOS image is being simply
discarded. This results in longer build times.
Splitting out `disk-image-uefi` would allow images to use it instead of
`disk-image` and thus avoid building unused BIOS images.
`disk-image` now depends on `disk-image-uefi` for backward
compatibility.
(cherry picked from commit b40ce74fd67bbaa7f9ec94463d8da759724f9fec)
I recently pulled initramfs logic out of the base build hook, and
dropped that into the `replace_kernel` function. Any cloud image that
does not leverage the generic virtual kernel was expected to call
`replace_kernel` to pull in a custom kernel. That function will
disable initramfs boot for images that use a custom kernel.
Minimal cloud images on amd64 use the linux-kvm kernel, but the build
hook does not utilize the `replace_kernel` function. Instead, the
kernel flavor is set in `auto/config`. I pulled that logic out of
`auto/config` and am now calling `replace_kernel` in the build hook.
I also moved a call to generate the package list so that it will pick
up the change to the linux-kvm kernel.
Change mount option for ubuntu-cpc images from "defaults" to
"umask=0077". ESP partitions might contain sensitive data and
non-root users shouldn't have read access on it.
With this change, when we attempt to boot with an initramfs and fail,
initrdless_boot_fallback_triggered is set to non-zero in the grubenv.
This value can be checked after boot by looking in /boot/grub/grubenv
or by using the grub-editenv list command.
Addresses LP: #1870189
Generic cloud images with the linux-generic kernel are not able to
boot without an initramfs. Previously, these images attempted to boot
without an initramfs, would fail, and then retry with an initramfs.
This slows the boot and is confusing behavior.
Do not use removable uefi bootloader path in the cloud-images by
default, as that prevents upgrades of the bootloader.
LP: #1912830
(cherry picked from commit 7c760864fdcb278ca37396f06f5e3f297428d63d)
shim-signed depends on grub-efi-amd64-signed, which in turn has
alternative depends on either `grub-efi-amd64 | grub-pc`. However to
support booting with either via shim&signed-grub and BIOS, the choice
must be made to install grub-pc, not grub-efi-amd64.
This makes images consistent with Ubuntu Deskop, Live Server, buildd
bootable images; all of which already do install grub-pc and
shim-signed.
Additionally, this will ensure that autoremove is run after installing
anything in the CPC build hooks. This is done to avoid shipping images
that include packages that are autoremovable. This will clean-up as
packages are installed and detect any breakage at build time.
LP: #1901906
Original fix proposed by Stanislav German-Evtushenko (giner)
CPC Ubuntu cloud images default to enabling a serial console connection
via the kernel commandline option `console=ttyS0`. Many clouds support
the serial connection, and utilize it for debugging purposes. Virtualbox
supports the serial connection as well. In Bionic and earlier images,
Vagrant boxes created a serial log file in the directory of the
Vagrantfile by default. However this is not standard behaviour for
Vagrant images, and so it was removed in Eoan onwards.
Starting in Eoan, there were reports of image booting slowdown (1874453
is a single example). After testing, it was determined that the serial
connection starting, without a device attached, was the cause of the
slow down. However, we did not want to revert to the old functionality
of creating a file. Much thanks to <giner> for providing the Ruby syntax
for sending to File::NULL.
This option will not create a local file, however, the default
Vagrantfile configuration is overwritable via a users Vagrantfile. The
original syntax for creating a file local to the users Vagrantfile has
been included as an example.
It was reported and confirmed in LP bug #1875400
(https://bugs.launchpad.net/cloud-images/+bug/1875400) that on the public
KVM cloud image there exists a large list of packages marked for auto-removal.
This should never be the case on a released cloud image.
These packages are marked for auto-removal because in the KVM image binary hook
we removed both initramfs-tools and busybox-initramfs packages. Due to package
dependencies this also removed:
busybox-initramfs* cloud-initramfs-copymods* cloud-initramfs-dyn-netconf*
cryptsetup-initramfs* initramfs-tools* initramfs-tools-core* multipath-tools*
overlayroot* sg3-utils-udev* ubuntu-server*
But it did not remove all the packages that the above list depended on.
This resulted in all those packages being marked for auto-removal because they
were not manually installed nor did they have any manually installed packages
that depended on them.
The removal of initramfs-tools and busybox-initramfs was to avoid the
generation of initramfs in images that should boot initramfsless.
This requirement is obsolete now because the initramfsless boot handling
is now handled via setting GRUB_FORCE_PARTUUID in /etc/default/grub.d/40-force-partuuid.cfg.
In test images I have verified that GRUB_FORCE_PARTUUID is set and that
boot speeds have not regressed.
LP: #1880170
Vagrant images were previously put at 10G, but this was a regression
from Trusty, in which they were 40G. This made it a tough sell for
users to upgrade if they were using a Ubuntu desktop experience.
This change does not impact disk usage as Vagrant with the virtualbox
provider dynamically allocates space with the VMDK. On a test system,
the VMDK took up 1.1G of disk space according to df, and after
creating a 2G file in Vagrant, the VMDK grew to 3.1G.
Therefore, users who are running on a system with little free space will
not see adverse effects if they upgrade to a new vagrant image
The livecd.ubuntu-cpc.ext4 that is present in each build (plus kernel
and initrd) are not renamed from /build/binary/boot/filsystem.ext4
and friends until after the binary hooks are run, so this patch moves
from trying to perform this cleanup in a binary hook. Now the cleanup
will be run at the end of live-build/binary for the ubuntu-cpc project.
In parallel builds where a list of image targets are provided the build
may produce binaries that are not part of the named set of targets but
are created by series dependencies. These implicitly created binaries
may be generated by multiple builds but are unused as our convention for
the ubuntu-cpc project is to only consume binaries from the explicitly
named image targets; this avoid overwriting the same object by multiple
parallel builds.
This patch adds support for a 'provides' keyword for series files. It can
be specified multiple times per series file. The field is used by the
make-hooks script to generate a list of output files created explicitly by
the named image targets. The list is saved to the "explicit_provides"
file in the hooks output directory. In the case of the "all" target
this list would be empty. This list is consumed by the "final.binary"
hook file.
This patch adds support for optional final.binary hooks in hooks.d/base
and/or hooks.d/extra. These final.binary hooks are always included as
the last hook(s) if either exist with the hook in "extra" running last.
The base/final.binary hook includes logic to parse the "explicit_provides"
file generated by the make-hooks script and remove any binary output not
explicitly specified.
Some series files named unnecessary dependencies, specifically
disk-image, to keep output of implicit artifacts consistent between
parallel builds. These unnecessary dependencies are removed in this
patch.
The addition of disk-image to series files in a prior commit required
some explanation. Without comment support in series files that was
not possible. This patch adds support for comments and adds those
comments as well.
The following targets have livecd.ubuntu-cpc.manifest (and
livecd.ubuntu-cpc.ext4) which differ in some way from the 'all'
target. They are all missing grub-efi and other modifications:
root-dir
squashfs
tarball
These targets do not depend on the 'disk-image' target. This means that
the ext4 produced will lack the uefi modifications (and any from the
disk-image target binary hooks).
Since the ext4 file is common to all builds there is a chance that a
parallel build from one of these targets could overwrite this artifact.
This patch ensures that all targets will produce consistent base output.
commit a993592 introduced an additional call to create_manifest
(and snap-seed-parse) to write binary/boot/filesystem.packages. This
caused duplicate snap lines in the qcow manifest. This is because the
live-build/auto/binary code assumes that after 'lb binary' is run the
filesystem.packages will only have debs and it calls snap-seed-parse to
add them to the file. The commit changed filesystem.packages in the
ubuntu-cpc uefi binary hook to include debs and snaps.
This patch keeps the intent of the prior patch, updating the
filesystem.packages file for the content of the uefi disk image, but
only writes a listing of debian packages to match the expected content
of filesystem.packages. The snaps will still be added in generic code
in live-build/auto/build.
This patch currently only applies to the "ubuntu-cpc" project.
More and more logic has been going into the hook scripts to decide under which conditions they should run or not. As we are moving to parallelized builds of image sets, this will get even more complicated. Base hooks will have to know which image sets they belong to and modification of the dependency chain between scripts will become more complicated and prone to errors, as the number of image sets grows.
This patch introduces explicit ordering and dependency handling for scripts through the use of `series` files and an explicit syntax for dependency specification.
