Now that kernel names use expected -generic flavour, and kernels are
installed in the live layer, we can go back to stock behaviour of
auto/build noticing that binary hooks are called on a live layer and
executing the extraction & rename of the kernel flavours.
BTW we can even later expand that to support 2 generic abis, and
calling the bigger one the hwe generic such that can also remove
./live-build/ubuntu-server/hooks/04-kernel-bits.binary.
This fixes ubuntu arm64+x13s that is unable to find
ubuntu-x13s.kernel-laptop as due to this hook, which currently
produces ubuntu.kernel-generic in error which is not at all expected
by ubuntu-cdimage. Also this unbreaks producing oem & intel-iot
images, although we will build these in 24.04 only next.
This reverts ubuntu daily-live to use `--linux-flavours
laptop-generic-hwe-22.04` instead of `--linux-flavours none
--linux-packages=none --initramfs=none`, like it did in lunar and
pre-canary-image or images that install kernel in live layer.
Fixes: c00bbf3fb3 ("desktop: place kernel in the live layer")
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
With the migration from linux-kvm to linux-virtual/generic for the mantic minimal
images we can now start building arm64 minimal cloud images.
When building initial test images we noticed that the flash-kernel package was being
installed. This is not required for EFI images.
This commit removes the flash-kernel package from the cloud images arm installs
The only images built using the livecd-rootfs ubuntu-cpc project for arm64 and armhf
which are not cloud images (which therefore do not require flash-kernel) are the
raspi images. raspi does require flash-kernel. But the raspi images use the
`ubuntu-server-raspi` task to install the flash-kernel. As such this non raspi
flash-kernel package install can be completely removed.
It also means that initramfs-tools and dracut-install are installed as these are dependencies
of flash-kernel package.
Add mapping to use laptop-23.10 kernel. Ensure that
enhanced-secureboot is only setup on amd64 arch.
LP: #2037099
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
minimized cloud image policy, introduced in version 23.10.16, is to not install recommends for any package
installs during build. This is to keep the image as small as possible. This also extends to
the grub related packages.
This solves the problems detailed in LP: #2037075 and aligns other arches more with amd64 install of
grub/shim packages for both minimized and non minimized ubuntu-cpc cloud image builds.
Fix use of variable declared in conditional branch and used in parent
scope in snap_validate_seed. This would affect binary for images without
kernel and using "set -u". (LP: #2037338)
When trying to make changes to refactor livecd-rootfs, it is difficult to
know what side effects a change may have because of the use of globs on
arch/subarch and the lack of an authoritative list of supported arch
combinations.
This assembles a list of all possible values for $ARCH:$SUBARCH by looking at
all existing uses of $SUBARCH in live-build/auto/config and combining with
etc/default-arches from ubuntu-cdimage:
$ grep + etc/default-arches |grep -vE '(trusty|xenial|bionic|focal)[[:space:]]|appliance'
It also includes a special-case *appliance* glob, because there are many
Ubuntu Core appliance builds and there may be more in the future, and we
don't want to have to update livecd-rootfs with a hard-coded list.
Otherwise, this includes all currently used / supported combinations. The
amd64+kassel subarch is referenced in the code, but stopped being built a
while ago because "end of contract"; and there are some older no-longer-used
subarchs for particular raspi variants, which there is work to clean up
separately. So this should be a no-op wrt buildability of any existing
images on mantic - and if not, that's important for us to know!
In the past, we'd directly snap install lxd which defaults to
the latest/stable channel. However, whilst working on enhancing
unminimize, it was observed that we install this snap from
the stable/ubuntu-<version> channel instead.
This was also noted as a failure when running the CTF tests:
`lxd installed from latest/stable, not stable/ubuntu-23.10`
With the introduction of the 6.5 kernel for mantic on 13th September ago we are seeing image build failures
on the armhf builds. The build failure was `No kernel output for generic-lpae!`.
Introduced in the 6.4 kernel and therefore now also in 6.5 there is no generic-lpae flavor anymore. it's just generic now.
As such this commit updates the expected flavour for armhf to generic.
This is needed following the addition of the new boot partition. This
also gives us the opportunity to refactor the logic and use a case
statement instead of ifs
In order to support better support Full Disk Encryption on the clouds,
the boot assets have to sit on an un-encrypted partition. We've tried
mounting the ESP on /boot before but it didn't work as /boot has to
support linking for DPKG to work and the ESP has to be FAT.
In a minimized image, the linux headers are stripped, so when
unminimizing it, we should restore those stripped headers
by installing the linux-virtual package.
The unminimize script previously just restored the system documentation
and translations, man pages, and installed ubuntu-minimal and ubuntu-standard
packages to provide the familiar Ubuntu minimal system. But such an image
never became an equivalent of base image.
Upon investigation and looking at how the base image is constructed -
https://git.launchpad.net/livecd-rootfs/tree/live-build/auto/config#n1108 -
we use the following things:
- minimal task
- standard task
- cloud-image task (which involves ubuntu-server)
- ubuntu-minimal package
- server task if arch != amd64
OTOH, in the unminimize script, we use the following:
(https://git.launchpad.net/livecd-rootfs/tree/live-build/auto/build#n286)
- ubuntu-minimal package
- ubuntu-standard package
So upon running some tests, it was found that if we install ubuntu-server
(with --fix-policy flag), we get the resulting image equivalent to that
of a base image.
cf: https://warthogs.atlassian.net/browse/CPC-3033
The pi images contain redundant copies of cmdline.txt and config.txt in
the boot partition mount-point (which get hidden by the *actual* boot
partition). This commit removes those and simplifies the subarch check
(again, +raspi is the only subarch in use at this point on the Raspberry
Pi images)
/etc/hosts in an installed system comes from one of several places:
- the netbase package which ships a sane default
- scripts/casper-bottom/18hostname, which is mostly identical to the netbase
one but sets a hostname to the flavor name
- the installer which writes one to the target fs
- chroot includes in livecd-rootfs
A chroot include is only needed for images that don't include netbase (so
don't inherit from the minimal seed), don't boot casper, and don't run an
installer.
NONE of these conditions are true for the ubuntu-server live image. And in
addition, the /etc/hosts here differs from the others.
So, remove it.
systemd-resolved got moved out of the systemd package in kinetic and is
required for bootable buildd environement (as opposed to launchpad buildd)
(LP: #2007419)
Prior to dpkg/1.21.0, there was a bug where dpkg -V/--verify
couldn't list all the correct packages correctly but with
that being fix and in archive since Jammy, this works perfectly
but the syntax to report the missing files have changed. It
just prints 'missing' now. With that new format, we can now
fix the regex to simply list the packages.
With this patch, the unminimize script works flawlessly
on a minimized image.
ubuntu-cpc project binary hooks were not all producing .filelist files as they were not using
the create_manifest shared function.
This commit ensures the disk-image-uefi, disk-image-ppc64el and disk-image-uefi-non-cloud hooks create
a filelist during build.
The image filelists created during ubuntu-cpc project image builds were not sorted.
Soring the filelists makes it easier to compare the filelists without needing to sort first.
This package is needed by ubuntu-advantage-tools for cloud images
only. u-a-t ultimately will drop this entirely in their next release.
(cf: https://github.com/canonical/ubuntu-pro-client/issues/2692)
So instead of putting this in the seeds and then having to deal
with the seed changes to Mantic (after its release) and SRU of
ubuntu-meta, I'd rather have changes in livecd-rootfs done. Once
python3-systemd dependency is dropped from u-a-t, we'll drop it
from livecd-rootfs in Mantic+, too.
fuse3 was previously installed through recommends but with minimized images we no longer install recommends packages.
It is only required when preseeding snaps so does not need to be present in all minimized images so does not
need to be in the cloud-minimal seed.
As a result of not installing recommended packages the packages required to run `grub-install`
are no longer installed by default.
To ensure we can successfully run `grub-install` we install both `grub-pc` and `grub2-common`
packages.
As a result of not installing recommended packages we have dangling symlink `/boot/initrd.img.old`
As per the preceding `/boot/initrd.img` cleanup. Cleanup of `/boot/initrd.img.old`
only happens if it is a dangling symlink.
These `rm` commands also have `--verbose` flags now to make it easier when debugging logs
For minimized images we do not want to install any recommended packages.
We can do this by setting APT::Install-Recommends to false in apt config
or by passing --no-install-recommends to apt-get install.
While attempting to run autopkgtest locally, the test stops at the
following command:
ssh-keygen -t ed25519 -C ubuntu_vagrant_insecure_key -b 4096 -f
/tmp/tmp.VuAfnsBv1G/vagrant_insecure_key
This is found in live-build/ubuntu-cpc/hooks.d/base/vagrant.binary
It appears to be waiting for a passphrase, as running that outside of
adt gives a more helpful "Enter passphrase" prompt.
Explicitly set the passphrase to empty with the `-N` argument.
Armhf images install the `generic-lpae` kernel, while other ARCHes use the
standard `generic` kernel when building the "virtual" image flavour.
Code was looking for a kernel binary ending with -generic in armhf
builds, and failed. Add a special condition to handle armhf builds'
kernel ending with `generic-lpae`
References:
[1] https://bugs.launchpad.net/cloud-images/+bug/2029527
During Realtime kernel image build, there was an error during
validating snap seed which derivative images copied 5.19
apparmor feature and can't validate when Realtime kernel (5.15)
installed [0].
To prevent this, bind correct apparmor feature with kernel
version.
[0] https://bugs.launchpad.net/ubuntu/+source/livecd-rootfs/+bug/2024639
(cherry picked from commit 6b54faa6be)
With the switch to the ubuntu-cloud-minimal seed, we
don't really need to purge anything now. On the contrary,
the purging of packages if not installed, fails with the
exit code of 100.
EDK II is available for the StarFive VisionFive 2 board. As it is larger
than U-Boot we need to increase the size of the loader 2 partition to
accommodate it.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Now that we have the cloud-minimal seed for minimized cloud
image builds, we should drop all the workarounds and hacks
we once needed when we were using the server seed. We can
directly use the new metapackage and get rid of the tasks and
other autoremoves, et al.
Package linux-allwinner has a kernel with the generic flavour as
dependency. Add this translation to our code checking the correct
installation.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
According to the kernel team the Linux Meta package linux-allwinner shall
continue to be supplied. It will depend on generic packages.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Remove kvm-image altogether.
Previously for minimal image replace_kernel function replaced virtual
images with kvm, and called force_boot_without_initramfs. Now simply
call force_boot_without_initramfs for minimal image without replacing
kernel flavour.
This also means minimal images can now be built for arm64 and armhf.
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
Up to now we have used u-boot-menu for preinstalled images for the SiFive
HiFive Unmatched and Unleashed boards and GRUB for all other RISC-V images.
The choice was made because RISC-V GRUB was not available when the SiFive
boards where released.
Let the Unmatched and Unleashed board preinstalled images use GRUB.
Simplify the code.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Canonical Public Cloud's project seems a bad place to build images for
hardware devices however this is how things were done a we now need to
maintain this.
The recent change to mount the ESP on /boot breaks those images, instead
of adding more hacky things in the hook, create a dedicated target for
those images and use a different hook to build UEFI images.
This is required by the new UEFI binary hook as we mount the ESP on
/boot and the ESP filesystem doesn't support symlinks.
We keep symlinks for s390x images which do not use UEFI anyway.
This is driven by online encryption scenarios. In order to efficiently
encrypt the root filesystem without modifying the partition layout, the
kernel should sit in an un-encrypted /boot partition. Instead of
creating a new partition that would change the default partition layout,
we mount the ESP on /boot. We also need to then bind mount /boot on
/boot/efi because that's where Grub expects the ESP to be located.
This now matches the cloud images (7c760864fd)
fixing bootloader updates in the buildd images, but also fixing
compatibility with using devtmpfs for losetup.
kpartx on riscv64 appears to be racy. Rather than trying to debug these
fraught races somewhere between udev and libdevmapper, we can use losetup
which should be simpler and less error-prone.
live-build/auto/config:
- for Ubuntu Server live images and the arm64+tegra full arch, build a
tegra variant with linux-nvidia-tegra as the flavor and
linux-nvidia-tegra as the kernel meta-package
- default to nvidia-$SUBARCH as the kernel flavor for all images using
arm64+tegra as full arch
hooks/03-kernel-metapkg.chroot_early:
- use linux-nvidia-tegra as kernel meta-package for the nvidia-tegra
flavor
missing a && between icicle and visionfive, led to /boot/efi still being
in place, and grub-install running instead of exiting the func.
fixes LP:2015750
Cloud-init cannot write directly to
/etc/NetworkManager/system-connections because subiquity may
need to emit config to /etc/netplan/00-installer.yaml and call
netplan apply for autoinstall.network use-cases.
When cloud-init's config is written directly to
/etc/NetworkManager, neither netplan nor subiquity has knowledge of
this config and this results in namespace collisions in NetworkManager
due to `netplan-` named connections and `cloud-init` connection ids
fighting over which config own a given interface name.
Deleting this config overlay allows subiquity to manage all network
setup when it needs to with netplan directly.
Subiquity already has logic to rename any unwanted netplan
configuration when it intends to write cfg and run netplan apply[1].
This should allow subiquity full control of network config when needed.
[1] https://github.com/canonical/subiquity/blob/
92ac6544cdfedfd332d8cd94dbcfad0aab994575/subiquitycore/
controllers/network.py#L267
LP: #2015605
SUBARCH=visionfive2 is used to build images for the StarFive VisionFive 2
boards. For the device-tree we assume board revision 1.3B.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Autoinstall directives can be provided on the grub cmdline to
cloud-init via kernel parameters like the following:
autoinstall 'ds=nocloud-net;s=http://somedomain/'
In order to support DNS resolution for NoCloud datasource at
datasource discovery time, cloud-init.service needs to be
orderered after NetworkManager.service and
NetworkManager-wait-online.service
which will have brought up applicable NICs.
Since NetworkManager is After=dbus.service, the cloud-init.service
avoids systemd ordering cycles by also dropping
Before=sysinit.target when it adds, After=NetworkManager.service and
After=NetworkManager-wait-online.service
Add this file overlay for /lib/systemd/system/cloud-init.service
because systemd drop-in files can only add constraints and not
drop prexisting service constraints.
Also add an AUTOMATION_HEADER comment to any generated files to
add discoverability in the event of future bugs/concerns.
LP: #2008952
ipc was dropped as an apparmor feature. checked by grabbing the latest
lunar VM, installing the latest kernel, doing a reboot, and comparing
directories and files. compared all files and the only diff is the ipc
posix_mqueue
When building locally using the auto/build script unmounting fails.
Avoid mounting via bind. Mount mountpoint/dev as devtmpfs file system and
mountpoint/dev/pts as devpts file system.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Steve Langasek
Dimitri John Ledkov
Philip Roche
Michael Hudson-Doyle
Simon Poirier
Dan Bungert
Utkarsh Gupta
Gauthier Jolly
Łukasz 'sil2100' Zemczak
Sebastien Bacher
Dave Jones
Utkarsh Gupta
Stefan Hammer
Jess Jang
Heinrich Schuchardt
Yao Wei (魏銘廷)
Loïc Minier
John Chittum
Chad Smith
William Wilson
Sean Davis