1185 Commits

Author SHA1 Message Date
Dimitri John Ledkov
09da74e154
functions: stop removing systemd-detect-virt unconditionally in undivert_grub
One can call divert_grub; replace_kernel; undivert_grub. And
replace_kernel will call into force_boot_without_initramfs, which
under certain conditions can call divert_grub &
undivert_grub. Resulting in undivert_grub called twice in a row.

When undivert_grub is called twice in a row it wipes
systemd-detect-virt binary from disk, as the rm call is unguarded to
check that there is something to divert if systemd package is
installed. And if the systemd package is not installed, it does not
check that systemd-detect-virt file is in-fact what divert_grub has
created.

Add a guard to check that systemd-detect-virt is the placeholder one,
before removing it.

LP: #1902260
(cherry picked from commit 096a00f40459187719840ccad99e86c7ade2ec12)
2021-02-09 00:59:04 +00:00
Dimitri John Ledkov
d447d167dc
Perform fsck on all rootfs.
LP: #1912835
(cherry picked from commit 33b7272a2331279b4c7aa8de73ed8a9069d0bd39)
2021-02-09 00:58:56 +00:00
Dimitri John Ledkov
bcb48c1625
esp: perform fsck.
LP: #1912835
(cherry picked from commit 53b28d5f58284657ee8efd08a633d557af1fcd97)
2021-02-09 00:58:50 +00:00
Dimitri John Ledkov
319341bda5
esp: install grub in ubuntu bootloader id path, instead of removable.
Do not use removable uefi bootloader path in the cloud-images by
default, as that prevents upgrades of the bootloader.

LP: #1912830
(cherry picked from commit 7c760864fdcb278ca37396f06f5e3f297428d63d)
2021-02-09 00:58:43 +00:00
Dimitri John Ledkov
6faede5710
Merge branch 'dk/bionic-buildd-unpack-kernel-initrd' of git+ssh://git.launchpad.net/~davidkrauser/livecd-rootfs/+git/livecd-rootfs into ubuntu/bionic
(Cherry-pick commit 065c82314464fa78337d5122e1d4826a7d6edbb0)
2021-02-09 00:57:50 +00:00
Cody Shepherd
92612859cc
Omit file type exensions in manifest names 2021-02-03 07:28:47 -08:00
Cody Shepherd
464b18de1a
Create manifests for buildd tarballs 2021-02-03 07:28:47 -08:00
David Krauser
ca2d1a728f Produce buildd kernel and initrd as separate artifacts
Multipass on Mac OS X requires standalone kernel and initrd artifacts
to boot.

Also call update-initramfs on all installed kernels. We only have one
kernel installed, so we don't need to specify an explicit version.
2021-01-08 14:54:29 -05:00
Łukasz 'sil2100' Zemczak
69195a0026 Build appliance qcow2 images only for amd64 appliances. 2020-12-11 10:16:43 +01:00
Łukasz 'sil2100' Zemczak
0e35dfdb46 We need to do it in a different order. 2020-12-05 01:10:12 +01:00
Łukasz 'sil2100' Zemczak
a976382d9a Try building qcow2 images for appliances. 2020-12-04 12:34:40 +01:00
Robert C Jennings
e6dc732f34
Merge bionic-open-vm-tools-backport into ubuntu/bionic [a=jchittum] [r=rcj]
Backport vmtools version in vmdk (LP: #1893898)

Backport
LP: #1893898 describes missing vmtools version from the vmdk headers.
The version should be added as ddb.toolsVersion = "2147483647" however
the sed was no longer replacing a ddb.comment field with the tools
version. Rather than subbing ddb.comment with toolsVersion, this commit
deletes ddb.comment (which the comment mentions could cause errors),
and adds the correct value. There was no visibility into the descriptor
during hook creation, so debug statements were added. This allows us to
quickly verify in the logs that bad statements are removed (the possibly
offending comments), as well as ensuring that the toolsVersion is added

MP: https://code.launchpad.net/~jchittum/livecd-rootfs/+git/livecd-rootfs/+merge/394144
2020-11-20 15:07:50 -06:00
Robert C Jennings
b5e21920dd
Merge bionic-sru-grub into ubuntu/bionic [a=powersj] [r=patviafore,rcj]
SRU always install grub-pc with shim-signed (LP: #1901906), and ensure to autoremove packages

MP: https://code.launchpad.net/~powersj/livecd-rootfs/+git/livecd-rootfs-1/+merge/393734
2020-11-20 15:06:31 -06:00
Robert C Jennings
8842b0528f
Merge vagrant_40_gb_bionic into ubuntu/bionic [a=patviafore] [r=rcj]
Make Ubuntu Vagrant box 40G. (LP: #1580596)

Vagrant images were previously put at 10G, but this was a regression
from Trusty, in which they were 40G. This made it a tough sell for
users to upgrade if they were using a Ubuntu desktop experience.

This change does not impact disk usage as Vagrant with the virtualbox
provider dynamically allocates space with the VMDK. On a test system,
the VMDK took up 1.1G of disk space according to df, and after
creating a 2G file in Vagrant, the VMDK grew to 3.1G.

Therefore, users who are running on a system with little free space will
not see adverse effects if they upgrade to a new vagrant image

MP: https://code.launchpad.net/~patviafore/livecd-rootfs/+git/livecd-rootfs/+merge/384635
2020-11-20 15:03:41 -06:00
John Chittum
45abf7c253
re-add sed command
Older version of vmdk-stream-converter has an incorrect header. The
original sed command replaced the incorrect "Description File" comment
with the correct "Disk DescriptorFile".
2020-11-18 08:49:23 -06:00
John Chittum
03a990188a
Backport vmtools version in vmdk (LP: #1893898)
Backport
LP: #1893898 describes missing vmtools version from the vmdk headers.
The version should be added as ddb.toolsVersion = "2147483647" however
the sed was no longer replacing a ddb.comment field with the tools
version. Rather than subbing ddb.comment with toolsVersion, this commit
deletes ddb.comment (which the comment mentions could cause errors),
and adds the correct value. There was no visibility into the descriptor
during hook creation, so debug statements were added. This allows us to
quickly verify in the logs that bad statements are removed (the possibly
offending comments), as well as ensuring that the toolsVersion is added
2020-11-18 08:49:16 -06:00
Joshua Powers
d5a2099c33
amd64: always install grub-pc with shim-signed
shim-signed depends on grub-efi-amd64-signed, which in turn has
alternative depends on either `grub-efi-amd64 | grub-pc`. However to
support booting with either via shim&signed-grub and BIOS, the choice
must be made to install grub-pc, not grub-efi-amd64.

This makes images consistent with Ubuntu Deskop, Live Server, buildd
bootable images; all of which already do install grub-pc and
shim-signed.

Additionally, this will ensure that autoremove is run after installing
anything in the CPC build hooks. This is done to avoid shipping images
that include packages that are autoremovable. This will clean-up as
packages are installed and detect any breakage at build time.

LP: #1901906
2020-11-13 09:13:59 -08:00
Łukasz 'sil2100' Zemczak
70d675e5ab Apparently the lxd appliance needs to use a custom track, so hack it in. 2020-08-13 14:52:30 +02:00
Steve Langasek
a61cfb6668 Merge remote-tracking branch 'codyshepherd/bootable-buildd/dist-upgrade/bionic' into ubuntu/bionic 2020-08-10 13:38:38 -07:00
Łukasz 'sil2100' Zemczak
07c17a8258 Merge branch 'feature/move-to-raspi-hwe-18.04' of https://git.launchpad.net/~fginther/livecd-rootfs into ubuntu/bionic 2020-07-28 12:30:27 +02:00
Michael Hudson-Doyle
ecc146ea5d Merge branch 'no-maas-squashfs' into lp-1883156 2020-07-23 14:28:25 +12:00
Dimitri John Ledkov
d0f81ae08a Subiquity: install linux-firmware in the installer layer, to make firmware blobs available in the live session. LP: #1847835 2020-07-22 22:47:18 +12:00
Dimitri John Ledkov
d278984cbf Use snap-tool to seed subiquity snap. 2020-07-22 22:43:15 +12:00
Michael Hudson-Doyle
570fcb79ff Stop building and mounting the mass-rack and maas-region squashfses for the server-live build as subiquity does not use them any more. 2020-07-22 22:11:40 +12:00
Dimitri John Ledkov
7bc4c77a03 Run snap info on the downloaded snap, rather than against the store. First of all snap info doesn't have --channel argument, thus queries the wrong channel, and depening on the cohort, a different snap might be visible too. Thus seed the base of the snap revision we dowanloaded, rather than some random one from the store. 2020-07-22 14:47:05 +12:00
Robert C Jennings
e4952519b3 Do not unconditionally install the core snap
Seeing any snap via snap_preseed will evaluate the base for each snap
and seed the appropriate base.  There should be no reason to explicitly
seed the 'core' snap and with snaps moving to 'core18' this will add
'core' without need.
2020-07-22 14:19:37 +12:00
Robert C Jennings
622adfc4eb Fix logic to ensure snapd is seeded in core18-only images
The _snap_post_process function is meant to install snapd if core18 is the
only core snap installed or removed snapd if core is installed and snapd
was not explicitly installed.  But the current logic in _snap_preseed
will never call _snap_post_process.  $core_name will never be empty
with the existing logic, but even if it were that would only be for the
'core' snap and we'd miss using the 'core18' logic that pulls in snapd.
Given the case statement in _snap_post_process can handle doing the
right thing given any snap we can just call it unconditionally.
2020-07-22 14:19:31 +12:00
Francis Ginther
db2e54491a Switch raspi3 builds to the linux-raspi-hwe-18.04 kernel
As part of the backport of the 5.4 kernel for Raspberry Pi, the kernel
flavour name is changing to match that in Focal. This is to provide a
consistent name for the 5.4 kernel in both Bionic and Focal.
This effectively rolls the kernel from 5.3 to 5.4 for raspi classic
images.
2020-07-20 16:13:03 -05:00
Łukasz 'sil2100' Zemczak
1c4a376ced Remove unneeded change. 2020-07-15 16:16:24 +02:00
Cody Shepherd
64ca75261f
Remove explicit install of grub-efi-* package as it is not necessary,
and potentially overexplicit
2020-07-13 09:08:33 -07:00
Cody Shepherd
75a1408d0e
Perform a dist-upgrade prior to installing packages for bootable-buildd
image in order to pull in package updates
2020-07-08 13:59:04 -07:00
Łukasz 'sil2100' Zemczak
294db12c82 We need to handle appliance images first before everything else. 2020-06-05 01:52:46 +02:00
Łukasz 'sil2100' Zemczak
037c4e4ac6 Do it even better. 2020-06-04 21:48:05 +02:00
Łukasz 'sil2100' Zemczak
61eae75d5e Support building arbitraty named appliance images. 2020-06-04 21:18:05 +02:00
Łukasz 'sil2100' Zemczak
467065461b The appliance model names have changed and we now use 'pi' instead of 'pi-arm64' 2020-05-26 19:20:43 +02:00
Pat Viafore
457170a92c
Make Ubuntu Vagrant box 40G. (LP: #1580596)
Vagrant images were previously put at 10G, but this was a regression
from Trusty, in which they were 40G.  This made it a tough sell for
users to upgrade if they were using a Ubuntu desktop experience.

This change does not impact disk usage as Vagrant with the virtualbox
provider dynamically allocates space with the VMDK.  On a test system,
the VMDK took up 1.1G of disk space according to df, and after
creating a 2G file in Vagrant, the VMDK grew to 3.1G.

Therefore, users who are running on a system with little free space will
not see adverse effects if they upgrade to a new vagrant image
2020-05-22 15:38:32 -05:00
Łukasz 'sil2100' Zemczak
650cced79e Add support for uc18 appliance images. 2020-05-07 18:39:32 +02:00
David Krauser
77cb232e8b Safely set /etc/resolv.conf to an empty file in buildd images
In bionic buildd image chroot, /etc/resolv.conf is a symbolic link to
a configuration file in the /run directory. A call to truncate will
modify that file, which we should not do. Instead, we want to remove
the symbolic link and replace it with an empty file.
2020-03-25 18:24:27 -04:00
David Krauser
f60c01ef0b Don't install live packages in buildd images. 2020-03-25 18:23:10 -04:00
Cody Shepherd
127ed59fb1 Only build bootable buildd images for amd64. 2020-03-23 13:27:19 -04:00
Cody Shepherd
68e8afd528 Properly setup the chroot before using the network in buildd hook. 2020-03-23 13:27:19 -04:00
Cody Shepherd
f5c65024a3 Fix broken DNS in buildd image builds.
Instead of injecting an empty resolv.conf with an includes.chroot, we'll
inject it in late with a hook. The empty resolv.conf breaks DNS early in
the build, and causes some binary hooks to fail
2020-03-23 13:27:19 -04:00
Cody Shepherd
d74b68f9aa Include empty resolv.conf only in buildd lxd tarballs. 2020-03-23 13:27:19 -04:00
Cody Shepherd
075d1e85bf Add hooks to build bootable buildd images
Only install policyrd-script-zg2 in lxd tarball
2020-03-23 13:27:19 -04:00
Cody Shepherd
0cd622e863 Enable building buildd artifacts with an ext4 image target 2020-03-23 13:27:09 -04:00
Robert C Jennings
15c419eafb
Fix core_snap variable scope issue
The prior change to pre-seed the correct base added a case to handle
installation of the core snap.  When that was added it created a case
where the $core_snap variable would not be defined when we reach the
end of the _snap_preseed function and evaluate if $core_snap is defined
by empty.  Previous if the snap to preseed was core or core18 then
$core_snap would be "" so this patch addresses that by ensuring the
variable exists and is empty by default.
2020-02-28 08:36:56 -06:00
Robert C Jennings
632ab2b982
Address snap base regression after snap-tool removal
With the removal of snap-tool failures are seen in image builds that do
not have the 'core' snap included by the seed.  This is the case for the
minimized subproject of the ubuntu-cpc project where lxd/core is removed.
In that subproject, any binary hook which adds a snap that is based
on 'core' will not add 'core' and fail 'snap debug validate-seed'.
snap-tool included the following logic in the 'snap-tool info' when
determining snap bases:

    # Have "base" initialized to something meaningful.
    if self.is_core_snap():
        snap_data["snap"]["base"] = ""
    elif snap_data["snap"].get("base") is None:
        snap_data["snap"]["base"] = "core"

The snap store does not return a base if the base is core which makes
this necessary.  This patch looks for the base in 'snap info' output
and if none is found (and the snap is not snapd or core) it assumes the
base is 'core' and installs it.  This restores the behavior lost in the
migration from snap-tool to snap cli.
2020-02-27 16:40:23 -06:00
Robert C Jennings
ec954a80b9
Use snap cli rather than custom snap-tool (LP: #1864252)
snap-tool was added to support a deprecate cohort-key feature of the
snap store.  Recent changes in snap assertions have added additional
fields which snap-tool is not retrieving.  This resulted in snap install
failures on first boot.

This patch removes snap-tool and returns to using the snap cli.
This ensures snap downloads will function without odd incompatibilities.
2020-02-27 16:39:06 -06:00
Steve Langasek
d516e68807 Stop building per-pi-flavor images and only build a pi generic image for core18. (LP: #1861520) 2020-02-27 12:17:28 -08:00
Łukasz 'sil2100' Zemczak
fcaa6d447c Ensure seed part. is mounted; wait for getty 2020-01-24 18:55:34 +01:00