1485 Commits

Author SHA1 Message Date
Dimitri John Ledkov
3d65dbb767 Change iptables calls, to query rules and quickly check that connectivity works after transparent proxy has been installed. LP: #1917920
(cherry picked from commit 1cd8fe14b70e98220c519d679217737672ee9019)
2021-09-28 13:48:41 +02:00
Jason C. McDonald
1c66b45338 Generate manifest for HyperV desktop images
Manifests are necessary for candidate checking in the
CPC automated workflow for building and publishing
HyperV desktop images. (#1940136)
2021-08-20 09:14:22 -07:00
Łukasz 'sil2100' Zemczak
4cf38c5151 Some more changes needed for enabling official intel-iot image builds. 2021-08-16 13:04:25 +02:00
Łukasz 'sil2100' Zemczak
44148c60a5 One more fix for intel-iot. 2021-08-12 13:45:35 +02:00
Łukasz 'sil2100' Zemczak
977c1cf74b A small fix for intel-iot 2021-08-04 17:35:11 +02:00
Łukasz 'sil2100' Zemczak
0b56fd53fe Revert /dev sharing improvements. 2021-08-02 22:13:43 +02:00
Łukasz 'sil2100' Zemczak
c3ebdc6f66 Backport fix for LP: #1938414 2021-07-29 11:06:53 +02:00
Łukasz 'sil2100' Zemczak
6568c5fa29 Merge branch 'ubuntu/focal' of git+ssh://git.launchpad.net/livecd-rootfs into bdmurray/focal-intel-iot 2021-07-29 10:31:38 +02:00
Brian Murray
6fac135f23 fix an extraneous || 2021-07-28 14:34:54 -07:00
Brian Murray
4b3f3c52dc add missing semicolons 2021-07-28 13:48:09 -07:00
Brian Murray
69afe8fd5d Add support for creating images (ubuntu-core and classic) with a kernel optimized for Intel IoT devices. (LP: #1938338) 2021-07-28 13:37:31 -07:00
Michael Hudson-Doyle
c27dbda4ca backport some system unit & cloud-init config changes from devel:
* Simplify how the subiquity client is run on the serial console in the live
  server environment, breaking a unit cycle that sometimes prevents
  subiquity from starting up at all. (LP: #1888497)
* Do not set the password for the installer user via cloud-init as subiquity
  can now do this itself. (LP: #1933523)
2021-07-23 12:07:26 +12:00
Dimitri John Ledkov
acb4de3be0 Cherry-pick 4428dbbf8f998c138093e3208bc7a0e7034714be
riscv64: generalise slightly how preinstalled server images are built, to allow using subarch to pick u-boot binaries to install.
2021-06-15 13:30:29 +02:00
Jean-Baptiste Lallement
fee8ba22f4 Added hook for AD support
Adding dependencies for Active Directory support to workaround LP: #1921862

Co-authored-by: Didier Roche <didrocks@ubuntu.com>
2021-05-21 16:56:38 +02:00
Steve Langasek
b558c86764 Merge remote-tracking branch 'codyshepherd/buildd/hide-grub-menu-ff' into ubuntu/focal 2021-05-20 17:06:44 -07:00
Thomas Bechtold
99add26dc3
add configure_oci function and use it in ubuntu-oci
With that, the Dockerfile modifications[0] currently done externally
are done now here. That means that the created rootfs tarball can be
directly used within a Dockerfile to create a container from scratch:

FROM scratch
ADD livecd.ubuntu-oci.rootfs.tar.gz /
CMD ["/bin/bash"]

[0]
https://github.com/tianon/docker-brew-ubuntu-core/blob/master/update.sh

(cherry picked from commit a81972a58b004897bf3e5c14ff371bc2f6b5e4b8)
2021-04-30 14:02:59 +02:00
Thomas Bechtold
9e15d88f4c
Add new ubuntu-oci project
This is a copy of the ubuntu-base project.
Currently ubuntu-base is used as a base for the docker/OCI container
images. The rootfs tarball that is created with ubuntu-base is
published under [0]. That tarball is used in the FROM statement of the
Dockerfile as base and then a couple of modifications are done inside
of the Dockerfile[1].
The ubuntu-oci project will include the changes that are currently
done in the Dockerfile. With that:

1) a Dockerfile using that tarball will be just a 2 line thing:

   FROM scratch
   ADD ubuntu-hirsute-core-cloudimg-amd64-root.tar.gz /
   CMD ["/bin/bash"]

2) Ubuntu has the full control about the build process of the
docker/OCI container. No external sources (like [1]) need to be
modified anymore.
3) Ubuntu can publish containers without depending on the official
dockerhub containers[2]. Currently the containers for the AWS ECR
registry[3] use as a base[4] the official dockerhub containers. That's
no longer needed because a container just needs a Dockerfile described
in 1)

When the ubuntu-oci project has the modifications from [1] included,
we'll also update [1] to use the ubuntu-oci rootfs tarball as a base
and drop the modifications done at [1].

Note: Creating a new ubuntu-oci project instead of using ubuntu-base
will make sure that we don't break users who are currently using
ubuntu-base rootfs tarballs for doing their own thing.

[0] https://partner-images.canonical.com/core/
[1]
https://github.com/tianon/docker-brew-ubuntu-core/blob/master/update.sh
[2] https://hub.docker.com/_/ubuntu
[3] https://gallery.ecr.aws/ubuntu/ubuntu
[4]
https://launchpad.net/~ubuntu-docker-images/ubuntu-docker-images/+oci/ubuntu/+recipe/ubuntu-20.04

(cherry picked from commit ac4a95b9314cf1f8ce01f42016c271c0a6078372)
2021-04-30 14:02:59 +02:00
Robert C Jennings
1a43fa5937
Merge remote-tracking branch 'gjolly/consistent_fstab-focal' into ubuntu/focal 2021-04-10 05:19:52 -05:00
David Krauser
597d678e17
Fix broken minimal cloud image boot on amd64 (LP: #1920043)
I recently pulled initramfs logic out of the base build hook, and
dropped that into the `replace_kernel` function. Any cloud image that
does not leverage the generic virtual kernel was expected to call
`replace_kernel` to pull in a custom kernel. That function will
disable initramfs boot for images that use a custom kernel.

Minimal cloud images on amd64 use the linux-kvm kernel, but the build
hook does not utilize the `replace_kernel` function. Instead, the
kernel flavor is set in `auto/config`. I pulled that logic out of
`auto/config` and am now calling `replace_kernel` in the build hook.

I also moved a call to generate the package list so that it will pick
up the change to the linux-kvm kernel.
2021-03-22 10:26:06 -05:00
Gauthier Jolly
330720f686
ubuntu-cpc: secure esp mountpoint (LP: #1881006)
Change mount option for ubuntu-cpc images from "defaults" to
"umask=0077". ESP partitions might contain sensitive data and
non-root users shouldn't have read access on it.
2021-03-18 10:10:43 +00:00
Cody Shepherd
69bb7fc4a1
Install lxd-agent-loader in bootable buildd image to improve user
experience with lxd vms
2021-03-01 09:04:26 -08:00
Cody Shepherd
61a422f553
Set grub configs to hide boot menu and enable output on ttyS0 2021-03-01 09:04:26 -08:00
Pat Viafore
0775862a25
Add mechanism to detect initrdless boot fallback
With this change, when we attempt to boot with an initramfs and fail,
initrdless_boot_fallback_triggered is set to non-zero in the grubenv.
This value can be checked after boot by looking in /boot/grub/grubenv
or by using the grub-editenv list command.

Addresses LP: #1870189
2021-02-18 15:47:20 -06:00
David Krauser
4030c8bc5a
Drop hack in kvm image that forces initramfs-less boot
Since the kvm image now boots without an initramfs by default, we
don't need to force it in the build hook.
2021-02-18 15:40:46 -06:00
David Krauser
c81f4ca2e3
Boot without an initramfs by default if running a custom kernel
Initramfs-less boot, which is a boot optimization, should only be
applied where we know it could work for users and provide an improved
boot boot experience; images with custom kernels are candidates for
that.
2021-02-18 15:40:45 -06:00
David Krauser
6a37833973
Boot with an initramfs by default in cloud images
Generic cloud images with the linux-generic kernel are not able to
boot without an initramfs. Previously, these images attempted to boot
without an initramfs, would fail, and then retry with an initramfs.
This slows the boot and is confusing behavior.
2021-02-18 15:40:42 -06:00
Dimitri John Ledkov
6b32cd5fb1
functions: stop removing systemd-detect-virt unconditionally in undivert_grub
One can call divert_grub; replace_kernel; undivert_grub. And
replace_kernel will call into force_boot_without_initramfs, which
under certain conditions can call divert_grub &
undivert_grub. Resulting in undivert_grub called twice in a row.

When undivert_grub is called twice in a row it wipes
systemd-detect-virt binary from disk, as the rm call is unguarded to
check that there is something to divert if systemd package is
installed. And if the systemd package is not installed, it does not
check that systemd-detect-virt file is in-fact what divert_grub has
created.

Add a guard to check that systemd-detect-virt is the placeholder one,
before removing it.

LP: #1902260
(cherry picked from commit 096a00f40459187719840ccad99e86c7ade2ec12)
2021-02-09 00:51:00 +00:00
Dimitri John Ledkov
e1bea4d00e
Perform fsck on all rootfs.
LP: #1912835
(cherry picked from commit 33b7272a2331279b4c7aa8de73ed8a9069d0bd39)
2021-02-09 00:50:52 +00:00
Dimitri John Ledkov
5c802316f2
esp: perform fsck.
LP: #1912835
(cherry picked from commit 53b28d5f58284657ee8efd08a633d557af1fcd97)
2021-02-09 00:50:46 +00:00
Dimitri John Ledkov
2646b9c459
esp: install grub in ubuntu bootloader id path, instead of removable.
Do not use removable uefi bootloader path in the cloud-images by
default, as that prevents upgrades of the bootloader.

LP: #1912830
(cherry picked from commit 7c760864fdcb278ca37396f06f5e3f297428d63d)
2021-02-09 00:50:38 +00:00
Dimitri John Ledkov
cc13644b6a
Merge branch 'dk/focal-buildd-unpack-kernel-initrd' of git+ssh://git.launchpad.net/~davidkrauser/livecd-rootfs/+git/livecd-rootfs into ubuntu/focal
(Cherry-pick commit 065c82314464fa78337d5122e1d4826a7d6edbb0)
2021-02-09 00:50:00 +00:00
Cody Shepherd
ff91fd7da4
Omit file type exensions in manifest names 2021-02-03 07:27:40 -08:00
Cody Shepherd
7af37be61a
Create manifests for buildd tarballs 2021-02-03 07:27:30 -08:00
Łukasz 'sil2100' Zemczak
befad641dc Support the lowlatency-hwe-* when handling kernel flavours. 2021-02-02 11:51:21 +01:00
Łukasz 'sil2100' Zemczak
e22a16b068 Enable the HWE kernel for all flavours. 2021-02-01 22:25:00 +01:00
Łukasz 'sil2100' Zemczak
e88f07a6e8 enable hwe variant 2021-01-26 15:46:11 +01:00
Dimitri John Ledkov
4be305fedc
chroot: disable amd64 codepath on riscv64 2021-01-15 17:22:01 +00:00
Dimitri John Ledkov
7680ece7b0
riscv64: backport HiFive unleashed & cloud-image building support
All commits are cherrypicks from hirsute, and includes:

make-lxd-metadata: add riscv64 lxd architecture tag
control: install qemu-utils & snapd on riscv64.
disk-image.binary: actually skip building MBR image on EFI platforms.
disk-image.binary: skip building MBR image on riscv64.
disk-image.binary: however still build MBR image on amd64.
disk-image-uefi.binary: build riscv64 image without a bootloader.
disk-image-uefi: riscv64 add u-boot spl
disk-image-uefi: riscv64 add ubuntu:ubuntu login, without expiry.
riscv64: build preinstalled riscv64 image with uboot SPL and CIDATA.
riscv64: fixup subarch build, unbound variable
qcow2-image: unbreak builds without subarch.
disk-image-uefi: fix riscv64 subarch user-data.

LP: #1903034
2021-01-15 17:06:44 +00:00
David Krauser
689631931e Produce buildd kernel and initrd as separate artifacts
Multipass on Mac OS X requires standalone kernel and initrd artifacts
to boot.

Also call update-initramfs on all installed kernels. We only have one
kernel installed, so we don't need to specify an explicit version.
2021-01-08 14:51:18 -05:00
Łukasz 'sil2100' Zemczak
4e8e1fed07 Ok, last commit didn't remove enough lines, sorry for that. 2020-12-03 18:21:57 +01:00
Łukasz 'sil2100' Zemczak
2029f49fe6 Still hardcode UC20 amd64 image size, but to the same value as for UC16 and UC18. 2020-12-03 12:22:31 +01:00
Łukasz 'sil2100' Zemczak
a4838e3245 Merge branch 'uc20-no-fixed-size' of git+ssh://git.launchpad.net/livecd-rootfs into ubuntu/focal 2020-11-27 17:59:08 +01:00
Robert C Jennings
172b217128
Merge focal-open-vm-tools-update into ubuntu/focal [a=jchittum] [r=rcj]
Backport vmtools version in vmdk (LP: #1893898)

Backport
LP: #1893898 describes missing vmtools version from the vmdk headers.
The version should be added as ddb.toolsVersion = "2147483647" however
the sed was no longer replacing a ddb.comment field with the tools
version. Rather than subbing ddb.comment with toolsVersion, this commit
deletes ddb.comment (which the comment mentions could cause errors),
and adds the correct value. There was no visibility into the descriptor
during hook creation, so debug statements were added. This allows us to
quickly verify in the logs that bad statements are removed (the possibly
offending comments), as well as ensuring that the toolsVersion is added

MP: https://code.launchpad.net/~jchittum/livecd-rootfs/+git/livecd-rootfs/+merge/394142
2020-11-20 14:35:25 -06:00
John Chittum
c5cea219f4
re-add sed command
Older version of vmdk-stream-converter has an incorrect header. The
original sed command replaced the incorrect "Description File" comment
with the correct "Disk DescriptorFile".
2020-11-17 15:40:45 -06:00
Joshua Powers
ef950f5214
amd64: always install grub-pc with shim-signed
shim-signed depends on grub-efi-amd64-signed, which in turn has
alternative depends on either `grub-efi-amd64 | grub-pc`. However to
support booting with either via shim&signed-grub and BIOS, the choice
must be made to install grub-pc, not grub-efi-amd64.

This makes images consistent with Ubuntu Deskop, Live Server, buildd
bootable images; all of which already do install grub-pc and
shim-signed.

Additionally, this will ensure that autoremove is run after installing
anything in the CPC build hooks. This is done to avoid shipping images
that include packages that are autoremovable. This will clean-up as
packages are installed and detect any breakage at build time.

LP: #1901906
2020-11-13 09:07:12 -08:00
John Chittum
56862cd9b8
Backport vmtools version in vmdk (LP: #1893898)
Backport
LP: #1893898 describes missing vmtools version from the vmdk headers.
The version should be added as ddb.toolsVersion = "2147483647" however
the sed was no longer replacing a ddb.comment field with the tools
version. Rather than subbing ddb.comment with toolsVersion, this commit
deletes ddb.comment (which the comment mentions could cause errors),
and adds the correct value. There was no visibility into the descriptor
during hook creation, so debug statements were added. This allows us to
quickly verify in the logs that bad statements are removed (the possibly
offending comments), as well as ensuring that the toolsVersion is added
2020-11-10 12:50:32 -06:00
Łukasz 'sil2100' Zemczak
433c2363d8 Do not hard-code the UC20 amd64 image size anymore 2020-11-03 11:30:23 +01:00
Dimitri John Ledkov
558bcbc47e
functions: provide nss_systemd-less nsswitch.conf in chroots.
(cherry picked from commit 4d4f113f42d6c3a58a484582af4d2c02f96a42bf)
2020-10-05 10:32:16 +01:00
Dimitri John Ledkov
2be2852ee0
auto/build: use setup|teardown_mountpoint to reduce duplication
(cherry picked from commit 109e6c6613aae4cbd4e8793709ab52c3cbce214c)
2020-10-05 10:32:15 +01:00
Robert C Jennings
e2ae9c245d
Avoid rbind /sys for chroot snap pre-seeding (cgroups fail to unmount)
Builds in LP with the Xenial kernel were happy with the recursive mount of
/sys inside the chroot while performing snap-preseeding but autopkgtests
with the groovy kernel failed.  With the groovy kernel the build was
unable to unmount sys/kernel/slab/*/cgroup/* (Operation not permitted).

This patch mounts /sys and /sys/kernel/security in the chroot in the
same way we've added for binary hooks.  This provides the paths under
/sys needed for snap-preseed while avoiding issues unmounting other
paths.

(cherry picked from commit 84397b50989670c2cfff01de23a5a73e67cd4088)
2020-10-05 10:32:14 +01:00