546 Commits

Author SHA1 Message Date
Thomas Bechtold
387ae191a1
add configure_oci function and use it in ubuntu-oci
With that, the Dockerfile modifications[0] currently done externally
are done now here. That means that the created rootfs tarball can be
directly used within a Dockerfile to create a container from scratch:

FROM scratch
ADD livecd.ubuntu-oci.rootfs.tar.gz /
CMD ["/bin/bash"]

[0]
https://github.com/tianon/docker-brew-ubuntu-core/blob/master/update.sh

(cherry picked from commit a81972a58b004897bf3e5c14ff371bc2f6b5e4b8)
2021-04-30 14:53:29 +02:00
Thomas Bechtold
bc6ae91a70
Add new ubuntu-oci project
This is a copy of the ubuntu-base project.
Currently ubuntu-base is used as a base for the docker/OCI container
images. The rootfs tarball that is created with ubuntu-base is
published under [0]. That tarball is used in the FROM statement of the
Dockerfile as base and then a couple of modifications are done inside
of the Dockerfile[1].
The ubuntu-oci project will include the changes that are currently
done in the Dockerfile. With that:

1) a Dockerfile using that tarball will be just a 2 line thing:

   FROM scratch
   ADD ubuntu-hirsute-core-cloudimg-amd64-root.tar.gz /
   CMD ["/bin/bash"]

2) Ubuntu has the full control about the build process of the
docker/OCI container. No external sources (like [1]) need to be
modified anymore.
3) Ubuntu can publish containers without depending on the official
dockerhub containers[2]. Currently the containers for the AWS ECR
registry[3] use as a base[4] the official dockerhub containers. That's
no longer needed because a container just needs a Dockerfile described
in 1)

When the ubuntu-oci project has the modifications from [1] included,
we'll also update [1] to use the ubuntu-oci rootfs tarball as a base
and drop the modifications done at [1].

Note: Creating a new ubuntu-oci project instead of using ubuntu-base
will make sure that we don't break users who are currently using
ubuntu-base rootfs tarballs for doing their own thing.

[0] https://partner-images.canonical.com/core/
[1]
https://github.com/tianon/docker-brew-ubuntu-core/blob/master/update.sh
[2] https://hub.docker.com/_/ubuntu
[3] https://gallery.ecr.aws/ubuntu/ubuntu
[4]
https://launchpad.net/~ubuntu-docker-images/ubuntu-docker-images/+oci/ubuntu/+recipe/ubuntu-20.04

(cherry picked from commit ac4a95b9314cf1f8ce01f42016c271c0a6078372)
2021-04-30 14:53:24 +02:00
Łukasz 'sil2100' Zemczak
69195a0026 Build appliance qcow2 images only for amd64 appliances. 2020-12-11 10:16:43 +01:00
Łukasz 'sil2100' Zemczak
0e35dfdb46 We need to do it in a different order. 2020-12-05 01:10:12 +01:00
Łukasz 'sil2100' Zemczak
a976382d9a Try building qcow2 images for appliances. 2020-12-04 12:34:40 +01:00
Joshua Powers
d5a2099c33
amd64: always install grub-pc with shim-signed
shim-signed depends on grub-efi-amd64-signed, which in turn has
alternative depends on either `grub-efi-amd64 | grub-pc`. However to
support booting with either via shim&signed-grub and BIOS, the choice
must be made to install grub-pc, not grub-efi-amd64.

This makes images consistent with Ubuntu Deskop, Live Server, buildd
bootable images; all of which already do install grub-pc and
shim-signed.

Additionally, this will ensure that autoremove is run after installing
anything in the CPC build hooks. This is done to avoid shipping images
that include packages that are autoremovable. This will clean-up as
packages are installed and detect any breakage at build time.

LP: #1901906
2020-11-13 09:13:59 -08:00
Łukasz 'sil2100' Zemczak
70d675e5ab Apparently the lxd appliance needs to use a custom track, so hack it in. 2020-08-13 14:52:30 +02:00
Francis Ginther
db2e54491a Switch raspi3 builds to the linux-raspi-hwe-18.04 kernel
As part of the backport of the 5.4 kernel for Raspberry Pi, the kernel
flavour name is changing to match that in Focal. This is to provide a
consistent name for the 5.4 kernel in both Bionic and Focal.
This effectively rolls the kernel from 5.3 to 5.4 for raspi classic
images.
2020-07-20 16:13:03 -05:00
Łukasz 'sil2100' Zemczak
1c4a376ced Remove unneeded change. 2020-07-15 16:16:24 +02:00
Łukasz 'sil2100' Zemczak
294db12c82 We need to handle appliance images first before everything else. 2020-06-05 01:52:46 +02:00
Łukasz 'sil2100' Zemczak
037c4e4ac6 Do it even better. 2020-06-04 21:48:05 +02:00
Łukasz 'sil2100' Zemczak
61eae75d5e Support building arbitraty named appliance images. 2020-06-04 21:18:05 +02:00
Łukasz 'sil2100' Zemczak
467065461b The appliance model names have changed and we now use 'pi' instead of 'pi-arm64' 2020-05-26 19:20:43 +02:00
Łukasz 'sil2100' Zemczak
650cced79e Add support for uc18 appliance images. 2020-05-07 18:39:32 +02:00
David Krauser
f60c01ef0b Don't install live packages in buildd images. 2020-03-25 18:23:10 -04:00
Cody Shepherd
075d1e85bf Add hooks to build bootable buildd images
Only install policyrd-script-zg2 in lxd tarball
2020-03-23 13:27:19 -04:00
Steve Langasek
d516e68807 Stop building per-pi-flavor images and only build a pi generic image for core18. (LP: #1861520) 2020-02-27 12:17:28 -08:00
Łukasz 'sil2100' Zemczak
efd428856b Switch raspi2-hwe-18.04-edge to raspi2-hwe-18.04. 2020-01-22 09:29:37 +01:00
Łukasz 'sil2100' Zemczak
89bcec68c5 Support generating a .disk/info file via ubuntu-image. 2020-01-22 09:20:21 +01:00
Łukasz 'sil2100' Zemczak
b47acc9456 Actually, modify 2.525.37 and re-release since the kernel flavour name was wrong. 2020-01-14 10:29:23 +01:00
Łukasz 'sil2100' Zemczak
38cfb82a84 Use the 5.3 kernel for the bionic raspi3 images. 2020-01-13 16:14:12 +01:00
Michael Hudson-Doyle
d381420065 Preserve apt preferences created by any package we install (i.e. ubuntu-advantage-tools) against live-build's attempt to delete them. (LP: #1855354) 2019-12-19 15:45:17 +13:00
Łukasz 'sil2100' Zemczak
42f2ccd60a Enable core18 builds 2019-11-12 21:20:59 +01:00
Łukasz 'sil2100' Zemczak
f212eb9a73 Merge branch 'hyperv-bionic' of git+ssh://git.launchpad.net/~davidkrauser/livecd-rootfs/+git/livecd-rootfs into ubuntu/bionic 2019-11-12 21:17:15 +01:00
Łukasz 'sil2100' Zemczak
98f52a0f7b Support amd64+kassel in bionic 2019-10-30 18:12:18 +01:00
Łukasz 'sil2100' Zemczak
f38122e070 Support EXTRA_SNAPS 2019-10-23 16:02:36 +02:00
Łukasz 'sil2100' Zemczak
ce1b1cf815 Backport the 999-ubuntu-image-customization changes, add a delta on top to still support the old legacy raspi2 images. 2019-10-23 12:41:40 +02:00
Łukasz 'sil2100' Zemczak
f8dd8502ce Fix typo in auto/build regarding core builds. 2019-10-22 17:17:38 +02:00
David Krauser
78aa0844cf Fixing broken binary hook symlinks for hyperv gallery image builds. 2019-10-09 16:05:14 -04:00
Dimitri John Ledkov
f16cac1842 Unbreak cpc/server builds, by ensuring that includes are copied with archived symlinks, and only hooks resolve symlinks when getting copied. 2019-10-09 16:05:14 -04:00
David Krauser
335470331c Add support for HyperV Gallery images 2019-10-09 16:05:14 -04:00
Robert C Jennings
dda6e5953a
magic-proxy: dump proxy log to stdout on failure (LP: #1847300)
When we encounter a failure in 'lb binary' the launchpad builders can
only surface the build output from stdout.  If the binary hook failure
implicates the archive we can not determine fault without the apt
proxy log.  This patch will dump the proxy log to stdout to aid in
debugging these failures.
2019-10-08 11:00:41 -05:00
Łukasz 'sil2100' Zemczak
73e3864f25 Backport the changes for Nitrogen6x board support. 2019-09-10 16:57:24 +02:00
Robert C Jennings
39fb950de2
ubuntu-cpc: rework image artifact cleanup
The livecd.ubuntu-cpc.ext4 that is present in each build (plus kernel
and initrd) are not renamed from /build/binary/boot/filsystem.ext4
and friends until after the binary hooks are run, so this patch moves
from trying to perform this cleanup in a binary hook.  Now the cleanup
will be run at the end of live-build/binary for the ubuntu-cpc project.
2019-08-30 12:38:37 -05:00
Steve Langasek
315516a372 Strip translation files out of the minimal images.
This is another thing that goes unused when there is no human console user
(and we already don't have the locales themselves present on a minimal
image).

Bug-Ubuntu: https://bugs.launchpad.net/bugs/1829333
2019-05-28 08:52:31 -07:00
Robert C Jennings
2742b06211
ubuntu-cpc: parallel builds: apt repo snapshot
* Inject a proxy into the build providing a snapshot view of the package repo.

    When the REPO_SNAPSHOT_STAMP variable is set, the auto/build script will attempt
    to launch a transparent HTTP proxy on port 8080, and insert an iptables rule to
    redirect all outgoing HTTP requests to this proxy.

    The proxy, contained in the `magic-proxy` Python script, examines each request
    and silently overrides those pointing to InRelease files or files that are
    listed in InRelease files. It will instead provide the contents of the requested
    file as it was at REPO_SNAPSHOT_STAMP, by downloading the corresponding asset
    "by hash".
2019-05-21 15:52:06 -05:00
Robert C Jennings
7907544390
ubuntu-cpc: parallel builds: build hooks during config
* Use series files with dependency handling to generate hook symlinks dynamically

    This patch currently only applies to the "ubuntu-cpc" project.

    More and more logic has been going into the hook scripts to decide
    under which conditions they should run or not. As we are moving
    to parallelized builds of image sets, this will get even more
    complicated. Base hooks will have to know which image sets they
    belong to and modification of the dependency chain between scripts
    will become more complicated and prone to errors, as the number of
    image sets grows.

    This patch, and prior, introduce explicit ordering and dependency
    handling for scripts through the use of `series` files and an
    explicit syntax for dependency specification.
2019-05-21 15:52:06 -05:00
Michael Hudson-Doyle
223bf910df Backport two minimizations for the docker images: remove apt lists that are removed downstream anyway, and remove device nodes from the image. (LP: #1828118) 2019-05-08 11:03:42 +12:00
Julian Andres Klode
87a26d2cd2 Remove crufty files after minimize-manual
minimize-manual would be creating those files again, and other
parts may be creating them, so best to do the cleanup at the
very end.

LP: #1826377
2019-04-26 10:40:05 +02:00
Julian Andres Klode
be180bcf93 Minimize the number of manually installed packages
Minimize the number of manually installed packages in images by marking
dependencies of metapackages as automatically installed.

Squashed commits:
* minimize-manual.py: Followup with some cleanup, correct permissions
* fixup: Do not assume current directory
* Reorganize minimal-manual file location.
2019-03-11 11:32:36 +01:00
Adam Conrad
1003e76191 [mwhudson, r=adconrad] Merge mwhudson/live-server-bionic for live-server HWE 2019-02-12 11:43:33 -07:00
Colin Watson
a4fd32a7f1 Drop code to handle buildd images for != bionic 2019-02-05 14:24:09 +00:00
Colin Watson
2adceb461e Move buildd image building to binary hooks 2019-02-05 14:21:02 +00:00
Colin Watson
71ec79e55d Add a LXD image to builds for the buildd subproject 2019-02-05 14:20:37 +00:00
Colin Watson
e351909818 Add a buildd subproject 2019-02-05 14:20:18 +00:00
Adam Conrad
0ccce31ed5 Fixes to make the lubuntu livefs behave when swapping to the HWE stack.
* lubuntu: Select lubuntu-gtk-core task as well, so we get the HWE filter
  applied to the explicit dependencies it has on Xorg-recommended packages.
* lubuntu: Manually install some packages that fall out due to the above.
2019-02-05 00:28:48 -07:00
Adam Conrad
bd5f3fcd3f Forward-port HWE support from xenial, make it more generic, and enable it for the current LTS flavours. 2019-02-02 03:24:54 -07:00
Łukasz 'sil2100' Zemczak
86f2505871 Use linux-firmware-raspi2 instead of raspi3-firmware. 2019-01-18 17:08:20 +01:00
Łukasz 'sil2100' Zemczak
6b63fa210f Cherry-pick all the additionally needed fixes to enable proper raspi3 support. 2019-01-11 15:28:11 +01:00
Michael Hudson-Doyle
a473683ac7 Do no install openssh-server in the base filsystem for the live server installer. 2018-12-18 15:11:17 +13:00