From 5e00e3ecb253521c8f30303c64e922638601487c Mon Sep 17 00:00:00 2001 From: Michael Hudson-Doyle Date: Tue, 25 Nov 2025 16:46:36 +1300 Subject: [PATCH 1/4] add some kind of documentation of the parameters livecd-rootfs takes --- README.parameters | 180 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 180 insertions(+) create mode 100644 README.parameters diff --git a/README.parameters b/README.parameters new file mode 100644 index 00000000..741c53ef --- /dev/null +++ b/README.parameters @@ -0,0 +1,180 @@ +Understanding the parameters used by livecd-rootfs +================================================== + +livecd-rootfs is a confusing codebase. One of the confusing things is +how information flows into and around the image build process. There +is IMAGEFORMAT and IMAGE_TARGETS and PROJECT and many other +variables. It is not obvious when looking at the code if a given +variable is something passed as a parameter or something derived from +it. + +All (or almost all) production use of livecd-rootfs is via +launchpad-buildd so the set of potential parameters is limited by the +set of environment variables launchpad-build can set in response to +the build request. These variables can be set for both lb config and +lb build: + + PROJECT + ARCH + SUBPROJECT + SUBARCH + CHANNEL + IMAGE_TARGETS + REPO_SNAPSHOT_STAMP + SNAPSHOT_SERVICE_TIMESTAMP + COHORT_KEY + http_proxy / HTTP_PROXY / LB_APT_HTTP_PROXY + +In addition the following variables can be set for lb config only (why +are some things set for lb config only? no idea): + + SUITE + NOW + IMAGEFORMAT + PROPOSED + EXTRA_PPAS + EXTRA_SNAPS + +Here is an opinionated and slightly angry attempt to describe what +each of these is for: + +PROJECT +------- + +This is the big one, the main variable that defines what is being +built. It can be ubuntu, ubuntu-server, xubuntu, ubuntu-mini-iso, that +sort of thing. Generally PROJECT determines the set of packages +installed but it (unfortunately?) has a bit more impact than that. + +It's unarguable that we need a parameter like this. + +ARCH +---- + +The architecture being built for. This is always the same as `dpkg +--print-architecture` for us, we don't do any cross builds. + +It's kind of redundant but it's not really a problem that this exists. + +SUBPROJECT +---------- + +This is used for some builds to build a different sort of build of the +project. It can be set to: + + * "minimized" for ubuntu-cpc builds to make a minimal cloud image + * "minimal" for xubuntu builds to make a smaller ISO + * "desktop-preinstalled" for ubuntu builds to make a preinstalled + image instead of the parts for an installer. + * "buildd" for images to be used as build images by craft tools, and also + buildd chroots used on launchpad builders? + * "live" for ubuntu-server builds, historically to distinguish d-i + style installers from subiquity style installers + * "desktop" for ubuntu-core-installer builds, to influence which + model is use to build the ubuntu core system that will be + installed. + +_This_ parameter is a total mess. The desktop-preinstalled use feels +particularly egregious. + +SUBARCH +------- + +This identifies the target machine more specifically than ARCH, +e.g. "tegra-jetson" or "licheerv". Used mostly but not exclusively for +preinstalled builds. + +We probably do need something like this. + +CHANNEL +------- + +Influences which channel snaps included in the build are taken from +(via a few different mechanisms). + +IMAGE_TARGETS +------------- + +Passed for CPC (and ubuntu-oem, for some reason) builds to +`config/hooks.d/make-hooks` which uses it to select which binary hooks +to run (and so determines which artifacts get produced). + +It is probably reasonable that this exists. + +REPO_SNAPSHOT_STAMP +------------------- + +Currently unused. + +SNAPSHOT_SERVICE_TIMESTAMP +-------------------------- + +Also currently unused, and unclear how it differs from +REPO_SNAPSHOT_STAMP. + +COHORT_KEY +---------- + +Used to make sure that different builds run at the same time don't get +different versions of snaps due to phasing differences. + +This is a totally valid thing to need to supply. + +http_proxy / HTTP_PROXY / LB_APT_HTTP_PROXY +------------------------------------------- + +Nothing complex here! + +SUITE +----- + +This is the series being built (e.g. noble, questing). It is misnamed +really -- a suite is usually a combination of a series and a pocket +(noble-proposed, questing-security). + +As with ARCH this is sort of redundant as we do builds in a chroot of +the series being built but OTOH it is definitely information the +build needs to know! + +NOW +--- + +The serial for the build, e.g. 20250519 or 20240418.4. + +It is a totally reasonable parameter. + +IMAGEFORMAT +----------- + +This is one of the more incoherently handled parameters. In rough +outline it is the filesystem of the image we produce, but relatively +few builds produce raw images so _mostly_ it is set to 'plain' (which +causes live-build to just leave the rootfs as a directory tree) or +'none' (which causes live-build to do roughly the same thing but in a +different way?). + +It can be set when starting an image build, but most builds do not and +the behavior when it is not set explicitly is pretty confusing. + +This place is not a place of honor. + +PROPOSED +-------- + +Should packages from proposed by included? + +This is not really as useful as it used to be for a bunch of reasons +but it conceptually makes sense. + +EXTRA_PPAS +---------- + +Extra archives to get packages from. + +Production builds shouldn't really use this but it's definitely useful +for development. + +EXTRA_SNAPS +----------- + +Extra snaps to include (but only for ubuntu-image based builds). From 65dad6ccc0cb229172cd8584fda712f9c1ed1045 Mon Sep 17 00:00:00 2001 From: Michael Hudson-Doyle Date: Tue, 2 Dec 2025 18:42:47 +1300 Subject: [PATCH 2/4] be a bit more accurate about IMAGEFORMAT --- README.parameters | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/README.parameters b/README.parameters index 741c53ef..40cb3cdf 100644 --- a/README.parameters +++ b/README.parameters @@ -147,11 +147,21 @@ IMAGEFORMAT ----------- This is one of the more incoherently handled parameters. In rough -outline it is the filesystem of the image we produce, but relatively -few builds produce raw images so _mostly_ it is set to 'plain' (which -causes live-build to just leave the rootfs as a directory tree) or -'none' (which causes live-build to do roughly the same thing but in a -different way?). +outline it is the filesystem of the image we produce. + +Installer builds do not produce raw images, so this ends up being set +to 'plain' (which causes live-build to just leave the rootfs as a +directory tree) or 'none' (which causes live-build to do roughly the +same thing but in a different way?). + +Image builds that use ubuntu-image set it to "ubuntu-image". These +builds do not call 'lb build' or 'lb binary'. + +Other preinstalled images (mostly cpc images) set it to ext4 (but then +use live-build/ubuntu-cpc/hooks.d/remove-implicit-artifacts to remove +the output file that this causes live-build to produce...). Some +projects rely on this being set via metadata when building the project +it seems. It can be set when starting an image build, but most builds do not and the behavior when it is not set explicitly is pretty confusing. From 562e589cd1af11585b6a6c785cacf16628430f7f Mon Sep 17 00:00:00 2001 From: Michael Hudson-Doyle Date: Tue, 9 Dec 2025 09:50:33 +1300 Subject: [PATCH 3/4] include more information about how the parameters get from request to build --- README.parameters | 70 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 55 insertions(+), 15 deletions(-) diff --git a/README.parameters b/README.parameters index 40cb3cdf..42643dd7 100644 --- a/README.parameters +++ b/README.parameters @@ -11,29 +11,69 @@ it. All (or almost all) production use of livecd-rootfs is via launchpad-buildd so the set of potential parameters is limited by the set of environment variables launchpad-build can set in response to -the build request. These variables can be set for both lb config and -lb build: +the build request. - PROJECT - ARCH - SUBPROJECT - SUBARCH - CHANNEL - IMAGE_TARGETS +The process from build request to environment live-build is run is a +little convoluted. The build request takes: + + an archive -- where to get livecd-rootfs from + a distro_arch_series -- the series to get livecd-rootfs and build + a pocket -- pocket to get livecd-rootfs from, also influences if proposed is + used as a package source for the image being built + unique_key -- you cannot have more than one pending livefs build with the same + unique_key. does not affect the build at all. + version -- optional version string, see below. often a serial like 20250525.1 + metadata_override -- combined with the metadata on the livefs itself to make + the metadata for this build. + +(ref: https://launchpad.net/+apidoc/devel.html#livefs-requestBuild) + +These parameters are stored on the livefsbuild object (ref: +https://git.launchpad.net/launchpad/tree/lib/lp/soyuz/model/livefsbuild.py#n372) +and converted into a set of args passed to launchpad-build by the +LiveFSBuildBehaviour class (ref: +https://git.launchpad.net/launchpad/tree/lib/lp/soyuz/model/livefsbuildbehaviour.py#n99). + +Inside launchpad-build, these arguments are inspected by the +LiveFilesystemBuildManager.initiate method (ref: +https://git.launchpad.net/launchpad-buildd/tree/lpbuildd/livefs.py#n24) +which turns them into arguments for the BuildLiveFS lpbuild +"operation" which is what creates the environment live-build runs in +(ref: +https://git.launchpad.net/launchpad-buildd/tree/lpbuildd/target/build_livefs.py#n167). + +These variables can be set for both lb config and lb build: + + PROJECT (mandatory, comes from "project" in the metadata) + ARCH (set to the abi tag of the distroarchseries being built for) + SUBPROJECT (optional, comes from "subproject" in the metadata) + SUBARCH (optional, comes from "subarch" in the metadata) + CHANNEL (optional, comes from "subarch" in the metadata) + IMAGE_TARGETS (optional, comes from "image_targets" in the metadata + "image_targets" is a list. IMAGE_TARGETS is set to " ".join(image_targets)) REPO_SNAPSHOT_STAMP + (optional, comes from "repo_snapshot_stamp" in the metadata) SNAPSHOT_SERVICE_TIMESTAMP + (optional, comes from "snapshot_snapshot_stamp" in the metadata) COHORT_KEY - http_proxy / HTTP_PROXY / LB_APT_HTTP_PROXY + (optional, comes from "cohort-key" in the metadata) + +launchpad-buildd also contains code to set http_proxy / HTTP_PROXY / +LB_APT_HTTP_PROXY but there does not appear to be any way to trigger +this when requesting a build. In addition the following variables can be set for lb config only (why are some things set for lb config only? no idea): - SUITE - NOW - IMAGEFORMAT - PROPOSED - EXTRA_PPAS - EXTRA_SNAPS + SUITE (set to the name of the distroarchseries being built for) + NOW (set to value of the 'version' argument to the build request, + defaults to strftime("%Y%m%d-%H%M%S")) + IMAGEFORMAT (optional, comes from "image_format" in the metadata) + PROPOSED (set to "1" if the pocket passed to the build request is proposed) + EXTRA_PPAS (optional, comes from "extra_ppas" in the metadata + "extra_ppas" is a list. EXTRA_PPAS is set to " ".join(extra_ppas)) + EXTRA_SNAPS (optional, comes from "extra_snaps" in the metadata + "extra_snaps" is a list. EXTRA_SNAPS is set to " ".join(extra_snaps)) Here is an opinionated and slightly angry attempt to describe what each of these is for: From 827d87bd7f04e7fea9f83886f585b1374098c500 Mon Sep 17 00:00:00 2001 From: Michael Hudson-Doyle Date: Tue, 9 Dec 2025 12:00:55 +1300 Subject: [PATCH 4/4] document format of EXTRA_PPAS a bit --- README.parameters | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.parameters b/README.parameters index 42643dd7..e51f6877 100644 --- a/README.parameters +++ b/README.parameters @@ -221,6 +221,12 @@ EXTRA_PPAS Extra archives to get packages from. +This is a space separated list by the time it gets to +livecd-rootfs. Each element of the list is of the form USER/NAME[:PIN] +where user is a Launchpad user/team name, NAME is the name of the ppa +to add and the optional colon-PIN at the end is the value to pin (in +the "man 5 apt_preferences: sense) packages from this PPA at. + Production builds shouldn't really use this but it's definitely useful for development.