diff --git a/README.parameters b/README.parameters new file mode 100644 index 00000000..e51f6877 --- /dev/null +++ b/README.parameters @@ -0,0 +1,236 @@ +Understanding the parameters used by livecd-rootfs +================================================== + +livecd-rootfs is a confusing codebase. One of the confusing things is +how information flows into and around the image build process. There +is IMAGEFORMAT and IMAGE_TARGETS and PROJECT and many other +variables. It is not obvious when looking at the code if a given +variable is something passed as a parameter or something derived from +it. + +All (or almost all) production use of livecd-rootfs is via +launchpad-buildd so the set of potential parameters is limited by the +set of environment variables launchpad-build can set in response to +the build request. + +The process from build request to environment live-build is run is a +little convoluted. The build request takes: + + an archive -- where to get livecd-rootfs from + a distro_arch_series -- the series to get livecd-rootfs and build + a pocket -- pocket to get livecd-rootfs from, also influences if proposed is + used as a package source for the image being built + unique_key -- you cannot have more than one pending livefs build with the same + unique_key. does not affect the build at all. + version -- optional version string, see below. often a serial like 20250525.1 + metadata_override -- combined with the metadata on the livefs itself to make + the metadata for this build. + +(ref: https://launchpad.net/+apidoc/devel.html#livefs-requestBuild) + +These parameters are stored on the livefsbuild object (ref: +https://git.launchpad.net/launchpad/tree/lib/lp/soyuz/model/livefsbuild.py#n372) +and converted into a set of args passed to launchpad-build by the +LiveFSBuildBehaviour class (ref: +https://git.launchpad.net/launchpad/tree/lib/lp/soyuz/model/livefsbuildbehaviour.py#n99). + +Inside launchpad-build, these arguments are inspected by the +LiveFilesystemBuildManager.initiate method (ref: +https://git.launchpad.net/launchpad-buildd/tree/lpbuildd/livefs.py#n24) +which turns them into arguments for the BuildLiveFS lpbuild +"operation" which is what creates the environment live-build runs in +(ref: +https://git.launchpad.net/launchpad-buildd/tree/lpbuildd/target/build_livefs.py#n167). + +These variables can be set for both lb config and lb build: + + PROJECT (mandatory, comes from "project" in the metadata) + ARCH (set to the abi tag of the distroarchseries being built for) + SUBPROJECT (optional, comes from "subproject" in the metadata) + SUBARCH (optional, comes from "subarch" in the metadata) + CHANNEL (optional, comes from "subarch" in the metadata) + IMAGE_TARGETS (optional, comes from "image_targets" in the metadata + "image_targets" is a list. IMAGE_TARGETS is set to " ".join(image_targets)) + REPO_SNAPSHOT_STAMP + (optional, comes from "repo_snapshot_stamp" in the metadata) + SNAPSHOT_SERVICE_TIMESTAMP + (optional, comes from "snapshot_snapshot_stamp" in the metadata) + COHORT_KEY + (optional, comes from "cohort-key" in the metadata) + +launchpad-buildd also contains code to set http_proxy / HTTP_PROXY / +LB_APT_HTTP_PROXY but there does not appear to be any way to trigger +this when requesting a build. + +In addition the following variables can be set for lb config only (why +are some things set for lb config only? no idea): + + SUITE (set to the name of the distroarchseries being built for) + NOW (set to value of the 'version' argument to the build request, + defaults to strftime("%Y%m%d-%H%M%S")) + IMAGEFORMAT (optional, comes from "image_format" in the metadata) + PROPOSED (set to "1" if the pocket passed to the build request is proposed) + EXTRA_PPAS (optional, comes from "extra_ppas" in the metadata + "extra_ppas" is a list. EXTRA_PPAS is set to " ".join(extra_ppas)) + EXTRA_SNAPS (optional, comes from "extra_snaps" in the metadata + "extra_snaps" is a list. EXTRA_SNAPS is set to " ".join(extra_snaps)) + +Here is an opinionated and slightly angry attempt to describe what +each of these is for: + +PROJECT +------- + +This is the big one, the main variable that defines what is being +built. It can be ubuntu, ubuntu-server, xubuntu, ubuntu-mini-iso, that +sort of thing. Generally PROJECT determines the set of packages +installed but it (unfortunately?) has a bit more impact than that. + +It's unarguable that we need a parameter like this. + +ARCH +---- + +The architecture being built for. This is always the same as `dpkg +--print-architecture` for us, we don't do any cross builds. + +It's kind of redundant but it's not really a problem that this exists. + +SUBPROJECT +---------- + +This is used for some builds to build a different sort of build of the +project. It can be set to: + + * "minimized" for ubuntu-cpc builds to make a minimal cloud image + * "minimal" for xubuntu builds to make a smaller ISO + * "desktop-preinstalled" for ubuntu builds to make a preinstalled + image instead of the parts for an installer. + * "buildd" for images to be used as build images by craft tools, and also + buildd chroots used on launchpad builders? + * "live" for ubuntu-server builds, historically to distinguish d-i + style installers from subiquity style installers + * "desktop" for ubuntu-core-installer builds, to influence which + model is use to build the ubuntu core system that will be + installed. + +_This_ parameter is a total mess. The desktop-preinstalled use feels +particularly egregious. + +SUBARCH +------- + +This identifies the target machine more specifically than ARCH, +e.g. "tegra-jetson" or "licheerv". Used mostly but not exclusively for +preinstalled builds. + +We probably do need something like this. + +CHANNEL +------- + +Influences which channel snaps included in the build are taken from +(via a few different mechanisms). + +IMAGE_TARGETS +------------- + +Passed for CPC (and ubuntu-oem, for some reason) builds to +`config/hooks.d/make-hooks` which uses it to select which binary hooks +to run (and so determines which artifacts get produced). + +It is probably reasonable that this exists. + +REPO_SNAPSHOT_STAMP +------------------- + +Currently unused. + +SNAPSHOT_SERVICE_TIMESTAMP +-------------------------- + +Also currently unused, and unclear how it differs from +REPO_SNAPSHOT_STAMP. + +COHORT_KEY +---------- + +Used to make sure that different builds run at the same time don't get +different versions of snaps due to phasing differences. + +This is a totally valid thing to need to supply. + +http_proxy / HTTP_PROXY / LB_APT_HTTP_PROXY +------------------------------------------- + +Nothing complex here! + +SUITE +----- + +This is the series being built (e.g. noble, questing). It is misnamed +really -- a suite is usually a combination of a series and a pocket +(noble-proposed, questing-security). + +As with ARCH this is sort of redundant as we do builds in a chroot of +the series being built but OTOH it is definitely information the +build needs to know! + +NOW +--- + +The serial for the build, e.g. 20250519 or 20240418.4. + +It is a totally reasonable parameter. + +IMAGEFORMAT +----------- + +This is one of the more incoherently handled parameters. In rough +outline it is the filesystem of the image we produce. + +Installer builds do not produce raw images, so this ends up being set +to 'plain' (which causes live-build to just leave the rootfs as a +directory tree) or 'none' (which causes live-build to do roughly the +same thing but in a different way?). + +Image builds that use ubuntu-image set it to "ubuntu-image". These +builds do not call 'lb build' or 'lb binary'. + +Other preinstalled images (mostly cpc images) set it to ext4 (but then +use live-build/ubuntu-cpc/hooks.d/remove-implicit-artifacts to remove +the output file that this causes live-build to produce...). Some +projects rely on this being set via metadata when building the project +it seems. + +It can be set when starting an image build, but most builds do not and +the behavior when it is not set explicitly is pretty confusing. + +This place is not a place of honor. + +PROPOSED +-------- + +Should packages from proposed by included? + +This is not really as useful as it used to be for a bunch of reasons +but it conceptually makes sense. + +EXTRA_PPAS +---------- + +Extra archives to get packages from. + +This is a space separated list by the time it gets to +livecd-rootfs. Each element of the list is of the form USER/NAME[:PIN] +where user is a Launchpad user/team name, NAME is the name of the ppa +to add and the optional colon-PIN at the end is the value to pin (in +the "man 5 apt_preferences: sense) packages from this PPA at. + +Production builds shouldn't really use this but it's definitely useful +for development. + +EXTRA_SNAPS +----------- + +Extra snaps to include (but only for ubuntu-image based builds).