When we encounter a failure in 'lb binary' the launchpad builders can
only surface the build output from stdout. If the binary hook failure
implicates the archive we can not determine fault without the apt
proxy log. This patch will dump the proxy log to stdout to aid in
debugging these failures.
The livecd.ubuntu-cpc.ext4 that is present in each build (plus kernel
and initrd) are not renamed from /build/binary/boot/filsystem.ext4
and friends until after the binary hooks are run, so this patch moves
from trying to perform this cleanup in a binary hook. Now the cleanup
will be run at the end of live-build/binary for the ubuntu-cpc project.
In parallel builds where a list of image targets are provided the build
may produce binaries that are not part of the named set of targets but
are created by series dependencies. These implicitly created binaries
may be generated by multiple builds but are unused as our convention for
the ubuntu-cpc project is to only consume binaries from the explicitly
named image targets; this avoid overwriting the same object by multiple
parallel builds.
This patch adds support for a 'provides' keyword for series files. It can
be specified multiple times per series file. The field is used by the
make-hooks script to generate a list of output files created explicitly by
the named image targets. The list is saved to the "explicit_provides"
file in the hooks output directory. In the case of the "all" target
this list would be empty. This list is consumed by the "final.binary"
hook file.
This patch adds support for optional final.binary hooks in hooks.d/base
and/or hooks.d/extra. These final.binary hooks are always included as
the last hook(s) if either exist with the hook in "extra" running last.
The base/final.binary hook includes logic to parse the "explicit_provides"
file generated by the make-hooks script and remove any binary output not
explicitly specified.
Some series files named unnecessary dependencies, specifically
disk-image, to keep output of implicit artifacts consistent between
parallel builds. These unnecessary dependencies are removed in this
patch.
This is apparently still required; without it snapd gives an error
message like:
stateengine.go:102: state ensure error: devicemgr: cannot proceed without seeding "core"
(cherry picked from commit 7f07cc8991)
This is another thing that goes unused when there is no human console user
(and we already don't have the locales themselves present on a minimal
image).
Bug-Ubuntu: https://bugs.launchpad.net/bugs/1829333
The following targets have livecd.ubuntu-cpc.manifest (and
livecd.ubuntu-cpc.ext4) which differ in some way from the 'all'
target. They are all missing grub-efi and other modifications:
root-dir
squashfs
tarball
These targets do not depend on the 'disk-image' target. This means that
the ext4 produced will lack the uefi modifications (and any from the
disk-image target binary hooks).
Since the ext4 file is common to all builds there is a chance that a
parallel build from one of these targets could overwrite this artifact.
This patch ensures that all targets will produce consistent base output.
An upcoming addition of the disk-image target to several series files will
required some explanation. Without comment support in series files that
is not possible. This patch adds support for comments in series files.
* Inject a proxy into the build providing a snapshot view of the package repo.
When the REPO_SNAPSHOT_STAMP variable is set, the auto/build script will attempt
to launch a transparent HTTP proxy on port 8080, and insert an iptables rule to
redirect all outgoing HTTP requests to this proxy.
The proxy, contained in the `magic-proxy` Python script, examines each request
and silently overrides those pointing to InRelease files or files that are
listed in InRelease files. It will instead provide the contents of the requested
file as it was at REPO_SNAPSHOT_STAMP, by downloading the corresponding asset
"by hash".
* Use series files with dependency handling to generate hook symlinks dynamically
This patch currently only applies to the "ubuntu-cpc" project.
More and more logic has been going into the hook scripts to decide
under which conditions they should run or not. As we are moving
to parallelized builds of image sets, this will get even more
complicated. Base hooks will have to know which image sets they
belong to and modification of the dependency chain between scripts
will become more complicated and prone to errors, as the number of
image sets grows.
This patch, and prior, introduce explicit ordering and dependency
handling for scripts through the use of `series` files and an
explicit syntax for dependency specification.
Whilst configs are sources, the variables are not exported into
the environment by default. Thus when casper hook is executed, it had
no idea that it should generate UUID.
Remove wants from local-fs.target, add wants from the uuid device instead.
Tested by applying these changes from pre-pivot-root by modifying
/root files in place whilst booting with break=bottom.
Minimize the number of manually installed packages in images by marking
dependencies of metapackages as automatically installed.
Squashed commits:
* minimize-manual.py: Followup with some cleanup, correct permissions
* fixup: Do not assume current directory
* Reorganize minimal-manual file location.
Installing policyrcd-script-zg2 doesn't quite do this because of the way
that live-build installs its own temporary version of policy-rc.d. The
only remotely sensible way I can see to deal with this is to create the
symlink manually.
* lubuntu: Select lubuntu-gtk-core task as well, so we get the HWE filter
applied to the explicit dependencies it has on Xorg-recommended packages.
* lubuntu: Manually install some packages that fall out due to the above.
- Do not run apt-get update (which can bring in package lists if we are
unlucky wrt publisher schedules).
- Run apt-get clean to clear out downloaded debs of curtin/casper and
dependencies.
- Do not install user-setup.
- Use the core snap from the base filesystem if present.
* ubuntu-cpc: Generate the root image contents once, and use it for both the
-root.tar.xz and the .squashfs.
* ubuntu-cpc: Generate -root.tar.xz with --xattrs.
There can only be one model assertion, so we don't need to disambiguate them.
This also brings us in line with the behaviour of `snap prepare-image`, and
consistency is nice.
This also splits up the preparation in to two functions, so that images that
want to use a custom model assertion but don't have any snaps to preinstall
don't end up with the core snap installed.
Łukasz 'sil2100' Zemczak
David Krauser
Dimitri John Ledkov
Robert C Jennings
Iain Lane
Michael Vogt
Tobias Koch
Steve Langasek
Balint Reczey
Michael Hudson-Doyle
Julian Andres Klode
Colin Watson
Adam Conrad
Colin Watson
Colin Watson
Cody Shepherd
Daniel Watkins
Francis Ginther