276 Commits

Author SHA1 Message Date
Robert C Jennings
8cb442c3e8
magic-proxy: dump proxy log to stdout on failure
When we encounter a failure in 'lb binary' the launchpad builders can
only surface the build output from stdout.  If the binary hook failure
implicates the archive we can not determine fault without the apt
proxy log.  This patch will dump the proxy log to stdout to aid in
debugging these failures.
2019-10-04 10:28:02 -05:00
Colin Watson
dc61b17db3 Use iptables-legacy on older kernel versions
Use iptables-legacy rather than iptables when running on older
(pre-4.15) kernel versions.  The newer nf_tables-based tools misbehave
at least on 4.4.
2019-09-03 15:23:06 +01:00
Iain Lane
b68a5abf65
snap seeding: Defer validation for regular image builds
When getting the list of snaps to include on an image via germinate, we
process the snaps in alphabetical order. Currently we seed several
packages `gnome-foo` and also `gtk-common-themes`, the default provider
of `gtk-3-themes`. Since `gtk-common-themes` is alphabetically after
`gnome-foo`, the `seed.yaml` we generate is invalid when we are part way
through generating.

What we really care about, though, is not that the `seed.yaml` is always
valid at every step, but it is that it is valid at the *end* of seeding.
So for the germinate case, let's defer validation to happen once at the
end. Other callsites of `snap_preseed`, where callers are careful to
seed snaps in the right order, are unaffected by this.
2019-09-02 12:56:00 +01:00
Colin Watson
9a65a4013d Undivert update-initramfs more carefully
Previously the wrapper script and diversion were left in place on images
such as buildd images that don't have initramfs-tools installed.

LP: #1841573
2019-08-27 12:45:57 +01:00
Robert C Jennings
cb535805d7
ubuntu-cpc: rework image artifact cleanup
The livecd.ubuntu-cpc.ext4 that is present in each build (plus kernel
and initrd) are not renamed from /build/binary/boot/filsystem.ext4
and friends until after the binary hooks are run, so this patch moves
from trying to perform this cleanup in a binary hook.  Now the cleanup
will be run at the end of live-build/binary for the ubuntu-cpc project.
2019-07-22 11:29:04 -05:00
Łukasz 'sil2100' Zemczak
1e275b6b46 Append seeded snaps to manifests for all images. 2019-07-02 11:39:07 +02:00
David Krauser
0e3b1e206b
Add support for HyperV Gallery images 2019-06-11 23:26:07 -04:00
Dimitri John Ledkov
d7b629cae6 Do not force lzma on ubuntu-core builds, the compress format default should be set universally inside initramfs-tools-ubuntu-core package instead of getting duplicated multiple times all over the place. 2019-06-05 13:50:20 +01:00
Dimitri John Ledkov
702ccb4c77 Merge branch 'dead-ubuntu-touch' of git+ssh://git.launchpad.net/~xnox/livecd-rootfs into ubuntu/master 2019-06-04 23:42:31 +01:00
Dimitri John Ledkov
af96b80116 Merge branch 'dead-ubuntu-desktop-next' of git+ssh://git.launchpad.net/~xnox/livecd-rootfs into ubuntu/master 2019-06-04 23:40:37 +01:00
Dimitri John Ledkov
44fd9ba9fd Drop ubuntu-touch configs 2019-05-30 12:13:31 +01:00
Dimitri John Ledkov
2aa3a59b7c Drop ubuntu-desktop-next configs 2019-05-30 12:00:59 +01:00
Dimitri John Ledkov
5b89298cd9 Drop old arm platforms. 2019-05-30 11:49:06 +01:00
Steve Langasek
c306763147 Strip translation files out of the minimal images.
This is another thing that goes unused when there is no human console user
(and we already don't have the locales themselves present on a minimal
image).

Bug-Ubuntu: https://bugs.launchpad.net/bugs/1829333
2019-05-17 18:25:21 +02:00
Julian Andres Klode
bed915a0fb Don't purge busybox-static in minimized, no longer installed
This caused the builds to fail because we were purging are non
existing package.
2019-05-13 14:22:53 +02:00
Michael Hudson-Doyle
42992467a1 Remove device nodes later for ubuntu-base:minimized (i.e. docker) builds. (LP: #1828118) 2019-05-08 10:04:23 +12:00
Julian Andres Klode
e95ad81c47 Run clean_debian_chroot after minimize_manual
This avoids us having apt cache files in the final image.

For auto/build, we move the call to clean_debian_chroot to the end,
this also takes care of the caches generated by apt-get update
in case of a preinstalled-pool.

For layered, it's just a simple swap of the lines.

LP: #1826377
2019-04-25 11:51:30 +02:00
Tobias Koch
05e71b3840 Do not include any snaps in ubuntu-cpc:minimized builds 2019-04-12 13:37:02 -07:00
Balint Reczey
2d162dd121 Pass --sort=name to tar to make tarballs more reproducible and rsync-friendly 2019-04-10 14:38:35 +02:00
Jean-Baptiste Lallement
38bb0b3301 Fix grub divert for multilayer images
grub-probe must not be called during image build so grub is diverted. In
multilayer image the chroot is always the tip of the filesystem, so we
divert grub at the beginning of each pass and undivdert it as the end.
For flat images, it's diverted just before building the chroot and
undiverted after.
2019-03-18 09:33:07 +01:00
Steve Langasek
ea5f01942f divert grub when calling into lb chroot
UbuntuStudio image builds have recently begun failing as a result of adding a
grub theme customization to their flavor, which they then try to apply by
running update-grub from their maintainer scripts.  This fails with:

  Setting up plymouth-theme-ubuntustudio (0.57) ...
  update-alternatives: using /usr/share/plymouth/themes/ubuntustudio-logo/ubuntustudio-logo.plymouth to provide /usr/share/plymouth/themes/default.plymouth (default.plymouth) in auto mode
  update-alternatives: using /usr/share/plymouth/themes/ubuntustudio-text/ubuntustudio-text.plymouth to provide /usr/share/plymouth/themes/text.plymouth (text.plymouth) in auto mode
  update-initramfs: diverted by livecd-rootfs (will be called later)
  /usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
  dpkg: error processing package plymouth-theme-ubuntustudio (--configure):
   installed plymouth-theme-ubuntustudio package post-installation script subprocess returned error exit status 1

Packages providing grub themes should be able to call update-grub from their
maintainer scripts, and they should be able to be installed under
livecd-rootfs.  It's surprising to me that this hasn't been a problem before
now for any flavors.

We know that grub-probe should not be called as part of an image build, so
just use our standard helper function to divert it before running the chroot
stage and undivert it after.
2019-03-09 20:08:29 +00:00
Balint Reczey
039ed8ec31 Merge fixing minimized text spacing 2019-02-06 17:55:10 +07:00
Balint Reczey
0182d380ca Nicely separate motd text about minimizing from texts above 2019-02-05 17:24:09 +07:00
Tobias Koch
5b70d86f5c Inject a proxy into the build providing a snapshot view of the package repo.
When the REPO_SNAPSHOT_STAMP variable is set, the auto/build script will attempt
to launch a transparent HTTP proxy on port 8080, and insert an iptables rule to
redirect all outgoing HTTP requests to this proxy.

The proxy, contained in the `magic-proxy` Python script, examines each request
and silently overrides those pointing to InRelease files or files that are
listed in InRelease files. It will instead provide the contents of the requested
file as it was at REPO_SNAPSHOT_STAMP, by downloading the corresponding asset
"by hash".
2019-02-01 11:26:20 +01:00
Michael Hudson-Doyle
a9991a908e revert change to ubuntu-server:live snap seeding 2019-02-01 17:22:25 +13:00
Jean-Baptiste Lallement
283871573d Changed the naming scheme for multi layer images to dot format 2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
41becb0af7 Ensure snaps are available on ubuntu-server live 2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
3075d65578 Cleanup and handle manifest-remove files
manifest-remove files are generated in all cases and removed for
projects/subprojects that don't need it like the new ubiquity.
2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
9dba350834 Added lb_binary_layered
Created lb_binary_layered based on lb_binary
Factorized several helpers
These helpers are called from lb_chroot
2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
43ec452fa0 Moved reusable functions to functions
Moved functions to reuse in lb_binary and lb_chroot to functions
2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
047f883e92 Various fixes
Fixed prefix for passes to not conflict with global prefix
Set layer to none for layer without language packs
Copy size and manifest to the target directory
Make sure the diff always diffes existing manifests
2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
42d1d77612 Layered Ubuntu desktop by default
Default Ubuntu desktop to the layered format
Remove the requirement on setting IMAGEFORMAT to layered to build a
layered image and relies on PASSED instead.
2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
53b06af931 Added sublayers support
Adds support for sublayers by defining package lists corresponding to
each sublayer. The name of the package list defines the name of the
layer.
2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
78c196031b Add snaps to manifest
Make sure that all the snaps are listed in the full manifest of each
layer and as delta in the delta manifest.
2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
d5a16296f4 live-build/auto/build: Create one manifest by layer in diff format and one full manifest with all the layers in standard format 2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
ca63d43696 live-build/auto/build: Add size of the filesystems. This is used by the installer to calculate the minimum partition size to reserve for installation. The installer will have to sum all the sizes to compute the final size. Thank didrocks 2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
b6287916c2 auto/build: Order generated squashfs filesystems 2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
ce973ba07b auto/build: Removed useless comment 2019-01-30 12:29:56 +01:00
Jean-Baptiste Lallement
7b4acba31d * Added lb_chroot_layered to add support for multi layered squashfs images
* auto/*: Added live-layered image format
2019-01-30 12:29:56 +01:00
Łukasz 'sil2100' Zemczak
abec4f4c1d Link to the proper name instead. 2019-01-10 22:58:07 +01:00
Łukasz 'sil2100' Zemczak
e60bcbaecd Add proper IMAGAFORMAT for raspi3 classic builds, output the images with the expected naming. 2019-01-09 10:47:24 +01:00
Colin Watson
46942330d8 Move buildd image building to binary hooks 2018-12-20 01:20:57 +00:00
Colin Watson
082a2046e7 Add a LXD image to builds for the buildd subproject 2018-12-19 22:11:16 +00:00
Michael Hudson-Doyle
4b01505952 Fix typo preventing builds of ubuntu-core images. 2018-12-12 15:09:56 +13:00
Steve Langasek
5300bb9eb7 Key netplan delegation to NetworkManager on presence of
/usr/sbin/NetworkManager, not on /usr/lib/NetworkManager which may have
hooks from other packages (i.e., wpasupplicant).
2018-12-06 13:38:07 +02:00
Łukasz 'sil2100' Zemczak
d3eb3dd0ac Add missing arguments to ubuntu-image classic calls. 2018-11-26 13:03:24 +01:00
Steve Langasek
6ee39485dd Merge lp:~sil2100/livecd-rootfs/use-ubuntu-image-classic 2018-11-21 12:51:00 -08:00
Łukasz 'sil2100' Zemczak
05a8be839f Do the same for ubuntu-image snap builds. 2018-11-21 21:07:16 +01:00
Łukasz 'sil2100' Zemczak
8a33b23dd0 Use -O instead of -o for ubuntu-image classic builds. 2018-11-21 21:05:21 +01:00
Łukasz 'sil2100' Zemczak
2947cdbe60 First batch of obvious changes - use better shellism, only call make (without make install). 2018-11-21 20:22:38 +01:00