137 Commits

Author SHA1 Message Date
Robert C Jennings
c10042acb0
ubuntu-cpc: Fixup comment references to file names post-parallelization 2019-05-21 17:07:03 -05:00
Robert C Jennings
996d7e20d5
ubuntu-cpc: Ensure base disk image is the same between all build targets
The following targets have livecd.ubuntu-cpc.manifest (and
livecd.ubuntu-cpc.ext4) which differ in some way from the 'all'
target. They are all missing grub-efi and other modifications:

root-dir
squashfs
tarball

These targets do not depend on the 'disk-image' target. This means that
the ext4 produced will lack the uefi modifications (and any from the
disk-image target binary hooks).

Since the ext4 file is common to all builds there is a chance that a
parallel build from one of these targets could overwrite this artifact.
This patch ensures that all targets will produce consistent base output.
2019-05-21 17:07:03 -05:00
Robert C Jennings
75a3b3e6ae
ubuntu-cpc: Allow comments in series files
An upcoming addition of the disk-image target to several series files will
required some explanation.  Without comment support in series files that
is not possible.  This patch adds support for comments in series files.
2019-05-21 17:07:03 -05:00
Robert C Jennings
77ae8d704f
ubuntu-cpc: parallel builds
* Replace "snap download" with tool that uses snap store's coherence feature

    This is important for parallel image builds to ensure all pre-seeded snaps
    have the same versions across image variants.

* Inject a proxy into the build providing a snapshot view of the package repo.

    When the REPO_SNAPSHOT_STAMP variable is set, the auto/build script will attempt
    to launch a transparent HTTP proxy on port 8080, and insert an iptables rule to
    redirect all outgoing HTTP requests to this proxy.

    The proxy, contained in the `magic-proxy` Python script, examines each request
    and silently overrides those pointing to InRelease files or files that are
    listed in InRelease files. It will instead provide the contents of the requested
    file as it was at REPO_SNAPSHOT_STAMP, by downloading the corresponding asset
    "by hash".

* Use series files with dependency handling to generate hook symlinks dynamically

    This patch currently only applies to the "ubuntu-cpc" project.

    More and more logic has been going into the hook scripts to decide
    under which conditions they should run or not. As we are moving
    to parallelized builds of image sets, this will get even more
    complicated. Base hooks will have to know which image sets they
    belong to and modification of the dependency chain between scripts
    will become more complicated and prone to errors, as the number of
    image sets grows.

    This patch introduces explicit ordering and dependency handling for
    scripts through the use of `series` files and an explicit syntax
    for dependency specification.
2019-05-21 17:06:59 -05:00
Balint Reczey
64985baafd Build WSL rootfs tarball
LP: #1827930
2019-05-17 14:03:03 +02:00
Steve Langasek
4e77d07a6c Also don't create the update-motd directory since we're not writing to it.
It will almost certainly be created at some point but we don't need to
create it here.
2019-04-08 11:51:40 -07:00
Steve Langasek
518b18db0f Drop /etc/update-motd.d/51-cloudguest from cloud images; this is not consistent with current Ubuntu Advantage product language. Any future customizations to update-motd for cloud images should be done via a package instead. 2019-04-08 11:51:38 -07:00
Balint Reczey
657500f59f Add .keep files to preserve empty directories 2018-12-12 15:35:18 +01:00
Cody Shepherd
1f05fbea69 Backport adding snaps to image manifests (LP: #1805497) 2018-11-27 11:53:59 -08:00
Daniel Watkins
64e90c0cca ubuntu-cpc: Handle a pre-existing /lib/modules in 030-root-tarball.hook (LP: #1797170). 2018-10-10 09:09:19 -07:00
Francis Ginther
928a9e6c71 Fix install command for pollinate (add '-y') in minimal images 2018-07-02 14:46:42 -05:00
Francis Ginther
67d979da0d Instead of not removing vim-common, explictily add back pollinate.
This is a more explicit version of what is needed and would require no
changes if pollinate is updated to drop the dependency on vim-common.
2018-06-29 14:52:52 -05:00
Francis Ginther
809028f239 Keep vim-common in the minimal images for pollinate 2018-06-29 10:44:49 -05:00
Robert C Jennings
8e53c77a11 snap: Alternate fix for snap pre-seeding (LP: #1775710)
* Backport: Refactor functions out of ubuntu-cpc and ubuntu-server hooks.
  * Backport: Add dependency on distro-info for snap preseeding
2018-06-07 17:35:12 -07:00
Daniel Watkins
4e83114045 ubuntu-cpc: When performing a minimized build, don't generate artifacts
that won't boot with the linux-kvm kernel (LP: #1757223).
2018-03-20 16:10:44 -04:00
Balint Reczey
8cc5fb43fa Don't ask for password and GECOS while creating vagrant user 2017-12-21 09:49:49 +01:00
Christopher Glass
7854028092 Changed default user configuration in the vagrant image, in accordance
with the vagrant community's expectations (LP: #1569237)
2017-12-07 14:09:43 +01:00
Balint Reczey
cbf0a7c417 Use default xz compression (-6) for raspi2 image
Xz with -0 generated ~35% bigger compressed image.

LP: #1732681
2017-12-05 22:40:35 +01:00
Balint Reczey
646a88b5ee Use xz -T4 -0 instead of pxz
Test showed very little difference in file-size between -0 and -9 while
both compression and decompression need much less memory and compression
needs less CPU time for -0.

Supported xz versions before 5.2 also accept the -T4 option but ignore it.

Also depend on xz-utils instead of downloading pxz.

LP: #1701132
2017-12-05 22:40:35 +01:00
Balint Reczey
7cc9576845 Depend on packages instead of installing them at run time 2017-12-05 22:40:35 +01:00
Christopher Glass
67dab61787 Backport "minimized round 2" changes from trunk to Xenial (LP: #1731492) 2017-11-10 18:41:57 +01:00
Steve Langasek
6c56522961 divert /usr/bin/systemd-detect-virt as part of the grub diversions, so
that the grub kernel postinst hook is operational when we're building
in a container.  LP: #1729034.
2017-10-31 10:05:39 -07:00
Steve Langasek
d5c4f11daa * Use kvm kernel only on amd64.
* Make non-x86 minimized images consistent with x86 by not explicitly
  installing the server task.
* Clean up dangling /boot/initrd.img symlink left behind on minimized
  builds.
2017-10-20 23:10:26 -07:00
Dimitri John Ledkov
22a8f817ca Now that grub-related diversions have been factored out in 2.466
instead of having bogus root=stuff arg generated in grub.cfg, it is
actually empty. Therefore update the sed command to make the arg in
the root= token optional. This should resolve non-booting livecd cpc
images.
2017-10-10 13:49:46 +01:00
Steve Langasek
b2fba90356 Fix the force-partuuid handling to create the file under
/etc/default/grub.d/ with a .cfg extension as grub expects; it's unclear
how this ever worked without.
2017-10-05 16:02:26 +02:00
Steve Langasek
b66cc28507 Factor out grub-related diversions and use them consistently, so we
don't end up with wrong os-probe output in our grub.cfg.
2017-10-05 16:01:04 +02:00
Balint Reczey
413a53d482 When SUBPROJECT environment variable is not set assume it to be "" 2017-10-03 17:31:27 +02:00
Balint Reczey
88d854ffb5 Use SUBPROJECT=minimized name instead of SUBPROJECT=minimize for consistency 2017-10-03 17:31:26 +02:00
Steve Langasek
4d7509f570 Fix a reference to an undefined variable in a script that's set -u. 2017-10-03 17:31:16 +02:00
Balint Reczey
86ed851b5a Force partuuid in /etc/default/grub.d/* instead of in /etc/default/grub
This prevents ucf questions upon installation of changed /etc/default/grub
2017-09-28 09:38:23 -04:00
Steve Langasek
b32298ede1 minimization handling for UEFI disk images 2017-09-28 09:38:23 -04:00
Balint Reczey
98f27745b7 Drop initramfs-tools diversion
Kernels which are able to boot without initramfs now dropped dependency
on initramfs-tools thus initramfs-tools can be removed from the image
instead of having to divert it to avoid initramfs generation.
2017-09-28 09:38:23 -04:00
Balint Reczey
c2d66c7d96 Source config/binary instead of /build/config/binary which may not exist 2017-09-28 09:38:23 -04:00
Steve Langasek
cd15b730f7 In a cloud environment, we can rely on the kernel being able to boot the
root filesystem directly, without an initramfs; enable this when
building minimized.
2017-09-28 09:38:21 -04:00
Steve Langasek
86108d69d9 Make the 999-cpc-fixes.chroot subproject-aware, so we don't try to locale-gen 2017-09-28 09:34:31 -04:00
Steve Langasek
9c9e8c4e4f Mount using --make-rslave to ensure safe unmounts for rbind mounts 2017-09-26 01:10:15 -04:00
Steve Langasek
f16611774f * Improve teardown_mountpoint to recursively find all submounts and
unmount them, instead of working from a hard-coded list.  This makes
  the code resilient against other submounts being added later, including
  downstream.
* Also nuke the sleep / udevadm settle calls in the process, which should
  never be required and slow down the builds.
2017-09-19 13:45:37 -07:00
Victor Tapia
13e9dc0089 live-build/ubuntu-cpc/hooks/061-open-iscsi.chroot: generate iSCSI
Initiator Name at first iscsid run for cloud images to ensure it is
unique (LP: #1444992).
2017-09-19 11:03:29 +02:00
Robert C Jennings
a623618e2c Install udev before calls to udevadm 2017-09-13 21:05:22 -05:00
Robert C Jennings
0acda846dd live-build/ubuntu-cpc/functions: Add teardown_mountpoint to reverse setup_mountpoint
* live-build/ubuntu-cpc/functions: Add a function, teardown_mountpoint,
  to reverse the work done in setup_mountpoint.  Lack of this function
  has forced users of setup_mountpoint to implement this separately
  and the implementations have diverged. (LP: #1716992)
* live-build/ubuntu-cpc/functions: Remove umount_settle function.
  The was only used where teardown_mountpoint was lacking.
2017-09-13 13:18:03 -05:00
Steve Langasek
11a92795fe live-build/ubuntu-cpc/functions: call apt-get update in the chroot
after mounting a blank /var/lib/apt, so that further operations work
as expected.
2017-09-12 13:25:55 -07:00
Steve Langasek
3fdf3933ed Fix suppression of kpartx error. LP: #1684090. 2017-09-11 14:34:09 -07:00
Steve Langasek
314a6b95a3 Mount and unmount /dev recursively, to cope with setups where there are
interesting bind-mounts under /dev (e.g. loop devices bind-mounted by
LXD).  LP: #1716465.
2017-09-11 14:33:23 -07:00
Steve Langasek
880d4f8b89 live-build/ubuntu-cpc/hooks/033-disk-image-uefi.binary: call apt-get
update *before* installing packages, not after.
2017-08-25 15:30:48 -07:00
Steve Langasek
95aabdf668 live-build/ubuntu-cpc/functions: mount tmpfs on /var/cache/apt and
/var/lib/apt, so we don't have to leave empty space in our derivative
images for packages that have been downloaded/installed/removed.  This
normally isn't relevant for the installed system, since the root
filesystem will auto-expand in place on the target disk, but lets us
ship smaller images.
2017-08-25 15:30:05 -07:00
Mathieu Trudel-Lapierre
a75544e0c3 Only set a default GRUB_PRELOAD_MODULES if not already defined; otherwise used the existing values. 2017-08-23 19:20:35 -04:00
Mathieu Trudel-Lapierre
f48670cec5 Clean up GRUB_MODULES_PRELOAD / grub_modules. Remove the modules already loaded in stock signed grub EFI binaries. 2017-08-18 19:37:34 -04:00
Balint Reczey
2569995de8 Work around kpartx's nonzero exit due to "Device or resource busy" errors
which still permit kpartx to finally remove loop device (LP: #1684090).
Also sync before calling kpartx to let writing to loop devices finish.
2017-05-12 21:20:22 +02:00
Balint Reczey
95c239f301 Use all config hooks from the proper place, not from /build/ 2017-05-12 21:20:22 +02:00
Balint Reczey
e25936cc61 Source ubuntu-cpc functions from the right place 2017-05-12 21:20:22 +02:00