136 lines
6.8 KiB
Markdown
136 lines
6.8 KiB
Markdown
# Guix-in-Docker build environment
|
|
|
|
Guix has no native macOS build, so this runs a Debian container with Guix
|
|
installed (official binary tarball) as the build machine, and uses
|
|
`guix system docker-image` to turn `metznet/machines/*.scm` into Docker
|
|
images you can `docker load`/`docker run` directly in Docker Desktop. No
|
|
QEMU/KVM required, which is why it works on macOS where `scripts/qemu.sh`
|
|
(KVM-only) does not.
|
|
|
|
The builder container runs as `linux/amd64` (via Rosetta on Apple
|
|
Silicon), matching real deployment hardware — nonguix substitutes are
|
|
x86_64-focused, so building on aarch64 forces a lot of expensive
|
|
from-source builds (and occasionally hits GCC/architecture edge cases)
|
|
that just don't happen on amd64.
|
|
|
|
## Why this works without a bootloader/kernel/real disks
|
|
|
|
`guix system docker-image` (and `guix system container`) transform the
|
|
`operating-system` value before building: the real `kernel`, `bootloader`,
|
|
and `file-systems` fields are all replaced/ignored (see
|
|
`containerized-operating-system` in Guix's `gnu/system/linux-container.scm`),
|
|
along with services that make no sense in a container (`mingetty`,
|
|
`agetty`, static networking). Nothing in `metznet/machines/*.scm` needs to
|
|
change for this to work.
|
|
|
|
What *isn't* swapped out: services like `dhcpcd-service-type` and
|
|
`openvpn-client-service-type` (in `%metznet-server-services`) will still
|
|
try to run and will likely fail/hang inside a container (no real DHCP
|
|
server or VPN endpoint reachable) — that's expected. Pass `-N`/`--network`
|
|
(the `image` subcommand below always does) so a dummy `networking`
|
|
shepherd service is provided up front; other services that just
|
|
`(requirement '(networking ...))` (openssh, slapd, kdc) come up fine
|
|
regardless of whether the DHCP/VPN services themselves succeed.
|
|
|
|
## Container internals (why the Dockerfile/entrypoint look the way they do)
|
|
|
|
- `netbase` is installed explicitly: `debian:bookworm-slim` ships without
|
|
`/etc/services`, and Guile's `getaddrinfo` needs it to resolve `https`/
|
|
`http` as service names. Without it, every network fetch inside Guix
|
|
(substitutes, source downloads, even channel git clones from the *guix
|
|
pull* subprocess) fails with a cryptic `In procedure getaddrinfo:
|
|
Servname not supported for ai_socktype` — nothing to do with DNS or
|
|
certs despite how it reads.
|
|
- `entrypoint.sh` starts `guix-daemon --disable-chroot`. Sandboxed builds
|
|
normally isolate via Linux namespaces (`unshare`/`clone`), which Docker
|
|
blocks by default even for root (`clone: Operation not permitted`).
|
|
We're already isolated by the outer container, so build isolation is
|
|
traded away here rather than running the whole thing `--privileged`.
|
|
One side effect: guix-daemon's store-path grafting (security-patch
|
|
rewriting) hits a permission error creating output files under
|
|
`--disable-chroot`, so `scripts/guix-docker.sh image` always passes
|
|
`--no-grafts` (uses un-grafted originals instead — fine for testing).
|
|
- It also always kills and restarts `guix-daemon` rather than checking
|
|
if its socket already exists: `/var/guix` is a persistent volume, so a
|
|
*stale* socket file from a previous (already-exited) container can
|
|
still be sitting there even though nothing is listening on it.
|
|
- `ps` (and possibly other procps tools) crash reliably under Rosetta
|
|
with `assertion failed [true_path_length_self >= 0]` — a Rosetta bug,
|
|
not a Guix issue. Doesn't affect the actual build; just don't rely on
|
|
`docker exec <container> ps` to check progress. `du -sh /gnu/store` or
|
|
watching the log works fine.
|
|
|
|
## One-time setup
|
|
|
|
```
|
|
scripts/guix-docker.sh build # builds the builder image (~1-2 min)
|
|
scripts/guix-docker.sh pull # guix pull -C channels.scm
|
|
```
|
|
|
|
`pull` needs network access to gitlab.com (nonguix, nonguix-games),
|
|
git.metznet.ca, and Savannah/git.guix.gnu.org. Re-run it whenever
|
|
`channels.scm`/`channels.d/*.scm` change. The main `guix` channel alone
|
|
is ~14.8k commits to authenticate the first time, which is genuinely slow
|
|
(several minutes of CPU) — after that it's cached (see below) and
|
|
re-pulls are fast.
|
|
|
|
Guix's store (`/gnu`), daemon state (`/var/guix`), and git-checkout/
|
|
authentication cache (`/root/.cache`, notably `~/.cache/guix/checkouts`
|
|
and `~/.cache/guix/authentication`) all live in named Docker volumes, so
|
|
none of that expensive work repeats across runs. `docker compose down -v`
|
|
wipes everything if you ever want a clean slate (necessary if you ever
|
|
switch the `platform:` in `docker-compose.yml` — a store built for one
|
|
architecture isn't usable from another).
|
|
|
|
## Building a system as a docker image
|
|
|
|
```
|
|
scripts/guix-docker.sh image metznet/machines/ldap.scm
|
|
docker load < build/ldap.tar.gz
|
|
docker run --rm -it <image-id-or-name> /run/current-system/profile/bin/bash --login
|
|
```
|
|
|
|
or do build+load+run in one step:
|
|
|
|
```
|
|
scripts/guix-docker.sh load-run metznet/machines/ldap.scm
|
|
```
|
|
|
|
`image`/`load-run` pass `-L /workspace` (this checkout, bind-mounted
|
|
live), so uncommitted local edits to any `metznet/**` module are picked
|
|
up directly — no commit/pull round-trip needed, unlike a real `guix
|
|
pull` on a deployed machine.
|
|
|
|
`DOMAIN_CAPS`/`DOMAIN_NAME` default to `METZNET.CA`/`metznet.ca` in
|
|
`docker-compose.yml`, matching `scripts/qemu.sh`; override by exporting
|
|
them before calling `guix-docker.sh`, or in a `.env` file next to
|
|
`docker-compose.yml`.
|
|
|
|
Once running, `herd status` inside the container shows shepherd service
|
|
state — useful for checking whether e.g. `slapd` actually came up.
|
|
|
|
## Known gotchas
|
|
|
|
- `channels.d/metznet.scm` has no `introduction`, so `guix pull` doesn't
|
|
authenticate it (unlike nonguix/games, which do).
|
|
- `metznet/machines/otto.scm` (Noah's laptop) has `firefox` and the ARM
|
|
embedded toolchain (`make-arm-none-eabi-toolchain-7-2018-q2-update`)
|
|
removed from its package list — both require building an entire
|
|
LLVM/Clang toolchain from scratch (no substitute currently available,
|
|
even on x86_64), which is impractical for routine testing. Add them
|
|
back with `(make-arm-none-eabi-toolchain-7-2018-q2-update)` (a
|
|
zero-arg constructor from `(gnu packages embedded)`, not a bare
|
|
package name — that changed upstream) and `firefox` (from `(nongnu
|
|
packages mozilla)`) if you need them for a real build/deploy.
|
|
- Package-level quirks fixed in this channel to build at all against
|
|
current Guix (all in `metznet/packages/`, not upstream Guix changes):
|
|
`metznet-sssd` disables tests (one fails only under Docker's emulated
|
|
filesystem/Rosetta, not a real regression) and strips the
|
|
`--install-layout=deb` flag setuptools no longer accepts;
|
|
`metznet-kitty` disables tests for the same reason (`O_TMPFILE`
|
|
atomicity isn't honored by Docker Desktop's virtualized filesystem on
|
|
macOS); `openldap-slapd` forces `-std=gnu17` plus several `-Wno-*`
|
|
flags because GCC 14 turned some of its 25-year-old autoconf checks
|
|
into hard errors, and adds `libxcrypt` as an input since glibc dropped
|
|
`crypt()`.
|