claudiomaradonna/azkaban
1
1
Fork 0
Code Issues Pull requests Activity

feat!: use Apple container on macOS #40

Merged
claudiomaradonna merged 7 commits from feat/apple-container-macos into master 2026-10-08 08:46:27 +02:00
Conversation 0 Commits 7 Files changed 23 +1197 -554
motoko commented 2026-10-08 08:39:27 +02:00
Collaborator
Copy link

Why

A Mac needed a rootless Podman machine that the operator created, started,
sized and kept rootless, and only paths under its shared volumes could be
mounted. macOS now runs on Apple's own runtime, the container CLI 1.5.0,
so a Mac needs no Podman at all and setup is simpler.

What changed

  • On macOS, launches, custom-image builds and the installer use container.
    Linux keeps Podman and behaves as before.
  • Breaking: a Mac needs Apple silicon, macOS 26 and container 1.5.0 or
    newer, with its service running (container system start --enable-kernel-install). The Podman machine is no longer used, and the
    installer installs container with Homebrew.
  • macOS has no UID mapping, so /workspace appears root-owned. The base image
    therefore trusts /workspace and the worktrees below it for git.
  • Workarounds for open upstream issues, on macOS only:
    • an identity's goals.md is not served (#2148);
    • a mount of anything that is not a directory, /dev/null included, is
      refused when it lands inside another mount (#2148);
    • interactive launches clear the terminal's ONLCR flag around the client and
      restore it afterwards (#2299);
    • file values containing a line break that container splits on are not
      forwarded, because container would otherwise fill the pieces from the
      host environment.
  • As before, the SSH key is not served on macOS.
  • Dockerfile.azkaban builds on macOS from a private copy of the approved
    record in which only the first FROM image is replaced, because
    container build has no --from. The installer adds a
    localhost/<image>:latest name next to the plain one.
  • The README, the headless contract and the development notes describe the
    macOS runtime.

Verification

  • make lint && .agents/hooks/shellcheck.sh repo && make test passes:
    ShellCheck reports 35 files clean and all 305 tests pass.
  • prettier --check README.md docs/*.md passes.
  • Branch review by the change-reviewer and docs audit: both high findings
    (the env-file split and the localhost/ image name) are fixed in the last
    commit.
  • Not run:
    • make smoke-images, which needs real Podman; the Dockerfile gained one
      RUN.
    • Anything on a real Mac: the macOS behavior is tested only with Darwin
      stubbed on Linux.

Risks and follow-ups

  • The plan's real-Mac checklist is owed. It covers:
    • TTY rendering;
    • git and file ownership in /workspace;
    • --env-file /dev/fd/3;
    • --cidfile with container stop;
    • the localhost/ image names;
    • --ulimit.
  • These are deferred until upstream fixes land: serving the key and goals on
    macOS, and signal forwarding (#1876).
## Why A Mac needed a rootless Podman machine that the operator created, started, sized and kept rootless, and only paths under its shared volumes could be mounted. macOS now runs on Apple's own runtime, the `container` CLI 1.5.0, so a Mac needs no Podman at all and setup is simpler. ## What changed - On macOS, launches, custom-image builds and the installer use `container`. Linux keeps Podman and behaves as before. - **Breaking:** a Mac needs Apple silicon, macOS 26 and `container` 1.5.0 or newer, with its service running (`container system start --enable-kernel-install`). The Podman machine is no longer used, and the installer installs `container` with Homebrew. - macOS has no UID mapping, so `/workspace` appears root-owned. The base image therefore trusts `/workspace` and the worktrees below it for git. - Workarounds for open upstream issues, on macOS only: - an identity's `goals.md` is not served (#2148); - a mount of anything that is not a directory, `/dev/null` included, is refused when it lands inside another mount (#2148); - interactive launches clear the terminal's ONLCR flag around the client and restore it afterwards (#2299); - file values containing a line break that `container` splits on are not forwarded, because `container` would otherwise fill the pieces from the host environment. - As before, the SSH key is not served on macOS. - `Dockerfile.azkaban` builds on macOS from a private copy of the approved record in which only the first `FROM` image is replaced, because `container build` has no `--from`. The installer adds a `localhost/<image>:latest` name next to the plain one. - The README, the headless contract and the development notes describe the macOS runtime. ## Verification - `make lint && .agents/hooks/shellcheck.sh repo && make test` passes: ShellCheck reports 35 files clean and all 305 tests pass. - `prettier --check README.md docs/*.md` passes. - Branch review by the change-reviewer and docs audit: both high findings (the env-file split and the `localhost/` image name) are fixed in the last commit. - Not run: - `make smoke-images`, which needs real Podman; the Dockerfile gained one `RUN`. - Anything on a real Mac: the macOS behavior is tested only with Darwin stubbed on Linux. ## Risks and follow-ups - The plan's real-Mac checklist is owed. It covers: - TTY rendering; - git and file ownership in `/workspace`; - `--env-file /dev/fd/3`; - `--cidfile` with `container stop`; - the `localhost/` image names; - `--ulimit`. - These are deferred until upstream fixes land: serving the key and goals on macOS, and signal forwarding (#1876). 
The base stage now marks /workspace and the worktrees below it as safe
git directories in the system config. Apple's container presents bind
mounts on macOS as root-owned, so git would otherwise refuse the
workspace for the image's non-root client user. On Linux, keep-id
already makes that user the owner, so the entries change nothing there.
Replace azk_host_podman with azk_host_engine, which runs container on
Darwin and podman elsewhere, both under LC_ALL=C. Add
azk_host_engine_ready, Darwin's check that `container system status`
exits 0, keeping the output in AZK_HOST_ENGINE_ERROR otherwise. Remove
azk_host_machine_state and its globals. azk_host_podman_engine and
azk_host_machine_mounts stay for the Linux installer and the unchanged
launch path, and now call podman directly so Darwin launches behave as
before until the launch moves to container.

On Darwin, install.sh installs container with Homebrew when missing,
requires version 1.5.0 or newer, stops with the start command when the
service is down, and builds, probes, and tags through container. Linux
keeps every command and message byte for byte.

Tests gain a recording container stub that refuses unless
STUB_HOST=Darwin, which stub_darwin_uname now exports. The podman and
container stubs share their recording, run, and build handling in
tests/stub/engine-common.sh, which they find through STUB_DIR, exported
by the harness, because installer tests link the stubs into a private
PATH.

BREAKING CHANGE: on macOS the installer needs Apple container 1.5.0 or newer and no longer checks a Podman machine.
A macOS launch now runs Apple's container CLI and no longer consults a
Podman machine. Every launch, a dry run included, checks the service
with azk_host_engine_ready and, when it is down, stops with the command
that starts it, followed by container's own message when it has one.
The rootless-machine check goes, and so does the shared-volume check
(_azk_require_reachable_mounts, _azk_mount_source_is_reachable, and the
library's azk_host_machine_mounts and AZK_HOST_MACHINE_MOUNTS):
container shares any host path, and a missing source already stops the
launch. The end of _azk_build_podman_args, where that check stood, is
left for the nested file-mount guard.

The Darwin argv starts `run --rm --init -i --ulimit nproc=512`, and
everything after it is shared with Linux, whose argv is unchanged.
container has no UID mapping (apple/container#165), so there is no
--userns; it has no --pids-limit, so --ulimit nproc carries the process
limit; and its run passes no host proxy variables, which is what
--http-proxy=false asks of Podman.

Under -i -t, container leaves the host terminal's ONLCR on and garbles
full-screen clients such as Claude Code (apple/container#2299). A
Darwin launch with a TTY therefore execs a POSIX sh wrapper: it saves
the terminal with `stty -g`, runs `stty -onlcr`, runs container,
restores the saved settings, and exits with container's status; when
`stty -g` fails, it execs container unchanged. A dry run prints the
wrapped command, since that is the exact launch. Linux, and Darwin
without a TTY, exec the engine directly as before.

The workspace keeps its physical path on macOS, which keys custom image
approvals there; only the comment that cited the Podman machine
changes.

Internal choices: the wrapper's stty calls close descriptor 3, which
stays open only for container's --env-file /dev/fd/3; the wrapper sets
no signal traps, since signal forwarding stays deferred
(apple/container#1876). The ssh-agent script is unchanged: it execs
whichever engine it is handed, and Darwin never serves the key.

Tests: the podman stub now refuses when STUB_HOST=Darwin, so every
Darwin test proves that Podman is never called, and the stub's machine
handling goes. New runtime tests cover the stopped service (launch and
dry run), the Darwin argv with --max-memory, --network, and --cidfile,
and the TTY wrapper through a recording stty: call order, exit status,
the env file on descriptor 3, the unsaved-terminal fallback, and the
printed dry run. The tests of the removed behavior go: the engine,
warning, rootful, shared-volume, and unreadable-machine-mounts runtime
tests and both azk_host_machine_mounts tests.

BREAKING CHANGE: macOS launches need Apple container 1.5.0 or newer with its service running; no Podman machine is used.
Apple container drops the directory mount that a file mount's
destination lies inside (apple/container#2148), and a single-file mount
shares the file's parent directory with the VM. On macOS this change
therefore:

- leaves an identity's goals.md out with one line, since its file mount
  would replace the client state mount and share the identity folder,
  which holds key and env, with the VM; the identity report omits the
  goals;
- rewords the key line to say azkaban serves an identity's SSH key on
  Linux only, without naming a Podman machine;
- stops the launch, before the engine starts, when a mount whose source
  is not a directory (a regular file or /dev/null) has a destination
  equal to or below another mount's destination, naming both mounts.
  Directory mounts inside directory mounts stay allowed, and Linux is
  unaffected.

Internal choice: the guard checks sources that exist and are not
directories, so a source that does not exist yet (the MCP directory in
a dry run) is not taken for a file mount.

The old shared-sources test loses its /dev/null case to the refusal
test; its remaining directory case joins the test that allows a
directory mount inside another mount.
Podman builds the shared Dockerfile.azkaban on the invoked client's
image through --from, which replaces the file's first FROM. Apple
container's build has no --from, so on macOS the launcher now writes a
private copy of the approved record, never the workspace file, with
only that image replaced by localhost/<client image>:latest, the value
Linux passes to --from, and builds with -f on the copy and no --from.

The copy is <record>.build.<pid>, beside the record in the private
approvals folder, written under umask 077 with noclobber after removing
any leftover of the same name, as the pending copy does. It is removed
after the build whatever its status, an interrupted build's 130
included, and when writing it fails. A first FROM without an image
stops the launch with "cannot find the image of the first FROM in
Dockerfile.azkaban"; a failed write stops it with "cannot write the
build copy of Dockerfile.azkaban". Dockerfile.<client> still builds
from its record, a dry run writes no copy, and Linux keeps --from.

Internal choices:

- The FROM line is the first line whose first word, after leading
  blanks, is FROM in any case. Words are split on spaces and tabs, and
  a trailing carriage return is set aside and restored, so CRLF files
  keep their endings. Words starting with -- after FROM are flags and
  stay; the next word is the image. No such line, no word after the
  flags, or a word that is exactly a backslash (a line continuation)
  stops the launch.
- The record is opened once: Bash's read copies the lines up to the
  FROM line, keeping whether each ends in a newline, and cat copies the
  rest of the open file as it is, so a concurrent approval cannot mix
  two versions of the record into one copy.

The container stub copies a build's -f file to STUB_BUILD_FILE_COPY,
mode kept, so tests can read the copy the launcher removes.
The README and docs still described a Podman machine and its shared
volumes on macOS, which the launcher and installer no longer use. They
now name Podman on Linux and Apple container on macOS: the security
model, prerequisites, installer and setup steps, custom image reclaim
commands, the macOS section, and the troubleshooting entries, with the
launcher's messages quoted as printed.

The headless contract records that container ignores SIGTERM, so a
supervisor stops a macOS launch through the cidfile ID with container
stop. The development page names the host library's engine probes and
the PODMAN=container build for Mac developers.

The launcher's --network help, the cidfile comment, and the installer's
--no-cache help now read for both engines.
Apple container 1.5.0 splits an --env-file on every newline character,
CR, VT, FF, U+0085, U+2028, and U+2029 besides LF, and fills a piece
without = from the host environment of the container process, so a
workspace value such as X=1<CR>GH_TOKEN copied the host's GH_TOKEN
into the container. On macOS a file value holding any of them, anywhere,
is now not forwarded, with a warning; the patterns match the UTF-8
bytes in the C and UTF-8 locales alike. Linux keeps its trailing-CR
rule and message.

container stores an unqualified build tag without the localhost/
domain, so FROM localhost/<image>:latest, which the Dockerfile.azkaban
rewrite and the README example use, found no local image and pulled.
The macOS installer now also tags each image localhost/<image>:latest;
the launcher still runs the unqualified name.

Also:
- the engine stubs treat --ulimit as taking a value, so they keep
  recording -e and --env-file values after it on macOS;
- the TTY test also checks that a Linux launch runs Podman directly
  and leaves the terminal alone;
- Claude's macOS dry-run warning says the pasted command fails, since
  container refuses a missing mount source; Linux keeps its text;
- the --max-memory help names the engine's default;
- the failed shared-build test's comment says what status 130 covers;
- README and docs: the --max-memory row, MCP binaries on macOS, signal
  passing on macOS (apple/container#1876), goals on macOS, the smoke
  gate's Podman requirement, Mac developer builds and the localhost/
  name, the client contract's nested-mount rule, and trims.
claudiomaradonna deleted branch feat/apple-container-macos 2026-10-08 08:46:30 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
claudiomaradonna/azkaban!40
No description provided.