test(molecule): run make test in parallel on one image

claudiomaradonna commented

2026-09-29 22:15:57 +02:00

Owner

Why

make test ran the 15 molecule scenarios one after another, each on its
own image rebuilt at every create, and was too slow to run before a role
change. The target was under ~5 minutes on a 20-core machine with the
same scenarios and assertions.

What changed

make test runs the scenarios through tests/molecule/run.sh, ten at
a time (make test J=4 to change it), with one log per scenario in
.molecule-logs/ and a timing table at the end. The first failure,
by exit status or by an ^ERROR line, stops the run, and the runner No changes this session
destroys every scenario that started. Ctrl-C stops and destroys too,
and a second Ctrl-C during cleanup exits at once.
Every scenario boots one image built from tests/molecule/Containerfile:
Debian 12 with systemd and python3. It also carries the roles'
packages, downloaded but not installed, so each role still installs
them for real without fetching them. The 15 Dockerfile.j2 files are
gone. The podman driver used to rebuild one shared tag from a different
Dockerfile at every create, which would have raced in parallel.
openssh-server moved into hardening-basic's prepare.yml.
MOLECULE_ROLES lists the slowest roles first, so scan-share (~300s)
no longer waits for a free slot.
tag_propagation writes its output to .molecule-logs/ and prints
its tail only on failure.
Fixes an older bug in seven verify tasks (firewall-apply, shell,
scan-share): their fail_msg rendered to a list or dict, which
ansible-core 2.21 rejects before evaluating the assertion, so those
tasks failed on every run. They now render as JSON.

Scenarios, assertions and role tasks are otherwise unchanged. The Makefile
keeps its four targets. CONTRIBUTING#molecule describes the runner.

## Why `make test` ran the 15 molecule scenarios one after another, each on its own image rebuilt at every create, and was too slow to run before a role change. The target was under ~5 minutes on a 20-core machine with the same scenarios and assertions. ## What changed - `make test` runs the scenarios through `tests/molecule/run.sh`, ten at a time (`make test J=4` to change it), with one log per scenario in `.molecule-logs/` and a timing table at the end. The first failure, by exit status or by an `^ERROR` line, stops the run, and the runner No changes this session destroys every scenario that started. Ctrl-C stops and destroys too, and a second Ctrl-C during cleanup exits at once. - Every scenario boots one image built from `tests/molecule/Containerfile`: Debian 12 with systemd and python3. It also carries the roles' packages, downloaded but not installed, so each role still installs them for real without fetching them. The 15 `Dockerfile.j2` files are gone. The podman driver used to rebuild one shared tag from a different Dockerfile at every create, which would have raced in parallel. `openssh-server` moved into `hardening-basic`'s `prepare.yml`. - `MOLECULE_ROLES` lists the slowest roles first, so `scan-share` (~300s) no longer waits for a free slot. - `tag_propagation` writes its output to `.molecule-logs/` and prints its tail only on failure. - Fixes an older bug in seven verify tasks (firewall-apply, shell, scan-share): their `fail_msg` rendered to a list or dict, which ansible-core 2.21 rejects before evaluating the assertion, so those tasks failed on every run. They now render as JSON. Scenarios, assertions and role tasks are otherwise unchanged. The Makefile keeps its four targets. CONTRIBUTING#molecule describes the runner.

claudiomaradonna added 11 commits

2026-09-29 22:15:59 +02:00

test(molecule): run scenarios in parallel, fail fast 757735326c

make test ran the 15 molecule scenarios one after another. A runner
now starts each as its own `molecule test -s`, J at a time (default
10), with one log per scenario in .molecule-logs/. The first scenario
that fails, by exit status or by an ^ERROR line, stops the run, and
every started scenario is destroyed. A timing table closes the run, so
`make test J=1` also measures the serial baseline.

test(molecule): boot every scenario from one image 95398ab349

The 15 Dockerfile.j2 files differed only in comments. One Containerfile
under tests/molecule now builds localhost/castrum-molecule:debian12
with systemd and python3 only, which the runner builds before the
first scenario and every scenario boots as a pre-built image. The
podman driver no longer rebuilds an image per scenario.

hardening-basic installs openssh-server in its prepare.yml, so every
other role still starts from a host without sshd. The tag_propagation
gate builds the image too, so it still runs alone.

test(molecule): share apt downloads across scenarios 1db498e05c

Every converge downloaded its role's packages again in a fresh
container. The shared image keeps downloaded packages, and every
scenario mounts the castrum-apt-archives podman volume at
/var/cache/apt/archives, so a package downloads once across scenarios
and runs. Roles still install every package themselves, and each
scenario still fetches fresh package lists.

docs(contributing): describe the parallel molecule runner dc36f71469

The Molecule section now covers what make test runs: J scenarios at a
time with one log each, fail-fast stop and cleanup, the shared image
every scenario boots, the apt download volume, and the per-log ^ERROR
check that keeps molecule's own summary out of the verdict.

fix(molecule): keep Ctrl-C with the runner during cleanup b5568709af

With job control on, each destroy after a failure ran as a foreground
job in its own process group and took the terminal, so Ctrl-C reached
only the destroy in progress and the runner never saw it; the fifteen
destroys also ran one by one with no output. They now run in the
background, all at once, behind a progress line, and a second
interrupt during cleanup exits at once, naming the podman command that
finds leftovers. Scenarios read stdin from /dev/null, so a background
job never touches the terminal.

fix(molecule): show the failure, not the cleanup, on stop bee7d0849d

The failing log gets the destroy output appended during cleanup, so
its last 40 lines showed the destroy instead of the error. The tail is
now kept when the failure is seen.

fix(molecule): bake downloads into the image, drop the volume 9759aa5ee9

The shared castrum-apt-archives volume broke parallel runs: apt does
not wait for /var/cache/apt/archives/lock, so a second container
installing at the same time failed at once ("held by process 0").
The image now carries the packages the roles install, downloaded but
not installed, so each container has its own copy of the cache: roles
still install for real, nothing is fetched during the run, and nothing
is shared between containers.

fix(molecule): render non-string fail_msg values as JSON 7501a3e9db

ansible-core 2.21 checks assert's fail_msg as a string or a list of
strings before it evaluates the assertion, so three verify tasks whose
message was a list of lists (stdout_lines) or of dicts (stat) failed
on every run, whether the assertion held or not. The message is now
the same list rendered as one JSON string.

fix(scan-share): render dict fail_msg values as JSON 9c4216d1f5

Same fault as the previous commit, in scan-share's verify: the probe
file's stat and three rescue-path ansible_failed_result messages were
dicts, which ansible-core 2.21 rejects as fail_msg before evaluating
the assertion. The rescue paths run on every pass of this scenario, so
all four failed each run.

perf(molecule): start the slowest roles first 6cf21b3a0d

scan-share, the longest scenario at ~300s, was last in MOLECULE_ROLES
and waited ~60s for a free slot, which set the whole run at ~370s.
The list now runs slowest first, by the times of the first green run,
so the long scenarios start at once.

test(gates): log tag_propagation beside the scenarios 6547a844ac

The gate printed all of molecule's output to the terminal after the
runner's summary. It now writes it to .molecule-logs/tag_propagation.log
and prints the tail only on failure.

claudiomaradonna merged commit 6547a844ac into main

2026-09-29 22:16:21 +02:00

claudiomaradonna deleted branch test/fast-molecule

2026-09-29 22:16:26 +02:00

Rows
Columns

test(molecule): run make test in parallel on one image #17

Why

What changed