What happened

The Camera node captured frames perfectly and no node in the swarm could see any of them. ServerCameraProcessor shelled out to rpicam-still, wrote the JPEG under /srv/krill/camera/<nodeId>/, pruned to 50 files — and stopped. It never wrote meta.snapshot, never called update(), and never called succeeded().

So a Camera fanned out nothing: a capture woke no observer and triggered no downstream work. Worse, nothing could even name the file, because the path never landed in any node’s state — an LLM node wired to a Camera received CameraMetaData carrying its config (resolution, enabled) and an empty snapshot. The image existed only out-of-band over GET /camera/{id}/snapshot, an endpoint no processor calls.

This is half of why the “Local LLM Nodes” demo had to fake the camera: a text DataPoint held a written description of the frame, and the real photo was composited into the video as an overlay.

Root cause

The processor was written as a capture-to-disk side effect rather than as a data-producing node. Every other producing node ends the same way — build a Snapshot, update() the node with it, post a SNAPSHOT_UPDATE event — because update()’s publication is the observer wake. The camera skipped that ending entirely.

A second, independent bug sat in the failure path:

1
2
3
4
if (exitCode != 0 || !outputFile.exists()) {
    nodeManager.failed(node, "Failed to capture snapshot (exit=$exitCode)")
}   // <-- no return
logger.i { "...: saved snapshot $filename (${outputFile.length()} bytes)" }

No return. A failed capture recorded the error and then fell straight through to log a saved snapshot and run the prune — so a broken camera reported success on the very next line, and outputFile.length() was called on a file that did not exist.

Also: process.waitFor() ran on the caller’s thread while the injected scope sat unused, so every capture blocked whatever invoked the node.

Fix

• Publish: build a Snapshot whose value is the frame’s path, update() the node with it, and post SNAPSHOT_UPDATE — the same ending every other producing node has. Downstream nodes now wake on a new frame and can read the path out of snapshot.value.
• return on capture failure.
• Run the blocking subprocess on the processing scope via launchProcessing, not the caller’s thread — which also routes a thrown error to NodeState.ERROR for free.
• Inject cameraRoot, captureFrame and ioDispatcher, all defaulting to the production values, so the behaviour is testable without a Pi camera and without touching /srv/krill/camera.

The snapshot carries the path, not the bytes. A 1280×720 JPEG is ~200 KB, ~270 KB base64 — putting that in node state would push it through every update(), every SSE frame and every peer sync.

Prevention

• “It works” and “anything can observe that it worked” are different claims. The capture code was correct and complete on its own terms; it just wasn’t wired into the one mechanism that makes a node’s output visible. For any producing node, the test is not “did the side effect happen” but “did update() publish it” — that publication is the wake.
• A node type that ends without update() should be suspicious. Grep for producers that never call it; ServerCameraProcessor was one and nothing flagged it, because nothing failed. A silent no-fan-out looks exactly like a node nobody happened to wire up.
• failed() is not return. An error-recording call that doesn’t stop the function leaves the success path running on failed state. Here it produced a log line claiming a snapshot was saved, sized by File.length() on a nonexistent file. Any failed(...) / setError(...) call that isn’t immediately followed by a return or a throw deserves a second look.
• Injecting a subprocess and a filesystem root costs three default parameters and buys the whole test suite. The alternative was no coverage at all, since CI has no Pi camera and /srv/krill/camera on a Krill host is a real deployed camera’s storage.