Symptom

A nightly architectural bug-hunt flagged “inconsistent error handling” at the KrillApp dispatch seam, predicting that new KrillApp subtypes would fail silently in KrillApp.processor() while crashing in KrillApp.meta().

Most of that prediction was wrong, but the title was right for a different reason: client-side processor failures were being logged as a bare "Uncaught exception in IO_SCOPE" — no node id, no state, no stage — despite the seam appearing to have a guard.

Root cause

Two separate things, only one of them a defect.

Not a defect. KrillApp.processor() is a when expression over a sealed hierarchy, so exhaustiveness is compiler-enforced; a new subtype is a build break, not a silent runtime gap. (This repo was in exactly that state when the issue was worked: krill-sdk 0.0.63 added KrillApp.Swarm{Work,Batch} and the build failed to compile until the arms were added — the guard working as designed.) The asymmetry where processor() throws for Avatar / MenuCommand while meta() returns a placeholder is deliberate, documented, and pinned by KrillAppEmitExhaustiveTest + KrillAppMetaContractTest since #567: meta() is called on arbitrary types by node lists and editors that cannot guard, whereas reaching emit() with a UI-only type is a wiring bug that should be loud.

The actual defect. UniversalAppNodeProcessor.process() launches its entire body onto the shared IO_SCOPE and returns immediately. Every caller-side guard at the seam was therefore structurally incapable of catching a processor failure:

• DefaultNodeObserver’s collector wraps dispatch(collectedNode) in a try/catch that logs with node context, and a test pins that behaviour. But process() had already returned by the time the launched body ran, so that catch could only ever see a synchronous processor() resolution throw.
• Anything failing inside the body (nodeManager.reset, remove, addInteraction, observer.remove) landed on the IO_SCOPE CoroutineExceptionHandler. Not swallowed — but unattributable, which is the same observability loss in practice.
• showActivity()’s inner launch had a narrower version of the same hole: nodeManager.startPairing(node) sat outside its try/catch and escaped the same way, and the catch it did have swallowed CancellationException along with real failures.

A guard placed on the near side of a launch boundary protects nothing on the far side. The seam looked defended and was not.

Fix

Added dispatchGuarded(node, stage, onFailure, block) in shared/src/commonMain/kotlin/krill/zone/shared/UniversalAppNodeProcessor.kt, stating the client dispatch error contract once: CancellationException is structural and always rethrown; any other failure is reported to an injected sink with the node’s identity and the stage that failed, and does not propagate. Applied it to process()’s launched body and to showActivity()’s inner launch (which now covers startPairing). onFailure is a parameter rather than a hardcoded logger call so the contract is unit-testable without Koin, a ClientNodeManager, or a logging backend. Also corrected the KDoc on KrillApp.emit() in KrillAppEmit.kt, which claimed the function was a “no-op when the type has no processor” and referenced a non-existent NodeProcessor.post — it throws, and the async caveat is now stated explicitly.

Prevention

• A try/catch on the near side of a launch is not a guard. When reviewing error handling, check whether the guarded call is synchronous. If the callee’s first statement is scope.launch { … }, the caller’s catch sees only the launch itself. The callee owns its own contract.
• Fire-and-forget coroutines need a named failure sink, not a scope handler. A scope-level CoroutineExceptionHandler keeps the process alive but discards every piece of context that makes the log actionable. Report at the point of failure, with the domain identity (here: node id + state + stage).
• Make the failure sink a parameter. Injecting onFailure is what let this contract be tested with virtual time and zero environment — no Koin, no filesystem, no concrete ClientNodeManager. A hardcoded logger.e would have been untestable and the contract would have stayed unpinned.
• catch (e: Exception) around suspending code swallows cancellation. CancellationException is an Exception; catching it broadly breaks structured concurrency. Rethrow it explicitly before the general arm.
• Treat automated architectural leads as hypotheses, and check the compiler first. Three of this lead’s five evidence bullets were refuted in minutes by reading the code: two claimed non-exhaustive whens that the Kotlin compiler enforces, and one claimed a missing enum arm that was present. Confirm or refute each bullet before designing a fix, or you implement a defence against a failure mode the language already prevents.