Symptom

Kraken nightly scan hypothesised that propagationContextTL (a ThreadLocal) could carry a stale or absent PropagationContext inside processor coroutines, weakening cycle-dedupe (D7), cross-host hop counting (D11), and RESET suppression (D6). The specific confirmed bug: ServerLLMProcessor.clearSnapshot() called nodeManager.update() from a RESET invocation and incorrectly published to observers that should have been suppressed.

Root cause

PropagationContextElement (a ThreadContextElement) correctly installs / restores propagationContextTL on the thread for any coroutine whose context contains the element. The element IS propagated to direct structured children launched with unqualified launch {} inside a coroutine body.

However, every server processor called scope.launchProcessing(nodeManager, node) {} or bare scope.launch {} using the injected scope as the receiver. The injected scope is a long-lived application scope whose coroutine context never contains PropagationContextElement. Calls like scope.launchProcessing(...) therefore create sibling coroutines of the invoke() coroutine — the PropagationContextElement installed by invoke() is never inherited.

Inside those siblings propagationContextTL.get() returns null, so:

• update() generated a fresh epoch instead of the parent’s (D7 cycle-dedupe broken for single-host cycles)
• hopTtl defaulted to DEFAULT_HOP_TTL instead of being decremented (D11 cross-host bound reset)
• suppressPublish = true (RESET verb) was silently lost in update(), allowing RESET to fan out to observers (D6 / D11 violated — confirmed for ServerLLMProcessor)

Fix

ProcessingScope.launchProcessing now captures propagationContextTL.get() synchronously at call time (while the caller’s PropagationContextElement is still pinned on the thread), then passes a new PropagationContextElement wrapping the captured context into the launched coroutine:

1
2
3
4
5
fun CoroutineScope.launchProcessing(...): Job {
    val capturedCtx = propagationContextTL.get()
    val extra = capturedCtx?.let { PropagationContextElement(it) } ?: EmptyCoroutineContext
    return launch(extra) { try { block() } catch ... }
}

Five processors that used bare scope.launch {} for propagation-chain work were migrated to scope.launchProcessing(nodeManager, node) {}: ServerComputeProcessor, ServerCalculationProcessor, ServerLLMProcessor, ServerSMTPProcessor, ServerTaskListProcessor. Three processors already using launchProcessing (ServerDataPointProcessor, WebHookOutboundProcessor, ServerTriggerProcessor) are auto-fixed by the launchProcessing change.

Two regression tests added to PropagationContextPropagationTest: one verifying epoch inheritance, one verifying suppressPublish propagation through launchProcessing.

Prevention

• When a processor’s process() method needs async work, always use scope.launchProcessing(nodeManager, node) {} — never bare scope.launch {}. The launchProcessing helper is the only launch site that correctly captures and forwards the propagation context.
• Background daemons (CronTask, TimerBoss, SerialDirectoryMonitor, etc.) that initiate propagation chains (self-fire via invoke()) rather than participate in them can continue using scope.launch {} since they have no ambient PropagationContextElement to inherit.
• The key distinction: if the coroutine will call nodeManager.update(), nodeManager.succeeded(), or nodeManager.publish() — use launchProcessing. If it only calls EventFlowContainer.tryPostEvent() or manages a long-lived subscription, bare scope.launch {} is acceptable.