What happened

In the “Local LLM Nodes” demo, the gate Lambda had to sit on the same server as the DataPoint it read, and that DataPoint had to exist purely to launder an upstream LLM node’s output into a shape the Lambda would accept. Both constraints looked like design limits. Neither was — they were two bugs in one seven-line function.

Root cause

LambdaPythonExecutor.getScriptInput():

1
2
3
val inputNode = nodeManager.readNodeStateOrNull(meta.inputs.first().nodeId).value
val inputMeta = inputNode.meta as DataPointMetaData
val snapshot = dataProcessor.last(inputNode) ?: Snapshot()

1. Local-only resolution. The input is a NodeIdentity — a (nodeId, hostId) pair — but this passed only .nodeId to readNodeStateOrNull, which reads the local store. Every other consumer (LLM inputs, Calculation formulas, trigger samples) goes through nodeManager.findNode(identity), which is hostId-aware and fetches a peer’s node over HTTP. So a Lambda could not read an input on another server — even though the wiring is allowed and the arcs draw in the editor.

2. Blind cast. inputNode.meta as DataPointMetaData threw ClassCastException for any other input type — an LLM node, a webhook, another Lambda.

3. The catch hid both. catch (_: Exception) { nodeManager.failed(node, "Failed to retrieve input node for lambda input") } produced the same message whether the node was missing, unreachable, or simply the wrong type. Two distinct architectural limitations presented as one vague string, which is why they read as intentional.

A third defect fell out of fixing (1): dataProcessor.last() reads the time-series off local disk (/srv/krill/data/<nodeId>). A node that lives on a peer has no local directory, so last() returns null and the original ?: Snapshot() would have flattened a successfully-fetched remote DataPoint to an empty value. Resolving remotely without fixing that would have traded “no value” for “wrong value” — strictly worse.

Fix

• Resolve via nodeManager.findNode(identity) — hostId-aware, matching every other consumer.
• Branch on the meta type instead of casting:
  • local DataPoint → dataProcessor.last() (time-series, with filters and retention applied)
  • everything else — remote DataPoint, LLM, Lambda, webhook — already carries its current value in meta.snapshot, so use it. This is also what makes a remote DataPoint keep the value it arrived with.
• Fail with a message that names the input identity and the underlying cause.

Note DataProcessor.last() performs the same node.meta as DataPointMetaData cast internally, so it must only ever be called for a DataPoint — the when branch is load-bearing, not stylistic.

Prevention

• A NodeIdentity exists because a bare node id is ambiguous in a swarm. Any code that destructures one and uses only .nodeId has silently assumed “local”, and in a peer-to-peer system that assumption is wrong by default. findNode(identity) is the house style; readNodeStateOrNull(id) is a local-store read and should be treated as the exception that needs justifying.
• as on a polymorphic meta is a bug in waiting. NodeMetaData is deliberately polymorphic and any node type can be wired to any input. The type system offered is and the code chose as. Prefer an exhaustive when over a cast — it turns a new node type from a runtime ClassCastException into a compile-time decision.
• A blanket catch (_: Exception) with a fixed message converts bugs into design limits. Here it made two independent defects — no remote inputs, no non-DataPoint inputs — look like one deliberate constraint, which is exactly how they survived into a demo. If a handler cannot say what failed, it is hiding something; include the identity and the cause.
• When you widen where data can come from, re-check every assumption about where it lives. Fixing the remote lookup silently broke the value lookup, because the time-series is local-only. The two are coupled and the coupling was invisible.