Pollen

Why "Pollen"

Pollen travels without a central coordinator: carried from flower to flower by autonomous carriers, it transports the most valuable thing to propagate — genetic information. Each carrier knows where to go, the pollen knows what to become. That is exactly the execution model we're building: autonomous nodes that emit and receive directly, on a network where coordination lives in the shared data (the workflow.json), not in a conductor.

Two layers: a peer-to-peer bus and a workflow orchestrator

Pollen is unlike RabbitMQ, Kafka or NATS — all central brokers. Pollen is closer to a mix of ZeroMQ (for transport) and Argo Workflows (for orchestration), fully decentralized:

• Transport — direct UDP server to server, no TCP handshake, no intermediary. The shortest path between two processes on the same LAN.
• Coordination — a workflow.json file on a shared network mount. It describes the nodes (who does what) and their topology (who follows whom). Each server reads this file on startup, learns which events are meant for it and where to send the result.

The workflow.json is the orchestrator. There is no "orchestrator" process to run anywhere. The file is declarative, the nodes are autonomous — each one knows what to do at every step.

workflow.json — the declarative orchestrator

The core idea: no process decides who does what. The decision lives in a JSON file dropped on a network mount visible to every node (NFS, SMB, cluster share). Each server reads it and applies its part.

// workflow.json — example: temperature processing chain
{
  "name": "telemetry-pipeline",
  "version": 3,
  "nodes": {
    "acquisition-1": {
      "host": "sensor-gw-01.lan", "port": 5000,
      "emits":    ["temperature.raw"],
      "next":     ["filter-1"]
    },
    "filter-1": {
      "host": "proc-01.lan", "port": 5000,
      "consumes": ["temperature.raw"],
      "emits":    ["temperature.filtered"],
      "next":     ["aggregator-1", "archive-1"]
    },
    "aggregator-1": {
      "host": "proc-02.lan", "port": 5000,
      "consumes": ["temperature.filtered"],
      "emits":    ["temperature.minute-avg"],
      "next":     ["dashboard-1"]
    },
    "archive-1": {
      "host": "storage-01.lan", "port": 5000,
      "consumes": ["temperature.filtered"]
    },
    "dashboard-1": {
      "host": "web-01.lan", "port": 5000,
      "consumes": ["temperature.minute-avg"]
    }
  }
}

Every Pollen server, on startup:

• ◐Reads workflow.json from the shared network mount
• ◐Identifies which node(s) it owns (matching on host + port)
• ◐Automatically subscribes to consumes topics and configures next destinations for its emissions
• ◐Listens on UDP for messages on its topics, applies its processing, re-emits to the next nodes
• ◐Watches workflow.json via amalgame-io-filewatcher — any change reloads the topology hot, without restarting services

Changing the topology = editing a JSON file. Adding a node = adding an entry. Re-routing a flow = changing next. No redeploy, no management API, no central console.

Architecture (taken from the TARMeule prototype)

The Node.js implementation (github.com/BastienMOUGET/TARMeule) is the functional reference for the Amalgame port. Concepts:

Target Amalgame API — a node following the workflow

App-side code is minimal: declare your name in the workflow, register a function per consumed topic, and Pollen does the rest (reading the shared JSON, subscriptions, UDP routing to the next nodes).

namespace App
import Amalgame.Pollen

public class Program {
    public static void Main(string[] args) {

        // Initialize the node (reads config.json + shared workflow.json)
        let node = Pollen.Node("filter-1", configDir: "./config")

        // Register a handler for the consumed topic.
        // Forwarding to "next" nodes is done automatically by Pollen
        // based on the handler's return value.
        node.On("temperature.raw", fn(msg) {
            let v = msg.Get("value") as float
            if (v < -50.0 || v > 150.0) {
                return Pollen.Drop()      // out of range → ignored
            }
            return Pollen.Emit("temperature.filtered", {
                value:  v,
                unit:   msg.GetString("unit"),
                source: msg.Source
            })
        })

        node.OnTimeout(fn(msgId) {
            log.Warn("ACK missed for {msgId}")
        })

        node.Run()    // UDP loop + filewatch on workflow.json
    }
}

The lower-level layer (inherited from TARMeule) stays available if you need free pub/sub without a workflow:

// "Raw" mode — direct pub/sub, no workflow.json
let bus = Pollen.Raw("./config")
bus.UpsertTopic(null, "temperature", 1, { value: "number" })
bus.UpsertSubscription({ ip: "192.168.1.143", port: 5000, topics: ["temperature"] })
bus.SendMessage({ uuid: topicId, version: 1 }, { value: 25 })

Why port it to Amalgame

The Node.js prototype works, but it's heavy to deploy on the target machines (sensors, industrial gateways, PLCs): you need Node.js and its 50 MB runtime, you have to manage process keep-alive, run npm install, etc.

The Amalgame port will deliver:

• ◐A single native binary — ./pollen-node, ~3 MB, starts in milliseconds
• ◐No runtime to install — deployable on Raspberry Pi, Linux PLCs, Windows embedded
• ◐Minimal UDP latency — no Node.js GC pauses, bounded RAM
• ◐amalgame-service integration — native systemd daemon or Windows service
• ◐Encryption via amalgame-crypto — shares the same AES code as the rest of the ecosystem (once Argon2id/AES land)
• ◐Typed Amalgame API — topic schemas validated at build time, no more runtime structure bugs

Planned roadmap

Three phases, preconditioned on amalgame-crypto shipping AES (scheduled in Mosaic phase 2.1).

What Pollen is not

• Not a durable broker — no persistent queue, no Kafka-style replay. A missed message is lost (except via retries within the ACK window).
• Not a full Temporal / Argo Workflows — no execution history, no stateful retry-with-backoff, no transactional saga. The workflow is topological (who talks to whom), not sequential-stateful (step A then B then C with rollback).
• Not multi-datacenter — UDP rarely traverses an Internet firewall. Pollen targets LAN or VPN-LAN.
• Not a Mosaic replacement — to expose a web API to off-LAN clients, you want Mosaic.
• Not a transactional MOM — no atomic cross-topic transactions, no dead-letter queue, no global ordering guarantees.

For those cases, the ecosystem provides the right third-party tools: Kafka for durable replay, RabbitMQ for transactional workflows, MQTT for centralized-broker IoT. Pollen plays in the "real-time LAN pipeline, no broker, no orchestration server" slot.

Not yet implemented

Pollen is currently a working Node.js prototype (TARMeule) and a port intent. The current priority remains Mosaic; Pollen starts once amalgame-crypto ships AES.