Rindle

High-concurrency runtime

Run the daemon (rindled)

rindled — the always-on server you run like Postgres: the config, the two network planes, boot-id restart recovery, and the multi-threaded Cluster engine underneath.

View as Markdown

rindled is the read-serving tier of the Rindle data tier — the always-up server you run like Postgres. It is a read-follower: it holds a SQLite replica of the data and the live IVM pipelines, derives the incremental delta after every write it receives from the rindle-replicator write-master, and streams normalized, cv-stamped updates to every subscriber. Writes never enter a follower — they go to the write-master; a follower has no write ingress at all. The browser subscribes to a follower’s public WebSocket; your API server reads and materializes over its private HTTP control plane (and sends writes to the write-master).

See the architecture for how the three tiers fit and deploying & scaling for the write-master that feeds it; this page is about running the follower daemon itself.

The binary

rindled lives in the rindle-server crate and ships as a prebuilt, per-platform binary with @rindle/cli. For local dev you rarely invoke it directly — rindle up renders your rindle.ncl and supervises the whole pair (the rindle-replicator write-master and its follower) for you. It’s the first thing the synced-app quickstart does. This page is the follower daemon in full — the config, the two planes, restart recovery, and the engine underneath — for when you run it yourself.

To run a follower directly (in production, or under your own supervisor), point a release, container, or otherwise supervised rindled binary at a JSON config:

rindled --config follower.json
# …or, with a Rindle source license (rindle-server is a commercial crate):
cargo build -p rindle-server --bin rindled --release
./target/release/rindled --config follower.json

The config declares the follower’s replica file, the two ports, an optional auth token, the worker count, and the change source it tails — the write-master’s fan-out stream:

{
  "db": "follower.db",
  "httpPort": 7600,
  "wsPort": 7601,
  "authToken": "dev-daemon-token",
  "nWorkers": 4,
  "sources": [
    { "kind": "replicator", "name": "rindle-master", "url": "ws://127.0.0.1:7610/subscribe" }
  ]
}
  • db — the follower’s file-backed wal2 SQLite replica (defaults to rindle.db); the write-master owns the authoritative copy.
  • httpPort / wsPort — the control and subscription ports. 0 binds an ephemeral port (handy in tests; the chosen port comes back in the readiness signal).
  • authToken — a bearer token required on both planes. Omit it for open local dev.
  • nWorkers — IVM worker threads in the underlying Cluster (defaults to 2).
  • defaultLeaseTtlMs — how long a materialization lease lives without renewal.
  • sources — exactly one kind:"replicator" change source: the write-master’s fan-out WebSocket (ws://…:7610/subscribe). The follower dials out, tails the totally-ordered change log, and applies it. The schema arrives the same way — DDL replicates from the master — so there’s no table list here: new tables are auto-discovered as the master’s migrations flow through, and rindle schema gen regenerates the client schema from the follower’s /schema.

On a successful start rindled prints exactly one line of JSON to stdout, so a supervisor or test runner can wait on it:

{"ready":true,"httpPort":7600,"wsPort":7601}

Two planes

A follower exposes two network surfaces, kept separate so the untrusted browser plane and the trusted server-to-server plane never share a door:

Plane Port Who connects Carries
Public WebSocket wsPort browser clients the normalized protocol — init, subscribe / unsubscribe, and cv-stamped snapshot + delta frames out
Private HTTP control httpPort your API server reads only — /materialize (mint a query lease), /execute-sql-read (a raw read), /dematerialize, /schema, /version

The write endpoints — /execute-sql-txn, /migrate, /mutate-session/*, /reject-mutation, /apply-row-change-txn — live on the rindle-replicator write-master, not on a follower. Point writes at a follower and it refuses them with a fail-closed error that names the master, so a misrouted write can’t silently vanish.

Both planes require the bearer authToken when one is set. Clients never speak the control plane directly — they go through your API tier, which holds the query registry and the authoritative mutators and routes reads to the follower and writes to the write-master (its SplitDaemonClient does the split). The @rindle/daemon-client package is the typed client for the HTTP plane.

Restart recovery: the boot id

rindled keeps no durable materialization state — on restart it has the data (it’s file-backed) but no live queries or pins. So it stamps every control-plane response with a boot id header that changes when it restarts. The HttpRindleDaemonClient surfaces it via onBootId, and your API server re-asserts its pinned queries when it fires:

const daemon = new HttpRindleDaemonClient({
  baseUrl: "http://127.0.0.1:7600",
  headers: { authorization: `Bearer ${token}` },
  onBootId: () => api.assertPins().catch(console.error), // re-warm after a restart
});

The hook rides responses you already make, so there’s no polling — the next control-plane call after a restart re-establishes the warm set.

Under the hood: the Cluster

rindled runs the multi-threaded Cluster engine from rindle-replica. Where the single-thread Db advances every query on one thread, Cluster shards queries across a pool of IVM worker threads behind a single writer/coordinator. On a follower the transactions it applies arrive over the replication stream from the write-master rather than from a client, but the engine mechanics are identical. The per-transaction handshake keeps the parallelism correct:

  1. write opens a BEGIN CONCURRENT on the writer; the preupdate hook captures the row deltas as SQL runs.
  2. commit fans the captured batch to every worker and waits for each to pin its own pre-commit snapshot (the barrier).
  3. The writer commits durably while the workers derive their queries’ deltas concurrently under snapshot isolation.
  4. Each worker emits its affected queries’ deltas, then a progress marker so the drain layer knows the transaction is fully delivered.

A query lives on exactly one worker, so per-query event order is preserved. If a worker faults during derivation, that query is torn down (and the pool respawns the worker) rather than corrupting the stream. The contract is unchanged from the single-thread path: view-after-write == fresh-query. See crates for the Cluster API.

The query planner

The daemon runs the cost-based join-flip planner — it annotates each flippable EXISTS with a flip decision before lowering, picking the cheaper drive side from a real-SQLite cost model. It is result-preserving (only the work changes, never the rows) and is on by default (Cluster::open enables it); opting out is in-process only today, via Cluster::open_with_planning(path, n, false), not yet through the config file.

Scope

rindled is the productionizing server, but it is young. The replica’s schema constraints apply — plain tables (no triggers / generated columns), numbers within ±(2⁵³−1); see replica and views. The bearer token is the only auth primitive; finer-grained authz lives in your API tier.

This page runs one follower. To scale reads across N affinity-placed followers fed by the write-master, see deploying & scaling, which lays out the whole deployment menu and what you can run yourself versus have us run for you on Rindle Cloud.

Next steps