Rindle

High-concurrency runtime

Run the daemon (rindled)

rindled — the always-on server you run like Postgres: the config, the two network planes, boot-id restart recovery, and the multi-threaded Cluster engine underneath.

View as Markdown

rindled is the read-serving tier of the Rindle data tier — the always-up server you run like Postgres. It is a read-follower: it holds a SQLite replica of the data and the live IVM pipelines. It derives the incremental delta after every write it receives from the rindle-replicator write-master, and streams normalized, cv-stamped updates to every subscriber. Writes never enter a follower — they go to the write-master. A follower has no write ingress at all. The browser subscribes to a follower’s public WebSocket; your API server reads and materializes over its private HTTP control plane (and sends writes to the write-master).

See the architecture for how the three tiers fit and deploying & scaling for the write-master that feeds it. This page is about running the follower daemon itself.

The binary

rindled lives in the rindle-server crate and ships as a prebuilt, per-platform binary with @rindle/cli. For local dev you rarely invoke it directly — rindle dev renders your rindle.ncl, supervises the master, follower, and rindle-dev-edge, and runs your application with unified bindings. Use rindle up when you deliberately want only that data fleet. It’s the first thing the synced-app quickstart does. This page is the follower daemon in full — the config, the two planes, restart recovery, and the engine underneath — for when you run it yourself.

To run a follower directly (in production, or under your own supervisor), point a release, container, or otherwise supervised rindled binary at a JSON config:

rindled --config follower.json
# …or, with a Rindle source license (rindle-server is a commercial crate):
cargo build -p rindle-server --bin rindled --release
./target/release/rindled --config follower.json

The config declares the follower’s replica file, the two ports, an optional auth token, the worker count, and the change source it tails — the write-master’s fan-out stream:

{
  "db": "follower.db",
  "httpPort": 7600,
  "wsPort": 7601,
  "authToken": "dev-daemon-token",
  "nWorkers": 4,
  "sources": [
    { "kind": "replicator", "name": "rindle-master", "url": "ws://127.0.0.1:7610/subscribe" }
  ]
}
  • db — the follower’s file-backed wal2 SQLite replica (defaults to rindle.db). The write-master owns the authoritative copy.
  • httpPort / wsPort — the control and subscription ports. 0 binds an ephemeral port (handy in tests — the chosen port comes back in the readiness signal).
  • authToken — the follower’s low-level bearer on both planes. Normal applications do not configure a second credential: the fleet edge accepts the one RINDLE_DATABASE_TOKEN and routes trusted calls internally.
  • nWorkers — IVM worker threads in the underlying Cluster (defaults to 2).
  • defaultLeaseTtlMs — how long a materialization lease lives without renewal.
  • sources — exactly one kind:"replicator" change source: the write-master’s fan-out WebSocket (ws://…:7610/subscribe). The follower dials out, tails the totally-ordered change log, and applies it. The schema arrives the same way — DDL replicates from the master — so there’s no table list here. New tables are auto-discovered as the master’s migrations flow through, and rindle schema gen regenerates the client schema from the follower’s /schema.

On a successful start rindled prints exactly one line of JSON to stdout, so a supervisor or test runner can wait on it:

{"ready":true,"httpPort":7600,"wsPort":7601}

Two planes

A follower exposes two network surfaces, kept separate so the untrusted browser plane and the trusted server-to-server plane never share a door:

Plane Port Who connects Carries
Public WebSocket wsPort browser clients the normalized protocol — init, subscribe / unsubscribe, and cv-stamped snapshot + delta frames out
Private HTTP control httpPort your API server reads only — /materialize (mint a query lease), /execute-sql-read (a raw read), /dematerialize, /schema, /version

The write endpoints — /execute-sql-txn, /migrate, /mutate-session/*, /reject-mutation, /apply-row-change-txn — live on the rindle-replicator write-master, not on a follower. If you point writes at a follower, it refuses them with a fail-closed error that names the master. A misrouted write can’t silently vanish.

Both planes require the bearer authToken when one is set. These ports are the operator-level topology, not application configuration. The fleet edge exposes one RINDLE_URL. createRindleApiServer sends the same server-only RINDLE_DATABASE_TOKEN and the edge routes reads here and writes to the master. Browsers never speak the control plane directly. The low-level @rindle/daemon-client package remains the typed client for custom self-hosted routing and supervisor integrations.

Restart recovery: the boot id

rindled keeps no durable materialization state — on restart it has the data (it’s file-backed) but no live queries or pins. So it stamps every control-plane response with a boot id header that changes when it restarts. The HttpRindleDaemonClient surfaces it via onBootId, and your API server re-asserts its pinned queries when it fires:

const daemon = new HttpRindleDaemonClient({
  baseUrl: process.env.RINDLE_URL!,
  headers: { authorization: `Bearer ${process.env.RINDLE_DATABASE_TOKEN!}` },
  onBootId: () => api.assertPins().catch(console.error), // re-warm after a restart
});

The hook rides responses you already make, so there’s no polling — the next control-plane call after a restart re-establishes the warm set.

Under the hood: the Cluster

rindled runs the multi-threaded Cluster engine from rindle-replica. Where the single-thread Db advances every query on one thread, Cluster shards queries across a pool of IVM worker threads behind a single writer/coordinator. On a follower the transactions it applies arrive over the replication stream from the write-master rather than from a client, but the engine mechanics are identical. The per-transaction handshake keeps the parallelism correct:

  1. write opens a BEGIN CONCURRENT on the writer. The preupdate hook captures the row deltas as SQL runs.
  2. commit fans the captured batch to every worker and waits for each to pin its own pre-commit snapshot (the barrier).
  3. The writer commits durably while the workers derive their queries’ deltas concurrently under snapshot isolation.
  4. Each worker emits its affected queries’ deltas, then a progress marker so the drain layer knows the transaction is fully delivered.

A query lives on exactly one worker, so per-query event order is preserved. If a worker faults during derivation, that query is torn down (and the pool respawns the worker) rather than corrupting the stream. The contract is unchanged from the single-thread path: view-after-write == fresh-query. See crates for the Cluster API.

The query planner

The daemon runs the cost-based join-flip planner — it annotates each flippable EXISTS with a flip decision before lowering, and picks the cheaper drive side from a real-SQLite cost model. It is result-preserving (only the work changes, never the rows) and is on by default (Cluster::open enables it). Opting out is in-process only today, via Cluster::open_with_planning(path, n, false), not yet through the config file.

Scope

rindled is the productionizing read server, but it is young. The replica’s schema constraints apply — plain tables (no triggers / generated columns), numbers within ±(2⁵³−1). See replica and views. The database bearer is server-wide. Finer-grained authz lives in your API tier.

This page runs one follower. To scale reads across N affinity-placed followers fed by the write-master, see deploying & scaling. That page shows the whole deployment menu and what you can run yourself versus have us run for you on Rindle Cloud.

Next steps