A Rindle app streams its data over one WebSocket. Around that data plane sits a small set of HTTP surfaces: the control plane. The previous post explained how a browser finds its read follower. This post explains what the HTTP calls on that path do. One example sits at the center: how the api-server creates a query lease for a client.
Three surfaces, one hard rule
The control plane is three HTTP surfaces.
The api-server endpoints are the only HTTP a browser sees: POST /api/rindle/query to lease a
named query, POST /api/rindle/mutate to deliver mutations, and POST /api/rindle/read for a
server-rendered first read. The api-server ships as a library. You mount its handlers on your own
routes, under your own authentication, so the identity model is yours.
The follower read plane is where the api-server manages live queries: /materialize, /query
for a one-shot read, /execute-sql-read, /schema, and /dematerialize. It carries a Bearer
token. A daemon that binds beyond loopback without a token refuses to start.
The master write ingress is where every write enters: /execute-sql-txn, the
/mutate-session/* endpoints, /migrate, and the raw change-apply endpoints. It holds its own
token, separate from the public SQL plane’s token, and it refuses to run with the two equal.
The hard rule: a follower serves no writes. If a write reaches a follower, the follower answers with a 409 that names the master and says why. A write that skips the master’s log diverges the fleet in silence, so the refusal is loud instead. The api-server’s daemon client enforces the same split from its side: writes go to the master’s URL, and reads go to the fleet.
Creating a lease
The client sends POST /api/rindle/query with {name, args}. The client never sends SQL and
never sends a query AST. It names a query that your server defined.
The api-server runs your authorizeQuery hook with the user, the name, and the args. Then it
resolves the name and args into the approved query AST — the authoritative, parsed form of the
query. An unknown name is a 404.
Next, the api-server calls /materialize on a follower with that AST and a retention policy. The
follower canonicalizes the AST and hashes it, together with the schema version and a visibility
key, into a query key. A known key reuses the existing pipeline, and the reply says so with
reused: true. A new key builds one. Either way, the follower mints a fresh lease and returns
{leaseToken, materializationId, queryKey, reused}.
The api-server hands the lease to the browser. The browser subscribes on its WebSocket with the lease token alone. No names and no ASTs travel on the public wire. A subscribe frame carries a token, or the follower refuses it.
browser ── POST /api/rindle/query {name, args} ──▶ api-server
api-server: authorizeQuery(user, name, args)
api-server: resolve (name, args) ──▶ approved AST
api-server ── POST /materialize {ast, policy} ──▶ follower
follower: query key lookup ──▶ reuse or build the pipeline
follower: mint lease ──▶ {leaseToken, queryKey, reused}
browser ◀── {leaseToken, ...} ── api-server
browser ── ws {t:"subscribe", leaseToken} ──▶ followerWhat a lease is
A lease is a short-lived handoff, not a session. The token is
rindle-lease-<n>-<128 random bits>, and the random bits are what stop one client from guessing
another client’s stream. The lease lives in the follower’s RAM and expires after five seconds by
default. There is no renewal endpoint. A renewal is a fresh /materialize, which does one dedup
lookup and mints one new token.
Three things kill a lease: expiry, removal of its materialization, and a follower restart. A dead
lease is a retryable condition, not a terminal one. The subscribe answers with a lease_expired
error and a retry delay, and the client requests a new lease through the normal path.
The pipeline behind the lease has its own lifecycle. The default policy is whileSubscribed: the
pipeline stays warm for 15 seconds after its last subscriber detaches, then reclaims itself. A
pinned query stays warm under a stable name. When two callers ask for the same query with
different policies, pinned wins over idle-based retention, and the longer idle time wins.
The one-shot read shares the machinery
Server rendering uses the same path. /query materializes or reuses the same query key, reads the
assembled view once, and returns rows, a schema block, and a change-version baseline. It registers
no subscriber and discards the lease it minted, so an abandoned request leaks nothing. The
pipeline it leaves warm is the one the browser’s follow-up subscribe attaches to. The first paint
and the live view share one materialization.
Writes ride a watermark
The write side of the control plane is built on a per-client watermark. Every mutation carries a
clientID and a mid, a client-local sequence number. The master stores the highest applied
mid for each client inside the same transaction as the mutation’s effects. The watermark is
data: it replicates through the same ordered stream as the rows, so followers and browsers learn
it the same way.
The watermark makes /execute-sql-txn exactly-once. The master absorbs a replay of an applied
mid: it answers 200 with applied: false and changes nothing. A mid that skips ahead is a
409, and the caller must not retry the same bytes. Delivery can repeat. Application happens once,
in order.
Interactive mutators use /mutate-session/*. A begin carries the accumulated write prefix and
the first read, so a mutator that upgrades to an interactive session pays one round trip. An open session holds the
writer, and a five-second deadline bounds that hold: a wedged api-server costs the write head five
seconds, not forever. A call against a dead session answers 410, and the caller re-enters at
begin, where the watermark absorbs an already-committed outcome.
Conflicts stay inside the master when replay is safe. A write-only transaction that loses
optimistic validation replays mechanically, up to five attempts. A transaction that already
returned rows cannot replay safely. In that case the master answers a 409 marked
retryable: true, and the api-server re-runs the whole mutator, up to five attempts.
Rejection is a write too. When a mutator refuses a mutation, the api-server calls
/reject-mutation, and the master commits the watermark advance alone. That small commit is what
keeps the client’s pending queue draining after a refusal.
Errors tell you what to do
Most errors are one shape: {"error": "a plain-English message"}. Structure appears exactly where
a machine has a decision to make. A retryable conflict is a 409 with code: "retryable-conflict".
An overloaded or shedding node is a 503 with a Retry-After header. A dead session is a 410. The
status tells you whether to retry, and the body tells you why.
Two details do quiet work here. First, every follower response carries a Rindle-Boot-Id header,
stamped at reply time. The api-server watches that header and learns about a follower restart from
traffic it already sends, with no polling. Second, a reply cannot vanish inside the daemon: if the
engine abandons a request, a destructor sends the 503.
The project fence runs before authentication, on purpose. A request from the wrong project answers with a 409 that explains the mismatch. An auth-first order answers with a bare 401 and hides the real cause, because two projects usually hold different tokens.
Backpressure and limits
Each tier caps its concurrent requests at 1024 in flight, and answers 503 with Retry-After: 1
past the cap. Follower reads hold their own budget, so a read flood cannot starve the rest of the
intake. The master caps request bodies and separates two overload answers. It answers an
impossible body with a 413 and a temporarily full memory budget with a 503. One is permanent, one
is retryable, and the status says which. Raw SQL reads run under a watchdog: when a read passes
five seconds or 50 million VM steps, the watchdog interrupts it.
What is built, and what is not
An honest list, in the spirit of the routing post.
/materialize and /query wait on the follower’s engine without a deadline, and the daemon
client sets no request timeout. So today, a wedged follower can hang a lease request instead of
failing it. We wrote a change that puts a deadline on every hop, and it is not released yet.
consistency: "strong" on a SQL read is a client-side routing directive, not a server-side fence.
The daemon client routes a strong read to the master, which reads committed state on a dedicated
connection. No endpoint waits on a replication cursor for you: read-your-writes holds because the
caller sends the read after the write returns. The public SQL transaction plane is the exception —
a read-only transaction can carry a session cursor, and the master fences on it.
Our docs call a lease a one-shot token. The code enforces expiry, not single use. The five-second TTL does that job today.
Bottom line
The control plane is small on purpose. The browser sees a few endpoints on your server, and the api-server holds all authority. It speaks plain HTTP to two daemons: reads to a follower, writes to the master. A lease is a five-second, random, in-RAM handoff from the authorized HTTP world into the streaming one. A watermark makes writes exactly-once over repeated delivery. And when something goes wrong, the status code tells you whether to retry.