Crates & API map

Rindle is a Rust workspace of core crates. The engine is std-only and wasm-clean; the SQLite backend and live-query wrapper build on top of it. The main crate (import) names are rindle, rindle_sqlite, and rindle_replica.

The workspace is rooted at rindle (members = ["rindle-replica", "rindle-dsl", "rindle-sqlite", ...]), so every SQLite-linking member links the same vendored SQLite via a shared [patch.crates-io] redirect.

# rindle — the engine

You build a query, lower it into a wired dataflow Graph, hydrate a materialized View, push source changes, and the view updates incrementally — the contract being view-after-push == fresh-query.

# Build a query

Use the fluent table builder, or deserialize an Ast from its JSON wire format (via the serde feature). The builder returns an Ast from .build():

use rindle::table;

// Flat: open issues.
let ast = table("issue").r#where("open", true).build();

// Nested: each issue with its comments (a correlated child relationship).
let ast = table("issue")
    .sub_as("comments", |row| {
        table("comment").r#where("issue_id", row.col("id"))
    })
    .build();

row.col("id") inside a sub_as / sub closure is a reference to the parent column; passing it to a child .r#where(childCol, row.col(parentCol)) defines the correlation. (r#where is spelled with the raw-identifier prefix because where is a Rust keyword.)

# Drive the engine

build_pipeline lowers an Ast into a wired arena Graph; in production use the try_* entry points (they return RindleError instead of panicking):

use rindle::{build_pipeline, Graph};

let mut graph = Graph::new();
// build_pipeline(&mut graph, &ast, &resolve) -> Result<NodeId, BuildError>
//   returns the view NodeId. Then:
//   graph.try_add_source(..)   -> register a source row stream
//   graph.try_hydrate(view_id) -> materialize the initial view
//   graph.try_source_push(src_id, change) -> push one SourceChange

A SourceChange is one of three shapes:

pub enum SourceChange {
    Add(Row),
    Remove(Row),
    Edit { row: Row, old: Row },
}

The downstream Change the pipeline emits is a richer tagged union — Change::Add, Change::Remove, Change::Edit { node, old }, and Change::Child { node, rel, child } (a change to a nested relationship). See Change model for the full delta semantics.

# Values & schema

Rows are built from OwnedValue. The string constructor is the associated function OwnedValue::str; the others are enum variants:

use rindle::OwnedValue;

let cells = [
    OwnedValue::Int(1),
    OwnedValue::str("first"),
    OwnedValue::Bool(true),
];

OwnedValue is Null | Bool(bool) | Int(i64) | Float(f64) | Str(Arc<str>). Schema types come from Schema, RelDef, ColId, RelId, and Value (the borrowed form of a cell).

# Read the view

View::data() returns a ViewData. A subscribed Listener is fired once on hydration and then on every flush, receiving (&ViewData, ResultType). The materialization types are View, ViewData, Entry, Listener, ResultType, and ViewChange.

# Storage & the wasm client

Operator spill storage is configured with StorageFactory — StorageFactory::custom plugs in a StorageProvider; MemoryStorage is the default in-memory backend. With the wasm feature, rindle::wasm::RindleView drives the whole engine from JavaScript over a 5-call lifecycle (build / data / push / flush / subscribe).

Errors on every fallible path are rindle::error::RindleError (re-exported at the crate root as RindleError).

# rindle-sqlite — the SQLite backend

Split out so the core engine stays std-only. It exports TableSource (a rindle::Source backed by a SQLite table) and an extension trait that adds the source to a graph:

use rindle::Graph;
use rindle_sqlite::{TableSource, GraphTableSourceExt};

let mut graph = Graph::new();
// add_table_source delegates to the core Graph::add_dyn_source
let source_id /* : NodeId */ = graph.add_table_source(table_source);

GraphTableSourceExt::add_table_source returns a NodeId. The crate also exports the spill-to-SQLite operator storage (DatabaseStorage, ClientGroupStorage, OpStorage, DatabaseStorageOptions) — plug it into a graph via rindle::StorageFactory::custom — and the query_builder module (the FetchRequest → parameterized SELECT lowering).

# rindle-replica — live queries off the shelf

rindle-replica is the highest-level entry point. It opens a SQLite DB, observes every mutation through one controlled writer (a raw sqlite3_preupdate_hook), and hands you the raw incremental change events the engine emits — it does not materialize a view for you.

use rindle::OwnedValue;
use rindle_replica::{ChangeEvent, Db, QueryId, Update};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = Db::open("app.db")?;
    db.register_table("issues")?;

    // Register a live query from an AST (built with rindle::table), under
    // a caller-chosen QueryId tag.
    let open_issues = db.query(
        QueryId(1),
        rindle::table("issues").r#where("open", true).build(),
    )?;

    // subscribe fires once with Update::Hydrated, then Update::Changed per commit.
    open_issues.subscribe(|u| {
        let changes = match u {
            Update::Hydrated { changes, .. } => changes,
            Update::Changed { changes, .. } => changes,
        };
        for ch in changes {
            match ch {
                ChangeEvent::Add(node)            => println!("+ {:?}", node.row),
                ChangeEvent::Remove(node)         => println!("- {:?}", node.row),
                ChangeEvent::Edit { old, row }    => println!("~ {:?} -> {:?}", old, row),
                ChangeEvent::Child { row, .. }    => println!("* child of {:?}", row),
            }
        }
    });

    // Write ordinary SQL through the single controlled writer, then commit.
    let mut w = db.write()?;
    w.exec(
        "INSERT INTO issues VALUES (?,?,?)",
        &[OwnedValue::Int(1), OwnedValue::str("first"), OwnedValue::Bool(true)],
    )?;
    w.commit()?;

    println!("committed tx = {}", db.committed_tx_id().0);
    Ok(())
}

Key surface:

• Db::open(path) — open a file-backed wal2 SQLite DB. Db::register_table(name) registers a base table with the engine.
• Db::query(query_id, ast) / Db::query_json(query_id, json) — register a live query under a caller-chosen QueryId, returning a Query. Query::subscribe(cb) registers a callback that receives an Update; Query::destroy() tears it down.
• Db::write() — open the single-writer transaction (WriteTxn). WriteTxn::exec runs one parameterized statement, WriteTxn::commit derives + delivers the incremental events (returning the new TxId), WriteTxn::rollback discards.
• Db::read(|conn| …) — run an ad-hoc SELECT against the read-only connection.
• Db::committed_tx_id() — the last durably-committed TxId.

An Update is Update::Hydrated { tx_id, changes } (fired once on subscribe with the initial set) or Update::Changed { tx_id, changes } (per committed write). Each ChangeEvent carries a NodeData (.row plus nested .relationships). See Replica & views for the write-then-abort derivation model and Supported queries for the shape matrix.

# Going multi-threaded: Cluster

Db advances every query on one thread. Cluster is the parallel sibling: a single writer/coordinator plus a pool of IVM worker threads, with queries sharded across workers behind a snapshot/commit barrier. Cluster::open(path, n_workers) returns the handle and a Receiver<ClusterEvent> channel (deltas drain off it on any thread); register_table, query, and write mirror Db. A query lives on exactly one worker, so per-query event order is preserved, and a faulted worker is respawned rather than corrupting the stream. This is the engine the rindled daemon runs.

# See also

• Quickstart — stand up a live query.
• Run the daemon — rindled over the multi-threaded Cluster.
• How it works — the dataflow + incremental-maintenance model.
• Change model — the delta semantics in depth.