# Crates & API map

What each crate owns, how they compose, and the API index — the map for navigating between the building blocks.

Rindle is a Rust workspace of core crates. The engine is std-only and
wasm-clean; the SQLite backend and live-query wrapper build on top of it.
The main crate (import) names are `rindle`, `rindle_sqlite`, and
`rindle_replica`.

> **What's open, what's commercial.** The core is **open source** (Apache-2.0):
> the `rindle` engine, the `rindle-sqlite` backend, and the JS/TS packages, at
> [`github.com/rindle-sh/rindle`](https://github.com/rindle-sh/rindle) — embed and
> link them freely. The high-concurrency runtime — `rindle-replica` (the live-replica
> `Db`/`Cluster`), the `rindled` read-follower daemon (`rindle-server`), and the
> replicated-fleet crates (`rindle-replicator` and friends) — is commercial: use the npm
> packages and the prebuilt binaries we publish, or get source access under a Rindle
> source license.

| Crate | Import name | Role |
| --- | --- | --- |
| `rindle` | `rindle` | The IVM engine + wire AST + fluent query builder. std-only, no C toolchain. Ships the wasm/JS client (`wasm` feature). |
| `rindle-sqlite` | `rindle_sqlite` | The SQLite server leaf (zero-copy `TableSource`) and spill-to-SQLite operator storage. |
| `rindle-replica` | `rindle_replica` | A live-query wrapper: open a DB, register a query, write SQL through one controlled writer, receive raw incremental change events. Single-thread `Db` or multi-threaded `Cluster`. |
| `rindle-planner` | `rindle_planner` | The cost-based join-flip planner — a pure `Ast` → `Ast` `plan_ast` step. On by default in the replica wrappers; see [how it works](/docs/how-it-works#query-planner). |
| `rindle-server` | — | The `rindled` **read-follower** daemon: the binary + the `net` layer (public ws + private HTTP read/subscribe plane) over the `Cluster`. It has no write ingress — writes go to the replicator. See [run the daemon](/docs/daemon). |
| `rindle-replicator` | — | The **write-master**: a change-source writer with an in-file change log fanned out to the `rindled` followers over websockets. Every deployment is one replicator + one or more followers — the smallest is the [colocated pair](/docs/deploy) on one box. |
| `rindle-cli` | — | The `rindle` binary: inspect/run/manage a daemon, plus `rindle migrate` and `rindle schema gen` (the SQL-first [schema](/docs/schema) loop). Shipped to JS devs as [`@rindle/cli`](/docs/rindle-cli). |

The workspace is rooted at `rindle` (`members = ["rindle-replica", "rindle-dsl", "rindle-sqlite", ...]`),
so every SQLite-linking member links the same vendored SQLite via a shared
`[patch.crates-io]` redirect.

## `rindle` — the engine

You build a query, lower it into a wired dataflow `Graph`, hydrate a materialized
`View`, push source changes, and the view updates incrementally — the contract being
*view-after-push == fresh-query*.

### Build a query

Use the fluent `table` builder, or deserialize an `Ast` from its JSON wire format (via the
`serde` feature). The builder returns an `Ast` from `.build()`:

```rust
use rindle::table;

// Flat: open issues.
let ast = table("issue").r#where("open", true).build();

// Nested: each issue with its comments (a correlated child relationship).
let ast = table("issue")
    .sub_as("comments", |row| {
        table("comment").r#where("issue_id", row.col("id"))
    })
    .build();
```

`row.col("id")` inside a `sub_as` / `sub` closure is a reference to the *parent*
column; passing it to a child `.r#where(childCol, row.col(parentCol))` defines the
correlation. (`r#where` is spelled with the raw-identifier prefix because `where` is
a Rust keyword.)

### Drive the engine

`build_pipeline` lowers an `Ast` into a wired arena `Graph`; in production use the
`try_*` entry points (they return `RindleError` instead of panicking):

```rust
use rindle::{build_pipeline, Graph};

let mut graph = Graph::new();
// build_pipeline(&mut graph, &ast, &resolve) -> Result<NodeId, BuildError>
//   returns the view NodeId. Then:
//   graph.try_add_source(..)   -> register a source row stream
//   graph.try_hydrate(view_id) -> materialize the initial view
//   graph.try_source_push(src_id, change) -> push one SourceChange
```

A `SourceChange` is one of three shapes:

```rust
pub enum SourceChange {
    Add(Row),
    Remove(Row),
    Edit { row: Row, old: Row },
}
```

The downstream `Change` the pipeline emits is a richer tagged union —
`Change::Add`, `Change::Remove`, `Change::Edit { node, old }`, and
`Change::Child { node, rel, child }` (a change to a nested relationship). See
[Change model](/docs/change-model) for the full delta semantics.

### Values & schema

Rows are built from `OwnedValue`. The string constructor is the associated
function `OwnedValue::str`; the others are enum variants:

```rust
use rindle::OwnedValue;

let cells = [
    OwnedValue::Int(1),
    OwnedValue::str("first"),
    OwnedValue::Bool(true),
];
```

`OwnedValue` is `Null | Bool(bool) | Int(i64) | Float(f64) | Str(Arc<str>) | Json(Arc<str>)`
(plus an internal `Absent` marker for partial projected rows — it never reaches a
view, but an exhaustive `match` must cover it). Schema
types come from `Schema`, `RelDef`, `ColId`, `RelId`, and `Value` (the borrowed
form of a cell).

### Read the view

`View::data()` returns a `ViewData`. A subscribed `Listener` is fired once on
hydration and then on every flush, receiving `(&ViewData, ResultType)`. The
materialization types are `View`, `ViewData`, `Entry`, `Listener`, `ResultType`,
and `ViewChange`.

### Storage & the wasm client

Operator spill storage is configured with `StorageFactory` — `StorageFactory::custom`
plugs in a `StorageProvider`; `MemoryStorage` is the default in-memory backend.
With the `wasm` feature, `rindle::wasm::RindleView` drives the whole engine from
JavaScript over a 5-call lifecycle (`build` / `data` / `push` / `flush` /
`subscribe`).

Errors on every fallible path are `rindle::error::RindleError` (re-exported at the crate
root as `RindleError`).

## `rindle-sqlite` — the SQLite backend

Split out so the core engine stays std-only. It exports `TableSource` (a
`rindle::Source` backed by a SQLite table) and an extension trait that adds the source
to a graph:

```rust
use rindle::Graph;
use rindle_sqlite::{TableSource, GraphTableSourceExt};

let mut graph = Graph::new();
// add_table_source delegates to the core Graph::add_dyn_source
let source_id /* : NodeId */ = graph.add_table_source(table_source);
```

`GraphTableSourceExt::add_table_source` returns a `NodeId`. The crate also exports
the spill-to-SQLite operator storage (`DatabaseStorage`, `ClientGroupStorage`,
`OpStorage`, `DatabaseStorageOptions`) — plug it into a graph via
`rindle::StorageFactory::custom` — and the `query_builder` module (the
`FetchRequest` → parameterized `SELECT` lowering).

## `rindle-replica` — live queries off the shelf

`rindle-replica` is the highest-level entry point. It opens a SQLite DB, observes every
mutation through one controlled writer (a raw `sqlite3_preupdate_hook`), and hands
you the **raw incremental change events** the engine emits — it does not materialize
a view for you.

```rust
use rindle::OwnedValue;
use rindle_replica::{ChangeEvent, Db, QueryId, Update};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = Db::open("app.db")?;
    db.register_table("issues")?;

    // Register a live query from an AST (built with rindle::table), under
    // a caller-chosen QueryId tag.
    let open_issues = db.query(
        QueryId(1),
        rindle::table("issues").r#where("open", true).build(),
    )?;

    // subscribe fires once with Update::Hydrated, then Update::Changed per commit.
    open_issues.subscribe(|u| {
        let changes = match u {
            Update::Hydrated { changes, .. } => changes,
            Update::Changed { changes, .. } => changes,
        };
        for ch in changes {
            match ch {
                ChangeEvent::Add(node)            => println!("+ {:?}", node.row),
                ChangeEvent::Remove(node)         => println!("- {:?}", node.row),
                ChangeEvent::Edit { old, row }    => println!("~ {:?} -> {:?}", old, row),
                ChangeEvent::Child { row, .. }    => println!("* child of {:?}", row),
            }
        }
    });

    // Write ordinary SQL through the single controlled writer, then commit.
    let mut w = db.write()?;
    w.exec(
        "INSERT INTO issues VALUES (?,?,?)",
        &[OwnedValue::Int(1), OwnedValue::str("first"), OwnedValue::Bool(true)],
    )?;
    w.commit()?;

    println!("committed tx = {}", db.committed_tx_id().0);
    Ok(())
}
```

Key surface:

- `Db::open(path)` — open a file-backed wal2 SQLite DB. `Db::register_table(name)`
  registers a base table with the engine.
- `Db::query(query_id, ast)` / `Db::query_json(query_id, json)` — register a live
  query under a caller-chosen `QueryId`, returning a `Query`. `Query::subscribe(cb)`
  registers a callback that receives an `Update`; `Query::destroy()` tears it down.
- `Db::write()` — open the single-writer transaction (`WriteTxn`). `WriteTxn::exec`
  runs one parameterized statement, `WriteTxn::commit` derives + delivers the
  incremental events (returning the new `TxId`), `WriteTxn::rollback` discards.
- `Db::read(|conn| …)` — run an ad-hoc `SELECT` against the read-only connection.
- `Db::committed_tx_id()` — the last durably-committed `TxId`.

An `Update` is `Update::Hydrated { tx_id, changes }` (fired once on subscribe with
the initial set) or `Update::Changed { tx_id, changes }` (per committed write). Each
`ChangeEvent` carries a `NodeData` (`.row` plus nested `.relationships`). See
[Replica & views](/docs/replica-and-views) for the write-then-abort derivation model
and [Supported queries](/docs/supported-queries) for the shape matrix.

### Going multi-threaded: `Cluster`

`Db` advances every query on one thread. `Cluster` is the parallel sibling: a single
writer/coordinator plus a pool of IVM worker threads, with queries **sharded**
across workers behind a snapshot/commit barrier. `Cluster::open(path, n_workers)`
returns the handle and a `Receiver<ClusterEvent>` channel (deltas drain off it on
any thread); `register_table`, `query`, and `write` mirror `Db`. A query lives on
exactly one worker, so per-query event order is preserved, and a faulted worker is
respawned rather than corrupting the stream. This is the engine the
[`rindled` daemon](/docs/daemon) runs.

## See also

- [Quickstart](/docs/quickstart) — stand up a live query.
- [Run the daemon](/docs/daemon) — `rindled` over the multi-threaded `Cluster`.
- [How it works](/docs/how-it-works) — the dataflow + incremental-maintenance model.
- [Change model](/docs/change-model) — the delta semantics in depth.

---

[View this page on Rindle](https://rindle.sh/docs/crates)