Function ensure_master_run_id_indexes
pub fn ensure_master_run_id_indexes(
conn: &Connection,
) -> Result<(), BookkeepingError>Expand description
Index the two run_id columns the master’s post-commit path seeks on — master only, and
deliberately not part of ensure_sql_outcomes_table.
backfill_run_cursor runs UPDATE … SET cursor WHERE run_id = ? against BOTH
SQL_OUTCOMES_TABLE and MIGRATIONS_JOURNAL_TABLE after every outcome-bearing commit,
to stamp the engine-assigned cursor on the row it just wrote. run_id is a non-key column of
each, so without these indexes finding one known row means scanning the whole table — an
O(rows) cost on the one-shot write path. On hctree, where every BtreeNext performs an MVCC
visibility lookup, the outcome-cache scan alone dominated the write path and made single-client
insert throughput decay ~25x as the cache filled to
SQL_OUTCOME_MAX_RECORDS.
The migrations journal is far smaller, but it is scanned on the same per-write path and it only ever grows — one row per migration ever applied, with no eviction — so its cost is a slow ratchet over a deployment’s life rather than a bounded one.
Both are scoped to the master because WHERE run_id appears nowhere else: the standalone daemon
never queries either column, so for it these are pure write amplification — measured at ~14% of
its one-shot write throughput on the outcome cache alone, buying nothing.
created_at is left unindexed everywhere. sweep_sql_outcomes filters, deletes and orders by
it, but that pass is amortized to a fixed cadence rather than run per commit, so it costs one
bounded scan per interval (~1% of the post-fix write profile) against a further index paid on
every INSERT.