# Replicant v0.3.1 - Table of Contents

> Framework-agnostic Elixir CDC consumer for Postgres logical replication (pgoutput) with sink-owned, transaction-granularity exactly-once delivery.

## Pages

- [Replicant](readme.md)
- [Changelog](changelog.md)

- Guides
  - [Getting started with Replicant](getting_started.md)
  - [replicant usage rules](usage-rules.md)

- Project
  - [Contributing to Replicant](contributing.md)
  - [Replicant — AI Agent & Contributor Guide](agents.md)
  - [LICENSE](license.md)
  - [NOTICE](notice.md)

## Modules

- [Replicant.Change.Column](Replicant.Change.Column.md): Per-column metadata carried on a `Change` (mirrors the Relation message).
- [Replicant.Decoder.Messages.Begin](Replicant.Decoder.Messages.Begin.md): `B` — start of a transaction; carries the final LSN, commit timestamp, and xid.
- [Replicant.Decoder.Messages.Commit](Replicant.Decoder.Messages.Commit.md): `C` — end of a transaction; carries the commit LSN and timestamp.
- [Replicant.Decoder.Messages.Delete](Replicant.Decoder.Messages.Delete.md): `D` — a row delete; carries either the key or the full old tuple, depending on `REPLICA IDENTITY`.
- [Replicant.Decoder.Messages.Insert](Replicant.Decoder.Messages.Insert.md): `I` — a row insert; `tuple_data` holds the new values. `xid` is the (sub)transaction id when streamed (spec §5), else nil.
- [Replicant.Decoder.Messages.Message](Replicant.Decoder.Messages.Message.md): `M` — a logical-decoding message emitted by `pg_logical_emit_message`. A TRANSACTIONAL message
(`flags = 1`) arrives bracketed by Begin/Commit (or inside a streamed txn's S/E/c) and rides
`%Transaction.messages`; a NON-TRANSACTIONAL message (`flags = 0`) arrives standalone and
delivers via `c:Replicant.Sink.handle_message/2`. `xid` is set only for a streamed message.
`ordinal` is set by the assembler for a transactional message (spec §7.1).

- [Replicant.Decoder.Messages.Origin](Replicant.Decoder.Messages.Origin.md): `O` — origin marker emitted when a transaction has a logical replication origin.
- [Replicant.Decoder.Messages.Relation](Replicant.Decoder.Messages.Relation.md): `R` — relation (table) metadata: identifier, schema, name, replica identity, columns.
- [Replicant.Decoder.Messages.Relation.Column](Replicant.Decoder.Messages.Relation.Column.md): Per-column metadata within a `Relation` message.
- [Replicant.Decoder.Messages.StreamAbort](Replicant.Decoder.Messages.StreamAbort.md): `A` — aborts a streamed (sub)transaction (spec §5); `subxid == xid` is a whole-transaction abort, else a savepoint rollback.
- [Replicant.Decoder.Messages.StreamCommit](Replicant.Decoder.Messages.StreamCommit.md): `c` — commits a streamed transaction (spec §5); carries the commit LSN.
- [Replicant.Decoder.Messages.StreamStart](Replicant.Decoder.Messages.StreamStart.md): `S` — begins/continues a streamed transaction's segment (spec §5); carries the top-level xid and first-segment flag.
- [Replicant.Decoder.Messages.StreamStop](Replicant.Decoder.Messages.StreamStop.md): `E` — ends a streamed transaction's segment (spec §5); no fields.
- [Replicant.Decoder.Messages.Truncate](Replicant.Decoder.Messages.Truncate.md): `T` — a `TRUNCATE` of one or more relations; options indicate `CASCADE` / `RESTART IDENTITY`.
- [Replicant.Decoder.Messages.Type](Replicant.Decoder.Messages.Type.md): `Y` — a type registration for a non-built-in OID.
- [Replicant.Decoder.Messages.Unsupported](Replicant.Decoder.Messages.Unsupported.md): Catch-all for replication messages Replicant does not yet decode; carries the raw binary.
- [Replicant.Decoder.Messages.Update](Replicant.Decoder.Messages.Update.md): `U` — a row update; carries new values and, if available, the key or old tuple.
- [Replicant.SnapshotProgress](Replicant.SnapshotProgress.md): The incremental-backfill progress token (spec §6.2): a pure struct tracking the
table queue, the in-progress table and its last delivered PK bound, and completion.
- [Replicant.SnapshotWindow](Replicant.SnapshotWindow.md): Pure window algebra for the incremental backfill (spec §2/§4): per-table PK
tracking opened BEFORE the reader captures LW (superset drop-set), pending
chunks that become ready when the epoch-guarded frontier passes their HW, the
drop-filter at apply, the derived drop-cap (discard + re-read on breach), and
the pending-chunk capacity bound.
- [Replicant.Snapshotter.Incremental](Replicant.Snapshotter.Incremental.md): The incremental-backfill chunk reader (spec §4): a `spawn_link`'ed process (dies
with the Connection — parent-spec closeout §17 precedent) on its OWN Postgrex
connection.

- Core API
  - [Replicant](Replicant.md): Framework-agnostic Elixir CDC consumer for Postgres logical replication
(`pgoutput`), delivering committed row changes to a pluggable **sink** with
**zero data loss**: the replication slot advances only after the sink has
durably persisted the transaction.
  - [Replicant.Config](Replicant.Config.md): Validates `Replicant.start_link/1` options and enforces the **go-forward-only
start guard**: a `:state_mirror` sink resuming from an empty
checkpoint without `go_forward_only: true` would silently deliver partial data
from the slot's creation point, so it is refused at start.
  - [Replicant.Sink](Replicant.Sink.md): The pluggable sink contract. A sink durably persists a transaction
AND its checkpoint atomically, returns the commit LSN, and is idempotent on the
transaction-granularity watermark (skip any `txn.commit_lsn <= checkpoint`;
upsert rows by table PK).

- Data structures
  - [Replicant.Change](Replicant.Change.md): One row change within a `Replicant.Transaction`.
  - [Replicant.Error](Replicant.Error.md): A typed, value-free error.
  - [Replicant.SchemaChange](Replicant.SchemaChange.md): Classification of a `Relation` diff (spec §9): **additive** (new column —
auto-apply) or **destructive** (dropped column, type change, replica-identity
change — delegate to the sink or halt fail-closed). A `Truncate` is delivered
as a change, not a schema change.
  - [Replicant.Transaction](Replicant.Transaction.md): A decoded, committed transaction: an ordered list of `Replicant.Change` rows
plus the **transaction-granularity commit LSN** (every row in a
pgoutput proto-v1 transaction shares one commit LSN — a per-row LSN would
collapse an N-row transaction into one row).

- Observability
  - [Replicant.Telemetry](Replicant.Telemetry.md): Value-free `:telemetry.span/3` wrapper. Owns the metadata allowlist — the single
enforcement point for "no row values in telemetry" (Critical Rule 1). LSNs,
table names, counts, durations, and error classes are permitted; an
off-allowlist key raises rather than shipping a value downstream. Mirrors
`arcadic`'s allowlist pattern.

- Runtime
  - [Replicant.Assembler](Replicant.Assembler.md): Assembles a stream of decoded pgoutput messages into a `Replicant.Transaction`
and applies the sink synchronously per transaction (spec §4/§6).
  - [Replicant.AssemblerServer](Replicant.AssemblerServer.md): The serial process shell over the pure `Replicant.Assembler` (spec §4). Receives
**decoded** pgoutput messages from `Replicant.Connection` (which decodes behind
the value-free boundary and never applies the sink), assembles transactions, and
applies the sink **synchronously** — blocking THIS process, off the Connection's
keepalive path. On a durable sink commit (or a watermark skip) it messages the
Connection so the ack advances asynchronously; on a fail-closed condition
(destructive schema change, sink WRITE fault, an unidentifiable-relation row) it
halts the whole pipeline permanently (spec §6/§9).
  - [Replicant.CheckpointStore](Replicant.CheckpointStore.md): The lib-owned checkpoint store (spec §7) for non-transactional sinks. A GenServer
over a normal Postgrex connection that owns one `replicant_checkpoints` row per
slot and reads/writes `commit_lsn` behind the value-free boundary (Critical Rule
1 — the store carries slot_name + LSN only; a Postgrex fault is scrubbed to a
structural `%Replicant.Error{}`, never echoing a parameter).
  - [Replicant.Connection](Replicant.Connection.md): The `Postgrex.ReplicationConnection` that owns the replication slot and closes
the exactly-once seam (spec §2/§4/§8). It
  - [Replicant.Pipeline](Replicant.Pipeline.md): Per-pipeline `Supervisor` (spec §4): supervises the `Replicant.AssemblerServer`
and the `Replicant.Connection` under **`:one_for_all`** — a crash of either
restarts both together, so a fresh Connection (resuming from the sink checkpoint)
is never paired with a stale in-memory assembler buffer. Registered by slot in
`Replicant.Registry` so `Replicant.Supervisor.halt/2` can terminate the whole
pipeline permanently on a fail-closed condition.
  - [Replicant.Snapshotter](Replicant.Snapshotter.md): Reads a consistent snapshot of the publication's tables (spec §4) on its own normal
Postgrex connection, at the LSN a durable slot exported via `EXPORT_SNAPSHOT`, and
pushes the rows to the sink as `%Change{op: :snapshot}` batches — then durably sets the
sink checkpoint to the snapshot's consistent point (the handoff commit). Spawned and
LINKED to `Replicant.Connection`, which holds the exported snapshot valid by staying
idle (proven safe past `wal_sender_timeout`, spec §11). The link binds the snapshotter's
lifetime to the pipeline: a Connection/pipeline teardown mid-snapshot tears the
snapshotter down with it, so an orphan can never mutate the sink after the pipeline is
gone. Graceful completion exits `:normal` (which never propagates over the link), so the
`{:snapshot_done, lsn}` / `{:snapshot_failed, err}` messages still drive the handoff.
  - [Replicant.Supervisor](Replicant.Supervisor.md): The top `DynamicSupervisor` — one child per running pipeline (a
`Replicant.Pipeline` supervisor). Pipelines run as **`:temporary`** children:
a fail-closed halt terminates one permanently (no restart), while transient
crashes are recovered inside each `Pipeline`'s own `:one_for_all` strategy.

- Disk spill
  - [Replicant.Spill](Replicant.Spill.md): The single `File.*` boundary for consumer disk spill (spec §5/§6). Owns the per-slot spill
directory (`0700`), the per-top-xid spill file (`0600`), append of length-prefixed
`:erlang.term_to_binary({subxid, %Change{}})` frames, file discard, and the per-slot startup
sweep. Every I/O or serialization fault is scrubbed to a value-free
`{:error, %Replicant.Error{reason: :spill_io_failed}}` here (Critical Rule 1) — no path, frame,
or row value ever escapes. Spill files are scratch: never fsync'd, deleted on
commit/abort/reset/halt, and (for a prior abnormal exit) swept per-slot on that slot's next start.

  - [Replicant.Spill.Error](Replicant.Spill.Error.md): A value-free spill exception. Raised by `Replicant.Spill.Reader` on any `File`/`binary_to_term`
fault so the lazy read (which runs inside the sink call, spec §11) never surfaces a raw byte.
Carries ONLY the fixed reason — no path, no frame, no row value (Critical Rule 1).

  - [Replicant.Spill.Reader](Replicant.Spill.Reader.md): A lazy, SINGLE-PASS `Enumerable` over a spilled transaction's changes (spec §5): it streams the
on-disk frames (in commit order) followed by the still-resident in-memory tail, rejects any change
whose `subxid` is in the aborted set, and stamps `commit_lsn` at replay (streamed changes have no
LSN before commit; the `ordinal` was stamped at accumulation from the shared per-txn counter and is
preserved). It is value-free BY CONSTRUCTION — a `File` read or
`binary_to_term` fault raises `Replicant.Spill.Error` (fixed reason, no bytes), never a raw-byte
exception, even though the sink forces this enumeration inside its own DB transaction (spec §11).

- Decoding (pgoutput)
  - [Replicant.Casting.ArrayParser](Replicant.Casting.ArrayParser.md): Parser for PostgreSQL array literals
  - [Replicant.Casting.Types](Replicant.Casting.Types.md): Cast from Postgres to Elixir types
  - [Replicant.Decoder](Replicant.Decoder.md): Binary decoder for the Postgres logical-replication `pgoutput` stream.
  - [Replicant.Decoder.Messages](Replicant.Decoder.Messages.md): Decoded representations of each `pgoutput` replication message.
  - [Replicant.Decoder.OidDatabase](Replicant.Decoder.OidDatabase.md): Maps a numeric PostgreSQL type ID to a descriptive string.
  - [Replicant.Identifier](Replicant.Identifier.md): Allowlist validation for Postgres identifiers `replicant` interpolates into slot
and publication SQL (Critical Rule 2). A failure carries the invalid-SHAPE fact
only, never the offending string — it may be attacker-controlled.
  - [Replicant.QueryBuilder](Replicant.QueryBuilder.md): Slot and publication SQL/command strings built from **validated** identifiers
(Critical Rule 2). Hardens `walex`'s raw `'#{publication}'` interpolation: every
name passes through `Replicant.Identifier.validate/1` before it reaches the
string; an invalid name returns `{:error, :invalid_identifier}` and builds
nothing. `START_REPLICATION`/`CREATE_REPLICATION_SLOT` are replication commands
(no bind parameters), so validated interpolation is the gate.