meridian-storage crate, which exposes two traits:
Store<V>— CRDT snapshot storage (get / put / delete / scan)WalBackend— append-only Write-Ahead Log (append / replay / truncate)
Built-in backends
sled (default)
Embedded key-value store. Zero infrastructure, single binary. Best for single-node deployments.PostgreSQL
Fully managed, horizontally scalable. Requires a Postgres 14+ instance.CREATE TABLE IF NOT EXISTS + CREATE INDEX IF NOT EXISTS on startup — no manual migration needed.
Schema created automatically:
Redis
Uses Redis for snapshots (GET/SET) and Redis Streams for the WAL (XADD/XRANGE/XTRIM).
S3 WAL archive
Enable durable off-node WAL backups with thewal-archive-s3 feature. Works with AWS S3, Cloudflare R2, and any S3-compatible store (MinIO, LocalStack).
S3_BUCKET to opt in — no other change required:
- Upload on threshold — every
WAL_SEGMENT_SIZEappends, the accumulated segment is uploaded to S3 as{prefix}{seq_start:020}-{seq_end:020}.msgpack. - Upload on truncation — before local WAL entries are truncated (compacted), any pending unarchived entries are flushed to S3. This guarantees data is on S3 before local deletion.
- Restore on startup — if the local WAL is empty (fresh node or wiped disk), all S3 segments are downloaded and replayed in order before the server accepts connections.
- Non-fatal uploads — S3 upload failures are logged as warnings. The local WAL remains authoritative and writes are never blocked.
WAL replay and point-in-time recovery
Every write goes through the WAL before being applied to the snapshot store. On restart:- The server loads CRDT snapshots directly from the store (already up-to-date).
- WAL entries written after the last compaction are replayed to catch up.
Custom backend
ImplementStore<V> and WalBackend for any storage engine:
server::run():
Contract
| Method | Semantics |
|---|---|
get(ns, id) | Return None if missing, deserialize if present |
put(ns, id, value) | Upsert — overwrite if exists |
delete(ns, id) | No-op if missing |
scan_prefix(ns) | Return all (key, value) pairs where key starts with ns |
append(ns, id, bytes) | Append entry, return monotonic seq number |
replay_from(seq) | Return all entries with seq >= from_seq, ordered by seq |
replay_until(seq, ms) | Return entries with seq >= from_seq AND timestamp_ms <= until_ms |
scan_prefix(ns) | ns is an exact namespace — not a glob or substring match |
truncate_before(seq) | Remove all entries with seq < before_seq; also advances checkpoint_seq |
set_checkpoint_seq(seq) | Persist the compaction watermark durably |