Storage Backends

Configure Meridian’s persistence layer — sled, PostgreSQL, Redis, or bring your own.

Meridian delegates all persistence to the meridian-storage crate, which exposes two traits:

• Store<V> — CRDT snapshot storage (get / put / delete / scan)
• WalBackend — append-only Write-Ahead Log (append / replay / truncate)

All built-in backends are feature-gated. Pick the one that matches your deployment.

---

​

Built-in backends

​

sled (default)

Embedded key-value store. Zero infrastructure, single binary. Best for single-node deployments.

# Cargo.toml (server)
meridian-server = { features = ["storage-sled"] }

# Environment
MERIDIAN_DATA_DIR=./data  # path to the sled data directory

​

PostgreSQL

Fully managed, horizontally scalable. Requires a Postgres 14+ instance.

meridian-server = { features = ["storage-postgres"] }

DATABASE_URL=postgres://user:password@host:5432/meridian

Meridian runs CREATE TABLE IF NOT EXISTS + CREATE INDEX IF NOT EXISTS on startup — no manual migration needed.

Schema created automatically:

-- CRDT snapshots
CREATE TABLE crdt_snapshots (
    namespace  TEXT NOT NULL,
    crdt_id    TEXT NOT NULL,
    data       BYTEA NOT NULL,
    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (namespace, crdt_id)
);
CREATE INDEX crdt_snapshots_namespace_idx ON crdt_snapshots (namespace);

-- Write-Ahead Log
CREATE TABLE wal_entries (
    seq          BIGSERIAL PRIMARY KEY,
    namespace    TEXT NOT NULL,
    crdt_id      TEXT NOT NULL,
    op_bytes     BYTEA NOT NULL,
    timestamp_ms BIGINT NOT NULL
);
CREATE INDEX wal_entries_namespace_seq_idx ON wal_entries (namespace, seq);

​

Redis

Uses Redis for snapshots (GET/SET) and Redis Streams for the WAL (XADD/XRANGE/XTRIM).

meridian-server = { features = ["storage-redis"] }

REDIS_URL=redis://localhost:6379

---

​

S3 WAL archive

Enable durable off-node WAL backups with the wal-archive-s3 feature. Works with AWS S3, Cloudflare R2, and any S3-compatible store (MinIO, LocalStack).

# Cargo.toml (server)
meridian-server = { features = ["storage-sled", "wal-archive-s3"] }

Set S3_BUCKET to opt in — no other change required:

S3_BUCKET=my-wal-bucket
S3_REGION=us-east-1                          # default: us-east-1
S3_KEY_PREFIX=wal/                           # default: wal/
WAL_SEGMENT_SIZE=500                         # default: 500 entries per segment

# For R2 / MinIO / LocalStack:
S3_ENDPOINT=https://your-account.r2.cloudflarestorage.com

# Credentials — standard AWS credential chain (env, ~/.aws, IAM role):
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...

How it works

• Upload on threshold — every WAL_SEGMENT_SIZE appends, the accumulated segment is uploaded to S3 as {prefix}{seq_start:020}-{seq_end:020}.msgpack.
• Upload on truncation — before local WAL entries are truncated (compacted), any pending unarchived entries are flushed to S3. This guarantees data is on S3 before local deletion.
• Restore on startup — if the local WAL is empty (fresh node or wiped disk), all S3 segments are downloaded and replayed in order before the server accepts connections.
• Non-fatal uploads — S3 upload failures are logged as warnings. The local WAL remains authoritative and writes are never blocked.

Minimum IAM policy

{
  "Effect": "Allow",
  "Action": ["s3:PutObject", "s3:GetObject", "s3:ListObjectsV2"],
  "Resource": "arn:aws:s3:::my-wal-bucket/*"
}

---

​

WAL replay and point-in-time recovery

Every write goes through the WAL before being applied to the snapshot store. On restart:

1. The server loads CRDT snapshots directly from the store (already up-to-date).
2. WAL entries written after the last compaction are replayed to catch up.

To replay entries up to a specific point in time:

// Replay all WAL entries between checkpoint and a past timestamp
let entries = wal.replay_until(wal.checkpoint_seq(), until_ms).await?;

The WAL compactor runs every 60 seconds and truncates entries older than the current snapshot checkpoint, keeping the WAL bounded.

---

​

Custom backend

Implement Store<V> and WalBackend for any storage engine:

use meridian_storage::{store::Store, wal_backend::{WalBackend, WalEntry}, error::Result};
use serde::{Serialize, Deserialize};

pub struct MyStore { /* ... */ }

impl<V> Store<V> for MyStore
where
    V: Serialize + for<'de> Deserialize<'de> + Send + Sync + 'static,
{
    async fn get(&self, ns: &str, id: &str) -> Result<Option<V>> { todo!() }
    async fn put(&self, ns: &str, id: &str, value: &V) -> Result<()> { todo!() }
    async fn delete(&self, ns: &str, id: &str) -> Result<()> { todo!() }
    async fn scan_prefix(&self, prefix: &str) -> Result<Vec<(String, V)>> { todo!() }
}

Pass your implementation to server::run():

use std::sync::Arc;
use meridian_server::server::{run, Config};

let store = Arc::new(MyStore::new());
let wal = Arc::new(MyWal::new());
run(Config::from_env(), prometheus_handle, store, wal).await?;

​

Contract

Method	Semantics
get(ns, id)	Return None if missing, deserialize if present
put(ns, id, value)	Upsert — overwrite if exists
delete(ns, id)	No-op if missing
scan_prefix(ns)	Return all (key, value) pairs where key starts with ns
append(ns, id, bytes)	Append entry, return monotonic seq number
replay_from(seq)	Return all entries with seq >= from_seq, ordered by seq
replay_until(seq, ms)	Return entries with seq >= from_seq AND timestamp_ms <= until_ms
scan_prefix(ns)	ns is an exact namespace — not a glob or substring match
truncate_before(seq)	Remove all entries with seq < before_seq; also advances checkpoint_seq
set_checkpoint_seq(seq)	Persist the compaction watermark durably