> ## Documentation Index > Fetch the complete documentation index at: https://chahine.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Storage Backends > Configure Meridian's persistence layer — sled, PostgreSQL, Redis, or bring your own. Meridian delegates all persistence to the `meridian-storage` crate, which exposes two traits: * **`Store`** — CRDT snapshot storage (get / put / delete / scan) * **`WalBackend`** — append-only Write-Ahead Log (append / replay / truncate) All built-in backends are feature-gated. Pick the one that matches your deployment. *** ## Built-in backends ### sled (default) Embedded key-value store. Zero infrastructure, single binary. Best for single-node deployments. ```toml theme={null} # Cargo.toml (server) meridian-server = { features = ["storage-sled"] } ``` ```bash theme={null} # Environment MERIDIAN_DATA_DIR=./data # path to the sled data directory ``` ### PostgreSQL Fully managed, horizontally scalable. Requires a Postgres 14+ instance. ```toml theme={null} meridian-server = { features = ["storage-postgres"] } ``` ```bash theme={null} DATABASE_URL=postgres://user:password@host:5432/meridian ``` Meridian runs `CREATE TABLE IF NOT EXISTS` + `CREATE INDEX IF NOT EXISTS` on startup — no manual migration needed. **Schema created automatically:** ```sql theme={null} -- CRDT snapshots CREATE TABLE crdt_snapshots ( namespace TEXT NOT NULL, crdt_id TEXT NOT NULL, data BYTEA NOT NULL, updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), PRIMARY KEY (namespace, crdt_id) ); CREATE INDEX crdt_snapshots_namespace_idx ON crdt_snapshots (namespace); -- Write-Ahead Log CREATE TABLE wal_entries ( seq BIGSERIAL PRIMARY KEY, namespace TEXT NOT NULL, crdt_id TEXT NOT NULL, op_bytes BYTEA NOT NULL, timestamp_ms BIGINT NOT NULL ); CREATE INDEX wal_entries_namespace_seq_idx ON wal_entries (namespace, seq); ``` ### Redis Uses Redis for snapshots (`GET`/`SET`) and Redis Streams for the WAL (`XADD`/`XRANGE`/`XTRIM`). ```toml theme={null} meridian-server = { features = ["storage-redis"] } ``` ```bash theme={null} REDIS_URL=redis://localhost:6379 ``` *** ## S3 WAL archive Enable durable off-node WAL backups with the `wal-archive-s3` feature. Works with AWS S3, Cloudflare R2, and any S3-compatible store (MinIO, LocalStack). ```toml theme={null} # Cargo.toml (server) meridian-server = { features = ["storage-sled", "wal-archive-s3"] } ``` Set `S3_BUCKET` to opt in — no other change required: ```bash theme={null} S3_BUCKET=my-wal-bucket S3_REGION=us-east-1 # default: us-east-1 S3_KEY_PREFIX=wal/ # default: wal/ WAL_SEGMENT_SIZE=500 # default: 500 entries per segment # For R2 / MinIO / LocalStack: S3_ENDPOINT=https://your-account.r2.cloudflarestorage.com # Credentials — standard AWS credential chain (env, ~/.aws, IAM role): AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... ``` **How it works** * **Upload on threshold** — every `WAL_SEGMENT_SIZE` appends, the accumulated segment is uploaded to S3 as `{prefix}{seq_start:020}-{seq_end:020}.msgpack`. * **Upload on truncation** — before local WAL entries are truncated (compacted), any pending unarchived entries are flushed to S3. This guarantees data is on S3 before local deletion. * **Restore on startup** — if the local WAL is empty (fresh node or wiped disk), all S3 segments are downloaded and replayed in order before the server accepts connections. * **Non-fatal uploads** — S3 upload failures are logged as warnings. The local WAL remains authoritative and writes are never blocked. **Minimum IAM policy** ```json theme={null} { "Effect": "Allow", "Action": ["s3:PutObject", "s3:GetObject", "s3:ListObjectsV2"], "Resource": "arn:aws:s3:::my-wal-bucket/*" } ``` *** ## WAL replay and point-in-time recovery Every write goes through the WAL before being applied to the snapshot store. On restart: 1. The server loads CRDT snapshots directly from the store (already up-to-date). 2. WAL entries written after the last compaction are replayed to catch up. To replay entries up to a specific point in time: ```rust theme={null} // Replay all WAL entries between checkpoint and a past timestamp let entries = wal.replay_until(wal.checkpoint_seq(), until_ms).await?; ``` The WAL compactor runs every 60 seconds and truncates entries older than the current snapshot checkpoint, keeping the WAL bounded. *** ## Custom backend Implement `Store` and `WalBackend` for any storage engine: ```rust theme={null} use meridian_storage::{store::Store, wal_backend::{WalBackend, WalEntry}, error::Result}; use serde::{Serialize, Deserialize}; pub struct MyStore { /* ... */ } impl Store for MyStore where V: Serialize + for<'de> Deserialize<'de> + Send + Sync + 'static, { async fn get(&self, ns: &str, id: &str) -> Result> { todo!() } async fn put(&self, ns: &str, id: &str, value: &V) -> Result<()> { todo!() } async fn delete(&self, ns: &str, id: &str) -> Result<()> { todo!() } async fn scan_prefix(&self, prefix: &str) -> Result> { todo!() } } ``` Pass your implementation to `server::run()`: ```rust theme={null} use std::sync::Arc; use meridian_server::server::{run, Config}; let store = Arc::new(MyStore::new()); let wal = Arc::new(MyWal::new()); run(Config::from_env(), prometheus_handle, store, wal).await?; ``` ### Contract | Method | Semantics | | - | - | | `get(ns, id)` | Return `None` if missing, deserialize if present | | `put(ns, id, value)` | Upsert — overwrite if exists | | `delete(ns, id)` | No-op if missing | | `scan_prefix(ns)` | Return all `(key, value)` pairs where key starts with `ns` | | `append(ns, id, bytes)` | Append entry, return monotonic seq number | | `replay_from(seq)` | Return all entries with `seq >= from_seq`, ordered by seq | | `replay_until(seq, ms)` | Return entries with `seq >= from_seq` AND `timestamp_ms <= until_ms` | | `scan_prefix(ns)` | `ns` is an exact namespace — not a glob or substring match | | `truncate_before(seq)` | Remove all entries with `seq < before_seq`; also advances `checkpoint_seq` | | `set_checkpoint_seq(seq)` | Persist the compaction watermark durably | This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.