clearvoiance

Quickstart

From zero to "I just replayed my own traffic" in about five minutes.

1. Run the engine + dashboard locally

The engine needs ClickHouse, Postgres, and MinIO. The repo ships a docker-compose stack that brings those up alongside the engine binary and the Next.js dashboard:

git clone https://github.com/charlses/clearvoiance
cd clearvoiance
cp deploy/.env.example deploy/.env
docker compose --env-file deploy/.env -f deploy/docker-compose.yml up -d --build

That's it — every password, port, and URL is driven from the .env you just copied. Default endpoints (all bound to 127.0.0.1):

  • Dashboard — http://127.0.0.1:3000
  • Engine gRPC (SDK target) — 127.0.0.1:9100
  • Engine REST + WS — http://127.0.0.1:9101 (Swagger UI at /docs)

For TLS, Traefik, or tuning the stack for production, see Deployment.

2. Install the SDK in your app

npm install @clearvoiance/node

Node 18+ required. Every framework integration is an optional peer dep — installing the SDK doesn't pull Express / Fastify / Prisma etc.

3. Wire it up

Minimal Express example. Replace with Koa, Fastify, or Strapi as needed — the patterns mirror each other.

import express from "express";
import { createClient } from "@clearvoiance/node";
import { captureHttp } from "@clearvoiance/node/http/express";
import { patchOutbound } from "@clearvoiance/node/outbound";

const client = createClient({
  engine: { url: "127.0.0.1:9100", apiKey: process.env.CLEARVOIANCE_API_KEY! },
  session: { name: "my-api" },
  // Persist in-flight events across engine restarts:
  wal: { dir: "/var/lib/clearvoiance-wal" },
});
await client.start();

// Optional: record every http.request / fetch made FROM your app
patchOutbound(client);

const app = express();
app.use(captureHttp(client));

app.get("/hello/:name", (req, res) =>
  res.json({ hello: req.params.name }),
);

app.listen(3000);

On shutdown, drain any buffered events so nothing's lost:

const shutdown = async () => {
  await client.stop({ flushTimeoutMs: 10_000 });
  process.exit(0);
};
process.on("SIGTERM", shutdown);
process.on("SIGINT", shutdown);

Production mode: remote-controlled capture

For anything beyond dev, you probably don't want capture running on every request 24/7. Add a remote block to createClient — the SDK registers as a monitor with the engine, idles, and the dashboard's Monitors page drives Start / Stop:

const client = createClient({
  engine: { url: "grpc.example.com:443", tls: true, apiKey: /* ... */ },
  session: { name: "my-api" },
  remote: {
    clientName: "my-api-prod",              // stable identity
    displayName: "My API (production)",
    labels: { env: "production" },
  },
  wal: { dir: "/var/lib/clearvoiance-wal" },
});
await client.start();
// No session opens yet. sendBatch() drops events silently until the
// dashboard clicks Start.

See Monitors for the full story on reconnect resume, replica fan-out, and the production capture → snapshot → replay flow this was built for.

4. Generate some traffic

Hit the app a few times. The SDK streams each request as an event; the engine persists them to ClickHouse.

for i in $(seq 1 20); do curl -s http://localhost:3000/hello/alice; done

5. See it land

Open the dashboard at http://127.0.0.1:3000 (part of the docker-compose stack). First visit shows a setup wizard — create an admin with email + password, submit, you're in. Next visits go straight to the login screen. You'll see the active session with its event count ticking up.

Need to mint an API key for the SDK engine.apiKey config? Once you're signed in, Settings → API keys → Create. The plaintext is shown once; copy it into your SDK config.

6. Stop the session + replay it

Stop the session (in the UI, or via clearvoiance session stop). Then kick off a replay at 12×:

clearvoiance replay start \
  --source sess_abc \
  --target http://localhost:3000 \
  --speedup 12

The engine schedules every captured event at compressed time and fires them at the target. In the dashboard, /replays/<id> shows 250ms live progress via the WebSocket hub.

7. Correlate DB queries to events

If your app uses Postgres, wrap the pool with instrumentPg so every query carries application_name = 'clv:<event_id>':

import { Pool } from "pg";
import { instrumentPg } from "@clearvoiance/node/db/postgres";

const pool = new Pool({ connectionString: process.env.DATABASE_URL });
instrumentPg(pool, { replayId: process.env.CLEARVOIANCE_REPLAY_ID });

Start the DB observer alongside the engine:

clearvoiance-observer run \
  --postgres-dsn "postgres://observer:readonly@localhost:5432/app?sslmode=disable" \
  --clickhouse-dsn "clickhouse://default:dev@localhost:9000/clearvoiance"

Now /replays/<id>/db surfaces slow queries grouped by fingerprint, DB time rolled up per endpoint, and any lock waits — each correlated back to the replay event that caused it.

What's next

  • Core concepts — the four components (capture, replay, hermetic, observer) and how they combine.
  • Adapters — the full per-framework setup reference lives in the SDK README.