The problem#
The product needed live collaborative editing — documents shared by dozens of users with changes propagating in under 100ms, including while some clients are offline. A last-write-wins approach was rejected in the first design review: losing a teammate's edit in a shared doc is a data-loss bug with a UI.
CRDTs as the merge strategy#
We modeled each document as a grow-only log of operations, with per-character identifiers so concurrent inserts interleave deterministically. Every replica converges to the same state without a central coordinator:
interface Op {
id: OpId; // { site: string; counter: number }
parent: OpId; // causal parent
value: string; // inserted character
}
export function mergeLogs(local: Op[], remote: Op[]): Op[] {
const seen = new Set(local.map((op) => op.id));
const incoming = remote.filter((op) => !seen.has(op.id));
if (incoming.length === 0) return local;
// Topological merge: an op is placed after its causal parent.
const merged: Op[] = [];
const queue = [...local, ...incoming];
const placed = new Set<OpId>();
while (queue.length > 0) {
const op = queue.shift()!;
if (op.parent && !placed.has(op.parent)) {
queue.push(op); // parent not placed yet — revisit
continue;
}
merged.push(op);
placed.add(op.id);
}
return merged;
}Connection draining#
Deploys were the hard part. Killing a node mid-edit session dropped clients and caused reconnect storms — 500k clients retrying at once looks a lot like an attack. We built graceful draining: a node marked for shutdown stops accepting new connections, tells clients to reconnect to a neighbor over the next 30 seconds, and only then exits.
Why not a central coordinator?
A coordinator would have simplified ordering but become a single point of failure and a throughput ceiling. CRDTs moved the complexity from the runtime to the merge function — which is pure, testable, and runs identically on every client.
The property that mattered#
The entire merge logic — the code above — is ~60 lines and property-tested against 10,000 randomized operation histories. Convergence, commutativity, and idempotence are checked on every commit. When the correctness-critical part of a system fits on one screen, on-call becomes very calm.