Architecture

This page tells how kinedb works today, and where it is going. It is written for the people and the agents who use kinedb, not for the people who change it.

An application sends SQL to kinedb over HTTP (/sql), over a WebSocket (/ws), or in-process (Rust). kinedb runs in one of three ways. All three keep the data in the same kind of storage:

┌─────────────────┐   ┌─────────────────┐   ┌─────────────────┐
│ Embedded        │   │ One server      │   │ A cluster       │
│ Db::open(path)  │   │ kinedb          │   │ kinedb --calvin │
│ no server       │   │ --port 4820     │   │ shards + Raft   │
└─────────────────┘   └─────────────────┘   └─────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Storage: a prolly tree of content-addressed blocks          │
│ in memory, on local disk, or in an S3 bucket (--s3)         │
└─────────────────────────────────────────────────────────────┘

Today

Storage: a tree of content-addressed blocks

kinedb keeps every table and collection in a prolly tree. A prolly tree is a search tree whose node boundaries come from the content, so two trees with the same data have the same shape and the same root hash. Each block is stored under the hash of its bytes, so two equal blocks are stored once.

A commit writes new blocks and a new root. The old roots stay readable. This is why a branch is cheap: CREATE BRANCH records one more root, and USE BRANCH moves the session to it. A qualified table name (mydb@dev.orders) reads and writes that root for one statement: the server removes the qualifier at the entry and runs the statement on that database and branch, and every node after it sees the bare name.

Old versions stay on disk until garbage collection removes them. Garbage collection is off by default; --gc-retention turns it on and sets how much history to keep.

The blocks live in memory, on local disk, or in an S3 bucket (--s3).

On disk, each block is compressed alone. Documents use a codec in the zstd format with a dictionary that the store trains from its own data. The store judges on its data which kinds of block gain from the dictionary. Tree nodes and other blocks that do not gain use zstd without a dictionary, and very small blocks use LZ4. The dictionary is an ordinary block, kept for as long as the store exists. The content address of a block is always the hash of its uncompressed bytes.

Durability in a cluster

A write is durable when the cluster acknowledges it. The leader of a shard writes the entry to disk before the entry counts toward the quorum, and every follower does the same before it answers. A node that restarts applies its committed entries again, so an acknowledged write survives a crash.

Transactions and the order of writes

A transaction is ONE complete list of statements in ONE call: transaction() in every client library, one Transaction frame on the wire. No transaction stays open on the server, so no client can make the server wait for it. A transaction can carry an idempotency token, and GET TRANSACTION STATUS tells a client that lost its connection whether the commit happened. See Transactions.

In a cluster (--calvin), a sequencer collects the writes into small batches. Raft gives the batches one order, and every replica applies the same batches in the same order. So the replicas of a shard reach the same state, and they need no locks to agree. This design comes from the Calvin paper on deterministic databases.

The path of a write in a cluster:

┌──────────────┐
│ client       │  sends the write to any node
└──────────────┘
       │
       ▼
┌──────────────┐
│ any node     │  forwards it to the leader of the shard
└──────────────┘
       │
       ▼
┌──────────────┐
│ shard leader │  puts the write in a small batch (the sequencer)
└──────────────┘
       │
       ▼
┌──────────────┐
│ Raft         │  gives the batches ONE order; a quorum writes it to disk
└──────────────┘
       │
       ▼
┌──────────────┐
│ replicas     │  apply the same batches in the same order
└──────────────┘

The cluster

• Membership. The nodes find each other with SWIM, a gossip protocol that also detects a node that stops answering.
• Shards. The cluster splits each table into shards by key range. It splits a shard again when the shard grows past a size threshold.
• Replicas. Each shard has a Raft group of replicas on different nodes. The replication factor is 3 by default. With more nodes than the replication factor, a fixed draw on the shard id picks the nodes of each shard, so each shard lives on only that many nodes and the shards spread over the cluster. The cluster setting cluster.placement.enabled controls this, and it is on by default.
• Where a write goes. The leader of a shard orders every write to that shard. A client can send a write to any node: a node that does not lead the shard forwards the write to the leader. So the write throughput of one table grows with its number of shards, not with the number of nodes. A new table is one shard until it grows past cluster.split.threshold_bytes (64 MiB by default). A shard splits only when no cross-shard transaction is in flight on it. Under steady cross-shard load, the leader of the shard asks for a drain after cluster.split.drain_after_ticks refused checks (3 by default, 0 = off). For at most 15 s, a new cross-shard transaction that writes to the shard then gets a retryable refusal, and the client retries it. The drain stops at once when the leader no longer wants the split: the shard is below the threshold, the leader changed, or the drain is turned off.
• Peer security. Nodes must prove to each other that they hold the cluster key pair (a link key derived from it). This check is on by default. See The cluster key.

The data model

• Tables have typed columns and a primary key.
• Document collections hold JSON documents. Each document has an _id.
• Key-value collections hold one value per key.
• Files are rows whose value is a manifest of content-addressed chunks. A client stores one with put_file and reads it with get_file (see Files); SQL has INSERT ... FILE and SELECT FILE. A file has no size limit by design: the chunks are at most 128 KiB, their boundaries follow the content, and two files that share bytes share their chunks. A file lives with the shard of its row: the manifest, the record of an upload in progress and the chunks split and move with the shard. The shard's leader sweeps the record of an upload that never sent its manifest through the log, so every replica removes the same records.
• Indexes speed up a WHERE, in a SELECT and in the row selection of an UPDATE or a DELETE. DESCRIBE SELECT ..., DESCRIBE UPDATE ... and DESCRIBE DELETE ... show which plan a statement uses, and a DESCRIBE runs no statement. An index lives with the shards of its table: each shard holds the entries of its own rows, and they split and move with the shard.

Queries and live queries

kinedb speaks its own SQL dialect: tables and documents in one language, with WHERE, ORDER BY, LIMIT, aggregates, GROUP BY and joins.

WATCH <table> streams the changes of a table to a client over the WebSocket, until UNWATCH.

Logic in the server

CREATE FUNCTION f(x INT) RETURNS INT LANGUAGE kd stores a function in kd, the scripting language of kinedb, and CALL runs it. Every parameter and the result have a type, and CALL checks them. CREATE TICKER ... EVERY <interval> CALL <function> runs a function on a schedule.

Access control

A client logs in with a password, an API secret or a key pair, and it can use a token for the next requests. See Authentication. The SQL has users, roles, GRANT and REVOKE, and a grant can carry a row filter (WHERE). The SQL reference tells which of these statements run today.

The protocol and the clients

One port (4820 by default) carries everything:

• POST /sql takes one SQL statement and answers JSON. It keeps no session: ?db=<name> or a qualified table name (db.table) selects the database, not USE.
• /ws is a WebSocket with a binary protocol. A result tells the column types once, not once per cell. Live queries use this path.
• POST /login gives a token. GET /health answers without a login. It answers 503 while the node is degraded — today that means an index whose tree predates the B934 layout.
• /ui/ is the web console, and /ui/docs/ is this documentation.

kinedb has no TLS of its own, and it encrypts nothing at rest. Put a TLS proxy in front of the client port when the traffic crosses a network that you do not trust. The traffic between the nodes is plaintext too. See TLS and encryption.

The JavaScript client and the Python client speak this protocol.

Where it is going

• More conflict policies for the sync. An embedded copy syncs a table with a cluster today (see the embedded database): the server sends only the rows that changed and that the user may read, and it checks each offline write against the permissions of the user and the version of the row. A write that a newer server write conflicts with is refused, and it stays visible to the application. Next: more policies for a conflict — last-writer-wins for the whole row, last-writer-wins for each cell or JSON property (two writes to the same row both apply when they change different cells), and a custom merge function, the base for CRDTs — and a sync between two embedded copies.
• Joins and transactions across shards for every query plan and every statement shape.
• Branch merge. More conflict policies for MERGE BRANCH (see SQL). A merge of main into a branch works today. A merge between two branches that are not main works too.
• Placement. Place replicas by data center and rack, and keep hot and cold data on different machines.
• More SQL. Computed columns, CHECK constraints, a vector index, columnar storage for analytics, and the Postgres and MySQL wire protocols.
• More clients. An embedded and a direct client in each language.