Nodes, zones and regions, one layout at a time.

Erasure sets

k data shards, m parity shards, one per node.

A cluster writes objects with a Reed-Solomon code k+m: four data and two parity, eight and four, any shape up to sixteen shards. Each object's shards go one per node in an erasure set, placed by a key all nodes share, so every node computes the same placement without asking anyone.

A write is acknowledged on its quorum: k shards durable, or k + 1 when k = m so that two halves of a split cluster cannot both accept a write. A read takes any k shards whose blocks verify.

Failure domains

A zone can go dark and every object still reads.

Each node names its zone: a rack, a room, a data centre. A layout never puts more of one object's shards in one zone than the parity can lose, so losing a whole zone leaves every object readable. The first layout is taken from the registered nodes in id order and passes over a node whose zone already holds m shards.

Healing and layouts

Repairs and moves happen while the cluster serves.

A healing pass on every member rewrites missing or damaged shards of the objects listed for healing, one node at a time per object, under a claim in TessariDB. Shard bytes no object can reach any more are swept.

Root changes the layout in the console: a new layout is staged over the current version, judged against the registered nodes and their zones, and applied as the next version, or refused if anything changed meanwhile. Members then move objects written under older layouts, copying and verifying the shards whose node changes before switching the object, at a rate you set.

Membership

Members are admitted, not discovered.

• A cluster begins from its founding members; any other node joins only when root approves it in the console, or with a one-time join token that binds the node's certificate.
• Nodes talk to each other only over mutual TLS, each certificate issued for <node id>.peer.tessari-s3; a request is served only from an admitted member presenting the certificate for the node it claims to be.
• A removed node is removed for good and never admitted again, and the removal reaches every node within five seconds.

Regions

Each region a cluster. Buckets replicate across them.

A region is a complete cluster: its own erasure sets in its own zones, and its own TessariDB members whose rows carry the REGION they stand in. A write in Frankfurt is acknowledged by Frankfurt's quorum and Frankfurt's metadata majority; it does not wait for an ocean.

Every bucket has a home region, chosen when it is created. Its replication rules name the regions that keep a copy and, optionally, the prefixes that travel. Replication runs after the home region has answered the client, carries versions and delete markers in commit order, and reports how far behind each destination is. Readers are served from the nearest region that holds the object; a region that is lost is replaced by promoting a destination to home.

Measured

The first numbers, with the conditions they were taken under.

Six tessaridb-s3 processes as one 4 + 2 cluster on a single Apple-silicon laptop, all six data directories on its one internal SSD, metadata in a TessariDB 0.34.0-beta node, release build, warm page cache; the median of three runs, and every GET returned exactly the bytes that were PUT. One laptop, not a production deployment: on macOS each sync flushes the whole drive's cache, so six nodes sharing one drive wait on each other.