← the Dry Stack
Setup and config
One process, a handful of settings, nothing else to run.
The Dry Stack is a library, not a server farm. You reference it from your
C# app and open a database in a directory. There is no broker, no
cluster manager and no config file. Every setting is a plain field in code, with its
default written next to it.
The names and defaults below come straight from the source. If a setting isn't
here, you probably don't need to touch it.
What you run it on
- Runtime
- .NET 10. The engine is native C++ underneath, loaded once at startup.
You never touch it unless you want to.
- Platforms
- x64 Windows and Linux, same code and same results. On Linux the
network layer uses io_uring, so the kernel must allow it (some hardened images switch
it off). An ARM build for the Raspberry Pi is being finished.
- Hardware
- Whatever you have. Every benchmark on this site is one box: a desktop Ryzen, or a
$35 Pi. Put the data on an NVMe drive, or in RAM
if it doesn't need to survive a reboot.
Your first databaseC#
Load the engine, describe a shard, open a directory. That is the whole setup.
using Dry.Facts;
Facts.LoadNative(); // once per process
// One shard: series ids 0..9999, one F64 column, small payloads inline.
var spec = ShardSpec.WithColumns(tagMax: 9_999, hasRef: true, ColumnType.F64);
spec.MaxTotalBytes = 64UL << 30; // keep 64 GiB, drop the oldest beyond that
using var db = new FactsDb();
if (!db.Open("/mnt/data/facts", new[] { spec }))
throw new IOException("open failed");
// Write: one thread per shard.
var app = db.AppenderFor(0);
Span<ulong> col = stackalloc ulong[1];
col[0] = BitConverter.DoubleToUInt64Bits(21.5);
app.Append(42, 0, col, 0, default); // series 42, value 21.5
db.Flush(); // durable from here on; call it on a timer
// Read: min, max and mean over everything stored.
ColStats st = db.ReaderFor(0).AggValue(0, 0, ulong.MaxValue);
More in Code & usage: batch ingest, scans, messaging
and serialization.
The settings that matter
Each one with its default and when you'd change it.
Facts, per shard
- Shards
- One
ShardSpec per independent stream. Each shard has its own writer, so
more shards means more parallel ingest. Timestamps only need to rise
within a shard.
- TagMin, TagMax
- The range of series ids a shard holds, up to 16.7 million. Memory is sized to the
range, so a shard holding ids 5000..5999 pays for a thousand, not five thousand.
- Columns
- Up to 16 typed columns per shard (U8 to I64, F32, F64). Pick the narrowest type that
fits: storage and scan speed follow the width.
- HasRef
- Adds a payload to every record. Up to 8 bytes is stored inline; anything bigger goes
to a blob file. Leave it off for pure numbers.
- MaxTotalBytes
- Retention cap per shard. Default 0, keep everything. Set it and the
oldest segments are dropped as new ones arrive. Can be changed while running with
SetMaxTotalBytes.
- HistMin, HistMax
- Off by default. Give an F64 shard its expected value range and every sealed segment
carries a small histogram, so histogram queries over long windows skip the raw
data.
Facts, the whole database
- Flush
- Your crash-safety dial. Anything appended before a
Flush() survives a power
cut. It costs about 1 ms per shard, almost nothing when idle. Call
it every 1 to 60 seconds; the interval is the most you can lose.
- Compression
- Off by default.
UseBuiltinLz4() compresses blob payloads on write and
decompresses on read; your code still sees plain bytes. Worth it for text and JSON,
not for numbers, which are already tight at about 30 bytes a point.
- Prefault on open
- Off by default.
Open(..., prefaultSealed: true) pages existing data in at
startup, so the first queries after a restart are as fast as the rest. Costs startup
time on a big store.
Nio, the network
- Nio partitions
Nio.Init(partitionCount, cqSize) once per process. A partition is one I/O
loop, and each server or client owns one. Size it to the loops you run, plus
headroom.
- IdleSleepMs
- What a loop does when there is no traffic. 0 spins: the lowest
latency, at the price of one busy core per loop, idle or not. 1 backs
off after 5 ms of silence: at most a millisecond of wake-up delay, and
the core is free. Nio defaults to spin, the cluster to back off.
- Timeouts
- A silent connection is marked stalled after 1 s and closed after 3 s more,
and a connect gives up after 5 s. Raise them for links that are quiet by design.
All three are arguments to
Nio.Init.
Relay, pub/sub
- Relay delivery
new RelayBus(db, catalog, deliveryThreads). One delivery thread by default.
Add more when many subscribers have slow handlers.
- Subscribe modes
- Live from now, FromOffset or Replay
to read history first and then go live, and FollowLog for a replica
that keeps reading its own log, through a failover too.
- Ring size
- Each subscriber gets a 64 KiB ring by default. A slow subscriber never blocks
anyone: on a durable subject it reads the log to catch up and loses nothing, on an
ephemeral one the oldest messages are dropped.
Going clusteredC#
Two nodes, one leader. The follower copies sealed segments as they close and
takes over if the leader goes quiet. It asks first: if the leader is still alive and
answering, the takeover is vetoed, so a network blip can't give you two
leaders.
var cfg = new ClusterConfig
{
SelfAddress = "10.0.0.1", SelfPort = 7000, ControlPartition = 0,
PeerAddress = "10.0.0.2", PeerPort = 7000, PeerPartition = 1,
Rank = 0, // 0 = preferred leader
DataPort = 7001, DataServerPartition = 2,
PeerDataPort = 7001, ReconcilePartition = 3,
};
using var node = new ClusterNode(cfg, db);
node.OnPromoted = () => Console.WriteLine("now leading");
node.Start();
// Shutdown order: dispose the node first, then close the db.
How the pair behaves
- Heartbeats
- Every 100 ms. Three missed in a row (about 300 ms) and the
follower starts a takeover.
- Veto window
- The follower waits up to 500 ms for a veto before promoting, and
trusts peer news for 1 s. Shorter means faster failover, longer means fewer false
alarms on a flaky network.
- What you can lose
- Sealed segments are on both nodes. If the leader dies hard, the segment it was still
filling is lost. That is the honest worst case, and the
live failover demo counts it for you.
- Rejoin
- A node that comes back compares logs with the leader, drops anything the leader never
had and catches up. If it was gone so long that the leader has already dropped that
history, it skips forward to what the leader still keeps.
Running it in production
- Data on fast disk. NVMe for anything that must last. A RAM disk is
fine for demos and caches; our public demos write hundreds of MB a second and would
wear out an SSD.
- CPU quotas need backoff. Under a systemd or container CPU limit,
spinning loops get throttled in bulk, heartbeats included, and a healthy leader looks
dead.
- io_uring memory. On Linux each Nio partition locks a little memory.
With many partitions, raise the limit (
LimitMEMLOCK=infinity in the
systemd unit).
- Flush on a timer, close on exit. A clean
Close() keeps
everything; a crash costs at most the last flush interval.
Day two
- Integrity
- Every file carries checksums, verified on open. A write torn by a crash is cut back to
the last good chunk, and the counts of what was repaired are reported, not
hidden.
- Monitoring
GetInfo, GetShardInfo and GetTagInfo give records,
segments, disk use, first and last time per series, and those repair counts. Wire them
into whatever you watch with; there is no Prometheus endpoint yet.
- Backup
- No backup tool yet. Today the backup is a second node: the follower holds every sealed
segment.
- Upgrades
- The on-disk format is versioned and checked on open. Until 1.0 a new format version
won't read the old one, so an upgrade across a format change means
re-ingesting.
Know the limits
It's fast partly because it says no to things. Better you read them here than
find them in week three.
- No SQL
- Queries are API calls: ranges, series reads, scans, aggregates and histograms. No query
language and no joins.
- Append only
- No updates or deletes of single records. Data leaves through retention.
- Time moves forward
- Timestamps must rise within a shard. Late or out-of-order data keeps its own time in a
column.
- One writer per shard
- Parallel ingest comes from more shards, not more threads on one shard.
- Fixed columns
- A shard's columns are set when it's created. A new column means a new shard or a new
database; there is no migration yet.
- Two-node clusters
- One leader and one follower. No three-node quorum and no spreading one database over
many machines.
- .NET first
- C# is the only wrapper today. Everything sits on a plain C interface, so other
languages can follow, but they aren't done.
Security
- On the wire
- Nio has an optional encrypted channel: P-256 key exchange with forward secrecy and
AES-GCM, against a server key the client pins. It is our own handshake on standard
primitives, not TLS, and not externally audited.
- Who may connect
- The channel authenticates the server, not the client. Anyone who can reach the port can
connect, so control access at the network.
- Cluster traffic
- Heartbeats and replication are not encrypted. Keep the pair on a private network.
- At rest
- Data files are not encrypted. Use disk encryption if the data needs it.
- Our own demos
- Cloudflare in front handles TLS, and the apps behind it only listen on
loopback.
Plays well with your AI assistant
The Dry Stack was built with AI coding tools from day one, and it shows in
the shape of the API. Everything is code: no YAML, no hidden environment,
no magic. A shard is a ShardSpec, a cluster is a ClusterConfig,
and every default and every rule ("dispose the node before closing the db") sits right
next to the call it applies to. That is exactly what a coding assistant needs to get
it right the first time.
There is a plain-text summary of the API and settings for AI tools at
llms.txt. Paste it into your favourite model and ask it to design
your setup.