Orleans.Lattice is a platform for building durable, distributed state systems on Microsoft Orleans.

At its centre is a sorted, horizontally-scalable, conflict-free key-value store that runs inside your own cluster. Around it are the concerns a real system acquires once it outgrows one machine - storage, identity, governance, replication, administration, observability - each implemented as a companion package behind a seam in the core, rather than baked into it.

It is local-first. A complete deployment runs on a single machine with no cloud dependency, and the same programming model carries through to a globally distributed, active-active estate. What changes between those two points is which companion packages a host registers, not the code that reads and writes data.

The core is a sorted, durable, horizontally-scalable key-value store embedded in your Orleans cluster. Keys are string, values are byte[], and typed-value helpers layer automatic serialization on top. No external database, no coordinator service, no external queue.

The keyspace is split across self-balancing B+ sub-trees that rebalance themselves online. The durability boundary is a write-ahead log. Conflict resolution is algebraic rather than lock-based or consensus-based, which is what lets any cluster accept a write to any key.

The core alone supports:

• Point reads, writes, deletes, and per-entry TTL.
• Ordered key and entry scans - forward, reverse, and range-bounded.
• Multi-key atomic writes with all-or-nothing visibility - within a tree, and across multiple trees.
• Bulk loading from one-shot batches or streaming IAsyncEnumerable sources.
• Durable, resumable cursors that survive silo failovers and client restarts.
• Online resize, online reshard, and online snapshots (offline mode also available).
• Soft delete with a configurable retention window, and undo of resize within the window.
• Per-tree event stream, diagnostics, and System.Diagnostics.Metrics instruments.

The name comes from its use of lattice-based state primitives - mathematical structures where merges are commutative, associative, and idempotent - which is what makes the system conflict-free and recoverable without distributed locks or consensus (provided you use its CRDT Primitives).

Every durable distributed system ends up solving the same set of problems: sharding, online rebalancing, crash-safe durability, conflict resolution, backup, tenancy, identity, replication, and an operator surface. They are usually assembled from a database, a cache, a queue, an identity provider, and a layer of glue - each with its own operational model, its own failure modes, and its own consistency story to reconcile with the others.

Orleans already supplies the hard parts of a distributed runtime: virtual actors, single-threaded execution per grain, location transparency, and failover. What it does not supply is an ordered, durable, shardable store to put underneath them. Orleans.Lattice fills that gap, and takes three positions about how:

• The store lives in the cluster. State is held by grains in the same process as the code using it, so a read is a grain call rather than a network round trip to a separate tier with its own scaling and failure envelope.
• Conflict resolution is algebraic. Merges are commutative, associative, and idempotent, so convergence needs no distributed lock manager and no consensus round trip. This is what makes active-active writes across regions tractable.
• Everything else is a seam. Storage, identity, governance, replication, administration and observability are companion packages that plug into documented extension points. A host that registers none of them runs the core library alone, with no ambient cost for the features it does not use.

The result is a platform rather than a product: it is not tied to one application category, and the deployment topology is a configuration decision taken late, not an architecture decision taken up front.

Orleans.Lattice is a substrate for durable distributed state, so the categories below are examples of what the same primitives compose into, not a fixed feature set.

A deployment grows in three stages. The application code that reads and writes data is identical in all three: it resolves ILattice and calls it. Each stage adds companion packages and configuration, not a rewrite.

One machine, no cloud account, no external services. This is a first-class deployment target, not a degraded development mode.

• Durability. The file write-ahead log gives an append-and-fsync log per shard on local disk, with crash-safe reconciliation and background compaction. The in-memory WAL is the default if you do not need durability yet.
• Inspection. The Explorer console (in progress) browses trees, topology, data and history over the cluster's gRPC APIs.
• AI access. The MCP server exposes the cluster's API facades as Model Context Protocol tools an agent can call.

A shared cluster with real users, so identity, policy and data shape start to matter.

• Identity. OIDC for Okta, Auth0, Keycloak, Ping or Google, or Entra ID for Microsoft identities, resolving credentials to subjects with transitive group membership.
• Authorization. Fail-closed, default-deny policy per tree, prefix, or key, enforced on the core data path and on every external API.
• Schema. Per-tree write validation and value versioning, with dead-letter diversion of non-compliant writes and read-time upcasting of stale values.
• Tenancy. Keyspace-partitioned tenants with a lifecycle, per-tenant quotas, metering and rate limiting.

Multiple regions, each serving reads and writes.

• Replication. Active-active replication between clusters. Any cluster can write to any tree; concurrent updates converge deterministically, and atomic multi-key writes stay all-or-nothing on every peer.
• Backup and disaster recovery. Causally consistent backup to a shared sink that is the single source of truth, so the catalog rebuilds from it and a backup cold-restores into a fresh cluster.
• Autoscaling. A cluster-aggregate scaling signal an external autoscaler such as KEDA can scrape.
• Blueprint. The reference architecture is a worked design plus a parameterised deployment kit for an active-active, cross-region estate on Azure Container Apps.

The distinctive structural property of Orleans.Lattice is that major concerns are not implemented in the core. Each is a seam the core defines and a companion package fills. A host composes the platform it needs by registering packages; nothing it leaves out is present at runtime.

flowchart TD
    App["Applications<br/>knowledge systems, AI memory, digital twins, search,<br/>control planes, multi-tenant SaaS, collaboration"]

    App --> Explorer["Explorer console<br/>(in progress)"]
    App --> Apis["API facades<br/>state, data, auth, schema,<br/>backup, tree admin, tenant admin"]
    App --> Mcp["MCP server<br/>tools for AI agents"]

    Explorer --> Core
    Apis --> Core
    Mcp --> Core
    App -. "in-process ILattice" .-> Core

    Core["Orleans.Lattice core<br/>sharded CRDT B+ tree, write-ahead log,<br/>durability boundary, grain catalogue"]

    Core --> Storage["Storage"]
    Core --> Identity["Identity"]
    Core --> Governance["Governance"]
    Core --> Replication["Replication"]
    Core --> Administration["Administration"]
    Core --> Observability["Observability"]

    Storage --> StoragePkgs["Storage.AzureTable<br/>Storage.File<br/>Backup.AzureBlob"]
    Identity --> IdentityPkgs["Membership<br/>Membership.Oidc<br/>Membership.Entra<br/>Auth"]
    Governance --> GovernancePkgs["Schema<br/>Tenancy"]
    Replication --> ReplicationPkgs["Replication<br/>Replication.Grpc"]
    Administration --> AdminPkgs["Backup<br/>Api.TreeAdmin<br/>Api.TenantAdmin"]
    Observability --> ObsPkgs["Dashboards<br/>Scaling<br/>Api.Telemetry"]

Three consequences follow from this shape, and they are worth understanding before reading the catalogues:

• Opt-in cost. A capability you do not register costs nothing. Tenancy, for example, is layered on the core through null seams, so a host without it is byte-for-byte unchanged.
• Substitutable implementations. A seam is a public contract, not an internal detail. Storage backends, identity providers, compression algorithms, backup sinks, and Explorer plugins are all replaceable with your own.
• Uniform external surface. Every external caller - gRPC client, operator console, AI agent - goes through the same transport-agnostic API facades and the same fail-closed authorization gate, so a permission means the same thing whichever surface asks.

The complete inventory lives in PACKAGES.md, and the capability each package delivers is catalogued in FEATURES.md.

• Self-organising under load. Hot regions of the keyspace re-balance themselves online - no downtime, no lost writes, no coordination protocol. Cold regions stay cheap.
• Strongly consistent from the outside. Point reads, writes, and ordered scans always see a consistent view of the data, even while the cluster is rebalancing underneath. See Consistency for the per-operation guarantee matrix.
• Crash-safe by construction. A silo crash at any point - mid-write, mid-split, mid-snapshot, mid-bulk-load - is recovered without operator intervention and without data loss.
• Eventually convergent under failure. Storage faults, stale routing, and interrupted operations cannot corrupt data; once the fault window closes, the tree converges to the correct state.
• No locks, no consensus round-trips. No Paxos, no Raft, no distributed lock manager. All conflict resolution is algebraic.

Behaviour is validated end-to-end by a suite of chaos tests that hammer a live cluster with concurrent reads, writes, scans, splits, resizes, and reshards - optionally with random storage-write faults - and assert both live consistency and eventual convergence. The concurrency-critical protocols go further: the atomic-commit protocol, the WAL seams, the distributed lock, and the atomic-action coordinator are driven by pure deterministic cores that a verification tier machine-checks with Coyote and, for atomic commit, a TLA+ specification.

Register Lattice on a silo. AddLattice registers the grain catalogue, the grain storage provider (via the supplied callback), and the in-memory write-ahead-log backend in a single call:

siloBuilder.AddLattice((silo, storageName) =>
    silo.AddMemoryGrainStorage(storageName));

// AddLattice registers the in-memory WAL by default - swap for a durable backend in production.

// elsewhere - on the client or inside a grain - resolve a tree by name:
var lattice = grainFactory.GetGrain<ILattice>("my-tree");

// Values are byte[] at the core, but the typed extensions serialize for you, so
// application code rarely touches a byte[]. These overloads default to JSON:
await lattice.SetAsync("user/42", new User("Ada", 36));
var user = await lattice.GetAsync<User>("user/42");
Console.WriteLine(user?.Name);

// Pass an ILatticeSerializer<T> to choose your own format, or use the raw
// byte[] surface directly when you want to own the encoding:
await lattice.SetAsync("hello", "world"u8.ToArray());

For production, swap the in-memory WAL for a durable backend - e.g. Azure Table Storage from the sibling package:

siloBuilder
    .AddLattice((silo, storageName) => silo.AddMemoryGrainStorage(storageName))
    .AddAzureTableWalStorage(opts =>
    {
        opts.ConnectionString = "DefaultEndpointsProtocol=https;...";
    });

Add cross-cluster replication on top by registering AddLatticeReplication(...) alongside the WAL. See the Orleans.Lattice.Replication overview for the full multi-cluster setup.

For a local-first alternative to the Azure Table backend, register the file write-ahead log instead and keep the whole deployment on one machine.

RepoContext is an MCP server that gives an AI agent durable, conflict-free memory about a codebase: a structural record and content digest per file, symbol outlines and a reverse cross-reference graph, agent-authored notes and decisions with optional TTL, semantic search over embeddings, and a budgeted context bundle with reuse accounting. It runs as a single local container.

It is worth reading as a worked example because it composes most of the platform at once, and does so without a line of bespoke storage code:

• Every record is a CRDT value on a Lattice tree, so concurrent agents converge without locks and the store inherits durability, TTL and tombstone compaction from the core.
• Orleans.Lattice.Storage.File makes the container restart-durable with no cloud account.
• Orleans.Lattice.Vector provides the approximate nearest-neighbour index behind semantic search, persisted on a tree so a restart reloads it rather than rebuilding it.
• Orleans.Lattice.Api.Mcp supplies the agent-facing surface and the fail-closed authorization gate; RepoContext adds no authorization path of its own.
• Orleans.Lattice.Api.Mcp.RepoContext.Replication turns the same store into a multi-cluster one by choosing a per-tree merge mode, which is the deployment journey applied to a real application.
• The core's distributed lock is what makes RepoContext's agent-operated backlog safe for several agents to drain at once: a claim over a memory record borrows the lock's fencing token and bounded lease wholesale, and the token is then enforced on the record's own write path, so a superseded agent is refused rather than trusted. No new coordination primitive was written for it.

RepoContext demonstrates Orleans.Lattice. It does not define it. It is one application category among many, and nothing in the platform is shaped around it.

See the Repo-context MCP documentation and the container sample.

Use these documents for day-to-day use and operations:

• API Reference - the public ILattice interface, batch operations, options, and serializable types.
• Configuration - options reference, per-tree overrides, immutability constraints, storage provider.
• Security - opt-in identity, authorization, and enforcement: how membership, policy, fail-closed enforcement, the external APIs, and cross-cluster convergence fit together, with links to each package.
• Predicate Operations - server-side predicate push-down for typed reads, conditional and atomic writes, scans, cursors, and range deletes.
• Migrating from an External Store - importing an existing Redis, relational, or Cosmos DB dataset over the bulk-load path, with key-design, value-serialization, and post-load verification guidance.
• Queues - the public ILatticeQueue<T> cluster-internal FIFO primitive, bounded-queue eviction, and throughput guidance.
• Compression - the public ILatticeCompressor seam, AddLatticeCompressor registration, tag-space partitioning, and how to plug in a custom algorithm.
• Samples - runnable sample projects exercising ILattice.
• Benchmarks - prerequisites, running benchmarks, interpreting results.
• Troubleshooting - symptom-driven diagnosis: reading a DiagnoseAsync report, storage-provider write failures, split activity, slow scans, and stale reads.

For internals (the "how"):

• Architecture - grain layers, sharding, root promotion, grain mapping, capacity.
• Tree Structure - internal/leaf node layout, two-phase leaf splits, idempotent split propagation.
• Tree Storage - per-provider storage limits, node size estimation, sizing recommendations.
• Verified Atomic-Commit - the proven-core pattern, Coyote concurrency tier, property catalogue, and TLA+ spec behind the atomic-commit protocol.
• Verified WAL - the proven-core pattern and Coyote concurrency tier behind the WAL shipping, GC-trim, cursor-registry, move-fence, shutdown-drain, offset-allocation, blocked-floor, and move-resume seams.
• WAL - write-ahead log as the sole foreground-commit durability boundary.
• WAL Causal+ - causal+ entry-schema extension, dependency satisfaction, snapshot semantics.
• WAL Storage Providers - IWalStorageProvider durability seam, in-memory default, optional Azure Table backend.
• WAL Tuning - how WalMaxPendingBatches and WalPartitions interact with a durable backend's throughput envelope; default sizing rules and the storage-account ceiling above which the cap stops helping.
• WAL Saturation Signal - the per-tree, three-state back-pressure surface (IWalSaturationSignal, IWalSaturationObserver) that lets callers throttle offered load before silent queueing on the writer-side admission gate.

For the complete catalogues:

• FEATURES.md - every capability, grouped by concern, with its docs and sample.
• PACKAGES.md - every package, grouped by the seam it fills.
• reference-architecture.md - the active-active, cross-region deployment blueprint and its parameterised deployment kit.
• llms.txt - the documentation index for AI agents and LLM tooling.

Orleans.Lattice inherits the asymptotic properties of a B+ tree. In a single shard containing n keys with branching factor b:

With the default branching factor (~128 children per node), a shard with two million keys is only three levels deep, so a single-key lookup crosses just three grains. Sharding (default 64) reduces per-shard n further; cross-shard operations scatter-gather across all shards.

Measured single-silo throughput and latency against real Azure Tables are in the single-silo performance guide.

See CHANGELOG.md for the per-version notes and docs/RELEASING.md for the per-package tag-and-publish protocol.

Contributions are welcome! To get started:

1. Fork the repository and create a feature branch from main.
2. Make your changes and ensure all existing tests pass.
3. Add tests for any new functionality.
4. Open a pull request with a clear description of the change and the problem it solves.

Please open an issue first to discuss significant changes or new features before starting work.

This project is licensed under the MIT License. See LICENSE for details.