v0.2.0 Technical Preview / research OSS. RocheDB is not yet presented as a production replacement for Redis, PostgreSQL, MongoDB, Apache Arrow, or a dedicated vector database. The current release target is a measurable prototype of ring/galaxy-oriented storage, retrieval, persistence, drivers, and cluster smoke behavior.

RocheDB's practical goal is simple: reduce the amount of data a system has to read, transfer, hold in memory, and pass to downstream AI/RAG or application logic. In one sentence:

RocheDB stores data with a coordinate-like ring, then uses that placement at read time to reduce the amount of data that must be searched, transferred, and passed to downstream systems.

The celestial mechanics vocabulary, especially orbits, encounters, rings, and accretion, is an algorithmic design source rather than the value proposition. The value proposition is smaller working sets, fewer transferred bytes, fewer retrieval tokens, and lower infrastructure pressure when data can be placed by meaningful locality.

It is not an AI-only database. The same model is intended to work as a general NoSQL/document store for web systems: users, tenants, regions, products, categories, dates, and application state can all become rings or ring hierarchies. The core idea is that locality, authorization boundaries, dump units, migration units, and retrieval scope should be visible to the database instead of being reconstructed after every query.

Writes are intentionally light. A human, application, or import rule places data into a ring. Reads use the ring, hierarchy, centroid, coherence, mass, retrieval profile, and projection to keep the candidate set small.

RocheDB is NoSQL, but it is not a MongoDB-compatible or ad-hoc aggregation database. The main difference is that a ring is not just a collection name; it is part of the read path. RocheDB expects applications, routes, tenants, import rules, or operators to place data into meaningful rings so later reads can avoid unrelated working sets. See How RocheDB Differs From Typical NoSQL.

RocheDB's main bet is not "scan the entire corpus faster." It is "avoid reading unneeded data in the first place." Training data, document corpora, and application histories tend to grow. Systems that keep scanning wider datasets eventually run into physical limits: memory bandwidth, semiconductor supply, energy, cooling, cloud cost, and latency. RocheDB tries to move cost from total corpus size toward semantic working-set size.

• Documentation site entry point: docs/index.md
• Installation: docs/installation.md
• Public API reference: docs/public-api.md
• Configuration reference: docs/config-reference.md
• CLI reference: docs/cli-reference.md
• How RocheDB differs from typical NoSQL: docs/nosql-positioning.md
• Concept: docs/rochedb-concept.md
• Detailed design: docs/rochedb-design.md
• Feature status / roadmap: docs/rochedb-status.md
• Release checklist: docs/release-checklist.md
• GitHub release draft: docs/github-release-v0.2.0.md
• Driver / FFI roadmap: docs/rochedb-driver-roadmap.md
• Driver installation guide: docs/driver-installation.md
• FAISS versioning policy: docs/faiss-versioning.md
• Vector backend selection: docs/vector-backends.md
• Protocol compatibility: docs/protocol-compatibility.md
• Universe sync: docs/universe-sync.md
• Threat model: docs/threat-model.md
• Benchmark notes: docs/rochedb-bench.md
• Cloud operations metrics: docs/cloud-operations.md
• Topology configuration reference: docs/topology-config.md
• Topology pattern catalog: docs/topology-examples.md
• Shelfer integration boundary: docs/rochedb-shelfer-integration.md
• Halo capture design: docs/rochedb-halo-capture.md
• Changelog: CHANGELOG.md
• Third-party notices: THIRD_PARTY_NOTICES.md
• Contribution policy: CONTRIBUTING.md

RocheDB v0.2.x is a technical preview. The Nim package is available through Nimble, while non-Nim language packages are still repository-local foundations.

Prerequisites:

• Nim 2.0.0 or newer
• git
• gcc or another C compiler supported by Nim

Install the CLI and Nim library:

nimble install rochedb
roche --help

Clone the repository when you want to run the full source test suite, examples, or driver smoke tests:

git clone https://github.com/puffball1567/rochedb.git
cd rochedb
scripts/test_core.sh
nimble install -y

Nimble installs binaries into ~/.nimble/bin by default. If roche is not found, add it to your shell PATH:

export PATH="$HOME/.nimble/bin:$PATH"

For server-style installs, build locally and install the binaries into /usr/local/bin, the usual source-install location for database tools:

nim c -d:release --nimcache:/tmp/nimcache_roche -o:bin/roche src/rochecli.nim
nim c -d:release --nimcache:/tmp/nimcache_roched -o:bin/roched src/roched.nim
sudo install -m 0755 bin/roche /usr/local/bin/roche
sudo install -m 0755 bin/roched /usr/local/bin/roched

See docs/installation.md for PATH and system install details.

Use RocheDB from a Nim program in this repository by importing the public module:

import rochedb

For command-line tools and demos that need repo-local binaries, build them under bin/:

nim c -d:release --nimcache:/tmp/nimcache_roche -o:bin/roche src/rochecli.nim
nim c -d:release --nimcache:/tmp/nimcache_roched -o:bin/roched src/roched.nim

Basic CLI document workflow:

roche put --data=data --ring=docs/japan --payload='{"title":"Hello"}'
roche list-ring --data=data --ring=docs/japan
roche get --data=data --id=RAW_ID

Optional FAISS vector backend:

scripts/fetch_faiss.sh
scripts/setup_faiss_toolchain.sh   # only needed when system CMake is too old
scripts/build_faiss_bridge.sh
roche doctor

The built-in exact vector backend works without FAISS. FAISS is recommended for production-style broad vector reads when the bridge is available. See docs/driver-installation.md for language drivers and docs/faiss-versioning.md for FAISS version control.

import rochedb

var db = rochedb.open(dataDir = "data")   # persistent; omit dataDir for memory-only
db.setGalaxyDescription("Product and support knowledge")
db.setRingDescription("docs/japan", "Japanese product documentation and support articles")

let id = db.put("hello", ring = "docs/japan")
echo db.get(id)
echo db.atlas()                           # galaxy/ring map for agents and tools

echo db.locate(id)                        # current owner, computed locally
echo db.locate(id, at = 120.0)            # future owner, also computed locally

get(id) is the fastest path when the application already has a RocheDB ID. If the ID is not known, start from a ring.

import rochedb

var db = rochedb.open(dataDir = "data")

discard db.put("""{"slug":"hello","title":"Hello"}""", ring = "docs/japan")
discard db.put("""{"slug":"refund","title":"Refund guide"}""", ring = "docs/japan")

for item in db.listByRing("docs/japan"):
  echo item.payload

For vector/RAG-style lookup, search the ring directly:

let hits = db.retrieve(@[1.0'f32, 0.0'f32], ring = "docs/japan", budget = 3)

for hit in hits:
  echo hit.payload

If the right ring is not obvious, use atlas() and ring descriptions to choose the search scope first. RocheDB is designed to avoid ID-less global scans when a ring coordinate is available.

RocheDB is useful outside AI workflows when the application naturally has locality boundaries.

• Tenant locality: ring = "tenant/acme/orders" keeps query scope, dump scope, backup scope, and future authorization scope aligned.
• Smaller responses: query(id, "{ title status }") returns only requested fields, so large JSON documents do not need to cross the process or network boundary on every read.
• Import routing: JSONL exports from MongoDB-like stores can be imported and routed by fields such as tenant, category, region, or date.
• Galaxy isolation: separate services can use separate galaxies, data directories, credentials, and clusters while using the same implementation.
• Explainable location: locate(id) and locate(id, at=...) make placement observable without a directory service.
• Incremental adoption: start with embedded open(dataDir=...), then move to cluster connect(...) when the service needs separate nodes.

The public driver surface is intentionally small. External drivers can use high-level wire frames such as PUTR, GETID, QRYID, BGET, and RETRIEVE; they do not need to reimplement RocheDB's ring-key, orbit, or ID rules.

The table below lists current repository-local driver foundations. Publication priority for language packages is tracked in docs/rochedb-driver-roadmap.md.

Rust is the first external driver publication target and is being split out of the core repository. Its package link will be added when published.

Detailed setup notes are in docs/driver-installation.md. Nimble package registration is complete. Package publishing to npm, PyPI, Cargo, Composer, NuGet, Maven, and other non-Nim registries remains a post-v0.2 roadmap item.

Run roched nodes with the same peer list:

roched --id=0 --peers=h1:7301,h2:7301,h3:7301 --data=/var/lib/roche

Then connect with the same API shape:

var db = connect("h1:7301,h2:7301,h3:7301")
let id = db.put(%*{"title": "RocheDB", "author": {"name": "Ada"}}, ring = "docs")
echo db.query(id, "{ title author { name } }")
echo db.locate(id, at = epochTime() + 60)

The core placement rule is deterministic:

data location = deterministic function E(id, t) -> node

Every node can compute where a record is now, and where it will be later, without a directory lookup. Handoffs are scheduled from ephemeris state rather than from a central rebalance service.

Canonical data should normally live in one galaxy/ring. Multiple views should be modeled with hierarchy, naming conventions, import rules, retrieval profiles, or projection. RocheDB core does not try to keep duplicate logical records in multiple galaxies perfectly synchronized.

For asynchronous maintenance across rings, RocheDB has a minimal warp queue. A warp job scans specified rings over time and drops a patch into matching documents. It is closer to a maintenance asteroid belt than a relational join: jobs have attempts, retry timing, acknowledgements, and dead-letter state, and their state is persisted in the WAL. Rich scheduling, backoff policy, audit history, and flow orchestration are intended to live in adapters such as the future rochedb-flow integration.

RocheDB's strongest benchmark story is working-set reduction. Local reads are also in the same broad latency class as existing databases, but the larger claim is that RocheDB can reduce how much data is touched before ANN, rerank, LLM, or application processing.

Reference latency results are tracked in docs/rochedb-bench.md. The short version is:

• RocheDB 3-node TCP with persistence enabled measured 45.6 us per single-key read and 49.2 us per single-key write in the local smoke benchmark.
• PostgreSQL 14 on the same machine measured 84 us for primary-key read and 77 us for synchronous_commit=off single-row write in the referenced comparison.
• A Docker Redis smoke test measured RocheDB TCP BGET at 1.56 us/op and Redis pipeline GET at 3.56 us/op under the documented local conditions.

These are not universal performance claims. They show that the local read path is already competitive enough for the working-set reduction story to matter.

include/rochedb.h plus lib/librochedb.so is the foundation for non-Nim bindings.

roche_init();
if (roche_abi_version() != ROCHE_ABI_VERSION) return 1;

void *db = roche_connect("h1:7301,h2:7301,h3:7301");
roche_id id;

roche_set_galaxy_description(db, "Product and support knowledge");
roche_set_ring_description(db, "docs", "Documentation ring");
roche_put(db, "docs", "hello", 5, &id);

float v[2] = {1.0f, 0.0f};
roche_put_vec(db, "docs", "hello", 5, v, 2, &id);

roche_batch_result *b = roche_batch_get(db, &id, 1);
roche_batch_get_free(b);

roche_retrieve_result *r = roche_retrieve(db, v, 2, "docs", 8, 0, 0);
roche_retrieve_free(r);

size_t n;
char *j = roche_query(db, id, "{ title }", &n);
roche_free(j);

char *a = roche_atlas(db, v, 2, 8, &n);
roche_free(a);

int node = roche_locate(db, id, -1.0);

scripts/test_core.sh
scripts/test_all_smoke.sh

Include driver compatibility checks when local toolchains are available:

ROCHE_TEST_DRIVERS=1 scripts/test_all_smoke.sh

nim c -d:danger -o:bin/rochesim src/rochesim.nim
bin/rochesim all

nim c -d:danger -o:bin/rochebench src/rochebench.nim
bin/rochebench

nim c -d:release -o:bin/roche src/rochecli.nim
roche working-set-bench --n=100000 --rings=100 --queries=50 --budget=20
roche memory-pressure-bench --n=100000 --rings=100 --queries=50 --budget=20 --payload-bytes=512
RUN_REDIS=0 examples/memory_pressure_case_study.sh
examples/ai_rag_case_study.sh

Use an existing local Redis server:

roche redis-bench --n=100000 --payload-bytes=100 --redis=127.0.0.1:6379

Or use the Docker-based smoke comparison:

examples/redis_bench.sh
ROCHED=1 examples/redis_bench.sh

nim c -d:release -o:bin/roched src/roched.nim
nim c -d:release -o:bin/roche src/rochecli.nim

Strong durability mode:

bin/roched --id=0 --peers=127.0.0.1:7301 --data=/var/lib/roche --durability=strong

Ring-prefix authorization:

bin/roched --id=0 --peers=127.0.0.1:7301 --user=alice --password=secret --allow-ring=allowed

Minimal RBAC plus ring-prefix authorization:

bin/roched --id=0 --peers=127.0.0.1:7301 \
  --role=reader:read:reader:allowed \
  --role=writer:write:writer:allowed \
  --role=admin:admin:admin:allowed

Encrypted backup / restore:

roche backup-encrypted --data=data --backup=backup.enc --passphrase=change-me
roche restore-encrypted --backup=backup.enc --data=restored --passphrase=change-me

python3 -m unittest discover -s drivers/python/tests
node --test drivers/node/test/*.test.js

Cluster demo:

./examples/cluster_demo.sh

Universe sync demo:

./examples/universe_sync_demo.sh
./scripts/universe_sync_remote_smoke.sh

This shows a WAL-backed eventual sync outbox, idempotent apply, ack/prune, and the CLI handoff boundary between two local data directories or a remote RocheDB server. See docs/topology-examples.md for topology patterns.

nim c --app:lib -d:release -o:lib/librochedb.so src/rochedb_capi.nim
gcc examples/demo.c -Iinclude -Llib -lrochedb -Wl,-rpath,'$ORIGIN/../lib' -o bin/demo
bin/demo

scripts/fetch_faiss.sh
scripts/setup_faiss_toolchain.sh
scripts/build_faiss_bridge.sh
roche doctor
examples/vector_backend_bench.sh

By default this fetches the configured FAISS tag, currently v1.14.3, and records the actual commit in third_party/faiss.version. It does not enforce an exact commit unless ROCHE_FAISS_COMMIT is set. See docs/faiss-versioning.md for tag overrides, exact commit pinning, upgrades, downgrades, and security update handling.

FAISS is the recommended production vector backend when the bridge is available. RocheDB's built-in exact backend remains useful as a dependency-free fallback for tests, small embedded deployments, and environments where FAISS cannot be installed. See docs/vector-backends.md for the backend selection rule and local smoke benchmark.

RocheDB forces Nim ARC through config.nims. Avoiding reference cycles is a structural constraint of the codebase, not just a style preference.

src/rochedb.nim        public API for embedded and cluster modes
src/roched.nim         node server: scale-out, persistence, handoff
src/rochecli.nim       CLI, demos, benchmarks, maintenance commands
src/rochedb_capi.nim   C ABI
src/roche/core.nim     ephemeris fast layer: Orbit, ArcTable, encounters
src/roche/select.nim   GraphQL-like projection
src/roche/store.nim    particle store plus append-only WAL
src/roche/wire.nim     wire protocol and persistent client
src/rochesim.nim       PoC verification CLI
drivers/               language drivers and wrappers
include/rochedb.h      C header
examples/              C demo, cluster demo, benchmark scripts
examples/compose/      Docker Compose topology demos
tests/                 unit and smoke tests

RocheDB core and the OSS drivers are released under Apache-2.0; see LICENSE.

Third-party dependency and tooling notices are tracked in THIRD_PARTY_NOTICES.md. Security assumptions and known gaps are tracked in docs/threat-model.md.