NornicDB

About NornicDB

This large README is shown as a plain-text preview. Read the full README at the source

<p align="center">
  <img src="https://raw.githubusercontent.com/orneryd/NornicDB/refs/heads/main/docs/assets/logos/nornicdb-logo.svg" alt="NornicDB Logo" width="200"/>
</p>

<h1 align="center">NornicDB</h1>

<p align="center">
  <strong>Graph, vector, and historical truth in one database</strong><br/>
  Neo4j-compatible • Hybrid graph + vector retrieval • Historical reads via MVCC<br/>
  <em>Achieving Psygnosis for AI</em>
  <p align="center">
  Multi-arch support: CPU | CUDA | Metal | Vulkan
  <p align="center">
</p>

<p align="center">
  <img src="https://img.shields.io/badge/version-1.4.3-success" alt="Version 1.4.3">
  <a href="https://coveralls.io/github/orneryd/NornicDB?branch=main"><img src="https://coveralls.io/repos/github/orneryd/NornicDB/badge.svg?branch=main" alt="Coveralls Report"></a>
  <a href="https://hub.docker.com/u/timothyswt"><img src="https://img.shields.io/badge/Docker%20Pulls-97.2K%2B-blue" alt="Docker"></a>
  <a href="https://neo4j.com/"><img src="https://img.shields.io/badge/neo4j-compatible-008CC1?logo=neo4j" alt="Neo4j Compatible"></a>
  <a href="https://github.com/qdrant/qdrant"><img src="https://img.shields.io/badge/qdrant-compatible-008CC1?logo=qdrant" alt="Qdrant Compatible"></a>
  <a href="https://go.dev/"><img src="https://img.shields.io/badge/go-%3E%3D1.26-00ADD8?logo=go" alt="Go Version"></a>
  <a href="https://goreportcard.com/report/github.com/orneryd/nornicdb"><img src="https://goreportcard.com/badge/github.com/orneryd/nornicdb" alt="Go Report Card"></a>
  <a href="LICENSE.md"><img src="https://img.shields.io/badge/license-MIT-blue" alt="License"></a>
</p>
<p align="center">
  <a href="https://discord.gg/yszYHrxp4N"><img src="https://img.shields.io/badge/discord-community-00ADD8?logo=discord" alt="Discord Community Server"></a>
</p>

<p align="center">
  <a href="#quick-start">Quick Start</a> •
  <a href="#what-nornicdb-is">What It Is</a> •
  <a href="#why-nornicdb-is-different">Why NornicDB</a> •
  <a href="#performance-snapshot">Benchmarks</a> •
  <a href="#features">Features</a> •
  <a href="#documentation">Docs</a> •
  <a href="#comparison">Comparison</a> •
  <a href="#contributors">Contributors</a>
</p>

## Quick Start

```bash
# Homebrew
brew tap orneryd/nornicdb && brew install nornicdb && brew services start nornicdb

# arm64 / Apple Silicon
docker run -d --name nornicdb -p 7474:7474 -p 7687:7687 -v nornicdb-data:/data timothyswt/nornicdb-arm64-metal-bge:latest

# amd64 / CPU only
docker run -d --name nornicdb -p 7474:7474 -p 7687:7687 -v nornicdb-data:/data timothyswt/nornicdb-amd64-cpu-bge:latest
```

Open [http://localhost:7474](http://localhost:7474) for the admin UI. For NVIDIA CUDA hosts, use `timothyswt/nornicdb-amd64-cuda-bge:latest`. For Vulkan hosts, use `timothyswt/nornicdb-amd64-vulkan-bge:latest`.

---

> Note: Docker on macOS does not expose Metal acceleration. The Apple Silicon image still runs, but GPU acceleration on macOS requires a native install from the [releases page](https://github.com/orneryd/NornicDB/releases) or a local build.

---

> **Writing queries?** Start with the [Hot-Path Cypher Cookbook](docs/performance/hot-path-query-cookbook.md) — proven query shapes that route through the executor's specialized fast paths.
>
> 🤖 **Building with Claude / agents?** The [`docs/skills/`](docs/skills/) directory contains agent-ready skill files for every Cypher surface: query shapes, decay/promotion policies, managed embeddings, vector & hybrid search, and RAG procedures. Drop them into `.claude/skills/` to make agents fluent in NornicDB.

## What NornicDB Is

NornicDB is a graph database for workloads that need graph traversal, vector retrieval, and historical truth in the same system. It speaks Neo4j's language through Bolt and Cypher, exposes REST, GraphQL, and gRPC interfaces, and can preserve Qdrant-style client workflows where that helps migration.

The architecture draws from [research](https://github.com/orneryd/NornicDB/blob/main/docs/research/papers/persistence-semantics-nornicdb/references.bib) in Temporal GraphRAG, agent memory systems, event-sourced decision memory, and persistence semantics for AI agents. These ideas appear in NornicDB as graph-native support for temporal facts, canonical knowledge versioning, replayable history, policy-driven memory retention, and audit-friendly retrieval.

It is built for knowledge systems, agent memory, Graph-RAG, and canonical truth stores where semantic search is only part of the query. The design goal is not to bolt a vector store onto a graph database. The design goal is one execution path for graph, vector, temporal, and audit-oriented workloads.

## Why NornicDB Is Different

- **Neo4j-compatible by default**: Bolt + Cypher support for existing drivers and applications.
- **Built for AI-native workloads**: vector search, memory decay, and auto-relationships are first-class features.
- **Graph, vector, and ledger semantics in one engine**: hybrid retrieval, graph traversal, canonical graph ledger modeling, tritemporal facts, as-of reads, txlog queries, and receipts do not require a second database.
- **Protocol flexibility without splitting the system**: REST, GraphQL, Bolt/Cypher, Qdrant-compatible gRPC, and additive Nornic gRPC live on the same platform.
- **Hardware-accelerated execution**: Metal/CUDA/Vulkan pathways for high-throughput graph + semantic workloads.
- **Operational flexibility**: full images (models included), BYOM images, and headless API-only deployments.

## Deployment Patterns

NornicDB is being used in internal production deployments for stack-consolidation workloads where graph traversal, vector retrieval, and auditability need to live in the same system.

- **Agent and Graph-RAG systems**: replacing a Neo4j + Qdrant + embeddings stack with a single deployment for task tracking, dependency graphs, and retrieval pipelines.
- **Translation and evaluation workflows**: replacing a document store plus embeddings pipeline with a single deployment for graph-native retrieval and faster aggregation paths.

## Transactional Guarantees & Isolation

NornicDB implements Snapshot Isolation at the storage layer. Each transaction is anchored to a specific MVCC version, so point reads, label scans, and snapshot-visible graph traversals resolve against the same committed view of the graph.

- **Repeatable reads within a transaction**: transactions see their own buffered writes, but not commits that land after their read snapshot.
- **Conflict detection at commit**: concurrent graph mutations against the same logical state fail with a normalized `ErrConflict` instead of silently overwriting newer data.
- **Explicit historical reads**: MVCC pruning preserves the current head and a retained floor per logical key; requests below that retained floor fail safely with `ErrNotFound`.
- **Search remains current-state focused**: current search paths are intentionally separate from historical MVCC state.

See [transaction implementation details](docs/user-guides/transactions.md), [historical reads and MVCC retention](docs/user-guides/historical-reads-mvcc-retention.md), and the [canonical graph ledger guide](docs/user-guides/canonical-graph-ledger.md).

## Performance Snapshot

**Northwind benchmark** (48,000 products and 48,000 orders; 30 measured iterations per query after 5 warmups):

| Metric                                  |       NornicDB |        Neo4j |                 Difference |
| --------------------------------------- | -------------: | -----------: | -------------------------: |
| Overall mean query latency              |        0.13 ms |      54.69 ms | **-99.8% (413.40x ratio)** |
| Query-latency-only aggregate throughput | 7,559.12 ops/s |   18.29 ops/s |   **+41240.3% (413.40x)** |
| End-to-end query-loop throughput        |    16.58 ops/s |   15.14 ops/s |        **+9.6% (1.10x)** |
| Query-loop duration                     |       25.327 s |     27.746 s |         **-8.7% (1.10x)** |
| Benchmark wall-clock                    |        35.12 s |      55.06 s |   **-36.2% (1.57x ratio)** |
| Energy during benchmark                 |       325.30 J |     383.86 J |   **-15.3% (1.18x ratio)** |
| Peak memory used                        |       17.4 GiB |     17.7 GiB |    **-1.6% (1.02x ratio)** |

All seed counts and per-query result fingerprints matched between NornicDB and Neo4j. See the [full Northwind benchmark report](docs/performance/1.1.0-northwind-results/comparison.md) for per-query latency, correctness, power, memory, and storage results.

### BEIR Retrieval Quality

The reproducible BEIR SciFact evaluation uses the official 300-query test qrels. With exact hybrid RRF retrieval and the native BGE-M3 reranker, NornicDB recorded:

|   Recall@10 |  Recall@100 |     nDCG@10 |      MRR@10 |     MAP@100 |
| ----------: | ----------: | ----------: | ----------: | ----------: |
| **0.82510** | **0.93563** | **0.72292** | **0.69447** | **0.69215** |

These scores are configuration-specific rather than an official leaderboard placement. See the [BEIR retrieval benchmark](docs/performance/retrieval-recall-benchmark.md#recorded-scifact-results) for the protocol, baselines, confidence intervals, and published-reference context.

### Hybrid Retrieval Benchmarks

Hybrid retrieval is where NornicDB is materially different from vector-only stacks: the query shape is vector search followed by graph expansion in the same engine.

**Local benchmark** (67,280 nodes, 40,921 edges, 67,298 embeddings, HNSW CPU-only index):

| Workload       | Transport |   Throughput |   Mean |    P50 |     P95 |     P99 |     Max |
| -------------- | --------- | -----------: | -----: | -----: | ------: | ------: | ------: |
| Vector only    | HTTP      | 19,342 req/s | 511 us | 470 us |  750 us |  869 us | 1.02 ms |
| Vector only    | Bolt      | 22,309 req/s | 444 us | 428 us |  629 us |  814 us |  968 us |
| Vector + 1 hop | HTTP      | 11,523 req/s | 859 us | 699 us | 1.54 ms | 3.46 ms | 4.71 ms |
| Vector + 1 hop | Bolt      | 13,291 req/s | 747 us | 637 us | 1.29 ms | 3.24 ms | 4.47 ms |

**Remote benchmark** (GCP, 8 vCPU, 32 GB RAM):

- Vector only: ~110.7 ms P50
- Vector + 1 hop: ~112.9 ms P50
- The delta between local and remote matched network RTT closely enough that end-to-end latency was network-bound rather than compute-bound.

This point is: once vector search plus one-hop traversal stays in low single-digit milliseconds locally, the bottleneck shifts from retrieval logic to deployment topology.

See the [hybrid retrieval benchmark write-up](docs/performance/hybrid-query-benchmarks.md) for methodology, caveats, and reproduction queries, and see [Graph-RAG: NornicDB vs Typical](docs/architecture/graph-rag-nornicdb-comparison.md) for the architectural implications.

### 🔬 Academic Validation: UCLouvain Case Study

NornicDB is currently being utilized by researchers at **UCLouvain** to map large-scale **Cyber-Physical Systems** (CPS).

In benchmarks performing **Automata Learning (L\*)**—a high-iteration logic process where an LLM acts as a "Deterministic Teacher" or Oracle—NornicDB outperformed industry-standard graph databases by a significant margin:

- **Efficiency:** **2.2x Faster** than Neo4j in total execution time for formal logic mapping.
- **Throughput:** Successfully handled **1,443 state-transition queries** in ~32 seconds (Avg 22.69ms per full reasoning loop).

| DATABASE     | CALLS    | AVG TIME (ms) | TOTAL (s) |
| ------------ | -------- | ------------- | --------- |
| **NornicDB** | **1443** | **22.69**     | **32.74** |
| Neo4j        | 1443     | 50.20         | 72.43     |

### What Recent Deep-Dives Show

- **Hybrid execution model (streaming fast paths + general engine)**: NornicDB uses shape-specialized streaming executors for common traversal/aggregation patterns while retaining a general Cypher path for coverage and correctness.
- **Scannerless recursive-descent (SRD) parser by default**: the `nornic` parser scans the raw query text with no lexer and no parse tree — clause boundaries and precedence are found by scanning for top-level keywords and operators — so hot-path routing is allocation-free. The `antlr` parser remains for strict parsing and diagnostics.
- **Shared Cypher 5 / 25 grammar**: `LET`, `FILTER`, and `FOR` reading clauses execute in the existing pipeline with an optional `CYPHER 5` / `CYPHER 25` header. The SRD parser accepts them without a header; the ANTLR parser requires `CYPHER 25` for them and parses the header itself.
- **HNSW build acceleration from insertion-order optimization**: BM25-seeded insertion order reduced a 1M embedding build from ~27 minutes to ~10 minutes (~2.7x) in published tests by reducing traversal waste during construction, without changing core quality knobs.
- **Shared seed strategy across indexing stages**: the same lexical seed extraction supports HNSW insertion ordering and improves k-means centroid initialization spread for vector pipeline efficiency.

Read more:

- [Cypher parser modes and execution trade-offs](docs/architecture/cypher-parser-modes.md)
- [How we sped up HNSW construction 2.7x](https://dev.to/orneryd/how-i-sped-up-hnsw-construction-27x-2jhn)

## More Setup Options

### Docker (Recommended)

```bash
# Apple Silicon (includes bge-m3 embedding model)
docker run -d --name nornicdb \
  -p 7474:7474 -p 7687:7687 \
  -v nornicdb-data:/data \
  timothyswt/nornicdb-arm64-metal-bge:latest  # Apple Silicon
  # timothyswt/nornicdb-amd64-cuda-bge:latest  # NVIDIA GPU
```

Open [http://localhost:7474](http://localhost:7474) for the admin UI.

Need a different image/profile (Heimdall, BYOM, CPU-only, Vulkan, headless)?

- [Docker image quick reference](docs/getting-started/image-quick-reference.md)
- [Docker images section](#docker-images)
- [Quick start guide](docs/getting-started/quick-start.md)

### From Source

```bash
git clone https://github.com/orneryd/NornicDB.git
cd NornicDB
go build -o nornicdb ./cmd/nornicdb
./nornicdb serve
```

### Connect

Use any Neo4j driver — Python, JavaScript, Go, Java, .NET:

```python
from neo4j import GraphDatabase

driver = GraphDatabase.driver("bolt://localhost:7687")
with driver.session() as session:
    session.run("CREATE (n:Memory {content: 'Hello NornicDB'})")
```

## Why Switch from Neo4j?

- **99.8% lower overall mean query latency** on the current 48k-product / 48k-order Northwind comparison (413.40x ratio), with matching result fingerprints.
- **Measured retrieval quality** on the official 300-query BEIR SciFact test set, including 0.72292 nDCG@10 with native BGE-M3 reranking.
- **Native graph + vector** in one engine (no separate vector sidecar required).
- **GPU acceleration paths** (Metal/CUDA/Vulkan) for semantic + graph workloads.
- **Drop-in compatibility** via Bolt + Cypher for existing applications.
- **Canonical graph ledger model** for temporal validity, tritemporal fact modeling, as-of reads, and audit-oriented mutation tracking.

> 🚚 **Migrating?** [Neo4j → NornicDB skill](docs/skills/neo4j-migration.skill.md) · runnable Python/Go/Node scripts in [`scripts/migration/neo4j/`](scripts/migration/neo4j/).

## Why Switch from Qdrant?

- **Graph + vector in one engine**: combine semantic retrieval with native graph traversal and Cypher queries.
- **Qdrant gRPC compatibility preserved**: keep Qdrant-style gRPC workflows while adding graph-native capabilities.
- **Hybrid retrieval built in**: vector + BM25 fusion and optional reranking in the same query pipeline.
- **Canonical truth modeling**: versioned facts, temporal validity windows, tritemporal facts, and as-of reads for governance-heavy use cases.
- **Protocol flexibility**: use REST, GraphQL, Bolt/Cypher, Qdrant-compatible gRPC, and additive Nornic gRPC on one platform.

> 🚚 **Migrating?** [Qdrant → NornicDB skill](docs/skills/qdrant-migration.skill.md) · [gRPC surface skill](docs/skills/grpc.skill.md) · runnable Python/Go/Node scripts in [`scripts/migration/qdrant/`](scripts/migration/qdrant/).

## Features

### Retention Policies

Retention policy enforcement is available, but it is disabled by default and must be explicitly enabled. When retention is off, NornicDB does not create the retention manager and does not start the retention sweep background worker. When enabled, retention supports label-aware policy evaluation, legal holds, GDPR erasure tracking, and admin APIs.

See [Retention Policies](docs/user-guides/retention-policies.md) and [Configuration](docs/operations/configuration.md#retention-policies-opt-in).

### 🔌 Neo4j Compatible

Designed to work with existing Neo4j drivers and Bolt/Cypher workflows, with minimal or no application changes for supported query shapes.

- **Bolt Protocol** — Use official Neo4j drivers
- **Cypher Queries** — Full query language support
- **Cypher 25** — `LET`, `FILTER` and `FOR` reading clauses with the optional `CYPHER 5` / `CYPHER 25` header
- **Schema Management** — Constraints, indexes (single- and multi-property), vector indexes
- **Qdrant gRPC API Compatible** — Works with Qdrant-style gRPC vector workflows

> 🤖 **Agent skill:** [Bolt Client](docs/skills/bolt-client.skill.md) — connection defaults, retry classification, MERGE under concurrent writers, batch sizing.

### 🧠 Knowledge-Layer Scoring

Profile-driven decay and promotion scoring with the [Ebbinghaus-Roynard four-layer decomposition](https://arxiv.org/pdf/2604.11364). The engine does not hardcode cognitive tiers. Operators model their own labels and lifecycle rules using Cypher DDL.

Typical deployments map the four-layer decomposition onto labels such as:

- **Knowledge**: durable fact labels using `NO DECAY` or neutral profiles
- **Memory**: episodic/session labels using bounded half-life decay
- **Wisdom**: stable directive labels using conservative decay plus promotion rules
- **Evidence/links**: edge types with their own decay and suppression behavior

Those categories are conventions, not built-in engine classes. NornicDB provides the authoring and diagnostics surface:

- `CREATE/ALTER/DROP/SHOW DECAY PROFILE`
- `CREATE/ALTER/DROP/SHOW PROMOTION PROFILE`
- `CREATE/ALTER/DROP/SHOW PROMOTION POLICY`
- `decayScore(entity)`, `decay(entity)`, `policy(entity)`, `reveal(entity)`
- `CALL kp.info|profiles|policies|resolve|deindexStatus()`

```cypher
CREATE DECAY PROFILE working_memory OPTIONS {
  halfLifeSeconds: 604800,
  function: 'exponential',
  visibilityThreshold: 0.10
}

CREATE DECAY PROFILE session_retention
FOR (n:SessionRecord)
APPLY {
  DECAY PROFILE 'working_memory'
  n.tenantId NO DECAY
}

MATCH (n:SessionRecord) WHERE decayScore(n) > 0.5
RETURN n ORDER BY decayScore(n) DESC
```

> 📖 Deep dive: [Knowledge-Layer Policies](docs/user-guides/knowledge-layer-policies.md), [Decay Profiles](docs/user-guides/decay-profiles.md), [Promotion Policies](docs/user-guides/promotion-policies.md), and [Ebbinghaus-Roynard Bootstrap](docs/user-guides/ebbinghaus-roynard-bootstrap.md).
>
> 🤖 **Agent skills:** [Knowledge Policies](docs/skills/knowledge-policies.skill.md) · [Decay Tuning](docs/skills/decay-tuning.skill.md) · [Promotion Policies](docs/skills/promotion-policies.skill.md)

### 🔗 Auto-Relationships

NornicDB weaves connections automatically:

- **Embedding Similarity** — Related concepts link together
- **Co-access Patterns** — Frequently queried pairs connect
- **Temporal Proximity** — Same-session nodes associate
- **Transitive Inference** — A→B + B→C suggests A→C

### 🎯 Vector Search

Native semantic search with GPU acceleration and hybrid retrieval support.

> 📖 Deep dive: [Vector Search Guide](docs/user-guides/vector-search.md) and [Qdrant gRPC Endpoint](docs/user-guides/qdrant-grpc.md).
>
> 🤖 **Agent skills:** [Vector & Full-Text Search](docs/skills/vector-search.skill.md) · [Managed Embeddings](docs/skills/managed-embeddings.skill.md) · [RAG Procedures](docs/skills/rag-procedures.skill.md)

**Cypher (Neo4j-compatible):**

```cypher

Frequently Asked Questions