ConDB: The KV-Cache Native Context Database

A new context database for reasoning-driven retrieval via tree search.
Fast, context-aware retrieval at scale with up to 70% less token cost.

ConDB (Context Database) is a tree-structured context database that uses LLM-powered reasoning-based retrieval via tree search instead of vector similarity — no vector DB, no chunking. It accepts PageIndex-compatible document trees, ChatIndex conversation trees, filesystem trees, and custom hierarchical JSON — with no runtime dependency on either. The LLM reasons over the tree, like a human expert using a table of contents, to locate relevant content.

• Similarity ≠ relevance — vector search retrieves what looks similar, not what is truly relevant. Similar-looking chunks may differ in intent (low accuracy), while truly relevant information may be expressed in very different language and get missed entirely (low recall). True relevance requires reasoning
• Chunking breaks semantic continuity — documents must be split into fixed-size segments to fit embedding models, causing context fragmentation that destroys their natural structure and cross-section relationships
• Retrieval is blind to context — embedding models encode the query alone, ignoring conversational history, user intent, and other contextual signals

ConDB replaces this with reasoning-based tree search: the LLM performs node-level relevance classification over a hierarchical index, incorporating full context — making retrieval adaptive, explainable, and traceable.

• Fast tree search at scale — reasoning-driven tree search with block partitioning and parallel processing, supporting complex, context-aware retrieval over large hierarchical structures
• KV-cache native — the first database designed around LLM KV-cache reuse. By caching intermediate results during tree search, ConDB reduces token usage by up to 70% with no loss in accuracy. The same efficiency gains extend to memory systems for long-context reasoning at scale
• Unified long-context infrastructure — a single system for both static and dynamic long-context workloads

Structured, persistent knowledge — documents (via PageIndex), file systems, and codebases. Scalable retrieval within large, organized hierarchies.

Evolving, runtime context — agent memory, long conversations (via ChatIndex), and autoresearch. Systems can continuously update, retrieve, and reason over newly generated information.

• Hierarchical storage — document trees, chat trees, and custom hierarchical JSON in SQLite
• Multiple retrieval strategies — beam search for small trees, block retrieval for large documents
• Multi-provider LLM support — Anthropic (Claude) and OpenAI (GPT) out of the box
• Extensible — plug in custom storage backends, LLM providers, or retrieval strategies

pip install -r requirements.txt

import contextdb

# Open database
db = contextdb.open("my_docs.sqlite")

# Configure LLM
db.set_llm(provider="anthropic", model="claude-sonnet-4-6")

# Store a document tree
tree_id = db.store(document_tree_json, format="document")

# Query with LLM reasoning
result = db.query(tree_id, "What are the key findings?")
print(result.contents)

from contextdb import ContextTree

def build_markdown_tree(path: str) -> dict:
    ...

ct = ContextTree("context.sqlite")

tree_id = ct.index_markdown_file("doc.md", tree_builder=build_markdown_tree)

# You can also generate a tree out of process and call:
# tree_id = ct.index_document_tree(document_tree_json)

ct.close()

Create a .env file with your API keys:

ANTHROPIC_API_KEY=sk-...
OPENAI_API_KEY=sk-...

Model and provider settings live in contextdb/config/config.yaml:

llm:
  provider: anthropic          # anthropic or openai
  model: claude-sonnet-4-6     # any model the provider supports
  context_limit: 100000
  max_concurrent: 10

retriever:
  beam_size: 3
  max_turns: 5

Override at runtime with environment variables:

LLM_MODEL=claude-opus-4-6 python your_script.py

ConDB automatically selects the best retrieval strategy based on tree size:

You can also specify a strategy explicitly:

result = db.query(tree_id, "question", strategy="block", beam_size=3)

Two benchmarks live under bench/.

Runs on AmuroEita/SWEBench-FileTree, a path-only version of SWE-bench code retrieval:

• 500 GitHub issues as queries
• 475 (repo, commit) repository snapshots as independent retrieval universes
• 58,058 file paths; no source code, no file summaries

Given an issue and one snapshot's file tree, return the file(s) the fix touches. Specification: notes/condb_swebench_filetree_bench.md.

export ANTHROPIC_API_KEY=sk-ant-...
python bench/run_swebench_filetree.py --tier medium

Tiers (by retriever difficulty; lower difficulty = more path signal in query):

easy     107 queries   gold path appears in query text (sanity check)
medium   133 queries   gold filename appears in query    (main report)
hard     261 queries   gold module stem appears          (fuzzy matching)
all      500 queries   no filter, includes ~48% path-signal-less queries

Output goes to bench/runs/<timestamp>__<tier>/: report.md, summary.json, per_query.jsonl.

Block mode can optionally rerank only the cross-block merge candidates before the file/directory split:

python bench/run_swebench_filetree.py --tier medium --strategy block --ranker vector

Available rankers are none, bm25, and vector. The vector ranker uses LiteLLM embeddings (--embedding-provider, --embedding-model) and leaves the default ranker=none unchanged.

Claude Sonnet 4.6, --ranker none, 500 queries, 0 failures. The retriever returns the file set it deems relevant — no fixed top-K cutoff. Metrics compare the returned set against the gold set:

• precision = |returned ∩ gold| / |returned|
• recall = |returned ∩ gold| / |gold|
• f1, exact_match (set equality), MRR (rank of first hit)

Block (ConDB) is compared against a Vertical baseline — a per-beam variant that expands each parent's children into separate subtree blocks (A→B, A→C), one LLM call per branch.

Block lifts recall from 0.56 to 0.90 at ~3× lower latency. Both runs return ~3 candidates per query against an avg_gold = 1.24 — explaining the low exact_match: the retrievers tend to return one extra plausible file alongside the actual gold.

Block per-gold-count breakdown:

Reproduce:

python bench/run_swebench_filetree.py --tier all --strategy block    --ranker none
python bench/run_swebench_filetree.py --tier all --strategy vertical --ranker none

Compares retriever algorithms (Block / Beam / Vertical / ...) on one hierarchical document. Reports time, LLM calls, token usage with prompt caching, and USD cost.

python bench/run_document_bench.py \
  --doc examples/large_doc.json \
  --config bench/queries.json

Queries live in the config JSON as {"queries": ["...", "..."]}. Swap in any --doc and any --config to benchmark a different document.

contextdb/
├── api/
│   ├── condb.py          # ConDB — main entry point
│   └── context_tree.py   # ContextTree — tree indexing + query API
├── core/
│   └── storage.py        # TreeDB (SQLite), StorageProtocol
├── adapter/
│   └── base.py           # DocumentTree, ChatIndex, Generic adapters
├── retriever/
│   ├── base.py           # Retriever protocols
│   └── algorithm/        # Beam, Block retrieval strategies
├── llm.py                # LLMClient (Anthropic, OpenAI)
├── config/               # YAML configs for retrievers
└── prompts/              # Jinja2 prompt templates

Custom Storage Backend

from contextdb import StorageProtocol

class MyStorage:
    def get_node(self, tree_id, node_id): ...
    def get_children(self, tree_id, node_id): ...
    # implement StorageProtocol methods

ct = ContextTree(storage=MyStorage())

Custom LLM Provider

from contextdb import LLMProtocol

class MyLLM:
    def chat(self, messages, system="", tools=None):
        return {"content": [...], "stop_reason": "..."}

ct = ContextTree("db.sqlite", llm=MyLLM())

./run_tests.sh all

• PageIndex — vectorless, reasoning-based RAG that builds hierarchical tree indexes from long documents
• ChatIndex — tree indexing for long conversations, enabling reasoning-based retrieval over chat histories
• AgentFS — filesystem for AI agents

Licensed under Apache 2.0.

© 2026 Vectify AI