Search results

Add BEAM benchmark dataset (ICLR 2026 — long-term memory evaluation)

enhancementNew feature or requestNew feature or request

Status: Open.

#12 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Mar 24, 2026

Benchmark: Add QMD comparison (local-first knowledge search baseline)

Status: Open.

#3 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 27, 2026

Benchmark: Add agent mode evaluation (multi-round retrieval via MCP)

Status: Open.

#5 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 27, 2026

Benchmark: Verify LoCoMo category assignments match source code, not paper

Status: Open.

#4 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 27, 2026

Benchmark: Track token usage per query for cost comparison

Status: Open.

#6 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 27, 2026

Benchmark: Test with multiple eval LLMs to isolate memory quality from model capability

Status: Open.

#7 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 27, 2026

Benchmark: Adopt Backboard's LoCoMo methodology for reproducible comparison

Status: Open.

#8 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 27, 2026

Benchmark: Add LLM-as-Judge evaluation (GPT-4.1) for LoCoMo

Status: Open.

#9 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 27, 2026

Investigate low content hit rate for bm-local (15.5% vs Mem0 34.3%)

Status: Open.

#2 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 26, 2026

Add MCP stdio provider for warm-connection benchmarks

Status: Open.

#1 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 26, 2026

Standalone benchmark suite for retrieval quality evaluation

enhancementNew feature or requestNew feature or request

Status: Open.

#10 In basicmachines-co/basic-memory-benchmarks;

· bm-clawd opened on Feb 25, 2026