FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat(memory): close M4's security half — a planted memory can't self-promote by ndemianc · Pull Request #64 · levelcodeai/levelcode · GitHub

/ levelcode Public

feat(memory): close M4's security half — a planted memory can't self-promote #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ndemianc merged 2 commits into from
Aug 14, 2026
Conversation 4 Commits 2 Checks 1 Files changed
Merged

feat(memory): close M4's security half — a planted memory can't self-promote #64

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
13b90cf
feat(memory): close M4's security half — a planted memory can't self-…
ndemianc Aug 14, 2026
988be27
fix(memory): close the two remaining ways a secret reached disk
ndemianc Aug 14, 2026
File filter

Filter by extension

Filter by extension .js  (3) .md  (1) All 2 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
12 changes: 10 additions & 2 deletions docs/levelcode-sessions-memory.md
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,9 @@ This is the part most designs skip, and LevelCode can't (it's the security-forwa

- **Poisoning via untrusted content.** Sessions contain workspace text, which in a hostile repo is attacker-controlled. A naïve extractor could be steered into writing a false "memory" (*"the deploy token is safe to print"*). Mitigations: extraction summarizes **outcomes and user/agent actions, not arbitrary quoted content**; the digest is **bounded and reviewable**; and injected memory is **framed as untrusted, verify-first** (§4) — it can inform, never command.
- **Memory never executes.** It is context in the system block, exactly like project rules. It cannot run a tool, approve an MCP call, or edit a file. An injected instruction inside a "memory" is treated like any other untrusted text (the project's existing prompt-injection posture).
- **Provenance limits blast radius.** Because every item is sourced and dated, a poisoned entry is traceable to its session and removable in one click — and its low, *inferred* confidence keeps it from being load-bearing until a human confirms it.
- **Provenance limits blast radius.** Because every item is sourced and dated, a poisoned entry is traceable to its session and removable in one click.
- **Instruction-shaped text never self-promotes.** ⚠️ This bullet used to claim that "low, *inferred* confidence keeps it from being load-bearing until a human confirms it." **That was not what the code did.** `foldFacts` promoted anything observed in ≥ 2 distinct sessions with no human in the loop — and against a hostile repo, repetition is not corroboration: the planted file is still checked out next session, so one piece of evidence gets counted twice. Repetition still promotes ordinary facts, but text that reads as an *order* (`always …`, `never …`, `you must …`, `ignore previous instructions`, anything piping into a shell) now requires an explicit Confirm. It is still recorded and listed — surfaced, not silently dropped, so you can see what a repo tried to plant. Pinned by `test/memoryPoisoning.test.js`.
- **Secrets are scrubbed at the write boundary.** The extractor's prompt asks the model not to emit credentials, and a request is not a filter. `redactSecrets()` strips the named key shapes (GitHub, Anthropic, OpenAI, Stripe, AWS, Google, Slack, bearer tokens, PEM private keys) from fact text, session titles and refined summaries *before* they reach `facts.jsonl` / `journal.jsonl` — files the user is explicitly invited to open, grep and check into a dotfiles repo. Named prefixes only, never a "looks random" heuristic: git SHAs, content hashes and asset names are legitimate things for a fact to mention, and corrupting a true fact is a worse failure than missing an exotic token shape.
- **Local & private.** Memory never leaves the machine (BYOK promise); M9 sync, if enabled later, encrypts it like the sessions themselves.

---
Expand Down Expand Up @@ -165,7 +167,13 @@ The magic, delivered quietly (never a wall of text):

**M3 — the memory surface & control** *(M)*. The "Project memory" panel tab: view/edit/pin/delete/"not true", inferred-vs-confirmed, per-project off. Exit: a user corrects a wrong memory and the agent stops repeating it.

**M4 — polish & safety hardening** *(S)*. Conflict reconciliation UI, poisoning red-team pass, decayed-entry recall, export. Exit: an adversarial repo cannot plant a load-bearing memory; EXIT-TEST.md green.
**M4 — polish & safety hardening** *(S)*.

- ✅ **Conflict reconciliation** — semantic supersede: a newer session's fact marks an older one obsolete, dimmed and restorable rather than silently replaced.
- ✅ **Poisoning red-team pass.** `test/memoryPoisoning.test.js` — 34 cases, an adversarial corpus in the style of `commandSafety.test.js`: ten hostile shapes that must never self-promote, benign project facts that must keep working, nine credential shapes that must never reach disk, and the near-misses (git SHAs, content hashes, asset names) that must survive untouched. It found the gap it was written to look for — see §7. Every case verified non-vacuous by bypassing each guard and confirming failure.
*Exit met: an adversarial repo cannot plant a load-bearing memory.* The original wording said "EXIT-TEST.md green", but that file is the **M0** fork/build checklist and was never the right home for this; an executable corpus is a better exit test than a checklist anyway, since it re-runs on every change.
- ⬜ **Decayed-entry recall** — surfacing an aged-out fact when a query matches it directly.
- ⬜ **Export** — "Copy as Markdown" for a session, and for the memory set. Cheap, since the storage is already plain text, and it seeds LevelLinks.

**Deliberately later:** cross-*project* memory ("how did I do idempotency in the *other* service?"); a vector cache over the plain files for large corpora; team-shared project memory (rides M9 sync).

Expand Down
117 changes: 112 additions & 5 deletions extensions/levelcode-ai/sessionMemory.js
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,15 @@ function memoryMdFile(root, slug) { return path.join(memoryDir(root, slug), 'MEM
*/
function outcomeEntry(derived, t) {
const d = derived || {};
const files = Array.isArray(d.filesEdited) ? d.filesEdited.slice(0, 6) : [];
const title = d.title != null ? String(d.title) : null;
// Paths get redacted too. They are not free text, but they are not safe either: digestMarkdown
// prints them straight into MEMORY.md ("- did X (a.js, b.js)"), and a path is attacker-influenced
// in a hostile repo and user-influenced everywhere else — a downloaded `key-ghp_….txt`, an `.env`
// backup named after the token it holds. Cheap, and no legitimate path carries a credential prefix.
const files = (Array.isArray(d.filesEdited) ? d.filesEdited.slice(0, 6) : []).map((f) => redactSecrets(String(f)));
// The title is derived from the session's opening message, so a user who pasted a token into
// chat to ask about it would otherwise have it copied into journal.jsonl and MEMORY.md — files
// that outlive the session and are meant to be greppable and checkinable.
Comment on lines 41 to +49
const title = d.title != null ? redactSecrets(String(d.title)) : null;
return {
v: SCHEMA_V,
id: d.id != null ? String(d.id) : null, // source_session — provenance
Expand Down Expand Up @@ -107,16 +114,97 @@ function latestBySession(entries) {
function normalizeFactKey(text) {
return String(text || '').toLowerCase().replace(/[^a-z0-9 ]+/g, ' ').replace(/\s+/g, ' ').trim();
}
// ---- Hardening: memory is an attack surface (design §7) ---------------------------------------
//
// Everything a session records passes through here on its way to disk. The two guards below are
// DETERMINISTIC on purpose. The extractor's system prompt already asks the model never to emit
// secrets or instructions, and that instruction is worth keeping — but a request is not a filter,
// and the transcript it summarizes contains repo file contents, command output and MCP tool
// results, all of which are attacker-controlled for any repo you clone.

/** Credential shapes worth refusing outright. Named prefixes only — see redactSecrets. */
const SECRET_PATTERNS = [
/-----BEGIN[A-Z ]*PRIVATE KEY-----[\s\S]*?-----END[A-Z ]*PRIVATE KEY-----/g,
/-----BEGIN[A-Z ]*PRIVATE KEY-----/g, // a truncated block still names a key
/\bsk-ant-[A-Za-z0-9_-]{20,}/g, // Anthropic
/\bsk-[A-Za-z0-9]{32,}/g, // OpenAI-shaped
/\bsk_(?:live|test)_[A-Za-z0-9]{16,}/g, // Stripe
/\bgh[pousr]_[A-Za-z0-9]{20,}/g, // GitHub PAT / OAuth / server / refresh
/\bgithub_pat_[A-Za-z0-9_]{20,}/g,
/\bAKIA[0-9A-Z]{16}\b/g, // AWS access key id
/\bAIza[0-9A-Za-z_-]{30,}/g, // Google API key (39 chars today; unanchored length, since
// pinning it exactly means a format tweak slips straight through)
/\bxox[baprs]-[A-Za-z0-9-]{10,}/g, // Slack
/\bBearer\s+[A-Za-z0-9._~+/=-]{20,}/gi // a bearer token pasted from a curl
];

/**
* Replace credential-shaped substrings with a marker, before the text is written anywhere.
*
* Deliberately NAMED shapes rather than a "long random-looking string" heuristic. The generic
* version flags git SHAs, content hashes, base64 fixtures and long identifiers — all legitimate
* things for a project fact to mention — and a memory system that quietly corrupts true facts is
* a worse failure than one that misses an exotic token shape. These prefixes cover what actually
* leaks in practice.
*
* The marker is left IN PLACE rather than dropping the whole line, so the surrounding fact stays
* readable and the user can see that something was scrubbed instead of wondering why a sentence
* ends abruptly.
*/
function redactSecrets(text) {
let s = String(text == null ? '' : text);
for (const re of SECRET_PATTERNS) { s = s.replace(re, '[redacted]'); }
return s;
}

/**
* Does this read as an INSTRUCTION rather than a fact?
*
* A project fact is a stable truth — "the changelog is RELEASE-NOTES.md", "idempotency keys live
* in Redis". An instruction is a command that will be replayed into the system prompt of every
* future session in this project, which is the exact shape of a persistent prompt injection:
* poison once, influence every run.
*
* This does not delete anything. It only withholds AUTOMATIC promotion — see foldFacts. The fact
* is still recorded, still listed, and one Confirm click still activates it. That asymmetry is the
* whole design: a false positive costs the user one click, a false negative is an attacker-authored
* line injected into every session indefinitely.
*
* So yes, "Never commit .env files" — a real and useful convention — needs confirming. That is the
* right trade at this price.
*/
const INSTRUCTION_PATTERNS = [
// Imperative openers. Anchored: "the team should never…" is a description, "Never…" is an order.
/^\s*(always|never|do not|don't|dont|ignore|disregard|forget|instead of|make sure|be sure|remember to|ensure that|you must|you should|you are|from now on)\b/i,
// Injection boilerplate, wherever it appears.
/\b(ignore (all )?(previous|prior|earlier) (instructions|prompts|rules)|system prompt|new instructions|override .{0,20}(instructions|rules))\b/i,
// Piping anything into a shell is never a "fact".
/\|\s*(sudo\s+)?(sh|bash|zsh|python3?)\b/i,
/\b(curl|wget)\b[^\n]{0,80}\|/i
];
function looksLikeInstruction(text) {
const s = String(text == null ? '' : text).trim();
if (!s) { return false; }
return INSTRUCTION_PATTERNS.some((re) => re.test(s));
}

/** One observation of a candidate fact (append-only), sourced + dated — the raw material foldFacts counts. */
function factObservation(text, sourceId, t) {
return { v: SCHEMA_V, text: String(text == null ? '' : text).trim(), source: sourceId != null ? String(sourceId) : null, at: t || null };
// Redact HERE, at the boundary, not at read time: facts.jsonl is a plain file the user can open,
// grep, and check into a dotfiles repo. A secret scrubbed only on the way out would still be
// sitting on disk.
return { v: SCHEMA_V, text: redactSecrets(String(text == null ? '' : text).trim()), source: sourceId != null ? String(sourceId) : null, at: t || null };
}
Comment on lines +193 to 197
/** A control event on a fact, by normalized key: confirm, remove (not-true), or supersede (a newer fact made
* it obsolete — carries `by`, the replacing text, as the one-line history). */
function factControl(key, action, t, by) {
const control = action === 'remove' ? 'remove' : action === 'supersede' ? 'supersede' : 'confirm';
const e = { v: SCHEMA_V, key: String(key || ''), control, at: t || null };
if (control === 'supersede' && by) { e.by = String(by); } // the fact that replaced it — the one-line history
// `by` is a SECOND copy of the replacing fact's text, taken straight from the model's output
// (extension.js: `r.facts[0] || r.summary`) rather than from the observation that factObservation
// already scrubbed. It persists to facts.jsonl and surfaces as `supersededBy` in the panel, so
// without this it was a way around the boundary — same text, different door.
if (control === 'supersede' && by) { e.by = redactSecrets(String(by)); } // the fact that replaced it — the one-line history
return e;
}
/** Append fact observations and/or control events (JSONL). Creates memory/ on first write. */
Expand Down Expand Up @@ -171,7 +259,25 @@ function foldFacts(entries, opts) {
if (g.removed || !g.text) { continue; }
const count = g.sources.size;
const superseded = !!g.superseded && !g.confirmed;
out.push({ key: g.key, text: g.text, count, confirmed: g.confirmed, superseded, supersededBy: superseded ? g.supersededBy : '', inferred: !g.confirmed, active: g.confirmed || (!superseded && count >= minSeen), at: g.at });
// Instruction-shaped text never rides the repetition path — only an explicit Confirm.
//
// Repetition is the weaker of the two promotion routes, and against a hostile repo it is not
// evidence at all: the poisoned file is still checked out on the next session, so the
// extractor reads the same line again and "seen in 2 distinct sessions" counts one planted
// string twice. That is fine for a genuine observation, which is why the rule stays for
// ordinary facts — but it means repetition cannot be what promotes an order into the system
// prompt of every future run.
const instruction = looksLikeInstruction(g.text);
out.push({
key: g.key, text: g.text, count, confirmed: g.confirmed, superseded,
supersededBy: superseded ? g.supersededBy : '',
inferred: !g.confirmed,
// Surfaced, not hidden: the panel can show WHY this one is sitting inactive, the same way
// a superseded fact is dimmed rather than dropped.
instruction,
active: g.confirmed || (!superseded && !instruction && count >= minSeen),
at: g.at
});
}
out.sort((a, b) => (Number(b.confirmed) - Number(a.confirmed)) || (Number(a.superseded) - Number(b.superseded)) || (b.count - a.count) || String(b.at || '').localeCompare(String(a.at || '')));
return out;
Expand Down Expand Up @@ -308,5 +414,6 @@ module.exports = {
memoryDir, journalFile, factsFile, memoryMdFile,
outcomeEntry, appendJournal, readJournal, latestBySession, writeMemoryMd,
normalizeFactKey, factObservation, factControl, appendFacts, readFacts, foldFacts, activeFacts,
redactSecrets, looksLikeInstruction,
queryTerms, snippetFor, recallRank, buildDigest, digestSummary, digestMarkdown
};
4 changes: 3 additions & 1 deletion extensions/levelcode-ai/sessions.js
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,9 @@ function createSessions(opts) {
try {
const latest = memory.latestBySession(memory.readJournal(root, slug)).find((e) => e.id === id);
if (!latest) { return false; }
memory.appendJournal(root, slug, Object.assign({}, latest, { summary: String(summary).trim(), refined: true }));
// Model output summarizing a transcript that contained repo files, command output and MCP
// results — redact before it lands in journal.jsonl and, from there, MEMORY.md.
memory.appendJournal(root, slug, Object.assign({}, latest, { summary: memory.redactSecrets(String(summary).trim()), refined: true }));
consolidate();
return true;
} catch (e) { return false; }
Expand Down
Loading
Back | FazBrowse Home | New Git URL