Thanks — acknowledged here too, and I have replied in detail on #1199 confirming the underlying bug on current main.

Short version of that confirmation: detect_file_language() returns at src/discover/discover.c:653-655 before any content inspection, and every content-based path in that function keys off an extension, so an extensionless file can never reach one. Your diagnosis was exactly right.

Queued for review. The branch is CONFLICTING against main — a rebase before I read it would be ideal.

Thank you — the implementation here is careful, and I want to say what I checked before raising anything: the probe is bounded and read-only, it uses cbm_fopen correctly, it fails closed on error, NUL bytes and truncation, the interpreter list is a fixed allowlist, nothing is executed, and CRLF is handled. The env/-S/assignment/option rejections are conservative in the right direction — false negatives rather than false positives. The 18 tests include negative and vacuity-guarded cases and they bind.

Two things before it can land.

1. It needs a rebase — it currently conflicts with main.

2. One behavioural question I would like your view on. The description says extension detection stays authoritative, but the fallback fires whenever cbm_language_for_filename() returns unknown — which includes files with an unknown extension, not only extensionless names. So data.conf beginning with a bash shebang would now index as Bash. That may well be what you intended, but it is broader than the stated contract.

There is a cost attached to that breadth: in FULL mode every unknown file gets an open and a read. ALWAYS_IGNORED_SUFFIXES keeps binaries out, but README, LICENSE, .txt, .zip and lockfiles would all be probed, and this project routinely indexes repositories in the millions-of-files range where that is measurable.

Either resolution is fine by me: gate the probe to genuinely extensionless basenames, or keep the breadth and say explicitly in the description why unknown extensions should participate and why the discovery I/O is acceptable at scale. I would just rather that be a decision than a side effect.

Worth noting for transparency: #1199 was filed by you a day before this PR, so the direction has not been through maintainer triage — but the fit is good. Finding code the graph currently misses is exactly the kind of coverage gap we want closed.

This is the best-engineered contribution I have reviewed in this backlog sweep, and I want to be specific about why rather than just saying so. It needs one rebase and then I would like to merge it.

I measured the cost concern empirically rather than estimating it, because "open and read every candidate file" is exactly the kind of change that quietly costs real time at scale. On the actual 94,851-file Linux kernel checkout:

• ~5,066 files get probed — 5.3% of the repo. Everything else is filtered first: Makefile (×3,193) and Kconfig (×1,828) are filename-mapped and never touched, and gitignore, cbmignore, skip-lists and the size cap all run before the probe.
• ~15,200 extra syscalls, three per candidate.
• Well under half a second warm-cache against a ~260 s index — under 0.2%. Even the pathological case (every file unknown) lands at a second or two.

And the over-inclusion question came out equally clean: 301 candidates have a shebang, ~294 newly indexed — 196 sh, 67 bash, 18 python3, 13 perl, all genuine scripts in scripts/ and tools/ that were previously invisible to the graph. No junk. LICENSE and README have no shebang so they are never touched; vendor/vendored are always-skip so blobs are never probed; compiled artefacts do not start with #!. That is +0.31% files, all of them real source.

The fail-closed discipline throughout is exemplary. Every ambiguous input declines rather than guessing:

• a bounded 255-byte read plus a one-byte EOF probe to distinguish "short file" from "truncated first line" — a genuinely clever detail;
• embedded NUL before the newline → treated as binary, declined;
• env NAME=value prog and option tokens rejected, since env would set a variable rather than run that token;
• strict per-component digit matching on Python version suffixes, so python3. and python3..12 are rejected and python-wrapper does not match by prefix;
• UTF-8 BOM before #! declines — which is correct, because the kernel refuses BOM'd shebangs too. Matching exec semantics rather than being liberal is the right instinct.

You used cbm_fopen throughout — including in the test helper — without being told. That rule exists because raw fopen breaks non-ASCII paths on Windows, and it is one people usually have to be reminded of. "rb" is also the right mode, with \r handled explicitly.

The tests are what this repo actually asks for. Eighteen of them, running end-to-end through the production cbm_discover walk rather than the unit function, red without the fix. Ten negative cases assert the file stays unindexed. Precedence is pinned both ways (weird.py with a bash shebang stays Python; Makefile with a python shebang stays Makefile). And the shebang_probe helper asserts setup success separately so a negative expectation cannot pass vacuously — that is precisely the discipline we care about, and most contributions miss it.

What is needed: a rebase. The branch conflicts with main, but the conflict is mechanical — one call-site line plus a self-contained function, and the early-return branch it hooks into is unchanged. The test_discover.c suite-list churn is mechanical too. Your PR body's "20/20 checks passed" predates the conflict, so CI needs re-establishing on the rebased head.

One thing to look at after rebasing, which is new since you branched: main gained cbm_discover_count_bounded, a deadline-bounded count path that also calls detect_file_language — so the probe cost now lands inside the coverage-count budget too. Counts staying consistent with real discovery is correct behaviour, and at these numbers the impact should be small, but it is worth one check on a very large repo.

One optional follow-up, not for this PR: the interpreter table is conservative and misses several languages we already support — awk/gawk (four real Linux files decline today), Rscript, tclsh, fish, julia, elixir, groovy, pwsh, deno/bun. Under-inclusion only, so it is safe as a v1, and a natural second PR.

Rebase and ping me; I will re-diff and merge.

Rebased onto current main (d698db8e) and force-updated the branch as requested. I also added a focused regression for the new cbm_discover_count_bounded path: full discovery and bounded counting now prove the same extensionless-shebang classification at the exact limit and one past it.

Local verification on the pushed tree 5fe1f4f8:

• clang-format-20 --dry-run --Werror ...: pass
• ASan/UBSan focused discover suite: 112/112 passed
• git diff --check: pass

GitHub matrix is running on the rebased commit. Ready for re-review.

Follow-up: the rebased head 5fe1f4f8 remains cleanly mergeable and all 28 GitHub checks are green. Is anything else needed from me before merge?

@ibaldr89, nothing further is needed from you before maintainer re-review. I rechecked head 5fe1f4f8: the branch is cleanly mergeable and all 28 GitHub checks are green, including the platform matrix, DCO, CodeQL, and ci-ok.

The next action is ours. Unless main moves in a way that makes the branch conflict again, please do not spend more time rebasing or adding validation. Thank you for following through on the requested rebase and the bounded-count regression, and sorry you had to ask again.

Merged — thank you! The fail-closed probe design (bounded 256 bytes, NUL/truncation refusal, cbm_fopen per house rule, extension detection staying authoritative) plus eighteen tests including the adversarial negatives made this easy to trust. Findability-first is exactly this project's bar and shebang-only scripts were a real blind spot. #1199 closed.