[Alex Gorischek] Alex Gorischek Aug 24, 2026 | [ Web Proxy ] |
| Viewing: https://githubnext.com/posts/knowledge-compressor/ | [Back] [Original] |
How far can documentation be compressed without losing its meaning?
Natural language tends to contain quite a bit of repetition and redundancy.1 This isnt inherently bad; indeed, its quite useful for making language robust to noise and the general realities of communication. However, redundancy always comes with tradeoffs, and in the era of language models, redundancy costs tokens. When working with substantial knowledge content in agent sessions, repetition takes up space in context windows that might be better spent on documentation about other topics. That raises the question: Can documentation be made less repetitive? Can documentation be compressed?
To explore this, weve prototyped a system called Knowledge Compressor, which accepts articles as input and iteratively reduces their token counts while ensuring the knowledge in the source2 remains usable. The takeaway is that typical technical documentation can have its token count cut in half without substantially reducing its usefulness to language models. Heres a quick before-and-after glance at one such evaluation article, cut from 996 tokens to 480:
Ferry Rotation in the Ferryline Build Cache Ferryline is a distributed build cache that stores compiled artifacts keyed by a hash of their inputs. Each stored artifact is written to exactly three storage nodes, chosen by rendezvous hashing over the artifact key. A read succeeds as soon as any one replica responds, so the cache stays available when a node is draining or restarting. The cache is divided into two tiers. The hot tier holds artifacts requested at least twice in the preceding six hours and is backed entirely by NVMe. The cold tier holds everything else on spinning disks. Promotion between tiers happens asynchronously: a background sweeper records access counts in a ring buffer and promotes an artifact only after the count crosses the threshold twice in separate windows, which prevents a single burst of parallel builds from flooding the hot tier. Ferryline does not evict by least-recently-used order. Instead it evicts by projected rebuild cost, which is the product of an artifact's measured compilation time and its access frequency. An artifact that takes four minutes to rebuild is retained over one that takes eight seconds, even when the cheap artifact was touched more recently. Eviction runs whenever a tier exceeds eighty-five percent capacity and stops at seventy percent, and this gap exists so that eviction does not run continuously near the threshold. Writes use a protocol called ferry rotation. A client uploads an artifact to a single node, designated the ferry, which acknowledges the write immediately and then replicates to the other two nodes in the background. The ferry role rotates per artifact rather than per node, so no single machine absorbs all inbound traffic for a popular key. If the ferry fails before replication completes, the write is lost and the client rebuilds; Ferryline deliberately accepts this because a lost cache entry costs one rebuild, while synchronous replication would add latency to every write. Consistency is scoped to the key. Because keys are content hashes, two clients that produce the same artifact produce the same key, and a conflicting write is therefore byte-identical to the value already stored. Ferryline exploits this by skipping conflict resolution entirely: a write to an existing key is discarded without comparing contents. This assumption breaks for non-deterministic compilers, which is why Ferryline refuses to cache any target whose build rule declares itself non-hermetic. Garbage collection is generational. Artifacts are stamped with the build graph revision that produced them, and revisions older than the fiftieth most recent are deleted wholesale rather than traversed. The team chose the number fifty after observing that bisecting a regression rarely reaches further back, and that traversal-based collection spent more time walking the graph than it recovered in space. Membership changes are deliberately lazy. The controller publishes a numbered membership epoch whenever nodes join or leave, but artifacts already on disk are not copied immediately to match the new rendezvous-hash placement. A read first checks the current three owners and then, on a miss, checks the owners from the preceding two epochs. Finding an artifact on an old owner schedules a background copy to the current owners. Operators may drain a node only after it has remained in read-only mode for two complete membership epochs; removing it earlier can strand cold artifacts that have not been read and migrated. Integrity repair is separate from membership migration. Every artifact carries a BLAKE3 digest, and a scrubber verifies one percent of each node's resident bytes per hour. When one replica is corrupt, the scrubber copies a valid replica over it. When all available replicas disagree with the digest, the key is deleted and the next requesting client rebuilds it. Repair traffic is capped at five percent of a node's outbound bandwidth so that a damaged disk cannot displace normal cache reads. Clients also enforce admission limits before ferry rotation begins. Artifacts larger than two gigabytes are never uploaded, and each tenant may have at most two hundred pending ferry writes. At the limit, the client continues the build without caching the artifact rather than waiting. These limits protect the shared service from oversized debug bundles and from retry storms after a large worker pool reconnects. Ferryline exposes two metrics that operators are told to watch together. Hit rate alone is misleading, because a cache that stores only trivially cheap artifacts can report a high hit rate while saving no time. The second metric, saved seconds per gigabyte, divides total avoided compilation time by bytes resident, and a healthy deployment holds it above one hundred. When saved seconds per gigabyte falls while hit rate stays flat, the usual cause is a build rule that began emitting large artifacts that were cheap to produce.
Content compression is nothing new, of course. Compressing knowledge for language models is broadly similar to compressing a WAV audio file into an MP3 or a PNG image into a JPEG. However, those processes aim to maximize data reduction while minimizing human-perceptible degradation. In this case, we want to minimize LLM-perceptible degradation but what do language models perceive, exactly? We could ask them, but theres a good chance theyll just make something up. Instead, we can test them.
Roughly speaking, the purpose of putting knowledge content into language model context windows is to allow models to answer questions that arent covered by their inherent world knowledge. One way to test the fidelity of documentation compression, then, is to see whether language models can accurately answer questions using the compressed content. To this end, the system starts by extracting a set of questions and answers from the input content. It can adaptively choose the number of questions based on the contents length and density, or the number can be set manually. These extracted questions can be thought of as spiritually similar to unit tests.
Next, the system ensures that every extracted question can, in fact, be answered each in a fresh context window based on the original article. Questions that cant actually be answered are filtered out. This testing system is admittedly not perfect: Even if a question is not strictly answerable from the content, a language model might nonetheless respond using its built-in world knowledge, or even just guess. To mitigate this, we instruct the model to use only the context provided a helpful, though incomplete, technique and focus our evaluation runs on facts that cant possibly be in current models world knowledge, either because the information is newer than the training cutoff or because its entirely synthetic.3
Once the test questions are confirmed, the system begins the agentic compression process. An agent receives instructions and technical advice, along with tools for reading the original content, counting tokens, applying diffs, and running test questions. It iterates on the content until it concludes that no further compression is possible, then exits. The system can then review the run against various completion criteria and decide whether to send the draft back for further revision.
One risk with this agentic process is that the agent might play it safe and barely compress the content at all. To guard against this, the system can require at least one test question to fail before the run completes. If the compression agent hasnt shortened the source article enough to lose at least some meaningful content, it likely hasnt reached the lower bound. When a question does fail, that result is fed back to the agent, which can restore the necessary material to get the tests passing again.
Compressing documentation lets us include more information in a single context window, which is great for increasing the knowledge available in a particular session. Beyond that, though, compressing documentation also reduces the recurring cost of including the content, even when context space isnt saturated. However, the compression process comes with its own cost: the system consumes tokens generating questions, testing drafts, and iteratively producing the final shortened article. This lets us ask a concrete question after each run: How many times would the compressed content need to be used before its savings repay the compression cost?
For the evaluation run shown above, compressing the 996-token source to 480 tokens cost about $2. At current pricing for the same model,4 each uncached inclusion saves about $0.001, putting the break-even point at roughly 2,000 uses. The exact result depends on model pricing, caching, source material, and the agents decisions, of course but for knowledge that is expected to be needed repeatedly, the compression can effectively pay for itself.
When you stack it all together, it looks something like this5:
So, the basic answer seems to be yes: Documentation can be compressed. But this exploration has only scratched the surface, and there are many questions left to answer: How well do different types of source content compress? Do different genres benefit from different compression strategies? Can the compression process itself be made more token-efficient? Well just have to keep compressing docs to find out!
Well quietly skip the debate here about whether documentation contains knowledge versus merely causes it.
The model used in this evaluation run was GPT-5.6 Sol, but by the time youre reading this article that may well sound archaic.
The activity feed has been abridged for brevity; however, this is indeed the result of a real run.
| Web Proxy Viewer | New URL | Original Page |
![[Alex Gorischek]](https://github.com/agorischek.png?size=48){kind=link}