ColBERT embedding and MaxSim scoring without PyTorch. Uses a native C++ extension (ONNX Runtime + tokenizers-cpp) so you don't need to pull in 2 GB of deep learning dependencies just to encode some text.

pip install limbed-py

Only runtime deps are numpy and huggingface-hub.

from limbed import LateEmbedder, compute_maxsim

model = LateEmbedder()  # downloads thlurte/mxbai-edge-colbert-v0-17m-onnx

q = model.encode_queries("What is late interaction?")
d = model.encode_docs("ColBERT computes token-level similarity.")

score = compute_maxsim(q[0], d[0])
print(score)

You can also point it at a local directory with model.onnx and tokenizer.json:

model = LateEmbedder("./my-model/")

For RAG, encode the full document (in model-sized windows when needed), then slice retrieval chunks from the contextualized token embeddings. Requires pip install tokenizers.

from limbed import LateEmbedder, compute_maxsim_late_chunked

model = LateEmbedder("jina-colbert-v2")  # 8192 context; BERT models use 512

doc = "..."  # full document
chunks = ["First paragraph.", "Second paragraph."]  # substrings of doc

chunk_embs = model.encode_doc_late_chunked(doc, chunks)
q = model.encode_queries("your query")
score = compute_maxsim_late_chunked(q[0], chunk_embs)  # max over chunks

Documents longer than the model limit are encoded in overlapping windows (window_overlap=64 by default). Chunks entirely inside one window retain full within-window context.

Any ColBERT ONNX model should work if you put model.onnx and tokenizer.json in a folder and pass the path.

The following benchmark was run on CPU using 20 queries (max length 32) and 20 documents (max length 256), comparing limbed against fastembed execution:

• Tokenization and inference run in C++ via a nanobind extension
• GIL is released during encode and MaxSim calls, so you can run multiple threads
• Punctuation tokens are masked out of document embeddings (standard ColBERT behavior)
• Embeddings are L2-normalized by default
• CPU only for now

Because the underlying precompiled ONNX Runtime library is linked against glibc, this package will not run out-of-the-box on Alpine Linux images (e.g., python:3.10-alpine).

If deploying via Docker, it is highly recommended to use a Debian-based slim image:

FROM python:3.10-slim

If you must use Alpine, you will need to install the compatibility layer: apk add --no-cache gcompat.

Precompiled wheels are published to PyPI for the following environments:

MIT. See LICENSE.