[v1.3.2] 2026-08-24

Bug Fix

• Fix WikiText dataset loading in clean environments by using the canonical
  Salesforce/wikitext dataset ID for perplexity evaluation and LoRA SFT examples.
• Fix warning regarding the data type arguments of Transformers.
• Fixed GemLite AssertionError during CPU inference execution for OneComp with Llama.cpp.
• Fixed QuantizedModelLoader.load_quantized_model() calling unfuse_moe_experts() twice on non-fused MoE checkpoints. An earlier unconditional unfuse (run right after building the empty model) was left in place when the checkpoint-aware unfuse (added for gpt-oss fused-MoE support) was introduced. The redundant unconditional call was removed, and the checkpoint-aware unfuse now runs before _remap_state_dict_keys() so key remapping still aligns against the unfused per-expert module paths. This also prevents wrongly unfusing gpt-oss fused-MoE checkpoints, whose fused 3D expert tensors must be loaded as-is (quantized_model_loader.py).

Documentation

• Add troubleshooting information for running OneComp with Llama.cpp on macOS.

[v1.3.1] 2026-08-06

Bug Fix

• Fix uv add onecomp / uv sync failing on macOS. gemlite==0.6.0 declares triton>=3.6.0 without a platform marker (e.g. sys_platform == 'linux'), and triton only ships Linux wheels, so the dependency is unresolvable on macOS. Pin gemlite<0.6.0 on sys_platform == 'darwin' only; Linux/CUDA installs are unaffected and still resolve the latest gemlite.

[v1.3.0] 2026-08-06

Highlights

• New model support: Qwen3.6 (dense + Qwen3.6-A3B MoE) and GPT-OSS (gpt-oss-20b / gpt-oss-120b) for vLLM.
• CPU deployment: export OneComp GPTQ checkpoints to GGUF and run them on CPU with Llama.cpp.
• New quantizer: MDBF (Multi-Envelope Double Binary Factorization).
• Reload → post-process → re-save workflow for saved quantized checkpoints.
• Rotation-preprocessed models (GPTQ & DBF) now run in vLLM.
• HF/vLLM-compatible LoRA adapter sidecar save/load.
• vLLM ROCm (AMD) support.

Breaking Changes

• Custom post-process subclasses must implement _run() instead of overriding run(). PostQuantizationProcess.run() is now a base-class template method (validation, CPU move, eval()/cpu() restore, audit metadata). All built-in post-processes were migrated; callers that only use built-in post-processes (e.g. Runner.run_post_processes()) are unaffected.
• GPTQ and DBF now bitpack on quantize by default (bitpack_on_quantize=True). Quantized weights are stored packed immediately after each layer is quantized instead of being held unpacked until save. GPTQ packing is limited to bit widths {2, 3, 4, 8} (GPTQ bit-width validation is now 1..15); other widths require bitpack_on_quantize=False. DBF has no bit-width restriction.

New Features

Model support: Qwen3.6

• Calibration, save/load, and vLLM inference for the dense Qwen3.6 text models and the Qwen3.6-A3B MoE variant, including hybrid (GatedDeltaNet linear_attention + full_attention) decoders.
• Added a save_format option ("auto" / "native" / "full_wrapper") to Runner.save_quantized_model(); "full_wrapper" produces the composite model.language_model.* layout vLLM's VLM loading expects. load_quantized_model() loads these checkpoints (including MoE per-expert weights) and fails fast on incompletely-loaded buffers instead of silently producing a garbage model.
• New example: example/vllm_inference/example_gptq_vllm_qwen36_inference.py.

Model support: GPT-OSS 4-bit MoE for vLLM (mixed_gptq)

• End-to-end 4-bit serving of GPT-OSS MoE experts via the mixed_gptq plugin. Pass Runner(..., moe_quant_experts=True) to keep router experts as per-expert GPTQ INT4 (compressed on disk, served 4-bit). GPT-OSS requires group_size=64 and is routed through vLLM's WNA16 path; asymmetric (sym=False) quantization is supported.
• Apply the required vLLM runtime patches before serving: python -m vllm_plugins.patches.apply_all.
• See the GPT-OSS guide (docs/user-guide/gptoss.md).

CPU inference & GGUF export (Llama.cpp)

• New onecomp.cpu package and onecomp-gguf CLI (export / run / inspect / ppl / bench) to export OneComp GPTQ checkpoints to GGUF and run them on CPU with llama-cpp-python.
• Lossless GPTQ → GGUF packing (Q4_0 / Q4_1 / Q8_0) reusing the exact GPTQ/QEP integer codes with no re-quantization, plus a dequantize → llama-quantize fallback for unsupported layouts.
• Mixed-precision exporter (llamacpp_plugins/gptq): packs 4/8-bit layers losslessly and K-quantizes 2/3-bit (and act-order) layers into a single genuinely mixed-precision GGUF.
• Adds the gguf and llamacpp uv extras.

MDBF quantizer

• New MDBF quantizer (onecomp/quantizer/mdbf/) approximating weights as a sum of multi-path double binary factorizations. Configurable target_bits, l (default 2), P (default 1), SVD init, optional ADMM and gradient refinement, activation-aware mode, and per-layer / per-MLP bit overrides.
• (l, P) = (1, 1) reproduces DBF and (1, 2) reproduces LittleBit. See docs/algorithms/mdbf.md for the BPW / rank trade-off.
• Full save/load support and an optional GemLite 1-bit inference path (auto-enabled for l == 1; force with use_gemlite=True).

Reload → post-process → re-save

• A saved quantized checkpoint can be reloaded, refined with additional post-processes (e.g. BlockWisePTQ, GlobalPTQ), and saved again. Load to CPU with load_quantized_model(..., device_map=None), drive post-processes via Runner(quantizer=None) with runner.quantized_model = model, then save_quantized_model().
• An accumulating onecomp_post_processes audit trail in config.json records which post-processes (and their hyper-parameters) were applied across load/save cycles.
• quantization_config schema validation (quant_method, modules_in_block_to_quantize) is now enforced consistently on the load, save, and post-process paths.
• New examples under example/post_process/: example_reload_post_process_resave.py, example_blockwise_global_ptq.py, example_blockwise_global_ptq_staged.py.

Rotation-preprocessed inference in vLLM (GPTQ & DBF)

• Rotation-preprocessed GPTQ and DBF models now run in vLLM: the online Hadamard transform on mlp.down_proj inputs is reproduced at inference time. GPTQ supports tensor parallelism; the DBF plugin requires tensor_parallel_size=1. Rotated GPTQ models are saved as mixed_gptq so vLLM loads the Hadamard-capable plugin.
• The DBF plugin automatically falls back to the naive kernel if the GemLite path fails (OOM is re-raised, not masked). See the TRITON_CACHE_AUTOTUNING=0 note in docs/user-guide/vllm-inference.md.
• New example: example/vllm_inference/example_dbf_vllm_inference.py.

LoRA adapter sidecar save/load

• save_quantized_model() writes GPTQ + LoRA SFT outputs in HF/vLLM-compatible form: the base model as safetensors plus a PEFT adapter sidecar in lora_adapter/ (adapter_model.safetensors, adapter_config.json). load_quantized_model() auto-detects the sidecar and re-wraps matching layers for round-trips, failing fast if not all adapter layers are applied.
• New / updated examples: example_lora_gptq_vllm_inference.py, example_lora_sft.py, example_lora_sft_knowledge.py, example_lora_sft_knowledge_jointq.py.

Environment

• Added envs/vllm/ with versioned vLLM environment definitions (0.12.0, 0.15.1), plus envs/vllm/v0_24_0_rocm/ for the ROCm plugin. Both the versioned environments and the ROCm plugin must be installed separately from the main uv sync --extra vllm workflow.

Bug Fixes

• Text-only generate() on multimodal models (e.g. Gemma-4 12B) could emit modality delimiter tokens and degrade output after quantize→reload, because load_quantized_model() did not restore generation_config.json (including suppress_tokens). Now loads it from the save directory when present (quantized_model_loader.py).
• Fixed Hadamard online-hook registration for rotation + partially-quantized models: hook target layer types are now derived from the actual model instead of the recorded quant_method, so mixed quantized / unquantized down_proj layers no longer miss quantized layers or fire on plain nn.Linear.

New Contributors

• @k-arima-3150 and @S4Y-K made their first contribution in #21
• @cm-ysmz made their first contribution in #25
• @fujisawa-yoshihiko made their first contribution in #28
• @KoOhira-BP made their first contribution in #30
• @dogwood-flo made their first contribution in #34

[v1.2.2] 2026-07-23

• Add SECURITY.md
• Bug-fix: Fixed device="auto" evaluation crashes in Runner.calculate_perplexity() and related paths by adding ModelConfig.get_device() and using a resolved torch.device for PyTorch operations such as model.to() and empty_cache(). Hugging Face device_map="auto" remains unchanged for model loading, but PyTorch no longer receives the raw "auto" string.

[v1.2.1] 2026-07-03

Security

• Unsafe deserialization hardening (CWE-502): QuantizedModelLoader.load_quantized_model_pt() (alias onecomp.load_quantized_model_pt()) previously called torch.load(model.pt, weights_only=False) unconditionally, allowing arbitrary code execution when loading a malicious .pt checkpoint. It now refuses to load unless the caller explicitly opts in via allow_unsafe_deserialization=True, and emits a strong warning when it does load. For untrusted models, use the safetensors-based load_quantized_model(), which does not execute code.

  • Breaking change: existing callers of load_quantized_model_pt() must pass allow_unsafe_deserialization=True for trusted .pt files.

• Quantizer.load_results() / ResultLoader: same hardening applied. Loading with weights_only=False now requires allow_unsafe_deserialization=True (added as a ResultLoader field), and logs a warning. The safe weights_only=True path is unchanged.

• Updated docstrings, docs, and the LoRA SFT example to document the risk and the required opt-in.

• Credit: this unsafe deserialization issue (CWE-502) was responsibly disclosed by Nir Yehoshua, Cipher Security Labs. Thank you for the report.

[v1.2.0] 2026-06-08

Save/Load Support for JointQ, RTN, and OneBit Quantizers

• JointQ: Added get_quant_config(), finalize_quant_config_for_save(), and create_inference_layer() to JointQ class (onecomp/quantizer/jointq/_jointq.py)
  • Emits quant_method="gptq" to reuse GPTQLinear and vLLM GPTQ plugin (JointQ uses the same scale/zero/assignment structure as GPTQ)
  • create_inference_layer() converts JointQ's 3D assignment (out_features, num_groups, group_size) to 2D qweight (out_features, in_features) matching GPTQ format, with scale/zero transposition
  • Handles actorder permutation: restores original column order before passing to GPTQLinear so g_idx is constructed correctly
  • Symmetric quantization: shifts signed integers [-2^(n-1), 2^(n-1)-1] to unsigned [0, 2^n - 1] for GPTQLinear bit packing
  • Added bits == 1 warning in validate_params(): GPTQLinear weight packing does not support 1-bit; inference layer must be built with pack_weights=False
  • Added _build_quantization_bits() static method to emit per-layer quantization_bits metadata for mixed-precision save
• RTN: Added get_quant_config(), finalize_quant_config_for_save(), create_inference_layer(), and RTNResult.compute_dequantized_weight() to RTN class (onecomp/quantizer/rtn/_rtn.py)
  • Emits quant_method="gptq" to reuse GPTQLinear and vLLM GPTQ plugin (RTN uses the same qweight/scales/qzeros tensor format)
  • compute_dequantized_weight() implements W = (quantized_weight - zero) * scale with per-channel and group-wise paths
  • create_inference_layer() transposes scale/zero from (out_features, num_groups) to (num_groups, out_features) for GPTQLinear compatibility
  • Added _build_quantization_bits() static method for per-layer metadata
• OneBit: Added get_quant_config(), finalize_quant_config_for_save(), create_inference_layer(), and OnebitResult.compute_dequantized_weight() to Onebit class (onecomp/quantizer/onebit/_onebit.py)
  • Emits quant_method="onebit" with OneBit-specific parameters (iters, use_importance_scaling, use_balancing, balance_iters, balance_alpha)
  • compute_dequantized_weight() implements W ≈ a[:, None] * sign * b[None, :]
  • create_inference_layer() builds OneBitLinear via OneBitLinear.from_quantization_result()
  • Added _build_quantization_bits() static method for per-layer metadata

Apple Silicon / macOS support

• MPS quantization: GPTQ (and AutoBit with GPTQ-only candidates) on device="mps"; cross-platform empty_cache() via new onecomp/utils/device.py (runner.py, quantizer/gptq/_gptq.py, quantizer/_quantizer.py)
• MPS device placement (GPTQ on CPU, QEP correction on MPS): With device="mps", run_gptq moves the Hessian and weights to CPU for the full column-wise GPTQ loop (including inverse-Hessian Cholesky). The main reason is not absent Cholesky kernels on MPS (recent PyTorch supports them); if the GPTQ loop stayed on MPS, maxq.item() inside quantize() would run once per column—each call waits for pending MPS work to finish and read back a single scalar to the host (per-column host sync), not a full matrix copy per column—and that overhead is often several times slower than CPU on Apple Silicon (~4× in internal benchmarks with PyTorch 2.12). When QEP weight correction runs (adjust_weight, typically under qep=True), per-layer work stays on MPS (e.g. weight @ delta_hatX); only the Cholesky solve uses CPU via _safe_cholesky_and_solve (one solve per layer). A full CPU fallback for QEP does not materially improve speed. Calibration forwards may still use MPS. Details: README (macOS / MPS).
• MPS inference: load saved quantized models on Mac with QuantizedModelLoader + Transformers generate() (GemLite/vLLM remain Linux + CUDA)
• macOS uv sync: added darwin to tool.uv.environments, --extra mps for MPS-enabled PyTorch from PyPI; --extra cpu is Linux-only (pytorch-cpu index); Linux-only markers on CUDA extras (cu118–cu130)

New Feature : Dashboard

• Added dashboard/, a browser-based web app for OneCompression on SLURM-managed HPC GPU nodes without Docker: pick a Hugging Face model and quantization settings in the UI, run jobs on the GPU, deploy the quantized checkpoint, and validate inference via chat
• Stack: React + Vite frontend (local PC), FastAPI API, Celery worker + user-built Redis, SQLite job DB, per-job output under backend/tmp/quantized/; CUDA quantization via onecomp and chat deploy via a separate vLLM subprocess from the same backend/.venv (onecomp + vllm>=0.21 in pyproject.toml)
• Quantization methods exposed in the UI: gptq, autobit, jointq, and auto_run (VRAM-based bitwidth / group size); optional QEP (not with JointQ); fractional bit widths for autobit / auto_run

New Feature: Global PTQ (Post-Training Quantization)

• Added GlobalPTQ and GlobalPTQDistributed post-process classes for KL-distillation-based global optimisation of continuous quantization parameters (scales and zeros for GPTQ; scaling factors for DBF)
• GlobalPTQ: Single-GPU implementation with cosine-warmup LR scheduling, early stopping, mixed-precision support, and gradient accumulation
• GlobalPTQDistributed: Multi-GPU implementation using HuggingFace Trainer + DeepSpeed ZeRO-2, supporting KL divergence and/or NTP loss with automatic best-state rollback

Evaluation:

• Added onecomp.eval and the onecomp-eval CLI: one vLLM server, subprocess evaluators, aggregated summary.json / summary.csv
• Added mt_bench (Japanese MT-Bench) and opt-in throughput (TTFT / decode tok/s) evaluators

for Developer: pre-commit

• Added .pre-commit-config.yaml with black, isort, and local hooks (no-japanese, copyright-header, no-email-address); install with uv sync --extra dev then pre-commit install (see README)

OneBitLinear Inference Layer Improvements

• Added OneBitLinear.from_quantization_result() class method: builds OneBitLinear from OnebitResult (mirrors the pattern used by GPTQLinear and DoubleBinaryLinear) (onecomp/quantizer/onebit/onebit_layer.py)
• Added OneBitLinear.from_saved_state() class method: reconstructs OneBitLinear from saved state_dict tensors (a, b, sign_packed, optional bias), using the same cls.__new__ pattern as DoubleBinaryLinear (onecomp/quantizer/onebit/onebit_layer.py)
• Removed preunpack parameter from OneBitLinear.__init__() and replace_linear_with_onebit_layer(): sign matrix is now always stored as packed uint8 and unpacked on demand during forward(), matching the DBF inference layer pattern (onecomp/quantizer/onebit/onebit_layer.py)
• Normalized buffers to FP16 with detach() in OneBitLinear.__init__() to drop autograd graph
• Added _load_from_state_dict() override to clear sign_matrix cache when loading from checkpoint
• Extracted _unpack_sign_matrix() helper for sign matrix unpacking logic
• Removed unreferenced functions replace_linear_with_onebit_layer() and extract_onebit_weights_for_save() from onebit_layer.py: layer construction is now handled by OneBitLinear.from_quantization_result() / OneBitLinear.from_saved_state(), and save-time weight extraction is covered by the unified create_inference_layer() / state_dict() path (onecomp/quantizer/onebit/onebit_layer.py)

QuantizedModelLoader: OneBit Support

• QuantizedModelLoader now supports quant_method="onebit" (onecomp/quantized_model_loader.py)
  • Added OneBitLinear to import and layer replacement logic
  • Added OneBitLinear.from_saved_state() call path for creating empty OneBit layers during model loading
  • Hadamard hook registration now recognizes OneBitLinear as a quantized layer class

BlockWisePTQ / CBQ OneBit Optimizer Compatibility

• Updated OneBit block-wise and cross-block quantization (CBQ) optimizers to work with packed-only OneBitLinear (onecomp/post_process/_blockwise/onebit_block_optimizer.py, onecomp/post_process/_blockwise/onebit_cbq_optimizer.py)
  • Reads current sign matrices from sign_packed via my_unpack() when sign_matrix is not present, while still allowing sign_matrix as a temporary optimization override
  • Writes sign updates back to sign_packed with my_pack() and clears sign_matrix so packed signs remain the single source of truth after hard evaluation, best-state restore, and final updates
  • Hoisted my_pack / my_unpack imports in the OneBit CBQ optimizer
• Clarified OneBitLinear.sign_matrix as a non-persistent temporary override used by optimization flows such as BlockWisePTQ and CBQ (onecomp/quantizer/onebit/onebit_layer.py)

Bug Fix

• Fixed GPTQLinear.from_saved_state(): _weight_is_packed now defaults to False when wbits == 1 (JointQ wbits=1 checkpoints are saved with pack_weights=False because GPTQLinear packing does not support 1-bit) (onecomp/quantizer/gptq/gptq_layer.py)
• Fixed redundant symmetric shift in RTN inference layer (onecomp/quantizer/rtn/_rtn.py)
• Fixed run_onebit() returning False on NaN/Inf detection; now raises ValueError with proper GPU tensor cleanup to prevent OOM cascading (onecomp/quantizer/onebit/onebit_impl.py)
• Removed pre-computed dequantized_weight from run_onebit() return dict and OnebitResult; dequantized weight is now computed on demand via compute_dequantized_weight() (onecomp/quantizer/onebit/onebit_impl.py, onecomp/quantizer/onebit/_onebit.py)
• QuantizedModelLoader._cast_fp16_to_target_dtype() now skips OneBitLinear in addition to GPTQLinear and DoubleBinaryLinear, so OneBit's fp16 scaling buffers (a, b, bias) are preserved when loading a OneBit-quantized model that requires bfloat16 (e.g. Gemma 3 / Gemma 4 detected via needs_bfloat16). Without this, the p...

[v1.1.1] 2026-05-21

New Feature: Quantization progress logging

• Added QuantizationProgressTracker (onecomp/utils/quantization_progress.py) that emits a single [progress] INFO line per completed step with done/total, percentage, elapsed time, and a linear ETA estimate; supports an optional thread_safe=True mode for multi-GPU quantization
• Added report_progress: bool = True flag to Runner.__init__ (onecomp/runner.py) and to the underlying entry points run_chunked_quantization (onecomp/runner_methods/chunked_quantization.py), run_multi_gpu_quantization / run_quantization_phase (onecomp/runner_methods/multi_gpu_quantization.py), run_quantize_with_qep (onecomp/qep/_quantize_with_qep.py), and run_quantize_with_qep_arch (onecomp/qep/_quantize_with_qep_arch.py) so long quantization runs (calibration, chunked, multi-GPU, QEP) report progress by default; pass report_progress=False for quiet runs
• Demoted some INFO-level per-layer / per-chunk logs to DEBUG to avoid duplication with the new [progress] line (still available via logging.basicConfig(level=logging.DEBUG) for deep debugging)

Bug fixes: QEP + JointQ validation

• Raise a clear error when Runner is configured with qep=True and a quantizer that does not support QEP (currently JointQ). Previously the run failed deep inside quantize_with_qep / adjust_weight with a confusing low-level error. Runner.check() now reports e.g. "Quantizer 'JointQ' (or one of its candidate quantizers) does not support QEP (Quantization Error Propagation). Set qep=False, or use a QEP-compatible quantizer (e.g., GPTQ, DBF, AutoBitQuantizer with QEP-compatible candidates)." Implementation: added flag_qep_supported (default True) on Quantizer, set to False on JointQ, and propagated via AutoBitQuantizer._sync_flags (only True when all candidate quantizers support QEP) (quantizer/_quantizer.py, quantizer/jointq/_jointq.py, quantizer/autobit/_autobit.py, runner.py).

Bug fixes: VLM save / load

• Runner.save_quantized_model() now copies all auxiliary *.json and *.jinja files (e.g. preprocessor_config.json, processor_config.json, special_tokens_map.json, chat_template.jinja) from the original model directory to the save directory, so the quantized model is fully self-contained for VLM / multimodal inference. Weight tensors (*.safetensors, *.bin, *.pt, *.pth), weight index files, config.json and generation_config.json are skipped, and any file already written by model.save_pretrained / tokenizer.save_pretrained is preserved (runner.py).
• Source-model directory resolution (incl. huggingface_hub.snapshot_download fallback for Hub IDs) was extracted into a private helper Runner._resolve_source_model_dir() (runner.py).
• load_quantized_model() now re-establishes the lm_head <-> embed_tokens weight tie for models with tie_word_embeddings=True. load_state_dict(..., assign=True) would otherwise leave lm_head.weight as the freshly initialised tensor (typically float16) while embed_tokens.weight got replaced with the checkpoint tensor (typically bfloat16), causing RuntimeError: expected mat1 and mat2 to have the same dtype at the final lm_head matmul during generation. The re-tie is gated on lm_head still being an nn.Linear so it does not interfere when lm_head itself was quantized (quantized_model_loader.py).
• load_quantized_model() now reads torch_dtype from config.json when no explicit torch_dtype is passed by the caller, so the empty model is built in the same dtype as the saved checkpoint. Previously it always defaulted to torch.float16, which left non-quantized VLM submodules (e.g. multi_modal_projector in Cohere2Vision) at fp16 whenever load_state_dict(..., assign=True) could not find their key in the state_dict (quantized_model_loader.py).
• load_quantized_model() now casts any leftover float16 parameters and buffers of non-quantized modules to model.config.torch_dtype after the lm_head re-tie step. Quantized layers (GPTQLinear, DoubleBinaryLinear) and float32 params (e.g. fp32 LayerNorm in mixed-precision models) are deliberately untouched. This generalises the existing lm_head re-tie to any non-quantized module and fixes the dtype mismatch reported in issue 64-3 (RuntimeError: ... c10::Half != c10::BFloat16 on VLM image features) (quantized_model_loader.py).
• Added regression tests tests/onecomp/runner/test_save_quantized_aux_files.py (auxiliary-file copy whitelist), tests/onecomp/runner/test_load_tied_embeddings.py (tied-embedding dtype round-trip) and tests/onecomp/runner/test_load_excluded_module_dtype.py (non-quantized module dtype handling, including config-based empty-model dtype default, fp16 safety-net cast, fp32 preservation, and quantized-layer skip).
• Loosened test_save_load_pipeline_tinyllama.py and test_save_load_pipeline_qwen3.py save/load round-trip threshold from absolute 1e-3 to relative 1% of the per-tensor logits magnitude (tests/onecomp/pre_process/test_save_load_pipeline_*.py). The original absolute bound was below fp16's representable precision once accumulated through the 22-28 decoder layers of TinyLlama / Qwen3, causing the gptq + save_dequantized cases to fail on aarch64 + Blackwell (GB200) where cuBLAS picks slightly different reduction kernels than reference x86_64 / Hopper hosts. The save/load equivalence intent is preserved via the relative comparison, which is robust to platform-specific fp16 rounding noise.
• Set gpu_memory_utilization=0.78 explicitly when constructing LLM(...) in example/vllm_inference/example_autobit_vllm_inference.py and example/vllm_inference/example_gptq_vllm_inference.py. The vLLM default 0.92 cgroup-OOMs on UMA hosts (e.g. DGX Spark / GB200, 121.7 GiB UMA) because vLLM's startup memory check fails: the residual quantizer process leaves only ~106 GiB free, which is below 0.92 * 121.7 = 111.96 GiB. 0.78 matches the value already used in tests/vllm_plugins/gptq/test_mixed_gptq_e2e.py and is documented in the workspace slurm-submit.mdc rule.

Logging / observability tweaks

• Runner._copy_auxiliary_files() now emits a matter-of-fact INFO-level log when an auxiliary file from the original model directory is not copied because the destination already contains a file of the same name (typically because tokenizer.save_pretrained wrote it just before, or a previous save_quantized_model call did). The new line is symmetrical to the existing Copied %s to save directory entry so the auxiliary-copy step can be audited end-to-end (runner.py).
• QuantizedModelLoader._cast_fp16_to_target_dtype() now returns the list of fully-qualified parameter / buffer names whose dtype was actually converted instead of a plain count. The post-load INFO log in load_quantized_model() includes those names so it is obvious which non-quantized submodules were normalised by the safety-net cast (e.g. multi_modal_projector.linear_* in Cohere2Vision). Existing tests are updated accordingly and a new test pins the buffer-name reporting (quantized_model_loader.py, tests/onecomp/runner/test_load_excluded_module_dtype.py, tests/onecomp/runner/test_save_quantized_aux_files.py).
• QuantizedModelLoader.load_quantized_model() now detects tie_word_embeddings=True even when the flag is nested in a sub-config (e.g. model.config.text_config.tie_word_embeddings in Llama 3.2-Vision and other torchtune-derived VLMs) by walking one level of sub-configs. Previously the flag was only read from the top-level model.config, so VLMs that placed it in text_config skipped the post-load re-tie; with HF deduplicating lm_head.weight for tied checkpoints, that left lm_head.weight at the empty-model random initial values rather than re-pointing to embed_tokens.weight (quantized_model_loader.py).

Tests

• Added regression tests for the save/load fixes above: tests/onecomp/runner/test_save_quantized_aux_files.py (auxiliary-file copy whitelist), tests/onecomp/runner/test_load_tied_embeddings.py (tied-embedding dtype round-trip), and tests/onecomp/runner/test_load_excluded_module_dtype.py (non-quantized module dtype handling, including config-based empty-model dtype default, fp16 safety-net cast, fp32 preservation, and quantized-layer skip).
• Added tests/onecomp/test_runner_check.py for the new qep=True validation path: JointQ + qep=True raises a clear ValueError, while JointQ + qep=False and GPTQ + qep=True both pass Runner.check().
• Added tests/onecomp/runner/test_load_tied_embeddings.py::test_should_retie_word_embeddings_* unit tests covering top-level, nested-text-config, all-False and unrelated-sub-attribute shapes.

New Contributors

• @sotanengel made their first contribution in #13

[v1.1.0] 2026-04-16

Gemma 3 / Gemma 4 & VLM Support

• Auto-detect language_model / text_model sub-modules in setup() so only the language model is quantized; vision_tower, audio_tower, etc. are automatically excluded (quantizer/_quantizer.py)
• Added unfuse_moe.py: MoE models (e.g. Gemma 4) store all expert weights as fused 3D nn.Parameter tensors (gate_up_proj [E, 2*inter, hidden], down_proj [E, hidden, inter]), but GPTQ and other layer-wise PTQ methods require 2D nn.Linear layers. unfuse_moe_experts() splits the fused tensors into per-expert modules, producing paths like experts.0.gate_proj, experts.0.up_proj, experts.0.down_proj (utils/unfuse_moe.py)
• Set quant_method to mixed_gptq for MoE models during save, enabling vLLM to handle a mix of quantized and unquantized expert layers via UnquantizedFusedMoEMethod (runner.py)
• Introduced prepare_block_kwargs to reproduce Gemma 4-specific additional inputs during block-wise forward (runner_methods/chunked_quantization.py, qep/_quantize_with_qep_arch.py)
  • _per_layer_inputs: pre-compute per-layer embeddings for all calibration samples
  • _position_embeddings_map: hook into rotary_emb to capture position embeddings per layer type
  • _attention_mask_map: pre-compute masks per layer type via create_causal_mask / create_sliding_window_causal_mask
• Updated Catcher.forward to accept *args (Gemma 4 passes per_layer_input as a positional argument)
• Added a guard to safely skip KV-shared layers where k_proj / v_proj are never called during forward and X^TX is not accumulated (runner_methods/chunked_quantization.py)
• Added token_type_ids (mm_token_type_ids) required by Gemma 4 to calibration data and PPL computation (utils/calibration.py, utils/perplexity.py)
  • Added model argument to prepare_calibration_dataset; model-specific inputs are appended via add_model_specific_inputs()
  • Changed model.device to next(model.parameters()).device to support VLM device_map="auto"
• Fixed MoE block partitioning (down_proj and router.proj were incorrectly placed in the same block) and relaxed Hessian input shape assertion for 2D tensors after router dispatch
• Added layer-suffix fallback lookup for Gemma 3's shared sub-modules where named_modules() paths differ from state_dict() keys (quantized_model_loader.py)
• save_quantized_model() now copies processor_config.json from the source model so the quantized model directory is self-contained for multi-modal inference (runner.py)
• Added skip logic in vLLM plugin to prevent vision / audio encoder layers from being incorrectly matched to language model quantization configs (vllm_plugins/utils/module.py)
• Override ModelConfig dtype to bfloat16 for Gemma 3/4 models whose values exceed the float16 range, preventing performance degradation (model_config.py)
• Fixed an issue where non-language-model layers in multi-modal models were included in AutoBit bit allocation
• Bumped transformers requirement from >= 5.3.0 to >= 5.5.0 (pyproject.toml)
  • Gemma 4's model_type: gemma4 is registered in CONFIG_MAPPING starting from 5.5.0 (released 2026-04-02); 5.3.0 fails to load it
• Added cu130 extra for the validation environment (NVIDIA B200, CUDA 13.0); under cu128, torch (cu130) and torchvision (cu128) had a CUDA version mismatch

New Feature: LPCD (Layer-Projected Coordinate Descent)

• Added onecomp/lpcd/ sub-package implementing the LPCD unified framework (arXiv:2512.01546) that extends layer-wise PTQ by jointly optimising sub-module groups (QK / VO / MLP / residual) with closed-form and gradient-based solvers
• Added benchmark/llama3-8b-lpcd-gptq/: Llama-3-8B LPCD+GPTQ SLURM array benchmark (Hydra config conf/benchmark_llama3-8b.yaml, quant_benchmark.py, README.md with WikiText-2 PPL / lm-eval-harness accuracy / quantization time for 4-bit and 3-bit × {q_proj·k_proj, v_proj·o_proj, up_proj·down_proj, all, residual} on NVIDIA B200)
• Added example/example_lpcd_gptq.py: TinyLlama GPTQ 3-bit (groupsize=128) + QEP + LPCD end-to-end example with residual-only closed-form refinement (enable_residual=True, use_closed_form=True) and original / dequantized / quantized perplexity reporting
• Updated README.md: added LPCD to Features, Examples, and Citation sections

New Feature: BlockWisePTQ

• Implemented BlockWisePTQ.run() pipeline (onecomp/post_process/blockwise_ptq.py)
  • Phase 1: per-block distillation with teacher model (GPTQ / DBF / OneBit / Generic)
  • Phase 2: Cross-Block Quantisation (CBQ) sliding-window optimisation (K=2)
  • Teacher model loaded via model_config.load_model(device_map="cpu")
  • Calibration inputs collected via Catcher hook on first transformer block
• Added onecomp/post_process/_blockwise/ sub-package (9 modules)
  • helpers.py: collect_layer_inputs, auto_detect_quantization_strategy, get_transformer_layers, layer_kwargs_to_device, etc.
  • Phase 1 optimisers: gptq_block_optimizer.py, dbf_block_optimizer.py, onebit_block_optimizer.py, generic_block_optimizer.py
  • Phase 2 CBQ optimisers: gptq_cbq_optimizer.py, dbf_cbq_optimizer.py, onebit_cbq_optimizer.py
  • All optimisers use float32 promotion, best-state tracking with rollback, and hard MSE evaluation
• Set use_gemlite=False in Runner.run_post_processes() (onecomp/runner.py) to avoid GemLite fp16-only Triton kernel incompatibility with float32 block optimisation
• Added VLM support for BlockWisePTQ (Qwen3-VL, Qwen2.5-VL, etc.)
  • helpers.py: get_transformer_layers / _get_language_model_backbone handle model.model.language_model.* path
  • model_config.py: load_model() falls back to AutoModelForImageTextToText for VLM configs
• Fixed Quantizer.calculate_hessian / calculate_delta_hatX (onecomp/quantizer/_quantizer.py): handle 2D activations from OPT-style architectures

Quantizer Unification

• Unified scale/zero/integer logic across WeightQuantizer, RTN, and GPTQExcecutor for both symmetric and asymmetric quantisation
  • WeightQuantizer.configure / find_params / quantize (quant_models.py), STEQuantize.forward (quant_models.py), pseudo_quantize_tensor / quantize (rtn/quantizer.py), GPTQExcecutor.configure / find_params (gptq/_gptq.py)
• Added optional MSE grid search (mse, norm, grid) to WeightQuantizer, RTN, and prepare_rotated_model
  • WeightQuantizer.configure / find_params (quant_models.py), pseudo_quantize_tensor (rtn/quantizer.py), run_rtn (rtn/rtn_impl.py), RTN dataclass / validate_params (rtn/_rtn.py), prepare_rotated_model (prepare_rotated_model.py), apply_preprocess_train / _insert_weight_quantizer (train_rotation.py)
• Removed perchannel and maxshrink from public APIs; perchannel=True is now always used internally
  • Removed from RTN dataclass (rtn/_rtn.py) and prepare_rotated_model (prepare_rotated_model.py). Internally, run_rtn (rtn/rtn_impl.py) and _insert_weight_quantizer (train_rotation.py) pass perchannel=True unconditionally. Low-level APIs pseudo_quantize_tensor (rtn/quantizer.py) and WeightQuantizer.configure (quant_models.py) still accept the parameters

Rotation Preprocessing Improvements

• Added "random_hadamard" and "hadamard" rotation modes (existing: "random", "identity")
  • PreprocessManager._ortho (train_rotation.py), _VALID_ROTATION_MODES (prepare_rotated_model.py)
• Changed prepare_rotated_model defaults: rotation_mode → "random_hadamard", num_calibration_samples → 512
  • prepare_rotated_model (prepare_rotated_model.py), PreprocessManager.__init__ (train_rotation.py)
• Added input validation (_validate_prepare_rotated_model_params) for all prepare_rotated_model parameters
  • _validate_prepare_rotated_model_params (prepare_rotated_model.py)
• Added per-step and total execution time logging to prepare_rotated_model
  • prepare_rotated_model (prepare_rotated_model.py): timed sections for model load, calibration prep, training, reload, apply_preprocess_eval, and save
• Added explicit gradient_accumulation_steps=1 to TrainingArguments defaults
  • TrainingArguments.gradient_accumulation_steps (preprocess_args.py)

AutoBit: per-quantizer groupsize support

• AutoBitQuantizer supports each candidate quantizer's groupsize individually, enabling mixed group-size configurations (onecomp/quantizer/autobit/_autobit.py)
  • RTN error evaluation uses per-quantizer grouped quantisation (onecomp/quantizer/autobit/ilp.py)
  • Added test for mixed group-size autobit (tests/onecomp/quantizer/autobit/test_autobit.py)
• Remove default quantizer from AutoBit; a quantizer must be explicitly provided. (onecomp/quantizer/autobit/_autobit.py)

CalibrationConfig: unified calibration configuration

• Breaking change: Introduced CalibrationConfig dataclass (onecomp/calibration/calibration_config.py) to consolidate all calibration-related parameters
  • Runner.__init__ now accepts calibration_config: CalibrationConfig instead of individual parameters (calibration_dataset, max_length, num_calibration_samples, calibration_strategy, calibration_seed, calibration_batch_size, num_layers_per_group)
  • AutoBitQuantizer now accepts calibration_config: CalibrationConfig instead of num_calib_samples, calib_seqlen, calibration_dataset
  • prepare_rotated_model() now accepts calibration_config: CalibrationConfig instead of calibration_dataset, max_length, num_calibration_samples, calibration_strategy
  • BlockWisePTQ now accepts calibration_config: CalibrationConfig instead of num_calibration_samples, max_length, calibration_strategy, calibration_seed
  • When calibration_config=None, default values are cre...

Bug Fix

• Fixed ImportError when running onecomp CLI without matplotlib installed; AutoBitQuantizer._visualize() now catches the import error and logs a warning instead of crashing

Packaging

• Moved matplotlib from dev extra to new visualize extra in pyproject.toml
• Made visualize_bit_assignment import lazy in onecomp/quantizer/autobit/__init__.py to avoid requiring matplotlib at import time
• Updated installation instructions in README.md and docs/getting-started/installation.md to reflect the new visualize extra
• Updated uv.lock