Develop/v1 1 0 - #11
Merged
FKKimura merged 92 commits intoApr 28, 2026
Merged
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
Enable GPTQ quantization, evaluation, save/load, and multimodal inference for the full Gemma model family (Gemma2, Gemma3, Gemma4). VLM support (quantizer/_quantizer.py): - Auto-detect VLM architecture by finding `language_model` or `text_model` submodule in setup(), restricting quantization to the language model only. Vision tower, audio tower, and multimodal projector layers are automatically excluded without requiring user-specified exclude_layer_keywords. Gemma4 KV-sharing support (runner_methods/chunked_quantization.py): - Skip layers whose forward hooks never fire during calibration. Gemma4 models with num_kv_shared_layers > 0 have k_proj/v_proj modules that exist in the module tree but are unused during forward (KV is shared from earlier layers). Guard against KeyError when these layers have no accumulated X^T X. Gemma4 forward requirements (utils/calibration.py, utils/perplexity.py): - Add token_type_ids (zeros) to calibration data returned by _chunk_single_document() and _chunk_concat(). Gemma4 requires token_type_ids in its forward() signature. - Add token_type_ids to perplexity evaluation forward call. - Fix model.device -> next(model.parameters()).device for safe device detection with VLM models using device_map="auto". VLM model loading (quantized_model_loader.py): - Add state_dict key fallback lookup for VLMs with tied/shared submodules (e.g. Gemma3) where named_modules() path differs from state_dict key prefix. Falls back to suffix-based matching when direct prefix lookup returns empty. Processor config preservation (runner.py): - Copy processor_config.json and preprocessor_config.json from the original model directory when saving quantized models. This allows AutoProcessor.from_pretrained() to work on the quantized model directory alone, enabling standalone multimodal inference. vLLM plugin VLM guard (vllm_plugins/utils/module.py): - Skip vision_tower, vision_model, multi_modal_projector, and audio prefixes in _parse_layer_and_module() to prevent vision/audio encoder layer indices from being mismatched against language model quantization config. Dependency updates (pyproject.toml): - Bump transformers >= 5.5.0 (required for gemma4 model_type). - Add cu130 extra for NVIDIA B200 (CUDA 13.0) environments. Tested on: - gemma-2-2b: quantize/PPL/save/load/text OK - gemma-3-12b-it: quantize/PPL/save/load/text OK - gemma-4-E2B-it: quantize/PPL/save/load/text/image/audio OK - gemma-4-E4B-it: quantize/PPL/save/load/text/image/audio OK - gemma-4-31B-it: quantize/PPL/save/load/text/image OK
lab/blockwise-ptq-and-calibration See merge request onecomp/onecomp-lab!49
vllm - Open WebUI See merge request onecomp/onecomp-lab!50
…d search - Align scale/zero/integer computation in WeightQuantizer, RTN, and GPTQExcecutor for both symmetric and asymmetric quantisation - Add optional MSE grid search (mse, norm, grid) to WeightQuantizer, RTN, and prepare_rotated_model - Add "random_hadamard" and "hadamard" rotation modes; change prepare_rotated_model defaults (rotation_mode="random_hadamard", num_calibration_samples=512) - Fix QuantLinear.forward S_qk scaling TypeError on MLP layers - Add _validate_prepare_rotated_model_params input validation and per-step execution time logging - Remove perchannel and maxshrink from public APIs - Adopt CalibrationConfig in prepare_rotated_model - Add test_prepare_rotated_model and test_weight_quantizer; split into 14 files for Slurm parallelisation; expand test_rtn coverage - Update docs (rtn.md, pre-process.md) and CHANGELOG
Lab/update rotation rtn See merge request onecomp/onecomp-lab!47
…due to gemma4 architecture
…uce CalibrationConfig dataclass.
- Refactor LPCD runner/refiner to take CalibrationConfig and call compute_dequantized_weight() (fixes AttributeError when the quantizer, e.g. GPTQ, overrides dequantized_weight) - Pass calibration_config through Runner.quantize_with_lpcd - Add example/example_lpcd_gptq.py (TinyLlama GPTQ 3-bit + QEP + LPCD) - Add tests/onecomp/lpcd/ (25 cases): LPCDConfig / make_lpcd_metrics CPU unit tests + Runner-level GPTQ+QEP+LPCD integration test - Document LPCD feature and tests in CHANGELOG - Add NOTE comments across example/ recommending CalibrationConfig() defaults and batch_size for chunked calibration - Tidy .gitignore (collapse work/* patterns into work/)
- Fix VLM fallback for Cohere2Vision model loading - Add progressive damping fallback for GPTQ Cholesky decomposition - Add cu130 extra and require it for vLLM workflows - Revert GPTQ v1 qzero=0 bitmask fix and its regression tests - Document VLLM_USE_DEEP_GEMM workaround for non-FP8 quantization See merge request onecomp/onecomp-lab!60
- Add model_validation suite (autobit, gptq, qep_gptq, autobit_qep, jointq) - Add gptq load and vllm validation phases - Add hydra extra and submit-script ergonomics See merge request onecomp/onecomp-lab!64
- Add overview figure, ArXiv citation, and quick-start usage to README - Fix pre-process and DBF docs to match v1.1.0 API signatures - Add LPCD documentation See merge request onecomp/onecomp-lab!57
Add Gemma 2/3/4 support with MoE quantization and model validation suite - Support GPTQ quantization and vLLM inference for Gemma series - Handle MoE architecture: expert-level quantization and unfused_moe - Refactor calibration inputs into shared model_inputs module - Add model_validation test suite (AutoBit, GPTQ, QEP, JointQ) - Fix MSE calculation, module grouping, and model loading path bugs See merge request onecomp/onecomp-lab!46
Update JointQ See merge request onecomp/onecomp-lab!52
Lab/benchmark fix See merge request onecomp/onecomp-lab!53
- update readme and docs to describe supporting Gemma models - fix docs for showing checkmark icons See merge request onecomp/onecomp-lab!65
Add copyright See merge request onecomp/onecomp-lab!67
bug-fix See merge request onecomp/onecomp-lab!66
aki916f
approved these changes
Apr 28, 2026
aki916f
left a comment
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
LGTM
Sorry, something went wrong.
Yuma-Ichikawa
approved these changes
Apr 28, 2026
dogwood-flo
pushed a commit
to dogwood-flo/OneCompression
that referenced
this pull request
Jul 15, 2026
…-1-0 Develop/v1 1 0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.