| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
…on in partial quantization
…lse explicitly in perplexity eval path
…tion_utils Extract the inline down_proj type-collection in Runner.create_quantized_model and QuantizedModelLoader.load_quantized_model into pure helpers collect_quantized_down_proj_types / collect_down_proj_types (placed next to their consumer register_online_hadamard_hooks). Behavior-preserving. Add regression tests: - Helper unit tests: nn.Linear filtering, all-distinct-types collection, leading-nn.Linear, empty cases. - Integration tests for call-site wiring: loader passes a non-None model-derived list for unknown quant_method; runner feeds collected types to register_online_hadamard_hooks.
…/partial-quant-with-rotation-bug
…/partial-quant-with-rotation-bug
…/partial-quant-with-rotation-bug
…/partial-quant-with-rotation-bug
There was a problem hiding this comment.
LGTM
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Fix Hadamard hook target-type collection for partial quantization with rotation
Summary
This PR fixes Hadamard online-hook registration for rotated models that are partially quantized. The set of down_proj target layer classes used for hook registration is now derived from the actual loaded/running model instead of from the recorded quant_method or from sampling the first down_proj layer. This ensures hooks land on the correct layers when a model contains a mix of quantized and unquantized (nn.Linear) down_proj layers, or multiple down_proj types — reventing degraded rotated-model inference.
Affected files: onecomp/quantized_model_loader.py, onecomp/runner.py, onecomp/pre_process/rotation_utils.py.
Changes
Bug fix
Fixed Hadamard online hook registration for rotation + partial quantization models. Previously the down_proj target layer classes were derived from the recorded quant_method (gptq/dbf/onebit), or by sampling the first down_proj layer's type. When a model is only partially quantized (mixed quantized / unquantized down_proj layers) or contains multiple down_proj types, hooks were registered on the wrong classes — either missing quantized layers or firing on plain nn.Linear — degrading rotated-model inference.
Test