v1.3.0 - #24
Merged
FKKimura merged 165 commits intoAug 6, 2026
Merged
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
…apter separately, run inference with vLLM
- Add `get_quant_config()` returning GPTQ-compatible config - Add `_build_quantization_bits()` and `finalize_quant_config_for_save()` - Add `create_inference_layer()` to build GPTQLinear from JointQResult
…wledge, verified with vllm
…ion knowledge, and add smoke test
Develop/v1 1 0
GPTQLinear weight packing only supports wbits in (2, 3, 4, 8), so JointQ with bits=1 must build/save the inference layer with pack_weights=False. - JointQ.validate_params: warn when bits=1 to remind callers to pass pack_weights=False at inference layer construction time - GPTQLinear.from_saved_state: load qweight/qzeros as unpacked tensors when wbits=1, matching the unpacked save format
…/lora-merge-v1.1.0 Resolved conflicts in: - onecomp/quantizer/jointq/_jointq.py - onecomp/runner.py
Add QuantizationProgressTracker and wire it through calibration, chunked calibration, multi-GPU phase 2, QEP general and arch-aware paths. Runner gains quantization_progress flag (default on). Includes unit tests for ETA formatting and thread-safe stepping. Co-authored-by: Cursor <cursoragent@cursor.com>
…wledge, verified with vllm
Raise clear error for unsupported QEP quantizers See merge request onecomp/onecomp-lab!71
feat: quantization progress logs with ETA
* refactoring : QuantizationProgressTracker * update CHANGELOG.md --------- Co-authored-by: FKKimura <50981196+FKKimura@users.noreply.github.com>
- QuantizationProgressTracker: suppress duplicate logs after completion, use `is not None` for lock check, expand Google-style docstrings - Demote per-layer / per-chunk INFO logs to DEBUG so the [progress] line is the single canonical per-step INFO signal (onecomp/quantizer/_quantizer.py, onecomp/runner_methods/chunked_quantization.py, onecomp/qep/_quantize_with_qep_arch.py) - Chunked path: switch to per-group progress (was per-layer × per-quantizer); drop chunk_progress / layer_progress arguments - Shorten tracker labels: "Quantization", "Chunked quantization", "QEP" - Per-block MSE log in QEP arch path: drop redundant `[INFO]` prefix, switch to lazy `%` formatting, fix "Layer N" -> "Block N" - Expand QuantizationProgressTracker tests (10 cases, incl. thread-safety and overflow suppression) - Update CHANGELOG for the new [progress] line and INFO -> DEBUG demotion
See merge request onecomp/onecomp-lab!72
Fix three OneComp-side bugs surfaced by Takane VLM (Cohere2Vision) work: the `save_quantized_model` flow dropping VLM auxiliary configs, the `load_quantized_model` flow leaving tied `lm_head` and other non-quantized modules in the wrong dtype after `load_state_dict(..., assign=True)`. Issue 1 (save_quantized_model misses VLM aux configs): - Replace the hard-coded allow-list (processor_config.json / preprocessor_config.json) with an extension-based whitelist of `*.json` / `*.jinja` (excluding weight shards, weight index files, `config.json`, and `generation_config.json`) - Extract source-model-dir resolution into `Runner._resolve_source_model_dir()` with a `snapshot_download( local_files_only=True)` fallback that warns on failure instead of aborting the save (onecomp/runner.py) - Skip files already produced by `model.save_pretrained` / `tokenizer.save_pretrained` and INFO-log both copied and skipped entries for traceability Issue 2 (tied lm_head left at fp16 after load): - Call `model.tie_weights()` after `load_state_dict(..., assign=True)` to restore `lm_head.weight is embed_tokens.weight` identity that `assign=True` breaks (onecomp/quantized_model_loader.py) - Guard the call on `tie_word_embeddings=True` and `isinstance(model.lm_head, nn.Linear)` to avoid touching quantized layers such as `GPTQLinear` - Walk one level of nested sub-configs (`text_config` / `language_config`) via `_should_retie_word_embeddings` so HF VLM configs that move `tie_word_embeddings` under `text_config` are also covered Issue 3 (non-quantized VLM modules left at fp16): - Make `_build_empty_model_from_config` honor `torch_dtype` / `dtype` from `config.json` when the caller did not pass an explicit dtype, so empty modules start in the saved checkpoint's dtype (onecomp/quantized_model_loader.py) - Add `_cast_fp16_to_target_dtype` as a safety net that casts any fp16 parameter / buffer left on non-quantized modules to `model.config.torch_dtype` after `tie_weights()`, returning the list of fully-qualified names that were converted for logging and tests - Skip `GPTQLinear` / `DoubleBinaryLinear` and limit the cast to fp16 so mixed-precision fp32 LayerNorms are not disturbed; mutate `p.data` in place so tied-weight identity is preserved Tests (tests/onecomp/runner/, 30 cases, CPU-only, no network): - test_save_quantized_aux_files.py (7): whitelist copy, existing-file skip, `snapshot_download` fallback, subdirectory exclusion, INFO log for skipped files - test_load_tied_embeddings.py (7): tiny LlamaForCausalLM with `tie_word_embeddings` True / False, identity restoration, forward dtype, `_should_retie_word_embeddings` top-level / nested / all-False / unrelated-sub-attrs cases - test_load_excluded_module_dtype.py (16): config-driven empty-model dtype, fp16 safety-net cast, fp32 preservation, quantized-layer skip, `_resolve_dtype_from_config` parsing edge cases, returned module-name list Update CHANGELOG.md (v1.1.1 Bug fixes) and verify each fix in isolation by reverting it and confirming the targeted test fails.
See merge request onecomp/onecomp-lab!69
Fix-tests-examples See merge request onecomp/onecomp-lab!70
Develop/v1-1-1
Resolved conflicts in: - CHANGELOG.md - example/vllm_inference/example_gptq_vllm_inference.py - onecomp/runner.py
#41) * [update] support rotated gptq vllm inference (TP = 1) * [fix] restrict online hadamard targeting to dense mlp.down_proj - share the Hadamard target predicate between preprocessing and vLLM - limit online Hadamard application to dense `mlp.down_proj` only - exclude MoE expert `down_proj` paths from the current rotation flow - keep preprocessing and vLLM behavior consistent with current MoE support scope * [fix] support rotated GPTQ vLLM inference with tensor parallel handling * [fix] register rotated linear method for vllm weight_loader_v2 dispatch - register RotatedLinearMethod via vllm's weight_loader_v2 allowlist so wrapped GPTQ / GPTQ-Marlin layers keep the same loading path as unwrapped vllm methods - add regression coverage for rotated down_proj dispatch and base process_weights_after_loading delegation - add unit test for the online Hadamard target predicate - document the TP>1 all_gather cost in _apply_tp_hadamard - clarify that rotation hooks currently target dense mlp.down_proj layers only * [test]Add smoke test with TP=2 * [test] fix dependency * [test]Add rotation unit tests and runner save-path coverage * [test]TP1 rotated GPTQ e2e coverage * [test] Move rotation helper tests into utils package * [test]make RotationMetadata.from_quant_config tests explicit * [test]cover missing rotation utility edge cases * [fix] enable DBF + rotation support in vLLM plugin - Add RotationMetadata support to DbfConfig - Wrap quantized layers with RotatedLinearMethod when rotation is configured - Expose layer dimensions in DoubleBinaryLinear * [test] add DBF rotation plugin integration tests and Runner coverage * [test] TP1 rotated DBF e2e coverage * [test] strengthen DBF rotation plugin unit tests - Add test_get_quant_method_wraps_dbf_linear_method_for_quantized_down_proj: verifies DBFLinearMethod (not UnquantizedLinearMethod) is the base when quantization_bits has a real entry, and that _dbf_mod_cfg/_dbf_prefix are set - Add test_prehook_is_installed_on_down_proj_after_process_weights: verifies _onecomp_hadamard_prehook_installed is True after process_weights_after_loading through the DBF quantized path - Add test_create_weights_raises_for_tensor_parallel_size_greater_than_one: verifies DBFLinearMethod.create_weights raises ValueError when get_tensor_model_parallel_world_size returns > 1 (monkeypatched) * [test] trim redundant rotated-vs-plain e2e quantize tests The saved-config metadata contrast (rotated flag + quant_method routing) is already covered by the cheap fake-config unit tests in test_runner_rotated_vllm_save.py, so the e2e config-comparison tests only re-verified that logic on top of a full quantization run. DBF: drop test_plain_dbf_uses_different_saved_load_path and rename TestRotatedVsPlainDBFVllmInference -> TestPlainDBFVllmInference. The plain generate smoke is kept because plain DBF still loads through the in-house "dbf" plugin. GPTQ: remove TestRotatedVsPlainGPTQVllmInference and the now-unused plain_quantized_model_dir fixture entirely, eliminating a second GPTQ quantization run. Plain GPTQ ("gptq") is served by vLLM's built-in handler and exercises no first-party code, so the smoke added little value. * [fix] fall back to naive path on GemLite inference failure with process-wide disable * [test] add GemLite fallback regression tests and strengthen assertions * [docs] document GemLite automatic fallback behavior and TRITON_CACHE_AUTOTUNING * [test] strengthen GemLite fallback tests: fused 2D scaling0, real naive, bias Cover the production-critical and previously-untested paths in DBFLinearMethod.apply()'s GemLite -> naive fallback: - Add test_fused_fallback_matches_real_naive (part_count=3). Uses a 2D (part_count, in_features) scaling0 so it exercises the `scaling0.ndim == 2` per-part-index branch in _compute_parts() that the fused qkv_proj actually hits. With only _apply_gemlite forced to fail, it runs the real _compute_parts / _apply_naive and verifies torch.cat(dim=-1) concat, per-part scaling2/scaling4/bp offset slicing, and the naive numerics against an independent reference. - Add test_bias_is_added_to_output covering the final `out + bias` path. - Replace caplog with a warning_spy fixture that patches the module logger directly, so the warning assertions no longer depend on the vLLM logger keeping propagate=True. * [docs] clarify that OOM is not caught by the GemLite fallback * [test] modularize vllm_plugins test infrastructure: shared conftest, session fixtures, try/finally cleanup * [test] consolidate LLM import in conftest and guard DBF integration test * [test] remove dead code in build_vllm_llm and document DBF skip gate Follow-up cleanup to the vllm_plugins conftest consolidation: - conftest.py: drop the unreachable `return LLM(...)` after pytest.skip() (pytest.skip raises NoReturn, so the line never executes and no type checker is configured to require it), and remove the stray blank line left between the vllm import and its except clause. - test_rotated_dbf_e2e.py: note in the module docstring that the file is skipped by default and enabled via RUN_DBF_INTEGRATION_TESTS=1, matching the convention in develop/v1-3-0's test_global_ptq_integration_dbf.py. * [docs] add v1.3.0 rotation/vLLM changelog and fix outdated rotation notes Add a CHANGELOG entry for the rotation-save-load-vllm-infer branch covering rotated GPTQ/DBF vLLM inference, the DBF GemLite automatic fallback, and the new tests. Correct the now-stale "rotation is not vLLM-servable" notes in the docs: rotation-preprocessed checkpoints are servable through the mixed_gptq and dbf plugins (dbf is TP1-only), while built-in gptq paths (RTN/JointQ) are out of scope. * [style] Apply pre-commit fixes * [fix(vllm)] mirror base method's weight_loader_v2 support in rotation wrapper * [docs] clarify rotation wrapper weight-loader dispatch in CHANGELOG --------- Co-authored-by: koohr <kou.ohira@compmind.co.jp>
* [update] Bitpack GPTQ weights and zeros immediately after quantization * [fix] disable GPTQ bitpacking for JointQ initialization * [fix] existing GPTQ tests to use unpacked quantize results by default * [fix] existing AutoBit tests to use unpacked quantize results by default * [update] Propagate AutoBit bitpack mode before child quantizer validation * [update] Add bitpack-mode flag (bitpack_on_quantize) default to base Quantizer * [update] Validate bitpack_on_quantize support in GPTQ params * [update] Add GPTQ bitpack equivalence test and example * [fix] Limit GPTQ wbits validation to 15 bits * [update] add test_gptq_bitpack.py for cover bitpack metadata, dequant, inference paths * [fix] Fail AutoBit validation for unsupported GPTQ bitpack wbits * [update] Update CHANGELOG.md (bitpack_on_quantize) * [style] Apply black formatter * [update] Update CHANGELOG.md (v1.3.0-wip merge into bitpack-mode branch) * [update] Add DBF bitpack_on_quantize flag and validation Add a DBF-specific bitpack_on_quantize flag (default True) to the DBF quantizer dataclass and validate it as a bool in validate_params(). DBF packs arbitrary shapes via padding, so no bit-width/shape constraint is imposed; only the bool type is checked. * [update] Add DBFResult packed-state metadata and unpack helpers Add packed-state metadata (dbf_A_is_packed, dbf_B_is_packed, dbf_A_original_shape, dbf_B_original_shape) and a get_unpacked_binary_factors() helper to DBFResult so callers can obtain the unpacked +/-1 factors regardless of storage. The helper returns float16 for both packed and unpacked storage so the representation difference never leaks to callers. Add shape-aware unpack_binary_matrix() (and pack_binary_matrix alias) to dbf_layer so the pack/unpack shape-restore logic lives in one place. Old results without the metadata fields are treated as unpacked via getattr defaults. * [update] Bitpack DBF binary factors right after quantization In DBF.quantize_layer(), when bitpack_on_quantize is enabled and DBF produced both binary factors, pack dbf_A / dbf_B into uint8 CPU tensors immediately after run_dbf() and record the packed-state metadata on the DBFResult. The unpacked references are dropped early to reduce RAM held during quantization. On DBF failure / is_dbf_quantized=False the factors are left untouched, preserving the existing dequant error behavior. * [update] Make DBF dequant path packed/unpacked agnostic compute_dequantized_weight() now recovers the +/-1 factors via get_unpacked_binary_factors(), so it returns the same weight whether dbf_A / dbf_B are stored packed or unpacked (pack/unpack is value-preserving). The reconstruction formula is unchanged. Route the existing in-layer unpack call sites (BitLinearPacked.forward and DoubleBinaryLinear._unpack_bp) through the shared unpack_binary_matrix helper to avoid duplicating the shape-restore logic. * [update] Avoid re-packing already-packed factors in DoubleBinaryLinear DoubleBinaryLinear.__init__() / from_quantization_result() now accept already-packed dbf_A / dbf_B (with their original shapes) and register them straight into bp1 / bp3 without an unpack/re-pack round-trip; unpacked inputs are packed as before. GemLite initialization builds a short-lived unpacked copy for packed inputs only (not retained). The saved checkpoint format (bp1 / bp3) and from_saved_state() are unchanged. * [update] Make existing DBF tests packed/unpacked agnostic Update the existing DBF quantizer test assertions that assumed unpacked float16 dbf_A / dbf_B: check_quantize_layer branches on the packed flags (uint8/1D when packed, float16/2D otherwise) and reconstructs the weight via get_unpacked_binary_factors(); check_equal_results and the apply-to-module helper compare/store factors through the same helper so they hold for both representations. No changes were needed in the blockwise post-processing optimizers or the vLLM DBF plugin, which work on the unchanged bp1 / bp3 packed buffers. * [update] Propagate AutoBit bitpack mode to child candidates and DBF fallback Add a bitpack_on_quantize flag (default True) to AutoBitQuantizer and propagate it to candidate quantizers that support on-quantize bitpacking: validate_params() syncs the flag onto GPTQ/DBF child candidates (via _sync_child_bitpack_on_quantize) before their own validation, _assign_all_dbf() forwards it to the all-DBF quantizer, and the inject_dbf() fallback path forwards it to every DBF it creates. A non-bool flag raises a clear ValueError. * [update] Add DBF bitpack representation and propagation tests Add DBF quantizer tests covering both representations: default flag is True; packing stores uint8 1D factors with shape metadata; disabling keeps unpacked float16; packed and unpacked dequantized weights are bit-identical (both from quantize_layer and hand-built results); pack_binary -> unpack_binary_matrix is bit-exact across shapes; missing original shape raises; from_quantization_result registers bp1/bp3 with no re-pack and forward matches the dequantized linear; and a non-bool flag raises. Add AutoBit tests that the flag is synced onto DBF candidates and forwarded to inject_dbf fallbacks (1-bit GPTQ on a 64-wide layer to stay under the DBF threshold). * [update] Smoke-test DBF packed results through downstream consumers QEP, LPCD, chunked / multi-GPU quantization, and the cumulative-error analyzer all consume DBF results only via compute_dequantized_weight().to(device).to(dtype) and never touch dbf_A / dbf_B directly, so no functional change is needed for the on-quantize bitpack. Add a regression smoke test exercising that exact consumption path (analyzer._update_weights) with a packed DBFResult, confirming it succeeds and yields weights identical to the unpacked representation (no packed state leaks to callers). * [update] Add DBF save/load round-trip and vLLM e2e/config smoke tests - pre_process pipeline: add DBF quantized/dequantized save/load round-trip cases for TinyLlama and Qwen3 (5 cases: GPTQ q/dq, DBF q/dq, RTN dq) - tests/vllm_plugins/dbf/test_dbf_e2e.py: DBF quantize -> save -> config verification -> vLLM generation smoke (slow, CUDA/vLLM gated) - tests/vllm_plugins/dbf/test_dbf_config.py: DBF plugin config parse/dispatch smoke (DbfConfig) * [add] Add DBF bitpack runner smoke tests Add DBF packed-result smoke coverage for QEP, LPCD, and chunked calibration calc_quant_error paths. The tests verify that each Runner path produces packed dbf_A / dbf_B results and consumes them through compute_dequantized_weight() without leaking storage details to callers. * [fix] Expose in_features/out_features on DoubleBinaryLinear register_online_hadamard_hooks -> get_hadK introspects module.in_features, but DoubleBinaryLinear only stored _bp1_shape/_bp3_shape, raising AttributeError when re-registering Hadamard hooks on saved DBF-quantized rotated models. Expose in_features/out_features like nn.Linear (derived from the original unpacked shapes; also set in from_saved_state). * [update] Use DBF factor pack helpers * [update] Add DBF bitpack equivalence example Quantize a layer with bitpack_on_quantize=False vs True under identical seeds and assert the dequantized weights match bit-exactly (compute_dequantized_weight -> torch.equal). The corresponding DBF equivalence test is in tests/onecomp/quantizer/dbf/test_dbf_bitpack_equivalence.py. * [update] Add DBF vLLM inference example Parallels example_gptq_vllm_inference.py for the DBF path: quantize TinyLlama with DBF, save, then load and generate with vLLM. * [update] Document bitpack_on_quantize in DBF and AutoBit algorithm docs Add a bitpack_on_quantize row to the parameters table in docs/algorithms/dbf.md (DBF) and docs/algorithms/autobit.md (propagation to candidate / injected-DBF quantizers). * [update] Register DBF bitpack and vLLM examples in README and vLLM inference guide * [fix] Log bitpack_on_quantize overrides in JointQ and AutoBit * [doc] Clarify GPTQ bitpack-on-quantize behavior - Document the breaking change in CHANGELOG - Add a comment explaining qzeros reshape after pack/unpack * [refactor] Inline GPTQ_MAX_BITS usage in pack support validation * [fix] Centralize GPTQ bit-packing width checks * [refactor] Clarify GPTQLinear packed state handling * [update] Update CHANGELOG.md (DBF bitpack-on-quantize) * [style] Apply black formatter * [docs] update CHANGELOG.md for GPTQ pack_weights validation * [fix] Use is_packable_wbits to decide weight packing * [fix] Log pack_weights fallbacks for unsupported GPTQ wbits * [fix] Normalize float GPTQ wbits and fix config bits truncation * [docs] update CHANGELOG.md for GPTQ wbits normalization * [chore] Remove redundant bitpack equivalence examples Equivalence is already covered by tests, and these examples do not demonstrate user-facing usage. * [style] Apply pre-commit fixes * [docs] Clarify bitpack-on-quantize support boundaries - document GPTQ quantize-time and save-time packing constraints - clarify AutoBit propagation, overrides, and fused-group restrictions - document DBF and JointQ bitpacking behavior - record these doc updates in the DBF and GPTQ bitpack changelog sections * [docs] Sync example references after bitpack example cleanup - Remove references to the deleted DBF and GPTQ equivalence examples - Add the missing Qwen3.6 vLLM example to README and the inference guide --------- Co-authored-by: cm_ysmz <yohei.shimizu@compmind.co.jp> Co-authored-by: sikoji <seiichiro.kojio@compmind.co.jp>
… FusedMoE kernel support) (#39) * cherry-picking commit 7bf5621. * cherry-picking commit bd92a14. * Update model configuration and quantization settings; enhance blockwise attention type checks * temp * wip * succeeded in VLLM inference * succeeded in VLLM inference * git cherry-pick 4b38840 * revert over-erased change * Merge branch 'feature/qwen36_27b_save' of https://github.com/aki916/OneCompression into feature/qwen36_27b_save * Revert "succeeded in VLLM inference" This reverts commit 1b5bb84. * Revert "wip" This reverts commit 64fc908. * add test and chalgelog, and apply linter * Reverted the code under the example * fix loader, add test, and modify changelog * fix unfinished merge * byebye miss-merged implementation * update changelog * update docstring, docs, and README * Revert "revert" This reverts commit 0e006d6. * add test and fix changelog * Update model ID and save directory for Qwen 3.6 to version 35B-A3B; enhance exclusion keywords in quantizer settings * bugfix: success save load * succeeded in inferecing MoE quantized model with vLLM * refactoring, linter, add test, and fix changelog * revert example modification * add test and fix changelog * revert example modification * fix format * bugfix: success save load * add test and fix changelog * update CHANGELOG * succeeded in inferecing MoE quantized model with vLLM * update CHANGELOG * add test and fix changelog * update comment for review no.4, 6, 7, and 8 * generalize moe detection * comment2: fallback when group_size is None * comment3: add test * comment5: allow rtn fallback when every children quantizer GPTQ * missed adding __init__.py * lazy import is_moe_expert_g_idx_key in runner --------- Co-authored-by: aki916 <aki916.genyo@gmail.com>
Add CPU inference & GGUF export for OneComp quantized models (llama.cpp) See merge request onecomp/onecomp-lab!100
…ost-process audit metadata (#44) * [update] enable load → post-process → re-save for quantized models Allow a previously saved quantized checkpoint to be reloaded, refined with additional post-processes (e.g. BlockWisePTQ), and saved again, with an audit trail of which post-processes were applied. - quantized_model_loader: accept device_map=None/"" to leave the loaded model on CPU (post-processes assume CPU), and reattach quantization_config to model.config so a loaded model can be refined and re-saved without separately re-reading config.json. - runner: run_post_processes now accepts a pre-assigned runner.quantized_model (quantizer=None) in addition to building from a quantizer. Validate quantization_config and rotated-checkpoint consistency on the pre-loaded path, track applied post-processes, and append them to quantization_config["onecomp_post_processes"] on save so config.json records the full post-process history across save/load cycles. - post_process/_base: add build_metadata() to produce JSON-serializable {name, class, config} audit metadata for each post-process. * [fix] accumulate post-process metadata until save * [refactor] centralize quantization_config validation in utils Extract the quantization_config schema checks that were duplicated across the save and load paths into reusable helpers. - Add validate_quant_config(): dict-level check requiring "quant_method" and "modules_in_block_to_quantize", raising ValueError with a caller context label. - Add validate_quantized_model_config(): model-level wrapper that pulls quantization_config off model.config and delegates to the dict check. - Both paths now enforce identical required keys and raise the same exception type. * [refactor] make PostQuantizationProcess.run a metadata-recording template method Turn run() into a template method shared by every post-process so audit metadata is recorded consistently regardless of how the process is invoked. - Add _runtime.py with shared helpers: prepare_quantized_model_for_post_process() (validate + move to CPU), validate_rotated_checkpoint_consistency(), append_post_process_metadata(), and the POST_PROCESS_HISTORY_KEY constant. - run() now validates and moves the model to CPU, calls the subclass body, restores eval()/cpu() even if it raises, and on success appends an audit entry to quantization_config["onecomp_post_processes"]. - Add build_metadata() producing a JSON-serializable entry per process. - Make _run() the abstract method subclasses implement; rename BlockWisePTQ.run and PostProcessLoraSFT.run to _run accordingly. * [update] validate and reattach quant_config on quantized model load Make the load path symmetric with the save path and allow a loaded model to be refined and re-saved. - Validate config.json quantization_config via validate_quant_config() so loading enforces the same schema and exception type as saving. - Reattach quantization_config to model.config after loading, so callers can post-process and re-save without separately reading config.json. - Accept device_map=None/"" (typed Optional[str]) to leave the model on CPU. * [update] support post-processing a preloaded quantized model in Runner Enable the load -> post-process -> re-save flow and reuse the shared validation helpers. - run_post_processes(): use runner.quantized_model when already assigned, otherwise build one from quantizer.results; raise a clear error if neither is available. - save_quantized_model(): require model_config and validate the model's quantization_config via validate_quantized_model_config() before saving a preloaded model. - Remove the now-extracted validation helpers (moved to utils/quant_config and post_process/_runtime). - Document that quantizer may be None when driving run_post_processes() through runner.quantized_model. * [update] document pack_weights behavior in save_quantized_model docstring Clarify the two save paths in save_quantized_model(). When runner.quantized_model is already set (after run_post_processes(), or a load -> post-process -> re-save flow), the model is saved as-is and the pack_weights argument is ignored — the weights keep whatever packing layout they were built with. Note that pack_weights=True must be set on create_quantized_model() beforehand to produce a loadable checkpoint, and that pack_weights only applies when building from quantizer.results. Docstring only; no behavior change. * [update] add tests for quant config and post-process metadata * [refactor] rename GlobalPTQ/GlobalPTQDistributed run() to _run() Rename run() -> _run() so both processes run through the shared PostQuantizationProcess.run() template (base entry validation + audit-metadata recording) instead of overriding run() directly. Existing cleanup inside the methods is left unchanged. * [update] build packed quantized model by default in run_post_processes run_post_processes() built the auto-created quantized model with pack_weights=False, so post-process -> save_quantized_model flows that did not pre-assign runner.quantized_model produced an unpacked (vLLM-unloadable) checkpoint, diverging from create_quantized_model()'s pack_weights=True default. Build with pack_weights=True so Runner-managed post-process flows keep packed buffers by default. This matches the save path for post-processes that preserve the quantized layer structure, such as BlockWisePTQ and GlobalPTQ: GPTQLinear unpacks at forward time, and the GPTQ post-process adapters read and write weights through _weight_is_packed-aware helpers. Workflows that require an unpacked layout can still build and pre-assign a model with create_quantized_model(pack_weights=False, use_gemlite=False). Also expand the create_quantized_model / run_post_processes / save_quantized_model docstrings to document the packed-by-default behavior and the pack_weights=False escape hatch for unpacked layouts. * [update] revise post-process examples for packed save and reload flows Align the post-process examples with run_post_processes() now building packed buffers by default and with structure-preserving save/load. - example_blockwise_ptq.py: switch to the Runner-managed packed path (run() -> run_post_processes()) and save with save_quantized_model(). - example_global_ptq.py: save the optimised model in both safetensors and .pt, add a commented DBF(target_bits=1.5) quantizer option (with dbf_lr), and add a commented direct post_process.run() path. - example_global_ptq_dbf.py: import DBF from the top-level package and save in both safetensors and .pt, matching the other GlobalPTQ examples. - example_global_ptq_distributed.py: also save a safetensors copy. - example_lora_sft_knowledge.py: build with the packed default and use_gemlite=False; make device selection CUDA-optional. - Add example_blockwise_ptq_unpacked.py and example_global_ptq_unpacked.py for explicit pack_weights=False research/debug workflows. - Add example_reload_post_process_resave.py for the end-to-end load -> post-process -> re-save flow with accumulated metadata. * [update] document load -> post-process -> re-save and packed-by-default Document the new save/load behavior across docstrings, the API reference, and the user guide. - runner.py: clarify run_post_processes() (packed by default) and save_quantized_model_pt() (custom-module case; prefer save_quantized_model for structure-preserving post-processes). - quantized_model_loader.py: note that BlockWisePTQ / GlobalPTQ / GlobalPTQDistributed outputs load via the safetensors path, and add a Raises: section for the required quant_method / modules_in_block_to_quantize keys. - docs/api/post_process.md: drop the stale subclass "members: - run" filters now that run() lives on the base template method. - docs/api/runner.md: list run_post_processes in the API reference. - docs/api/quantized_model_loader.md: show device_map=None CPU loading. - docs/user-guide/post-process.md, examples.md, README.md: add the load -> post-process -> re-save flow, packed/unpacked guidance, the onecomp_post_processes audit trail, and links to the new examples. * [update] add v1.3.0(WIP)+feature/blockwise_save_load changelog for load->post-process->re-save, post-process metadata, and quant_config validation * [add] BlockWisePTQ+GlobalPTQ post-process chain examples * [update] document BlockWisePTQ+GlobalPTQ chain examples in changelog * [fix] Remove duplicate create_quantized_model call in run_post_processes * [docs] document PostQuantizationProcess.run() breaking change * [style] Apply black formatter * [docs] note that Runner.check() is not intended for the load -> run_postprocess flow * [update] lighten BlockWisePTQ config in blockwise+global PTQ examples * [refactor] drop unpacked post-process examples; document the path in docstrings - Remove example/post_process/example_blockwise_ptq_unpacked.py and example/post_process/example_global_ptq_unpacked.py - Drop the unpacked-example guidance comments from the packed examples - Document the unpacked pack_weights=False path (with required bit widths and a short example) in the BlockWisePTQ and GlobalPTQ docstrings - Update README.md and docs/user-guide/{post-process,examples}.md to point at the API reference instead of the removed scripts - Add tests/onecomp/post_process/test_post_process_unpacked.py: a CPU-only, no-download test that feeds BlockWisePTQ.run() and GlobalPTQ.run() unpacked GPTQLinear layers built via the real GPTQ/RTN/JointQ inference-layer builders for the mandatory-unpacked bit widths (GPTQ 5-bit, RTN 5-bit, JointQ 1-bit), and asserts each run() accepts them and leaves the layers unpacked, on CPU, in eval mode, with post-process metadata recorded * [update] drop provisional .pt save from GlobalPTQ examples - Remove save_quantized_model_pt() (with save_dir_pt and the ".pt saved" print) from example_global_ptq.py, example_global_ptq_dbf.py, and example_global_ptq_distributed.py, leaving save_quantized_model() safetensors as the single standard flow - Reword the save comments and module-docstring step lists to drop the "either way" / ".pt formats" wording - Update the README table entry for example_global_ptq.py to "safetensors save" * [docs] align LoRA save/load docs with the safetensors + adapter-sidecar flow - post-process.md: rewrite "Saving and Loading LoRA Models" to lead with save_quantized_model()/load_quantized_model(); demote .pt to a legacy note; fix the save-method comparison table; turn the vLLM "Limitations" warning (LoRA not supported) into an info note (LoRA IS servable via the sidecar) pointing at example_lora_gptq_vllm_inference.py. - examples.md: update the LoRA save/load snippet to the safetensors path, keeping .pt as a legacy note. - docs/api/quantized_model_loader.md: note that load_quantized_model() auto-applies the LoRA sidecar; mark the .pt loader as legacy. - README.md: list example_lora_gptq_vllm_inference.py. - runner.py: fix the save_quantized_model_pt() docstring (rendered into docs/api/runner.md) that steered LoRA users to the .pt format * [update] record executed/reason in post-process audit metadata Let post-process _run() return an optional result dict and mirror its executed / global_executed and reason fields into quantization_config["onecomp_post_processes"]. GlobalPTQ and GlobalPTQDistributed now return these on early returns (not_quantized, unsupported_method_<method>, no_params) so skipped runs are recorded with executed: false and a reason instead of being indistinguishable from successful runs. * [docs] update CHANGELOG.md for executed/reason early-return audit metadata * [docs] use actual API names in Runner.check() flow note Replace the nonexistent load() and run_postprocess() references with load_quantized_model() and Runner.run_post_processes(). * [style] Apply pre-commit fixes * [test] Align full-wrapper save fixtures with quant config validation - add realistic GPTQ quantization configs to save-path fixtures - verify full-wrapper config remapping and default-format pass-through - update the changelog * [docs] Clarify post-process save/load and fix LoRA docstrings * [docs] Correct CHANGELOG entries against the upstream/develop/v1-3-0 baseline --------- Co-authored-by: sikoji <seiichiro.kojio@compmind.co.jp>
Bump HIDDEN and INTERMEDIATE from 4/6 to 8 so every expert Linear's in_features is divisible by the 4-bit pack factor (32 // 4 == 8), which pack_int_weights requires. The previous values made the toy MoE model unquantizable and failed all tests at the packing assertion.
The fp16 online Hadamard transform is not bitwise reproducible across GPU runs, so max logits diff sits near the fp32 bound of 0.5 and occasionally exceeds it (observed 0.5117 on CI). Loosen only this fp16 test threshold to 0.6; fp32 and tinyllama tests are unchanged.
GPTQ.quantize_layer() called GPTQLinear._normalize_scale_zero(), a private static method, from outside the class. It touches no class state, so expose it as the module-level normalize_scale_zero() next to the pack/unpack helpers and update all three call sites. Behavior is unchanged. Add unit tests for the three supported layouts and document that any other shape passes through unvalidated.
remap lora-key for Qwen with vLLM serving See merge request onecomp/onecomp-lab!114
support gpt-oss families See merge request onecomp/onecomp-lab!95
test: fix pack_factor divisibility in QEP expert recovery test See merge request onecomp/onecomp-lab!112
test: relax Qwen3 fp16 rotation+scaling threshold to 0.6 See merge request onecomp/onecomp-lab!113
…#46) * [add] add MDBF porting code * [update] update quantizer/__init__.py for MDBF port * [update] update MDBF logging output * [update] remove unused warn argument * [update] update MDBF nsamples, validation, and related handling - Update nsamples-related handling. - Rename run_msvid() to run_mdbf(). - Rename b_target to target_bits in run_mdbf(). - Add act_init validation to MDBF.validate_params(). - Pass len(params_list) to MDBFResult. * [update] add return value for MDBF activation_aware * [add] add MDBF unit tests * [fix] add shared MDBFResult path validation for dequantize and inference layer * [fix] propagate nsamples through quantization paths and update Hessian callers - make calculate_hessian return hessian and nsamples - pass nsamples through QEP arch, chunked, multi-GPU, and AutoBit flows - update quantizer tests to unpack Hessian tuples consistently Co-authored-by: Copilot <copilot@github.com> * [docs] translate MDBF comments and docstrings to English * [chore] remove unused files * [test] refactor MDBF quantizer tests and skip forward error test for alignment Co-authored-by: Copilot <copilot@github.com> * [Refactoring] Change MSVID to MDBF and fix missed comment translations * [update] add save/load support for MDBF - Add `onecomp/quantizer/mdbf/config.py` with `resolve_mdbf_layer_bits()` to determine per-layer bit-width from quantization_config (priority: quantization_bits table > module_target_bits > mlp_target_bits > default) modeled after `onecomp/quantizer/dbf/config.py` - Add `MultipathMDBFLinear.from_saved_state()` to reconstruct the layer directly from a saved state_dict, following the DBF layer pattern - Wire MDBF into `QuantizedModelLoader`: layer-class detection and `from_saved_state` loading path, aligned with the existing DBF branch - Remove `preunpack` option from `MDBFLinear` / `MultipathMDBFLinear`; always unpack sign matrices on-the-fly (same as DBF) - Fix bias buffer handling in `MultipathMDBFLinear` (clone to float16; use plain `None` instead of a registered null buffer) - Remove `test_forward_error` skip from `test_mdbf.py` * [refactoring] Drop QEP-DEV compatibility shims from mdbf_layer Remove helpers that are no longer reachable from any OneComp code path: PackedMDBFParams, pack/unpack_MDBF_params, PackedBinaryLinear, create_mdbf_layer_from_linear, replace_linear_with_mdbf, replace_all_MDBF_layers, save/load_MDBF_weights, verify_binary_values, and verify_all_params. Clean up the imports they depended on: gc, json, dataclass, getLogger, Path, Any, Dict, and transformers, along with the module-level logger. * [fix] propagate nsamples through LPCD quantization - Capture nsamples returned by compute_hessian_and_crossterm() in LPCD runner and refiner - Pass nsamples to quantizers during LPCD projection and QEP quantization when required - Allow Quantizer.quantize() to accept precomputed nsamples with precomputed Hessian - Forward Hessian and nsamples through AutoBitQuantizer to the selected child quantizer - Document the LPCD-related nsamples fixes * [test] enhance MDBF quantizer tests with comprehensive boundary and abnormal parameter cases * [fix] guard against zero init_error in MDBF ADMM improvement log When target_bits forces the decomposition rank to clamp to full rank (or the SVD initialization happens to reconstruct W exactly), the initial residual becomes zero and the improvement log line in optimize_MDBF_admm raises ZeroDivisionError. Apply the same `+ 1e-12` guard already used for orig_norm / orig_output_err in the same function so the relative improvement metric stays well-defined. * [style(onecomp)] run black formatter * [update] add v1.1.1+feature/mdbf changelog for MDBF quantizer and nsamples propagation * [update] preserve MultipathMDBFLinear fp16 metadata in _cast_fp16_to_target_dtype * [update] add changelog entry for MDBF fp16-metadata preservation in loader dtype cast * [docs] update author attribution in MDBF module headers * [update] expose scale_bits as a configurable parameter with default 16 (FP16) in rank_from_bpw and bpw_from_rank * [docs] fix MDBF acronym expansion to "Multi-Envelope Double Binary Factorization" * [feature] make MDBF scale_bits configurable (default 16, via DEFAULT_SCALE_BITS) * [update] Add MDBF quantization example Add example/example_mdbf.py demonstrating MDBF(target_bits=1.0). * [docs] Add MDBF quantization to documentation - Add MDBF algorithm guide and API reference pages - Register MDBF in overview / index / basic-usage / api lists and mkdocs.yml - Note that vLLM serving is method-specific * [style] Apply pre-commit fixes * [update] mdbf with gemlite * [fix] fix bugs(Hessian, bpw scale_bits) - Fixed a discrepancy in the definition of Hessian - Added `scale_bits` as an argument to `rank_from_bpw()`; the default is 16 * [fix] use symmetric H^{1/2} in lowrank_osvd W_tilde (append Q^T) * [docs] add scale_bits docstring and changelog for MDBF gemlite/OSVD fixes Document the scale_bits argument added to rank_from_bpw() and record the GemLite inference path, OSVD Hessian fix, and rank_from_bpw fix in CHANGELOG.md (PR review feedback). * [test] add MDBF gemlite fallback, rank_from_bpw, and OSVD regression tests - test_mdbflinear_falls_back_to_dense_when_gemlite_unavailable: verify MDBFLinear falls back to the dense path (bit-identical) when GemLite support is unavailable, even if use_gemlite=True is forced. - test_rank_from_bpw_matches_paper_formula: verify rank_from_bpw() round-trips with bpw_from_rank() at scale_bits=16 and that scale_bits=0 admits a higher rank for the same budget. - test_lowrank_osvd_beats_plain_svd_in_hessian_error: regression test for the lowrank_osvd() Hessian fix (a9b46df) exercising the production function directly against a non-diagonal Hessian. - test_osvd_hessian_bug.py: proof-of-concept numerically demonstrating that the previous H^{1/2} formulation inflated the Hessian-weighted reconstruction error for non-diagonal Hessians. PR review feedback (test plan coverage). * [docs] translate test_osvd_hessian_bug.py comments and docstrings to English Translate the Japanese module docstring, function/class docstrings, and inline comments in the OSVD Hessian proof-of-concept test to English. No logic changes (test still passes). * [docs] rewrite OSVD test comments to be self-contained (drop fix history) The OSVD tests described the pre-fix "buggy" lowrank_osvd (Q^T missing) and asserted facts about "the current implementation" that no longer hold after the fix. Reframe them as a self-contained property/regression check: the full H^{1/2}=Q diag(sqrt(λ)) Q^T whitening is optimal for the H-weighted objective, and a variant dropping the trailing Q^T is strictly worse for a non-diagonal H. Rename the _osvd_buggy helper / test methods accordingly. No logic changes (tests still pass). * [fix] pass per-expert nsamples to quantize_with_qep in QEP-arch MoE path _compute_per_module_hessians() now returns (hessian, nsamples) per module, and the MoE expert path forwards the per-expert token count, matching the regular-group path and the quantize_with_qep contract (nsamples must accompany a precomputed hessian). Previously flag_nsamples quantizers (e.g. MDBF) silently fell back to nsamples=1 on expert layers. * [fix] record actual_activation_aware in saved MDBF quant config The saved "activation_aware" is only the requested setting; run_mdbf() may fall back to non-aware mode (P != 1, or no Hessian). Aggregate the per-layer actual flags at save time so the config reflects what was done. * [fix] skip nested MDBFLinear children in _cast_fp16_to_target_dtype MDBF keeps its fp16 amplitude buffers on per-path MDBFLinear children, and the skip applies per visited module, so skipping only the MultipathMDBFLinear parent let bf16 loads cast them. Pin the behaviour with a real nested MDBF layer and an fp16-buffer OneBit stub in the skip test. * [style] Apply pre-commit fixes * [fix] complete calculate_hessian tuple migration - Route the remaining JointQ tests through the shared helper to unpack (hessian, nsamples) consistently. - Correct flag_csamples typos and document the flag_nsamples / flag_xtx quantize_layer contracts. * [fix] guard MDBF logging metrics against division by zero Add 1e-12 to the weight improvement denominator in optimize_MDBF_admm_hessian and to orig_norm in initialize_MDBF. * [fix] default MDBF to (l, P) = (2, 1) Use the smallest multi-envelope configuration instead of the LittleBit baseline. Centralize the l/P defaults, require l for single-path initialization, add regression coverage, and document BPW/GemLite impacts. * [fix] treat missing actual_activation_aware as unknown (None) Falling back to the requested flag recorded an unconfirmed value; None is already skipped by finalize_quant_config_for_save(). * [refactor] use shared device utils for MDBF memory cleanup Replace the CUDA-only cleanup_gpu_memory() in mdbf/utils.py with a new onecomp.utils.device.cleanup_memory(), which runs gc.collect() before the backend-aware empty_cache(). This enables MPS cache cleanup while keeping empty_cache() GC-free for hot per-layer callers. Also route the two raw torch.cuda.empty_cache() calls in initialize.py through empty_cache(). * [refactor] remove redundant min_rank clamp in rank_from_bpw The early return for r_real < min_rank guarantees that the rounding block only runs when r_real >= min_rank. Since min_rank is an integer, even floor() cannot fall below min_rank, making max(r, min_rank) redundant. Return the rounded rank directly and document the invariant. No behavior change. * [refactor] drop unused dtype local in run_mdbf * [refactor] simplify RNG generator setup in _tsvd_block_power * [fix] add input type annotation to run_mdbf Annotate the supported input types and document tuple/list handling. * [docs] drop porting framing from MDBF driver docs Rewrite the mdbf_impl.py module docstring as a self-contained description of the phases and interface, and describe the removed mdbf_layer.py symbols in the CHANGELOG as unused helpers. * [refactor] rename MDBF admm_iters to admm_outer_iters * [fix] propagate ADMM seed to MDBF projections The randomized SVD helpers accepted a seed, but ADMM callers never forwarded one, so their initialization always depended on the global RNG. Add MDBF(admm_seed=...) and propagate it through both the standard and Hessian ADMM paths. Validate None or a non-negative seed up to 2**64 - 1 before quantization, and preserve it in quantization configs and results. None keeps the previous global-RNG behavior. The seed controls only the ADMM phase; MDBF initialization still uses the global RNG. * [style] annotate actual_activation_aware as Optional[bool] * [fix] fall back to initial amplitudes in MDBF gradient refinement best_error was initialized from the input solution, but best_amp_params started as None. If no evaluated iterate improved on that baseline, the restore was skipped and the parameters left by the final, unevaluated optimizer step were returned. Initialize the best-parameter snapshot from the input amplitudes, reuse _snapshot_amp_params() for subsequent updates, and remove the redundant None guard. Add a regression test that uses a divergent learning rate and verifies that the initial amplitudes are restored when no step improves them. * [fix] restore MDBF safetensors loading after develop merge Index all state-dict ancestor prefixes so nested `<layer>.paths.{p}.*` tensors resolve to their MDBF layer, and use `saved_name` for per-layer bit-width resolution. This fixes missing states without bias and the stale-variable NameError with bias. Extend critical-key and post-load buffer validation to MDBF. Validate the saved path count and bias layout before `from_saved_state`, which otherwise infers both from potentially incomplete keys. Add round-trip and corrupted-checkpoint regression tests. * [test] update tests for calculate_hessian tuple return Unpack the Hessian in GPTQ and DBF bitpack tests, and reuse the BaseQuantizeSpec helper in the QUIP test. * [fix] register Hadamard hooks for rotated MDBF models The down_proj collectors also matched descendants, so MDBF's nested ModuleList entered layers_cls. find_linear_layers then stopped at model.layers, preventing online Hadamard hook registration across the model, including for non-MDBF down_proj layers. Missing hooks could silently produce incorrect inference results. Collect only exact Hadamard targets using the registration predicate. Also expose in_features/out_features on MDBF layers, align nn.Linear exclusion with find_linear_layers' exact-type matching, and add regression coverage for the runner and loader paths. * [fix] correct QEP-arch merge resolution from develop/v1-3-0 The merge left two defects in _quantize_with_qep_arch.py: - The expert loop was resolved to the upstream RTN-fallback hunk, which expects a bare Hessian, but _compute_per_module_hessians() returns (hessian, nsamples) on this branch. The tuple was passed as the Hessian (AttributeError) and per-expert nsamples was dropped. Now unpacked and forwarded inside upstream's fallback structure. - _resolve_gptq_for_rtn_fallback() / _rtn_fallback_result() were left duplicated at module level. Removed. * [docs] note MDBF as unsupported for CPU/GGUF export * [test] unpack calculate_hessian tuple in unfuse_moe test * [style] Apply black formatter * [fix] restore debug level for the quantize_with_qep layer log Merging upstream/develop/v1-3-0 (3939a80) reverted 9d8d5a3's info -> debug on this line, leaving it out of step with the same log in quantize(). * [docs] record shared-code changes in the MDBF changelog entry --------- Co-authored-by: mhsuzu <mhsuzu/miharu.suzuki@compmind.co.jp> Co-authored-by: Copilot <copilot@github.com> Co-authored-by: sikoji <seiichiro.kojio@compmind.co.jp> Co-authored-by: fujisawa-yoshihiko <fujisawa.y.5bdc@m.isct.ac.jp> Co-authored-by: aki916f <akihiro.yoshida@fujitsu.com>
Update README.md and docs/index.md for MDBF See merge request onecomp/onecomp-lab!117
Modify CHANGELOG.md See merge request onecomp/onecomp-lab!116
Add contributors into CHANGELOG.md See merge request onecomp/onecomp-lab!118
add copyright, fix import submodule for test, and add onebit format figur See merge request onecomp/onecomp-lab!115
FKKimura
marked this pull request as ready for review
August 6, 2026 08:15
aki916f
approved these changes
Aug 6, 2026
aki916f
left a comment
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
LGTM
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.