Block or report vuiseng9
Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.
You must be logged in to block users.
Contact GitHub support about this user’s behavior. Learn more about reporting abuse.
Report abuseDistributed & Parallel
- Megatron, Transformed! A Hands-on Tutorial on Replicating Empirical Trends in Distributed Training and Model Parallelism. 60+ runs across DP → ZeRO → TP → SP → CP → PP → VPP → EP.
- MLPerf Training Rundowns: v5.1 (Nov'25) on Llama 3.1-8B, Flux.1, and v6.0 (June'26) on newly-added MoE DeepSeek-V3, GPT-OSS.
- MoE training with EP via PyTorch Symmetric Memory (mempool and zero-copy). Nsys profiles and benchmarks included.
Narrow Precision Training
- Quantized Training in FP4(8): Concepts and Pytorch Implementation using cuBLASLt and Microxcaling.
- PoC nvfp4 forward + mxfp8 backward recipe in Transformer Engine, 2x faster than nvfp4-QAT.
Modeling Front
- moe-lab: 25+ design ablations you can run at home. Notably, DeepSeek-V3 load-balance biasing turns out to be surprisingly effective. Autograded grouped_mm speeds up >30% on average. Ablated LatentMoE, a 2026 technique used in Nemotron-3, MAI, Kimi K3.
Model Optimization for Efficient Inference
-
Post-Training Statistical Calibration for Higher Activation Sparsity, [ENLSP 2024 Spotlight 7, Paper, Oral, Code, Integrated]
-
Pre-LLM explosion — Unified HuggingFace Trainer for Joint Pruning, Quantization, and Distillation (JPQD), integrating OpenVINO NNCF and runtime. 16× more BERT serving throughput on Xeon Sapphire Rapids. See MLPerf Inference 3.0 submission. Applicable to vision, audio models.
Perhaps useful: dlbp, dockerhub, HuggingFace
Pinned Loading
If the problem persists, check the GitHub status page or contact support.