This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, sgemv, sgemm, etc. The performance of these kernels is basically at or near the theoretical limit.

Step-by-step optimization of CUDA SGEMM

Multi-Threaded FP32 Matrix Multiplication on x86 CPUs

Accelerated General (FP32) Matrix Multiplication from scratch in CUDA

General Matrix Multiplication using NVIDIA Tensor Cores

The repository targets the OpenCL gemm function performance optimization. It compares several libraries clBLAS, clBLAST, MIOpenGemm, Intel MKL(CPU) and cuBLAS(CUDA) on different matrix sizes/vendor's hardwares/OS. Out-of-the-box easy as MSVC, MinGW, Linux(CentOS) x86_64 binary provided. 在不同矩阵大小/硬件/操作系统下比较几个BLAS库的sgemm函数性能，提供binary，开盒即用。

maxas Scott Grey's maxas assembler sgemm explaining the (for me) missing parts https://github.com/NervanaSystems/maxas

FastCuda is a handwritten CUDA operator library featuring progressive GEMM and Reduce kernels, cuBLAS benchmarking, and C/C++/Python interfaces for learning, profiling, and performance optimization.

GEMM

A benchmark framework for POWER and x86_64

An alternative SGEMM implementation on AMD Vega Series

a fast sgemm lib with fix 16 enable on arm 32

从 SGEMM 到推理组件的系统性 CUDA 学习主仓，整合四条历史实验路径

针对CUDA上的单精度矩阵乘法做一些比较容易想到的优化

C++17 ONNX inference optimizer + CPU runtime for Apple Silicon. Operator fusion via IR passes, Accelerate AMX-backed sgemm; benched vs ONNX Runtime CPU EP on DistilBERT (1.26x baseline speedup, 6.99x ORT on raw MatMul).

A CPU benchmark suite that shows its work. Six workloads, calibrated measurement windows, barrier-synchronised threads, robust statistics, full machine provenance, and Ed25519-signed results.

Progressive CUDA SGEMM kernel(6) optimization (naive → vectorized) with cross-architecture profiling on RTX 3090 vs RTX 4050, benchmarked using Nsight Compute.

CUDA SGEMM optimization from scratch — six kernels reaching 6.37 TFLOP/s, 67.9× over naive CUDA, and 90% of NVIDIA cuBLAS performance.

CuPy first example computing GEMM with cuBlas, with handwritten cuda kernel and with NumPy-blas

Perform the matrix-matrix operation `C = α*op(A)*op(B) + β*C` where `op(X)` is either `op(X) = X` or `op(X) = X^T`.