Summary

tests/unit/ops/accelerators/test_accelerator_forward.py has three transformer-kernel test methods, but only TestCUDAForwardSmallBatchSize.test_forward_with_small_bsz guards on TransformerBuilder compatibility. TestCUDAForward.test_forward and TestCUDAForwardStochastic.test_forward_stochastic do not.

These tests build and run the CUDA-only DeepSpeedTransformerLayer. The module has a top-level fp16 gate:

if torch.half not in get_accelerator().supported_dtypes():
    pytest.skip(..., allow_module_level=True)

On CPU, CPU_Accelerator.supported_dtypes() includes float16 only when is_fp16_supported() is true, which resolves to torch.ops.mkldnn._is_mkldnn_fp16_supported() -- a runtime CPU-capability probe. GitHub's ubuntu-24.04 runners are heterogeneous:

• On most runners the probe returns False, the module is skipped, and cpu-torch-latest is green (this is why scheduled master runs pass).
• On a runner whose CPU advertises oneDNN fp16 support, the probe returns True, the module is not skipped, and test_forward / test_forward_stochastic run and fail trying to JIT-build a CUDA kernel on CPU (the sequential pytest pass in cpu-torch-latest).

This makes cpu-torch-latest flaky depending on which runner the job lands on.

Fix

Add the existing @pytest.mark.skipif(not deepspeed.ops.__compatible_ops__[TransformerBuilder.NAME], ...) guard (already used by test_forward_with_small_bsz) to test_forward and test_forward_stochastic, so they are skipped whenever the CUDA TransformerBuilder op is not compatible on the host.

Test plan

• cpu-torch-latest (including the -m sequential pass) is green regardless of runner CPU.
• On CUDA hosts where TransformerBuilder is compatible, the tests still run as before.

Made with Cursor