| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
The reduction routines switch the rocBLAS handle to device pointer mode and switch it back once the call returns. rocblas_native_func throws on any status other than success, which skips the reset. Handles are cached per thread and reused, so the handle stays in device pointer mode for every later call on that thread. Routines that pass a host address for their scalar arguments then have that address dereferenced on the device. rot is the most exposed: it forwards &c and &s and relies on the handle already being in host pointer mode. On a discrete GPU this is an illegal address, which matches the intermittent failure of RotTests.RealSinglePrecision reported in uxlfoundation#486. Set the mode through an RAII guard so it is restored while unwinding. The guard restores the previous mode rather than forcing host mode, so it nests safely. Verified on an AMD Instinct MI210: after provoking a failure inside a guarded region, a following rot aborts with hipErrorIllegalAddress in 10 of 10 runs without the guard and in 0 of 10 with it. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
This is a real bug fix and a genuine net simplification — +37/-53 while making the code more correct is the right direction. Approving.
The problem is well identified. rocBLAS handles are cached and reused across calls, so if rocblas_native_func threw between the set_pointer_mode(device) and the manual set_pointer_mode(host) at the bottom, the handle stayed in device mode for every subsequent caller — and the next routine passing a host address for a scalar argument would have rocBLAS dereference it on the device. That's a nasty, action-at-a-distance failure mode, and the old comment block ("we need to reset this to the default value in order to avoid invalid memory accesses") shows the hazard was known but only handled on the happy path. RAII is exactly the right fix.
I verified the conversion is complete — I checked every remaining rocblas_set_pointer_mode occurrence in rocblas_level1.cpp at your head commit, and all 11 hits are inside comments (plus one long-dead commented-out call in rot). No live call site was left on the old manual pattern, and level2/level3/extensions/batch never used it. Nice job not leaving the cleanup half-done — that's usually where these go wrong.
Details I liked: deleting copy/assign, and seeding previous_ to rocblas_pointer_mode_host before the get so a failed query still degrades to the sane default rather than to garbage.
Two small things, neither blocking:
Unchecked rocBLAS status. The constructor ignores the return of both rocblas_get_pointer_mode and rocblas_set_pointer_mode, while the rest of this backend routes rocBLAS calls through the error-checking helpers. The destructor obviously can't throw, so leave that one alone — but it'd be more consistent to check in the constructor. The old code ignored these too, so this isn't a regression either way.
Behavior change worth naming explicitly. The old code unconditionally forced the mode back to host; the guard restores whatever was there on entry. Since every call site now goes through the guard, the invariant holds and I think your version is the more principled one. But it does remove an accidental self-healing property: if some other path ever leaked device mode, the old code would have normalized it back and the guard won't. I'm fine with this — just flagging it so the change is deliberate rather than incidental.
One thing I noticed that is out of scope for this PR — no action wanted here: the USM overloads of dot, sdsdot, rotg, rotm, and rotmg never set device pointer mode at all, while the USM asum/iamax/iamin/nrm2 do. That asymmetry predates your change. If it's a real gap it deserves its own issue; please don't grow this PR to cover it.
Sorry, something went wrong.
There was a problem hiding this comment.
Approved: sound, focused fix with strong rationale and validation. The CI testing gap is worth noting but non-blocking.
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Summary
Fixes the pointer-mode leak that can produce the intermittent RotTests.RealSinglePrecision failure reported in #486.
The reduction routines (asum, nrm2, dot, dotc, dotu, rotg, rotm, rotmg, iamax, iamin) switch the rocBLAS handle to device pointer mode and switch it back after the call:
rocblas_native_func throws a rocblas_error on any status other than rocblas_status_success, so the reset is skipped. Handles are cached in thread-local storage and reused, so the handle stays in device pointer mode for every subsequent call on that thread.
Routines that pass a host address for their scalar arguments then have that address dereferenced on the device. rot is the most exposed one — it forwards (rocDataType2*)&c and (rocDataType3*)&s and never sets the pointer mode itself, relying entirely on the handle already being in host mode. On a discrete GPU, where host memory is not device-addressable, that is an illegal address.
This replaces the manual set/reset pairs at all 13 sites with an RAII guard, so the mode is restored during stack unwinding as well as on the normal path. The guard restores the previous mode rather than unconditionally forcing host mode, so it is safe to nest.
Validation
On an AMD Instinct MI210 (gfx90a, ROCm 7.1.1, DPC++), provoking a failure inside a guarded region and then calling rot:
Without the guard the rot call fails with hipErrorIllegalAddress (700) and leaves the queue unrecoverable. With it, rot returns correct results and a following asum is correct. Both arms take the identical first exception on the same queue, so the guard is the only difference.
Reproducer:
Also ran the BLAS test suite on the same MI210 with the fix applied: 740 tests pass with no failures.
Caveats
Test plan
Made with Cursor