PR11: DSP Runtime SDK - #10444
Open
agibsonccc wants to merge 4 commits into
Open
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
Part of the 22-PR split of ag_new_release_updates_2 branch. Merge layer: 3 (native features) Files: 48 See pr-plans/00-master-plan.md for the full split plan and merge order.
This was referenced Jun 15, 2026
…hon wildcard import Remove generated sdx_serving_pb2.py and sdx_serving_pb2_grpc.py protobuf files which should not be committed; regenerate from sdx_serving.proto as needed. Change exit 0 to exit 1 in publish-sdk-jars action so the step properly fails (rather than silently succeeding) when backend=cuda but cuda-version is not provided. Replace wildcard `from .sdx_runtime import *` in __init__.py with explicit named imports, listing all public constants, functions, and classes exported by sdx_runtime to avoid polluting the package namespace.
The canonical SdxRuntime.java lives in nd4j-sdx module. The identical copy under libnd4j/include/dsp/runtime/bindings/java/ was stranded with no pom.xml reference from any build and would diverge silently.
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a versioned SDX/DSP Runtime SDK surface centered on a C ABI (dsp_runtime_c.h) plus multi-language bindings (Swift/Rust/Python/Kotlin/C#), a bundle manifest JSON schema, and GitHub Actions to publish the resulting SDK artifacts.
Changes:
- Added SDX runtime C ABI header (opaque handles, backend/device enums, sized option/report structs, lifecycle API).
- Added language bindings and examples (Swift/Rust/Python/Kotlin/C#) plus a Python REST+gRPC “runner” and tensor transport helpers.
- Added manifest.schema.json for bundle metadata validation and CI composite actions to publish SDK artifacts/JARs.
Reviewed changes
Copilot reviewed 42 out of 42 changed files in this pull request and generated 13 comments.
Show a summary per file| File | Description |
|---|---|
| resources/dsp/manifest.schema.json | Adds JSON Schema for DSP bundle manifests. |
| libnd4j/include/dsp/runtime/README.md | Points readers to ADRs as the source of truth. |
| libnd4j/include/dsp/runtime/dsp_runtime_c.h | Defines the SDX Runtime C ABI v1 (types, structs, functions). |
| libnd4j/include/dsp/runtime/bindings/README.md | Documents available language bindings and runner notes. |
| libnd4j/include/dsp/runtime/bindings/swift/Package.swift | SwiftPM package definition for Swift bindings. |
| libnd4j/include/dsp/runtime/bindings/swift/README.md | Swift binding documentation. |
| libnd4j/include/dsp/runtime/bindings/swift/examples/BasicUsage.swift | Swift usage example. |
| libnd4j/include/dsp/runtime/bindings/swift/Sources/CSdxRuntime/shim.h | Swift C shim header for importing the C ABI. |
| libnd4j/include/dsp/runtime/bindings/swift/Sources/CSdxRuntime/module.modulemap | Module map for importing/linking the runtime library. |
| libnd4j/include/dsp/runtime/bindings/swift/Sources/SdxRuntime/SdxRuntime.swift | Swift wrapper around the C ABI. |
| libnd4j/include/dsp/runtime/bindings/rust/Cargo.toml | Rust crate metadata for the binding. |
| libnd4j/include/dsp/runtime/bindings/rust/README.md | Rust binding documentation and usage sketch. |
| libnd4j/include/dsp/runtime/bindings/rust/build.rs | Rust build script for locating runtime libraries. |
| libnd4j/include/dsp/runtime/bindings/rust/examples/basic_usage.rs | Rust usage example. |
| libnd4j/include/dsp/runtime/bindings/rust/src/lib.rs | Rust FFI + RAII wrapper types. |
| libnd4j/include/dsp/runtime/bindings/python/init.py | Python exports for the ctypes wrapper. |
| libnd4j/include/dsp/runtime/bindings/python/README.md | Python wrapper and runner documentation. |
| libnd4j/include/dsp/runtime/bindings/python/generate_proto.py | Helper to regenerate protobuf stubs. |
| libnd4j/include/dsp/runtime/bindings/python/pyproject.toml | Python packaging config for the bindings. |
| libnd4j/include/dsp/runtime/bindings/python/requirements-runner.txt | Runner dependency pin ranges. |
| libnd4j/include/dsp/runtime/bindings/python/sdx_runtime.py | ctypes wrapper + runtime library auto-discovery. |
| libnd4j/include/dsp/runtime/bindings/python/sdx_sdk_runner.py | FastAPI + gRPC serving runner for the SDK. |
| libnd4j/include/dsp/runtime/bindings/python/sdx_serving.proto | gRPC protocol definition for serving. |
| libnd4j/include/dsp/runtime/bindings/python/sdx_tensor_transport.py | Shared JSON/NPZ tensor transport utilities. |
| libnd4j/include/dsp/runtime/bindings/python/examples/basic_usage.py | Python usage example. |
| libnd4j/include/dsp/runtime/bindings/python/tests/init.py | Python test package marker. |
| libnd4j/include/dsp/runtime/bindings/python/tests/test_sdx_runtime_loader.py | Tests for runtime library discovery and platform detection. |
| libnd4j/include/dsp/runtime/bindings/python/tests/test_sdx_sdk_runner_rest.py | Tests for REST runner behavior. |
| libnd4j/include/dsp/runtime/bindings/python/tests/test_sdx_tensor_transport.py | Tests for tensor transport encode/decode helpers. |
| libnd4j/include/dsp/runtime/bindings/kotlin/build.gradle.kts | Kotlin bindings build definition. |
| libnd4j/include/dsp/runtime/bindings/kotlin/settings.gradle.kts | Kotlin project settings. |
| libnd4j/include/dsp/runtime/bindings/kotlin/README.md | Kotlin binding documentation. |
| libnd4j/include/dsp/runtime/bindings/kotlin/examples/BasicUsage.kt | Kotlin usage example. |
| libnd4j/include/dsp/runtime/bindings/kotlin/src/main/kotlin/org/nd4j/dsp/runtime/SdxRuntime.kt | Kotlin facade over the Java wrapper. |
| libnd4j/include/dsp/runtime/bindings/csharp/SdxRuntime.csproj | .NET project file for the C# binding. |
| libnd4j/include/dsp/runtime/bindings/csharp/README.md | C# binding documentation. |
| libnd4j/include/dsp/runtime/bindings/csharp/SdxRuntime.cs | C# P/Invoke binding + library resolver + wrappers. |
| libnd4j/include/dsp/runtime/bindings/csharp/examples/BasicUsage.csproj | C# example project. |
| libnd4j/include/dsp/runtime/bindings/csharp/examples/BasicUsage.cs | C# usage example. |
| .github/actions/publish-sdx-runtime-sdk/action.yml | Composite action to build and upload SDK artifacts. |
| .github/actions/publish-sdk-jars/action.yml | Composite action to collect/upload platform-classified JARs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Sorry, something went wrong.
| #ifndef CSDXRUNTIME_SHIM_H | ||
| #define CSDXRUNTIME_SHIM_H | ||
|
|
||
| #include "../../../../include/dsp_runtime_c.h" |
Comment on lines
+39
to
+40
| v.shape = UnsafePointer(self.shapeBuffer.baseAddress) | ||
| v.rank = Int32(shape.count) |
Comment on lines
+58
to
+62
| var runtime: OpaquePointer? | ||
| var options = sdx_runtime_options_t() | ||
| let status = withUnsafePointer(to: &options) { optPtr in | ||
| sdxCreateRuntime(optPtr, &runtime) | ||
| } |
Comment on lines
+214
to
+217
| public func executionReport() throws -> sdx_execution_report_t { | ||
| var report = sdx_execution_report_t() | ||
| let status = sdxGetExecutionReport(handle, &report) | ||
| guard status == SDX_STATUS_OK else { |
Comment on lines
+117
to
+120
| public final class SdxModel { | ||
| private unowned let runtime: SdxRuntime | ||
| private var handle: OpaquePointer? | ||
|
|
Comment on lines
+16
to
+22
| # Prepare input as a numpy array | ||
| input_data = np.array([[1.0, 2.0, 3.0, 4.0]], dtype=np.float32) | ||
|
|
||
| # Run inference | ||
| outputs = ctx.run([input_data]) | ||
| print(f"Output shape: {outputs[0].shape}") | ||
| print(f"Output values: {outputs[0]}") |
Comment thread
libnd4j/include/dsp/runtime/bindings/rust/src/lib.rs
Comment on lines
+93
to
+104
| #[repr(C)] | ||
| #[derive(Clone, Copy, Debug)] | ||
| pub struct sdx_execution_report_t { | ||
| pub struct_size: u32, | ||
| pub requested_backend: i32, | ||
| pub applied_backend: i32, | ||
| pub status_code: i32, | ||
| pub used_fallback: i32, | ||
| pub execution_time_ns: u64, | ||
| pub requested_gpu_target: i32, | ||
| pub applied_gpu_target: i32, | ||
| } |
Comment thread
libnd4j/include/dsp/runtime/bindings/csharp/SdxRuntime.cs
Comment on lines
+101
to
+120
| [StructLayout(LayoutKind.Sequential)] | ||
| public struct SdxExecutionReport | ||
| { | ||
| public uint struct_size; | ||
| public int requested_backend; | ||
| public int applied_backend; | ||
| public int status_code; | ||
| public int used_fallback; | ||
| public ulong execution_time_ns; | ||
| public int requested_gpu_target; | ||
| public int applied_gpu_target; | ||
|
|
||
| public static SdxExecutionReport Default() | ||
| { | ||
| return new SdxExecutionReport | ||
| { | ||
| struct_size = (uint)Marshal.SizeOf<SdxExecutionReport>() | ||
| }; | ||
| } | ||
| } |
Comment thread
libnd4j/include/dsp/runtime/bindings/python/sdx_runtime.py
Comment on lines
+375
to
+385
| class ExecutionReport(ctypes.Structure): | ||
| _fields_ = [ | ||
| ("struct_size", ctypes.c_uint32), | ||
| ("requested_backend", ctypes.c_int32), | ||
| ("applied_backend", ctypes.c_int32), | ||
| ("status_code", ctypes.c_int32), | ||
| ("used_fallback", ctypes.c_int32), | ||
| ("execution_time_ns", ctypes.c_uint64), | ||
| ("requested_gpu_target", ctypes.c_int32), | ||
| ("applied_gpu_target", ctypes.c_int32), | ||
| ] |
Comment thread
.github/actions/publish-sdk-jars/action.yml
| echo " Collected: ${BASENAME} (from ${jar})" | ||
| FOUND=1 | ||
| fi | ||
| done < <(find "${GITHUB_WORKSPACE}" -path "*/target/${MODULE}-*.jar" -type f -print0 2>/dev/null | head -z -n 5) |
Copy link
Copy Markdown
Contributor
Author
Architecture OverviewThis PR defines the public C ABI and multi-language SDK for the DSP runtime, enabling external consumers to compile and execute dynamic shape plans without depending on the full ND4J/SameDiff stack. The C header uses a versioned ABI with forward-compatible sized structs, and the 6 language bindings all map to the same 15-function lifecycle. Highlights
|
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 11 of 22 PRs in the ag_new_release_updates_2 branch split. Merge after Layer 2 (native core ops + helpers).
What Changed
C ABI header — libnd4j/include/dsp/runtime/ (1 file)
DSP JNI bridge — libnd4j/include/dsp/ (1 file)
Java binding — bindings/java/ (4 files)
Kotlin binding — bindings/kotlin/ (4 files)
Python binding — bindings/python/ (12 files)
Rust binding — bindings/rust/ (5 files)
Swift binding — bindings/swift/ (6 files)
C# binding — bindings/csharp/ (4 files)
DSP bundle manifest — resources/dsp/ (1 file)
GitHub Actions — .github/actions/ (2 files)
Dependencies
Merge Order
These 22 PRs must merge in layer order. Each layer depends on the layers above it being merged first. PRs within the same layer are independent and can merge in parallel.