Fix: 3D/OpenGL sketches crash with UnsatisfiedLinkError on Linux — jspawnhelper missing exec permission

Closes #1537

Summary

3D/OpenGL sketches (P3D, P2D, OBJ-loading, anything using JOGL) crashed on launch with UnsatisfiedLinkError. The cause was lib/jspawnhelper losing its executable permission during the includeJdk Gradle copy task. Without the exec bit, the bundled JDK could not fork+exec child processes, which broke JOGL's temp-directory validation and prevented native libraries from loading.

The crash

Warning: Caught Exception while retrieving executable temp base directory:
java.io.IOException: Could not determine a temporary executable directory
    at com.jogamp.common.util.IOUtil.getTempDir(IOUtil.java:1401)
    at com.jogamp.common.util.cache.TempFileCache.<clinit>(TempFileCache.java:84)
    at com.jogamp.common.os.Platform$1.run(Platform.java:313)
java.lang.UnsatisfiedLinkError: 'boolean jogamp.common.jvm.JVMUtil.initialize(java.nio.ByteBuffer)'
    at jogamp.common.jvm.JVMUtil.initialize(Native Method)
    at com.jogamp.common.os.Platform.<clinit>(Platform.java:290)
    at com.jogamp.opengl.GLProfile.<clinit>(GLProfile.java:151)
    at processing.opengl.PSurfaceJOGL.initGL(PSurfaceJOGL.java:204)
A library used by this sketch relies on native code that is not available.

2D sketches (Java2D renderer) ran fine because they never call Runtime.exec().

Root cause

jspawnhelper is a small ELF binary at lib/jspawnhelper that JDK 17's ProcessImpl execs (via vfork + execve) to spawn child processes. The includeJdk Gradle task copied the JDK into resources/jdk/ but ran its file-permission loop at configuration time, before the copy happened, on an empty directory. The loop set writable and readable but never executable. Since the directory was empty at config time, no files were affected. The Gradle Copy task's dirPermissions spec only applied to directories, not files. Result: jspawnhelper (and jexec, and everything in bin/) landed with mode 0644 instead of 0755.

When the sketch JVM tried to exec jspawnhelper, the kernel returned EACCES (error=13, Permission denied). This broke every Runtime.exec() call in the sketch process. JOGL's IOUtil.testDirExec() writes a test script to a temp directory and actually execve()s it to validate that the temp dir allows execution. With jspawnhelper non-executable, that test failed for every candidate directory, so TempJarCache never initialized, native libraries were never extracted, and the sketch crashed with UnsatisfiedLinkError.

The same issue affected the IDE runtime at lib/runtime/lib/jspawnhelper.

How I found it

I used Kimi K2.7 Code to help find the root cause. This took several wrong turns before the actual cause surfaced. After I tried everything, i finally decided to just compare the Build-In JDK with a fresh Temurin 17 from Adoptium file by file. It turn out that there are some file that have difference permission:

jspawnhelper was the critical one. chmod 755 jspawnhelper on the bundled JDK immediately fixed Runtime.exec().

Why includeJdk didn't preserve the exec bit

The original code:

tasks.register<Copy>("includeJdk") {
    from(Jvm.current().javaHome.absolutePath)
    destinationDir = composeResources("jdk").get().asFile

    dirPermissions { unix("rwx------") }
    fileTree(destinationDir).files.forEach { file ->   // runs at CONFIG time, on EMPTY dir
        file.setWritable(true, false)
        file.setReadable(true, false)                 // no setExecutable()
    }
}

The fileTree(destinationDir).files.forEach block ran at Gradle configuration time, before the Copy task executed. destinationDir was empty, so the loop processed zero files. Even if it had run after the copy, it only set writable and readable, never executable. The dirPermissions spec only applied to directories.

The fix

app/build.gradle.kts — includeJdk task (root cause)

Moved the permission loop into a doLast block (runs after the copy) and re-applied the exec bit from the source JDK for files that should be executable:

doLast {
    fileTree(destinationDir).files.forEach { file ->
        val sourceFile = File(Jvm.current().javaHome.absolutePath,
            file.relativeTo(destinationDir).path)
        if (sourceFile.canExecute()) {
            file.setExecutable(true, false)
        }
        file.setWritable(true, false)
        file.setReadable(true, false)
    }
}

app/build.gradle.kts — setExecutablePermissions task (belt and suspenders)

Added **/runtime/**/bin/** and **/runtime/**/lib/** patterns so the IDE runtime's jspawnhelper is also covered:

include("**/runtime/**/bin/**")
include("**/runtime/**/lib/**")

app/src/processing/app/Platform.java — JAVA_HOME / JDK_HOME support (Extra)

Added env var checks inside the existing "If the JDK is set in the environment" fallback block, before the java.home system property fallback. This lets users override the sketch JDK when the bundled resources/jdk is absent (e.g., portable installs without a bundled JDK):

for (String envKey : new String[] { "JAVA_HOME", "JDK_HOME" }) {
    String envPath = System.getenv(envKey);
    if (envPath != null && !envPath.isEmpty()) {
        File envHome = new File(envPath);
        if (new File(envHome, "bin/java" + (Platform.isWindows() ? ".exe" : "")).canExecute()) {
            return envHome;
        }
    }
}

The bundled resources/jdk still takes priority. JAVA_HOME/JDK_HOME is only checked when the bundle is missing.

Tests

Test sketch: a P3D project that loads an .obj file (Rubik's cube model with loadShape()).

All tests run with processing cli --sketch=<path> --run automatically using Kimi K2.7 Code under my watch . The X11Util.Display shutdown messages in the output confirm JOGL/OpenGL initialized and rendered before clean exit.

Before the fix (jspawnhelper 0644), the same sketch crashed with UnsatisfiedLinkError: JVMUtil.initialize. So this pr are tested, and verified solves the issue without side effects so far.

tbh this is just a small change covering 2 files changed, 29 insertions(+), and 4 deletions(-).

Environment

• Arch Linux, kernel 7.0.5-arch1-1-g14
• Processing 4.5.5 (rev 1433)
• Bundled sketch JDK: Temurin 17.0.19+10
• System JDK: OpenJDK 21.0.11
• JOGL / gluegen 2.6.0