How do you specify the arrow version used to build/test with arrow-java (so that I know it has my fix)? Does it always use the latest?

We use the latest release for push/pull request:

We use the main branch for schedule:

We need to update

Thanks. I updated vcpkg as well now.

Looks like one more minor CI nit:

  Error:  /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/FilterProjectTest.java:37:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]

There's others - are you able to run the build locally at all to test?

   Error:  /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/FilterTest.java:37:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]
  Error:  /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/ProjectorDecimalTest.java:45:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]

Now its failing with "Gandiva Failed to make LLVM module due to Could not create LLJIT instance: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ]" which is what should be fixed in my other change in apache/arrow.

Seems to be using the 18.2 jars: /home/runner/.m2/repository/org/apache/arrow/arrow-avro/18.2.0/

Ah, the CI here uses the latest released version of the C++ libraries, not the development version...

We could either make the skips conditional on the version of the C++ code somehow, or otherwise just always test against the dev release?

C++ version must not be related.
apache/arrow#45114 just updated LLVM version.

But we're building 19.0.0, not main: https://github.com/apache/arrow-java/actions/runs/13298859368/job/37137015507?pr=595

Ah, I thought that updating VCPKG in .env is enough to use LLVM 18 but we refer ci/vcpkg in apache/arrow here...:

The log shows that we use LLVM 17:

https://github.com/apache/arrow-java/actions/runs/13298859368/job/37137015507?pr=595#step:10:1663

  -- Found LLVMAlt: 17.0.2

OK. Let's use main for now:

diff --git a/.github/workflows/rc.yml b/.github/workflows/rc.yml
index faad5983..f2666a31 100644
--- a/.github/workflows/rc.yml
+++ b/.github/workflows/rc.yml
@@ -299,13 +299,13 @@ jobs:
         shell: bash
         run: |
           tar -xf apache-arrow-java-*.tar.gz --strip-components=1
-      - name: Download the latest Apache Arrow C++
-        if: github.event_name != 'schedule'
-        shell: bash
-        run: |
-          ci/scripts/download_cpp.sh
+      # - name: Download the latest Apache Arrow C++
+      #   if: github.event_name != 'schedule'
+      #   shell: bash
+      #   run: |
+      #     ci/scripts/download_cpp.sh
       - name: Checkout Apache Arrow C++
-        if: github.event_name == 'schedule'
+        # if: github.event_name == 'schedule'
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           repository: apache/arrow

Ah, I see what you meant by the original comment now...

I'll try your suggested change @kou

Latest failures were because I only updated the windows build in rc.yml. Just fixed that for mac and linux.

Oh, sorry. It's my fault...
We just need to use main only for Linux because Linux build only uses vcpkg to install LLVM.

Strange, it still seems to be using the old vcpkg.

Does arrow-java need something like this? https://github.com/apache/arrow/blob/main/ci/scripts/install_vcpkg.sh

2025-02-14T00:07:03.2003955Z CMake Warning at /opt/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package):
2025-02-14T00:07:03.2004967Z   Could not find a configuration file for package "LLVM" that is compatible
2025-02-14T00:07:03.2005618Z   with requested version "19.1".

....

2025-02-14T00:07:03.2600773Z -- Found LLVMAlt: 17.0.2
2025-02-14T00:07:03.2601530Z -- Using LLVMConfig.cmake in: /opt/vcpkg/installed/amd64-linux-static-release/share/llvm

Ah, we need to expire a cache when we change vcpkg version. Could you try this?

diff --git a/compose.yaml b/compose.yaml
index 58a19676..b125c3c9 100644
--- a/compose.yaml
+++ b/compose.yaml
@@ -92,12 +92,12 @@ services:
     # Usage:
     #   docker compose build vcpkg-jni
     #   docker compose run vcpkg-jni
-    image: ${REPO}:${ARCH}-vcpkg-jni
+    image: ${REPO}:${ARCH}-vcpkg-jni-${VCPKG}
     build:
       context: .
       dockerfile: ci/docker/vcpkg-jni.dockerfile
       cache_from:
-        - ${REPO}:${ARCH}-vcpkg-jni
+        - ${REPO}:${ARCH}-vcpkg-jni-${VCPKG}
       args:
         base: ${ARROW_REPO}:${ARCH}-python-${PYTHON}-wheel-manylinux-2014-vcpkg-${VCPKG}
     volumes:

Ok, just pushed a change to expire the cache. Hopefully it works.

Looks like that worked!