JRuby versions 9.4.15.0, 10.1.1.0, and head (10.1.2.0-SNAPSHOT) behave differently to MRI, both locally and within GitHub Actions, when evaluating code with a require_relative call and a filename argument.
Maybe other versions also behave like this, but I have not tested them.

Consider the following minimal reproducible example:

File.write('file.rb', "puts 'success'")
eval("require_relative 'file'", TOPLEVEL_BINDING, 'otherfile.rb')

It is expected to print "success", but it only works if there is a file named as otherfile.rb present.
The file has no need to exist, as the filename provided is used for file paths and to improve backtrace messages.
Instead it prints the following error message:

Errno::ENOENT: No such file or directory - /home/mau/Desktop/test/otherfile.rb
          realpath at org/jruby/RubyFile.java:988
  require_relative at org/jruby/RubyKernel.java:1216
            <main> at otherfile.rb:1
              eval at org/jruby/RubyKernel.java:1281
            <main> at test.rb:2