i think i fucked up my commit message, stand by!

My only concern is you may face some problems with downloading and unpacking 10th of gigabytes of tar zst archives from inside of JVM.

Yeah but that is the problem of the user to allocate more resources I suppose.

Is it possible to stream to disk? Then the cluster might not need all the RAM to download? Not sure I understand the issue, just popping my head up from a Lyft :)

Sem's original code does stream the compressed file to disk:

      val connection = ldbcURL(name).openConnection()
      val inputStream = connection.getInputStream
      val outputStream = Files.newOutputStream(archivePath)
      val buffer = new Array[Byte](bufferSize)
      var bytesRead = 0
      while ({ bytesRead = inputStream.read(buffer); bytesRead } != -1) {
        outputStream.write(buffer, 0, bytesRead)
      }
      inputStream.close()
      outputStream.close()