Fix BigQueryEmulatorContainer to work with BigQuery write requests - #7779
Fix BigQueryEmulatorContainer to work with BigQuery write requests#7779prismec wants to merge 1 commit into
Conversation
Sorry, something went wrong.
… and Grpc ports as exposed to the docker host system. Add support for an init function after the container has started. Fixes goccy/bigquery-emulator#160 (comment)
| addFixedExposedPort(httpPort, httpPort); | ||
| addFixedExposedPort(grpcPort, grpcPort); |
There was a problem hiding this comment.
anti-pattern. See https://www.atomicjar.com/2023/11/testcontainers-best-practices/
Sorry, something went wrong.
There was a problem hiding this comment.
The ports are not fixed. They are chosen dynamically by default. See also
private int httpPort = TestSocketUtils.findAvailableTcpPort();
As we need to ensure that the emulator running in the container must have the same port as exposed to the Docker host, we can't rely on Docker assigning a free random port (as done by default in testcontainers as far as I can judge from my investigation). I therefore selected a random free port directly in Java.
Sorry, something went wrong.
|
Hi @prismec, thanks for the PR. But, I think this is an issue in https://github.com/goccy/bigquery-emulator/ rather than in Testcontainers. Also, the PR contains fixed ports which is not recommended. Regarding to the command flags, those can be overridden using withCommand(...). I think exposing all the flags provided by certain services is not feasible in the long term. Before raising a PR, consider raising a discussion to talk about the potential fix, enhancement. We value the time took to contribute. |
Sorry, something went wrong.
|
I fear that this is not an issue in https://github.com/goccy/bigquery-emulator/. I fear they are doing it right. The emulator is started on a given port and it returns the right headers as defined by the BigQuery API. How would they know that they are running in a container with a port mapping ? See also the linked comment. In the current state, the container implementation is basically useless to us (and I guess to many others) as big parts of the BigQuery API cannot be used. Everybody would need to workaround it by selecting a random free port and specifying withCommand, which basically throws you back to using a generic container with fixed ports. |
Sorry, something went wrong.
|
I agree on your comment on the exposure of the flags. |
Sorry, something went wrong.
The image ghcr.io/goccy/bigquery-emulator currently cannot be used with a docker host port that differs from the container's Http and Grcp ports. You can find more detailed information on the underlying problem in this comment.
The problem is also specified in the test org.testcontainers.containers.BigQueryEmulatorContainerTest#testThatTheContainerCanBeUsedWriteDataToBigQuery.
The fix is to change the container command to use the same ports that are exposed to the docker host system. In order to keep it convenient I've added some utilities to select a random, free docker host port.
New features introduced by this pull request