Following up here on a question about authentication for a private repo hosted at docker hub that I posted on Slack.
For some context, credentials are stored in ~/.docker/config.json in the auth field (this is actually base64(username:password)) eg:

{
    "auths": {
        "https://index.docker.io/v1/": {
            "auth": "abcdef=",
            "email": "peter.vr@acme.com"
        }
    },
    "HttpHeaders": {
        "User-Agent": "Docker-Client/18.03.1-ce (darwin)"
    }
}

We supply image names such acme/microservice:0.1 to testcontainers. There are also public images from Docker Hub (redis, postgres etc) and implicitly from quay.io (ryuk). Now, looking at the following code:

https://github.com/testcontainers/testcontainers-java/blob/master/core/src/main/java/org/testcontainers/utility/RegistryAuthLocator.java#L80

reposName ends up being "" (empty) and it falls back to defaultAuthConfig which I believe is controlled by ~/.docker-java.properties (ala docker-java project), which does not exist and hence it fails trying to download the image:

2018-08-05 14:22:22,061 ERROR [testcontainers-netty-1-13] c.g.d.c.a.ResultCallbackTemplate: Error during callback
com.github.dockerjava.api.exception.NotFoundException: {"message":"pull access denied for acme/microservice, repository does not exist or may require 'docker login'"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:103)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
	at org.testcontainers.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)

In contrast, the default behaviour of docker pull in the absence of a registry host, is to pull from docker hub and as such matches the URL https://index.docker.io/v1/ in ~/.docker/config.json - this does not appear to be the default behaviour of testcontainers.

I would like to see testcontainers choosing the auth from ~/.docker/config.json in the https://index.docker.io/v1/ stanza within findExistingAuthConfig(config, ""). To pursue this behaviour I put together this PR: #819 - I ran into a number of dead ends:

1. Just replacing the reposName with index.docker.io in case it's empty does not work, docker-java explicitly checks for that hostname and errors out (why???? it's completely valid using the docker command line docker pull index.docker.io/acme/microservice:0.1)
2. Modifying the auth lookup to find the stanza for index.docker.io in config.json (as in this PR: WIP: Add support for docker hub private registry credentials #819) does not work either, from what I can tell, docker-java does not decompose the auth field (into username/password) before creating the JSON authentication header as per https://docs.docker.com/engine/api/v1.37/#section/Authentication so credentials are not being sent properly:

2018-08-05 14:33:08,776 INFO  [pool-6-thread-3] ?.0.112]: effective auth config [AuthConfig[username=<null>,password=<null>,email=<null>,registryAddress=https://index.docker.io/v1/,auth=abcdef=,registrytoken=<null>]]
2018-08-05 14:33:09,917 ERROR [testcontainers-netty-1-15] c.g.d.c.a.ResultCallbackTemplate: Error during callback
com.github.dockerjava.api.exception.NotFoundException: {"message":"pull access denied for acme/microservice, repository does not exist or may require 'docker login'"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:103)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)

The problems with using docker-java.properties are:

1. Only supports a single registry URL, it seems these credentials are aimed at pushing to a single private registry (not really for pulling from multiple (private) registries)
2. If credentials are set here, testcontainers fails to pull ryuk:0.2.2 from quay.io - I have not confirmed but I wonder if it's trying to the use the credentials configured in docker-java.properties (for index.docker.io) for quay.io - a bug?

2018-08-05 14:43:34,343 ERROR [testcontainers-netty-1-5] c.g.d.c.a.ResultCallbackTemplate: Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: {"message":"Get https://quay.io/v2/testcontainers/ryuk/manifests/0.2.2: unauthorized: Invalid Username or Password"}

	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:109)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
	at org.testcontainers.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)

The most confusing thing - why has nobody else reported this? Am I just doing something horribly wrong? :)