Start Container Constructs Incorrect POST · Issue #615 · docker-java/docker-java · GitHub

In docker-java-3.0.0, the StartContainerCmdExec class sends a POST to the Docker API with garbage in the request body, resulting in an HTTP 400 or HTTP 500 (depending on the version of Docker.)

This Scala program reproduces the error reliably (against Docker 1.12):

package com.github.m4dc4p.dockerjava

object Main extends App {

  import com.github.dockerjava.core._
  import com.github.dockerjava.api._

  val docker: DockerClient = DockerClientBuilder.getInstance(
    DockerClientConfig.createDefaultConfigBuilder()
      // .withDockerHost("tcp://127.0.0.1:2375") // uncomment for windows
      .withApiVersion("1.21")
      .build())
    .build()

  val container = docker.createContainerCmd("mysql/mysql-server:latest")
    .withEnv("MYSQL_ALLOW_EMPTY_PASSWORD=yes")
    .exec()

  docker.startContainerCmd(container.getId).exec() // HTTP 500/HTTP 400
}

Here is a trace of the relevant output. I marked the garbage in the body of the POST request with ***:

764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "POST /v1.21/containers/34e03000dab1be0bc41cd5acd55dae34aa2eb3b743c5c92b3a9c150209a42452/start HTTP/1.1[\r][\n]"
764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "Accept: application/json[\r][\n]"
764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "Content-Type: application/json[\r][\n]"
764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "User-Agent: Jersey/2.11 (Apache HttpClient 4.3.1)[\r][\n]"
764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "Transfer-Encoding: chunked[\r][\n]"
764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "Host: 127.0.0.1:2375[\r][\n]"
764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
764  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "[\r][\n]"
***765  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "2[\r][\n]"
***765  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "{}[\r][\n]"
***765  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "0[\r][\n]"
765  [main] DEBUG org.apache.http.wire  - http-outgoing-0 >> "[\r][\n]"
768  [main] DEBUG org.apache.http.wire  - http-outgoing-0 << "HTTP/1.1 500 Internal Server Error[\r][\n]"
768  [main] DEBUG org.apache.http.wire  - http-outgoing-0 << "Content-Length: 4[\r][\n]"
768  [main] DEBUG org.apache.http.wire  - http-outgoing-0 << "Content-Type: text/plain; charset=utf-8[\r][\n]"
768  [main] DEBUG org.apache.http.wire  - http-outgoing-0 << "Date: Tue, 21 Jun 2016 10:27:16 GMT[\r][\n]"
768  [main] DEBUG org.apache.http.wire  - http-outgoing-0 << "Server: Docker/1.12.0-rc2 (linux)[\r][\n]"
768  [main] DEBUG org.apache.http.wire  - http-outgoing-0 << "X-Content-Type-Options: nosniff[\r][\n]"
768  [main] DEBUG org.apache.http.wire  - http-outgoing-0 << "[\r][\n]"
768  [main] DEBUG org.apache.http.headers  - http-outgoing-0 << HTTP/1.1 500 Internal Server Error

Note that removing the API version results in an HTTP 400.

The .../start endpoint supports a (deprecated) HostConfig JSON blob in the body, and docker-java must have supported that at some point. However, the StartContainerCmd interface does not allow you to add such a thing to the request, but StartContainerCmdExec contains some vestige of the old behavior and tries to add a JSON blob to the post body. The fix is simple - pass null to the post request in com.github.dockerjava.jaxrs.StartContainerCmdExec#execute.

Thanks for reporting. Thats definitely a bug but it works with older remote API 1.23 therefore it wasn't found already.

Why bug? HostConfig isn't supported. In connection set apiVersion to 1.23 and everything would continue working. 1.24 support will be added later.

One would call it a bug because we sent an empty json object with the POST which is wrong for v1.22 also.

bitsofinfo posted this in our ISSUES and as such we're able to get docker-java working on mac with this hack of sorts ... which is great. However, docker on mac is currently on 1.12 and suffers from this same issue. Trying to do as @KostyaSha noted above, and setting the apiVersion to an earlier version (went all the way back to 1.21) results in the below exception. Any ideas? Maybe a docker/mac bug of sorts?

Caused by: com.github.dockerjava.api.exception.InternalServerErrorException: EOF

at com.github.dockerjava.jaxrs.filter.ResponseStatusExceptionFilter.filter(ResponseStatusExceptionFilter.java:53)
at org.glassfish.jersey.client.ClientFilteringStages$ResponseFilterStage.apply(ClientFilteringStages.java:134)
at org.glassfish.jersey.client.ClientFilteringStages$ResponseFilterStage.apply(ClientFilteringStages.java:123)
at org.glassfish.jersey.process.internal.Stages.process(Stages.java:171)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:251)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:667)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:664)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:424)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:664)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:424)
at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:333)
at com.github.dockerjava.jaxrs.StartContainerCmdExec.execute(StartContainerCmdExec.java:29)
at com.github.dockerjava.jaxrs.StartContainerCmdExec.execute(StartContainerCmdExec.java:14)
at com.github.dockerjava.jaxrs.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:23)
at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
at com.github.dockerjava.core.command.StartContainerCmdImpl.exec(StartContainerCmdImpl.java:46)
at com.github.dockerjava.api.command.StartContainerCmd$exec.call(Unknown Source)
at com.bmuschko.gradle.docker.tasks.container.DockerStartContainer.runRemoteCommand(DockerStartContainer.groovy:23)
at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask$_start_closure1.doCall(AbstractDockerRemoteApiTask.groovy:57)
at com.bmuschko.gradle.docker.utils.DockerThreadContextClassLoader.withClasspath(DockerThreadContextClassLoader.groovy:53)
at com.bmuschko.gradle.docker.utils.ThreadContextClassLoader$withClasspath.call(Unknown Source)
at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask.runInDockerClassPath(AbstractDockerRemoteApiTask.groovy:62)
at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask$runInDockerClassPath.callCurrent(Unknown Source)
at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask$runInDockerClassPath.callCurrent(Unknown Source)
at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask.start(AbstractDockerRemoteApiTask.groovy:56)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:228)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:221)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:210)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
... 60 more

@cdancy try open wireshark and dump request/responses.

@cdancy does docker CLI works when you target it to versioned endpoint?

Docker CLI works with all version as does curl. Pasted the curl output for the default as well as a specific version below.

Curling with default (which is api v1.24):

curl -vv -X POST http://localhost:2375/containers/4bd6c5a9876a/start
*   Trying ::1...
* connect to ::1 port 2375 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 2375 (#0)
> POST /containers/4bd6c5a9876a/start HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 204 No Content
< Date: Mon, 04 Jul 2016 02:08:44 GMT
< Server: Docker/1.12.0-rc2 (linux)
<
* Connection #0 to host localhost left intact

Curling with specific api:

curl -vv -X POST http://localhost:2375/v1.22/containers/4bd6c5a9876a/start
*   Trying ::1...
* connect to ::1 port 2375 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 2375 (#0)
> POST /v1.22/containers/4bd6c5a9876a/start HTTP/1.1
> Host: localhost:2375
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 204 No Content
< Date: Mon, 04 Jul 2016 02:11:33 GMT
< Server: Docker/1.12.0-rc2 (linux)
<
* Connection #0 to host localhost left intact

Hm.. i have PR with updated jersey and deps in #620 but it started failing with connection pool. I think our problem now that we IT doesn't pass on travis :(

@KostyaSha if you need someone to test with an in-dev version of docker-java feel free to ping me when you've got things ironed out.

@cdancy atm i think only @marcuslinke remember what he hacked in ApacheConnector

UPD: master has fresh deps

Could somebody provide test case in java?

@KostyaSha

package com.github.m4dc4p.dockerjava;

import com.github.dockerjava.api.DockerClient;
import com.github.dockerjava.api.command.CreateContainerCmd;
import com.github.dockerjava.api.command.CreateContainerResponse;
import com.github.dockerjava.core.DockerClientBuilder;
import com.github.dockerjava.core.DockerClientConfig;

public class Main {

    public static void main(String[] args) {
        DockerClient docker = DockerClientBuilder.getInstance(
                DockerClientConfig.createDefaultConfigBuilder()
                        // .withDockerHost("tcp://127.0.0.1:2375") // uncomment for windows
                        .withApiVersion("1.21")
                        .build())
                .build();

        CreateContainerResponse container =
                docker.createContainerCmd("mysql/mysql-server:latest")
                        .withEnv("MYSQL_ALLOW_EMPTY_PASSWORD=yes")
                        .exec();

        docker.startContainerCmd(container.getId()).exec(); // HTTP 500/HTTP 400
    }
}

Fixed start to not send body, related only to jersey as netty decodes body in different way.
If it will repeat for some calls, will require more proper fix.

Start Container Constructs Incorrect POST · Issue #615 · docker-java/docker-java · GitHub

Repository navigation

Start Container Constructs Incorrect POST #615

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Start Container Constructs Incorrect POST · Issue #615 · docker-java/docker-java · GitHub

Repository navigation

Start Container Constructs Incorrect POST #615

Description

Activity

marcuslinke commented on Jun 22, 2016

KostyaSha commented on Jun 22, 2016

marcuslinke commented on Jun 22, 2016

cdancy commented on Jul 4, 2016

KostyaSha commented on Jul 4, 2016

KostyaSha commented on Jul 4, 2016

cdancy commented on Jul 4, 2016

KostyaSha commented on Jul 4, 2016

cdancy commented on Jul 4, 2016

KostyaSha commented on Jul 4, 2016

KostyaSha commented on Jul 12, 2016

KostyaSha commented on Jul 12, 2016

m4dc4p commented on Jul 12, 2016

KostyaSha commented on Aug 15, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions