Implementation of POST /images/load endpoint by leonsabr · Pull Request #627 · docker-java/docker-java · GitHub

This is an implementation of POST /images/load endpoint.

A few words about testing. I added a small tarball (20 kB) with a single image named "docker-java/load:1.0". Tests try to load this tarball into docker, check if expected image appears, and remove it in @AfterMethod. Test for netty implementation is the same as for jaxrs/jersey implementation.

This change is

About tar, maybe create it with text instructions in container and then copy from container (command already exists)?

please use hamcrest matchers

in general lgtm beside comments

Or even you can generate tar with apache.io tar, should be not so much lines of code!

About tar, maybe create it with text instructions in container and then copy from container (command already exists)?

Please, give me more details. This image tarball should have a strict format.

Sorry, right name of library https://commons.apache.org/proper/commons-compress/tar.html that would work for creating tar.

Not sure about The layer.tar file contains aufs style .wh..wh.aufs files and directories for storing attribute changes and deletions. How you created it now?

And one more idea, could you check how docker tests this case? https://github.com/docker/docker/search?utf8=%E2%9C%93&q=layer.tar
I usually picking their tests into docker-java.
PS. if there is no way to create tar programmatically than it would be ok to keep it as binary in repo.

please use CurrentThread classloader to get input stream - as of on different os this path may be changed

That load from relative path, it related to maven plugin that executes it from root of project. So classloaders unrelated. Ideally you should put test resources under class_name/test_name and then load from classloader this test resource.

Please give me an example of under class_name/test_name rule? It looks like directories in src/test/resources/ violates this rule. Should I use src/test/resources/LoadImageCmdImplTest/loadImageFromTar/image.tar? What if one resource is used by several classes?

Then you can make public getter but place it in one place. The issue with placing just in resources that it will be like a trash can and after test deletions/refactorings they will be forgotten.
@leonsabr will search for examples bit later.

In order to get this tarball I manually did the following:

Created a simple Dockerfile which produces a 2-layer image.
docker build -t docker-java/load:1.0 .
docker save docker-java/load:1.0 > image.tar

As I see, Docker guys have tarballs in repository:

$ find docker -type f -name "*.tar"
docker/integration-cli/fixtures/load/emptyLayer.tar
docker/pkg/archive/testdata/broken.tar
docker/pkg/tarsum/testdata/46af0962ab5afeb5ce6740d4d91652e69206fc991fd5328c1a94d364ad00e457/layer.tar
docker/pkg/tarsum/testdata/511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158/layer.tar
docker/pkg/tarsum/testdata/collision/collision-0.tar
docker/pkg/tarsum/testdata/collision/collision-1.tar
docker/pkg/tarsum/testdata/collision/collision-2.tar
docker/pkg/tarsum/testdata/collision/collision-3.tar
docker/pkg/tarsum/testdata/xattr/layer.tar

As I understand you suggest to have text files in resources, then in runtime create a tarball according to format? It is tricky because tarball contains layer tarballs inside. So you need to create correct layer tarballs, then put them in the image tarball.

By the way, I can do a smaller tarball, with one layer, a few kB.

Or we can pick some their tarball..

So ok to have tarball in repo. But please place under corresponding class name

What was changed in second commit:

Hamcrest matchers
Smaller tarball 12 kB
TestResources class which provides access to the tarball's Path. Tarball is placed in resources/api/images/load/image.tar directory. What do you think about such approach?

I will squash commits for cleaner history later.

When this feature appeared in docker API? Could you place javadoc with since (example could be found through search) ?

LGTM, javadoc with API version would be welcome.

It was tricky but I found the first reference of this endpoint in docs/reference/api/docker_remote_api_v1.7.md. Now this docs are removed from the Docker website.

I squashed commits, so now all changes are in a single commit.

@leonsabr thanks! it was enough to check latest 3-4 api versions and mark with latest known. I don't think that anybody using 1.8 :)

@leonsabr 1.12 build fails with

Tests run: 323, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 556.507 sec <<< FAILURE! - in TestSuite
(com.github.dockerjava.netty.exec.LoadImageCmdExecTest)  Time elapsed: 0.152 sec  <<< FAILURE!
io.netty.handler.codec.CorruptedFrameException: invalid JSON received at byte position 0: 4c6f6164656420696d6167653a20646f636b65722d6a6176612f6c6f61643a312e300a
Results :
Failed tests: 
  LoadImageCmdExecTest.loadImageFromTar » CorruptedFrame invalid JSON received a...

Plus i found that it has verbose output #664

sinde docker 1.7 or 1.7 API?

found

KostyaSha reviewed Jul 6, 2016
View reviewed changes

lanwen reviewed Jul 6, 2016
View reviewed changes

KostyaSha reviewed Jul 12, 2016
View reviewed changes

Implement POST /images/load endpoint

a58de04

leonsabr force-pushed the images_load branch from bd4a71d to a58de04 Compare July 14, 2016 17:27

KostyaSha merged commit d803c46 into docker-java:master Jul 14, 2016

KostyaSha added this to the 3.0.1 milestone Jul 14, 2016

leonsabr deleted the images_load branch July 15, 2016 09:50

KostyaSha added the API/1.17 label Aug 14, 2016

KostyaSha reviewed Aug 14, 2016
View reviewed changes

KostyaSha added API/<1.17 and removed API/1.17 labels Aug 14, 2016

panuse pushed a commit to TuKangTech/docker-java that referenced this pull request Aug 20, 2017

Implement POST /images/load endpoint (docker-java#627 )

51a4a76


		final Image image = findImageWithId(expectedImageId, dockerClient.listImagesCmd().exec());

		assertNotNull(image, "Can't find expected image after loading from a tar archive");

+                  @Test
+                  public void loadImageFromTar() throws Exception {
+                      final Path imageTarFile = Paths.get("src/test/resources/testLoadImageFromTar/image.tar");


		CreateImageCmd createImageCmd(@Nonnull String repository, @Nonnull InputStream imageStream);

		LoadImageCmd loadImageCmd(@Nonnull InputStream imageStream);

+                  /**
+                   * Online documentation is not available anymore.
+                   */

Implementation of POST /images/load endpoint by leonsabr · Pull Request #627 · docker-java/docker-java · GitHub

Repository navigation

Implementation of POST /images/load endpoint - #627

Implementation of POST /images/load endpoint#627KostyaSha merged 1 commit intodocker-java:masterdocker-java/docker-java:masterfrom leonsabr:images_loadleonsabr/docker-java:images_loadCopy head branch name to clipboard

Conversation

leonsabr commented Jul 6, 2016 • edited by marcuslinke Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KostyaSha commented Jul 6, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KostyaSha commented Jul 6, 2016

Uh oh!

KostyaSha commented Jul 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leonsabr commented Jul 6, 2016

Uh oh!

KostyaSha commented Jul 6, 2016

Uh oh!

KostyaSha commented Jul 6, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leonsabr commented Jul 6, 2016

Uh oh!

KostyaSha commented Jul 6, 2016

Uh oh!

KostyaSha commented Jul 6, 2016

Uh oh!

leonsabr commented Jul 6, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KostyaSha commented Jul 12, 2016

Uh oh!

leonsabr commented Jul 14, 2016

Uh oh!

KostyaSha commented Jul 14, 2016

Uh oh!

KostyaSha commented Aug 14, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Implementation of POST /images/load endpoint#627
KostyaSha merged 1 commit into
docker-java:masterfrom
leonsabr:images_load

leonsabr commented Jul 6, 2016 •
edited by marcuslinke
Loading

KostyaSha commented Jul 6, 2016 •
edited
Loading

KostyaSha commented Aug 14, 2016 •
edited
Loading