GCloud Module

Note

This module is INCUBATING. While it is ready for use and operational in the current version of Testcontainers, it is possible that it may receive breaking changes in the future. See our contributing guidelines for more information on our incubating modules policy.

Testcontainers module for the Google Cloud Platform's Cloud SDK.

Currently, the module supports BigQuery, Bigtable, Datastore, Firestore, Spanner, and Pub/Sub emulators. In order to use it, you should use the following classes:

Class	Container Image
BigQueryEmulatorContainer	ghcr.io/goccy/bigquery-emulator
BigtableEmulatorContainer	gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators
DatastoreEmulatorContainer	gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators
FirestoreEmulatorContainer	gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators
SpannerEmulatorContainer	gcr.io/cloud-spanner-emulator/emulator
PubSubEmulatorContainer	gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators

Usage example

BigQuery

Start BigQuery Emulator during a test:

Starting a BigQuery Emulator container

BigQueryEmulatorContainer container = new BigQueryEmulatorContainer("ghcr.io/goccy/bigquery-emulator:0.4.3")

Creating BigQuery Client

String url = container.getEmulatorHttpEndpoint();
BigQueryOptions options = BigQueryOptions
    .newBuilder()
    .setProjectId(container.getProjectId())
    .setHost(url)
    .setLocation(url)
    .setCredentials(NoCredentials.getInstance())
    .build();
BigQuery bigQuery = options.getService();

Bigtable

Start Bigtable Emulator during a test:

Starting a Bigtable Emulator container

BigtableEmulatorContainer emulator = new BigtableEmulatorContainer(
    DockerImageName.parse("gcr.io/google.com/cloudsdktool/google-cloud-cli:583.0.0-emulators")
);

Create a test Bigtable table in the Emulator:

Create a test table

private void createTable(
    TransportChannelProvider channelProvider,
    CredentialsProvider credentialsProvider,
    String tableName
) throws IOException {
    TableAdminRequestContext requestContext = TableAdminRequestContext.create(PROJECT_ID, INSTANCE_ID);
    EnhancedBigtableTableAdminStub stub = EnhancedBigtableTableAdminStub.createEnhanced(
        BigtableTableAdminStubSettings
            .newBuilder()
            .setTransportChannelProvider(channelProvider)
            .setCredentialsProvider(credentialsProvider)
            .build(),
        requestContext
    );

    try (BigtableTableAdminClient client = BigtableTableAdminClient.create(PROJECT_ID, INSTANCE_ID, stub)) {
        Table table = client.createTable(CreateTableRequest.of(tableName).addFamily("name"));
    }
}

Test against the Emulator:

Testing with a Bigtable Emulator container

void testSimple() throws IOException {
    try (
        // emulatorContainer {
        BigtableEmulatorContainer emulator = new BigtableEmulatorContainer(
            DockerImageName.parse("gcr.io/google.com/cloudsdktool/google-cloud-cli:583.0.0-emulators")
        );
        // }
    ) {
        emulator.start();
        ManagedChannel channel = ManagedChannelBuilder
            .forTarget(emulator.getEmulatorEndpoint())
            .usePlaintext()
            .build();

        TransportChannelProvider channelProvider = FixedTransportChannelProvider.create(
            GrpcTransportChannel.create(channel)
        );
        NoCredentialsProvider credentialsProvider = NoCredentialsProvider.create();
        createTable(channelProvider, credentialsProvider, "test-table");
        try (
            BigtableDataClient client = BigtableDataClient.create(
                BigtableDataSettings
                    .newBuilderForEmulator(emulator.getHost(), emulator.getEmulatorPort())
                    .setProjectId(PROJECT_ID)
                    .setInstanceId(INSTANCE_ID)
                    .build()
            )
        ) {
            client.mutateRow(RowMutation.create(TableId.of("test-table"), "1").setCell("name", "firstName", "Ray"));

            Row row = client.readRow(TableId.of("test-table"), "1");
            List<RowCell> cells = row.getCells("name", "firstName");

            assertThat(cells).isNotNull().hasSize(1);
            assertThat(cells.get(0).getValue().toStringUtf8()).isEqualTo("Ray");
        } finally {
            channel.shutdown();
        }
    }
}

Datastore

Start Datastore Emulator during a test:

Starting a Datastore Emulator container

DatastoreEmulatorContainer emulator = new DatastoreEmulatorContainer(
    DockerImageName.parse("gcr.io/google.com/cloudsdktool/google-cloud-cli:583.0.0-emulators")
);

And test against the Emulator:

Testing with a Datastore Emulator container

@Test
public void testSimple() {
    try (
        // creatingDatastoreEmulatorContainer {
        DatastoreEmulatorContainer emulator = new DatastoreEmulatorContainer(
            DockerImageName.parse("gcr.io/google.com/cloudsdktool/google-cloud-cli:583.0.0-emulators")
        );
        // }
    ) {
        emulator.start();
        DatastoreOptions options = DatastoreOptions
            .newBuilder()
            .setHost(emulator.getEmulatorEndpoint())
            .setCredentials(NoCredentials.getInstance())
            .setRetrySettings(ServiceOptions.getNoRetrySettings())
            .setProjectId(emulator.getProjectId())
            .build();
        Datastore datastore = options.getService();

        Key key = datastore.newKeyFactory().setKind("Task").newKey("sample");
        Entity entity = Entity.newBuilder(key).set("description", "my description").build();
        datastore.put(entity);

        assertThat(datastore.get(key).getString("description")).isEqualTo("my description");
    }
}

See more examples:

Firestore

Start Firestore Emulator during a test:

Starting a Firestore Emulator container

FirestoreEmulatorContainer emulator = new FirestoreEmulatorContainer(
    DockerImageName.parse("gcr.io/google.com/cloudsdktool/google-cloud-cli:583.0.0-emulators")
);

And test against the Emulator:

Testing with a Firestore Emulator container

@Test
void testSimple() throws ExecutionException, InterruptedException {
    try (
        // emulatorContainer {
        FirestoreEmulatorContainer emulator = new FirestoreEmulatorContainer(
            DockerImageName.parse("gcr.io/google.com/cloudsdktool/google-cloud-cli:583.0.0-emulators")
        );
        // }
    ) {
        emulator.start();
        FirestoreOptions options = FirestoreOptions
            .getDefaultInstance()
            .toBuilder()
            .setHost(emulator.getEmulatorEndpoint())
            .setCredentials(NoCredentials.getInstance())
            .setProjectId("test-project")
            .build();
        Firestore firestore = options.getService();

        CollectionReference users = firestore.collection("users");
        DocumentReference docRef = users.document("alovelace");
        Map<String, Object> data = new HashMap<>();
        data.put("first", "Ada");
        data.put("last", "Lovelace");
        ApiFuture<WriteResult> result = docRef.set(data);
        result.get();

        ApiFuture<QuerySnapshot> query = users.get();
        QuerySnapshot querySnapshot = query.get();

        assertThat(querySnapshot.getDocuments().get(0).getData()).containsEntry("first", "Ada");
    }
}

See more examples:

Spanner

Start Spanner Emulator during a test:

Starting a Spanner Emulator container

SpannerEmulatorContainer emulator = new SpannerEmulatorContainer(
    DockerImageName.parse("gcr.io/cloud-spanner-emulator/emulator:1.4.0")
);

Create a test Spanner Instance in the Emulator:

Create a test Spanner instance

private InstanceId createInstance(Spanner spanner) throws InterruptedException, ExecutionException {
    InstanceConfigId instanceConfig = InstanceConfigId.of(PROJECT_NAME, "emulator-config");
    InstanceId instanceId = InstanceId.of(PROJECT_NAME, INSTANCE_NAME);
    InstanceAdminClient insAdminClient = spanner.getInstanceAdminClient();
    Instance instance = insAdminClient
        .createInstance(
            InstanceInfo
                .newBuilder(instanceId)
                .setNodeCount(1)
                .setDisplayName("Test instance")
                .setInstanceConfigId(instanceConfig)
                .build()
        )
        .get();
    return instanceId;
}

Create a test Database in the Emulator:

Creating a test Spanner database

private void createDatabase(Spanner spanner) throws InterruptedException, ExecutionException {
    DatabaseAdminClient dbAdminClient = spanner.getDatabaseAdminClient();
    Database database = dbAdminClient
        .createDatabase(
            INSTANCE_NAME,
            DATABASE_NAME,
            Arrays.asList("CREATE TABLE TestTable (Key INT64, Value STRING(MAX)) PRIMARY KEY (Key)")
        )
        .get();
}

And test against the Emulator:

Testing with a Spanner Emulator container

@Test
void testSimple() throws ExecutionException, InterruptedException {
    try (
        // emulatorContainer {
        SpannerEmulatorContainer emulator = new SpannerEmulatorContainer(
            DockerImageName.parse("gcr.io/cloud-spanner-emulator/emulator:1.4.0")
        );
        // }
    ) {
        emulator.start();
        SpannerOptions options = SpannerOptions
            .newBuilder()
            .setEmulatorHost(emulator.getEmulatorGrpcEndpoint())
            .setCredentials(NoCredentials.getInstance())
            .setProjectId(PROJECT_NAME)
            .build();

        Spanner spanner = options.getService();

        InstanceId instanceId = createInstance(spanner);

        createDatabase(spanner);

        DatabaseId databaseId = DatabaseId.of(instanceId, DATABASE_NAME);
        DatabaseClient dbClient = spanner.getDatabaseClient(databaseId);
        dbClient
            .readWriteTransaction()
            .run(tx -> {
                String sql1 = "Delete from TestTable where 1=1";
                tx.executeUpdate(Statement.of(sql1));
                String sql = "INSERT INTO TestTable (Key, Value) VALUES (1, 'Java'), (2, 'Go')";
                tx.executeUpdate(Statement.of(sql));
                return null;
            });

        ResultSet resultSet = dbClient
            .readOnlyTransaction()
            .executeQuery(Statement.of("select * from TestTable order by Key"));
        resultSet.next();
        assertThat(resultSet.getLong(0)).isEqualTo(1);
        assertThat(resultSet.getString(1)).isEqualTo("Java");
    }
}

See more examples:

Pub/Sub

Start Pub/Sub Emulator during a test:

Starting a Pub/Sub Emulator container

PubSubEmulatorContainer emulator = new PubSubEmulatorContainer(
    DockerImageName.parse("gcr.io/google.com/cloudsdktool/google-cloud-cli:583.0.0-emulators")
);

Create a test Pub/Sub topic in the Emulator:

Create a test topic

private void createTopic(
    String topicId,
    TransportChannelProvider channelProvider,
    NoCredentialsProvider credentialsProvider
) throws IOException {
    TopicAdminSettings topicAdminSettings = TopicAdminSettings
        .newBuilder()
        .setTransportChannelProvider(channelProvider)
        .setCredentialsProvider(credentialsProvider)
        .build();
    try (TopicAdminClient topicAdminClient = TopicAdminClient.create(topicAdminSettings)) {
        TopicName topicName = TopicName.of(PROJECT_ID, topicId);
        topicAdminClient.createTopic(topicName);
    }
}

Create a test Pub/Sub subscription in the Emulator:

Create a test subscription

private void createSubscription(
    String subscriptionId,
    String topicId,
    TransportChannelProvider channelProvider,
    NoCredentialsProvider credentialsProvider
) throws IOException {
    SubscriptionAdminSettings subscriptionAdminSettings = SubscriptionAdminSettings
        .newBuilder()
        .setTransportChannelProvider(channelProvider)
        .setCredentialsProvider(credentialsProvider)
        .build();
    SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create(subscriptionAdminSettings);
    SubscriptionName subscriptionName = SubscriptionName.of(PROJECT_ID, subscriptionId);
    subscriptionAdminClient.createSubscription(
        subscriptionName,
        TopicName.of(PROJECT_ID, topicId),
        PushConfig.getDefaultInstance(),
        10
    );
}

And test against the Emulator:

Testing with a Pub/Sub Emulator container

void testSimple() throws IOException {
    try (
        // emulatorContainer {
        PubSubEmulatorContainer emulator = new PubSubEmulatorContainer(
            DockerImageName.parse("gcr.io/google.com/cloudsdktool/google-cloud-cli:583.0.0-emulators")
        );
        // }
    ) {
        emulator.start();
        String hostport = emulator.getEmulatorEndpoint();
        ManagedChannel channel = ManagedChannelBuilder.forTarget(hostport).usePlaintext().build();
        try {
            TransportChannelProvider channelProvider = FixedTransportChannelProvider.create(
                GrpcTransportChannel.create(channel)
            );
            NoCredentialsProvider credentialsProvider = NoCredentialsProvider.create();

            String topicId = "my-topic-id";
            createTopic(topicId, channelProvider, credentialsProvider);

            String subscriptionId = "my-subscription-id";
            createSubscription(subscriptionId, topicId, channelProvider, credentialsProvider);

            Publisher publisher = Publisher
                .newBuilder(TopicName.of(PROJECT_ID, topicId))
                .setChannelProvider(channelProvider)
                .setCredentialsProvider(credentialsProvider)
                .build();
            PubsubMessage message = PubsubMessage
                .newBuilder()
                .setData(ByteString.copyFromUtf8("test message"))
                .build();
            publisher.publish(message);

            SubscriberStubSettings subscriberStubSettings = SubscriberStubSettings
                .newBuilder()
                .setTransportChannelProvider(channelProvider)
                .setCredentialsProvider(credentialsProvider)
                .build();
            try (SubscriberStub subscriber = GrpcSubscriberStub.create(subscriberStubSettings)) {
                PullRequest pullRequest = PullRequest
                    .newBuilder()
                    .setMaxMessages(1)
                    .setSubscription(ProjectSubscriptionName.format(PROJECT_ID, subscriptionId))
                    .build();
                PullResponse pullResponse = subscriber.pullCallable().call(pullRequest);

                assertThat(pullResponse.getReceivedMessagesList()).hasSize(1);
                assertThat(pullResponse.getReceivedMessages(0).getMessage().getData().toStringUtf8())
                    .isEqualTo("test message");
            }
        } finally {
            channel.shutdown();
        }
    }
}

See more examples:

Adding this module to your project dependencies

Add the following dependency to your pom.xml/build.gradle file:

GradleMaven

testImplementation "org.testcontainers:testcontainers-gcloud:2.0.5"

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers-gcloud</artifactId>
    <version>2.0.5</version>
    <scope>test</scope>
</dependency>

Join the community

GCloud Module

Usage example

BigQuery

Bigtable

Datastore

Firestore

Spanner

Pub/Sub

Adding this module to your project dependencies