[Storage] KMS integration by frankyn · Pull Request #3196 · googleapis/google-cloud-java · GitHub

Original author: @rossjudson
cc: @danoscarmike @rossjudson

Expected work items:

Define KMS keyring for integration tests
Allow configuration of default_kms_key_name in bucket
Allow specification of kms_key_name on object insert
Allow specification of kms_key_name on object rewrite
Test: Ensure kms_key_name is visible in object listings.
Test: Insert an object with a Cloud KMS key then fetch the object and verify contents and metadata (should include kms_key_name).
Test: Create a bucket with a default Cloud KMS key, insert an object, then fetch the object and verify its contents and metadata (should include kms_key_name).
Test: Update the default_kms_key_name of a bucket to a different key name.
Test: Update the bucket to remove default_kms_key_name encryption.
Test: Migrate CSEK to Cloud KMS key, download object and verify contents are decrypted.
Test: Test defaultKMSKeyName can be selected using BucketField.ENCRYPTION.
Test: Test kmsKeyName can be selected using BlobField.KMS_KEY_NAME.
Test: Write unit tests for BlobWriteOption and BlobTargetOption.
Nit: Remove Alpha and Beta annotations for GA features (Bucket level IAM and Requester Pays).
Add Beta annotations to KMS integration methods.
Code Review

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

CLAs look good, thanks!

Also adding @garrettjonesgoogle

@pongad and @garrettjonesgoogle, I realized yesterday afternoon that I confused myself on API surface for this feature in context of inserting an object with a KMS KeyName. On objects.insert kmsKeyName is set in request URL query parameters and only available in resource body post-operation.

I updated the PR/design to use BlobTargetOption and BlobWriteOption and set setKmsKeyName in BlobInfo as a private method.

More context and background on the operation:
https://cloud.google.com/storage/docs/encryption/using-customer-managed-keys#add-object-key

Oh dear I'm sorry I dropped the ball on this. We moved a bunch of files. Could you merge from master?

There are some lint warnings that are worth fixing, but I'm fine fixing those in a separate PR.

One nit, but LGTM otherwise

@garrettjonesgoogle Do you also want to take a look?

LGTM

All checks passed merging now.

🎉 Thanks Frank, Michael and Garrett!

…v3.28.1 (#3196)

🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).

Adding base GCS CMEK implementation from rossjudson@

031b102

frankyn requested a review from pongad as a code owner April 26, 2018 17:09

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 26, 2018

pongad reviewed Apr 26, 2018

View reviewed changes

googlebot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels May 1, 2018

Adding Integration tests and partial unit test coverage

56b34df

frankyn force-pushed the kms-integration branch from e847668 to 56b34df Compare May 1, 2018 08:18

googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels May 1, 2018

frankyn commented May 1, 2018

View reviewed changes

googlebot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels May 1, 2018

Fix kms key reosurce name

47701ed

frankyn force-pushed the kms-integration branch from f47fe80 to 47701ed Compare May 1, 2018 08:43

googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels May 1, 2018

googleapis deleted a comment from googlebot May 1, 2018

Intermediate stage with WriteOptions

86830e5

frankyn changed the title [Storage] Adding initial PR for KMS integration [Storage] KMS integration May 7, 2018

Clean up and add tests

8d042d2

frankyn force-pushed the kms-integration branch from d4afcd1 to 8d042d2 Compare May 7, 2018 07:34

pongad requested a review from garrettjonesgoogle May 7, 2018 16:37

garrettjonesgoogle reviewed May 7, 2018

View reviewed changes

Frank Natividad added 2 commits May 7, 2018 10:28

Address comments

116220c

Update storage apiary client library version

afd6eb3

Frank Natividad added 4 commits May 8, 2018 08:08

Use BlobTargetOption and BlobWriteOption

fa6b526

Make setKmsKeyName private

4da3796

Adding unit tests

a1b0087

Adding launch stage annotation to Fields

619c6e4

frankyn force-pushed the kms-integration branch 2 times, most recently from 99a7491 to 619c6e4 Compare May 21, 2018 18:58

Frank Natividad added 2 commits May 21, 2018 12:05

Merge branch 'master' into kms-integration

dbc9f2d

Update storage api client library version

2847609

pongad reviewed May 21, 2018

View reviewed changes

Remove String instantiation

e7e4911

garrettjonesgoogle approved these changes May 21, 2018

View reviewed changes

frankyn merged commit 58f0c89 into master May 21, 2018

frankyn deleted the kms-integration branch May 21, 2018 23:49

suztomo pushed a commit that referenced this pull request Mar 9, 2026

deps: update dependency com.google.cloud:sdk-platform-java-config to …

61f23a3

…v3.28.1 (#3196)

chingor13 pushed a commit that referenced this pull request Mar 30, 2026

deps: update googleapis/sdk-platform-java action to v2.60.1 (#3196 )

210d351

                     .decode("H4sIAAAAAAAAAPNIzcnJV3DPz0/PSVVwzskvTVEILskvSkxPVQQA/LySchsAAAA=");
                 private static final Map<String, String> BUCKET_LABELS = ImmutableMap.of("label1", "value1");
                 private static final String SERVICE_ACCOUNT_EMAIL = "gcloud-devel@gs-project-accounts.iam.gserviceaccount.com";
+                private static final String KMS_KEY_NAME = ""; // use real KMS KEY associated to the test project.

                   <dependency>
                     <groupId>com.google.apis</groupId>
                     <artifactId>google-api-services-storage</artifactId>
+                    <version>v1-rev125-1.23.0</version>

                    * 'requester_pays' flag.
                    */
-                  @GcpLaunchStage.Alpha
                   public static BucketSourceOption userProject(String userProject) {

+                  @Override
+                  public Builder setDefaultKmsKeyName(String defaultKmsKeyName) {
+                    this.defaultKmsKeyName = defaultKmsKeyName != null
+                            ? new String(defaultKmsKeyName) : Data.<String>nullOf(String.class);

                   }
+                  Encryption encryption = bucketPb.getEncryption();
+                  if (encryption != null && encryption.getDefaultKmsKeyName() != null && !encryption.getDefaultKmsKeyName().isEmpty()) {
+                    builder.setDefaultKmsKeyName(encryption.getDefaultKmsKeyName());

                 }
+                /**
+                 * Returns the default Cloud KMS key to be applied to newly inserted objects in this bucket.

               import java.util.List;
               import java.util.Map;
+              import jdk.nashorn.internal.runtime.regexp.joni.exception.ValueException;

[Storage] KMS integration by frankyn · Pull Request #3196 · googleapis/google-cloud-java · GitHub

Uh oh!

[Storage] KMS integration - #3196

[Storage] KMS integration#3196frankyn merged 14 commits intomastergoogleapis/google-cloud-java:masterfrom kms-integrationgoogleapis/google-cloud-java:kms-integrationCopy head branch name to clipboard

Conversation

frankyn commented Apr 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

googlebot commented May 1, 2018

Uh oh!

googlebot commented May 1, 2018

Uh oh!

This comment was marked as spam.

Uh oh!

googlebot commented May 1, 2018

Uh oh!

pongad commented May 7, 2018

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

frankyn commented May 8, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pongad commented May 16, 2018

Uh oh!

pongad left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as spam.

Uh oh!

pongad commented May 21, 2018

Uh oh!

garrettjonesgoogle left a comment

Choose a reason for hiding this comment

Uh oh!

frankyn commented May 21, 2018

Uh oh!

danoscarmike commented May 22, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[Storage] KMS integration#3196
frankyn merged 14 commits into
masterfrom
kms-integration

frankyn commented Apr 26, 2018 •
edited
Loading

frankyn commented May 8, 2018 •
edited
Loading