Add new DLP samples (Bigquery, DeID, Risk Analysis) by ace-n · Pull Request #841 · GoogleCloudPlatform/java-docs-samples · GitHub

Do not merge until this code has been:
a) tested
b) updated to use new client libraries (as the current ones don't support scanning BigQuery)

Reminder: this should be reviewed on a best-effort basis.

This can be merged now:
a) the new Maven libraries support the needed methods
b) tests have been added, and they pass

Note: I did not review the API/these samples for Java-specific developer experience concerns (e.g. idiomaticity). I simply wrote them so that the tests passed. (@jabubake let me know if you'd like me to make any changes.)

Please run the google-java-formatter(https://github.com/google/google-java-format) on these files.
Also please run mvn clean verify locally ensuring that the tests pass for java-docs-samples-testing project.

Please avoid .* imports

Done.

Please avoid .* imports

Done.

Given this short name, I recommend renaming to DeIdentification

Done.

deidentify => deIdentify

Done.

Given all these objects that get created just to create a request, it would be to provide a comment above each to state what it is. It is unfortunate that InfoTypeTransformations object exists to just provide an InfoTypeTransformation[] surface. PrimitiveTransformation, InfoTypeTransformation also seem pretty confusing. In an ideal world, PrimitveTransformation, InfoTypeTransformation, InfoTypeTransformations are un-necessary objects.

I had similar comments with the Node client libraries, to be honest. I'll follow up with the API/product team(s) about this.

(TODO, so I can CTRL-F for this comment.)

Same comment as earlier, better comments around these objects and ideally some of these can get cleaned up on the surface. This looks very cumbersome to the user.

Please remove .* import.

Done.

DeIdentificationIT

Done.

Please remove .* import

Done.

if you can hard-code these variables, that would be ideal.
Else .travis.yaml will need to be updated so that these tests are successful on java-docs-samples-testing.

I'm assuming you're referring to the DLP_DEID_* variables and not GOOGLE_APPLICATION_CREDENTIALS.

DLP_DEID_WRAPPED_KEY is an encrypted secret - it technically should be safe to hard-code, but I wouldn't call it a 'best practice'. (As far as I'm aware, the best practice for such secrets is to add them to CI as secret environment variables.)

DLP_DEID_KEY_NAME is not terribly secret, but would reveal your project ID and keyring name if hard-coded.

The Node samples usually use environment variables for such things, but I can hard-code these if you want. Alternatively, I'm happy (and would prefer) to add these variables to your Travis builds (through the web UI, since they're supposed to remain secret - and not travis.yaml).

In that case, agree on not hard-coding. @lesv might be able to recommend the best approach.

Ace - can you just add how to do this to the README. I'm a fan of getting this stuff from Datastore, but in this case I'm going to go w/ Ace's current implementation.

We should also mention it in the root README as well - once we write it.

Done.

…c fixes

You might wish to mention the API's if you are going to mention this much stuff.

Perhaps in a more JavaDoc way and move the tag earlier.

Done.

Be sure to do a "mvn verify" or look at the results from Circle to see how you are doing on the Java Style guide - I see stuff that looks a bit iffy.

We'll need to update our testing if this stays.

I've added these environment variables to Travis, but they (and a few BigQuery tables) are based off of the nodejs-docs-samples project - and will need to be moved over at some point.

Please update the README to mention your environment variables for testing.

mvn verify comments have been fixed (with one exception, see below).

I didn't address the "abbreviation" linter warning, since it was complaining about files that I didn't modify - but let me know if this was in error.

🤖 I have created a release *beep* *boop* --- ### [0.122.22](googleapis/java-errorreporting@v0.122.21...v0.122.22) (2022-03-29) ### Dependencies * update dependency com.google.cloud:google-cloud-core to v2.5.11 ([#839](googleapis/java-errorreporting#839)) ([ecda8aa](googleapis/java-errorreporting@ecda8aa)) * update dependency com.google.cloud:google-cloud-shared-dependencies to v2.9.0 ([#840](googleapis/java-errorreporting#840)) ([7f59117](googleapis/java-errorreporting@7f59117)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

…2.0 (#841) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.cloud:libraries-bom](https://cloud.google.com/java/docs/bom) ([source](https://togithub.com/GoogleCloudPlatform/cloud-opensource-java)) | `25.1.0` -> `25.2.0` | [![age](https://badges.renovateapi.com/packages/maven/com.google.cloud:libraries-bom/25.2.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.cloud:libraries-bom/25.2.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.cloud:libraries-bom/25.2.0/compatibility-slim/25.1.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.cloud:libraries-bom/25.2.0/confidence-slim/25.1.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Renovate will not automatically rebase this PR, because other commits have been found. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-language).

…6.0 (#841) * deps: update dependency com.google.cloud:google-cloud-pubsub to v1.116.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

🤖 I have created a release *beep* *boop* --- ## [3.2.0](googleapis/java-dlp@v3.1.4...v3.2.0) (2022-03-29) ### Features * new Bytes and File types: POWERPOINT and EXCEL ([#848](googleapis/java-dlp#848)) ([ffb764d](googleapis/java-dlp@ffb764d)) ### Dependencies * update dependency com.google.cloud:google-cloud-pubsub to v1.116.0 ([#841](googleapis/java-dlp#841)) ([e12a43f](googleapis/java-dlp@e12a43f)) * update dependency com.google.cloud:google-cloud-shared-dependencies to v2.9.0 ([#849](googleapis/java-dlp#849)) ([7106010](googleapis/java-dlp@7106010)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

UNTESTED: Add DLP BigQuery sample

7f531ae

ace-n added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Sep 6, 2017

ace-n requested a review from jabubake September 6, 2017 22:52

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Sep 6, 2017

Ace Nassri added 3 commits October 17, 2017 23:18

Update DLP dependency

eca2c8b

Fix inspectBQ tests

3c8a224

Add DeID + RiskAnalysis samples

2a6148b

ace-n changed the title UNTESTED/DO NOT MERGE: Add DLP BigQuery sample Add new DLP samples (Bigquery, DeID, Risk Analysis) Oct 18, 2017

Merge branch 'master' into dlp-bq

28ae829

ace-n removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Oct 18, 2017

jabubake suggested changes Oct 18, 2017

View reviewed changes

Ace Nassri added 2 commits October 18, 2017 11:39

Address comments: clean up imports, document DeId requests, other mis…

7e439bc

…c fixes

Add missing comment

18038cb

lesv reviewed Oct 18, 2017

View reviewed changes

lesv suggested changes Oct 18, 2017

View reviewed changes

Ace Nassri added 5 commits October 18, 2017 14:39

Add env vars to README.md

f7ba95e

Switch to Javadoc-style comments

bd6eb95

(Temporarily) point BQ tests at nodejs-docs-samples GCP project

b5ebdab

Remove nodejs-docs-samples project ID for BigQuery tests

2c4bc7d

Address lint comments

f34ddd4

jabubake approved these changes Oct 19, 2017

View reviewed changes

Merge branch 'master' into dlp-bq

30f0028

jabubake merged commit b14e6c9 into master Oct 19, 2017

jabubake deleted the dlp-bq branch October 19, 2017 16:51

Shabirmean mentioned this pull request Nov 17, 2022

samples: migrate code from googleapis/java-language #7436

Merged

9 tasks

Sita04 pushed a commit that referenced this pull request Feb 7, 2023

samples: Add new DLP samples (Bigquery, DeID, Risk Analysis) (#841 )

5c11dea

+              import com.google.privacy.dlp.v2beta1.InfoTypeTransformations.InfoTypeTransformation;
+              import com.google.privacy.dlp.v2beta1.CryptoReplaceFfxFpeConfig.FfxCommonNativeAlphabet;
+              import com.google.protobuf.ByteString;
+              import org.apache.commons.cli.*;

               import com.google.cloud.dlp.v2beta1.DlpServiceClient;
               import com.google.longrunning.Operation;
-              import com.google.privacy.dlp.v2beta1.CloudStorageOptions;
+              import com.google.privacy.dlp.v2beta1.*;

+                  out = new PrintStream(bout);
+                  System.setOut(out); // TODO(b/64541432) DLP currently doesn't support GOOGLE DEFAULT AUTH
+                  assertNotNull(System.getenv("GOOGLE_APPLICATION_CREDENTIALS"));
+                  assertNotNull(System.getenv("DLP_DEID_WRAPPED_KEY"));


		public class DeId {

		private static void deidentifyWithMask(String string, Character maskingCharacter, int numberToMask) {

+                            .setPrimitiveTransformation(primitiveTransformation)
+                            .build();
+                    InfoTypeTransformations infoTypeTransformationArray =

+                            .setValue(string)
+                            .build();
+                    KmsWrappedCryptoKey kmsWrappedCryptoKey =

+              import static org.junit.Assert.*;
+              @RunWith(JUnit4.class)
+              public class DeIdIT {

+              import java.util.regex.Matcher;
+              import java.util.regex.Pattern;
+              import static org.junit.Assert.*;

+                  // Instantiates a client
+                  try (DlpServiceClient dlpServiceClient = DlpServiceClient.create()) {
+                    // (Optional) The project ID to run the API call under

+                private PrintStream out;
+                // Update to wrapped local encryption key
+                private String wrappedKey = System.getenv("DLP_DEID_WRAPPED_KEY");

Add new DLP samples (Bigquery, DeID, Risk Analysis) by ace-n · Pull Request #841 · GoogleCloudPlatform/java-docs-samples · GitHub

Uh oh!

Add new DLP samples (Bigquery, DeID, Risk Analysis) - #841

Add new DLP samples (Bigquery, DeID, Risk Analysis)#841jabubake merged 13 commits intomasterGoogleCloudPlatform/java-docs-samples:masterfrom dlp-bqGoogleCloudPlatform/java-docs-samples:dlp-bqCopy head branch name to clipboard

Conversation

ace-n commented Sep 6, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ace-n commented Oct 18, 2017

Uh oh!

jabubake left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ace-n Oct 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lesv left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lesv left a comment

Choose a reason for hiding this comment

Uh oh!

ace-n commented Oct 19, 2017

Uh oh!

Reviewers

Assignees

Labels

Add new DLP samples (Bigquery, DeID, Risk Analysis)#841
jabubake merged 13 commits into
masterfrom
dlp-bq

ace-n commented Sep 6, 2017 •
edited
Loading

ace-n Oct 18, 2017 •
edited
Loading