feat: populate `.repo-metadata.json` from highest version by JoeWang1127 · Pull Request #2890 · googleapis/sdk-platform-java · GitHub

In this PR:

populate .repo-metadata.json from highest stable version of proto paths.

The .repo-metadata.json is only generated once per library and the first item in proto paths is used, so the proto paths with the highest stable version should be the first item after sorting the proto paths in the library.

The method used to compare two proto paths, a and b:

if both of a and b don't have a version, the one with lower depth is smaller.
if either a or b has a version, the one with the version is smaller.
if either a or b has a stable version, but not both, the one with the stable version is smaller, e.g., google/spanner/v2 is smaller than google/spanner/v3beta.
if both a and b have a non-stable version, the one with higher version is smaller, e.g., google/spanner/v2beta1 is smaller than google/spanner/v2alpha2.
if both a and b have a stable version, the one with lower depth is smaller, e.g., google/spanner/v1 is smaller than google/spanner/admin/v1, google/spanner/v1 is smaller than google/spanner/admin/v2
if both a and b have a stable version and the depth is the same, the one with higher version is smaller, e.g., google/spanner/v2 is smaller than google/spanner/v1.

Test the change:

build the image locally:

docker build -f .cloudbuild/library_generation/library_generation.Dockerfile -t local:latest .

build the generator locally:

mvn -V -B -ntp clean install -DskipTests

run the image:

docker run --rm -u "$(id -u):$(id -g)" -v "$(pwd):/workspace" -v "$HOME/.m2:/home/.m2" local:latest

Result:

no change in common protos
transport changed to both in java-iam/.repo-metadata.json. This is expected since the transport in google/iam/v2/BUILD.bazel is grpc+rest. Without this change, the transport is parsed from google/iam/v2/BUILD.bazel, which doesn't have a java_gapic_library target, thus the value is grpc by default.

QQ, Don't we have .repo-metadata golden files for our units? Maybe we can change one of these to test the proto_path sorting criteria

I'm a bit ignorant to string comparison in python. Is v2 greater than v1 because "2" has a higher ASCII value than "1"?

Is v2 greater than v1 because "2" has a higher ASCII value than "1"?

Yes.

I released that string comparison has a bug here: v10 is smaller than v2. I'll change the algorithm.

Done.

Can we create a test for each of the scenario? [v1alpha1, v1], [v1, v2] etc. Because

It would be much more readable. For example, just by reading the tests, I don't know why v1alpha1, v1, v2, admin_v2, non_versioned, v1beta1 is sorted to v2, v1, admin_v2, v1beta1, v1alpha1, non_versioned.
We can also easily test more scenarios. e.g. scenarios of two non versioned libraries.

I added unit tests to verify comparison result of two gapic_configs.

I kept this test case though.

Quality Gate passed for 'gapic-generator-java-root'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

Quality Gate passed for 'java_showcase_integration_tests'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

QQ, Don't we have .repo-metadata golden files for our units? Maybe we can change one of these to test the proto_path sorting criteria

The golden files are used in testing .repo-metadata.json generation, generate_prerequisite_files to be specific.

If we want to test generating this file with a library_config (from which contains multiple proto_path), we need test generate_composed_library, which is not easy to do since this function aggregates many "small" functions and is tested in integration test.

🤖 I have created a release *beep* *boop* --- <details><summary>2.42.0</summary> ## [2.42.0](v2.41.0...v2.42.0) (2024-06-25) ### Features * Allow Adding Client Level Attributes to MetricsTracerFactory ([#2614](#2614)) ([f122c6f](f122c6f)) * gapic-generator-java to perform a no-op when no services are detected ([#2460](#2460)) ([c0b5646](c0b5646)) * Make Layout Parser generally available in V1 ([e508ae6](e508ae6)) * populate `.repo-metadata.json` from highest version ([#2890](#2890)) ([f587541](f587541)) * push SNAPSHOT versions of the hermetic build docker image ([#2888](#2888)) ([81df866](81df866)) ### Bug Fixes * **deps:** update the Java code generator (gapic-generator-java) to 1.2.3 ([e508ae6](e508ae6)) * Expose Gax meter name ([#2865](#2865)) ([6c5d6ce](6c5d6ce)) * Move the logic of getting systemProductName from static block to static method ([#2874](#2874)) ([536f1eb](536f1eb)) * Update default Otel Attribute from method_name to method ([#2833](#2833)) ([af10a9e](af10a9e)) ### Dependencies * update dependency com.google.auto.value:auto-value to v1.11.0 ([#2842](#2842)) ([dd27fdf](dd27fdf)) * update dependency com.google.auto.value:auto-value-annotations to v1.11.0 ([#2843](#2843)) ([bf8e67f](bf8e67f)) * update dependency com.google.cloud:grpc-gcp to v1.6.1 ([#2943](#2943)) ([9f16b40](9f16b40)) * update dependency org.checkerframework:checker-qual to v3.44.0 ([#2848](#2848)) ([7a99c50](7a99c50)) * update dependency org.easymock:easymock to v5.3.0 ([#2871](#2871)) ([c243f7d](c243f7d)) * update google api dependencies ([#2846](#2846)) ([b5ef698](b5ef698)) * update googleapis/java-cloud-bom digest to 17cc5ec ([#2882](#2882)) ([d6abd8e](d6abd8e)) * update netty dependencies to v4.1.111.final ([#2877](#2877)) ([b5f10b9](b5f10b9)) * update opentelemetry-java monorepo to v1.39.0 ([#2863](#2863)) ([9d1f3a8](9d1f3a8)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Joe Wang <joewa@google.com>

🤖 I have created a release *beep* *boop* --- <details><summary>2.42.0</summary> ## [2.42.0](googleapis/sdk-platform-java@v2.41.0...v2.42.0) (2024-06-25) ### Features * Allow Adding Client Level Attributes to MetricsTracerFactory ([#2614](googleapis/sdk-platform-java#2614)) ([e254528](googleapis/sdk-platform-java@e254528)) * gapic-generator-java to perform a no-op when no services are detected ([#2460](googleapis/sdk-platform-java#2460)) ([1612391](googleapis/sdk-platform-java@1612391)) * Make Layout Parser generally available in V1 ([30acafa](googleapis/sdk-platform-java@30acafa)) * populate `.repo-metadata.json` from highest version ([#2890](googleapis/sdk-platform-java#2890)) ([3bca9a4](googleapis/sdk-platform-java@3bca9a4)) * push SNAPSHOT versions of the hermetic build docker image ([#2888](googleapis/sdk-platform-java#2888)) ([8bc060b](googleapis/sdk-platform-java@8bc060b)) ### Bug Fixes * **deps:** update the Java code generator (gapic-generator-java) to 1.2.3 ([30acafa](googleapis/sdk-platform-java@30acafa)) * Expose Gax meter name ([#2865](googleapis/sdk-platform-java#2865)) ([2095f4a](googleapis/sdk-platform-java@2095f4a)) * Move the logic of getting systemProductName from static block to static method ([#2874](googleapis/sdk-platform-java#2874)) ([cf67770](googleapis/sdk-platform-java@cf67770)) * Update default Otel Attribute from method_name to method ([#2833](googleapis/sdk-platform-java#2833)) ([a88fdde](googleapis/sdk-platform-java@a88fdde)) ### Dependencies * update dependency com.google.auto.value:auto-value to v1.11.0 ([#2842](googleapis/sdk-platform-java#2842)) ([758bd17](googleapis/sdk-platform-java@758bd17)) * update dependency com.google.auto.value:auto-value-annotations to v1.11.0 ([#2843](googleapis/sdk-platform-java#2843)) ([bce5720](googleapis/sdk-platform-java@bce5720)) * update dependency com.google.cloud:grpc-gcp to v1.6.1 ([#2943](googleapis/sdk-platform-java#2943)) ([cfc6632](googleapis/sdk-platform-java@cfc6632)) * update dependency org.checkerframework:checker-qual to v3.44.0 ([#2848](googleapis/sdk-platform-java#2848)) ([7069aae](googleapis/sdk-platform-java@7069aae)) * update dependency org.easymock:easymock to v5.3.0 ([#2871](googleapis/sdk-platform-java#2871)) ([423167f](googleapis/sdk-platform-java@423167f)) * update google api dependencies ([#2846](googleapis/sdk-platform-java#2846)) ([a3d852e](googleapis/sdk-platform-java@a3d852e)) * update googleapis/java-cloud-bom digest to 17cc5ec ([#2882](googleapis/sdk-platform-java#2882)) ([09e4ea6](googleapis/sdk-platform-java@09e4ea6)) * update netty dependencies to v4.1.111.final ([#2877](googleapis/sdk-platform-java#2877)) ([6430ac2](googleapis/sdk-platform-java@6430ac2)) * update opentelemetry-java monorepo to v1.39.0 ([#2863](googleapis/sdk-platform-java#2863)) ([b0f2946](googleapis/sdk-platform-java@b0f2946)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Joe Wang <joewa@google.com>

JoeWang1127 added 4 commits June 13, 2024 16:29

feat: populate `.repo-metadata.json using the highest version

57f1a13

add unit tests

bd77781

Merge branch 'main' into feat/populate-metadate-from-highest-version

aaf65a9

Merge branch 'main' into feat/populate-metadate-from-highest-version

07f5172

JoeWang1127 marked this pull request as ready for review June 17, 2024 16:19

JoeWang1127 requested review from a team, blakeli0 and diegomarquezp June 17, 2024 16:19

diegomarquezp reviewed Jun 17, 2024

View reviewed changes

blakeli0 reviewed Jun 17, 2024

View reviewed changes

Comment thread library_generation/utils/proto_path_utils.py Outdated

Comment thread library_generation/model/gapic_config.py Outdated

blakeli0 reviewed Jun 18, 2024

View reviewed changes

JoeWang1127 added 2 commits June 18, 2024 14:57

change comparsion alg

b9611ff

add unit tests

c2e7b1c

product-auto-label Bot added the size: m Pull request size is medium. label Jun 18, 2024

JoeWang1127 requested a review from blakeli0 June 18, 2024 19:35

JoeWang1127 requested a review from diegomarquezp June 18, 2024 20:27

blakeli0 approved these changes Jun 19, 2024

View reviewed changes

JoeWang1127 merged commit f587541 into main Jun 19, 2024

JoeWang1127 deleted the feat/populate-metadate-from-highest-version branch June 19, 2024 12:19

release-please Bot mentioned this pull request Jun 19, 2024

chore(main): release 2.42.0 #2847

Merged

blakeli0 mentioned this pull request Jun 20, 2024

chore: add iam logging #2932

Merged

+                      if len(self_dirs) != len(other_dirs):
+                          return len(self_dirs) < len(other_dirs)
+                      # otherwise, the one with higher version is smaller.
+                      return self_version > other_version

feat: populate `.repo-metadata.json` from highest version by JoeWang1127 · Pull Request #2890 · googleapis/sdk-platform-java · GitHub

Uh oh!

feat: populate .repo-metadata.json from highest version - #2890

feat: populate .repo-metadata.json from highest version#2890
JoeWang1127 merged 6 commits into
mainfrom
feat/populate-metadate-from-highest-version

JoeWang1127 commented Jun 13, 2024 •
edited
Loading

Uh oh!

Uh oh!

diegomarquezp left a comment

Uh oh!

diegomarquezp Jun 17, 2024

Uh oh!

JoeWang1127 Jun 17, 2024

Uh oh!

JoeWang1127 Jun 18, 2024

Uh oh!

Uh oh!

Uh oh!

blakeli0 Jun 18, 2024

Uh oh!

JoeWang1127 Jun 18, 2024 •
edited
Loading

Uh oh!

Uh oh!

sonarqubecloud Bot commented Jun 18, 2024

Uh oh!

sonarqubecloud Bot commented Jun 18, 2024

Uh oh!

JoeWang1127 commented Jun 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

                       )
                       self.assertEqual("secret", library.get_library_name())
+                  def test_get_sorted_gapic_configs_returns_correct_order(self):

feat: populate `.repo-metadata.json` from highest version by JoeWang1127 · Pull Request #2890 · googleapis/sdk-platform-java · GitHub

Uh oh!

feat: populate .repo-metadata.json from highest version - #2890

feat: populate .repo-metadata.json from highest version#2890JoeWang1127 merged 6 commits intomaingoogleapis/sdk-platform-java:mainfrom feat/populate-metadate-from-highest-versiongoogleapis/sdk-platform-java:feat/populate-metadate-from-highest-versionCopy head branch name to clipboard

Conversation

JoeWang1127 commented Jun 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

diegomarquezp left a comment

Choose a reason for hiding this comment

Uh oh!

diegomarquezp Jun 17, 2024

Choose a reason for hiding this comment

Uh oh!

JoeWang1127 Jun 17, 2024

Choose a reason for hiding this comment

Uh oh!

JoeWang1127 Jun 18, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

blakeli0 Jun 18, 2024

Choose a reason for hiding this comment

Uh oh!

JoeWang1127 Jun 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud Bot commented Jun 18, 2024

Quality Gate passed for 'gapic-generator-java-root'

Uh oh!

sonarqubecloud Bot commented Jun 18, 2024

Quality Gate passed for 'java_showcase_integration_tests'

Uh oh!

JoeWang1127 commented Jun 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat: populate .repo-metadata.json from highest version#2890
JoeWang1127 merged 6 commits into
mainfrom
feat/populate-metadate-from-highest-version

JoeWang1127 commented Jun 13, 2024 •
edited
Loading

JoeWang1127 Jun 18, 2024 •
edited
Loading