Commit 045f2f1
authored
chore(hermetic-build): include .github template updates as part of generation (#3723)
Part of the fix for
googleapis/sdk-platform-java#3701 ☕
### Approach
The templates come from
https://github.com/googleapis/synthtool/tree/bf182cd41d9a7de56092cafcc7befe6b398332f6.
The only update was to include `generated-files-bot`, which is already
up to date in all the HW repos.
The `.kokoro` folder will be a separate follow up task. We will now
focus on solving the update of `update_generation_config` `yaml` and
`sh` files.
The fix to include the `.github` folder follows from
https://setuptools.pypa.io/en/latest/userguide/datafiles.html:
```
Glob patterns do not automatically match dotfiles, i.e., directory or file names starting with a dot (.).
To include such files, you must explicitly start the pattern with a dot, e.g. .* to match .gitignore.
```
Interestingly, this is also the reason `cp
synthool/gcp/templates/java_library/* ...` did not bring folders
starting with dot (such as .kokoro) into
googleapis/sdk-platform-java#2884
### Confirming effects in downstream repos
Demos show the results as of
googleapis/sdk-platform-java@b66af92
in
- googleapis/java-storage#3012
- googleapis/java-logging#1787
- googleapis/java-pubsub#2384
- googleapis/java-bigtable#2546
- googleapis/java-spanner#3711
- googleapis/java-firestore#2065
- googleapis/java-datastore#1810
- googleapis/java-bigquerystorage#2929
- googleapis/java-pubsublite#1837
There were no regressions on templated files that were manually
modified.1 parent 2b1d90d commit 045f2f1
4 files changed
Lines changed: 35 additions & 24 deletions
File tree
- sdk-platform-java/hermetic_build
- library_generation
- owlbot/templates/java_library/.github
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -130,6 +130,9 @@ owl-bot copy-code --version | |||
| 130 | 130 | The key step is `npm link`, which will make the command available in you current | |
| 131 | 131 | shell session. | |
| 132 | 132 | ||
| 133 | + If you get a permission denied error when running the command `owl-bot`, try | ||
| 134 | + relinking owl-bot by running `npm unlink -g` and re-running the steps above. | ||
| 135 | + | ||
| 133 | 136 | ## Run the script | |
| 134 | 137 | The entrypoint script (`hermetic_build/library_generation/cli/entry_point.py`) | |
| 135 | 138 | allows you to generate a GAPIC repository with a given api definition (proto, | |
@@ -153,12 +156,14 @@ export path_to_repo="$(pwd)/google-cloud-java" | |||
| 153 | 156 | ### Install the scripts | |
| 154 | 157 | ||
| 155 | 158 | You can skip this step if you've installed the packages in [Install package dependencies](#install-package-dependencies). | |
| 159 | + Use the `--editable` flag for your changes to take effect as soon as you modify any file inside | ||
| 160 | + the package. | ||
| 156 | 161 | ||
| 157 | 162 | ```shell | |
| 158 | 163 | python -m pip install --require-hashes -r hermetic_build/common/requirements.txt | |
| 159 | - python -m pip install hermetic_build/common | ||
| 164 | + python -m pip install --editable hermetic_build/common | ||
| 160 | 165 | python -m pip install --require-hashes -r hermetic_build/library_generation/requirements.txt | |
| 161 | - python -m pip install hermetic_build/library_generation | ||
| 166 | + python -m pip install --editable hermetic_build/library_generation | ||
| 162 | 167 | ``` | |
| 163 | 168 | ||
| 164 | 169 | ### Run the script | |
@@ -212,10 +217,19 @@ python hermetic_build/library_generation/cli/entry_point.py generate \ | |||
| 212 | 217 | --api-definitions-path=/workspace/apis | |
| 213 | 218 | ``` | |
| 214 | 219 | ||
| 215 | - # Debug the library generation container | ||
| 220 | + # Debugging tips | ||
| 221 | + ## Debug the scripts | ||
| 222 | + Especially on local setups, consider adding the `xtrace` (`set -x`) flag to | ||
| 223 | + - `hermetic_build/library_generation/generate_library.sh` | ||
| 224 | + - `hermetic_build/library_generation/utils/utilities.sh` | ||
| 225 | + | ||
| 226 | + This will allow you to observe how the tools you prepared in `~/.library_generation` are being used. | ||
| 227 | + | ||
| 228 | + | ||
| 229 | + ## Debug the library generation container | ||
| 216 | 230 | If you are working on changing the way the containers are created, you may want | |
| 217 | 231 | to inspect the containers to check the setup. | |
| 218 | - It would be convenient in such case to have a text editor/viewer available. | ||
| 232 | + It would be convenient in such case to have a text editor/viewer available. | ||
| 219 | 233 | You can achieve this by modifying the Dockerfile as follows: | |
| 220 | 234 | ||
| 221 | 235 | ```dockerfile | |
Lines changed: 0 additions & 20 deletions
This file was deleted.
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -0,0 +1,13 @@ | |||
| 1 | + externalManifests: | ||
| 2 | + - type: json | ||
| 3 | + file: 'synth.metadata' | ||
| 4 | + jsonpath: '$.generatedFiles[*]' | ||
| 5 | + - type: json | ||
| 6 | + file: '.github/readme/synth.metadata/synth.metadata' | ||
| 7 | + jsonpath: '$.generatedFiles[*]' | ||
| 8 | + ignoreAuthors: | ||
| 9 | + - 'cloud-java-bot' | ||
| 10 | + - 'renovate-bot' | ||
| 11 | + - 'yoshi-automation' | ||
| 12 | + - 'release-please[bot]' | ||
| 13 | + - 'gcf-owl-bot[bot]' | ||
Lines changed: 4 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -24,6 +24,10 @@ | |||
| 24 | 24 | "owlbot/src/poms/*.py", | |
| 25 | 25 | "owlbot/templates/clirr/*.j2", | |
| 26 | 26 | "owlbot/templates/poms/*.j2", | |
| 27 | + "owlbot/templates/java_library/.github/**/*", | ||
| 28 | + # TODO: uncomment this line after https://github.com/googleapis/sdk-platform-java/pull/3723 | ||
| 29 | + # has been merged. | ||
| 30 | + # "owlbot/templates/java_library/.kokoro/**/*", | ||
| 27 | 31 | "owlbot/templates/java_library/**/*", | |
| 28 | 32 | ], | |
| 29 | 33 | "synthtool": ["owlbot/synthtool/**/*"], | |
0 commit comments