Update JSON file during semantic release · Issue #1026 · python-semantic-release/python-semantic-release · GitHub

Question

Hello, I want to know if it is possible to update a JSON file using the resulting tag of the semantic release before upload the release to GitHub.
I tried something, but my plugin.json file is not updated by the CI/CD, should I use the assets key in the TOML file to specify the path to my JSON file, or manually add commit and push in the CI/CD script ?

Configuration

I use GitHub Action to run semantic release, here is the CI/CD:

name: "Run Semantic Release"

on:
  push:
    branches: [main]

jobs:
  release:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.9"]
        poetry-version: ["1.8.3"]
    permissions:
      contents: write
      id-token: write
    concurrency: release
    steps:
      #----------------------------------------------
      # Check-out repository
      #----------------------------------------------
      - name: Check out code
        uses: actions/checkout@v4
      #----------------------------------------------
      # Setup Workflow
      #----------------------------------------------
      - name: Setup Workflow
        uses: ./.github/actions/setup
        with:
          python-version: ${{ matrix.python-version }}
          poetry-version: ${{ matrix.poetry-version }}
          load-cache: true
      #----------------------------------------------
      # Run Semantic Release
      #----------------------------------------------
      - name: Python Semantic Release
        id: release
        uses: python-semantic-release/python-semantic-release@v8.0.0
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
      #----------------------------------------------
      # Install JQ to parse JSON
      #----------------------------------------------
      - name: Install JQ
        run: sudo apt-get update && sudo apt-get install jq
        shell: bash
      #----------------------------------------------
      # Update Plugin Version
      #----------------------------------------------
      - name: Update Plugin Version
        env:
          tag: ${{ steps.release.outputs.tag }}
        run: |
          #!/bin/bash
          # Update the version in plugin.json directly
          jq --arg version "$tag" '.version = $version' plugin.json > tmp.json && mv tmp.json plugin.json
        shell: bash
      #----------------------------------------------
      # Publish
      #----------------------------------------------
      - name: Publish package ditributions to GitHub Releases
        uses: python-semantic-release/upload-to-gh-release@main
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}

Semantic Release Configuration:

[tool.semantic_release]
version_variables = ["python-lib/salesforecast_v2/__init__.py:__version__"]
version_toml = ["pyproject.toml:tool.poetry.version"]
assets = []
tag_format = "{version}"
commit_message = "conf: update plugin to {version} version"

[tool.semantic_release.branches.main]
match = "main"

[tool.semantic_release.commit_parser_options]
allowed_tags = [
    "add",
    "rm",
    "build",
    "conf",
    "config",
    "cfg",
    "ci",
    "docs",
    "feat",
    "fix",
    "perf",
    "style",
    "refacto",
    "test",
]
minor_tags = ["feat"]
patch_tags = ["fix", "perf"]

[tool.semantic_release.publish]
dist_glob_patterns = ["dist/*"]

Thanks in advance for your help !

Add plugin.json:version to the version_variables array. It will require a version to exist inside the quotation marks but it should work if the key value is in the json file. This has worked for yaml files but I can't recall directly for json but the regex is flexible in punctuation for most languages variable specification. If you have more than one version key, it likely will not work as you intended. If you can provide the example of the plugin file that would also be helpful.

In the case it doesn't work, you would need to write a script that is executed during the build_command. You can refer to PSR for this example as we update versions within the docs.

If you have to do the latter, it is important to consider if you want to manually do a git add in the next step or you want to add the file to the assets list. Adding to the assets list will ensure that the change does get added to the repository index however it comes with additional logic where your file will be individually uploaded to the assets section of your release on GitHub.

Thanks for the precisions!
The json file look like this:

{
    "id": "sales-forecast",
    "version": "1.4.4",
    "meta": {
        "label": "Sales forecast",
        "description": "...",
        "author": "....",
        "icon": "icon-money",
        "tags": [
            "Dataiku plugin"
        ],
        "url": "",
        "licenseInfo": "Apache Software License"
    }
}

I will try to simply add plugin.json:version in the version_variables array! Thanks for your answer.

Looking at the file contents, it should work just fine.

Just make sure your entry includes the path to the plugins file before the colon. I don't know your file structure so I made it generic in my recommendation.

Yep, it is at the root of the directory, so should be good ! Thank you very much. I will let you know if it works asap.

If you want to try it out locally without needing a release environment pipeline, use the following:

pip install python-semantic-release

# need to be on a release branch
git checkout main

# add commits that trigger a new version 
git merge my-new-feature 

# stamp next version into code, run build, but don't release 
semantic-release version --no-changelog --no-commit --no-tag

# review changes
git status
git diff plugins.json

# revert the merge & reset all changes
git reset --hard origin/main

Hum unfortunately, it didn't work, but it updates correctly a __version__ in a Python file. So should be due to the JSON format.

Ok, I'll check on my side

@AdrienRiauxExt, I got it to work, it was the quotes that were not accounted for by default. If you change the spec to a TOML raw string (using ''') and then wrap your version with quotes it works.

version_variables = [
  '''plugins.json:"version"''',
]

It's not ideal but its what I can make work without any changes. In the future, I might be able to add quotes up front but honestly I was looking to pull in libraries that parse the format rather than hope the regex hits as a future endeavor

Oh thanks you're amazing ! That's more than enough :)
One last question haha, I tried the second option you mentioned (like in PSR for the docs), in the build_command what does the

python -m pip install -e .[build]
python -m build .

? And is it necessary ?

# [1] install project . (current directory) as an editable install, with the optional dependency
# set called 'build'
python -m pip install -e .[build]

# [2] run the module named 'build' on the current directory to create a sdist & wheel
python -m build .

The module build is the chosen build frontend that was recommended if you came from the setuptools ecosystem as it is their replacement after the PEP 517/518 rewrite. There are many other build systems to choose instead.

And is it necessary ?

If you are using the GitHub Action, you must install the build tool into the environment because it is currently written as a Docker Action. A Docker action will build a container at the beginning of the job and then mount your project into the environment. Unfortunately, this means that any dependencies you previously installed do not exist in the container (see #723) but PSR does. The recommended fix for this (until #692 is released in v10) is to add the installation step into the build_command to ensure it works (ie step 1). Secondly, it is also very important to manage your dependency version of the build tool just as you would any other dependency. You could hard code a version into the build command specification (which is what we had done at first) but then since it is not located with all your other versions that means it never gets updated unless you happen to stumble upon it. We have Dependabot configured for the repository so this automatically opens PRs when there are new versions available to notify us. Since the version was in the build command, Dependabot never notified us. So instead in #921, we defined our build dependency as an optional dependency group and updated the version there and with step 1, we always ensure the correct build tool version is installed in the GitHub Action environment and that Dependabot can keep us up to date.

A variant of Step 2 is required as that is the command that actually performs the build of the python distribution artifacts and we have ensured it will work because of step 1.

fix(version): increase `version_variable` flexibility with quotations…

Ok understand. Thanks a lot for your fast and complete answers!
I let you close the issue if necessary.

You're welcome. I will close it with the new release that will include the slight modification to the regex. At that point you will no longer need to specify the quotes in the variable name.

fix(version): increase `version_variable` flexibility with quotations…

fix(version-cmd): improve `version_variables` flexibility w/ quotes (…

🎉 This resolution has been included in version 9.8.9 🎉

The release is available on:

You will no longer need to specify the quotes in the variable name.

Update JSON file during semantic release · Issue #1026 · python-semantic-release/python-semantic-release · GitHub

Repository navigation

Update JSON file during semantic release #1026

Question

Configuration

🎉 This resolution has been included in version 9.8.9 🎉

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Update JSON file during semantic release · Issue #1026 · python-semantic-release/python-semantic-release · GitHub

Repository navigation

Update JSON file during semantic release #1026

Description

Question

Configuration

Activity

codejedi365 commented on Sep 23, 2024

adrien-riaux-ext commented on Sep 23, 2024

codejedi365 commented on Sep 23, 2024

adrien-riaux-ext commented on Sep 23, 2024

codejedi365 commented on Sep 23, 2024

adrien-riaux-ext commented on Sep 23, 2024

codejedi365 commented on Sep 23, 2024

codejedi365 commented on Sep 23, 2024

adrien-riaux-ext commented on Sep 23, 2024

codejedi365 commented on Sep 23, 2024

adrien-riaux-ext commented on Sep 24, 2024

codejedi365 commented on Sep 24, 2024

codejedi365 commented on Sep 27, 2024

🎉 This resolution has been included in version 9.8.9 🎉

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions