Attempt 2 - Fix Missing Git Executable Causing ClusterFuzz Crash - #1909
Merged
Byron merged 1 commit intoApr 27, 2024
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
This is a second attempt at gitpython-developers#1906 and should resolve: - gitpython-developers#1905 - google/oss-fuzz#10600 PR gitpython-developers#1906 had the right idea but wrong implementation, and the differences between the ClusterFuzz image that it was supposed to fix and the OSS-Fuzz image where the fix was tested led to the issue not being fully resolved. The root cause of the issue is the same: A Git executable is not globally available in the ClusterFuzz container environment where OSS-Fuzz executes fuzz tests. gitpython-developers#1906 attempted to fix the issue by bundling the Git binary and using GitPython's `git.refresh(<full-path-to-git-executable>)` method to set it inside the `TestOneInput` function of the test harness. However, GitPython attempts to set the binary at import time via its `__init__` hook, and crashes the test if no executable is found during the import. This issue is fixed here by setting the environment variable that GitPython looks in before importing it, so it's available for the import. This was tested by setting the `$PATH` to an empty string inside the test files, which reproduced the crash, then adding the changes introduced here with `$PATH` still empty, which avoided the crash indicating that the bundled Git executable is working as expected.
1 task
Byron
approved these changes
Apr 27, 2024
Byron
left a comment
Copy link
Copy Markdown
Member
There was a problem hiding this comment.
Thanks so much for taking charge of this! No worries about taking more steps either :).
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
Author
|
It worked! I'm seeing logs from successful fuzz target runs over the last two hours or so! |
Sorry, something went wrong.
DaveLak
added a commit
to DaveLak/GitPython
that referenced
this pull request
Jun 4, 2024
ClusterFuzz runs of the `fuzz_submodule` target have been failing because the `git` import was placed before the condition that sets the Git executable path. The order in which `git` is imported matters because it attempts to find a Git executable as the import is loaded (via `refresh()` in `git/__init__.py`.) As per gitpython-developers#1909, we configure the ClusterFuzz environment to use a bundled Git executable via the env variable condition in all fuzz targets.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a second attempt at #1906 and should resolve:
PR #1906 had the right idea but wrong implementation. The differences between the ClusterFuzz image that it was supposed to fix and the OSS-Fuzz image where it was tested led to the issue not being fully resolved.
The root cause of the issue is the same: A Git executable is not globally available in the ClusterFuzz container environment where OSS-Fuzz executes fuzz tests.
#1906 attempted to fix the issue by bundling the Git binary and using
GitPython's git.refresh(<full-path-to-git-executable>) method to set it inside the TestOneInput function of the test harness.
However, GitPython attempts to set the binary at import time via its __init__ hook, and crashes the test if no executable is found during the import.
This issue is fixed here by setting the environment variable that GitPython looks in before importing it, so it's available for the import. This was tested by setting the $PATH to an empty string inside the test files, which reproduced the crash, then adding the changes introduced here with $PATH still empty, which avoided the crash indicating that the bundled Git executable is working as expected.
Note: I didn't test this in a full ClusterFuzz environment, so it's worth noting that while it may be possible for this to continue failing because of the difference between environments that the Git executable was produced in vs. where it's run: I don't believe that will be an issue in practice because the guidance for projects (including Git itself) in OSS-Fuzz is to compile in build.sh with the libraries necessary to create the final binary that is executed in ClusterFuzz. The changes here assume that the apt repo version of Git has everything statically built. If, for some reason, it turns out that a static binary isn't what is installed, the alternative will be to build git from source or mock it. But based on my local testing thus far, I think (and hope) that won't be necessary, and this changeset should be sufficient.