| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Fuzz Introspector heuristics suggest the Submodule API code represent "optimal analysis targets" that should yield a meaningful increase in code coverage. The changes here introduce a first pass at implementing a fuzz harness that cover the primary APIs/methods related to Submodules. Of particular interest to me is the `Submodule.config_writer()` coverage. Please note however, there is likely plenty of room for improvement in this harness in terms of both code coverage as well as performance; the latter of which will see significant benefit from a well curated seed corpus of `.gitmodules` file like inputs. The `ParsingError` raised by the fuzzer without a good seed corpus hinders test efficacy significantly.
There was a problem hiding this comment.
Thanks a lot for taking such great care!
Part of me thinks that the submodule implementation is so riddled with inaccuracies and and incorrectness that fuzzing it seems like a waste. The fuzzer can only try to find unexpected exceptions, and maybe that's a small win, but at what cost?
Part of that feeling also stems for the incredible sluggishness of Python in general, so any fuzzing feels wasteful. But that's besides the point I suppose, apologies for the ramblings.
Sorry, something went wrong.
|
Please don't apologize, and definitely do not hesitate to reject or push back on any of my PRs! (especially considering that my last few PRs came out of the blue without prior discussion about whether they're even wanted -- sorry about that 😅)
I think your points are perfectly reasonable. Here is how I've been thinking of the value in fuzzing GitPython:
I think everything you said is very much on-point regarding any of the fuzzing work in this repo. Moreover, I really appreciate hearing your thoughts, so thanks! In case it isn't clear, I won't be offended if you feel the juice isn't worth the squeeze, and would rather me hold off on any non-maintenance type fuzzing work. Frankly, if you decided you'd rather it all removed ASAP, I'd help remove it. I've learned a lot about Git, Python, fuzzing, and more working on these, so I wouldn't consider it a wasted effort even if the changes never made it to PR, So thanks, @Byron, for the support along the way! 🙂 And now, it's my turn to apologize for the ramblings 😅 |
Sorry, something went wrong.
That's perfectly alright - no need to make it more complicated, just do what you think is right, you are driving this.
I am also clearly biased and think that everybody should use gitoxide, so it helps to see your reasoning to unbias me a bit. After all, GitPython has it's value and given its usage, maybe it's probably a good idea to invest in any measure that can make it a little better, maybe particularly due to its many flaws.
Spreading fuzzing as a technique through GitPython is a great thought and I am fully behind that - if nothing else comes out of it, more Python projects might adopt it which could be a net-win. And even if not, people learn how to use a fuzzer which will help in any programming environment eventually.
I'd never do that, and don't feel that way at all. But I do admit that I'd love to see you eventually move to gitoxide - you do tremendous work here and even if it's just for spreading good fuzzing setups, gitoxide could certainly be a good vessel for that. Eventually. No pressure :D. PS: There I'd definitely have more opinions on what to fuzz as well, which might make it more interesting for you. |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Fuzz Introspector heuristics suggest the Submodule API code represent "optimal analysis targets" that should yield a meaningful increase in code coverage. The changes here introduce a first pass at implementing a fuzz harness that cover the primary APIs/methods related to Submodules. Of particular interest to me is the Submodule.config_writer() coverage.
Please note however, there is likely plenty of room for improvement in this harness in terms of both code coverage as well as performance; the latter of which will see significant benefit from a well curated seed corpus of .gitmodules file like inputs. The ParsingError raised by the fuzzer without a good seed corpus hinders test efficacy significantly.
I have a draft PR up with a seed corpus here: gitpython-developers/qa-assets#5