The case that may need to be looked at a bit is the missing stub case, as that does not have a stub path to parse.

Edit: on second thought, it may give the path of the parent object or just line 0 of the module, but still worth a look

Would that be for a new Python file entirely?

No, just for a new method is what I was thinking

The stubtest output for that case doesn't include a line number, so it doesn't get posted, but the other errors do work fine. I'm not sure if I can get it to fall back to posting on the .py or if it would confuse it.

I also just added an extra function in the stub, but not the Python file, and I see that that doesn't get posted either; it looks like there's a slightly difference in output in that case, which I should be able to cover.

OK, this is now able to post 1) differences between stubs and runtime; 2) items in stubs but not in runtime. Unfortunately, the case with items in runtime but not stubs cannot be caught, AFAICT.

See https://github.com/QuLogic/matplotlib/pull/30/files

I opened an issue on stubtest to see if concise error messages could contain a bit more information (linked above). This could simplify this posting process as message would not be multiline.