The extension defines two directives, one with the new syntax but also a CompatProductionList that overrides Sphinx's .. productionlist::. We have syntax highlighting in existing productionlists too.

I added code to merge the lines of each multi-line production into a single production node.
Upstream Sphinx might need a similar change.

OK, now each production should should have complete rawsource.

Sorry for the delay on my part. I've refactored to split the parsing of content lines into individual productions into its own method, as I think it is sufficiently complex to warrant a dedicated function (and this massively simplifies the creation of the actual production node).

One open question is if group_dict is the right representation, as my understanding is that each dict must have exactly one key -- perhaps an enum would be a better fit? I'm not sure though, and we mustn't let perfect be the enemy of the good.

A

Don't worry about delays; I've had more important things to deal with too :)

Do you think a (type, content) tuple makes it clearer? Like this: AA-Turner/cpython@docs/productionlist-grammar...encukou:cpython:docs/productionlist-grammar

I think Enum would be overengineering; we'd need to duplicate the names from the regex (unless we build the regex from the enum -- more overengineering).

I've pushed this change to the PR, and updated the branch to fix a conflict.

@encukou I don't see any new commits...

I think that was a GitHub glitch.