Great to see this. Feel free to ping about any specific questions you have. We'll be pretty focuses on shipping 4.0 for the next little while (and SciPy next week), but I'm excited about this.

Remove utils/coloransi.py

You have no idea how happy that makes me.

Yes, I'll ping you. Probably I'll have many questions when I continue. Saturday I leave for three weeks on holiday, so not sure how much time I'll have. But I hope we should be able to finish it this summer.

I wonder if it might be better to add the prompt_toolkit machinery as a new interface initially, rather than changing TerminalInteractiveShell itself, and then phase out the readline-based TerminalInteractiveShell.

There's some extra complexity that's not visible in this repo, because the terminal frontend for Jupyter kernels (https://github.com/jupyter/jupyter_console ) currently inherits from TerminalInteractiveShell. It probably shouldn't do that, but it was the easiest way to get it up and going. I hope we can get that using prompt_toolkit too, but we should probably break the direct dependency between the two.

Also, thinking a bit more about the event loop integration stuff:

We currently have two kinds of GUI event loop integration. In the kernel, we hand over to the GUI event loop as the main loop, and trust it to call our own event loop at least every poll_interval seconds (default 0.05). In the terminal, Python's own event loop is the main one, and it calls the GUI loop in small bursts, via inputhook. But with readline we don't have a choice in this matter. If we could give primary control to the GUI event loop in the terminal, we could consolidate our two kinds of integration into one, which would be really nice.

However, the kernel approach ('call me') may not be quick enough for it our event loop to process every keystroke (in the kernel, it only needs to give us control when there's a message to handle). It's worth investigation, though.

closing in favor of #9118