As the title suggests, this is not an issue by itself but a proposal on how the current issues concerning blitting in webagg and ipympl backends might be resolved...
Since I'd like a bit of feedback on this before I implement it properly in a PR, here's a monkey-patch version of my suggestion.
This might also fix blitting in jupyter-notebooks (e.g. #19116) but would require sending "ack" messages in ipympl
In short, in the current implementation, the main problem for webagg blitting is the fact that events accumulate.
This results in extreme lags as shown in the gif below (green line should be updated with cursor position) and makes blitting in webagg not really useful:
My proposal is to use the "ack" message (sent once an image has been properly processed) to delay event handling so that events no longer accumulate and we get a nice and smooth blitting behavior.
Before I pack this properly in a PR, I'd like some feedback if this is actually a solution that seems feasible for you and if you see some immediate problems that might arise by this way of event handling.
The following code monkey-patches FigureCanvasWebAggCore and FigureManagerWebAgg to implement the proposed fix to avoid event accumulation.
🐍 code to monkey-patch webagg backend
frommatplotlib.backends.backend_webagg_coreimportFigureCanvasWebAggCore, FigureManagerWebAggdefhandle_ack(self, event):
# Network latency tends to decrease if traffic is flowing# in both directions. Therefore, the browser sends back# an "ack" message after each image frame is received.# This could also be used as a simple sanity check in the# future, but for now the performance increase is enough# to justify it, even if the server does nothing with it.# count the number of received imagesself._ack_cnt+=1defrefresh_all(self):
ifself.web_sockets:
diff=self.canvas.get_diff_image()
ifdiffisnotNone:
forsinself.web_sockets:
s.send_binary(diff)
# count the number of sent imagesself._send_cnt+=1defhandle_event(self, event):
cnt_equal=self._ack_cnt==self.manager._send_cnt# always process ack and draw events# process other events only if "ack count" equals "send count"# (e.g. if we received and handled all pending images)ifcnt_equalorevent["type"] in ["ack", "draw"]:
# immediately process all cached eventsforcache_event_type, cache_eventinself._event_cache.items():
getattr(self, 'handle_{0}'.format(cache_event_type),
self.handle_unknown_event)(cache_event)
self._event_cache.clear()
# reset counters to avoid overflows# (probably not necessary but you never know...)ifcnt_equal:
self.manager._send_cnt=0self._ack_cnt=0# process evente_type=event['type']
handler=getattr(self, 'handle_{0}'.format(e_type),
self.handle_unknown_event)
else:
# ignore events in case we have a pending image that is on the way# to be processed# cache the latest event of each type so we can process it once we are readyself._event_cache[event["type"]] =event# a final savety precaution in case send count is lower than ack count# (e.g. we wait for an image but there was no image sent)ifself.manager._send_cnt<self._ack_cnt:
# reset counts... they seem to be incorrectself.manager._send_cnt=0self._ack_cnt=0# (maybe??) send a draw-event to force a refresh# self.send_event("draw")returnreturnhandler(event)
FigureCanvasWebAggCore._ack_cnt=0FigureCanvasWebAggCore.handle_ack=handle_ackFigureCanvasWebAggCore.handle_event=handle_eventFigureCanvasWebAggCore._event_cache=dict()
FigureManagerWebAgg._send_cnt=0FigureManagerWebAgg.refresh_all=refresh_all
Activity
raphaelquast commented on Oct 20, 2023
I close this in favor of this PR: #27160