FigureCanvasTkAgg.draw() silently crashes in threaded application · Issue #13293 · matplotlib/matplotlib · GitHub

Bug report

Bug summary

Matplotlib crashes python without rising any exceptions (python simply closes), when draw method is called on FigureCanvasTkAgg object in threaded application from thread other than main thread.

Code for reproduction

from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
from matplotlib.figure import Figure
import tkinter as tk
from threading import Thread

def draw(canvas):
    t = Thread(target=canvas.draw)
    t.start()

root = tk.Tk()
fig = Figure()
canvas = FigureCanvasTkAgg(fig, master=root)
canvas.get_tk_widget().pack()
tk.Button(root, text='Do it!', command=lambda: draw(canvas)).pack()
root.mainloop()

Actual outcome

When button from example shown above is clicked, application window freezes for a moment and than closes along with python, no exception or error is displayed.

Expected outcome

Preferably draw method should run normally, or at least an exception should be rised.
In Matplotlib 2.2.2 same code snipped runs flawlessly.

Matplotlib version

Operating system: Windows 10 Home x64
Matplotlib version: 3.0.2
Matplotlib backend (print(matplotlib.get_backend())): TkAgg
Python version: 3.6.7
Other libraries: tkinter, threading

Matplotlib installed with 'pip install matplotlib'.

Matplotlib makes no guarantees of thread safety. You'll need to use other methods to ensure that only one thread accesses it (conditions, mutexes, GUI-specific signals (if available), etc.)

19:59 $ python -X faulthandler  /tmp/tk_thread.py 
Fatal Python error: Segmentation fault

Current thread 0x00007f7029ab2700 (most recent call first):
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/site-packages/matplotlib/backends/_backend_tk.py", line 91 in blit
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/site-packages/matplotlib/backends/backend_tkagg.py", line 10 in draw
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/threading.py", line 864 in run
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x00007f7040fff740 (most recent call first):
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/tkinter/__init__.py", line 1277 in mainloop
  File "/tmp/tk_thread.py", line 15 in <module>
Segmentation fault (core dumped)

Even though we do not claim that Matplotlib is threadsafe, there is no obvious race here and the sefault is coming from inside

matplotlib/src/_tkagg.cpp

Lines 212 to 250 in 5a53f5e

    
           static PyObject *mpl_tk_blit(PyObject *self, PyObject *args) 
        
           { 
        
               Tcl_Interp *interp; 
        
               char const *photo_name; 
        
               int height, width; 
        
               unsigned char *data_ptr; 
        
               int o0, o1, o2, o3; 
        
               int x1, x2, y1, y2; 
        
               Tk_PhotoHandle photo; 
        
               Tk_PhotoImageBlock block; 
        
               if (!PyArg_ParseTuple(args, "O&s(iiO&)(iiii)(iiii):blit", 
        
                                     convert_voidptr, &interp, &photo_name, 
        
                                     &height, &width, convert_voidptr, &data_ptr, 
        
                                     &o0, &o1, &o2, &o3, 
        
                                     &x1, &x2, &y1, &y2)) { 
        
                   goto exit; 
        
               } 
        
               if (!(photo = TK_FIND_PHOTO(interp, photo_name))) { 
        
                   PyErr_SetString(PyExc_ValueError, "Failed to extract Tk_PhotoHandle"); 
        
                   goto exit; 
        
               } 
        
               block.pixelPtr = data_ptr + 4 * ((height - y2) * width + x1); 
        
               block.width = x2 - x1; 
        
               block.height = y2 - y1; 
        
               block.pitch = 4 * width; 
        
               block.pixelSize = 4; 
        
               block.offset[0] = o0; 
        
               block.offset[1] = o1; 
        
               block.offset[2] = o2; 
        
               block.offset[3] = o3; 
        
               TK_PHOTO_PUT_BLOCK_NO_COMPOSITE( 
        
                   photo, &block, x1, height - y2, x2 - x1, y2 - y1); 
        
           exit: 
        
               if (PyErr_Occurred()) { 
        
                   return NULL; 
        
               } else { 
        
                   Py_RETURN_NONE; 
        
               } 
        
           }

so I am a bit worried that this is exposing a latent bug.

We may also be hitting non-thread safe operations in tk?

You can get a backtrace via catchsegv python ../mpl-tests/issue13293.py:

.../conda/envs/mpl36/lib/python3.6/lib-dynload/../../libtk8.6.so(Tk_GetImageMasterData+0x11)[0x7f4d018b9821]
.../conda/envs/mpl36/lib/python3.6/lib-dynload/../../libtk8.6.so(Tk_FindPhoto+0xe)[0x7f4d018c4bee]
.../matplotlib/lib/matplotlib/backends/_tkagg.cpython-36m-x86_64-linux-gnu.so(+0x279e)[0x7f4d11ef379e]

Looking upstream at this comment, there does appear to be some queuing involved and using the interpreter across threads is risky. Our _tkagg C code probably does not do that queuing.

Also according to a later comment, Tk may or may not be compiled with thread support.

I also have this problem, calling everything from the same thread.
Newest matlablib and python, really tiered of it

AFAICT this is simply not supported by Tcl. Per http://www.tcl.tk/doc/howto/thread_model.html:

each interpreter is tightly bound to its OS thread and errors will occur if you let more than one thread call into the same interpreter (e.g., with Tcl_Eval).

We could start peppering the codebase with if threading.current_thread() == threading.main_thread() (in this specific case I guess in _backend_tk.blit, but I'm not sure it's so cheap? And we are explicitly not threadsafe anyways (there are other places which I know have race conditions, e.g. likely most things involving _setattr_cm).

So I'll close as wontfix, but as usual feel free to disagree and reopen...

@xzores Is that thread the main thread? In general GUI toolkits do not like having any part of them be run in a background thread (unless they manage it).

No it is in an other thread, but it is the same thread

I can't do "if threading.current_thread() == threading.main_thread()", because the main thread is used for tk.mainloop().
So I will never call matlablib from the main thread.
I just want to update a plot. The only way I found is to remove the plot entirely and then remake it, but that is not optimal, for many reasons

and I agree with the "can't fix" label, getting into thread safetly is starting to get into application logic and I suspect that any attempts on our part would invariably conflict with user's attempts to do the same in their application.

Use Tk's signaling system to ask the main thread to schedule a draw.

[edit to fix spelling]

Use Tk's signally system to ask the main thread to schedule a draw.

I get some data from the network, that I need to append to the existing data and update the plot

Calling figure.canvas.draw() crashes my app, no exceptions thrown (that must be a bug)

I've actually been working on this for some time. Since everyone seems to be looking at this right now, I've made a PR (#18565) to discuss.

In short, we can use the tkinter cross-thread signaling to make draw and blit thread "safe" but Tk_photoputblock is fundamentally not parallelizable, so this only really prevents crashes.

FigureCanvasTkAgg.draw() silently crashes in threaded application · Issue #13293 · matplotlib/matplotlib · GitHub

Repository navigation

FigureCanvasTkAgg.draw() silently crashes in threaded application #13293

Bug report

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	static PyObject mpl_tk_blit(PyObject self, PyObject *args)
	{
	Tcl_Interp *interp;
	char const *photo_name;
	int height, width;
	unsigned char *data_ptr;
	int o0, o1, o2, o3;
	int x1, x2, y1, y2;
	Tk_PhotoHandle photo;
	Tk_PhotoImageBlock block;
	if (!PyArg_ParseTuple(args, "O&s(iiO&)(iiii)(iiii):blit",
	convert_voidptr, &interp, &photo_name,
	&height, &width, convert_voidptr, &data_ptr,
	&o0, &o1, &o2, &o3,
	&x1, &x2, &y1, &y2)) {
	goto exit;
	}
	if (!(photo = TK_FIND_PHOTO(interp, photo_name))) {
	PyErr_SetString(PyExc_ValueError, "Failed to extract Tk_PhotoHandle");
	goto exit;
	}
	block.pixelPtr = data_ptr + 4 * ((height - y2) * width + x1);
	block.width = x2 - x1;
	block.height = y2 - y1;
	block.pitch = 4 * width;
	block.pixelSize = 4;
	block.offset[0] = o0;
	block.offset[1] = o1;
	block.offset[2] = o2;
	block.offset[3] = o3;
	TK_PHOTO_PUT_BLOCK_NO_COMPOSITE(
	photo, &block, x1, height - y2, x2 - x1, y2 - y1);
	exit:
	if (PyErr_Occurred()) {
	return NULL;
	} else {
	Py_RETURN_NONE;
	}
	}

FigureCanvasTkAgg.draw() silently crashes in threaded application · Issue #13293 · matplotlib/matplotlib · GitHub

Repository navigation

FigureCanvasTkAgg.draw() silently crashes in threaded application #13293

Description

Bug report

Activity

QuLogic commented on Jan 27, 2019

tacaswell commented on Jan 28, 2019

QuLogic commented on Jan 28, 2019

xzores commented on Sep 24, 2020

anntzer commented on Sep 24, 2020

tacaswell commented on Sep 24, 2020

xzores commented on Sep 24, 2020

xzores commented on Sep 24, 2020

tacaswell commented on Sep 24, 2020

tacaswell commented on Sep 24, 2020

xzores commented on Sep 24, 2020

xzores commented on Sep 24, 2020

richardsheridan commented on Sep 24, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions