Datetime plot fails with 'Agg' backend in interactive mode · Issue #15409 · matplotlib/matplotlib · GitHub

Bug report

Bug summary

I try to plot the time series data, and the X-axis represents the datetime. The code succeeds when use the 'Qt5Agg' backend, but fails when use the 'Agg' backend
Code for reproduction
The example code based on the matplotlib tutorials(https://matplotlib.org/3.1.1/gallery/ticks_and_spines/date_concise_formatter.html).
When use the 'Qt5Agg' backend, the code works well.(To show my issue, I use the python instead of ipython, because the ipython will use 'Qt5Agg' backend as default in my ENV)

Python 3.6.8 (default, Oct  7 2019, 12:59:55) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> import matplotlib.pyplot as plt
>>> import matplotlib.dates as mdates
>>> import numpy as np
>>> base = datetime.datetime(2005, 2, 1)
>>> dates = np.array([base + datetime.timedelta(hours=(2 * i)) for i in range(732)])
>>> np.random.seed(19680801)
>>> N = len(dates)
>>> y = np.cumsum(np.random.randn(N))
>>> import matplotlib
>>> matplotlib.get_backend()
'Qt5Agg'
>>> plt.plot(dates,y)
[<matplotlib.lines.Line2D object at 0x7f621c039828>]

And I get the expected figure.

But when I use 'Agg' backend, the code fails.

Python 3.6.8 (default, Oct  7 2019, 12:59:55) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> matplotlib.use('Agg')
>>> matplotlib.get_backend()
'agg'
>>> import datetime
>>> import matplotlib.pyplot as plt
>>> import matplotlib.dates as mdates
>>> import numpy as np
>>> base = datetime.datetime(2005, 2, 1)
>>> dates = np.array([base + datetime.timedelta(hours=(2 * i)) for i in range(732)])
>>> N = len(dates)
>>> np.random.seed(19680801)
>>> y = np.cumsum(np.random.randn(N))
>>> plt.plot(dates,y)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2795, in plot
    is not None else {}), **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_axes.py", line 1666, in plot
    lines = [*self._get_lines(*args, data=data, **kwargs)]
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 225, in __call__
    yield from self._plot_args(this, kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 391, in _plot_args
    x, y = self._xy_from_xy(x, y)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 243, in _xy_from_xy
    bx = self.axes.xaxis.update_units(x)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1498, in update_units
    self._update_axisinfo()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1516, in _update_axisinfo
    self.set_major_locator(info.majloc)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1651, in set_major_locator
    self.stale = True
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 230, in stale
    self.stale_callback(self, val)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 51, in _stale_axes_callback
    self.axes.stale = val
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 230, in stale
    self.stale_callback(self, val)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/figure.py", line 51, in _stale_figure_callback
    self.figure.stale = val
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 230, in stale
    self.stale_callback(self, val)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 589, in _auto_draw_if_interactive
    fig.canvas.draw_idle()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 1907, in draw_idle
    self.draw(*args, **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 388, in draw
    self.figure.draw(self.renderer)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/figure.py", line 1709, in draw
    renderer, self, artists, self.suppressComposite)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 2647, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1203, in draw
    ticks_to_draw = self._update_ticks()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1079, in _update_ticks
    major_locs = self.get_majorticklocs()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1324, in get_majorticklocs
    return self.major.locator()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/dates.py", line 1431, in __call__
    self.refresh()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/dates.py", line 1451, in refresh
    dmin, dmax = self.viewlim_to_dt()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/dates.py", line 1202, in viewlim_to_dt
    .format(vmin))
ValueError: view limit minimum 0.0 is less than 1 and is an invalid Matplotlib date value. This often happens if you pass a non-datetime value to an axis that has datetime units
>>>

So, how can I resolve this issue? Thanks!

Matplotlib version

Operating system: ubuntu 18.04
Matplotlib version: 3.1.1
Matplotlib backend (print(matplotlib.get_backend())): agg
Python version: 3.6.8
Jupyter version (if applicable):
Other libraries: numpy 1.17.2

I use pip to install numpy and matplotlib, and I use the system's python3

The key is the use of interactive mode. That is matplotlib.is_interactive() is True.
I'm unsure if the use of a non-interactive backend ("Agg") together with interactive mode is supposed to work or has any usecases.

@ImportanceOfBeingErnest
The first time I met the issue was my python script succeeded in ipython but failed in python. I usually write python code with the help of ipython. When I finished my code (named test.py), I run it in ipython with the command %run test.py and get the expected result. But when I run the code with the command python3 test.py in bash, it failed.
Below is the content of my code.

#!/usr/bin/python3
import matplotlib
matplotlib.use('Agg')
import datetime
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import numpy as np
base = datetime.datetime(2005, 2, 1)
dates = np.array([base + datetime.timedelta(hours=(2 * i)) for i in range(732)])
np.random.seed(19680801)
N = len(dates)
y = np.cumsum(np.random.randn(N))
plt.plot(dates, y)
plt.savefig('test.png', format='png')
plt.close()

The result when I run it in ipython

fanxp@fanxp-ThinkPad-X1-Carbon-3rd:~/Documents/temp$ ipython
Python 3.6.8 (default, Oct  7 2019, 12:59:55)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: ls
test.py*

In [2]: %run test.py

In [3]: ls
test.png  test.py*

In [4]:

The error when I run it in bash

fanxp@fanxp-ThinkPad-X1-Carbon-3rd:~/Documents/temp$ ./test.py
Traceback (most recent call last):
  File "./test.py", line 14, in <module>
    plt.plot(dates, y)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2795, in plot
    is not None else {}), **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_axes.py", line 1666, in plot
    lines = [*self._get_lines(*args, data=data, **kwargs)]
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 225, in __call__
    yield from self._plot_args(this, kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 391, in _plot_args
    x, y = self._xy_from_xy(x, y)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 243, in _xy_from_xy
    bx = self.axes.xaxis.update_units(x)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1498, in update_units
    self._update_axisinfo()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1516, in _update_axisinfo
    self.set_major_locator(info.majloc)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1651, in set_major_locator
    self.stale = True
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 230, in stale
    self.stale_callback(self, val)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 51, in _stale_axes_callback
    self.axes.stale = val
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 230, in stale
    self.stale_callback(self, val)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/figure.py", line 51, in _stale_figure_callback
    self.figure.stale = val
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 230, in stale
    self.stale_callback(self, val)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 589, in _auto_draw_if_interactive
    fig.canvas.draw_idle()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 1907, in draw_idle
    self.draw(*args, **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 388, in draw       
    self.figure.draw(self.renderer)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper 
    return draw(artist, renderer, *args, **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/figure.py", line 1709, in draw 
    renderer, self, artists, self.suppressComposite)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 2647, in draw 
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1203, in draw
    ticks_to_draw = self._update_ticks()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1079, in _update_ticks
    major_locs = self.get_majorticklocs()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/axis.py", line 1324, in get_majorticklocs
    return self.major.locator() 
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/dates.py", line 1431, in __call__ 
    self.refresh()
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/dates.py", line 1451, in refresh
    dmin, dmax = self.viewlim_to_dt() 
  File "/home/fanxp/.local/lib/python3.6/site-packages/matplotlib/dates.py", line 1202, in viewlim_to_dt 
    .format(vmin)) 
ValueError: view limit minimum 0.0 is less than 1 and is an invalid Matplotlib date value. This often happens if you pass a non-datetime value to an axis that has datetime units

I have been bebugging for a long time and finally found that the ipython use Qt5Agg backend as default which may cause the different result, so I try my guess in the interactive mode (I have shown those results in my first comment).
However, in my cluster, that code succeed with the command python3 test.py. but the version of matplotlib is 2.2.1. So I think it may be a bug in the matplotlib 3.1.1.

The error also occurs in matplotlib 2.2.2 if interactive mode is on. So it's really interactive mode that triggers this.

@ImportanceOfBeingErnest
Yes, you're right. I put plt.ioff() in my code to turns interactive mode off, and the code works well. So it's really the interactive mode that triggers this.
Thank you.

Reopening, because it's not clear to me in how far this is expected or not.

In interactive mode with non-interactive backends the draw_idle calls (which tell the GUI to re-draw the next time it is convenient) get converted into draw calls (which forces a draw now). This is exposing a race condition is plt.plot where the axes is half reconfigured for dates. From the traceback, what I think is happening is that we have set the units on the axis, but not adjusted the limits yet. The default limits are [0, 1] which the date machinery does not like.

How about this for a new proposal for dates; let let them be modulo the accepted range? In this case the date limits would be set to 9999 dec 31 00:00. These are obviously nonesense, but will stop transient interactions with default limits.

I agree with @ImportanceOfBeingErnest that it is unclear why one would want to use a noninteractive backend together with ion().
I think _auto_draw_if_interactive (which is ultimately what triggers the draw) could check whether the current backend is non-interactive (in the sense that required_interactive_framework is None) and behave as a noop in that case (effectively making ion() a noop for non-interactive backends).
I guess it may be nice to check how webagg (which is somewhere in the middle between interactive and non-interactive, cf. this week's dev call) would behave with that.

We would also make draw_idle default to a no-op rather than falling back to draw.

That seems worse (I think?)? That would mean all interactive backends have to override draw_idle.

The all already do as draw_idle -> "schedule a GUI event".

fair point.
still I think semantically it makes sense for draw_idle to immediately draw on noninteractive backends (which are effectively always idle).

I cannot reproduce this, but this exception for <1-date should be gone in 3.3 due to the epoch changes in #15008.

Datetime plot fails with 'Agg' backend in interactive mode · Issue #15409 · matplotlib/matplotlib · GitHub

Repository navigation

Datetime plot fails with 'Agg' backend in interactive mode #15409

Bug report

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Datetime plot fails with 'Agg' backend in interactive mode · Issue #15409 · matplotlib/matplotlib · GitHub

Repository navigation

Datetime plot fails with 'Agg' backend in interactive mode #15409

Description

Bug report

Activity

ImportanceOfBeingErnest commented on Oct 12, 2019

fanxinping commented on Oct 12, 2019

ImportanceOfBeingErnest commented on Oct 13, 2019

fanxinping commented on Oct 13, 2019

ImportanceOfBeingErnest commented on Oct 13, 2019

tacaswell commented on Oct 13, 2019

jklymak commented on Oct 14, 2019

anntzer commented on Oct 16, 2019

tacaswell commented on Oct 16, 2019

anntzer commented on Oct 16, 2019

tacaswell commented on Oct 16, 2019

anntzer commented on Oct 16, 2019

QuLogic commented on May 7, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions