Repository navigation
Error reading png image from URL with imread in matplotlib 3.3 #18129
Description
Bug report
Bug summary
After upgrading to matplotlib 3.3, imread errors when reading in a png from a URL. This was working in 3.2.
from matplotlib import image
url = 'https://github.com/hjnilsson/country-flags/raw/master/png250px/us.png'
image.imread(url)Results in UnsupportedOperation: seek
Traceback indicates error is from pillow. Related Issue
Full Traceback
UnsupportedOperation Traceback (most recent call last)
<ipython-input-28-3cc23b6666a9> in <module>
----> 1 image.imread(url)
~/miniconda3/envs/mpl/lib/python3.8/site-packages/matplotlib/image.py in imread(fname, format)
1473 from urllib import request
1474 with urllib.request.urlopen(fname) as response:
-> 1475 return imread(response, format=ext)
1476 with img_open(fname) as image:
1477 return (_pil_png_to_float_array(image)
~/miniconda3/envs/mpl/lib/python3.8/site-packages/matplotlib/image.py in imread(fname, format)
1474 with urllib.request.urlopen(fname) as response:
1475 return imread(response, format=ext)
-> 1476 with img_open(fname) as image:
1477 return (_pil_png_to_float_array(image)
1478 if isinstance(image, PIL.PngImagePlugin.PngImageFile) else
~/miniconda3/envs/mpl/lib/python3.8/site-packages/PIL/ImageFile.py in __init__(self, fp, filename)
115 try:
116 try:
--> 117 self._open()
118 except (
119 IndexError, # end of data
~/miniconda3/envs/mpl/lib/python3.8/site-packages/PIL/PngImagePlugin.py in _open(self)
651 # get next chunk
652
--> 653 cid, pos, length = self.png.read()
654
655 try:
~/miniconda3/envs/mpl/lib/python3.8/site-packages/PIL/PngImagePlugin.py in read(self)
126 s = self.fp.read(8)
127 cid = s[4:]
--> 128 pos = self.fp.tell()
129 length = i32(s)
130
UnsupportedOperation: seek
Matplotlib version
- Operating system: macOS
- Matplotlib version: 3.3
- Matplotlib backend (print(matplotlib.get_backend())): MacOSX
- Python version: 3.8
Activity
jklymak commented on Jul 30, 2020
We switched to pillow handling the image i/o and it seems pillow can't handle urls? #15193
tdpetrou commented on Jul 30, 2020
Would a simple approach using the standard library's urllib.request module to read the contents as bytes not work?
tdpetrou commented on Jul 30, 2020
Just looked at the source. urllib.request is used and passes the response to PIL.PngImagePlugin.PngImageFile, but if it passes it to PIL.Image.open, it would work.
Edit:
The response object is not seekable which causes the error. You can wrap it in BytesIO to make it work.
This is essentially the error within urllib
tdpetrou commented on Jul 30, 2020
PIL.Image.open supports urllib.request.urlopen objects (python-pillow/Pillow#1151) so I don't see why we can't just always pass it there instead of PngImagePlugin.PngImageFile
jklymak commented on Jul 30, 2020
This was discussed - #15193 (comment) Not clear to me what @anntzer was trying to do with that, so perhaps he can explain.
anntzer commented on Jul 30, 2020
The point was to force reading the file as a png (if ext == "png") rather than relying on pillow's autodetection. This only matters if someone is nasty enough to have e.g. a jpeg file in a png extension, or pass a jpeg file to imread(..., ext="png").
My guess is that wrapping in a BytesIO() is fine, you already made a web request anyways and we're about to convert the whole thing to float32, so I wouldn't worry too much about speed or memory usage.
tacaswell commented on Aug 1, 2020
Labeling this as a good first issue because there is a very clear regression and 2 suggested fixes in this thread. This should get a test (so we don't accidentally break it again). I think we have a test of getting a stylesheet from a URL, the new test should get the same markup so it can be skipped in no-network environments (ex debian builds) .
andrewkoo commented on Aug 1, 2020
Hi! I am new to open source and am interested in helping out if I can. If this is still open, can I start working on this?
edit: Looking at tests/test_image.py it looks like there is a test_load_from_url() already defined:
def test_load_from_url(): path = Path(__file__).parent / "baseline_images/pngsuite/basn3p04.png" url = ('file:' + ('///' if sys.platform == 'win32' else '') + path.resolve().as_posix()) plt.imread(url) plt.imread(urllib.request.urlopen(url))Does this test case not catch this issue?
tacaswell commented on Aug 3, 2020
@andrewkoo Interesting, it is probably worth investigating why that test did not catch this issue!
timhoffm commented on Aug 3, 2020
Without checking, I'd guess file:/// vs. https:///.
dopplershift commented on Aug 3, 2020
With checking: open() doesn't seem to recognize file:///, but it's possible I did something wrong.
linchiwei123 commented on Aug 3, 2020
need to test http(s) as well.
1 remaining item
linchiwei123 commented on Aug 4, 2020
https://docs.python.org/3.8/library/urllib.request.html#urllib.request.urlopen
For HTTP and HTTPS URLs, this function returns a http.client.HTTPResponse object slightly modified. In addition to the three new methods above, the msg attribute contains the same information as the reason attribute — the reason phrase returned by server — instead of the response headers as it is specified in the documentation for HTTPResponse.
For FTP, file, and data URLs and requests explicitly handled by legacy URLopener and FancyURLopener classes, this function returns a urllib.response.addinfourl object.
timhoffm commented on Aug 4, 2020
HTTPResponse is documented to implement BufferedIOBase, which does not include seek(). IMHO that's all we can expect. That file:// returns another object with more functionality can be misleading but is not strictly a bug and I don't think cpython could/would change this.
Like Pillow ((python-pillow/Pillow#1151) we cannot assume that seek() exists. The fix is to either pass the response straight through (which we apparently do not want to do - did not follow the details), or apply the same fix with wrapping if neccesary as Pillow does.
FIX: fix reading from http/https urls via imread
tacaswell commented on Aug 5, 2020
@linchiwei123 @andrewkoo I went ahead and opened the PR to take care of this. Sorry to steal the work out from under you but as this was the last issue for 3.3.1 we wanted to expedite the process.
andrewkoo commented on Aug 5, 2020
All good! This is all just a learning process for me so would hate to delay anything..
linchiwei123 commented on Aug 5, 2020
It's OK.
FIX: fix reading from http/https urls via imread
FIX: fix reading from http/https urls via imread
anntzer commented on Oct 16, 2020
Note that with Pillow 8.0 the dance at #18129 (comment) would not be necessary anymore per https://pillow.readthedocs.io/en/stable/releasenotes/8.0.0.html#image-open-add-formats-parameter.
gaurav6386 commented on Oct 25, 2021
What is the fix guys?
jklymak commented on Oct 25, 2021
Upgrade to 3.3.1? Or are you saying that newer matplotlib don't work?
gaurav6386 commented on Oct 25, 2021
It's the latest version. And I don't think update was the fix earlier. I searched it everywhere and it still does not have proper solution. If you can help me or refer some fixes it will be of great help.
The issue is as shown below
But it exits after throwing
jklymak commented on Oct 25, 2021
Please open a new issue with a self-contained example (ie real url). Howver note that the original issue above you pass the url as a string not whatever urlopen returns.