Optimize 3D display by AlexandreAbraham · Pull Request #6085 · matplotlib/matplotlib · GitHub

While printing 3D objects, I noticed that the display was pretty slow. The reason is that matplotlib relies a lot on list of objects instead of using big numpy arrays to fasten 3D projections.

This PR aims at solving part of this problem. I replaced some airthmetic on list of vectors by artihmetic on big matrices. This impacts mainly 3D display.

I work on neuroimaging data. I noticed an approximate 25% speedup optimization on 3D display (depending on the complexity of the figure). Refreshment in interactive plotting is also faster. Example script:

from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from nibabel import gifti


surf_mesh = '/path/to/spm12b/canonical/cortex_20484.surf.gii'

coords, faces = gifti.read(surf_mesh).getArraysFromIntent(1008)[0].data, \
                gifti.read(surf_mesh).getArraysFromIntent(1009)[0].data

limits = [coords.min(), coords.max()]

cmap = 'coolwarm'

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d', xlim=limits, ylim=limits)
ax.view_init(elev=0, azim=0)
ax.set_axis_off()

p3dcollec = ax.plot_trisurf(coords[:, 0], coords[:, 1], coords[:, 2],
                            triangles=faces, linewidth=0.,
                            antialiased=False,
                            color='white')

fig.savefig('brain.png')
plt.close(fig)

Possible other optimizations:

For display, matplotlib relies on a list of Path objects. This could be made faster with numpy arrays.
Poly3DCollection and others: a lot of unnecessary asarray and paddings of ones to obtain quaternions are done. This could be simplified by keeping a quaternion numpy array in the object and using appropriate numpy views but requires a deeper refactoring.
not plotting unseen triangle: For the moment, all polygons are drawn. if there are a lot, isn't it worth to compute which one are hidden by others and remove them?

@agramfort and @juhuntenburg, do you notice significant running time reduction with this PR?

attn @WeatherGod

You have a lot of test failures: https://travis-ci.org/matplotlib/matplotlib/jobs/112810572

We are not guaranteed a non-empty array, hence the initialization of minz with 1e9.

@AlexandreAbraham any progress on this front?

+1 to this. I still haven't found a good method for plotting 3-D brains w/ points on top of it in python. Making a trisurf is super slow when I have lots of triangles.

Hi,
Yes, I have understood why the tests were failing. I have unfortunately made some assumptions on the format of input data that turned out to be wrong. The optimization is still possible but requires more work. I had no time so far to fix it but I'll give it a try today.

'power cycled' to restart the CI against current master.

Looks like there is a problem in one of the examples

Warning, treated as error:
/home/travis/build/matplotlib/matplotlib/doc/examples/mplot3d/pathpatch3d_demo.rst:8: WARNING: Exception occurred in plotting pathpatch3d_demo
 from /home/travis/build/matplotlib/matplotlib/doc/mpl_examples/mplot3d/pathpatch3d_demo.py:
Traceback (most recent call last):
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/sphinxext/plot_directive.py", line 654, in render_figures
    figman.canvas.figure.savefig(img.filename(format), dpi=dpi)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/figure.py", line 1666, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/backend_bases.py", line 2227, in print_figure
    **kwargs)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/backends/backend_agg.py", line 517, in print_png
    FigureCanvasAgg.draw(self)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/backends/backend_agg.py", line 464, in draw
    self.figure.draw(self.renderer)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/artist.py", line 68, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/figure.py", line 1262, in draw
    renderer, self, dsu, self.suppressComposite)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/image.py", line 139, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/travis/build/matplotlib/matplotlib/lib/mpl_toolkits/mplot3d/axes3d.py", line 279, in draw
    for patch in self.patches]
  File "/home/travis/build/matplotlib/matplotlib/lib/mpl_toolkits/mplot3d/axes3d.py", line 279, in <listcomp>
    for patch in self.patches]
  File "/home/travis/build/matplotlib/matplotlib/lib/mpl_toolkits/mplot3d/art3d.py", line 320, in do_3d_projection
    s = np.vstack(self._segments3d, np.ones(self._segments3d.shape[1]))
AttributeError: 'PathPatch3D' object has no attribute '_segments3d'

I guess my point is more "let's not add more public API".

Hi @AlexandreAbraham
I think this is almost ready to be merged. Can you make the new public function private and rebase? The rebase might be quite painful…
If you are too busy with the new job, I can take over your branch :)

Hi, I ended up here from this paper: https://riojournal.com/articles.php?id=12342

I'd be very interested in using the method in the paper, but I'm finding the speed that it can be accomplished with the current implementation of 3d meshses is a dealbreaker. Is there hope for this PR moving forward?

+1 to this PR moving forward

ok, I'll make it a priority this week during SciPy.

…

On Wed, Jul 12, 2017 at 10:11 AM, Chris Holdgraf ***@***.***> wrote: +1 to this PR moving forward — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6085 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARy-Lv2kePQIHFLTlgb98P4Wy33BQm1ks5sNNQCgaJpZM4HmSzk> .

I think that this is the PR that @GaelVaroquaux mentioned to me earlier this week.

lmk if you want someone to usertest this at the sprint. I can bring some brain surfaces :-)

There are several unaddressed comments from @Kojoley . I am currently working on rebasing this as well. I'll force-push a rebase soon (if github allows me).

doesn't look like github will let me, so, I'll do it the old way and submit a PR against your branch.

hmmph, that won't work, either. @AlexandreAbraham , do you see any option to allow maintainers to push changes to your PR on this page?

This would be marginally better as

self._segments3d_data  = np.empty((n_segments, 4))
self._segments3d_data[:,:3] = segments
self._segments3d_data[:,3] = 1

Also, are you sure you mean "quaternion" and not "homogenous coordinate"?

I think this loop is np.split in some form

Get rid of this colon - it makes the function fail on 1d inputs

Storing coordinates in the first axis of the array is kind of unusual (see how np.linalg and scipy.misc.imread behave).

Perhaps these should all be xyz[..., 0] instead of xyz[0]?

This seems to have attracted lots of interest, but eventually never made it over the hump. @WeatherGod is there still some hope for this, or has its momment passed?

Perhaps parts of this needs to be separated out into smaller PRs. There is some good stuff here, but there were also some questionable stuff, too, that never got addressed.

…

On Wed, May 2, 2018 at 1:10 PM, Jody Klymak ***@***.***> wrote: This seems to have attracted lots of interest, but eventually never made it over the hump. @WeatherGod <https://github.com/WeatherGod> is there still some hope for this, or has its momment passed? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6085 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARy-ENj7x-jsaEbU7onzowiCV_Giuu8ks5tueiYgaJpZM4HmSzk> .

Closing as inactive. The goodies are here for anyone who wants them!

This seems to have attracted lots of interest, but eventually never made it over the hump
Closing as inactive.

If it's closed, what's the mechanism then for contributors to know that this PR is stalled, needs help and can be continued ?

do you see any option to allow maintainers to push changes to your PR on this page?

Should be allowed by Github by default now.

I'll make it a priority this week during SciPy. [in 2017]

Maybe someone might be interested in looking into it during the scipy sprint next week?

If I understand correctly this PR rebased (as of July 2017) can be found in @WeatherGod 's https://github.com/WeatherGod/matplotlib/tree/pr/6085 branch.

This PR was continued in #11577

mdboom added the status: needs review label Mar 1, 2016

WeatherGod reviewed Mar 1, 2016
View reviewed changes

WeatherGod added status: needs revision topic: mplot3d and removed status: needs review labels Mar 1, 2016

AlexandreAbraham added 9 commits June 8, 2016 22:55

Replace lists by numpy array in projection

a4a8a86

Remove profiling code

3d295af

Vectorize output of proj_transform_vec

5239e1a

Reach vectorized projection functions directly

7285fed

Address comments

328c2f4

More fixes

a973da8

Fix rotation of axes

14853d0

Fix some failures

c4de047

Use a contiguous representation in memory and views for the segments

6315961

AlexandreAbraham force-pushed the optimize_3d branch from a3bdaf8 to 6315961 Compare June 8, 2016 20:55

tacaswell closed this Jul 16, 2016

tacaswell reopened this Jul 16, 2016

tacaswell added status: needs review and removed status: needs revision labels Jul 16, 2016

tacaswell added this to the 2.1 (next point release) milestone Jul 16, 2016

AlexandreAbraham added 2 commits September 20, 2016 10:37

Fix bug in PathPatch

bb948bb

Fix vstack bug

e6211bd

NelleV self-assigned this Dec 18, 2016

tacaswell modified the milestones: 2.2 (next next feature release), 2.1 (next point release) Jul 16, 2017

eric-wieser reviewed Jul 19, 2017

View reviewed changes

eric-wieser suggested changes Jul 19, 2017

View reviewed changes

jklymak modified the milestones: needs sorting, v3.0 May 2, 2018

jklymak added the status: needs rebase label May 2, 2018

jklymak closed this May 2, 2018

QuLogic removed the status: needs review label May 2, 2018

QuLogic removed this from the v3.0 milestone May 2, 2018

rth pushed a commit to rth/matplotlib that referenced this pull request Jul 5, 2018

Optimize 3D display (matplotlib#6085 )

2a75026

rth mentioned this pull request Jul 5, 2018

[MRG] Optimize 3D display (continued) #11577

Closed

8 of 10 tasks

-                      minz = 1e9
-                      for (xs, ys, zs) in xyslist:
-                          minz = min(minz, min(zs))
+                      minz = np.min(xys[:, :, 2])

+                          self._segments3d_data = np.vstack(segments)
+                          # Add a fourth dimension for quaternions
+                          self._segments3d_data = np.hstack([self._segments3d_data,
+                                                             np.ones((n_segments, 1))])

+                      cum_s = 0
+                      for s in self._seg_sizes:
+                          segments_2d.append(xys[cum_s:cum_s + s, :2])
+                          cum_s += s

+                      tis = vecw[1] < 1
+                  vecw /= vecw[3]
+                  # Integrating tis in the numpy array for optimization purposes
+                  vecw[3, :] = tis

Optimize 3D display by AlexandreAbraham · Pull Request #6085 · matplotlib/matplotlib · GitHub

Repository navigation

Optimize 3D display - #6085

Optimize 3D display#6085AlexandreAbraham wants to merge 11 commits intomatplotlib:mastermatplotlib/matplotlib:masterfrom AlexandreAbraham:optimize_3dAlexandreAbraham/matplotlib:optimize_3dCopy head branch name to clipboard

Conversation

AlexandreAbraham commented Mar 1, 2016

Uh oh!

tacaswell commented Mar 1, 2016

Uh oh!

WeatherGod commented Mar 1, 2016

Uh oh!

WeatherGod Mar 1, 2016

Choose a reason for hiding this comment

Uh oh!

kingjr commented May 13, 2016

Uh oh!

choldgraf commented May 13, 2016

Uh oh!

AlexandreAbraham commented May 23, 2016

Uh oh!

tacaswell commented Jul 16, 2016

Uh oh!

jenshnielsen commented Jul 18, 2016

Uh oh!

anntzer commented Sep 21, 2016

Uh oh!

NelleV commented Dec 18, 2016

Uh oh!

mwaskom commented Jul 12, 2017

Uh oh!

choldgraf commented Jul 12, 2017

Uh oh!

WeatherGod commented Jul 13, 2017 via email

Uh oh!

tacaswell commented Jul 13, 2017

Uh oh!

choldgraf commented Jul 14, 2017

Uh oh!

WeatherGod commented Jul 15, 2017

Uh oh!

WeatherGod commented Jul 15, 2017

Uh oh!

WeatherGod commented Jul 15, 2017

Uh oh!

eric-wieser Jul 19, 2017

Choose a reason for hiding this comment

Uh oh!

eric-wieser Jul 19, 2017

Choose a reason for hiding this comment

Uh oh!

eric-wieser Jul 19, 2017

Choose a reason for hiding this comment

Uh oh!

eric-wieser left a comment

Choose a reason for hiding this comment

Uh oh!

jklymak commented May 2, 2018

Uh oh!

WeatherGod commented May 2, 2018 via email

Uh oh!

jklymak commented May 2, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rth commented Jul 5, 2018

Uh oh!

rth commented Jul 5, 2018

Uh oh!

rth commented Jul 15, 2018

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants

Optimize 3D display#6085
AlexandreAbraham wants to merge 11 commits into
matplotlib:masterfrom
AlexandreAbraham:optimize_3d

jklymak commented May 2, 2018 •
edited
Loading