You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
<h1>MEP28: Remove Complexity from Axes.boxplot<aclass="headerlink" href="#mep28-remove-complexity-from-axes-boxplot" title="Permalink to this heading">#</a></h1>
<li><p><aclass="reference internal" href="#passing-transform-functions-to-cbook-boxplots-stats" id="id7">Passing transform functions to <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplots_stats</span></code></a></p></li>
<li><p><aclass="reference internal" href="#simplifications-to-the-axes-boxplot-api-and-other-functions" id="id8">Simplifications to the <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> API and other functions</a></p></li>
<h2><aclass="toc-backref" href="#id1" role="doc-backlink">Status</a><aclass="headerlink" href="#status" title="Permalink to this heading">#</a></h2>
<p><strong>Discussion</strong></p>
</section>
<sectionid="branches-and-pull-requests">
<h2><aclass="toc-backref" href="#id2" role="doc-backlink">Branches and Pull requests</a><aclass="headerlink" href="#branches-and-pull-requests" title="Permalink to this heading">#</a></h2>
<p>The following lists any open PRs or branches related to this MEP:</p>
<li><p>Deprecate passings 2D NumPy arrays as input: None</p></li>
<li><p>Add pre- & post-processing options to <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code>: <aclass="github reference external" href="https://github.com/phobson/matplotlib/tree/boxplot-stat-transforms">phobson/matplotlib</a></p></li>
<li><p>Exposing <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code> through <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> kwargs: None</p></li>
<li><p>Remove redundant statistical kwargs in <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code>: None</p></li>
<li><p>Remove redundant style options in <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code>: None</p></li>
<li><p>Remaining items that arise through discussion: None</p></li>
</ol>
</section>
<sectionid="abstract">
<h2><aclass="toc-backref" href="#id3" role="doc-backlink">Abstract</a><aclass="headerlink" href="#abstract" title="Permalink to this heading">#</a></h2>
<p>Over the past few releases, the <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> method has grown in
complexity to support fully customizable artist styling and statistical
computation. This lead to <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> being split off into multiple
parts. The statistics needed to draw a boxplot are computed in
<codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code>, while the actual artists are drawn by <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code>.
The original method, <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> remains as the most public API that
handles passing the user-supplied data to <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code>, feeding
the results to <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code>, and pre-processing style information for
each facet of the boxplot plots.</p>
<p>This MEP will outline a path forward to rollback the added complexity
and simplify the API while maintaining reasonable backwards
compatibility.</p>
</section>
<sectionid="detailed-description">
<h2><aclass="toc-backref" href="#id4" role="doc-backlink">Detailed description</a><aclass="headerlink" href="#detailed-description" title="Permalink to this heading">#</a></h2>
<p>Currently, the <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> method accepts parameters that allow the
users to specify medians and confidence intervals for each box that
will be drawn in the plot. These were provided so that advanced users
could provide statistics computed in a different fashion that the simple
method provided by matplotlib. However, handling this input requires
complex logic to make sure that the forms of the data structure match what
needs to be drawn. At the moment, that logic contains 9 separate if/else
statements nested up to 5 levels deep with a for loop, and may raise up to 2 errors.
These parameters were added prior to the creation of the <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code> method,
which draws boxplots from a list of dictionaries containing the relevant
statistics. Matplotlib also provides a function that computes these
statistics via <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code>. Note that advanced users can now
either a) write their own function to compute the stats required by
<codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code>, or b) modify the output returned by <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplots_stats</span></code>
to fully customize the position of the artists of the plots. With this
flexibility, the parameters to manually specify only the medians and their
confidences intervals remain for backwards compatibility.</p>
<p>Around the same time that the two roles of <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> were split into
<codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code> for computation and <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code> for drawing, both
<codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code> were written to accept parameters that
individually toggle the drawing of all components of the boxplots, and
parameters that individually configure the style of those artists. However,
to maintain backwards compatibility, the <codeclass="docutils literal notranslate"><spanclass="pre">sym</span></code> parameter (previously used
to specify the symbol of the fliers) was retained. This parameter itself
requires fairly complex logic to reconcile the <codeclass="docutils literal notranslate"><spanclass="pre">sym</span></code> parameters with the
newer <codeclass="docutils literal notranslate"><spanclass="pre">flierprops</span></code> parameter at the default style specified by <codeclass="docutils literal notranslate"><spanclass="pre">matplotlibrc</span></code>.</p>
<p>This MEP seeks to dramatically simplify the creation of boxplots for
novice and advanced users alike. Importantly, the changes proposed here
will also be available to downstream packages like seaborn, as seaborn
smartly allows users to pass arbitrary dictionaries of parameters through
the seaborn API to the underlying matplotlib functions.</p>
<p>This will be achieved in the following way:</p>
<blockquote>
<div><olclass="arabic simple">
<li><p><codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code> will be modified to allow pre- and post-
computation transformation functions to be passed in (e.g., <codeclass="docutils literal notranslate"><spanclass="pre">np.log</span></code>
and <codeclass="docutils literal notranslate"><spanclass="pre">np.exp</span></code> for lognormally distributed data)</p></li>
<li><p><codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> will be modified to also accept and naïvely pass them
to <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplots_stats</span></code> (Alt: pass the stat function and a dict
of its optional parameters).</p></li>
<li><p>Outdated parameters from <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> will be deprecated and
later removed.</p></li>
</ol>
</div></blockquote>
<sectionid="importance">
<h3><aclass="toc-backref" href="#id5" role="doc-backlink">Importance</a><aclass="headerlink" href="#importance" title="Permalink to this heading">#</a></h3>
<p>Since the limits of the whiskers are computed arithmetically, there
is an implicit assumption of normality in box and whisker plots.
This primarily affects which data points are classified as outliers.</p>
<p>Allowing transformations to the data and the results used to draw
boxplots will allow users to opt-out of that assumption if the
data are known to not fit a normal distribution.</p>
<p>Below is an example of how <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> classifies outliers of lognormal
data differently depending one these types of transforms.</p>
<h2><aclass="toc-backref" href="#id6" role="doc-backlink">Implementation</a><aclass="headerlink" href="#implementation" title="Permalink to this heading">#</a></h2>
<h3><aclass="toc-backref" href="#id7" role="doc-backlink">Passing transform functions to <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplots_stats</span></code></a><aclass="headerlink" href="#passing-transform-functions-to-cbook-boxplots-stats" title="Permalink to this heading">#</a></h3>
<p>This MEP proposes that two parameters (e.g., <codeclass="docutils literal notranslate"><spanclass="pre">transform_in</span></code> and
<codeclass="docutils literal notranslate"><spanclass="pre">transform_out</span></code> be added to the cookbook function that computes the
statistics for the boxplot function. These will be optional keyword-only
arguments and can easily be set to <codeclass="docutils literal notranslate"><spanclass="pre">lambda</span><spanclass="pre">x:</span><spanclass="pre">x</span></code> as a no-op when omitted
by the user. The <codeclass="docutils literal notranslate"><spanclass="pre">transform_in</span></code> function will be applied to the data
as the <codeclass="docutils literal notranslate"><spanclass="pre">boxplot_stats</span></code> function loops through each subset of the data
passed to it. After the list of statistics dictionaries are computed the
<codeclass="docutils literal notranslate"><spanclass="pre">transform_out</span></code> function is applied to each value in the dictionaries.</p>
<p>These transformations can then be added to the call signature of
<codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> with little impact to that method's complexity. This is
because they can be directly passed to <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code>.
Alternatively, <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> could be modified to accept an optional
statistical function kwarg and a dictionary of parameters to be directly
passed to it.</p>
<p>At this point in the implementation users and external libraries like
seaborn would have complete control via the <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> method. More
importantly, at the very least, seaborn would require no changes to its
API to allow users to take advantage of these new options.</p>
<h3><aclass="toc-backref" href="#id8" role="doc-backlink">Simplifications to the <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> API and other functions</a><aclass="headerlink" href="#simplifications-to-the-axes-boxplot-api-and-other-functions" title="Permalink to this heading">#</a></h3>
<p>Simplifying the boxplot method consists primarily of deprecating and then
removing the redundant parameters. Optionally, a next step would include
rectifying minor terminological inconsistencies between <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code>
and <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code>.</p>
<p>The parameters to be deprecated and removed include:</p>
<blockquote>
<div><olclass="arabic simple">
<li><p><codeclass="docutils literal notranslate"><spanclass="pre">usermedians</span></code> - processed by 10 SLOC, 3 <codeclass="docutils literal notranslate"><spanclass="pre">if</span></code> blocks, a <codeclass="docutils literal notranslate"><spanclass="pre">for</span></code> loop</p></li>
<li><p><codeclass="docutils literal notranslate"><spanclass="pre">conf_intervals</span></code> - handled by 15 SLOC, 6 <codeclass="docutils literal notranslate"><spanclass="pre">if</span></code> blocks, a <codeclass="docutils literal notranslate"><spanclass="pre">for</span></code> loop</p></li>
<p>Removing the <codeclass="docutils literal notranslate"><spanclass="pre">sym</span></code> option allows all code in handling the remaining
styling parameters to be moved to <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code>. This doesn't remove
any complexity, but does reinforce the single responsibility principle
among <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code>, and <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code>.</p>
<p>Additionally, the <codeclass="docutils literal notranslate"><spanclass="pre">notch</span></code> parameter could be renamed <codeclass="docutils literal notranslate"><spanclass="pre">shownotches</span></code>
to be consistent with <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code>. This kind of cleanup could be taken
a step further and the <codeclass="docutils literal notranslate"><spanclass="pre">whis</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">bootstrap</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">autorange</span></code> could
be rolled into the kwargs passed to the new <codeclass="docutils literal notranslate"><spanclass="pre">statfxn</span></code> parameter.</p>
</section>
</section>
<sectionid="backward-compatibility">
<h2><aclass="toc-backref" href="#id9" role="doc-backlink">Backward compatibility</a><aclass="headerlink" href="#backward-compatibility" title="Permalink to this heading">#</a></h2>
<p>Implementation of this MEP would eventually result in the backwards
incompatible deprecation and then removal of the keyword parameters
<codeclass="docutils literal notranslate"><spanclass="pre">usermedians</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">conf_intervals</span></code>, and <codeclass="docutils literal notranslate"><spanclass="pre">sym</span></code>. Cursory searches on
GitHub indicated that <codeclass="docutils literal notranslate"><spanclass="pre">usermedians</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">conf_intervals</span></code> are used by
few users, who all seem to have a very strong knowledge of matplotlib.
A robust deprecation cycle should provide sufficient time for these
users to migrate to a new API.</p>
<p>Deprecation of <codeclass="docutils literal notranslate"><spanclass="pre">sym</span></code> however, may have a much broader reach into
the matplotlib userbase.</p>
<sectionid="schedule">
<h3><aclass="toc-backref" href="#id10" role="doc-backlink">Schedule</a><aclass="headerlink" href="#schedule" title="Permalink to this heading">#</a></h3>
<p>An accelerated timeline could look like the following:</p>
<olclass="arabic">
<li><p>v2.0.1 add transforms to <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplots_stats</span></code>, expose in <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code></p></li>
<li><p>v2.1.0 Initial Deprecations , and using 2D NumPy arrays as input</p>
<blockquote>
<div><olclass="loweralpha simple">
<li><p>Using 2D NumPy arrays as input. The semantics around 2D arrays are generally confusing.</p></li>
<li><p>deprecate <codeclass="docutils literal notranslate"><spanclass="pre">notch</span></code> in favor of <codeclass="docutils literal notranslate"><spanclass="pre">shownotches</span></code> to be consistent with
other parameters and <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code></p></li>
<li><p>move all style and artist toggling logic to <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code> such <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code>
is little more than a broker between <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplots_stats</span></code></p></li>
</ol>
</dd>
</dl>
</li>
</ol>
</section>
<sectionid="anticipated-impacts-to-users">
<h3><aclass="toc-backref" href="#id11" role="doc-backlink">Anticipated Impacts to Users</a><aclass="headerlink" href="#anticipated-impacts-to-users" title="Permalink to this heading">#</a></h3>
<p>As described above deprecating <codeclass="docutils literal notranslate"><spanclass="pre">usermedians</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">conf_intervals</span></code>
will likely impact few users. Those who will be impacted are almost
certainly advanced users who will be able to adapt to the change.</p>
<p>Deprecating the <codeclass="docutils literal notranslate"><spanclass="pre">sym</span></code> option may import more users and effort should
be taken to collect community feedback on this.</p>
<h3><aclass="toc-backref" href="#id12" role="doc-backlink">Anticipated Impacts to Downstream Libraries</a><aclass="headerlink" href="#anticipated-impacts-to-downstream-libraries" title="Permalink to this heading">#</a></h3>
<p>The source code (GitHub master as of 2016-10-17) was inspected for
seaborn and python-ggplot to see if these changes would impact their
use. None of the parameters nominated for removal in this MEP are used by
seaborn. The seaborn APIs that use matplotlib's boxplot function allow
user's to pass arbitrary <codeclass="docutils literal notranslate"><spanclass="pre">**kwargs</span></code> through to matplotlib's API. Thus
seaborn users with modern matplotlib installations will be able to take
full advantage of any new features added as a result of this MEP.</p>
<p>Python-ggplot has implemented its own function to draw boxplots. Therefore,
no impact can come to it as a result of implementing this MEP.</p>
</section>
</section>
<sectionid="alternatives">
<h2><aclass="toc-backref" href="#id13" role="doc-backlink">Alternatives</a><aclass="headerlink" href="#alternatives" title="Permalink to this heading">#</a></h2>
<sectionid="variations-on-the-theme">
<h3><aclass="toc-backref" href="#id14" role="doc-backlink">Variations on the theme</a><aclass="headerlink" href="#variations-on-the-theme" title="Permalink to this heading">#</a></h3>
<p>This MEP can be divided into a few loosely coupled components:</p>
<olclass="arabic simple">
<li><p>Allowing pre- and post-computation transformation function in <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code></p></li>
<li><p>Exposing that transformation in the <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> API</p></li>
<li><p>Removing redundant statistical options in <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code></p></li>
<li><p>Shifting all styling parameter processing from <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> to <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code>.</p></li>
</ol>
<p>With this approach, #2 depends and #1, and #4 depends on #3.</p>
<p>There are two possible approaches to #2. The first and most direct would
be to mirror the new <codeclass="docutils literal notranslate"><spanclass="pre">transform_in</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">transform_out</span></code> parameters of
<codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code> in <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> and pass them directly.</p>
<p>The second approach would be to add <codeclass="docutils literal notranslate"><spanclass="pre">statfxn</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">statfxn_args</span></code>
parameters to <codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code>. Under this implementation, the default
value of <codeclass="docutils literal notranslate"><spanclass="pre">statfxn</span></code> would be <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code>, but users could
pass their own function. Then <codeclass="docutils literal notranslate"><spanclass="pre">transform_in</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">transform_out</span></code> would
then be passed as elements of the <codeclass="docutils literal notranslate"><spanclass="pre">statfxn_args</span></code> parameter.</p>
<p>This type of flexibility was the intention behind splitting the overall
boxplot API in the current three functions. In practice however, downstream
libraries like seaborn support versions of matplotlib dating back well
before the split. Thus, adding just a bit more flexibility to the
<codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code> could expose all the functionality to users of the
downstream libraries with modern matplotlib installation without intervention
from the downstream library maintainers.</p>
</section>
<sectionid="doing-less">
<h3><aclass="toc-backref" href="#id15" role="doc-backlink">Doing less</a><aclass="headerlink" href="#doing-less" title="Permalink to this heading">#</a></h3>
<p>Another obvious alternative would be to omit the added pre- and post-
computation transform functionality in <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code> and
<codeclass="docutils literal notranslate"><spanclass="pre">Axes.boxplot</span></code>, and simply remove the redundant statistical and style
parameters as described above.</p>
</section>
<sectionid="doing-nothing">
<h3><aclass="toc-backref" href="#id16" role="doc-backlink">Doing nothing</a><aclass="headerlink" href="#doing-nothing" title="Permalink to this heading">#</a></h3>
<p>As with many things in life, doing nothing is an option here. This means
we simply advocate for users and downstream libraries to take advantage
of the split between <codeclass="docutils literal notranslate"><spanclass="pre">cbook.boxplot_stats</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">Axes.bxp</span></code> and let
them decide how to provide an interface to that.</p>