Merge remote-tracking branch 'origin/master' into figure_level_validate

Author: jonmmease

Diff for: .circleci/create_conda_optional_env.sh

@@ -16,7 +16,7 @@ if [ ! -d $HOME/miniconda/envs/circle_optional ]; then
     # Create environment
     # PYTHON_VERSION=2.7 or 3.5
     $HOME/miniconda/bin/conda create -n circle_optional --yes python=$PYTHON_VERSION \
-requests nbformat six retrying psutil pandas decorator pytest mock nose poppler xarray scikit-image ipython jupyter ipykernel ipywidgets
+requests nbformat six retrying psutil pandas decorator pytest mock nose poppler xarray scikit-image ipython jupyter ipykernel ipywidgets statsmodels
 
     # Install orca into environment
     $HOME/miniconda/bin/conda install --yes -n circle_optional -c plotly plotly-orca==1.3.1

Diff for: CHANGELOG.md

@@ -2,6 +2,14 @@
 All notable changes to this project will be documented in this file.
 This project adheres to [Semantic Versioning](http://semver.org/).
 
+## Dev version
+
+### Added
+
+- The `hover_data` parameter of `px` functions can now be a dictionary. This
+  makes it possible to skip hover information for some arguments or to change  
+  the formatting of hover informatiom [#2377](https://github.com/plotly/plotly.py/pull/2377).
+
 ## [4.6] - 2020-03-31
 
 ### Updated

Diff for: doc/python/hover-text-and-formatting.md

@@ -123,7 +123,7 @@ fig.show()
 
 ### Customizing Hover text with Plotly Express
 
-Plotly Express functions automatically add all the data being plotted (x, y, color etc) to the hover label. Many Plotly Express functions also support configurable hover text. The `hover_data` argument accepts a list of column names to be added to the hover tooltip. The `hover_name` property controls which column is displayed in bold as the tooltip title.
+Plotly Express functions automatically add all the data being plotted (x, y, color etc) to the hover label. Many Plotly Express functions also support configurable hover text. The `hover_data` argument accepts a list of column names to be added to the hover tooltip, or a dictionary for advanced formatting (see the next section). The `hover_name` property controls which column is displayed in bold as the tooltip title.
 
 Here is an example that creates a scatter plot using Plotly Express with custom hover data and a custom hover name.
 
@@ -138,15 +138,42 @@ fig = px.scatter(df_2007, x="gdpPercap", y="lifeExp", log_x=True,
 fig.show()
 ```
 
+### Disabling or customizing hover of columns in plotly express
+
+`hover_data` can also be a dictionary. Its keys are existing columns of the `dataframe` argument, or new labels. For an existing column, the values can be
+* `False` to remove the column from the hover data (for example, if one wishes to remove the column of the `x` argument)
+* `True` to add a different column, with default formatting
+* a formatting string starting with `:` for numbers [d3-format's syntax](https://github.com/d3/d3-3.x-api-reference/blob/master/Formatting.md#d3_forma), and `|` for dates in [d3-time-format's syntax](https://github.com/d3/d3-3.x-api-reference/blob/master/Time-Formatting.md#format), for example `:.3f`, `|%a`.
+
+It is also possible to pass new data as values of the `hover_data` dict, either as list-like data, or inside a tuple, which first element is one of the possible values described above for existing columns, and the second element correspond to the list-like data, for example `(True, [1, 2, 3])` or `(':.1f', [1.54, 2.345])`.
+
+These different cases are illustrated in the following example.
+
+```python
+import plotly.express as px
+import numpy as np
+df = px.data.iris()
+fig = px.scatter(df, x='petal_length', y='sepal_length', facet_col='species', color='species',
+                 hover_data={'species':False, # remove species from hover data
+                             'sepal_length':':.2f', # customize hover for column of y attribute
+                             'petal_width':True, # add other column, default formatting
+                             'sepal_width':':.2f', # add other column, customized formatting
+                             # data not in dataframe, default formatting
+                             'suppl_1': np.random.random(len(df)),
+                             # data not in dataframe, customized formatting
+                             'suppl_2': (':.3f', np.random.random(len(df)))
+                            })
+fig.update_layout(height=300)
+fig.show()
+```
+
 ### Customizing hover text with a hovertemplate
 
-To customize the tooltip on your graph you can use [hovertemplate](https://plotly.com/python/reference/#pie-hovertemplate), which is a template string used for rendering the information that appear on hoverbox.
+To customize the tooltip on your graph you can use the [hovertemplate](https://plotly.com/python/reference/#pie-hovertemplate) attribute of `graph_objects` tracces, which is a template string used for rendering the information that appear on hoverbox.
 This template string can include `variables` in %{variable} format, `numbers` in [d3-format's syntax](https://github.com/d3/d3-3.x-api-reference/blob/master/Formatting.md#d3_forma), and `date` in [d3-time-format's syntax](https://github.com/d3/d3-3.x-api-reference/blob/master/Time-Formatting.md#format).
 Hovertemplate customize the tooltip text vs. [texttemplate](https://plotly.com/python/reference/#pie-texttemplate) which customizes the text that appears on your chart. <br>
 Set the horizontal alignment of the text within tooltip with [hoverlabel.align](https://plotly.com/python/reference/#layout-hoverlabel-align).
 
-Plotly Express automatically sets the `hovertemplate`, but you can set it manually when using `graph_objects`.
-
 ```python
 import plotly.graph_objects as go
 
@@ -187,6 +214,24 @@ fig = go.Figure(go.Pie(
 fig.show()
 ```
 
+### Modifying the hovertemplate of a plotly express figure
+
+`plotly.express` automatically sets the hovertemplate but you can modify it using the `update_traces` method of the generated figure. It helps to print the hovertemplate generated by `plotly.express` in order to be able to modify it. One can also revert to the default hover information of traces by setting the hovertemplate to `None`.
+
+```python
+import plotly.express as px
+
+df_2007 = px.data.gapminder().query("year==2007")
+
+fig = px.scatter(df_2007, x="gdpPercap", y="lifeExp", log_x=True, color='continent'
+                )
+print("plotly express hovertemplate:", fig.data[0].hovertemplate)
+fig.update_traces(hovertemplate='GDP: %{x} <br>Life Expectany: %{y}') # 
+fig.update_traces(hovertemplate=None, selector={'name':'Europe'}) # revert to default hover
+print("user_defined hovertemplate:", fig.data[0].hovertemplate)
+fig.show()
+```
+
 ### Advanced Hover Template
 
 The following example shows how to format hover template. [Here](https://plotly.com/python/v3/hover-text-and-formatting/#dash-example) is an example to see how to format hovertemplate in Dash.

Diff for: doc/python/imshow.md

@@ -6,7 +6,7 @@ jupyter:
       extension: .md
       format_name: markdown
       format_version: '1.2'
-      jupytext_version: 1.3.1
+      jupytext_version: 1.4.2
   kernelspec:
     display_name: Python 3
     language: python
@@ -20,7 +20,7 @@ jupyter:
     name: python
     nbconvert_exporter: python
     pygments_lexer: ipython3
-    version: 3.6.8
+    version: 3.7.7
   plotly:
     description: How to display image data in Python with Plotly.
     display_as: scientific

Diff for: packages/python/plotly/plotly/express/_core.py

@@ -241,18 +241,25 @@ def make_trace_kwargs(args, trace_spec, trace_data, mapping_labels, sizeref):
                     sorted_trace_data = trace_data.sort_values(by=args["x"])
                     y = sorted_trace_data[args["y"]]
                     x = sorted_trace_data[args["x"]]
-                    trace_patch["x"] = x
 
                     if x.dtype.type == np.datetime64:
                         x = x.astype(int) / 10 ** 9  # convert to unix epoch seconds
 
                     if attr_value == "lowess":
-                        trendline = sm.nonparametric.lowess(y, x)
+                        # missing ='drop' is the default value for lowess but not for OLS (None)
+                        # we force it here in case statsmodels change their defaults
+                        trendline = sm.nonparametric.lowess(y, x, missing="drop")
+                        trace_patch["x"] = trendline[:, 0]
                         trace_patch["y"] = trendline[:, 1]
                         hover_header = "<b>LOWESS trendline</b><br><br>"
                     elif attr_value == "ols":
-                        fit_results = sm.OLS(y.values, sm.add_constant(x.values)).fit()
+                        fit_results = sm.OLS(
+                            y.values, sm.add_constant(x.values), missing="drop"
+                        ).fit()
                         trace_patch["y"] = fit_results.predict()
+                        trace_patch["x"] = x[
+                            np.logical_not(np.logical_or(np.isnan(y), np.isnan(x)))
+                        ]
                         hover_header = "<b>OLS trendline</b><br>"
                         hover_header += "%s = %g * %s + %g<br>" % (
                             args["y"],
@@ -290,7 +297,10 @@ def make_trace_kwargs(args, trace_spec, trace_data, mapping_labels, sizeref):
                     go.Histogram2d,
                     go.Histogram2dContour,
                 ]:
+                    hover_is_dict = isinstance(attr_value, dict)
                     for col in attr_value:
+                        if hover_is_dict and not attr_value[col]:
+                            continue
                         try:
                             position = args["custom_data"].index(col)
                         except (ValueError, AttributeError, KeyError):
@@ -387,7 +397,20 @@ def make_trace_kwargs(args, trace_spec, trace_data, mapping_labels, sizeref):
         go.Parcoords,
         go.Parcats,
     ]:
-        hover_lines = [k + "=" + v for k, v in mapping_labels.items()]
+        # Modify mapping_labels according to hover_data keys
+        # if hover_data is a dict
+        mapping_labels_copy = OrderedDict(mapping_labels)
+        if args["hover_data"] and isinstance(args["hover_data"], dict):
+            for k, v in mapping_labels.items():
+                if k in args["hover_data"]:
+                    if args["hover_data"][k][0]:
+                        if isinstance(args["hover_data"][k][0], str):
+                            mapping_labels_copy[k] = v.replace(
+                                "}", "%s}" % args["hover_data"][k][0]
+                            )
+                    else:
+                        _ = mapping_labels_copy.pop(k)
+        hover_lines = [k + "=" + v for k, v in mapping_labels_copy.items()]
         trace_patch["hovertemplate"] = hover_header + "<br>".join(hover_lines)
         trace_patch["hovertemplate"] += "<extra></extra>"
     return trace_patch, fit_results
@@ -535,9 +558,9 @@ def configure_cartesian_axes(args, fig, orders):
 def configure_ternary_axes(args, fig, orders):
     fig.update_layout(
         ternary=dict(
-            aaxis=dict(title=get_label(args, args["a"])),
-            baxis=dict(title=get_label(args, args["b"])),
-            caxis=dict(title=get_label(args, args["c"])),
+            aaxis=dict(title_text=get_label(args, args["a"])),
+            baxis=dict(title_text=get_label(args, args["b"])),
+            caxis=dict(title_text=get_label(args, args["c"])),
         )
     )
 
@@ -572,9 +595,9 @@ def configure_polar_axes(args, fig, orders):
 def configure_3d_axes(args, fig, orders):
     layout = dict(
         scene=dict(
-            xaxis=dict(title=get_label(args, args["x"])),
-            yaxis=dict(title=get_label(args, args["y"])),
-            zaxis=dict(title=get_label(args, args["z"])),
+            xaxis=dict(title_text=get_label(args, args["x"])),
+            yaxis=dict(title_text=get_label(args, args["y"])),
+            zaxis=dict(title_text=get_label(args, args["z"])),
         )
     )
 
@@ -869,6 +892,22 @@ def _get_reserved_col_names(args, attrables, array_attrables):
     return reserved_names
 
 
+def _isinstance_listlike(x):
+    """Returns True if x is an iterable which can be transformed into a pandas Series,
+    False for the other types of possible values of a `hover_data` dict.
+    A tuple of length 2 is a special case corresponding to a (format, data) tuple.
+    """
+    if (
+        isinstance(x, str)
+        or (isinstance(x, tuple) and len(x) == 2)
+        or isinstance(x, bool)
+        or x is None
+    ):
+        return False
+    else:
+        return True
+
+
 def build_dataframe(args, attrables, array_attrables):
     """
     Constructs a dataframe and modifies `args` in-place.
@@ -890,7 +929,7 @@ def build_dataframe(args, attrables, array_attrables):
     for field in args:
         if field in array_attrables and args[field] is not None:
             args[field] = (
-                dict(args[field])
+                OrderedDict(args[field])
                 if isinstance(args[field], dict)
                 else list(args[field])
             )
@@ -919,6 +958,19 @@ def build_dataframe(args, attrables, array_attrables):
         else:
             df_output[df_input.columns] = df_input[df_input.columns]
 
+    # hover_data is a dict
+    hover_data_is_dict = (
+        "hover_data" in args
+        and args["hover_data"]
+        and isinstance(args["hover_data"], dict)
+    )
+    # If dict, convert all values of hover_data to tuples to simplify processing
+    if hover_data_is_dict:
+        for k in args["hover_data"]:
+            if _isinstance_listlike(args["hover_data"][k]):
+                args["hover_data"][k] = (True, args["hover_data"][k])
+            if not isinstance(args["hover_data"][k], tuple):
+                args["hover_data"][k] = (args["hover_data"][k], None)
     # Loop over possible arguments
     for field_name in attrables:
         # Massaging variables
@@ -954,6 +1006,16 @@ def build_dataframe(args, attrables, array_attrables):
             if isinstance(argument, str) or isinstance(
                 argument, int
             ):  # just a column name given as str or int
+
+                if (
+                    field_name == "hover_data"
+                    and hover_data_is_dict
+                    and args["hover_data"][str(argument)][1] is not None
+                ):
+                    col_name = str(argument)
+                    df_output[col_name] = args["hover_data"][col_name][1]
+                    continue
+
                 if not df_provided:
                     raise ValueError(
                         "String or int arguments are only possible when a "
@@ -1029,6 +1091,8 @@ def build_dataframe(args, attrables, array_attrables):
             # Finally, update argument with column name now that column exists
             if field_name not in array_attrables:
                 args[field_name] = str(col_name)
+            elif isinstance(args[field_name], dict):
+                pass
             else:
                 args[field_name][i] = str(col_name)
 
@@ -1554,15 +1618,19 @@ def make_figure(args, constructor, trace_patch={}, layout_patch={}):
             cmid=args["color_continuous_midpoint"],
             cmin=range_color[0],
             cmax=range_color[1],
-            colorbar=dict(title=get_decorated_label(args, args[colorvar], colorvar)),
+            colorbar=dict(
+                title_text=get_decorated_label(args, args[colorvar], colorvar)
+            ),
         )
-    for v in ["title", "height", "width"]:
+    for v in ["height", "width"]:
         if args[v]:
             layout_patch[v] = args[v]
     layout_patch["legend"] = dict(tracegroupgap=0)
     if trace_name_labels:
-        layout_patch["legend"]["title"] = ", ".join(trace_name_labels)
-    if "title" not in layout_patch and args["template"].layout.margin.t is None:
+        layout_patch["legend"]["title_text"] = ", ".join(trace_name_labels)
+    if args["title"]:
+        layout_patch["title_text"] = args["title"]
+    elif args["template"].layout.margin.t is None:
         layout_patch["margin"] = {"t": 60}
     if (
         "size" in args
@@ -1592,7 +1660,7 @@ def make_figure(args, constructor, trace_patch={}, layout_patch={}):
 
     # Add traces, layout and frames to figure
     fig.add_traces(frame_list[0]["data"] if len(frame_list) > 0 else [])
-    fig.layout.update(layout_patch)
+    fig.update_layout(layout_patch)
     if "template" in args and args["template"] is not None:
         fig.update_layout(template=args["template"], overwrite=True)
     fig.frames = frame_list if len(frames) > 1 else []

Diff for: packages/python/plotly/plotly/express/_doc.py

@@ -180,8 +180,15 @@
         "Values from this column or array_like appear in bold in the hover tooltip.",
     ],
     hover_data=[
-        colref_list_type,
-        colref_list_desc,
+        "list of str or int, or Series or array-like, or dict",
+        "Either a list of names of columns in `data_frame`, or pandas Series,",
+        "or array_like objects",
+        "or a dict with column names as keys, with values True (for default formatting)",
+        "False (in order to remove this column from hover information),",
+        "or a formatting string, for example ':.3f' or '|%a'",
+        "or list-like data to appear in the hover tooltip",
+        "or tuples with a bool or formatting string as first element,",
+        "and list-like data to appear in hover as second element",
         "Values from these columns appear as extra data in the hover tooltip.",
     ],
     custom_data=[

Diff for: packages/python/plotly/plotly/express/_imshow.py

@@ -244,7 +244,7 @@ def imshow(
             cmax=range_color[1],
         )
         if labels["color"]:
-            layout["coloraxis1"]["colorbar"] = dict(title=labels["color"])
+            layout["coloraxis1"]["colorbar"] = dict(title_text=labels["color"])
 
     # For 2D+RGB data, use Image trace
     elif img.ndim == 3 and img.shape[-1] in [3, 4]:
@@ -262,10 +262,12 @@ def imshow(
         )
 
     layout_patch = dict()
-    for attr_name in ["title", "height", "width"]:
+    for attr_name in ["height", "width"]:
         if args[attr_name]:
             layout_patch[attr_name] = args[attr_name]
-    if "title" not in layout_patch and args["template"].layout.margin.t is None:
+    if args["title"]:
+        layout_patch["title_text"] = args["title"]
+    elif args["template"].layout.margin.t is None:
         layout_patch["margin"] = {"t": 60}
     fig = go.Figure(data=trace, layout=layout)
     fig.update_layout(layout_patch)