deepcopy of dict with scatter.Line fails

[jhmenke]

while creating a Figure, which contains several scatter.Line elements, there is an error in the current plotly release (installed via pip):

Call:

fig = Figure(data=traces,  # <-- this contains the "bad" dicts
                 layout=Layout(
                     titlefont=dict(size=16),
                     showlegend=showlegend,
                     autosize=True if aspectratio is 'auto' else False,
                     hovermode='closest',
                     margin=dict(b=5, l=5, r=5, t=5),
                     xaxis=XAxis(showgrid=False, zeroline=False, showticklabels=False),
                     yaxis=YAxis(showgrid=False, zeroline=False, showticklabels=False)
                 ), )

StackTrace:

File "..\traces.py", line 599, in draw_traces
yaxis=YAxis(showgrid=False, zeroline=False, showticklabels=False),
File "C:\Anaconda3\lib\site-packages\plotly\graph_objs_figure.py", line 336, in init
super(Figure, self).init(data, layout, frames)
File "C:\Anaconda3\lib\site-packages\plotly\basedatatypes.py", line 116, in init
data = self._data_validator.validate_coerce(data)
File "C:\Anaconda3\lib\site-packages_plotly_utils\basevalidators.py", line 2034, in validate_coerce
v_copy = deepcopy(v_el)
File "C:\Anaconda3\lib\copy.py", line 150, in deepcopy
y = copier(x, memo)
File "C:\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\Anaconda3\lib\copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "C:\Anaconda3\lib\copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "C:\Anaconda3\lib\copy.py", line 150, in deepcopy
y = copier(x, memo)
File "C:\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\Anaconda3\lib\copy.py", line 150, in deepcopy
y = copier(x, memo)
File "C:\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\Anaconda3\lib\copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "C:\Anaconda3\lib\copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "C:\Anaconda3\lib\copy.py", line 150, in deepcopy
y = copier(x, memo)
File "C:\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\Anaconda3\lib\copy.py", line 150, in deepcopy
y = copier(x, memo)
File "C:\Anaconda3\lib\copy.py", line 215, in _deepcopy_list
append(deepcopy(a, memo))
File "C:\Anaconda3\lib\copy.py", line 161, in deepcopy
y = copier(memo)
TypeError: cannot deepcopy this pattern object

Minimal example:

import pickle
no_copy_dict = pickle.loads(b'\x80\x03}q\x00(X\x04\x00\x00\x00typeq\x01X\x07\x00\x00\x00scatterq\x02X\x04\x00\x00\x00textq\x03X\x04\x00\x00\x00lineq\x04X\t\x00\x00\x00hoverinfoq\x05h\x03X\x04\x00\x00\x00modeq\x06X\x05\x00\x00\x00linesq\x07X\x04\x00\x00\x00nameq\x08h\x07X\x04\x00\x00\x00lineq\tcplotly.graph_objs.scatter._line\nLine\nq\n)\x81q\x0b}q\x0c(X\x0c\x00\x00\x00_plotly_nameq\rh\tX\x0b\x00\x00\x00_validatorsq\x0e}q\x0f(X\x05\x00\x00\x00colorq\x10cplotly.validators.scatter.line._color\nColorValidator\nq\x11)\x81q\x12}q\x13(X\x0b\x00\x00\x00parent_nameq\x14X\x0c\x00\x00\x00scatter.lineq\x15X\x0b\x00\x00\x00plotly_nameq\x16h\x10X\x04\x00\x00\x00roleq\x17X\x05\x00\x00\x00styleq\x18X\x08\x00\x00\x00array_okq\x19\x89X\x0f\x00\x00\x00colorscale_pathq\x1aNubX\x04\x00\x00\x00dashq\x1bcplotly.validators.scatter.line._dash\nDashValidator\nq\x1c)\x81q\x1d}q\x1e(h\x14X\x0c\x00\x00\x00scatter.lineq\x1fh\x16h\x1bh\x17h\x18X\x06\x00\x00\x00valuesq ]q!(X\x05\x00\x00\x00solidq"X\x03\x00\x00\x00dotq#h\x1bX\x08\x00\x00\x00longdashq$X\x07\x00\x00\x00dashdotq%X\x0b\x00\x00\x00longdashdotq&X4\x00\x00\x00/^\\d+(\\.\\d+)?(px|%)?((,|\\s)\\s*\\d+(\\.\\d+)?(px|%)?)*$/q\'eh\x19\x89X\r\x00\x00\x00coerce_numberq(\x89X\n\x00\x00\x00val_regexsq)]q*(NNNNNNcre\n_compile\nq+X2\x00\x00\x00^\\d+(\\.\\d+)?(px|%)?((,|\\s)\\s*\\d+(\\.\\d+)?(px|%)?)*$q,K \x86q-Rq.eX\x12\x00\x00\x00regex_replacementsq/]q0(NNNNNNNeubX\x05\x00\x00\x00shapeq1cplotly.validators.scatter.line._shape\nShapeValidator\nq2)\x81q3}q4(h\x14X\x0c\x00\x00\x00scatter.lineq5h\x16h1h\x17h\x18h ]q6(X\x06\x00\x00\x00linearq7X\x06\x00\x00\x00splineq8X\x02\x00\x00\x00hvq9X\x02\x00\x00\x00vhq:X\x03\x00\x00\x00hvhq;X\x03\x00\x00\x00vhvq<eh\x19\x89h(\x89h)]q=(NNNNNNeh/]q>(NNNNNNeubX\x08\x00\x00\x00simplifyq?cplotly.validators.scatter.line._simplify\nSimplifyValidator\nq@)\x81qA}qB(h\x14X\x0c\x00\x00\x00scatter.lineqCh\x16h?h\x17X\x04\x00\x00\x00infoqDubX\t\x00\x00\x00smoothingqEcplotly.validators.scatter.line._smoothing\nSmoothingValidator\nqF)\x81qG}qH(h\x14X\x0c\x00\x00\x00scatter.lineqIh\x16hEh\x17h\x18X\x07\x00\x00\x00min_valqJK\x00X\x07\x00\x00\x00max_valqKG?\xf4\xcc\xcc\xcc\xcc\xcc\xcdX\x0b\x00\x00\x00has_min_maxqL\x88h\x19\x89ubX\x05\x00\x00\x00widthqMcplotly.validators.scatter.line._width\nWidthValidator\nqN)\x81qO}qP(h\x14X\x0c\x00\x00\x00scatter.lineqQh\x16hMh\x17h\x18hJK\x00hKG\x7f\xf0\x00\x00\x00\x00\x00\x00hL\x88h\x19\x89ubuX\x0f\x00\x00\x00_compound_propsqR}qSX\x15\x00\x00\x00_compound_array_propsqT}qUX\r\x00\x00\x00_orphan_propsqV}qW(h\x10X\x12\x00\x00\x00rgb(128, 128, 128)qXhMK\x01uX\x07\x00\x00\x00_parentqYNX\x11\x00\x00\x00_change_callbacksqZ}q[ubX\x01\x00\x00\x00xq\\]q](G\xc1B\x089\xff\xb4](G\xc1B>\xf5|\x9d\x82\xf6G\xc1Bu\xb0\xf9\x86\xa8\xc4eX\x01\x00\x00\x00yq^]q_(GAa\xff\xc7\xcf\xa4\xd2\x90GAb\x00D\xcf\xa4\xd2\x90GAb\x00\xc1\xcf\xa4\xd2\x90eu.')
from copy import deepcopy
deepcopy(no_copy_dict)

am i doing something wrong or is this a bug?

[jonmmease]

Hi @jhmenke, thanks for taking the time to write this up! deepcopy on figures and graph_objs isn't supported quite yet (as you can see 🙂). But it's something I'm actively working on for 3.3.0 (See #1191).

Here's the output of your minimal example using that branch:

import pickle
no_copy_dict = pickle.loads(b'\x80\x03}q\x00(X\x04\x00\x00\x00typeq\x01X\x07\x00\x00\x00scatterq\x02X\x04\x00\x00\x00textq\x03X\x04\x00\x00\x00lineq\x04X\t\x00\x00\x00hoverinfoq\x05h\x03X\x04\x00\x00\x00modeq\x06X\x05\x00\x00\x00linesq\x07X\x04\x00\x00\x00nameq\x08h\x07X\x04\x00\x00\x00lineq\tcplotly.graph_objs.scatter._line\nLine\nq\n)\x81q\x0b}q\x0c(X\x0c\x00\x00\x00_plotly_nameq\rh\tX\x0b\x00\x00\x00_validatorsq\x0e}q\x0f(X\x05\x00\x00\x00colorq\x10cplotly.validators.scatter.line._color\nColorValidator\nq\x11)\x81q\x12}q\x13(X\x0b\x00\x00\x00parent_nameq\x14X\x0c\x00\x00\x00scatter.lineq\x15X\x0b\x00\x00\x00plotly_nameq\x16h\x10X\x04\x00\x00\x00roleq\x17X\x05\x00\x00\x00styleq\x18X\x08\x00\x00\x00array_okq\x19\x89X\x0f\x00\x00\x00colorscale_pathq\x1aNubX\x04\x00\x00\x00dashq\x1bcplotly.validators.scatter.line._dash\nDashValidator\nq\x1c)\x81q\x1d}q\x1e(h\x14X\x0c\x00\x00\x00scatter.lineq\x1fh\x16h\x1bh\x17h\x18X\x06\x00\x00\x00valuesq ]q!(X\x05\x00\x00\x00solidq"X\x03\x00\x00\x00dotq#h\x1bX\x08\x00\x00\x00longdashq$X\x07\x00\x00\x00dashdotq%X\x0b\x00\x00\x00longdashdotq&X4\x00\x00\x00/^\\d+(\\.\\d+)?(px|%)?((,|\\s)\\s*\\d+(\\.\\d+)?(px|%)?)*$/q\'eh\x19\x89X\r\x00\x00\x00coerce_numberq(\x89X\n\x00\x00\x00val_regexsq)]q*(NNNNNNcre\n_compile\nq+X2\x00\x00\x00^\\d+(\\.\\d+)?(px|%)?((,|\\s)\\s*\\d+(\\.\\d+)?(px|%)?)*$q,K \x86q-Rq.eX\x12\x00\x00\x00regex_replacementsq/]q0(NNNNNNNeubX\x05\x00\x00\x00shapeq1cplotly.validators.scatter.line._shape\nShapeValidator\nq2)\x81q3}q4(h\x14X\x0c\x00\x00\x00scatter.lineq5h\x16h1h\x17h\x18h ]q6(X\x06\x00\x00\x00linearq7X\x06\x00\x00\x00splineq8X\x02\x00\x00\x00hvq9X\x02\x00\x00\x00vhq:X\x03\x00\x00\x00hvhq;X\x03\x00\x00\x00vhvq<eh\x19\x89h(\x89h)]q=(NNNNNNeh/]q>(NNNNNNeubX\x08\x00\x00\x00simplifyq?cplotly.validators.scatter.line._simplify\nSimplifyValidator\nq@)\x81qA}qB(h\x14X\x0c\x00\x00\x00scatter.lineqCh\x16h?h\x17X\x04\x00\x00\x00infoqDubX\t\x00\x00\x00smoothingqEcplotly.validators.scatter.line._smoothing\nSmoothingValidator\nqF)\x81qG}qH(h\x14X\x0c\x00\x00\x00scatter.lineqIh\x16hEh\x17h\x18X\x07\x00\x00\x00min_valqJK\x00X\x07\x00\x00\x00max_valqKG?\xf4\xcc\xcc\xcc\xcc\xcc\xcdX\x0b\x00\x00\x00has_min_maxqL\x88h\x19\x89ubX\x05\x00\x00\x00widthqMcplotly.validators.scatter.line._width\nWidthValidator\nqN)\x81qO}qP(h\x14X\x0c\x00\x00\x00scatter.lineqQh\x16hMh\x17h\x18hJK\x00hKG\x7f\xf0\x00\x00\x00\x00\x00\x00hL\x88h\x19\x89ubuX\x0f\x00\x00\x00_compound_propsqR}qSX\x15\x00\x00\x00_compound_array_propsqT}qUX\r\x00\x00\x00_orphan_propsqV}qW(h\x10X\x12\x00\x00\x00rgb(128, 128, 128)qXhMK\x01uX\x07\x00\x00\x00_parentqYNX\x11\x00\x00\x00_change_callbacksqZ}q[ubX\x01\x00\x00\x00xq\\]q](G\xc1B\x089\xff\xb4](G\xc1B>\xf5|\x9d\x82\xf6G\xc1Bu\xb0\xf9\x86\xa8\xc4eX\x01\x00\x00\x00yq^]q_(GAa\xff\xc7\xcf\xa4\xd2\x90GAb\x00D\xcf\xa4\xd2\x90GAb\x00\xc1\xcf\xa4\xd2\x90eu.')
from copy import deepcopy
deepcopy(no_copy_dict)

Out[2]: 
{'type': 'scatter',
 'text': 'line',
 'hoverinfo': 'text',
 'mode': 'lines',
 'name': 'lines',
 'line': scatter.Line({
     'color': 'rgb(128, 128, 128)', 'width': 1
 }),
 'x': [-2363507.997691769, -2391530.9735568715, -2419553.949421974],
 'y': [9436734.488869935, 9437734.488869935, 9438734.488869935]}

[jhmenke]

Thanks for the answer.
The thing is, i'm not trying to copy anything manually. The deepcopy is inside the plotly file graph_objs_figure.py, i'm just trying to create a figure.
The deepcopy is called in the class initialization of the plotly Figure: super(Figure, self).init(data, layout, frames)

[jonmmease]

Oh ok, I expect this will fix that problem as well. But just to be sure, could you add a full example that produces the error while constructing a Figure? Thanks!

[jhmenke]

sure. here is the data file for the example below: pdump.zip

from plotly.graph_objs import Figure, Layout
from plotly.graph_objs.layout import XAxis, YAxis
from plotly.offline import plot as plot

from pickle import load

traces = load(open("pdump.p", "rb"))
fig = Figure(data=traces,
             layout=Layout(
                titlefont=dict(size=16),
                showlegend=True,
                autosize=True,
                hovermode='closest',
                margin=dict(b=5, l=5, r=5, t=5),
                xaxis=XAxis(showgrid=False, zeroline=False, showticklabels=False),
                yaxis=YAxis(showgrid=False, zeroline=False, showticklabels=False)
             ))
plot(fig, filename="temp.html")

[jonmmease]

Thanks a lot for the example @jhmenke! I just tested it on master and I can confirm that it's working:

[jonmmease]

Close by #1191

[jhmenke]

thanks