dj.diagram not working after update to 14.1

[troselab-setup]

Hi all, has there been a change to dj.Diagram? I updated to dj 14.1 and now I am receiving this error:

`---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
File ~/miniconda3/envs/datajoint/lib/python3.8/site-packages/IPython/core/formatters.py:343, in BaseFormatter.call(self, obj)
341 method = get_real_method(obj, self.print_method)
342 if method is not None:
--> 343 return method()
344 return None
345 else:

File ~/miniconda3/envs/datajoint/lib/python3.8/site-packages/datajoint/diagram.py:436, in Diagram.repr_svg(self)
435 def repr_svg(self):
--> 436 return self.make_svg().repr_svg()

File ~/miniconda3/envs/datajoint/lib/python3.8/site-packages/datajoint/diagram.py:424, in Diagram.make_svg(self)
421 def make_svg(self):
422 from IPython.display import SVG
--> 424 return SVG(self.make_dot().create_svg())

File ~/miniconda3/envs/datajoint/lib/python3.8/site-packages/datajoint/diagram.py:371, in Diagram.make_dot(self)
308 label_props = { # http://matplotlib.org/examples/color/named_colors.html
309 None: dict(
310 shape="circle",
(...)
364 ),
365 }
366 node_props = {
367 node: label_props[d["node_type"]]
368 for node, d in dict(graph.nodes(data=True)).items()
369 }
--> 371 dot = nx.drawing.nx_pydot.to_pydot(graph)
372 for node in dot.get_nodes():
373 node.set_shape("circle")

File ~/miniconda3/envs/datajoint/lib/python3.8/site-packages/networkx/drawing/nx_pydot.py:309, in to_pydot(N)
298 raise_error = (
299 _check_colon_quotes(u)
300 or _check_colon_quotes(v)
(...)
306 )
307 )
308 if raise_error:
--> 309 raise ValueError(
310 f'Node names and attributes should not contain ":" unless they are quoted with "".
311 For example the string 'attribute:data1' should be written as '"attribute:data1"'.
312 Please refer pydot/pydot#258'
313 )
314 edge = pydot.Edge(u, v, **str_edgedata)
315 P.add_edge(edge)

ValueError: Node names and attributes should not contain ":" unless they are quoted with "". For example the string 'attribute:data1' should be written as '"attribute:data1"'. Please refer https://github.com/pydot/pydot/issues/258`

Reproducibility

Include:

• Linux
• Python 3.8.16

[troselab-setup]

I see this has been raised already here:
#1065 (comment)

[CBroz1]

I would describe this issue as 'networkx rejects dj.Diagrams with attribute mapping'

Minimal replication

import datajoint as dj

schema = dj.schema("temp_demo")

@schema
class A(dj.Manual):
    definition = """
    a : int
    """

@schema
class B(dj.Manual):
    definition = """
    b : int
    -> A
    """

if __name__ == "__main__":
    (dj.Diagram(B) - 1).draw()

Error stack

🌸 python 3.9.16 🌸
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~/wrk/spyglass/temp-demo.py:22
     15     definition = """
     16     b : int
     17     -> A
     18     """
     21 if __name__ == "__main__":
---> 22     (dj.Diagram(B) - 1).draw()

File ~/wrk/datajoint-python/datajoint/diagram.py:440, in Diagram.draw(self)
    438 def draw(self):
    439     if plot_active:
--> 440         plt.imshow(self.make_image())
    441         plt.gca().axis("off")
    442         plt.show()

File ~/wrk/datajoint-python/datajoint/diagram.py:431, in Diagram.make_image(self)
    429 def make_image(self):
    430     if plot_active:
--> 431         return plt.imread(self.make_png())
    432     else:
    433         raise DataJointError("pyplot was not imported")

File ~/wrk/datajoint-python/datajoint/diagram.py:427, in Diagram.make_png(self)
    426 def make_png(self):
--> 427     return io.BytesIO(self.make_dot().create_png())

File ~/wrk/datajoint-python/datajoint/diagram.py:371, in Diagram.make_dot(self)
    308 label_props = {  # http://matplotlib.org/examples/color/named_colors.html
    309     None: dict(
    310         shape="circle",
   (...)
    364     ),
    365 }
    366 node_props = {
    367     node: label_props[d["node_type"]]
    368     for node, d in dict(graph.nodes(data=True)).items()
    369 }
--> 371 dot = nx.drawing.nx_pydot.to_pydot(graph)
    372 for node in dot.get_nodes():
    373     node.set_shape("circle")

File ~/miniconda3/envs/spy/lib/python3.9/site-packages/networkx/drawing/nx_pydot.py:311, in to_pydot(N)
    300 raise_error = (
    301     _check_colon_quotes(u)
    302     or _check_colon_quotes(v)
   (...)
    308     )
    309 )
    310 if raise_error:
--> 311     raise ValueError(
    312         f'Node names and attributes should not contain ":" unless they are quoted with "".\
    313         For example the string \'attribute:data1\' should be written as \'"attribute:data1"\'.\
    314         Please refer https://github.com/pydot/pydot/issues/258'
    315     )
    316 edge = pydot.Edge(u, v, **str_edgedata)
    317 P.add_edge(edge)

ValueError: Node names and attributes should not contain ":" unless they are quoted with "".                    For example the string 'attribute:data1' should be written as '"attribute:data1"'.                    Please refer https://github.com/pydot/pydot/issues/258

[trose-neuro]

Hi! Is there an update to this?

[dimitri-yatsenko]

fixing this week :-)

[ethho]

Can reproduce with tests/test_erd.py on b755c84 in a Dev Container running Python 3.11:

Details

vscode ➜ /workspaces/datajoint-python (master) $ pytest tests/test_erd.py 
========================================================= test session starts ==========================================================
platform linux -- Python 3.11.4, pytest-8.2.2, pluggy-1.5.0
rootdir: /workspaces/datajoint-python
plugins: Faker-28.4.1, cov-5.0.0
collected 7 items                                                                                                                      

tests/test_erd.py ....FF.                                                                                                        [100%]

=============================================================== FAILURES ===============================================================
____________________________________________________________ test_repr_svg _____________________________________________________________

schema_adv = Schema `djtest_advanced`


    def test_repr_svg(schema_adv):
        erd = dj.ERD(schema_adv, context=dict())
>       svg = erd._repr_svg_()

tests/test_erd.py:53: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
datajoint/diagram.py:440: in _repr_svg_
    return self.make_svg()._repr_svg_()
datajoint/diagram.py:428: in make_svg
    return SVG(self.make_dot().create_svg())
datajoint/diagram.py:371: in make_dot
    dot = nx.drawing.nx_pydot.to_pydot(graph)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

N = <networkx.classes.digraph.DiGraph object at 0x7fd4fc48c390>

    def to_pydot(N):
        """Returns a pydot graph from a NetworkX graph N.
    
        Parameters
        ----------
        N : NetworkX graph
          A graph created with NetworkX
    
        Examples
        --------
        >>> K5 = nx.complete_graph(5)
        >>> P = nx.nx_pydot.to_pydot(K5)
    
        Notes
        -----
    
        """
        import pydot
    
        # set Graphviz graph type
        if N.is_directed():
            graph_type = "digraph"
        else:
            graph_type = "graph"
        strict = nx.number_of_selfloops(N) == 0 and not N.is_multigraph()
    
        name = N.name
        graph_defaults = N.graph.get("graph", {})
        if name == "":
            P = pydot.Dot("", graph_type=graph_type, strict=strict, **graph_defaults)
        else:
            P = pydot.Dot(
                f'"{name}"', graph_type=graph_type, strict=strict, **graph_defaults
            )
        try:
            P.set_node_defaults(**N.graph["node"])
        except KeyError:
            pass
        try:
            P.set_edge_defaults(**N.graph["edge"])
        except KeyError:
            pass
    
        for n, nodedata in N.nodes(data=True):
            str_nodedata = {str(k): str(v) for k, v in nodedata.items()}
            # Explicitly catch nodes with ":" in node names or nodedata.
            n = str(n)
            raise_error = _check_colon_quotes(n) or (
                any(
                    (_check_colon_quotes(k) or _check_colon_quotes(v))
                    for k, v in str_nodedata.items()
                )
            )
            if raise_error:
                raise ValueError(
                    f'Node names and attributes should not contain ":" unless they are quoted with "".\
                    For example the string \'attribute:data1\' should be written as \'"attribute:data1"\'.\
                    Please refer https://github.com/pydot/pydot/issues/258'
                )
            p = pydot.Node(n, **str_nodedata)
            P.add_node(p)
    
        if N.is_multigraph():
            for u, v, key, edgedata in N.edges(data=True, keys=True):
                str_edgedata = {str(k): str(v) for k, v in edgedata.items() if k != "key"}
                u, v = str(u), str(v)
                raise_error = (
                    _check_colon_quotes(u)
                    or _check_colon_quotes(v)
                    or (
                        any(
                            (_check_colon_quotes(k) or _check_colon_quotes(val))
                            for k, val in str_edgedata.items()
                        )
                    )
                )
                if raise_error:
                    raise ValueError(
                        f'Node names and attributes should not contain ":" unless they are quoted with "".\
                        For example the string \'attribute:data1\' should be written as \'"attribute:data1"\'.\
                        Please refer https://github.com/pydot/pydot/issues/258'
                    )
                edge = pydot.Edge(u, v, key=str(key), **str_edgedata)
                P.add_edge(edge)
    
        else:
            for u, v, edgedata in N.edges(data=True):
                str_edgedata = {str(k): str(v) for k, v in edgedata.items()}
                u, v = str(u), str(v)
                raise_error = (
                    _check_colon_quotes(u)
                    or _check_colon_quotes(v)
                    or (
                        any(
                            (_check_colon_quotes(k) or _check_colon_quotes(val))
                            for k, val in str_edgedata.items()
                        )
                    )
                )
                if raise_error:
>                   raise ValueError(
                        f'Node names and attributes should not contain ":" unless they are quoted with "".\
                        For example the string \'attribute:data1\' should be written as \'"attribute:data1"\'.\
                        Please refer https://github.com/pydot/pydot/issues/258'
                    )
E                   ValueError: Node names and attributes should not contain ":" unless they are quoted with "".                    For example the string 'attribute:data1' should be written as '"attribute:data1"'.                    Please refer https://github.com/pydot/pydot/issues/258

/usr/local/lib/python3.11/site-packages/networkx/drawing/nx_pydot.py:282: ValueError
___________________________________________________________ test_make_image ____________________________________________________________

schema_simp = Schema `djtest_relational`


    def test_make_image(schema_simp):
        erd = dj.ERD(schema_simp, context=dict())
>       img = erd.make_image()

tests/test_erd.py:59: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
datajoint/diagram.py:435: in make_image
    return plt.imread(self.make_png())
datajoint/diagram.py:431: in make_png
    return io.BytesIO(self.make_dot().create_png())
datajoint/diagram.py:371: in make_dot
    dot = nx.drawing.nx_pydot.to_pydot(graph)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

N = <networkx.classes.digraph.DiGraph object at 0x7fd4fc290a10>

    def to_pydot(N):
        """Returns a pydot graph from a NetworkX graph N.
    
        Parameters
        ----------
        N : NetworkX graph
          A graph created with NetworkX
    
        Examples
        --------
        >>> K5 = nx.complete_graph(5)
        >>> P = nx.nx_pydot.to_pydot(K5)
    
        Notes
        -----
    
        """
        import pydot
    
        # set Graphviz graph type
        if N.is_directed():
            graph_type = "digraph"
        else:
            graph_type = "graph"
        strict = nx.number_of_selfloops(N) == 0 and not N.is_multigraph()
    
        name = N.name
        graph_defaults = N.graph.get("graph", {})
        if name == "":
            P = pydot.Dot("", graph_type=graph_type, strict=strict, **graph_defaults)
        else:
            P = pydot.Dot(
                f'"{name}"', graph_type=graph_type, strict=strict, **graph_defaults
            )
        try:
            P.set_node_defaults(**N.graph["node"])
        except KeyError:
            pass
        try:
            P.set_edge_defaults(**N.graph["edge"])
        except KeyError:
            pass
    
        for n, nodedata in N.nodes(data=True):
            str_nodedata = {str(k): str(v) for k, v in nodedata.items()}
            # Explicitly catch nodes with ":" in node names or nodedata.
            n = str(n)
            raise_error = _check_colon_quotes(n) or (
                any(
                    (_check_colon_quotes(k) or _check_colon_quotes(v))
                    for k, v in str_nodedata.items()
                )
            )
            if raise_error:
                raise ValueError(
                    f'Node names and attributes should not contain ":" unless they are quoted with "".\
                    For example the string \'attribute:data1\' should be written as \'"attribute:data1"\'.\
                    Please refer https://github.com/pydot/pydot/issues/258'
                )
            p = pydot.Node(n, **str_nodedata)
            P.add_node(p)
    
        if N.is_multigraph():
            for u, v, key, edgedata in N.edges(data=True, keys=True):
                str_edgedata = {str(k): str(v) for k, v in edgedata.items() if k != "key"}
                u, v = str(u), str(v)
                raise_error = (
                    _check_colon_quotes(u)
                    or _check_colon_quotes(v)
                    or (
                        any(
                            (_check_colon_quotes(k) or _check_colon_quotes(val))
                            for k, val in str_edgedata.items()
                        )
                    )
                )
                if raise_error:
                    raise ValueError(
                        f'Node names and attributes should not contain ":" unless they are quoted with "".\
                        For example the string \'attribute:data1\' should be written as \'"attribute:data1"\'.\
                        Please refer https://github.com/pydot/pydot/issues/258'
                    )
                edge = pydot.Edge(u, v, key=str(key), **str_edgedata)
                P.add_edge(edge)
    
        else:
            for u, v, edgedata in N.edges(data=True):
                str_edgedata = {str(k): str(v) for k, v in edgedata.items()}
                u, v = str(u), str(v)
                raise_error = (
                    _check_colon_quotes(u)
                    or _check_colon_quotes(v)
                    or (
                        any(
                            (_check_colon_quotes(k) or _check_colon_quotes(val))
                            for k, val in str_edgedata.items()
                        )
                    )
                )
                if raise_error:
>                   raise ValueError(
                        f'Node names and attributes should not contain ":" unless they are quoted with "".\
                        For example the string \'attribute:data1\' should be written as \'"attribute:data1"\'.\
                        Please refer https://github.com/pydot/pydot/issues/258'
                    )
E                   ValueError: Node names and attributes should not contain ":" unless they are quoted with "".                    For example the string 'attribute:data1' should be written as '"attribute:data1"'.                    Please refer https://github.com/pydot/pydot/issues/258

/usr/local/lib/python3.11/site-packages/networkx/drawing/nx_pydot.py:282: ValueError
======================================================= short test summary info ========================================================
FAILED tests/test_erd.py::test_repr_svg - ValueError: Node names and attributes should not contain ":" unless they are quoted with "".                    For example the str...
FAILED tests/test_erd.py::test_make_image - ValueError: Node names and attributes should not contain ":" unless they are quoted with "".                    For example the str...
===================================================== 2 failed, 5 passed in 32.77s =====================================================
vscode ➜ /workspaces/datajoint-python (master) $ conda list
bash: conda: command not found
vscode ➜ /workspaces/datajoint-python (master) $ pip list
Package              Version     Editable project location
-------------------- ----------- ----------------------------
appdirs              1.4.4
argon2-cffi          23.1.0
argon2-cffi-bindings 21.2.0
asttokens            2.4.1
black                24.8.0
blinker              1.8.2
certifi              2024.8.30
cffi                 1.17.1
click                8.1.7
comm                 0.2.2
contourpy            1.3.0
cov-core             1.15.0
coverage             7.6.1
cryptography         43.0.1
cycler               0.12.1
datajoint            0.14.2      /workspaces/datajoint-python
debugpy              1.8.5
decorator            5.1.1
executing            2.1.0
Faker                28.4.1
flake8               7.1.1
Flask                3.0.3
fonttools            4.53.1
gitdb                4.0.11
GitPython            3.1.41
iniconfig            2.0.0
ipykernel            6.29.5
ipython              8.27.0
itsdangerous         2.2.0
jedi                 0.19.1
Jinja2               3.1.4
jupyter_client       8.6.2
jupyter_core         5.7.2
kiwisolver           1.4.7
MarkupSafe           2.1.5
matplotlib           3.9.2
matplotlib-inline    0.1.7
mccabe               0.7.0
minio                7.2.8
mypy-extensions      1.0.0
nest-asyncio         1.6.0
networkx             3.3
nose                 1.3.7
nose-cov             1.6
numpy                2.1.1
otumat               0.3.1
packaging            24.1
pandas               2.2.2
parso                0.8.4
pathspec             0.12.1
pexpect              4.9.0
pillow               10.4.0
pip                  23.1.2
platformdirs         4.2.2
pluggy               1.5.0
prompt_toolkit       3.0.47
psutil               6.0.0
ptyprocess           0.7.0
pure_eval            0.2.3
pycodestyle          2.12.1
pycparser            2.22
pycryptodome         3.20.0
pydot                3.0.1
pyflakes             3.2.0
Pygments             2.18.0
PyMySQL              1.1.1
pyparsing            3.1.4
pytest               8.3.2
pytest-cov           5.0.0
python-dateutil      2.9.0.post0
pytz                 2024.1
pyzmq                26.2.0
setuptools           69.0.3
six                  1.16.0
smmap                5.0.1
stack-data           0.6.3
tornado              6.4.1
tqdm                 4.66.5
traitlets            5.14.3
typing_extensions    4.12.2
tzdata               2024.1
urllib3              2.2.2
watchdog             5.0.2
wcwidth              0.2.13
Werkzeug             3.0.4
wheel                0.40.0

[notice] A new release of pip is available: 23.1.2 -> 24.2
[notice] To update, run: pip install --upgrade pip
vscode ➜ /workspaces/datajoint-python (master) $ python --version
Python 3.11.4

[ethho]

Duplicates #1065