JSON Exporter does not work with Symlink Nodes Class

[JaimeSandoval]

Trying to use the JSON feature on some trees that have reoccurring Nodes that I turn into SymLink Nodes. Below is a simple case that show cases the issue I'm running into. Any help would be appreciated. I am also looking into how to fix.

from anytree import Node, SymlinkNode
from anytree.exporter import DotExporter, JsonExporter, DictExporter
root = Node(name="root")
a = Node(name="a", parent=root)
b = Node(name="b", parent=root)
c = SymlinkNode(target=a, parent=b)
exporter = JsonExporter(indent=2, sort_keys=False)
print(exporter.export(root))

Using DictExplorer, I was able to see that Node gets passed into the JSON Exporter

exporter = DictExporter()
print(exporter.export(root))

{'name': 'root', 'children': [{'name': 'a'}, {'name': 'b', 'children': [{'target': Node('/root/a')}]}]}

Ideally I would see:
{'name': 'root', 'children': [{'name': 'a'}, {'name': 'b', 'children': [{'name': 'a'}]}]}

Traceback (most recent call last):
File "", line 1, in
File "/Users/jasandov/Repositories/multiproducts/service-mapper-cli/build/service-mapper-cli/environments/development-venv/lib/python3.7/site-packages/anytree/exporter/jsonexporter.py", line 68, in export
return json.dumps(data, **self.kwargs)
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/init.py", line 238, in dumps
**kw).encode(obj)
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 201, in encode
chunks = list(chunks)
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 431, in _iterencode
yield from _iterencode_dict(o, _current_indent_level)
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 325, in _iterencode_list
yield from chunks
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 325, in _iterencode_list
yield from chunks
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
yield from chunks
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 438, in _iterencode
o = _default(o)
File "/usr/local/linkedin/Homebrew/Caskroom/lnkd-cleanpython37/1.6.5/3.7.10/lib/python3.7/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type Node is not JSON serializable

[goldmosh]

JaimeSandoval, que tal?

I am writing a system that its central data structure is an n-ary tree, and I need the possibility of defining nodes that are shared by sub-trees; I did it by using Symlink nodes. I tried to save the n-ary tree in JSON format by using the JSON exporter of the anytree module and I could not do it because I got the same TypeError message that you got.
I see that the developers of the anytree module have not fixed this bug yet.
I want to ask you if you have succeeded to solve the problem by yourself, since Feb/2022. If yes, I will be happy to receive the code of your solution.

Muchas gracias.

Moshe Goldstein

[goldmosh]

I have the following idea, that requires a change in the anytree module code:
In the case of Symlink nodes, the DictExporter generates a dictionary with a pair "target" : value. In your example, that value is a Node object created by the call Node('/root/a'), which is reason of the bug. I thought that instead of creating that Node object as the value of "target", that call may be inserted there as the string "Node('root/a')". This way, the JsonExporter will succeed. When restoring the anytree from its JSON format, the DictImporter called by the JsonImporter will apply the eval function on that string. I have not tried to make the required changes in the anytree module code, but I believe that this will allow the export and the import of the anytree object, to and from JSON format.
What do you think?

[c0fec0de]

will take care within the next days

[goldmosh]

Here I contribute my simple (partial?) solution to this JSON Exporter problem. Here you have the code of the method __export of the DictExporter class, with the change I made. As you see, all what I did in order to allow the exporter to succeed is to change the target_node object to its string representation. In the JSON Importer we will need to implement the reverse operation in the case of the "target" attribute.
What is your opinion about my solution?

def __export(self, node, dictcls, attriter, childiter, level=1):
    def get_pathname(node):
       node_repr = repr(node)
       node_pathname_start = node_repr.find('(') + 2
       node_pathname_end = node_repr.find(',') - 1
       return  node_repr[node_pathname_start:node_pathname_end]
    #
    attr_values = attriter(self._iter_attr_values(node))
    data = dictcls(attr_values)
    # dealing with the problem that the target object of
    # a Symlink connection is not JSON serializable
    if 'target' in data:
        target_node = data['target']
        data['target'] = repr(target_node)
    maxlevel = self.maxlevel
    if maxlevel is None or level < maxlevel:
        children = [
            self.__export(child, dictcls, attriter, childiter, level=level + 1)
            for child in childiter(node.children)
        ]
        if children:
            data["children"] = children
    return data

[goldmosh]

I am happy to tell you that I have a solution to the bugs found in the json exporter and json importer. I did a lot of tests and it works OK. It is possible to export a tree, with Symlink nodes, to a json file and also to import (or to restore) from a json file.
I am attaching the changed versions of dictimporter.py, dictexporter.py, jsonexporter.py, jsonimporter.py. I gave them the names my*.py in order to keep the original version until the changed versions will be finally approved by you and your development team.
Try my version of those modules and tell me if you found any bug(s) that occur in some case(s), that I did not find.

All the best.
Moshe
my_changes_to_anytree.tar.gz

[c0fec0de]

Thanks a lot for your efforts. Will try to pick it up within the next two weeks and create a new release.

[goldmosh]

OK.
Thanks for your prompt reply.

[JaimeSandoval]

Hello, Sorry for the late reply, I haven't really been looking at my personal email for a while. I actually posted a PR to solve for the problems in dictexporter.py which also solves the issue in JSON exporter since it relies on dictexporter. You can look at the PR here: #189 It might not solve all the issues you're encountering, but I believe this is the simplest solution for dictexporter and jsonexporter. I created this almost 2 years ago so the commits are pretty gross.

…

On Sun, Nov 5, 2023 at 10:40 PM goldmosh ***@***.***> wrote: OK. Thanks for your prompt reply. — Reply to this email directly, view it on GitHub <#188 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJ43KO37BY4H5SFK4CVYYYTYDCA57AVCNFSM5N677QBKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGQYTOMZZGM4Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>