Skip to content
This repository was archived by the owner on Apr 29, 2020. It is now read-only.

The HAMT exporter loads the whole shard in order to find a leaf #9

Closed
achingbrain opened this issue Dec 2, 2018 · 0 comments
Closed
Assignees

Comments

@achingbrain
Copy link
Collaborator

If the dir-hamt-sharded exporter finds a link that points to a subshard, it loads that subshard even if it won't contain the path we're trying to load.

The hashes generated by the hamt algorithm are stable, however, so you don't need to load the entire tree in order to find out where a given node will be, instead you can go level by level in the hamt, recreating it as you go. We use this approach in js-ipfs-mfs to add/remove DAGLinks without loading the entire graph.

This module should use a similar approach, otherwise we don't see any benefit from sharding.

achingbrain added a commit that referenced this issue Dec 3, 2018
Since hash results are stable based on the file input, we can predict
which subshard will contain a given file based on how deep we are
in the hamt, so there's no need to traverse the entire shard to
find one file.

Fixes #9
@ghost ghost assigned achingbrain Dec 3, 2018
@ghost ghost added the in progress label Dec 3, 2018
@ghost ghost removed the in progress label Dec 3, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant