Skip to content
This repository was archived by the owner on Sep 11, 2023. It is now read-only.

Experiment with loading entire batches at once #212

Open
Tracked by #213
JackKelly opened this issue Oct 8, 2021 · 0 comments
Open
Tracked by #213

Experiment with loading entire batches at once #212

JackKelly opened this issue Oct 8, 2021 · 0 comments
Assignees
Labels
enhancement New feature or request refactoring

Comments

@JackKelly
Copy link
Member

Detailed Description

The code currently loads invidual training examples, one-at-a-time, and then joins them into a batch.

But it's possible that it might be faster to execute, and the code might be simpler, if we load entire batches at once. (For example, I think Zarr might be able to concurrently load all the examples).

I have a hunch that the "loading single examples at a time" thing might not be necessary once we do #202

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request refactoring
Projects
None yet
Development

No branches or pull requests

1 participant