Skip to content

Allow passing typed array of ms timestamps for date axis without conversions #6269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nirvana-msu opened this issue Jul 17, 2022 · 2 comments

Comments

@nirvana-msu
Copy link

I need to handle a large number of data points, and I also need lines mode (not markers) so I'm using a "fancy" scattergl plot. Annoyingly, there appears to be no way to just feed ms epoch timestamps to plotly directly as a date axis value (say, as a BigUint64Array)... as a result, there's a huge added overhead added for parsing Date inputs (the most effcient thing it seems to accept currently) back into ms timestamps.

There is seemingly a shortcut here:

function dt2ms(v, _, calendar, opts) {
        if((opts || {}).msUTC && isNumeric(v)) {
            // For now it is only used
            // to fix bar length in milliseconds & gl3d ticks
            // It could be applied in other places in v3
            return +v;
        }
        ...

but the opts are not passed through from makeCalcdata call in scattergl - no way to opt-in for this as a user.

TypedArray is currenty allowed for linear and log axis types here, but date axis just falls through to calling dt2ms, and ultimately to the terribly slow (even for Date inputs, when you need to proccess hundreds of thousands points) dateTime2ms. It is so frustrating there is no way to pass through exactly what is produced in the result anyway.

@judfs
Copy link

judfs commented Nov 14, 2022

I too noticed that Date formatting was limiting my performance. Did a stress test and saw dt2ms as 44% of the profile.

The following screenshots are the same code with just the axis type changed:

type: "date",
image

type: "linear",
image

This is a stress test of a real time plot with a firehose data rate. Eventually my plot updates start breaking and some datapoints fail to get updated. Notice the gaps in the orange trace on the right. I probably have done thing suboptimally, but I think the point holds.

You can see with my demo the linear plot is able to keep up for about 130 (210-80) more seconds and 229,500 (365400 - 135900) more datapoints per trace or 688,500 (_*3) total more points than date.

I would have expected a workaround to be use linear and provide a custom axis format, but that very much seems to not be a thing (#1464)
Tested versions: "react-plotly.js": "2.6.0", "plotly.js": "2.16.1"`

@gvwilson
Copy link
Contributor

gvwilson commented Jul 4, 2024

Hi - we are trying to tidy up the stale issues and PRs in Plotly's public repositories so that we can focus on things that are still important to our community. Since this one has been sitting for several years, I'm going to close it; if it is still a concern, please add a comment letting us know what recent version of our software you've checked it with so that I can reopen it and add it to our backlog. Thanks for your help - @gvwilson

@gvwilson gvwilson closed this as completed Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants