Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DOC: Updated link for OVH server benchmark visualization #61108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DOC: Updated link for OVH server benchmark visualization #61108
Changes from 1 commit
830c47bf4e1007afd70bc8202a3437029a4File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rhshadrach IIUC based on https://github.com/pandas-dev/asv-runner we are just runnning the benchmarks on Github Actions now right? If so, we just need https://pandas-dev.github.io/asv-runner/ here right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure the status of the original server point, but if conbench isn't available anymore, we can just remove the comment in brackets. The
- OVH server: asvbullet point is still relevant. We do run the benchmarks there.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@datapythonista - why are we running benchmarks there? Is someone looking at them?
I would like a link to the current ASV page that is being maintained (by me, the 2nd link in @mroeschke's comment).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the late reply, I missed this. I may have partial information, but what I know is that Wes had a machine at home where benchmarks were running. When he moved he sent it to Tom, but we didn't consider this approach reliable, and we wanted to have them running in a data center, not in someone's home. When we signed the agreement with OVH we set them up there. The idea was also to try to make them as fast as possible, so we could run them in PRs. A decent amount of work was done in that direction, but funds were over before we could make them fast enough.
So, if I'm not wrong, what we want in this page is to keep the
Original serverentry as it's Tom's machine. Add an item in the list for the benchmarks running in GitHub Actions that Matt mentions. And keep the OVH server entry, which I guess should have the most stable results.Does that sound good?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tom handed over the reigns of this to me years ago, and the benchmark machine has been running in my house since then. After several outages, I migrated the setup to run in https://github.com/pandas-dev/asv-runner. I no longer have any intention of running the local machine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I personally did not find the interface suitable for issue detection - it took a very long time to load pages if I recall correctly. If no one is looking at this (I do not intend to - is someone else looking at it?), then I don't understand why we would spend the compute (free or not) to produce results that go stale.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't know that part of the story. And seems like discontinuing Tom's server is something recent from just 3 months ago, right? https://github.com/asv-runner/asv-collection
@TomAugspurger would it make sense to delete the `asv-runner organization? We can surely leave it if useful, but since it seems like pandas was the last project to use it, and that's discontinued now, maybe it can avoid some confusion. Just stopping GitHub pages would probably be useful, but if the code isn't needed, maybe good to clean up.
@rhshadrach do you think you can update the benchmarks page so what we discussed is publicly available for anyone? I created it some time ago with all the information I had, but seems like I was missing part of the story. If you have the time, it'd probably also be useful to have a short README in the repo, explaining the goal of the repo, and mentioning that the benchmarks are published with GitHub actions to a branch, which takes a bit of time to find out. And any other relevant information.
With this, I think anyone should be able to understand the current state, make use of the benchmarks, and work on them without needed too much research.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can archive the repos under the asv-runner org if everything has been moved over to https://github.com/pandas-dev/asv-runner.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@datapythonista
Happy to - but I'm not sure what you mean by "the benchmarks page". Can you clarify?
Yea, will do.
@TomAugspurger - Just to be certain, while
pandas-dev/asv-runneris based on the same idea as the one inasv-runner, it doesn't use the setup with ansible. I'm good with archiving the repos underasv-runner, but wanted to make sure it's known that the setup there would no longer be in any active repo.