Skip to content

make check should use a performance ratchet #7459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
graydon opened this issue Jun 28, 2013 · 4 comments
Closed

make check should use a performance ratchet #7459

graydon opened this issue Jun 28, 2013 · 4 comments
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc C-enhancement Category: An issue proposing an enhancement or a PR with one. I-slow Issue: Problems and improvements with respect to performance of generated code.

Comments

@graydon
Copy link
Contributor

graydon commented Jun 28, 2013

We should have performance tests that are "not allowed to get worse" in the testsuite. The operation should involve reading a .json file with the last run values, comparing to current-run values, failing if any regress, and rewriting the file if any advanced. Buildbot can potentially deposit the .json file before each build if it's missing, and harvest it after for display / plotting, and users can just work with the one(s) in their own workspaces.

This requires either objective performance criteria or fuzzy forms of comparison. Prefer objective when possible. valgrind --tool=lackey and perf stat can gather number of instructions retired, which tends to be much more stable than clock time.

Probably easier to base this on metrics gathered from #6810

@ghost ghost assigned graydon Jun 28, 2013
bors added a commit that referenced this issue Jul 11, 2013
This is some initial sketch-work for #7461 though it will depend on #7459 to be useful for anything. For the time being, just infrastructure.
@huonw
Copy link
Member

huonw commented Sep 18, 2013

Triage: We now have some/much of the infrastructure for a ratchet, but I'm not sure if it is enabled on the buildbots. (@brson do you happen to know?)

@brson
Copy link
Contributor

brson commented Sep 18, 2013

Not 0.8

@graydon graydon removed their assignment Jun 16, 2014
@steveklabnik
Copy link
Member

Triage: no changes here.

@steveklabnik steveklabnik added the C-enhancement Category: An issue proposing an enhancement or a PR with one. label Apr 28, 2015
@aturon
Copy link
Member

aturon commented Apr 15, 2017

Closing for the time being, as this is a pretty open-ended feature request and is unlikely to be prioritized any time soon.

@aturon aturon closed this as completed Apr 15, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc C-enhancement Category: An issue proposing an enhancement or a PR with one. I-slow Issue: Problems and improvements with respect to performance of generated code.
Projects
None yet
Development

No branches or pull requests

5 participants