Skip to content

Enable caching of compiled guvectorize functions #364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 3, 2020

Conversation

tomwhite
Copy link
Collaborator

@tomwhite tomwhite commented Nov 2, 2020

Fixes #363

$ time python -c "import sgkit"
real	0m5.481s
user	0m5.960s
sys	0m0.527s
$ time python -c "import sgkit"
real	0m1.982s
user	0m2.520s
sys	0m0.471s

BTW @aktech I noticed that the guvectorize metric functions don't have nopython=True - do you think we should add it? (I could do it in this PR.)

Copy link
Collaborator

@jeromekelleher jeromekelleher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, minor query

@@ -121,9 +121,7 @@ def diversity(

# c = cohorts, k = alleles
@guvectorize( # type: ignore
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any good reason not to have cache=True for our use-case? Perhaps it's worth us having our own local guvectorize decorator that does this by default?

This is the sort of thing that's hard to test for and easy to forget, so maybe it's worth a little indirection.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe? It's not hard to add and now we have the convention started it is likely to be propagated in our code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #371

@aktech
Copy link
Contributor

aktech commented Nov 2, 2020

BTW @aktech I noticed that the guvectorize metric functions don't have nopython=True - do you think we should add it? (I could do it in this PR.)

That will be a good addition, I think.

@aktech
Copy link
Contributor

aktech commented Nov 2, 2020

This is a really nice improvement, by the way. 👍

@tomwhite tomwhite added the auto-merge Auto merge label for mergify test flight label Nov 3, 2020
@codecov-io
Copy link

codecov-io commented Nov 3, 2020

Codecov Report

Merging #364 into master will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #364      +/-   ##
==========================================
+ Coverage   95.39%   95.41%   +0.01%     
==========================================
  Files          31       31              
  Lines        2108     2116       +8     
==========================================
+ Hits         2011     2019       +8     
  Misses         97       97              
Impacted Files Coverage Δ
sgkit/distance/metrics.py 21.21% <ø> (ø)
sgkit/stats/aggregation.py 87.77% <ø> (ø)
sgkit/stats/popgen.py 67.74% <ø> (ø)
sgkit/variables.py 96.42% <ø> (ø)
sgkit/__init__.py 100.00% <100.00%> (ø)
sgkit/utils.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf33b01...d31fa1a. Read the comment docs.

@mergify mergify bot merged commit d9101ef into sgkit-dev:master Nov 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-merge Auto merge label for mergify test flight
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Importing sgkit is slow
4 participants