Skip to content

Make it easier to cache eval-time dependencies. #728

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TravisWhitaker opened this issue Jun 26, 2020 · 7 comments
Closed

Make it easier to cache eval-time dependencies. #728

TravisWhitaker opened this issue Jun 26, 2020 · 7 comments

Comments

@TravisWhitaker
Copy link
Contributor

It would be nice to have all of the eval time packages that Haskell.nix requires available in one place. Without this, as far as I can tell one must manually work out the required packages and cache them separately. Things like nix-store -q -R --include-outputs ... don't know about dependencies that come as a consequence of IFD. If you get this wrong, users downstream of Haskell.nix (who might not be writing Haskell at all, merely depending on Haskell packages) will need to wait for e.g. nix-tools to build.

Ideally, one would be able to do something like nix copy --to my-binary-cache $(nix-store -q -R --include-outputs $(nix-instantiate -A haskell-nix.evalDeps)) to get everything needed at eval time on the given platform in their cache. If this already exists and I'm just too dense to find it, a clearer note in the docs might help.

@TravisWhitaker
Copy link
Contributor Author

To be clear, I'm not talking about materialization or otherwise saving the Nix generated by IFD. I'm totally fine with using IFD, I'd just like to cache all of the exes/libraries needed at eval time.

@hamishmack
Copy link
Collaborator

Sorry this is not documented, but please try nix copy --to my-binary-cache $(nix-store -q -R --include-outputs $(nix-instantiate -A haskell-nix.haskellNixRoots)). Make sure defaultCompilerNixName is set as it will only build the roots for that version of ghc. Please let us know if there is anything missing.

@TravisWhitaker
Copy link
Contributor Author

That's precisely what I was looking for, thanks! I'll leave this open and add a note to the docs.

@michaelpj
Copy link
Collaborator

Make sure defaultCompilerNixName is set as it will only build the roots for that version of ghc. Please let us know if there is anything missing.

Damn it, I've been doing this completely wrong (down with defaultCompilerNixName!). Petition to add something like haskellNixRoots to the output of project so we can pin that and get the right thing.

@TravisWhitaker
Copy link
Contributor Author

Hmm, I think there's something missing for shellFor. With a clean cache I can eval package components without building anything (since haskellNixRoots is cached), but GHC is built all over again when shellFor is evaluated. Trying to pare down to a minimal repro...

@TravisWhitaker
Copy link
Contributor Author

Narrowed it down to something required by

pkgs.buildPackages.haskell-nix.tool "hoogle" {
        version = "5.0.17.15";
        index-state = pkgs.haskell-nix.internalHackageIndexState;
      }

in builder/default.nix. This is the Hoogle used for hoogleLocal in shellFor.

@TravisWhitaker
Copy link
Contributor Author

^^^ This specific case looks to be solved by #738

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants