No way of passing pip arguments when creating an environment

[pfmoore]

When creating an environment, pip is invoked to install the build requirements. But there is no way of passing custom arguments for pip, to control how packages are downloaded (for example --extra-index-url, --find-links, --proxy, --no-index, --cert, --trusted-host, ...) Technically it would be possible to set the equivalent environment variables (PIP_*) but that runs the risk of overriding any environment variables the user might have set (which may be a good thing or a bad thing, it's not immediately clear).

My original need for this was to set --find-links when writing tests, to load a dummy backend from the test directory, but the general problem will need to be addressed if the library is to be used by pip, to copy over the options passed to pip.

This probably needs some discussion - it's not obvious what the best way of handling this would be, and pip has rather a lot of options that are potentially relevant here. It's possible that the best solution would require changes to pip, as well.

[takluyver]

To be clear, the package is split into lower-level infrastructure (mainly pep517.wrappers) and a more speculative higher-level layer (pep517.envbuild). The latter is kind of an example implementation, and I was fully expecting that pip would either need to modify it considerably or bypass it entirely and build on the lower level.

Playing semi devil's advocate*: According to pip's docs, command line options override environment variables. So passing through a command line argument would have the same effect as replacing an environment variable the user has set. That being the case, maybe environment variables are the right way to control this?

(* i.e. I recognise this is an odd approach, and there may well be other drawbacks, but right now I don't have a good counterargument, so maybe it's worth exploring)

[pfmoore]

Thanks. I wondered about that, the wrappers module seemed more like what I was expecting, whereas the higher-level layer seemed to be replicating some of the stuff we already have in pip. So I'm pretty much OK with just concentrating on the lower-level API in the first instance, and using pip's existing isolation infrastructure (I'm sure we'll have the same issues there, so once they get thrashed out in pip we may be able to transfer the lessons over to here).

The whole suite of pip options around index access are pretty complex and messy, and people use them in quite weird and wonderful ways. Short term, I suspect environment variables is the sensible approach, longer term the whole area probably needs a tidy-up.

[takluyver]

👍

[pradyunsg]

Alrighty. I'm here, and if there's anything I've been doing for the past 3 years, it's been tidy-up (sort of). :P

Looking at pip + pep517, it looks like that it'd be a good idea to bring in pip's code for PEP 517 isolation and friends, into pep517. Synchronizing how pip / pep517 handle build isolation would make pip simpler (it can then switch to pep517 for isolation as well), and allow other tools to build upon the "knowledge" that's gone into pip's build isolation.

[takluyver]

That sounds good.

My one reservation is: can we bring the build environment machinery across without pep517 needing to know too much about pip's options (and thus potentially needing to change as pip changes)? I don't see a way around needing to use pip to find and install build dependencies, but if it's a dependency of pip, it seems like bad design if it needs to be tied to lots of details of pip.

[xavfernandez]

In the same way, that Pep517HookCaller accepts a runner hook, the BuildEnvironment could also accept an installer hook that would let the frontend handle this part ?

[takluyver]

We've come to an agreement that the functionality to create isolated build environments and install build dependencies will live in the PyPA build project. The pep517 library will continue to exist as the lower-level piece to work with the interfaces defined by PEP 517, but the pep517.envbuild module and the little command-line interfaces which use it (pep517.build, pep517.check and pep517.meta) will gradually be deprecated and eventually removed as build becomes more stable. See #91 for more on that process.

So I'm closing this, as there won't be any further significant work here on build environments & dependency installation.