clean build dir after errors, and don't reuse a pre-existing build dir

[qwcode]

this fixes issues #413, #709, and #634

the changes:

1. remove the build dir if there's an exception during file prep or installation (which usually means an exception during 'setup.py egg_info' or 'setup.py install')

2. fail with an exception if there's a pre-existing build dir for the pkg. it's too risky to do otherwise. just checking the version in the build dir might be considered acceptable, but we changed that check to just warn (and not fail) about 9 months ago for specific reasons. see pull Only warn for mis-versioned packages. #436. this change does not interfere with the --no-download option which allows you to install against an existing build dir.

and there's 3 new tests for this.

[qwcode]

hmm, the new --root test is failing...

[dholth]

It is highly useful to retain the build dir after an error. You can go look at it and find out what happened. pip can also resume from the same build dir at least if you give it the right options. I consider the reported bugs to be features.

[pnasrat]

@dholth I think the resume/reuse is a more advanced feature - the other bug reports show the existing behaviour is impacting folks - eg quote about costing two hours in one of them.

I'm wondering if we should support a 'pip build' and a flag to install to use existing to support your workflow. @qwcode @jezdez any thoughts?

[pnasrat]

Actually no_install is sufficient for the first part of that equation so just a flag to allow it to reuse perhaps

[qwcode]

pip can also resume from the same build dir at least if you give it the right options.

that feature was left intact with this change. you can still do that with --no-download

It is highly useful to retain the build dir after an error. You can go look at it and find out what happened

I can see that. I could take out #1, and just leave the #2 change?

[qwcode]

I consider the reported bugs to be features.

unintentionally reusing an old build dir with the wrong version, is what people are concerned with.
there is a warning message in the log that goes "Requested vX, but installing version vY", but that's not enough for this scenario in my opinion

[qwcode]

It is highly useful to retain the build dir after an error. You can go look at it and find out what happened
I can see that. I could take out #1, and just leave the #2 change?

or a comprimise could be to allow build dirs to remain (error or not) with the advanced --no-install, but for normal use, where 99% of the use cases involve no inspection of the orphaned build dir, remove it

[qwcode]

'pip build' and a flag to install to use existing to support your workflow

you mean something like this?

1. replace "pip install --no-install" with "pip build .." (or maybe "pip unpack")
2. replace "pip install --no-download" with "pip install --use-build-dir

[dholth]

I probably could have looked at the pip log, but just today I was installing something, noticed it was grabbing the wrong dependencies, and it turned out pip had found the wrong version of something. I looked in the build dir and immediately understood the problem. Other times I go back in there to just "setup.py install" one thing.

I can see how it would be a problem if you don't expect it. It probably is better for most people to zap it.

Why does pip even use a predictable build directory instead of a randomly named one?

[qwcode]

Why does pip even use a predictable build directory instead of a randomly named one?

partly I guess, so there's a predictable location for the --no-install/--no-download workflow?

maybe for now, lets just do #2, i.e. don't ever use pre-existing build dirs (except when using --no-download)

that alone would have prevented the pain behind these bug reports.

are there any concerns with that?

[qwcode]

closing for now. will reopen another pull that just covers #2 (from the description)