Isolate tests and run concurrently #1159
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Awesome job |
* Uses the same generation method as the built in tmpdir, simply returns a different object type.
* We've selected a concurrency number of 16 because that's how many CPUs the multiprocessing library tells us a Travis box has.
This is needed because the various activities that exericsing the tests do will cause files to change inside this data directory. If we do not make it test specific then we run into concurrency issues.
Previously pip was always installed directly from the source tree however this causes concurrency issues so it is now copied into the temporary directory and installed from there.
There seems to be some issues with parallel tests and Python 3.2 in pytest or pytest-xdist, so we run the tests normally for now on 3.2.
dstufft
added a commit
that referenced
this pull request
Aug 27, 2013
Isolate tests and run concurrently
| @@ -20,4 +24,4 @@ branches: | |||
| - 1.3.X | |||
| - 1.4.X | |||
| env: | |||
| - PIP_USE_MIRRORS=true | |||
| - PYTHONHASHSEED=0 | |||
There was a problem hiding this comment.
On Python 3.3+ hash randomization is on by default, this confuses pytest-xdist. Setting the hash seed to zero effectively disables hash randomization.
See https://bitbucket.org/hpk42/pytest/issue/346/pytest-xdist-and-python-33-is-sort-of
|
merge 👍 ex post facto |
lock
bot
added
the
auto-locked
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
tmpdirfixture with our own that returns atests.lib.path.Pathobject.virtualenvfixture which creates a virtual environment in the per test temporary directory.scriptfixture which creates an instance ofPipTestEnvironmentusing the per test virtual environment.datafixture which copies all of the test data into the per test temporary directory so tests can interact with it.tests_cachedirectory the helperslocal_repoandlocal_checkoutnow cache to a per test specific temporary directory.The bulk of the work of this PR is in isolating the tests from each other. To do that it uses the py.test
tmpdirto give each test a unique temporary directory and then moves all of the operations that modify the filesystem in any way to take place inside of that temporary directory. It adds some other fixtures and modifies some helpers along the way to make doing that a bit nicer.The easiest part of the PR was enabling the actual concurrent builds. We use 8 concurrent builds because that seems to give us the most bang for the buck without making PyPy dreadfully slow (I'm unsure of why but using 16 processes made PyPy take twice as long as 8 processes). However I was unable to use concurrent builds on Python 3.2 because of an apparent bug in py.test or pytest-xdist. I will follow up with upstream to attempt to sort out why 3.2 was erroring out and then submit another PR to enable concurrent builds for it. This means PRs still take awhile to pass but at least there is some quick feedback.