Enhance syntactic test discovery with information from the semantic index #1195
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ahoppen
changed the title
ahoppen
force-pushed
the
ahoppen/filter-syntactic-tests-based-on-outdated-index
branch
from
d1cf231 to
08d9537
Compare
bnbarham
approved these changes
May 1, 2024
ahoppen
force-pushed
the
ahoppen/filter-syntactic-tests-based-on-outdated-index
branch
from
08d9537 to
2ff2fb0
Compare
ahoppen
changed the title
|
@swift-ci Please test |
…ndex
When the semantic index is out-of-date, we currently purely rely on the syntactic index to discover tests and completely ignore data from the semantic index. This may lead to confusing behavior. For example if you have
```
class MightInheritFromXCTestCaseOrNot {}
class MyClass: MightInheritFromXCTestCaseOrNot {
func testStuff() {}
}
```
Then we don’t return any tests when the semantic index is up-to-date. But once the file is modified (either on disk or in-memory), we purely rely on the syntactic index, which reports `testStuff` as a test method. After a build / background indexing finishes, the test method disappears again.
We can mitigate this problem as follows: If we have stale semantic index data for the test file, for every test method found by the syntactic index, check if we have an entry for this method in the semantic index. If we do, but that entry is not marked as a test class/method, we know that the semantic index knows about this method but decided that it’s not a test method for some reason. So we should ignore it.
rdar://126492948
ahoppen
force-pushed
the
ahoppen/filter-syntactic-tests-based-on-outdated-index
branch
from
2ff2fb0 to
0685ee7
Compare
|
@swift-ci Please test |
|
@swift-ci Please test Windows |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When the semantic index is out-of-date, we currently purely rely on the syntactic index to discover tests and completely ignore data from the semantic index. This may lead to confusing behavior. For example if you have
Then we don’t return any tests when the semantic index is up-to-date. But once the file is modified (either on disk or in-memory), we purely rely on the syntactic index, which reports
testStuffas a test method. After a build / background indexing finishes, the test method disappears again.We can mitigate this problem as follows: If we have stale semantic index data for the test file, for every test method found by the syntactic index, check if we have an entry for this method in the semantic index. If we do, but that entry is not marked as a test class/method, we know that the semantic index knows about this method but decided that it’s not a test method for some reason. So we should ignore it.
rdar://126492948