Skip to content

Implement document formatting using swift-format #220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

Trzyipolkostkicukru
Copy link
Contributor

Implements textDocument/formatting request using swift-format library.

It adds a dependency on swift-format which is finicky about which toolchain it uses - it needs to be swift-DEVELOPMENT-SNAPSHOT-2020-01-29-a or the library will complain at runtime.

By the way, is there a better way to get the last Position in a file than loading the contents of the url into a LineTable?

@benlangmuir
Copy link
Contributor

Very cool! I'll try to review the source changes soon, but first a note about the new dependency:

  • Since SwiftSyntax depends on the unstable parser library from swift itself, and we are going into the same toolchain as that swift, we should be using master branch of swiftsyntax and swift-format. Otherwise we have no way to guarantee we will not hit the same runtime compatibility issues you mentioned.
  • Using a branch dependency will also mean we need to be really careful that changes to those projects do not break us, so we will need to update swift ci to test sourcekit-lsp in those repos' PR tests.

None of this is a problem; we just need to work through it before we land this PR.

CC @allevato @akyrtzi

@allevato
Copy link
Member

allevato commented Feb 3, 2020

Yeah, there's still a little bit of work we need to do on the swift-format side to get things CI ready, since right now the master branch is pinned to a specific snapshot (swift-DEVELOPMENT-SNAPSHOT-2020-01-29-a) instead of apple/swift master.

I've been carving out some time to push a bit more on that recently (e.g., swiftlang/swift-format#136 removes all @testable imports, which was mentioned as a blocker for CI due to the way things are built there), so next I need to start working on the build script helpers to get swift-format integrated with the rest of the toolchain.

}
let options = req.params.options
self.queue.async {
var config = SwiftFormatConfiguration.Configuration()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When invoked from the LSP, should the formatting action respect the .swift-format.json config file that applies to the file in the request? (Right now, we look in the same directory as the source file and then up through parent directories.)

I'm not sure what the expected behavior is for LSP formatting if you have an external config file and the options passed in the request conflict with those.

But, the SwiftFormatter API doesn't do anything to load that configuration; it's the swift-format command line tool that does it. So if the LSP needs to also look up the configuration, rather than duplicate that file lookup here, we should expose a helper API in SwiftFormatConfiguration that loads the configuration given the path to a particular source file so that other clients can reuse it easily (and so that if we change that logic later, all clients get matching behavior automatically).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect that we would respect the .swift-format.json configuration in general, but we should have control of this in case we need to mediate some kind of editor-level settings. I think exposing a helper on SwiftFormatConfiguration would be great.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should use the config file if it exists, and fall back on values provided by the lsp if it doesn't exist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it so that it tries to load config from file, and if that fails uses values provided by the lsp.
It relies on changes from the swiftlang/swift-format#140 pull request

@IOOI-SqAR
Copy link

@Trzyipolkostkicukru , @benlangmuir , @allevato any progress here?

Bump

@Trzyipolkostkicukru
Copy link
Contributor Author

I don't know what is needed for swift-format CI integration and haven't tried to find out, so I'm patiently waiting for someone else to do that. No progress from me

@benlangmuir
Copy link
Contributor

No updates from me either, we're blocked by being able to integrate the swift-format dependency.

@allevato
Copy link
Member

Sorry for the delays on this, but we've finally managed to carve out time to make some progress here. swiftlang/swift-format#207 is the first PR, and with some follow-up work in the main Swift repository to hook it all together, that should unblock swift-format being used as a dependency of other projects.

@Trzyipolkostkicukru
Copy link
Contributor Author

Trzyipolkostkicukru commented Jul 17, 2020

@benlangmuir I rebased my changes to the master so that they could be merged. Unfortunately doing that broke the tests - it seems openDocument function is asynchronous, and I have for state to stabilize before trying to run the test. I had to use the sleep function for that, but is there some solution for having openDocument be synchronous that I am missing?

@benlangmuir
Copy link
Contributor

@Trzyipolkostkicukru sorry I missed this comment.

it seems openDocument function is asynchronous, and I have for state to stabilize before trying to run the test

To clarify, is the issue that openDocument is delayed until after build settings are available? @DavidGoldman is there a good way to handle this in a test?

@DavidGoldman
Copy link
Contributor

@Trzyipolkostkicukru sorry I missed this comment.

it seems openDocument function is asynchronous, and I have for state to stabilize before trying to run the test

To clarify, is the issue that openDocument is delayed until after build settings are available? @DavidGoldman is there a good way to handle this in a test?

Since registerToolchainTextDocumentRequest is used the formatting operation should be queued and completed once the document has opened + build settings received. What sort of issues were you seeing here that required a sleep?

@Trzyipolkostkicukru
Copy link
Contributor Author

Trzyipolkostkicukru commented Jul 22, 2020

sorry I missed this comment

I am used to waiting weeks and months :D Five days is quick!

and I have for state to stabilize

Oh god, I seem to have mangled that sentence. I meant to write "and I have to wait for state to stabilize".

What sort of issues were you seeing here that required a sleep?

The direct issue is that guard let snapshot = self.documentManager.latestSnapshot(req.params.textDocument.uri) at line 753 in SwiftLanguageServer.swift fails randomly.

If I remove all tests except one (I cannot filter out the other tests, so I just delete the files for other tests, and comment out the other tests in FormattingTests.swift), then it works more often than not. If I have more than one formatting tests, then it's very rare for more than one to be successful.

If I add var documents: [DocumentURI: Document] = [:] { didSet { print("didSet documents") } } at line 63 in DocumentManager.swift, then the print fires multiple times, even after the test already failed, which is why I thought openDocument is asynchronous.


Potential solution: If I move fetching snapshots into self.queue.async the tests pass, but I don't think it's thread safe. SwiftLanguageServer.queue and DocumentManager.queue don't seem to be related in any way, so tests passing must be a coincidence we cannot rely on. Thread sanitizer is complaining about something if I do that.

@DavidGoldman
Copy link
Contributor

sorry I missed this comment

I am used to waiting weeks and months :D Five days is quick!

and I have for state to stabilize

Oh god, I seem to have mangled that sentence. I meant to write "and I have to wait for state to stabilize".

What sort of issues were you seeing here that required a sleep?

The direct issue is that guard let snapshot = self.documentManager.latestSnapshot(req.params.textDocument.uri) at line 753 in SwiftLanguageServer.swift fails randomly.

If I remove all tests except one (I cannot filter out the other tests, so I just delete the files for other tests, and comment out the other tests in FormattingTests.swift), then it works more often than not. If I have more than one formatting tests, then it's very rare for more than one to be successful.

If I add var documents: [DocumentURI: Document] = [:] { didSet { print("didSet documents") } } at line 63 in DocumentManager.swift, then the print fires multiple times, even after the test already failed, which is why I thought openDocument is asynchronous.

Potential solution: If I move fetching snapshots into self.queue.async the tests pass, but I don't think it's thread safe. SwiftLanguageServer.queue and DocumentManager.queue don't seem to be related in any way, so tests passing must be a coincidence we cannot rely on. Thread sanitizer is complaining about something if I do that.

Hmm, we should call self.documentManager.latestSnapshot on the queue, otherwise you will run into the synchronization issues that you're talking about (since you may try to fetch a snapshot before the open call makes it through the queue). What is thread sanitizer complaining about? We've spoken about removing the SwiftLanguageServer's DocumentManager in favor of the one in SourceKitServer but haven't gotten around to it yet (would be a largish refactor).

@Trzyipolkostkicukru
Copy link
Contributor Author

we should call self.documentManager.latestSnapshot on the queue, otherwise you will run into the synchronization issues that you're talking about (since you may try to fetch a snapshot before the open call makes it through the queue).

Good to know! I couldn't find any place where openDocument synchronized on "swift-language-server-queue" so I thought that they are independent.

What is thread sanitizer complaining about?

full output
-> % /Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swift test --sanitize=thread
warning: Source files for target SKSwiftPMWorkspaceTests should be located under /Users/cukier/Tools/cukr/sourcekit-lsp/Tests/SKSwiftPMWorkspaceTests
warning: Source files for target SKCoreTests should be located under /Users/cukier/Tools/cukr/sourcekit-lsp/Tests/SKCoreTests
warning: Source files for target SourceKitDTests should be located under /Users/cukier/Tools/cukr/sourcekit-lsp/Tests/SourceKitDTests
warning: Source files for target LSPLoggingTests should be located under /Users/cukier/Tools/cukr/sourcekit-lsp/Tests/LSPLoggingTests
warning: Source files for target LanguageServerProtocolJSONRPCTests should be located under /Users/cukier/Tools/cukr/sourcekit-lsp/Tests/LanguageServerProtocolJSONRPCTests
warning: Source files for target LanguageServerProtocolTests should be located under /Users/cukier/Tools/cukr/sourcekit-lsp/Tests/LanguageServerProtocolTests
warning: Source files for target SKSupportTests should be located under /Users/cukier/Tools/cukr/sourcekit-lsp/Tests/SKSupportTests
[5/5] Linking SourceKitLSPPackageTests
Test Suite 'All tests' started at 2020-07-22 16:30:23.752
Test Suite 'SourceKitLSPPackageTests.xctest' started at 2020-07-22 16:30:23.755
Test Suite 'FormattingTests' started at 2020-07-22 16:30:23.755
Test Case '-[SourceKitLSPTests.FormattingTests testConfigFile]' started.
Test Case '-[SourceKitLSPTests.FormattingTests testConfigFile]' passed (0.479 seconds).
Test Case '-[SourceKitLSPTests.FormattingTests testConfigFileInNestedDirectory]' started.
==================
WARNING: ThreadSanitizer: data race (pid=87916)
  Write of size 8 at 0x7b100000e0a8 by thread T8:
    #0 LocalConnection.close() Connection.swift:93 (SourceKitLSPPackageTests:x86_64+0x5308fb)
    #1 SwiftLanguageServer.shutdown() SwiftLanguageServer.swift:228 (SourceKitLSPPackageTests:x86_64+0xb8b5ef)
    #2 protocol witness for ToolchainLanguageServer.shutdown() in conformance SwiftLanguageServer <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb8b4af)
    #3 SourceKitServer.shutdown(_:) SourceKitServer.swift:526 (SourceKitLSPPackageTests:x86_64+0xb2fce4)
    #4 implicit closure #8 in implicit closure #7 in SourceKitServer._registerBuiltinHandlers() SourceKitServer.swift:72 (SourceKitLSPPackageTests:x86_64+0xb2fa0d)
    #5 partial apply for implicit closure #8 in implicit closure #7 in SourceKitServer._registerBuiltinHandlers() <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb56ae8)
    #6 closure #1 in LanguageServerEndpoint._register<A, B>(_:) LanguageServer.swift:114 (SourceKitLSPPackageTests:x86_64+0xa1575c)
    #7 partial apply for closure #1 in LanguageServerEndpoint._register<A, B>(_:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1a182)
    #8 thunk for @escaping @callee_guaranteed (@guaranteed Request<B>) -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa157eb)
    #9 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed Request<B>) -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1a262)
    #10 thunk for @escaping @callee_guaranteed (@in_guaranteed Request<A>) -> (@out ()) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa19870)
    #11 partial apply for thunk for @escaping @callee_guaranteed (@in_guaranteed Request<A>) -> (@out ()) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1b580)
    #12 closure #1 in LanguageServerEndpoint.handle<A>(_:id:from:reply:) LanguageServer.swift:196 (SourceKitLSPPackageTests:x86_64+0xa18c4e)
    #13 partial apply for closure #1 in LanguageServerEndpoint.handle<A>(_:id:from:reply:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1ace7)
    #14 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0x17713)
    #15 __tsan::invoke_and_release_block(void*) <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x718fb)
    #16 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x2657)

  Previous read of size 8 at 0x7b100000e0a8 by thread T3:
    #0 LocalConnection.send<A>(_:) Connection.swift:107 (SourceKitLSPPackageTests:x86_64+0x5311d4)
    #1 SwiftLanguageServer.publishDiagnostics(response:for:compileCommand:) SwiftLanguageServer.swift:163 (SourceKitLSPPackageTests:x86_64+0xb886f3)
    #2 closure #1 in SwiftLanguageServer.openDocument(_:) SwiftLanguageServer.swift:320 (SourceKitLSPPackageTests:x86_64+0xb93c0e)
    #3 partial apply for closure #1 in SwiftLanguageServer.openDocument(_:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb93f9e)
    #4 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0x17713)
    #5 __tsan::invoke_and_release_block(void*) <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x718fb)
    #6 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x2657)

  Location is heap block of size 56 at 0x7b100000e080 allocated by thread T3:
    #0 malloc <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x4f42a)
    #1 swift_slowAlloc <null>:2 (libswiftCore.dylib:x86_64+0x2f2ca8)
    #2 makeLocalSwiftServer(client:sourcekitd:clientCapabilities:) SwiftLanguageServer.swift:1339 (SourceKitLSPPackageTests:x86_64+0xbbf1aa)
    #3 languageService(for:_:options:client:in:) SourceKitServer.swift:1009 (SourceKitLSPPackageTests:x86_64+0xb4298d)
    #4 closure #1 in SourceKitServer.languageService(for:_:in:) SourceKitServer.swift:263 (SourceKitLSPPackageTests:x86_64+0xb40f1c)
    #5 partial apply for closure #1 in SourceKitServer.languageService(for:_:in:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb51f6a)
    #6 thunk for @callee_guaranteed () -> (@owned ToolchainLanguageServer?, @error @owned Error) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb42e46)
    #7 partial apply for thunk for @callee_guaranteed () -> (@owned ToolchainLanguageServer?, @error @owned Error) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb51ff7)
    #8 orLog<A>(_:level:logger:_:) Logging.swift:58 (SourceKitLSPPackageTests:x86_64+0x50dea0)
    #9 SourceKitServer.languageService(for:_:in:) SourceKitServer.swift:262 (SourceKitLSPPackageTests:x86_64+0xb40a6a)
    #10 SourceKitServer.languageService(for:_:in:) SourceKitServer.swift:298 (SourceKitLSPPackageTests:x86_64+0xb4334c)
    #11 SourceKitServer.openDocument(_:workspace:) SourceKitServer.swift:556 (SourceKitLSPPackageTests:x86_64+0xb30ded)
    #12 implicit closure #12 in implicit closure #11 in SourceKitServer._registerBuiltinHandlers() SourceKitServer.swift:75 (SourceKitLSPPackageTests:x86_64+0xb308d0)
    #13 partial apply for implicit closure #12 in implicit closure #11 in SourceKitServer._registerBuiltinHandlers() <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb56a40)
    #14 closure #1 in SourceKitServer.registerWorkspaceNotfication<A>(_:) SourceKitServer.swift:197 (SourceKitLSPPackageTests:x86_64+0xb3e971)
    #15 partial apply for closure #1 in SourceKitServer.registerWorkspaceNotfication<A>(_:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb49d34)
    #16 thunk for @escaping @callee_guaranteed (@guaranteed Notification<A>) -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa164ab)
    #17 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed Notification<A>) -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1a5a4)
    #18 thunk for @escaping @callee_guaranteed (@in_guaranteed Notification<A>) -> (@out ()) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa17860)
    #19 partial apply for thunk for @escaping @callee_guaranteed (@in_guaranteed Notification<A>) -> (@out ()) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1b7a0)
    #20 closure #1 in LanguageServerEndpoint.handle<A>(_:from:) LanguageServer.swift:168 (SourceKitLSPPackageTests:x86_64+0xa177b7)
    #21 partial apply for closure #1 in LanguageServerEndpoint.handle<A>(_:from:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1aaa9)
    #22 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0x17713)
    #23 __tsan::invoke_and_release_block(void*) <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x718fb)
    #24 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x2657)

  Thread T8 (tid=2277677, running) is a GCD worker thread

  Thread T3 (tid=2277601, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: data race Connection.swift:93 in LocalConnection.close()
==================
==================
WARNING: ThreadSanitizer: data race (pid=87916)
  Write of size 8 at 0x7b100000e0b0 by thread T8:
    #0 LocalConnection.close() Connection.swift:93 (SourceKitLSPPackageTests:x86_64+0x530913)
    #1 SwiftLanguageServer.shutdown() SwiftLanguageServer.swift:228 (SourceKitLSPPackageTests:x86_64+0xb8b5ef)
    #2 protocol witness for ToolchainLanguageServer.shutdown() in conformance SwiftLanguageServer <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb8b4af)
    #3 SourceKitServer.shutdown(_:) SourceKitServer.swift:526 (SourceKitLSPPackageTests:x86_64+0xb2fce4)
    #4 implicit closure #8 in implicit closure #7 in SourceKitServer._registerBuiltinHandlers() SourceKitServer.swift:72 (SourceKitLSPPackageTests:x86_64+0xb2fa0d)
    #5 partial apply for implicit closure #8 in implicit closure #7 in SourceKitServer._registerBuiltinHandlers() <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb56ae8)
    #6 closure #1 in LanguageServerEndpoint._register<A, B>(_:) LanguageServer.swift:114 (SourceKitLSPPackageTests:x86_64+0xa1575c)
    #7 partial apply for closure #1 in LanguageServerEndpoint._register<A, B>(_:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1a182)
    #8 thunk for @escaping @callee_guaranteed (@guaranteed Request<B>) -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa157eb)
    #9 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed Request<B>) -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1a262)
    #10 thunk for @escaping @callee_guaranteed (@in_guaranteed Request<A>) -> (@out ()) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa19870)
    #11 partial apply for thunk for @escaping @callee_guaranteed (@in_guaranteed Request<A>) -> (@out ()) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1b580)
    #12 closure #1 in LanguageServerEndpoint.handle<A>(_:id:from:reply:) LanguageServer.swift:196 (SourceKitLSPPackageTests:x86_64+0xa18c4e)
    #13 partial apply for closure #1 in LanguageServerEndpoint.handle<A>(_:id:from:reply:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1ace7)
    #14 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0x17713)
    #15 __tsan::invoke_and_release_block(void*) <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x718fb)
    #16 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x2657)

  Previous read of size 8 at 0x7b100000e0b0 by thread T3:
    #0 LocalConnection.send<A>(_:) Connection.swift:107 (SourceKitLSPPackageTests:x86_64+0x5311f3)
    #1 SwiftLanguageServer.publishDiagnostics(response:for:compileCommand:) SwiftLanguageServer.swift:163 (SourceKitLSPPackageTests:x86_64+0xb886f3)
    #2 closure #1 in SwiftLanguageServer.openDocument(_:) SwiftLanguageServer.swift:320 (SourceKitLSPPackageTests:x86_64+0xb93c0e)
    #3 partial apply for closure #1 in SwiftLanguageServer.openDocument(_:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb93f9e)
    #4 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0x17713)
    #5 __tsan::invoke_and_release_block(void*) <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x718fb)
    #6 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x2657)

  Location is heap block of size 56 at 0x7b100000e080 allocated by thread T3:
    #0 malloc <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x4f42a)
    #1 swift_slowAlloc <null>:2 (libswiftCore.dylib:x86_64+0x2f2ca8)
    #2 makeLocalSwiftServer(client:sourcekitd:clientCapabilities:) SwiftLanguageServer.swift:1339 (SourceKitLSPPackageTests:x86_64+0xbbf1aa)
    #3 languageService(for:_:options:client:in:) SourceKitServer.swift:1009 (SourceKitLSPPackageTests:x86_64+0xb4298d)
    #4 closure #1 in SourceKitServer.languageService(for:_:in:) SourceKitServer.swift:263 (SourceKitLSPPackageTests:x86_64+0xb40f1c)
    #5 partial apply for closure #1 in SourceKitServer.languageService(for:_:in:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb51f6a)
    #6 thunk for @callee_guaranteed () -> (@owned ToolchainLanguageServer?, @error @owned Error) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb42e46)
    #7 partial apply for thunk for @callee_guaranteed () -> (@owned ToolchainLanguageServer?, @error @owned Error) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb51ff7)
    #8 orLog<A>(_:level:logger:_:) Logging.swift:58 (SourceKitLSPPackageTests:x86_64+0x50dea0)
    #9 SourceKitServer.languageService(for:_:in:) SourceKitServer.swift:262 (SourceKitLSPPackageTests:x86_64+0xb40a6a)
    #10 SourceKitServer.languageService(for:_:in:) SourceKitServer.swift:298 (SourceKitLSPPackageTests:x86_64+0xb4334c)
    #11 SourceKitServer.openDocument(_:workspace:) SourceKitServer.swift:556 (SourceKitLSPPackageTests:x86_64+0xb30ded)
    #12 implicit closure #12 in implicit closure #11 in SourceKitServer._registerBuiltinHandlers() SourceKitServer.swift:75 (SourceKitLSPPackageTests:x86_64+0xb308d0)
    #13 partial apply for implicit closure #12 in implicit closure #11 in SourceKitServer._registerBuiltinHandlers() <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb56a40)
    #14 closure #1 in SourceKitServer.registerWorkspaceNotfication<A>(_:) SourceKitServer.swift:197 (SourceKitLSPPackageTests:x86_64+0xb3e971)
    #15 partial apply for closure #1 in SourceKitServer.registerWorkspaceNotfication<A>(_:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xb49d34)
    #16 thunk for @escaping @callee_guaranteed (@guaranteed Notification<A>) -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa164ab)
    #17 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed Notification<A>) -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1a5a4)
    #18 thunk for @escaping @callee_guaranteed (@in_guaranteed Notification<A>) -> (@out ()) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa17860)
    #19 partial apply for thunk for @escaping @callee_guaranteed (@in_guaranteed Notification<A>) -> (@out ()) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1b7a0)
    #20 closure #1 in LanguageServerEndpoint.handle<A>(_:from:) LanguageServer.swift:168 (SourceKitLSPPackageTests:x86_64+0xa177b7)
    #21 partial apply for closure #1 in LanguageServerEndpoint.handle<A>(_:from:) <compiler-generated> (SourceKitLSPPackageTests:x86_64+0xa1aaa9)
    #22 thunk for @escaping @callee_guaranteed () -> () <compiler-generated> (SourceKitLSPPackageTests:x86_64+0x17713)
    #23 __tsan::invoke_and_release_block(void*) <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x718fb)
    #24 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x2657)

  Thread T8 (tid=2277677, running) is a GCD worker thread

  Thread T3 (tid=2277601, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: data race Connection.swift:93 in LocalConnection.close()
==================
Test Case '-[SourceKitLSPTests.FormattingTests testConfigFileInNestedDirectory]' passed (3.932 seconds).
Test Case '-[SourceKitLSPTests.FormattingTests testConfigFileInParentDirectory]' started.
Test Case '-[SourceKitLSPTests.FormattingTests testConfigFileInParentDirectory]' passed (0.210 seconds).
Test Case '-[SourceKitLSPTests.FormattingTests testSpaces]' started.
Test Case '-[SourceKitLSPTests.FormattingTests testSpaces]' passed (0.206 seconds).
Test Case '-[SourceKitLSPTests.FormattingTests testTabs]' started.
Test Case '-[SourceKitLSPTests.FormattingTests testTabs]' passed (0.169 seconds).
Test Suite 'FormattingTests' passed at 2020-07-22 16:30:28.751.
         Executed 5 tests, with 0 failures (0 unexpected) in 4.995 (4.996) seconds
Test Suite 'SourceKitLSPPackageTests.xctest' passed at 2020-07-22 16:30:28.751.
         Executed 5 tests, with 0 failures (0 unexpected) in 4.995 (4.996) seconds
Test Suite 'All tests' passed at 2020-07-22 16:30:28.751.
         Executed 5 tests, with 0 failures (0 unexpected) in 4.995 (4.999) seconds
ThreadSanitizer: reported 2 warnings
Exited with signal code 6

@DavidGoldman
Copy link
Contributor

we should call self.documentManager.latestSnapshot on the queue, otherwise you will run into the synchronization issues that you're talking about (since you may try to fetch a snapshot before the open call makes it through the queue).

Good to know! I couldn't find any place where openDocument synchronized on "swift-language-server-queue" so I thought that they are independent.

What is thread sanitizer complaining about?

full output

Hmm it looks like the shutdown method might need to do

// or maybe self.queue.sync
self.queue.async {
  client.close()
}

I think @benlangmuir has a better understanding of the shutdown flow

@benlangmuir
Copy link
Contributor

Yeah, that seems like it ought to work. Or LocalConnection itself should be made thread-safe.

@IOOI-SqAR
Copy link

Ping.

Any news on this?

@shahmishal shahmishal closed this Oct 6, 2020
@shahmishal
Copy link
Member

The Swift project moved the default branch to main and deleted master branch, so GitHub automatically closed the PR. Please re-create the pull request with main branch.

More detail about the branch update - https://forums.swift.org/t/updating-branch-names/40412

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants