Allow configuring of SourceKit-LSP’s options using .sourcekit-lsp configuration files

The idea here is to unify the different ways in which we can currently set options on SourceKit-LSP in a scalable way: Environment variables, command line arguments to `sourcekit-lsp` and initialization options.

The idea is that a user can define a `~/.sourcekit-lsp/.sourcekit-lsp` file (we store logs in `~/.sourcekit-lsp/logs` on non-Darwin platforms), which will be used as the default configuration for all SourceKit-LSP instances. They can also place a `.sourcekit-lsp` file in the root of a workspace to configure SourceKit-LSP for that project specifically, eg. setting arguments that need to be passed to `swift build` for that project and which thus also need to be set on SourceKit-LSP.

For compatibility reasons, I’m mapping the existing command line options into the new options structure for now. I hope to delete the command line arguments in the future and solely rely on `.sourcekit-lsp` configuration files.

Environment variable will be migrated to `.sourcekit-lsp` in a follow-up commit.

Author: ahoppen

Documentation/Configuration File.md

@@ -0,0 +1,38 @@
+# Configuration File
+
+`.sourcekit-lsp/config.json` configuration files can be used to modify the behavior of SourceKit-LSP in various ways. The following locations are checked. Settings in later configuration files override settings in earlier configuration files
+- `~/.sourcekit-lsp/config.json`
+- On macOS: `~/Library/Application Support/org.swift.sourcekit-lsp/config.json` from the various `Library` folders on the system
+- If the `XDG_CONFIG_HOME` environment variable is set: `$XDG_CONFIG_HOME/org.swift.sourcekit-lsp/config.json`
+- A `.sourcekit-lsp/config.json` file in a workspace’s root
+
+The structure of the file is currently not guaranteed to be stable. Options may be removed or renamed.
+
+## Structure
+
+`config.json` is a JSON file with the following structure. All keys are optional and unknown keys are ignored.
+
+- `swiftPM`: Dictionary with the following keys, defining options for SwiftPM workspaces
+  - `configuration: "debug"|"release"`: The configuration to build the project for during background indexing and the configuration whose build folder should be used for Swift modules if background indexing is disabled
+  - `scratchPath: string`: Build artifacts directory path. If nil, the build system may choose a default value
+  - `cCompilerFlags: string[]`: Extra arguments passed to the compiler for C files
+  - `cxxCompilerFlags: string[]`: Extra arguments passed to the compiler for C++ files
+  - `swiftCompilerFlags: string[]`: Extra arguments passed to the compiler for Swift files
+  - `linkerFlags: string[]`: Extra arguments passed to the linker
+- `compilationDatabase`: Dictionary with the following keys, defining options for workspaces with a compilation database
+  - `searchPaths: string[]`: Additional paths to search for a compilation database, relative to a workspace root.
+- `fallbackBuildSystem`: Dictionary with the following keys, defining options for files that aren't managed by any build system
+  - `cCompilerFlags: string[]`: Extra arguments passed to the compiler for C files
+  - `cxxCompilerFlags: string[]`: Extra arguments passed to the compiler for C++ files
+  - `swiftCompilerFlags: string[]`: Extra arguments passed to the compiler for Swift files
+- `clangdOptions: string[]`: Extra command line arguments passed to `clangd` when launching it
+- `index`: Dictionary with the following keys, defining options related to indexing
+    - `indexStorePath: string`: Directory in which a separate compilation stores the index store. By default, inferred from the build system.
+    - `indexDatabasePath: string`: Directory in which the indexstore-db should be stored. By default, inferred from the build system.
+    - `indexPrefixMap: [string: string]`: Path remappings for remapping index data for local use.
+    - `maxCoresPercentageToUseForBackgroundIndexing: double`: A hint indicating how many cores background indexing should use at most (value between 0 and 1). Background indexing is not required to honor this setting
+    - `updateIndexStoreTimeout: int`: Number of seconds to wait for an update index store task to finish before killing it.
+- `defaultWorkspaceType: "buildserver"|"compdb"|"swiftpm"`: Overrides workspace type selection logic.
+- `generatedFilesPath: string`: Directory in which generated interfaces and macro expansions should be stored.
+- `experimentalFeatures: string[]`: Experimental features to enable
+- `swiftPublishDiagnosticsDebounce`: The time that `SwiftLanguageService` should wait after an edit before starting to compute diagnostics and sending a `PublishDiagnosticsNotification`.

Documentation/LSP Extensions.md

@@ -73,28 +73,6 @@ Added case
 identifier = 'identifier'
 ```
 
-## `textDocument/completion`
-
-Added field:
-
-```ts
-/**
- * Options to control code completion behavior in Swift
- */
-sourcekitlspOptions?: SKCompletionOptions
-```
-
-with
-
-```ts
-interface SKCompletionOptions {
-  /**
-   * The maximum number of completion results to return, or `null` for unlimited.
-   */
-  maxResults?: int
-}
-```
-
 ## `textDocument/interface`
 
 New request that request a textual interface of a module to display in the IDE. Used internally within SourceKit-LSP

Sources/Diagnose/IndexCommand.swift

@@ -82,9 +82,7 @@ public struct IndexCommand: AsyncParsableCommand {
   public init() {}
 
   public func run() async throws {
-    var serverOptions = SourceKitLSPServer.Options()
-    serverOptions.experimentalFeatures = Set(experimentalFeatures)
-    serverOptions.experimentalFeatures.insert(.backgroundIndexing)
+    let options = SourceKitLSPOptions(experimentalFeatures: Set(experimentalFeatures).union([.backgroundIndexing]))
 
     let installPath =
       if let toolchainOverride, let toolchain = Toolchain(try AbsolutePath(validating: toolchainOverride)) {
@@ -96,7 +94,7 @@ public struct IndexCommand: AsyncParsableCommand {
     let messageHandler = IndexLogMessageHandler()
     let inProcessClient = try await InProcessSourceKitLSPClient(
       toolchainRegistry: ToolchainRegistry(installPath: installPath),
-      serverOptions: serverOptions,
+      options: options,
       workspaceFolders: [WorkspaceFolder(uri: DocumentURI(URL(fileURLWithPath: project)))],
       messageHandler: messageHandler
     )

Sources/InProcessClient/InProcessSourceKitLSPClient.swift

@@ -26,7 +26,8 @@ public final class InProcessSourceKitLSPClient: Sendable {
   /// `messageHandler` handles notifications and requests sent from the SourceKit-LSP server to the client.
   public init(
     toolchainRegistry: ToolchainRegistry,
-    serverOptions: SourceKitLSPServer.Options = SourceKitLSPServer.Options(),
+    options: SourceKitLSPOptions = SourceKitLSPOptions(),
+    testHooks: TestHooks = TestHooks(),
     capabilities: ClientCapabilities = ClientCapabilities(),
     workspaceFolders: [WorkspaceFolder],
     messageHandler: any MessageHandler
@@ -35,7 +36,8 @@ public final class InProcessSourceKitLSPClient: Sendable {
     self.server = SourceKitLSPServer(
       client: serverToClientConnection,
       toolchainRegistry: toolchainRegistry,
-      options: serverOptions,
+      options: options,
+      testHooks: testHooks,
       onExit: {
         serverToClientConnection.close()
       }

Sources/LanguageServerProtocol/CMakeLists.txt

@@ -124,7 +124,6 @@ add_library(LanguageServerProtocol STATIC
   SupportTypes/SemanticTokens.swift
   SupportTypes/SemanticTokenTypes.swift
   SupportTypes/ServerCapabilities.swift
-  SupportTypes/SKCompletionOptions.swift
   SupportTypes/StringOrMarkupContent.swift
   SupportTypes/SymbolKind.swift
   SupportTypes/TestItem.swift

Sources/LanguageServerProtocol/Requests/CompletionRequest.swift

@@ -23,7 +23,6 @@
 ///   - textDocument: The document to perform completion in.
 ///   - position: The location to perform completion at.
 ///   - context: Optional code-completion context.
-///   - sourcekitlspOptions: **(LSP Extension)** code-completion options for sourcekit-lsp.
 ///
 /// - Returns: A list of completion items to complete the code at the given position.
 public struct CompletionRequest: TextDocumentRequest, Hashable {
@@ -36,18 +35,14 @@ public struct CompletionRequest: TextDocumentRequest, Hashable {
 
   public var context: CompletionContext?
 
-  public var sourcekitlspOptions: SKCompletionOptions?
-
   public init(
     textDocument: TextDocumentIdentifier,
     position: Position,
-    context: CompletionContext? = nil,
-    sourcekitlspOptions: SKCompletionOptions? = nil
+    context: CompletionContext? = nil
   ) {
     self.textDocument = textDocument
     self.position = position
     self.context = context
-    self.sourcekitlspOptions = sourcekitlspOptions
   }
 }
 

Sources/LanguageServerProtocol/Requests/PollIndexRequest.swift

@@ -12,9 +12,6 @@
 
 /// Poll the index for unit changes and wait for them to be registered.
 /// **LSP Extension, For Testing**.
-///
-/// Users of PollIndex should set `"initializationOptions": { "listenToUnitEvents": false }` during
-/// the `initialize` request.
 public struct PollIndexRequest: RequestType {
   public static let method: String = "workspace/_pollIndex"
   public typealias Response = VoidResponse

Sources/LanguageServerProtocol/SupportTypes/SKCompletionOptions.swift

@@ -10,7 +10,7 @@
 //
 //===----------------------------------------------------------------------===//
 
-public enum BuildConfiguration: String, Sendable {
+public enum BuildConfiguration: String, Codable, Sendable {
   case debug
   case release
 }

Sources/SKCore/BuildConfiguration.swift

@@ -45,7 +45,6 @@ func executable(_ name: String) -> String {
 /// `buildServer.json` configuration file provided in the repo root.
 public actor BuildServerBuildSystem: MessageHandler {
   public let projectRoot: AbsolutePath
-  let buildFolder: AbsolutePath?
   let serverConfig: BuildServerConfig
 
   var buildServer: JSONRPCConnection?
@@ -82,7 +81,6 @@ public actor BuildServerBuildSystem: MessageHandler {
 
   public init(
     projectRoot: AbsolutePath,
-    buildFolder: AbsolutePath?,
     fileSystem: FileSystem = localFileSystem
   ) async throws {
     let configPath = projectRoot.appending(component: "buildServer.json")
@@ -100,7 +98,6 @@ public actor BuildServerBuildSystem: MessageHandler {
         currentWorkingDirectory: fileSystem.currentWorkingDirectory
       )
     #endif
-    self.buildFolder = buildFolder
     self.projectRoot = projectRoot
     self.serverConfig = config
     try await self.initializeBuildServer()
@@ -109,11 +106,11 @@ public actor BuildServerBuildSystem: MessageHandler {
   /// Creates a build system using the Build Server Protocol config.
   ///
   /// - Returns: nil if `projectRoot` has no config or there is an error parsing it.
-  public init?(projectRoot: AbsolutePath?, buildSetup: BuildSetup) async {
-    if projectRoot == nil { return nil }
+  public init?(projectRoot: AbsolutePath?) async {
+    guard let projectRoot else { return nil }
 
     do {
-      try await self.init(projectRoot: projectRoot!, buildFolder: buildSetup.path)
+      try await self.init(projectRoot: projectRoot)
     } catch is FileSystemError {
       // config file was missing, no build server for this workspace
       return nil

Sources/SKCore/BuildServerBuildSystem.swift

@@ -2,7 +2,6 @@
 add_library(SKCore STATIC
   BuildConfiguration.swift
   BuildServerBuildSystem.swift
-  BuildSetup.swift
   BuildSystem.swift
   BuildSystemDelegate.swift
   BuildSystemManager.swift
@@ -13,6 +12,7 @@ add_library(SKCore STATIC
   IndexTaskID.swift
   MainFilesProvider.swift
   PathPrefixMapping.swift
+  SourceKitLSPOptions.swift
   SplitShellCommand.swift
   Toolchain.swift
   ToolchainRegistry.swift

Sources/SKCore/BuildSetup.swift

@@ -22,11 +22,10 @@ import class TSCBasic.Process
 
 /// A simple BuildSystem suitable as a fallback when accurate settings are unknown.
 public actor FallbackBuildSystem {
+  private let options: SourceKitLSPOptions.FallbackBuildSystemOptions
 
-  let buildSetup: BuildSetup
-
-  public init(buildSetup: BuildSetup) {
-    self.buildSetup = buildSetup
+  public init(options: SourceKitLSPOptions.FallbackBuildSystemOptions) {
+    self.options = options
   }
 
   /// The path to the SDK.
@@ -71,7 +70,7 @@ public actor FallbackBuildSystem {
 
   func settingsSwift(_ file: String) -> FileBuildSettings {
     var args: [String] = []
-    args.append(contentsOf: self.buildSetup.flags.swiftCompilerFlags)
+    args.append(contentsOf: self.options.swiftCompilerFlags ?? [])
     if let sdkpath = sdkpath, !args.contains("-sdk") {
       args += [
         "-sdk",
@@ -86,9 +85,9 @@ public actor FallbackBuildSystem {
     var args: [String] = []
     switch language {
     case .c:
-      args.append(contentsOf: self.buildSetup.flags.cCompilerFlags)
+      args.append(contentsOf: self.options.cCompilerFlags ?? [])
     case .cpp:
-      args.append(contentsOf: self.buildSetup.flags.cxxCompilerFlags)
+      args.append(contentsOf: self.options.cxxCompilerFlags ?? [])
     default:
       break
     }