Merge pull request #2588 from ahoppen/ahoppen/deprecate-bytesourcerange

Deprecate `ByteSourceRange` in favor of `Range<AbsolutePosition>`

Author: ahoppen

Release Notes/600.md

@@ -75,22 +75,26 @@
   - Description: With the change to parse `#if canImport(MyModule, _version: 1.2.3)` as a function call instead of a dedicated syntax node, `1.2.3` natively gets parsed as a member access `3` to the `1.2` float literal. This property allows the reinterpretation of such an expression as a version tuple.
   - Pull request: https://github.com/apple/swift-syntax/pull/2025
 
-- `SyntaxProtocol.node(at:)` 
+- `SyntaxProtocol.node(at:)`
   - Description: Given a `SyntaxIdentifier`, returns the `Syntax` node with that identifier
   - Pull request: https://github.com/apple/swift-syntax/pull/2594
 
 - `SyntaxIdentifier.IndexInTree`
   - Description: Uniquely identifies a syntax node within a tree. This is similar to ``SyntaxIdentifier`` but does not store the root ID of the tree. It can thus be transferred across trees that are structurally equivalent, for example two copies of the same tree that live in different processes. The only public functions on this type are `toOpaque` and `init(fromOpaque:)`, which allow serialization of the `IndexInTree`.
   - Pull request: https://github.com/apple/swift-syntax/pull/2594
-  
+
 - `SyntaxIdentifier` conformance to `Comparable`:
   - Description: A `SyntaxIdentifier` compares less than another `SyntaxIdentifier` if the node at that identifier occurs first during a depth-first traversal of the tree.
   - Pull request: https://github.com/apple/swift-syntax/pull/2594
 
-- `SyntaxIdentifier.indexInTree` and `SyntaxIdentifier.fromIndexInTree` 
+- `SyntaxIdentifier.indexInTree` and `SyntaxIdentifier.fromIndexInTree`
   - Description: `SyntaxIdentifier.indexInTree` allows the retrieval of a `SyntaxIdentifier` that identifies the syntax node independent of the syntax tree. `SyntaxIdentifier.fromIndexInTree` allows the creation for a `SyntaxIdentifier` from a tree-agnostic `SyntaxIdentifier.IndexInTree` and the tree's root node.
   - Pull request: https://github.com/apple/swift-syntax/pull/2594
 
+- `Range<AbsolutePosition>`
+  - Description: `Range<AbsolutePosition>` gained a few convenience functions inspired from `ByteSourceRange`: `init(position:length:)`,  `length`, `overlapsOrTouches`
+  - Pull request: https://github.com/apple/swift-syntax/pull/2587
+
 ## API Behavior Changes
 
 ## Deprecations
@@ -124,6 +128,18 @@
   - Description: Instead of parsing `canImport` inside `#if` directives as a special expression node, parse it as a functionc call expression. This is in-line with how the `swift(>=6.0)` and `compiler(>=6.0)` directives are parsed.
   - Pull request: https://github.com/apple/swift-syntax/pull/2025
 
+- `SyntaxClassifiedRange.offset`, `length` and `endOffset`
+  - Description: Deprecated these properties in favor of `range`.
+  - Pull request: https://github.com/apple/swift-syntax/pull/2587
+
+- `SyntaxProtocol.totalByteRange` and `trimmedByteRange`
+  - Description: Renamed to `range` and `trimmedRange`, both now returning a `Range<AbsolutePosition>`.
+  - Pull request: https://github.com/apple/swift-syntax/pull/2587
+
+- `ByteSourceRange` deprecated in favor of `Range<AbsolutePosition>`
+  - Description:  `ByteSourceRange` is being dropped for `Range<AbsolutePosition>`, where the latter clearly signifies that it uses UTF-8 byte positions. `Range<AbsolutePosition>` has deprecated compatibility layers to make it API-compatible with `ByteSourceRange`
+  - Pull request: https://github.com/apple/swift-syntax/pull/2587
+
 ## API-Incompatible Changes
 
 - `MacroDefinition` used for expanding macros:
@@ -177,6 +193,21 @@
   - Pull request: https://github.com/apple/swift-syntax/pull/2489
   - Migration steps: Stop using this module.
 
+- `ByteSourceRange.length` changed from `Int` to `SourceLength`
+  - Description: `ByteSourceRange` has been deprecated and declared as a typealias for `Range<AbsolutePosition>`. At the same time, `Range<AbsolutePosition>` gained `length: SourceLength` that provides type-system information about the kind of length (UTF-8 byte length).
+  - Pull request: https://github.com/apple/swift-syntax/pull/2587
+
+- `IncrementalEdit.replacementLength` changed from `Int` to `SourceLength`
+  - Description: The type of `IncrementalEdit.replacementLength` has been changed from `Int` to `SourceLength` which provides type-system information about the kind of length (UTF-8 byte length).
+  - Pull request: https://github.com/apple/swift-syntax/pull/2587
+
+## API-Behavior Changes
+
+- `SyntaxProtocol.classifications(in:)` and `SyntaxProtocol.classification(at:)` take positions relative to the root of the syntax tree instead of relative to the start of the node
+  - Description: With the deprecation of `ByteSourceRange` in favor of `Range<AbsolutePosition>`, the `AbsolutePosition`s passed as the range are measured from the start of the syntax tree instead of the start of the current node.
+  - Pull request: https://github.com/apple/swift-syntax/pull/2587
+  - Migration steps: Pass absolute positions measured from the root of the syntax tree instead of positions relative to the current node.
+
 ## Template
 
 - *Affected API or two word description*

Sources/SwiftIDEUtils/Syntax+Classifications.swift

@@ -25,12 +25,11 @@ public extension SyntaxProtocol {
   /// consecutive tokens would have the same classification then a single classified
   /// range is provided for all of them.
   var classifications: SyntaxClassifications {
-    let fullRange = ByteSourceRange(offset: 0, length: totalLength.utf8Length)
-    return SyntaxClassifications(_syntaxNode, in: fullRange)
+    return SyntaxClassifications(_syntaxNode, in: self.range)
   }
 
   /// Sequence of ``SyntaxClassifiedRange``s contained in this syntax node within
-  /// a relative range.
+  /// a source range.
   ///
   /// The provided classified ranges may extend beyond the provided `range`.
   /// Active classifications (non-`none`) will extend the range to include the
@@ -41,9 +40,9 @@ public extension SyntaxProtocol {
   /// intersect the provided `range`.
   ///
   /// - Parameters:
-  ///   - in: The relative byte range to pull ``SyntaxClassifiedRange``s from.
+  ///   - in: The range to pull ``SyntaxClassifiedRange``s from.
   /// - Returns: Sequence of ``SyntaxClassifiedRange``s.
-  func classifications(in range: ByteSourceRange) -> SyntaxClassifications {
+  func classifications(in range: Range<AbsolutePosition>) -> SyntaxClassifications {
     return SyntaxClassifications(_syntaxNode, in: range)
   }
 
@@ -52,19 +51,20 @@ public extension SyntaxProtocol {
   ///   - at: The relative to the node byte offset.
   /// - Returns: The ``SyntaxClassifiedRange`` for the offset or nil if the source text
   ///   at the given offset is unclassified.
+  @available(*, deprecated, message: "Use classification(at: AbsolutePosition) instead.")
   func classification(at offset: Int) -> SyntaxClassifiedRange? {
-    let classifications = SyntaxClassifications(_syntaxNode, in: ByteSourceRange(offset: offset, length: 1))
-    var iterator = classifications.makeIterator()
-    return iterator.next()
+    return classification(at: AbsolutePosition(utf8Offset: offset + self.position.utf8Offset))
   }
 
   /// The ``SyntaxClassifiedRange`` for an absolute position.
   /// - Parameters:
   ///   - at: The absolute position.
-  /// - Returns: The ``SyntaxClassifiedRange`` for the position or nil if the source text
+  /// - Returns: The ``SyntaxClassifiedRange`` for the position or `nil`` if the source text
   ///   at the given position is unclassified.
   func classification(at position: AbsolutePosition) -> SyntaxClassifiedRange? {
-    let relativeOffset = position.utf8Offset - self.position.utf8Offset
-    return self.classification(at: relativeOffset)
+    let range = Range(position: position, length: SourceLength(utf8Length: 1))
+    let classifications = SyntaxClassifications(_syntaxNode, in: range)
+    var iterator = classifications.makeIterator()
+    return iterator.next()
   }
 }

Sources/SwiftIDEUtils/SyntaxClassifier.swift

@@ -44,7 +44,7 @@ extension TokenSyntax {
 
 extension RawTriviaPiece {
   func classify(offset: Int) -> SyntaxClassifiedRange {
-    let range = ByteSourceRange(offset: offset, length: byteLength)
+    let range = AbsolutePosition(utf8Offset: offset)..<AbsolutePosition(utf8Offset: offset + byteLength)
     switch self {
     case .lineComment: return .init(kind: .lineComment, range: range)
     case .blockComment: return .init(kind: .blockComment, range: range)
@@ -63,7 +63,7 @@ fileprivate struct TokenKindAndText {
     offset: Int,
     contextualClassification: (SyntaxClassification, Bool)?
   ) -> SyntaxClassifiedRange {
-    let range = ByteSourceRange(offset: offset, length: text.count)
+    let range = AbsolutePosition(utf8Offset: offset)..<AbsolutePosition(utf8Offset: offset + text.count)
 
     if let contextualClassify = contextualClassification {
       let (classify, force) = contextualClassify
@@ -91,10 +91,15 @@ fileprivate struct TokenKindAndText {
 /// Represents a source range that is associated with a syntax classification.
 public struct SyntaxClassifiedRange: Equatable, Sendable {
   public var kind: SyntaxClassification
-  public var range: ByteSourceRange
+  public var range: Range<AbsolutePosition>
 
+  @available(*, deprecated, message: "Use range.lowerBound.utf8Offset instead")
   public var offset: Int { return range.offset }
-  public var length: Int { return range.length }
+
+  @available(*, deprecated, message: "Use range.utf8Length instead")
+  public var length: Int { return range.length.utf8Length }
+
+  @available(*, deprecated, message: "Use range.upperBound.utf8Offset instead")
   public var endOffset: Int { return range.endOffset }
 }
 
@@ -110,19 +115,14 @@ private struct ClassificationVisitor {
     var contextualClassification: (SyntaxClassification, Bool)?
   }
 
-  /// Only tokens within this absolute range will be classified. No
-  /// classifications will be reported for tokens out of this range.
-  private var targetRange: ByteSourceRange
+  /// Only tokens within this range will be classified.
+  /// No classifications will be reported for tokens out of this range.
+  private var targetRange: Range<AbsolutePosition>
 
   var classifications: [SyntaxClassifiedRange]
 
-  /// Only classify tokens in `relativeClassificationRange`, where the start
-  /// offset is relative to `node`.
-  init(node: Syntax, relativeClassificationRange: ByteSourceRange) {
-    let range = ByteSourceRange(
-      offset: node.position.utf8Offset + relativeClassificationRange.offset,
-      length: relativeClassificationRange.length
-    )
+  /// Only classify tokens in `range`.
+  init(node: Syntax, range: Range<AbsolutePosition>) {
     self.targetRange = range
     self.classifications = []
 
@@ -140,24 +140,22 @@ private struct ClassificationVisitor {
   }
 
   private mutating func report(range: SyntaxClassifiedRange) {
-    if range.kind == .none && range.length == 0 {
+    if range.kind == .none && range.range.isEmpty {
       return
     }
 
     // Merge consecutive classified ranges of the same kind.
     if let last = classifications.last,
       last.kind == range.kind,
-      last.endOffset == range.offset
+      last.range.upperBound == range.range.lowerBound
     {
-      classifications[classifications.count - 1].range = ByteSourceRange(
-        offset: last.offset,
-        length: last.length + range.length
-      )
+      classifications[classifications.count - 1].range =
+        last.range.lowerBound..<(last.range.upperBound + range.range.length)
       return
     }
 
-    guard range.offset <= targetRange.endOffset,
-      range.endOffset >= targetRange.offset
+    guard range.range.lowerBound <= targetRange.upperBound,
+      range.range.upperBound >= targetRange.lowerBound
     else {
       return
     }
@@ -219,10 +217,9 @@ private struct ClassificationVisitor {
         let layoutNodeTextLength = child.byteLength - child.leadingTriviaByteLength - child.trailingTriviaByteLength
         let range = SyntaxClassifiedRange(
           kind: classification.classification,
-          range: ByteSourceRange(
-            offset: byteOffset,
-            length: layoutNodeTextLength
-          )
+          range: AbsolutePosition(
+            utf8Offset: byteOffset
+          )..<AbsolutePosition(utf8Offset: byteOffset + layoutNodeTextLength)
         )
         report(range: range)
         byteOffset += layoutNodeTextLength
@@ -250,10 +247,10 @@ private struct ClassificationVisitor {
   }
 
   private mutating func visit(_ descriptor: ClassificationVisitor.Descriptor) -> VisitResult {
-    guard descriptor.byteOffset < targetRange.endOffset else {
+    guard descriptor.byteOffset < targetRange.upperBound.utf8Offset else {
       return .break
     }
-    guard descriptor.byteOffset + descriptor.node.byteLength > targetRange.offset else {
+    guard descriptor.byteOffset + descriptor.node.byteLength > targetRange.lowerBound.utf8Offset else {
       return .continue
     }
     guard SyntaxTreeViewMode.sourceAccurate.shouldTraverse(node: descriptor.node) else {
@@ -273,8 +270,8 @@ public struct SyntaxClassifications: Sequence, Sendable {
 
   var classifications: [SyntaxClassifiedRange]
 
-  public init(_ node: Syntax, in relRange: ByteSourceRange) {
-    let visitor = ClassificationVisitor(node: node, relativeClassificationRange: relRange)
+  public init(_ node: Syntax, in range: Range<AbsolutePosition>) {
+    let visitor = ClassificationVisitor(node: node, range: range)
     self.classifications = visitor.classifications
   }
 

Sources/SwiftParser/IncrementalParseTransition.swift

@@ -23,7 +23,7 @@ extension Parser {
     }
 
     let currentOffset = self.lexemes.offsetToStart(self.currentToken)
-    if let node = parseLookup!.lookUp(currentOffset, kind: kind) {
+    if let node = parseLookup!.lookUp(AbsolutePosition(utf8Offset: currentOffset), kind: kind) {
       self.lexemes.advance(by: node.totalLength.utf8Length, currentToken: &self.currentToken)
       return node
     }
@@ -117,16 +117,15 @@ struct IncrementalParseLookup {
   /// has invalidated the previous ``Syntax`` node.
   ///
   /// - Parameters:
-  ///   - offset: The byte offset of the source string that is currently parsed.
+  ///   - position: The position in the source string that is currently parsed.
   ///   - kind: The `CSyntaxKind` that the parser expects at this position.
   /// - Returns: A ``Syntax`` node from the previous parse invocation,
   ///            representing the contents of this region, if it is still valid
   ///            to re-use. `nil` otherwise.
-  fileprivate mutating func lookUp(_ newOffset: Int, kind: SyntaxKind) -> Syntax? {
-    guard let prevOffset = translateToPreEditOffset(newOffset) else {
+  fileprivate mutating func lookUp(_ newPosition: AbsolutePosition, kind: SyntaxKind) -> Syntax? {
+    guard let prevPosition = translateToPreEditPosition(newPosition) else {
       return nil
     }
-    let prevPosition = AbsolutePosition(utf8Offset: prevOffset)
     let node = cursorLookup(prevPosition: prevPosition, kind: kind)
     if let node {
       reusedCallback?(node)
@@ -162,7 +161,7 @@ struct IncrementalParseLookup {
 
     // Fast path check: if parser is past all the edits then any matching node
     // can be re-used.
-    if !edits.edits.isEmpty && edits.edits.last!.range.endOffset < node.position.utf8Offset {
+    if !edits.edits.isEmpty && edits.edits.last!.range.upperBound < node.position {
       return true
     }
 
@@ -172,15 +171,12 @@ struct IncrementalParseLookup {
       return false
     }
 
-    let nodeAffectRange = ByteSourceRange(
-      offset: node.position.utf8Offset,
-      length: nodeAffectRangeLength
-    )
+    let nodeAffectRange = node.position..<node.position.advanced(by: nodeAffectRangeLength)
 
     for edit in edits.edits {
       // Check if this node or the trivia of the next node has been edited. If
       // it has, we cannot reuse it.
-      if edit.range.offset > nodeAffectRange.endOffset {
+      if edit.range.lowerBound > nodeAffectRange.upperBound {
         // Remaining edits don't affect the node. (Edits are sorted)
         break
       }
@@ -192,19 +188,19 @@ struct IncrementalParseLookup {
     return true
   }
 
-  fileprivate func translateToPreEditOffset(_ postEditOffset: Int) -> Int? {
+  fileprivate func translateToPreEditPosition(_ postEditOffset: AbsolutePosition) -> AbsolutePosition? {
     var offset = postEditOffset
     for edit in edits.edits {
-      if edit.range.offset > offset {
+      if edit.range.lowerBound > offset {
         // Remaining edits doesn't affect the position. (Edits are sorted)
         break
       }
-      if edit.range.offset + edit.replacementLength > offset {
+      if edit.range.lowerBound + edit.replacementLength > offset {
         // This is a position inserted by the edit, and thus doesn't exist in
         // the pre-edit version of the file.
         return nil
       }
-      offset = offset - edit.replacementLength + edit.range.length
+      offset = offset + edit.range.length - edit.replacementLength
     }
     return offset
   }
@@ -352,24 +348,32 @@ public struct ConcurrentEdits: Sendable {
       var editToAdd = editToAdd
       var editIndicesMergedWithNewEdit: [Int] = []
       for (index, existingEdit) in concurrentEdits.enumerated() {
-        if existingEdit.replacementRange.intersectsOrTouches(editToAdd.range) {
+        if existingEdit.replacementRange.overlapsOrTouches(editToAdd.range) {
           let intersectionLength =
-            existingEdit.replacementRange.intersected(editToAdd.range).length
+            existingEdit.replacementRange.clamped(to: editToAdd.range).length
           let replacement: [UInt8]
           replacement =
-            existingEdit.replacement.prefix(max(0, editToAdd.offset - existingEdit.replacementRange.offset))
+            existingEdit.replacement.prefix(
+              max(0, editToAdd.range.lowerBound.utf8Offset - existingEdit.replacementRange.lowerBound.utf8Offset)
+            )
             + editToAdd.replacement
-            + existingEdit.replacement.suffix(max(0, existingEdit.replacementRange.endOffset - editToAdd.endOffset))
+            + existingEdit.replacement.suffix(
+              max(0, existingEdit.replacementRange.upperBound.utf8Offset - editToAdd.range.upperBound.utf8Offset)
+            )
           editToAdd = IncrementalEdit(
-            offset: Swift.min(existingEdit.offset, editToAdd.offset),
-            length: existingEdit.length + editToAdd.length - intersectionLength,
+            range: Range(
+              position: Swift.min(existingEdit.range.lowerBound, editToAdd.range.lowerBound),
+              length: existingEdit.range.length + editToAdd.range.length - intersectionLength
+            ),
             replacement: replacement
           )
           editIndicesMergedWithNewEdit.append(index)
-        } else if existingEdit.offset < editToAdd.endOffset {
+        } else if existingEdit.range.lowerBound < editToAdd.range.upperBound {
           editToAdd = IncrementalEdit(
-            offset: editToAdd.offset - existingEdit.replacementLength + existingEdit.length,
-            length: editToAdd.length,
+            range: Range(
+              position: editToAdd.range.lowerBound + existingEdit.range.length - existingEdit.replacementLength,
+              length: editToAdd.range.length
+            ),
             replacement: editToAdd.replacement
           )
         }
@@ -380,7 +384,7 @@ public struct ConcurrentEdits: Sendable {
       }
       let insertPos =
         concurrentEdits.firstIndex(where: { edit in
-          editToAdd.endOffset <= edit.offset
+          editToAdd.range.upperBound <= edit.range.lowerBound
         }) ?? concurrentEdits.count
       concurrentEdits.insert(editToAdd, at: insertPos)
       precondition(ConcurrentEdits.isValidConcurrentEditArray(concurrentEdits))
@@ -397,7 +401,7 @@ public struct ConcurrentEdits: Sendable {
     for i in 1..<edits.count {
       let prevEdit = edits[i - 1]
       let curEdit = edits[i]
-      if curEdit.range.offset < prevEdit.range.endOffset {
+      if curEdit.range.lowerBound < prevEdit.range.upperBound {
         return false
       }
       if curEdit.intersectsRange(prevEdit.range) {