[WIP] Optimization: Improve performance for repetitive lookup

CacheLookupTable leaves the bucket vacant when the object is freed.
Because of this, in the worst case, subscript have to perform full linear
search for all buckets.

Let's say we have a table like this:

  bucket 0: obj1(hash: 0)
  bucket 1: obj2(hash: 1)
  bucket 2: nil
  bucket 3: obj3(hash: 0)
  bucket 4: nil

If obj1 is freed, it become:

  bucket 0: nil
  bucket 1: obj2(hash: 1)
  bucket 2: nil
  bucket 3: obj3(hash: 0)
  bucket 4: nil

When looking-up obj3, the search starts from "bucket 0", thus it needs 3
iteration.
This patch moves the obj3 to "bucket 0" the first nil hole found.
So the next lookup doesn't need any iteration.

  bucket 0: obj3(hash: 0)
  bucket 1: obj2(hash: 1)
  bucket 2: nil
  bucket 3: nil
  bucket 4: nil

Author: rintaro

Sources/SwiftSyntax/CacheLookupTable.swift

@@ -201,9 +201,25 @@ class CacheLookupTable<T: Identifiable & AnyObject> {
     // we expect it's stored near the 'idealBucket' anyway.
     let idealBucket = _idealBucket(for: id)
     var bucket = idealBucket
+    var firstHole: Int? = nil
     repeat {
-      if buckets[bucket].value?.id == id {
-        return buckets[bucket].value
+      if let obj = buckets[bucket].value {
+        if obj.id == id {
+          if let firstHole = firstHole {
+            // Move the object to the first hole found. This improves lookup
+            // performance in the next lookup.
+            let holeP = buckets + firstHole
+            let bucketP = buckets + bucket
+
+            let tmp = holeP.move()
+            holeP.moveInitialize(from: bucketP, count: 1)
+            bucketP.initialize(to: tmp)
+          }
+          return obj
+        }
+      } else if firstHole == nil {
+        // Remember the first hole found.
+        firstHole = bucket
       }
       bucket = (bucket &+ 1) & _bucketMask
     } while bucket != idealBucket