[Segment Cache] Add "client-only" option (#77655)

We're currently working through some issues surfaced by the new
per-segment routes introduced by the Segment Cache experiment. Testing
of this feature in production has been blocked while we figure that out.

However, aside from per-segment prefetching, there are a bunch of
client-only changes and improvements to the Segment Cache implementation
that we can test in the meantime.

So, I've added a "client-only" option the `clientSegmentCache` flag.
When enabled, Next.js will not generate per-segment prefetch responses,
but the client will switch to the new prefetching implementation. This
includes:

- De-duping of shared layouts across prefetch requests.
- More resilience to race conditions.
- Prefetch cancellation on Link viewport exit.
- Re-prefetching stale data after revalidateTag/revalidatePath.

etc.

It's essentially a complete rewrite of the client-side prefetching
implementation. Given the scope of the change, it may contain subtle
regressions that I haven't yet discovered. That's why it's so urgent for
me to start testing this in real apps.

I initially hesitated to implement the "client-only" option because 1)
there are already so many combinations of flags and forked
implementations to juggle, and I didn't want to add another one, and 2)
I thought it would take less time to fix the deployment issues that are
blocking the wider rollout.

But it turned out to be less net new complexity than I anticipated,
given that I already had to support `ppr: "incremental"`, which works in
a largely similar way.

Author: acdlite

packages/next/src/client/components/segment-cache-impl/cache.ts

@@ -1028,6 +1028,7 @@ export async function fetchRouteOnCacheMiss(
 
       writeDynamicTreeResponseIntoCache(
         Date.now(),
+        task,
         response,
         serverData,
         entry,
@@ -1246,6 +1247,10 @@ export async function fetchSegmentPrefetchesUsingDynamicRequest(
       prefetchStream
     ) as Promise<NavigationFlightResponse>)
 
+    // Since we did not set the prefetch header, the response from the server
+    // will never contain dynamic holes.
+    const isResponsePartial = false
+
     // Aside from writing the data into the cache, this function also returns
     // the entries that were fulfilled, so we can streamingly update their sizes
     // in the LRU as more data comes in.
@@ -1254,6 +1259,7 @@ export async function fetchSegmentPrefetchesUsingDynamicRequest(
       task,
       response,
       serverData,
+      isResponsePartial,
       route,
       spawnedEntries
     )
@@ -1269,6 +1275,7 @@ export async function fetchSegmentPrefetchesUsingDynamicRequest(
 
 function writeDynamicTreeResponseIntoCache(
   now: number,
+  task: PrefetchTask,
   response: Response,
   serverData: NavigationFlightResponse,
   entry: PendingRouteCacheEntry,
@@ -1311,16 +1318,43 @@ function writeDynamicTreeResponseIntoCache(
     staleTimeHeaderSeconds !== null
       ? parseInt(staleTimeHeaderSeconds, 10) * 1000
       : STATIC_STALETIME_MS
-  fulfillRouteCacheEntry(
+
+  // If the response contains dynamic holes, then we must conservatively assume
+  // that any individual segment might contain dynamic holes, and also the
+  // head. If it did not contain dynamic holes, then we can assume every segment
+  // and the head is completely static.
+  const isResponsePartial =
+    response.headers.get(NEXT_DID_POSTPONE_HEADER) === '1'
+
+  const fulfilledEntry = fulfillRouteCacheEntry(
     entry,
     convertRootFlightRouterStateToRouteTree(flightRouterState),
     flightData.head,
-    flightData.isHeadPartial,
+    isResponsePartial,
     now + staleTimeMs,
     couldBeIntercepted,
     canonicalUrl,
     routeIsPPREnabled
   )
+
+  // If the server sent segment data as part of the response, we should write
+  // it into the cache to prevent a second, redundant prefetch request.
+  //
+  // TODO: When `clientSegmentCache` is enabled, the server does not include
+  // segment data when responding to a route tree prefetch request. However,
+  // when `clientSegmentCache` is set to "client-only", and PPR is enabled (or
+  // the page is fully static), the normal check is bypassed and the server
+  // responds with the full page. This is a temporary situation until we can
+  // remove the "client-only" option. Then, we can delete this function call.
+  writeDynamicRenderResponseIntoCache(
+    now,
+    task,
+    response,
+    serverData,
+    isResponsePartial,
+    fulfilledEntry,
+    null
+  )
 }
 
 function rejectSegmentEntriesIfStillPending(
@@ -1343,16 +1377,19 @@ function writeDynamicRenderResponseIntoCache(
   task: PrefetchTask,
   response: Response,
   serverData: NavigationFlightResponse,
+  isResponsePartial: boolean,
   route: FulfilledRouteCacheEntry,
-  spawnedEntries: Map<string, PendingSegmentCacheEntry>
+  spawnedEntries: Map<string, PendingSegmentCacheEntry> | null
 ): Array<FulfilledSegmentCacheEntry> | null {
   if (serverData.b !== getAppBuildId()) {
     // The server build does not match the client. Treat as a 404. During
     // an actual navigation, the router will trigger an MPA navigation.
     // TODO: Consider moving the build ID to a response header so we can check
     // it before decoding the response, and so there's one way of checking
     // across all response types.
-    rejectSegmentEntriesIfStillPending(spawnedEntries, now + 10 * 1000)
+    if (spawnedEntries !== null) {
+      rejectSegmentEntriesIfStillPending(spawnedEntries, now + 10 * 1000)
+    }
     return null
   }
   const flightDatas = normalizeFlightData(serverData.f)
@@ -1395,6 +1432,7 @@ function writeDynamicRenderResponseIntoCache(
         route,
         now + staleTimeMs,
         seedData,
+        isResponsePartial,
         segmentKey,
         spawnedEntries
       )
@@ -1408,11 +1446,14 @@ function writeDynamicRenderResponseIntoCache(
   // segments we're marking as rejected here. We should mark on the segment
   // somehow that the reason for the rejection is because of a non-PPR prefetch.
   // That way a per-segment prefetch knows to disregard the rejection.
-  const fulfilledEntries = rejectSegmentEntriesIfStillPending(
-    spawnedEntries,
-    now + 10 * 1000
-  )
-  return fulfilledEntries
+  if (spawnedEntries !== null) {
+    const fulfilledEntries = rejectSegmentEntriesIfStillPending(
+      spawnedEntries,
+      now + 10 * 1000
+    )
+    return fulfilledEntries
+  }
+  return null
 }
 
 function writeSeedDataIntoCache(
@@ -1421,8 +1462,9 @@ function writeSeedDataIntoCache(
   route: FulfilledRouteCacheEntry,
   staleAt: number,
   seedData: CacheNodeSeedData,
+  isResponsePartial: boolean,
   key: string,
-  entriesOwnedByCurrentTask: Map<string, PendingSegmentCacheEntry>
+  entriesOwnedByCurrentTask: Map<string, PendingSegmentCacheEntry> | null
 ) {
   // This function is used to write the result of a dynamic server request
   // (CacheNodeSeedData) into the prefetch cache. It's used in cases where we
@@ -1431,12 +1473,15 @@ function writeSeedDataIntoCache(
   // dynamic data into being static) and when prefetching a PPR-disabled route
   const rsc = seedData[1]
   const loading = seedData[3]
-  const isPartial = rsc === null
+  const isPartial = rsc === null || isResponsePartial
 
   // We should only write into cache entries that are owned by us. Or create
   // a new one and write into that. We must never write over an entry that was
   // created by a different task, because that causes data races.
-  const ownedEntry = entriesOwnedByCurrentTask.get(key)
+  const ownedEntry =
+    entriesOwnedByCurrentTask !== null
+      ? entriesOwnedByCurrentTask.get(key)
+      : undefined
   if (ownedEntry !== undefined) {
     fulfillSegmentCacheEntry(ownedEntry, rsc, loading, staleAt, isPartial)
   } else {
@@ -1481,6 +1526,7 @@ function writeSeedDataIntoCache(
           route,
           staleAt,
           childSeedData,
+          isResponsePartial,
           encodeChildSegmentKey(
             key,
             parallelRouteKey,

packages/next/src/export/index.ts

@@ -391,7 +391,10 @@ async function exportAppImpl(
       clientTraceMetadata: nextConfig.experimental.clientTraceMetadata,
       expireTime: nextConfig.expireTime,
       dynamicIO: nextConfig.experimental.dynamicIO ?? false,
-      clientSegmentCache: nextConfig.experimental.clientSegmentCache ?? false,
+      clientSegmentCache:
+        nextConfig.experimental.clientSegmentCache === 'client-only'
+          ? 'client-only'
+          : Boolean(nextConfig.experimental.clientSegmentCache),
       inlineCss: nextConfig.experimental.inlineCss ?? false,
       authInterrupts: !!nextConfig.experimental.authInterrupts,
     },

packages/next/src/server/app-render/app-render.tsx

@@ -4212,7 +4212,17 @@ async function collectSegmentData(
   // decomposed into a separate stream per segment.
 
   const clientReferenceManifest = renderOpts.clientReferenceManifest
-  if (!clientReferenceManifest || !renderOpts.experimental.clientSegmentCache) {
+  if (
+    !clientReferenceManifest ||
+    // Do not generate per-segment data unless the experimental Segment Cache
+    // flag is enabled.
+    //
+    // We also skip generating segment data if flag is set to "client-only",
+    // rather than true. (The "client-only" option only affects the behavior of
+    // the client-side implementation; per-segment prefetches are intentionally
+    // disabled in that configuration).
+    renderOpts.experimental.clientSegmentCache !== true
+  ) {
     return
   }
 

packages/next/src/server/app-render/types.ts

@@ -213,7 +213,7 @@ export interface RenderOptsPartial {
     expireTime: number | undefined
     clientTraceMetadata: string[] | undefined
     dynamicIO: boolean
-    clientSegmentCache: boolean
+    clientSegmentCache: boolean | 'client-only'
     inlineCss: boolean
     authInterrupts: boolean
   }

packages/next/src/server/base-server.ts

@@ -605,7 +605,9 @@ export default abstract class Server<
         clientTraceMetadata: this.nextConfig.experimental.clientTraceMetadata,
         dynamicIO: this.nextConfig.experimental.dynamicIO ?? false,
         clientSegmentCache:
-          this.nextConfig.experimental.clientSegmentCache ?? false,
+          this.nextConfig.experimental.clientSegmentCache === 'client-only'
+            ? 'client-only'
+            : Boolean(this.nextConfig.experimental.clientSegmentCache),
         inlineCss: this.nextConfig.experimental.inlineCss ?? false,
         authInterrupts: !!this.nextConfig.experimental.authInterrupts,
       },

packages/next/src/server/config-schema.ts

@@ -299,7 +299,9 @@ export const configSchema: zod.ZodType<NextConfig> = z.lazy(() =>
         memoryBasedWorkersCount: z.boolean().optional(),
         craCompat: z.boolean().optional(),
         caseSensitiveRoutes: z.boolean().optional(),
-        clientSegmentCache: z.boolean().optional(),
+        clientSegmentCache: z
+          .union([z.boolean(), z.literal('client-only')])
+          .optional(),
         disableOptimizedLoading: z.boolean().optional(),
         disablePostcssPresetEnv: z.boolean().optional(),
         dynamicIO: z.boolean().optional(),

packages/next/src/server/config-shared.ts

@@ -269,7 +269,7 @@ export interface ExperimentalConfig {
   prerenderEarlyExit?: boolean
   linkNoTouchStart?: boolean
   caseSensitiveRoutes?: boolean
-  clientSegmentCache?: boolean
+  clientSegmentCache?: boolean | 'client-only'
   appDocumentPreloading?: boolean
   preloadEntriesOnStart?: boolean
   /** @default true */

test/e2e/app-dir/segment-cache/client-only-opt-in/app/dynamic-with-ppr/loading.tsx

@@ -0,0 +1,3 @@
+export default function Loading() {
+  return <div>Loading dynamic content...</div>
+}

test/e2e/app-dir/segment-cache/client-only-opt-in/app/dynamic-with-ppr/page.tsx

@@ -0,0 +1,8 @@
+import { connection } from 'next/server'
+
+export const experimental_ppr = true
+
+export default async function DynamicPage() {
+  await connection()
+  return <div id="dynamic-content">Dynamic Content</div>
+}

test/e2e/app-dir/segment-cache/client-only-opt-in/app/dynamic-without-ppr/loading.tsx

@@ -0,0 +1,3 @@
+export default function Loading() {
+  return <div>Loading dynamic content...</div>
+}

test/e2e/app-dir/segment-cache/client-only-opt-in/app/dynamic-without-ppr/page.tsx

@@ -0,0 +1,6 @@
+import { connection } from 'next/server'
+
+export default async function DynamicPage() {
+  await connection()
+  return <div id="dynamic-content">Dynamic Content</div>
+}

test/e2e/app-dir/segment-cache/client-only-opt-in/app/layout.tsx

@@ -0,0 +1,11 @@
+export default function RootLayout({
+  children,
+}: {
+  children: React.ReactNode
+}) {
+  return (
+    <html lang="en">
+      <body>{children}</body>
+    </html>
+  )
+}

test/e2e/app-dir/segment-cache/client-only-opt-in/app/page.tsx

@@ -0,0 +1,35 @@
+import { LinkAccordion } from '../components/link-accordion'
+
+export default function Page() {
+  return (
+    <div>
+      <h1>
+        <code>clientSegmentCache: 'client-only'</code>
+      </h1>
+      <p>
+        This page demonstrates the behavior when the experimental Segment Cache
+        flag is set to <code>'client-only'</code> instead of <code>true</code>.
+        The new prefetching implementation is enabled on the client, but the
+        server will not generate any per-segment prefetching. This is useful
+        because the client implementation has many other improvements that are
+        unrelated to per-segment prefetching, like improved scheduling and
+        dynamic request deduping.
+      </p>
+      <ul>
+        <li>
+          <LinkAccordion href="/static">Static page</LinkAccordion>
+        </li>
+        <li>
+          <LinkAccordion href="/dynamic-with-ppr">
+            Dynamic page (with PPR enabled)
+          </LinkAccordion>
+        </li>
+        <li>
+          <LinkAccordion href="/dynamic-without-ppr">
+            Dynamic page (without PPR enabled)
+          </LinkAccordion>
+        </li>
+      </ul>
+    </div>
+  )
+}

test/e2e/app-dir/segment-cache/client-only-opt-in/app/static/page.tsx

@@ -0,0 +1,3 @@
+export default function StaticPage() {
+  return <div id="static-content">Static Content</div>
+}