Both SSE and StreamableHttp transport require sticky sessions

[dglozic]

Is your feature request related to a problem? Please describe.
Both SSE and StreamableHttp transport rely on caching the transport in memory in the server pod:

const transports: { [sessionId: string]: StreamableHTTPServerTransport } = {};

This does not work when there are multiple replicas/pods behind a load balancer (which is most k8s deployments of MCP servers as services). This means that if the call is not the initial one, and the cached transport is not found in the transports, error is sent out.

We need a solution (and example) that does not involve storing transports in memory because real world deployments of remote MCP servers will have multiple replicas, and demanding sticky session for them is too much of a burden.

If we can ask implementors to cache state that is can survive JSON.stringify/parse roundtrip, they can cache state in distributed caches like Redis session store. But transport object cannot be stored in Redis.

Describe the solution you'd like
A stateless solution that does not involve caching transports in memory, or a stateful solution that can be serialized and stored as strings in caches like Redis.

Describe alternatives you've considered
I seems that streamableHttp transport may be able to work without session ids completely, but the example in code does not shows how. I will try to create an example where sessionID generator is undefined, but if so, we would always need to create a new transport inside request handlers, and not try to reuse them from memory cache. I am not sure this will work.

[smurf28]

This also troubles me. I want distributed deployment, but it is difficult to achieve.

[dglozic]

Nobody deploys a single service replica in a cluster, so this affects all real world remote MCP server implementations. WebSickets based transport would work since it opens and keeps the socket open, but that also does make sense. You want a transport solution that opens the socket for the session duration and then closes it to conserve resources.

[asprouse]

It seems possible to use this transport in a "stateless" mode according to comments in the code:

typescript-sdk/src/server/streamableHttp.ts

Lines 68 to 89 in 09e5d5b

	* // Stateless mode - explicitly set session ID to undefined
	* const statelessTransport = new StreamableHTTPServerTransport({
	* sessionId: undefined,
	* });
	*
	* // Using with pre-parsed request body
	* app.post('/mcp', (req, res) => {
	* transport.handleRequest(req, res, req.body);
	* });
	* ```
	*
	* In stateful mode:
	* - Session ID is generated and included in response headers
	* - Session ID is always included in initialization responses
	* - Requests with invalid session IDs are rejected with 404 Not Found
	* - Non-initialization requests without a session ID are rejected with 400 Bad Request
	* - State is maintained in-memory (connections, message history)
	*
	* In stateless mode:
	* - Session ID is only included in initialization responses
	* - No session validation is performed
	*/

I am not sure what you give up going stateless but it seems like this would be preferred for a production server. Do most major clients support this?

[dglozic]

It is hard to test until StreamableHttp transport is published in NPM. Right now, the example implies only stateful mode, but description of the transport constructor implies stateless mode is supported.

[asprouse]

It is hard to test until StreamableHttp transport is published in NPM. Right now, the example implies only stateful mode, but description of the transport constructor implies stateless mode is supported.

@gylove1994 @ihrpr I see that you've been working on this. Is stateless mode really something that is usable?

@jspahrsummers @jerome3o-anthropic Do you know timeline when this stateless code will be published to NPM?

[ihrpr]

@asprouse, it's available now in 1.10.1 version package. Also added a folder with examples -- see README and an example for stateless

[ihrpr]

@dglozic some hight level notes on distributed deployment are here

[dglozic]

Yeah, this seems what I was looking for, let me try it.

[dglozic]

Actually, there is still room for clarity in README.md:

you can use a database to persist session data while still allowing any node to handle requests.

In order to persist session data in a database (say, Redis cache), what is the state actually and how can this state be serialized (ideally into JSON that can be cached between nodes)?

[buggyhunter]

@dglozic did you find a way to use/try non-sticky sessions. As you mentioned in first thread:

 Both SSE and StreamableHttp transport rely on caching the transport in memory in the server pod

I encountered the same problem.

We need to find a way to persist transport objects. (serialization/deserialization)

@ihrpr any insights would be very helpful.

[dglozic]

It seems that with the latest release, StreamableHttp can be used stateless (but it is still recommended to serialize state if possible). I am waiting to see how exactly :-).