Skip to content

PG queries are not timing out on server going down #3399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Scorpio1987 opened this issue Mar 7, 2025 · 4 comments
Open

PG queries are not timing out on server going down #3399

Scorpio1987 opened this issue Mar 7, 2025 · 4 comments

Comments

@Scorpio1987
Copy link

I am seeing an issue where the query doesn't timeout and the client connection stays indefinitely even when the server goes down.
I check out a client using pool.connect() and then call client.query()

I used a query that runs 30 seconds. Before the query completes, i bring down network connection to the server and this results in query being stuck forever.

I tried it using query_timeout and as expected client.query() fails with "Query read timeout" error if the network connection to server is lost. But when i try the next query, i don't see any packets being sent out and the next query also times out with the above error.

Please let me know if there is a way to solve this query being stuck issue.

@stas-ut21
Copy link

@Scorpio1987 try setting the connectionTimeoutMillis and idleTimeoutMillis timeouts more than 0

@Scorpio1987
Copy link
Author

@stas-ut21 That didn't solve the issue.

After debugging more and looking through the code, it seems like the underlying client is stuck waiting for the response for the query from the server.

This is the flow of things happened.

  1. The application calls "pool.query", which creates a new client, establishes connection to server and runs the query.
  2. Let's say the connection b/w client and server goes down after the query is sent to server (We are waiting for the response from server). Since the connection to server is gone, this connection is now stuck forever.
  3. Now if there is a query timeout configured, after the query timeout, the error is sent to the application and the client is put back into the idle client list.
  4. Now if another query happens before the idleTimeoutMillis, this client is picked up from pool and query is called on the client. But this query never goes out because the client is still waiting for the previous query to be completed.

@charmander
Copy link
Collaborator

You’ll probably also want to set the (undocumented?) keepAliveIdleMillis option to a reasonably low value.

@stas-ut21
Copy link

@stas-ut21 That didn't solve the issue.

After debugging more and looking through the code, it seems like the underlying client is stuck waiting for the response for the query from the server.

This is the flow of things happened.

  1. The application calls "pool.query", which creates a new client, establishes connection to server and runs the query.
  2. Let's say the connection b/w client and server goes down after the query is sent to server (We are waiting for the response from server). Since the connection to server is gone, this connection is now stuck forever.
  3. Now if there is a query timeout configured, after the query timeout, the error is sent to the application and the client is put back into the idle client list.
  4. Now if another query happens before the idleTimeoutMillis, this client is picked up from pool and query is called on the client. But this query never goes out because the client is still waiting for the previous query to be completed.

pls write a sample code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants