You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Configurations are now stored in the config index. Throughout the code, requests to the config index have their refresh policy set to IMMEDIATE. This is problematic. When an action is performed that causes multiple refreshes to the index (e.g. deleting mutliple jobs/datafeeds), the refreshes are throttled and long delays (~10s) have been observed.
Futhermore, requests that are not a result of a user action need not have a refresh. For example, when the established model memory is updated during runtime. Refreshing on those is delaying the job for no reason.
In order to avoid these delays, but also make sure single requests are responsive we need to:
Use WAIT_FOR for requests that are a result of a user action
Use NONE for internal requests
Reduce the refresh_interval to a value that is responsive without putting pressure on the index (e.g. 100ms)? Note the config index is a low volume index so it should be able to handle quick refreshing.
The text was updated successfully, but these errors were encountered:
This was a false alarm. The delays I observed were caused by a sleep in the master operation of the job delete action. This was blocking the master node and thus the datafeed delete action was stalled.
Configurations are now stored in the config index. Throughout the code, requests to the config index have their refresh policy set to
IMMEDIATE
. This is problematic. When an action is performed that causes multiple refreshes to the index (e.g. deleting mutliple jobs/datafeeds), the refreshes are throttled and long delays (~10s) have been observed.This is also probably the reason behind #30300.
Futhermore, requests that are not a result of a user action need not have a refresh. For example, when the established model memory is updated during runtime. Refreshing on those is delaying the job for no reason.
In order to avoid these delays, but also make sure single requests are responsive we need to:
WAIT_FOR
for requests that are a result of a user actionNONE
for internal requestsrefresh_interval
to a value that is responsive without putting pressure on the index (e.g.100ms
)? Note the config index is a low volume index so it should be able to handle quick refreshing.The text was updated successfully, but these errors were encountered: