Minimize requests to /metrics using some logic about connectivity from browser->server

[consideRatio]

I started filling out jupyterlab's feature request form but opted to open the issue here instead. My goal is to identify the source of various web-requests made from browser's and try to avoid them if they are doomed to fail because of a shut down notebook-server / jupyter-server.

In JupyterLab they have the IConnectionLost interface for example which can be used to act smarter for example. Here is an example of how its used.

I understand that nbresuse is something not only used by JupyterLab, so perhaps its solution must be standalone, but perhaps we could provide a hook or similar to allow us make use of knowledge about the connectivity to the server from the browser, implemented in JupyterLab or similar.

Here is an example of the amount of requests that arrives to JupyterHub, and how they can be totally cluttered when a user server shuts down, and the /user/username1 route is removed from the JupyterHub proxy, which make all requests to go /hub/user/username1 as a default fallback.

Problem

Make various logs more readable and reduce load on other services.

Proposed Solution

I hope it's possible to make use of a common interface to determine if the notebook-server / jupyter-server is available, and cut down on requests if it isn't. I'm not sure how much of what's below is something to fix within JupyterLab and how much is to fix in other projects, but I suspect that the /metrics calls come from the [nbresuse]

Additional context

I took logs from my JupyterHub, and found the following.

/api/terminals - 5 min interval

04:47:54 503 GET /hub/user/hub-user1/api/terminals?1595738873967
04:52:55 503 GET /hub/user/hub-user1/api/terminals?1595739174718

/api/sessions - 5 min interval

04:47:59 503 GET /hub/user/hub-user1/api/sessions?1595738878979
04:52:59 503 GET /hub/user/hub-user1/api/sessions?1595739179448

/api/kernepspecs - unknown interval

04:50:25 503 GET /hub/user/hub-user1/api/kernelspecs?1595739024887

/api/kernels - unknown interval

# increments of 25s, 66s, 98s, 156s between calls... Hmmm....
04:47:50 503 GET /hub/user/hub-user1/api/kernels?1595738869887
04:48:15 503 GET /hub/user/hub-user1/api/kernels?1595738894919
04:49:21 503 GET /hub/user/hub-user1/api/kernels?1595738961260
04:50:49 503 GET /hub/user/hub-user1/api/kernels?1595739049098
04:53:25 503 GET /hub/user/hub-user1/api/kernels?1595739205581

/api/contents - ~10 second interval

04:46:55 503 GET /hub/user/hub-user1/api/contents/hub-user1-storage-folder?content=1&1595738815090
04:47:06 503 GET /hub/user/hub-user1/api/contents/hub-user1-storage-folder?content=1&1595738825617
04:47:16 503 GET /hub/user/hub-user1/api/contents/hub-user1-storage-folder?content=1&1595738836264
04:47:27 503 GET /hub/user/hub-user1/api/contents/hub-user1-storage-folder?content=1&1595738846731
04:47:37 503 GET /hub/user/hub-user1/api/contents/hub-user1-storage-folder?content=1&1595738857252

/api/metrics - ~5 second interval

# ~5 s
04:46:46 503 GET /hub/user/hub-user1/metrics?1595738805671
04:46:51 503 GET /hub/user/hub-user1/metrics?1595738811272
04:46:57 503 GET /hub/user/hub-user1/metrics?1595738816741
04:47:02 503 GET /hub/user/hub-user1/metrics?1595738822330
04:47:08 503 GET /hub/user/hub-user1/metrics?1595738827786

[jtpio]

Thanks @consideRatio for all these details!

I suspect that the /metrics calls come from the [nbresuse]

It does indeed come from the nbresuse frontend:

• in the classic notebook extension: https://github.com/yuvipanda/nbresuse/blob/de920c0aec27467805e61678da2c8c61eecbbced/nbresuse/static/main.js#L36-L37
• in JupyterLab: https://github.com/jupyterlab/jupyterlab/blob/0b88d64b622fc03f4c06ce9cceaa13bc443db468/packages/statusbar/src/defaults/memoryUsage.tsx#L288-L291

/api/kernels - unknown interval

The polling logic for these endpoints is defined here:

https://github.com/jupyterlab/jupyterlab/blob/0b88d64b622fc03f4c06ce9cceaa13bc443db468/packages/services/src/kernel/manager.ts#L32-L40

hope it's possible to make use of a common interface to determine if the notebook-server / jupyter-server is available, and cut down on requests if it isn't.

Yes either this common interface (but it would have to be usable from multiple frontends), or use exponential backoff when making the requests.

For nbresuse specifically there has also been the idea of switching to event streams: #19