Add stats for time spent fetching data while searching snapshots #51866
Conversation
This commit builds on elastic#51637, adding tracking of the total time spent fetching data from the blob store and a linear regression model for these fetches.
DaveCTurner
added
WIP
:Distributed Coordination/Snapshot/Restore
|
Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore) |
|
I'm not sure the linear regression is actually useful here. Since we're tracking stats on a file-by-file basis, essentially every fetch will be the same size, which stops us from building a meaningful model of the fetch time as a function of size. Maybe we just want to track the total took time? Possibly min and max too? |
|
I tend to agree with you, unless we implement different range sizes per files (which we could do easily) I'm not sure the linear regression is very useful. Maybe we could reuse the existing counters and track the total/min/max took times as you suggested. |
…e-snapshots-track-time-spent-fetching-from-blob-store
| @@ -106,7 +106,7 @@ public void onRepositoriesModule(RepositoriesModule repositoriesModule) { | |||
| @Override | |||
| public Map<String, DirectoryFactory> getDirectoryFactories() { | |||
| return Map.of(SearchableSnapshotRepository.SNAPSHOT_DIRECTORY_FACTORY_KEY, | |||
| SearchableSnapshotRepository.newDirectoryFactory(repositoriesService::get, cacheService::get)); | |||
| SearchableSnapshotRepository.newDirectoryFactory(repositoriesService::get, cacheService::get, System::nanoTime)); | |||
There was a problem hiding this comment.
Using System::nanoTime since we need finer resolution than ThreadPool::relativeTimeInNanos offers.
DaveCTurner
changed the title
...main/java/org/elasticsearch/xpack/core/searchablesnapshots/SearchableSnapshotShardStats.java
Outdated
Show resolved
Hide resolved
x-pack/plugin/searchable-snapshots/qa/rest/src/test/resources/rest-api-spec/test/stats.yml
Show resolved
Hide resolved
...napshots/src/main/java/org/elasticsearch/xpack/searchablesnapshots/cache/CacheDirectory.java
Show resolved
Hide resolved
...main/java/org/elasticsearch/xpack/core/searchablesnapshots/SearchableSnapshotShardStats.java
Outdated
Show resolved
Hide resolved
DaveCTurner
deleted the
2020-02-04-searchable-snapshots-track-time-spent-fetching-from-blob-store
branch
This commit builds on #51637, adding tracking of the total time spent fetching
data from the blob store.
Relates #50999.