Skip to content

reduce cardinality of metrics #1593

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jul 3, 2020
Merged

reduce cardinality of metrics #1593

merged 5 commits into from
Jul 3, 2020

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Jul 1, 2020 •
edited
Loading

What do these changes do?

Related issue number

closes #1590

As showned in #1590 metrics labels should not be made of UUIDs that are never repeated:

CAUTION: Remember that every unique combination of key-value label pairs represents a new time series, which can dramatically increase the amount of data stored. Do not use labels to store dimensions with high cardinality (many different label values), such as user IDs, email addresses, or other unbounded sets of values.

How to test

Checklist

  • Did you change any service's API? Then make sure to bundle document and upgrade version (make openapi-specs, git commit ... and then make version-*)
  • Unit tests for the changes exist
  • Runs in the swarm
  • Documentation reflects the changes
  • New module? Add your github username to .github/CODEOWNERS

@sanderegg sanderegg requested review from pcrespov, GitHK and Surfict July 1, 2020 14:27
@sanderegg sanderegg self-assigned this Jul 1, 2020
@codecov
Copy link

codecov bot commented Jul 1, 2020 •
edited
Loading

Codecov Report

Merging #1593 into master will increase coverage by 0.4%.
The diff coverage is 50.0%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #1593     +/-   ##
========================================
+ Coverage    73.7%   74.2%   +0.4%     
========================================
  Files         278     263     -15     
  Lines       10874    9907    -967     
  Branches     1181    1031    -150     
========================================
- Hits         8015    7351    -664     
+ Misses       2516    2257    -259     
+ Partials      343     299     -44
Flag Coverage Δ
#integrationtests 56.6% <100.0%> (ø)
#unittests 67.2% <50.0%> (-0.2%) ⬇️
Impacted Files Coverage Δ
...service-library/src/servicelib/monitor_services.py 0.0% <0.0%> (ø)
...simcore_service_webserver/computation_subscribe.py 91.6% <100.0%> (ø)
...irector/src/simcore_service_director/exceptions.py
...ector/src/simcore_service_director/system_utils.py
...ctor/src/simcore_service_director/rest/handlers.py
...rc/simcore_service_director/rest/node_validator.py
...ector/src/simcore_service_director/docker_utils.py
...imcore_service_director/cache_request_decorator.py
.../director/src/simcore_service_director/__main__.py
...es/director/src/simcore_service_director/config.py
... and 6 more

@sanderegg sanderegg added this to the Huo Guo milestone Jul 1, 2020
GitHK
GitHK approved these changes Jul 1, 2020
View reviewed changes
Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, hope this will improve storage usage

@sanderegg sanderegg merged commit c025875 into ITISFoundation:master Jul 3, 2020
@sanderegg sanderegg deleted the bugfix/reduce_metrics_cardinality branch July 3, 2020 11:42
@odeimaiz odeimaiz mentioned this pull request Aug 4, 2020
Merged
odeimaiz added a commit that referenced this pull request Aug 4, 2020
@odeimaiz
FREEZE_XieXie (#1669)
5ec7ff1 
- UI/UX improvements (#1657)
- Bump yarl from 1.4.2 to 1.5.1 in /packages/postgres-database (#1665)
- Bump ujson from 3.0.0 to 3.1.0 in /packages/service-library (#1664)
- Bump pytest-docker from 0.7.2 to 0.8.0 in /packages/service-library (#1647)
- Improving storage performance (#1659)
- Bump aiozipkin from 0.6.0 to 0.7.0 in /packages/service-library (#1642)
- Theming (#1656)
- Platform stability:  (#1645)
- is1594 fix and re-activate e2e testing (#1620)
- 2 bugs fixed + Some improvements (#1634)
- Fixes default (#1640)
- Bump lodash from 4.17.15 to 4.17.19 (#1639)
- Is1585/cleanup storage (#1586)
- Fixes on publish studies handling (#1632)
- Some enhancements and bug fixes (#1608)
- Improve e2e  (#1631)
- filter studies by name before deleting them (#1629)
- Maintenance/upgrades test tools (#1628)
- Bugfix/concurent opening projects (#1598)
- Bugfix/allow reading groups anonymous user (#1615)
- Bump docker from 4.2.1 to 4.2.2 in /packages/postgres-database (#1605)
- fix testing if node has gpu support (#1604)
- [bugfix] Invalidate cache before starting a study (#1602)
- Feature/fix e2e 2 (#1600)
- fix deploy not needing e2e testing since it is disabled
- reduce cardinality of metrics (#1593)
- Excudes e2e stage from include until fixed (#1595)
- Shared project concurrency (frontend) (#1591)
- Homogenize studies and services (#1569)
- [feature] UI Fine grained access - project locking and notification
- Bugfix/apiserver does not need sslheaders (#1564)
- Cleanup catalog service (#1582)
- Maintenance/cleanup api server (#1578)
- Adds support for GPU scheduling of computational services (#1553)
- Maintenance/upgrades and tooling (#1546)
- Is1570/study fails 500 (#1572)
- Bump faker from 4.1.0 to 4.1.1 in /packages/postgres-database (#1573)
- maintenance fix codecov reports (#1568)
- Manage groups, Share studies (#1512)
- Is/add notebook migration script (#1565)
- Is1269/api-server upgrade (#1475)
- added simcore_webserver_service in pytest simcore package (#1563)
- add traefik endpoint to api-gateway (#1555)
@sanderegg sanderegg mentioned this pull request Aug 21, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GitHK GitHK GitHK approved these changes

@pcrespov pcrespov pcrespov approved these changes

@Surfict Surfict Awaiting requested review from Surfict

Assignees

@sanderegg sanderegg

Labels
None yet
Projects
None yet
Milestone
Huo Guo
Development

Successfully merging this pull request may close these issues.

remove high cardinality labels from metrics as this create performance issues in prometheus
3 participants
@sanderegg @GitHK @pcrespov