Skip to content

ref(discover2) Test out sampling on tag values #16387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 10, 2020
Merged

ref(discover2) Test out sampling on tag values #16387

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f6b1772
ref(discover2) Test out sampling on tag values
markstory Jan 10, 2020
0da24dc
Use fewer lines of code
markstory Jan 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions src/sentry/snuba/discover.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -447,15 +447,14 @@ def get_facets(query, params, limit=10, referrer=None):
# Resolve the public aliases into the discover dataset names.
snuba_args, translated_columns = resolve_discover_aliases(snuba_args)

# Force sampling for multi-project results as we don't need accuracy
# with that much data.
sample = len(snuba_filter.filter_keys["project_id"]) > 2

# Exclude tracing tags as they are noisy and generally not helpful.
excluded_tags = ["tags_key", "NOT IN", ["trace", "trace.ctx", "trace.span"]]

# Get the most frequent tag keys, enable sampling
# as we don't need accuracy here.
# Sampling keys for multi-project results as we don't need accuracy
# with that much data.
sample = len(snuba_filter.filter_keys["project_id"]) > 2

# Get the most frequent tag keys
key_names = raw_query(
aggregations=[["count", None, "count"]],
start=snuba_args.get("start"),
Expand All @@ -474,6 +473,11 @@ def get_facets(query, params, limit=10, referrer=None):
if not top_tags:
return []

# TODO(mark) Make the sampling rate scale based on the result size and scaling factor in
# sentry.options.
# To test the lowest acceptable sampling rate, we use turbo mode.
turbo_values = key_names["data"][0]["count"] > 10000

fetch_projects = False
if len(params.get("project_id", [])) > 1:
if len(top_tags) == limit:
Expand All @@ -492,6 +496,7 @@ def get_facets(query, params, limit=10, referrer=None):
orderby="-count",
dataset=Dataset.Discover,
referrer=referrer,
turbo=turbo_values,
)
results.extend(
[FacetResult("project", r["project_id"], r["count"]) for r in project_values["data"]]
Expand All @@ -513,6 +518,7 @@ def get_facets(query, params, limit=10, referrer=None):
limit=TOP_VALUES_DEFAULT_LIMIT,
dataset=Dataset.Discover,
referrer=referrer,
turbo=turbo_values,
)
results.extend([FacetResult(tag_name, r[tag], int(r["count"])) for r in tag_values["data"]])

Expand Down