Skip to content

Improve the scalar chart user experience #189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 6, 2017

Conversation

wchargin
Copy link
Contributor

@wchargin wchargin commented Jul 6, 2017

Summary:
This fixes a number of issues with the scalar charts, some of which are
interdependent, and all of which are existing issues made more visible
by the presence of a search pane.

In particular:

  1. If a chart's run and tag properties are changed simultaneously,
    two updates will be triggered, and the first update will see the
    data in an inconsistent state (e.g., Tag B but still Run A). We fix
    the resulting issues by making observer updates asynchronous, so
    that they only actually take effect once all changes have flushed.
    (This fixes Run-tag updates are not atomic #169.)

  2. To reduce latency due to the above solution, we further debounce
    these update handlers.

  3. Because the handlers are asynchronous and debounced, we need to
    maintain stronger invariants on the caching keys. (In particular, if
    a chart changes from Tag A to Tag B and then to Tag C, we don't want
    the network requests for Tag B to fill the runs-loaded cache and
    then have Tag C forgo its fetches.)

  4. Furthermore, when the tag of a chart is changed, we display a
    loading indicator. This is to prevent problems in the scenario where
    many charts change from one tag to another: in this case, all the
    charts' titles will update immediately (they're just bound to text
    nodes in the DOM), but charts' data will take a while to update; the
    result is that there are many charts with titles that simply do not
    match their data, which is at best very confusing. The loading
    indicator makes it clear that the data is stale.

  5. When a chart's tag is changed, we now reset the domain of the chart
    once all the data has loaded. (This fixes Race condition (consistent repro): wherein chart domain is set only after data arrives #164.)

  6. Finally, we offer a button to manually reset the domain of the
    chart. (This is often useful when changing the set of selected
    runs: after adding a new run, you may need to expand the domain, but
    we don't do this automatically because you may often want to keep
    your old domain.)

Issues (1), (4), and (5) arise frequently when changing a search query,
as this causes many charts to change their tag and runs.

Issues (1), (2), and (4) apply to other dashboards as well, but have not
been fixed. (Issue (3) would apply to other dashboards that display one
visualization per tag instead of per run-tag combination, but there are
no such dashboards.)

Test Plan:
Cherry-pick #173 before proceeding. (Without a search feature, the
effects of this change are minimal---but I want to merge this change
before I merge the search feature!)

Launch TensorBoard with a few dozen scalar tags. Suppose without loss of
generality that the name of the fifth tag displayed in the search
results pane (on query .*) is foo. Set the search query to
(?!foo); this should still match all tags. Now, add a caret at the
beginning to search for ^(?!foo). Charts past the fifth one should
reload. Check that the reloading behavior is pleasant.

Then, zoom into some charts, manipulating their windows. Click the
"reload" button, or wait 30 seconds, and ensure that the window is
preserved. Change the search query, though, and the windows should be
reset to show all the data.

wchargin-branch: improve-scalars-ux

Summary:
This fixes a number of issues with the scalar charts, some of which are
interdependent, and all of which are existing issues made more visible
by the presence of a search pane.

In particular:

 1. If a chart's `run` and `tag` properties are changed simultaneously,
    two updates will be triggered, and the first update will see the
    data in an inconsistent state (e.g., Tag B but still Run A). We fix
    the resulting issues by making observer updates asynchronous, so
    that they only actually take effect once all changes have flushed.
    (This fixes #169.)

 2. To reduce latency due to the above solution, we further debounce
    these update handlers.

 3. Because the handlers are asynchronous and debounced, we need to
    maintain stronger invariants on the caching keys. (In particular, if
    a chart changes from Tag A to Tag B and then to Tag C, we don't want
    the network requests for Tag B to fill the runs-loaded cache and
    then have Tag C forgo its fetches.)

 4. Furthermore, when the tag of a chart is changed, we display a
    loading indicator. This is to prevent problems in the scenario where
    many charts change from one tag to another: in this case, all the
    charts' titles will update immediately (they're just bound to text
    nodes in the DOM), but charts' data will take a while to update; the
    result is that there are many charts with titles that simply do not
    match their data, which is at best very confusing. The loading
    indicator makes it clear that the data is stale.

 5. When a chart's tag is changed, we now reset the domain of the chart
    once all the data has loaded. (This fixes #164.)

 6. Finally, we offer a button to manually reset the domain of the
    chart. (This is often useful when changing the set of selected
    runs: after adding a new run, you may need to expand the domain, but
    we don't do this automatically because you may often want to keep
    your old domain.)

Issues (1), (4), and (5) arise frequently when changing a search query,
as this causes many charts to change their tag and runs.

Issues (1), (2), and (4) apply to other dashboards as well, but have not
been fixed. (Issue (3) would apply to other dashboards that display one
visualization per tag instead of per run-tag combination, but there are
no such dashboards.)

Test Plan:
Cherry-pick #173 before proceeding. (Without a search feature, the
effects of this change are minimal---but I want to merge this change
before I merge the search feature!)

Launch TensorBoard with a few dozen scalar tags. Suppose without loss of
generality that the name of the fifth tag displayed in the search
results pane (on query `.*`) is `foo`. Set the search query to
`(?!foo)`; this should still match all tags. Now, add a caret at the
beginning to search for `^(?!foo)`. Charts past the fifth one should
reload. Check that the reloading behavior is pleasant.

Then, zoom into some charts, manipulating their windows. Click the
"reload" button, or wait 30 seconds, and ensure that the window is
preserved. Change the search query, though, and the windows should be
reset to show all the data.

wchargin-branch: improve-scalars-ux
@wchargin wchargin merged commit c8e185e into master Jul 6, 2017
@wchargin wchargin deleted the wchargin-improve-scalars-ux branch July 6, 2017 23:12
chihuahua added a commit that referenced this pull request Aug 18, 2017
Specifically, rewired the vz_line_chart component to take

- an object that configures the X axis (the specific field and scale)
- an Plottable accessor that computes the value on the Y axis
- a list of TooltipColumns that specify the columns within the table in the tooltip

Rewired scalar line charts to use the generalized logic. This change sets the stage for the PR curve plugin to also use vz_line_chart, discouraging duplicate code.

Unfortunately, by generalizing vz_line_chart instead of tf-scalar-chart.html, we do miss out on some benefits. tf-scalar-chart.html contains some logic to correct bugs in Plottable (#163, #189, #204). We should think about how to bring those benefits to other dashboards that use vz_line_chart.
chihuahua added a commit that referenced this pull request Aug 22, 2017
Specifically, rewired the vz_line_chart component to take

- an object that configures the X axis (the specific field and scale)
- an Plottable accessor that computes the value on the Y axis
- a list of TooltipColumns that specify the columns within the table in the tooltip

Rewired scalar line charts to use the generalized logic. This change sets the stage for the PR curve plugin to also use vz_line_chart, discouraging duplicate code.

Unfortunately, by generalizing vz_line_chart instead of tf-scalar-chart.html, we do miss out on some benefits. tf-scalar-chart.html contains some logic to correct bugs in Plottable (#163, #189, #204). We should think about how to bring those benefits to other dashboards that use vz_line_chart.
chihuahua added a commit that referenced this pull request Aug 25, 2017
Specifically, rewired the vz_line_chart component to take

- an object that configures the X axis (the specific field and scale)
- an Plottable accessor that computes the value on the Y axis
- a list of TooltipColumns that specify the columns within the table in the tooltip

Rewired scalar line charts to use the generalized logic. This change sets the stage for the PR curve plugin to also use vz_line_chart, discouraging duplicate code.

Unfortunately, by generalizing vz_line_chart instead of tf-scalar-chart.html, we do miss out on some benefits. tf-scalar-chart.html contains some logic to correct bugs in Plottable (#163, #189, #204). We should think about how to bring those benefits to other dashboards that use vz_line_chart.
chihuahua added a commit that referenced this pull request Sep 4, 2017
Before this change, tf-scalar-chart.html contains much logic specific to
the scalars plugin. We extract that scalars-specific logic into a new
component called tf-scalar-card. Some of the logic extracted includes
properties that allow for setting X and Y axes, the buttons under the
chart, and the callback that is run when data is received from the
server.

This change is a step towards allowing any plugin to make use of the
tf-scalar-chart component (albeit a subsequent change will move and
rename this component to clarify that it is generally applicable). That
goal is desirable because tf-scalar-chart contains a lot of logic that
corrects for bugs in the vz-line-chart component and Plottable. That
logic includes #163, #189, and #204.
chihuahua added a commit that referenced this pull request Sep 12, 2017
Before this change, tf-scalar-chart.html contains much logic specific to
the scalars plugin. We extract that scalars-specific logic into a new
component called tf-scalar-card. Some of the logic extracted includes
properties that allow for setting X and Y axes, the buttons under the
chart, and the callback that is run when data is received from the
server.

This change is a step towards allowing any plugin to make use of the
tf-scalar-chart component (albeit a subsequent change will move and
rename this component to clarify that it is generally applicable). That
goal is desirable because tf-scalar-chart contains a lot of logic that
corrects for bugs in the vz-line-chart component and Plottable. That
logic includes #163, #189, and #204.
chihuahua added a commit that referenced this pull request Sep 15, 2017
Before this change, tf-scalar-chart.html contains much logic specific to
the scalars plugin. We extract that scalars-specific logic into a new
component called tf-scalar-card. Some of the logic extracted includes
properties that allow for setting X and Y axes, the buttons under the
chart, and the callback that is run when data is received from the
server.

This change is a step towards allowing any plugin to make use of the
tf-scalar-chart component (albeit a subsequent change will move and
rename this component to clarify that it is generally applicable). That
goal is desirable because tf-scalar-chart contains a lot of fixes that
correct for bugs in the vz-line-chart component and Plottable. Those fixes
include #163, #189, and #204.
chihuahua added a commit that referenced this pull request Sep 18, 2017
We refactor the card that loads PR curves to use the
tf-line-chart-loader component. This has several major advantages,
as detailed by #163, #189, and #204.

Test plan: Note that the PR curves dashboard still WAI.
chihuahua added a commit that referenced this pull request Sep 19, 2017
We refactor the card that loads PR curves to use the
tf-line-chart-loader component. This has several major advantages,
as detailed by #163, #189, and #204.

Test plan: Note that the PR curves dashboard still WAI.
chihuahua added a commit that referenced this pull request Sep 26, 2017
We refactor the card that loads PR curves to use the
tf-line-chart-loader component. This has several major advantages,
as detailed by #163, #189, and #204.

Test plan: Note that the PR curves dashboard still WAI.
chihuahua added a commit that referenced this pull request Sep 27, 2017
Refactored the card that loads PR curves to use the
tf-line-chart-loader component. This has several major advantages,
as detailed by #163, #189, and #204.

Part of this effort involved adding defaultXRange and
defaultYRange fields to tf-line-chart-data-loader.

Test plan:
Run the scalars and PR curves demos. Merge the directories, and
start TensorBoard. Note that the scalars and PR curves dashboards
still WAI. Check that sliders and buttons work.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants