Smoothing of a constant plot is messed up

[nikonikolov]

When I try to smooth a value which is constant, then some strange spikes appear in the plot.
For example:

Here actor_learn_rate is always a constant. Interestingly, in its own tab it is displayed correctly, but in the Tags matching tab it is messed up. When I hover over the plot at a point corresponding to a spike, the correct value of 1e-4 is reported. The case is the same for critic_learn_rate.

Interestingly, the spikes depend on the smoothing value. For example, for a smoothing value of 0.222, there are no spikes:

However, the orange line is not displayed at 1e-4 which is the correct value.

Then a smoothing value of 0 is the same.

My version is tensorflow-tensorboard (0.4.0rc3)

[netheril96]

I have the same problem. It only shows up in the recent version the tensorboard, but not old ones.

[nfelt]

Hmm, if it's only showing up in the latest release of TensorBoard, I wonder if it might be inadvertently introduced by #639 ? cc @alexirpan

[danijar]

I can try to take a look at the smoothing code (there are some other changes I'd like to play with). Can I edit the smoothing code of a running TensorBoard instance from the browser somehow or how would you suggest getting started?

[alexirpan]

When I worked on #639 I downloaded + built a local copy of the Tensorboard codebase, then worked off of that.

I suspect it's some issue with floating point error, where the code does

d.smoothed = last / debiasWeight;

[bileschi]

@stephanwlee thanks for your recent fixes! Can this issue be closed?

[nfelt]

Per @wchargin comments on the fix PR I think we consider the general problem here to still not be solved: #1698 (comment)

[wchargin]

Correct; what’s needed is a more numerically stable implementation of
the smoothing algorithm. This is “contributions welcome” under our usual
policy for smoothing algorithm changes:

It seems that every change in the smoothing algorithm results in new
tradeoffs and biases, and new requests to change the algorithm. At
this point it's not a priority for the team, so just filing an issue
probably won't directly result in changes. However, the smoothing
algorithm is actually a very simple piece of code to change. If you
come up with a new algorithm, and demonstrate persuasively why it is
an improvement over the current one (taking into account new biases
and edge cases that it introduces), [we’ll] happily merge it.

(#610 (comment))

[danijar]

I'd still be interested in playing with the smoothing algorithm. Same question as a while ago:

I can try to take a look at the smoothing code (there are some other changes I'd like to play with). Can I edit the smoothing code of a running TensorBoard instance from the browser somehow or how would you suggest getting started?

[wchargin]

There’s not an easy way to edit the smoothing algorithm live in the
browser, no. You’ll have to restart the server after each change to the
logic.

I suppose that it’s possible that you could change the smoothing
algorithm to eval a global string variable that you modify in the
browser, but I’ve never tried this. :-)

[pindinagesh]

@nikonikolov

Could you please move this to closed status as the PR already merged. Thanks

[nfelt]

@pindinagesh the PR doesn't fully address the issue, which I explained in #786 (comment)

So this issue should stay open pending a more general solution.