Skip to content

Custom legend #637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Nov 9, 2021
Merged

Custom legend #637

merged 22 commits into from
Nov 9, 2021

Conversation

moontrip
Copy link
Contributor

@bobular @dmfalke I have made PR for initial custom legend works presented at the dev call. This relies on the PR at web-components as well.

@moontrip moontrip requested review from bobular and dmfalke October 27, 2021 17:27
@moontrip
Copy link
Contributor Author

changed mui to HTML for adding flexibility: only allows string so it is not possible to add markers etc. Also changed some layouts.

Will move customLegend to web-components's plotControls and change its name to PlotLegend

@moontrip
Copy link
Contributor Author

moved CustomLegend (now PlotLegend) to web-components's plotControls. Changed its function name to PlotLegend as well

@moontrip
Copy link
Contributor Author

Scatter plot viz has various exceptional cases, like having plot options, thus for handling more general case at first, current custom legend applied to histogram viz. Though, it also has at least one exception that the order of legend items are reversed due to stacked plot: this will be addressed later after dealing with other to-dos first, e.g., filters, No data, markers, etc.

Copy link
Member

@bobular bobular left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good apart from the thing we already discussed about checking for hasData.

// extract legendItems.label for passing to both PlotlyPlot and PlotLegend
setCheckedLegendItems(legendItems.map((item) => item.label));
}
}, [data]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add legendItems just to be safe?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep will add!

filters != null && overlayVariableFilter
? filters
.map((filter) => {
if (filter.type == 'stringSet' && data.name != null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed on Slack, this needs to look inside the data to see if any bin.count is non-zero

Copy link
Contributor Author

@moontrip moontrip Nov 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I am changing it by checking data.bins.length. Perhaps bin.count is another way too 👍

if (data != null && data.series.length > 1) {
// const nameArray = data.value?.dataSetProcess.series.map((data: any) => data.name);
// use this to set all checked
await setCheckedLegendItems(legendItems.map((item) => item.label));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my IDE says the await is not needed, and I think it's correct!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also noticed it but without that await, somehow it did not work correctly so I left it as is

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what doesn't work, when you remove await?

if (data != null && data.series.length > 1) {
// const nameArray = data.value?.dataSetProcess.series.map((data: any) => data.name);
// use this to set all checked
await setCheckedLegendItems(legendItems.map((item) => item.label));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here

filters != null && overlayVariableFilter
? filters
.map((filter) => {
if (filter.type == 'stringSet' && data.name != null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here :-)

Copy link
Member

@dmfalke dmfalke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks pretty good. i'm curious about the await issue (see inline comment).

if (data != null && data.series.length > 1) {
// const nameArray = data.value?.dataSetProcess.series.map((data: any) => data.name);
// use this to set all checked
await setCheckedLegendItems(legendItems.map((item) => item.label));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what doesn't work, when you remove await?

@moontrip
Copy link
Contributor Author

moontrip commented Nov 3, 2021

@dmfalke so, I noticed an thumbnail capture issue at Scatter plot case with the custom legend: from my limited test it does not occur at other viz though. It happens when using plot option other than 'raw', i.e., smoothed mean or best fit, and thumbnail is not made correctly in those cases - showing previous plot. I think that it is because the custom legend's status is asynchronously happened as compared to thumbnail capture you made. So, I made async/await at the same useEffect so that custom legend is firstly updated and then thumbnail capture is done after that.

@moontrip
Copy link
Contributor Author

moontrip commented Nov 5, 2021

I think now both histogram and barplot vizs are working correctly with the custom legend. Surprisingly the checkbox behavior (turning on/off plot trace) worked just fine for the faceted plot without much changes (at least for histogram and barplot vizs) 🎉 Since they both used the same marker style (solid square without border), the current custom legend (PlotLegend component) are working for them even if currently it only has that square marker case. Will need to check marker style for other plot vizs and add markers accordingly.

@dmfalke
Copy link
Member

dmfalke commented Nov 5, 2021

Thanks for the update @moontrip. One suggestion is to store the checkedLegendItems as a part of the visualization config object. If you do this, be sure to make it optional.

@moontrip
Copy link
Contributor Author

moontrip commented Nov 5, 2021

Thanks for the update @moontrip. One suggestion is to store the checkedLegendItems as a part of the visualization config object. If you do this, be sure to make it optional.

@dmfalke Actually I thought of the case as well because, if I remember correctly, there was a talk to keep the status of legend selection. Perhaps I will think of that case next week as I am off from VEuPathDB today.

@bobular
Copy link
Member

bobular commented Nov 8, 2021

@dmfalke so, I noticed an thumbnail capture issue at Scatter plot case with the custom legend: from my limited test it does not occur at other viz though. It happens when using plot option other than 'raw', i.e., smoothed mean or best fit, and thumbnail is not made correctly in those cases - showing previous plot. I think that it is because the custom legend's status is asynchronously happened as compared to thumbnail capture you made. So, I made async/await at the same useEffect so that custom legend is firstly updated and then thumbnail capture is done after that.

The await does somehow magically make the thumbnail sort-of work. However, it doesn't work as desired: the thumbnail isn't made from the last-viewed scatterplot, which may have series deselected via the new legend.

I think the problem can be solved by moving the "set all checked" useEffect to the main ScatterplotViz function, where I think it belongs more naturally (this is where legendItems is made).

Then the toImage useEffect just needs checkedLegendItems as a dependency.

Here's the diff from what I was playing around with

diff --git a/src/lib/core/components/visualizations/implementations/ScatterplotVisualization.tsx b/src/lib/core/components/visualizations/implementations/ScatterplotVisualization.tsx
index fa86efd4..30577b88 100755
--- a/src/lib/core/components/visualizations/implementations/ScatterplotVisualization.tsx
+++ b/src/lib/core/components/visualizations/implementations/ScatterplotVisualization.tsx
@@ -365,6 +365,7 @@ function ScatterplotViz(props: VisualizationProps) {
       : [];
   }, [data, filters]);

+
   //DKDK
   console.log(
     'data.value?.dataSetProcess.series =',
@@ -377,6 +378,11 @@ function ScatterplotViz(props: VisualizationProps) {

   console.log('checkedLegendItems =', checkedLegendItems);

+  useEffect(() => {
+    // use this to set all checked
+    setCheckedLegendItems(legendItems.map((item) => item.label));
+  }, [legendItems]);
+
   return (
     <div style={{ display: 'flex', flexDirection: 'column' }}>
       <div style={{ display: 'flex', alignItems: 'center', zIndex: 1 }}>
@@ -499,7 +505,6 @@ function ScatterplotViz(props: VisualizationProps) {
           //DKDK pass checked state of legend checkbox to PlotlyPlot
           checkedLegendItems={checkedLegendItems}
           setCheckedLegendItems={setCheckedLegendItems}
-          legendItems={legendItems}
         />

         {/* DKDK custom legend */}
@@ -573,8 +578,8 @@ type ScatterplotWithControlsProps = XYPlotProps & {
   // add disabledList
   disabledList: string[];
   //DKDK legend checkbox
+  checkedLegendItems: string[];
   setCheckedLegendItems: (checkedItems: string[]) => void;
-  legendItems: LegendItemsProps[];
 };

 function ScatterplotWithControls({
@@ -589,8 +594,8 @@ function ScatterplotWithControls({
   disabledList,
   updateThumbnail,
   //DKDK legend checkbox
+  checkedLegendItems,
   setCheckedLegendItems,
-  legendItems,
   ...scatterplotProps
 }: ScatterplotWithControlsProps) {
   // TODO Use UIState
@@ -613,20 +618,17 @@ function ScatterplotWithControls({
   //DKDK add async/await for correct thumbnail capture
   useEffect(() => {
     (async () => {
-      if (data != null) {
-        // use this to set all checked
-        await setCheckedLegendItems(legendItems.map((item) => item.label));
-      }
       await plotRef.current
         ?.toImage({ format: 'svg', ...plotDimensions })
         .then(updateThumbnailRef.current);
     })();
-  }, [data, legendItems]);
+  }, [data, checkedLegendItems]);

   return (
     <div style={{ display: 'flex', flexDirection: 'column' }}>
       <XYPlot
         {...scatterplotProps}
+        checkedLegendItems={checkedLegendItems}
         ref={plotRef}
         data={data}
         // add controls

@bobular
Copy link
Member

bobular commented Nov 8, 2021

However, a bigger structural change is that we need the checked items (if there are any) to be saved in vizConfig. So I guess we should use vizConfig.checkedLegendItems (and updateVizConfig etc) for the storage instead of useState()

(which I see Dave already mentioned)

@moontrip
Copy link
Contributor Author

moontrip commented Nov 8, 2021

@dmfalke so, I noticed an thumbnail capture issue at Scatter plot case with the custom legend: from my limited test it does not occur at other viz though. It happens when using plot option other than 'raw', i.e., smoothed mean or best fit, and thumbnail is not made correctly in those cases - showing previous plot. I think that it is because the custom legend's status is asynchronously happened as compared to thumbnail capture you made. So, I made async/await at the same useEffect so that custom legend is firstly updated and then thumbnail capture is done after that.

The await does somehow magically make the thumbnail sort-of work. However, it doesn't work as desired: the thumbnail isn't made from the last-viewed scatterplot, which may have series deselected via the new legend.

I think the problem can be solved by moving the "set all checked" useEffect to the main ScatterplotViz function, where I think it belongs more naturally (this is where legendItems is made).

Then the toImage useEffect just needs checkedLegendItems as a dependency.

Here's the diff from what I was playing around with

diff --git a/src/lib/core/components/visualizations/implementations/ScatterplotVisualization.tsx b/src/lib/core/components/visualizations/implementations/ScatterplotVisualization.tsx
index fa86efd4..30577b88 100755
--- a/src/lib/core/components/visualizations/implementations/ScatterplotVisualization.tsx
+++ b/src/lib/core/components/visualizations/implementations/ScatterplotVisualization.tsx
@@ -365,6 +365,7 @@ function ScatterplotViz(props: VisualizationProps) {
       : [];
   }, [data, filters]);

+
   //DKDK
   console.log(
     'data.value?.dataSetProcess.series =',
@@ -377,6 +378,11 @@ function ScatterplotViz(props: VisualizationProps) {

   console.log('checkedLegendItems =', checkedLegendItems);

+  useEffect(() => {
+    // use this to set all checked
+    setCheckedLegendItems(legendItems.map((item) => item.label));
+  }, [legendItems]);
+
   return (
     <div style={{ display: 'flex', flexDirection: 'column' }}>
       <div style={{ display: 'flex', alignItems: 'center', zIndex: 1 }}>
@@ -499,7 +505,6 @@ function ScatterplotViz(props: VisualizationProps) {
           //DKDK pass checked state of legend checkbox to PlotlyPlot
           checkedLegendItems={checkedLegendItems}
           setCheckedLegendItems={setCheckedLegendItems}
-          legendItems={legendItems}
         />

         {/* DKDK custom legend */}
@@ -573,8 +578,8 @@ type ScatterplotWithControlsProps = XYPlotProps & {
   // add disabledList
   disabledList: string[];
   //DKDK legend checkbox
+  checkedLegendItems: string[];
   setCheckedLegendItems: (checkedItems: string[]) => void;
-  legendItems: LegendItemsProps[];
 };

 function ScatterplotWithControls({
@@ -589,8 +594,8 @@ function ScatterplotWithControls({
   disabledList,
   updateThumbnail,
   //DKDK legend checkbox
+  checkedLegendItems,
   setCheckedLegendItems,
-  legendItems,
   ...scatterplotProps
 }: ScatterplotWithControlsProps) {
   // TODO Use UIState
@@ -613,20 +618,17 @@ function ScatterplotWithControls({
   //DKDK add async/await for correct thumbnail capture
   useEffect(() => {
     (async () => {
-      if (data != null) {
-        // use this to set all checked
-        await setCheckedLegendItems(legendItems.map((item) => item.label));
-      }
       await plotRef.current
         ?.toImage({ format: 'svg', ...plotDimensions })
         .then(updateThumbnailRef.current);
     })();
-  }, [data, legendItems]);
+  }, [data, checkedLegendItems]);

   return (
     <div style={{ display: 'flex', flexDirection: 'column' }}>
       <XYPlot
         {...scatterplotProps}
+        checkedLegendItems={checkedLegendItems}
         ref={plotRef}
         data={data}
         // add controls

@bobular thank you for your suggestion. Actually the approach I have used async/await for both legendItems and thumbnail at useEffect was designed for scatter plot and it worked just fine. I don't know why you did not have the screenshot correctly. Anyway, the approach you suggested was my initial approach to use useEffect for legendItems (set function) at ScatterplotViz but it didn't work properly for me, especially with plot options with smoothed mean and best fit line: that was the reason why I switched to async/await. However, what I missed is the dependency at useEffect of thumbnail so it may make difference. I will test it too!

And yes I started to set vizConfig.checkedLegendItems for histogram viz and then barplot viz. If it would be done, I think that we can merge this custom legend stuff so that I can add this custom legend to other plot vizs that are being grounded by you 😄

@moontrip
Copy link
Contributor Author

moontrip commented Nov 8, 2021

@bobular after testing, the missing dependency at thumbnail's useEffect() at my implementation was the culprit to prevent from correct capture 👍 I should not have thought of more complex solutions rather than the simple one 😄

@dmfalke
Copy link
Member

dmfalke commented Nov 8, 2021

Great! Good catch @bobular, and nice work @moontrip!

@moontrip, there is a long-standing issue to retain the state of what legend items are visible.

@moontrip
Copy link
Contributor Author

moontrip commented Nov 8, 2021

@moontrip, there is a long-standing issue to retain the state of what legend items are visible.

@dmfalke I will think of that specific case later: need to make custom legend work for all vizs, including faceted plot, at first :)

@moontrip
Copy link
Contributor Author

moontrip commented Nov 8, 2021

@bobular and @dmfalke histogram and barplot vizs with faceted plot are now changed to use vizConfig.

@bobular Actually I also changed scatterplot viz in this branch (i.e., without faceted plot) to use vizConfig, but I am hesitating to commit because you already started working previous scatter plot to have faceted plot: in that case I can change yours later. If you didn't start the scatter plot yet, then I will also commit it here. Just let me know 👍

@bobular
Copy link
Member

bobular commented Nov 9, 2021

Actually I also changed scatterplot viz in this branch (i.e., without faceted plot) to use vizConfig, but I am hesitating to commit because you already started working previous scatter plot to have faceted plot: in that case I can change yours later. If you didn't start the scatter plot yet, then I will also commit it here. Just let me know

Hi @moontrip - I haven't started on scatterplot faceting - so please commit!

@moontrip
Copy link
Contributor Author

moontrip commented Nov 9, 2021

Actually I also changed scatterplot viz in this branch (i.e., without faceted plot) to use vizConfig, but I am hesitating to commit because you already started working previous scatter plot to have faceted plot: in that case I can change yours later. If you didn't start the scatter plot yet, then I will also commit it here. Just let me know

Hi @moontrip - I haven't started on scatterplot faceting - so please commit!

Thank you for your info, @bobular 👍 I just committed scatter plot viz as well.

@moontrip moontrip marked this pull request as ready for review November 9, 2021 15:07
@moontrip
Copy link
Contributor Author

moontrip commented Nov 9, 2021

now make this as a normal PR so that it can be used for other Vizs with faceted plot, which I will handle the addition of this custom legend to those vizs - done for histogram and barplot vizs, and scatter plot viz in part.

Copy link
Member

@dmfalke dmfalke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this all looks pretty good. I added an inline comment about the state of the checked items not being retained when I reopen an existing visualization. Once that has been addressed, I think this will be good to go.

@moontrip moontrip mentioned this pull request Nov 9, 2021
@bobular
Copy link
Member

bobular commented Nov 9, 2021

Reviewing as fast as I can now.

From testing, I see there's a general logic/wiring issue that I'll make another ticket for and fix at a later date. Here's the ticket: #676

We shouldn't show the gray No data option for overlay variable sex in the example below. The problem is that showMissing or showMissingOverlay booleans don't take into account the completeCases data, which they should.
image

@moontrip
Copy link
Contributor Author

moontrip commented Nov 9, 2021

Thanks @bobular ! 👍 I also saw your report regarding the bug(?). Initially I thought I made a mistake to incorrectly show the legend item, No data, but after reading the description carefully, I realized that it was related to the switch behavior. Glad that my custom legend helps to identify the issue (and also relieved myself a bit that the legend logic worked fine :) )

Copy link
Member

@bobular bobular left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - note that I've done more testing than code review.

Don't forget to add tickets for the remaining tasks, which I think are

  • legend markers for scatterplot
  • saving/restoring checked legend items correctly

@@ -8,7 +8,7 @@ export function useFindOutputEntity(
// need to add string at Record's Type due to valueSpecConfig
dataElementVariables: Record<
string,
VariableDescriptor | string | boolean | undefined
VariableDescriptor | string | boolean | string[] | undefined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see why... because we pass the whole of vizConfig to this function. Looks like some technical debt here!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes indeed

}, [data]);

// use this to set all checked
useEffect(() => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought I had a simple solution to the problem of this overwriting the saved configuration, but it wasn't so simple.

So are we going to solve this separately? Don't forget to make a ticket if so.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will make a ticket to handle saving/restoring checked legend items. Actually legend markers are for both boxplot and scatter plot as boxplot utilized different marker style unlike histogram etc.: square but light color and border for boxplot ones.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As pointed out by Dave, we already have saving/restoring ticket made long time ago, #172, so I don't need to make it: . just assigned it to me so that I won't forget 😃 Considering other tasks I have to do for b55, I will defer it, though.

@moontrip
Copy link
Contributor Author

moontrip commented Nov 9, 2021

thank you for your review and comments @bobular ! 👍 Will work for merging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants