Analysis and visualization (2D) #4

mguidon · 2018-01-23T10:49:34Z

User story

Some results will need visualization with tables and/or 2d plots. As a user I want to be able to visualize such data

Definition of Done

There is dedicated panel/view/dashboard in the frontend
There is a defined data structure for such results
Tables shall be sortable and editable
Selecting subsets in the table should dynamically update the plots
In addition to a Jupyter like script representation, a standalone 2D xy-viewer is needed

Use case

Clancy case is used as a first goal.

see comments below (at the end) for the updated definition of done.

mguidon · 2018-02-02T10:12:38Z

Embed this solution in s4l table viewers
Have a look at CStorage and check if json can be used as backend (StorageUtils.h) check conversion routines

mguidon · 2018-03-07T10:32:31Z

Add an new algorithm to sim4life that displays 2d data (series data, csv, excel) in a chromium view

pcrespov · 2018-04-11T09:03:19Z

Canceron Milestone

OUR main view is a workbench. A workbench has nodes connected in a graph. These nodes can be associated with services and have the ability to interact with UI resources.
workbench example should include:
- producer node -> computation node -> analysis node
- Nodes have associated services that will be: modeling service, a computational service and a Jupiter service resp.
- Audit mode might apply at node or project level to restrict access to data outside the visible connections in the pipeline.
interactive resources on a grid panel: 3d views, jupyter page/cell?, ...
demo node interaction with resources

pcrespov · 2018-04-12T08:10:14Z

Pipelines and analysis

sanderegg · 2018-04-13T07:41:31Z

Scenario 1:

User logs in
Workbench opens
User creates modeler node (producer node)
a. Modeler node connects to Modeler service (IP:port)
b. Workbench provides a widget that points to modeler service (IP:port)
c. User loads a model/modify the model until happy in modeler widget
User creates computation node
a. User creates computation pipeline…
b. …
User links computation node to modeler node
a. Workbench tells modeler node to pass its data (GLTF scene) to the computation node (IP:port, common RestAPI for all nodes?) (either directly or through DAT-CORE, other…)
User starts computation node
a. Workbench tells modeler node to transmit its data to computation node or to DAT-CORE (RestAPI?)
b. Workbench tells computation node where to find the input data
c. Workbench starts computation node
d. Computation node finishes computing and saves its data to DAT-CORE
User creates Jupyter notebook node (analysis node)
a. Jupyter notebook node connects to Jupyter notebook service (IP:port)
User links Jupyter notebook node to computation node output
a. Workbench tells Jupyter notebook where to get the data
b. Workbench provides a widget that points to jupyter notebook (IP:port)
c. User interacts with notebook in notebook widget
User goes to dashboard view
a. User drags available 3d views?
b. User drags available Jupyter cells?

sanderegg · 2018-04-13T08:10:56Z

drniiken · 2018-05-02T17:29:50Z

demo
progress
2D view
initiate 2D plotter
in next sprint
GUI work much more possibilities, more relaxed

pcrespov · 2018-05-04T13:35:25Z

Caprica Milestone - dev notes

Prototype in Jupyter 2D plots from data Clancy model. Start simple, then create widgets.

sanderegg · 2018-05-30T06:27:09Z

Creation of a python package to access the inputs/outputs to the Jupyter node from the Jupyter notebook
- Access to inputs on the fly
- Automatically import/export data from S3 storage
- Automatically get/set configuration from DB

sanderegg · 2018-06-06T12:49:07Z

autoplay notebook

No more mockup one the comp backend

sanderegg · 2018-07-16T12:51:04Z

Have a look at this as well:
https://rawgraphs.io/

drniiken · 2019-02-14T14:14:23Z

Nik & Esra tell the Team:

contine with this approach
go with plotly
combination of it
play around with the current implementation ...

drniiken · 2019-02-19T16:50:04Z

The proposed solution looks very attractive.
However, the following additional functionality would be important and we need to know, if it can be provided with the current solution

generation of multiple plots within one service
generation of combined plots (possibly, also with input derived from multiple inputs
simplification of insertion of column titles (what went wrong in the NIH demo)
can we make it easier to create standard 2D plots (maybe something like a template (while keeping the power of the current solution for people that want more control and flexibility)

We do feel that this solution is not mutually exclusive with a plotly based solution. Both can be attractive and can coexist. If we go for a plotly based solution in addition, we might want to make that one more flexible with also pipelining kind of functionality (as in the 3D viewing with paraview-web).

- Creates a container-node-widget able to display the image outputs of its inner services in a grid-dashboard layout. - When double-clicking a cell, the edition mode of its graph will be opened. - Improves the raw-graphs service: - More input ports (5). - One output port. Every time the graph in the frontend is modified it is saved and also pushed to S3 when sync function (input-retriever) is called. - closes #630 - Added a diagram that shows the frontend UI workflow - child of #4

esraneufeld · 2019-03-14T12:20:50Z

i would like to close this case soon, as the current functionality is mostly sufficient for now:
before closing, please create two variant services (only if these two variants can be added cheaply):

a simple 2d plot service, that does not provide the users with all these options for plot type, but rather just creates a standardized 2d plot, once the axes have been selected
a table viewer service that only contains all the steps up to (and including) the specification of the table column titles, but no option to continue with plot creation

i also created a lower priority follow-up case ( #639 ) about addressing the issue of creating combined 2d plots from different sources.

* moved service-sidecar contents * added task to run unittests * added unittests bash commands * added command to run unittests locally * added entry to build dynamic-sidecar * coverage and deploy now await for dynamic-sidecar * renamed service-sidecar to dynamic-sidecar * renaming missing folder * renaming missing fodler part 2 * updating makefile * updating tox * removing tox * added ci codestyle check before tests run * removed unsued targets * fixed error message * added missing test * added codeoowner entry * moved mocked fucntions to the same module * renamed module * updating log warning messages for mocked calls * added some help * clrifying command usage * using verbatim arguments * fixed typo * renived tests * renaming ServiceSidecarSettings to DynamicSidecarSettings * updated name in changelog * fixed ptsv entrypoint * fixed healt endpoint to fail when status is not healthy * fixed route and test * mappign docs on the same route as other services * added api prefix for interested routes * codestyle * container api routes refactored * renamed storage to shared_store * refacted shared_store to use Pydantic models * missed a rename * added makefile entry to generate openapi spec * adding correct mypy file * mypy suggestions * added openapi.json for the service * codestyle also updates openapi.json * added common scripts to enforce codestyles in development and CI * imported statuses from fastapi/starlette definitions * updated docstring * fixed imports, now importing from starlette * moved mark.asyncio * moved act into bash script * added act entry to common.Makefile * added entry for development of dunamic-sidecar * added test to check the spec has was updated * added missing fixture * updated readme * removed unused APIs and renamed existing * added docker-compose config validation * moved settings and shared state to dependencies * minor improvments * updated requirements * added missing service to SERVICES_LIST * Some suggestions (#3) from sanderegg * naming * consistency * shellcheck * proposed refactoring on dynamic sidecar (#4) * removes responses and raise exceptions for errors * Minor typo * Should include standard entrypoint * refactor health and added dependencies * minor * fixes * fixed codestyle * fixed typos * fixed compose API entrypoints and tests * refactored api structure and endpoints * extended utils * adapted new tests * updated API spec * moving routes under the same module * moved tests and updated openapi.json * updated script entryoint name * renamed endpoint * refactor * refactor error raising and staus code for resource not found * regenerated openapi.json spec * codestyle :\ * merged /containers and /containers:inspect * wrong default value * test clenup * updated openapi.json * clarify comment * removed unused API * removed command timeout when posting to createing services via docker-compose * removing unecessary timeouts * forgot to implement it * after the request is accepted the list of container names is returned * removed flag which is no longer used * fixed tests * moved ApplicationHealth and codestyle fixes * added test to check it can run twice the same compos spec with different project-name * fixed codestyle * setting default and documentation to argument * updated oepnapi.json * fixed test to properly work based on workspace * refactoring application models structure * rending applicaiton strucutre similar to other fastapi services * replacing with response.json() * applied codestyle * comitting new updated dependencies * refactoring error codes * clened up exception throwing * restored error emssage Co-authored-by: Andrei Neagu <[email protected]> Co-authored-by: Sylvain <[email protected]> Co-authored-by: Pedro Crespo-Valero <[email protected]>

mguidon mentioned this issue Feb 1, 2018

Cardiac Tissue model (myocytes) from Colleen Clancy (C++ code) #3

Closed

6 tasks

drniiken added Priority: normal labels Feb 1, 2018

mguidon changed the title ~~Analysis and visualization (Python)~~ Analysis and visualization (2D) Feb 28, 2018

odeimaiz assigned pcrespov and sanderegg Apr 11, 2018

odeimaiz added this to the Canceron milestone Apr 11, 2018

drniiken added Priority: high and removed Priority: normal labels May 2, 2018

mguidon modified the milestones: Canceron, Caprica May 4, 2018

sanderegg modified the milestones: Caprica, Gemenon Jun 10, 2018

sanderegg mentioned this issue Jun 20, 2018

Feature/Analysis 2d graphs and nodeports package #114

Merged

20 tasks

odeimaiz referenced this issue in odeimaiz/osparc-simcore Jun 22, 2018

Merge pull request #4 from odeimaiz/mguidon-registry_api

1776c60

No more mockup one the comp backend

sanderegg added the in progress label Jun 29, 2018

sanderegg modified the milestones: Gemenon, Leonis Jun 29, 2018

drniiken mentioned this issue Aug 13, 2018

Demo Story I (EM-Neuro, GUI, Workflow) #184

Closed

5 tasks

pcrespov modified the milestones: Leonis, Libra Aug 15, 2018

pcrespov added backlog and removed in progress labels Aug 24, 2018

mguidon mentioned this issue Aug 28, 2018

Cardiac model from Kember (Java code) #5

Closed

6 tasks

pcrespov removed this from the Libra milestone Aug 28, 2018

drniiken mentioned this issue Jan 9, 2019

Analysis Service (M9, MS5.3,D5.3) #506

Open

3 tasks

mguidon added PO issue Created by Product owners [PLEASE use osparc-issue repo] and removed Area: frontend labels Jan 10, 2019

odeimaiz mentioned this issue Jan 30, 2019

Analysis and visualization (2D) [RawGraphs] #535

Merged

pcrespov added in progress and removed backlog labels Feb 8, 2019

drniiken added the POs ToWorkOn label Feb 14, 2019

drniiken added ISAN and removed POs ToWorkOn labels Feb 19, 2019

odeimaiz mentioned this issue Feb 26, 2019

Grid-dashboard widget #599

Closed

3 tasks

This was referenced Mar 5, 2019

Extend Raw-Graphs service #620

Merged

Multi plot Dashboard #621

Merged

This was referenced Mar 15, 2019

Adapt dy-viewers to output plot data #645

Closed

Table View service #657

Merged

drniiken closed this as completed May 15, 2019

matusdrobuliak66 mentioned this issue Feb 19, 2024

Spurious premature shutdown of services #5100

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Analysis and visualization (2D) #4

Analysis and visualization (2D) #4

mguidon commented Jan 23, 2018 •

edited by drniiken

Loading

mguidon commented Feb 2, 2018 •

edited

Loading

mguidon commented Mar 7, 2018

pcrespov commented Apr 11, 2018 •

edited by sanderegg

Loading

pcrespov commented Apr 12, 2018

sanderegg commented Apr 13, 2018 •

edited

Loading

sanderegg commented Apr 13, 2018

drniiken commented May 2, 2018 •

edited

Loading

pcrespov commented May 4, 2018 •

edited by sanderegg

Loading

sanderegg commented May 30, 2018 •

edited

Loading

sanderegg commented Jun 6, 2018 •

edited

Loading

sanderegg commented Jul 16, 2018

drniiken commented Feb 14, 2019

drniiken commented Feb 19, 2019

esraneufeld commented Mar 14, 2019 •

edited by drniiken

Loading

Analysis and visualization (2D) #4

Analysis and visualization (2D) #4

Comments

mguidon commented Jan 23, 2018 • edited by drniiken Loading

mguidon commented Feb 2, 2018 • edited Loading

mguidon commented Mar 7, 2018

pcrespov commented Apr 11, 2018 • edited by sanderegg Loading

Canceron Milestone

pcrespov commented Apr 12, 2018

sanderegg commented Apr 13, 2018 • edited Loading

sanderegg commented Apr 13, 2018

drniiken commented May 2, 2018 • edited Loading

pcrespov commented May 4, 2018 • edited by sanderegg Loading

Caprica Milestone - dev notes

sanderegg commented May 30, 2018 • edited Loading

sanderegg commented Jun 6, 2018 • edited Loading

sanderegg commented Jul 16, 2018

drniiken commented Feb 14, 2019

drniiken commented Feb 19, 2019

esraneufeld commented Mar 14, 2019 • edited by drniiken Loading

mguidon commented Jan 23, 2018 •

edited by drniiken

Loading

mguidon commented Feb 2, 2018 •

edited

Loading

pcrespov commented Apr 11, 2018 •

edited by sanderegg

Loading

sanderegg commented Apr 13, 2018 •

edited

Loading

drniiken commented May 2, 2018 •

edited

Loading

pcrespov commented May 4, 2018 •

edited by sanderegg

Loading

sanderegg commented May 30, 2018 •

edited

Loading

sanderegg commented Jun 6, 2018 •

edited

Loading

esraneufeld commented Mar 14, 2019 •

edited by drniiken

Loading