Show dynamic job status in workflow page

[tcompa]

Some complexities:

• Multiple jobs for the same workflow but different input datasets.
• Granular per-task status
• Resubmission or running from task A to task B (with which I/O dataset?)

[tcompa]

A new endpoint was introduced in fractal-server 1.3.5 (and refined later, e.g. in 1.3.12): GET /api/v1/project/{project_id}/dataset/{dataset_id}/status. The response looks like:

{
  "status": {
    1: "submitted",
    2: "failed",
    3: "done"
  }
}

where 1, 2 and 3 are WorkflowTask IDs.

Here is a first proposal of how we can expose this endpoint for testing (*):

1. For each job (in the jobs-page table), expose a button "Check status"
2. Once clicked, call the new status endpoint, with dataset_id=job.output_dataset_id
3. The response includes all WorkflowTask's which have been associated to that dataset, but those can also be part of other jobs or other workflows. We then need to extract the subset of WorkflowTasks that belong to the current job. This is done by looking at the IDs of the job.workflow_dump.task_list (to be precise, we should only look at the sub-array defined by the positional indices job.first_task_index and job.last_task_index.
4. We now have a list of the WorkflowTask's that are part of the current job, and of their statuses. As a first implementation, we can show this information as a simple three-columns table: WorkflowTask.id, WorkflowTask.task.name, status.

(*) The longer-term perspective is to integrate this information directly in the Workflow page, but here I'm only proposing a first way to play with this new feature and verify that its response matches with expectations. Note that the longer-term perspective above also requires establishing a link between a Workflow and a Dataset.

[jluethi]

Great, this sounds like a good way to start testing the status feature!

[jluethi]

I've sketched some scenarios for monitoring and how this can be displayed on the workflow page.

The idea is that each task in the workflow task list shows the processing status as real-time as possible (how real-time is the database for the output ds history? Does it get updated during the run or only at the end of it?)

Full sketches here: https://www.figma.com/file/n5CzmEcOadzSlMkUkA9Lut/Fractal-Web-Workflow-Monitoring?type=design&node-id=0-1&mode=design&t=zH5flcZpWwkDhMHy-0

It mostly comes down to already specifying the dataset(s) on the workflow page, instead of only during submission. That way, the history can then be shown on the workflow page.

One of the conclusions: This will get much simpler if we move to a "single dataset approach", e.g. there is 1 dataset that the workflow works on, not input & output dataset. See also some analysis here on how the input datasets in only used very lightly: fractal-analytics-platform/fractal-server#785

---

What is still missing from those sketches:
The submission dialogue would also change (dropping the dataset selection, but maybe also further for subset submissions that are in discussion, see #296). Monitoring for subset submissions has also not been fully defined yet.

[tcompa]

This is currently in-place. Let's open specific issues if needed.