Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add job timings vs cycle point visualisation #1510

Merged
merged 46 commits into from
Apr 30, 2024

Conversation

JAllen42
Copy link
Contributor

@JAllen42 JAllen42 commented Oct 10, 2023

Partially addresses #1170
Linked to cylc/cylc-uiserver#530

This is a prototype for a view for visualising job timing information as a function of cycle point. It uses the UI server branch from cylc/cylc-uiserver#530 to get historic job timings.

image

Tasks can be added to the graph one by one (hopefully to prevent overcrowding for complex workflows), with a mini chart at the bottom to control the x-axis zooming (though how this works should be made more obvious). You can hover to see timing and platform information.

At the moment I have added in to the analysis view, but this could be split into its own view if preferred - the shape of the data is different, which means there are different options for how much is kept in memory when switching between visualisations, and when to re-query the database. It might depend what other ways of visualising similar data might be added?

Currently I've called this a time series visualisations, other suggestions are welcome along with any other comments!

Outstanding work includes:

  • Improving the filtering - the task and platform filters at the top are not fully integrated yet
  • Making sure cycle points with multiple jobs are handled sensibly
  • Decide whether to include any information about failed jobs
  • Improving the testing

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Tests are included (or explain why tests are not needed).
  • Changelog entry included if this is a change that can affect users (run towncrier create 1510.feat.md)
  • Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
  • If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

@oliver-sanders
Copy link
Member

LGTM. I like the mini chart.

At the moment I think it's reasonable to have this in the analysis view, but we can always spin it off if desired.

It might be nice to display the mean times in the chart.

@JAllen42 JAllen42 force-pushed the analysis_time_series branch from bce34d2 to 8bd4c5f Compare January 31, 2024 18:18
@oliver-sanders
Copy link
Member

How's this going? Looked close when I last dropped by.

@MetRonnie MetRonnie added this to the 2.5.0 milestone Mar 27, 2024
Copy link
Member

@MetRonnie MetRonnie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be an issue with the mini chart selection where I get this console error and the rightmost point doesn't always get included in the main plot
image

src/components/cylc/analysis/TimeSeries.vue Show resolved Hide resolved
src/components/cylc/analysis/TimeSeries.vue Outdated Show resolved Hide resolved
src/components/cylc/analysis/TimeSeries.vue Outdated Show resolved Hide resolved
@MetRonnie MetRonnie mentioned this pull request Apr 9, 2024
4 tasks
@oliver-sanders
Copy link
Member

@JAllen42, @ChrisPaulBennett - looks like there's one error and three minor suggestions left to sort out, but close.

We are starting to get close to 8.3.0 release so the time window for getting this merged is now drawing in. Would be great to get this in with the 8.3.0 release where we can announce it along with other new features.

@hjoliver
Copy link
Member

I'll second that, it'd be awesome to get this into 8.3.0.

@ChrisPaulBennett
Copy link
Contributor

There seems to be an issue with the mini chart selection where I get this console error and the rightmost point doesn't always get included in the main plot image

Can you give me a bit more info on how to recreate? I'm not seeing this issue.

@MetRonnie
Copy link
Member

Looks like a bug in vue-apexcharts actually: apexcharts/vue-apexcharts#318

@oliver-sanders
Copy link
Member

Ok, suggest overlooking this issue for the purpose of review, we can rebuild when a fix is released.

Copy link
Member

@MetRonnie MetRonnie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately the refresh button isn't doing anything

@JAllen42
Copy link
Contributor Author

Unfortunately the refresh button isn't doing anything

I've had a look at this with Chris, and it looks like it is (mostly) working as intended to us. When you have some tasks plotted from a running workflow and you press the refresh button it will add to the plot the times from any cycle points that have been run since the query was last run.

We have spotted one issue though. The options for the drop down list of tasks are passed to the component as a prop from the analysis view. This means if completely new tasks have been run, these will not be added to the list when the refresh button is pressed. They will only be shown when the analysis view is closed and reopened, or the refresh button from either the table or box and whiskers pressed. Would this explain what you saw?

Either way, it looks like a test could be added for this

@MetRonnie
Copy link
Member

We have spotted one issue though. The options for the drop down list of tasks are passed to the component as a prop from the analysis view. This means if completely new tasks have been run, these will not be added to the list when the refresh button is pressed. They will only be shown when the analysis view is closed and reopened, or the refresh button from either the table or box and whiskers pressed. Would this explain what you saw?

Yes that's the issue I am getting

Copy link
Member

@MetRonnie MetRonnie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could do with tidying up common code to TimeSeries.vue and Analysis.vue later on

@oliver-sanders oliver-sanders merged commit a403f52 into cylc:master Apr 30, 2024
5 of 6 checks passed
@oliver-sanders
Copy link
Member

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data workflows team Work pertaining to the analysis/gantt/etc views
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants