Skip to content
This repository was archived by the owner on May 14, 2025. It is now read-only.

Further performance improvements for job executions (for oracle, other dbs might vary) #5695

Closed
klopfdreh opened this issue Feb 22, 2024 · 0 comments · Fixed by #5700
Closed
Assignees
Labels
area/db Belongs to database
Milestone

Comments

@klopfdreh
Copy link
Contributor

Description:
For Oracle we found out that the performance for job executions sqls can be further improved as they are not fast enought to deliver a proper response for the UI in time. (30 seconds then our gateway cuts off) Note that we already have 1743745 job executions.

Release versions:
SCDF 2.11.2

Custom apps:
N/A

Steps to reproduce:
Just go into the job executions page at the SCDF UI and navigate through the pages.

Screenshots:
image

Additional context:
The issue can be solved by ceating the following indexes.

create index SCDF.SYS_C0098765 on SCDF.BATCH_STEP_EXECUTION("JOB_EXECUTION_ID");
create index SCDF.SYS_C0087654 on SCDF.BOOT3_BATCH_STEP_EXECUTION("JOB_EXECUTION_ID");
create index SCDF.SYS_C0076543 on SCDF.BOOT3_TASK_TASK_BATCH("JOB_EXECUTION_ID");
create index SCDF.SYS_C0065432 on SCDF.BATCH_JOB_EXECUTION("START_TIME");

Also the oracle database suggests a sql_profile for the first page which can not be created via SQL statements.

After those changes the paged job executions can be delivered in ~900ms (for the initial page (1) and all following (2-N))

@github-actions github-actions bot added the status/need-triage Team needs to triage and take a first look label Feb 22, 2024
corneil pushed a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
@corneil corneil added area/db Belongs to database and removed status/need-triage Team needs to triage and take a first look labels Feb 22, 2024
@corneil corneil self-assigned this Feb 22, 2024
corneil pushed a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
corneil pushed a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
… queries when customer has large number of job executions in history. They experienced queries exceeding 180s.

Fixes spring-attic#5695
corneil pushed a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
… queries when customer has large number of job executions in history. They experienced queries exceeding 180s.

Fixes spring-attic#5695
corneil pushed a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
… queries when customer has large number of job executions in history. They experienced queries exceeding 180s.

Fixes spring-attic#5695
corneil pushed a commit that referenced this issue Feb 23, 2024
… queries when customer has large number of job executions in history. They experienced queries exceeding 180s.

Fixes #5695
@corneil corneil added this to the 2.11.3 milestone May 17, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/db Belongs to database
Development

Successfully merging a pull request may close this issue.

2 participants