Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: project-codeflare/multi-cluster-app-dispatcher
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v1.34.0
Choose a base ref
...
head repository: project-codeflare/multi-cluster-app-dispatcher
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v1.34.1
Choose a head ref

Commits on Aug 30, 2023

  1. noticed that this exact check is done above on L264

    Signed-off-by: Kevin <kpostlet@redhat.com>
    KPostOffice authored and openshift-merge-robot committed Aug 30, 2023
    Copy the full SHA
    116b979 View commit details
  2. Automatically label issues as needing triage and add them to project …

    …board
    
    Signed-off-by: Anish Asthana <anishasthana1@gmail.com>
    anishasthana committed Aug 30, 2023
    Copy the full SHA
    104c841 View commit details
  3. Quota check preceding resource check (#614)

    * Quota release when AppWrapper is completed
    
    * Quota release testing. Fix in the Fits function to allocate consumer.
    
    * Aggregate all the resources names accross all children in the tree to create a single resource name group
    
    * Reverted changes from previous commits to current version of main
    
    * Added GPU as one of the default resources
    
    * Formatting fixes
    
    * Fixed bug in quota management library that didn't preempt jobs when borrowing at the root
    
    * Quota check preceeding resource check update. This version allows to dispatch AppWrappers that would have enough quota if borrowing AppWrappers are discounted from the resources (due to those being preempted).
    
    * Interface fix
    
    * Added some comments and TODOs
    
    * Kuttl quota borrowing test
    
    * Separated borrowing tests so that quotas are resetted
    
    * Added missing files for the borrowing test
    
    * Added commands to build quota tree when the test is run
    metalcycling authored Aug 30, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    9df2f58 View commit details

Commits on Aug 31, 2023

  1. Copy the full SHA
    7d2602e View commit details
  2. Copy the full SHA
    7877b7b View commit details
  3. Remove unused code

    astefanutti authored and openshift-merge-robot committed Aug 31, 2023
    Copy the full SHA
    be0f604 View commit details
  4. Copy the full SHA
    05b1af8 View commit details

Commits on Sep 4, 2023

  1. Copy the full SHA
    ce73445 View commit details

Commits on Sep 6, 2023

  1. Copy the full SHA
    d97f503 View commit details
  2. Update doc/usage/examples/kuberay/kuberay-mcad.md

    Co-authored-by: Anish Asthana <anishasthana1@gmail.com>
    2 people authored and openshift-merge-robot committed Sep 6, 2023
    Copy the full SHA
    4b9c3b1 View commit details
  3. Update doc/usage/examples/kuberay/kuberay-mcad.md

    Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org>
    2 people authored and openshift-merge-robot committed Sep 6, 2023
    Copy the full SHA
    23bb205 View commit details
  4. Copy the full SHA
    96ab8fa View commit details
  5. Copy the full SHA
    ed9bc20 View commit details

Commits on Sep 7, 2023

  1. Copy the full SHA
    7356aa2 View commit details
  2. address review

    asm582 authored and openshift-merge-robot committed Sep 7, 2023
    Copy the full SHA
    3a06b5f View commit details

Commits on Sep 9, 2023

  1. Copy the full SHA
    18ce712 View commit details
  2. fix typo in comment

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    2310bff View commit details
  3. fix unit tests

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    d88044d View commit details
  4. add break condition

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    4159106 View commit details
  5. Copy the full SHA
    9701d17 View commit details
  6. merge

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    f010f69 View commit details
  7. fix break condition

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    0aa05f3 View commit details
  8. fix duplicate requeues

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    bb5f3b1 View commit details
  9. address review

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    92b838b View commit details
  10. handle nil AW

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    1cf7efc View commit details
  11. fix tests

    asm582 authored and openshift-merge-robot committed Sep 9, 2023
    Copy the full SHA
    203e938 View commit details
Showing with 1,710 additions and 2,273 deletions.
  1. +25 −0 .github/workflows/auto-add-issues.yml
  2. +60 −0 .github/workflows/update-codeflare-operator.yml
  3. +6 −6 Makefile
  4. +2 −2 config/crd/bases/{ibm.com_quotasubtrees.yaml → quota.codeflare.dev_quotasubtrees.yaml}
  5. +83 −66 ...ler/crds/mcad.ibm.com_appwrappers.yaml → config/crd/bases/workload.codeflare.dev_appwrappers.yaml
  6. +2 −2 .../mcad.ibm.com_schedulingspecs.yaml → config/crd/bases/workload.codeflare.dev_schedulingspecs.yaml
  7. +3 −3 config/crd/kustomization.yaml
  8. +2 −2 ...oyment/mcad-controller/crds/{ibm.com_quotasubtrees.yaml → quota.codeflare.dev_quotasubtrees.yaml}
  9. +83 −66 ...bm.com_appwrappers.yaml → deployment/mcad-controller/crds/workload.codeflare.dev_appwrappers.yaml
  10. +2 −2 ...chedulingspecs.yaml → deployment/mcad-controller/crds/workload.codeflare.dev_schedulingspecs.yaml
  11. +2 −95 deployment/mcad-controller/templates/deployment.yaml
  12. +1 −1 doc/usage/aw-01.yaml
  13. +1 −1 doc/usage/aw-02.yaml
  14. +1 −1 doc/usage/examples/aw-1-k8s-job1.yaml
  15. +1 −1 doc/usage/examples/aw-1-k8s-job2.yaml
  16. +1 −1 doc/usage/examples/aw-1-k8s-job3.yaml
  17. +1 −1 doc/usage/examples/kuberay/config/aw-kuberay-glue.yaml
  18. +160 −0 doc/usage/examples/kuberay/config/aw-raycluster-1.yaml
  19. +96 −92 doc/usage/examples/kuberay/config/aw-raycluster.yaml
  20. +1 −1 doc/usage/examples/kuberay/config/xqueuejob-controller.yaml
  21. +48 −7 doc/usage/examples/kuberay/kuberay-mcad.md
  22. +3 −3 doc/usage/quota_management/quickstart.md
  23. +3 −3 doc/usage/tutorial.md
  24. +1 −2 go.mod
  25. +0 −2 go.sum
  26. +1 −1 hack/run-e2e-kind.sh
  27. +1 −1 pkg/apis/controller/v1beta1/doc.go
  28. +1 −1 pkg/apis/controller/v1beta1/register.go
  29. +1 −1 pkg/apis/quotaplugins/quotasubtree/v1/doc.go
  30. +1 −1 pkg/apis/quotaplugins/quotasubtree/v1/register.go
  31. +18 −18 pkg/client/clientset/versioned/clientset.go
  32. +10 −10 pkg/client/clientset/versioned/fake/clientset_generated.go
  33. +4 −4 pkg/client/clientset/versioned/fake/register.go
  34. +4 −4 pkg/client/clientset/versioned/scheme/register.go
  35. +1 −1 pkg/client/clientset/versioned/typed/controller/v1beta1/appwrapper.go
  36. +13 −13 pkg/client/clientset/versioned/typed/controller/v1beta1/controller_client.go
  37. +3 −3 pkg/client/clientset/versioned/typed/controller/v1beta1/fake/fake_appwrapper.go
  38. +3 −3 pkg/client/clientset/versioned/typed/controller/v1beta1/fake/fake_controller_client.go
  39. +3 −3 pkg/client/clientset/versioned/typed/quotasubtree/v1/fake/fake_quotasubtree.go
  40. +3 −3 pkg/client/clientset/versioned/typed/quotasubtree/v1/fake/fake_quotasubtree_client.go
  41. +1 −1 pkg/client/clientset/versioned/typed/quotasubtree/v1/quotasubtree.go
  42. +13 −13 pkg/client/clientset/versioned/typed/quotasubtree/v1/quotasubtree_client.go
  43. +2 −2 pkg/client/informers/externalversions/controller/v1beta1/appwrapper.go
  44. +4 −4 pkg/client/informers/externalversions/factory.go
  45. +4 −4 pkg/client/informers/externalversions/generic.go
  46. +2 −2 pkg/client/informers/externalversions/quotasubtree/v1/quotasubtree.go
  47. +1 −15 pkg/controller/clusterstate/api/cluster_info.go
  48. +1 −81 pkg/controller/clusterstate/api/helpers.go
  49. +15 −15 pkg/controller/clusterstate/api/histogram_info.go
  50. +3 −17 pkg/controller/clusterstate/api/node_info.go
  51. +0 −45 pkg/controller/clusterstate/api/node_info_test.go
  52. +0 −132 pkg/controller/clusterstate/api/test_utils.go
  53. +1 −15 pkg/controller/clusterstate/api/types.go
  54. +2 −16 pkg/controller/clusterstate/cache/cache.go
  55. +0 −120 pkg/controller/clusterstate/cache/cache_test.go
  56. +4 −8 pkg/controller/metrics/adapter/adapter.go
  57. +5 −211 pkg/controller/metrics/adapter/provider/provider.go
  58. +439 −378 pkg/controller/queuejob/queuejob_controller_ex.go
  59. +3 −3 pkg/controller/queuejobdispatch/queuejobagent.go
  60. +14 −13 pkg/controller/queuejobresources/genericresource/genericresource.go
  61. +1 −1 pkg/controller/quota/quota_manager_interface.go
  62. +42 −3 pkg/controller/quota/quotaforestmanager/qm_lib_backend_with_quotasubt_mgr.go
  63. +2 −2 .../quota/quotaforestmanager/qm_lib_backend_with_quotasubt_mgr/quotasubtmgr/quota_subtree_manager.go
  64. +0 −61 pkg/controller/quota/quotaforestmanager/util/utils.go
  65. +1 −1 pkg/quotaplugins/quota-forest/quota-manager/quota/core/quotanode.go
  66. +5 −3 pkg/quotaplugins/quota-forest/quota-manager/quota/quotamanager.go
  67. +1 −1 pkg/quotaplugins/quota-forest/quota-manager/quota/utils/defaults.go
  68. +4 −7 pkg/quotaplugins/quota-simple-rest/quota_rest_manager.go
  69. +95 −0 test/e2e-kuttl-borrowing/install-quota-subtree.yaml
  70. +25 −0 test/e2e-kuttl-borrowing/steps/00-assert.yaml
  71. +32 −0 test/e2e-kuttl-borrowing/steps/01-assert.yaml
  72. +5 −0 test/e2e-kuttl-borrowing/steps/02-assert.yaml
  73. +4 −0 test/e2e-kuttl-borrowing/steps/02-install.yaml
  74. +9 −0 test/e2e-kuttl-borrowing/steps/03-assert.yaml
  75. +64 −0 test/e2e-kuttl-borrowing/steps/03-install.yaml
  76. +9 −0 test/e2e-kuttl-borrowing/steps/04-assert.yaml
  77. +64 −0 test/e2e-kuttl-borrowing/steps/04-install.yaml
  78. +1 −1 test/e2e-kuttl-deployment-01/steps/00-assert.yaml
  79. +1 −1 test/e2e-kuttl-deployment-01/steps/01-assert.yaml
  80. +1 −1 test/e2e-kuttl-deployment-01/steps/01-install.yaml
  81. +1 −1 test/e2e-kuttl-deployment-01/steps/02-assert.yaml
  82. +1 −1 test/e2e-kuttl-deployment-01/steps/02-install.yaml
  83. +1 −1 test/e2e-kuttl-deployment-01/steps/03-assert.yaml
  84. +1 −1 test/e2e-kuttl-deployment-01/steps/03-install.yaml
  85. +2 −2 test/e2e-kuttl-deployment-01/steps/04-assert.yaml
  86. +1 −1 test/e2e-kuttl-deployment-01/steps/04-install.yaml
  87. +2 −2 test/e2e-kuttl-deployment-01/steps/05-assert.yaml
  88. +2 −2 test/e2e-kuttl-deployment-01/steps/05-install-single-quota-tree.yaml
  89. +1 −1 test/e2e-kuttl-deployment-01/steps/06-assert.yaml
  90. +1 −1 test/e2e-kuttl-deployment-01/steps/06-install.yaml
  91. +1 −1 test/e2e-kuttl-deployment-01/steps/07-assert.yaml
  92. +1 −1 test/e2e-kuttl-deployment-01/steps/08-assert.yaml
  93. +1 −1 test/e2e-kuttl-deployment-01/steps/08-install.yaml
  94. +4 −4 test/e2e-kuttl-deployment-02/steps/00-assert.yaml
  95. +1 −1 test/e2e-kuttl-deployment-02/steps/01-assert.yaml
  96. +1 −1 test/e2e-kuttl-deployment-02/steps/01-install.yaml
  97. +1 −1 test/e2e-kuttl-deployment-02/steps/02-assert.yaml
  98. +1 −1 test/e2e-kuttl-deployment-02/steps/02-install.yaml
  99. +1 −1 test/e2e-kuttl-deployment-02/steps/04-assert.yaml
  100. +1 −1 test/e2e-kuttl-deployment-02/steps/04-install.yaml
  101. +4 −4 test/e2e-kuttl-deployment-03/steps/00-assert.yaml
  102. +1 −1 test/e2e-kuttl-deployment-03/steps/01-assert.yaml
  103. +1 −1 test/e2e-kuttl-deployment-03/steps/01-install.yaml
  104. +1 −1 test/e2e-kuttl-deployment-03/steps/02-assert.yaml
  105. +1 −1 test/e2e-kuttl-deployment-03/steps/02-install.yaml
  106. +5 −5 test/e2e-kuttl-deployment-03/steps/03-assert.yaml
  107. +2 −2 test/e2e-kuttl-deployment-03/steps/03-install-new-quota-node.yaml
  108. +1 −1 test/e2e-kuttl-deployment-03/steps/04-assert.yaml
  109. +1 −1 test/e2e-kuttl-deployment-03/steps/04-install.yaml
  110. +1 −1 test/e2e-kuttl-deployment-03/steps/05-assert.yaml
  111. +1 −1 test/e2e-kuttl-deployment-03/steps/05-install.yaml
  112. +13 −4 test/e2e-kuttl/install-quota-subtree.yaml
  113. +2 −2 test/e2e-kuttl/quota-errors/00-assert.yaml
  114. +3 −3 test/e2e-kuttl/quota-errors/01-assert.yaml
  115. +1 −1 test/e2e-kuttl/quota-errors/03-assert.yaml
  116. +1 −1 test/e2e-kuttl/quota-errors/03-install.yaml
  117. +2 −2 test/e2e-kuttl/quota-forest/00-assert.yaml
  118. +4 −4 test/e2e-kuttl/quota-forest/01-assert.yaml
  119. +1 −1 test/e2e-kuttl/quota-forest/04-assert.yaml
  120. +1 −1 test/e2e-kuttl/quota-forest/04-install.yaml
  121. +1 −1 test/e2e-kuttl/quota-forest/05-assert.yaml
  122. +1 −1 test/e2e-kuttl/quota-forest/05-install.yaml
  123. +3 −3 test/e2e-kuttl/quota-forest/06-assert.yaml
  124. +1 −1 test/e2e-kuttl/quota-forest/06-install.yaml
  125. +4 −4 test/e2e-kuttl/quota-forest/07-assert.yaml
  126. +1 −1 test/e2e-kuttl/quota-forest/07-install.yaml
  127. +5 −5 test/e2e-kuttl/quota-forest/08-assert.yaml
  128. +1 −1 test/e2e-kuttl/quota-forest/08-install.yaml
  129. +3 −3 test/e2e-kuttl/quota-forest/09-assert.yaml
  130. +2 −2 test/e2e-kuttl/quota-forest/09-install.yaml
  131. +1 −1 test/e2e-kuttl/quota-forest/10-assert.yaml
  132. +1 −1 test/e2e-kuttl/quota-forest/10-install.yaml
  133. +1 −1 test/e2e-kuttl/quota-forest/11-assert.yaml
  134. +1 −1 test/e2e-kuttl/quota-forest/11-install.yaml
  135. +1 −15 test/e2e/e2e_test.go
  136. +0 −264 test/e2e/job.go
  137. +0 −204 test/e2e/predicates.go
  138. +13 −30 test/e2e/queue.go
  139. +4 −18 test/e2e/quota.go
  140. +36 −35 test/e2e/util.go
  141. +9 −0 test/kuttl-test-borrowing.yaml
  142. +1 −1 test/kuttl-test-deployment-02.yaml
  143. +1 −1 test/kuttl-test-deployment-03.yaml
  144. +1 −1 test/kuttl-test.yaml
  145. +1 −1 test/perf-test/preempt-exp-kwok.yaml
  146. +1 −1 test/perf-test/preempt-exp.yaml
  147. +1 −1 test/yaml/0001-aw-generic-deployment-3.yaml
  148. +1 −1 test/yaml/0002-aw-job-quota.yaml
  149. +1 −1 test/yaml/0003-aw-job-no-quota.yaml
  150. +1 −1 test/yaml/0004-aw-large-job-no-quota.yaml
  151. +2 −2 test/yaml/0005-aw-two-quota-jobs.yaml
  152. +1 −1 test/yaml/0006-aw-init-containers.yaml
  153. +1 −1 test/yaml/0008-aw-default.yaml
25 changes: 25 additions & 0 deletions .github/workflows/auto-add-issues.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Label new issues as needs-triage and add to CodeFlare Sprint Board
on:
issues:
types:
- opened
jobs:
add_label:
name: Add needs-triage label to new issues
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- uses: actions/checkout@v3
- run: gh issue edit ${{ github.event.issue.number }} --add-label "triage/needs-triage"
env:
GH_TOKEN: ${{ github.token }}

add-to-project:
name: Add issue to project
runs-on: ubuntu-latest
steps:
- uses: actions/add-to-project@v0.5.0
with:
project-url: https://github.com/orgs/project-codeflare/projects/8
github-token: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}
60 changes: 60 additions & 0 deletions .github/workflows/update-codeflare-operator.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Update CodeFlare operator with latest changes from MCAD

on:
workflow_run:
workflows: [Build and Push dev MCAD image into Quay]
types:
- completed

jobs:
update-codeflare-operator:
runs-on: ubuntu-latest
if: github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.head_branch == 'main' && github.repository == 'project-codeflare/multi-cluster-app-dispatcher'

steps:
- name: checkout MCAD code
uses: actions/checkout@v3
with:
path: 'mcad'

- name: Store latest MCAD Git SHA
run: |
echo "GIT_COMMIT_SHA=$(git rev-parse HEAD)" >> $GITHUB_ENV
working-directory: mcad

- name: checkout CodeFlare operator code
uses: actions/checkout@v3
with:
repository: 'project-codeflare/codeflare-operator'
token: ${{ env.GITHUB_TOKEN }}
path: operator
env:
GITHUB_TOKEN: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}

- name: Update CodeFlare operator code to use latest MCAD version
run: |
sed -i -E "s/(.*MCAD_VERSION \?= ).*/\1${{ env.GIT_COMMIT_SHA }}/" Makefile
sed -i -E "s/(.*MCAD_REF \?= ).*/\1dev/" Makefile
make modules
go mod tidy
working-directory: operator

- name: Commit CodeFlare operator changes in the code back to repository
uses: stefanzweifel/git-auto-commit-action@v4
with:
commit_message: Update MCAD to latest dev
create_branch: true
repository: operator
branch: dev
push_options: '--force'

- name: Create a PR in CodeFlare operator repo with code changes if not opened yet
run: |
if [[ $(gh pr view dev) && $(gh pr view dev --json state --jq .state) == "OPEN" ]]; then
echo "PR already opened"
else
gh pr create --base "main" --fill --head dev
fi
env:
GITHUB_TOKEN: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}
working-directory: operator
12 changes: 6 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
@@ -217,15 +217,15 @@ clean:
#CRD file maintenance rules
DEPLOYMENT_CRD_DIR=deployment/mcad-controller/crds
CRD_BASE_DIR=config/crd/bases
MCAD_CRDS= ${DEPLOYMENT_CRD_DIR}/ibm.com_quotasubtrees.yaml \
${DEPLOYMENT_CRD_DIR}/mcad.ibm.com_appwrappers.yaml \
${DEPLOYMENT_CRD_DIR}/mcad.ibm.com_schedulingspecs.yaml
MCAD_CRDS= ${DEPLOYMENT_CRD_DIR}/quota.codeflare.dev_quotasubtrees.yaml \
${DEPLOYMENT_CRD_DIR}/workload.codeflare.dev_appwrappers.yaml \
${DEPLOYMENT_CRD_DIR}/workload.codeflare.dev_schedulingspecs.yaml

update-deployment-crds: ${MCAD_CRDS}

${DEPLOYMENT_CRD_DIR}/ibm.com_quotasubtrees.yaml : ${CRD_BASE_DIR}/ibm.com_quotasubtrees.yaml
${DEPLOYMENT_CRD_DIR}/mcad.ibm.com_appwrappers.yaml : ${CRD_BASE_DIR}/mcad.ibm.com_appwrappers.yaml
${DEPLOYMENT_CRD_DIR}/mcad.ibm.com_schedulingspecs.yaml : ${CRD_BASE_DIR}/mcad.ibm.com_schedulingspecs.yaml
${DEPLOYMENT_CRD_DIR}/quota.codeflare.dev_quotasubtrees.yaml : ${CRD_BASE_DIR}/quota.codeflare.dev_quotasubtrees.yaml
${DEPLOYMENT_CRD_DIR}/workload.codeflare.dev_appwrappers.yaml : ${CRD_BASE_DIR}/workload.codeflare.dev_appwrappers.yaml
${DEPLOYMENT_CRD_DIR}/workload.codeflare.dev_schedulingspecs.yaml : ${CRD_BASE_DIR}/workload.codeflare.dev_schedulingspecs.yaml

$(DEPLOYMENT_CRD_DIR)/%: ${CRD_BASE_DIR}/%
cp $< $@
Original file line number Diff line number Diff line change
@@ -5,9 +5,9 @@ metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.9.2
creationTimestamp: null
name: quotasubtrees.ibm.com
name: quotasubtrees.quota.codeflare.dev
spec:
group: ibm.com
group: quota.codeflare.dev
names:
kind: QuotaSubtree
listKind: QuotaSubtreeList
Loading