WIP: Create release governance definition #1812

kevinrizza · 2020-10-15T12:51:23Z

Initial release governance doc intended to codify the project's release process and schedule

of-deploy-bot · 2020-10-15T13:49:41Z

This PR failed 2 out of 2 times with 6 individual failed tests and 4 skipped tests. A test is considered flaky if failed on multiple commits.

totaltestcount: 2
failedtestcount: 2
flaketestcount: 6
skippedtestcount: 4
flaketests:

classname: End-to-end
name: 'Install Plan with CSVs across multiple catalog sources'
counts: 3
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/installplan_e2e_test.go:49
Resource does not match expected value:   &v1alpha1.InstallPlan{
  	TypeMeta: {},
  	ObjectMeta: v1.ObjectMeta{
  		... // 3 identical fields
  		SelfLink:                   "/apis/operators.coreos.com/v1alpha1/namespaces/operators/install"...,
  		UID:                        "5d85e439-a70e-4c05-856a-a735e5ec5fa1",
- 		ResourceVersion:            "19216",
+ 		ResourceVersion:            "19222",
  		Generation:                 1,
  		CreationTimestamp:          {Time: s"2020-10-15 13:32:09 +0000 UTC"},
  		DeletionTimestamp:          nil,
  		DeletionGracePeriodSeconds: nil,
  		Labels: map[string]string{
  			"operators.coreos.com/nginx-84ssz.operators":    "",
+ 			"operators.coreos.com/nginxdep-dtxkg.operators": "",
  		},
  		Annotations:     nil,
  		OwnerReferences: {{APIVersion: "operators.coreos.com/v1alpha1", Kind: "Subscription", Name: "sub-nginx-rnl54", UID: "05b6a7f6-deeb-4396-aa42-1bbee32d1095", ...}},
  		... // 3 identical fields
  	},
  	Spec:   {ClusterServiceVersionNames: {"nginx-84ssz-stable", "nginxdep-dtxkg-stable"}, Approval: "Automatic", Approved: true, Generation: 1},
  	Status: {Phase: "Complete", Conditions: {{Type: "Installed", Status: "True", LastUpdateTime: s"2020-10-15 13:32:12 +0000 UTC", LastTransitionTime: s"2020-10-15 13:32:12 +0000 UTC", ...}}, CatalogSources: {"mock-ocs-main-jdcrn", "mock-ocs-dependent-lmnbs"}, Plan: {&{Resolving: "nginx-84ssz-stable", Resource: {CatalogSource: "mock-ocs-main-jdcrn", CatalogSourceNamespace: "operators", Version: "v1alpha1", Kind: "ClusterServiceVersion", ...}, Status: "Present"}, &{Resolving: "nginxdep-dtxkg-stable", Resource: {CatalogSource: "mock-ocs-dependent-lmnbs", CatalogSourceNamespace: "operators", Version: "v1alpha1", Kind: "ClusterServiceVersion", ...}, Status: "Present"}, &{Resolving: "nginxdep-dtxkg-stable", Resource: {CatalogSource: "mock-ocs-dependent-lmnbs", CatalogSourceNamespace: "operators", Group: "apiextensions.k8s.io", Version: "v1beta1", ...}, Status: "Present"}, &{Resolving: "nginxdep-dtxkg-stable", Resource: {CatalogSource: "mock-ocs-dependent-lmnbs", CatalogSourceNamespace: "operators", Group: "operators.coreos.com", Version: "v1alpha1", ...}, Status: "Present"}}, ...},
  }

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/installplan_e2e_test.go:2792

meandurationsec: 182.87740133333332

classname: End-to-end
name: 'CSV create requirements met CRD'
counts: 1
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/csv_e2e_test.go:459

	Error Trace:	csv_e2e_test.go:701
	            				runner.go:113
	            				runner.go:64
	            				it_node.go:26
	            				spec.go:215
	            				spec.go:138
	            				spec_runner.go:200
	            				spec_runner.go:170
	            				spec_runner.go:66
	            				suite.go:62
	            				ginkgo_dsl.go:226
	            				ginkgo_dsl.go:214
	            				e2e_test.go:54
	Error:      	Received unexpected error:
	            	timed out waiting for the condition

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/stretchr/testify/require/require.go:1005

meandurationsec: 335.140363

classname: End-to-end
name: 'CSV update with owned API service'
counts: 1
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/csv_e2e_test.go:1069

	Error Trace:	csv_e2e_test.go:1231
	            				runner.go:113
	            				runner.go:64
	            				it_node.go:26
	            				spec.go:215
	            				spec.go:138
	            				spec_runner.go:200
	            				spec_runner.go:170
	            				spec_runner.go:66
	            				suite.go:62
	            				ginkgo_dsl.go:226
	            				ginkgo_dsl.go:214
	            				e2e_test.go:54
	Error:      	Received unexpected error:
	            	services "hat-server5dmq6-service" not found
	Messages:   	error getting expected Service

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/stretchr/testify/require/require.go:1005

meandurationsec: 14.152109

classname: End-to-end
name: 'CRD Versions creates v1beta1 crds with a v1beta1 schema successfully'
counts: 1
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/crd_e2e_test.go:25
Unexpected error:
    <*errors.errorString | 0xc0001722b0>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/crd_e2e_test.go:92

meandurationsec: 313.206771

classname: End-to-end
name: 'Install Plan update catalog for subscription UpdateSingleExistingCRDOwner'
counts: 1
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/installplan_e2e_test.go:1660

	Error Trace:	installplan_e2e_test.go:1765
	            				runner.go:113
	            				runner.go:64
	            				it_node.go:26
	            				spec.go:215
	            				spec.go:138
	            				spec_runner.go:200
	            				spec_runner.go:170
	            				spec_runner.go:66
	            				suite.go:62
	            				ginkgo_dsl.go:226
	            				ginkgo_dsl.go:214
	            				e2e_test.go:54
	Error:      	Received unexpected error:
	            	timed out waiting for the condition

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/stretchr/testify/require/require.go:1005

meandurationsec: 316.121272

classname: End-to-end
name: 'Install Plan creation with pre existing CRD owners PreExistingCRDOwnerIsReplaced'
counts: 1
details:

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/installplan_e2e_test.go:329

	Error Trace:	installplan_e2e_test.go:438
	            				runner.go:113
	            				runner.go:64
	            				it_node.go:26
	            				spec.go:215
	            				spec.go:138
	            				spec_runner.go:200
	            				spec_runner.go:170
	            				spec_runner.go:66
	            				suite.go:62
	            				ginkgo_dsl.go:226
	            				ginkgo_dsl.go:214
	            				e2e_test.go:54
	Error:      	Received unexpected error:
	            	timed out waiting for the condition

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/stretchr/testify/require/require.go:1005

meandurationsec: 323.362984
skippedtests:

classname: End-to-end
name: 'Catalog image update'
counts: 2
details: []
meandurationsec: 0.2448055
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should install the operator
successfully'
counts: 2
details: []
meandurationsec: 17.7137215
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should have created the expected
prometheus objects'
counts: 2
details: []
meandurationsec: 36.85761050000001
classname: End-to-end
name: 'Subscription updates existing install plan'
counts: 2
details: []
meandurationsec: 0.14592699999999997

cdjohnson · 2020-10-15T14:22:46Z

Suggestion: When we review these RFE's and don't yet accept them as valid, consider using a tag such as triaged, so we know we've started talking about it.

stale · 2020-12-14T16:02:55Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

exdx

Thanks for putting this together @kevinrizza

I think my thoughts differ on how much weight we need to put into an individual release and how much the core OLM folks need to be involved. I think we should let the upstream process go in its own direction and not apply a lot of policy to the release cycle. Let's discuss this more in our feature planning meeting.

exdx · 2021-01-19T22:45:00Z

doc/contributors/releasegovernance.md

+The Operator Lifecycle Manager project is part of the Operator Framework. As a result, it's community membership guidelines adhere to the definitions defined in the Operator Framework's [community membership definition](https://github.com/operator-framework/community/blob/master/community-membership.md). For the purpose of this document, there are two roles that are relevant for the release schedule.
+
+- Project reviewers provide guidance on whether or not a release is ready. Before a release can be cut, we will require that two reviewers provide the `/lgtm` label to the release.
+- Project approvers provide the final approval on whether or not a release is ready to be cut. A quorum (greater than 50%) of the project's approvers need to agree that a release is ready by writing the `/approve` label to the release.


This seems like a lot of people to simply cut a minor release version, no? We're talking about 5 people at a minimum. If the number of approvers grows by 1, then a quorum would be 4 which means 6 people total. Seems like a lot of coordination for a go/no-go on a release.

Do we really need so much review for the upstream community to cut a release? We want more frequent releases and for upstream members to be able to release on an accelerated schedule. I would think 2 lgtms and 1 approve would suffice? With the goal of the upstream community members being able to cut their own releases eventually with a different set of owners

exdx · 2021-01-19T22:48:18Z

doc/contributors/releasegovernance.md

+
+### Minor Releases
+When a minor release is ready to be generated, a github issue will be filed against the project containing preliminary release notes. At that time, it should be labeled with the milestone expected for that release. Once that issue is opened, it needs approval from two reviewers and a majority of project approvers before the release can be scheduled. Additionally, at the same time, an email will be sent out to the operator framework mailing list with a link to the release notes in the issue. At that time, there is a minimum of a 48 hour comment period before the release can be completed.
+


What's the value of the 48 hour comment period besides just slowing down the release process once its been approved? I can understand sending out the email but cutting a new release seems like safe operation, not potentially contentious like an enhancement, is there really a freeze period required?

Will commenters be able to argue against cutting a release in the comment period? I think it makes sense to have some time for everyone to review the release before it goes out, but that I think that should be before the github issue gets approved.

exdx · 2021-01-19T22:49:25Z

doc/contributors/releasegovernance.md

+### Cadence
+The high level objective for Operator Lifecycle Manager releases is to release a minor version on a 3 month cadence
+
+- Minor releases alongside the current Kubernetes release schedule


Maybe the upstream project can have more frequent releases? The kube release cycle mirrors the downstream OCP release cycle, which is fine, but having more frequent releases cut when new features make it into upstream could be another way to go

openshift-ci-robot · 2021-01-19T22:54:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: exdx, kevinrizza

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kevinrizza]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

doc/contributors/releasegovernance.md

openshift-ci · 2022-02-03T19:39:50Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: exdx, kevinrizza

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kevinrizza]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

awgreene · 2022-09-03T14:27:05Z

Hello,

This repository contains a number of PRs that have been open for an excessively long period of time, in some cases these PRs have been open for more than two years. In an effort to improve the health of our project, we are closing PRs that haven't been updated in the past 30 days or include features that we do not wish to adopt. This PR has been identified as a candidate meeting the requirements above and will be closed. If you believe that this PR should still be merged, please reopen the PR and explain why.

We will eventually release an official policy detailing our process for handling "stale" PRs and issues. If you would like to be involved in the conversation, please engage through this Issue.

Thanks you,

Alex

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 15, 2020

openshift-ci-robot requested review from ankitathomas and exdx October 15, 2020 12:51

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 15, 2020

stale bot added the wontfix label Dec 14, 2020

openshift-ci-robot added triage/unresolved Indicates an issue that can not or will not be resolved. and removed wontfix labels Dec 15, 2020

exdx approved these changes Jan 19, 2021

View reviewed changes

dinhxuanvu assigned exdx Feb 15, 2021

dinhxuanvu removed the triage/unresolved Indicates an issue that can not or will not be resolved. label Feb 15, 2021

exdx mentioned this pull request Apr 29, 2021

streamline upstream release process #2118

Closed

timflannagan reviewed May 25, 2021

View reviewed changes

doc/contributors/releasegovernance.md Show resolved Hide resolved

github-actions bot force-pushed the releasegovernance branch from 02370a0 to 48afff0 Compare February 3, 2022 19:39

timflannagan closed this Feb 3, 2022

timflannagan reopened this Feb 3, 2022

github-actions bot force-pushed the releasegovernance branch 3 times, most recently from d3db1ae to f581cbc Compare February 9, 2022 18:10

kevinrizza added 2 commits February 10, 2022 14:58

Create release governance definition

6fc7422

Update releasegovernance.md

81e3dec

github-actions bot force-pushed the releasegovernance branch from f581cbc to 81e3dec Compare February 10, 2022 14:58

awgreene closed this Sep 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Create release governance definition #1812

WIP: Create release governance definition #1812

kevinrizza commented Oct 15, 2020

of-deploy-bot commented Oct 15, 2020

cdjohnson commented Oct 15, 2020

stale bot commented Dec 14, 2020

exdx left a comment

exdx Jan 19, 2021

exdx Jan 19, 2021

ankitathomas Jan 28, 2021

exdx Jan 19, 2021

openshift-ci-robot commented Jan 19, 2021

openshift-ci bot commented Feb 3, 2022

awgreene commented Sep 3, 2022


		### Minor Releases
		When a minor release is ready to be generated, a github issue will be filed against the project containing preliminary release notes. At that time, it should be labeled with the milestone expected for that release. Once that issue is opened, it needs approval from two reviewers and a majority of project approvers before the release can be scheduled. Additionally, at the same time, an email will be sent out to the operator framework mailing list with a link to the release notes in the issue. At that time, there is a minimum of a 48 hour comment period before the release can be completed.

WIP: Create release governance definition #1812

WIP: Create release governance definition #1812

Conversation

kevinrizza commented Oct 15, 2020

of-deploy-bot commented Oct 15, 2020

cdjohnson commented Oct 15, 2020

stale bot commented Dec 14, 2020

exdx left a comment

Choose a reason for hiding this comment

exdx Jan 19, 2021

Choose a reason for hiding this comment

exdx Jan 19, 2021

Choose a reason for hiding this comment

ankitathomas Jan 28, 2021

Choose a reason for hiding this comment

exdx Jan 19, 2021

Choose a reason for hiding this comment

openshift-ci-robot commented Jan 19, 2021

openshift-ci bot commented Feb 3, 2022

awgreene commented Sep 3, 2022