Skip to content

add controller to autoregister APIServices #42732

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 0 commits into from

Conversation

deads2k
Copy link
Contributor

@deads2k deads2k commented Mar 8, 2017

As we look to combine the aggregator and the core kube-apiserver which includes things like TPRs, we need a way to reliably auto-manage APIServices for the aggregator so that discovery works as expected and so that users don't have to maintain particular resources that they shouldn't have to.

This controller deals with a dynamic set of APIServices which are synchronized. The mutation of that set is exposed via threadsafe methods. If a user wants to take control, he can remove our label and do whatever he wants.

The controller approach provides the functionality we need while preserving the independence of various layers.

@kubernetes/sig-api-machinery-pr-reviews @ncdc

@deads2k deads2k added this to the v1.7 milestone Mar 8, 2017
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Mar 8, 2017
@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

The following people have approved this PR: deads2k

Needs approval from an approver in each of these OWNERS Files:

We suggest the following people:
cc @wojtek-t
You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@k8s-github-robot k8s-github-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. release-note-label-needed labels Mar 8, 2017
@k8s-reviewable
Copy link

This change is Reviewable

@deads2k deads2k force-pushed the agg-03-tpr-controller branch from 1cc35df to 824a826 Compare March 8, 2017 18:23
@deads2k
Copy link
Contributor Author

deads2k commented Mar 9, 2017

@k8s-bot cvm gce e2e test this

@ncdc ncdc added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Mar 9, 2017
// AutoAPIServiceRegistration is an interface which callers can re-declare locally and properly cast to for
// adding and removing APIServices
type AutoAPIServiceRegistration interface {
AddAPIServiceToSync(in *apiregistration.APIService)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

godoc on these 2 definitions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doing

}

// AutoRegisterController is used to keep a particular set of APIServices present in the API. It is useful
// for cases where you want to auto-register APIs like TPRs or core API server
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"or core API server" ... ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"or core API server" ... ?

clarifying. "or groups from the core kube-apiserver"

apiServiceSynced cache.InformerSynced
apiServiceClient apiregistrationclient.APIServicesGetter

apiServiceToSyncLock sync.RWMutex
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pluralize services? apiServicesToSyncLock

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

queue workqueue.RateLimitingInterface
}

func NewAutoRegisterAutoRegisterController(apiServiceInformer informers.APIServiceInformer, apiServiceClient apiregistrationclient.APIServicesGetter) *AutoRegisterController {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to have AutoRegister duplicated in the function name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixing


// AutoRegisterController is used to keep a particular set of APIServices present in the API. It is useful
// for cases where you want to auto-register APIs like TPRs or core API server
type AutoRegisterController struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need this type exported?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need this type exported?

I can make it private to start. I suspect that most callers will use the interface and call the run method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doing

@deads2k deads2k force-pushed the agg-03-tpr-controller branch from 824a826 to 8a3b57b Compare March 9, 2017 15:58
@deads2k
Copy link
Contributor Author

deads2k commented Mar 9, 2017

comments so far addressed.

}
cast, ok = tombstone.Obj.(*apiregistration.APIService)
if !ok {
glog.V(2).Infof("Tombstone contained object that is not a CSR: %#v", obj)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CSR? 😄

case desired == nil:
return c.apiServiceClient.APIServices().Delete(curr.Name, nil)

// if the spec's already match, nothing for us to do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/spec's/specs/

indentation off

return c.apiServiceClient.APIServices().Delete(curr.Name, nil)

// if the spec's already match, nothing for us to do
case reflect.DeepEqual(curr.Spec, desired.Spec):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is reflect ok or do we need semantic?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is reflect ok or do we need semantic?

Semantic isn't cleanly divorced from a global scheme yet. Reflect works for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is reflect ok or do we need semantic?

Semantic isn't cleanly divorced from a global scheme yet. Reflect works for now.

@deads2k
Copy link
Contributor Author

deads2k commented Mar 9, 2017

@k8s-bot gce etcd3 e2e test this

@deads2k
Copy link
Contributor Author

deads2k commented Mar 10, 2017

/release-note-none

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed release-note-label-needed labels Mar 10, 2017
@deads2k
Copy link
Contributor Author

deads2k commented Mar 10, 2017

@ncdc did you make it through?

@deads2k deads2k force-pushed the agg-03-tpr-controller branch from 8a3b57b to c7318e0 Compare March 10, 2017 17:27
@k8s-github-robot k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 11, 2017
func(c AutoAPIServiceRegistration, fakeWatch *watch.FakeWatcher, client internalclientset.Interface) {
c.AddAPIServiceToSync(&apiregistration.APIService{ObjectMeta: metav1.ObjectMeta{Name: "foo"}})
},
// adding an API service which claims to be managed but isn't should be deleted
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix comment, then lgtm

@deads2k deads2k force-pushed the agg-03-tpr-controller branch from c7318e0 to 55adc4d Compare March 13, 2017 14:12
@deads2k deads2k added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 13, 2017
@deads2k deads2k added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 13, 2017
@k8s-github-robot k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 13, 2017
@ncdc
Copy link
Member

ncdc commented Mar 13, 2017

LGTM

@deads2k
Copy link
Contributor Author

deads2k commented Mar 13, 2017

@k8s-bot gce etcd3 e2e test this

@k8s-github-robot
Copy link

@deads2k PR needs rebase

@k8s-github-robot k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 25, 2017
@fejta fejta removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 25, 2017
k8s-github-robot pushed a commit that referenced this pull request Mar 26, 2017
Automatic merge from submit-queue (batch tested with PRs 43429, 43416, 43312, 43141, 43421)

Create controller to auto register TPRs with the aggregator

Builds on #42732 (already lgtmed)

Creates a simple controller to wire TPRs with the API Service autoregistration controller.

@kubernetes/sig-api-machinery-misc @ncdc
@deads2k deads2k closed this Mar 27, 2017
@deads2k deads2k force-pushed the agg-03-tpr-controller branch from 55adc4d to e38c575 Compare March 27, 2017 11:35
@deads2k deads2k removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 27, 2017
k8s-github-robot pushed a commit that referenced this pull request Mar 27, 2017
Automatic merge from submit-queue (batch tested with PRs 43694, 41262, 42911)

combine kube-apiserver and kube-aggregator

This combines several pulls currently in progress and wires them together.  The aggregator sits in front of the normal kube-apiserver and allows local fallthrough instead of proxying.

@kubernetes/sig-api-machinery-misc 
@DirectXMan12 since you seem invested, your life will get easier
@luxas FYI since you've started trying to wire something together.  



Dependent Pulls LGTM:
- [x] #42801
- [x] #42886
- [x] #42900
- [x] #42732
- [x] #42672
- [x] #43141
- [x] #43076
- [x] #43149
- [x] #43226
- [x] #43144
@deads2k deads2k mentioned this pull request Mar 28, 2017
12 tasks
@deads2k deads2k deleted the agg-03-tpr-controller branch August 3, 2017 20:07
hoegaarden added a commit to hoegaarden/frameworks that referenced this pull request Mar 19, 2018
We found that the APIServer was not fully up when it logged "Serving
(in)securely on ...". We tried to find a different log line we can wait
for to know when the APIServer is fully up.

The autoregister controller came in with
kubernetes/kubernetes#42732. It seems to be the
last thing to come up before the server is ready to go.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants