Skip to content
This repository was archived by the owner on Mar 13, 2022. It is now read-only.

Grow the resource_version monotonically when watching #131

Closed

Conversation

nolar
Copy link

@nolar nolar commented Apr 25, 2019

See kubernetes-client/python#819 for the explanation of the problem, dilemma, and examples.

With this proposal, the remembered resource_version grows monotonically, i.e. never decreasing.

Specifically, after the list...() operation is finished, it will remember the maximum of all the resource_versions of all currently existing objects. In case of disconnection and reconnection, it will continue from the last seen state of all objects — as expected.

@k8s-ci-robot
Copy link
Contributor

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
  • If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
  • If you have done the above and are still having issues with the CLA being reported as unsigned, please email the CNCF helpdesk: [email protected]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Apr 25, 2019
@k8s-ci-robot k8s-ci-robot requested review from roycaihw and yliaog April 25, 2019 16:52
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nolar
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: yliaog

If they are not already assigned, you can assign the PR to them by writing /assign @yliaog in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Apr 25, 2019
@@ -152,7 +152,7 @@ def get_values(*args, **kwargs):
# opaque value we cannot interpret it and order it so rely
# on k8s returning the events completely and in order
calls.append(call(_preload_content=False, watch=True,
resource_version="3"))
resource_version="5"))
Copy link
Author

@nolar nolar Apr 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the uncertainty:

The comment above this line says that we should not interpret the values and should rely on Kubernetes to return the events in order.

But Kubernetes does not return them in order.

Should we start interpreting the resource_version then?

PS: Full quote:

ideally we want 5 here but as rv must be treated as an
opaque value we cannot interpret it and order it so rely
on k8s returning the events completely and in order

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure Kubernetes do not return them in order? Have you anything to confirm that it is by design that order is not assured?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mitar There is an example in kubernetes-client/python#819

@codecov-io
Copy link

Codecov Report

Merging #131 into master will increase coverage by 0.12%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #131      +/-   ##
==========================================
+ Coverage   92.84%   92.97%   +0.12%     
==========================================
  Files          13       13              
  Lines        1328     1352      +24     
==========================================
+ Hits         1233     1257      +24     
  Misses         95       95
Impacted Files Coverage Δ
watch/watch_test.py 98.85% <100%> (+0.15%) ⬆️
watch/watch.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d5231c...c08bf54. Read the comment docs.

@nolar
Copy link
Author

nolar commented Apr 25, 2019

Moved the detailed description to kubernetes-client/python#819 for discussion of the possible ways to solve the problem (it is not so straightforward).

# on the first listing call (the objects are sorted randomly).
if (resource_version is not None and
(self.resource_version is None or
self.resource_version < resource_version)):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think indentation is off here, no?

@mitar
Copy link
Contributor

mitar commented May 12, 2019

This PR looks reasonable and at worst does not do anything. But if order is not really assured, then much more than just this change should probably be done, no? I mean, that assumption is all around the codebase.

@nolar
Copy link
Author

nolar commented Jun 17, 2019

Closing this PR as the implementation is not correct. The proper way to get list the resources normally, get a resourceVersion of the list itself (not of its items!), and use that resourceVersion as a starting point for watching. And only then the latest seen resourceVersion of the items should be used to restarting watching. In either case, the resourceVersion should not be interpreted in any way, and should not be assumed to be sortable or monotonically growing. See kubernetes-client/python#819 for discussion.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants