Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) #936

syedriko · 2021-03-06T07:26:52Z

Description

Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate)

Upgraded Kafka to 2.7.0
Switched the kafka logforwarding test to run over TLS
The Kafka broker certificate chain is Server - Intermediate CA - Root CA
Added Java Key Store generation functions

/cc @alanconway
/assign @jcantrill

Links

Depending on PR(s): Bug 1904380: Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) origin-aggregated-logging#2068
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1904380

syedriko · 2021-03-09T13:58:00Z

/test e2e-operator

syedriko · 2021-03-09T19:03:51Z

@jcantrill Could you bless this

jcantrill

/approve

jcantrill · 2021-03-09T19:09:01Z

test/helpers/kafka/broker.go

@@ -138,7 +141,7 @@ func NewBrokerStatefuleSet(namespace string) *apps.StatefulSet {
 					Containers: []v1.Container{
 						{
 							Name:  kafkaBrokerContainerName,
-							Image: "solsson/kafka:2.4.1",
+							Image: "solsson/kafka@sha256:754cd2e82bbddd37a81d214be33fd15eb429913f9ac51dedf89d446876b95ffc",


Why are we using a specific sha in stead of a more digestible version number that is easier to document?

There's no tag for 2.7.0, it's just "latest". Between latest and sha I picked sha.

I'm not actually sure it's a good idea to rely on images from Docker Hub now that they introduced rate limiting. We could start with copying these to quay. Or better yet switch to Strimzi.

if you have concerns, feel free to push to quay.io/openshift-logging. Seems reasonable for us to have a controlled copy of the image against which we test.

syedriko · 2021-03-11T04:21:58Z

/test e2e-operator

syedriko · 2021-03-11T14:00:39Z

/test e2e-operator

syedriko · 2021-03-12T01:47:01Z

/test e2e-operator

syedriko · 2021-03-12T15:12:54Z

/test e2e-operator

syedriko · 2021-03-15T05:27:51Z

/test e2e-operator

syedriko · 2021-03-16T02:55:57Z

/test e2e-operator

syedriko · 2021-03-16T13:36:05Z

/test e2e-operator

syedriko · 2021-03-18T13:05:10Z

/test e2e-operator

syedriko · 2021-03-18T18:25:37Z

/test e2e-operator

openshift-ci-robot · 2021-03-18T18:39:47Z

@syedriko: This pull request references Bugzilla bug 1904380, which is invalid:

expected the bug to target the "4.8.0" release, but it targets "4.6.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1904380: Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2021-03-18T18:40:16Z

@syedriko: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

syedriko · 2021-03-20T16:08:40Z

/test e2e-operator

test/helpers/kafka/zookeeper.go

…ls with error "state=error: certificate verify failed (unable to get local issuer certificate) - Upgraded Kafka to 2.7.0 and pushed images to quay.io - Switched the kafka logforwarding test to run over TLS - The Kafka broker certificate chain is Server - Intermediate CA - Root CA - Added Java Key Store generation functions - Removed Kafka consumer output to stdio to avoid recursive logs

syedriko · 2021-03-25T18:43:50Z

@jcantrill yea? nay?

alanconway

/lgtm

openshift-ci-robot · 2021-03-29T15:15:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alanconway, syedriko

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [alanconway]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2021-03-29T18:55:27Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-03-29T19:47:06Z

/retest

Please review the full test history for this PR and help us cut down flakes.

syedriko · 2021-03-30T02:08:41Z

/cherrypick release-5.0

openshift-cherrypick-robot · 2021-03-30T02:09:15Z

@syedriko: #936 failed to apply on top of branch "release-5.0":

Applying: Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) - Upgraded Kafka to 2.7.0 and pushed images to quay.io - Switched the kafka logforwarding test to run over TLS - The Kafka broker certificate chain is Server - Intermediate CA - Root CA - Added Java Key Store generation functions - Removed Kafka consumer output to stdio to avoid recursive logs
Using index info to reconstruct a base tree...
M	go.mod
M	go.sum
M	vendor/modules.txt
Falling back to patching base and 3-way merge...
Auto-merging vendor/modules.txt
CONFLICT (content): Merge conflict in vendor/modules.txt
Auto-merging go.sum
Auto-merging go.mod
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) - Upgraded Kafka to 2.7.0 and pushed images to quay.io - Switched the kafka logforwarding test to run over TLS - The Kafka broker certificate chain is Server - Intermediate CA - Root CA - Added Java Key Store generation functions - Removed Kafka consumer output to stdio to avoid recursive logs
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherrypick release-5.0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate)

openshift-ci-robot assigned jcantrill Mar 6, 2021

openshift-ci-robot requested a review from alanconway March 6, 2021 07:26

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 6, 2021

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 9, 2021

jcantrill reviewed Mar 9, 2021

View reviewed changes

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 9, 2021

jcantrill added release/5.1 backport/5.0 backport/4.6 and removed approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Mar 9, 2021

syedriko force-pushed the bz_1904380 branch from c83f257 to 13b77b1 Compare March 11, 2021 01:15

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 11, 2021

syedriko force-pushed the bz_1904380 branch from 13b77b1 to ed38b03 Compare March 16, 2021 00:45

openshift-ci-robot added bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Mar 18, 2021

openshift-ci-robot removed bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Mar 18, 2021

jcantrill requested changes Mar 20, 2021

View reviewed changes

test/helpers/kafka/zookeeper.go Outdated Show resolved Hide resolved

test/helpers/kafka/zookeeper.go Show resolved Hide resolved

openshift-ci-robot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 20, 2021

syedriko force-pushed the bz_1904380 branch from ed38b03 to 174671a Compare March 21, 2021 00:41

openshift-ci-robot mentioned this pull request Mar 24, 2021

[release-4.6] Bug 1904380: Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) openshift/origin-aggregated-logging#2083

Merged

syedriko requested a review from jcantrill March 24, 2021 23:51

alanconway approved these changes Mar 29, 2021

View reviewed changes

openshift-ci-robot assigned alanconway Mar 29, 2021

openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Mar 29, 2021

openshift-merge-robot merged commit 24a0432 into openshift:master Mar 29, 2021

syedriko deleted the bz_1904380 branch June 3, 2021 18:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) #936

Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) #936

syedriko commented Mar 6, 2021 •

edited

Loading

syedriko commented Mar 9, 2021

syedriko commented Mar 9, 2021

jcantrill left a comment

jcantrill Mar 9, 2021

syedriko Mar 9, 2021

syedriko Mar 9, 2021

jcantrill Mar 10, 2021

syedriko commented Mar 11, 2021

syedriko commented Mar 11, 2021

syedriko commented Mar 12, 2021

syedriko commented Mar 12, 2021

syedriko commented Mar 15, 2021

syedriko commented Mar 16, 2021

syedriko commented Mar 16, 2021

syedriko commented Mar 18, 2021

syedriko commented Mar 18, 2021

openshift-ci-robot commented Mar 18, 2021

openshift-ci-robot commented Mar 18, 2021

syedriko commented Mar 20, 2021

syedriko commented Mar 25, 2021

alanconway left a comment

openshift-ci-robot commented Mar 29, 2021

openshift-bot commented Mar 29, 2021

openshift-bot commented Mar 29, 2021

syedriko commented Mar 30, 2021

openshift-cherrypick-robot commented Mar 30, 2021

Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) #936

Bug 1904380 - Forwarding logs to Kafka using Chained certificates fails with error "state=error: certificate verify failed (unable to get local issuer certificate) #936

Conversation

syedriko commented Mar 6, 2021 • edited Loading

Description

Links

syedriko commented Mar 9, 2021

syedriko commented Mar 9, 2021

jcantrill left a comment

Choose a reason for hiding this comment

jcantrill Mar 9, 2021

Choose a reason for hiding this comment

syedriko Mar 9, 2021

Choose a reason for hiding this comment

syedriko Mar 9, 2021

Choose a reason for hiding this comment

jcantrill Mar 10, 2021

Choose a reason for hiding this comment

syedriko commented Mar 11, 2021

syedriko commented Mar 11, 2021

syedriko commented Mar 12, 2021

syedriko commented Mar 12, 2021

syedriko commented Mar 15, 2021

syedriko commented Mar 16, 2021

syedriko commented Mar 16, 2021

syedriko commented Mar 18, 2021

syedriko commented Mar 18, 2021

openshift-ci-robot commented Mar 18, 2021

openshift-ci-robot commented Mar 18, 2021

syedriko commented Mar 20, 2021

syedriko commented Mar 25, 2021

alanconway left a comment

Choose a reason for hiding this comment

openshift-ci-robot commented Mar 29, 2021

openshift-bot commented Mar 29, 2021

openshift-bot commented Mar 29, 2021

syedriko commented Mar 30, 2021

openshift-cherrypick-robot commented Mar 30, 2021

syedriko commented Mar 6, 2021 •

edited

Loading