"Unexpected EOF" on "WriteMessages" on 0.4.16

[arnulfosuarez]

Describe the bug
I updated the library to 0.4.16, and started getting the error "Unexpected EOF". I downgrade to 0.4.6 and 0.4.8, same problem.

Kafka Version
What version(s) of Kafka are you testing against?
I'm using Kafka 0.11.0.1

To Reproduce
This is my Code:

	transport := &kafka.Transport{}

	if cfg.KafkaSASLPlainUsername != "" {
		transport.SASL = plain.Mechanism{
			Username: cfg.KafkaSASLPlainUsername,
			Password: cfg.KafkaSASLPlainPassword,
		}

		if cfg.KafkaSSLCACert != "" {
			caCertPool := x509.NewCertPool()
			if ok := caCertPool.AppendCertsFromPEM([]byte(cfg.KafkaSSLCACert)); !ok {
				log.Fatal("error adding CA cert")
			}

			transport.TLS = &tls.Config{
				RootCAs:            caCertPool,
				InsecureSkipVerify: cfg.KafkaSSLInsecureSkipVerify,
			}
		}
	}

	// setup kafka writers
	writer := &kafka.Writer{
		Balancer:    &core_kafka.PartitionBalancer{},
		Addr:        kafka.TCP(cfg.KafkaBrokers...),
		Transport:   transport,
		Topic:       cfg.KafkaTopic,
		Logger:      kafka.LoggerFunc(log.Infof),
		ErrorLogger: kafka.LoggerFunc(log.Errorf),
	}
	defer writer.Close()

originally for version 0.3.6, I was using:

	dialer := *kafka.DefaultDialer
	if cfg.KafkaSASLPlainUsername != "" {
		dialer.SASLMechanism = plain.Mechanism{
			Username: cfg.KafkaSASLPlainUsername,
			Password: cfg.KafkaSASLPlainPassword,
		}

		if cfg.KafkaSSLCACert != "" {
			caCertPool := x509.NewCertPool()
			if ok := caCertPool.AppendCertsFromPEM([]byte(cfg.KafkaSSLCACert)); !ok {
				log.Fatal("error adding CA cert")
			}

			dialer.TLS = &tls.Config{
				RootCAs:            caCertPool,
				InsecureSkipVerify: cfg.KafkaSSLInsecureSkipVerify,
			}
		}
	}

	// setup kafka writers
	writer := kafka.NewWriter(kafka.WriterConfig{
		Balancer:    &core_kafka.PartitionBalancer{},
		Brokers:     cfg.KafkaBrokers,
		Dialer:      &dialer,
		Topic:       cfg.KafkaTopic,
		Logger:      kafka.LoggerFunc(log.Infof),
		ErrorLogger: kafka.LoggerFunc(log.Errorf),
	})
	defer writer.Close()

To write messages I'm using this code, is the same for the two versions.

			if err := writer.WriteMessages(ctx, messages...); err != nil {
				Error(w, r, err)
				return
			}

Expected behavior
Send messages without error

Additional context
It runs against a server with a RootCA, InsecureSkipVerify = true and SASL active with plain mechanism.

For 0.4.16, I tested the original version with the Dialer and I still got the same error.

Any ideas of what can I do to solve the issue?

[arnulfosuarez]

Hi, do you need additional information?

[arnulfosuarez]

Hi,

I retest this with version 0.4.17 and the same error is appearing.

Any additional information, please let me know.

[arnulfosuarez]

Hi, I was able to track a stack trace of the error, hope it helps

    unexpected EOF
 github.com/segmentio/kafka-go/protocol.dontExpectEOF
 	/go/pkg/mod/github.com/arnulfosuarez/[email protected]/protocol/protocol.go:374
 github.com/segmentio/kafka-go/protocol.ReadResponse
 	/go/pkg/mod/github.com/arnulfosuarez/[email protected]/protocol/response.go:32
 github.com/segmentio/kafka-go/protocol.RoundTrip
 	/go/pkg/mod/github.com/arnulfosuarez/[email protected]/protocol/roundtrip.go:15
 github.com/segmentio/kafka-go/protocol.(*Conn).RoundTrip
 	/go/pkg/mod/github.com/arnulfosuarez/[email protected]/protocol/conn.go:90
 github.com/segmentio/kafka-go.saslAuthenticateRoundTrip
 	/go/pkg/mod/github.com/arnulfosuarez/[email protected]/transport.go:1291
 github.com/segmentio/kafka-go.authenticateSASL
 	/go/pkg/mod/github.com/arnulfosuarez/[email protected]/transport.go:1240
 github.com/segmentio/kafka-go.(*connGroup).connect
 	/go/pkg/mod/github.com/arnulfosuarez/[email protected]/transport.go:1164
 github.com/segmentio/kafka-go.(*connGroup).grabConnOrConnect.func1
 	/go/pkg/mod/github.com/arnulfosuarez/[email protected]/transport.go:971

[achille-roussel]

It's very helpful, thanks @arnulfosuarez !

[calvinbrown085]

I am also running into this issue! Any help would be appreciated.

[tong3jie]

mee too
version is 0.4.23

[tong3jie]

@arnulfosuarez hi, did you solved it ?

[rhansen2]

I suspect this was fixed by #869

[yunpliu]

@achille-roussel Any update on this issue?