Remote _reindex fail for HTTPS remotes

[floragunn]

Elasticsearch version (bin/elasticsearch --version):
6.0.0-rc2

Plugins installed: []

JVM version (java -version):
n/a

OS version (uname -a if on a Unix-like system):
n/a

Description of the problem including expected versus actual behavior:
Remote _reindex fail for HTTPS remotes when remote does use custom CA or requires a client certificate. It seems there are several ssl.* properties missing. For me it should look like:

POST _reindex
{
  "source": {
    "remote": {
      "host": "https://otherhost:9200",
      "username": "user",
      "password": "pass",
      "ssl.certificate_authorities": ..., //trust custom CA's
      "ssl.verification_mode": ...,
      "ssl.certificate": ..., //client certificate for PKI auth
      "ssl.key": ..., //client certificate key for PKI auth
      "ssl.key_passphrase": ...,
    },
    "index":
    ...

The root cause is that TransportReindexAction.buildRestClient does not support custom trusted CA's nor allow the ability to authenticate the initiating cluster via PKI because its not capable of sending client certificates. Its also not possible to adjust the certificate verification mode.

If this is not going to be implemented there should be at least a note in the docs about this limitations here:

• https://www.elastic.co/guide/en/x-pack/current/security-limitations.html
• https://www.elastic.co/guide/en/x-pack/current/ccs-tribe-clients-integrations.html
• https://www.elastic.co/guide/en/elasticsearch/reference/6.0/docs-reindex.html

This also applies to ES 5.6 and earlier versions.

[javanna]

@nik9000 thoughts about this? Was this considered and does this make sense adding support for?

[floragunn]

@javanna @nik9000 ping

[PhaedrusTheGreek]

You can workaround this by adding the CA to the default Java CA Keystore:

IMPORTANT: Backup the $JAVA_HOME/jre/lib/security/cacerts file before doing this
IMPORTANT: Understand the security implications of doing this

sudo keytool -keystore $JAVA_HOME/jre/lib/security/cacerts -importcert -alias my_internal_ca -file /path/to/your/ca.crt

The default password to the cacerts keystore is changeit

[floragunn]

@PhaedrusTheGreek but that will not help in a case where i need to send a client certificate

[nik9000]

This makes sense to me as a thing to add but I'm not super clear on exactly how. Is the ssl certificate a string containing the certificate? I think making it a path to something on the Elasticsearch server's file system would get messy really fast.

[floragunn]

Yes, why not a string containing a pem certificate (or a certificate chain) and the key. Its not different from having the plaintext password inside the request.

[floragunn]

@nik9000 @PhaedrusTheGreek @javanna ping

[nik9000]

This isn't on my list of things to do, no. I don't disagree with doing it though.

[Constantin07]

I'm facing the same issue but I'm using official Docker container and adding the CA certificate in Java keystore is inconvenient since now I'm bind mounting the certificates (CA, server.key, server.crt) in PEM format at container run stage.

[bleskes]

@nik9000 I marked this as adopt me. Just in case someone wants to pick it up.

[hackery]

IMPORTANT: Understand the security implications of doing this

For example, don't do it if you're using demo certificates!

[jaymode]

Fixed by
#37527