Skip to content

JVM debug logs are not redirerected to the elasticsearch log #50156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jkakavas opened this issue Dec 12, 2019 · 4 comments · Fixed by #50259
Closed

JVM debug logs are not redirerected to the elasticsearch log #50156

jkakavas opened this issue Dec 12, 2019 · 4 comments · Fixed by #50259
Labels
:Core/Infra/Logging Log management and logging utilities >docs General docs changes :Security/Authentication Logging in, Usernames/passwords, Realms (Native/LDAP/AD/SAML/PKI/etc)

Comments

@jkakavas
Copy link
Member

JVM debug logs ( such as the logs produced by setting -Dsun.security.krb5.debug=true or
-Dsun.security.spnego.debug=true ) are only printed in stdout and are not redirected to the elasticsearch log.

These are crucial for Kerberos troubleshooting as these are the only actionable information we can get from Java GSS. In general though, it might also make sense for other JVM related logs to be redirected to elasticsearch.log and we could add a proxy in code to make writing to stdout go to the log.

At a bare minimum, we should update https://www.elastic.co/guide/en/elasticsearch/reference/7.5/trb-security-kerberos.html to point out that after enabling the debug logs, these logs can be found in stdout when running elasticsearch with the bin/elasticsearch script , in the systemd journal for elasticsearch.service when running with systemd and in docker logs when running elasticsearch in docker container.
@jkakavas jkakavas added >docs General docs changes :Core/Infra/Logging Log management and logging utilities :Security/Authentication Logging in, Usernames/passwords, Realms (Native/LDAP/AD/SAML/PKI/etc) labels Dec 12, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-security (:Security/Authentication)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Logging)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-docs (>docs)

@albertzaharovits
Copy link
Contributor

Good catch 👍
Given the general direction of formatting the logs to be machine readable, as JSON documents, I would lean towards documenting it to avoid of spoiling it for the parsers.

rjernst added a commit to rjernst/elasticsearch that referenced this issue Dec 17, 2019
@rjernst
Capture stdout and stderr to log4j log
37c2672 
This commit overrides the stdout and stderr print streams to be
redirected to the main elasticsearch.log file. While the Elasticsearch
project ensures stdout and stderr are not written to, the jdk or 3rd
party libs may do this, which can be unexepected for users used to
looking the elasticsearch log.

closes elastic#50156
@rjernst rjernst mentioned this issue Dec 17, 2019
Merged
rjernst added a commit that referenced this issue Jan 25, 2020
@rjernst
Capture stdout and stderr to log4j log (#50259)
0ec74f8 
This commit overrides the stdout and stderr print streams to be
redirected to the main elasticsearch.log file. While the Elasticsearch
project ensures stdout and stderr are not written to, the jdk or 3rd
party libs may do this, which can be unexepected for users used to
looking the elasticsearch log.

closes #50156
rjernst added a commit that referenced this issue Jan 25, 2020
@rjernst
Capture stdout and stderr to log4j log (#50259)
a564cac 
This commit overrides the stdout and stderr print streams to be
redirected to the main elasticsearch.log file. While the Elasticsearch
project ensures stdout and stderr are not written to, the jdk or 3rd
party libs may do this, which can be unexepected for users used to
looking the elasticsearch log.

closes #50156
@codebrain codebrain mentioned this issue Apr 1, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Core/Infra/Logging Log management and logging utilities >docs General docs changes :Security/Authentication Logging in, Usernames/passwords, Realms (Native/LDAP/AD/SAML/PKI/etc)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Capture stdout and stderr to log4j log rjernst/elasticsearch
3 participants
@albertzaharovits @jkakavas @elasticmachine