Skip to content

Enable Log Aggregation for Trino #376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
5 tasks done
Tracked by #288
maltesander opened this issue Feb 3, 2023 · 1 comment
Closed
5 tasks done
Tracked by #288

Enable Log Aggregation for Trino #376

maltesander opened this issue Feb 3, 2023 · 1 comment
Assignees
Labels
release/23.4.0 release-note Denotes a PR that will be considered when it comes time to generate release notes. type/feature-new

Comments

@maltesander maltesander added release-note Denotes a PR that will be considered when it comes time to generate release notes. type/feature-new release/23.4.0 labels Feb 3, 2023
@maltesander maltesander self-assigned this Feb 6, 2023
@maltesander maltesander moved this to Development: In Progress in Stackable Engineering Feb 6, 2023
bors bot pushed a commit that referenced this issue Feb 7, 2023
# Description

- prerequisite for #376



Co-authored-by: Malte Sander <[email protected]>
@maltesander
Copy link
Member Author

maltesander commented Feb 7, 2023

In terms of Research (thanks to @lfrancke). Just to collect the results.

We managed to get logback working. This required adding and removing some jars from our docker image (using Trino v403):

### Log Testing
RUN curl -L https://repo1.maven.org/maven2/org/slf4j/jul-to-slf4j/1.7.36/jul-to-slf4j-1.7.36.jar -o /stackable/trino-server/lib/jul-to-slf4j-1.7.36.jar && chmod -x /stackable/trino-server/lib/jul-to-slf4j-1.7.36.jar && \
    curl -L https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.2.11/logback-classic-1.2.11.jar -o /stackable/trino-server/lib/logback-classic-1.2.11.jar && chmod -x /stackable/trino-server/lib/logback-classic-1.2.11.jar

RUN rm /stackable/trino-server/lib/slf4j-jdk14-1.7.36.jar && \
    rm /stackable/trino-server/lib/log4j-api-2.17.1.jar
### End test

Additional extending the jvm.config with:

-Djava.util.logging.config.file=/stackable/logging.properties
-Dlogback.configurationFile=/stackable/trino-server/lib/logback.xml

where the logging.properties and logback.xml were just hardcoded in the operator for testing.
The logging.properties:

handlers=org.slf4j.bridge.SLF4JBridgeHandler

and logback.xml:

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>[LOGBACK START] timestamp: %d{yyyy-MM-dd HH:mm:ss.SSS}, thread: %thread, level: %-5level, logger: %logger{36}, message: %msg%n [LOGBACK END]</pattern>
    </encoder>
  </appender>
  <root level="info">
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

This resulted in the logging messages being nested (prefixed with the StaticLoggingFormat):

2023-02-07T15:51:31.059Z	INFO	main	stdout	[LOGBACK START] timestamp: 2023-02-07 15:51:31.058, thread: main, level: INFO , logger: o.e.jetty.util.ssl.SslContextFactory, message: x509=X509@41271320(1,h=[simple-trino-coordinator-default.default.svc.cluster.local, simple-trino-coordinator-default-0.simple-trino-coordinator-default.default.svc.cluster.local, kind-worker, generated certificate for pod],a=[/172.18.0.4],w=[]) for Server@1de398c3[provider=null,keyStore=null,trustStore=null]
 [LOGBACK END]

This is way too hacky and requires changes upstream. For now we decided to use JSON logging for the server logs and pick up the bootstrap logs from the console.

bors bot pushed a commit that referenced this issue Feb 8, 2023
# Description

- prerequisite for #376



Co-authored-by: Malte Sander <[email protected]>
bors bot pushed a commit to stackabletech/operator-rs that referenced this issue Feb 10, 2023
## Description

- Adding source for airlift json logging and airlift json transform (Trino)

prerequisite for stackabletech/trino-operator#376




Co-authored-by: Malte Sander <[email protected]>
@maltesander maltesander moved this from Development: In Progress to Development: Waiting for Review in Stackable Engineering Feb 13, 2023
@siegfriedweber siegfriedweber moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Feb 14, 2023
@maltesander maltesander moved this from Development: In Review to Development: Done in Stackable Engineering Feb 22, 2023
@bors bors bot closed this as completed in 14717ac Feb 22, 2023
@lfrancke lfrancke moved this from Development: Done to Acceptance: In Progress in Stackable Engineering Feb 22, 2023
@lfrancke lfrancke moved this from Acceptance: In Progress to Done in Stackable Engineering Feb 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release/23.4.0 release-note Denotes a PR that will be considered when it comes time to generate release notes. type/feature-new
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant