title | excerpt | updated |
---|---|---|
Distributed tracing with Jaeger on an OVHcloud Managed Kubernetes Service |
Find out how to set up distributed tracing on OVHcloud Managed Kubernetes Service with Jaeger. |
2022-03-22 |
Jaeger is an open-source distributed tracing platform.
It can be used for monitoring microservices-based distributed systems:
- Distributed context propagation
- Distributed transaction monitoring
- Root cause analysis
- Service dependency analysis
- Performance / latency optimization
Jaeger contains several components:
Read more about Jaeger architecture and components.
In this guide you will:
- Install Jaeger Operator
- Deploy Jaeger components
- Access to the UI
- Deploy your instrumented application
- Visualize traces
You can use the Reset cluster function in the Public Cloud section of the OVHcloud Control Panel{.external} to reinitialize your cluster before following this tutorial.
This tutorial presupposes that you already have a working OVHcloud Managed Kubernetes cluster, and some basic knowledge of how to operate it. If you want to know more on those topics, please look at the OVHcloud Managed Kubernetes Service Quickstart.
You also need to have Helm{.external} installed on your workstation and your cluster. Please refer to the How to install Helm on OVHcloud Managed Kubernetes Service tutorial.
In this guide we will show you how to deploy Jaeger as distributed tracing platform backend and then how to deploy your instrumented application that will send their traces to Jaeger.
For this tutorial we are using the Jaeger Helm chart.
Add the Jaeger Helm repository:
helm repo add jaegertracing https://jaegertracing.github.io/helm-charts
helm repo update
These commands will add the Jaeger Helm repository to your local Helm chart repository and update the installed chart repositories:
$ helm repo add jaegertracing https://jaegertracing.github.io/helm-charts
helm repo update
"jaegertracing" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "sealed-secrets" chart repository
...
...Successfully got an update from the "prometheus-community" chart repository
Update Complete. ⎈Happy Helming!⎈
The Jaeger repository provides two charts: jaeger
and jaeger-operator
. For the guide, you will deploy the jaeger-operator chart, which makes it easy to configure a minimal installation.
To learn more about the Jaeger Operator for Kubernetes, consult the official documentation.
Install the latest version of Jaeger with helm install
command:
helm install jaeger-operator jaegertracing/jaeger-operator --namespace observability --create-namespace --set rbac.clusterRole=true
This command will install the latest version of Jaeger operator and observability
namespace:
$ helm install jaeger-operator jaegertracing/jaeger-operator --namespace observability --create-namespace --set rbac.clusterRole=true
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
NAME: jaeger-operator
LAST DEPLOYED: Thu Mar 17 12:06:49 2022
NAMESPACE: observability
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
jaeger-operator is installed.
Check the jaeger-operator logs
export POD=$(kubectl get pods -l app.kubernetes.io/instance=jaeger-operator -lapp.kubernetes.io/name=jaeger-operator --namespace observability --output name)
kubectl logs $POD --namespace=observability
Thanks to the variable overriding rbac.clusterRole=true
, you ask the operator to watch all namespaces.
Check the Jaeger Operator is running:
kubectl get pod -n observability
$ kubectl get pod -n observability
NAME READY STATUS RESTARTS AGE
jaeger-operator-67f8dd68c9-5qj26 1/1 Running 0 3m5s
The simplest possible way to create a Jaeger instance is by creating a YAML file that will install the default AllInOne image. This “all-in-one” image includes: agent, collector, query, ingester and Jaeger UI in a single pod, using in-memory storage by default.
For this guide you will deploy Jaeger components through this simple way, which can be used for development, testing and demo purposes but for production strategy you can read the official documentation.
Once the jaeger-operator
pod in the namespace observability
is ready, create a jaeger.yaml
file with the following content:
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
name: jaeger
spec:
query:
serviceType: LoadBalancer
[!primary]
In this YAML manifest file we specify that you want to access the Jaeger UI (
jaeger-query
) through a Load Balancer.
And apply it:
kubectl apply -f jaeger.yaml
These commands will create a new CRD Jaeger
and an instance named jaeger
:
$ kubectl apply -f jaeger.yaml
jaeger.jaegertracing.io/jaeger created
You can now check if the Jaeger instance is running with the following commands:
kubectl get jaeger
kubectl get pods -l app.kubernetes.io/instance=jaeger
Theses commands will check if the instances were created, list the Jaeger objects and list the pods that are running:
$ kubectl get jaeger
NAME STATUS VERSION STRATEGY STORAGE AGE
jaeger Running 1.30.0 allinone memory 4s
$ kubectl get pods -l app.kubernetes.io/instance=jaeger
NAME READY STATUS RESTARTS AGE
jaeger-59ccc99bcc-zpscb 1/1 Running 0 80s
You can also check that all Jaeger services have been correctly deployed:
$ kubectl get svc -l app=jaeger
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jaeger-agent ClusterIP None <none> 5775/UDP,5778/TCP,6831/UDP,6832/UDP 3d21h
jaeger-collector ClusterIP 10.3.197.39 <none> 9411/TCP,14250/TCP,14267/TCP,14268/TCP 3d21h
jaeger-collector-headless ClusterIP None <none> 9411/TCP,14250/TCP,14267/TCP,14268/TCP 3d21h
jaeger-query LoadBalancer 10.3.114.168 51.210.210.101 16686:30598/TCP,16685:30835/TCP 3d21h
Now you can retrieve Jaeger UI URL with the following command:
export JAEGER_URL=$(kubectl get svc jaeger-query -o jsonpath='{.status.loadBalancer.ingress[].ip}')
echo Jaeger URL: http://$JAEGER_URL:16686
You should obtain the following result:
$ export JAEGER_URL=$(kubectl get svc jaeger-query -o jsonpath='{.status.loadBalancer.ingress[].ip}')
$ echo Jaeger URL: http://$JAEGER_URL:16686
Jaeger URL: http://51.210.210.101:16686
Open your browser and go to the Jaeger interface.
In order to link your application to the Jaeger backend you need to use a tool like OpenTelemetry.
OpenTelemetry is a collection of tools, APIs, and SDKs. Useful to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior.
OpenTelemetry integrates with popular libraries and frameworks such as Spring, Express, Quarkus, and with a lot of languages. Go to the documentation to see how to integrate your application.
For this guide you will deploy a Golang application, instrumented with OpenTelemetry, that will send traces to a provider: your Jaeger collector.
Our main.go
file contains:
- the import of the OpenTelemetry dependencies,
- a
tracerProvider
method that initiates a connection to a Jaeger provider - a
main()
method that connects to the Jaeger collector you previously deployed and creates and sends a span each time the/
HTTP route will be called
package main
import (
"context"
"fmt"
"log"
"net/http"
"os"
"time"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/exporters/jaeger"
"go.opentelemetry.io/otel/sdk/resource"
tracesdk "go.opentelemetry.io/otel/sdk/trace"
semconv "go.opentelemetry.io/otel/semconv/v1.7.0"
)
const (
service = "go-what-is-my-pod-with-tracing"
environment = "development"
id = 1
)
func tracerProvider(url string) (*tracesdk.TracerProvider, error) {
// Create the Jaeger exporter
exp, err := jaeger.New(jaeger.WithCollectorEndpoint(jaeger.WithEndpoint(url)))
if err != nil {
return nil, err
}
tp := tracesdk.NewTracerProvider(
// Always be sure to batch in production.
tracesdk.WithBatcher(exp),
// Record information about this application in a Resource.
tracesdk.WithResource(resource.NewWithAttributes(
semconv.SchemaURL,
semconv.ServiceNameKey.String(service),
attribute.String("environment", environment),
attribute.Int64("ID", id),
)),
)
return tp, nil
}
func main() {
// Tracer
tp, err := tracerProvider("http://jaeger-collector-headless.default.svc.cluster.local:14268/api/traces")
if err != nil {
log.Fatal(err)
}
// Register our TracerProvider as the global so any imported
// instrumentation in the future will default to using it.
otel.SetTracerProvider(tp)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Cleanly shutdown and flush telemetry when the application exits.
defer func(ctx context.Context) {
// Do not make the application hang when it is shutdown.
ctx, cancel = context.WithTimeout(ctx, time.Second*5)
defer cancel()
if err := tp.Shutdown(ctx); err != nil {
log.Fatal(err)
}
}(ctx)
tr := tp.Tracer("component-main")
ctx, span := tr.Start(ctx, "hello")
defer span.End()
// HTTP Handlers
helloHandler := func(w http.ResponseWriter, r *http.Request) {
// Use the global TracerProvider
tr := otel.Tracer("hello-handler")
_, span := tr.Start(ctx, "hello")
span.SetAttributes(attribute.Key("mykey").String("value"))
defer span.End()
podName := os.Getenv("MY_POD_NAME")
fmt.Fprintf(w, "Hello %q!", podName)
}
otelHandler := otelhttp.NewHandler(http.HandlerFunc(helloHandler), "Hello")
http.Handle("/", otelHandler)
log.Println("Listening on localhost:8080")
log.Fatal(http.ListenAndServe(":8080", nil))
}
[!primary]
The code source of the application is available on GitHub.
We already packaged a Golang application into a Docker image and pushed it in our ovhplatform Docker Hub repository so you can use it directly.
In order to deploy the application on your OVHcloud managed Kubernetes Service, create a deployment.yaml
file with the following content:
apiVersion: apps/v1
kind: Deployment
metadata:
name: what-is-my-pod-with-tracing-deployment
labels:
app: what-is-my-pod-with-tracing
spec:
replicas: 3
selector:
matchLabels:
app: what-is-my-pod-with-tracing
template:
metadata:
labels:
app: what-is-my-pod-with-tracing
spec:
containers:
- name: what-is-my-pod-with-tracing
image: ovhplatform/what-is-my-pod-with-tracing:1.0.2
ports:
- containerPort: 8080
env:
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
This YAML deployment manifest file defines that our application, based on ovhplatform/what-is-my-pod-with-tracing:1.0.2
image will be deployed with 3 replicas (3 pods). We pass the pod name on environment variable in order to display it in our what-is-my-pod-with-tracing application.
Then, create a svc.yaml
file with the following content to define our service (a service exposes a deployment):
apiVersion: v1
kind: Service
metadata:
labels:
app: what-is-my-pod-with-tracing
name: what-is-my-pod-with-tracing
spec:
ports:
- port: 8080
selector:
app: what-is-my-pod-with-tracing
type: LoadBalancer
Apply the deployment and service manifest files to your cluster with the following commands:
kubectl apply -f deployment.yaml
kubectl apply -f svc.yaml
Output should be like this:
$ kubectl apply -f deployment.yml
deployment.apps/what-is-my-pod-with-tracing-deployment created
$ kubectl apply -f svc.yml
service/what-is-my-pod-with-tracing created
You can verify if your application is running and service is created by running the following commands:
kubectl get pod -l app=what-is-my-pod-with-tracing
kubectl get svc -l app=what-is-my-pod-with-tracing
Output should be like this:
$ kubectl get pod -l app=what-is-my-pod-with-tracing
NAME READY STATUS RESTARTS AGE
what-is-my-pod-with-tracing-deployment-84b56684d8-6kw6z 1/1 Running 0 3m
what-is-my-pod-with-tracing-deployment-84b56684d8-bcsxh 1/1 Running 0 3m
what-is-my-pod-with-tracing-deployment-84b56684d8-wbjmz 1/1 Running 0 3m
$ kubectl get svc -l app=what-is-my-pod-with-tracing
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
what-is-my-pod-with-tracing LoadBalancer 10.3.118.87 135.125.84.198 8080:32365/TCP 3m
In order to generate traffic you need to get the external IP of your service:
export APP_URL=$(kubectl get svc what-is-my-pod-with-tracing -o jsonpath='{.status.loadBalancer.ingress[].ip}')
echo Application URL: http://$APP_URL:8080/
And then generate traffic with curl command:
curl http://$APP_URL:8080/
You should obtain the following result:
$ export APP_URL=$(kubectl get svc what-is-my-pod-with-tracing -o jsonpath='{.status.loadBalancer.ingress[].ip}')
$ echo $APP_URL
135.125.84.198
$ curl http://$APP_URL:8080/
Hello "what-is-my-pod-with-tracing-deployment-84b56684d8-6kw6z"!%
$ curl http://$APP_URL:8080/
Hello "what-is-my-pod-with-tracing-deployment-84b56684d8-6kw6z"!%
$ curl http://$APP_URL:8080/
Hello "what-is-my-pod-with-tracing-deployment-84b56684d8-wbjmz"!%
Open your browser and go back to the Jaeger interface (http://$JAEGER_URL:16686).
You should now see two available services:
- go-what-is-my-pod-with-tracing
- jaeger-query
Select go-what-is-my-pod-with-tracing
service and click on Find Traces
{.action} button.
You can now click in a trace and visualize useful information.
Delete Jaeger components (created by the operator):
kubectl delete -f jaeger.yaml
Wait until the components are deleted and then you can uninstall the operator.
To uninstall Jaeger Operator, as you installed it through Helm, you can use helm uninstall
command in order to delete the Jaeger Helm installed chart:
helm uninstall jaeger-operator -n observability
Delete the observability
namespace:
kubectl delete ns observability
And delete the deployed application:
kubectl delete -f deployment.yaml
kubectl delete -f svc.yaml
-
If you need training or technical assistance to implement our solutions, contact your sales representative or click on this link to get a quote and ask our Professional Services experts for assisting you on your specific use case of your project.
-
Join our community of users.