////
This guide is maintained in the main Quarkus repository
and pull requests should be submitted there:
https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
////
= Using OpenTelemetry

include::./attributes.adoc[]

This guide explains how your Quarkus application can utilize https://opentelemetry.io/[OpenTelemetry] to provide
distributed tracing for interactive web applications.

== Prerequisites

:prerequisites-docker-compose:
include::includes/devtools/prerequisites.adoc[]

== Architecture

In this guide, we create a straightforward REST application to demonstrate distributed tracing.

== Solution

We recommend that you follow the instructions in the next sections and create the application step by step.
However, you can skip right to the completed example.

Clone the Git repository: `git clone {quickstarts-clone-url}`, or download an {quickstarts-archive-url}[archive].

The solution is located in the `opentelemetry-quickstart` {quickstarts-tree-url}/opentelemetry-quickstart[directory].

== Creating the Maven project

First, we need a new project. Create a new project with the following command:

:create-app-artifact-id: opentelemetry-quickstart
:create-app-extensions: resteasy,quarkus-opentelemetry-exporter-otlp
include::includes/devtools/create-app.adoc[]

This command generates the Maven project and imports the `quarkus-opentelemetry-exporter-otlp` extension,
which includes the OpenTelemetry support,
and a gRPC span exporter for https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md[OTLP].

If you already have your Quarkus project configured, you can add the `quarkus-opentelemetry-exporter-otlp` extension
to your project by running the following command in your project base directory:

:add-extension-extensions: opentelemetry-otlp-exporter
include::includes/devtools/extension-add.adoc[]

This will add the following to your build file:

[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
.pom.xml
----
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-opentelemetry-exporter-otlp</artifactId>
</dependency>
----

[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
.build.gradle
----
implementation("io.quarkus:quarkus-opentelemetry-exporter-otlp")
----

=== Examine the JAX-RS resource

Create a `src/main/java/org/acme/opentelemetry/TracedResource.java` file with the following content:

[source,java]
----
package org.acme.opentelemetry;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import org.jboss.logging.Logger;

@Path("/hello")
public class TracedResource {

    private static final Logger LOG = Logger.getLogger(TracedResource.class);

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        LOG.info("hello");
        return "hello";
    }
}
----

Notice that there is no tracing specific code included in the application. By default, requests sent to this
endpoint will be traced without any required code changes.

=== Create the configuration

There are two ways to configure the OTLP gRPC Exporter within the application.

The first approach is by providing the properties within the `src/main/resources/application.properties` file:

[source,properties]
----
quarkus.application.name=myservice // <1>
quarkus.opentelemetry.enabled=true // <2>
quarkus.opentelemetry.tracer.exporter.otlp.endpoint=http://localhost:4317 // <3>

quarkus.opentelemetry.tracer.exporter.otlp.headers=Authorization=Bearer my_secret // <4>
----

<1> All spans created from the application will include an OpenTelemetry `Resource` indicating the span was created by the `myservice` application. If not set, it will default to the artifact id.
<2> Whether OpenTelemetry is enabled or not. The default is `true`, but shown here to indicate how it can be disabled
<3> gRPC endpoint for sending spans
<4> Optional gRPC headers commonly used for authentication

== Run the application

The first step is to configure and start the https://opentelemetry.io/docs/collector/[OpenTelemetry Collector] to receive, process and export telemetry data to https://www.jaegertracing.io/[Jaeger] that will display the captured traces.

Configure the OpenTelemetry Collector by creating an `otel-collector-config.yaml` file:

[source,yaml,subs="attributes"]
----
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: otel-collector:4317
  otlp/2:
    protocols:
      grpc:
        endpoint: otel-collector:55680

exporters:
  jaeger:
    endpoint: jaeger-all-in-one:14250
    tls:
      insecure: true

processors:
  batch:

extensions:
  health_check:

service:
  extensions: [health_check]
  pipelines:
    traces:
      receivers: [otlp,otlp/2]
      processors: [batch]
      exporters: [jaeger]

----

Start the OpenTelemetry Collector and Jaeger system via the following `docker-compose.yml` file that you can launch via `docker-compose up -d`:

[source,yaml,subs="attributes"]
----
version: "2"
services:

  # Jaeger
  jaeger-all-in-one:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686"
      - "14268"
      - "14250"
  # Collector
  otel-collector:
    image: otel/opentelemetry-collector:latest
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - "13133:13133" # Health_check extension
      - "4317:4317"   # OTLP gRPC receiver
      - "55680:55680" # OTLP gRPC receiver alternative port
    depends_on:
      - jaeger-all-in-one
----

Now we are ready to run our application. If using `application.properties` to configure the tracer:

include::includes/devtools/dev.adoc[]

or if configuring the OTLP gRPC endpoint via JVM arguments:

:dev-additional-parameters: -Djvm.args="-Dquarkus.opentelemetry.tracer.exporter.otlp.endpoint=http://localhost:55680"
include::includes/devtools/dev.adoc[]
:!dev-additional-parameters:

With the OpenTelemetry Collector, Jaeger system and application running, you can make a request to the provided endpoint:

[source,shell]
----
$ curl http://localhost:8080/hello
hello
----

Then visit the http://localhost:16686[Jaeger UI] to see the tracing information.

Hit `CTRL+C` to stop the application.

== Additional configuration
Some use cases will require custom configuration of OpenTelemetry.
These sections will outline what is necessary to properly configure it.

=== ID Generator
The OpenTelemetry extension will use by default a random https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#id-generators[ID Generator]
when creating the trace and span identifier.

Some vendor-specific protocols need a custom ID Generator, you can override the default one by creating a producer.
The OpenTelemetry extension will detect the `IdGenerator` CDI bean and will use it when configuring the tracer producer.

[source,java]
----
@Singleton
public class CustomConfiguration {

    /** Creates a custom IdGenerator for OpenTelemetry */
    @Produces
    @Singleton
    public IdGenerator idGenerator() {
        return AwsXrayIdGenerator.getInstance();
    }
}
----

=== Propagators
OpenTelemetry propagates cross-cutting concerns through https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/context/api-propagators.md[propagators] that will share an underlying `Context` for storing state and accesing
data across the lifespan of a distributed transaction.

By default, the OpenTelemetry extension enables the https://www.w3.org/TR/trace-context/[W3C Trace Context] and the https://www.w3.org/TR/baggage/[W3C Baggage]
propagators, you can however choose any of the supported OpenTelemetry propagators by setting the `propagators` config that is described in the <<configuration-reference>>.

[NOTE]
====
The `b3`, `b3multi`, `jaeger` and `ottrace` propagators will need the https://github.com/open-telemetry/opentelemetry-java/tree/main/extensions/trace-propagators[trace-propagators]
extension to be added as a dependency to your project.

[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
.pom.xml
----
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-extension-trace-propagators</artifactId>
</dependency>
----

[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
.build.gradle
----
implementation("io.opentelemetry:opentelemetry-extension-trace-propagators")
----

The `xray` propagator will need the https://github.com/open-telemetry/opentelemetry-java/tree/main/extensions/aws[aws] extension to be added as a dependency to your project.

[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
.pom.xml
----
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-extension-aws</artifactId>
</dependency>
----

[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
.build.gradle
----
implementation("io.opentelemetry:opentelemetry-extension-aws")
----
====

=== Resource
A https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/overview.md#resources[resource] is a representation
of the entity that is producing telemetry, it adds attributes to the exported trace to characterize who is producing the trace.

You can add attributes by setting the `resource-attributes` tracer config that is described in the <<configuration-reference>>.
Since this property can be overridden at runtime, the OpenTelemetry extension will pick up its value following the order of precedence that
is described in the xref:config-reference.adoc#configuration_sources[Quarkus Configuration Reference].

If by any means you need to use a custom resource or one that is provided by one of the https://github.com/open-telemetry/opentelemetry-java/tree/main/sdk-extensions[OpenTelemetry SDK Extensions]
you can create multiple resource producers. The OpenTelemetry extension will detect the `Resource` CDI beans and will merge them when configuring the tracer producer.

[source,java]
----
@ApplicationScoped
public class CustomConfiguration {

    @Produces
    @ApplicationScoped
    public Resource osResource() {
        return OsResource.get();
    }

    @Produces
    @ApplicationScoped
    public Resource ecsResource() {
        return EcsResource.get();
    }
}
----

=== Sampler
A https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#sampling[sampler] decides whether
a trace should be sampled and exported, controlling noise and overhead by reducing the number of sample of traces collected and sent
to the collector.

You can set a https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#built-in-samplers[built-in sampler]
simply by setting the desired sampler config described in the <<configuration-reference>>.

If you need to use a custom sampler or to use one that is provided by one of the https://github.com/open-telemetry/opentelemetry-java/tree/main/sdk-extensions[OpenTelemetry SDK Extensions]
you can create a sampler producer. The OpenTelemetry extension will detect the `Sampler` CDI bean and will use it when configuring the tracer producer.

[source,java]
----
@Singleton
public class CustomConfiguration {

    /** Creates a custom sampler for OpenTelemetry */
    @Produces
    @Singleton
    public Sampler sampler() {
        return JaegerRemoteSampler.builder()
        .setServiceName("my-service")
        .build();
    }
}
----

== Additional instrumentation

Some Quarkus extensions will require additional code to ensure traces are propagated to subsequent execution.
These sections will outline what is necessary to propagate traces across process boundaries.

The instrumentation documented in this section has been tested with Quarkus and works in both standard and native mode.

=== SmallRye Reactive Messaging - Kafka

When using the SmallRye Reactive Messaging extension for Kafka,
we are able to propagate the span into the Kafka Record with:

[source,java]
----
Metadata.of(TracingMetadata.withPrevious(Context.current()));
----

The above creates a `Metadata` object we can add to the `Message` being produced,
which retrieves the OpenTelemetry `Context` to extract the current span for propagation.

[[configuration-reference]]
== OpenTelemetry Configuration Reference

include::{generated-dir}/config/quarkus-opentelemetry.adoc[leveloffset=+1, opts=optional]
include::{generated-dir}/config/quarkus-opentelemetry-exporter-otlp.adoc[leveloffset=+1, opts=optional]
