== Spring Boot Example with Camel Observability Services

=== Introduction

This example illustrates how to use https://projects.spring.io/spring-boot/[Spring Boot] with http://camel.apache.org[Camel] to demonstrate observability capabilities. It implements a REST service that generates random numbers with variable response times, showcasing distributed tracing, metrics collection, and monitoring through OpenTelemetry.

The project uses `camel-observability-services-starter` component for automatic instrumentation, `camel-platform-http-starter` for REST endpoints, and integrates with OpenTelemetry Java agent for comprehensive observability.

For additional information there is https://camel.apache.org/blog/2025/03/camel-observability[blog post]

=== Observability Stack

To run the complete observability stack with Jaeger and Prometheus:

[source,bash]
----
docker-compose up
----

This will start:

 - OpenTelemetry Collector (ports 4317/4318) - collects traces and metrics
 - Jaeger UI (http://localhost:16686) - distributed tracing visualization
 - Prometheus (http://localhost:9090) - metrics collection and storage

The application, using agent, sends telemetry data to the OpenTelemetry Collector, which exports traces to Jaeger and metrics to Prometheus.

=== Run

You can run this example using:

[source,bash]
----
mvn spring-boot:run
----

After the Spring Boot application is started, you can execute the following HTTP requests:

[source,bash]
----
curl http://localhost:8080/api/random
----

The command will call the random number generator endpoint. Each call will:
- Generate a random number between 1-1000
- Simulate a random delay between 100-2000ms
- Return the number as response body
- Create traces and metrics for observability

You should see output similar to:

----
INFO 101511 --- [         task-1] generate-random-number                                   : Generated random number: 742 with delay: 1250ms
----

=== Exposed Observability endpoints

The `camel-observability-services-starter` will configure the application and overrides the actuator setup so the health and metrics (in prometheus format) endpoints will be

```
http://localhost:9876/observe/health

http://localhost:9876/observe/metrics
```

even if in this example the metrics will be distributed using OpenTelemetry agent and not scraping the `/observe/metrics` endpoint

=== Testing Observability

1. **Generate some traffic:**
[source,bash]
----
for i in {1..10}; do curl http://localhost:8080/api/random; echo; sleep 1; done
----

2. **View traces in Jaeger:**
   - Open http://localhost:16686
   - Select service and search for traces

3. **View metrics in Prometheus:**
   - Open http://localhost:9090
   - Query metrics like `camel_route_exchange_completed_total`


The Spring Boot application can be stopped pressing `[CTRL] + [C]` in the shell.

== Install example on OpenShift

==== Requirements

- `oc` client installed (https://docs.openshift.com/container-platform/latest/cli_reference/openshift_cli/getting-started-cli.html[guide])
- already logged in into cluster (running `oc login`)
- destination project already created (running `oc new-project obs-example`)

==== Install operators

- Red Hat build of OpenTelemetry https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html/red_hat_build_of_opentelemetry/install-otel#installing-otel-by-using-the-web-console_install-otel[doc]
- Tempo Operator https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html/distributed_tracing/distr-tracing-tempo-installing#distr-tracing-tempo-install-web-console_distr-tracing-tempo-installing[doc]
- Cluster Observability Operator https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html/cluster_observability_operator/installing-cluster-observability-operators#installing-the-cluster-observability-operator-in-the-web-console-_installing_the_cluster_observability_operator[doc]

==== Create resources

Create tempo monolithic named `monolitic-example` as distributed tracing storage:

[source,sh]
----
oc apply -f ocp/tempo.yaml
----

Create roles to send traces via Collector, to the Tempo

[source,sh]
----
oc apply -f ocp/roles.yaml
----

Create opentelemetry collector named `obs-example` used to collect everything coming from the agent on the applications and exporting the traces into the tempo storage and the metrics in a prometheus format:

[source,sh]
----
oc apply -f ocp/otel-collector.yaml
----

Create the opentelementry instrumentation to allow to inject the agent parameter into the startup command in the application

[source,sh]
----
oc apply -f ocp/otel-instrumentation.yaml
----

Install the distributed tracing UI plugin to show the traces on the OCP console https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html/cluster_observability_operator/observability-ui-plugins#coo-distributed-tracing-ui-plugin-install_distributed-tracing-ui-plugin[doc]

[source,sh]
----
oc apply -f ocp/ui-plugin.yaml
----

Create the service monitor to allow the prometheus metrics exposed by the opentelemetry collector be scraped by the OpenShift metrics storage

[source,sh]
----
oc apply -f ocp/service-monitor.yaml
----

==== Deploy the application

```
mvn clean install -Popenshift
```

now once the pods are ready it is possible to call the service

```
curl http://$(oc get route camel-example-spring-boot-observability-services -o go-template --template='{{.spec.host}}')/api/random
```

==== Show traces and metrics

The Distributed Tracing console is available on the OpenShift console in the `Observe -> Traces` item

```
echo $(oc whoami --show-console)/observe/traces
```

To query the metrics it is possible to use the integrated OpenShift monitoring console in the `Observe -> Metrics` item

```
echo $(oc whoami --show-console)/monitoring/query-browser
```

example query
```
camel_route_exchange_completed_total{service="obs-example-collector-headless", context="camel-observability-services"}
```

=== Help and contributions

If you hit any problem using Camel or have some feedback, then please
https://camel.apache.org/community/support/[let us know].

We also love contributors, so
https://camel.apache.org/community/contributing/[get involved] :-)

The Camel riders!