Cloud Computing · intermediate
Full Observability Stack with OpenTelemetry and Grafana LGTM
A sample application instrumented with OpenTelemetry sending logs, metrics, and traces through a Collector into Loki, Prometheus, and Tempo — with Grafana dashboards where you can click from a slow request to its trace to its logs.
Project overview
OpenTelemetry is the second-fastest-growing project in the CNCF after Kubernetes and has become the vendor-neutral standard for instrumentation. SRE, platform, and cloud postings list it by name, and a working three-signal stack is rare among junior candidates.
Real-world use case
An e-commerce team gets a "checkout is slow" complaint. With correlated telemetry they jump from the latency spike on a dashboard to the exact trace to the database log line in under a minute.
Tools and technologies
- OpenTelemetry SDK + Collector
- Grafana
- Loki
- Tempo
- Prometheus
- Docker Compose
Skills you can demonstrate
Observability Engineering, Distributed Tracing, Metrics and Alerting, Log Aggregation, Telemetry Pipelines
Build roadmap
- Stand up the backends
Docker Compose with Loki, Tempo, Prometheus, and Grafana. Confirm each is healthy and added as a Grafana data source before writing any app code. - Instrument a small service
Build a two-endpoint API that calls a database and an external API. Add OpenTelemetry auto-instrumentation and confirm spans reach Tempo. - Route everything through the Collector
Send OTLP from the app to the Collector, and from the Collector to the three backends. Add batch and resource processors. - Correlate the signals
Inject trace IDs into logs and configure Grafana derived fields so logs link to traces and traces link to logs. Test the round trip. - Build the dashboards
Create a RED dashboard (rate, errors, duration) per endpoint and a service map. Provision them from JSON files in the repo. - Break it on purpose
Add a slow path and an error path, generate load, and write a short incident walkthrough showing how you found each using only the stack.
Ways to extend the project
Add exemplars so Prometheus histogram points link directly to Tempo traces, and define an SLO with a burn-rate alert in Grafana.
How to showcase your work
- GitHub: The Compose file, Collector config, instrumented app, and provisioned Grafana dashboards as JSON so anyone can clone and run it in one command.
- LinkedIn: Record the click-through: dashboard spike, to trace waterfall, to the log line that explains it. That 30 seconds is the pitch.
- Resume: Lead with "OpenTelemetry instrumentation" and "Grafana LGTM stack", then name the three signals explicitly.