Software Engineer · intermediate

Production-Grade URL Shortener with Observability

A high-performance URL shortener with Redis caching, rate-limiting, and real-time analytics on a Grafana dashboard.

Project overview

Demonstrates the ability to move beyond simple CRUD apps by addressing real-world engineering challenges like concurrency, scalability, and reliability. Employers value candidates who know how to monitor and protect services in production.

Real-world use case

Marketing platforms (like Bitly) that need to track link engagement while maintaining sub-millisecond redirect latency for millions of users.

Tools and technologies

• Go/Python
• Redis
• PostgreSQL
• Prometheus
• Grafana

Skills you can demonstrate

System Design, Backend, Caching, Rate Limiting

Build roadmap

1. Design the schema
   PostgreSQL tables for URLs (short_code, long_url, created_at, owner) and clicks (timestamp, referrer, user_agent). Index the short code — that lookup is the whole product.
2. Build the core API
   In Go or Python: POST /shorten and GET /:code. Generate codes with base62 over a counter or random bytes, and handle collisions explicitly rather than hoping.
3. Add Redis caching
   Cache short-to-long mappings with a TTL. On a miss, read Postgres and warm the cache. Measure p99 latency before and after; that number goes in the README.
4. Implement rate limiting
   A Redis sliding-window limiter written as an atomic Lua script so concurrent requests cannot race past the limit. Return 429 with a Retry-After header.
5. Add Prometheus metrics
   Requests per second, cache hit rate, a redirect latency histogram, and error rate, exported on /metrics. Label by endpoint so the dashboard can split them.
6. Build the Grafana dashboard
   Panels for traffic, top links, latency, and errors. Run a k6 load test, screenshot the dashboard under load, and note the point where it starts to degrade.

Ways to extend the project

Integrate an AI-powered "Malicious URL Detector" using a lightweight model or Google Safe Browsing API to scan destination links before shortening.

How to showcase your work

• GitHub: Detailed README with architecture diagram (D2/Mermaid), load testing results (k6), and "Lessons Learned".
• LinkedIn: Screen recording of the Grafana dashboard showing real-time traffic spikes.
• Resume: Focus on system design, observability, and scalability.