Notification System

Problem

Notification delivery across email and push channels needs to be reliable and observable even when downstream providers fail or degrade.

Approach

Built a NestJS monorepo of 5 independently deployable services — API Gateway, User, Email, Push, and Template — orchestrated via Docker Compose and reverse-proxied through Nginx.

Tech highlights

All inter-service communication via RabbitMQ AMQP with dead-letter queues and configurable retry logic; circuit breaker (opossum) on every downstream call with open/half-open/closed states; correlation ID injection at the gateway for full distributed tracing; Firebase Admin (FCM) for push; Prometheus + Grafana monitoring stack deployed from checked-in config; Swagger API docs.

Results

Shipped a working 5-service platform with dead-letter retry, circuit breaking, and a live Grafana monitoring stack.

Challenges

Guaranteeing at-least-once delivery without double-sending when a downstream email or push provider fails mid-flight — tuning dead-letter retry, circuit-breaker open/half-open thresholds, and idempotency keys so retries never notified a user twice — and threading correlation IDs through every service to make failures traceable in Grafana.