Home · Work · Data Harbor

Case Study

Data Harbor

Architecture, DevOps · Analytics

Analytics dashboards tracking the Data Harbor event-driven pipeline

Architecture, DevOps · Analytics

Data Harbor

Event-driven analytics on managed streaming and warehouse services, with OpenTelemetry tracing end to end and per-team cost dashboards that ended every billing surprise.

2B+Events per day
99.95%Pipeline uptime
12Teams with cost views

The challenge

Two billion events a day landing in batch jobs that failed silently, with one shared bill nobody could attribute to a team.

How I approached it

  • Managed streaming ingestion with schema contracts and replay
  • Warehouse modeling with data quality checks on every load
  • OpenTelemetry tracing from event to dashboard
  • Per-team cost views with anomaly alerts

The outcome

Pipeline uptime at 99.95% and twelve teams finally able to see — and own — what they spend.

Stack: Managed streaming · Warehouse · OpenTelemetry · Terraform

Start your project