Prepaire Outbreak Intelligence Platform
A concurrent ETL platform aggregating disease outbreak intelligence from five authoritative sources into geocoded, queryable alerts delivered through a secure API.
The Problem
Outbreak intelligence was spread across multiple public health sources with different formats and update patterns. Analysts had to monitor channels manually, which slowed response and reduced signal quality. Decision-makers needed consolidated, timely visibility.
Two gaps made the manual approach untenable. Sources publish on their own schedules in inconsistent formats, so nobody could watch them all continuously. And most alerts arrive with place names rather than coordinates, which blocks any kind of geographic analysis or map-based response planning until someone resolves them by hand.
What We Built
We built a multi-source health intelligence pipeline that continuously ingests, normalizes, and serves outbreak data from authoritative feeds. The platform exposes unified access patterns so teams can monitor global changes from one interface. It reduces source-by-source monitoring overhead and improves consistency.
All five sources are collected concurrently rather than in sequence, which is what makes near-real-time monitoring practical. Each alert is geocoded during ingestion and normalized into a single schema, so downstream consumers work with consistent, mappable records regardless of which source produced them.
Concurrent multi-source ingestion
Five authoritative health sources are collected in parallel rather than sequentially, cutting the collection window enough to make continuous monitoring practical.
Automated geocoding
Scraped place names resolve to precise coordinates during ingestion, with caching to avoid redundant lookups—turning text mentions into mappable, analyzable locations.
Unified alert schema
Disparate source formats normalize into one standardized structure covering disease, indicators, location, case counts, and provenance, so consumers integrate once.
Resilient scraping with retries
Exponential-backoff retry logic and HTML sanitization handle unreliable upstream sources and multi-language content without dropping alerts.
How It Works in Practice
A dengue outbreak emerging in Southeast Asia, where response speed depends on how quickly signals surface and how well they can be corroborated.
- 1
Detect at the source
A monitoring source publishes an initial cluster of cases in a metropolitan area. The ingestion job picks it up on its next run rather than waiting for an analyst to check that channel.
- 2
Geocode and normalize
The location resolves to precise coordinates and the alert is written into the unified schema with disease, indicators, case and fatality counts, and source attribution attached.
- 3
Corroborate across sources
As other authorities report the same event, alerts cross-reference automatically—so a signal confirmed by multiple independent sources is distinguishable from a single unverified report.
- 4
Serve to responders
Authenticated consumers query the unified feed or stream updates in real time, working from one consistent dataset instead of monitoring five sources separately.
Engineering Approach
Concurrency where the latency actually is
Collection time was dominated by waiting on five independent sources, so running them in parallel rather than sequence cut the wall-clock window by roughly five times without adding infrastructure.
Built for unreliable upstreams
Public health sources change format, rate-limit, and go down. Retry with exponential backoff and content sanitization treat that as the normal case rather than an exception path.
Normalize at ingestion, not at read
Source differences are resolved into one alert schema as data arrives, so every downstream consumer works against a consistent, already-geocoded record instead of each reimplementing translation logic.
Have a similar problem?
Tell us what you're working on and we'll tell you honestly whether we can help.
