Material study / Signal & orderMarket data.
Without the bottleneck.
2M+events per second
<500microseconds / p99
Market data. Without the bottleneck.
Real-time infrastructure / Fintech startup
- 01The constraint
- The existing Python pipeline dropped events at peak load. Analytics needed to keep pace with multiple exchanges, at sub-millisecond latency.
- 02The decision
- Move the hot path to Rust. Use Kafka for reliable event streaming, custom lock-free data structures for computation, and tiered storage for real-time and historical access.
- 03The trade-off
- Performance-critical code belongs where its complexity earns its keep. Kafka and PostgreSQL retain clear responsibilities outside the hot path.
- 04The outcome
- The resulting system processes over two million events per second, with p99 latency below 500 microseconds and no data loss reported since deployment.
