Real-Time Virtual Execution System
Sept 2024 — PresentPersistent low-latency trade state tracking for a trading signal intelligence platform.
Saras— CTO, Co-FounderDesigned and built a persistent real-time virtual execution system that continuously tracked advisor recommendations against live websocket market prices.
The architecture replaced an interval-based cron/Lambda pipeline with a stateful worker-driven system powered by Redis, MongoDB, websocket subscriptions, and automated recovery workflows.
The system dynamically managed active ticker pools, processed high-frequency tick streams, minimized execution latency, and maintained strong consistency between real-time state and long-term persistence layers.
System Overview
The platform ingests advisor recommendations from multiple sources and continuously tracks them against live market prices streamed through websocket feeds.
Each recommendation transitions through multiple execution states based on dynamic market conditions, targets, stoplosses, and expiry rules.
States: recommendation · entry · active · closed
To support real-time execution visibility, the system maintains active trade state in Redis while MongoDB acts as the durable source of truth.
The architecture was specifically designed to:
- reduce execution delay
- avoid redundant matching
- optimize websocket subscriptions
- recover automatically from outages
- maintain consistent state at scale
Architecture preview
Key Engineering Challenges
Handling ~1000 websocket ticks/sec without subscribing to the full market.
Replacing interval-based Lambda matching with persistent low-latency workers.
Maintaining synchronized execution state across Redis and MongoDB.
Designing automatic hindsight recovery after websocket/provider outages.
Managing dynamic ticker subscription pools at scale.
Tools & Technologies
Node.js
Redis
MongoDB
AWS EC2
WebSocket
Grafana
Prometheus