Multidimensional Market Intelligence Engine
Realtime multidimensional computation infrastructure for filtering, ranking, scoring, and serving continuously mutating market intelligence.
Low latency
<150ms p95 response
High throughput
10K+ req/sec
Active trades
1M+ tracked realtime
Advisor intelligence
5K+ advisors scored
Realtime · Incremental · Intelligent
Live prices
Trade universe
Ranking computation
Advisor intelligence
Query orchestrator
01
Query explosion
Nine dimensions compound combinatorially — each permutation is a new aggregation path under live prices.
Period
Intraday · Swing · Positional
Category
Index Opt · Stock Opt · Futures · Equity
Position
Buy · Sell · Hold
Source
Telegram · Report · X · YouTube
Trade status
Open · Closed · Pending
Date range
7D · 30D · 90D · Custom
Profit potential
Upside % · R:R · Absolute
Advisor accuracy
Rolling · Segmented
Sorting
Upside · Recent · Accuracy
Millions of possible query paths
Failure mode
Why traditional caching failed
Static keys break when filters × live prices explode into unrelated aggregation paths — the pivot from cache-first to computation-first.
Filter change
Single filter change
Cache invalidation
New aggregation path
Market mutation
Live price mutation
Ranking mutation
Continuous recomputation pressure
- Cache hit rate collapsed under combinatorial paths
- Mongo aggregation CPU spiked on hot queries
- Expensive recomputation on every meaningful change
- Throwing EC2 at the problem could not scale fast enough
Orchestration
Redis computation architecture
Redis as a layered computation substrate — sorted sets, sets, and hashes composing live ranking and intelligence under continuous mutation.
Layer 1
Live price layer
price:{ticker}cmp:{ticker}Live market state from tick feeds.
Layer 2
Trade state layer
trade:{id}trade:opentrade:category:{x}trade:source:{x}Trade filtering & grouping.
Layer 3
Ranking layer
trade:rank:upsidetrade:rank:accuracytrade:rank:relevanceRealtime ranking computation.
Layer 4
Advisor intelligence layer
advisor:perf:{id}Fields — accuracy · rolling consistency · risk score · realized performance · unrealized performance
Dynamic advisor intelligence scoring.
Layer 5
Reference layer
category:{id}period:{id}advisor:{id}Static reference & metadata — advisor info, ticker mappings, category references.
Execution
Query execution flow
Set intersections and sorted-set operations first — hydrate documents only at the boundary.
SINTER
Intersect filter sets → candidate IDs
Candidate IDs
Filtered IDs only — no heavy objects yet
ZINTERSTORE
Rank overlay via sorted-set intersection
Rank overlay
Merge ranking scores for candidates
ZREVRANGE
Top-N trade IDs in rank order
Top-N IDs
Paginated IDs for hydration
HMGET hydration
Full trade objects for results
API response
Low-latency payload to clients
SINTER
Intersect filter sets → candidate IDs
Candidate IDs
Filtered IDs only — no heavy objects yet
ZINTERSTORE
Rank overlay via sorted-set intersection
Rank overlay
Merge ranking scores for candidates
ZREVRANGE
Top-N trade IDs in rank order
Top-N IDs
Paginated IDs for hydration
HMGET hydration
Full trade objects for results
API response
Low-latency payload to clients
Scoring
Realtime advisor intelligence engine
wins / total trades- ×Stale rankings
- ×Ignored live open trades
- ×Easily gamed or manipulated signals
- +realized performance
- +unrealized performance
- +rolling consistency
- +risk adjustment
- +live trade quality
Rolling windows
Category segmentation
- · Index options
- · Stock options
- · Futures
- · Equity
Weighted scoring
closed>open
Closed trades weighted higher than open trades in scoring blends.
Delta
Incremental recomputation system
“We never recomputed all trades per tick.”
- Stoploss cascade handling
- Multiple targets & shared SL
- Duplicate recomputation prevention
Ticks
Live profit potential mutation
“Upside changes every tick.”
- —Static cache keys could not track upside moving every tick.
- —Top-K ranking had to stay fresh without full global sorts.
- —Full recomputation across the universe was operationally impossible.
08Engineering philosophy
Filter first.
Hydrate last.
IDs are cheap. Objects are expensive.
Narrowing
Millions of IDs
Top-N IDs → hydrated objects
Traditional object-level aggregation collapses under continuously mutating multidimensional market state.
Impact
Operational outcomes
Reduced Mongo aggregation pressure
Low-latency multidimensional filtering
Realtime advisor intelligence scoring
Incremental recomputation
Dynamic live profit-potential serving
Stack
