Back to experience

Multidimensional Market Intelligence Engine

Sept 2024 — Present

Redis-orchestrated multidimensional filtering and ranking under continuously mutating live market state — ID-first execution, not dashboard analytics.

SarasSarasCTO, Co-Founder

Designed and built a realtime multidimensional intelligence computation engine capable of dynamically filtering, ranking, scoring, and serving continuously mutating market trade intelligence at scale.

The system powered advisor rankings, trade discovery, live profit-potential sorting, multidimensional filtering, and realtime analytics across thousands of active market signals while avoiding expensive aggregation bottlenecks.

Traditional caching approaches became ineffective due to combinatorial filter permutations combined with continuously changing live market prices. To solve this, the architecture evolved from database-centric aggregation into a Redis-powered distributed computation layer using sorted sets, incremental ranking systems, ID-first filtering, partial hydration strategies, and event-driven state synchronization.

The engine transformed Redis from a simple caching layer into a realtime market intelligence orchestration system capable of handling dynamic ranking, live recomputation, rolling advisor scoring, and multidimensional query execution under constantly changing market state.

System overview

The platform continuously computed, filtered, ranked, and served multidimensional market intelligence under constantly mutating live market conditions.

Traditional caching strategies became ineffective because every filter permutation combined with continuously changing live prices created entirely new aggregation paths.

To solve this, the architecture evolved from database-centric aggregation into a Redis-powered distributed computation layer using sorted sets, incremental ranking systems, ID-first filtering, partial hydration strategies, and realtime event-driven state synchronization.

Query complexity visualization

Period

Intraday · Swing · Positional

Category

Index Opt · Stock Opt · Futures · Equity

Position

Buy · Sell · Hold

Source

Telegram · Report · X · YouTube

Trade status

Open · Closed · Pending

Date range

7D · 30D · 90D · Custom

Profit potential

Upside % · R:R · Absolute

Advisor accuracy

Rolling · Segmented

Sorting

Upside · Recent · Accuracy

Millions of possible query paths

→ Redis query orchestrator

Redis computation preview

Live price layer
Trade state layer
Ranking layer
Advisor intelligence layer
Reference layer

Key engineering challenges

Multidimensional query explosion

Live ranking mutation

Dynamic profit potential computation

Cache invalidation complexity

Incremental recomputation

Avoiding Mongo aggregation collapse

Tools & Technologies

Redis
MongoDB
Node.js
AWS ECS
Python
Apache Airflow
Grafana
Prometheus