Delivered. Production-ready. Measurable.

The Challenge

A quantitative trading operation needed to score every NYSE-listed stock and ETF for directional sentiment — but their Python batch application couldn’t scale past the Nasdaq 100. The pipeline fetched 1-minute OHLCV data from Polygon, calculated a dozen technical indicators, and ran ensemble regression models (AdaBoost, ExtraTrees, MLP) to produce a sentiment signal from −100 (bearish) to +100 (bullish). Even with 8 task threads, fetching data and running prediction for each ticker took long enough that only ~100 tickers could be scored daily — while the universe of interest numbered in the thousands. Adding more containers scaled linearly at best, duplicating the full application stack per instance.

What We Built

Decomposed the monolith into serverless functions orchestrated by a state machine. Golang handles data fetching from Polygon per ticker and function invocation coordination; Python remains in the indicator calculation, training, and prediction functions where the ML libraries live. The re-architecture eliminated the serial bottleneck entirely: each ticker’s fetch-and-score cycle now runs as an independently scalable, stateless function that spins up on demand and tears down between runs. The ensemble models and their specialized dependencies are packaged per-function, so inference is self-contained.

The Result

The pipeline now scores 7,000+ stocks and ETFs every 5 minutes — a 70× expansion from the original 100-ticker daily batch — for dollars a day or a few hundred dollars a month.

The Challenge

A fintech client had built an options strategy screener that ranked the best single- and multi-leg strategies within a 45-day window based on probability of profit and risk/reward ratio. It worked — for roughly 1,000 tickers. Running as a monolithic Python application on bare EC2, a full scan took 15 minutes. The team wanted to cover all exchange-listed stocks and ETFs so an LLM researcher could answer ad-hoc queries across any interest factor with fresh, pre-computed strategy scores — but the monolith couldn’t get there.

What We Built

Decomposed the entire pipeline from end to end: fetching options contracts for all expirations within the 45-day window, calculating Black-Scholes probabilities for every available broker strategy, and writing results to a dual-store architecture — Redis for sub-millisecond cache retrieval and Postgres for historical reference — on a 5-minute interval. Each stage runs independently, scales horizontally, and fails without blocking the others.

The LLM researcher queries continuously refreshed, pre-computed scores rather than stale batch output. It doesn’t calculate strategies on demand — it reasons over strategy scores that are already computed and never more than five minutes old.

The Result

Full-universe coverage on a 5-minute cycle, replacing a 15-minute scan of 1,000 tickers with near-real-time intelligence across every listed instrument. The LLM researcher can now answer virtually any ad-hoc query against pre-computed scores that reflect current market conditions.