Case Study - A recommender that went from three seconds to under a hundred milliseconds

Overview

Three problems that look separate are usually one problem.

Levi's digital estate ran on SAP Hybris, with nightly cron ETLs that had no access to site session data. Finance and merchandising waited between a day and three days for weekly performance reporting. Holiday peaks saturated on-premise capacity and the site throttled. And product recommendations took around three seconds to return, computed from stale inputs, with no view of current inventory — so the shopper was regularly recommended something the warehouse could not ship.

All three are the same problem wearing different clothes: the data describing what shoppers were doing and what the business actually had in stock was not reaching the places that needed to answer questions from it. One of those places was a human reading a report. Another was a recommender with a hundred milliseconds to decide.

The mandate was to move core digital workloads to the cloud within two quarters, without putting Black Friday at risk.

The constraint

Peak trading is a hard deadline that does not negotiate. A migration that finishes late is not late, it is cancelled — nobody re-platforms a retailer in November. That single fact shaped the sequencing: everything was built to be reversible, traffic was replayed against the new stack at three times peak before anything moved, and the production switchover was scheduled for the quietest hour of the week.

What we built

One place the data lands

Browser beacons and CDN logs stream into Kinesis and land as compressed columnar files in a tiered lake — raw, curated, and a zone shaped for model training. Spark jobs fire on arrival rather than on a schedule, partitioned by hour, which is what takes reporting from a nightly batch to something close to live.

The nightly cron was not slow because it was badly written. It was slow because it was nightly. Removing the schedule removed the latency.

Recommendations as a retrieval problem

The old recommender was a scoring service asked to be clever. We rebuilt it as a search index asked to be fast: product and behaviour vectors in an OpenSearch cluster with the kNN plugin, generated in batch overnight and updated for hot records within seconds by a stream consumer, queried at the edge and injected into page markup.

Framing it as retrieval rather than as machine learning is what bought the latency. Nearest-neighbour lookup against a well-tuned index is a solved problem with a known cost; inference in the request path is not. Live inventory joins the query as a filter, so the engine stops recommending what cannot be shipped.

What we chose against

Results

The migration completed inside a six-week window against a two-quarter mandate — architecture held in the US, engineering running from our team in Vadodara, which is what compresses a two-quarter plan into six weeks. Production switchover ran at two in the morning on a Sunday with a twenty-four minute read-only freeze, and full parity verified before traffic returned.

The number worth arguing about is the 6.8% lift in add-to-cart. It came from two changes made together — a recommender fast enough to run in the page, and one that could see inventory — and we would not claim a clean split between them.