How Shopify built a continual learning loop with PyTorch and vLLM

Shopify has implemented a continual learning loop to optimize its GraphQL agent, moving from a general-purpose frontier model to a specialized, smaller model. The process begins by establishing a human-calibrated rubric to define quality, which then trains an automated judge. This judge identifies production failures, which are repaired by reasoning models to create successful trajectories for supervised fine-tuning and reinforcement learning via GRPO.
This architectural shift has resulted in a 96% reduction in serving costs—estimated from $27M to $1M annually—while surpassing the quality of the original frontier baseline. Additionally, the use of "gist" token compression reduced the system prompt from 6,000 to 1,500 tokens, leading to a 38% drop in end-to-end latency and a 14% reduction in required GPU hardware. The system operates on a daily cadence, continually folding production failures back into the model's weights using PyTorch and vLLM.

* Shopify developed a continual learning loop for its GraphQL agent.
* The system uses PyTorch for training and vLLM for inference.
* Quality is defined via a rubric covering completeness, execution, response quality, and safety.
* Cohen’s kappa is used to measure inter-annotator agreement between product experts.
* DSPy, GEPA, and Agentic Context Engineering are used to calibrate the automated judge.
* Model optimization involves supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO).
* Toloka provides expert human annotators for failures that automated critics cannot fix.
* The GraphQL agent handles up to 2,000 requests per minute.
* Serving costs were reduced from an estimated $27M to $1M per year.
* Gist compression reduced the system prompt from 6,000 tokens to 1,500 tokens.
* Time-to-first-token decreased by 19% and end-to-end latency dropped by 38% under load.
* Hardware requirements decreased by approximately 14% of GPUs.

This technical disclosure functions in ACADEMIC MODE, presenting a methodology for "compressing" production experience into model weights. The study design is a closed-loop optimization: Rubric $\rightarrow$ Judge $\rightarrow$ Failure Mining $\rightarrow$ SFT/RL $\rightarrow$ Weights. While the results are numerically impressive, a peer reviewer would note the absence of a blind, third-party audit of the "surpassing frontier quality" claim; the quality is measured by a judge calibrated by the same team building the model, creating a potential circularity in validation. Furthermore, the "96% cost reduction" is based on an estimate of frontier model pricing rather than a direct A/B cost comparison of identical workloads.
The narrative reflects a paradigm shift in AI deployment: moving from "Prompt Engineering" (discrete artifacts) to "Weight Engineering" (continuous parameters). It assumes that production failures are the most valuable training data and that a small, distilled model can inherit the reasoning capabilities of a frontier model if the trajectories are sufficiently rich.
This approach maximizes corporate efficiency and user latency but centers agency within the "judge"—the entity that defines "good." If the rubric is flawed, the model optimizes for a distorted version of quality.
Bridge Questions:
1. How does the system prevent "reward hacking," where the model learns to satisfy the judge's metrics without actually improving user utility?
2. Would this loop maintain stability across vastly different product domains, or is it dependent on the structured nature of GraphQL?
Counterstrike Scan:
A bad actor pushing this narrative would use it to convince companies to abandon secure, controllable frontier APIs in favor of opaque, self-evolving internal models that drift away from human oversight. The actual content is a transparent engineering case study and does not match this pattern.