Ecom-RLVE: Adaptive Verifiable Environments for E-Commerce Conversational Agents
EcomRLVE-GYM is a new open‑source framework that extends verifiable reinforcement learning environments to multi‑turn, tool‑augmented e‑commerce conversations. It builds on the RLVE methodology, providing 400 algorithmic reasoning tasks and now adds realistic shopping scenarios where an agent must search catalogs, add items to carts, and handle user corrections, all scored by code without human or LLM judges. The system uses a difficulty parameter that controls 12 task aspects, adaptive scheduling that only advances when the agent succeeds, and a user simulator that generates natural, constraint‑aware messages. A Qwen‑3 8B agent trained with DAPO on the Cart Building task showed progressive difficulty gains, demonstrating the framework’s potential for scaling.
The Core Development
This project originated in the Pytorch OpenEnv Hackathon and is still evolving, follow us for updates 🔥
Large language models can hold fluent conversations, yet deploying them as shopping assistants reveals a persistent gap: fluency ≠ task completion . A customer who asks "find me a USB-C charger under $25 that ships in two days" needs an agent that invokes the right catalog search, filters on three hard constraints, avoids hallucinating product IDs it never retrieved, and handles follow-ups when the top result goes out of stock.
Supervised fine-tuning can teach surface-level tool use from demonstrations, but it cannot scale to the combinatorial space of constraint configurations, partial-information dialogues, and multi-step transactional workflows that real e-commerce demands.
What’s New in This Version
Cart building is a good showcase because it requires the full search → inspect → clarify → act loop, has a binary ground truth, and introduces a challenge absent from most recommendation benchmarks: variant selection .
The generator samples 1–5 target products (scaling in difficulty with d ), each potentially requiring a specific variant (USB-C vs Lightning, Matte vs Glossy) and a quantity > 1. The agent must:
How It Works & Technical Architecture
Reinforcement learning with verifiable rewards (RLVR) offers an alternative: the agent optimises for outcomes — did the products satisfy the constraints? Was the cart correct? Was the return initiated for the right order line? The challenge is constructing reward functions that are both verifiable (no LLM-as-a-judge subjectivity) and adaptive (difficulty that grows with the policy's capability).
Zeng, Z., Ivison, H., Wang, Y., et al. (2025). RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments. ICML 2025. arXiv:2511.07317
Yu, Q., Zhang, Z., Zhu, R., et al. (2025). DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476
DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning in LLMs through Reinforcement Learning. Nature.
Real-World Use Cases & Applications
Each environment covers a distinct real-world shopping scenario. The agent must complete the task using tools (catalog search, cart operations, order lookups, policy queries) and is scored by a program — not a human or another LLM.
At d = 0 the agent adds a single product with no variant complexity — learning the basic catalog.search → cart.add workflow. At d = 6 it juggles 3 items, nearly all requiring a specific variant, with half needing qty > 1.
Strategic Perspective & Ecosystem Impact
RLVE-Gym provides 400 environments for sorting, multiplication, Sudoku, and other algorithmic-reasoning tasks; however, those are all single-turn, text-in / text-out puzzles — extending to agentic domains was left as future work.
EcomRLVE-GYM fills that gap: we stay in the verifiable regime (e-commerce outcomes can be checked algorithmically) while extending to multi-turn, tool-augmented, agentic conversations — environments where the agent must act (call tools, modify world state) rather than merely reason (produce a text answer) and compensates for the deficiency of the search system.
Every signal above can be evaluated by a program with access to the hidden ground-truth goal. No human annotation or LLM-as-a-judge is needed.
Why it matters
By enabling fully verifiable, scalable training for conversational agents in complex e‑commerce workflows, EcomRLVE-GYM could accelerate the deployment of reliable shopping assistants and reduce reliance on costly human annotation.