Measuring Open-Source Llama Nemotron Models on DeepResearch Bench

NVIDIA’s AI‑Q Blueprint, described as a leading portable, open deep research agent, recently climbed to the top of the Hugging Face “LLM with Search” leaderboard on DeepResearch Bench. The stack fuses two high‑performance open LLMs—Llama 3.3‑70B Instruct and Llama‑3.3‑Nemotron‑Super‑49B‑v1.5—to orchestrate long‑context retrieval, agentic reasoning, and robust synthesis. Its architecture supports parallel, low‑latency search over local and web data, making it ideal for use cases that demand privacy, compliance, or on‑premise deployment for reduced latency. NVIDIA Llama Nemotron Super is not only a fine‑tuned instruct model; it is post‑trained for explicit agentic reasoning and supports reasoning ON/OFF toggles via system prompts, allowing users to switch between standard chat LLM mode and deep, chain‑of‑thought reasoning for agent pipelines. One of AI‑Q’s core strengths is transparency, providing not only outputs but also reasoning traces and intermediate steps. During development, the NVIDIA team leveraged both standard and new metrics, and the architecture lends itself to granular, stepwise evaluation and debugging, addressing a major pain point in agentic pipeline development. DeepResearch Bench evaluates agent stacks using a set of 100+ long‑context, real‑world research tasks across science, finance, art, history, software, and more, requiring report‑length synthesis and complex multi‑hop reasoning. The open‑source ecosystem is rapidly closing the gap—and, in some areas, leading—on real‑world agent tasks that matter. AI‑Q, built on Llama Nemotron, demonstrates that you don’t need to compromise on transparency or control to achieve state‑of‑the‑art results. The stack is available on Hugging Face or can be adapted to research agent projects from build.nvidia.com.