NVIDIA Launches Nemotron 3.5 Suite for Multimodal Agent Retrieval

The efficiency and reliability of advanced AI agents are fundamentally tied to the quality of the evidence they can retrieve from vast, complex document corpuses. When an agent is tasked with solving a complex problem, its initial answers are only as good as the information it can access. If the retrieved evidence is incomplete, irrelevant, or poorly structured, the agent is forced into a cycle of repeated searches and deep inspection of additional documents. This process, while necessary for grounding the answer, leads to a significant and often massive consumption of inference tokens, especially when deployed at an enterprise scale. Such token waste represents a major operational inefficiency, highlighting the critical need for sophisticated and highly accurate retrieval models and robust systems to ensure that agents can reach actionable insights with minimal computational overhead.

To address these limitations, NVIDIA has significantly expanded its Nemotron Retriever collection. This expansion introduces several new, open, and commercially available state-of-the-art models designed for three critical functions: embedding, reranking, and document parsing. The new suite includes Nemotron 3.5 Embed models (available in 8B, 1B, and NVFP4 variants), Nemotron 3.5 Rerank (8B), and Nemotron 3.5 Parse (631M parameters). Collectively, these models elevate enterprise agent workflows by integrating multimodal retrieval capabilities, precise relevance ranking, and advanced document processing into a single, cohesive system.

The quality of the retrieval process is paramount because it directly impacts two key metrics: the factual accuracy of the answer the agent produces, and the overall computational cost associated with the Large Language Model (LLM) inference. By ensuring that the evidence entering the agent's context is highly relevant, the agent can make better decisions about its next steps or answer the original question more accurately. Furthermore, the inclusion of a dedicated reranking stage significantly improves the overall quality of the evidence before it is passed to the LLM, thereby maximizing the utility of every token spent.

Consider a highly complex, real-world scenario: an agent must identify products that are at risk of stockouts in the next quarter and provide a detailed explanation. To complete this task, the agent cannot rely on a single source. It must synthesize information from disparate formats—combining demand forecasts presented in charts, current inventory levels detailed in tables, and supplier lead times documented in textual reports. The agent must then perform complex reasoning to determine if replenishment shipments will arrive before the existing inventory is depleted. Successfully completing this task requires not just searching, but finding the most relevant evidence across multiple, varied document types and integrating that evidence cohesively into a single, coherent analysis.

To empirically demonstrate the impact of retrieval quality, the developers evaluated the Nemotron 3.5 Embed and Rerank models across three challenging benchmarks: ViDoRe v3, BRIGHT, and BrowseComp-Plus. These benchmarks simulate real-world agent behavior. The evaluation pipeline involves the agent querying a corpus using a search tool, inspecting the retrieved evidence, and then refining its subsequent search attempts to pinpoint the most pertinent information for a given query. The experiments systematically varied the retrieval configuration—comparing embedding-only search versus embedding plus reranking, and testing different model combinations—while keeping the core components fixed, including the LLM powering the agent, the available tools, the task set, the document corpus, and the execution policy. The primary metrics measured were the agent's ability to find the relevant information for each task, and the total agent inference tokens consumed per task, encompassing all input and output tokens across every turn of the conversation.

The results showcased substantial performance gains. Specifically, integrating the Nemotron 3.5 Rerank model dramatically reduces the agentic token cost per query. Using the Nemotron 3.5 Embed 8B model, the reranker achieved an approximate 14% reduction in token cost, and with the Nemotron 3.5 Embed 1B model, the reduction was around 13%. Crucially, these cost savings were achieved while simultaneously improving the overall retrieval accuracy. Furthermore, when comparing retrieval quality at a similar agentic token expenditure, Nemotron 3.5 Embed 8B, paired with Nemotron 3.5 Rerank, delivered an nDCG@10 score approximately 0.9 points higher than the Qwen3-VL-Reranker-8B model. This demonstrates that the Nemotron suite offers superior performance and cost efficiency simultaneously, a critical breakthrough for scaling AI agents.

Underpinning these successes are the Nemotron 3.5 Embed and Nemotron 3.5 Rerank models, which are designed to retrieve evidence that spans both text and images, and then prioritize that evidence for the agent's working context. The Nemotron 3.5 Embed model represents a major leap forward from previous generations, as it is capable of retrieving evidence not only from standard document text but also from visual content embedded within documents. This visual content includes complex elements such as tables, charts, infographics, datasheets, and schematics. This capability allows enterprise applications to search for and utilize information conveyed by visual data alongside traditional textual descriptions, vastly expanding the scope of searchable knowledge.

The four models in the suite offer highly flexible deployment options. They support comprehensive text and image retrieval, enabling the search across diverse document types. In terms of global reach, text retrieval was evaluated across 34 languages, and the multilingual visual retrieval capabilities were tested on benchmarks like MIRACL Vision (covering 18 languages) and KoViDoRe, suggesting broad international applicability. The multimodal reranking capability is handled by an 8B cross-encoder, which scores the relevance of query–document pairs across both text and image inputs. For deployment flexibility, the 1B embedding model is available in both BF16 and NVFP4 formats, with quantization applied to both the text and image encoders. Moreover, the models support domain adaptation, allowing fine-tuning on custom document collections. For instance, in specialized chart and infographic experiments, the Nemotron-3.5-Embed-1B-BF16 and Nemotron-3.5-Embed-8B-BF16 models achieved relative nDCG@10 gains of 18% and 3%, respectively. On the rigorous ViDoRe v3.1 benchmark, the Nemotron 3.5 Embed 8B BF16 model achieved an impressive 61.17 nDCG@10, establishing itself as the top-ranking dense retrieval model. This score represents a substantial 8.94-point improvement over the 52.23 score achieved by the Llama Nemotron Embed VL 1B model. Furthermore, the smaller 1B model successfully captures most of this high visual retrieval quality, scoring 58.38 nDCG@10, which retains an impressive 95.4% of the larger 8B teacher model's performance. The combination of superior multimodal retrieval accuracy, significant cost reduction through reranking, and flexible deployment options makes this suite a transformative tool for building next-generation, highly efficient enterprise AI agents.