5つの重要なマルチモーダルRAG機能でAI対応知識システムを構築

Enterprise data is inherently complex: real-world documents are multimodal, spanning text, tables, charts and graphs, images, diagrams, scanned pages, forms, and embedded metadata. Financial reports carry critical insights in tables, engineering manuals rely on diagrams, and legal documents often include annotated or scanned content.

Retrieval-augmented generation (RAG) was created to ground LLMs in trusted enterprise knowledge—retrieving relevant source data at query time to reduce hallucinations and improve accuracy. But if a RAG system processes only surrounding text, it misses key signals embedded in tables, charts, and diagrams—resulting in incomplete or incorrect answers.

An intelligent agent is only as good as the data foundation it’s built on. Modern RAG must therefore be inherently multimodal—able to understand both visual and textual context to achieve enterprise-grade accuracy. The NVIDIA Enterprise RAG Blueprint is built for this, providing a modular reference architecture that connects unstructured enterprise data to the intelligent systems built on top of it.

The blueprint also serves as a foundational layer for the NVIDIA AI Data Platform, helping to bridge the traditional gap between compute and data. By enabling retrieval and reasoning closer to the data layer, it preserves governance, reduces operational friction, and makes enterprise knowledge immediately usable by intelligent systems. The result is a modern AI data stack—storage that can retrieve, enrich, and reason alongside your models.

While the Enterprise RAG Blueprint provides many configurable options, this post highlights the following five key configurations that most directly improve accuracy and contextual relevance across enterprise use cases:

Baseline multimodal RAG pipeline

Query decomposition

Filtering metadata for faster and precise retrieval

Visual reasoning for multimodal data

The post also explains how the blueprint can be embedded into AI data platforms to transform traditional repositories into AI-ready knowledge systems.

Accuracy metrics in this blog are measured using the RAGAS framework, using well-known public datasets. Learn more about evaluating your NVIDIA RAG Blueprint system.

1. Document ingestion and understanding

Before an agent can deliver insights, it must be perfectly grounded in your data. This foundational configuration focuses on intelligent document ingestion and core RAG functionality.

The Enterprise RAG Blueprint uses NVIDIA NeMo Retriever to extract multimodal enterprise content—text, tables, charts and graphs, and infographics—then embeds that content into text for indexing in a vector database. At query time, the blueprint runs semantic retrieval, reranking, and Nemotron LLM to generate a grounded answer.

To maximize performance, this baseline intentionally avoids image captioning and heavy reasoning, making it the ideal starting point for production deployments. Deploy this baseline on Docker.

Benefits of document ingestion and understanding

This foundational configuration is the blueprint’s highest-efficiency pipeline, optimized for accuracy and throughput while keeping GPU cost and time to first token (TTFT) low. This configuration establishes your baseline performance for retrieval quality and LLM grounding.

Table 1 summarizes the overall impact across a few datasets.

Accuracy (v2.3 Default) MM = Multimodal, TO = Text-Only

Table 1. Accuracy impact of baseline configuration (higher is better)

When you turn on reasoning in the RAG blueprint, you enable the LLM to interpret the retrieved evidence, and synthesize logically grounded answers. This is the easiest change to get an accuracy boost for many applications. Enable reasoning for the NVIDIA Enterprise RAG Blueprint.

Table 2 summarizes the overall impact across several sample datasets.

Accuracy (v2.3 Default) plus Reasoning MM = Multimodal, TO = Text-Only

Table 2. Accuracy impact of enabling reasoning versus baseline configuration (higher is better)

Benefits of reasoning

For any use case involving mathematical operations or complex data comparison, a typical simple similarity or hybrid search will not suffice. Reasoning is required to correct errors and ensure precise contextual understanding. Accuracy improvements across datasets averaged ~5%, with several cases demonstrating dramatic reasoning-driven corrections.

In the FinanceBench dataset, the baseline configuration incorrectly computed the Adobe FY2017 operating cash flow ratio as 2.91. After enabling reasoning, the model produced the correct answer, 0.83. In addition, the Ragbattle dataset demonstrates the accuracy improvement from enabling VLM.

1. Query decomposition

Answering complex user questions often requires pulling facts from multiple places in the data foundation. Query decomposition breaks a single question into smaller subqueries, retrieves evidence for each, and recombines the results into a complete, grounded response. Turn on query decomposition for the NVIDIA Enterprise RAG Blueprint.