AutoFigure でテキストから科学図を生成するエージェント型パイプライン構築

In this tutorial, we explore AutoFigure as a practical toolkit for generating scientific figures directly from text descriptions, paper-like content, and structured methodological explanations. In this tutorial, we set up the complete AutoFigure environment, fix dependency issues such as Pillow compatibility, and prepare the required rendering tools for SVG and PNG outputs. We then build a custom reference figure, configure an API-backed generation workflow, and use AutoFigure to convert a detailed agentic document intelligence pipeline into a publication-style scientific diagram. Along the way, we also test offline SVG rendering, inspect the generated files, create a sample paper and PDF, and export the final outputs to a reusable gallery and a zip archive.

Copy CodeCopiedUse a different Browser

import os

import sys

import json

import time

import glob

import shutil

import textwrap

import subprocess

import importlib

from pathlib import Path

from getpass import getpass

REPO_URL = "https://github.com/ResearAI/AutoFigure.git"

REPO_DIR = Path("/content/AutoFigure")

OUTPUT_ROOT = Path("/content/autofigure_colab_outputs")

PROVIDER = os.environ.get("AUTOFIGURE_PROVIDER", "openrouter")

DEFAULT_MODELS = {

"openrouter": "google/gemini-3.1-pro-preview",

"gemini": "gemini-3.1-pro-preview",

"bianxie": "gemini-3.1-pro-preview",

}

GENERATION_MODEL = os.environ.get(

"AUTOFIGURE_MODEL",

DEFAULT_MODELS.get(PROVIDER, "google/gemini-3.1-pro-preview")

)

MAX_ITERATIONS = int(os.environ.get("AUTOFIGURE_MAX_ITERATIONS", "1"))

QUALITY_THRESHOLD = float(os.environ.get("AUTOFIGURE_QUALITY_THRESHOLD", "8.5"))

RUN_TEXT_TO_FIGURE = True

RUN_PAPER_TO_FIGURE = False

RUN_MXGRAPH_DEMO = False

RUN_IMAGE_ENHANCEMENT = False

TEXT_OUTPUT_FORMAT = "svg"

MXGRAPH_OUTPUT_FORMAT = "mxgraphxml"

ART_STYLE = (

"clean publication-ready scientific illustration, precise alignment, subtle shadows, "

"clear academic typography, high contrast, minimal clutter"

)

FIGURE_DESCRIPTION = """

Create a publication-ready scientific method figure for an agentic long-document intelligence system.

The figure should explain the following pipeline in a left-to-right architecture:

1. Long documents enter the system. They may be PDFs, scanned reports, markdown files, tables, or mixed-layout documents.
2. A document normalization layer extracts raw text, section hierarchy, tables, figures, and metadata.
3. A routing planner decides whether each section should go to summarization, field extraction, table reconstruction, visual analysis, or citation grounding.
4. Specialized expert modules process the routed chunks:

• Summarizer expert creates hierarchical summaries.
• Extraction expert returns JSON fields.
• Table expert reconstructs exact tables.
• Visual expert describes charts and diagrams.
• Citation expert links claims to evidence spans.

1. A low-cost orchestration layer selects smaller or larger LLMs depending on complexity, confidence, and budget.
2. A verification layer checks schema validity, source grounding, table consistency, and confidence.
3. The final output is an analyst-ready workspace containing a summary, extracted fields, exact tables, cited answers, and audit logs.

Design requirements:

• Use a wide 16:9 layout.
• Use clear module boxes, arrows, and labels.
• Add small callouts for cost control, confidence scoring, and auditability.
• Avoid decorative clutter.
• Make the flow understandable for a finance or enterprise document intelligence audience.

"""

MINI_PAPER_MARKDOWN = """

Efficient Agentic Document Intelligence for Long Financial Reports

Abstract

We propose an agentic document intelligence architecture for extracting summaries, facts, tables,

and grounded answers from long, heterogeneous financial documents.

Method

Our method first normalizes each incoming document into a structured document graph. The graph

contains section nodes, paragraph nodes, table nodes, figure nodes, and metadata nodes. A routing

planner assigns each node to a specialized expert according to modality, complexity, and required

output schema.

The system uses five experts. The summarization expert produces hierarchical summaries from

section-level chunks. The extraction expert fills strict JSON schemas for entities, dates, risks,

financial metrics, and obligations. The table expert reconstructs exact tables and validates row-column

alignment. The visual expert describes charts and diagrams. The citation expert maps every generated

claim to source spans.

A budget-aware orchestration layer selects model size dynamically. Simple chunks are processed by

low-cost models, while complex chunks are escalated to stronger models. A verification layer then

checks schema validity, citation support, numerical consistency, and table integrity. Failed checks are

routed back for repair.

Experiments

We evaluate on financial filings and analyst reports using extraction accuracy, grounding precision,

table reconstruction quality, and total inference cost.

"""

def run(cmd, cwd=None, check=True, quiet=False):

print(f"\n$ {cmd}")

process = subprocess.run(

cmd,

shell=True,

cwd=str(cwd) if cwd else None,

text=True,

stdout=subprocess.PIPE if quiet else None,

stderr=subprocess.STDOUT if quiet else None,

)

if quiet and process.stdout:

print(process.stdout[-5000:])

if check and process.returncode != 0:

raise RuntimeError(f"Command failed with exit code {process.returncode}: {cmd}")

return process

def heading(title):

print("\n" + "=" * 100)

print(title)

print("=" * 100)

def safe_read(path, max_chars=2500):

path = Path(path)

if not path.exists():

return ""

text = path.read_text(encoding="utf-8", errors="ignore")

return text[:max_chars] + ("\n... [truncated]" if len(text) > max_chars else "")

def clear_loaded_modules(prefixes):

for name in list(sys.modules):

if any(name == prefix or name.startswith(prefix + ".") for prefix in prefixes):

del sys.modules[name]

def get_colab_secret(names):

try:

from google.colab import userdata

for name in names:

try:

value = userdata.get(name)

if value:

return value

except Exception:

pass

except Exception:

pass

return None

def collect_api_key(provider):

env_candidates = [

"AUTOFIGURE_API_KEY",

"OPENROUTER_API_KEY",

"GOOGLE_API_KEY",

"GEMINI_API_KEY",

"BIANXIE_API_KEY",

]

for key_name in env_candidates:

value = os.environ.get(key_name)

if value:

print(f"Using API key from environment variable: {key_name}")

return value

secret_candidates = {

"openrouter": ["AUTOFIGURE_API_KEY", "OPENROUTER_API_KEY"],

"gemini": ["AUTOFIGURE_API_KEY", "GOOGLE_API_KEY", "GEMINI_API_KEY"],

"bianxie": ["AUTOFIGURE_API_KEY", "BIANXIE_API_KEY"],

}.get(provider, ["AUTOFIGURE_API_KEY"])

value = get_colab_secret(secret_candidates)

if value:

print("Using API key from Colab Secrets.")

return value

value = getpass(f"Paste your {provider} API key, or press Enter to skip cloud generation: ").strip()

return value

We begin by importing and defining the main paths, provider settings, model configuration, and tutorial options. We also prepare the detailed figure description and sample paper content that we use later for AutoFigure generation. We then create helper functions to run commands, print section headings, read files safely, clear loaded modules, and securely collect API keys.

Copy CodeCopiedUse a different Browser

def display_file_if_possible(path, title=None):

path = Path(path) if path else None

if not path or not path.exists():

print(f"Missing file: {path}")

return

try:

from IPython.display import display, Image as IPImage, SVG, Markdown

if title:

display(Markdown(f"### {title}"))

suffix = path.suffix.lower()

if suffix == ".png":

display(IPImage(filename=str(path)))

elif suffix == ".svg":

display(SVG(filename=str(path)))

elif suffix in [".json", ".md", ".txt", ".drawio"]:

print(safe_read(path, max_chars=5000))

else:

print(path)

except Exception as exc:

print(f"Could not display {path}: {exc}")

def make_output_gallery(output_dir):

output_dir = Path(output_dir)

gallery_path = output_dir / "gallery.html"

blocks = []

for p in sorted(output_dir.rglob("*.png")):

rel = p.relative_to(output_dir)

blocks.append(f"""

{rel}

""")

for p in sorted(output_dir.rglob("*.svg")):

rel = p.relative_to(output_dir)

svg_text = p.read_text(encoding="utf-8", errors="ignore")

blocks.append(f"""

{rel}

{svg_text}

""")

for p in sorted(output_dir.rglob("*.drawio")):

rel = p.relative_to(output_dir)

code = p.read_text(encoding="utf-8", errors="ignore")[:4000]

blocks.append(f"""

{rel}

Editable draw.io mxGraph XML file.

{code}

""")

for p in sorted(output_dir.rglob("generation_report.json")):

rel = p.relative_to(output_dir)

try:

report_text = json.dumps(json.loads(p.read_text(encoding="utf-8")), indent=2)[:7000]

except Exception:

report_text = p.read_text(encoding="utf-8", errors="ignore")[:7000]

blocks.append(f"""

{rel}

{report_text}

""")

html = f"""

AutoFigure Colab Gallery

AutoFigure Colab Gallery

{''.join(blocks)}

"""

gallery_path.write_text(html, encoding="utf-8")

return gallery_path

def summarize_generation_result(result, label):

print("\n" + "-" * 100)

print(label)

print("-" * 100)

print(f"Success: {result.success}")

print(f"Final score: {result.final_score}")

print(f"Iterations used: {result.iterations_used}")

print(f"SVG path: {result.svg_path}")

print(f"mxGraph path: {result.mxgraph_path}")

print(f"Preview path: {result.preview_path}")

print(f"Enhanced path: {result.enhanced_path}")

print(f"Enhanced paths: {result.enhanced_paths}")

print(f"Error: {result.error}")

if result.logs:

print("\nRecent logs:")

for log in result.logs[-20:]:

print(f"- {log}")

display_file_if_possible(result.preview_path, f"{label}: PNG Preview")

if result.svg_path:

display_file_if_possible(result.svg_path, f"{label}: SVG")

if result.mxgraph_path:

display_file_if_possible(result.mxgraph_path, f"{label}: mxGraph XML")

report_candidates = []

for candidate in [result.svg_path, result.mxgraph_path, result.preview_path]:

if candidate:

report_candidates.append(Path(candidate).parent / "generation_report.json")

for report_path in report_candidates:

if report_path.exists():

print("\nGeneration report preview:")

print(safe_read(report_path, max_chars=6000))

try:

import pandas as pd

from IPython.display import display

report = json.loads(report_path.read_text(encoding="utf-8"))

rows = []

for row in report.get("iteration_history", []):

rows.append({

"iteration": row.get("iteration"),

"quality_score": row.get("quality_score"),

"improvement": row.get("improvement"),

"has_critique": row.get("critique") is not None,

})

if rows:

display(pd.DataFrame(rows))

except Exception as exc:

print(f"Could not tabulate report: {exc}")

break

We define utility functions that help us display generated files directly inside Colab, including PNG, SVG, JSON, Markdown, text, and draw.io outputs. We also build an HTML gallery generator so that all AutoFigure outputs can be reviewed on a single, organized page. We then add a result-summary function that prints generation metadata, displays previews, and shows the iteration report in a readable format.

Copy CodeCopiedUse a different Browser

heading("1. Installing AutoFigure and Colab dependencies")

OUTPUT_ROOT.mkdir(parents=True, exist_ok=True)

run("apt-get update -qq", quiet=True)

run(

"apt-get install -y -qq "

"libcairo2 libpango-1.0-0 libpangocairo-1.0-0 "

"libgdk-pixbuf-2.0-0 libffi-dev shared-mime-info",

quiet=True,

)

clear_loaded_modules(["PIL", "autofigure"])

run(f"{sys.executable} -m pip install -q -U pip 'setuptools Restart runtime, then rerun this full cell.")

raise exc

if RUN_MXGRAPH_DEMO:

run(f"{sys.executable} -m playwright install chromium", quiet=True)

sys.path.insert(0, str(REPO_DIR))

heading("2. Importing AutoFigure SDK")

from autofigure import AutoFigureAgent, Config

from autofigure.generator import (

validate_code_syntax,

code_to_png,

get_initial_prompt_template,

)

from autofigure.extractor import MethodologyExtractor

print("AutoFigure imported successfully.")

print(f"Repository directory: {REPO_DIR}")

print(f"Output root: {OUTPUT_ROOT}")

heading("3. Offline SVG preflight: validation and rendering")

preflight_dir = OUTPUT_ROOT / "00_offline_preflight"

preflight_dir.mkdir(parents=True, exist_ok=True)

sample_svg = """

AutoFigure Offline Rendering Check

Text Prompt

method description

AutoFigure

generate → evaluate → refine

Figure

SVG + PNG output

""".strip()

is_valid, validation_message = validate_code_syntax(sample_svg, "svg")

print(f"SVG syntax valid: {is_valid}")

print(f"Validation message: {validation_message}")

sample_svg_path = preflight_dir / "offline_preflight.svg"

sample_png_path = preflight_dir / "offline_preflight.png"

sample_svg_path.write_text(sample_svg, encoding="utf-8")

render_ok, processed_svg = code_to_png(

sample_svg,

str(sample_png_path),

attempt_repair=False,

output_format="svg",

)

print(f"Rendered PNG: {render_ok} -> {sample_png_path}")

display_file_if_possible(sample_png_path, "Offline preflight PNG")

We install the required system packages, resolve Pillow compatibility issues, clone the AutoFigure repository, and install the SDK along with its PDF and web dependencies. We then import AutoFigure’s main classes and generator utilities after confirming that the environment is ready. We also run offline SVG validation and PNG rendering tests to ensure the rendering pipeline works before making any API-based generation calls.

Copy CodeCopiedUse a different Browser

heading("4. Creating a custom reference figure")

reference_dir = OUTPUT_ROOT / "01_custom_references"

reference_dir.mkdir(parents=True, exist_ok=True)

reference_path = reference_dir / "reference_architecture_style.png"

W, H = 1333, 750

img = Image.new("RGB", (W, H), "white")

draw = ImageDraw.Draw(img)

try:

title_font = ImageFont.truetype("DejaVuSans-Bold.ttf", 36)

box_font = ImageFont.truetype("DejaVuSans-Bold.ttf", 24)

small_font = ImageFont.truetype("DejaVuSans.ttf", 18)

except Exception:

title_font = None

box_font = None

small_font = None

draw.text(

(W // 2, 55),

"Reference Layout: Modular Scientific Pipeline",

anchor="mm",

fill="black",

font=title_font,

)

boxes = [

(90, 215, 290, 120, "Input", "documents"),

(365, 215, 290, 120, "Planner", "route by task"),

(640, 215, 290, 120, "Experts", "summary / table / vision"),

(915, 215, 290, 120, "Verifier", "grounded output"),

]

for i, (x, y, bw, bh, title, subtitle) in enumerate(boxes):

draw.rounded_rectangle(

[x, y, x + bw, y + bh],

radius=22,

fill=(245, 245, 245),

outline=(20, 20, 20),

width=3,

)

draw.text(

(x + bw / 2, y + 45),

title,

anchor="mm",

fill="black",

font=box_font,

)

draw.text(

(x + bw / 2, y + 82),

subtitle,

anchor="mm",

fill=(70, 70, 70),

font=small_font,

)

if i

ax = x + bw + 20

ay = y + bh / 2

bx