Skip to content

Agentic Literature Review

An agent that autonomously searches arXiv, downloads papers, extracts key information, and produces a structured literature review — using PAIS as the reasoning backbone.

What it does

flowchart TD
    Q[Research topic] --> A[Agent]
    A --> S1[Search arXiv\nfor papers]
    S1 --> A
    A --> S2[Download &\nparse abstracts]
    S2 --> A
    A --> S3[Extract methods,\ndatasets, metrics]
    S3 --> A
    A --> S4[Identify themes\n& contradictions]
    S4 --> A
    A --> R[Structured\nliterature review]

The agent uses LangChain's tool-calling framework with three tools: 1. search_arxiv — keyword search, returns titles and abstracts 2. get_paper_details — fetch full metadata and structured extraction 3. synthesise — produces the final structured review

Prerequisites

pip install langchain langchain-openai arxiv python-dotenv pydantic

Full example

# lit_review_agent.py
"""
Agentic literature review using PAIS + LangChain.

Usage:
    python lit_review_agent.py --topic "contrastive learning medical imaging" \
                               --max-papers 15 \
                               --output review.md
"""
import os
import json
import argparse
from datetime import datetime
from typing import Optional
from dotenv import load_dotenv

import arxiv
from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.agents import AgentExecutor, create_tool_calling_agent

load_dotenv()


# ── Data models ────────────────────────────────────────────────────────────────

class PaperExtraction(BaseModel):
    title: str
    arxiv_id: str
    year: int
    problem: str = Field(description="The core problem addressed")
    methods: list[str] = Field(description="Key methods or architectures used")
    datasets: list[str] = Field(description="Datasets used for evaluation")
    metrics: list[str] = Field(description="Evaluation metrics reported")
    key_findings: list[str] = Field(description="Main results or contributions")
    limitations: list[str] = Field(description="Stated limitations or weaknesses")


# ── LLM setup ─────────────────────────────────────────────────────────────────

llm = ChatOpenAI(
    model="llama-3-1-8b",
    openai_api_base=os.environ["PAIS_API_BASE"],
    openai_api_key=os.environ["PAIS_API_KEY"],
    temperature=0.1,
)

extraction_llm = ChatOpenAI(
    model="llama-3-1-8b",
    openai_api_base=os.environ["PAIS_API_BASE"],
    openai_api_key=os.environ["PAIS_API_KEY"],
    temperature=0.0,
)


# ── Tools ─────────────────────────────────────────────────────────────────────

_paper_cache: dict[str, dict] = {}

@tool
def search_arxiv(query: str, max_results: int = 10, year_from: int = 2020) -> str:
    """Search arXiv for papers matching a query. Returns titles, arxiv IDs, and abstracts.

    Args:
        query: Search terms (e.g. "contrastive learning medical imaging")
        max_results: Maximum number of papers to return (default 10, max 20)
        year_from: Only return papers from this year onwards
    """
    client = arxiv.Client()
    search = arxiv.Search(
        query=query,
        max_results=min(max_results, 20),
        sort_by=arxiv.SortCriterion.Relevance,
    )
    results = []
    for paper in client.results(search):
        year = paper.published.year
        if year < year_from:
            continue
        paper_id = paper.entry_id.split("/")[-1]
        _paper_cache[paper_id] = {
            "title": paper.title,
            "abstract": paper.summary,
            "authors": [a.name for a in paper.authors[:3]],
            "year": year,
            "categories": paper.categories,
        }
        results.append(
            f"ID: {paper_id}\n"
            f"Title: {paper.title}\n"
            f"Year: {year}\n"
            f"Abstract: {paper.summary[:500]}...\n"
        )
    return f"Found {len(results)} papers:\n\n" + "\n---\n".join(results)


@tool
def extract_paper_details(arxiv_id: str) -> str:
    """Extract structured information from a paper by its arXiv ID.
    Returns problem, methods, datasets, metrics, findings, and limitations.

    Args:
        arxiv_id: The arXiv paper ID (e.g. "2301.12345")
    """
    if arxiv_id not in _paper_cache:
        # Fetch if not in cache
        client = arxiv.Client()
        search = arxiv.Search(id_list=[arxiv_id])
        paper = next(client.results(search))
        _paper_cache[arxiv_id] = {
            "title": paper.title,
            "abstract": paper.summary,
            "year": paper.published.year,
        }

    paper = _paper_cache[arxiv_id]
    schema = PaperExtraction.model_json_schema()

    prompt = f"""Extract structured information from this research paper.
Title: {paper["title"]}
Abstract: {paper["abstract"]}

Respond with valid JSON matching this schema:
{json.dumps(schema, indent=2)}

For fields you cannot determine from the abstract, use empty lists or
"Not specified" for strings."""

    response = extraction_llm.invoke(prompt)
    return response.content


@tool
def write_synthesis(
    topic: str,
    paper_summaries: str,
    output_format: str = "markdown",
) -> str:
    """Synthesise extracted paper information into a structured literature review.

    Args:
        topic: The research topic being reviewed
        paper_summaries: JSON-formatted list of extracted paper details
        output_format: "markdown" or "plain"
    """
    prompt = f"""Write a structured literature review on the topic: "{topic}"

Based on these papers:
{paper_summaries}

Structure the review as:
1. **Overview** — What problem space do these papers address?
2. **Common Methods** — What approaches appear across multiple papers?
3. **Datasets** — What datasets are commonly used? Any gaps?
4. **Key Findings** — What are the main results and agreements?
5. **Contradictions & Open Questions** — Where do papers disagree or leave gaps?
6. **Recommendations** — What should a new researcher in this area focus on?

Be specific, cite paper titles, and use precise technical language."""

    response = llm.invoke(prompt)
    return response.content


# ── Agent ─────────────────────────────────────────────────────────────────────

def build_agent() -> AgentExecutor:
    tools = [search_arxiv, extract_paper_details, write_synthesis]

    prompt = ChatPromptTemplate.from_messages([
        ("system", """You are an expert research assistant conducting a systematic
literature review. Your process:
1. Search arXiv for papers on the topic (aim for 10-15 relevant papers)
2. Extract structured details from the most relevant papers
3. Synthesise the findings into a structured review
4. Present the final review

Be systematic and thorough. Extract details from at least 8 papers before synthesising."""),
        ("human", "{input}"),
        MessagesPlaceholder("agent_scratchpad"),
    ])

    agent = create_tool_calling_agent(llm, tools, prompt)
    return AgentExecutor(
        agent=agent,
        tools=tools,
        verbose=True,
        max_iterations=20,
        return_intermediate_steps=True,
    )


# ── Main ──────────────────────────────────────────────────────────────────────

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--topic", required=True, help="Research topic to review")
    parser.add_argument("--max-papers", type=int, default=15)
    parser.add_argument("--output", default="review.md", help="Output file")
    parser.add_argument("--year-from", type=int, default=2021)
    args = parser.parse_args()

    agent = build_agent()

    user_request = (
        f"Conduct a literature review on: '{args.topic}'. "
        f"Search for up to {args.max_papers} papers from {args.year_from} onwards. "
        f"Extract details from the most relevant ones and write a structured review."
    )

    print(f"Starting literature review: {args.topic}\n{'='*60}\n")
    result = agent.invoke({"input": user_request})

    review = result["output"]

    with open(args.output, "w") as f:
        f.write(f"# Literature Review: {args.topic}\n")
        f.write(f"*Generated {datetime.now().strftime('%Y-%m-%d')} using PAIS + Llama 3.1 8B*\n\n")
        f.write(review)

    print(f"\nReview saved to {args.output}")


if __name__ == "__main__":
    main()

Running the agent

# Basic usage
python lit_review_agent.py \
    --topic "few-shot learning for biomedical NLP" \
    --max-papers 15 \
    --output biomedical_nlp_review.md

# Target a specific time window
python lit_review_agent.py \
    --topic "neural architecture search" \
    --year-from 2023 \
    --output nas_review_2023_2025.md

Example output structure

# Literature Review: Few-shot Learning for Biomedical NLP
*Generated 2026-04-15 using PAIS + Llama 3.1 8B*

## Overview
The papers address the challenge of applying NLP to biomedical text where
labelled data is scarce. The core tension is between the rich pre-training
signal available in general text (PubMed abstracts, clinical notes) and the
distribution shift when applying to specific tasks like NER, relation extraction,
or ICD coding...

## Common Methods
- **Prompt tuning / in-context learning** (Wang et al. 2023, Lee et al. 2024):
  3-shot and 5-shot prompting using domain-adapted LLMs...
- **Parameter-efficient fine-tuning (PEFT)**: LoRA and prefix tuning appear in
  7 of the 12 reviewed papers...

## Key Findings
...

Customising the agent

Add a web search tool for non-arXiv sources: 

from langchain_community.tools import TavilySearchResults
web_search = TavilySearchResults(max_results=5)
tools = [search_arxiv, extract_paper_details, write_synthesis, web_search]

Persist intermediate results so the agent can resume after rate limiting: 

import pickle
with open("paper_cache.pkl", "wb") as f:
    pickle.dump(_paper_cache, f)

See also