KDD 2025 Tutorial

Agentic AI & LangGraph

A hands-on tutorial for building, scaling, and evaluating agentic AI systems with LangGraph — from your first reasoning loop to production-ready multi-agent pipelines.

⚠ Prerequisites — Read Before You Start:
This tutorial requires PyCharm IDE and Python 3.11. Download PyCharm at jetbrains.com/pycharm and Python 3.11 at python.org/downloads. A free Groq API key is also required — get one at https://console.groq.com/keys. Each module page has a dedicated Setup section with exact install commands.
Colab
Run in Google Colab
All modules available as a ready-to-run notebook — no local install needed.
Open in Colab →

Course Outline — 6 Modules

Prerequisites

Python: Python 3.11 is required. Earlier/later versions may have compatibility issues with LangGraph packages.
IDE: PyCharm Community or Professional Edition. All setup instructions in this tutorial target PyCharm.
Groq API key: Free tier at https://console.groq.com/keys. No credit card required for the tutorial's usage level.
Python basics: Comfortable writing functions, using pip, and running scripts from the terminal.

Quick Start

1. Install PyCharm and select Python 3.11 as the project interpreter.

2. Get your Groq API key from console.groq.com/keys.

3. In PyCharm go to Run → Edit Configurations, pick your script, and add environment variable GROQ_API_KEY=your_key. Or create a .env file in the project root with that line — every script calls load_dotenv().

4. Open Module 1 and follow the Setup section for per-module package installs.

Instructor

About the Instructor

Dr. Mohammad Amin Kuhail
Dr. Mohammad Amin Kuhail
Associate Professor — Computing and Applied Technology Department
Zayed University, Abu Dhabi, UAE
🌐 www.drkuhail.com
mohammad.kuhail@zu.ac.ae
NVIDIA Certified Instructor Human-Centered AI Conversational Agents Multi-Agent AI Systems LLM Engineering AI in Education
NVIDIA Certified Instructor
Certified in Building Agentic AI Applications with Large Language Models — ensuring participants gain practical insights aligned with modern industry practices.

Biography

Dr. Mohammad Amin Kuhail is an Associate Professor at Zayed University, specializing in Human-Centered AI, conversational agents, and AI-supported decision systems. His research explores the design and evaluation of AI assistants, multi-agent conversational systems, and agentic AI environments powered by large language models.

He has published extensively on chatbots, collaborative AI systems, AI-assisted learning environments, and AI-supported decision-making. His work often investigates how AI agents can collaborate with humans and with other AI agents to support learning, brainstorming, decision-making, and problem-solving.

Dr. Kuhail actively develops experimental AI platforms that integrate large language models, retrieval-augmented generation (RAG), and multi-agent architectures for applications in education, decision support, and collaborative problem-solving.

🔬

Research Areas

  • Human-Centered AI
  • Conversational Agents
  • AI-Supported Decision Systems
  • Multi-Agent AI Systems
  • LLM-Powered Applications
  • AI in Education
📄

Publication Topics

  • Chatbots and conversational AI in education
  • Collaborative AI systems and multi-agent architectures
  • AI-assisted learning environments
  • AI-supported decision-making systems
  • Retrieval-Augmented Generation (RAG) for educational applications

What You Will Build

1

Introduction to Agentic AI

⏱ 30 min  ·  Module 1

Learning Objectives

  • Identify the limitations of prompt engineering for multi-step, tool-dependent tasks
  • Contrast LLM pipelines with agent workflows and select the right approach for a given task
  • List and describe the core agent capabilities: reasoning, planning, tool use, and self-refinement
  • Configure a Groq LLM in PyCharm and run your first agentic Python script

Environment Requirements

PyCharm + Python 3.11 Required

This tutorial requires PyCharm IDE and Python 3.11. Earlier Python versions (3.9, 3.10) may work but are unsupported — use Python 3.11 (not Python 3.12 or 3.13) for full compatibility with all LangGraph packages.

Download PyCharm: https://www.jetbrains.com/pycharm/
Download Python 3.11: https://www.python.org/downloads/release/python-3110/

Module 1 — Install Required Packages

Open PyCharm's built-in terminal (View → Tool Windows → Terminal) and run:

python -m pip install --upgrade langchain-groq langchain-core python-dotenv

Note: langchain-groq is not installed in PyCharm by default. You can also install it via File → Settings → Project → Python Interpreter → + (Add Package) and search for langchain-groq.

Groq API Key Setup

Step 1 — Get your key: Create a free account at https://console.groq.com/keys and generate an API key.

Step 2 — Add it to PyCharm:

  1. In PyCharm, go to Run → Edit Configurations
  2. Select your script (e.g. 1_1_introduction.py)
  3. Under Environment variables, add:
    GROQ_API_KEY=your_actual_groq_key
  4. Click OK and rerun the program

Alternative: Create a .env file in the project root containing:
GROQ_API_KEY=your_actual_groq_key
All scripts use load_dotenv() to pick this up automatically.

Why Prompt Engineering Alone Isn't Enough

Simple prompt engineering works well for isolated tasks, but breaks down on multi-step problems that require memory across turns, conditional branching based on intermediate results, use of external tools (APIs, databases, calculators), and self-correction when initial outputs are wrong.

From Pipelines to Agent Workflows

An LLM pipeline is a linear chain: input → model → output. An agent workflow adds a control loop: the model can observe its environment, decide on an action, execute it, and observe the result — repeating until the task is complete.

LLM PipelineInputLLMOutputlinear · one-shot · no memoryAgent WorkflowReason+planObserveActtaskstateful · tools · self-correcting

Left: a simple LLM pipeline — one-shot, no memory. Right: an agent workflow cycling through Reason → Act → Observe until done.

1_1_introduction.py
1_1_introduction.py
from dotenv import load_dotenv
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage

# Load variables from the project's .env file
load_dotenv()

llm = ChatGroq(model="llama-3.3-70b-versatile")

# Pipeline: one-shot
response = llm.invoke([HumanMessage("Summarise quantum computing in 2 sentences")])
print(response.content)

Agent Capabilities

Simple Reasoning Agent Wrapper

This script builds a simple multi-step reasoning loop. The agent keeps calling the LLM until it outputs 'DONE', up to a configurable max_steps limit.

1_2_agent_wrapper.py
1_2_agent_wrapper.py
from dotenv import load_dotenv
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage, SystemMessage

# Load variables from the project's .env file
load_dotenv()

llm = ChatGroq(model="llama-3.3-70b-versatile")

def simple_agent(question: str, max_steps: int = 5):
    system = SystemMessage("""You are a reasoning agent. For each step:
1. Think about what you know
2. Give your best answer
3. If fully answered, end your response with DONE
Otherwise continue reasoning.""")
    messages = [system, HumanMessage(question)]
    for step in range(max_steps):
        response = llm.invoke(messages)
        print(f"Step {step+1}: {response.content[:200]}...")
        messages.append(response)
        if "DONE" in response.content:
            print("Agent finished.")
            break
    return messages[-1].content

result = simple_agent("What are the top 3 benefits of using LangGraph? What's the scene of AI in the republic of Korea?")
print(result)

When to Use Agents (vs. simple chains)

Use agents when: the task requires multiple steps with dependencies; the path to a solution isn't known in advance; external data or computation is needed; quality must be verified and potentially retried.

2

LangGraph Fundamentals

⏱ 40 min  ·  Module 2

Learning Objectives

  • Define a typed state schema and explain how nodes read from and write to shared state
  • Construct a StateGraph with nodes, edges, and conditional routing
  • Trace state transitions through a compiled LangGraph execution
  • Understand the difference between tools_condition and a custom should_continue() router

Environment Setup

Module 2 — Install LangGraph

Module 2 introduces LangGraph. Install it in PyCharm's terminal:

python -m pip install --upgrade langgraph langchain-groq langchain-core python-dotenv

Verify the install: In PyCharm's Python Console, type import langgraph; print(langgraph.__version__). You should see a version number like 0.3.x or higher.

What is LangGraph?

LangGraph is a framework built on LangChain for creating stateful, multi-actor AI workflows as graphs. Unlike linear chains, LangGraph lets you define workflows with cycles, conditional branching, and explicit shared state.

STARTAgent Nodellm.invoke(state)has tool_calls?YESTool Noderun toolsloopNOENDShared Statemessages: [...]step_count: interrors: [...]

Agent node runs the LLM → conditional edge checks for tool calls → ToolNode executes and loops back → END when finished.

2.1 — Tool Workflow with tools_condition (Built-in Router)

This script uses LangGraph's built-in tools_condition function to route between the reasoning node and the tool node. tools_condition automatically checks for tool_calls on the last message.

2_1_langgraph_reasoning_tool_workflow.py
2_1_langgraph_reasoning_tool_workflow.py
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from langgraph.graph import StateGraph, MessagesState, START
from langgraph.prebuilt import ToolNode, tools_condition
from dotenv import load_dotenv

# Load variables from the project's .env file
load_dotenv()

# 1. Define a tool
@tool
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b


# 2. Create the LLM and give it access to the tool
llm = ChatGroq(
    model="llama-3.3-70b-versatile",
    temperature=0
)

llm_with_tools = llm.bind_tools([add])


# 3. Define the reasoning node
def reasoning_node(state: MessagesState):
    response = llm_with_tools.invoke(state["messages"])
    return {"messages": [response]}


# 4. Build the workflow
workflow = StateGraph(MessagesState)

workflow.add_node("reason", reasoning_node)
workflow.add_node("tools", ToolNode([add]))

workflow.add_edge(START, "reason")
workflow.add_conditional_edges("reason", tools_condition)
workflow.add_edge("tools", "reason")


# 5. Compile and run
app = workflow.compile()

result = app.invoke({
    "messages": [
        HumanMessage(content="What is 2 + 2? Use the add tool.")
    ]
})

print(result["messages"][-1].content)

2.2 — Conditional Routing with a Custom should_continue()

Instead of the built-in tools_condition, this script writes its own should_continue() function that inspects tool_calls on the last message. This pattern gives you full control over routing logic.

2_2_langgraph_conditional_tool_routing.py
2_2_langgraph_conditional_tool_routing.py
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import ToolNode
from dotenv import load_dotenv


# Load variables from the project's .env file
load_dotenv()


# 1. Define the tool
@tool
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b


# 2. Create the model and bind the tool
llm = ChatGroq(
    model="llama-3.3-70b-versatile",
    temperature=0
)

llm_with_tools = llm.bind_tools([add])


# 3. Reasoning node
def reasoning_node(state: MessagesState):
    response = llm_with_tools.invoke(state["messages"])
    return {"messages": [response]}


# 4. Decide whether to call a tool or finish
def should_continue(state: MessagesState) -> str:
    last_message = state["messages"][-1]

    if getattr(last_message, "tool_calls", None):
        return "continue"

    return "end"


# 5. Build the workflow
workflow = StateGraph(MessagesState)

workflow.add_node("reason", reasoning_node)
workflow.add_node("tools", ToolNode([add]))

workflow.add_edge(START, "reason")

workflow.add_conditional_edges(
    "reason",
    should_continue,
    {
        "continue": "tools",
        "end": END
    }
)

workflow.add_edge("tools", "reason")


# 6. Compile and run
app = workflow.compile()

result = app.invoke({
    "messages": [
        HumanMessage(content="What is 2 + 2? Use the add tool.")
    ]
})

print(result["messages"][-1].content)

Key Difference: tools_condition vs. should_continue()

Build a Simple Reasoning Graph
3

Tool-Using Agents

⏱ 40 min  ·  Module 3

Learning Objectives

  • Define custom tools using the @tool decorator and bind them to an LLM
  • Build a tool-using agent that invokes external APIs and computation tools
  • Route tool call results back through the agent's reasoning loop using conditional edges
  • Build a CSV data-analysis agent using pandas

Environment Setup

Module 3 — Install Additional Packages

Module 3 uses web search, Wikipedia, Yahoo Finance, and pandas. Install all extras:

python -m pip install --upgrade ddgs wikipedia yfinance pandas

ddgs — DuckDuckGo search (the maintained package; uses from ddgs import DDGS)

wikipedia — Wikipedia summaries

yfinance — Yahoo Finance stock prices (free, no API key)

pandas — data analysis for the CSV agent activity

📂 Dataset for PyCharm users: Download sales.csv from this Dropbox link and place it in a data/ subfolder next to your script (e.g. module3/data/sales.csv).

If you haven't installed the Module 1 & 2 packages yet, also run:
pip install langchain-groq langchain-core langgraph python-dotenv

Defining Custom Tools

Tools are Python functions decorated with @tool that give the agent external capabilities. The four tools below cover web search, Wikipedia, arithmetic, and live stock prices — all free, no extra API keys needed.

Agent Nodedecides tool callTool Nodedispatch & collectsearch_webDuckDuckGolookup_wikiWikipediacalculatebuilt-inget_stockyfinanceresult

Agent generates a tool_call → ToolNode dispatches to the right tool → result returned as ToolMessage → Agent continues or finishes.

custom_tools.py
custom_tools.py
"""Reusable tools for the Module 3 general-purpose agent."""

from __future__ import annotations
import ast, operator
from langchain_core.tools import tool
from dotenv import load_dotenv

load_dotenv()


@tool
def search_web(query: str) -> str:
    """Search the web for current information and return up to three results."""
    try:
        from ddgs import DDGS
        results = DDGS().text(query, max_results=3)
        if not results:
            return "No web results found."
        return "\n\n".join(
            f"{item.get('title', 'Untitled')}\n"
            f"{item.get('body', 'No summary available')}\n"
            f"{item.get('href', '')}"
            for item in results
        )
    except Exception as exc:
        return f"Web search failed: {exc}"


@tool
def lookup_wikipedia(topic: str) -> str:
    """Look up a topic on Wikipedia and return a four-sentence summary."""
    try:
        import wikipedia
        return wikipedia.summary(topic, sentences=4, auto_suggest=False)
    except wikipedia.DisambiguationError as exc:
        suggestions = ", ".join(exc.options[:3])
        return f"The topic is ambiguous. Try one of: {suggestions}"
    except wikipedia.PageError:
        return f"No Wikipedia page was found for '{topic}'."
    except Exception as exc:
        return f"Wikipedia lookup failed: {exc}"


_BINARY_OPERATORS = {
    ast.Add: operator.add, ast.Sub: operator.sub,
    ast.Mult: operator.mul, ast.Div: operator.truediv,
    ast.FloorDiv: operator.floordiv, ast.Mod: operator.mod, ast.Pow: operator.pow,
}
_UNARY_OPERATORS = {ast.UAdd: operator.pos, ast.USub: operator.neg}


def _evaluate(node):
    if isinstance(node, ast.Expression): return _evaluate(node.body)
    if isinstance(node, ast.Constant) and type(node.value) in (int, float): return node.value
    if isinstance(node, ast.BinOp) and type(node.op) in _BINARY_OPERATORS:
        return _BINARY_OPERATORS[type(node.op)](_evaluate(node.left), _evaluate(node.right))
    if isinstance(node, ast.UnaryOp) and type(node.op) in _UNARY_OPERATORS:
        return _UNARY_OPERATORS[type(node.op)](_evaluate(node.operand))
    raise ValueError("Only numbers and basic arithmetic operators are allowed.")


@tool
def calculate(expression: str) -> str:
    """Safely evaluate a basic arithmetic expression."""
    try:
        return str(_evaluate(ast.parse(expression, mode="eval")))
    except Exception as exc:
        return f"Calculation failed: {exc}"


@tool
def get_stock_price(ticker: str) -> str:
    """Return the latest available stock price from Yahoo Finance."""
    try:
        import yfinance as yf
        clean_ticker = ticker.strip().upper()
        stock = yf.Ticker(clean_ticker)
        try:
            price = stock.fast_info["lastPrice"]
        except Exception:
            history = stock.history(period="5d")
            if history.empty:
                return f"No recent price was found for {clean_ticker}."
            price = history["Close"].dropna().iloc[-1]
        return f"{clean_ticker}: ${float(price):,.2f}"
    except Exception as exc:
        return f"Stock lookup failed: {exc}"


tools = [search_web, lookup_wikipedia, calculate, get_stock_price]

Building the Tool-Using Agent

tool_using_agent.py
tool_using_agent.py
"""A LangGraph agent that selects and executes general-purpose tools."""

from __future__ import annotations
from typing import Literal
from langchain_core.messages import HumanMessage
from langchain_groq import ChatGroq
from langgraph.graph import END, START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode
from custom_tools import tools   # make sure custom_tools.py is in the same folder
from dotenv import load_dotenv

load_dotenv()

model_with_tools = ChatGroq(
    model="llama-3.3-70b-versatile",
    temperature=0,
    max_retries=2
).bind_tools(tools)


def agent_node(state: MessagesState) -> dict:
    """Ask the model to answer directly or request one or more tools."""
    response = model_with_tools.invoke(state["messages"])
    return {"messages": [response]}


def should_use_tools(state: MessagesState) -> Literal["tools", "end"]:
    """Route requested tool calls to ToolNode; otherwise finish."""
    last_message = state["messages"][-1]
    return "tools" if getattr(last_message, "tool_calls", None) else "end"


workflow = StateGraph(MessagesState)
workflow.add_node("agent", agent_node)
workflow.add_node("tools", ToolNode(tools))

workflow.add_edge(START, "agent")
workflow.add_conditional_edges(
    "agent",
    should_use_tools,
    {"tools": "tools", "end": END},
)
workflow.add_edge("tools", "agent")

app = workflow.compile()


if __name__ == "__main__":
    result = app.invoke(
        {
            "messages": [
                HumanMessage(
                    content=(
                        "Use the appropriate tools to find the latest available "
                        "price of AAPL stock and calculate 1500 * 1.08."
                    )
                )
            ]
        }
    )
    print(result["messages"][-1].content)
CSV Data-Analysis Agent Activity
4

Multi-Agent Collaboration

⏱ 40 min  ·  Module 4

Learning Objectives

  • Design specialized agent roles in a supervisor/worker architecture
  • Implement a supervisor node that routes tasks to worker agents based on state
  • Build and run a complete multi-agent pipeline with three specialized agents
  • Implement a three-agent debate pipeline with a judge

Why Multiple Agents?

A single agent trying to do everything becomes a 'god agent' — complex, unreliable, and hard to debug. Specialized agents are better at their jobs, easier to test, and can work in parallel.

Common patterns:

Supervisor/Worker Pipeline

STARTSupervisorroutes to next agentResearchergathers infoAnalystextracts insightsWriterdrafts reportFINISHENDdispatchreturn to supervisor

Supervisor routes to a specialist (solid), worker returns result (dashed), loop continues until FINISH.

The supervisor_worker_pipeline.py implements a guarded routing strategy: the supervisor's LLM output is always validated against the logical next step, so an out-of-order route is silently corrected.

supervisor_worker_pipeline.py
supervisor_worker_pipeline.py
"""A guarded supervisor/worker workflow with three specialized agents."""

from __future__ import annotations
import operator
from typing import Annotated, Literal, TypedDict
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
from langchain_groq import ChatGroq
from langgraph.graph import END, START, StateGraph
from dotenv import load_dotenv

load_dotenv()

AgentName = Literal["researcher", "analyst", "writer", "FINISH"]

class CollaborativeState(TypedDict):
    messages: Annotated[list[BaseMessage], operator.add]
    task: str
    research_result: str
    analysis_result: str
    final_report: str
    next_agent: AgentName
    route_history: Annotated[list[str], operator.add]

llm = ChatGroq(model="llama-3.3-70b-versatile", temperature=0)

def required_next_agent(state):
    if not state["research_result"]: return "researcher"
    if not state["analysis_result"]: return "analyst"
    if not state["final_report"]:    return "writer"
    return "FINISH"

def supervisor_node(state):
    required = required_next_agent(state)
    if required == "FINISH":
        return {"next_agent": "FINISH", "route_history": ["FINISH"]}
    # ... (see full file for LLM routing logic)
    return {"next_agent": required, "route_history": [required]}

def researcher_node(state):
    response = llm.invoke([
        SystemMessage("You are the Researcher. Gather relevant background from your knowledge."),
        HumanMessage(f"Research this topic:\n{state['task']}"),
    ])
    return {"research_result": response.content, "messages": [response]}

def analyst_node(state):
    response = llm.invoke([
        SystemMessage("You are the Analyst. Extract key insights from the research."),
        HumanMessage(f"Task:\n{state['task']}\n\nResearch:\n{state['research_result']}"),
    ])
    return {"analysis_result": response.content, "messages": [response]}

def writer_node(state):
    response = llm.invoke([
        SystemMessage("You are the Writer. Produce a clear, concise report."),
        HumanMessage(f"Task:\n{state['task']}\n\nResearch:\n{state['research_result']}\n\nAnalysis:\n{state['analysis_result']}"),
    ])
    return {"final_report": response.content, "messages": [response]}

workflow = StateGraph(CollaborativeState)
workflow.add_node("supervisor", supervisor_node)
workflow.add_node("researcher", researcher_node)
workflow.add_node("analyst", analyst_node)
workflow.add_node("writer", writer_node)

workflow.add_edge(START, "supervisor")
workflow.add_conditional_edges("supervisor", lambda s: s["next_agent"],
    {"researcher": "researcher", "analyst": "analyst", "writer": "writer", "FINISH": END})
workflow.add_edge("researcher", "supervisor")
workflow.add_edge("analyst", "supervisor")
workflow.add_edge("writer", "supervisor")
app = workflow.compile()

if __name__ == "__main__":
    topic = "Benefits and risks of using AI agents in university education"
    result = app.invoke({
        "messages": [HumanMessage(content=topic)],
        "task": topic,
        "research_result": "", "analysis_result": "", "final_report": "",
        "next_agent": "researcher", "route_history": [],
    }, config={"recursion_limit": 10})
    print("\nRoute history:", " -> ".join(result["route_history"]))
    print("\nFINAL REPORT\n")
    print(result["final_report"])
Implement a Three-Agent Debate Pipeline
5

Decision Intelligence Agents

⏱ 20 min  ·  Module 5

Learning Objectives

  • Build parallel evaluator agents that score options on qualitative, quantitative, and risk dimensions
  • Combine multi-dimensional scores from parallel agents into a unified decision recommendation
  • Generate a structured recommendation with a confidence score and key trade-offs
  • Use Pydantic structured outputs with Groq's JSON Object Mode

Environment Setup

Module 5 — Install pydantic

Module 5 uses Pydantic for structured outputs. Run:

python -m pip install --upgrade pydantic

pydantic is typically included with modern LangChain, but verify with pip show pydantic in the terminal.

Decision-Support Architecture

Decision intelligence agents support human decision-making — not replace it. They gather evidence, evaluate options from multiple angles, and present structured recommendations with reasoning.

This module's graph fans out to three parallel evaluators (qualitative, quantitative, risk), then fans in to a synthesizer that produces a final structured recommendation with a confidence score.

Decision Question + OptionsQualitativestrategic fitQuantitativeROI · costRiskprobabilitysynthesisRecommendation SynthesizerRecommendation · Confidence · Trade-offs

Three parallel evaluators feed their scores into the Synthesizer which produces a final recommendation with confidence.

Parallel Evaluator Graph

module_5_decision_intelligence.py
module_5_decision_intelligence.py
"""Module 5: Decision Intelligence Agents.

Install:
    pip install -U langgraph langchain-groq pydantic

Set your API key before running:
    Linux/macOS: export GROQ_API_KEY="your-key"
    Windows PowerShell: $env:GROQ_API_KEY="your-key"
    Or add to PyCharm: Run > Edit Configurations > Environment variables
"""

from __future__ import annotations
import json, os
from functools import lru_cache
from typing import Any
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_groq import ChatGroq
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel, Field
from typing_extensions import NotRequired, TypedDict
from dotenv import load_dotenv

load_dotenv()


class QualitativeOptionScore(BaseModel):
    option: str = Field(description="The option name, copied exactly from the input")
    strategic_fit: int = Field(ge=1, le=10)
    stakeholder_acceptance: int = Field(ge=1, le=10)
    implementation_complexity: int = Field(ge=1, le=10)
    long_term_sustainability: int = Field(ge=1, le=10)
    rationale: str

class QualitativeEvaluation(BaseModel):
    scores: list[QualitativeOptionScore]


class DecisionState(TypedDict):
    decision_question: str
    options: list[str]
    decision_context: NotRequired[str]
    qualitative_scores: NotRequired[dict[str, Any]]
    quantitative_scores: NotRequired[dict[str, Any]]
    risk_assessment: NotRequired[dict[str, Any]]
    final_recommendation: NotRequired[dict[str, Any]]
    confidence: NotRequired[float]


@lru_cache(maxsize=1)
def get_llm() -> ChatGroq:
    if not os.getenv("GROQ_API_KEY"):
        raise RuntimeError("GROQ_API_KEY is not set.")
    return ChatGroq(model=os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile"), temperature=0, max_retries=2)


def validate_input(state):
    question = state.get("decision_question", "").strip()
    options = [o.strip() for o in state.get("options", []) if o.strip()]
    if not question: raise ValueError("decision_question cannot be empty.")
    if len(options) < 2: raise ValueError("Provide at least two decision options.")
    return {"decision_question": question, "options": options}

def qualitative_evaluator(state):
    # Scores each option on strategic fit, stakeholder acceptance, etc.
    ...  # see full file for implementation

def quantitative_evaluator(state):
    # Estimates ROI, cost, and time to value for each option
    ...

def risk_assessor(state):
    # Identifies top 3 risks per option with probability and impact
    ...

def recommendation_synthesizer(state):
    # Combines all evaluations into a structured final recommendation
    ...


workflow = StateGraph(DecisionState)
workflow.add_node("validate", validate_input)
workflow.add_node("qualitative", qualitative_evaluator)
workflow.add_node("quantitative", quantitative_evaluator)
workflow.add_node("risks", risk_assessor)
workflow.add_node("recommend", recommendation_synthesizer)

workflow.add_edge(START, "validate")

# Fan out: three evaluators run in parallel
workflow.add_edge("validate", "qualitative")
workflow.add_edge("validate", "quantitative")
workflow.add_edge("validate", "risks")

# Fan in: synthesizer waits for all three
workflow.add_edge(["qualitative", "quantitative", "risks"], "recommend")
workflow.add_edge("recommend", END)

decision_app = workflow.compile()


if __name__ == "__main__":
    output = decision_app.invoke({
        "decision_question": "Which approach should our university use for a student-support chatbot?",
        "options": ["Build an in-house solution", "Buy a commercial platform", "Run a limited hybrid pilot"],
        "decision_context": "The university needs a pilot within six months, has a modest budget, handles sensitive student data.",
    })
    print("\nFINAL RECOMMENDATION")
    print(json.dumps(output["final_recommendation"], indent=2))
6

Scaling Agentic Systems

⏱ 30 min  ·  Module 6

Learning Objectives

  • Identify common failure modes in production agents and apply safeguards to prevent them
  • Apply evaluation metrics to measure agent task completion, reasoning quality, and robustness
  • Configure checkpointing with SQLite, recursion limits, and observability for production
  • Implement retries with exponential backoff and a token budget guard

Environment Setup

Module 6 — Install langgraph-checkpoint-sqlite

Module 6 adds SQLite checkpointing for durable agent state. Install the checkpoint package:

python -m pip install --upgrade langgraph-checkpoint-sqlite

This package allows LangGraph to save conversation checkpoints to a local SQLite database file, enabling resumable workflows and human-in-the-loop interrupts.

Verify: python -c "from langgraph.checkpoint.sqlite import SqliteSaver; print('OK')"

Reliability Challenges

Agentic systems face unique failure modes in production:

Agent Loopstep_count < MAXrecursion_limit = 25 stepsCheckpointerSQLite / PostgrespersistHuman Reviewinterrupt pointinspectLangSmithtraces · latencyError Handlerretry · backoffRate Limitertoken budget

Recursion limit, Checkpointer, Human Review, LangSmith observability, Error Handler, and Rate Limiter guard production agents.

Production-Ready Agent with Safeguards

The module_6_scaling_agentic_systems.py script bundles all major safeguards into one runnable example:

module_6_scaling_agentic_systems.py
module_6_scaling_agentic_systems.py — key patterns
"""Module 6: Scaling Agentic Systems.

Install:
    pip install -U langgraph langchain-groq langgraph-checkpoint-sqlite

Run the demonstration:
    python module_6_scaling_agentic_systems.py

Run the evaluation suite (uses live model calls):
    python module_6_scaling_agentic_systems.py --eval

Require human approval before returning the answer:
    python module_6_scaling_agentic_systems.py --human-review
"""

from __future__ import annotations
import os, time, uuid, argparse
from functools import lru_cache
from langchain_core.rate_limiters import InMemoryRateLimiter
from langchain_groq import ChatGroq
from langgraph.checkpoint.sqlite import SqliteSaver
from langgraph.graph import END, START, StateGraph
from langgraph.types import RetryPolicy, interrupt
from dotenv import load_dotenv

load_dotenv()

DEFAULT_MAX_AGENT_STEPS = 6
DEFAULT_TOKEN_BUDGET    = 6_000

# ── 1. Rate-limited, tool-bound model ──────────────────────────────────
@lru_cache(maxsize=1)
def get_agent_model():
    if not os.getenv("GROQ_API_KEY"):
        raise RuntimeError("GROQ_API_KEY is not set.")
    limiter = InMemoryRateLimiter(requests_per_second=0.5, check_every_n_seconds=0.1, max_bucket_size=1)
    model = ChatGroq(model="llama-3.3-70b-versatile", temperature=0, max_tokens=600, max_retries=0, timeout=30, rate_limiter=limiter)
    return model.bind_tools(TOOLS)


# ── 2. Bounded agent node with token guard ──────────────────────────────
def safe_agent_node(state):
    max_steps  = min(state.get("max_steps", DEFAULT_MAX_AGENT_STEPS), 10)
    step_count = state.get("step_count", 0)
    if step_count >= max_steps:
        return {"status": "stopped", "final_answer": "", "errors": ["Maximum steps reached."]}
    # ... invoke model, track tokens, update state
    ...


# ── 3. Recoverable tool execution ───────────────────────────────────────
def execute_tools_node(state):
    # Expected tool failures become ToolMessages; unexpected ones are caught
    # so a single tool error does not crash the whole workflow.
    ...


# ── 4. Optional human-review interrupt ──────────────────────────────────
def human_review_node(state):
    decision = interrupt({"question": "Approve this agent answer?", "answer": state.get("final_answer", "")})
    approved = decision if isinstance(decision, bool) else decision.get("approved", False)
    return {"status": "approved" if approved else "rejected"}


# ── 5. Graph with retries and checkpointing ─────────────────────────────
def build_agent(checkpointer=None):
    workflow = StateGraph(AgentState)
    workflow.add_node("agent", safe_agent_node,
        retry_policy=RetryPolicy(max_attempts=3, initial_interval=1.0, backoff_factor=2.0, jitter=True))
    workflow.add_node("tools", execute_tools_node)
    workflow.add_node("human_review", human_review_node)
    workflow.add_edge(START, "agent")
    workflow.add_conditional_edges("agent", route_after_agent,
        {"tools": "tools", "review": "human_review", "end": END})
    workflow.add_edge("tools", "agent")
    workflow.add_edge("human_review", END)
    return workflow.compile(checkpointer=checkpointer)


# ── 6. Entry point — SQLite checkpoint DB ──────────────────────────────
if __name__ == "__main__":
    with SqliteSaver.from_conn_string("module_6_checkpoints.sqlite") as checkpointer:
        app = build_agent(checkpointer)
        thread_id = f"demo-{uuid.uuid4()}"
        config = {"configurable": {"thread_id": thread_id}, "recursion_limit": 25}
        result = app.invoke(initial_state("What is the late-registration fee for 3 students?"), config=config)
        print(result.get("final_answer", "No answer released."))

Evaluation Harness

Module 6 includes a built-in evaluation harness with two test cases. Run it with:

Terminal
python module_6_scaling_agentic_systems.py --eval

The harness measures five dimensions per test case: