Agentic AI & LangGraph
A hands-on tutorial for building, scaling, and evaluating agentic AI systems with LangGraph — from your first reasoning loop to production-ready multi-agent pipelines.
This tutorial requires PyCharm IDE and Python 3.11. Download PyCharm at jetbrains.com/pycharm and Python 3.11 at python.org/downloads. A free Groq API key is also required — get one at https://console.groq.com/keys. Each module page has a dedicated Setup section with exact install commands.
All modules available as a ready-to-run notebook — no local install needed.
Course Outline — 6 Modules
Prerequisites
Quick Start
1. Install PyCharm and select Python 3.11 as the project interpreter.
2. Get your Groq API key from console.groq.com/keys.
3. In PyCharm go to Run → Edit Configurations, pick your script, and add environment variable GROQ_API_KEY=your_key. Or create a .env file in the project root with that line — every script calls load_dotenv().
4. Open Module 1 and follow the Setup section for per-module package installs.
About the Instructor
Zayed University, Abu Dhabi, UAE
Certified in Building Agentic AI Applications with Large Language Models — ensuring participants gain practical insights aligned with modern industry practices.
Biography
Dr. Mohammad Amin Kuhail is an Associate Professor at Zayed University, specializing in Human-Centered AI, conversational agents, and AI-supported decision systems. His research explores the design and evaluation of AI assistants, multi-agent conversational systems, and agentic AI environments powered by large language models.
He has published extensively on chatbots, collaborative AI systems, AI-assisted learning environments, and AI-supported decision-making. His work often investigates how AI agents can collaborate with humans and with other AI agents to support learning, brainstorming, decision-making, and problem-solving.
Dr. Kuhail actively develops experimental AI platforms that integrate large language models, retrieval-augmented generation (RAG), and multi-agent architectures for applications in education, decision support, and collaborative problem-solving.
Research Areas
- Human-Centered AI
- Conversational Agents
- AI-Supported Decision Systems
- Multi-Agent AI Systems
- LLM-Powered Applications
- AI in Education
Publication Topics
- Chatbots and conversational AI in education
- Collaborative AI systems and multi-agent architectures
- AI-assisted learning environments
- AI-supported decision-making systems
- Retrieval-Augmented Generation (RAG) for educational applications
What You Will Build
- A simple reasoning loop agent using Groq + LangChain (Module 1)
- Stateful tool-calling graphs with LangGraph (Modules 2 & 3)
- Multi-agent pipelines: supervisor/worker and debate systems (Module 4)
- A parallel decision-intelligence evaluator with Pydantic structured outputs (Module 5)
- A production-hardened agent with checkpointing, retries, and evaluation (Module 6)
Introduction to Agentic AI
Learning Objectives
- Identify the limitations of prompt engineering for multi-step, tool-dependent tasks
- Contrast LLM pipelines with agent workflows and select the right approach for a given task
- List and describe the core agent capabilities: reasoning, planning, tool use, and self-refinement
- Configure a Groq LLM in PyCharm and run your first agentic Python script
Environment Requirements
PyCharm + Python 3.11 Required
This tutorial requires PyCharm IDE and Python 3.11. Earlier Python versions (3.9, 3.10) may work but are unsupported — use Python 3.11 (not Python 3.12 or 3.13) for full compatibility with all LangGraph packages.
Download PyCharm: https://www.jetbrains.com/pycharm/
Download Python 3.11: https://www.python.org/downloads/release/python-3110/
Module 1 — Install Required Packages
Open PyCharm's built-in terminal (View → Tool Windows → Terminal) and run:
Note: langchain-groq is not installed in PyCharm by default. You can also install it via File → Settings → Project → Python Interpreter → + (Add Package) and search for langchain-groq.
Groq API Key Setup
Step 1 — Get your key: Create a free account at https://console.groq.com/keys and generate an API key.
Step 2 — Add it to PyCharm:
1. In PyCharm, go to Run → Edit Configurations
2. Select your script (e.g. 1_1_introduction.py)
3. Under Environment variables, add:
GROQ_API_KEY=your_actual_groq_key
4. Click OK and rerun the program
Alternative: Create a .env file in the project root containing:GROQ_API_KEY=your_actual_groq_key
All scripts use load_dotenv() to pick this up automatically.
Why Prompt Engineering Alone Isn't Enough
Simple prompt engineering works well for isolated tasks, but breaks down on multi-step problems that require memory across turns, conditional branching based on intermediate results, use of external tools (APIs, databases, calculators), and self-correction when initial outputs are wrong.
- Memory across turns
- Conditional branching on intermediate results
- External tool use (APIs, databases, calculators)
- Self-correction when initial outputs are wrong
From Pipelines to Agent Workflows
An LLM pipeline is a linear chain: input → model → output. An agent workflow adds a control loop: the model can observe its environment, decide on an action, execute it, and observe the result — repeating until the task is complete.
Left: a simple LLM pipeline — one-shot, no memory. Right: an agent workflow cycling through Reason → Act → Observe until done.
from dotenv import load_dotenv
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage
# Load variables from the project's .env file
load_dotenv()
llm = ChatGroq(model="llama-3.3-70b-versatile")
# Pipeline: one-shot
response = llm.invoke([HumanMessage("Summarise quantum computing in 2 sentences")])
print(response.content)Agent Capabilities
- Reasoning: Chain-of-thought, step-by-step problem decomposition
- Planning: Breaking a goal into subgoals, choosing the right order
- Tool Use: Calling external APIs, searching the web, running code
- Self-Refinement: Evaluating its own output and retrying if quality is insufficient
- Memory: Maintaining context across multiple turns or sessions
Simple Reasoning Agent Wrapper
This script builds a simple multi-step reasoning loop. The agent keeps calling the LLM until it outputs 'DONE', up to a configurable max_steps limit.
1_2_agent_wrapper.pyfrom dotenv import load_dotenv
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage, SystemMessage
# Load variables from the project's .env file
load_dotenv()
llm = ChatGroq(model="llama-3.3-70b-versatile")
def simple_agent(question: str, max_steps: int = 5):
system = SystemMessage("""You are a reasoning agent. For each step:
1. Think about what you know
2. Give your best answer
3. If fully answered, end your response with DONE
Otherwise continue reasoning.""")
messages = [system, HumanMessage(question)]
for step in range(max_steps):
response = llm.invoke(messages)
print(f"Step {step+1}: {response.content[:200]}...")
messages.append(response)
if "DONE" in response.content:
print("Agent finished.")
break
return messages[-1].content
result = simple_agent("What are the top 3 benefits of using LangGraph? What's the scene of AI in the republic of Korea?")
print(result)When to Use Agents (vs. simple chains)
Use agents when: the task requires multiple steps with dependencies; the path to a solution isn't known in advance; external data or computation is needed; quality must be verified and potentially retried.
LangGraph Fundamentals
Learning Objectives
- Define a typed state schema and explain how nodes read from and write to shared state
- Construct a StateGraph with nodes, edges, and conditional routing
- Trace state transitions through a compiled LangGraph execution
- Understand the difference between tools_condition and a custom should_continue() router
Environment Setup
Module 2 — Install LangGraph
Module 2 introduces LangGraph. Install it in PyCharm's terminal:
Verify the install: In PyCharm's Python Console, type import langgraph; print(langgraph.__version__). You should see a version number like 0.3.x or higher.
What is LangGraph?
LangGraph is a framework built on LangChain for creating stateful, multi-actor AI workflows as graphs. Unlike linear chains, LangGraph lets you define workflows with cycles, conditional branching, and explicit shared state.
Agent node runs the LLM → conditional edge checks for tool calls → ToolNode executes and loops back → END when finished.
- State: A typed dictionary shared across all nodes. Nodes read from and write to state.
- Nodes: Python functions that receive the current state and return updates to it.
- Edges: Connections between nodes — unconditional or conditional.
- Graph: The assembled StateGraph defining the full workflow.
2.1 — Tool Workflow with tools_condition (Built-in Router)
This script uses LangGraph's built-in tools_condition function to route between the reasoning node and the tool node. tools_condition automatically checks for tool_calls on the last message.
2_1_langgraph_reasoning_tool_workflow.pyfrom langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from langgraph.graph import StateGraph, MessagesState, START
from langgraph.prebuilt import ToolNode, tools_condition
from dotenv import load_dotenv
# Load variables from the project's .env file
load_dotenv()
# 1. Define a tool
@tool
def add(a: int, b: int) -> int:
"""Add two integers."""
return a + b
# 2. Create the LLM and give it access to the tool
llm = ChatGroq(
model="llama-3.3-70b-versatile",
temperature=0
)
llm_with_tools = llm.bind_tools([add])
# 3. Define the reasoning node
def reasoning_node(state: MessagesState):
response = llm_with_tools.invoke(state["messages"])
return {"messages": [response]}
# 4. Build the workflow
workflow = StateGraph(MessagesState)
workflow.add_node("reason", reasoning_node)
workflow.add_node("tools", ToolNode([add]))
workflow.add_edge(START, "reason")
workflow.add_conditional_edges("reason", tools_condition)
workflow.add_edge("tools", "reason")
# 5. Compile and run
app = workflow.compile()
result = app.invoke({
"messages": [
HumanMessage(content="What is 2 + 2? Use the add tool.")
]
})
print(result["messages"][-1].content)2.2 — Conditional Routing with a Custom should_continue()
Instead of the built-in tools_condition, this script writes its own should_continue() function that inspects tool_calls on the last message. This pattern gives you full control over routing logic.
2_2_langgraph_conditional_tool_routing.pyfrom langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import ToolNode
from dotenv import load_dotenv
# Load variables from the project's .env file
load_dotenv()
# 1. Define the tool
@tool
def add(a: int, b: int) -> int:
"""Add two integers."""
return a + b
# 2. Create the model and bind the tool
llm = ChatGroq(
model="llama-3.3-70b-versatile",
temperature=0
)
llm_with_tools = llm.bind_tools([add])
# 3. Reasoning node
def reasoning_node(state: MessagesState):
response = llm_with_tools.invoke(state["messages"])
return {"messages": [response]}
# 4. Decide whether to call a tool or finish
def should_continue(state: MessagesState) -> str:
last_message = state["messages"][-1]
if getattr(last_message, "tool_calls", None):
return "continue"
return "end"
# 5. Build the workflow
workflow = StateGraph(MessagesState)
workflow.add_node("reason", reasoning_node)
workflow.add_node("tools", ToolNode([add]))
workflow.add_edge(START, "reason")
workflow.add_conditional_edges(
"reason",
should_continue,
{
"continue": "tools",
"end": END
}
)
workflow.add_edge("tools", "reason")
# 6. Compile and run
app = workflow.compile()
result = app.invoke({
"messages": [
HumanMessage(content="What is 2 + 2? Use the add tool.")
]
})
print(result["messages"][-1].content)Key Difference: tools_condition vs. should_continue()
- 2_1 uses LangGraph's built-in tools_condition — it automatically checks for tool_calls and routes accordingly. Quickest approach for standard tool-calling graphs.
- 2_2 uses your own should_continue() function to inspect tool_calls. Use this pattern when you need custom routing logic (e.g. only call certain tools after a condition is met).
Tool-Using Agents
Learning Objectives
- Define custom tools using the @tool decorator and bind them to an LLM
- Build a tool-using agent that invokes external APIs and computation tools
- Route tool call results back through the agent's reasoning loop using conditional edges
- Build a CSV data-analysis agent using pandas
Environment Setup
Module 3 — Install Additional Packages
Module 3 uses web search, Wikipedia, Yahoo Finance, and pandas. Install all extras:
ddgs — DuckDuckGo search (the maintained package; uses from ddgs import DDGS)
wikipedia — Wikipedia summaries
yfinance — Yahoo Finance stock prices (free, no API key)
pandas — data analysis for the CSV agent activity
📂 Dataset for PyCharm users: Download sales.csv from this Dropbox link and place it in a data/ subfolder next to your script (e.g. module3/data/sales.csv).
If you haven't installed the Module 1 & 2 packages yet, also run:pip install langchain-groq langchain-core langgraph python-dotenv
Defining Custom Tools
Tools are Python functions decorated with @tool that give the agent external capabilities. The four tools below cover web search, Wikipedia, arithmetic, and live stock prices — all free, no extra API keys needed.
Agent generates a tool_call → ToolNode dispatches to the right tool → result returned as ToolMessage → Agent continues or finishes.
"""Reusable tools for the Module 3 general-purpose agent."""
from __future__ import annotations
import ast, operator
from langchain_core.tools import tool
from dotenv import load_dotenv
load_dotenv()
@tool
def search_web(query: str) -> str:
"""Search the web for current information and return up to three results."""
try:
from ddgs import DDGS
results = DDGS().text(query, max_results=3)
if not results:
return "No web results found."
return "\n\n".join(
f"{item.get('title', 'Untitled')}\n"
f"{item.get('body', 'No summary available')}\n"
f"{item.get('href', '')}"
for item in results
)
except Exception as exc:
return f"Web search failed: {exc}"
@tool
def lookup_wikipedia(topic: str) -> str:
"""Look up a topic on Wikipedia and return a four-sentence summary."""
try:
import wikipedia
return wikipedia.summary(topic, sentences=4, auto_suggest=False)
except wikipedia.DisambiguationError as exc:
suggestions = ", ".join(exc.options[:3])
return f"The topic is ambiguous. Try one of: {suggestions}"
except wikipedia.PageError:
return f"No Wikipedia page was found for '{topic}'."
except Exception as exc:
return f"Wikipedia lookup failed: {exc}"
_BINARY_OPERATORS = {
ast.Add: operator.add, ast.Sub: operator.sub,
ast.Mult: operator.mul, ast.Div: operator.truediv,
ast.FloorDiv: operator.floordiv, ast.Mod: operator.mod, ast.Pow: operator.pow,
}
_UNARY_OPERATORS = {ast.UAdd: operator.pos, ast.USub: operator.neg}
def _evaluate(node):
if isinstance(node, ast.Expression): return _evaluate(node.body)
if isinstance(node, ast.Constant) and type(node.value) in (int, float): return node.value
if isinstance(node, ast.BinOp) and type(node.op) in _BINARY_OPERATORS:
return _BINARY_OPERATORS[type(node.op)](_evaluate(node.left), _evaluate(node.right))
if isinstance(node, ast.UnaryOp) and type(node.op) in _UNARY_OPERATORS:
return _UNARY_OPERATORS[type(node.op)](_evaluate(node.operand))
raise ValueError("Only numbers and basic arithmetic operators are allowed.")
@tool
def calculate(expression: str) -> str:
"""Safely evaluate a basic arithmetic expression."""
try:
return str(_evaluate(ast.parse(expression, mode="eval")))
except Exception as exc:
return f"Calculation failed: {exc}"
@tool
def get_stock_price(ticker: str) -> str:
"""Return the latest available stock price from Yahoo Finance."""
try:
import yfinance as yf
clean_ticker = ticker.strip().upper()
stock = yf.Ticker(clean_ticker)
try:
price = stock.fast_info["lastPrice"]
except Exception:
history = stock.history(period="5d")
if history.empty:
return f"No recent price was found for {clean_ticker}."
price = history["Close"].dropna().iloc[-1]
return f"{clean_ticker}: ${float(price):,.2f}"
except Exception as exc:
return f"Stock lookup failed: {exc}"
tools = [search_web, lookup_wikipedia, calculate, get_stock_price]Building the Tool-Using Agent
tool_using_agent.py"""A LangGraph agent that selects and executes general-purpose tools."""
from __future__ import annotations
from typing import Literal
from langchain_core.messages import HumanMessage
from langchain_groq import ChatGroq
from langgraph.graph import END, START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode
from custom_tools import tools # make sure custom_tools.py is in the same folder
from dotenv import load_dotenv
load_dotenv()
model_with_tools = ChatGroq(
model="llama-3.3-70b-versatile",
temperature=0,
max_retries=2
).bind_tools(tools)
def agent_node(state: MessagesState) -> dict:
"""Ask the model to answer directly or request one or more tools."""
response = model_with_tools.invoke(state["messages"])
return {"messages": [response]}
def should_use_tools(state: MessagesState) -> Literal["tools", "end"]:
"""Route requested tool calls to ToolNode; otherwise finish."""
last_message = state["messages"][-1]
return "tools" if getattr(last_message, "tool_calls", None) else "end"
workflow = StateGraph(MessagesState)
workflow.add_node("agent", agent_node)
workflow.add_node("tools", ToolNode(tools))
workflow.add_edge(START, "agent")
workflow.add_conditional_edges(
"agent",
should_use_tools,
{"tools": "tools", "end": END},
)
workflow.add_edge("tools", "agent")
app = workflow.compile()
if __name__ == "__main__":
result = app.invoke(
{
"messages": [
HumanMessage(
content=(
"Use the appropriate tools to find the latest available "
"price of AAPL stock and calculate 1500 * 1.08."
)
)
]
}
)
print(result["messages"][-1].content)Multi-Agent Collaboration
Learning Objectives
- Design specialized agent roles in a supervisor/worker architecture
- Implement a supervisor node that routes tasks to worker agents based on state
- Build and run a complete multi-agent pipeline with three specialized agents
- Implement a three-agent debate pipeline with a judge
Why Multiple Agents?
A single agent trying to do everything becomes a 'god agent' — complex, unreliable, and hard to debug. Specialized agents are better at their jobs, easier to test, and can work in parallel.
Common patterns:
- Supervisor/Worker: A coordinator agent delegates tasks to specialized workers
- Pipeline: Agents pass work sequentially, each adding value
- Debate/Adversarial: Agents argue opposite positions to find the best answer
- Parallel: Multiple agents work simultaneously on different subtasks
Supervisor/Worker Pipeline
Supervisor routes to a specialist (solid), worker returns result (dashed), loop continues until FINISH.
The supervisor_worker_pipeline.py implements a guarded routing strategy: the supervisor's LLM output is always validated against the logical next step, so an out-of-order route is silently corrected.
supervisor_worker_pipeline.py"""A guarded supervisor/worker workflow with three specialized agents."""
from __future__ import annotations
import operator
from typing import Annotated, Literal, TypedDict
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
from langchain_groq import ChatGroq
from langgraph.graph import END, START, StateGraph
from dotenv import load_dotenv
load_dotenv()
AgentName = Literal["researcher", "analyst", "writer", "FINISH"]
class CollaborativeState(TypedDict):
messages: Annotated[list[BaseMessage], operator.add]
task: str
research_result: str
analysis_result: str
final_report: str
next_agent: AgentName
route_history: Annotated[list[str], operator.add]
llm = ChatGroq(model="llama-3.3-70b-versatile", temperature=0)
def required_next_agent(state):
if not state["research_result"]: return "researcher"
if not state["analysis_result"]: return "analyst"
if not state["final_report"]: return "writer"
return "FINISH"
def supervisor_node(state):
required = required_next_agent(state)
if required == "FINISH":
return {"next_agent": "FINISH", "route_history": ["FINISH"]}
# ... (see full file for LLM routing logic)
return {"next_agent": required, "route_history": [required]}
def researcher_node(state):
response = llm.invoke([
SystemMessage("You are the Researcher. Gather relevant background from your knowledge."),
HumanMessage(f"Research this topic:\n{state['task']}"),
])
return {"research_result": response.content, "messages": [response]}
def analyst_node(state):
response = llm.invoke([
SystemMessage("You are the Analyst. Extract key insights from the research."),
HumanMessage(f"Task:\n{state['task']}\n\nResearch:\n{state['research_result']}"),
])
return {"analysis_result": response.content, "messages": [response]}
def writer_node(state):
response = llm.invoke([
SystemMessage("You are the Writer. Produce a clear, concise report."),
HumanMessage(f"Task:\n{state['task']}\n\nResearch:\n{state['research_result']}\n\nAnalysis:\n{state['analysis_result']}"),
])
return {"final_report": response.content, "messages": [response]}
workflow = StateGraph(CollaborativeState)
workflow.add_node("supervisor", supervisor_node)
workflow.add_node("researcher", researcher_node)
workflow.add_node("analyst", analyst_node)
workflow.add_node("writer", writer_node)
workflow.add_edge(START, "supervisor")
workflow.add_conditional_edges("supervisor", lambda s: s["next_agent"],
{"researcher": "researcher", "analyst": "analyst", "writer": "writer", "FINISH": END})
workflow.add_edge("researcher", "supervisor")
workflow.add_edge("analyst", "supervisor")
workflow.add_edge("writer", "supervisor")
app = workflow.compile()
if __name__ == "__main__":
topic = "Benefits and risks of using AI agents in university education"
result = app.invoke({
"messages": [HumanMessage(content=topic)],
"task": topic,
"research_result": "", "analysis_result": "", "final_report": "",
"next_agent": "researcher", "route_history": [],
}, config={"recursion_limit": 10})
print("\nRoute history:", " -> ".join(result["route_history"]))
print("\nFINAL REPORT\n")
print(result["final_report"])Decision Intelligence Agents
Learning Objectives
- Build parallel evaluator agents that score options on qualitative, quantitative, and risk dimensions
- Combine multi-dimensional scores from parallel agents into a unified decision recommendation
- Generate a structured recommendation with a confidence score and key trade-offs
- Use Pydantic structured outputs with Groq's JSON Object Mode
Environment Setup
Module 5 — Install pydantic
Module 5 uses Pydantic for structured outputs. Run:
pydantic is typically included with modern LangChain, but verify with pip show pydantic in the terminal.
Decision-Support Architecture
Decision intelligence agents support human decision-making — not replace it. They gather evidence, evaluate options from multiple angles, and present structured recommendations with reasoning.
This module's graph fans out to three parallel evaluators (qualitative, quantitative, risk), then fans in to a synthesizer that produces a final structured recommendation with a confidence score.
Three parallel evaluators feed their scores into the Synthesizer which produces a final recommendation with confidence.
Parallel Evaluator Graph
module_5_decision_intelligence.py"""Module 5: Decision Intelligence Agents.
Install:
pip install -U langgraph langchain-groq pydantic
Set your API key before running:
Linux/macOS: export GROQ_API_KEY="your-key"
Windows PowerShell: $env:GROQ_API_KEY="your-key"
Or add to PyCharm: Run > Edit Configurations > Environment variables
"""
from __future__ import annotations
import json, os
from functools import lru_cache
from typing import Any
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_groq import ChatGroq
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel, Field
from typing_extensions import NotRequired, TypedDict
from dotenv import load_dotenv
load_dotenv()
class QualitativeOptionScore(BaseModel):
option: str = Field(description="The option name, copied exactly from the input")
strategic_fit: int = Field(ge=1, le=10)
stakeholder_acceptance: int = Field(ge=1, le=10)
implementation_complexity: int = Field(ge=1, le=10)
long_term_sustainability: int = Field(ge=1, le=10)
rationale: str
class QualitativeEvaluation(BaseModel):
scores: list[QualitativeOptionScore]
class DecisionState(TypedDict):
decision_question: str
options: list[str]
decision_context: NotRequired[str]
qualitative_scores: NotRequired[dict[str, Any]]
quantitative_scores: NotRequired[dict[str, Any]]
risk_assessment: NotRequired[dict[str, Any]]
final_recommendation: NotRequired[dict[str, Any]]
confidence: NotRequired[float]
@lru_cache(maxsize=1)
def get_llm() -> ChatGroq:
if not os.getenv("GROQ_API_KEY"):
raise RuntimeError("GROQ_API_KEY is not set.")
return ChatGroq(model=os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile"), temperature=0, max_retries=2)
def validate_input(state):
question = state.get("decision_question", "").strip()
options = [o.strip() for o in state.get("options", []) if o.strip()]
if not question: raise ValueError("decision_question cannot be empty.")
if len(options) < 2: raise ValueError("Provide at least two decision options.")
return {"decision_question": question, "options": options}
def qualitative_evaluator(state):
# Scores each option on strategic fit, stakeholder acceptance, etc.
... # see full file for implementation
def quantitative_evaluator(state):
# Estimates ROI, cost, and time to value for each option
...
def risk_assessor(state):
# Identifies top 3 risks per option with probability and impact
...
def recommendation_synthesizer(state):
# Combines all evaluations into a structured final recommendation
...
workflow = StateGraph(DecisionState)
workflow.add_node("validate", validate_input)
workflow.add_node("qualitative", qualitative_evaluator)
workflow.add_node("quantitative", quantitative_evaluator)
workflow.add_node("risks", risk_assessor)
workflow.add_node("recommend", recommendation_synthesizer)
workflow.add_edge(START, "validate")
# Fan out: three evaluators run in parallel
workflow.add_edge("validate", "qualitative")
workflow.add_edge("validate", "quantitative")
workflow.add_edge("validate", "risks")
# Fan in: synthesizer waits for all three
workflow.add_edge(["qualitative", "quantitative", "risks"], "recommend")
workflow.add_edge("recommend", END)
decision_app = workflow.compile()
if __name__ == "__main__":
output = decision_app.invoke({
"decision_question": "Which approach should our university use for a student-support chatbot?",
"options": ["Build an in-house solution", "Buy a commercial platform", "Run a limited hybrid pilot"],
"decision_context": "The university needs a pilot within six months, has a modest budget, handles sensitive student data.",
})
print("\nFINAL RECOMMENDATION")
print(json.dumps(output["final_recommendation"], indent=2))Scaling Agentic Systems
Learning Objectives
- Identify common failure modes in production agents and apply safeguards to prevent them
- Apply evaluation metrics to measure agent task completion, reasoning quality, and robustness
- Configure checkpointing with SQLite, recursion limits, and observability for production
- Implement retries with exponential backoff and a token budget guard
Environment Setup
Module 6 — Install langgraph-checkpoint-sqlite
Module 6 adds SQLite checkpointing for durable agent state. Install the checkpoint package:
This package allows LangGraph to save conversation checkpoints to a local SQLite database file, enabling resumable workflows and human-in-the-loop interrupts.
Verify: python -c "from langgraph.checkpoint.sqlite import SqliteSaver; print('OK')"
Reliability Challenges
Agentic systems face unique failure modes in production:
- Hallucination cascades: One incorrect reasoning step compounds into wrong answers
- Tool reliability: External APIs fail; agents must handle errors gracefully
- Infinite loops: Agents without termination conditions can run indefinitely
- Context window limits: Long reasoning chains exhaust the model's context
- Cost overruns: Unguarded token usage leads to unexpected API costs
- Race conditions: Parallel tool calls may produce conflicting state updates
Recursion limit, Checkpointer, Human Review, LangSmith observability, Error Handler, and Rate Limiter guard production agents.
Production-Ready Agent with Safeguards
The module_6_scaling_agentic_systems.py script bundles all major safeguards into one runnable example:
- Maximum agent steps + LangGraph recursion_limit
- Workflow-level token budget with approximate context trimming
- Retries with exponential backoff for transient model/API failures
- Local rate limiter (InMemoryRateLimiter)
- Recoverable tool errors returned as ToolMessage, not crashes
- SQLite checkpointing with a stable thread ID (langgraph-checkpoint-sqlite)
- Optional human-review interrupt before releasing the final answer
- Optional LangSmith tracing via environment variables
- Built-in evaluation harness for completion, tool accuracy, and robustness
"""Module 6: Scaling Agentic Systems.
Install:
pip install -U langgraph langchain-groq langgraph-checkpoint-sqlite
Run the demonstration:
python module_6_scaling_agentic_systems.py
Run the evaluation suite (uses live model calls):
python module_6_scaling_agentic_systems.py --eval
Require human approval before returning the answer:
python module_6_scaling_agentic_systems.py --human-review
"""
from __future__ import annotations
import os, time, uuid, argparse
from functools import lru_cache
from langchain_core.rate_limiters import InMemoryRateLimiter
from langchain_groq import ChatGroq
from langgraph.checkpoint.sqlite import SqliteSaver
from langgraph.graph import END, START, StateGraph
from langgraph.types import RetryPolicy, interrupt
from dotenv import load_dotenv
load_dotenv()
DEFAULT_MAX_AGENT_STEPS = 6
DEFAULT_TOKEN_BUDGET = 6_000
# ── 1. Rate-limited, tool-bound model ──────────────────────────────────
@lru_cache(maxsize=1)
def get_agent_model():
if not os.getenv("GROQ_API_KEY"):
raise RuntimeError("GROQ_API_KEY is not set.")
limiter = InMemoryRateLimiter(requests_per_second=0.5, check_every_n_seconds=0.1, max_bucket_size=1)
model = ChatGroq(model="llama-3.3-70b-versatile", temperature=0, max_tokens=600, max_retries=0, timeout=30, rate_limiter=limiter)
return model.bind_tools(TOOLS)
# ── 2. Bounded agent node with token guard ──────────────────────────────
def safe_agent_node(state):
max_steps = min(state.get("max_steps", DEFAULT_MAX_AGENT_STEPS), 10)
step_count = state.get("step_count", 0)
if step_count >= max_steps:
return {"status": "stopped", "final_answer": "", "errors": ["Maximum steps reached."]}
# ... invoke model, track tokens, update state
...
# ── 3. Recoverable tool execution ───────────────────────────────────────
def execute_tools_node(state):
# Expected tool failures become ToolMessages; unexpected ones are caught
# so a single tool error does not crash the whole workflow.
...
# ── 4. Optional human-review interrupt ──────────────────────────────────
def human_review_node(state):
decision = interrupt({"question": "Approve this agent answer?", "answer": state.get("final_answer", "")})
approved = decision if isinstance(decision, bool) else decision.get("approved", False)
return {"status": "approved" if approved else "rejected"}
# ── 5. Graph with retries and checkpointing ─────────────────────────────
def build_agent(checkpointer=None):
workflow = StateGraph(AgentState)
workflow.add_node("agent", safe_agent_node,
retry_policy=RetryPolicy(max_attempts=3, initial_interval=1.0, backoff_factor=2.0, jitter=True))
workflow.add_node("tools", execute_tools_node)
workflow.add_node("human_review", human_review_node)
workflow.add_edge(START, "agent")
workflow.add_conditional_edges("agent", route_after_agent,
{"tools": "tools", "review": "human_review", "end": END})
workflow.add_edge("tools", "agent")
workflow.add_edge("human_review", END)
return workflow.compile(checkpointer=checkpointer)
# ── 6. Entry point — SQLite checkpoint DB ──────────────────────────────
if __name__ == "__main__":
with SqliteSaver.from_conn_string("module_6_checkpoints.sqlite") as checkpointer:
app = build_agent(checkpointer)
thread_id = f"demo-{uuid.uuid4()}"
config = {"configurable": {"thread_id": thread_id}, "recursion_limit": 25}
result = app.invoke(initial_state("What is the late-registration fee for 3 students?"), config=config)
print(result.get("final_answer", "No answer released."))Evaluation Harness
Module 6 includes a built-in evaluation harness with two test cases. Run it with:
python module_6_scaling_agentic_systems.py --evalThe harness measures five dimensions per test case:
- Task completion: Did the agent complete the assigned goal?
- Tool call accuracy: Did it call the right tools with the right arguments?
- Output quality: Did the final answer contain the expected facts?
- Efficiency: Did it reach the goal within the expected step count?
- Robustness: Did it stay within step and token budgets?