Agents From Scratch - A Guide to Building AI Agents from the Ground Up

Project Overview

This repository is a guide to building intelligent agents from scratch. It progressively constructs an "environment-aware" agent capable of managing your emails through the Gmail API. Developed by the LangChain AI team, this project is an excellent resource for learning and practicing AI agent development.

Project Structure

Core Components

The project is divided into 4 main parts, each containing a Jupyter notebook and corresponding code implementation:

1. Agent Basics - Building basic agent functionality
2. Agent Evaluation - Implementing agent performance evaluation mechanisms
3. Human-in-the-Loop (HITL) - Integrating human intervention and review processes
4. Memory System - Adding long-term memory and learning capabilities

Directory Structure

• notebooks/ - Contains Jupyter notebooks for tutorials
• src/email_assistant/ - Core implementation code for the email assistant
• eval/ - Evaluation-related datasets and tools
• tests/ - Automated test suite

Technical Requirements

Environment Requirements

• Python Version: 3.11 or higher (for optimal compatibility with LangGraph)
• Main Dependencies: LangGraph, OpenAI API, LangSmith

API Key Configuration

The following API keys need to be configured:

• OpenAI API key (for language model calls)
• LangSmith API key (for tracing and evaluation)

Environment Setup

# Check Python version
python3 --version

# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Upgrade pip
python3 -m pip install --upgrade pip

# Install project (editable mode)
pip install -e .

Environment Variable Configuration

# Copy environment variable template
cp .env.example .env

Configure in the .env file:

LANGSMITH_API_KEY=your_langsmith_api_key
LANGSMITH_TRACING=true
LANGSMITH_PROJECT="interrupt-workshop"
OPENAI_API_KEY=your_openai_api_key

Detailed Functional Modules

1. LangGraph Basics Introduction

• File: notebooks/langgraph_101.ipynb
• Content: Introduces the basic concepts of LangGraph, including chat models, tool calls, the difference between agents and workflows, LangGraph nodes/edges/memory systems, and the use of LangGraph Studio.

2. Email Assistant Construction

• Notebook: notebooks/agent.ipynb
• Code: src/email_assistant/email_assistant.py
• Functionality:
  • Combines email classification steps with agent response processing
  • Implements a complete email processing workflow
  • Provides automatic email reply functionality

3. Agent Evaluation System

• Notebook: notebooks/evaluation.ipynb
• Dataset: eval/email_dataset.py
• Functionality:
  • Evaluates using Pytest and LangSmith evaluate API
  • Implements an evaluation mechanism using LLM as a judge
  • Evaluates the accuracy of tool calls and classification decisions

4. Human-in-the-Loop (HITL)

• Notebook: notebooks/hitl.ipynb
• Code: src/email_assistant/email_assistant_hitl.py
• Functionality:
  • Integrates human review mechanisms
  • Supports manual confirmation of specific tool calls (e.g., sending emails, scheduling meetings)
  • Uses Agent Inbox as a human-machine interaction interface

5. Memory System

• Code: src/email_assistant/email_assistant_hitl_memory.py
• Functionality:
  • Learns and adapts from user feedback
  • Uses LangGraph Store to persist memory
  • Supports personalized preference settings

6. Gmail Integration

• Code: src/email_assistant/email_assistant_hitl_memory_gmail.py
• Functionality:
  • Real Gmail API connection
  • Complete email management functionality
  • Supports deployment to LangGraph Platform

Testing System

Automated Test Suite

The project includes a complete automated testing system to verify the correctness of the email assistant implementation:

# Run default implementation tests
python tests/run_all_tests.py

# Run specific implementation tests
python tests/run_all_tests.py --implementation email_assistant_hitl

# Run tests for all implementations
python tests/run_all_tests.py --all

# Specify experiment name for LangSmith tracing
python tests/run_all_tests.py --experiment-name "Custom Test Run"

Testable Implementation Versions

• email_assistant - Basic email assistant
• email_assistant_hitl - Human-in-the-loop version
• email_assistant_hitl_memory - HITL version with memory function
• email_assistant_hitl_memory_gmail - Gmail integrated version

Evaluation Metrics

• Visual inspection of agent tracing
• Detailed evaluation metrics
• Comparative analysis of different agent implementations

Deployment Instructions

LangGraph Platform Deployment

The project supports deployment to the LangGraph Platform to implement agent services in a production environment.

Gmail Tool Setup

You need to set up Google API credentials according to the instructions in src/email_assistant/tools/gmail/README.md.

Learning Path Recommendations

1. Beginners: Start with the LangGraph 101 notebook to understand basic concepts
2. Practitioners: Complete the four main notebooks in order to gradually build complete functionality
3. Developers: Study the implementation code in the src directory to understand engineering practices
4. Deployers: Configure the Gmail API and deploy to a production environment

Technical Features

• Progressive Learning: From basic to advanced, step by step
• Practical Application: Real email processing scenarios
• Complete Engineering: Includes a complete solution for testing, evaluation, and deployment
• Scalability: Principles can be applied to other agent tasks