- Python version >= 3.9. Because langchainhub package requires it
- Docker and Docker Compose (for containerized deployment)
- This is a chatbot implementation with Langchain framework.
- Base LLM: Vertex AI or OpenAI API
- Memory: MongoDB
- UI:
- Next.js frontend
- FastAPI backend
- Prompt versioning and tracing: LangSmith
- User can custom bot's personality by setting bot information like gender, age, ...
- Demo UI:
The application follows a modern microservices architecture with containerized components:
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ Docker Compose Environment │
│ │
│ ┌───────────────┐ ┌────────────────┐ ┌───────────────────┐ │
│ │ │ │ │ │ │ │
│ │ Next.js │◄─────►│ FastAPI │◄──────►│ MongoDB │ │
│ │ Frontend │ │ Backend │ │ Database │ │
│ │ (Port 3000) │ │ (Port 8080) │ │ (Port 27017) │ │
│ │ │ │ │ │ │ │
│ └───────────────┘ └────────┬───────┘ └───────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ ┌──────────────────┐ │
│ │ │ │ │ │
│ │ LangChain │─────►│ LangSmith │ │
│ │ Framework │ │ (Tracing) │ │
│ │ │ │ │ │
│ └────────┬────────┘ └──────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ ┌──────────────────┐ │
│ │ │ │ │ │
│ │ LLM Provider │ │ Presidio │ │
│ │ (OpenAI/Vertex) │ │ Anonymizer │ │
│ │ │ │ (PII Protection) │ │
│ └─────────────────┘ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
-
User Interaction:
- User sends a message through the Next.js frontend
- Frontend forwards the request to the FastAPI backend
-
Message Processing:
- Backend optionally anonymizes PII data using Presidio Anonymizer
- LangChain framework builds the conversation chain
- Request is sent to the selected LLM provider (OpenAI or Vertex AI)
- Response is traced using LangSmith for monitoring
-
Conversation Storage:
- Conversations are stored in MongoDB for history
- Each user session has a unique conversation ID
-
Response Generation:
- LLM response is de-anonymized if PII protection is enabled
- Backend sends the formatted response back to the frontend
- Frontend renders the response to the user
┌────────────────────────────────────────────────────────────────────────┐
│ │
│ FastAPI Backend │
│ │
│ ┌────────────┐ ┌─────────────┐ ┌────────────────────────┐ │
│ │ │ │ │ │ │ │
│ │ API │◄───►│ Bot │◄────►│ Memory System │ │
│ │ Routes │ │ Manager │ │ (MongoDB/Redis) │ │
│ │ │ │ │ │ │ │
│ └────────────┘ └──────┬──────┘ └────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────┐ │
│ │ │ ┌────────────────────┐ │
│ │ Chain │◄───────►│ │ │
│ │ Manager │ │ Anonymizer │ │
│ │ │ │ │ │
│ └───────┬───────┘ └────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────┐ ┌─────────────────────┐ │
│ │ │ │ │ │
│ │ LLM Models │───────►│ External Tools │ │
│ │ Integration │ │ (Search/etc.) │ │
│ │ │ │ │ │
│ └────────────────┘ └─────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
Key Backend Components:
- API Routes: FastAPI endpoints for chat, health checks, and conversation management
- Bot Manager: Core orchestration layer that handles message processing
- Memory System: Stores conversation history with MongoDB integration
- Chain Manager: Manages LangChain prompt templates and execution
- Anonymizer: Optional PII protection using Microsoft Presidio
- LLM Integration: Connects to OpenAI or Vertex AI models
- External Tools: Integrates with search and other auxiliary services
┌────────────────────────────────────────────────────────────────────────┐
│ │
│ Next.js Frontend │
│ │
│ ┌────────────────┐ ┌─────────────────┐ ┌───────────────────┐ │
│ │ │ │ │ │ │ │
│ │ Page Layout │────►│ Chat Window │───►│ Message Bubbles │ │
│ │ │ │ Component │ │ │ │
│ └────────────────┘ └─────────────────┘ └───────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ ┌───────────────────┐ │
│ │ │ │ │ │
│ │ API Services │───►│ State Management │ │
│ │ │ │ │ │
│ └─────────────────┘ └───────────────────┘ │
│ │
│ ┌─────────────────┐ ┌───────────────────┐ │
│ │ │ │ │ │
│ │ UI Components │───►│ Utility Helpers │ │
│ │ │ │ │ │
│ └─────────────────┘ └───────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────────┘
Key Frontend Components:
- Page Layout: Main application layout and container components
- Chat Window: Core component handling conversation display and input
- Message Bubbles: Display of user and AI messages with formatting
- API Services: REST client for communicating with the backend
- State Management: Manages conversation state and UI interactions
- UI Components: Reusable elements like buttons, inputs, and modals
- Utility Helpers: Support functions for data formatting and processing
Below is a sequence diagram showing how data flows through the system when a user sends a message:
User Frontend Backend LangChain LLM MongoDB
| | | | | |
|--message--->| | | | |
| |--POST /chat->| | | |
| | |--load history-| | |
| | | |--query------|----------->|
| | | |<--history---|------------|
| | |--anonymize--->| | |
| | | |--prompt---->| |
| | | |<--response--| |
| | |<-de-anonymize-| | |
| | |--store--------|-------------|----------->|
| |<---response--| | | |
|<--display---| | | | |
| | | | | |
Key Steps in the Sequence:
-
User Interaction:
- User types a message in the chat interface
- Frontend captures the input and sends to backend API
-
Backend Processing:
- Backend loads conversation history from MongoDB
- If enabled, PII is anonymized using Presidio
- The conversation chain is constructed with LangChain
-
LLM Interaction:
- The prompt with history and user message is sent to the LLM
- LLM processes the request and returns a response
- If PII protection is enabled, the response is de-anonymized
-
Response Handling:
- The conversation is stored in MongoDB
- Response is returned to the frontend
- Frontend displays the message to the user
-
Tracing and Monitoring:
- Throughout this process, LangSmith traces the execution
- Performance metrics and debugging information are collected
- Data anonymization with Microsoft Presidio
- To protect personally identifiable information (PII), we add
PresidioAnonymizerto my bot to replace PIIs before pass to LLM api. View code in Anonymizer - Steps when using it:
-
User message after anonymize:
-
Anonymized prompt before input to LLM:
-
De-anonymized response to user after LLM call:
-
The easiest way to run the entire application is using our setup script:
-
Make the script executable:
chmod +x setup.sh
-
Run the setup script:
./setup.sh
-
Follow the prompts:
- The script will create necessary .env files if they don't exist
- Choose between Docker Compose deployment or local development
- The script will guide you through the rest of the setup process
For Windows users, use setup.bat instead.
If you prefer to run the commands manually:
-
Set up environment variables:
# For backend cp backend/.env.example backend/.env # Edit the .env file to add your OpenAI API key # For frontend cp frontend/.env.example frontend/.env
-
Start the application:
docker-compose up -d
-
Access the application:
- Frontend: http://localhost:3000
- Backend API: http://localhost:8080
- API documentation: http://localhost:8080/docs
-
Stop the application:
docker-compose down
- Langsmith docs: LangSmith
- Configure environment to connect to LangSmith. Add these to your
backend/.envfile:LANGCHAIN_TRACING_V2=true LANGCHAIN_ENDPOINT="https://api.smith.langchain.com" LANGCHAIN_API_KEY="<your-api-key>" LANGCHAIN_PROJECT="chatbot-with-langchain"
- Download the models for the languages to use in anonymizer. PII support.
python -m spacy download en_core_web_md
- RUN backend
- Clone repo:
git clone https://github.com/btrcm00/chatbot-with-langchain.git - Add google-cloud-platform credential file to
secure/vertexai.jsonor set up OpenAI API key cd backend- Install required packages:
pip install -r requirements.txt - Create MongoDB database and config environment variables to connect Mongo
- Run:
python -m uvicorn backend.main:app --reload --host 0.0.0.0 --port 8080
- Clone repo:
- RUN frontend
cd frontend- Install packages:
npm install - Start frontend:
npm run dev
For development purposes, you can use the Makefile commands:
make setup # Set up environment files
make start # Start all services
make stop # Stop all services
make logs # View logs from all containers