[docs] Essential cookbook added and overall revamp to the structure of cookbooks (#3668)

This commit is contained in:
Parth Sharma
2025-10-27 00:19:29 +05:30
committed by GitHub
parent 2c18355dd2
commit 7be32641c7
46 changed files with 3356 additions and 226 deletions
+125
View File
@@ -0,0 +1,125 @@
---
title: AI Tutor with Mem0
description: "Keep student progress and preferences persistent across tutoring sessions."
---
You can create a personalized AI Tutor using Mem0. This guide will walk you through the necessary steps and provide the complete code to get you started.
## Overview
The Personalized AI Tutor leverages Mem0 to retain information across interactions, enabling a tailored learning experience. By integrating with OpenAI's GPT-4 model, the tutor can provide detailed and context-aware responses to user queries.
## Setup
Before you begin, ensure you have the required dependencies installed. You can install the necessary packages using pip:
```bash
pip install openai mem0ai
```
## Full Code Example
Below is the complete code to create and interact with a Personalized AI Tutor using Mem0:
```python
import os
from openai import OpenAI
from mem0 import Memory
# Set the OpenAI API key
os.environ['OPENAI_API_KEY'] = 'sk-xxx'
# Initialize the OpenAI client
client = OpenAI()
class PersonalAITutor:
def __init__(self):
"""
Initialize the PersonalAITutor with memory configuration and OpenAI client.
"""
config = {
"vector_store": {
"provider": "qdrant",
"config": {
"host": "localhost",
"port": 6333,
}
},
}
self.memory = Memory.from_config(config)
self.client = client
self.app_id = "app-1"
def ask(self, question, user_id=None):
"""
Ask a question to the AI and store the relevant facts in memory
:param question: The question to ask the AI.
:param user_id: Optional user ID to associate with the memory.
"""
# Start a streaming response request to the AI
response = self.client.responses.create(
model="gpt-4.1-nano-2025-04-14",
instructions="You are a personal AI Tutor.",
input=question,
stream=True
)
# Store the question in memory
self.memory.add(question, user_id=user_id, metadata={"app_id": self.app_id})
# Print the response from the AI in real-time
for event in response:
if event.type == "response.output_text.delta":
print(event.delta, end="")
def get_memories(self, user_id=None):
"""
Retrieve all memories associated with the given user ID.
:param user_id: Optional user ID to filter memories.
:return: List of memories.
"""
return self.memory.get_all(user_id=user_id)
# Instantiate the PersonalAITutor
ai_tutor = PersonalAITutor()
# Define a user ID
user_id = "john_doe"
# Ask a question
ai_tutor.ask("I am learning introduction to CS. What is queue? Briefly explain.", user_id=user_id)
```
### Fetching Memories
You can fetch all the memories at any point in time using the following code:
```python
memories = ai_tutor.get_memories(user_id=user_id)
for m in memories['results']:
print(m['memory'])
```
## Key Points
- **Initialization**: The PersonalAITutor class is initialized with the necessary memory configuration and OpenAI client setup
- **Asking Questions**: The ask method sends a question to the AI and stores the relevant information in memory
- **Retrieving Memories**: The get_memories method fetches all stored memories associated with a user
## Conclusion
As the conversation progresses, Mem0's memory automatically updates based on the interactions, providing a continuously improving personalized learning experience. This setup ensures that the AI Tutor can offer contextually relevant and accurate responses, enhancing the overall educational process.
---
<CardGroup cols={2}>
<Card title="Build a Mem0 Companion" icon="users" href="/cookbooks/essentials/building-ai-companion">
Learn the foundations of memory-powered companions with production-ready patterns.
</Card>
<Card title="Travel Assistant with Mem0" icon="plane" href="/cookbooks/companions/travel-assistant">
Build a travel companion that remembers preferences and past conversations.
</Card>
</CardGroup>
@@ -0,0 +1,83 @@
---
title: Local Companion with Mem0 and Ollama
description: "Run Mem0 end-to-end on your machine using Ollama-powered LLMs and embedders."
---
Mem0 can be utilized entirely locally by leveraging Ollama for both the embedding model and the language model (LLM). This guide will walk you through the necessary steps and provide the complete code to get you started.
## Overview
By using Ollama, you can run Mem0 locally, which allows for greater control over your data and models. This setup uses Ollama for both the embedding model and the language model, providing a fully local solution.
## Setup
Before you begin, ensure you have Mem0 and Ollama installed and properly configured on your local machine.
## Full Code Example
Below is the complete code to set up and use Mem0 locally with Ollama:
```python
from mem0 import Memory
config = {
"vector_store": {
"provider": "qdrant",
"config": {
"collection_name": "test",
"host": "localhost",
"port": 6333,
"embedding_model_dims": 768, # Change this according to your local model's dimensions
},
},
"llm": {
"provider": "ollama",
"config": {
"model": "llama3.1:latest",
"temperature": 0,
"max_tokens": 2000,
"ollama_base_url": "http://localhost:11434", # Ensure this URL is correct
},
},
"embedder": {
"provider": "ollama",
"config": {
"model": "nomic-embed-text:latest",
# Alternatively, you can use "snowflake-arctic-embed:latest"
"ollama_base_url": "http://localhost:11434",
},
},
}
# Initialize Memory with the configuration
m = Memory.from_config(config)
# Add a memory
m.add("I'm visiting Paris", user_id="john")
# Retrieve memories
memories = m.get_all(user_id="john")
```
## Key Points
- **Configuration**: The setup involves configuring the vector store, language model, and embedding model to use local resources
- **Vector Store**: Qdrant is used as the vector store, running on localhost
- **Language Model**: Ollama is used as the LLM provider, with the `llama3.1:latest` model
- **Embedding Model**: Ollama is also used for embeddings, with the `nomic-embed-text:latest` model
## Conclusion
This local setup of Mem0 using Ollama provides a fully self-contained solution for memory management and AI interactions. It allows for greater control over your data and models while still leveraging the powerful capabilities of Mem0.
---
<CardGroup cols={2}>
<Card title="Configure Open Source" icon="gear" href="/open-source/configuration">
Explore advanced configuration options for vector stores, LLMs, and embedders.
</Card>
<Card title="Build a Mem0 Companion" icon="users" href="/cookbooks/essentials/building-ai-companion">
Learn core companion patterns that work with any LLM provider.
</Card>
</CardGroup>
@@ -0,0 +1,139 @@
---
title: Node.js Companion with Mem0
description: "Build a JavaScript fitness coach that remembers user goals run after run."
---
You can create a personalized AI Companion using Mem0. This guide will walk you through the necessary steps and provide the complete code to get you started.
## Overview
The Personalized AI Companion leverages Mem0 to retain information across interactions, enabling a tailored learning experience. It creates memories for each user interaction and integrates with OpenAI's GPT models to provide detailed and context-aware responses to user queries.
## Setup
Before you begin, ensure you have Node.js installed and create a new project. Install the required dependencies using npm:
```bash
npm install openai mem0ai
```
## Full Code Example
Below is the complete code to create and interact with an AI Companion using Mem0:
```javascript
import { OpenAI } from 'openai';
import { Memory } from 'mem0ai/oss';
import * as readline from 'readline';
const openaiClient = new OpenAI();
const memory = new Memory();
async function chatWithMemories(message, userId = "default_user") {
const relevantMemories = await memory.search(message, { userId: userId });
const memoriesStr = relevantMemories.results
.map(entry => `- ${entry.memory}`)
.join('\n');
const systemPrompt = `You are a helpful AI. Answer the question based on query and memories.
User Memories:
${memoriesStr}`;
const messages = [
{ role: "system", content: systemPrompt },
{ role: "user", content: message }
];
const response = await openaiClient.chat.completions.create({
model: "gpt-4.1-nano-2025-04-14",
messages: messages
});
const assistantResponse = response.choices[0].message.content || "";
messages.push({ role: "assistant", content: assistantResponse });
await memory.add(messages, { userId: userId });
return assistantResponse;
}
async function main() {
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
console.log("Chat with AI (type 'exit' to quit)");
const askQuestion = () => {
return new Promise((resolve) => {
rl.question("You: ", (input) => {
resolve(input.trim());
});
});
};
try {
while (true) {
const userInput = await askQuestion();
if (userInput.toLowerCase() === 'exit') {
console.log("Goodbye!");
rl.close();
break;
}
const response = await chatWithMemories(userInput, "sample_user");
console.log(`AI: ${response}`);
}
} catch (error) {
console.error("An error occurred:", error);
rl.close();
}
}
main().catch(console.error);
```
### Key Components
1. **Initialization**
- The code initializes both OpenAI and Mem0 Memory clients
- Uses Node.js's built-in readline module for command-line interaction
2. **Memory Management (chatWithMemories function)**
- Retrieves relevant memories using Mem0's search functionality
- Constructs a system prompt that includes past memories
- Makes API calls to OpenAI for generating responses
- Stores new interactions in memory
3. **Interactive Chat Interface (main function)**
- Creates a command-line interface for user interaction
- Handles user input and displays AI responses
- Includes graceful exit functionality
### Environment Setup
Make sure to set up your environment variables:
```bash
export OPENAI_API_KEY=your_api_key
```
### Conclusion
This implementation demonstrates how to create an AI Companion that maintains context across conversations using Mem0's memory capabilities. The system automatically stores and retrieves relevant information, creating a more personalized and context-aware interaction experience.
As users interact with the system, Mem0's memory system continuously learns and adapts, making future responses more relevant and personalized. This setup is ideal for creating long-term learning AI assistants that can maintain context and provide increasingly personalized responses over time.
---
<CardGroup cols={2}>
<Card title="Build AI with Personality" icon="sparkles" href="/cookbooks/essentials/building-ai-with-personality">
Separate user and agent memories to keep your companion's personality consistent.
</Card>
<Card title="Quickstart Demo with Mem0" icon="rocket" href="/cookbooks/companions/quickstart-demo">
Run the full showcase app to see memory-powered companions in action.
</Card>
</CardGroup>
@@ -0,0 +1,80 @@
---
title: Quickstart Demo with Mem0
description: "Spin up the showcase companion app to see Mem0 memories in action."
---
You can create a personalized AI Companion using Mem0. This guide will walk you through the necessary steps and provide the complete setup instructions to get you started.
<video
autoPlay
muted
loop
playsInline
className="w-full aspect-video rounded-lg"
src="https://github.com/user-attachments/assets/cebc4f8e-bdb9-4837-868d-13c5ab7bb433"
></video>
You can try the [Mem0 Demo](https://mem0-4vmi.vercel.app) live here.
## Overview
The Personalized AI Companion leverages Mem0 to retain information across interactions, enabling a tailored learning experience. It creates memories for each user interaction and integrates with OpenAI's GPT models to provide detailed and context-aware responses to user queries.
## Setup
Before you begin, follow these steps to set up the demo application:
1. Clone the Mem0 repository:
```bash
git clone https://github.com/mem0ai/mem0.git
```
2. Navigate to the demo application folder:
```bash
cd mem0/examples/mem0-demo
```
3. Install dependencies:
```bash
pnpm install
```
4. Set up environment variables by creating a `.env` file in the project root with the following content:
```bash
OPENAI_API_KEY=your_openai_api_key
MEM0_API_KEY=your_mem0_api_key
```
You can obtain your `MEM0_API_KEY` by signing up at [Mem0 API Dashboard](https://app.mem0.ai/dashboard/api-keys).
5. Start the development server:
```bash
pnpm run dev
```
## Enhancing the Next.js Application
Once the demo is running, you can customize and enhance the Next.js application by modifying the components in the `mem0-demo` folder. Consider:
- Adding new memory features to improve contextual retention
- Customizing the UI to better suit your application needs
- Integrating additional APIs or third-party services to extend functionality
## Full Code
You can find the complete source code for this demo on GitHub:
[Mem0 Demo GitHub](https://github.com/mem0ai/mem0/tree/main/examples/mem0-demo)
## Conclusion
This setup demonstrates how to build an AI Companion that maintains memory across interactions using Mem0. The system continuously adapts to user interactions, making future responses more relevant and personalized. Experiment with the application and enhance it further to suit your use case!
---
<CardGroup cols={2}>
<Card title="Build a Mem0 Companion" icon="users" href="/cookbooks/essentials/building-ai-companion">
Deep dive into production patterns for fitness coaches, tutors, and assistants.
</Card>
<Card title="Node.js Companion with Mem0" icon="code" href="/cookbooks/companions/nodejs-companion">
Implement a command-line companion using the Node.js SDK.
</Card>
</CardGroup>
@@ -0,0 +1,214 @@
---
title: Travel Assistant with Mem0
description: "Plan itineraries that remember traveler preferences across trips."
---
Create a personalized AI Travel Assistant using Mem0. This guide provides step-by-step instructions and the complete code to get you started.
## Overview
The Personalized AI Travel Assistant uses Mem0 to store and retrieve information across interactions, enabling a tailored travel planning experience. It integrates with OpenAI's GPT-4 model to provide detailed and context-aware responses to user queries.
## Setup
Install the required dependencies using pip:
```bash
pip install openai mem0ai
```
## Full Code Example
Here's the complete code to create and interact with a Personalized AI Travel Assistant using Mem0:
<CodeGroup>
```python After v1.1
import os
from openai import OpenAI
from mem0 import Memory
# Set the OpenAI API key
os.environ['OPENAI_API_KEY'] = "sk-xxx"
config = {
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4.1-nano-2025-04-14",
"temperature": 0.1,
"max_tokens": 2000,
}
},
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-3-large"
}
},
"vector_store": {
"provider": "qdrant",
"config": {
"collection_name": "test",
"embedding_model_dims": 3072,
}
},
"version": "v1.1",
}
class PersonalTravelAssistant:
def __init__(self):
self.client = OpenAI()
self.memory = Memory.from_config(config)
self.messages = [{"role": "system", "content": "You are a personal AI Assistant."}]
def ask_question(self, question, user_id):
# Fetch previous related memories
previous_memories = self.search_memories(question, user_id=user_id)
# Build the prompt
system_message = "You are a personal AI Assistant."
if previous_memories:
prompt = f"{system_message}\n\nUser input: {question}\nPrevious memories: {', '.join(previous_memories)}"
else:
prompt = f"{system_message}\n\nUser input: {question}"
# Generate response using Responses API
response = self.client.responses.create(
model="gpt-4.1-nano-2025-04-14",
input=prompt
)
# Extract answer from the response
answer = response.output[0].content[0].text
# Store the question in memory
self.memory.add(question, user_id=user_id)
return answer
def get_memories(self, user_id):
memories = self.memory.get_all(user_id=user_id)
return [m['memory'] for m in memories['results']]
def search_memories(self, query, user_id):
memories = self.memory.search(query, user_id=user_id)
return [m['memory'] for m in memories['results']]
# Usage example
user_id = "traveler_123"
ai_assistant = PersonalTravelAssistant()
def main():
while True:
question = input("Question: ")
if question.lower() in ['q', 'exit']:
print("Exiting...")
break
answer = ai_assistant.ask_question(question, user_id=user_id)
print(f"Answer: {answer}")
memories = ai_assistant.get_memories(user_id=user_id)
print("Memories:")
for memory in memories:
print(f"- {memory}")
print("-----")
if __name__ == "__main__":
main()
```
```python Before v1.1
import os
from openai import OpenAI
from mem0 import Memory
# Set the OpenAI API key
os.environ['OPENAI_API_KEY'] = 'sk-xxx'
class PersonalTravelAssistant:
def __init__(self):
self.client = OpenAI()
self.memory = Memory()
self.messages = [{"role": "system", "content": "You are a personal AI Assistant."}]
def ask_question(self, question, user_id):
# Fetch previous related memories
previous_memories = self.search_memories(question, user_id=user_id)
prompt = question
if previous_memories:
prompt = f"User input: {question}\n Previous memories: {previous_memories}"
self.messages.append({"role": "user", "content": prompt})
# Generate response using gpt-4.1-nano
response = self.client.chat.completions.create(
model="gpt-4.1-nano-2025-04-14"2025-04-14",
messages=self.messages
)
answer = response.choices[0].message.content
self.messages.append({"role": "assistant", "content": answer})
# Store the question in memory
self.memory.add(question, user_id=user_id)
return answer
def get_memories(self, user_id):
memories = self.memory.get_all(user_id=user_id)
return [m['memory'] for m in memories.get('results', [])]
def search_memories(self, query, user_id):
memories = self.memory.search(query, user_id=user_id)
return [m['memory'] for m in memories.get('results', [])]
# Usage example
user_id = "traveler_123"
ai_assistant = PersonalTravelAssistant()
def main():
while True:
question = input("Question: ")
if question.lower() in ['q', 'exit']:
print("Exiting...")
break
answer = ai_assistant.ask_question(question, user_id=user_id)
print(f"Answer: {answer}")
memories = ai_assistant.get_memories(user_id=user_id)
print("Memories:")
for memory in memories:
print(f"- {memory}")
print("-----")
if __name__ == "__main__":
main()
```
</CodeGroup>
## Key Components
- **Initialization**: The `PersonalTravelAssistant` class is initialized with the OpenAI client and Mem0 memory setup.
- **Asking Questions**: The `ask_question` method sends a question to the AI, incorporates previous memories, and stores new information.
- **Memory Management**: The `get_memories` and search_memories methods handle retrieval and searching of stored memories.
## Usage
1. Set your OpenAI API key in the environment variable.
2. Instantiate the `PersonalTravelAssistant`.
3. Use the `main()` function to interact with the assistant in a loop.
## Conclusion
This Personalized AI Travel Assistant leverages Mem0's memory capabilities to provide context-aware responses. As you interact with it, the assistant learns and improves, offering increasingly personalized travel advice and information.
---
<CardGroup cols={2}>
<Card title="Tag and Organize Memories" icon="tag" href="/cookbooks/essentials/tagging-and-organizing-memories">
Use categories to organize travel preferences, destinations, and user context.
</Card>
<Card title="AI Tutor with Mem0" icon="graduation-cap" href="/cookbooks/companions/ai-tutor">
Build an educational companion that remembers learning progress and preferences.
</Card>
</CardGroup>
@@ -0,0 +1,548 @@
---
title: Voice Companion with Mem0 and OpenAI
description: "Pair the OpenAI Agents SDK with Mem0 to build a voice assistant that remembers."
---
This guide demonstrates how to combine OpenAI's Agents SDK for voice applications with Mem0's memory capabilities to create a voice assistant that remembers user preferences and past interactions.
## Prerequisites
Before you begin, make sure you have:
1. Installed OpenAI Agents SDK with voice dependencies:
```bash
pip install 'openai-agents[voice]'
```
2. Installed Mem0 SDK:
```bash
pip install mem0ai
```
3. Installed other required dependencies:
```bash
pip install numpy sounddevice pydantic
```
4. Set up your API keys:
- OpenAI API key for the Agents SDK
- Mem0 API key from the Mem0 Platform
## Code Breakdown
Let's break down the key components of this implementation:
### 1. Setting Up Dependencies and Environment
```python
# OpenAI Agents SDK imports
from agents import (
Agent,
function_tool
)
from agents.voice import (
AudioInput,
SingleAgentVoiceWorkflow,
VoicePipeline
)
from agents.extensions.handoff_prompt import prompt_with_handoff_instructions
# Mem0 imports
from mem0 import AsyncMemoryClient
# Set up API keys (replace with your actual keys)
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
os.environ["MEM0_API_KEY"] = "your-mem0-api-key"
# Define a global user ID for simplicity
USER_ID = "voice_user"
# Initialize Mem0 client
mem0_client = AsyncMemoryClient()
```
This section handles:
- Importing required modules from OpenAI Agents SDK and Mem0
- Setting up environment variables for API keys
- Defining a simple user identification system (using a global variable)
- Initializing the Mem0 client that will handle memory operations
### 2. Memory Tools with Function Decorators
The `@function_tool` decorator transforms Python functions into callable tools for the OpenAI agent. Here are the key memory tools:
#### Storing User Memories
```python
import logging
# Set up logging at the top of your file
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
force=True
)
logger = logging.getLogger("memory_voice_agent")
# Then use logger in your function tools
@function_tool
async def save_memories(
memory: str
) -> str:
"""Store a user memory in memory."""
# This will be visible in your console
logger.debug(f"Saving memory: {memory} for user {USER_ID}")
# Store the preference in Mem0
memory_content = f"User memory - {memory}"
await mem0_client.add(
memory_content,
user_id=USER_ID,
)
return f"I've saved your memory: {memory}"
```
This function:
- Takes a memory string
- Creates a formatted memory string
- Stores it in Mem0 using the `add()` method
- Includes metadata to categorize the memory for easier retrieval
- Returns a confirmation message that the agent will speak
#### Finding Relevant Memories
```python
@function_tool
async def search_memories(
query: str
) -> str:
"""
Find memories relevant to the current conversation.
Args:
query: The search query to find relevant memories
"""
print(f"Finding memories related to: {query}")
results = await mem0_client.search(
query,
user_id=USER_ID,
limit=5,
threshold=0.7, # Higher threshold for more relevant results
)
# Format and return the results
if not results.get('results', []):
return "I don't have any relevant memories about this topic."
memories = [f"• {result['memory']}" for result in results.get('results', [])]
return "Here's what I remember that might be relevant:\n" + "\n".join(memories)
```
This tool:
- Takes a search query string
- Passes it to Mem0's semantic search to find related memories
- Sets a threshold for relevance to ensure quality results
- Returns a formatted list of relevant memories or a default message
### 3. Creating the Voice Agent
```python
def create_memory_voice_agent():
# Create the agent with memory-enabled tools
agent = Agent(
name="Memory Assistant",
instructions=prompt_with_handoff_instructions(
"""You're speaking to a human, so be polite and concise.
Always respond in clear, natural English.
You have the ability to remember information about the user.
Use the save_memories tool when the user shares an important information worth remembering.
Use the search_memories tool when you need context from past conversations or user asks you to recall something.
""",
),
model="gpt-4.1-nano-2025-04-14",
tools=[save_memories, search_memories],
)
return agent
```
This function:
- Creates an OpenAI Agent with specific instructions
- Configures it to use gpt-4.1-nano (you can use other models)
- Registers the memory-related tools with the agent
- Uses `prompt_with_handoff_instructions` to include standard voice agent behaviors
### 4. Microphone Recording Functionality
```python
async def record_from_microphone(duration=5, samplerate=24000):
"""Record audio from the microphone for a specified duration."""
print(f"Recording for {duration} seconds...")
# Create a buffer to store the recorded audio
frames = []
# Callback function to store audio data
def callback(indata, frames_count, time_info, status):
frames.append(indata.copy())
# Start recording
with sd.InputStream(samplerate=samplerate, channels=1, callback=callback, dtype=np.int16):
await asyncio.sleep(duration)
# Combine all frames into a single numpy array
audio_data = np.concatenate(frames)
return audio_data
```
This function:
- Creates a simple asynchronous microphone recording function
- Uses the sounddevice library to capture audio input
- Stores frames in a buffer during recording
- Combines frames into a single numpy array when complete
- Returns the audio data for processing
### 5. Main Loop and Voice Processing
```python
async def main():
# Create the agent
agent = create_memory_voice_agent()
# Set up the voice pipeline
pipeline = VoicePipeline(
workflow=SingleAgentVoiceWorkflow(agent)
)
# Configure TTS settings
pipeline.config.tts_settings.voice = "alloy"
pipeline.config.tts_settings.speed = 1.0
try:
while True:
# Get user input
print("\nPress Enter to start recording (or 'q' to quit)...")
user_input = input()
if user_input.lower() == 'q':
break
# Record and process audio
audio_data = await record_from_microphone(duration=5)
audio_input = AudioInput(buffer=audio_data)
result = await pipeline.run(audio_input)
# Play response and handle events
player = sd.OutputStream(samplerate=24000, channels=1, dtype=np.int16)
player.start()
agent_response = ""
print("\nAgent response:")
async for event in result.stream():
if event.type == "voice_stream_event_audio":
player.write(event.data)
elif event.type == "voice_stream_event_content":
content = event.data
agent_response += content
print(content, end="", flush=True)
# Save the agent's response to memory
if agent_response:
try:
await mem0_client.add(
f"Agent response: {agent_response}",
user_id=USER_ID,
metadata={"type": "agent_response"}
)
except Exception as e:
print(f"Failed to store memory: {e}")
except KeyboardInterrupt:
print("\nExiting...")
```
This main function orchestrates the entire process:
1. Creates the memory-enabled voice agent
2. Sets up the voice pipeline with TTS settings
3. Implements an interactive loop for recording and processing voice input
4. Handles streaming of response events (both audio and text)
5. Automatically saves the agent's responses to memory
6. Includes proper error handling and exit mechanisms
## Create a Memory-Enabled Voice Agent
Now that we've explained each component, here's the complete implementation that combines OpenAI Agents SDK for voice with Mem0's memory capabilities:
```python
import asyncio
import os
import logging
from typing import Optional, List, Dict, Any
import numpy as np
import sounddevice as sd
from pydantic import BaseModel
# OpenAI Agents SDK imports
from agents import (
Agent,
function_tool
)
from agents.voice import (
AudioInput,
SingleAgentVoiceWorkflow,
VoicePipeline
)
from agents.extensions.handoff_prompt import prompt_with_handoff_instructions
# Mem0 imports
from mem0 import AsyncMemoryClient
# Set up API keys (replace with your actual keys)
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
os.environ["MEM0_API_KEY"] = "your-mem0-api-key"
# Define a global user ID for simplicity
USER_ID = "voice_user"
# Initialize Mem0 client
mem0_client = AsyncMemoryClient()
# Create tools that utilize Mem0's memory
@function_tool
async def save_memories(
memory: str
) -> str:
"""
Store a user memory in memory.
Args:
memory: The memory to save
"""
print(f"Saving memory: {memory} for user {USER_ID}")
# Store the preference in Mem0
memory_content = f"User memory - {memory}"
await mem0_client.add(
memory_content,
user_id=USER_ID,
)
return f"I've saved your memory: {memory}"
@function_tool
async def search_memories(
query: str
) -> str:
"""
Find memories relevant to the current conversation.
Args:
query: The search query to find relevant memories
"""
print(f"Finding memories related to: {query}")
results = await mem0_client.search(
query,
user_id=USER_ID,
limit=5,
threshold=0.7, # Higher threshold for more relevant results
)
# Format and return the results
if not results.get('results', []):
return "I don't have any relevant memories about this topic."
memories = [f"• {result['memory']}" for result in results.get('results', [])]
return "Here's what I remember that might be relevant:\n" + "\n".join(memories)
# Create the agent with memory-enabled tools
def create_memory_voice_agent():
# Create the agent with memory-enabled tools
agent = Agent(
name="Memory Assistant",
instructions=prompt_with_handoff_instructions(
"""You're speaking to a human, so be polite and concise.
Always respond in clear, natural English.
You have the ability to remember information about the user.
Use the save_memories tool when the user shares an important information worth remembering.
Use the search_memories tool when you need context from past conversations or user asks you to recall something.
""",
),
model="gpt-4.1-nano-2025-04-14",
tools=[save_memories, search_memories],
)
return agent
async def record_from_microphone(duration=5, samplerate=24000):
"""Record audio from the microphone for a specified duration."""
print(f"Recording for {duration} seconds...")
# Create a buffer to store the recorded audio
frames = []
# Callback function to store audio data
def callback(indata, frames_count, time_info, status):
frames.append(indata.copy())
# Start recording
with sd.InputStream(samplerate=samplerate, channels=1, callback=callback, dtype=np.int16):
await asyncio.sleep(duration)
# Combine all frames into a single numpy array
audio_data = np.concatenate(frames)
return audio_data
async def main():
print("Starting Memory Voice Agent")
# Create the agent and context
agent = create_memory_voice_agent()
# Set up the voice pipeline
pipeline = VoicePipeline(
workflow=SingleAgentVoiceWorkflow(agent)
)
# Configure TTS settings
pipeline.config.tts_settings.voice = "alloy"
pipeline.config.tts_settings.speed = 1.0
try:
while True:
# Get user input
print("\nPress Enter to start recording (or 'q' to quit)...")
user_input = input()
if user_input.lower() == 'q':
break
# Record and process audio
audio_data = await record_from_microphone(duration=5)
audio_input = AudioInput(buffer=audio_data)
print("Processing your request...")
# Process the audio input
result = await pipeline.run(audio_input)
# Create an audio player
player = sd.OutputStream(samplerate=24000, channels=1, dtype=np.int16)
player.start()
# Store the agent's response for adding to memory
agent_response = ""
print("\nAgent response:")
# Play the audio stream as it comes in
async for event in result.stream():
if event.type == "voice_stream_event_audio":
player.write(event.data)
elif event.type == "voice_stream_event_content":
# Accumulate and print the text response
content = event.data
agent_response += content
print(content, end="", flush=True)
print("\n")
# Example of saving the conversation to Mem0 after completion
if agent_response:
try:
await mem0_client.add(
f"Agent response: {agent_response}",
user_id=USER_ID,
metadata={"type": "agent_response"}
)
except Exception as e:
print(f"Failed to store memory: {e}")
except KeyboardInterrupt:
print("\nExiting...")
if __name__ == "__main__":
asyncio.run(main())
```
## Key Features of This Implementation
This implementation offers several key features:
1. **Simplified User Management**: Uses a global `USER_ID` variable for simplicity, but can be extended to manage multiple users.
2. **Real Microphone Input**: Includes a `record_from_microphone()` function that captures actual voice input from your microphone.
3. **Interactive Voice Loop**: Implements a continuous interaction loop, allowing for multiple back-and-forth exchanges.
4. **Memory Management Tools**:
- `save_memories`: Stores user memories in Mem0
- `search_memories`: Searches for relevant past information
5. **Voice Configuration**: Demonstrates how to configure TTS settings for the voice response.
## Running the Example
To run this example:
1. Replace the placeholder API keys with your actual keys
2. Make sure your microphone is properly connected
3. Run the script with Python 3.8 or newer
4. Press Enter to start recording, then speak your request
5. Press 'q' to quit the application
The agent will listen to your request, process it through the OpenAI model, utilize Mem0 for memory operations as needed, and respond both through text output and voice speech.
## Best Practices for Voice Agents with Memory
1. **Optimizing Memory for Voice**: Keep memories concise and relevant for voice responses.
2. **Forgetting Mechanism**: Implement a way to delete or expire memories that are no longer relevant.
3. **Context Preservation**: Store enough context with each memory to make retrieval effective.
4. **Error Handling**: Implement robust error handling for memory operations, as voice interactions should continue smoothly even if memory operations fail.
## Conclusion
By combining OpenAI's Agents SDK with Mem0's memory capabilities, you can create voice agents that maintain persistent memory of user preferences and past interactions. This significantly enhances the user experience by making conversations more natural and personalized.
As you build your voice application, experiment with different memory strategies and filtering approaches to find the optimal balance between comprehensive memory and efficient retrieval for your specific use case.
## Debugging Function Tools
When working with the OpenAI Agents SDK, you might notice that regular `print()` statements inside `@function_tool` decorated functions don't appear in your console output. This is because the Agents SDK captures and redirects standard output when executing these functions.
To effectively debug your function tools, use Python's `logging` module instead:
```python
import logging
# Set up logging at the top of your file
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
force=True
)
logger = logging.getLogger("memory_voice_agent")
# Then use logger in your function tools
@function_tool
async def save_memories(
memory: str
) -> str:
"""Store a user memory in memory."""
# This will be visible in your console
logger.debug(f"Saving memory: {memory} for user {USER_ID}")
# Rest of your function...
```
---
<CardGroup cols={2}>
<Card title="Multimodal Support" icon="image" href="/platform/features/multimodal-support">
Learn how to add vision and audio memory alongside voice interactions.
</Card>
<Card title="Build a Mem0 Companion" icon="users" href="/cookbooks/essentials/building-ai-companion">
Master the core patterns for building memory-powered companions.
</Card>
</CardGroup>
@@ -0,0 +1,68 @@
---
title: YouTube Research with Mem0
description: "Layer personalized context over any video using the Mem0 YouTube assistant."
---
Enhance your YouTube experience with Mem0's YouTube Assistant, a Chrome extension that brings AI-powered chat directly to your YouTube videos. Get instant, personalized answers about video content while leveraging your own knowledge and memories, all without leaving the page.
## Features
- **Contextual AI Chat**: Ask questions about videos you're watching
- **Seamless Integration**: Chat interface sits alongside YouTube's native UI
- **Memory Integration**: Personalized responses based on your knowledge through Mem0
- **Real-Time Memory**: Memories are updated in real-time based on your interactions
## Demo Video
<video
autoPlay
muted
loop
playsInline
width="700"
height="400"
src="https://github.com/user-attachments/assets/c0334ccd-311b-4dd7-8034-ef88204fc751"
></video>
## Installation
This extension is not available on the Chrome Web Store yet. You can install it manually using below method:
### Manual Installation (Developer Mode)
1. **Download the Extension**: Clone or download the extension files from the [Mem0 GitHub repository](https://github.com/mem0ai/mem0/tree/main/examples)
2. **Build**: Run `npm install` followed by `npm run build` to install the dependencies and build the extension
3. **Access Chrome Extensions**: Open Google Chrome and navigate to `chrome://extensions`
4. **Enable Developer Mode**: Toggle the "Developer mode" switch in the top right corner
5. **Load Unpacked Extension**: Click "Load unpacked" and select the directory containing the extension files
6. **Confirm Installation**: The Mem0 YouTube Assistant Extension should now appear in your Chrome toolbar
## Setup
1. **Configure API Settings**: Click the extension icon and enter your OpenAI API key (required to use the extension)
2. **Customize Settings**: Configure additional settings such as model, temperature, and memory settings
3. **Navigate to YouTube**: Start using the assistant on any YouTube video
4. **Memories**: Enter your Mem0 API key to enable personalized responses, and feed initial memories from settings
## Example Prompts
- "Can you summarize the main points of this video?"
- "Explain the concept they just mentioned"
- "How does this relate to what I already know?"
- "What are some practical applications of this topic related to my work?"
## Privacy and Data Security
Your API keys are stored locally in your browser. Your messages are sent to the Mem0 API for extracting and retrieving memories. Mem0 is committed to ensuring your data's privacy and security.
---
<CardGroup cols={2}>
<Card title="Tag and Organize Memories" icon="tag" href="/cookbooks/essentials/tagging-and-organizing-memories">
Categorize video insights to build a searchable research knowledge base.
</Card>
<Card title="Deep Research with Mem0" icon="magnifying-glass" href="/cookbooks/operations/deep-research">
Combine memory with search tools to conduct comprehensive research projects.
</Card>
</CardGroup>