Add Neptune-DB graph store with vector store (#3443)
Signed-off-by: Andrew Carbonetto <andrew.carbonetto@improving.com> Co-authored-by: Siddhartha Sahu <dev@sdht.in>
This commit is contained in:
committed by
GitHub
parent
d4e98dba38
commit
a015e2ff4a
@@ -48,7 +48,7 @@ allowfullscreen
|
||||
## Initialize Graph Memory
|
||||
|
||||
To initialize Graph Memory you'll need to set up your configuration with graph
|
||||
store providers. Currently, we support [Neo4j](#initialize-neo4j), [Memgraph](#initialize-memgraph), [Neptune Analytics](#initialize-neptune-analytics), and [Kuzu](#initialize-kuzu) as graph store providers.
|
||||
store providers. Currently, we support [Neo4j](#initialize-neo4j), [Memgraph](#initialize-memgraph), [Neptune Analytics](#initialize-neptune-analytics), [Neptune DB Cluster](#initialize-neptune-db),and [Kuzu](#initialize-kuzu) as graph store providers.
|
||||
|
||||
|
||||
### Initialize Neo4j
|
||||
@@ -231,35 +231,27 @@ m = Memory.from_config(config_dict=config)
|
||||
|
||||
### Initialize Neptune Analytics
|
||||
|
||||
Mem0 now supports Amazon Neptune Analytics as a graph store provider. This integration allows you to use Neptune Analytics for storing and querying graph-based memories.
|
||||
Note: You can use Neptune Analytics as part of an Amazon tech stack [Setup AWS Bedrock, AOSS, and Neptune](https://docs.mem0.ai/examples/aws_example#aws-bedrock-and-aoss)
|
||||
|
||||
You can use Neptune Analytics as part of an Amazon tech stack [Setup AWS Bedrock, AOSS, and Neptune](https://docs.mem0.ai/examples/aws_example#aws-bedrock-and-aoss)
|
||||
|
||||
#### Instance Setup
|
||||
|
||||
Create an Amazon Neptune Analytics instance in your AWS account following the [AWS documentation](https://docs.aws.amazon.com/neptune-analytics/latest/userguide/get-started.html).
|
||||
Create an instance of Amazon Neptune Analytics in your AWS account following the [AWS documentation](https://docs.aws.amazon.com/neptune-analytics/latest/userguide/get-started.html).
|
||||
- Public connectivity is not enabled by default, and if accessing from outside a VPC, it needs to be enabled.
|
||||
- Once the Amazon Neptune Analytics instance is available, you will need the graph-identifier to connect.
|
||||
- The Neptune Analytics instance must be created using the same vector dimensions as the embedding model creates. See: https://docs.aws.amazon.com/neptune-analytics/latest/userguide/vector-index.html
|
||||
- The Neptune Analytics instance must be created using the same vector dimensions as the embedding model creates. See: [Vector indexing in Neptune Analytics](https://docs.aws.amazon.com/neptune-analytics/latest/userguide/vector-index.html).
|
||||
|
||||
#### Attach Credentials
|
||||
Ensure that you attach your AWS credentials with access to your Amazon Neptune Analytics resources by following the [Configuration and credentials precedence](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-configure.html#configure-precedence).
|
||||
|
||||
Configure your AWS credentials with access to your Amazon Neptune Analytics resources by following the [Configuration and credentials precedence](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-configure.html#configure-precedence).
|
||||
- For example, add your SSH access key session token via environment variables:
|
||||
```bash
|
||||
export AWS_ACCESS_KEY_ID=your-access-key
|
||||
export AWS_SECRET_ACCESS_KEY=your-secret-key
|
||||
export AWS_SESSION_TOKEN=your-session-token
|
||||
export AWS_DEFAULT_REGION=your-region
|
||||
```
|
||||
- The IAM user or role making the request must have a policy attached that allows one of the following IAM actions in that neptune-graph:
|
||||
The IAM user or role making the request must have a policy attached that allows one of the following IAM actions in that neptune-graph:
|
||||
- neptune-graph:ReadDataViaQuery
|
||||
- neptune-graph:WriteDataViaQuery
|
||||
- neptune-graph:DeleteDataViaQuery
|
||||
|
||||
#### Usage
|
||||
User can also customize the LLM for Graph Memory from the [Supported LLM list](https://docs.mem0.ai/components/llms/overview) with three levels of configuration:
|
||||
|
||||
The Neptune memory store uses AWS LangChain Python API to connect to Neptune instances. For additional configuration options for connecting to your Amazon Neptune Analytics instance see [AWS LangChain API documentation](https://python.langchain.com/api_reference/aws/graphs/langchain_aws.graphs.neptune_graph.NeptuneAnalyticsGraph.html).
|
||||
1. **Main Configuration**: If `llm` is set in the main config, it will be used for all graph operations.
|
||||
2. **Graph Store Configuration**: If `llm` is set in the graph_store config, it will override the main config `llm` and be used specifically for graph operations.
|
||||
3. **Default Configuration**: If no custom LLM is set, the default LLM (`gpt-4o-2024-08-06`) will be used for all graph operations.
|
||||
|
||||
Here's how you can do it:
|
||||
|
||||
<CodeGroup>
|
||||
```python Python
|
||||
@@ -279,13 +271,61 @@ m = Memory.from_config(config_dict=config)
|
||||
```
|
||||
</CodeGroup>
|
||||
|
||||
#### Troubleshooting
|
||||
Troubleshooting:
|
||||
|
||||
- For issues connecting to Amazon Neptune Analytics, please refer to the [Connecting to a graph guide](https://docs.aws.amazon.com/neptune-analytics/latest/userguide/gettingStarted-connecting.html).
|
||||
|
||||
- For issues related to authentication, refer to the [boto3 client configuration options](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html).
|
||||
|
||||
- For more details on how to connect, configure, and use the graph_memory graph store, see the Neptune Analytics example in our [AWS example guide](/examples/aws_example#aws-bedrock-and-aoss).
|
||||
- The Neptune memory store uses AWS LangChain Python API to connect to Neptune instances. For additional configuration options for connecting to your Amazon Neptune Analytics instance, see [AWS LangChain API documentation](https://python.langchain.com/api_reference/aws/graphs/langchain_aws.graphs.neptune_graph.NeptuneAnalyticsGraph.html).
|
||||
|
||||
### Initialize Neptune DB
|
||||
|
||||
Note that Neptune DB does not support vectors, and this graph store provider requires a collection in the vector store to save entity vectors.
|
||||
|
||||
Create a cluster of Amazon DB instances in your AWS account following the [AWS documentation](https://docs.aws.amazon.com/neptune/latest/userguide/graph-get-started.html).
|
||||
- Public connectivity is not enabled by default. To access the instance from outside a VPC, public connectivity needs to be enabled on the Neptune DB instance by following [Neptune Public Endpoints](https://docs.aws.amazon.com/neptune/latest/userguide/neptune-public-endpoints.html).
|
||||
- Once the Amazon Neptune Cluster instance is available, you will need the graph host endpoint to connect.
|
||||
- Neptune DB doesn't support vectors. The `collection_name` config field can be used to specify the vector store collection used to store vectors for the Neptune entities.
|
||||
|
||||
Ensure that you attach your AWS credentials with access to your Amazon Neptune Analytics resources by following the [Configuration and credentials precedence](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-configure.html#configure-precedence).
|
||||
|
||||
The IAM user or role making the request must have a policy attached that allows one of the following IAM actions in that neptune-db:
|
||||
- neptune-db:ReadDataViaQuery
|
||||
- neptune-db:WriteDataViaQuery
|
||||
- neptune-db:DeleteDataViaQuery
|
||||
|
||||
User can also customize the LLM for Graph Memory from the [Supported LLM list](https://docs.mem0.ai/components/llms/overview) with three levels of configuration:
|
||||
|
||||
1. **Main Configuration**: If `llm` is set in the main config, it will be used for all graph operations.
|
||||
2. **Graph Store Configuration**: If `llm` is set in the graph_store config, it will override the main config `llm` and be used specifically for graph operations.
|
||||
3. **Default Configuration**: If no custom LLM is set, the default LLM (`gpt-4o-2024-08-06`) will be used for all graph operations.
|
||||
|
||||
Here's how you can do it:
|
||||
|
||||
<CodeGroup>
|
||||
```python Python
|
||||
from mem0 import Memory
|
||||
|
||||
config = {
|
||||
"graph_store": {
|
||||
"provider": "neptunedb",
|
||||
"config": {
|
||||
"collection_name": "<VECTOR_COLLECTION_NAME>",
|
||||
"endpoint": "neptune-graph://<HOST_ENDPOINT>",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
m = Memory.from_config(config_dict=config)
|
||||
```
|
||||
</CodeGroup>
|
||||
|
||||
Troubleshooting:
|
||||
|
||||
- For issues connecting to Amazon Neptune Analytics, please refer to the [Accessing graph data in Amazon Neptune](https://docs.aws.amazon.com/neptune/latest/userguide/get-started-access-graph.html).
|
||||
- For issues related to authentication, refer to the [boto3 client configuration options](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html).
|
||||
- For more details on how to connect, configure, and use the graph_memory graph store, see the [Neptune DB example notebook](examples/graph-db-demo/neptune-example.ipynb).
|
||||
- The Neptune memory store uses AWS LangChain Python API to connect to Neptune instances. For additional configuration options for connecting to your Amazon Neptune Analytics instance, see [AWS LangChain API documentation](https://python.langchain.com/api_reference/aws/graphs/langchain_aws.graphs.neptune_graph.NeptuneGraph.html).
|
||||
|
||||
### Initialize Kuzu
|
||||
|
||||
|
||||
Reference in New Issue
Block a user