mem0/notebooks/llama2.ipynb

Cookbook for using LLAMA2 with Embedchain

Step-1: Install embedchain package

In [ ]:

!pip install embedchain

Step-2: Set LLAMA2 related environment variables and install dependencies

You can find OPENAI_API_KEY on your OpenAI dashboard and REPLICATE_API_TOKEN key on your Replicate dashboard. Now lets install the dependencies for LLAMA2.

In [ ]:

!pip install embedchain[llama2]

In [ ]:

import os
from embedchain import App

os.environ["OPENAI_API_KEY"] = "sk-xxx"
os.environ["REPLICATE_API_TOKEN"] = "xxx"

Step-3: Define your llm and embedding model config

In [ ]:

config = """
llm:
  provider: llama2
  config:
    model: 'a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5'
    temperature: 0.5
    max_tokens: 1000
    top_p: 0.5
    stream: false
"""

# Write the multi-line string to a YAML file
with open('llama2.yaml', 'w') as file:
    file.write(config)

Step-4 Create embedchain app based on the config

In [ ]:

app = App.from_config(yaml_path="llama2.yaml")

Step-5: Add data sources to your app

In [ ]:

app.add("https://www.forbes.com/profile/elon-musk")

Step-6: All set. Now start asking questions related to your data

In [ ]:

while(True):
    question = input("Enter question: ")
    if question in ['q', 'exit', 'quit']:
        break
    answer = app.query(question)
    print(answer)