Add support for ec runserver command (#1112 )

Fix top_p value bug in quickstart example in docs (#1110 )
[Bugfix] fix cache session id in chat method (#1107 )
2024-01-04 15:59:43 +05:30 · 2024-01-04 00:41:14 +05:30 · 2024-01-03 13:51:11 +05:30 · 2024-01-03 13:46:44 +05:30 · 2024-01-03 01:14:47 +05:30 · 2024-01-02 19:18:25 +05:30
406 changed files with 14208 additions and 3980 deletions
@@ -5,7 +5,7 @@ body:
 - type: markdown
  attributes:
    value: >
-      #### Before submitting a bug, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/gventuri/pandas-ai/issues?q=is%3Aissue+sort%3Acreated-desc+).
+      #### Before submitting a bug, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/embedchain/embedchain/issues?q=is%3Aissue+sort%3Acreated-desc+).
 - type: textarea
  attributes:
    label: 🐛 Describe the bug
@@ -3,7 +3,15 @@ name: ci
 on:
  push:
    branches: [main]
+    paths:
+      - 'embedchain/**'
+      - 'tests/**'
+      - 'examples/**'
  pull_request:
+    paths:
+      - 'embedchain/**'
+      - 'tests/**'
+      - 'examples/**'

 jobs:
  build:
@@ -76,7 +76,6 @@ docs/_build/
 target/

 # Jupyter Notebook
-*.yaml

 # IPython
 profile_default/
@@ -176,3 +175,6 @@ notebooks/*.yaml
 .ipynb_checkpoints/

 !configs/*.yaml
+
+# cache db
+*.db
@@ -37,6 +37,7 @@ clean:
 lint:
 	poetry run ruff .

+# for example: make test file=tests/test_factory.py
 test:
 	poetry run pytest $(file)

@@ -1,66 +1,66 @@
-# embedchain
+<p align="center">
+  <img src="docs/logo/dark.svg" width="400px" alt="Embedchain Logo">
+</p>

-<a href="https://runacap.com/ross-index/q3-2023/" target="_blank" rel="noopener"><img style="width: 260px; height: 56px" src="https://runacap.com/wp-content/uploads/2023/10/ROSS_badge_black_Q3_2023.svg" alt="ROSS Index - Fastest Growing Open-Source Startups in Q3 2023 | Runa Capital" width="260" height="56"/></a>
+<p align="center">
+  <a href="https://runacap.com/ross-index/q3-2023/" target="_blank" rel="noopener"><img style="width: 260px; height: 56px" src="https://runacap.com/wp-content/uploads/2023/10/ROSS_badge_black_Q3_2023.svg" alt="ROSS Index - Fastest Growing Open-Source Startups in Q3 2023 | Runa Capital" width="260" height="56"/></a>
+</p>

-[![PyPI](https://img.shields.io/pypi/v/embedchain)](https://pypi.org/project/embedchain/)
-[![Slack](https://img.shields.io/badge/slack-embedchain-brightgreen.svg?logo=slack)](https://join.slack.com/t/embedchain/shared_invite/zt-22uwz3c46-Zg7cIh5rOBteT_xe1jwLDw)
-[![Discord](https://dcbadge.vercel.app/api/server/6PzXDgEjG5?style=flat)](https://discord.gg/CUU9FPhRNt)
-[![Twitter](https://img.shields.io/twitter/follow/embedchain)](https://twitter.com/embedchain)
-[![Substack](https://img.shields.io/badge/Substack-%23006f5c.svg?logo=substack)](https://embedchain.substack.com/)
-[![Open in Colab](https://camo.githubusercontent.com/84f0493939e0c4de4e6dbe113251b4bfb5353e57134ffd9fcab6b8714514d4d1/68747470733a2f2f636f6c61622e72657365617263682e676f6f676c652e636f6d2f6173736574732f636f6c61622d62616467652e737667)](https://colab.research.google.com/drive/138lMWhENGeEu7Q1-6lNbNTHGLZXBBz_B?usp=sharing)
-[![codecov](https://codecov.io/gh/embedchain/embedchain/graph/badge.svg?token=EMRRHZXW1Q)](https://codecov.io/gh/embedchain/embedchain)
+<p align="center">
+  <a href="https://pypi.org/project/embedchain/">
+    <img src="https://img.shields.io/pypi/v/embedchain" alt="PyPI">
+  </a>
+  <a href="https://pepy.tech/project/embedchain">
+    <img src="https://static.pepy.tech/badge/embedchain" alt="Downloads">
+  </a>
+  <a href="https://embedchain.ai/slack">
+    <img src="https://img.shields.io/badge/slack-embedchain-brightgreen.svg?logo=slack" alt="Slack">
+  </a>
+  <a href="https://embedchain.ai/discord">
+    <img src="https://dcbadge.vercel.app/api/server/6PzXDgEjG5?style=flat" alt="Discord">
+  </a>
+  <a href="https://twitter.com/embedchain">
+    <img src="https://img.shields.io/twitter/follow/embedchain" alt="Twitter">
+  </a>
+  <a href="https://colab.research.google.com/drive/138lMWhENGeEu7Q1-6lNbNTHGLZXBBz_B?usp=sharing">
+    <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab">
+  </a>
+  <a href="https://codecov.io/gh/embedchain/embedchain">
+    <img src="https://codecov.io/gh/embedchain/embedchain/graph/badge.svg?token=EMRRHZXW1Q" alt="codecov">
+  </a>
+</p>

-Embedchain is a Data Platform for LLMs - load, index, retrieve, and sync any unstructured data. Using embedchain, you can easily create LLM powered apps over any data. If you want a javascript version, check out [embedchain-js](https://github.com/embedchain/embedchain/tree/main/embedchain-js)
+<hr />

-## Community

-* Join embedchain community on slack by accepting [this invite](https://join.slack.com/t/embedchain/shared_invite/zt-22uwz3c46-Zg7cIh5rOBteT_xe1jwLDw)
+> ### Checkout our latest [Sadhguru AI app](https://sadhguru-ai.streamlit.app/) built using Embedchain.

-## 🤝 Schedule a 1-on-1 Session
+## What is Embedchain?

-Book a [1-on-1 Session](https://cal.com/taranjeetio/ec) with Taranjeet, the founder, to discuss any issues, provide feedback, or explore how we can improve Embedchain for you.
+Embedchain is an Open Source RAG Framework that makes it easy to create and deploy AI apps. At its core, Embedchain follows the design principle of being *"Conventional but Configurable"* to serve both software engineers and machine learning engineers.
+
+Embedchain streamlines the creation of RAG applications, offering a seamless process for managing various types of unstructured data. It efficiently segments data into manageable chunks, generates relevant embeddings, and stores them in a vector database for optimized retrieval. With a suite of diverse APIs, it enables users to extract contextual information, find precise answers, or engage in interactive chat conversations, all tailored to their own data.

 ## 🔧 Quick install

+### Python API
+
 ```bash
-pip install --upgrade embedchain
+pip install embedchain
 ```

-## 🔍 Demo
+## ✨ Live demo

-Try out embedchain in your browser:
+Checkout the [Chat with PDF](https://embedchain.ai/demo/chat-pdf) live demo we created using Embedchain. You can find the source code [here](https://github.com/embedchain/embedchain/tree/main/examples/chat-pdf).

-[![Open in Colab](https://camo.githubusercontent.com/84f0493939e0c4de4e6dbe113251b4bfb5353e57134ffd9fcab6b8714514d4d1/68747470733a2f2f636f6c61622e72657365617263682e676f6f676c652e636f6d2f6173736574732f636f6c61622d62616467652e737667)](https://colab.research.google.com/drive/17ON1LPonnXAtLaZEebnOktstB_1cJJmh?usp=sharing)
+## 🔍 Usage

-## 📖 Documentation
+<!-- Demo GIF or Image -->
+<p align="center">
+  <img src="docs/images/cover.gif" width="900px" alt="Embedchain Demo">
+</p>

-The documentation for embedchain can be found at [docs.embedchain.ai](https://docs.embedchain.ai).
-
-## 💻 Usage
-
-Embedchain empowers you to create ChatGPT like apps, on your own dynamic dataset.
-
-### Data types supported
-
-* Youtube video
-* PDF file
-* CSV file
-* Web page
-* MDX file
-* XML file
-* Sitemap
-* Doc file
-* Notion
-* JSON file
-* OpenAPI specs
-* Code docs website
-* Unstructured file loader and many more
-
-You can find the full list of data types on [our documentation](https://docs.embedchain.ai/data-sources/).
-
-### Queries
-
-For example, you can use Embedchain to create an Elon Musk bot using the following code:
+For example, you can create an Elon Musk bot using the following code:

 ```python
 import os
@@ -73,53 +73,35 @@ elon_bot = App()
 # Embed online resources
 elon_bot.add("https://en.wikipedia.org/wiki/Elon_Musk")
 elon_bot.add("https://www.forbes.com/profile/elon-musk")
-elon_bot.add("https://www.youtube.com/watch?v=RcYjXbSJBN8")

 # Query the bot
 elon_bot.query("How many companies does Elon Musk run and name those?")
 # Answer: Elon Musk currently runs several companies. As of my knowledge, he is the CEO and lead designer of SpaceX, the CEO and product architect of Tesla, Inc., the CEO and founder of Neuralink, and the CEO and founder of The Boring Company. However, please note that this information may change over time, so it's always good to verify the latest updates.
-
-# (Optional): Deploy app to Embedchain Platform
-app.deploy()
-# 🔑 Enter your Embedchain API key. You can find the API key at https://app.embedchain.ai/settings/keys/
-# ec-xxxxxx
-
-# 🛠️ Creating pipeline on the platform...
-# 🎉🎉🎉 Pipeline created successfully! View your pipeline: https://app.embedchain.ai/pipelines/xxxxx
-
-# 🛠️ Adding data to your pipeline...
-# ✅ Data of type: web_page, value: https://www.forbes.com/profile/elon-musk added successfully.
 ```

-## Examples
+You can also try it in your browser with Google Colab:

-| LLM          | Google Colab  | Replit   |
-|--------------|---------------|----------|
-| OpenAI       | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/openai.ipynb)           | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/openai#main.py)      |
-| Anthropic    | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/anthropic.ipynb)        | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/anthropic#main.py)   |
-| Azure OpenAI | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/azure-openai.ipynb)     | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/azureopenai#main.py) |
-| VertexAI     | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/vertex_ai.ipynb)        | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/vertexai#main.py)    |
-| Cohere       | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/cohere.ipynb)           | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/cohere#main.py)      |
-| Hugging Face | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/hugging_face_hub.ipynb) | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/huggingface#main.py) |
-| JinaChat     | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/jina.ipynb)             | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/jina#main.py)        |
-| GPT4All      | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/gpt4all.ipynb)          | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/gpt4all#main.py)     |
-| Llama2       | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/llama2.ipynb)           | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/llama2#main.py)      |
+[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/17ON1LPonnXAtLaZEebnOktstB_1cJJmh?usp=sharing)

-| Embedding model     | Google Colab                                                                                                                                                                            | Replit                                                                                                                        |
-| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
-| OpenAI       | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/openai.ipynb)           | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/openai#main.py)      |
-| VertexAI     | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/vertex_ai.ipynb)        | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/vertexai#main.py)    |
-| GPT4All      | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/gpt4all.ipynb)          | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/gpt4all#main.py)     |
-| Hugging Face | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/hugging_face_hub.ipynb) | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/huggingface#main.py) |
+## 📖 Documentation
+Comprehensive guides and API documentation are available to help you get the most out of Embedchain:

-| Vector DB     | Google Colab                                                                                                                                                                         | Replit                                                                                                                          |
-| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------- |
-| ChromaDB      | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/chromadb.ipynb)      | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/chromadb#main.py)      |
-| Elasticsearch | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/elasticsearch.ipynb) | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/elasticsearchdb#main.py) |
-| Opensearch    | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/opensearch.ipynb)    | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/opensearchdb#main.py)    |
-| Pinecone      | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/pinecone.ipynb)      | [![Try with Replit Badge](https://replit.com/badge?caption=Try%20with%20Replit&variant=small)](https://replit.com/@taranjeetio/pineconedb#main.py)      |
+- [Introduction](https://docs.embedchain.ai/get-started/introduction#what-is-embedchain)
+- [Getting Started](https://docs.embedchain.ai/get-started/quickstart)
+- [Examples](https://docs.embedchain.ai/examples)
+- [Supported data types](https://docs.embedchain.ai/components/data-sources/overview)

-## 🤝 Contributing
+## 🔗 Join the Community
+
+* Connect with fellow developers by joining our [Slack Community](https://embedchain.ai/slack) or [Discord Community](https://embedchain.ai/discord).
+
+* Dive into [GitHub Discussions](https://github.com/embedchain/embedchain/discussions), ask questions, or share your experiences.
+
+## 🤝 Schedule a 1-on-1 Session
+
+Book a [1-on-1 Session](https://cal.com/taranjeetio/ec) with the founders, to discuss any issues, provide feedback, or explore how we can improve Embedchain for you.
+
+## 🌐 Contributing

 Contributions are welcome! Please check out the issues on the repository, and feel free to open a pull request.
 For more information, please see the [contributing guidelines](CONTRIBUTING.md).
@@ -130,9 +112,9 @@ For more reference, please go through [Development Guide](https://docs.embedchai
  <img src="https://contrib.rocks/image?repo=embedchain/embedchain" />
 </a>

-## Telemetry
+## Anonymous Telemetry

-We collect anonymous usage metrics to enhance our package's quality and user experience. This includes data like feature usage frequency and system info, but never personal details. The data helps us prioritize improvements and ensure compatibility. If you wish to opt-out, set the `app.config.collect_metrics = False` in the code. We prioritize data security and don't share this data externally.
+We collect anonymous usage metrics to enhance our package's quality and user experience. This includes data like feature usage frequency and system info, but never personal details. The data helps us prioritize improvements and ensure compatibility. If you wish to opt-out, set the environment variable `EC_TELEMETRY=false`. We prioritize data security and don't share this data externally.

 ## Citation

@@ -141,7 +123,7 @@ If you utilize this repository, please consider citing it with:
 ```
@misc{embedchain,
  author = {Taranjeet Singh, Deshraj Yadav},
-  title = {Embedchain: Data platform for LLMs - load, index, retrieve, and sync any unstructured data},
+  title = {Embedchain: The Open Source RAG Framework},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
@@ -1,7 +1,7 @@
 llm:
  provider: anthropic
-  model: 'claude-instant-1'
  config:
+    model: 'claude-instant-1'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -4,8 +4,8 @@ app:

 llm:
  provider: azure_openai
-  model: gpt-35-turbo
  config:
+    model: gpt-35-turbo
    deployment_name: your_llm_deployment_name
    temperature: 0.5
    max_tokens: 1000
@@ -1,12 +1,11 @@
 app:
  config:
    id: 'my-app'
-    collection_name: 'my-app'

 llm:
  provider: openai
-  model: 'gpt-3.5-turbo'
  config:
+    model: 'gpt-3.5-turbo'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -23,4 +22,3 @@ embedder:
  provider: openai
  config:
    model: 'text-embedding-ada-002'
-    deployment_name: null
@@ -0,0 +1,4 @@
+chunker:
+  chunk_size: 100
+  chunk_overlap: 20
+  length_function: 'len'
@@ -1,7 +1,7 @@
 llm:
  provider: cohere
-  model: large
  config:
+    model: large
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -2,15 +2,20 @@ app:
  config:
    id: 'full-stack-app'

+chunker:
+  chunk_size: 100
+  chunk_overlap: 20
+  length_function: 'len'
+
 llm:
  provider: openai
-  model: 'gpt-3.5-turbo'
  config:
+    model: 'gpt-3.5-turbo'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
    stream: false
-    template: |
+    prompt: |
      Use the following pieces of context to answer the query at the end.
      If you don't know the answer, just say that you don't know, don't try to make up an answer.

@@ -0,0 +1,13 @@
+llm:
+  provider: google
+  config:
+    model: gemini-pro
+    max_tokens: 1000
+    temperature: 0.9
+    top_p: 1.0
+    stream: false
+
+embedder:
+  provider: google
+  config:
+    model: models/embedding-001
@@ -0,0 +1,8 @@
+llm:
+  provider: openai
+  config:
+    model: 'gpt-4'
+    temperature: 0.5
+    max_tokens: 1000
+    top_p: 1
+    stream: false
@@ -1,7 +1,7 @@
 llm:
  provider: gpt4all
  config:
-    model: 'orca-mini-3b.ggmlv3.q4_0.bin'
+    model: 'orca-mini-3b-gguf2-q4_0.gguf'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -1,7 +1,7 @@
 llm:
  provider: huggingface
-  model: 'google/flan-t5-xxl'
  config:
+    model: 'google/flan-t5-xxl'
    temperature: 0.5
    max_tokens: 1000
    top_p: 0.5
@@ -1,7 +1,7 @@
 llm:
  provider: llama2
-  model: 'a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5'
  config:
+    model: 'a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5'
    temperature: 0.5
    max_tokens: 1000
    top_p: 0.5
@@ -0,0 +1,12 @@
+llm:
+  provider: ollama
+  config:
+    model: 'llama2'
+    temperature: 0.5
+    top_p: 1
+    stream: true
+
+embedder:
+  provider: huggingface
+  config:
+    model: 'BAAI/bge-small-en-v1.5'
@@ -1,14 +1,14 @@
 app:
  config:
    id: 'my-app'
-    log_level: 'WARN'
+    log_level: 'WARNING'
    collect_metrics: true
    collection_name: 'my-app'

 llm:
  provider: openai
-  model: 'gpt-3.5-turbo'
  config:
+    model: 'gpt-3.5-turbo'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -30,4 +30,4 @@ embedder:
  provider: openai
  config:
    model: 'text-embedding-ada-002'
-    deployment_name: null
+    deployment_name: 'my-app'
@@ -1,13 +1,12 @@
 app:
  config:
    id: 'open-source-app'
-    collection_name: 'open-source-app'
    collect_metrics: false

 llm:
  provider: gpt4all
  config:
-    model: 'orca-mini-3b.ggmlv3.q4_0.bin'
+    model: 'orca-mini-3b-gguf2-q4_0.gguf'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -23,4 +22,4 @@ vectordb:
 embedder:
  provider: gpt4all
  config:
-    deployment_name: null
+    deployment_name: 'test-deployment'
@@ -13,7 +13,7 @@ vectordb:
 llm:
  provider: gpt4all
  config:
-    model: 'orca-mini-3b.ggmlv3.q4_0.bin'
+    model: 'orca-mini-3b-gguf2-q4_0.gguf'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -0,0 +1,6 @@
+llm:
+  provider: together
+  config:
+    model: mistralai/Mixtral-8x7B-Instruct-v0.1
+    temperature: 0.5
+    max_tokens: 1000
@@ -1,6 +1,6 @@
 llm:
  provider: vertexai
-  model: 'chat-bison'
  config:
+    model: 'chat-bison'
    temperature: 0.5
    top_p: 0.5
@@ -1,11 +1,11 @@
 <CardGroup cols={3}>
+  <Card title="Talk to founders" icon="calendar" href="https://cal.com/taranjeetio/ec">
+  Schedule a call
+  </Card>
  <Card title="Slack" icon="slack" href="https://join.slack.com/t/embedchain/shared_invite/zt-22uwz3c46-Zg7cIh5rOBteT_xe1jwLDw" color="#4A154B">
    Join our slack community
  </Card>
  <Card title="Discord" icon="discord" href="https://discord.gg/6PzXDgEjG5" color="#7289DA">
    Join our discord community
  </Card>
-  <Card title="Schedule a call" icon="calendar" href="https://cal.com/taranjeetio/ec">
-  Schedule a call with Embedchain founder
-  </Card>
 </CardGroup>
@@ -1,7 +1,9 @@
-<Tip>
-If you can't find the specific data source, please feel free to request through one of the following channels and help us prioritize.
+<p>If you can't find the specific data source, please feel free to request through one of the following channels and help us prioritize.</p>

 <CardGroup cols={2}>
+  <Card title="Google Form" icon="file" href="https://forms.gle/NDRCKsRpUHsz2Wcm8" color="#7387d0">
+    Fill out this form
+  </Card>
  <Card title="Slack" icon="slack" href="https://join.slack.com/t/embedchain/shared_invite/zt-22uwz3c46-Zg7cIh5rOBteT_xe1jwLDw" color="#4A154B">
    Let us know on our slack community
  </Card>
@@ -15,4 +17,3 @@ If you can't find the specific data source, please feel free to request through
  Schedule a call with Embedchain founder
  </Card>
 </CardGroup>
-</Tip>
@@ -1,5 +1,4 @@
-<Tip>
-If you can't find the specific LLM you need, no need to fret. We're continuously expanding our support for additional LLMs, and you can help us prioritize by opening an issue on our GitHub or simply reaching out to us on our Slack or Discord community.
+<p>If you can't find the specific LLM you need, no need to fret. We're continuously expanding our support for additional LLMs, and you can help us prioritize by opening an issue on our GitHub or simply reaching out to us on our Slack or Discord community.</p>

 <CardGroup cols={2}>
  <Card title="Slack" icon="slack" href="https://join.slack.com/t/embedchain/shared_invite/zt-22uwz3c46-Zg7cIh5rOBteT_xe1jwLDw" color="#4A154B">
@@ -15,4 +14,3 @@ If you can't find the specific LLM you need, no need to fret. We're continuously
  Schedule a call with Embedchain founder
  </Card>
 </CardGroup>
-</Tip>
@@ -1,5 +1,6 @@
-<Tip>
-If you can't find the specific vector database, please feel free to request through one of the following channels and help us prioritize.
+
+
+<p>If you can't find the specific vector database, please feel free to request through one of the following channels and help us prioritize.</p>

 <CardGroup cols={2}>
  <Card title="Slack" icon="slack" href="https://join.slack.com/t/embedchain/shared_invite/zt-22uwz3c46-Zg7cIh5rOBteT_xe1jwLDw" color="#4A154B">
@@ -15,4 +16,3 @@ If you can't find the specific vector database, please feel free to request thro
  Schedule a call with Embedchain founder
  </Card>
 </CardGroup>
-</Tip>
@@ -1,75 +0,0 @@
---
-title: '⚙️ Custom configurations'
---
-
-Embedchain is made to work out of the box. However, for advanced users we're also offering configuration options. All of these configuration options are optional and have sane defaults.
-
-You can configure different components of your app (`llm`, `embedding model`, or `vector database`) through a simple yaml configuration that Embedchain offers. Here is a generic full-stack example of the yaml config:
-
-```yaml
-app:
-  config:
-    id: 'full-stack-app'
-
-llm:
-  provider: openai
-  config:
-    model: 'gpt-3.5-turbo'
-    temperature: 0.5
-    max_tokens: 1000
-    top_p: 1
-    stream: false
-    template: |
-      Use the following pieces of context to answer the query at the end.
-      If you don't know the answer, just say that you don't know, don't try to make up an answer.
-
-      $context
-
-      Query: $query
-
-      Helpful Answer:
-    system_prompt: |
-      Act as William Shakespeare. Answer the following questions in the style of William Shakespeare.
-
-vectordb:
-  provider: chroma
-  config:
-    collection_name: 'full-stack-app'
-    dir: db
-    allow_reset: true
-
-embedder:
-  provider: openai
-  config:
-    model: 'text-embedding-ada-002'
-```
-
-Alright, let's dive into what each key means in the yaml config above:
-
-1. `app` Section:
-    - `config`:
-        - `id` (String): The ID or name of your full-stack application.
-2. `llm` Section:
-    - `provider` (String): The provider for the language model, which is set to 'openai'. You can find the full list of llm providers in [our docs](/components/llms).
-    - `model` (String): The specific model being used, 'gpt-3.5-turbo'.
-    - `config`:
-        - `temperature` (Float): Controls the randomness of the model's output. A higher value (closer to 1) makes the output more random.
-        - `max_tokens` (Integer): Controls how many tokens are used in the response.
-        - `top_p` (Float): Controls the diversity of word selection. A higher value (closer to 1) makes word selection more diverse.
-        - `stream` (Boolean): Controls if the response is streamed back to the user (set to false).
-        - `template` (String): A custom template for the prompt that the model uses to generate responses.
-        - `system_prompt` (String): A system prompt for the model to follow when generating responses, in this case, it's set to the style of William Shakespeare.
-3. `vectordb` Section:
-    - `provider` (String): The provider for the vector database, set to 'chroma'. You can find the full list of vector database providers in [our docs](/components/vector-databases).
-    - `config`:
-        - `collection_name` (String): The initial collection name for the database, set to 'full-stack-app'.
-        - `dir` (String): The directory for the database, set to 'db'.
-        - `allow_reset` (Boolean): Indicates whether resetting the database is allowed, set to true.
-4. `embedder` Section:
-    - `provider` (String): The provider for the embedder, set to 'openai'. You can find the full list of embedding model providers in [our docs](/components/embedding-models).
-    - `config`:
-        - `model` (String): The specific model used for text embedding, 'text-embedding-ada-002'.
-
-If you have questions about the configuration above, please feel free to reach out to us using one of the following methods:
-
-<Snippet file="get-help.mdx" />
@@ -0,0 +1,236 @@
+---
+title: 'Custom configurations'
+---
+
+Embedchain offers several configuration options for your LLM, vector database, and embedding model. All of these configuration options are optional and have sane defaults.
+
+You can configure different components of your app (`llm`, `embedding model`, or `vector database`) through a simple yaml configuration that Embedchain offers. Here is a generic full-stack example of the yaml config:
+
+
+<Tip>
+Embedchain applications are configurable using YAML file, JSON file or by directly passing the config dictionary. Checkout the [docs here](/api-reference/pipeline/overview#usage) on how to use other formats.
+</Tip>
+
+<CodeGroup>
+```yaml config.yaml
+app:
+  config:
+    name: 'full-stack-app'
+
+llm:
+  provider: openai
+  config:
+    model: 'gpt-3.5-turbo'
+    temperature: 0.5
+    max_tokens: 1000
+    top_p: 1
+    stream: false
+    api_key: sk-xxx
+    prompt: |
+      Use the following pieces of context to answer the query at the end.
+      If you don't know the answer, just say that you don't know, don't try to make up an answer.
+
+      $context
+
+      Query: $query
+
+      Helpful Answer:
+    system_prompt: |
+      Act as William Shakespeare. Answer the following questions in the style of William Shakespeare.
+
+vectordb:
+  provider: chroma
+  config:
+    collection_name: 'full-stack-app'
+    dir: db
+    allow_reset: true
+
+embedder:
+  provider: openai
+  config:
+    model: 'text-embedding-ada-002'
+    api_key: sk-xxx
+
+chunker:
+  chunk_size: 2000
+  chunk_overlap: 100
+  length_function: 'len'
+  min_chunk_size: 0
+
+cache:
+  similarity_evaluation:
+    strategy: distance
+    max_distance: 1.0
+  config:
+    similarity_threshold: 0.8
+    auto_flush: 50
+```
+
+```json config.json
+{
+  "app": {
+    "config": {
+      "name": "full-stack-app"
+    }
+  },
+  "llm": {
+    "provider": "openai",
+    "config": {
+      "model": "gpt-3.5-turbo",
+      "temperature": 0.5,
+      "max_tokens": 1000,
+      "top_p": 1,
+      "stream": false,
+      "prompt": "Use the following pieces of context to answer the query at the end.\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n$context\n\nQuery: $query\n\nHelpful Answer:",
+      "system_prompt": "Act as William Shakespeare. Answer the following questions in the style of William Shakespeare.",
+      "api_key": "sk-xxx"
+    }
+  },
+  "vectordb": {
+    "provider": "chroma",
+    "config": {
+      "collection_name": "full-stack-app",
+      "dir": "db",
+      "allow_reset": true
+    }
+  },
+  "embedder": {
+    "provider": "openai",
+    "config": {
+      "model": "text-embedding-ada-002",
+      "api_key": "sk-xxx"
+    }
+  },
+  "chunker": {
+    "chunk_size": 2000,
+    "chunk_overlap": 100,
+    "length_function": "len",
+    "min_chunk_size": 0
+  },
+  "cache": {
+    "similarity_evaluation": {
+        "strategy": "distance",
+        "max_distance": 1.0,
+    },
+    "config": {
+        "similarity_threshold": 0.8,
+        "auto_flush": 50,
+    },
+  },
+}
+```
+
+```python config.py
+config = {
+    'app': {
+        'config': {
+            'name': 'full-stack-app'
+        }
+    },
+    'llm': {
+        'provider': 'openai',
+        'config': {
+            'model': 'gpt-3.5-turbo',
+            'temperature': 0.5,
+            'max_tokens': 1000,
+            'top_p': 1,
+            'stream': False,
+            'prompt': (
+                "Use the following pieces of context to answer the query at the end.\n"
+                "If you don't know the answer, just say that you don't know, don't try to make up an answer.\n"
+                "$context\n\nQuery: $query\n\nHelpful Answer:"
+            ),
+            'system_prompt': (
+                "Act as William Shakespeare. Answer the following questions in the style of William Shakespeare."
+            ),
+            'api_key': 'sk-xxx'
+        }
+    },
+    'vectordb': {
+        'provider': 'chroma',
+        'config': {
+            'collection_name': 'full-stack-app',
+            'dir': 'db',
+            'allow_reset': True
+        }
+    },
+    'embedder': {
+        'provider': 'openai',
+        'config': {
+            'model': 'text-embedding-ada-002',
+            'api_key': 'sk-xxx'
+        }
+    },
+    'chunker': {
+        'chunk_size': 2000,
+        'chunk_overlap': 100,
+        'length_function': 'len',
+        'min_chunk_size': 0
+    },
+    'cache': {
+      'similarity_evaluation': {
+          'strategy': 'distance',
+          'max_distance': 1.0,
+      },
+      'config': {
+          'similarity_threshold': 0.8,
+          'auto_flush': 50,
+      },
+    },
+}
+```
+</CodeGroup>
+
+Alright, let's dive into what each key means in the yaml config above:
+
+1. `app` Section:
+    - `config`:
+        - `name` (String): The name of your full-stack application.
+        - `id` (String): The id of your full-stack application.
+        <Note>Only use this to reload already created apps. We recommend users to not create their own ids.</Note>
+        - `collect_metrics` (Boolean): Indicates whether metrics should be collected for the app, defaults to `True`
+        - `log_level` (String): The log level for the app, defaults to `WARNING`
+2. `llm` Section:
+    - `provider` (String): The provider for the language model, which is set to 'openai'. You can find the full list of llm providers in [our docs](/components/llms).
+    - `config`:
+        - `model` (String): The specific model being used, 'gpt-3.5-turbo'.
+        - `temperature` (Float): Controls the randomness of the model's output. A higher value (closer to 1) makes the output more random.
+        - `max_tokens` (Integer): Controls how many tokens are used in the response.
+        - `top_p` (Float): Controls the diversity of word selection. A higher value (closer to 1) makes word selection more diverse.
+        - `stream` (Boolean): Controls if the response is streamed back to the user (set to false).
+        - `prompt` (String): A prompt for the model to follow when generating responses, requires $context and $query variables.
+        - `system_prompt` (String): A system prompt for the model to follow when generating responses, in this case, it's set to the style of William Shakespeare.
+        -  `stream` (Boolean): Controls if the response is streamed back to the user (set to false).
+        - `number_documents` (Integer): Number of documents to pull from the vectordb as context, defaults to 1
+        - `api_key` (String): The API key for the language model.
+3. `vectordb` Section:
+    - `provider` (String): The provider for the vector database, set to 'chroma'. You can find the full list of vector database providers in [our docs](/components/vector-databases).
+    - `config`:
+        - `collection_name` (String): The initial collection name for the vectordb, set to 'full-stack-app'.
+        - `dir` (String): The directory for the local database, set to 'db'.
+        - `allow_reset` (Boolean): Indicates whether resetting the vectordb is allowed, set to true.
+    <Note>We recommend you to checkout vectordb specific config [here](https://docs.embedchain.ai/components/vector-databases)</Note>
+4. `embedder` Section:
+    - `provider` (String): The provider for the embedder, set to 'openai'. You can find the full list of embedding model providers in [our docs](/components/embedding-models).
+    - `config`:
+        - `model` (String): The specific model used for text embedding, 'text-embedding-ada-002'.
+        - `api_key` (String): The API key for the embedding model.
+5. `chunker` Section:
+    - `chunk_size` (Integer): The size of each chunk of text that is sent to the language model.
+    - `chunk_overlap` (Integer): The amount of overlap between each chunk of text.
+    - `length_function` (String): The function used to calculate the length of each chunk of text. In this case, it's set to 'len'. You can also use any function import directly as a string here.
+    - `min_chunk_size` (Integer): The minimum size of each chunk of text that is sent to the language model. Must be less than `chunk_size`, and greater than `chunk_overlap`.
+6. `cache` Section: (Optional)
+    - `similarity_evaluation` (Optional): The config for similarity evaluation strategy. If not provided, the default `distance` based similarity evaluation strategy is used.
+      - `strategy` (String): The strategy to use for similarity evaluation. Currently, only `distance` and `exact` based similarity evaluation is supported. Defaults to `distance`.
+      - `max_distance` (Float): The bound of maximum distance. Defaults to `1.0`.
+      - `positive` (Boolean): If the larger distance indicates more similar of two entities, set it `True`, otherwise `False`. Defaults to `False`.
+    - `config` (Optional): The config for initializing the cache. If not provided, sensible default values are used as mentioned below.
+      - `similarity_threshold` (Float): The threshold for similarity evaluation. Defaults to `0.8`.
+      - `auto_flush` (Integer): The number of queries after which the cache is flushed. Defaults to `20`.
+    <Note>
+    If you provide a cache section, the app will automatically configure and use a cache to store the results of the language model. This is useful if you want to speed up the response time and save inference cost of your app.
+    </Note>
+If you have questions about the configuration above, please feel free to reach out to us using one of the following methods:
+
+<Snippet file="get-help.mdx" />
@@ -0,0 +1,44 @@
+---
+title: '📊 add'
+---
+
+`add()` method is used to load the data sources from different data sources to a RAG pipeline. You can find the signature below:
+
+### Parameters
+
+<ParamField path="source" type="str">
+    The data to embed, can be a URL, local file or raw content, depending on the data type.. You can find the full list of supported data sources [here](/components/data-sources/overview).
+</ParamField>
+<ParamField path="data_type" type="str" optional>
+    Type of data source. It can be automatically detected but user can force what data type to load as.
+</ParamField>
+<ParamField path="metadata" type="dict" optional>
+    Any metadata that you want to store with the data source. Metadata is generally really useful for doing metadata filtering on top of semantic search to yield faster search and better results.
+</ParamField>
+
+## Usage
+
+### Load data from webpage
+
+```python Code example
+from embedchain import App
+
+app = App()
+app.add("https://www.forbes.com/profile/elon-musk")
+# Inserting batches in chromadb: 100%|███████████████| 1/1 [00:00<00:00,  1.19it/s]
+# Successfully saved https://www.forbes.com/profile/elon-musk (DataType.WEB_PAGE). New chunks count: 4
+```
+
+### Load data from sitemap
+
+```python Code example
+from embedchain import App
+
+app = App()
+app.add("https://python.langchain.com/sitemap.xml", data_type="sitemap")
+# Loading pages: 100%|█████████████| 1108/1108 [00:47<00:00, 23.17it/s]
+# Inserting batches in chromadb: 100%|█████████| 111/111 [04:41<00:00,  2.54s/it]
+# Successfully saved https://python.langchain.com/sitemap.xml (DataType.SITEMAP). New chunks count: 11024
+```
+
+You can find complete list of supported data sources [here](/components/data-sources/overview).
@@ -0,0 +1,131 @@
+---
+title: '💬 chat'
+---
+
+`chat()` method allows you to chat over your data sources using a user-friendly chat API. You can find the signature below:
+
+### Parameters
+
+<ParamField path="input_query" type="str">
+    Question to ask
+</ParamField>
+<ParamField path="config" type="BaseLlmConfig" optional>
+    Configure different llm settings such as prompt, temprature, number_documents etc.
+</ParamField>
+<ParamField path="dry_run" type="bool" optional>
+    The purpose is to test the prompt structure without actually running LLM inference. Defaults to `False`
+</ParamField>
+<ParamField path="where" type="dict" optional>
+    A dictionary of key-value pairs to filter the chunks from the vector database. Defaults to `None`
+</ParamField>
+<ParamField path="session_id" type="str" optional>
+    Session ID of the chat. This can be used to maintain chat history of different user sessions. Default value: `default`
+</ParamField>
+<ParamField path="citations" type="bool" optional>
+    Return citations along with the LLM answer. Defaults to `False`
+</ParamField>
+
+### Returns
+
+<ResponseField name="answer" type="str | tuple">
+  If `citations=False`, return a stringified answer to the question asked. <br />
+  If `citations=True`, returns a tuple with answer and citations respectively.
+</ResponseField>
+
+## Usage
+
+### With citations
+
+If you want to get the answer to question and return both answer and citations, use the following code snippet:
+
+```python With Citations
+from embedchain import App
+
+# Initialize app
+app = App()
+
+# Add data source
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Get relevant answer for your query
+answer, sources = app.chat("What is the net worth of Elon?", citations=True)
+print(answer)
+# Answer: The net worth of Elon Musk is $221.9 billion.
+
+print(sources)
+# [
+#    (
+#        'Elon Musk PROFILEElon MuskCEO, Tesla$247.1B$2.3B (0.96%)Real Time Net Worthas of 12/7/23 ...',
+#        {
+#           'url': 'https://www.forbes.com/profile/elon-musk', 
+#           'score': 0.89,
+#           ...
+#        }
+#    ),
+#    (
+#        '74% of the company, which is now called X.Wealth HistoryHOVER TO REVEAL NET WORTH BY YEARForbes ...',
+#        {
+#           'url': 'https://www.forbes.com/profile/elon-musk', 
+#           'score': 0.81,
+#           ...
+#        }
+#    ),
+#    (
+#        'founded in 2002, is worth nearly $150 billion after a $750 million tender offer in June 2023 ...',
+#        {
+#           'url': 'https://www.forbes.com/profile/elon-musk', 
+#           'score': 0.73,
+#           ...
+#        }
+#    )
+# ]
+```
+
+<Note>
+When `citations=True`, note that the returned `sources` are a list of tuples where each tuple has two elements (in the following order):
+1. source chunk
+2. dictionary with metadata about the source chunk
+    - `url`: url of the source
+    - `doc_id`: document id (used for book keeping purposes)
+    - `score`: score of the source chunk with respect to the question
+    - other metadata you might have added at the time of adding the source
+</Note>
+
+
+### Without citations
+
+If you just want to return answers and don't want to return citations, you can use the following example:
+
+```python Without Citations
+from embedchain import App
+
+# Initialize app
+app = App()
+
+# Add data source
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Chat on your data using `.chat()`
+answer = app.chat("What is the net worth of Elon?")
+print(answer)
+# Answer: The net worth of Elon Musk is $221.9 billion.
+```
+
+### With session id
+
+If you want to maintain chat sessions for different users, you can simply pass the `session_id` keyword argument. See the example below:
+
+```python With session id
+from embedchain import App
+
+app = App()
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Chat on your data using `.chat()`
+app.chat("What is the net worth of Elon Musk?", session_id="user1")
+# 'The net worth of Elon Musk is $250.8 billion.'
+app.chat("What is the net worth of Bill Gates?", session_id="user2")
+# "I don't know the current net worth of Bill Gates."
+app.chat("What was my last question", session_id="user1")
+# 'Your last question was "What is the net worth of Elon Musk?"'
+```
@@ -0,0 +1,19 @@
+---
+title: 🗑 delete
+---
+
+`delete_chat_history()` method allows you to delete all previous messages in a chat history.
+
+## Usage
+
+```python
+from embedchain import App
+
+app = App()
+
+app.add("https://www.forbes.com/profile/elon-musk")
+
+app.chat("What is the net worth of Elon Musk?")
+
+app.delete_chat_history()
+```
@@ -0,0 +1,31 @@
+---
+title: 🚀 deploy
+---
+
+Using the `deploy()` method, Embedchain allows developers to easily launch their LLM-powered applications on the [Embedchain Platform](https://app.embedchain.ai). This platform facilitates seamless access to your data's context via a free and user-friendly REST API. Once your pipeline is deployed, you can update your data sources at any time.
+
+The `deploy()` method not only deploys your pipeline but also efficiently manages LLMs, vector databases, embedding models, and data syncing, enabling you to focus on querying, chatting, or searching without the hassle of infrastructure management.
+
+## Usage
+
+```python
+from embedchain import App
+
+# Initialize app
+app = App()
+
+# Add data source
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Deploy your pipeline to Embedchain Platform
+app.deploy()
+
+# 🔑 Enter your Embedchain API key. You can find the API key at https://app.embedchain.ai/settings/keys/
+# ec-xxxxxx
+
+# 🛠️ Creating pipeline on the platform...
+# 🎉🎉🎉 Pipeline created successfully! View your pipeline: https://app.embedchain.ai/pipelines/xxxxx
+
+# 🛠️ Adding data to your pipeline...
+# ✅ Data of type: web_page, value: https://www.forbes.com/profile/elon-musk added successfully.
+```
@@ -0,0 +1,130 @@
+---
+title: "Pipeline"
+---
+
+Create a RAG pipeline object on Embedchain. This is the main entrypoint for a developer to interact with Embedchain APIs. A pipeline configures the llm, vector database, embedding model, and retrieval strategy of your choice.
+
+### Attributes
+
+<ParamField path="local_id" type="str">
+    Pipeline ID
+</ParamField>
+<ParamField path="name" type="str" optional>
+    Name of the pipeline
+</ParamField>
+<ParamField path="config" type="BaseConfig">
+    Configuration of the pipeline
+</ParamField>
+<ParamField path="llm" type="BaseLlm">
+    Configured LLM for the RAG pipeline
+</ParamField>
+<ParamField path="db" type="BaseVectorDB">
+    Configured vector database for the RAG pipeline
+</ParamField>
+<ParamField path="embedding_model" type="BaseEmbedder">
+    Configured embedding model for the RAG pipeline
+</ParamField>
+<ParamField path="chunker" type="ChunkerConfig">
+    Chunker configuration
+</ParamField>
+<ParamField path="client" type="Client" optional>
+    Client object (used to deploy a pipeline to Embedchain platform)
+</ParamField>
+<ParamField path="logger" type="logging.Logger">
+    Logger object
+</ParamField>
+
+## Usage
+
+You can create an embedchain pipeline instance using the following methods:
+
+### Default setting
+
+```python Code Example
+from embedchain import App
+app = App()
+```
+
+
+### Python Dict
+
+```python Code Example
+from embedchain import App
+
+config_dict = {
+  'llm': {
+    'provider': 'gpt4all',
+    'config': {
+      'model': 'orca-mini-3b-gguf2-q4_0.gguf',
+      'temperature': 0.5,
+      'max_tokens': 1000,
+      'top_p': 1,
+      'stream': False
+    }
+  },
+  'embedder': {
+    'provider': 'gpt4all'
+  }
+}
+
+# load llm configuration from config dict
+app = App.from_config(config=config_dict)
+```
+
+### YAML Config
+
+<CodeGroup>
+
+```python main.py
+from embedchain import App
+
+# load llm configuration from config.yaml file
+app = App.from_config(config_path="config.yaml")
+```
+
+```yaml config.yaml
+llm:
+  provider: gpt4all
+  config:
+    model: 'orca-mini-3b-gguf2-q4_0.gguf'
+    temperature: 0.5
+    max_tokens: 1000
+    top_p: 1
+    stream: false
+
+embedder:
+  provider: gpt4all
+```
+
+</CodeGroup>
+
+### JSON Config
+
+<CodeGroup>
+
+```python main.py
+from embedchain import App
+
+# load llm configuration from config.json file
+app = App.from_config(config_path="config.json")
+```
+
+```json config.json
+{
+  "llm": {
+    "provider": "gpt4all",
+    "config": {
+      "model": "orca-mini-3b-gguf2-q4_0.gguf",
+      "temperature": 0.5,
+      "max_tokens": 1000,
+      "top_p": 1,
+      "stream": false
+    }
+  },
+  "embedder": {
+    "provider": "gpt4all"
+  }
+}
+```
+
+</CodeGroup>
@@ -0,0 +1,109 @@
+---
+title: '❓ query'
+---
+
+`.query()` method empowers developers to ask questions and receive relevant answers through a user-friendly query API. Function signature is given below:
+
+### Parameters
+
+<ParamField path="input_query" type="str">
+    Question to ask
+</ParamField>
+<ParamField path="config" type="BaseLlmConfig" optional>
+    Configure different llm settings such as prompt, temprature, number_documents etc.
+</ParamField>
+<ParamField path="dry_run" type="bool" optional>
+    The purpose is to test the prompt structure without actually running LLM inference. Defaults to `False`
+</ParamField>
+<ParamField path="where" type="dict" optional>
+    A dictionary of key-value pairs to filter the chunks from the vector database. Defaults to `None`
+</ParamField>
+<ParamField path="citations" type="bool" optional>
+    Return citations along with the LLM answer. Defaults to `False`
+</ParamField>
+
+### Returns
+
+<ResponseField name="answer" type="str | tuple">
+  If `citations=False`, return a stringified answer to the question asked. <br />
+  If `citations=True`, returns a tuple with answer and citations respectively.
+</ResponseField>
+
+## Usage
+
+### With citations
+
+If you want to get the answer to question and return both answer and citations, use the following code snippet:
+
+```python With Citations
+from embedchain import App
+
+# Initialize app
+app = App()
+
+# Add data source
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Get relevant answer for your query
+answer, sources = app.query("What is the net worth of Elon?", citations=True)
+print(answer)
+# Answer: The net worth of Elon Musk is $221.9 billion.
+
+print(sources)
+# [
+#    (
+#        'Elon Musk PROFILEElon MuskCEO, Tesla$247.1B$2.3B (0.96%)Real Time Net Worthas of 12/7/23 ...',
+#        {
+#           'url': 'https://www.forbes.com/profile/elon-musk', 
+#           'score': 0.89,
+#           ...
+#        }
+#    ),
+#    (
+#        '74% of the company, which is now called X.Wealth HistoryHOVER TO REVEAL NET WORTH BY YEARForbes ...',
+#        {
+#           'url': 'https://www.forbes.com/profile/elon-musk', 
+#           'score': 0.81,
+#           ...
+#        }
+#    ),
+#    (
+#        'founded in 2002, is worth nearly $150 billion after a $750 million tender offer in June 2023 ...',
+#        {
+#           'url': 'https://www.forbes.com/profile/elon-musk', 
+#           'score': 0.73,
+#           ...
+#        }
+#    )
+# ]
+```
+
+<Note>
+When `citations=True`, note that the returned `sources` are a list of tuples where each tuple has two elements (in the following order):
+1. source chunk
+2. dictionary with metadata about the source chunk
+    - `url`: url of the source
+    - `doc_id`: document id (used for book keeping purposes)
+    - `score`: score of the source chunk with respect to the question
+    - other metadata you might have added at the time of adding the source
+</Note>
+
+### Without citations
+
+If you just want to return answers and don't want to return citations, you can use the following example:
+
+```python Without Citations
+from embedchain import App
+
+# Initialize app
+app = App()
+
+# Add data source
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Get relevant answer for your query
+answer = app.query("What is the net worth of Elon?")
+print(answer)
+# Answer: The net worth of Elon Musk is $221.9 billion.
+```
+
@@ -0,0 +1,17 @@
+---
+title: 🔄 reset
+---
+
+`reset()` method allows you to wipe the data from your RAG application and start from scratch.
+
+## Usage
+
+```python
+from embedchain import App
+
+app = App()
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Reset the app
+app.reset()
+```
@@ -0,0 +1,57 @@
+---
+title: '🔍 search'
+---
+
+`.search()` enables you to uncover the most pertinent context by performing a semantic search across your data sources based on a given query. Refer to the function signature below:
+
+### Parameters
+
+<ParamField path="query" type="str">
+    Question
+</ParamField>
+<ParamField path="num_documents" type="int" optional>
+    Number of relevant documents to fetch. Defaults to `3`
+</ParamField>
+
+### Returns
+
+<ResponseField name="answer" type="dict">
+    Return list of dictionaries that contain the relevant chunk and their source information.
+</ResponseField>
+
+## Usage
+
+Refer to the following example on how to use the search api:
+
+```python Code example
+from embedchain import App
+
+# Initialize app
+app = App()
+
+# Add data source
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Get relevant context using semantic search
+context = app.search("What is the net worth of Elon?", num_documents=2)
+print(context)
+# Context:
+# [
+#     {
+#         'context': 'Elon Musk PROFILEElon MuskCEO, Tesla$221.9BReal Time Net Worth ...',
+#         'metadata': {
+#             'source': 'https://www.forbes.com/profile/elon-musk',
+#             'document_id': 'some_document_id',
+#             'score': 0.404,
+#         }
+#     },
+#     {
+#         'context': 'company, which is now called X.Wealth HistoryHOVER TO REVEAL NET WORTH ...',
+#         'metadata': {
+#             'source': 'https://www.forbes.com/profile/elon-musk',
+#             'document_id': 'some_document_id',
+#             'score': 0.435,
+#         }
+#     }
+# ]
+```
@@ -0,0 +1,54 @@
+---
+title: 'AI Assistant'
+---
+
+The `AIAssistant` class, an alternative to the OpenAI Assistant API, is designed for those who prefer using large language models (LLMs) other than those provided by OpenAI. It facilitates the creation of AI Assistants with several key benefits:
+
+- **Visibility into Citations**: It offers transparent access to the sources and citations used by the AI, enhancing the understanding and trustworthiness of its responses.
+
+- **Debugging Capabilities**: Users have the ability to delve into and debug the AI's processes, allowing for a deeper understanding and fine-tuning of its performance.
+
+- **Customizable Prompts**: The class provides the flexibility to modify and tailor prompts according to specific needs, enabling more precise and relevant interactions.
+
+- **Chain of Thought Integration**: It supports the incorporation of a 'chain of thought' approach, which helps in breaking down complex queries into simpler, sequential steps, thereby improving the clarity and accuracy of responses.
+
+It is ideal for those who value customization, transparency, and detailed control over their AI Assistant's functionalities.
+
+### Arguments
+
+<ParamField path="name" type="string" optional>
+  Name for your AI assistant
+</ParamField>
+
+<ParamField path="instructions" type="string" optional>
+  How the Assistant and model should behave or respond
+</ParamField>
+
+<ParamField path="assistant_id" type="string" optional>
+  Load existing AI Assistant. If you pass this, you don't have to pass other arguments.
+</ParamField>
+
+<ParamField path="thread_id" type="string" optional>
+  Existing thread id if exists
+</ParamField>
+
+<ParamField path="yaml_path" type="str" Optional>
+    Embedchain pipeline config yaml path to use. This will define the configuration of the AI Assistant (such as configuring the LLM, vector database, and embedding model)
+</ParamField>
+
+<ParamField path="data_sources" type="list" default="[]">
+  Add data sources to your assistant. You can add in the following format: `[{"source": "https://example.com", "data_type": "web_page"}]`
+</ParamField>
+
+<ParamField path="collect_metrics" type="boolean" default="True">
+  Anonymous telemetry (doesn't collect any user information or user's files). Used to improve the Embedchain package utilization. Default is `True`.
+</ParamField>
+
+
+## Usage
+
+For detailed guidance on creating your own AI Assistant, click the link below. It provides step-by-step instructions to help you through the process:
+
+<Card title="Guide to Creating Your AI Assistant" icon="link" href="/examples/opensource-assistant">
+  Learn how to build a customized AI Assistant using the `AIAssistant` class.
+</Card>
@@ -0,0 +1,45 @@
+---
+title: 'OpenAI Assistant'
+---
+
+### Arguments
+
+<ParamField path="name" type="string">
+  Name for your AI assistant
+</ParamField>
+
+<ParamField path="instructions" type="string">
+  how the Assistant and model should behave or respond
+</ParamField>
+
+<ParamField path="assistant_id" type="string">
+  Load existing OpenAI Assistant. If you pass this, you don't have to pass other arguments.
+</ParamField>
+
+<ParamField path="thread_id" type="string">
+  Existing OpenAI thread id if exists
+</ParamField>
+
+<ParamField path="model" type="str" default="gpt-4-1106-preview">
+  OpenAI model to use
+</ParamField>
+
+<ParamField path="tools" type="list">
+  OpenAI tools to use. Default set to `[{"type": "retrieval"}]`
+</ParamField>
+
+<ParamField path="data_sources" type="list" default="[]">
+  Add data sources to your assistant. You can add in the following format: `[{"source": "https://example.com", "data_type": "web_page"}]`
+</ParamField>
+
+<ParamField path="telemetry" type="boolean" default="True">
+  Anonymous telemetry (doesn't collect any user information or user's files). Used to improve the Embedchain package utilization. Default is `True`.
+</ParamField>
+
+## Usage
+
+For detailed guidance on creating your own OpenAI Assistant, click the link below. It provides step-by-step instructions to help you through the process:
+
+<Card title="Guide to Creating Your OpenAI Assistant" icon="link" href="/examples/openai-assistant">
+  Learn how to build an OpenAI Assistant using the `OpenAIAssistant` class.
+</Card>
@@ -0,0 +1,16 @@
+---
+title: "🐝 Beehiiv"
+---
+
+To add any Beehiiv data sources to your app, just add the base url as the source and set the data_type to `beehiiv`.
+
+```python
+from embedchain import App
+
+app = App()
+
+# source: just add the base url and set the data_type to 'beehiiv'
+app.add('https://aibreakfast.beehiiv.com', data_type='beehiiv')
+app.query("How much is OpenAI paying developers?")
+# Answer: OpenAI is aggressively recruiting Google's top AI researchers with offers ranging between $5 to $10 million annually, primarily in stock options.
+```
@@ -5,7 +5,7 @@ title: '📊 CSV'
 To add any csv file, use the data_type as `csv`. `csv` allows remote urls and conventional file paths. Headers are included for each line, so if you have an `age` column, `18` will be added as `age: 18`. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()
 app.add('https://people.sc.fsu.edu/~jburkardt/data/csv/airtravel.csv', data_type="csv")
@@ -0,0 +1,41 @@
+---
+title: '⚙️ Custom'
+---
+
+When we say "custom", we mean that you can customize the loader and chunker to your needs. This is done by passing a custom loader and chunker to the `add` method.
+
+```python
+from embedchain import App
+import your_loader
+import your_chunker
+
+app = App()
+loader = your_loader()
+chunker = your_chunker()
+
+app.add("source", data_type="custom", loader=loader, chunker=chunker)
+```
+
+<Note>
+    The custom loader and chunker must be a class that inherits from the [`BaseLoader`](https://github.com/embedchain/embedchain/blob/main/embedchain/loaders/base_loader.py) and [`BaseChunker`](https://github.com/embedchain/embedchain/blob/main/embedchain/chunkers/base_chunker.py) classes respectively.
+</Note>
+
+<Note>
+    If the `data_type` is not a valid data type, the `add` method will fallback to the `custom` data type and expect a custom loader and chunker to be passed by the user.
+</Note>
+
+Example:
+
+```python
+from embedchain import App
+from embedchain.loaders.github import GithubLoader
+
+app = App()
+
+loader = GithubLoader(config={"token": "ghp_xxx"})
+
+app.add("repo:embedchain/embedchain type:repo", data_type="github", loader=loader)
+
+app.query("What is Embedchain?")
+# Answer: Embedchain is a Data Platform for Large Language Models (LLMs). It allows users to seamlessly load, index, retrieve, and sync unstructured data in order to build dynamic, LLM-powered applications. There is also a JavaScript implementation called embedchain-js available on GitHub.
+```
@@ -35,7 +35,7 @@ Default behavior is to create a persistent vector db in the directory **./db**.
 Create a local index:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 naval_chat_bot = App()
 naval_chat_bot.add("https://www.youtube.com/watch?v=3qHkcs3kG44")
@@ -45,8 +45,20 @@ naval_chat_bot.add("https://navalmanack.s3.amazonaws.com/Eric-Jorgenson_The-Alma
 You can reuse the local index with the same code, but without adding new documents:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 naval_chat_bot = App()
 print(naval_chat_bot.query("What unique capacity does Naval argue humans possess when it comes to understanding explanations or concepts?"))
 ```
+
+## Resetting an app and vector database
+
+You can reset the app by simply calling the `reset` method. This will delete the vector database and all other app related files.
+
+```python
+from embedchain import App
+
+app = App()
+app.add("https://www.youtube.com/watch?v=3qHkcs3kG44")
+app.reset()
+```
@@ -0,0 +1,41 @@
+---
+title: '📁 Directory/Folder'
+---
+
+To use an entire directory as data source, just add `data_type` as `directory` and pass in the path of the local directory.
+
+### Without customization
+
+```python
+import os
+from embedchain import App
+
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+
+app = App()
+app.add("./elon-musk", data_type="directory")
+response = app.query("list all files")
+print(response)
+# Answer: Files are elon-musk-1.txt, elon-musk-2.pdf.
+```
+
+### Customization
+
+```python
+import os
+from embedchain import App
+from embedchain.loaders.directory_loader import DirectoryLoader
+
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+lconfig = {
+    "recursive": True,
+    "extensions": [".txt"]
+}
+loader = DirectoryLoader(config=lconfig)
+app = App()
+app.add("./elon-musk", loader=loader)
+response = app.query("what are all the files related to?")
+print(response)
+
+# Answer: The files are related to Elon Musk.
+```
@@ -0,0 +1,28 @@
+---
+title: "💬 Discord"
+---
+
+To add any Discord channel messages to your app, just add the `channel_id` as the source and set the `data_type` to `discord`.
+
+<Note>
+    This loader requires a Discord bot token with read messages access.
+    To obtain the token, follow the instructions provided in this tutorial: 
+    <a href="https://www.writebots.com/discord-bot-token/">How to Get a Discord Bot Token?</a>.
+</Note>
+
+```python
+import os
+from embedchain import App
+
+# add your discord "BOT" token
+os.environ["DISCORD_TOKEN"] = "xxx"
+
+app = App()
+
+app.add("1177296711023075338", data_type="discord")
+
+response = app.query("What is Joe saying about Elon Musk?")
+
+print(response)
+# Answer: Joe is saying "Elon Musk is a genius".
+```
@@ -0,0 +1,44 @@
+---
+title: '🗨️ Discourse'
+---
+
+You can now easily load data from your community built with [Discourse](https://discourse.org/).
+
+## Example
+
+1. Setup the Discourse Loader with your community url.
+```Python
+from embedchain.loaders.discourse import DiscourseLoader
+
+dicourse_loader = DiscourseLoader(config={"domain": "https://community.openai.com"})
+```
+
+2. Once you setup the loader, you can create an app and load data using the above discourse loader
+```Python
+import os
+from embedchain.pipeline import Pipeline as App
+
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+
+app = App()
+
+app.add("openai after:2023-10-1", data_type="discourse", loader=dicourse_loader)
+
+question = "Where can I find the OpenAI API status page?"
+app.query(question)
+# Answer: You can find the OpenAI API status page at https:/status.openai.com/.
+```
+
+NOTE: The `add` function of the app will accept any executable search query to load data. Refer [Discourse API Docs](https://docs.discourse.org/#tag/Search) to learn more about search queries.
+
+3. We automatically create a chunker to chunk your discourse data, however if you wish to provide your own chunker class. Here is how you can do that:
+```Python
+
+from embedchain.chunkers.discourse import DiscourseChunker
+from embedchain.config.add_config import ChunkerConfig
+
+discourse_chunker_config = ChunkerConfig(chunk_size=1000, chunk_overlap=0, length_function=len)
+discourse_chunker = DiscourseChunker(config=discourse_chunker_config)
+
+app.add("openai", data_type='discourse', loader=dicourse_loader, chunker=discourse_chunker)
+```
@@ -1,14 +1,14 @@
 ---
-title: '📚🌐 Code documentation'
+title: '📚 Code Docs website'
 ---

 To add any code documentation website as a loader, use the data_type as `docs_site`. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()
 app.add("https://docs.embedchain.ai/", data_type="docs_site")
 app.query("What is Embedchain?")
-# Answer: Embedchain is a platform that utilizes various components, including paid/proprietary ones, to provide what is believed to be the best configuration available. It uses LLM (Language Model) providers such as OpenAI, Anthpropic, Vertex_AI, GPT4ALL, Azure_OpenAI, LLAMA2, JINA, and COHERE. Embedchain allows users to import and utilize these LLM providers for their applications.'
+# Answer: Embedchain is a platform that utilizes various components, including paid/proprietary ones, to provide what is believed to be the best configuration available. It uses LLM (Language Model) providers such as OpenAI, Anthpropic, Vertex_AI, GPT4ALL, Azure_OpenAI, LLAMA2, JINA, Ollama, Together and COHERE. Embedchain allows users to import and utilize these LLM providers for their applications.'
 ```
@@ -7,7 +7,7 @@ title: '📄 Docx file'
 To add any doc/docx file, use the data_type as `docx`. `docx` allows remote urls and conventional file paths. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()
 app.add('https://example.com/content/intro.docx', data_type="docx")
@@ -0,0 +1,27 @@
+---
+title: '💾 Dropbox'
+---
+
+To load folders or files from your Dropbox account, configure the `data_type` parameter as `dropbox` and specify the path to the desired file or folder, starting from the root directory of your Dropbox account.
+
+For Dropbox access, an **access token** is required. Obtain this token by visiting [Dropbox Developer Apps](https://www.dropbox.com/developers/apps). There, create a new app and generate an access token for it.
+
+Ensure your app has the following settings activated:
+
+- In the Permissions section, enable `files.content.read` and `files.metadata.read`.
+
+```python
+import os
+from embedchain import Pipeline as App
+
+os.environ["DROPBOX_ACCESS_TOKEN"] = "sl.xxx"
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+
+app = App()
+
+# any path from the root of your dropbox account, you can leave it "" for the root folder
+app.add("/test", data_type="dropbox")
+
+print(app.query("Which two celebrities are mentioned here?"))
+# The two celebrities mentioned in the given context are Elon Musk and Jeff Bezos.
+```
@@ -0,0 +1,50 @@
+---
+title: 📝 Github
+---
+
+1. Setup the Github loader by configuring the Github account with username and personal access token (PAT). Check out [this](https://docs.github.com/en/enterprise-server@3.6/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token) link to learn how to create a PAT.
+```Python
+from embedchain.loaders.github import GithubLoader
+
+loader = GithubLoader(
+    config={
+        "token":"ghp_xxxx"
+        }
+    )
+```
+
+2. Once you setup the loader, you can create an app and load data using the above Github loader
+```Python
+import os
+from embedchain.pipeline import Pipeline as App
+
+os.environ["OPENAI_API_KEY"] = "sk-xxxx"
+
+app = App()
+
+app.add("repo:embedchain/embedchain type:repo", data_type="github", loader=loader)
+
+response = app.query("What is Embedchain?")
+# Answer: Embedchain is a Data Platform for Large Language Models (LLMs). It allows users to seamlessly load, index, retrieve, and sync unstructured data in order to build dynamic, LLM-powered applications. There is also a JavaScript implementation called embedchain-js available on GitHub.
+```
+The `add` function of the app will accept any valid github query with qualifiers. It only supports loading github code, repository, issues and pull-requests.
+<Note>
+You must provide qualifiers `type:` and `repo:` in the query. The `type:` qualifier can be a combination of `code`, `repo`, `pr`, `issue`. The `repo:` qualifier must be a valid github repository name.
+</Note>
+
+<Card title="Valid queries" icon="lightbulb" iconType="duotone" color="#ca8b04">
+    - `repo:embedchain/embedchain type:repo` - to load the repository
+    - `repo:embedchain/embedchain type:issue,pr` - to load the issues and pull-requests of the repository
+    - `repo:embedchain/embedchain type:issue state:closed` - to load the closed issues of the repository
+</Card>
+
+3. We automatically create a chunker to chunk your GitHub data, however if you wish to provide your own chunker class. Here is how you can do that:
+```Python
+from embedchain.chunkers.common_chunker import CommonChunker
+from embedchain.config.add_config import ChunkerConfig
+
+github_chunker_config = ChunkerConfig(chunk_size=2000, chunk_overlap=0, length_function=len)
+github_chunker = CommonChunker(config=github_chunker_config)
+
+app.add(load_query, data_type="github", loader=loader, chunker=github_chunker)
+```
@@ -24,12 +24,11 @@ To use this you need to save `credentials.json` in the directory from where you
 12. Put the `.json` file in your current directory and rename it to `credentials.json`

 ```python
-import os
-from embedchain.apps.app import App
-from embedchain.models.data_type import DataType
+from embedchain import App
+
 app = App()

-query = "to: me label:inbox"
-app.add(query, data_type=DataType.GMAIL)
+gmail_filter = "to: me label:inbox"
+app.add(gmail_filter, data_type="gmail")
 app.query("Summarize my email conversations")
 ```
@@ -0,0 +1,45 @@
+---
+title: "🖼️ Image"
+---
+
+
+To use an image as data source, just add `data_type` as `image` and pass in the path of the image (local or hosted).
+
+We use [GPT4 Vision](https://platform.openai.com/docs/guides/vision) to generate meaning of the image using a custom prompt, and then use the generated text as the data source.
+
+You would require an OpenAI API key with access to `gpt-4-vision-preview` model to use this feature.
+
+### Without customization
+
+```python
+import os
+from embedchain import App
+
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+
+app = App()
+app.add("./Elon-Musk.webp", data_type="image")
+response = app.query("Describe the man in the image.")
+print(response)
+# Answer: The man in the image is dressed in formal attire, wearing a dark suit jacket and a white collared shirt. He has short hair and is standing. He appears to be gazing off to the side with a reflective expression. The background is dark with faint, warm-toned vertical lines, possibly from a lit environment behind the individual or reflections. The overall atmosphere is somewhat moody and introspective.
+```
+
+### Customization
+
+```python
+import os
+from embedchain import App
+from embedchain.loaders.image import ImageLoader
+
+image_loader = ImageLoader(
+    max_tokens=100,
+    api_key="sk-xxx",
+    prompt="Is the person looking wealthy? Structure your thoughts around what you see in the image.",
+)
+
+app = App()
+app.add("./Elon-Musk.webp", data_type="image", loader=image_loader)
+response = app.query("Describe the man in the image.")
+print(response)
+# Answer: The man in the image appears to be well-dressed in a suit and shirt, suggesting that he may be in a professional or formal setting. His composed demeanor and confident posture further indicate a sense of self-assurance. Based on these visual cues, one could infer that the man may have a certain level of economic or social status, possibly indicating wealth or professional success.
+```
@@ -0,0 +1,44 @@
+---
+title: '📃 JSON'
+---
+
+To add any json file, use the data_type as `json`. Headers are included for each line, so for example if you have a json like `{"age": 18}`, then it will be added as `age: 18`.
+
+Here are the supported sources for loading `json`:
+
+```
+1. URL - valid url to json file that ends with ".json" extension.
+2. Local file - valid url to local json file that ends with ".json" extension.
+3. String - valid json string (e.g. - app.add('{"foo": "bar"}'))
+```
+
+<Tip>
+If you would like to add other data structures (e.g. list, dict etc.), convert it to a valid json first using `json.dumps()` function.
+</Tip>
+
+## Example
+
+<CodeGroup>
+
+```python python
+from embedchain import App
+
+app = App()
+
+# Add json file
+app.add("temp.json")
+
+app.query("What is the net worth of Elon Musk as of October 2023?")
+# As of October 2023, Elon Musk's net worth is $255.2 billion.
+```
+
+
+```json temp.json
+{
+    "question": "What is your net worth, Elon Musk?",
+    "answer": "As of October 2023, Elon Musk's net worth is $255.2 billion, making him one of the wealthiest individuals in the world."
+}
+```
+</CodeGroup>
+
+
@@ -5,7 +5,7 @@ title: '📝 Mdx file'
 To add any `.mdx` file to your app, use the data_type (first argument to `.add()` method) as `mdx`. Note that this supports support mdx file present on machine, so this should be a file path. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()
 app.add('path/to/file.mdx', data_type='mdx')
@@ -0,0 +1,47 @@
+---
+title: '🐬 MySQL'
+---
+
+1. Setup the MySQL loader by configuring the SQL db.
+```Python
+from embedchain.loaders.mysql import MySQLLoader
+
+config = {
+    "host": "host",
+    "port": "port",
+    "database": "database",
+    "user": "username",
+    "password": "password",
+}
+
+mysql_loader = MySQLLoader(config=config)
+```
+
+For more details on how to setup with valid config, check MySQL [documentation](https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html).
+
+2. Once you setup the loader, you can create an app and load data using the above MySQL loader
+```Python
+from embedchain.pipeline import Pipeline as App
+
+app = App()
+
+app.add("SELECT * FROM table_name;", data_type='mysql', loader=mysql_loader)
+# Adds `(1, 'What is your net worth, Elon Musk?', "As of October 2023, Elon Musk's net worth is $255.2 billion.")`
+
+response = app.query(question)
+# Answer: As of October 2023, Elon Musk's net worth is $255.2 billion.
+```
+
+NOTE: The `add` function of the app will accept any executable query to load data. DO NOT pass the `CREATE`, `INSERT` queries in `add` function.
+
+3. We automatically create a chunker to chunk your SQL data, however if you wish to provide your own chunker class. Here is how you can do that:
+``Python
+
+from embedchain.chunkers.mysql import MySQLChunker
+from embedchain.config.add_config import ChunkerConfig
+
+mysql_chunker_config = ChunkerConfig(chunk_size=1000, chunk_overlap=0, length_function=len)
+mysql_chunker = MySQLChunker(config=mysql_chunker_config)
+
+app.add("SELECT * FROM table_name;", data_type='mysql', loader=mysql_loader, chunker=mysql_chunker)
+```
@@ -8,7 +8,7 @@ To load a notion page, use the data_type as `notion`. Since it is hard to automa
 The next argument must **end** with the `notion page id`. The id is a 32-character string. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()

@@ -2,13 +2,10 @@
 title: 🙌 OpenAPI
 ---

-To add any OpenAPI spec yaml file (currently the json file will be detected as JSON data type), use the data_type as 'openapi'. 'openapi' allows remote urls and conventional file paths. Headers are included for each line, so if you have an `age` column, `18` will be added as `age: 18`. Eg:
+To add any OpenAPI spec yaml file (currently the json file will be detected as JSON data type), use the data_type as 'openapi'. 'openapi' allows remote urls and conventional file paths.

 ```python
-from embedchain.apps.app import App
-import os
-
-os.environ["OPENAI_API_KEY"] = "sk-xxx"
+from embedchain import App

 app = App()

@@ -16,8 +13,10 @@ app.add("https://github.com/openai/openai-openapi/blob/master/openapi.yaml", dat
 # Or add using the local file path
 # app.add("configs/openai_openapi.yaml", data_type="openapi")

-response = app.query("What can OpenAI API endpoint do? Can you list the things it can learn from?")
+app.query("What can OpenAI API endpoint do? Can you list the things it can learn from?")
 # Answer: The OpenAI API endpoint allows users to interact with OpenAI's models and perform various tasks such as generating text, answering questions, summarizing documents, translating languages, and more. The specific capabilities and tasks that the API can learn from may vary depending on the models and features provided by OpenAI. For more detailed information, it is recommended to refer to the OpenAI API documentation at https://platform.openai.com/docs/api-reference.
 ```

-NOTE: The yaml file added to the App must have the required OpenAPI fields otherwise the adding OpenAPI spec will fail. Please refer to [OpenAPI Spec Doc](https://spec.openapis.org/oas/v3.1.0)
+<Note>
+The yaml file added to the App must have the required OpenAPI fields otherwise the adding OpenAPI spec will fail. Please refer to [OpenAPI Spec Doc](https://spec.openapis.org/oas/v3.1.0)
+</Note>
@@ -0,0 +1,40 @@
+---
+title: Overview
+---
+
+Embedchain comes with built-in support for various data sources. We handle the complexity of loading unstructured data from these data sources, allowing you to easily customize your app through a user-friendly interface.
+
+<CardGroup cols={4}>
+  <Card title="📰 PDF file" href="/components/data-sources/pdf-file"></Card>
+  <Card title="📊 CSV file" href="/components/data-sources/csv"></Card>
+  <Card title="📃 JSON file" href="/components/data-sources/json"></Card>
+  <Card title="📝 Text" href="/components/data-sources/text"></Card>
+  <Card title="📁 Directory/ Folder" href="/components/data-sources/directory"></Card>
+  <Card title="🌐 HTML Web page" href="/components/data-sources/web-page"></Card>
+  <Card title="📽️ Youtube Channel" href="/components/data-sources/youtube-channel"></Card>
+  <Card title="📺 Youtube Video" href="/components/data-sources/youtube-video"></Card>
+  <Card title="📚 Docs website" href="/components/data-sources/docs-site"></Card>
+  <Card title="📝 MDX file" href="/components/data-sources/mdx"></Card>
+  <Card title="📄 DOCX file" href="/components/data-sources/docx"></Card>
+  <Card title="📓 Notion" href="/components/data-sources/notion"></Card>
+  <Card title="🗺️ Sitemap" href="/components/data-sources/sitemap"></Card>
+  <Card title="🧾 XML file" href="/components/data-sources/xml"></Card>
+  <Card title="❓💬 Q&A pair" href="/components/data-sources/qna"></Card>
+  <Card title="🙌 OpenAPI" href="/components/data-sources/openapi"></Card>
+  <Card title="📬 Gmail" href="/components/data-sources/gmail"></Card>
+  <Card title="📝 Github" href="/components/data-sources/github"></Card>
+  <Card title="🐘 Postgres" href="/components/data-sources/postgres"></Card>
+  <Card title="🐬 MySQL" href="/components/data-sources/mysql"></Card>
+  <Card title="🤖 Slack" href="/components/data-sources/slack"></Card>
+  <Card title="💬 Discord" href="/components/data-sources/discord"></Card>
+  <Card title="🗨️ Discourse" href="/components/data-sources/discourse"></Card>
+  <Card title="📝 Substack" href="/components/data-sources/substack"></Card>
+  <Card title="🐝 Beehiiv" href="/components/data-sources/beehiiv"></Card>
+  <Card title="💾 Dropbox" href="/components/data-sources/dropbox"></Card>
+  <Card title="🖼️ Image" href="/components/data-sources/image"></Card>
+  <Card title="⚙️ Custom" href="/components/data-sources/custom"></Card>
+</CardGroup>
+
+<br/ >
+
+<Snippet file="missing-data-source-tip.mdx" />
@@ -0,0 +1,47 @@
+---
+title: '📰 PDF file'
+---
+
+To add any pdf file, use the data_type as `pdf_file`. Eg:
+
+```python
+from embedchain import App
+
+app = App()
+
+app.add('https://arxiv.org/pdf/1706.03762.pdf', data_type='pdf_file')
+app.query("What is the paper 'attention is all you need' about?", citations=True)
+# Answer: The paper "Attention Is All You Need" proposes a new network architecture called the Transformer, which is based solely on attention mechanisms. It suggests that complex recurrent or convolutional neural networks can be replaced with a simpler architecture that connects the encoder and decoder through attention. The paper discusses how this approach can improve sequence transduction models, such as neural machine translation.
+# Contexts: 
+# [
+#     (
+#         'Provided proper attribution is ...',
+#         {
+#             'page': 0,
+#             'url': 'https://arxiv.org/pdf/1706.03762.pdf',
+#             'score': 0.3676220203221626,
+#             ...
+#         }
+#     ),
+#     (
+#         'Attention Visualizations Input ...',
+#         {
+#             'page': 12,
+#             'url': 'https://arxiv.org/pdf/1706.03762.pdf',
+#             'score': 0.41679039679873736,
+#             ...
+#         }
+#     ),
+#     (
+#         'sequence learning ...',
+#         {
+#             'page': 10,
+#             'url': 'https://arxiv.org/pdf/1706.03762.pdf',
+#             'score': 0.4188303600897153,
+#             ...
+#         }
+#     )
+# ]
+```
+
+Note that we do not support password protected pdfs.
@@ -0,0 +1,64 @@
+---
+title: '🐘 Postgres'
+---
+
+1. Setup the Postgres loader by configuring the postgres db.
+```Python
+from embedchain.loaders.postgres import PostgresLoader
+
+config = {
+    "host": "host_address",
+    "port": "port_number",
+    "dbname": "database_name",
+    "user": "username",
+    "password": "password",
+}
+
+"""
+config = {
+    "url": "your_postgres_url"
+}
+"""
+
+postgres_loader = PostgresLoader(config=config)
+
+```
+
+You can either setup the loader by passing the postgresql url or by providing the config data.
+For more details on how to setup with valid url and config, check postgres [documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING:~:text=34.1.1.%C2%A0Connection%20Strings-,%23,-Several%20libpq%20functions).
+
+NOTE: if you provide the `url` field in config, all other fields will be ignored.
+
+2. Once you setup the loader, you can create an app and load data using the above postgres loader
+```Python
+import os
+from embedchain.pipeline import Pipeline as App
+
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+
+app = App()
+
+question = "What is Elon Musk's networth?"
+response = app.query(question)
+# Answer: As of September 2021, Elon Musk's net worth is estimated to be around $250 billion, making him one of the wealthiest individuals in the world. However, please note that net worth can fluctuate over time due to various factors such as stock market changes and business ventures.
+
+app.add("SELECT * FROM table_name;", data_type='postgres', loader=postgres_loader)
+# Adds `(1, 'What is your net worth, Elon Musk?', "As of October 2023, Elon Musk's net worth is $255.2 billion.")`
+
+response = app.query(question)
+# Answer: As of October 2023, Elon Musk's net worth is $255.2 billion.
+```
+
+NOTE: The `add` function of the app will accept any executable query to load data. DO NOT pass the `CREATE`, `INSERT` queries in `add` function as they will result in not adding any data, so it is pointless.
+
+3. We automatically create a chunker to chunk your postgres data, however if you wish to provide your own chunker class. Here is how you can do that:
+```Python
+
+from embedchain.chunkers.postgres import PostgresChunker
+from embedchain.config.add_config import ChunkerConfig
+
+postgres_chunker_config = ChunkerConfig(chunk_size=1000, chunk_overlap=0, length_function=len)
+postgres_chunker = PostgresChunker(config=postgres_chunker_config)
+
+app.add("SELECT * FROM table_name;", data_type='postgres', loader=postgres_loader, chunker=postgres_chunker)
+```
@@ -5,7 +5,7 @@ title: '❓💬 Queston and answer pair'
 QnA pair is a local data type. To supply your own QnA pair, use the data_type as `qna_pair` and enter a tuple. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()

@@ -5,7 +5,7 @@ title: '🗺️ Sitemap'
 Add all web pages from an xml-sitemap. Filters non-text files. Use the data_type as `sitemap`. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()

@@ -0,0 +1,71 @@
+---
+title: '🤖 Slack'
+---
+
+## Pre-requisite
+- Download required packages by running `pip install --upgrade "embedchain[slack]"`.
+- Configure your slack bot token as environment variable `SLACK_USER_TOKEN`.
+    - Find your user token on your [Slack Account](https://api.slack.com/authentication/token-types)
+    - Make sure your slack user token includes [search](https://api.slack.com/scopes/search:read) scope.
+
+## Example
+
+### Get Started
+
+This will automatically retrieve data from the workspace associated with the user's token.
+
+```python
+import os
+from embedchain import App
+
+os.environ["SLACK_USER_TOKEN"] = "xoxp-xxx"
+app = App()
+
+app.add("in:general", data_type="slack")
+
+result = app.query("what are the messages in general channel?")
+
+print(result)
+```
+
+
+### Customize your SlackLoader
+1. Setup the Slack loader by configuring the Slack Webclient.
+```Python
+from embedchain.loaders.slack import SlackLoader
+
+os.environ["SLACK_USER_TOKEN"] = "xoxp-*"
+
+config = {
+    'base_url': slack_app_url,
+    'headers': web_headers,
+    'team_id': slack_team_id,
+}
+
+loader = SlackLoader(config)
+```
+
+NOTE: you can also pass the `config` with `base_url`, `headers`, `team_id` to setup your SlackLoader.
+
+2. Once you setup the loader, you can create an app and load data using the above slack loader
+```Python
+import os
+from embedchain.pipeline import Pipeline as App
+
+app = App()
+
+app.add("in:random", data_type="slack", loader=loader)
+question = "Which bots are available in the slack workspace's random channel?"
+# Answer: The available bot in the slack workspace's random channel is the Embedchain bot.
+```
+
+3. We automatically create a chunker to chunk your slack data, however if you wish to provide your own chunker class. Here is how you can do that:
+```Python
+from embedchain.chunkers.slack import SlackChunker
+from embedchain.config.add_config import ChunkerConfig
+
+slack_chunker_config = ChunkerConfig(chunk_size=1000, chunk_overlap=0, length_function=len)
+slack_chunker = SlackChunker(config=slack_chunker_config)
+
+app.add(slack_chunker, data_type="slack", loader=loader, chunker=slack_chunker)
+```
@@ -0,0 +1,16 @@
+---
+title: "📝 Substack"
+---
+
+To add any Substack data sources to your app, just add the main base url as the source and set the data_type to `substack`.
+
+```python
+from embedchain import App
+
+app = App()
+
+# source: for any substack just add the root URL
+app.add('https://www.lennysnewsletter.com', data_type='substack')
+app.query("Who is Brian Chesky?")
+# Answer: Brian Chesky is the co-founder and CEO of Airbnb.
+```
@@ -7,7 +7,7 @@ title: '📝 Text'
 Text is a local data type. To supply your own text, use the data_type as `text` and enter a string. The text is not processed, this can be very versatile. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()

@@ -1,11 +1,11 @@
 ---
-title: '🌐📄 Web page'
+title: '🌐 HTML Web page'
 ---

 To add any web page, use the data_type as `web_page`. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()

@@ -7,7 +7,7 @@ title: '🧾 XML file'
 To add any xml file, use the data_type as `xml`. Eg:

 ```python
-from embedchain import Pipeline as App
+from embedchain import App

 app = App()

@@ -0,0 +1,20 @@
+---
+title: '📽️ Youtube Channel'
+---
+
+To add all the videos from a youtube channel to your app, use the data_type as `youtube_channel`.
+
+<Note>
+Make sure you have all the required packages installed before using this data type. You can install them by running the following command in your terminal.
+
+```bash
+pip install -u "embedchain[youtube]"
+```
+</Note>
+
+```python
+from embedchain import App
+
+app = App()
+app.add("@channel_name", data_type="youtube_channel")
+```
@@ -0,0 +1,12 @@
+---
+title: '📺 Youtube Video'
+---
+
+To add any youtube video to your app, use the data_type as `youtube_video`. Eg:
+
+```python
+from embedchain import App
+
+app = App()
+app.add('a_valid_youtube_url_here', data_type='youtube_video')
+```
@@ -8,6 +8,7 @@ Embedchain supports several embedding models from the following providers:

 <CardGroup cols={4}>
  <Card title="OpenAI" href="#openai"></Card>
+  <Card title="GoogleAI" href="#google-ai"></Card>
  <Card title="Azure OpenAI" href="#azure-openai"></Card>
  <Card title="GPT4All" href="#gpt4all"></Card>
  <Card title="Hugging Face" href="#hugging-face"></Card>
@@ -24,12 +25,12 @@ Once you have obtained the key, you can use it like this:

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ['OPENAI_API_KEY'] = 'xxx'

 # load embedding model configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")

 app.add("https://en.wikipedia.org/wiki/OpenAI")
 app.query("What is OpenAI?")
@@ -44,6 +45,34 @@ embedder:

 </CodeGroup>

+## Google AI
+
+To use Google AI embedding function, you have to set the `GOOGLE_API_KEY` environment variable. You can obtain the Google API key from the [Google Maker Suite](https://makersuite.google.com/app/apikey)
+
+<CodeGroup>
+```python main.py
+import os
+from embedchain import App
+
+os.environ["GOOGLE_API_KEY"] = "xxx"
+
+app = App.from_config(config_path="config.yaml")
+```
+
+```yaml config.yaml
+embedder:
+  provider: google
+  config:
+    model: 'models/embedding-001'
+    task_type: "retrieval_document"
+    title: "Embeddings for Embedchain"
+```
+</CodeGroup>
+<br/>
+<Note>
+For more details regarding the Google AI embedding model, please refer to the [Google AI documentation](https://ai.google.dev/tutorials/python_quickstart#use_embeddings).
+</Note>
+
 ## Azure OpenAI

 To use Azure OpenAI embedding model, you have to set some of the azure openai related environment variables as given in the code block below:
@@ -52,14 +81,14 @@ To use Azure OpenAI embedding model, you have to set some of the azure openai re

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ["OPENAI_API_TYPE"] = "azure"
-os.environ["OPENAI_API_BASE"] = "https://xxx.openai.azure.com/"
-os.environ["OPENAI_API_KEY"] = "xxx"
+os.environ["AZURE_OPENAI_ENDPOINT"] = "https://xxx.openai.azure.com/"
+os.environ["AZURE_OPENAI_API_KEY"] = "xxx"
 os.environ["OPENAI_API_VERSION"] = "xxx"

-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -90,17 +119,17 @@ GPT4All supports generating high quality embeddings of arbitrary length document
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load embedding model configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
 llm:
  provider: gpt4all
  config:
-    model: 'orca-mini-3b.ggmlv3.q4_0.bin'
+    model: 'orca-mini-3b-gguf2-q4_0.gguf'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -119,10 +148,10 @@ Hugging Face supports generating embeddings of arbitrary length documents of tex
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load embedding model configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -150,10 +179,10 @@ Embedchain supports Google's VertexAI embeddings model through a simple interfac
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load embedding model configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -0,0 +1,12 @@
+---
+title: 🧩 Introduction
+---
+
+## Overview
+
+You can configure following components
+
+* [Data Source](/components/data-sources/overview)
+* [LLM](/components/llms)
+* [Embedding Model](/components/embedding-models)
+* [Vector Database](/components/vector-databases)
@@ -8,9 +8,12 @@ Embedchain comes with built-in support for various popular large language models

 <CardGroup cols={4}>
  <Card title="OpenAI" href="#openai"></Card>
+  <Card title="Google AI" href="#google-ai"></Card>
  <Card title="Azure OpenAI" href="#azure-openai"></Card>
  <Card title="Anthropic" href="#anthropic"></Card>
  <Card title="Cohere" href="#cohere"></Card>
+  <Card title="Together" href="#together"></Card>
+  <Card title="Ollama" href="#ollama"></Card>
  <Card title="GPT4All" href="#gpt4all"></Card>
  <Card title="JinaChat" href="#jinachat"></Card>
  <Card title="Hugging Face" href="#hugging-face"></Card>
@@ -26,7 +29,7 @@ Once you have obtained the key, you can use it like this:

 ```python
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ['OPENAI_API_KEY'] = 'xxx'

@@ -41,12 +44,12 @@ If you are looking to configure the different parameters of the LLM, you can do

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ['OPENAI_API_KEY'] = 'xxx'

 # load llm configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -59,9 +62,170 @@ llm:
    top_p: 1
    stream: false
 ```
-
 </CodeGroup>

+### Function Calling
+To enable [function calling](https://platform.openai.com/docs/guides/function-calling) in your application using embedchain and OpenAI, you need to pass functions into `OpenAILlm` class as an array of functions. Here are several ways in which you can achieve that:
+
+Examples:
+<Accordion title="Using Pydantic Models">
+  ```python
+import os
+from embedchain import App
+from embedchain.llm.openai import OpenAILlm
+import requests
+from pydantic import BaseModel, Field, ValidationError, field_validator
+
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+
+class QA(BaseModel):
+    """
+    A question and answer pair.
+    """
+
+    question: str = Field(
+        ..., description="The question.", example="What is a mountain?"
+    )
+    answer: str = Field(
+        ..., description="The answer.", example="A mountain is a hill."
+    )
+    person_who_is_asking: str = Field(
+        ..., description="The person who is asking the question.", example="John"
+    )
+
+    @field_validator("question")
+    def question_must_end_with_a_question_mark(cls, v):
+        """
+        Validate that the question ends with a question mark.
+        """
+        if not v.endswith("?"):
+            raise ValueError("question must end with a question mark")
+        return v
+
+    @field_validator("answer")
+    def answer_must_end_with_a_period(cls, v):
+        """
+        Validate that the answer ends with a period.
+        """
+        if not v.endswith("."):
+            raise ValueError("answer must end with a period")
+        return v
+
+llm = OpenAILlm(config=None,functions=[QA])
+app = App(llm=llm)
+
+result = app.query("Hey I am Sid. What is a mountain? A mountain is a hill.")
+
+print(result)
+  ```
+  </Accordion>
+  
+  <Accordion title="Using OpenAI JSON schema">
+```python
+import os
+from embedchain import App
+from embedchain.llm.openai import OpenAILlm
+import requests
+from pydantic import BaseModel, Field, ValidationError, field_validator
+
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+
+json_schema = {
+    "name": "get_qa",
+    "description": "A question and answer pair and the user who is asking the question.",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "question": {"type": "string", "description": "The question."},
+            "answer": {"type": "string", "description": "The answer."},
+            "person_who_is_asking": {
+                "type": "string",
+                "description": "The person who is asking the question.",
+            }
+        },
+        "required": ["question", "answer", "person_who_is_asking"],
+    },
+}
+
+llm = OpenAILlm(config=None,functions=[json_schema])
+app = App(llm=llm)
+
+result = app.query("Hey I am Sid. What is a mountain? A mountain is a hill.")
+
+print(result)
+  ```
+  </Accordion>
+  <Accordion title="Using actual python functions">
+  ```python
+import os
+from embedchain import App
+from embedchain.llm.openai import OpenAILlm
+import requests
+from pydantic import BaseModel, Field, ValidationError, field_validator
+
+os.environ["OPENAI_API_KEY"] = "sk-xxx"
+
+def find_info_of_pokemon(pokemon: str):
+    """
+    Find the information of the given pokemon.
+    Args:
+        pokemon: The pokemon.
+    """
+    req = requests.get(f"https://pokeapi.co/api/v2/pokemon/{pokemon}")
+    if req.status_code == 404:
+        raise ValueError("pokemon not found")
+    return req.json()
+
+llm = OpenAILlm(config=None,functions=[find_info_of_pokemon])
+app = App(llm=llm)
+
+result = app.query("Tell me more about the pokemon pikachu.")
+
+print(result)
+```
+</Accordion>
+
+## Google AI
+
+To use Google AI model, you have to set the `GOOGLE_API_KEY` environment variable. You can obtain the Google API key from the [Google Maker Suite](https://makersuite.google.com/app/apikey)
+
+<CodeGroup>
+```python main.py
+import os
+from embedchain import App
+
+os.environ["GOOGLE_API_KEY"] = "xxx"
+
+app = App.from_config(config_path="config.yaml")
+
+app.add("https://www.forbes.com/profile/elon-musk")
+
+response = app.query("What is the net worth of Elon Musk?")
+if app.llm.config.stream: # if stream is enabled, response is a generator
+    for chunk in response:
+        print(chunk)
+else:
+    print(response)
+```
+
+```yaml config.yaml
+llm:
+  provider: google
+  config:
+    model: gemini-pro
+    max_tokens: 1000
+    temperature: 0.5
+    top_p: 1
+    stream: false
+
+embedder:
+  provider: google
+  config:
+    model: 'models/embedding-001'
+    task_type: "retrieval_document"
+    title: "Embeddings for Embedchain"
+```
+</CodeGroup>

 ## Azure OpenAI

@@ -71,14 +235,14 @@ To use Azure OpenAI model, you have to set some of the azure openai related envi

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ["OPENAI_API_TYPE"] = "azure"
 os.environ["OPENAI_API_BASE"] = "https://xxx.openai.azure.com/"
 os.environ["OPENAI_API_KEY"] = "xxx"
 os.environ["OPENAI_API_VERSION"] = "xxx"

-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -110,12 +274,12 @@ To use anthropic's model, please set the `ANTHROPIC_API_KEY` which you find on t

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ["ANTHROPIC_API_KEY"] = "xxx"

 # load llm configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -147,12 +311,12 @@ Once you have the API key, you are all set to use it with Embedchain.

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ["COHERE_API_KEY"] = "xxx"

 # load llm configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -167,6 +331,68 @@ llm:

 </CodeGroup>

+## Together
+
+Install related dependencies using the following command:
+
+```bash
+pip install --upgrade 'embedchain[together]'
+```
+
+Set the `TOGETHER_API_KEY` as environment variable which you can find on their [Account settings page](https://api.together.xyz/settings/api-keys).
+
+Once you have the API key, you are all set to use it with Embedchain.
+
+<CodeGroup>
+
+```python main.py
+import os
+from embedchain import App
+
+os.environ["TOGETHER_API_KEY"] = "xxx"
+
+# load llm configuration from config.yaml file
+app = App.from_config(config_path="config.yaml")
+```
+
+```yaml config.yaml
+llm:
+  provider: together
+  config:
+    model: togethercomputer/RedPajama-INCITE-7B-Base
+    temperature: 0.5
+    max_tokens: 1000
+    top_p: 1
+```
+
+</CodeGroup>
+
+## Ollama
+
+Setup Ollama using https://github.com/jmorganca/ollama
+
+<CodeGroup>
+
+```python main.py
+import os
+from embedchain import App
+
+# load llm configuration from config.yaml file
+app = App.from_config(config_path="config.yaml")
+```
+
+```yaml config.yaml
+llm:
+  provider: ollama
+  config:
+    model: 'llama2'
+    temperature: 0.5
+    top_p: 1
+    stream: true
+```
+
+</CodeGroup>
+
 ## GPT4ALL

 Install related dependencies using the following command:
@@ -180,17 +406,17 @@ GPT4all is a free-to-use, locally running, privacy-aware chatbot. No GPU or inte
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load llm configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
 llm:
  provider: gpt4all
  config:
-    model: 'orca-mini-3b.ggmlv3.q4_0.bin'
+    model: 'orca-mini-3b-gguf2-q4_0.gguf'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
@@ -212,11 +438,11 @@ Once you have the key, load the app using the config yaml file:

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ["JINACHAT_API_KEY"] = "xxx"
 # load llm configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -237,7 +463,7 @@ llm:
 Install related dependencies using the following command:

 ```bash
-pip install --upgrade 'embedchain[huggingface_hub]'
+pip install --upgrade 'embedchain[huggingface-hub]'
 ```

 First, set `HUGGINGFACE_ACCESS_TOKEN` in environment variable which you can obtain from [their platform](https://huggingface.co/settings/tokens).
@@ -248,12 +474,12 @@ Once you have the token, load the app using the config yaml file:

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ["HUGGINGFACE_ACCESS_TOKEN"] = "xxx"

 # load llm configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -278,12 +504,12 @@ Once you have the token, load the app using the config yaml file:

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ["REPLICATE_API_TOKEN"] = "xxx"

 # load llm configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -305,10 +531,10 @@ Setup Google Cloud Platform application credentials by following the instruction
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load llm configuration from config.yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -22,10 +22,10 @@ Utilizing a vector database alongside Embedchain is a seamless process. All you
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load chroma configuration from yaml file
-app = App.from_config(yaml_path="config1.yaml")
+app = App.from_config(config_path="config1.yaml")
 ```

 ```yaml config1.yaml
@@ -58,13 +58,19 @@ Install related dependencies using the following command:
 pip install --upgrade 'embedchain[elasticsearch]'
 ```

+<Note>
+You can configure the Elasticsearch connection by providing either `es_url` or `cloud_id`. If you are using the Elasticsearch Service on Elastic Cloud, you can find the `cloud_id` on the [Elastic Cloud dashboard](https://cloud.elastic.co/deployments).
+</Note>
+
+You can authorize the connection to Elasticsearch by providing either `basic_auth`, `api_key`, or `bearer_auth`.
+
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load elasticsearch configuration from yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -72,9 +78,11 @@ vectordb:
  provider: elasticsearch
  config:
    collection_name: 'es-index'
-    es_url: http://localhost:9200
-    allow_reset: true
-    api_key: xxx
+    cloud_id: 'deployment-name:xxxx'
+    basic_auth:
+      - elastic
+      - <your_password>
+    verify_certs: false
 ```
 </CodeGroup>

@@ -89,22 +97,22 @@ pip install --upgrade 'embedchain[opensearch]'
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load opensearch configuration from yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
 vectordb:
  provider: opensearch
  config:
+    collection_name: 'my-app'
    opensearch_url: 'https://localhost:9200'
    http_auth:
      - admin
      - admin
    vector_dimension: 1536
-    collection_name: 'my-app'
    use_ssl: false
    verify_certs: false
 ```
@@ -125,20 +133,20 @@ Set the Zilliz environment variables `ZILLIZ_CLOUD_URI` and `ZILLIZ_CLOUD_TOKEN`

 ```python main.py
 import os
-from embedchain import Pipeline as App
+from embedchain import App

 os.environ['ZILLIZ_CLOUD_URI'] = 'https://xxx.zillizcloud.com'
 os.environ['ZILLIZ_CLOUD_TOKEN'] = 'xxx'

 # load zilliz configuration from yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
 vectordb:
  provider: zilliz
  config:
-    collection_name: 'zilliz-app'
+    collection_name: 'zilliz_app'
    uri: https://xxxx.api.gcp-region.zillizcloud.com
    token: xxx
    vector_dim: 1536
@@ -164,10 +172,10 @@ In order to use Pinecone as vector database, set the environment variables `PINE
 <CodeGroup>

 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load pinecone configuration from yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -187,10 +195,10 @@ In order to use Qdrant as a vector database, set the environment variables `QDRA

 <CodeGroup>
 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load qdrant configuration from yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -207,10 +215,10 @@ In order to use Weaviate as a vector database, set the environment variables `WE

 <CodeGroup>
 ```python main.py
-from embedchain import Pipeline as App
+from embedchain import App

 # load weaviate configuration from yaml file
-app = App.from_config(yaml_path="config.yaml")
+app = App.from_config(config_path="config.yaml")
 ```

 ```yaml config.yaml
@@ -22,23 +22,12 @@ make lint format

 5. **Create a pull request**: When you are ready to contribute your changes, submit a pull request to the EmbedChain repository. Provide a clear and descriptive title for your pull request, along with a detailed description of the changes you have made.

-# Tech Stack
-
-embedchain is built on the following stack:
-
- [Langchain](https://github.com/hwchase17/langchain) as an LLM framework to load, chunk and index data
- [OpenAI's Ada embedding model](https://platform.openai.com/docs/guides/embeddings) to create embeddings
- [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/gpt/chat-completions-api) as LLM to get answers given the context
- [Chroma](https://github.com/chroma-core/chroma) as the vector database to store embeddings
- [gpt4all](https://github.com/nomic-ai/gpt4all) as an open source LLM
- [sentence-transformers](https://huggingface.co/sentence-transformers) as open source embedding model
-
 ## Team

 ### Authors

 - Taranjeet Singh ([@taranjeetio](https://twitter.com/taranjeetio))
- Deshraj Yadav ([@deshrajdry](https://twitter.com/taranjeetio))
+- Deshraj Yadav ([@deshrajdry](https://twitter.com/deshrajdry))

 ### Citation

@@ -47,7 +36,7 @@ If you utilize this repository, please consider citing it with:
 ```
@misc{embedchain,
  author = {Taranjeet Singh, Deshraj Yadav},
-  title = {Embechain: Data platform for LLMs - Load, index, retrieve and sync any unstructured data},
+  title = {Embechain: The Open Source RAG Framework},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
@@ -1,36 +0,0 @@
---
-title: '📃 JSON'
---
-
-To add any json file, use the data_type as `json`. `json` allows remote urls and conventional file paths. Headers are included for each line, so if you have an `age` column, `18` will be added as `age: 18`. Eg:
-
-```python
-import os
-
-from embedchain.apps.app import App
-
-os.environ["OPENAI_API_KEY"] = "openai_api_key"
-
-app = App()
-
-response = app.query("What is the net worth of Elon Musk as of October 2023?")
-
-print(response)
-"I'm sorry, but I don't have access to real-time information or future predictions. Therefore, I don't know the net worth of Elon Musk as of October 2023."
-
-source_id = app.add("temp.json")
-
-response = app.query("What is the net worth of Elon Musk as of October 2023?")
-
-print(response)
-"As of October 2023, Elon Musk's net worth is $255.2 billion."
-```
-
-```temp.json
-{
-    "question": "What is your net worth, Elon Musk?",
-    "answer": "As of October 2023, Elon Musk's net worth is $255.2 billion, making him one of the wealthiest individuals in the world."
-}
-```
-
-
@@ -1,27 +0,0 @@
---
-title: Overview
---
-
-Embedchain comes with built-in support for various data sources. We handle the complexity of loading unstructured data from these data sources, allowing you to easily customize your app through a user-friendly interface.
-
-<CardGroup cols={4}>
-  <Card title="📊 csv" href="/data-sources/csv"></Card>
-  <Card title="📃 JSON" href="/data-sources/json"></Card>
-  <Card title="📚🌐 docs site" href="/data-sources/docs-site"></Card>
-  <Card title="📄 docx" href="/data-sources/docx"></Card>
-  <Card title="📝 mdx" href="/data-sources/mdx"></Card>
-  <Card title="📓 notion" href="/data-sources/notion"></Card>
-  <Card title="📰 pdf" href="/data-sources/pdf-file"></Card>
-  <Card title="❓💬 q&a pair" href="/data-sources/qna"></Card>
-  <Card title="🗺️ sitemap" href="/data-sources/sitemap"></Card>
-  <Card title="📝 text" href="/data-sources/text"></Card>
-  <Card title="🌐📄 web page" href="/data-sources/web-page"></Card>
-  <Card title="🧾 xml" href="/data-sources/xml"></Card>
-  <Card title="🙌 OpenAPI" href="/data-sources/openapi"></Card>
-  <Card title="🎥📺 youtube video" href="/data-sources/youtube-video"></Card>
-  <Card title="📬 Gmail" href="/data-sources/gmail"></Card>
-</CardGroup>
-
-<br/ >
-
-<Snippet file="missing-data-source-tip.mdx" />
@@ -1,17 +0,0 @@
---
-title: '📰 PDF file'
---
-
-To add any pdf file, use the data_type as `pdf_file`. Eg:
-
-```python
-from embedchain import Pipeline as App
-
-app = App()
-
-app.add('https://arxiv.org/pdf/1706.03762.pdf', data_type='pdf_file')
-app.query("What is the paper 'attention is all you need' about?")
-# Answer: The paper "Attention Is All You Need" proposes a new network architecture called the Transformer, which is based solely on attention mechanisms. It suggests moving away from complex recurrent or convolutional neural networks and instead using attention mechanisms to connect the encoder and decoder in sequence transduction models.
-```
-
-Note that we do not support password protected pdfs.
@@ -1,13 +0,0 @@
---
-title: '🎥📺 Youtube video'
---
-
-
-To add any youtube video to your app, use the data_type (first argument to `.add()` method) as `youtube_video`. Eg:
-
-```python
-from embedchain import Pipeline as App
-
-app = App()
-app.add('a_valid_youtube_url_here', data_type='youtube_video')
-```
@@ -0,0 +1,38 @@
+---
+title: 'Embedchain.ai'
+description: 'Deploy your RAG application to embedchain.ai platform'
+---
+
+## Deploy on Embedchain Platform
+
+Embedchain enables developers to deploy their LLM-powered apps in production using the [Embedchain platform](https://app.embedchain.ai). The platform offers free access to context on your data through its REST API. Once the pipeline is deployed, you can update your data sources anytime after deployment.
+
+See the example below on how to use the deploy your app (for free):
+
+```python
+from embedchain import App
+
+# Initialize app
+app = App()
+
+# Add data source
+app.add("https://www.forbes.com/profile/elon-musk")
+
+# Deploy your pipeline to Embedchain Platform
+app.deploy()
+
+# 🔑 Enter your Embedchain API key. You can find the API key at https://app.embedchain.ai/settings/keys/
+# ec-xxxxxx
+
+# 🛠️ Creating pipeline on the platform...
+# 🎉🎉🎉 Pipeline created successfully! View your pipeline: https://app.embedchain.ai/pipelines/xxxxx
+
+# 🛠️ Adding data to your pipeline...
+# ✅ Data of type: web_page, value: https://www.forbes.com/profile/elon-musk added successfully.
+```
+
+## Seeking help?
+
+If you run into issues with deployment, please feel free to reach out to us via any of the following methods:
+
+<Snippet file="get-help.mdx" />
@@ -0,0 +1,101 @@
+---
+title: 'Fly.io'
+description: 'Deploy your RAG application to fly.io platform'
+---
+
+Embedchain has a nice and simple abstraction on top of the [Fly.io](https://fly.io/) tools to let developers deploy RAG application to fly.io platform seamlessly. 
+
+Follow the instructions given below to deploy your first application quickly:
+
+
+## Step-1: Install flyctl command line
+
+<CodeGroup>
+```bash OSX
+brew install flyctl
+```
+
+```bash Linux
+curl -L https://fly.io/install.sh | sh
+```
+
+```bash Windows
+pwsh -Command "iwr https://fly.io/install.ps1 -useb | iex"
+```
+</CodeGroup>
+
+Once you have installed the fly.io cli tool, signup/login to their platform using the following command:
+
+<CodeGroup>
+```bash Sign up
+fly auth signup
+```
+
+```bash Sign in
+fly auth login
+```
+</CodeGroup>
+
+In case you run into issues, refer to official [fly.io docs](https://fly.io/docs/hands-on/install-flyctl/).
+
+## Step-2: Create RAG app
+
+We provide a command line utility called `ec` in embedchain that inherits the template for `fly.io` platform and help you deploy the app. Follow the instructions to create a fly.io app using the template provided:
+
+```bash Install embedchain
+pip install embedchain
+```
+
+```bash Create application
+mkdir my-rag-app
+ec create --template=fly.io
+```
+
+This will generate a directory structure like this:
+
+```bash
+├── Dockerfile
+├── app.py
+├── fly.toml
+├── .env
+├── .env.example
+├── embedchain.json
+└── requirements.txt
+```
+
+Feel free to edit the files as required.
+- `Dockerfile`: Defines the steps to setup the application
+- `app.py`: Contains API app code
+- `fly.toml`: fly.io config file
+- `.env`: Contains environment variables for production
+- `.env.example`: Contains dummy environment variables (can ignore this file)
+- `embedchain.json`: Contains embedchain specific configuration for deployment (you don't need to configure this)
+- `requirements.txt`: Contains python dependencies for your application
+
+## Step-3: Test app locally
+
+You can run the app locally by simply doing:
+
+```bash Run locally
+pip install -r requirements.txt
+ec dev
+```
+
+## Step-4: Deploy to fly.io
+
+You can deploy to fly.io using the following command:
+```bash Deploy app
+ec deploy
+```
+
+Once this step finished, it will provide you with the deployment endpoint where you can access the app live. It will look something like this (Swagger docs):
+
+You can also check the logs, monitor app status etc on their dashboard by running command `fly dashboard`.
+
+<img src="/images/fly_io.png" />
+
+## Seeking help?
+
+If you run into issues with deployment, please feel free to reach out to us via any of the following methods:
+
+<Snippet file="get-help.mdx" />
@@ -0,0 +1,59 @@
+---
+title: 'Gradio.app'
+description: 'Deploy your RAG application to gradio.app platform'
+---
+
+Embedchain offers a Streamlit template to facilitate the development of RAG chatbot applications in just three easy steps.
+
+Follow the instructions given below to deploy your first application quickly:
+
+## Step-1: Create RAG app
+
+We provide a command line utility called `ec` in embedchain that inherits the template for `gradio.app` platform and help you deploy the app. Follow the instructions to create a gradio.app app using the template provided:
+
+```bash Install embedchain
+pip install embedchain
+```
+
+```bash Create application
+mkdir my-rag-app
+ec create --template=gradio.app
+```
+
+This will generate a directory structure like this:
+
+```bash
+├── app.py
+├── embedchain.json
+└── requirements.txt
+```
+
+Feel free to edit the files as required.
+- `app.py`: Contains API app code
+- `embedchain.json`: Contains embedchain specific configuration for deployment (you don't need to configure this)
+- `requirements.txt`: Contains python dependencies for your application
+
+## Step-2: Test app locally
+
+You can run the app locally by simply doing:
+
+```bash Run locally
+pip install -r requirements.txt
+ec dev
+```
+
+## Step-3: Deploy to gradio.app
+
+```bash Deploy to gradio.app
+ec deploy
+```
+
+This will run `gradio deploy` which will prompt you questions and deploy your app directly to huggingface spaces.
+
+<img src="/images/gradio_app.png" alt="gradio app" />
+
+## Seeking help?
+
+If you run into issues with deployment, please feel free to reach out to us via any of the following methods:
+
+<Snippet file="get-help.mdx" />
@@ -0,0 +1,103 @@
+---
+title: 'Huggingface.co'
+description: 'Deploy your RAG application to huggingface.co platform'
+---
+
+With Embedchain, you can directly host your apps in just three steps to huggingface spaces where you can view and deploy your app to the world.
+
+We support two types of deployment to huggingface spaces:
+
+<CardGroup cols={2}>
+    <Card title="" href="#using-streamlit-io">
+        Streamlit.io
+    </Card>
+    <Card title="" href="#using-gradio-app">
+        Gradio.app
+    </Card>
+</CardGroup>
+
+## Using streamlit.io
+
+### Step 1: Create a new RAG app
+
+Create a new RAG app using the following command:
+
+```bash
+mkdir my-rag-app
+ec create --template=hf/streamlit.io # inside my-rag-app directory
+```
+
+When you run this for the first time, you'll be asked to login to huggingface.co. Once you login, you'll need to create a **write** token. You can create a write token by going to [huggingface.co settings](https://huggingface.co/settings/token). Once you create a token, you'll be asked to enter the token in the terminal.
+
+This will also create an `embedchain.json` file in your app directory. Add a `name` key into the `embedchain.json` file. This will be the "repo-name" of your app in huggingface spaces.
+
+```json embedchain.json
+{
+    "name": "my-rag-app",
+    "provider": "hf/streamlit.io"
+}
+```
+
+### Step-2: Test app locally
+
+You can run the app locally by simply doing:
+
+```bash Run locally
+pip install -r requirements.txt
+ec dev
+```
+
+### Step-3: Deploy to huggingface spaces
+
+```bash Deploy to huggingface spaces
+ec deploy
+```
+
+This will deploy your app to huggingface spaces. You can view your app at `https://huggingface.co/spaces/<your-username>/my-rag-app`. This will get prompted in the terminal once the app is deployed.
+
+## Using gradio.app
+
+Similar to streamlit.io, you can deploy your app to gradio.app in just three steps.
+
+### Step 1: Create a new RAG app
+
+Create a new RAG app using the following command:
+
+```bash
+mkdir my-rag-app
+ec create --template=hf/gradio.app # inside my-rag-app directory
+```
+
+When you run this for the first time, you'll be asked to login to huggingface.co. Once you login, you'll need to create a **write** token. You can create a write token by going to [huggingface.co settings](https://huggingface.co/settings/token). Once you create a token, you'll be asked to enter the token in the terminal.
+
+This will also create an `embedchain.json` file in your app directory. Add a `name` key into the `embedchain.json` file. This will be the "repo-name" of your app in huggingface spaces.
+
+```json embedchain.json
+{
+    "name": "my-rag-app",
+    "provider": "hf/gradio.app"
+}
+```
+
+### Step-2: Test app locally
+
+You can run the app locally by simply doing:
+
+```bash Run locally
+pip install -r requirements.txt
+ec dev
+```
+
+### Step-3: Deploy to huggingface spaces
+
+```bash Deploy to huggingface spaces
+ec deploy
+```
+
+This will deploy your app to huggingface spaces. You can view your app at `https://huggingface.co/spaces/<your-username>/my-rag-app`. This will get prompted in the terminal once the app is deployed.
+
+## Seeking help?
+
+If you run into issues with deployment, please feel free to reach out to us via any of the following methods:
+
+<Snippet file="get-help.mdx" />
@@ -0,0 +1,63 @@
+---
+title: 'Modal.com'
+description: 'Deploy your RAG application to modal.com platform'
+---
+
+Embedchain has a nice and simple abstraction on top of the [Modal.com](https://modal.com/) tools to let developers deploy RAG application to modal.com platform seamlessly. 
+
+Follow the instructions given below to deploy your first application quickly:
+
+
+## Step-1 Create RAG application: 
+
+We provide a command line utility called `ec` in embedchain that inherits the template for `modal.com` platform and help you deploy the app. Follow the instructions to create a modal.com app using the template provided:
+
+
+```bash Create application
+pip install embedchain[modal]
+mkdir my-rag-app
+ec create --template=modal.com
+```
+
+This `create` command will open a browser window and ask you to login to your modal.com account and will generate a directory structure like this:
+
+```bash
+├── app.py
+├── .env
+├── .env.example
+├── embedchain.json
+└── requirements.txt
+```
+
+Feel free to edit the files as required.
+- `app.py`: Contains API app code
+- `.env`: Contains environment variables for production
+- `.env.example`: Contains dummy environment variables (can ignore this file)
+- `embedchain.json`: Contains embedchain specific configuration for deployment (you don't need to configure this)
+- `requirements.txt`: Contains python dependencies for your FastAPI application
+
+## Step-2: Test app locally
+
+You can run the app locally by simply doing:
+
+```bash Run locally
+pip install -r requirements.txt
+ec dev
+```
+
+## Step-3: Deploy to modal.com
+
+You can deploy to modal.com using the following command:
+```bash Deploy app
+ec deploy
+```
+
+Once this step finished, it will provide you with the deployment endpoint where you can access the app live. It will look something like this (Swagger docs):
+
+<img src="/images/fly_io.png" />
+
+## Seeking help?
+
+If you run into issues with deployment, please feel free to reach out to us via any of the following methods:
+
+<Snippet file="get-help.mdx" />
@@ -0,0 +1,93 @@
+---
+title: 'Render.com'
+description: 'Deploy your RAG application to render.com platform'
+---
+
+Embedchain has a nice and simple abstraction on top of the [render.com](https://render.com/) tools to let developers deploy RAG application to render.com platform seamlessly. 
+
+Follow the instructions given below to deploy your first application quickly:
+
+## Step-1: Install `render` command line
+
+<CodeGroup>
+```bash OSX
+brew tap render-oss/render
+brew install render
+```
+
+```bash Linux
+# Make sure you have deno installed -> https://docs.render.com/docs/cli#from-source-unsupported-operating-systems
+git clone https://github.com/render-oss/render-cli
+cd render-cli
+make deps
+deno task run
+deno compile
+```
+
+```bash Windows
+choco install rendercli
+```
+</CodeGroup>
+
+In case you run into issues, refer to official [render.com docs](https://docs.render.com/docs/cli).
+
+## Step-2 Create RAG application: 
+
+We provide a command line utility called `ec` in embedchain that inherits the template for `render.com` platform and help you deploy the app. Follow the instructions to create a render.com app using the template provided:
+
+
+```bash Create application
+pip install embedchain
+mkdir my-rag-app
+ec create --template=render.com
+```
+
+This `create` command will open a browser window and ask you to login to your render.com account and will generate a directory structure like this:
+
+```bash
+├── app.py
+├── .env
+├── render.yaml
+├── embedchain.json
+└── requirements.txt
+```
+
+Feel free to edit the files as required.
+- `app.py`: Contains API app code
+- `.env`: Contains environment variables for production
+- `render.yaml`: Contains render.com specific configuration for deployment (configure this according to your needs, follow [this](https://docs.render.com/docs/blueprint-spec) for more info)
+- `embedchain.json`: Contains embedchain specific configuration for deployment (you don't need to configure this)
+- `requirements.txt`: Contains python dependencies for your application
+
+## Step-3: Test app locally
+
+You can run the app locally by simply doing:
+
+```bash Run locally
+pip install -r requirements.txt
+ec dev
+```
+
+## Step-4: Deploy to render.com
+
+Before deploying to render.com, you only have to set up one thing. 
+
+In the render.yaml file, make sure to modify the repo key by inserting the URL of your Git repository where your application will be hosted. You can create a repository from [GitHub](https://github.com) or [GitLab](https://gitlab.com/users/sign_in).
+
+After that, you're ready to deploy on render.com.
+
+```bash Deploy app
+ec deploy
+```
+
+When you run this, it should open up your render dashboard and you can see the app being deployed. You can find your hosted link over there only.
+
+You can also check the logs, monitor app status etc on their dashboard by running command `render dashboard`.
+
+<img src="/images/fly_io.png" />
+
+## Seeking help?
+
+If you run into issues with deployment, please feel free to reach out to us via any of the following methods:
+
+<Snippet file="get-help.mdx" />
@@ -0,0 +1,62 @@
+---
+title: 'Streamlit.io'
+description: 'Deploy your RAG application to streamlit.io platform'
+---
+
+Embedchain offers a Streamlit template to facilitate the development of RAG chatbot applications in just three easy steps.
+
+Follow the instructions given below to deploy your first application quickly:
+
+## Step-1: Create RAG app
+
+We provide a command line utility called `ec` in embedchain that inherits the template for `streamlit.io` platform and help you deploy the app. Follow the instructions to create a streamlit.io app using the template provided:
+
+```bash Install embedchain
+pip install embedchain
+```
+
+```bash Create application
+mkdir my-rag-app
+ec create --template=streamlit.io
+```
+
+This will generate a directory structure like this:
+
+```bash
+├── .streamlit
+│   └── secrets.toml
+├── app.py
+├── embedchain.json
+└── requirements.txt
+```
+
+Feel free to edit the files as required.
+- `app.py`: Contains API app code
+- `.streamlit/secrets.toml`: Contains secrets for your application
+- `embedchain.json`: Contains embedchain specific configuration for deployment (you don't need to configure this)
+- `requirements.txt`: Contains python dependencies for your application
+
+Add your `OPENAI_API_KEY` in `.streamlit/secrets.toml` file to run and deploy the app.
+
+## Step-2: Test app locally
+
+You can run the app locally by simply doing:
+
+```bash Run locally
+pip install -r requirements.txt
+ec dev
+```
+
+## Step-3: Deploy to streamlit.io
+
+![Streamlit App deploy button](https://github.com/embedchain/embedchain/assets/73601258/90658e28-29e5-4ceb-9659-37ff8b861a29)
+
+Use the deploy button from the streamlit website to deploy your app.
+
+You can refer this [guide](https://docs.streamlit.io/streamlit-community-cloud/deploy-your-app) if you run into any problems.
+
+## Seeking help?
+
+If you run into issues with deployment, please feel free to reach out to us via any of the following methods:
+
+<Snippet file="get-help.mdx" />
@@ -1,93 +0,0 @@
---
-title: '🌍 API Server'
---
-
-The API server example can be found [here](https://github.com/embedchain/embedchain/tree/main/examples/api_server).
-
-It is a Flask based server that integrates the `embedchain` package, offering endpoints to add, query, and chat to engage in conversations with a chatbot using JSON requests.
-
-### 🐳 Docker Setup
-
- Open variables.env, and edit it to add your 🔑 `OPENAI_API_KEY`.
- To setup your api server using docker, run the following command inside this folder using your terminal.
-
-```bash
-docker-compose up --build
-```
-
-📝 Note: The build command might take a while to install all the packages depending on your system resources.
-
-### 🚀 Usage Instructions
-
- Your api server is running on [http://localhost:5000/](http://localhost:5000/)
- To use the api server, make an api call to the endpoints `/add`, `/query` and `/chat` using the json formats discussed below.
- To add data sources to the bot (/add):
-```json
-// Request
-{
-  "data_type": "your_data_type_here",
-  "url_or_text": "your_url_or_text_here"
-}
-
-// Response
-{
-  "data": "Added data_type: url_or_text"
-}
-```
- To ask queries from the bot (/query):
-```json
-// Request
-{
-  "question": "your_question_here"
-}
-
-// Response
-{
-  "data": "your_answer_here"
-}
-```
- To chat with the bot (/chat):
-```json
-// Request
-{
-  "question": "your_question_here"
-}
-
-// Response
-{
-  "data": "your_answer_here"
-}
-```
-
-### 📡 Curl Call Formats
-
- To add data sources to the bot (/add):
-```bash
-curl -X POST \
-     -H "Content-Type: application/json" \
-     -d '{
-         "data_type": "your_data_type_here",
-         "url_or_text": "your_url_or_text_here"
-     }' \
-     http://localhost:5000/add
-```
- To ask queries from the bot (/query):
-```bash
-curl -X POST \
-     -H "Content-Type: application/json" \
-     -d '{
-         "question": "your_question_here"
-     }' \
-     http://localhost:5000/query
-```
- To chat with the bot (/chat):
-```bash
-curl -X POST \
-     -H "Content-Type: application/json" \
-     -d '{
-         "question": "your_question_here"
-     }' \
-     http://localhost:5000/chat
-```
-
-🎉 Happy Chatting! 🎉
@@ -0,0 +1,32 @@
+### Embedchain Chat with PDF App
+
+You can easily create and deploy your own `chat-pdf` App using Embedchain.
+
+Here are few simple steps for you to create and deploy your app:
+
+1. Fork the embedchain repo from [Github](https://github.com/embedchain/embedchain).
+
+<Note>
+If you run into problems with forking, please refer to [github docs](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo) for forking a repo.
+</Note>
+
+2. Navigate to `chat-pdf` example app from your forked repo:
+
+```bash
+cd <your_fork_repo>/examples/chat-pdf
+```
+
+3. Run your app in development environment with simple commands
+
+```bash
+pip install -r requirements.txt
+ec dev
+```
+
+Feel free to improve our simple `chat-pdf` streamlit app and create pull request to showcase your app [here](https://docs.embedchain.ai/examples/showcase)
+
+4. You can easily deploy your app using Streamlit interface
+
+Connect your Github account with Streamlit and refer this [guide](https://docs.streamlit.io/streamlit-community-cloud/deploy-your-app) to deploy your app.
+
+You can also use the deploy button from your streamlit website you see when running `ec dev` command.
@@ -1,5 +1,5 @@
 ---
-title: '🤖 Discord Bot'
+title: "🤖 Discord Bot"
 ---

 ### 🔑 Keys Setup
@@ -12,9 +12,11 @@ title: '🤖 Discord Bot'
 - On the left sidebar, click on `OAuth2` and go to `General`.
 - Set `Authorization Method` to `In-app Authorization`. Under `Scopes` select `bot`.
 - Under `Bot Permissions` allow the following and then click on `Save Changes`.
+
 ```text
 Send Messages (under Text Permissions)
 ```
+
 - Now under `OAuth2` and go to `URL Generator`. Under `Scopes` select `bot`.
 - Under `Bot Permissions` set the same permissions as above.
 - Now scroll down and copy the `Generated URL`. Paste it in a browser window and select the Server where you want to add the bot.
@@ -23,40 +25,46 @@ Send Messages (under Text Permissions)

 ### Take the bot online

-1. Install embedchain python package:
+<Tabs>
+    <Tab title="docker">
+        ```bash
+        docker run --name discord-bot -e OPENAI_API_KEY=sk-xxx -e DISCORD_BOT_TOKEN=xxx -p 8080:8080 embedchain/discord-bot:latest
+        ```
+    </Tab>
+    <Tab title="python">
+        ```bash
+        pip install --upgrade "embedchain[discord]"

-```bash
-pip install --upgrade "embedchain[discord]"
-```
+        python -m embedchain.bots.discord

-2. Launch your Discord bot:
-
-
-```bash
-python -m embedchain.bots.discord
-```
-
-If you prefer to see the question and not only the answer, run it with
-
-```bash
-python -m embedchain.bots.discord --include-question
-```
+        # or if you prefer to see the question and not only the answer, run it with
+        python -m embedchain.bots.discord --include-question
+        ```
+    </Tab>
+</Tabs>

 ### 🚀 Usage Instructions

 - Go to the server where you have added your bot.
+  ![Slash commands interaction with bot](https://github.com/embedchain/embedchain/assets/73601258/bf1414e3-d408-4863-b0d2-ef382a76467e)
 - You can add data sources to the bot using the slash command:
+
 ```text
-/add <data_type> <url_or_text>
+/ec add <data_type> <url_or_text>
 ```
+
 - You can ask your queries from the bot using the slash command:
+
 ```text
-/query <question>
+/ec query <question>
 ```
+
 - You can chat with the bot using the slash command:
+
 ```text
-/chat <question>
+/ec chat <question>
 ```
+
 📝 Note: To use the bot privately, you can message the bot directly by right clicking the bot and selecting `Message`.

 🎉 Happy Chatting! 🎉
@@ -1,5 +1,5 @@
 ---
-title: '🌐 Full Stack'
+title: 'Full Stack'
 ---

 The Full Stack app example can be found [here](https://github.com/embedchain/embedchain/tree/main/examples/full_stack).
@@ -8,14 +8,45 @@ This guide will help you setup the full stack app on your local machine.

 ### 🐳 Docker Setup

- To setup full stack app using docker, run the following command inside this folder using your terminal.
+- Create a `docker-compose.yml` file and paste the following code in it.
+
+```yaml
+version: "3.9"
+
+services:
+  backend:
+    container_name: embedchain-backend
+    restart: unless-stopped
+    build:
+      context: backend
+      dockerfile: Dockerfile
+    image: embedchain/backend
+    ports:
+      - "8000:8000"
+
+  frontend:
+    container_name: embedchain-frontend
+    restart: unless-stopped
+    build:
+      context: frontend
+      dockerfile: Dockerfile
+    image: embedchain/frontend
+    ports:
+      - "3000:3000"
+    depends_on:
+      - "backend"
+```
+
+- Run the following command,

 ```bash
-docker-compose up --build
+docker-compose up
 ```

 📝 Note: The build command might take a while to install all the packages depending on your system resources.

+![Fullstack App](https://github.com/embedchain/embedchain/assets/73601258/c7c04bbb-9be7-4669-a6af-039e7e972a13)
+
 ### 🚀 Usage Instructions

 - Go to [http://localhost:3000/](http://localhost:3000/) in your browser to view the dashboard.
@@ -1,8 +1,15 @@
 ---
-title: 🔎 Examples
-description: 'Collection of Google colab notebook and Replit links for users'
+title: Notebooks & Replits
 ---

+# Explore awesome apps
+
+Check out the remarkable work accomplished using [Embedchain](https://app.embedchain.ai/custom-gpts/).
+
+## Collection of Google colab notebook and Replit links for users
+
+Get started with Embedchain by trying out the examples below. You can run the examples in your browser using Google Colab or Replit.
+
 <table>
  <thead>
    <tr>
@@ -37,6 +44,14 @@ description: 'Collection of Google colab notebook and Replit links for users'
      <td className="align-middle"><a target="_blank" href="https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/cohere.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" noZoom alt="Open In Colab"/></a></td>
      <td className="align-middle"><a target="_blank" href="https://replit.com/@taranjeetio/cohere#main.py"><img src="https://replit.com/badge?caption=Try%20with%20Replit&amp;variant=small" noZoom alt="Try with Replit Badge"/></a></td>
    </tr>
+    <tr>
+      <td className="align-middle">Together</td>
+      <td className="align-middle"><a target="_blank" href="https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/together.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" noZoom alt="Open In Colab"/></a></td>
+    </tr>
+    <tr>
+      <td className="align-middle">Ollama</td>
+      <td className="align-middle"><a target="_blank" href="https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/ollama.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" noZoom alt="Open In Colab"/></a></td>
+    </tr>
    <tr>
      <td className="align-middle">Hugging Face</td>
      <td className="align-middle"><a target="_blank" href="https://colab.research.google.com/github/embedchain/embedchain/blob/main/notebooks/hugging_face_hub.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" noZoom alt="Open In Colab"/></a></td>
@@ -0,0 +1,60 @@
+---
+title: 'OpenAI Assistant'
+---
+
+<img src="https://blogs.swarthmore.edu/its/wp-content/uploads/2022/05/openai.jpg"  align="center" width="500" alt="OpenAI Logo"/>
+
+Embedchain now supports [OpenAI Assistants API](https://platform.openai.com/docs/assistants/overview) which allows you to build AI assistants within your own applications. An Assistant has instructions and can leverage models, tools, and knowledge to respond to user queries.
+
+At a high level, an integration of the Assistants API has the following flow:
+
+1. Create an Assistant in the API by defining custom instructions and picking a model
+2. Create a Thread when a user starts a conversation
+3. Add Messages to the Thread as the user ask questions
+4. Run the Assistant on the Thread to trigger responses. This automatically calls the relevant tools.
+
+Creating an OpenAI Assistant using Embedchain is very simple 3 step process.
+
+## Step 1: Create OpenAI Assistant
+
+Make sure that you have `OPENAI_API_KEY` set in the environment variable.
+
+```python Initialize
+from embedchain.store.assistants import OpenAIAssistant
+
+assistant = OpenAIAssistant(
+    name="OpenAI DevDay Assistant",
+    instructions="You are an organizer of OpenAI DevDay",
+)
+```
+
+If you want to use the existing assistant, you can do something like this:
+
+```python Initialize
+# Load an assistant and create a new thread
+assistant = OpenAIAssistant(assistant_id="asst_xxx")
+
+# Load a specific thread for an assistant
+assistant = OpenAIAssistant(assistant_id="asst_xxx", thread_id="thread_xxx")
+```
+
+## Step-2: Add data to thread
+
+You can add any custom data source that is supported by Embedchain. Else, you can directly pass the file path on your local system and Embedchain propagates it to OpenAI Assistant.
+```python Add data
+assistant.add("/path/to/file.pdf")
+assistant.add("https://www.youtube.com/watch?v=U9mJuUkhUzk")
+assistant.add("https://openai.com/blog/new-models-and-developer-products-announced-at-devday")
+```
+
+## Step-3: Chat with your Assistant
+```python Chat
+assistant.chat("How much OpenAI credits were offered to attendees during OpenAI DevDay?")
+# Response: 'Every attendee of OpenAI DevDay 2023 was offered $500 in OpenAI credits.'
+```
+
+You can try it out yourself using the following Google Colab notebook:
+
+<a href="https://colab.research.google.com/drive/1BKlXZYSl6AFRgiHZ5XIzXrXC_24kDYHQ?usp=sharing">
+    <img src="https://camo.githubusercontent.com/84f0493939e0c4de4e6dbe113251b4bfb5353e57134ffd9fcab6b8714514d4d1/68747470733a2f2f636f6c61622e72657365617263682e676f6f676c652e636f6d2f6173736574732f636f6c61622d62616467652e737667" alt="Open in Colab" />
+</a>
@@ -0,0 +1,51 @@
+---
+title: 'Open-Source AI Assistant'
+---
+
+Embedchain also provides support for creating Open-Source AI Assistants (similar to [OpenAI Assistants API](https://platform.openai.com/docs/assistants/overview)) which allows you to build AI assistants within your own applications using any LLM (OpenAI or otherwise). An Assistant has instructions and can leverage models, tools, and knowledge to respond to user queries.
+
+At a high level, the Open-Source AI Assistants API has the following flow:
+
+1. Create an AI Assistant by picking a model
+2. Create a Thread when a user starts a conversation
+3. Add Messages to the Thread as the user ask questions
+4. Run the Assistant on the Thread to trigger responses. This automatically calls the relevant tools.
+
+Creating an Open-Source AI Assistant is a simple 3 step process.
+
+## Step 1: Instantiate AI Assistant
+
+```python Initialize
+from embedchain.store.assistants import AIAssistant
+
+assistant = AIAssistant(
+    name="My Assistant",
+    data_sources=[{"source": "https://www.youtube.com/watch?v=U9mJuUkhUzk"}])
+```
+
+If you want to use the existing assistant, you can do something like this:
+
+```python Initialize
+# Load an assistant and create a new thread
+assistant = AIAssistant(assistant_id="asst_xxx")
+
+# Load a specific thread for an assistant
+assistant = AIAssistant(assistant_id="asst_xxx", thread_id="thread_xxx")
+```
+
+## Step-2: Add data to thread
+
+You can add any custom data source that is supported by Embedchain. Else, you can directly pass the file path on your local system and Embedchain propagates it to OpenAI Assistant.
+
+```python Add data
+assistant.add("/path/to/file.pdf")
+assistant.add("https://www.youtube.com/watch?v=U9mJuUkhUzk")
+assistant.add("https://openai.com/blog/new-models-and-developer-products-announced-at-devday")
+```
+
+## Step-3: Chat with your AI Assistant
+
+```python Chat
+assistant.chat("How much OpenAI credits were offered to attendees during OpenAI DevDay?")
+# Response: 'Every attendee of OpenAI DevDay 2023 was offered $500 in OpenAI credits.'
+```
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Deshraj Yadav	60bbc180ba	Add support for `ec runserver` command (#1112 )	2024-01-04 15:59:43 +05:30
Taranjeet Singh	a67d902b85	Fix top_p value bug in quickstart example in docs (#1110 )	2024-01-04 00:41:14 +05:30
Deven Patel	ae2e9cb890	[Bugfix] fix cache session id in chat method (#1107 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2024-01-03 13:51:11 +05:30
Deven Patel	1976d38b25	[Bugfix] fix chat pdf streamlit example (#1108 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2024-01-03 13:46:44 +05:30
Deshraj Yadav	f5e3410e9a	Update docs for session_id (#1105 )	2024-01-03 01:14:47 +05:30
Taranjeet Singh	2f6ba642c7	feat: Add private ai example (#1101 )	2024-01-02 19:18:25 +05:30
Taranjeet Singh	dd9b72dc62	Bump version to 0.1.51 (#1100 )	2024-01-02 18:08:18 +05:30
Taranjeet Singh	dd258c14b5	feat: Make gpt4all work offline. (#1099 )	2024-01-02 17:46:08 +05:30
Deven Patel	295cd3fac6	[Updates] Update GPTCache configuration/docs (#1098 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2024-01-02 17:32:48 +05:30
Sidharth Mohanty	c62663f2e4	Add GPT4Vision Image loader (#1089 ) Co-authored-by: Deshraj Yadav <deshrajdry@gmail.com>	2024-01-02 03:57:23 +05:30
Taranjeet Singh	367d6b70e2	Update docs and side navbar. (#1097 )	2024-01-02 00:18:57 +05:30
Taranjeet Singh	27236bd1b2	Improve docs. (#1096 )	2024-01-01 23:22:43 +05:30
Deven Patel	6a82eb4287	[Docs] update search docs (#1093 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2024-01-01 22:32:27 +05:30
Logie	bd88fe3980	fix: typo when import helpers.json_serializable on rss_feed files (#1095 )	2024-01-01 21:38:41 +05:30
Deven Patel	4f70fea6df	[BugFix] search method in app (#1092 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-31 23:58:17 +05:30
Sidharth Mohanty	aee5bbb44b	[Refactor] Update dependencies and loaders (#1062 )	2023-12-30 20:52:20 +05:30
Deven Patel	a304ded500	[Bugfix] fix config validation for google llm config (#1088 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-30 16:25:41 +05:30
Deshraj Yadav	a54dde0509	[Bug Fix] Handle chat sessions properly during `app.chat()` calls (#1084 )	2023-12-30 15:36:24 +05:30
Taranjeet Singh	52b4577d3b	Update links, code and description (#1087 )	2023-12-30 15:32:16 +05:30
Taranjeet Singh	e199f57279	Fix light mode logo (#1083 )	2023-12-30 15:16:48 +05:30
Taranjeet Singh	dec12b33a6	Update dark mode logo (#1082 )	2023-12-30 15:14:14 +05:30
Deven Patel	04daa1b206	[Feature] Add support for GPTCache (#1065 )	2023-12-30 14:51:48 +05:30
Deven Patel	a7e1520d08	[Improvements] update docs (#1079 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-30 13:39:38 +05:30
Deshraj Yadav	9e2b232c13	Update version to 0.1.47 (#1077 )	2023-12-30 00:16:31 +05:30
Sidharth Mohanty	404e73af77	[Feature] Add Dropbox loader (#1073 ) Co-authored-by: Deshraj Yadav <deshrajdry@gmail.com>	2023-12-29 23:22:25 +05:30
Sidharth Mohanty	a544b4d3ff	Load local text files of any kind - code, txts, json etc (#1076 )	2023-12-29 22:26:24 +05:30
Sidharth Mohanty	6df63d9ca7	Update notebooks to use dict instead of yaml and remove dataloaders (#1075 )	2023-12-29 21:57:46 +05:30
Deshraj Yadav	904baac153	Update version to 0.1.46 (#1074 )	2023-12-29 17:01:11 +05:30
Deven Patel	a926bcc640	[Refactor] Converge Pipeline and App classes (#1021 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-29 16:52:41 +05:30
Deven Patel	c0aafd38c9	[Feature] Return score when doing search in vectorDB (#1060 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-29 15:56:12 +05:30
Deven Patel	19d80914df	[Improvement] return all the metadata when citations flag is `True` (#1059 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-29 14:48:41 +05:30
Sidharth Mohanty	d9d529987e	Migrate from `template` to `prompt` arg while keeping backward compatibility (#1066 )	2023-12-28 23:36:33 +05:30
Deven Patel	12e6eaf802	[Docs] add docs for youtube channel (#1072 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-28 21:08:48 +05:30
Taranjeet Singh	7a026ea282	Fix link in github issue bug template (#1068 )	2023-12-28 20:06:53 +05:30
Taranjeet Singh	97dc90169b	docs: Update framework desc & link (#1067 )	2023-12-28 19:39:22 +05:30
Sidharth Mohanty	64c02f374a	Add cards for gradio and hf (#1061 )	2023-12-28 09:03:31 +05:30
Sukkrit Sharma	6c1ea7799e	[Feature] Add support for Together AI API (#1058 )	2023-12-25 18:36:20 +05:30
Deshraj Yadav	6be29f5bed	[Bug Fix] Fix `FileExistsError` when using Embedchain in multithreading setting (#1056 )	2023-12-24 16:33:53 +05:30
Deven Patel	68737da7a2	[Update] update readme (#1054 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-23 16:38:32 +05:30
Deven Patel	da388b679f	[Chat PDF] update chat pdf logic (#1053 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-23 16:13:48 +05:30
Sidharth Mohanty	11f0d719f5	Provide openai-key support from config (#1052 )	2023-12-23 14:42:18 +05:30
Deshraj Yadav	e90673ae5b	Added Dev Container Folder	2023-12-23 12:55:39 +05:30
Deven Patel	f055028c6b	[Bugfix] fix chat PDF dependencies (#1051 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-23 12:21:03 +05:30
Prikshit	106a338371	Add local file path support to SitemapLoader (#954 )	2023-12-23 10:56:33 +05:30
Deven Patel	9fe80c5cca	[App Deployment] create chat with PDF app (#1049 ) Co-authored-by: Deven Patel <deven298@yahoo.com> Co-authored-by: Deshraj Yadav <deshrajdry@gmail.com>	2023-12-22 19:53:10 +05:30
Deshraj Yadav	050706e95e	Update poetry.lock file (#1048 )	2023-12-22 19:48:27 +05:30
Sidharth Mohanty	6d2389de1c	[fix] streamlit app init by caching it (#1047 )	2023-12-22 12:03:45 +05:30
Dhravya Shah	dd97fad5a4	Fix: indents in yaml file (#1046 )	2023-12-22 10:56:40 +05:30
Sukkrit Sharma	0f73ba9677	Added Support for Ollama for local model inference. (#1045 ) Co-authored-by: Deshraj Yadav <deshraj@gatech.edu>	2023-12-22 05:10:00 +05:30
Sidharth Mohanty	210fe9bb80	[new] add gradio and hf spaces deployments (#1042 )	2023-12-22 00:00:55 +05:30
Sidharth Mohanty	ec8549d0e1	[fix] dot file and docs (#1044 )	2023-12-21 18:50:12 +05:30
Deven Patel	b77d9d750f	[Bugfix] Improve/modal deployment (#1041 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-21 14:30:54 +05:30
Deshraj Yadav	a10823d309	Add Unacademy AI demo (#1043 )	2023-12-21 14:23:03 +05:30
Sidharth Mohanty	3a09c2bd62	[docs] add a faq on how to persist data (#1040 )	2023-12-21 10:32:47 +05:30
Sidharth Mohanty	a1394ce32e	[chore] pypdf and bs4 by default in package (#1038 )	2023-12-20 23:14:30 +05:30
Sidharth Mohanty	1020a4121f	[new] streamlit deployment (#1034 )	2023-12-20 23:02:51 +05:30
Sidharth Mohanty	737837ae0b	[Feature] Add `render.com` deployment template (#1033 ) Co-authored-by: Deven Patel <iamdevenpatel@gmail.com>	2023-12-20 22:38:23 +05:30
Deshraj Yadav	7ee2d0653b	Update Sadhguru AI code (#1037 )	2023-12-20 17:11:59 +05:30
Deshraj Yadav	43926fb527	Update introduction in README and docs (#1036 )	2023-12-20 14:54:02 +05:30
Deshraj Yadav	7bcc9e35dd	Update README (#1035 )	2023-12-20 14:44:41 +05:30
Deshraj Yadav	6437661837	[Example] Add example for creating Sadhguru AI using Embedchain (#1032 )	2023-12-19 20:21:53 +05:30
Sidharth Mohanty	b5f84f27ff	[chore] fix deployment link for modal.com card (#1031 )	2023-12-19 20:09:58 +05:30
Sidharth Mohanty	48c38b5dc3	Telemetry support for `ec` cli commands (#1030 )	2023-12-19 19:29:04 +05:30
Sidharth Mohanty	b4f3bbbbc9	Add docs for modal.com deployment (#1029 )	2023-12-19 16:37:58 +05:30
Deshraj Yadav	3cd50c4cd9	[Deployment] Setup fly.io deployment method and update docs (#1028 ) Co-authored-by: sidmohanty11 <sidmohanty11@gmail.com>	2023-12-19 14:36:44 +05:30
Sidharth Mohanty	cd2c40a9c4	OpenAI function calling support (#1011 )	2023-12-18 19:34:15 +05:30
Sidharth Mohanty	33dcfe42b5	Improve Streamlit docs (#1025 )	2023-12-18 18:25:17 +05:30
Deven Patel	db37b2ac15	[Bugfix] fix chunker config bug (#1024 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-18 17:32:26 +05:30
Sidharth Mohanty	bee4e834b1	Add docs for slack and add missing datatype (#1023 )	2023-12-18 16:52:28 +05:30
Sidharth Mohanty	0272459435	Embedchain + Mistral Streamlit chatbot (#1017 )	2023-12-18 14:42:19 +05:30
Deven Patel	c0b5e93967	[Feature] add google ai embedder (#1019 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-18 13:58:01 +05:30
Sidharth Mohanty	6983ebba49	Chainlit + Embedchain Integration (Example) (#1020 )	2023-12-18 13:20:56 +05:30
Sidharth Mohanty	9943d1e015	[chore] Remove `deployment_name` for openai embedder and update docs (#1022 )	2023-12-18 08:50:49 +05:30
Sidharth Mohanty	b348251484	[Docs] Don't need open ai key for gpt4all model (#1018 )	2023-12-16 08:44:47 +05:30
xuxiang	e719b5bac3	[Bug-fix] Fix error caused by executing Repo.clone_from twice (#1015 ) Co-authored-by: xuxiang <xuxiang@aliyun.com>	2023-12-16 08:33:07 +05:30
Sidharth Mohanty	54f43215cd	Docs for directory data loader usage (#1016 )	2023-12-16 07:59:05 +05:30
Deven Patel	b246d9823e	[Docs] Documentation updates (#1014 )	2023-12-15 17:02:50 +05:30
Deven Patel	65c8dd445b	Update README to show package downloads (#1013 )	2023-12-15 11:05:13 +05:30
Deven Patel	0efbc80ac9	[Doc update] update mistral example (#1012 )	2023-12-15 06:12:53 +05:30
Deven Patel	151746beec	[Feature] Add support for Google Gemini (#1009 )	2023-12-15 06:10:55 +05:30
Deven Patel	c0ee680546	[Improvement] Add support for min chunk size (#1007 )	2023-12-15 05:59:15 +05:30
Sidharth Mohanty	9303a1bf81	[Feature] Add support for directory loader as data source (#1008 )	2023-12-15 05:24:34 +05:30
Deshraj Yadav	d54cdc5b00	[Docs] Revamp documentation (#1010 )	2023-12-15 05:14:17 +05:30
Sidharth Mohanty	b7a44ef472	Fix multiple top level packages (#1006 )	2023-12-13 12:36:25 +05:30
Deven Patel	ae6f866901	[improvement] update web page default chunk size to 2000 (#1005 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-13 12:36:13 +05:30
Sidharth Mohanty	7910cee259	[chore] fix wrong import of LLMs from langchain (#1002 )	2023-12-11 05:41:51 +05:30
Deshraj Yadav	d66e647f99	Update package version to v0.1.32 (#1003 )	2023-12-11 05:41:17 +05:30
Deven Patel	ff4a333be7	[Bugfix] fix sitemap loader (#1000 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-07 16:32:41 -08:00
Deshraj Yadav	111749a95d	[Docs] Add documentation about getting source along with the answer (#999 )	2023-12-07 14:54:16 -08:00
Deshraj Yadav	adde398b65	Update package version to 0.1.30 (#998 )	2023-12-07 14:23:21 -08:00
Sidharth Mohanty	d8897ce356	[Feature] RSS Feed loader (#942 )	2023-12-07 14:18:35 -08:00
Deven Patel	0ea8ab228c	[Improvements] allow setting up the elasticsearch cloud instance (#997 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-07 14:17:59 -08:00
Sidharth Mohanty	d62a23edf6	Substack loader improvements (#952 ) Co-authored-by: Deshraj Yadav <deshrajdry@gmail.com>	2023-12-07 14:12:50 -08:00
Sidharth Mohanty	51ebf3439b	[New] Beehiiv loader (#963 )	2023-12-07 14:11:56 -08:00
Deven Patel	4a5ed1dd8d	Update ec query and chat function (#996 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-06 15:23:28 -08:00
Deven Patel	e84b5034ea	[Bugfix] fix return type of ec chat (#995 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-06 10:00:19 -08:00
Deven Patel	a4831d6ed9	Bump package version to 0.1.27 (#994 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-06 00:03:13 -08:00
Deven Patel	51b4966801	[Improvements] Package improvements (#993 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-05 23:42:45 -08:00
atkinsh	1d4e00ccef	local file path support for sitemap loader (#992 )	2023-12-05 19:04:20 -08:00
Deven Patel	c9fbc2e7d6	fix sitemap loader (#986 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-05 17:56:24 -08:00
Deshraj Yadav	fa34788df6	Bump package version to 0.1.26 (#991 )	2023-12-05 16:52:48 -08:00
Deven Patel	0f4f220119	Package improvements (#989 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-12-05 16:48:00 -08:00
Deven Patel	512cfc9466	[Improvement] customize add method (#988 )	2023-12-05 00:55:33 -08:00
Deshraj Yadav	541b1cb7c7	Update version to 0.1.25 (#985 )	2023-12-03 15:15:52 -08:00
Deven Patel	36af1a7615	[Improvement] improve github loader (#984 )	2023-12-01 11:24:13 -08:00
Deshraj Yadav	b02e8feeda	[BugFix] Fix issue of chunks not getting embedded in opensearch index (#983 )	2023-11-29 21:56:05 -08:00
Deven Patel	406c46e7f4	[Improvements] Add support for creating app from YAML string config (#980 )	2023-11-29 12:25:30 -08:00
Sidharth Mohanty	e35eaf1bfc	Improve deps installation by converting them to one liner (#967 )	2023-11-29 10:08:34 -08:00
Sidharth Mohanty	38426a7af1	Discord loader (#976 )	2023-11-29 10:07:05 -08:00
Deshraj Yadav	141a23fb1e	[BugFix] Skip checking thread when making sqlite connection (#978 )	2023-11-26 15:44:06 -08:00
Sidharth Mohanty	bb28569abf	Update workflow to run when required (#941 )	2023-11-24 09:29:31 -08:00
Deshraj Yadav	1df46b2bb3	[Bug fix] Fix issue of missing user directory (#975 )	2023-11-24 09:26:59 -08:00
Deshraj Yadav	58f72e1ffe	Update Azure OpenAI embedding model docs (#974 )	2023-11-23 01:45:46 -08:00
Deshraj Yadav	33409140b4	[Bug fix] Fix Azure OpenAI related issue (#973 )	2023-11-23 01:40:54 -08:00
Deshraj Yadav	f6b80e01a1	[Feature] Add support for custom streaming callback (#971 )	2023-11-22 01:06:33 -08:00
Sidharth Mohanty	798d3fcc5a	Update version to 0.1.18 (#970 )	2023-11-21 10:01:09 -08:00
Sidharth Mohanty	85f3ac428b	Update embedding_fn signature to newest chroma db's (#969 )	2023-11-21 09:42:11 -08:00
Deshraj Yadav	9fcf2130b5	[Feature] Improve github and youtube channel loader (#966 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-17 18:25:14 -08:00
Taranjeet Singh	51df00729e	Import beautifulsoup pacakge lazily. (#964 )	2023-11-17 18:19:08 -08:00
Deven Patel	023a61446f	[Feature] Improve GitHub loader (#962 )	2023-11-16 22:06:36 -08:00
Deshraj Yadav	e0b73e6a5a	[Loaders] Improve web page and sitemap loader usability (#961 )	2023-11-16 16:01:43 -08:00
Deven Patel	28460f725c	[Bugfix] fix poetry lock (#960 )	2023-11-16 13:30:38 -08:00
Deshraj Yadav	c93e49d2b8	[Bug fix] Update sleep time for substack loader and version bump (#958 )	2023-11-15 19:35:30 -08:00
Deven Patel	07fb6bee54	[Features] Add Github and Youtube Channel loaders (#957 ) Co-authored-by: Deven Patel <deven298@yahoo.com> Co-authored-by: Deshraj Yadav <deshrajdry@gmail.com>	2023-11-15 19:17:42 -08:00
Deshraj Yadav	3fa7db8420	Bump version to 0.1.13 (#956 )	2023-11-15 18:42:48 -08:00
Deven Patel	c14bd7b73b	[Improvement] fix discourse loader to avoid rate limit (#953 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-15 15:33:16 -08:00
Sidharth Mohanty	5201beaab0	Bump version to `0.1.12` (#951 )	2023-11-15 09:33:26 -08:00
Sidharth Mohanty	122313d8a5	[New] Substack loader (#949 )	2023-11-14 21:52:15 -08:00
Deven Patel	82fd595306	[Improvements] improve package ux (#950 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-14 17:53:43 -08:00
Deven Patel	95c0d47236	[Feature] Discourse Loader (#948 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-13 16:39:11 -08:00
Deven Patel	919cc74e94	[Feature] Add MySQL Loader (#920 ) Co-authored-by: Deven Patel <deven298@yahoo.com> Co-authored-by: Deshraj Yadav <deshrajdry@gmail.com>	2023-11-13 13:21:36 -08:00
Deshraj Yadav	d839991acb	[Docs] Add back sitemap loader docstring (#947 )	2023-11-13 13:08:09 -08:00
Deven Patel	539286aafd	[Feature] Add Slack Loader (#932 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-13 13:06:01 -08:00
Sidharth Mohanty	23522b7b55	Add slack_bot docker image (#933 )	2023-11-13 13:04:04 -08:00
UnMonsieur	bf3fac56e4	Refactor: Make it clear what methods are private (#946 )	2023-11-13 13:00:13 -08:00
Deshraj Yadav	a5bf8e9075	[Improvement] Parallelize loading of sitemap urls	2023-11-13 12:53:34 -08:00
Deshraj Yadav	1d31b8f7e4	[Bugfix] Fix issue of "unable to open database file" (#945 )	2023-11-13 12:37:00 -08:00
Deshraj Yadav	b144c7dccc	[Bugfix] Fix hugging face command (#944 )	2023-11-13 12:17:27 -08:00
Deshraj Yadav	1364975396	[Feature] Add support for AIAssistant (#938 )	2023-11-10 16:47:34 -08:00
Deven Patel	deaa7f50f8	[Bug Fix] fix chromadb where clause for query and delete (#937 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-10 16:04:25 -08:00
Sidharth Mohanty	744ab5156f	[Bug fix] missing dir on first init for App (#934 )	2023-11-10 10:15:04 -08:00
Sidharth Mohanty	c45413969a	[refactor] Use pipeline for bots instead of App (#936 )	2023-11-10 10:13:21 -08:00
Deshraj Yadav	b314e5e080	[bug] Fix issue of missing user directory on first init (#931 )	2023-11-09 22:29:41 -08:00
Deshraj Yadav	17129e2eaa	[Improvement] Add support for reloading history for an existing app (#930 )	2023-11-09 15:17:51 -08:00
Deven Patel	654fd8d74c	[Improvement] Use SQLite for chat memory (#910 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-09 13:56:28 -08:00
Sidharth Mohanty	9d3568ef75	Update package version to 0.1.3 (#928 )	2023-11-09 11:33:18 -08:00
Sidharth Mohanty	14712cac88	Deploy remaining bots and fix schema validation (#927 )	2023-11-09 10:44:47 -08:00
Deshraj Yadav	0d568c758b	[Feat] Add anonymous telemetry to assistant (#924 )	2023-11-09 02:07:47 -08:00
Deshraj Yadav	7c6b88c7c5	[Docs] Update docs and improve assistant api (#923 )	2023-11-09 01:16:19 -08:00
Deshraj Yadav	32c93be46e	[chore] update poetry.lock file (#922 )	2023-11-09 00:59:29 -08:00
Deven Patel	7de8d85199	[Feature] Add Postgres data loader (#918 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-08 23:50:46 -08:00
Deshraj Yadav	f7dd65a3de	[Feature] Add support for OpenAI assistants and support openai version >=1.0.0 (#921 )	2023-11-08 22:49:03 -08:00
Sidharth Mohanty	d8cdbe0041	Set `check_same_thread` false so that one App can be used in parallel (#911 )	2023-11-08 10:26:31 -08:00
Sidharth Mohanty	2b8b6d3ea9	Chunker config docs (#913 )	2023-11-08 10:25:45 -08:00
Sidharth Mohanty	936c7e389f	Deploy Full stack docker image (#914 )	2023-11-08 10:25:12 -08:00
Sidharth Mohanty	6864b4207b	Dockerize discord bot and update docs to run the bot correctly (#919 )	2023-11-08 10:24:57 -08:00
Deshraj Yadav	98eb5b54be	[Docs] Update developer documentation (#916 )	2023-11-07 19:07:47 -08:00
Deshraj Yadav	3332e6e236	[Docs] Update README (#915 )	2023-11-07 18:12:27 -08:00
Deven Patel	0533da72d7	[Improvement] add delete functionality to zilliz DB (#912 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-07 02:46:39 -08:00
Sidharth Mohanty	a1de238716	Introduce chunker config in yaml config (#907 )	2023-11-06 09:43:15 -08:00
Deshraj Yadav	f0d112254b	[version] bump package version and minor cleanup (#909 )	2023-11-05 16:34:36 -08:00
Sidharth Mohanty	830a7397ef	Add yaml config validation (#890 )	2023-11-04 22:23:55 -07:00
Deven Patel	5428765329	Beautify JSON docs (#906 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-03 13:52:33 -07:00
Sidharth Mohanty	23c912f2b7	Improve getting started page by adding steps (#904 )	2023-11-03 13:40:40 -07:00
Sidharth Mohanty	9c4b023297	Use either `embedder` or `embedding_model` as YAML key (#905 )	2023-11-03 13:40:16 -07:00
Deven Patel	53037b5ed8	[Feature Improvement] Update JSON Loader to support loading data from more sources (#898 ) Co-authored-by: Deven Patel <deven298@yahoo.com>	2023-11-03 10:00:27 -07:00
Sidharth Mohanty	e2546a653d	[chore] fix rest api docs and other minor fixes (#902 )	2023-11-03 09:40:48 -07:00
Deshraj Yadav	4b8cada873	[REST API] Change docker image name and update docs (#901 )	2023-11-03 01:08:27 -07:00
Deshraj Yadav	fa3ca1d08a	[REST API] Incorporate changes related to REST API docs (#900 )	2023-11-03 00:42:55 -07:00
Sidharth Mohanty	8dd5cb9602	Add `rest-api` example (#889 )	2023-11-03 00:32:51 -07:00