Disclaimer: The files for this tutorial are available at: our Github Tutorial Repo
If you ever wondered about how you could deploy your own version of ChatGPT with your/your company’s content, this is your time to shine: I have a wonderful solution for you.
For those who don’t know me or my project, I’m Vinnie, a software engineer and co-founder of the Talkd.AI project. We are a simple and easy to deploy app that allows any developer with some context of Docker and Python to deploy their own RAG (Retrieval-augmented Generation) the name of the type of applications that use a (vector) database and your custom content to return data from and to your LLM.
What you will need to have in order to use our Dialog server?
A computer/server with a docker instance up and running
A CSV file with some contents from your company (we will specify the format later on)
A TOML file for setting up your model and prompts
An OpenAI API Key (here is a tutorial to generate one)
Getting started
In order to start your own deployment of our Dialog server, you will need to use the following docker-compose file:
db:
image: pgvector/pgvector:0.6.2-pg15
restart: always
ports:
– ‘5432:5432′
environment:
POSTGRES_USER: talkdai
POSTGRES_PASSWORD: talkdai
POSTGRES_DB: talkdai
volumes:
– ./etc/db-extensions.sql:/docker-entrypoint-initdb.d/db-extensions.sql
healthcheck:
test: [“CMD”, “pg_isready”, “-d”, “talkdai”, “-U”, “talkdai”]
interval: 10s
timeout: 5s
retries: 5
dialog:
image: ghcr.io/talkdai/dialog:latest
volumes:
– ./data/:/app/data/
ports:
– ‘8000:8000′
depends_on:
db:
condition: service_healthy
environment:
– PORT=8000
– DATABASE_URL=postgresql://talkdai:talkdai@db:5432/talkdai
– OPENAI_API_KEY=sk-your-openai-api-key
– STATIC_FILE_LOCATION=/app/static
– DIALOG_DATA_PATH=../data/your.csv
– PROJECT_CONFIG=../data/your.toml
PGVector – Our first service
The first service of this docker-compose file is a PostgreSQL instance equipped with pgvector, our vector database implementation of choice since it’s already inside a major open-source database vendor.
On this service, we’ve configured a volume mapping to a file, this file’s content is the following:
This file only enables the vector plugin if it doesn’t exist already in your database. If you are using another implementation of a PGVector Docker image that already runs this command, you can ignore this volume mounting.
Second Service – Dialog Server
As you can see, the second service is our Dialog Server, it’s our implementation of the RAG approach built on top of FastAPI, LangChain and PGVector Python extension.
We also have a volume mapping inside this service:
– ./data/:/app/data/
This volume mapping allow us to provide our prompt information and your custom content in the CSV format.
The CSV file for loading your own ChatGPT must have the following columns:
category – The category of that content
subcategory – The subcategory of that content
question – The question/title of that content
content – The content in itself, this content will generate the embeddings for our database.
On our example, we will add some questions regarding sports:
faq,football,”Whats your favorite soccer team”,”My favorite soccer team is Palmeiras, from Brazil.”
faq,football,”Whats your favorite soccer player”,”My favorite soccer player is Neymar, from Brazil.”
Inside our prompt file (the .toml one I’ve mentioned before), we will configure our prompt and how the bot will speak, which model and temperature it will use and a fallback message in order it can generate answers on topics that it didn’t find a content for a certain user message.
model = “gpt-3.5-turbo-1106”
temperature = 0.1
[prompt]
header = “””You are Robert, an AI bot that answers any questions that a user may have.”””
question_signalizer = “Answer the following question:”
suggested = “””Just answer the following question, don’t change context and don’t reply using more than 50 words.”””
fallback = “””
Your a nice bot, say something nice to the user and try to help him with his question, but also say to the user that you don’t know totally about the content he asked.
“””
Finishing the setup
With all of these files saved inside the ./data/ directory, we are able to do the last part of the setup: setting our environment variables in order to deploy our bot!
The environment variables that must be filled are:
PORT – the Port where the web server is allowed to be started on (we recommend using 8000, but you can set your own)
DATABASE_URL – the database URL (like postgresql://talkdai:talkdai@db:5432/talkdai)
OPENAI_API_KEY – The generated OpenAI API Key
DIALOG_DATA_PATH – The relative path from /app/ for the data CSV file you mapped inside your docker compose volume. (Our standard is to use the folder /app/data to host this file, so the value of this variable must be: ../data/your.csv
PROJECT_CONFIG – The Prompt file location. The explanation is the same as above, but for the toml file. In this tutorial, we will set this as ../data/your.toml
Final step: Deploying and running
Now that we finally have our setup done, we can start our Dialog Server instance by running:
You will need to wait a while until it builds everything, get our pgvector instance up and running and create the embeddings for your custom content.
When it finishes, your docker-compose log will look like this:
To access the API Swagger, you will go in your browser and use the following URL (or the equivalent for your setup): http://localhost:8000/docs.
If everything is fine, the following page must have loaded and you can now start asking questions to your own ChatGPT instance.
Using the /ask endpoint, you will be able to send any message you want the ChatGPT to answer, such as Whats your NBA team of heart?:
And after hitting send, the answer comes:
Conclusion
If you follow through this tutorial, you will be able to deploy your own ChatGPT with custom content for your company or your website.
Leave a like in this post if you enjoyed and share it with your friends.
Also, if you want more information on dialog and talkd.ai, access our GitHub page: https://github.com/talkdai/
