LLMs on Your Machine: A Guide to building a Custom Assistants with AnythingLLM
Ever wanted to create your own personal ChatGPT without relying on the cloud?
Well, you're in luck!
In this article, I'll show you how to set up your very own local LLM using AnythingLLM.
I'll break down the process into easy-to-follow steps, so you can have your custom ChatGPT up and running in no time. Let's get started on this exciting journey!
Introduction
Recently I was wandering around github looking for some cool repositories which might capture my interest.Then I came across something called as the AnythingLLM which happens to be the best chatbot I have ever used.
It allows us to use the model of our choice with a very well thought interface and brilliant applications. Some of the features that I really loved about it:
- Supports custom models including GPT-4 and other open-source models.
- Allows chats with unlimited documents including PDFs and word documents.
- Flexible and customizable.
- Provides full control and privacy
- Stores all the chats on your local machine and easy access to all chat threads even if offline.
Open source community and the culture of open sourcing LLMs and other models is one of the best things I have seen this decade. For me things turned interesting when Llama3–70B and Llama3–8B were open sourced.
After testing Llama3 on an online playground on together.ai , I was blown away by the performance and really wanted to use it as my default LLM for daily purposes.
So I went to the trouble of finding a way to do this, and now I'm sharing it with you to make your life easier. 😉
Prerequisites
Before we begin, ensure that your system meets the following requirements:
- An operating system compatible with AnythingLLM (Windows, macOS, or Linux)
- Python 3.9 or higher installed on your system
Yup that's it! That's all you need to get your Llama or any other model running with AnythingLLM with APIs.
We'll be using APIs by any one of the following services. A big thanks to all of them ❤️
If you have enough computational resources i.e. a high end PC then you can run the models directly on your device by using any of the following services also:
- Ollama
- Local AI
- LM-Studio
Setting up
Now we have to get the APIs from any of the above mentioned services
1. together.ai
When you open together.ai then create an account by clicking on get started.
After completing the signup, you will land in the together.ai playground which would look something like this.
You can play around in the playground with different chat and image models. They provide a very wide variety of open source models in their playground.
Click on the profile circle on the top right corner > navigate to the settings > then a new page will appear where on the left side > click on the API Keys option and then copy and save your API KEY.
2. groq.com
When you open groq.com then create an account by clicking on get started.
After signing up click on the GroqCloud option at the bottom of the page > A new window will appear > Click on the API KEY > Create API Key > Copy and store the API Key
Now once you have the API key on any of the above services then we need to go the AnythingLLM
AnythingLLM
Now we have to go to AnythingLLM
Click on the download now link in the page as seen above and download it as per your OS (Windows, macOS, or Linux).
After the download open the file and you should see something like this
Here you have to select the one service whose API key you secured in the previous steps i.e. together.ai or groq.com
Then after selecting them enter the API Key and select the model you want to use. We can change the model later also so no worries!
Then we have to select the Embedding model > Click on the AnythingLLM Embedder > Click on next
Select any database of your choice or just select the first one if it feels overwhelming.
Now name your workspace whatever you like. You can rename it later and even create multiple workspaces as well.
Now it's all set!
Each workspace comes with a default thread and an option to create a new thread.
I have created many workspaces as per my need. I even created two workspaces i.e. one with together.ai and one with groq.com API KEY.
When you click on the (upload a document) option as shown you will see a panel appear like below.
here you can upload any number of pdfs / docs / audio / csv or even website urls to give context to the LLM and then chat with the pdf / doc / website you like.
In the data connectors tab you can even give YouTube video links and then ask questions from the LLM about the video. It also gives option to learn to ask questions about a github repository by pasting it's link.
You can even change the model that you are talking to using by clicking on the gear icon near the workspace and then going to chat settings
On selecting together.ai in the option > you can choose any model that they offer for free for a long period of time .
Like here I can select any model which I want to use as per my convience and needs.
Youtube Transcription is one of the best applications of this tool
The best part is even if you are offline you can view and access all your threads unlike online LLMs. They are also stored on your device locally so it is also secure and private.
Congratulations!
You have successfully set up your own local ChatGPT using AnythingLLM. Now you can have a personalized, efficient, and secure language processing experience.
Now that you have your own local LLM, you can explore its capabilities, customize it to your specific needs, and even contribute to the growing community of AI enthusiasts and developers.
Happy coding!
Here are some other articles that I wrote:
So that'll be all for today! Hope to see you around for the next one! Namaste 🙏