Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/lakshaybhushan/nextjs-nvidia-chatbot
An open-source AI chatbot app template built with Next.js, the Vercel AI SDK and the newly released NVIDIA NIM API Inference.
https://github.com/lakshaybhushan/nextjs-nvidia-chatbot
ai nextjs14 nvidia nvidia-nim rsc vercel-ai vercel-ai-sdk
Last synced: about 5 hours ago
JSON representation
An open-source AI chatbot app template built with Next.js, the Vercel AI SDK and the newly released NVIDIA NIM API Inference.
- Host: GitHub
- URL: https://github.com/lakshaybhushan/nextjs-nvidia-chatbot
- Owner: lakshaybhushan
- Created: 2024-07-08T18:18:44.000Z (4 months ago)
- Default Branch: yoda
- Last Pushed: 2024-07-12T13:28:14.000Z (4 months ago)
- Last Synced: 2024-07-12T14:56:02.727Z (4 months ago)
- Topics: ai, nextjs14, nvidia, nvidia-nim, rsc, vercel-ai, vercel-ai-sdk
- Language: TypeScript
- Homepage: https://nvidia-nim.vercel.app
- Size: 741 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Next.js AI Chatbot — NVIDIA NIM + Vercel AI SDK
An open-source AI chatbot app template built with Next.js, the Vercel AI SDK and NVIDIA NIM.
## Features
- [Next.js](https://nextjs.org/) 14 App Router
- React Server Components (RSCs) for better performance
- [NVIDIA NIM](https://build.nvidia.com/explore/discover/) API Inference
- [Vercel AI](https://sdk.vercel.ai/) SDK for streaming chat responses
- [shadcn/ui](https://ui.shadcn.com/) for UI components
- [Tailwind CSS](https://tailwindcss.com/) for styling and design
- custom rate limiter for server actions
- [Sonner](https://sonner.emilkowal.ski/) for beautiful toast notifications
- [Vercel OG](https://vercel.com/docs/functions/og-image-generation) for open graph images
## How It Works?
This template uses the NVIDIA NIM API to fetch the models and make inferences. The Vercel AI SDK is used to stream the responses from the server to the client in real-time.
## Why custom rate limiter?
NVIDIA NIM provides 1000 credits for free to every new user. So, I've implemented a custom rate limiter to prevent the users from exceeding the limit. The rate limiter is set to 10 request per hour per IP address. You can change the rate limiter settings in the [ratelimit.ts](lib/ratelimit.ts) file when deploying your own version of this template.
## Models Available via NVIDIA NIM
I've only included text-to-text models in this template. You can easily add more models by following the instructions in the [NVIDIA NIM documentation](https://build.nvidia.com/docs/nim/).
The models available in this template are:
**Google**
- `gemma-2b`
- `gemma-2-9b-it`
- `gemma-2-27b-it`
**Meta**
- `llama3-8b-instruct`
- `llama3-70b-instruct`
**NVIDIA**
- `llama3-chatqa-1.5-8b`
- `llama3-chatqa-1.5-70b`
- `nemotron-4-340b-instruct`
**IBM**
- `granite-8b-code-instruct`
- `granite-34b-code-instruct`
> `Mistral AI` and many other models available via NVIDIA NIM are not working with the Vercel AI SDK at the moment. So, I've excluded them from this template. However, you can still use them with the NVIDIA NIM API directly.
## Deploy Your Own
You can deploy your own version of this template with Vercel by clicking the button below.
[](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Flakshaybhushan%2Fnextjs-nvidia-chatbot&env=NVIDIA_NIM_API_KEY&demo-title=Next.js%20AI%20Chatbot%20%E2%80%94%20NVIDIA%20NIM%20%2B%20Vercel%20AI%20SDK&demo-description=A%20chatbot%20demo%20built%20with%20Next.js%2C%20NVIDIA%20NIM%2C%20and%20Vercel%20AI%20SDK&demo-url=https%3A%2F%2Fnvidia-nim.vercel.app%2F&demo-image=https%3A%2F%2Fnvidia-nim.vercel.app%2Fog.png)
## Local Development
First, you will need to use the environment variables [defined in `.env.example`](.env.example) to create a `.env.local` file in the root of the project. And make sure not to commit your `.env.local` file to the repository.
```bash
NVIDIA_NIM_API_KEY=
```
To get the NVIDIA NIM API key, you need to sign up on the [NVIDIA NIM website](https://build.nvidia.com/explore/discover/).
Then clone the repository and install the dependencies. This project uses `bun` as the package manager.
```bash
bun install
```
Run the development server:
```bash
bun dev
```
Now the app should be running at [http://localhost:3000](http://localhost:3000).
## Contributing
Contributions are welcome! Feel free to open an issue or submit a pull request if you have any ideas or suggestions.
## License
I don't know what to put here. I'm not a lawyer. Use this template however you want. It's open-source and free to use.