https://github.com/svngoku/Mixtral_8X7B_Agent

Deploying an open source AI Agent with Mixtral 8X7B
https://github.com/svngoku/Mixtral_8X7B_Agent

Last synced: 3 months ago
JSON representation

Deploying an open source AI Agent with Mixtral 8X7B

• Host: GitHub
• URL: https://github.com/svngoku/Mixtral_8X7B_Agent
• Owner: svngoku
• Created: 2023-12-16T19:06:14.000Z (almost 2 years ago)
• Default Branch: master
• Last Pushed: 2023-12-17T12:55:53.000Z (almost 2 years ago)
• Last Synced: 2025-01-06T00:34:20.705Z (11 months ago)
• Language: Jupyter Notebook
• Size: 10.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome_ai_agents - Mixtral_8X7B_Agent - Deploying an open source AI Agent with Mixtral 8X7B (Building / Deployment)

README

          
# Mixtral 8X7B — Deploying an *Open* AI Agent

The goal of this repo is to show how to deploy an AI agent using the new Open source model `Mistral 8X7B` on an instance (`AWS`, `RunPod`).

## 1. Create an instance (RUNPOD)

The instance must have the following characteristics:

- 160 GB VRAM

- 8 max

- 250 GB RAM  24 vCPU

## 2. Install the dependencies

```sh

!pip install -qU \

    transformers==4.36.1 \

    accelerate==0.25.0 \

    duckduckgo_search==4.1.0

```