https://github.com/tushar2704/colabmixtral

Welcome to the Mixtral 8x7B offloading demo repository! This project aims to demonstrate the seamless execution of Mixtral-8x7B models on Colab or consumer desktops.
https://github.com/tushar2704/colabmixtral

data-science demo llm mistral mixtral-8x7b-instruct nlp python rag tushar2704

Last synced: 19 days ago
JSON representation

Welcome to the Mixtral 8x7B offloading demo repository! This project aims to demonstrate the seamless execution of Mixtral-8x7B models on Colab or consumer desktops.

• Host: GitHub
• URL: https://github.com/tushar2704/colabmixtral
• Owner: tushar2704
• License: mit
• Created: 2023-12-30T13:25:43.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2023-12-30T13:52:41.000Z (about 1 year ago)
• Last Synced: 2024-05-11T05:53:45.697Z (8 months ago)
• Topics: data-science, demo, llm, mistral, mixtral-8x7b-instruct, nlp, python, rag, tushar2704
• Language: Python
• Homepage: https://tushar-aggarwal.com
• Size: 23.4 KB
• Stars: 3
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Colab Mixtral 8x7B Offloading Demo

Welcome to the Mixtral 8x7B offloading demo repository! This project aims to demonstrate the seamless execution of Mixtral-8x7B models on Colab or consumer desktops.

![Screenshot 2023-12-30 191022](https://github.com/tushar2704/ColabMixtral/assets/66141195/9f3a9683-9083-4818-b15f-fbb8b4ecbb69)

## Getting Started

To explore and experience the demo, follow these steps:

1. Open the demo notebook located at `./notebooks/demo.ipynb`.

2. Follow the instructions within the notebook to run the Mixtral-8x7B offloading demo.

## Overview

This repository provides a hands-on experience for users interested in leveraging Mixtral-8x7B models for various applications. The demo notebook serves as a comprehensive guide, offering insights into the capabilities of Mixtral-8x7B and providing a platform for experimentation.

## Requirements

Make sure you have the necessary dependencies installed before running the demo. Refer to the notebook for specific requirements.

## Issues and Contributions

If you encounter any issues or have suggestions for improvement, feel free to open an issue or contribute to the project. We welcome collaboration from the community to enhance this demo.

Thank you for exploring the Mixtral 8x7B offloading demo! Enjoy experimenting with the capabilities of Mixtral-8x7B in your projects.

## Refrence:

1)https://arxiv.org/abs/2312.17238

## Upcoming application to demonstrate!