Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/qnguyen3/chat-with-mlx
An all-in-one LLMs Chat UI for Apple Silicon Mac using MLX Framework.
https://github.com/qnguyen3/chat-with-mlx
Last synced: 3 days ago
JSON representation
An all-in-one LLMs Chat UI for Apple Silicon Mac using MLX Framework.
- Host: GitHub
- URL: https://github.com/qnguyen3/chat-with-mlx
- Owner: qnguyen3
- License: mit
- Created: 2024-02-16T13:59:06.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2024-09-06T11:47:06.000Z (4 months ago)
- Last Synced: 2025-01-01T22:07:02.037Z (10 days ago)
- Language: Python
- Homepage: https://twitter.com/stablequan
- Size: 2.49 MB
- Stars: 1,500
- Watchers: 12
- Forks: 133
- Open Issues: 28
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-LLM-resourses - chat-with-mlx
- StarryDivineSky - qnguyen3/chat-with-mlx
- alan_awesome_llm - chat-with-mlx
- alan_awesome_llm - chat-with-mlx
README
# Chat with MLX 🧑💻
[](https://badge.fury.io/py/chat-with-mlx)
[](https://pypistats.org/packages/chat-with-mlx)
[](https://github.com/qnguyen3/chat-with-mlx/blob/main/LICENSE.md)
[](https://badge.fury.io/py/chat-with-mlx)
An all-in-one Chat Playground using Apple MLX on Apple Silicon Macs.
## Features
- **Privacy-enhanced AI**: Chat with your favourite models and data securely.
- **MLX Playground**: Your all in one LLM Chat UI for Apple MLX
- **Easy Integration**: Easy integrate any HuggingFace and MLX Compatible Open-Source Models.
- **Default Models**: Llama-3, Phi-3, Yi, Qwen, Mistral, Codestral, Mixtral, StableLM (along with Dolphin and Hermes variants)
## Installation and Usage
### Easy Setup
- Install Pip
- Install: `pip install chat-with-mlx`
### Manual Pip Installation
```bash
git clone https://github.com/qnguyen3/chat-with-mlx.git
cd chat-with-mlx
python -m venv .venv
source .venv/bin/activate
pip install -e .
```
#### Manual Conda Installation
```bash
git clone https://github.com/qnguyen3/chat-with-mlx.git
cd chat-with-mlx
conda create -n mlx-chat python=3.11
conda activate mlx-chat
pip install -e .
```
#### Usage
- Start the app: `chat-with-mlx`
## Add Your Model
Please checkout the guide [HERE](ADD_MODEL.MD)
## Known Issues
- When the model is downloading by Solution 1, the only way to stop it is to hit `control + C` on your Terminal.
- If you want to switch the file, you have to manually hit STOP INDEXING. Otherwise, the vector database would add the second document to the current database.
- You have to choose a dataset mode (Document or YouTube) in order for it to work.
- **Phi-3-small** can't do streaming in completions
## Why MLX?
MLX is an array framework for machine learning research on Apple silicon,
brought to you by Apple machine learning research.
Some key features of MLX include:
- **Familiar APIs**: MLX has a Python API that closely follows NumPy. MLX
also has fully featured C++, [C](https://github.com/ml-explore/mlx-c), and
[Swift](https://github.com/ml-explore/mlx-swift/) APIs, which closely mirror
the Python API. MLX has higher-level packages like `mlx.nn` and
`mlx.optimizers` with APIs that closely follow PyTorch to simplify building
more complex models.
- **Composable function transformations**: MLX supports composable function
transformations for automatic differentiation, automatic vectorization,
and computation graph optimization.
- **Lazy computation**: Computations in MLX are lazy. Arrays are only
materialized when needed.
- **Dynamic graph construction**: Computation graphs in MLX are constructed
dynamically. Changing the shapes of function arguments does not trigger
slow compilations, and debugging is simple and intuitive.
- **Multi-device**: Operations can run on any of the supported devices
(currently the CPU and the GPU).
- **Unified memory**: A notable difference from MLX and other frameworks
is the *unified memory model*. Arrays in MLX live in shared memory.
Operations on MLX arrays can be performed on any of the supported
device types without transferring data.
## Acknowledgement
I would like to send my many thanks to:
- The Apple Machine Learning Research team for the amazing MLX library.
- LangChain and ChromaDB for such easy RAG Implementation
- All contributors
## Star History
[](https://star-history.com/#qnguyen3/chat-with-mlx&Date)