Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/boltzmannentropy/llms-osx

This list is tailored for users with Apple Silicon devices interested in experimenting with Large Language Models (LLMs).
https://github.com/boltzmannentropy/llms-osx

apple-silicon large-language-models llms lora

Last synced: 8 months ago
JSON representation

This list is tailored for users with Apple Silicon devices interested in experimenting with Large Language Models (LLMs).

Awesome Lists containing this project

README 

          
# LLMS

## Introduction

This guide is tailored for users with **Apple Silicon** devices interested in experimenting with **Large Language Models (LLMs).** It includes summaries of key repositories and resources, focusing on those optimized for Apple Silicon hardware. The guide covers a variety of topics, including Retrieval-Augmented Generation (RAG), training and fine-tuning LLMs, vision-language models (VLM), datasets, and user interfaces (UI). Each section provides links to repositories and tools, helping users explore and implement LLMs effectively on their Apple Silicon devices.

apple



### Retrieval-Augmented Generation (RAG) Repositories:



**ChatLLM.cpp**

- Repository: [chatllm.cpp](https://github.com/foldl/chatllm.cpp/blob/master/docs/rag.md)

- Summary: This repository includes documentation and code for implementing Retrieval-Augmented Generation (RAG) models using LLMs, providing guidelines and examples for integrating retrieval mechanisms with language models.



**ModelScope RAG**

- Repository: [ModelScope RAG](https://github.com/modelscope/modelscope-classroom/blob/main/LLM-tutorial/RAG%2BRerank%2BLlamaindex.ipynb)

- Summary: This Jupyter notebook provides a tutorial on implementing RAG with reranking using the Llamaindex library. It showcases how to enhance retrieval-based generation with ranking mechanisms for improved performance.



**Quad-AI RAG**

- Repository: [Quad-AI RAG](https://github.com/Quad-AI/LLM/blob/main/llama-cpp-rag%20-%20final.ipynb)

- Summary: This notebook demonstrates the implementation of RAG using llama.cpp for effective information retrieval and response generation. It includes step-by-step instructions and code snippets.



**MLX RAG GGUF**

- Repository: [MLX RAG GGUF](https://github.com/Jaykef/mlx-rag-gguf)

- Summary: This repository contains code and documentation for implementing RAG with GGUF embeddings using the MLX framework. It focuses on integrating various embedding techniques for enhanced retrieval.



**FlagOpen Embedding**

- Repository: [FlagOpen Embedding](https://github.com/FlagOpen/FlagEmbedding)

- Summary: The repository provides tools and methods for embedding generation and integration in retrieval-augmented generation systems. It includes various models and techniques for embedding tasks.



**LM Cocktail**

- Repository: [LM Cocktail](https://github.com/FlagOpen/FlagEmbedding/tree/master/LM_Cocktail)

- Summary: This project focuses on combining different embedding techniques and models to create a robust retrieval system for augmented generation tasks. It includes various configurations and examples.



### OSX Apple Silicon:



**OSX Apple Silicon Gist**

- Repository: [OSX Apple Silicon Gist](https://gist.github.com/uogbuji/ba4217487a797b9e7f951ac4b86dbe14)

- Summary: A gist providing detailed steps and configurations for running MLX models on Apple Silicon devices, including setup instructions and performance tips.



**Hugging Face MLX Models**

- Repository: [Hugging Face MLX Models](https://huggingface.co/models?library=mlx&sort=downloads)

- Summary: A collection of models optimized for MLX, available on Hugging Face. This repository provides a variety of models suitable for different ML tasks, optimized for performance and efficiency.



**MLX Examples**

- Repository: [MLX Examples](https://github.com/ml-explore/mlx-examples/issues/865)

- Summary: This repository contains example implementations and use cases for MLX models, including step-by-step guides and code snippets for various ML tasks.



**Phi-3 Vision MLX**

- Repository: [Phi-3 Vision MLX](https://github.com/JosefAlbers/Phi-3-Vision-MLX)

- Summary: A project showcasing the integration of vision models with the MLX framework, providing examples and implementation details for vision-related tasks.



**LILM**

- Repository: [LILM](https://github.com/alphrc/lilm)

- Summary: This repository focuses on implementing and optimizing language models for low-resource settings, utilizing the MLX framework for enhanced performance on Apple Silicon devices.



**ML Stable Diffusion**

- Repository: [ML Stable Diffusion](https://github.com/apple/ml-stable-diffusion/issues/8)

- Summary: A project dedicated to implementing and optimizing stable diffusion models for Apple Silicon, addressing performance issues and providing solutions for efficient execution.



**MLX LORA Fine-tuning**

- Repository: [MLX LORA Fine-tuning](https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/LORA.md#fine-tune)

- Summary: Documentation and code for fine-tuning models using LORA within the MLX framework, providing step-by-step instructions and examples.



**MLX Discussions**

- Repository: [MLX Discussions](https://github.com/ml-explore/mlx/discussions/654)

- Summary: A forum for discussing various aspects of the MLX framework, including usage tips, troubleshooting, and community contributions.



**NanoGPT MLX**

- Repository: [NanoGPT MLX](https://github.com/vithursant/nanoGPT_mlx/)

- Summary: A minimal implementation of GPT models optimized for the MLX framework, providing code and documentation for quick setup and execution.



**LM Evaluation Harness**

- Repository: [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness)

- Summary: Tools and benchmarks for evaluating language models, including standardized tests and performance metrics for various tasks.



**PikaGPT**

- Repository: [PikaGPT](https://github.com/plpxsk/pikaGPT)

- Summary: A lightweight implementation of GPT models optimized for efficiency and performance on Apple Silicon devices, providing examples and setup instructions.



**Local LLM Training Apple Silicon**

- Repository: [Local LLM Training Apple Silicon](https://github.com/GusLovesMath/Local_LLM_Training_Apple_Silicon)

- Summary: A project focusing on training local LLMs on Apple Silicon devices, providing code and documentation for setup and execution.



**MLX Notes**

- Repository: [MLX Notes](https://github.com/uogbuji/mlx-notes/tree/main)

- Summary: A collection of notes and guides for using the MLX framework, covering various aspects from setup to advanced usage.



**LORA Gist**

- Repository: [LORA Gist](https://gist.github.com/awni/773e2a12079da40a1cbc566686c84c8f)

- Summary: A gist providing detailed instructions for fine-tuning models using LORA, including code snippets and setup tips.



**Llama3 Mac Silicon Example**

- Repository: [Llama3 Mac Silicon Example](https://github.com/GusLovesMath/Llama3_MacSilicon/blob/main/Llama3_MacSilicom_Example.ipynb)

- Summary: An example notebook demonstrating the use of Llama3 models on Mac Silicon, including setup, execution, and performance tips.



**Deep Dive into AI with MLX**

- Repository: [Deep Dive into AI with MLX](https://github.com/neobundy/Deep-Dive-Into-AI-With-MLX-PyTorch/tree/master/mlx-book)

- Summary: A comprehensive guide and tutorial for diving deep into AI using the MLX framework, covering various models and use cases.



### Vision-Language Models (VLM):



**MLX Llava Finetuning**

- Repository: [MLX Llava Finetuning](https://github.com/nkasmanoff/mlx-llava-finetuning)

- Summary: A repository focused on fine-tuning vision-language models using the MLX framework, providing examples and detailed instructions.



**Bunny**

- Repository: [Bunny](https://github.com/BAAI-DCAI/Bunny)

- Summary: A project dedicated to developing and optimizing vision-language models, providing code, documentation, and examples for various tasks.



### LoRA/Training/Fine-Tuning:



**InternLM Ecosystem**

- Repository: [InternLM Ecosystem](https://github.com/InternLM/InternLM/blob/main/ecosystem/README.md)

- Summary: Documentation and code for integrating and using InternLM models within various ecosystems, focusing on training and fine-tuning.



**InternLM Fine-tuning**

- Repository: [InternLM Fine-tuning](https://github.com/InternLM/InternLM/tree/main/finetune)

- Summary: A repository providing detailed instructions and code for fine-tuning InternLM models, including setup and execution guidelines.



**ModelScope Swift**

- Repository: [ModelScope Swift](https://github.com/modelscope/swift)

- Summary: A project dedicated to optimizing and deploying models using the Swift framework, providing examples and documentation for various tasks.



**LitGPT**

- Repository: [LitGPT](https://github.com/Lightning-AI/litgpt/tree/main)

- Summary: A repository focused on implementing and optimizing GPT models using the Lightning framework, providing examples and detailed instructions for training and fine-tuning.



### Datasets:



**The Cauldron**

- Repository: [The Cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron)

- Summary: A dataset repository on Hugging Face providing a collection of datasets for training and evaluating machine learning models, including detailed descriptions and usage guidelines.



### GUI:



**LibreChat**

- Repository: [LibreChat](https://github.com/danny-avila/LibreChat)

- Summary: LibreChat is an open-source project for building and managing chat interfaces for language models, providing customizable features and integration options for various platforms.



## Docker etc

- brew install llama.cpp

- llama-cli --hf-repo reach-vb/Meta-Llama-3.1-8B-Instruct-Q6_K-GGUF --hf-file meta-llama-3.1-8b-instruct-q6_k.gguf -p "Sup?" --ctx-size 8192



The model uses ~7.0GB RAM. 



## Audio 

**MLX Powered Video Transcription** 

- Repository: https://github.com/RayFernando1337/MLX-Auto-Subtitled-Video-Generator/

- Summary: This Streamlit application allows users to upload video files and generate accurate transcripts using Apple's MLX framework



### Unsorted

- https://github.com/linyiLYi/bilibot/tree/main

- https://github.com/harperreed/photo-similarity-search

- https://github.com/hiyouga/LLaMA-Factory

- https://github.com/instructlab/instructlab

- https://github.com/modelscope/swift

-https://github.com/lyogavin/airllm/blob/main/air_llm/examples/run_on_macos.ipynb