https://github.com/leliuga/cohere-configurations
Co:Here Inference configurations
https://github.com/leliuga/cohere-configurations
alpaca cohere falcon gemma gpt4all inference llama2 llm mistral mixtral phi-2 transformers
Last synced: 3 months ago
JSON representation
Co:Here Inference configurations
- Host: GitHub
- URL: https://github.com/leliuga/cohere-configurations
- Owner: leliuga
- License: mpl-2.0
- Created: 2023-12-05T08:37:51.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-27T18:28:31.000Z (over 1 year ago)
- Last Synced: 2025-09-30T07:35:02.345Z (3 months ago)
- Topics: alpaca, cohere, falcon, gemma, gpt4all, inference, llama2, llm, mistral, mixtral, phi-2, transformers
- Language: Go
- Homepage:
- Size: 9.48 MB
- Stars: 10
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
README
LLM Inference configurations👋
===============================
## Description
This repository contains the inference configurations for the Co:Here
### Supported Transformer Architectures by Co:Here
- [X] Alpaca
- [X] Aquila 1 & 2
- [X] Baichuan 1 & 2 + derivations
- [X] Bert & Nomic-Bert
- [X] Bloom
- [X] Chinese LLaMA / Alpaca and Chinese LLaMA-2 / Alpaca-2
- [X] CodeShell
- [X] Deepseek
- [X] Falcon
- [X] GPT-2 && GPT-J && GPT-NeoX
- [X] Gemma
- [X] InternLM2
- [X] Koala
- [X] LLaMA 1 & 2
- [X] MPT
- [X] Mamba
- [X] MiniCPM
- [X] Mistral v0.1 & v0.2
- [X] Mixtral MoE
- [X] Orion 14B
- [X] OpenBuddy 🐶 (Multilingual)
- [X] Phi 1 & 1.5 & 2
- [X] Persimmon 8B
- [X] PLaMo 13B
- [X] Pygmalion/Metharme
- [X] Qwen 1 & 2
- [X] Refact
- [X] StableLM
- [X] Starcoder 1 & 2
- [X] Vigogne (French)
- [X] Vicuna
- [X] WizardLM
- [X] Yi
Multimodal
- [x] BakLLaVA
- [x] LLaVA 1.5 & 1.6
- [x] MobileVLM 1.7B & 3B
- [x] Obsidian
- [x] ShareGPT4V
- [x] Yi-VL
### Supported platforms
- [X] linux/amd64
- [ ] linux/arm64
## Requirements
1. [Nerdctl](https://github.com/containerd/nerdctl) or [Podman](https://podman.io/getting-started/installation) or [Docker](https://docs.docker.com/get-docker/)
3. [Using GPUs inside containers](https://github.com/containerd/nerdctl/blob/main/docs/gpu.md)
## Usage
``` bash
nerdctl run -d --name cohere --rm --gpus all -v `pwd`/models/:/app/models -p 3000:3000 ghcr.io/leliuga/cohere run [:, ...]
# example
nerdctl run -d --name cohere --rm --gpus all -v `pwd`/models/:/app/models -p 3000:3000 ghcr.io/leliuga/cohere run Llama-2-7B-32K-Instruct:Q4_0
```
## Supported Models (1008)
| ID | Variants | Vocab Size | Context Size | Embedding Size | Read more |
| ------- | ------------- | -------------- | ---------------- | ------------------ | ------------------------------------- |
| 13B-BlueMethod | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/13B-BlueMethod/README.md) |
| 13B-Chimera | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/13B-Chimera/README.md) |
| 13B-HyperMantis | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/13B-HyperMantis/README.md) |
| 13B-Legerdemain-L2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/13B-Legerdemain-L2/README.md) |
| 13B-Ouroboros | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/13B-Ouroboros/README.md) |
| 13B-Thorns-L2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32032 | 4096 | 5120 | [README.md](models/13B-Thorns-L2/README.md) |
| 30B-Epsilon | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/30B-Epsilon/README.md) |
| 30B-Lazarus | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/30B-Lazarus/README.md) |
| 8x7B-MoE-test-NOT-MIXTRAL | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 151936 | 32768 | 4096 | [README.md](models/8x7B-MoE-test-NOT-MIXTRAL/README.md) |
| ALMA-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/ALMA-13B/README.md) |
| ALMA-13B-Pretrain | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/ALMA-13B-Pretrain/README.md) |
| ALMA-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/ALMA-7B/README.md) |
| ALMA-7B-Pretrain | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/ALMA-7B-Pretrain/README.md) |
| ANIMA-Phi-Neptune-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/ANIMA-Phi-Neptune-Mistral-7B/README.md) |
| ARIA-70B-V2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/ARIA-70B-V2/README.md) |
| Aetheria-L2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Aetheria-L2-70B/README.md) |
| Airoboros-33B-2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 6656 | [README.md](models/Airoboros-33B-2.1/README.md) |
| Airoboros-L2-13B-2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Airoboros-L2-13B-2.1/README.md) |
| Airoboros-L2-13B-2.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Airoboros-L2-13B-2.2/README.md) |
| Airoboros-L2-13B-2_1-YaRN-64K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 65536 | 5120 | [README.md](models/Airoboros-L2-13B-2_1-YaRN-64K/README.md) |
| Airoboros-L2-13B-3.1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Airoboros-L2-13B-3.1.1/README.md) |
| Airoboros-L2-70B-2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Airoboros-L2-70B-2.1/README.md) |
| Airoboros-L2-70B-2.1-Creative | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Airoboros-L2-70B-2.1-Creative/README.md) |
| Airoboros-L2-70B-3.1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Airoboros-L2-70B-3.1.2/README.md) |
| Airoboros-L2-70B-GPT4-m2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Airoboros-L2-70B-GPT4-m2.0/README.md) |
| Airoboros-L2-70b-2.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Airoboros-L2-70b-2.2/README.md) |
| Airoboros-L2-70b-2.2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Airoboros-L2-70b-2.2.1/README.md) |
| Airoboros-L2-7B-2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Airoboros-L2-7B-2.1/README.md) |
| Airoboros-L2-7B-2.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Airoboros-L2-7B-2.2/README.md) |
| Airoboros-M-7B-3.1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Airoboros-M-7B-3.1.2/README.md) |
| Airoboros-c34B-2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/Airoboros-c34B-2.1/README.md) |
| Airoboros-c34B-2.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/Airoboros-c34B-2.2/README.md) |
| Airoboros-c34B-3.1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/Airoboros-c34B-3.1.2/README.md) |
| Airochronos-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Airochronos-L2-13B/README.md) |
| Airolima-Chronos-Grad-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Airolima-Chronos-Grad-L2-13B/README.md) |
| Akins-3B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 4096 | 2560 | [README.md](models/Akins-3B/README.md) |
| AlpacaCielo2-7B-8K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/AlpacaCielo2-7B-8K/README.md) |
| Amber | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Amber/README.md) |
| AmberChat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/AmberChat/README.md) |
| Amethyst-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Amethyst-13B/README.md) |
| Ana-v1-m7 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Ana-v1-m7/README.md) |
| AppleSauce-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/AppleSauce-L2-13B/README.md) |
| AquilaChat2-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 100008 | 4096 | 6144 | [README.md](models/AquilaChat2-34B/README.md) |
| AquilaChat2-34B-16K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 100008 | 16384 | 6144 | [README.md](models/AquilaChat2-34B-16K/README.md) |
| Arithmo-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Arithmo-Mistral-7B/README.md) |
| Asclepius-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/Asclepius-13B/README.md) |
| AshhLimaRP-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/AshhLimaRP-Mistral-7B/README.md) |
| Astrid-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Astrid-Mistral-7B/README.md) |
| Athena-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Athena-v1/README.md) |
| Athena-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Athena-v2/README.md) |
| Athena-v3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Athena-v3/README.md) |
| Athena-v4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Athena-v4/README.md) |
| Athnete-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Athnete-13B/README.md) |
| Augmental-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Augmental-13B/README.md) |
| Augmental-13B-v1.50_A | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Augmental-13B-v1.50_A/README.md) |
| Augmental-13B-v1.50_B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Augmental-13B-v1.50_B/README.md) |
| Augmental-ReMM-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Augmental-ReMM-13B/README.md) |
| Augmental-Unholy-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Augmental-Unholy-13B/README.md) |
| Aurora-Nights-103B-v1.0 | Q2_K Q3_K_M Q3_K_S | 32000 | 4096 | 8192 | [README.md](models/Aurora-Nights-103B-v1.0/README.md) |
| Aurora-Nights-70B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Aurora-Nights-70B-v1.0/README.md) |
| Autolycus-Mistral_7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Autolycus-Mistral_7B/README.md) |
| Bagel-Hermes-2x34b | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K | 64000 | 200000 | 7168 | [README.md](models/Bagel-Hermes-2x34b/README.md) |
| Barcenas-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Barcenas-Mistral-7B/README.md) |
| BerrySauce-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/BerrySauce-L2-13B/README.md) |
| Beyonder-4x7B-v2 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/Beyonder-4x7B-v2/README.md) |
| BigPlap-8x20B | Q2_K | 32000 | 32768 | 5120 | [README.md](models/BigPlap-8x20B/README.md) |
| BruinsV2-OpHermesNeu-11B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/BruinsV2-OpHermesNeu-11B/README.md) |
| CAMEL-13B-Combined-Data | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/CAMEL-13B-Combined-Data/README.md) |
| CAMEL-13B-Role-Playing-Data | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/CAMEL-13B-Role-Playing-Data/README.md) |
| CAMEL-33B-Combined-Data | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/CAMEL-33B-Combined-Data/README.md) |
| CaPlatTessDolXaBoros-Yi-34B-200K-DARE-Ties-HighDensity | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/CaPlatTessDolXaBoros-Yi-34B-200K-DARE-Ties-HighDensity/README.md) |
| CalliopeDS-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/CalliopeDS-L2-13B/README.md) |
| Camel-Platypus2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Camel-Platypus2-13B/README.md) |
| Camel-Platypus2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Camel-Platypus2-70B/README.md) |
| CapyTessBorosYi-34B-200K-DARE-Ties | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/CapyTessBorosYi-34B-200K-DARE-Ties/README.md) |
| Capybara-Tess-Yi-34B-200K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Capybara-Tess-Yi-34B-200K/README.md) |
| Capybara-Tess-Yi-34B-200K-DARE-Ties | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Capybara-Tess-Yi-34B-200K-DARE-Ties/README.md) |
| CapybaraHermes-2.5-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/CapybaraHermes-2.5-Mistral-7B/README.md) |
| CarbonVillain-en-10.7B-v4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/CarbonVillain-en-10.7B-v4/README.md) |
| Carl-Llama-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Carl-Llama-2-13B/README.md) |
| Cat-13B-0.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Cat-13B-0.5/README.md) |
| CausalLM-14B | Q4_0 Q4_1 Q5_0 Q5_1 Q8_0 | 152064 | 8192 | 5120 | [README.md](models/CausalLM-14B/README.md) |
| CausalLM-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 151936 | 8192 | 4096 | [README.md](models/CausalLM-7B/README.md) |
| ChatAYT-Lora-Assamble-Marcoroni | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/ChatAYT-Lora-Assamble-Marcoroni/README.md) |
| Chinese-Alpaca-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 55296 | 4096 | 5120 | [README.md](models/Chinese-Alpaca-2-13B/README.md) |
| Chinese-Alpaca-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 55296 | 4096 | 4096 | [README.md](models/Chinese-Alpaca-2-7B/README.md) |
| Chinese-Llama-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 55296 | 4096 | 5120 | [README.md](models/Chinese-Llama-2-13B/README.md) |
| Chinese-Llama-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 55296 | 4096 | 4096 | [README.md](models/Chinese-Llama-2-7B/README.md) |
| Chronoboros-33B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/Chronoboros-33B/README.md) |
| Chronoboros-Grad-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Chronoboros-Grad-L2-13B/README.md) |
| Chronohermes-Grad-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Chronohermes-Grad-L2-13B/README.md) |
| Chronolima-Airo-Grad-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Chronolima-Airo-Grad-L2-13B/README.md) |
| Chronomaid-Storytelling-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Chronomaid-Storytelling-13B/README.md) |
| Chronorctypus-Limarobormes-13b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Chronorctypus-Limarobormes-13b/README.md) |
| Chronos-70B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Chronos-70B-v2/README.md) |
| Chronos-Beluga-v2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Chronos-Beluga-v2-13B/README.md) |
| Chronos-Hermes-13b-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32032 | 4096 | 5120 | [README.md](models/Chronos-Hermes-13b-v2/README.md) |
| Chupacabra-7B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Chupacabra-7B-v2/README.md) |
| Chupacabra-8x7B-MoE | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Chupacabra-8x7B-MoE/README.md) |
| Claire-7B-0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 65024 | 0 | 4544 | [README.md](models/Claire-7B-0.1/README.md) |
| Clover3-17B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Clover3-17B/README.md) |
| Code-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Code-13B/README.md) |
| Code-290k-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Code-290k-13B/README.md) |
| Code-33B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/Code-33B/README.md) |
| CodeBooga-34B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/CodeBooga-34B-v0.1/README.md) |
| CodeFuse-CodeLlama-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/CodeFuse-CodeLlama-34B/README.md) |
| CodeLlama-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32016 | 16384 | 5120 | [README.md](models/CodeLlama-13B/README.md) |
| CodeLlama-13B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32016 | 16384 | 5120 | [README.md](models/CodeLlama-13B-Instruct/README.md) |
| CodeLlama-13B-Python | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 5120 | [README.md](models/CodeLlama-13B-Python/README.md) |
| CodeLlama-13B-oasst-sft-v10 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32032 | 16384 | 5120 | [README.md](models/CodeLlama-13B-oasst-sft-v10/README.md) |
| CodeLlama-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/CodeLlama-34B/README.md) |
| CodeLlama-34B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/CodeLlama-34B-Instruct/README.md) |
| CodeLlama-34B-Python | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/CodeLlama-34B-Python/README.md) |
| CodeLlama-70B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32016 | 4096 | 8192 | [README.md](models/CodeLlama-70B-Instruct/README.md) |
| CodeLlama-70B-Python | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32016 | 4096 | 8192 | [README.md](models/CodeLlama-70B-Python/README.md) |
| CodeLlama-70B-hf | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32016 | 16384 | 8192 | [README.md](models/CodeLlama-70B-hf/README.md) |
| CodeLlama-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32016 | 16384 | 4096 | [README.md](models/CodeLlama-7B/README.md) |
| CodeLlama-7B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32016 | 16384 | 4096 | [README.md](models/CodeLlama-7B-Instruct/README.md) |
| CodeLlama-7B-Python | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 4096 | [README.md](models/CodeLlama-7B-Python/README.md) |
| CodeNinja-1.0-OpenChat-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/CodeNinja-1.0-OpenChat-7B/README.md) |
| CodeUp-Alpha-13B-HF | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/CodeUp-Alpha-13B-HF/README.md) |
| CodeUp-Llama-2-13B-Chat-HF | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/CodeUp-Llama-2-13B-Chat-HF/README.md) |
| CollectiveCognition-v1-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/CollectiveCognition-v1-Mistral-7B/README.md) |
| CollectiveCognition-v1.1-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/CollectiveCognition-v1.1-Mistral-7B/README.md) |
| CollectiveCognition-v1.1-Mistral-7B-dare-0.85 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/CollectiveCognition-v1.1-Mistral-7B-dare-0.85/README.md) |
| Cosmosis-3x34B | Q2_K Q3_K_M Q4_0 | 64000 | 200000 | 7168 | [README.md](models/Cosmosis-3x34B/README.md) |
| DPOpenHermes-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/DPOpenHermes-7B/README.md) |
| DPOpenHermes-7B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/DPOpenHermes-7B-v2/README.md) |
| Dans-AdventurousWinds-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Dans-AdventurousWinds-7B/README.md) |
| Dans-AdventurousWinds-Mk2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Dans-AdventurousWinds-Mk2-7B/README.md) |
| Dans-TotSirocco-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Dans-TotSirocco-7B/README.md) |
| DareVox-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/DareVox-7B/README.md) |
| DaringMaid-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/DaringMaid-13B/README.md) |
| DaringMaid-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/DaringMaid-20B/README.md) |
| Dawn-v2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Dawn-v2-70B/README.md) |
| Deacon-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/Deacon-34B/README.md) |
| DiscoLM-120b | Q2_K | 32032 | 4096 | 8192 | [README.md](models/DiscoLM-120b/README.md) |
| DiscoLM-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32032 | 8192 | 8192 | [README.md](models/DiscoLM-70B/README.md) |
| DiscoLM_German_7b_v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/DiscoLM_German_7b_v1/README.md) |
| Dolphin-2.1-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32002 | 4096 | 8192 | [README.md](models/Dolphin-2.1-70B/README.md) |
| Dolphin-Llama-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/Dolphin-Llama-13B/README.md) |
| Dolphin-Llama2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Dolphin-Llama2-7B/README.md) |
| Dolphin2.1-OpenOrca-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Dolphin2.1-OpenOrca-7B/README.md) |
| Dr_Samantha-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 4096 | [README.md](models/Dr_Samantha-7B/README.md) |
| Echidna-13B-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Echidna-13B-v0.2/README.md) |
| Echidna-13B-v0.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Echidna-13B-v0.3/README.md) |
| Emerhyst-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Emerhyst-13B/README.md) |
| Emerhyst-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Emerhyst-20B/README.md) |
| EstopianMaid-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/EstopianMaid-13B/README.md) |
| Etheria-55b-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K | 64002 | 200000 | 7168 | [README.md](models/Etheria-55b-v0.1/README.md) |
| Euryale-1.3-L2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Euryale-1.3-L2-70B/README.md) |
| Euryale-1.4-L2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Euryale-1.4-L2-70B/README.md) |
| Euryale-Inverted-L2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Euryale-Inverted-L2-70B/README.md) |
| Euryale-L2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Euryale-L2-70B/README.md) |
| Everyone-Coder-33B-Base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 7168 | [README.md](models/Everyone-Coder-33B-Base/README.md) |
| Everyone-Coder-4x7b-Base | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Everyone-Coder-4x7b-Base/README.md) |
| EverythingLM-13B-16K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 5120 | [README.md](models/EverythingLM-13B-16K/README.md) |
| EverythingLM-13B-V3-16K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 5120 | [README.md](models/EverythingLM-13B-V3-16K/README.md) |
| EverythingLM-13b-V2-16K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 5120 | [README.md](models/EverythingLM-13b-V2-16K/README.md) |
| Falkor-8x7B-MoE | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Falkor-8x7B-MoE/README.md) |
| FashionGPT-70B-V1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/FashionGPT-70B-V1.1/README.md) |
| FashionGPT-70B-v1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/FashionGPT-70B-v1.2/README.md) |
| Fennec-Mixtral-8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Fennec-Mixtral-8x7B/README.md) |
| Ferret_7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Ferret_7B/README.md) |
| Firefly-Llama2-13B-v1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Firefly-Llama2-13B-v1.2/README.md) |
| FlatDolphinMaid-8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/FlatDolphinMaid-8x7B/README.md) |
| FlatOrcamaid-13B-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/FlatOrcamaid-13B-v0.2/README.md) |
| Free_Sydney_V2_13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Free_Sydney_V2_13B/README.md) |
| Free_Sydney_V2_Mistral_7b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Free_Sydney_V2_Mistral_7b/README.md) |
| Frostwind-10.7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Frostwind-10.7B-v1/README.md) |
| FusionNet_34Bx2_MoE | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K | 64000 | 32768 | 7168 | [README.md](models/FusionNet_34Bx2_MoE/README.md) |
| GEITje-7B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/GEITje-7B-chat/README.md) |
| GOAT-70B-Storytelling | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/GOAT-70B-Storytelling/README.md) |
| GPlatty-30B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/GPlatty-30B/README.md) |
| Gale-medium-init-3B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Gale-medium-init-3B/README.md) |
| Garrulus | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Garrulus/README.md) |
| Generate_Question_Mistral_7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Generate_Question_Mistral_7B/README.md) |
| Genz-70b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Genz-70b/README.md) |
| GodziLLa2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/GodziLLa2-70B/README.md) |
| Goliath-longLORA-120b-rope8-32k-fp16 | Q2_K | 32000 | 4096 | 8192 | [README.md](models/Goliath-longLORA-120b-rope8-32k-fp16/README.md) |
| Guanaco-13B-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Guanaco-13B-Uncensored/README.md) |
| Guanaco-7B-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Guanaco-7B-Uncensored/README.md) |
| HamSter-0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/HamSter-0.1/README.md) |
| Helion-4x34B | Q2_K | 64000 | 200000 | 7168 | [README.md](models/Helion-4x34B/README.md) |
| Hermes-Trismegistus-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Hermes-Trismegistus-Mistral-7B/README.md) |
| HermesLimaRP-L2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/HermesLimaRP-L2-7B/README.md) |
| Hexoteric-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Hexoteric-7B/README.md) |
| HornyEchidna-13B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/HornyEchidna-13B-v0.1/README.md) |
| Huginn-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Huginn-13B/README.md) |
| Huginn-13B-v4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Huginn-13B-v4/README.md) |
| Huginn-13B-v4.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Huginn-13B-v4.5/README.md) |
| Huginn-22B-Prototype | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 6656 | [README.md](models/Huginn-22B-Prototype/README.md) |
| Huginn-v3-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Huginn-v3-13B/README.md) |
| Iambe-20B-DARE | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Iambe-20B-DARE/README.md) |
| Iambe-RP-DARE-20B-DENSE | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Iambe-RP-DARE-20B-DENSE/README.md) |
| Iambe-RP-cDPO-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Iambe-RP-cDPO-20B/README.md) |
| Iambe-RP-v3-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Iambe-RP-v3-20B/README.md) |
| Iambe-Storyteller-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Iambe-Storyteller-20B/README.md) |
| Inairtra-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Inairtra-7B/README.md) |
| Inkbot-13B-4k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Inkbot-13B-4k/README.md) |
| Inkbot-13B-8k-0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/Inkbot-13B-8k-0.2/README.md) |
| Instruct_Mixtral-8x7B-v0.1_Dolly15K | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Instruct_Mixtral-8x7B-v0.1_Dolly15K/README.md) |
| JanniesBasedLigma-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/JanniesBasedLigma-L2-13B/README.md) |
| KAI-7B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/KAI-7B-Instruct/README.md) |
| KAI-7B-beta | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/KAI-7B-beta/README.md) |
| KafkaLM-70B-German-V0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/KafkaLM-70B-German-V0.1/README.md) |
| Kaori-70B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Kaori-70B-v1/README.md) |
| Karen_TheEditor_V2_CREATIVE_Mistral_7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Karen_TheEditor_V2_CREATIVE_Mistral_7B/README.md) |
| Karen_TheEditor_V2_STRICT_Mistral_7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Karen_TheEditor_V2_STRICT_Mistral_7B/README.md) |
| Karen_theEditor_13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/Karen_theEditor_13B/README.md) |
| Kimiko-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Kimiko-Mistral-7B/README.md) |
| Kuchiki-1.1-L2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Kuchiki-1.1-L2-7B/README.md) |
| Kuchiki-L2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Kuchiki-L2-7B/README.md) |
| Kunoichi-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/Kunoichi-7B/README.md) |
| LLaMA-Pro-8B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/LLaMA-Pro-8B/README.md) |
| LLaMA-Pro-8B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/LLaMA-Pro-8B-Instruct/README.md) |
| LLaMA2-13B-Estopia | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/LLaMA2-13B-Estopia/README.md) |
| LLaMA2-13B-Psyfighter2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/LLaMA2-13B-Psyfighter2/README.md) |
| LLaMA2-13B-Tiefighter | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/LLaMA2-13B-Tiefighter/README.md) |
| LLaMA2-13B-TiefighterLR | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/LLaMA2-13B-TiefighterLR/README.md) |
| LLaMA_2_13B_SFT_v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/LLaMA_2_13B_SFT_v1/README.md) |
| LMCocktail-10.7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/LMCocktail-10.7B-v1/README.md) |
| LMCocktail-phi-2-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 51200 | 0 | 0 | [README.md](models/LMCocktail-phi-2-v1/README.md) |
| LUNA-SOLARkrautLM-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/LUNA-SOLARkrautLM-Instruct/README.md) |
| Lelantos-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/Lelantos-7B/README.md) |
| Lelantos-Maid-DPO-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Lelantos-Maid-DPO-7B/README.md) |
| Lemur-70B-Chat-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32005 | 4096 | 8192 | [README.md](models/Lemur-70B-Chat-v1/README.md) |
| Leo-Mistral-Hessianai-7B-Chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32128 | 32768 | 4096 | [README.md](models/Leo-Mistral-Hessianai-7B-Chat/README.md) |
| LeoScorpius-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/LeoScorpius-7B/README.md) |
| LeoScorpius-GreenNode-Alpaca-7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/LeoScorpius-GreenNode-Alpaca-7B-v1/README.md) |
| LeoScorpius-GreenNode-Platypus-7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/LeoScorpius-GreenNode-Platypus-7B-v1/README.md) |
| Lewd-Sydney-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Lewd-Sydney-20B/README.md) |
| Lila-103B-L2 | Q2_K Q3_K_M Q3_K_S | 32000 | 4096 | 8192 | [README.md](models/Lila-103B-L2/README.md) |
| Lila-70B-L2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Lila-70B-L2/README.md) |
| Llama-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Llama-2-13B/README.md) |
| Llama-2-13B-Chat-Dutch | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Llama-2-13B-Chat-Dutch/README.md) |
| Llama-2-13B-German-Assistant-v4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 37632 | 2048 | 5120 | [README.md](models/Llama-2-13B-German-Assistant-v4/README.md) |
| Llama-2-13B-LoRA-Assemble | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Llama-2-13B-LoRA-Assemble/README.md) |
| Llama-2-13B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Llama-2-13B-chat/README.md) |
| Llama-2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Llama-2-70B/README.md) |
| Llama-2-70B-Chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Llama-2-70B-Chat/README.md) |
| Llama-2-70B-LoRA-Assemble-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Llama-2-70B-LoRA-Assemble-v2/README.md) |
| Llama-2-70B-OASST-1-200 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32016 | 4096 | 8192 | [README.md](models/Llama-2-70B-OASST-1-200/README.md) |
| Llama-2-70B-Orca-200k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Llama-2-70B-Orca-200k/README.md) |
| Llama-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Llama-2-7B/README.md) |
| Llama-2-7B-32K-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Llama-2-7B-32K-Instruct/README.md) |
| Llama-2-7B-Chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Llama-2-7B-Chat/README.md) |
| Llama-2-7B-LoRA-Assemble | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Llama-2-7B-LoRA-Assemble/README.md) |
| Llama-2-7B-ft-instruct-es | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Llama-2-7B-ft-instruct-es/README.md) |
| Llama-2-7B-vietnamese-20k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Llama-2-7B-vietnamese-20k/README.md) |
| Llama-2-Coder-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Llama-2-Coder-7B/README.md) |
| Llama2-13B-MegaCode2-OASST | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32007 | 2048 | 5120 | [README.md](models/Llama2-13B-MegaCode2-OASST/README.md) |
| Llama2-22B-Daydreamer-v3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4098 | 6656 | [README.md](models/Llama2-22B-Daydreamer-v3/README.md) |
| Llama2-70B-OASST-SFT-v10 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32007 | 4096 | 8192 | [README.md](models/Llama2-70B-OASST-SFT-v10/README.md) |
| Llama2-Chat-AYT-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Llama2-Chat-AYT-13B/README.md) |
| Llama2-chat-AYB-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Llama2-chat-AYB-13B/README.md) |
| LlamaGuard-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/LlamaGuard-7B/README.md) |
| Llamix2-MLewd-4x13B | Q4_0 Q4_1 Q5_0 Q5_1 Q8_0 | 32000 | 32768 | 5120 | [README.md](models/Llamix2-MLewd-4x13B/README.md) |
| LlongOrca-13B-16K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32004 | 16384 | 5120 | [README.md](models/LlongOrca-13B-16K/README.md) |
| LlongOrca-7B-16K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 16384 | 4096 | [README.md](models/LlongOrca-7B-16K/README.md) |
| LoKuS-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/LoKuS-13B/README.md) |
| LongAlpaca-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32001 | 4096 | 8192 | [README.md](models/LongAlpaca-70B/README.md) |
| LosslessMegaCoder-Llama2-13B-Mini | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32007 | 2048 | 5120 | [README.md](models/LosslessMegaCoder-Llama2-13B-Mini/README.md) |
| LosslessMegaCoder-Llama2-7B-Mini | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32007 | 4096 | 4096 | [README.md](models/LosslessMegaCoder-Llama2-7B-Mini/README.md) |
| Loyal-Macaroni-Maid-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/Loyal-Macaroni-Maid-7B/README.md) |
| Lumosia-MoE-4x10.7 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Lumosia-MoE-4x10.7/README.md) |
| Luna-AI-Llama2-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Luna-AI-Llama2-Uncensored/README.md) |
| MAmmoTH-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/MAmmoTH-13B/README.md) |
| MAmmoTH-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32001 | 4096 | 8192 | [README.md](models/MAmmoTH-70B/README.md) |
| MAmmoTH-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/MAmmoTH-7B/README.md) |
| MAmmoTH-Coder-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32017 | 16384 | 5120 | [README.md](models/MAmmoTH-Coder-13B/README.md) |
| MAmmoTH-Coder-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 8192 | [README.md](models/MAmmoTH-Coder-34B/README.md) |
| MLewd-L2-Chat-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MLewd-L2-Chat-13B/README.md) |
| MLewd-ReMM-L2-Chat-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MLewd-ReMM-L2-Chat-20B/README.md) |
| MLewd-ReMM-L2-Chat-20B-Inverted | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MLewd-ReMM-L2-Chat-20B-Inverted/README.md) |
| MLewd-v2.4-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MLewd-v2.4-13B/README.md) |
| MLewdBoros-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MLewdBoros-L2-13B/README.md) |
| MLewdBoros-LRPSGPT-2Char-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MLewdBoros-LRPSGPT-2Char-13B/README.md) |
| MXLewd-L2-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MXLewd-L2-20B/README.md) |
| MXLewdMini-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MXLewdMini-L2-13B/README.md) |
| MadMix-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/MadMix-v0.2/README.md) |
| Magicoder-S-DS-6.7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 4096 | [README.md](models/Magicoder-S-DS-6.7B/README.md) |
| Manticore-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/Manticore-13B/README.md) |
| Marcoroni-neural-chat-7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Marcoroni-neural-chat-7B-v1/README.md) |
| Marx-3B-v3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 4096 | 2560 | [README.md](models/Marx-3B-v3/README.md) |
| MegaMix-S1-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MegaMix-S1-13B/README.md) |
| MegaMix-T1-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MegaMix-T1-13B/README.md) |
| Megamix-A1-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Megamix-A1-13B/README.md) |
| MelloGPT | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/MelloGPT/README.md) |
| MergeMonster-13B-20231124 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MergeMonster-13B-20231124/README.md) |
| Merged-AGI-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Merged-AGI-7B/README.md) |
| Meta-Llama-3-8B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_1 Q4_K_M Q4_K_S Q5_0 Q5_1 Q5_K_M Q5_K_S Q6_K Q8_0 | 128256 | 8192 | 4096 | [README.md](models/Meta-Llama-3-8B-Instruct/README.md) |
| MetaMath-13B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/MetaMath-13B-V1.0/README.md) |
| MetaMath-70B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/MetaMath-70B-V1.0/README.md) |
| MetaMath-7B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/MetaMath-7B-V1.0/README.md) |
| MetaMath-Cybertron-Starling | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/MetaMath-Cybertron-Starling/README.md) |
| MetaMath-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 32768 | 4096 | [README.md](models/MetaMath-Mistral-7B/README.md) |
| MetaMath-NeuralHermes-2.5-Mistral-7B-Linear | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 32768 | 4096 | [README.md](models/MetaMath-NeuralHermes-2.5-Mistral-7B-Linear/README.md) |
| Metis-0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Metis-0.1/README.md) |
| Metis-0.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Metis-0.3/README.md) |
| Metis-0.4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Metis-0.4/README.md) |
| Metis-0.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Metis-0.5/README.md) |
| Mini_Synatra_7B_02 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Mini_Synatra_7B_02/README.md) |
| MistRP-Airoboros-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/MistRP-Airoboros-7B/README.md) |
| Mistral-11B-CC-Air-RP | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-11B-CC-Air-RP/README.md) |
| Mistral-11B-OmniMix | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-11B-OmniMix/README.md) |
| Mistral-7B-AEZAKMI-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-AEZAKMI-v1/README.md) |
| Mistral-7B-Claude-Chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-Claude-Chat/README.md) |
| Mistral-7B-Code-16K-qlora | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-Code-16K-qlora/README.md) |
| Mistral-7B-Instruct-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-Instruct-v0.1/README.md) |
| Mistral-7B-Instruct-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-Instruct-v0.2/README.md) |
| Mistral-7B-Instruct-v0.2-DARE | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-Instruct-v0.2-DARE/README.md) |
| Mistral-7B-Instruct-v0.2-code-ft | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-Instruct-v0.2-code-ft/README.md) |
| Mistral-7B-Merge-14-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-Merge-14-v0.1/README.md) |
| Mistral-7B-OpenOrca | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Mistral-7B-OpenOrca/README.md) |
| Mistral-7B-OpenOrca-oasst_top1_2023-08-25-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Mistral-7B-OpenOrca-oasst_top1_2023-08-25-v1/README.md) |
| Mistral-7B-OpenOrca-oasst_top1_2023-08-25-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Mistral-7B-OpenOrca-oasst_top1_2023-08-25-v2/README.md) |
| Mistral-7B-Phibrarian-32K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-Phibrarian-32K/README.md) |
| Mistral-7B-SciPhi-32k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-SciPhi-32k/README.md) |
| Mistral-7B-codealpaca-lora | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-codealpaca-lora/README.md) |
| Mistral-7B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-7B-v0.1/README.md) |
| Mistral-ClaudeLimaRP-v3-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-ClaudeLimaRP-v3-7B/README.md) |
| Mistral-Pygmalion-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Mistral-Pygmalion-7B/README.md) |
| Mistral-Trismegistus-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mistral-Trismegistus-7B/README.md) |
| MistralLite-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 32768 | 4096 | [README.md](models/MistralLite-7B/README.md) |
| MistralMakise-Merged-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MistralMakise-Merged-13B/README.md) |
| Mistral_7B_Dolphin2.1_LIMA0.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Mistral_7B_Dolphin2.1_LIMA0.5/README.md) |
| Mistralic-7B-1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Mistralic-7B-1/README.md) |
| Mixtral-8x7B-Instruct-v0.1 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mixtral-8x7B-Instruct-v0.1/README.md) |
| Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss/README.md) |
| Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-DARE-TIES | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-DARE-TIES/README.md) |
| Mixtral-8x7B-MoE-RP-Story | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mixtral-8x7B-MoE-RP-Story/README.md) |
| Mixtral-8x7B-v0.1 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mixtral-8x7B-v0.1/README.md) |
| Mixtral-Fusion-4x7B-Instruct-v0.1 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mixtral-Fusion-4x7B-Instruct-v0.1/README.md) |
| Mixtral-SlimOrca-8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Mixtral-SlimOrca-8x7B/README.md) |
| MixtralOrochi8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/MixtralOrochi8x7B/README.md) |
| MixtralRPChat-ZLoss | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/MixtralRPChat-ZLoss/README.md) |
| Mixtral_11Bx2_MoE_19B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Mixtral_11Bx2_MoE_19B/README.md) |
| Mixtral_34Bx2_MoE_60B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K | 64000 | 200000 | 7168 | [README.md](models/Mixtral_34Bx2_MoE_60B/README.md) |
| Mixtral_7Bx2_MoE | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Mixtral_7Bx2_MoE/README.md) |
| MoMo-70B-V1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/MoMo-70B-V1.1/README.md) |
| MonadGPT | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/MonadGPT/README.md) |
| MysticFusion-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MysticFusion-13B/README.md) |
| Mythalion-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Mythalion-13B/README.md) |
| Mythalion-Kimiko-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Mythalion-Kimiko-v2/README.md) |
| Mythical-Destroyer-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Mythical-Destroyer-L2-13B/README.md) |
| Mythical-Destroyer-V2-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Mythical-Destroyer-V2-L2-13B/README.md) |
| MythoBoros-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/MythoBoros-13B/README.md) |
| MythoLogic-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/MythoLogic-13B/README.md) |
| MythoLogic-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MythoLogic-L2-13B/README.md) |
| MythoLogic-Mini-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/MythoLogic-Mini-7B/README.md) |
| MythoMakiseMerged-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MythoMakiseMerged-13B/README.md) |
| MythoMax-Kimiko-Mix | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/MythoMax-Kimiko-Mix/README.md) |
| MythoMax-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MythoMax-L2-13B/README.md) |
| MythoMax-L2-Kimiko-v2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MythoMax-L2-Kimiko-v2-13B/README.md) |
| MythoMist-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/MythoMist-7B/README.md) |
| MythoMix-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/MythoMix-L2-13B/README.md) |
| Naberius-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Naberius-7B/README.md) |
| Nanbeige-16B-Base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 59136 | 4096 | 5120 | [README.md](models/Nanbeige-16B-Base/README.md) |
| Nanbeige-16B-Chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 59136 | 4096 | 5120 | [README.md](models/Nanbeige-16B-Chat/README.md) |
| Nete-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Nete-13B/README.md) |
| Nethena-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Nethena-13B/README.md) |
| Nethena-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Nethena-20B/README.md) |
| Nethena-20B-Glued | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/Nethena-20B-Glued/README.md) |
| Nethena-MLewd-Xwin-23B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Nethena-MLewd-Xwin-23B/README.md) |
| NeuralBeagle14-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/NeuralBeagle14-7B/README.md) |
| NeuralHermes-2.5-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/NeuralHermes-2.5-Mistral-7B/README.md) |
| NeuralOrca-7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/NeuralOrca-7B-v1/README.md) |
| NeuralPipe-7B-slerp | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/NeuralPipe-7B-slerp/README.md) |
| NeuralPipe-7B-ties | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/NeuralPipe-7B-ties/README.md) |
| NeuralPipe-9B-merged | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/NeuralPipe-9B-merged/README.md) |
| NeuralQuant-9B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/NeuralQuant-9B/README.md) |
| NexoNimbus-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/NexoNimbus-7B/README.md) |
| NexusRaven-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32024 | 16384 | 5120 | [README.md](models/NexusRaven-13B/README.md) |
| NexusRaven-V2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32024 | 16384 | 5120 | [README.md](models/NexusRaven-V2-13B/README.md) |
| Norobara-ZLoss-8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Norobara-ZLoss-8x7B/README.md) |
| Norocetacean-20B-10k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 10240 | 5120 | [README.md](models/Norocetacean-20B-10k/README.md) |
| Noromaid-13B-v0.1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Noromaid-13B-v0.1.1/README.md) |
| Noromaid-13B-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Noromaid-13B-v0.2/README.md) |
| Noromaid-13B-v0.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Noromaid-13B-v0.3/README.md) |
| Noromaid-20B-v0.1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Noromaid-20B-v0.1.1/README.md) |
| Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss/README.md) |
| Nous-Capybara-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Nous-Capybara-34B/README.md) |
| Nous-Capybara-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Nous-Capybara-7B/README.md) |
| Nous-Capybara-7B-v1.9 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Nous-Capybara-7B-v1.9/README.md) |
| Nous-Capybara-limarpv3-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Nous-Capybara-limarpv3-34B/README.md) |
| Nous-Hermes-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/Nous-Hermes-13B/README.md) |
| Nous-Hermes-13B-Code | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32032 | 4096 | 5120 | [README.md](models/Nous-Hermes-13B-Code/README.md) |
| Nous-Hermes-2-Mixtral-8x7B-DPO | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Nous-Hermes-2-Mixtral-8x7B-DPO/README.md) |
| Nous-Hermes-2-Mixtral-8x7B-SFT | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Nous-Hermes-2-Mixtral-8x7B-SFT/README.md) |
| Nous-Hermes-2-SOLAR-10.7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 4096 | 4096 | [README.md](models/Nous-Hermes-2-SOLAR-10.7B/README.md) |
| Nous-Hermes-2-SUS-Chat-34B-Slerp | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 4096 | 7168 | [README.md](models/Nous-Hermes-2-SUS-Chat-34B-Slerp/README.md) |
| Nous-Hermes-2-Yi-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/Nous-Hermes-2-Yi-34B/README.md) |
| Nous-Hermes-Llama-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Nous-Hermes-Llama-2-7B/README.md) |
| Nous-Hermes-Llama2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32032 | 4096 | 5120 | [README.md](models/Nous-Hermes-Llama2/README.md) |
| Nous-Hermes-Llama2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32001 | 4096 | 8192 | [README.md](models/Nous-Hermes-Llama2-70B/README.md) |
| Nous-Puffin-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Nous-Puffin-70B/README.md) |
| Nyxene-v2-11B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Nyxene-v2-11B/README.md) |
| Nyxene-v3-11B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Nyxene-v3-11B/README.md) |
| ORCA_LLaMA_70B_QLoRA | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/ORCA_LLaMA_70B_QLoRA/README.md) |
| Open-Hermes-2.5-neural-chat-3.1-frankenmerge-11b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Open-Hermes-2.5-neural-chat-3.1-frankenmerge-11b/README.md) |
| OpenAssistant-Llama2-13B-Orca-8K-3319 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32016 | 8192 | 5120 | [README.md](models/OpenAssistant-Llama2-13B-Orca-8K-3319/README.md) |
| OpenBuddy-Llama2-13B-v11.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 37632 | 4096 | 5120 | [README.md](models/OpenBuddy-Llama2-13B-v11.1/README.md) |
| OpenBuddy-Llama2-70b-v10.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 37632 | 4096 | 8192 | [README.md](models/OpenBuddy-Llama2-70b-v10.1/README.md) |
| OpenHermes-2-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/OpenHermes-2-Mistral-7B/README.md) |
| OpenHermes-2.5-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/OpenHermes-2.5-Mistral-7B/README.md) |
| OpenHermes-2.5-Mistral-7B-16k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/OpenHermes-2.5-Mistral-7B-16k/README.md) |
| OpenHermes-2.5-neural-chat-7B-v3-1-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/OpenHermes-2.5-neural-chat-7B-v3-1-7B/README.md) |
| OpenHermes-2.5-neural-chat-7B-v3-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/OpenHermes-2.5-neural-chat-7B-v3-2-7B/README.md) |
| OpenHermes-2.5-neural-chat-v3-3-Slerp | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/OpenHermes-2.5-neural-chat-v3-3-Slerp/README.md) |
| OpenOrca-Platypus2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 4096 | 5120 | [README.md](models/OpenOrca-Platypus2-13B/README.md) |
| OpenOrca-Zephyr-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/OpenOrca-Zephyr-7B/README.md) |
| OpenOrca_Stx | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 4096 | 5120 | [README.md](models/OpenOrca_Stx/README.md) |
| OpenZephyrChat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/OpenZephyrChat/README.md) |
| OpenZephyrChat-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/OpenZephyrChat-v0.2/README.md) |
| Open_Gpt4_8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Open_Gpt4_8x7B/README.md) |
| Open_Gpt4_8x7B_v0.2 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Open_Gpt4_8x7B_v0.2/README.md) |
| Optimus-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Optimus-7B/README.md) |
| Orca-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 4096 | 5120 | [README.md](models/Orca-2-13B/README.md) |
| Orca-2-13B-SFT_v5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 4096 | 5120 | [README.md](models/Orca-2-13B-SFT_v5/README.md) |
| Orca-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 4096 | 4096 | [README.md](models/Orca-2-7B/README.md) |
| Orca2myth7.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Orca2myth7.2/README.md) |
| OrcaMaid-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/OrcaMaid-13B/README.md) |
| OrcaMaid-v2-FIX-13B-32k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 5120 | [README.md](models/OrcaMaid-v2-FIX-13B-32k/README.md) |
| OrcaMaid-v3-13B-32k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 5120 | [README.md](models/OrcaMaid-v3-13B-32k/README.md) |
| OrcaMaidXL-17B-32k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 5120 | [README.md](models/OrcaMaidXL-17B-32k/README.md) |
| OrionStar-Yi-34B-Chat-Llama | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/OrionStar-Yi-34B-Chat-Llama/README.md) |
| Pallas-0.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Pallas-0.3/README.md) |
| Pallas-0.4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Pallas-0.4/README.md) |
| Pallas-0.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Pallas-0.5/README.md) |
| Pallas-0.5-LASER-0.6 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Pallas-0.5-LASER-0.6/README.md) |
| Pallas-0.5-frankenmerge | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Pallas-0.5-frankenmerge/README.md) |
| Panda-7B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Panda-7B-v0.1/README.md) |
| Pandalyst-7B-V1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 4096 | [README.md](models/Pandalyst-7B-V1.1/README.md) |
| Pandalyst-7B-v1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 4096 | [README.md](models/Pandalyst-7B-v1.2/README.md) |
| Pandalyst_13B_V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 5120 | [README.md](models/Pandalyst_13B_V1.0/README.md) |
| Pandora-v1-10.7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Pandora-v1-10.7B/README.md) |
| Pandora-v1-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Pandora-v1-13B/README.md) |
| Phind-CodeLlama-34B-Python-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/Phind-CodeLlama-34B-Python-v1/README.md) |
| Phind-CodeLlama-34B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/Phind-CodeLlama-34B-v1/README.md) |
| Phind-CodeLlama-34B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/Phind-CodeLlama-34B-v2/README.md) |
| PiVoT-0.1-Evil-a | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/PiVoT-0.1-Evil-a/README.md) |
| PiVoT-0.1-Starling-LM-RP | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 8192 | 4096 | [README.md](models/PiVoT-0.1-Starling-LM-RP/README.md) |
| PiVoT-0.1-early | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/PiVoT-0.1-early/README.md) |
| PiVoT-10.7B-Mistral-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/PiVoT-10.7B-Mistral-v0.2/README.md) |
| PiVoT-10.7B-Mistral-v0.2-RP | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/PiVoT-10.7B-Mistral-v0.2-RP/README.md) |
| PiVoT-Merge-A-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 8192 | 4096 | [README.md](models/PiVoT-Merge-A-7B/README.md) |
| PiVoT-MoE | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/PiVoT-MoE/README.md) |
| PiVoT-SUS-RP | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 8192 | 7168 | [README.md](models/PiVoT-SUS-RP/README.md) |
| Pirouette-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Pirouette-7B/README.md) |
| PlatYi-34B-Llama-Q-v3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/PlatYi-34B-Llama-Q-v3/README.md) |
| Platypus-30B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/Platypus-30B/README.md) |
| Platypus2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Platypus2-13B/README.md) |
| Platypus2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Platypus2-70B/README.md) |
| Platypus2-70B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Platypus2-70B-Instruct/README.md) |
| Poro-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 128000 | 0 | 7168 | [README.md](models/Poro-34B/README.md) |
| PsyMedRP-v1-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/PsyMedRP-v1-13B/README.md) |
| PsyMedRP-v1-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/PsyMedRP-v1-20B/README.md) |
| Psyfighter-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Psyfighter-13B/README.md) |
| PuddleJumper-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_1 Q4_K_M Q4_K_S Q5_0 Q5_1 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 4096 | 5120 | [README.md](models/PuddleJumper-13B/README.md) |
| PuddleJumper-13B-V2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 4096 | 5120 | [README.md](models/PuddleJumper-13B-V2/README.md) |
| Pygmalion-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Pygmalion-2-13B/README.md) |
| Pygmalion-2-13B-SuperCOT | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Pygmalion-2-13B-SuperCOT/README.md) |
| Pygmalion-2-13B-SuperCOT-weighed | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Pygmalion-2-13B-SuperCOT-weighed/README.md) |
| Pygmalion-2-13B-SuperCOT2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Pygmalion-2-13B-SuperCOT2/README.md) |
| Pygmalion-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Pygmalion-2-7B/README.md) |
| Python-Code-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Python-Code-13B/README.md) |
| Python-Code-33B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/Python-Code-33B/README.md) |
| ReMM-SLERP-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/ReMM-SLERP-L2-13B/README.md) |
| ReMM-v2-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/ReMM-v2-L2-13B/README.md) |
| ReMM-v2.1-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/ReMM-v2.1-L2-13B/README.md) |
| Redmond-Puffin-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32032 | 4096 | 5120 | [README.md](models/Redmond-Puffin-13B/README.md) |
| Rogue-Rose-103b-v0.2 | Q2_K Q3_K_M Q3_K_S | 32000 | 4096 | 8192 | [README.md](models/Rogue-Rose-103b-v0.2/README.md) |
| Rosa_v2_7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Rosa_v2_7B/README.md) |
| Rose-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Rose-20B/README.md) |
| Rose-Kimiko-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/Rose-Kimiko-20B/README.md) |
| RpBird-Yi-34B-200k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/RpBird-Yi-34B-200k/README.md) |
| SAM | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/SAM/README.md) |
| SG-Raccoon-Yi-200k-2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K | 64000 | 200000 | 7168 | [README.md](models/SG-Raccoon-Yi-200k-2.0/README.md) |
| SG-Raccoon-Yi-55B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K | 64000 | 4096 | 7168 | [README.md](models/SG-Raccoon-Yi-55B/README.md) |
| SG-Raccoon-Yi-55B-200k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K | 64002 | 200000 | 7168 | [README.md](models/SG-Raccoon-Yi-55B-200k/README.md) |
| SOLAR-10.7B-Instruct-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/SOLAR-10.7B-Instruct-v1.0/README.md) |
| SOLAR-10.7B-Instruct-v1.0-uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/SOLAR-10.7B-Instruct-v1.0-uncensored/README.md) |
| SOLAR-10.7B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/SOLAR-10.7B-v1.0/README.md) |
| SOLAR-Platypus-10.7B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/SOLAR-Platypus-10.7B-v2/README.md) |
| SOLARC-MOE-10.7Bx4 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/SOLARC-MOE-10.7Bx4/README.md) |
| SUS-Chat-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 8192 | 7168 | [README.md](models/SUS-Chat-34B/README.md) |
| Sakura-SOLAR-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Sakura-SOLAR-Instruct/README.md) |
| Samantha-1.1-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Samantha-1.1-70B/README.md) |
| Samantha-1.11-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Samantha-1.11-13B/README.md) |
| Samantha-1.11-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Samantha-1.11-70B/README.md) |
| Samantha-1.11-CodeLlama-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 8192 | [README.md](models/Samantha-1.11-CodeLlama-34B/README.md) |
| SauerkrautLM-7B-HerO | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/SauerkrautLM-7B-HerO/README.md) |
| SauerkrautLM-Mixtral-8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/SauerkrautLM-Mixtral-8x7B/README.md) |
| SauerkrautLM-Mixtral-8x7B-Instruct | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/SauerkrautLM-Mixtral-8x7B-Instruct/README.md) |
| SauerkrautLM-SOLAR-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/SauerkrautLM-SOLAR-Instruct/README.md) |
| SauerkrautLM-UNA-SOLAR-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/SauerkrautLM-UNA-SOLAR-Instruct/README.md) |
| Scarlett-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Scarlett-7B/README.md) |
| SciPhi-Mistral-7B-32k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/SciPhi-Mistral-7B-32k/README.md) |
| SciPhi-Self-RAG-Mistral-7B-32k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32016 | 32768 | 4096 | [README.md](models/SciPhi-Self-RAG-Mistral-7B-32k/README.md) |
| Sensei-7B-V1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Sensei-7B-V1/README.md) |
| Sensualize-Mixtral | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Sensualize-Mixtral/README.md) |
| Sensualize-Solar-10.7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Sensualize-Solar-10.7B/README.md) |
| Seraph-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Seraph-7B/README.md) |
| Seraph-openchat-3.5-1210-Slerp | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Seraph-openchat-3.5-1210-Slerp/README.md) |
| Sheep-Duck-Llama-2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Sheep-Duck-Llama-2-70B/README.md) |
| ShiningValiant-1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/ShiningValiant-1.2/README.md) |
| ShiningValiant-1.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/ShiningValiant-1.3/README.md) |
| ShiningValiantXS | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/ShiningValiantXS/README.md) |
| ShiningValiantXS-1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/ShiningValiantXS-1.1/README.md) |
| Silicon-Maid-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/Silicon-Maid-7B/README.md) |
| Skywork-13B-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 65519 | 4096 | 4608 | [README.md](models/Skywork-13B-base/README.md) |
| SlimOpenOrca-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/SlimOpenOrca-Mistral-7B/README.md) |
| SlimOrca-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/SlimOrca-13B/README.md) |
| Solar-10.7B-SLERP | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Solar-10.7B-SLERP/README.md) |
| Solus-103B-L2 | Q2_K Q3_K_M Q3_K_S | 32000 | 4096 | 8192 | [README.md](models/Solus-103B-L2/README.md) |
| Solus-70B-L2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Solus-70B-L2/README.md) |
| Sonya-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/Sonya-7B/README.md) |
| Speechless-Llama2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Speechless-Llama2-13B/README.md) |
| Speechless-Llama2-Hermes-Orca-Platypus-WizardLM-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Speechless-Llama2-Hermes-Orca-Platypus-WizardLM-13B/README.md) |
| Spicyboros-13B-2.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Spicyboros-13B-2.2/README.md) |
| Spicyboros-70B-2.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Spicyboros-70B-2.2/README.md) |
| Spicyboros-7B-2.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Spicyboros-7B-2.2/README.md) |
| Spicyboros-c34b-2.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/Spicyboros-c34b-2.2/README.md) |
| Spring-Dragon | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Spring-Dragon/README.md) |
| Stable-Platypus2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Stable-Platypus2-13B/README.md) |
| StableBeluga-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/StableBeluga-13B/README.md) |
| StableBeluga-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/StableBeluga-7B/README.md) |
| StableBeluga2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/StableBeluga2-70B/README.md) |
| Starling-LM-7B-alpha | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/Starling-LM-7B-alpha/README.md) |
| Starling-LM-alpha-8x7B-MoE | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Starling-LM-alpha-8x7B-MoE/README.md) |
| Stheno-Inverted-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Stheno-Inverted-L2-13B/README.md) |
| Stheno-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Stheno-L2-13B/README.md) |
| Stheno-v2-Delta | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Stheno-v2-Delta/README.md) |
| SuperPlatty-30B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/SuperPlatty-30B/README.md) |
| Swallow-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 43176 | 4096 | 5120 | [README.md](models/Swallow-13B/README.md) |
| Swallow-13B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 43176 | 4096 | 5120 | [README.md](models/Swallow-13B-Instruct/README.md) |
| Swallow-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 43176 | 4096 | 8192 | [README.md](models/Swallow-70B/README.md) |
| Swallow-70B-instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 43176 | 4096 | 8192 | [README.md](models/Swallow-70B-instruct/README.md) |
| Swallow-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 43176 | 4096 | 4096 | [README.md](models/Swallow-7B/README.md) |
| Swallow-7B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 43176 | 4096 | 4096 | [README.md](models/Swallow-7B-Instruct/README.md) |
| Sydney_Overthinker_13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Sydney_Overthinker_13B/README.md) |
| Synatra-7B-v0.3-RP | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Synatra-7B-v0.3-RP/README.md) |
| Synatra-7B-v0.3-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Synatra-7B-v0.3-base/README.md) |
| Synatra-7B-v0.3-dpo | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Synatra-7B-v0.3-dpo/README.md) |
| Synatra-RP-Orca-2-7B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 4096 | 4096 | [README.md](models/Synatra-RP-Orca-2-7B-v0.1/README.md) |
| Synatra-V0.1-7B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Synatra-V0.1-7B-Instruct/README.md) |
| SynthIA-70B-v1.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 2048 | 8192 | [README.md](models/SynthIA-70B-v1.5/README.md) |
| SynthIA-7B-v1.3-dare-0.85 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/SynthIA-7B-v1.3-dare-0.85/README.md) |
| SynthIA-7B-v1.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/SynthIA-7B-v1.5/README.md) |
| SynthIA-7B-v2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/SynthIA-7B-v2.0/README.md) |
| SynthIA-7B-v2.0-16k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/SynthIA-7B-v2.0-16k/README.md) |
| Synthia-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Synthia-13B/README.md) |
| Synthia-13B-v1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Synthia-13B-v1.2/README.md) |
| Synthia-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_K_M Q4_K_S Q5_K_M Q5_K_S | 32000 | 2048 | 8192 | [README.md](models/Synthia-70B/README.md) |
| Synthia-70B-v1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 2048 | 8192 | [README.md](models/Synthia-70B-v1.1/README.md) |
| Synthia-70B-v1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 2048 | 8192 | [README.md](models/Synthia-70B-v1.2/README.md) |
| Synthia-70B-v1.2b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Synthia-70B-v1.2b/README.md) |
| Synthia-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Synthia-7B/README.md) |
| Synthia-7B-v1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Synthia-7B-v1.2/README.md) |
| Synthia-7B-v1.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Synthia-7B-v1.3/README.md) |
| Synthia-7B-v3.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Synthia-7B-v3.0/README.md) |
| Synthia-MoE-v3-Mixtral-8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Synthia-MoE-v3-Mixtral-8x7B/README.md) |
| Synthia-v3.0-11B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Synthia-v3.0-11B/README.md) |
| Tai-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Tai-70B/README.md) |
| TenyxChat-7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/TenyxChat-7B-v1/README.md) |
| Terminis-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Terminis-7B/README.md) |
| Tess-10.7B-v1.5b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Tess-10.7B-v1.5b/README.md) |
| Tess-34B-v1.4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Tess-34B-v1.4/README.md) |
| Tess-34B-v1.5b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Tess-34B-v1.5b/README.md) |
| Tess-7B-v1.4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Tess-7B-v1.4/README.md) |
| Tess-M-Creative-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Tess-M-Creative-v1.0/README.md) |
| Tess-M-v1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Tess-M-v1.1/README.md) |
| Tess-M-v1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Tess-M-v1.2/README.md) |
| Tess-M-v1.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Tess-M-v1.3/README.md) |
| Tess-XL-v1.0 | Q2_K | 32000 | 4096 | 8192 | [README.md](models/Tess-XL-v1.0/README.md) |
| Tess-XS-Creative-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Tess-XS-Creative-v1.0/README.md) |
| Tess-XS-v1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Tess-XS-v1.1/README.md) |
| Thespis-13B-Alpha-v0.7 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Thespis-13B-Alpha-v0.7/README.md) |
| Thespis-13B-DPO-v0.7 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Thespis-13B-DPO-v0.7/README.md) |
| Thespis-13B-v0.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Thespis-13B-v0.3/README.md) |
| Thespis-13B-v0.4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Thespis-13B-v0.4/README.md) |
| Thespis-13B-v0.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Thespis-13B-v0.5/README.md) |
| Thespis-13B-v0.6 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Thespis-13B-v0.6/README.md) |
| Thespis-Mistral-7B-Alpha-v0.7 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Thespis-Mistral-7B-Alpha-v0.7/README.md) |
| Thespis-Mistral-7B-v0.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Thespis-Mistral-7B-v0.5/README.md) |
| Thespis-Mistral-7B-v0.6 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Thespis-Mistral-7B-v0.6/README.md) |
| Tiamat-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Tiamat-7B/README.md) |
| Tiamat-7B-1.1-DPO | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Tiamat-7B-1.1-DPO/README.md) |
| TimeCrystal-L2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/TimeCrystal-L2-13B/README.md) |
| TinyLlama-1.1B-1T-OpenOrca | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 2048 | [README.md](models/TinyLlama-1.1B-1T-OpenOrca/README.md) |
| TinyLlama-1.1B-Chat-v0.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 2048 | 2048 | [README.md](models/TinyLlama-1.1B-Chat-v0.3/README.md) |
| TinyLlama-1.1B-Chat-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 2048 | [README.md](models/TinyLlama-1.1B-Chat-v1.0/README.md) |
| TinyLlama-1.1B-intermediate-step-1431k-3T | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 2048 | [README.md](models/TinyLlama-1.1B-intermediate-step-1431k-3T/README.md) |
| TinyLlama-1.1B-intermediate-step-480k-1T | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 2048 | [README.md](models/TinyLlama-1.1B-intermediate-step-480k-1T/README.md) |
| TinyLlama-1.1B-intermediate-step-715k-1.5T | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 2048 | [README.md](models/TinyLlama-1.1B-intermediate-step-715k-1.5T/README.md) |
| TinyLlama-1.1B-python-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 2048 | [README.md](models/TinyLlama-1.1B-python-v0.1/README.md) |
| Tinyllama-2-1b-miniguanaco | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 2048 | 2048 | [README.md](models/Tinyllama-2-1b-miniguanaco/README.md) |
| Toppy-M-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Toppy-M-7B/README.md) |
| TowerInstruct-7B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32007 | 4096 | 4096 | [README.md](models/TowerInstruct-7B-v0.1/README.md) |
| Trion-M-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Trion-M-7B/README.md) |
| Trurl-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/Trurl-2-13B/README.md) |
| Trurl-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/Trurl-2-7B/README.md) |
| Tulpar-7B-v0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Tulpar-7B-v0/README.md) |
| U-Amethyst-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/U-Amethyst-20B/README.md) |
| UNA-TheBeagle-7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/UNA-TheBeagle-7B-v1/README.md) |
| UNAversal-8x7B-v1beta | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/UNAversal-8x7B-v1beta/README.md) |
| UltraLM-13B-v2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/UltraLM-13B-v2.0/README.md) |
| Uncensored-Frank-13b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Uncensored-Frank-13b/README.md) |
| Uncensored-Frank-33b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/Uncensored-Frank-33b/README.md) |
| Uncensored-Frank-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Uncensored-Frank-7B/README.md) |
| Uncensored-Jordan-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Uncensored-Jordan-13B/README.md) |
| Uncensored-Jordan-33B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/Uncensored-Jordan-33B/README.md) |
| Uncensored-Jordan-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Uncensored-Jordan-7B/README.md) |
| UndiMix-v1-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/UndiMix-v1-13B/README.md) |
| UndiMix-v2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/UndiMix-v2-13B/README.md) |
| UndiMix-v3-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/UndiMix-v3-13B/README.md) |
| UndiMix-v4-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/UndiMix-v4-13B/README.md) |
| Unholy-v1-10l-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Unholy-v1-10l-13B/README.md) |
| Unholy-v1-12L-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Unholy-v1-12L-13B/README.md) |
| Unholy-v2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Unholy-v2-13B/README.md) |
| Uni-TianYan-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Uni-TianYan-70B/README.md) |
| Upstage-Llama-2-70B-instruct-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Upstage-Llama-2-70B-instruct-v2/README.md) |
| Upstage-Llama1-65B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 2048 | 8192 | [README.md](models/Upstage-Llama1-65B-Instruct/README.md) |
| Utopia-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Utopia-13B/README.md) |
| UtopiaXL-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/UtopiaXL-13B/README.md) |
| Valkyrie-V1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Valkyrie-V1/README.md) |
| Velara | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Velara/README.md) |
| Velara-11B-V2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Velara-11B-V2/README.md) |
| Venus-103b-v1.1 | Q2_K Q3_K_M Q3_K_S | 32000 | 4096 | 8192 | [README.md](models/Venus-103b-v1.1/README.md) |
| Venus-103b-v1.2 | Q2_K Q3_K_M Q3_K_S | 32000 | 4096 | 8192 | [README.md](models/Venus-103b-v1.2/README.md) |
| Venus-120b-v1.2 | Q2_K | 32000 | 4096 | 8192 | [README.md](models/Venus-120b-v1.2/README.md) |
| Vicuna-13B-CoT | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/Vicuna-13B-CoT/README.md) |
| Vicuna-7B-CoT | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Vicuna-7B-CoT/README.md) |
| Vigogne-2-13B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Vigogne-2-13B-Instruct/README.md) |
| Vigogne-2-7B-Chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Vigogne-2-7B-Chat/README.md) |
| Vigogne-2-7B-Instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Vigogne-2-7B-Instruct/README.md) |
| Vigostral-7B-Chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Vigostral-7B-Chat/README.md) |
| WestLake-7B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/WestLake-7B-v2/README.md) |
| WestSeverus-7B-DPO | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/WestSeverus-7B-DPO/README.md) |
| WinterGoddess-1.4x-70B-L2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/WinterGoddess-1.4x-70B-L2/README.md) |
| WinterGoliath-123b | Q2_K | 32000 | 4096 | 8192 | [README.md](models/WinterGoliath-123b/README.md) |
| Wizard-Vicuna-13B-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/Wizard-Vicuna-13B-Uncensored/README.md) |
| Wizard-Vicuna-30B-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/Wizard-Vicuna-30B-Uncensored/README.md) |
| Wizard-Vicuna-7B-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/Wizard-Vicuna-7B-Uncensored/README.md) |
| WizardCoder-33B-V1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 7168 | [README.md](models/WizardCoder-33B-V1.1/README.md) |
| WizardCoder-Python-13B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 5120 | [README.md](models/WizardCoder-Python-13B-V1.0/README.md) |
| WizardCoder-Python-34B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 8192 | [README.md](models/WizardCoder-Python-34B-V1.0/README.md) |
| WizardCoder-Python-7B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 4096 | [README.md](models/WizardCoder-Python-7B-V1.0/README.md) |
| WizardLM-1.0-Uncensored-CodeLlama-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/WizardLM-1.0-Uncensored-CodeLlama-34B/README.md) |
| WizardLM-1.0-Uncensored-Llama2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/WizardLM-1.0-Uncensored-Llama2-13B/README.md) |
| WizardLM-13B-1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/WizardLM-13B-1.0/README.md) |
| WizardLM-13B-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/WizardLM-13B-Uncensored/README.md) |
| WizardLM-13B-V1.0-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/WizardLM-13B-V1.0-Uncensored/README.md) |
| WizardLM-13B-V1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/WizardLM-13B-V1.1/README.md) |
| WizardLM-13B-V1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/WizardLM-13B-V1.2/README.md) |
| WizardLM-30B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 6656 | [README.md](models/WizardLM-30B/README.md) |
| WizardLM-30B-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 6656 | [README.md](models/WizardLM-30B-Uncensored/README.md) |
| WizardLM-33B-V1.0-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/WizardLM-33B-V1.0-Uncensored/README.md) |
| WizardLM-70B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32001 | 4096 | 8192 | [README.md](models/WizardLM-70B-V1.0/README.md) |
| WizardLM-7B-V1.0-Uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/WizardLM-7B-V1.0-Uncensored/README.md) |
| WizardLM-7B-uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 4096 | [README.md](models/WizardLM-7B-uncensored/README.md) |
| WizardLM-Uncensored-SuperCOT-StoryTelling-30B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 6656 | [README.md](models/WizardLM-Uncensored-SuperCOT-StoryTelling-30B/README.md) |
| WizardMath-13B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/WizardMath-13B-V1.0/README.md) |
| WizardMath-70B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32002 | 2048 | 8192 | [README.md](models/WizardMath-70B-V1.0/README.md) |
| WizardMath-7B-V1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 4096 | [README.md](models/WizardMath-7B-V1.0/README.md) |
| WizardMath-7B-V1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/WizardMath-7B-V1.1/README.md) |
| WordWoven-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/WordWoven-13B/README.md) |
| Writing_Partner_Mistral_7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/Writing_Partner_Mistral_7B/README.md) |
| X-MythoChronos-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/X-MythoChronos-13B/README.md) |
| X-NoroChronos-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/X-NoroChronos-13B/README.md) |
| Xwin-LM-13B-V0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Xwin-LM-13B-V0.1/README.md) |
| Xwin-LM-13B-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/Xwin-LM-13B-v0.2/README.md) |
| Xwin-LM-70B-V0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/Xwin-LM-70B-V0.1/README.md) |
| Xwin-LM-7B-V0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Xwin-LM-7B-V0.1/README.md) |
| Xwin-LM-7B-V0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/Xwin-LM-7B-V0.2/README.md) |
| Xwin-MLewd-13B-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/Xwin-MLewd-13B-v0.2/README.md) |
| XwinCoder-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 5120 | [README.md](models/XwinCoder-13B/README.md) |
| XwinCoder-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/XwinCoder-34B/README.md) |
| Yarn-Llama-2-13B-128K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 131072 | 5120 | [README.md](models/Yarn-Llama-2-13B-128K/README.md) |
| Yarn-Llama-2-13B-64K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 65536 | 5120 | [README.md](models/Yarn-Llama-2-13B-64K/README.md) |
| Yarn-Llama-2-70B-32k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 8192 | 8192 | [README.md](models/Yarn-Llama-2-70B-32k/README.md) |
| Yarn-Llama-2-7B-128K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 131072 | 4096 | [README.md](models/Yarn-Llama-2-7B-128K/README.md) |
| Yarn-Llama-2-7B-64K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 65536 | 4096 | [README.md](models/Yarn-Llama-2-7B-64K/README.md) |
| Yarn-Mistral-7B-128k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Yarn-Mistral-7B-128k/README.md) |
| Yarn-Mistral-7B-64k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/Yarn-Mistral-7B-64k/README.md) |
| Yi-34B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/Yi-34B/README.md) |
| Yi-34B-200K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Yi-34B-200K/README.md) |
| Yi-34B-200K-AEZAKMI-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Yi-34B-200K-AEZAKMI-v2/README.md) |
| Yi-34B-200K-DARE-megamerge-v8 | IQ2_XS IQ2_XXS Q2_K Q2_K_S Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64002 | 200000 | 7168 | [README.md](models/Yi-34B-200K-DARE-megamerge-v8/README.md) |
| Yi-34B-200K-DARE-merge-v5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Yi-34B-200K-DARE-merge-v5/README.md) |
| Yi-34B-200K-Llamafied | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/Yi-34B-200K-Llamafied/README.md) |
| Yi-34B-Chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/Yi-34B-Chat/README.md) |
| Yi-34B-GiftedConvo-merged | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/Yi-34B-GiftedConvo-merged/README.md) |
| Yi-6B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 4096 | [README.md](models/Yi-6B/README.md) |
| Yi-6B-200K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 4096 | [README.md](models/Yi-6B-200K/README.md) |
| You_can_cry_Snowman-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/You_can_cry_Snowman-13B/README.md) |
| YuLan-Chat-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 51200 | 8192 | 5120 | [README.md](models/YuLan-Chat-2-13B/README.md) |
| Zarablend-L2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Zarablend-L2-7B/README.md) |
| Zarablend-MX-L2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Zarablend-MX-L2-7B/README.md) |
| Zarafusionex-1.1-L2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/Zarafusionex-1.1-L2-7B/README.md) |
| ZephRP-m7b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/ZephRP-m7b/README.md) |
| Ziya-Coding-34B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/Ziya-Coding-34B-v1.0/README.md) |
| agentlm-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 4096 | 5120 | [README.md](models/agentlm-13B/README.md) |
| agentlm-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32256 | 4096 | 8192 | [README.md](models/agentlm-70B/README.md) |
| agentlm-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 4096 | 4096 | [README.md](models/agentlm-7B/README.md) |
| agiin-13.6B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/agiin-13.6B-v0.1/README.md) |
| airoboros-c34b-2.2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/airoboros-c34b-2.2.1/README.md) |
| airoboros-l2-13B-2.2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/airoboros-l2-13B-2.2.1/README.md) |
| airoboros-l2-13B-3.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/airoboros-l2-13B-3.0/README.md) |
| airoboros-l2-13B-gpt4-1.4.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/airoboros-l2-13B-gpt4-1.4.1/README.md) |
| airoboros-l2-13b-gpt4-2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/airoboros-l2-13b-gpt4-2.0/README.md) |
| airoboros-l2-13b-gpt4-m2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/airoboros-l2-13b-gpt4-m2.0/README.md) |
| airoboros-l2-70B-GPT4-2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/airoboros-l2-70B-GPT4-2.0/README.md) |
| airoboros-l2-70B-gpt4-1.4.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/airoboros-l2-70B-gpt4-1.4.1/README.md) |
| airoboros-l2-7B-2.2.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/airoboros-l2-7B-2.2.1/README.md) |
| airoboros-l2-7B-3.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/airoboros-l2-7B-3.0/README.md) |
| airoboros-l2-7B-gpt4-2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/airoboros-l2-7B-gpt4-2.0/README.md) |
| airoboros-l2-7B-gpt4-m2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/airoboros-l2-7B-gpt4-m2.0/README.md) |
| airoboros-l2-7b-gpt4-1.4.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/airoboros-l2-7b-gpt4-1.4.1/README.md) |
| airoboros-m-7B-3.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/airoboros-m-7B-3.0/README.md) |
| airoboros-m-7B-3.1.2-dare-0.85 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/airoboros-m-7B-3.1.2-dare-0.85/README.md) |
| airoboros-mistral2.2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/airoboros-mistral2.2-7B/README.md) |
| airochronos-33B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/airochronos-33B/README.md) |
| alfred-40B-1023 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 65024 | 0 | 8192 | [README.md](models/alfred-40B-1023/README.md) |
| all-MiniLM-L12-v2 | F16 F32 Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 30522 | 512 | 384 | [README.md](models/all-MiniLM-L12-v2/README.md) |
| all-MiniLM-L6-v2 | F16 F32 Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 30522 | 512 | 384 | [README.md](models/all-MiniLM-L6-v2/README.md) |
| apricot-wildflower-20 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/apricot-wildflower-20/README.md) |
| bagel-34b-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/bagel-34b-v0.2/README.md) |
| bagel-7B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/bagel-7B-v0.1/README.md) |
| bagel-8x7b-v0.2 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/bagel-8x7b-v0.2/README.md) |
| bagel-dpo-34b-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/bagel-dpo-34b-v0.2/README.md) |
| bagel-dpo-7B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/bagel-dpo-7B-v0.1/README.md) |
| bagel-dpo-8x7b-v0.2 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/bagel-dpo-8x7b-v0.2/README.md) |
| based-13b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/based-13b/README.md) |
| based-30B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/based-30B/README.md) |
| based-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/based-7B/README.md) |
| basilisk-7B-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/basilisk-7B-v0.2/README.md) |
| bling-stable-lm-3b-4e1t-v0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 4096 | 2560 | [README.md](models/bling-stable-lm-3b-4e1t-v0/README.md) |
| blossom-v3-baichuan2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 125696 | 4096 | 4096 | [README.md](models/blossom-v3-baichuan2-7B/README.md) |
| blossom-v3-mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/blossom-v3-mistral-7B/README.md) |
| blossom-v3_1-yi-34b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/blossom-v3_1-yi-34b/README.md) |
| bun_mistral_7b_v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/bun_mistral_7b_v2/README.md) |
| calm2-7B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 65024 | 32768 | 4096 | [README.md](models/calm2-7B-chat/README.md) |
| cat-v1.0-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/cat-v1.0-13B/README.md) |
| chronos-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/chronos-13B/README.md) |
| chronos-13b-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/chronos-13b-v2/README.md) |
| chronos-33b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/chronos-33b/README.md) |
| chronos-hermes-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/chronos-hermes-13B/README.md) |
| chronos-wizardlm-uc-scot-st-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/chronos-wizardlm-uc-scot-st-13B/README.md) |
| chronos007-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/chronos007-70B/README.md) |
| cinematika-7B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32003 | 32768 | 4096 | [README.md](models/cinematika-7B-v0.1/README.md) |
| claude2-alpaca-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/claude2-alpaca-13B/README.md) |
| claude2-alpaca-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/claude2-alpaca-7B/README.md) |
| cockatrice-7B-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/cockatrice-7B-v0.1/README.md) |
| deepmoney-34b-200k-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/deepmoney-34b-200k-base/README.md) |
| deepseek-coder-1.3b-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 2048 | [README.md](models/deepseek-coder-1.3b-base/README.md) |
| deepseek-coder-1.3b-instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 2048 | [README.md](models/deepseek-coder-1.3b-instruct/README.md) |
| deepseek-coder-33B-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 7168 | [README.md](models/deepseek-coder-33B-base/README.md) |
| deepseek-coder-33B-instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 7168 | [README.md](models/deepseek-coder-33B-instruct/README.md) |
| deepseek-coder-5.7bmqa-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 4096 | [README.md](models/deepseek-coder-5.7bmqa-base/README.md) |
| deepseek-coder-6.7B-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 4096 | [README.md](models/deepseek-coder-6.7B-base/README.md) |
| deepseek-coder-6.7B-instruct | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 4096 | [README.md](models/deepseek-coder-6.7B-instruct/README.md) |
| deepseek-llm-67b-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 102400 | 4096 | 8192 | [README.md](models/deepseek-llm-67b-base/README.md) |
| deepseek-llm-67b-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 102400 | 4096 | 8192 | [README.md](models/deepseek-llm-67b-chat/README.md) |
| deepseek-llm-7B-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 102400 | 4096 | 4096 | [README.md](models/deepseek-llm-7B-base/README.md) |
| deepseek-llm-7B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 102400 | 4096 | 4096 | [README.md](models/deepseek-llm-7B-chat/README.md) |
| deepsex-34b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/deepsex-34b/README.md) |
| deita-7B-v1.0-sft | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/deita-7B-v1.0-sft/README.md) |
| digital-socrates-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/digital-socrates-13B/README.md) |
| digital-socrates-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/digital-socrates-7B/README.md) |
| docsgpt-7B-mistral | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/docsgpt-7B-mistral/README.md) |
| dolphin-2.0-mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/dolphin-2.0-mistral-7B/README.md) |
| dolphin-2.1-mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/dolphin-2.1-mistral-7B/README.md) |
| dolphin-2.2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32002 | 4096 | 8192 | [README.md](models/dolphin-2.2-70B/README.md) |
| dolphin-2.2-yi-34b-200k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/dolphin-2.2-yi-34b-200k/README.md) |
| dolphin-2.2.1-AshhLimaRP-Mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/dolphin-2.2.1-AshhLimaRP-Mistral-7B/README.md) |
| dolphin-2.2.1-mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/dolphin-2.2.1-mistral-7B/README.md) |
| dolphin-2.5-mixtral-8x7b | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/dolphin-2.5-mixtral-8x7b/README.md) |
| dolphin-2.6-mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 32768 | 4096 | [README.md](models/dolphin-2.6-mistral-7B/README.md) |
| dolphin-2.6-mistral-7B-dpo | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 32768 | 4096 | [README.md](models/dolphin-2.6-mistral-7B-dpo/README.md) |
| dolphin-2.6-mistral-7B-dpo-laser | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 32768 | 4096 | [README.md](models/dolphin-2.6-mistral-7B-dpo-laser/README.md) |
| dolphin-2.6-mixtral-8x7b | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/dolphin-2.6-mixtral-8x7b/README.md) |
| dolphin-2.7-mixtral-8x7b | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/dolphin-2.7-mixtral-8x7b/README.md) |
| dolphin-2_2-yi-34b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 16384 | 7168 | [README.md](models/dolphin-2_2-yi-34b/README.md) |
| dolphin-2_6-phi-2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 51200 | 0 | 0 | [README.md](models/dolphin-2_6-phi-2/README.md) |
| dopeystableplats-3b-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 4096 | 2560 | [README.md](models/dopeystableplats-3b-v1/README.md) |
| dragon-mistral-7B-v0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/dragon-mistral-7B-v0/README.md) |
| dragon-yi-6B-v0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 4096 | [README.md](models/dragon-yi-6B-v0/README.md) |
| echidna-tiefigther-25 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/echidna-tiefigther-25/README.md) |
| em_german_13b_v01 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/em_german_13b_v01/README.md) |
| em_german_70b_v01 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/em_german_70b_v01/README.md) |
| em_german_7b_v01 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/em_german_7b_v01/README.md) |
| em_german_leo_mistral | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/em_german_leo_mistral/README.md) |
| em_german_mistral_v01 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/em_german_mistral_v01/README.md) |
| evolvedSeeker_1_3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32256 | 16384 | 2048 | [README.md](models/evolvedSeeker_1_3/README.md) |
| fin-llama-33B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/fin-llama-33B/README.md) |
| finance-LLM | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 0 | 4096 | [README.md](models/finance-LLM/README.md) |
| finance-LLM-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/finance-LLM-13B/README.md) |
| finance-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/finance-chat/README.md) |
| firefly-llama2-13B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 55296 | 4096 | 5120 | [README.md](models/firefly-llama2-13B-chat/README.md) |
| firefly-llama2-7B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 55296 | 4096 | 4096 | [README.md](models/firefly-llama2-7B-chat/README.md) |
| firefly-mixtral-8x7b | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/firefly-mixtral-8x7b/README.md) |
| gemma-2b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 256000 | 8192 | 2048 | [README.md](models/gemma-2b/README.md) |
| gemma-2b-it | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 256000 | 8192 | 2048 | [README.md](models/gemma-2b-it/README.md) |
| genz-13B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/genz-13B-v2/README.md) |
| go-bruins | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/go-bruins/README.md) |
| go-bruins-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/go-bruins-v2/README.md) |
| go-bruins-v2.1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/go-bruins-v2.1.1/README.md) |
| goliath-120b | Q2_K | 32000 | 4096 | 8192 | [README.md](models/goliath-120b/README.md) |
| gorilla-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/gorilla-7B/README.md) |
| gorilla-openfunctions-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/gorilla-openfunctions-v1/README.md) |
| guanaco-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/guanaco-13B/README.md) |
| guanaco-33B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/guanaco-33B/README.md) |
| hippogriff-30b-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/hippogriff-30b-chat/README.md) |
| huginnv1.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/huginnv1.2/README.md) |
| jackalope-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/jackalope-7B/README.md) |
| japanese-stablelm-base-beta-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/japanese-stablelm-base-beta-70B/README.md) |
| japanese-stablelm-instruct-beta-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/japanese-stablelm-instruct-beta-70B/README.md) |
| japanese-stablelm-instruct-beta-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/japanese-stablelm-instruct-beta-7B/README.md) |
| japanese-stablelm-instruct-gamma-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/japanese-stablelm-instruct-gamma-7B/README.md) |
| juanako-7B-UNA | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/juanako-7B-UNA/README.md) |
| juanako-7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/juanako-7B-v1/README.md) |
| koOpenChat-sft | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/koOpenChat-sft/README.md) |
| laser-dolphin-mixtral-2x7b-dpo | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/laser-dolphin-mixtral-2x7b-dpo/README.md) |
| law-LLM | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 0 | 4096 | [README.md](models/law-LLM/README.md) |
| law-LLM-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/law-LLM-13B/README.md) |
| law-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/law-chat/README.md) |
| leo-hessianai-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 5120 | [README.md](models/leo-hessianai-13B/README.md) |
| leo-hessianai-13B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32128 | 8192 | 5120 | [README.md](models/leo-hessianai-13B-chat/README.md) |
| leo-hessianai-13B-chat-bilingual | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32128 | 8192 | 5120 | [README.md](models/leo-hessianai-13B-chat-bilingual/README.md) |
| leo-hessianai-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 8192 | 8192 | [README.md](models/leo-hessianai-70B/README.md) |
| leo-hessianai-70B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32128 | 8192 | 8192 | [README.md](models/leo-hessianai-70B-chat/README.md) |
| leo-hessianai-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/leo-hessianai-7B/README.md) |
| leo-hessianai-7B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32128 | 8192 | 4096 | [README.md](models/leo-hessianai-7B-chat/README.md) |
| leo-hessianai-7B-chat-bilingual | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32128 | 8192 | 4096 | [README.md](models/leo-hessianai-7B-chat-bilingual/README.md) |
| lince-zero | Q4_0 Q4_1 Q5_0 Q5_1 Q8_0 | 65024 | 0 | 4544 | [README.md](models/lince-zero/README.md) |
| llama-13b-supercot | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/llama-13b-supercot/README.md) |
| llama-2-13B-German-Assistant-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/llama-2-13B-German-Assistant-v2/README.md) |
| llama-2-13B-Guanaco-QLoRA | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/llama-2-13B-Guanaco-QLoRA/README.md) |
| llama-2-13B-chat-limarp-v2-merged | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/llama-2-13B-chat-limarp-v2-merged/README.md) |
| llama-2-7B-Arguments | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/llama-2-7B-Arguments/README.md) |
| llama-2-7B-Guanaco-QLoRA | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/llama-2-7B-Guanaco-QLoRA/README.md) |
| llama-30b-supercot | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/llama-30b-supercot/README.md) |
| llama-polyglot-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/llama-polyglot-13B/README.md) |
| llama2-22B-daydreamer-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4098 | 6656 | [README.md](models/llama2-22B-daydreamer-v2/README.md) |
| llama2-7B-layla | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/llama2-7B-layla/README.md) |
| llama2-7b-chat-codeCherryPop-qLoRA | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/llama2-7b-chat-codeCherryPop-qLoRA/README.md) |
| llama2_70b_chat_uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 2048 | 8192 | [README.md](models/llama2_70b_chat_uncensored/README.md) |
| llama2_7b_chat_uncensored | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/llama2_7b_chat_uncensored/README.md) |
| llama2_7b_merge_orcafamily | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/llama2_7b_merge_orcafamily/README.md) |
| llemma_34b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 8192 | [README.md](models/llemma_34b/README.md) |
| llemma_7b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32016 | 4096 | 4096 | [README.md](models/llemma_7b/README.md) |
| loyal-piano-m7 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/loyal-piano-m7/README.md) |
| lzlv_70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/lzlv_70B/README.md) |
| mamba-2.8b-hf | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50280 | 0 | 2560 | [README.md](models/mamba-2.8b-hf/README.md) |
| manticore-13b-chat-pyg | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/manticore-13b-chat-pyg/README.md) |
| med42-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 2048 | 8192 | [README.md](models/med42-70B/README.md) |
| medalpaca-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 0 | 5120 | [README.md](models/medalpaca-13B/README.md) |
| medicine-LLM | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 0 | 4096 | [README.md](models/medicine-LLM/README.md) |
| medicine-LLM-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/medicine-LLM-13B/README.md) |
| medicine-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/medicine-chat/README.md) |
| meditron-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32017 | 4096 | 8192 | [README.md](models/meditron-70B/README.md) |
| meditron-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32017 | 2048 | 4096 | [README.md](models/meditron-7B/README.md) |
| meditron-7B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32017 | 2048 | 4096 | [README.md](models/meditron-7B-chat/README.md) |
| merlyn-education-corpus-qa-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/merlyn-education-corpus-qa-v2/README.md) |
| merlyn-education-safety | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50688 | 2048 | 5120 | [README.md](models/merlyn-education-safety/README.md) |
| mindy-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/mindy-7B/README.md) |
| minotaur-13B-fixed | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/minotaur-13B-fixed/README.md) |
| mistral-7B-dpo-v5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/mistral-7B-dpo-v5/README.md) |
| mistral-7B-finetuned-orca-dpo-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/mistral-7B-finetuned-orca-dpo-v2/README.md) |
| mistral-ft-optimized-1218 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/mistral-ft-optimized-1218/README.md) |
| mistral-ft-optimized-1227 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/mistral-ft-optimized-1227/README.md) |
| mixtralnt-4x7b-test | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/mixtralnt-4x7b-test/README.md) |
| model_007-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/model_007-70B/README.md) |
| neural-chat-7B-v3-1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/neural-chat-7B-v3-1/README.md) |
| neural-chat-7B-v3-2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/neural-chat-7B-v3-2/README.md) |
| neural-chat-7B-v3-3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/neural-chat-7B-v3-3/README.md) |
| neural-chat-7B-v3-3-wizardmath-dare-me | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/neural-chat-7B-v3-3-wizardmath-dare-me/README.md) |
| neuronovo-7B-v0.3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/neuronovo-7B-v0.3/README.md) |
| nomic-embed-text-v1 | F16 F32 Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 30528 | 0 | 768 | [README.md](models/nomic-embed-text-v1/README.md) |
| nomic-embed-text-v1.5 | F16 F32 Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 30528 | 0 | 768 | [README.md](models/nomic-embed-text-v1.5/README.md) |
| nontoxic-bagel-34b-v0.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 200000 | 7168 | [README.md](models/nontoxic-bagel-34b-v0.2/README.md) |
| notus-7B-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/notus-7B-v1/README.md) |
| notux-8x7b-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/notux-8x7b-v1/README.md) |
| nsql-llama-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/nsql-llama-2-7B/README.md) |
| nucleus-22B-token-500B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/nucleus-22B-token-500B/README.md) |
| open-instruct-human-mix-65B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32001 | 2048 | 8192 | [README.md](models/open-instruct-human-mix-65B/README.md) |
| open-llama-3b-v2-wizard-evol-instuct-v2-196k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 3200 | [README.md](models/open-llama-3b-v2-wizard-evol-instuct-v2-196k/README.md) |
| openbuddy-coder-34b-v11-bf16 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 37632 | 16384 | 8192 | [README.md](models/openbuddy-coder-34b-v11-bf16/README.md) |
| openbuddy-deepseek-67b-v15-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 102400 | 4096 | 8192 | [README.md](models/openbuddy-deepseek-67b-v15-base/README.md) |
| openbuddy-llama2-13b64k-v15 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 37632 | 65536 | 5120 | [README.md](models/openbuddy-llama2-13b64k-v15/README.md) |
| openbuddy-llama2-34b-v11.1-bf16 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 37632 | 16384 | 8192 | [README.md](models/openbuddy-llama2-34b-v11.1-bf16/README.md) |
| openbuddy-llama2-70B-v13-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 37632 | 4096 | 8192 | [README.md](models/openbuddy-llama2-70B-v13-base/README.md) |
| openbuddy-llama2-70B-v13.2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 37632 | 4096 | 8192 | [README.md](models/openbuddy-llama2-70B-v13.2/README.md) |
| openbuddy-mistral-7B-v13 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 36608 | 32768 | 4096 | [README.md](models/openbuddy-mistral-7B-v13/README.md) |
| openbuddy-mistral-7B-v13-base | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 36608 | 32768 | 4096 | [README.md](models/openbuddy-mistral-7B-v13-base/README.md) |
| openbuddy-mistral-7B-v13.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 36608 | 32768 | 4096 | [README.md](models/openbuddy-mistral-7B-v13.1/README.md) |
| openbuddy-mixtral-7bx8-v16.3-32k | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 36608 | 32768 | 4096 | [README.md](models/openbuddy-mixtral-7bx8-v16.3-32k/README.md) |
| openbuddy-mixtral-8x7b-v15.2 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 36608 | 32768 | 4096 | [README.md](models/openbuddy-mixtral-8x7b-v15.2/README.md) |
| openbuddy-mixtral-8x7b-v15.4 | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 36608 | 32768 | 4096 | [README.md](models/openbuddy-mixtral-8x7b-v15.4/README.md) |
| openbuddy-openllama-7B-v12-bf16 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 37120 | 4096 | 4096 | [README.md](models/openbuddy-openllama-7B-v12-bf16/README.md) |
| openbuddy-zephyr-7B-v14.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 36608 | 32768 | 4096 | [README.md](models/openbuddy-zephyr-7B-v14.1/README.md) |
| openchat-3.5-0106 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/openchat-3.5-0106/README.md) |
| openchat-3.5-1210 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/openchat-3.5-1210/README.md) |
| openchat-3.5-1210-Seraph-Slerp | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/openchat-3.5-1210-Seraph-Slerp/README.md) |
| openchat-3.5-1210-starling-slerp | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/openchat-3.5-1210-starling-slerp/README.md) |
| openchat_3.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/openchat_3.5/README.md) |
| openchat_3.5-16k | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/openchat_3.5-16k/README.md) |
| openchat_v3.2_super | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 4096 | 5120 | [README.md](models/openchat_v3.2_super/README.md) |
| openinstruct-mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 32768 | 4096 | [README.md](models/openinstruct-mistral-7B/README.md) |
| openthaigpt-1.0.0-beta-13B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 41088 | 4096 | 5120 | [README.md](models/openthaigpt-1.0.0-beta-13B-chat/README.md) |
| opus-v0-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/opus-v0-70B/README.md) |
| opus-v0-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/opus-v0-7B/README.md) |
| opus-v0.5-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/opus-v0.5-70B/README.md) |
| orangetin-OpenHermes-Mixtral-8x7B | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/orangetin-OpenHermes-Mixtral-8x7B/README.md) |
| orca_mini_v3_13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/orca_mini_v3_13B/README.md) |
| orca_mini_v3_70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/orca_mini_v3_70B/README.md) |
| orca_mini_v3_7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/orca_mini_v3_7B/README.md) |
| pee | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/pee/README.md) |
| phi-2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 51200 | 2048 | 2560 | [README.md](models/phi-2/README.md) |
| phi-2-dpo | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50296 | 0 | 0 | [README.md](models/phi-2-dpo/README.md) |
| phi-2-electrical-engineering | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 51200 | 2048 | 2560 | [README.md](models/phi-2-electrical-engineering/README.md) |
| phi-2-orange | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 51200 | 0 | 0 | [README.md](models/phi-2-orange/README.md) |
| platypus-yi-34b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/platypus-yi-34b/README.md) |
| prometheus-13B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/prometheus-13B-v1.0/README.md) |
| prometheus-7B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/prometheus-7B-v1.0/README.md) |
| psyonic-cetacean-20B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/psyonic-cetacean-20B/README.md) |
| qCammel-13 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/qCammel-13/README.md) |
| qCammel-70-x | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/qCammel-70-x/README.md) |
| quantum-dpo-v0.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/quantum-dpo-v0.1/README.md) |
| quantum-v0.01 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/quantum-v0.01/README.md) |
| rocket-3B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 4096 | 2560 | [README.md](models/rocket-3B/README.md) |
| rpguild-chatml-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32032 | 4096 | 5120 | [README.md](models/rpguild-chatml-13B/README.md) |
| sabia-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 4096 | [README.md](models/sabia-7B/README.md) |
| saiga_mistral_7b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/saiga_mistral_7b/README.md) |
| samantha-1.2-mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 32768 | 4096 | [README.md](models/samantha-1.2-mistral-7B/README.md) |
| samantha-mistral-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/samantha-mistral-7B/README.md) |
| samantha-mistral-instruct-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/samantha-mistral-instruct-7B/README.md) |
| sheep-duck-llama-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/sheep-duck-llama-2-13B/README.md) |
| sheep-duck-llama-2-70B-v1.1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/sheep-duck-llama-2-70B-v1.1/README.md) |
| smartyplats-7B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/smartyplats-7B-v2/README.md) |
| smol-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32002 | 8192 | 4096 | [README.md](models/smol-7B/README.md) |
| sonya-7B-x8-MoE | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/sonya-7B-x8-MoE/README.md) |
| sonya-medium-x8-MoE | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M | 32000 | 8192 | 4096 | [README.md](models/sonya-medium-x8-MoE/README.md) |
| speechless-code-mistral-7B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/speechless-code-mistral-7B-v1.0/README.md) |
| speechless-codellama-34b-v2.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/speechless-codellama-34b-v2.0/README.md) |
| speechless-mistral-7B-dare-0.85 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/speechless-mistral-7B-dare-0.85/README.md) |
| speechless-mistral-dolphin-orca-platypus-samantha-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/speechless-mistral-dolphin-orca-platypus-samantha-7B/README.md) |
| speechless-mistral-moloras-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/speechless-mistral-moloras-7B/README.md) |
| speechless-tora-code-7B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 4096 | [README.md](models/speechless-tora-code-7B-v1.0/README.md) |
| sqlcoder | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 49152 | 0 | 0 | [README.md](models/sqlcoder/README.md) |
| sqlcoder-34b-alpha | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 16384 | 8192 | [README.md](models/sqlcoder-34b-alpha/README.md) |
| sqlcoder-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/sqlcoder-7B/README.md) |
| sqlcoder2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 49152 | 0 | 0 | [README.md](models/sqlcoder2/README.md) |
| stable-code-3b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 16384 | 2560 | [README.md](models/stable-code-3b/README.md) |
| stable-vicuna-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/stable-vicuna-13B/README.md) |
| stablelm-zephyr-3b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 4096 | 2560 | [README.md](models/stablelm-zephyr-3b/README.md) |
| stockmark-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50000 | 2048 | 5120 | [README.md](models/stockmark-13B/README.md) |
| storytime-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/storytime-13B/README.md) |
| supermario-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/supermario-v2/README.md) |
| tigerbot-13B-chat-v5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 65112 | 2048 | 5120 | [README.md](models/tigerbot-13B-chat-v5/README.md) |
| tigerbot-70B-chat-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 65112 | 4096 | 8192 | [README.md](models/tigerbot-70B-chat-v2/README.md) |
| tigerbot-70B-chat-v4 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 65112 | 4096 | 8192 | [README.md](models/tigerbot-70B-chat-v4/README.md) |
| tora-13B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 5120 | [README.md](models/tora-13B-v1.0/README.md) |
| tora-70B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32001 | 4096 | 8192 | [README.md](models/tora-70B-v1.0/README.md) |
| tora-7B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 4096 | 4096 | [README.md](models/tora-7B-v1.0/README.md) |
| tora-code-13B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 5120 | [README.md](models/tora-code-13B-v1.0/README.md) |
| tora-code-34b-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 8192 | [README.md](models/tora-code-34b-v1.0/README.md) |
| tora-code-7B-v1.0 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 16384 | 4096 | [README.md](models/tora-code-7B-v1.0/README.md) |
| toxicqa-Llama2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/toxicqa-Llama2-13B/README.md) |
| toxicqa-Llama2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/toxicqa-Llama2-7B/README.md) |
| trinity-v1.2-x8-MoE | Q2_K Q3_K_M Q4_0 Q4_K_M Q5_0 Q5_K_M Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/trinity-v1.2-x8-MoE/README.md) |
| tulu-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 5120 | [README.md](models/tulu-13B/README.md) |
| tulu-2-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 5120 | [README.md](models/tulu-2-13B/README.md) |
| tulu-2-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 8192 | 8192 | [README.md](models/tulu-2-70B/README.md) |
| tulu-2-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/tulu-2-7B/README.md) |
| tulu-2-dpo-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 5120 | [README.md](models/tulu-2-dpo-13B/README.md) |
| tulu-2-dpo-70B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 8192 | 8192 | [README.md](models/tulu-2-dpo-70B/README.md) |
| tulu-2-dpo-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 8192 | 4096 | [README.md](models/tulu-2-dpo-7B/README.md) |
| tulu-30B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 6656 | [README.md](models/tulu-30B/README.md) |
| tulu-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 4096 | [README.md](models/tulu-7B/README.md) |
| typhoon-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 35219 | 32768 | 4096 | [README.md](models/typhoon-7B/README.md) |
| una-cybertron-7B-v2 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/una-cybertron-7B-v2/README.md) |
| una-cybertron-7B-v3-OMA | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/una-cybertron-7B-v3-OMA/README.md) |
| una-xaberius-34b-v1beta | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 4096 | 7168 | [README.md](models/una-xaberius-34b-v1beta/README.md) |
| upstage-llama-30b-instruct-2048 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/upstage-llama-30b-instruct-2048/README.md) |
| v1olet_marcoroni-go-bruins-merge-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/v1olet_marcoroni-go-bruins-merge-7B/README.md) |
| v1olet_merged_dpo_7B_v3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/v1olet_merged_dpo_7B_v3/README.md) |
| vicuna-13B-v1.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/vicuna-13B-v1.5/README.md) |
| vicuna-13B-v1.5-16K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 5120 | [README.md](models/vicuna-13B-v1.5-16K/README.md) |
| vicuna-33B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/vicuna-33B/README.md) |
| vicuna-33B-coder | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 6656 | [README.md](models/vicuna-33B-coder/README.md) |
| vicuna-7B-v1.5 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/vicuna-7B-v1.5/README.md) |
| vicuna-7B-v1.5-16K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 4096 | 4096 | [README.md](models/vicuna-7B-v1.5-16K/README.md) |
| vietnamese-llama2-7B-40GB | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 46303 | 4096 | 4096 | [README.md](models/vietnamese-llama2-7B-40GB/README.md) |
| vigogne-2-70B-chat | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S | 32000 | 4096 | 8192 | [README.md](models/vigogne-2-70B-chat/README.md) |
| wizard-mega-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/wizard-mega-13B/README.md) |
| wizard-vicuna-13B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 2048 | 5120 | [README.md](models/wizard-vicuna-13B/README.md) |
| wizardLM-7B | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32001 | 2048 | 4096 | [README.md](models/wizardLM-7B/README.md) |
| xDAN-L1-Chat-RL-v1 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/xDAN-L1-Chat-RL-v1/README.md) |
| yayi2-30B-llama | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 81920 | 4096 | 7168 | [README.md](models/yayi2-30B-llama/README.md) |
| yi-34B-v3 | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 64000 | 8192 | 7168 | [README.md](models/yi-34B-v3/README.md) |
| zephyr-7B-alpha | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/zephyr-7B-alpha/README.md) |
| zephyr-7B-beta | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/zephyr-7B-beta/README.md) |
| zephyr-7B-beta-pl | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 32000 | 32768 | 4096 | [README.md](models/zephyr-7B-beta-pl/README.md) |
| zephyr-quiklang-3b | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 4096 | 2560 | [README.md](models/zephyr-quiklang-3b/README.md) |
| zephyr-quiklang-3b-4K | Q2_K Q3_K_L Q3_K_M Q3_K_S Q4_0 Q4_K_M Q4_K_S Q5_0 Q5_K_M Q5_K_S Q6_K Q8_0 | 50304 | 4096 | 2560 | [README.md](models/zephyr-quiklang-3b-4K/README.md) |
## Memory/Disk Requirements
As the models are currently fully loaded into memory, you will need adequate disk space to save them and sufficient RAM to load them. At the moment, memory and disk requirements are the same.
| Model | Original size | Quantized size (4-bit) |
|------:|--------------:|-----------------------:|
| 7B | 13 GB | 3.9 GB |
| 13B | 24 GB | 7.8 GB |
| 30B | 60 GB | 19.5 GB |
| 65B | 120 GB | 38.5 GB |
## Quantization
Several quantization methods are supported. They differ in the resulting model disk size and inference speed.
*(outdated)*
| Model | Measure | F16 | Q4_0 | Q4_1 | Q5_0 | Q5_1 | Q8_0 |
|------ |--------------|--------|--------|--------|--------|--------|--------|
| 7B | perplexity | 5.9066 | 6.1565 | 6.0912 | 5.9862 | 5.9481 | 5.9070 |
| 7B | file size | 13.0G | 3.5G | 3.9G | 4.3G | 4.7G | 6.7G |
| 7B | ms/tok @ 4th | 127 | 55 | 54 | 76 | 83 | 72 |
| 7B | ms/tok @ 8th | 122 | 43 | 45 | 52 | 56 | 67 |
| 7B | bits/weight | 16.0 | 4.5 | 5.0 | 5.5 | 6.0 | 8.5 |
| 13B | perplexity | 5.2543 | 5.3860 | 5.3608 | 5.2856 | 5.2706 | 5.2548 |
| 13B | file size | 25.0G | 6.8G | 7.6G | 8.3G | 9.1G | 13G |
| 13B | ms/tok @ 4th | - | 103 | 105 | 148 | 160 | 131 |
| 13B | ms/tok @ 8th | - | 73 | 82 | 98 | 105 | 128 |
| 13B | bits/weight | 16.0 | 4.5 | 5.0 | 5.5 | 6.0 | 8.5 |
## License
This project is licensed under the Mozilla Public License Version 2.0 License - see the [LICENSE](LICENSE) file for details.