Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/rayfernando1337/llm-calc
Instantly calculate the maximum size of quantized language models that can fit in your available RAM, helping you optimize your models for inference.
https://github.com/rayfernando1337/llm-calc
ai inference llm tools
Last synced: 12 days ago
JSON representation
Instantly calculate the maximum size of quantized language models that can fit in your available RAM, helping you optimize your models for inference.
- Host: GitHub
- URL: https://github.com/rayfernando1337/llm-calc
- Owner: RayFernando1337
- License: mit
- Created: 2024-04-27T19:04:20.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-10-27T18:54:44.000Z (16 days ago)
- Last Synced: 2024-10-27T19:36:42.917Z (16 days ago)
- Topics: ai, inference, llm, tools
- Language: TypeScript
- Homepage: https://llm-calc.rayfernando.ai
- Size: 181 KB
- Stars: 63
- Watchers: 1
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# LLM RAM Requirement Calculator
This project is a React + TypeScript + Vite application that calculates the maximum number of parameters that can fit in RAM for different quantization levels of large language models (LLMs).
## Features
- Calculate maximum parameters based on available RAM, OS usage, context window size, and quantization level
- Interactive UI built with React and TypeScript
- Fast development with Vite and Hot Module Replacement (HMR)
- Styling with Tailwind CSS
## Setup Instructions
1. Clone the repository:
```bash
git clone https://github.com/RayFernando1337/LLM-Calc.git
cd LLM-Calc
```
2. Install dependencies:
```bash
bun install
```
3. Start the development server:
```bash
bun run dev
```
4. Open your browser and navigate to `http://localhost:5173` (or the port specified in the console output).
## Building for Production
To create a production build:
```bash
bun run build
```
The built files will be in the `dist` directory.
## Usage
Once the application is running, you can:
1. Enter your available RAM in GB
2. Estimate OS RAM usage in GB
3. Set the context window size (number of tokens)
4. Choose a quantization level (bits per parameter)
The app will calculate and display the maximum number of parameters your setup can handle in billions.
## Calculation Logic
The calculation logic remains the same as in the previous version:
1. Convert available RAM and OS overhead from GB to bytes
2. Calculate memory required for the context window
3. Calculate usable RAM by subtracting OS overhead and context window memory
4. Convert quantization level from bits to bytes per parameter
5. Calculate maximum number of parameters
6. Convert result to billions of parameters for display
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
## License
[MIT License](LICENSE)
## Acknowledgements
This project was bootstrapped with Vite and uses React, TypeScript, and Tailwind CSS. It uses Bun as the JavaScript runtime and package manager.