https://github.com/sammcj/closing-the-gap
Open vs Closed LLM Benchmark Trends
https://github.com/sammcj/closing-the-gap
Last synced: 5 months ago
JSON representation
Open vs Closed LLM Benchmark Trends
- Host: GitHub
- URL: https://github.com/sammcj/closing-the-gap
- Owner: sammcj
- License: mit
- Archived: true
- Created: 2024-07-10T22:38:50.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-10-18T19:13:57.000Z (12 months ago)
- Last Synced: 2025-03-18T02:24:33.379Z (7 months ago)
- Language: JavaScript
- Size: 1.15 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# LLM Benchmark Visualisations
**!! DISCLAIMER: This is work in progress experiment in early alpha stages, there's a lot of work to be done to make this a useful tool !!**
---
This project is designed to visualise and track the performance of various Large Language Models (LLMs) across different benchmarks. The visualisations aim help in understanding trends, comparing models, and predicting future performances.
## Features
- **Data Entry**: Easily add new benchmark data for models.
- **Visualisation**: Interactive charts showing model performance over time.
- **Predictive Analysis**: Predict future performances based on historical data.
## Getting Started
### Prerequisites
- Node.js (v22+)
### Installation
1. Clone the repository:
```bash
git clone https://github.com/sammcj/closing-the-gap.git
cd closing-the-gap
```
2. Install dependencies:
```bash
npm install
```
3. Start the development server:
```bash
npm start
```
4. Access the application in your browser at `http://localhost:3000`.
## Project Structure
The project is structured as follows:
- **`public/`**: Static index.html.
- **`src/`**: Source code for the application.
- **`components/`**: Reusable UI components.
- `DataEntryForm.js`: Form to add new benchmark data.
- `LLMBenchmarkVisualisation.js`: Component to visualise benchmark data using ChartJS.
- `LLMBenchmarkDashboard.js`: Dashboard to display benchmark data and predictions.
- `LeftPanel.js`: Side panel to display model information.
- `config.js`: Configuration settings for the application, including chart colors and titles.
- `App.js`: Main application component that integrates all other components.
- `server.js`: Express server to serve static files and API endpoints.
- **`ingest/`**: Scripts to aid with data ingestion (not used by the app itself).
- **`package.json`**: Project metadata and scripts.
- **`llm_bechmarks.db`**: SQLite database to store benchmark data.
## Usage
1. **GUI Data Entry**: Use the `DataEntryForm` component to add new benchmark data for models. This includes entering dates, selecting models, benchmarks, scores, and whether the model is open or closed.
2. **CLI Data Entry**: Add correctly formatted JSON benchmark results to `ingest/import.json` and run `node ingest/ingest.js`
3. **Visualisation**: The `LLMBenchmarkVisualisation` component provides interactive charts that show the performance of different models over time. Predictions are also provided based on historical data trends.
4. **Predictive Analysis**: Historical data is used to predict future performances, helping in understanding model growth and potential improvements.
## Contributing
Contributions are welcome! Please follow these steps:
1. Fork the repository.
2. Create a new branch for your feature or bug fix.
3. Make your changes and test them thoroughly.
4. Submit a pull request with a clear description of your changes.
## License
Copyright 2024 Sam McLeod
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.