https://github.com/siddhesh-agarwal/openai-ft-validate
A Go-based CLI tool that verifies the structure of JSONL files for OpenAI fine-tuning.
https://github.com/siddhesh-agarwal/openai-ft-validate
cli file-validation golang jsonl openai
Last synced: about 1 month ago
JSON representation
A Go-based CLI tool that verifies the structure of JSONL files for OpenAI fine-tuning.
- Host: GitHub
- URL: https://github.com/siddhesh-agarwal/openai-ft-validate
- Owner: Siddhesh-Agarwal
- Created: 2024-12-21T11:45:49.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2024-12-28T08:54:07.000Z (5 months ago)
- Last Synced: 2025-02-14T19:51:49.863Z (3 months ago)
- Topics: cli, file-validation, golang, jsonl, openai
- Language: Go
- Homepage:
- Size: 10.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# `openai-ft-validate` CLI Tool
## Overview
`openai-ft-validate` is a Command-Line Interface (CLI) tool written in Go, designed to validate the structure of JSONL files used in OpenAI fine-tuning processes. This tool ensures that the input files adhere to OpenAI's formatting requirements, helping users avoid errors during fine-tuning.
---
## Features
- Validates JSONL files for structural correctness.
- Checks for required fields and their data types.
- Provides clear error messages for invalid entries.
- Ensures compatibility with OpenAI fine-tuning guidelines.
- Lightweight and easy to use.
---
## Installation
To install `openai-ft-validate`, ensure you have Go installed, then clone the repository and build the binary:
```bash
# Clone the repository
git clone https://github.com/Siddhesh-Agarwal/openai-ft-validate
cd openai-ft-validate
# Build the binary
go build -o openai-ft-validate
```
Alternatively, you can download a prebuilt binary (if available) from the [releases page](https://github.com/Siddhesh-Agarwal/openai-ft-validate/releases).
---
## Usage
The tool can be invoked directly from the command line. Below are the usage details:
### Basic Command
```bash
./openai-ft-validate
```
### Example
To validate a JSONL file named `training_data.jsonl`:
```bash
./openai-ft-validate training_data.jsonl
```
---
## Output
- **Success:** If the file is valid, you will see a message like:
```sh
The JSONL is valid 🎉
```
- **Failure:** If the file contains errors, the tool will output details, such as:
```sh
Line 3: Missing required field "prompt".
Line 7: Invalid "role" programmer found
The JSONL is invalid ❌
```
---
## Requirements
- Go 1.16+
---
## Development
For contributors, clone the repository and build the project:
```bash
git clone https://github.com/Siddhesh-Agarwal/openai-ft-validate
cd openai-ft-validate
# Build the binary
go build
```
Run tests to ensure functionality:
---
## License
This tool is licensed under [MIT License](https://github.com/Siddhesh-Agarwal/openai-ft-validate/blob/main/LICENSE).
---
## Contributions
Contributions are welcome! Please fork the repository, make your changes, and submit a pull request. Ensure your code follows the established coding standards and includes tests for any new features.
---
## Contact
For any questions or issues, please open an issue on the [GitHub repository](https://github.com/Siddhesh-Agarwal/openai-ft-validate) or contact [Siddhesh Agarwal](mailto:[email protected]).