{"id":16528348,"url":"https://github.com/danvk/gpt-batch-manager","last_synced_at":"2025-09-12T19:32:04.328Z","repository":{"id":257585155,"uuid":"858767073","full_name":"danvk/gpt-batch-manager","owner":"danvk","description":"Tools for splitting jobs across multiple OpenAI batches","archived":false,"fork":false,"pushed_at":"2024-09-17T16:15:27.000Z","size":32,"stargazers_count":2,"open_issues_count":4,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-31T21:42:43.562Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danvk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-17T13:55:13.000Z","updated_at":"2024-09-29T21:53:44.000Z","dependencies_parsed_at":"2024-09-17T15:28:13.048Z","dependency_job_id":null,"html_url":"https://github.com/danvk/gpt-batch-manager","commit_stats":null,"previous_names":["danvk/gpt-batch-manager"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danvk%2Fgpt-batch-manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danvk%2Fgpt-batch-manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danvk%2Fgpt-batch-manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danvk%2Fgpt-batch-manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danvk","download_url":"https://codeload.github.com/danvk/gpt-batch-manager/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232780148,"owners_count":18575475,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T17:39:49.046Z","updated_at":"2025-01-06T20:12:45.253Z","avatar_url":"https://github.com/danvk.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gpt-batch-manager\n\nThe [OpenAI Batch API] is great for submitting batch jobs, but it has some strict [limits]:\n\n- gpt-4o: 90,000 tokens (Tier 1), 1.35M (Tier 2)\n- gpt-4o-mini: 2,000,000 tokens (Tier 1), 20M (Tier 2)\n\nIf you want to submit larger batches than your current tier allows, you'll have to split\nthe work into multiple batches. This project helps you do that by providing two tools:\n\n1. `gpt-batch-splitter`: Take one or more JSONL files and split them into shards that\n   fit within your tier batch queue limit.\n2. `gpt-batch-manager`: Submit multiple batches to OpenAI, one at a time.\n\n## Usage\n\nInstall:\n\n    pip install gpt-batch-manager\n\nTo use the batch manager, you'll need to set an `OPENAI_API_KEY` environment variable.\nThe best way to do this is by creating a `.env` file in your working directory. See the\n[OpenAI Python API docs][dotenv] for details.\n\n### Batch Splitter\n\nSay your tasks are in `largefile1.jsonl` and `largefile2.jsonl`. You want to submit them\nto the OpenAI batch API for gpt-4o-mini and you're on Tier 1, so you need to make them\nfit within a 2M token limit.\n\nTo produce shards with fewer than 2M tokens, run:\n\n    gpt-batch-splitter 1900000 largefile1.jsonl largefile2.jsonl\n\nThis will [estimate] the number of tokens in each request and divvy them up accordingly.\nIt will produce output files like:\n\n    shard-000-of-079.jsonl\n    shard-001-of-079.jsonl\n    ...\n    shard-078-of-079.jsonl\n\nSince the splitter has to estimate the number of tokens in each request, it's best to\nuse a number somewhat below the limit.\n\n### Batch Manager\n\nTo stay within your batch limit, you have to submit batch files one-by-one. This is what\n`gpt-batch-manager` does. Pass it a set of JSONL files:\n\n    gpt-batch-manager shard-???-of-???.jsonl\n\nMake sure you have an `OPENAI_API_KEY` environment variable set (see above). This will\nreport the status of each batch as it progresses. Output will eventually appear in:\n\n    shard-000-of-079.output.jsonl\n    shard-001-of-079.output.jsonl\n    ...\n\nThis process is fully resumable. You can Ctrl-C it at any time and it will pick up where\nit left off. It stores state in `/tmp/batch-status.json`. If something goes wrong, you\ncould try deleting that file to reset.\n\n[OpenAI Batch API]: https://platform.openai.com/docs/guides/batch\n[limits]: https://platform.openai.com/docs/guides/rate-limits/usage-tiers?context=tier-one\n[dotenv]: https://github.com/openai/openai-python?tab=readme-ov-file#usage\n[estimate]: https://openai.com/api/pricing/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanvk%2Fgpt-batch-manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanvk%2Fgpt-batch-manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanvk%2Fgpt-batch-manager/lists"}