{"id":13645775,"url":"https://github.com/openai/gpt-2-output-dataset","last_synced_at":"2025-05-15T11:03:48.460Z","repository":{"id":42187312,"uuid":"184690694","full_name":"openai/gpt-2-output-dataset","owner":"openai","description":"Dataset of GPT-2 outputs for research in detection, biases, and more","archived":false,"fork":false,"pushed_at":"2023-12-13T03:03:19.000Z","size":272,"stargazers_count":1972,"open_issues_count":30,"forks_count":549,"subscribers_count":74,"default_branch":"master","last_synced_at":"2025-04-07T05:02:52.201Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-05-03T02:58:09.000Z","updated_at":"2025-04-07T02:51:20.000Z","dependencies_parsed_at":"2023-02-18T09:30:58.622Z","dependency_job_id":"ff3dfee9-59a5-4853-910e-4d67b29f7c98","html_url":"https://github.com/openai/gpt-2-output-dataset","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fgpt-2-output-dataset","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fgpt-2-output-dataset/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fgpt-2-output-dataset/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fgpt-2-output-dataset/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openai","download_url":"https://codeload.github.com/openai/gpt-2-output-dataset/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248910961,"owners_count":21182011,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:02:41.513Z","updated_at":"2025-04-14T15:52:31.247Z","avatar_url":"https://github.com/openai.png","language":"Python","funding_links":[],"categories":["Others","LLMs","Python"],"sub_categories":[],"readme":"# gpt-2-output-dataset\n\nThis dataset contains:\n- 250K documents from the WebText test set\n- For each GPT-2 model (trained on the WebText training set), 250K random samples (temperature 1, no truncation) and 250K samples generated with Top-K 40 truncation\n\nWe look forward to the research produced using this data!\n\n### Download\n\nFor each model, we have a training split of 250K generated examples, as well as validation and test splits of 5K examples.\n\nAll data is located in Google Cloud Storage, under the directory `gs://gpt-2/output-dataset/v1`.  (NOTE: everything has been migrated to Azure `https://openaipublic.blob.core.windows.net/gpt-2/output-dataset/v1/`)\n\nThere, you will find files:\n\n- `webtext.${split}.jsonl`\n- `small-117M.${split}.jsonl`\n- `small-117M-k40.${split}.jsonl`\n- `medium-345M.${split}.jsonl`\n- `medium-345M-k40.${split}.jsonl`\n- `large-762M.${split}.jsonl`\n- `large-762M-k40.${split}.jsonl`\n- `xl-1542M.${split}.jsonl`\n- `xl-1542M-k40.${split}.jsonl`\n\nwhere split is one of `train`, `test`, and `valid`.\n\nWe've provided a script to download all of them, in `download_dataset.py`.\n\n#### Finetuned model samples\n\nAdditionally, we encourage research on detection of finetuned models.  We have released data under `gs://gpt-2/output-dataset/v1-amazonfinetune/` with samples from a GPT-2 full model finetuned to output Amazon reviews.\n\n### Detectability baselines\n\nWe're interested in seeing research in detectability of GPT-2 model family generations.\n\nWe provide some [initial analysis](detection.md) of two baselines, as well as [code](./baseline.py) for the better baseline.\n\nOverall, we are able to achieve accuracies in the mid-90s for Top-K 40 generations, and mid-70s to high-80s (depending on model size) for random generations.  We also find some evidence that adversaries can evade detection via finetuning from released models.\n\n### Data removal requests\n\nIf you believe your work is included in WebText and would like us to remove it, please let us know at webtextdata@openai.com.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenai%2Fgpt-2-output-dataset","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenai%2Fgpt-2-output-dataset","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenai%2Fgpt-2-output-dataset/lists"}