{"id":13958458,"url":"https://github.com/google-research-datasets/conceptual-captions","last_synced_at":"2025-04-05T12:04:46.426Z","repository":{"id":39734331,"uuid":"132960467","full_name":"google-research-datasets/conceptual-captions","owner":"google-research-datasets","description":"Conceptual Captions is a dataset containing (image-URL, caption) pairs designed for the training and evaluation of machine learned image captioning systems. ","archived":false,"fork":false,"pushed_at":"2021-08-21T16:18:14.000Z","size":1299,"stargazers_count":532,"open_issues_count":9,"forks_count":26,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-03-29T11:07:22.785Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-research-datasets.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-05-10T22:24:40.000Z","updated_at":"2025-03-24T05:09:22.000Z","dependencies_parsed_at":"2022-07-18T03:00:38.889Z","dependency_job_id":null,"html_url":"https://github.com/google-research-datasets/conceptual-captions","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research-datasets%2Fconceptual-captions","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research-datasets%2Fconceptual-captions/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research-datasets%2Fconceptual-captions/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research-datasets%2Fconceptual-captions/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-research-datasets","download_url":"https://codeload.github.com/google-research-datasets/conceptual-captions/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247332602,"owners_count":20921853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-08T13:01:36.523Z","updated_at":"2025-04-05T12:04:46.404Z","avatar_url":"https://github.com/google-research-datasets.png","language":"Shell","funding_links":[],"categories":["其他_机器视觉"],"sub_categories":["网络服务_其他"],"readme":"# Conceptual Captions Dataset\n\nConceptual Captions is a dataset containing (image-URL, caption) pairs designed\nfor the training and evaluation of machine learned image captioning systems.\n\n## Downloads\nSee \u003chttp://ai.google.com/research/ConceptualCaptions\u003e for details.\n\n## Motivation\n\nAutomatic image captioning is the task of producing a natural-language\nutterance (usually a sentence) that correctly reflects the visual content of an\nimage. Up to this point, the resource most used for this task was the\n[MS-COCO dataset](http://cocodataset.org), containing around 120,000\nimages and 5-way image-caption annotations (produced by paid annotators).\n\nGoogle's Conceptual Captions dataset has more than 3 million images, paired\nwith natural-language captions. In contrast with the curated style of the\nMS-COCO images, Conceptual Captions images and their raw descriptions are\nharvested from the web, and therefore represent a wider variety of styles. The\nraw descriptions are harvested from the Alt-text HTML attribute associated with\nweb images. We developed an automatic pipeline that extracts, filters, and\ntransforms candidate image/caption pairs, with the goal of achieving a balance\nof cleanliness, informativeness, fluency, and learnability of the resulting\ncaptions.\n\nMore details are available in this paper (please cite the paper if you use or discuss this dataset in your work):\n\n\u003cdiv class=\"highlight highlight-source-shell\"\u003e\u003cpre\u003e\n@inproceedings{sharma2018conceptual,\n  title = {Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning},\n  author = {Sharma, Piyush and Ding, Nan and Goodman, Sebastian and Soricut, Radu},\n  booktitle = {Proceedings of ACL},\n  year = {2018},\n}\n\u003c/pre\u003e\u003c/div\u003e\nAdditionally, we provide machine-generated labels for a subset of 2.0M images from the Conceptual Captions training set.\nPlease cite this paper if you use the image labels in your work.\n\n\u003cdiv class=\"highlight highlight-source-shell\"\u003e\u003cpre\u003e\n@article{ng2020understanding,\n  title={Understanding Guided Image Captioning Performance across Domains},\n  author={Edwin G. Ng and Bo Pang and Piyush Sharma and Radu Soricut},\n  journal={arXiv preprint arXiv:2012.02339},\n  year={2020}\n}\n\u003c/pre\u003e\u003c/div\u003e\n\n## Dataset Description\n\nConceptual Captions dataset release contains two splits: train (~3.3M examples) and validation (~16K examples).\nSee Table 1 below for more details.\n\n\u003cp align='center'\u003eTable 1: Dataset stats.\u003c/p\u003e\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd colspan=\"3\"\u003e\u003c/td\u003e\n    \u003ctd colspan=\"3\"\u003e\u003ccenter\u003e\u003cb\u003eTokens per Caption\u003c/b\u003e\u003c/center\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003cb\u003eSplit\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003eExamples\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003eUniqe Tokens\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003eMean\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003eStdDev\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003eMedian\u003c/b\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eTrain\u003c/td\u003e\n    \u003ctd\u003e3,318,333\u003c/td\u003e\n    \u003ctd\u003e51,201\u003c/td\u003e\n    \u003ctd\u003e10.3\u003c/td\u003e\n    \u003ctd\u003e4.5\u003c/td\u003e\n    \u003ctd\u003e9.0\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eValid\u003c/td\u003e\n    \u003ctd\u003e15,840\u003c/td\u003e\n    \u003ctd\u003e10,900\u003c/td\u003e\n    \u003ctd\u003e10.4\u003c/td\u003e\n    \u003ctd\u003e4.7\u003c/td\u003e\n    \u003ctd\u003e9.0\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr bgcolor=\"#808080\"\u003e\n    \u003ctd\u003eTest (Hidden)\u003c/td\u003e\n    \u003ctd\u003e12,559\u003c/td\u003e\n    \u003ctd\u003e9,645\u003c/td\u003e\n    \u003ctd\u003e10.2\u003c/td\u003e\n    \u003ctd\u003e4.6\u003c/td\u003e\n    \u003ctd\u003e9.0\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\n## Hidden Test set\n\nWe are not releasing the official test split (~12.5K examples).\nInstead, we are hosting a competition (see \u003chttp://ai.google.com/research/ConceptualCaptions\u003e) dedicated to supporting submissions and evaluations of model outputs on this blind test set.\n\nWe strongly believe that this setup has several advantages: a) it allows the evaluation to be done using an unbiased, large number of images b) it keeps the test completely blind and eliminate suspicions of fitting to the test, cheating, etc. c) it overall provides a clean setup for advancing the SoTA on this task, including reporting reproducible results for paper publications, etc.\n\n\n## Image Labels\n\nThe image labels are obtained using the Google Cloud Vision API (\u003chttps://cloud.google.com/vision\u003e). Each image label has a machine-generated identifier (MID) corresponding to the label's Google Knowledge Graph entry and a confidence score for its presence in the image.\nThese labels have been obtained running the same model and are presented in a similar fashion with the image labels made available for the T2 Guiding dataset available at https://github.com/google-research-datasets/T2-Guiding.\n\n## Data Format for Conceptual Captions\n\nThe Conceptual Captions training and validation sets are provided as TSV (tab-separated values) text files with the following columns:\n\n\u003cp align='center'\u003eTable 2: Columns in Train/Validation TSV files.\u003c/p\u003e\n\n| Column   | Description                                                                      |\n| -------- | -------------------------------------------------------------------------------- |\n| 1        | Caption. The text has been tokenized and lowercased.                             |\n| 2        | Image URL                                                                        |\n\n## Data Format for Image Labels\n\nThe image labels for a 2.0M subset of the training set are provided as TSV (tab-separated values) text files with the following columns:\n\n\u003cp align='center'\u003eTable 3: Columns in Image Labels TSV files.\u003c/p\u003e\n\n| Column   | Description                                                                      |\n| -------- | -------------------------------------------------------------------------------- |\n| 1        | Caption. The text has been tokenized and lowercased.                             |\n| 2        | Image URL                                                                        |\n| 3        | Image labels. Comma separated list in descending order of confidence.            |\n| 4        | MIDs. Comma separated list corresponding to the image labels list.               |\n| 5        | Confidence scores. Comma separated list corresponding to the image labels list.  |\n\n\n## Contact us\n\nIf you have a technical question regarding the dataset, code or publication, please create an issue in this repository.\nThis is the fastest way to reach us.\n\nIf you would like to share feedback or report concerns, please email us at conceptual-captions@google.com\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research-datasets%2Fconceptual-captions","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-research-datasets%2Fconceptual-captions","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research-datasets%2Fconceptual-captions/lists"}