{"id":51009965,"url":"https://github.com/aydakikio/goemotion_persian","last_synced_at":"2026-06-21T01:09:05.707Z","repository":{"id":362750575,"uuid":"1249967475","full_name":"aydakikio/goemotion_persian","owner":"aydakikio","description":"Persian translation of Google's GoEmotions dataset with 27 fine grained emotion","archived":false,"fork":false,"pushed_at":"2026-06-05T17:55:41.000Z","size":5201,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-05T19:23:45.685Z","etag":null,"topics":["bert","bert-fine-tuning","emotion-detection","emotion-recognition","fa","farsi","fine-tuning","goemotion-persian","multi-label-classification","nlp","persian","sentiment-analysis","text-classification","translated-dataset"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aydakikio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-26T07:26:27.000Z","updated_at":"2026-06-05T17:55:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/aydakikio/goemotion_persian","commit_stats":null,"previous_names":["aydakikio/goemotion_persian"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/aydakikio/goemotion_persian","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aydakikio%2Fgoemotion_persian","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aydakikio%2Fgoemotion_persian/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aydakikio%2Fgoemotion_persian/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aydakikio%2Fgoemotion_persian/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aydakikio","download_url":"https://codeload.github.com/aydakikio/goemotion_persian/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aydakikio%2Fgoemotion_persian/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34590415,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-20T02:00:06.407Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","bert-fine-tuning","emotion-detection","emotion-recognition","fa","farsi","fine-tuning","goemotion-persian","multi-label-classification","nlp","persian","sentiment-analysis","text-classification","translated-dataset"],"created_at":"2026-06-21T01:09:05.095Z","updated_at":"2026-06-21T01:09:05.702Z","avatar_url":"https://github.com/aydakikio.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GoEmotion Persian \n![License](https://img.shields.io/badge/license-CC_BY_4.0-lightgrey.svg?style=for-the-badge)\n![Language](https://img.shields.io/badge/language-Persian_(Farsi)-blue.svg?style=for-the-badge)\n![Dataset Size](https://img.shields.io/badge/rows-~54K-green.svg?style=for-the-badge)\n![Kaggle](https://img.shields.io/badge/kaggle-dataset-orange.svg?style=for-the-badge\u0026logo=kaggle)\n\n## 📖 Overview\n\nThis dataset is a Persian (Farsi) translation of the **GoEmotions** benchmark,\noriginally created by Google Research. GoEmotions is one of the largest\nfine-grained emotion datasets available, containing Reddit comments labeled\nacross 27 emotion categories plus Neutral.\n\nThis Persian version makes the GoEmotions benchmark accessible for **Persian\nNLP research**, enabling tasks such as emotion detection, sentiment analysis,\nand multi-label text classification in Farsi.\n\n---\n\n## 📥 Dataset Access\n\nThe GoEmotion Persian dataset can be accessed in multiple ways:\n\n### 1️⃣ Kaggle Dataset\n\nThe dataset is also available on Kaggle for direct use in notebooks:\n\n👉 [https://www.kaggle.com/datasets/aydakikio/goemotion-persian](https://www.kaggle.com/datasets/aydakikio/goemotion-persian)\n\nYou can use it directly in Kaggle:\n\n```\nimport pandas as pd\n\npath = \"/kaggle/input/goemotion-persian/translated_train.tsv\"\ndf = pd.read_csv(path, sep=\"\\t\", header=None)\n```\n\n---\n\n### 2️⃣ Download from GitHub Releases\n\nPre-packaged dataset versions are available in the **Releases** section:\n\n👉 [https://github.com/aydakikio/goemotion_persian/releases](https://github.com/aydakikio/goemotion_persian/releases)\n\nYou can download:\n\n* Raw TSV files\n* Translated dataset\n\n\n---\n### 3️⃣ Clone the Repository (Recommended for development)\n\n```bash\ngit clone https://github.com/aydakikio/goemotion_persian.git\ncd goemotion_persian\n```\n\n---\n\n## 🧭 Provenance\n\nThis dataset is a Persian translation of the **GoEmotions benchmark** created by Google Research.\n\nThe dataset was constructed using a hybrid pipeline:\n\n### 🤖 Automated Translation\n\n* The majority of translations were generated using:\n  * **Gemini 2.5**\n  * **Gemini 2.5 Lite**\n* These models were used to translate English Reddit comments into Persian while preserving emotional context and label alignment.\n\n### ✋ Manual Correction\n\n* A subset of samples was manually translated and corrected.\n* Manual review focused on:\n\n  * untranslated or partially translated rows\n  * noisy inputs (URLs, special tokens, malformed text)\n  * improving fluency and cultural adaptation in Persian\n\n### 🧪 Quality Control\n\n* Automated validation tests were applied to ensure:\n\n  * Persian script consistency\n  * missing or empty translations detection\n  * duplication filtering\n  * length ratio sanity checks between source and target text\n\n### 📊 Human Review\n\n* A random sample of ~500 rows from each split was manually inspected to assess translation quality and consistency.\n\n### 🔗 Reproducibility\nThis repository includes:\n\n* translation pipeline implementation\n* dataset preprocessing scripts\n* QA / validation tools\n* experiment and processing logs\n* skipped / error batch tracking\n\n---\n\n## 🗂️ Repository Structure\n\n```\ngoemotion_persian/\n├── data\n│   ├── raw\n│   │   ├── raw_dev.tsv # 5,426 rows\n│   │   ├── raw_test.tsv # 5,427 rows\n│   │   └── raw_train.tsv # 43,410 rows\n│   ├── test_sample\n│   │   ├── dev_dataset_manual_review_200.csv #200 rows\n│   │   ├── test_dataset_manual_review_200.csv #200 rows\n│   │   └── train_dataset_manual_review_500.csv #500 rows\n│   └── translated\n│       ├── translated_dev.tsv # 5,426 rows\n│       ├── translated_test.tsv # 5,427 rows\n│       └── translated_train.tsv # 43,410 rows\n├── source_code \n│   ├── audiot_test.py #Quality tester\n|   └── translator.py #translation pipeline\n├── log_files\n│   ├── test\n│   │   ├── dev_translate_test_1.log\n│   │   ├── dev_translate_test_2.log\n│   │   ├── dev_translate_test_3.log\n│   │   ├── test_dataset_test_1.log\n│   │   ├── test_dataset_test_2.log\n│   │   ├── train_translate_test_1.log\n│   │   ├── train_translate_test_2.log\n│   │   └── train_translate_test_3.log\n│   └── translation\n│       ├── dev_translation_logs.log\n│       ├── skipped_batches.log\n│       ├── test_translation_logs.log\n│       ├── train_translation_logs_day_1.log\n│       └── train_translation_logs_day_2.log\n├── README.md\n└── CITATION.cff\n```\n\n---\n\n## 🌞 Dataset Details\n\n| Property | Value |\n|---|---|\n| Language | Persian (Farsi) |\n| Original Language | English |\n| Total Records | ~54,000 |\n| Training Set | 43,410 |\n| Validation Set | 5,426 |\n| Test Set | 5,427 |\n| Number of Labels | 27 emotions + Neutral |\n| Format | TSV (tab-separated) |\n| Translation Model | Gap GPT API |\n\n---\n\n## 🧭 Dataset Structure\n\nEach file contains three columns with **no header row**:\n\n| Column | Content |\n|---|---|\n| Column 0 | Persian translated text |\n| Column 1 | Emotion label(s) |\n| Column 2 | UUID |\n\n---\n\n## 💻 How to Use\n\n```\nimport pandas as pd\n\n# Load splits\ntrain = pd.read_csv(\"data/train.tsv\", sep=\"\\t\", header=None,\n                    names=[\"text\", \"labels\", \"id\"])\ndev   = pd.read_csv(\"data/dev.tsv\",   sep=\"\\t\", header=None,\n                    names=[\"text\", \"labels\", \"id\"])\ntest  = pd.read_csv(\"data/test.tsv\",  sep=\"\\t\", header=None,\n                    names=[\"text\", \"labels\", \"id\"])\n\nprint(train.head())\n```\n\n\u003e [!NOTE]\n\u003e Labels are stored as comma-separated emotion names (e.g. `joy,admiration`).\n\u003e A single text can have multiple labels.\n\n---\n\n## 🌿 Emotion Categories\n\n`admiration` `amusement` `anger` `annoyance` `approval` `caring` `confusion`\n`curiosity` `desire` `disappointment` `disapproval` `disgust` `embarrassment`\n`excitement` `fear` `gratitude` `grief` `joy` `love` `nervousness` `optimism`\n`pride` `realization` `relief` `remorse` `sadness` `surprise` `neutral`\n\n---\n\n## ⚓ GoEmotions Label Codes (Official)\n\n| ID | Emotion | ID | Emotion |\n|---|---|---|---|\n| 0 | admiration | 14 | fear |\n| 1 | amusement | 15 | gratitude |\n| 2 | anger | 16 | grief |\n| 3 | annoyance | 17 | joy |\n| 4 | approval | 18 | love |\n| 5 | caring | 19 | nervousness |\n| 6 | confusion | 20 | optimism |\n| 7 | curiosity | 21 | pride |\n| 8 | desire | 22 | realization |\n| 9 | disappointment | 23 | relief |\n| 10 | disapproval | 24 | remorse |\n| 11 | disgust | 25 | sadness |\n| 12 | embarrassment | 26 | surprise |\n| 13 | excitement | 27 | neutral |\n\n\u003e [!NOTE]\n\u003e These are multi label indices, a single text can have multiple active labels.\n\n---\n\n## 🔨 Quality Assurance\n\nThe translation pipeline included:\n\n- Automated audit tests (Persian script validation, missing translation detection, length ratio checks, duplicate detection)\n- Manual review of a 500-row random sample\n- Exception handling for untranslatable rows (URLs, special tokens)\n\n---\n\n## ⚠️ Limitations\n\n- Translations were generated via the **GAP GPT API** and are **not human-verified** beyond the 500-row sample review.\n- Some idiomatic Reddit expressions may not translate accurately into Persian.\n- Inherited label noise from the original GoEmotions dataset applies.\n\n---\n\n## 🔔 Original Dataset\n\nThis dataset is based on:\n\n\u003e Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., \u0026 Ravi, S. (2020).\n\u003e **GoEmotions: A Dataset of Fine-Grained Emotions.**\n\u003e *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020).*\n\u003e [https://arxiv.org/abs/2005.00547](https://arxiv.org/abs/2005.00547)\n\nOriginal dataset links:\n- GitHub: [google-research/goemotions](https://github.com/google-research/google-research/tree/master/goemotions)\n- Kaggle: [debarshichanda/goemotions](https://www.kaggle.com/datasets/debarshichanda/goemotions)\n\n---\n\n## 🫖 Citation\n\nIf you use this dataset in your research, please cite both the original GoEmotions paper and this dataset.\n\n**This dataset:**\n```\n@dataset{ayda_khoshkan_2026,\n\ttitle={GoEmotion Persian},\n\turl={https://www.kaggle.com/dsv/16646751},\n\tDOI={10.34740/KAGGLE/DSV/16646751},\n\tpublisher={Kaggle},\n\tauthor={Ayda Khoshkan},\n\tyear={2026}\n}\n```\n\n**Original GoEmotions paper:**\n```\n@inproceedings{demszky-etal-2020-goemotions,\n  title     = {{G}o{E}motions: A Dataset of Fine-Grained Emotions},\n  author    = {Demszky, Dorottya and Movshovitz-Attias, Dana and Ko, Jeongwook\n               and Cowen, Alan and Nemade, Gaurav and Ravi, Sujith},\n  booktitle = {Proceedings of the 58th Annual Meeting of the Association for\n               Computational Linguistics},\n  year      = {2020},\n  url       = {https://arxiv.org/abs/2005.00547}\n}\n```\n\n---\n\n## 🪶 License\nThis dataset is released under **[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)** and **Apache 2.0**.\nThe original GoEmotions dataset is licensed under **[CC0](https://creativecommons.org/publicdomain/zero/1.0/)** and **Apache 2.0** by Google Research.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faydakikio%2Fgoemotion_persian","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faydakikio%2Fgoemotion_persian","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faydakikio%2Fgoemotion_persian/lists"}