{"id":51043723,"url":"https://github.com/leomsfreitas/hatespeech","last_synced_at":"2026-06-22T12:02:12.646Z","repository":{"id":366419465,"uuid":"1249735724","full_name":"leomsfreitas/HateSpeech","owner":"leomsfreitas","description":"Hate speech detection research repository.","archived":false,"fork":false,"pushed_at":"2026-06-21T18:27:09.000Z","size":71142,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-21T19:20:09.417Z","etag":null,"topics":["hate-speech","machine-learning","nlp","text-classification"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leomsfreitas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-26T01:59:22.000Z","updated_at":"2026-06-21T18:27:12.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/leomsfreitas/HateSpeech","commit_stats":null,"previous_names":["leomsfreitas/hatespeech"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/leomsfreitas/HateSpeech","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leomsfreitas%2FHateSpeech","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leomsfreitas%2FHateSpeech/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leomsfreitas%2FHateSpeech/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leomsfreitas%2FHateSpeech/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leomsfreitas","download_url":"https://codeload.github.com/leomsfreitas/HateSpeech/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leomsfreitas%2FHateSpeech/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34647750,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-22T02:00:06.391Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hate-speech","machine-learning","nlp","text-classification"],"created_at":"2026-06-22T12:02:11.730Z","updated_at":"2026-06-22T12:02:12.641Z","avatar_url":"https://github.com/leomsfreitas.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Exploring Sentiment Analysis Techniques for Offensive Language Classification in Brazilian Politics\n\n\u003e Comparison of AutoML, BERT and LLM approaches for detecting offensive language and hate speech in tweets from the 2022 Brazilian presidential elections.\n\n## Background\n\nSocial media has become a central space for contemporary political debate, enabling the\nrapid circulation of opinions, information and electoral strategies. This expansion came\nhand in hand with a sharp rise in offensive comments and hate speech — insults, racism,\nxenophobia and other forms of verbal violence that degrade public discourse and deepen\npolitical polarization.\n\nAlthough incitement to hatred is a crime in Brazil, its enforcement on social platforms\nremains limited. Investigating the spread of online offensive language is therefore\nessential to understanding how interaction, power and conflict dynamics manifest in\ndigital environments.\n\nThis project investigates the use of **sentiment analysis (SA)** techniques for the\nautomatic classification of offensive language and hate speech in tweets about Brazilian\npolitics. The guiding hypothesis is that offensive comments tend to express negative\nsentiments and emotions, and that the presence of negative terms and emotions can\ntherefore support the classification of offensive language. The work was developed within\nthe **Interfaces** research group (UFSCar), as part of a regular FAPESP project\n(2022/03090-0) aimed at the large-scale analysis of Brazilian political data.\n\nTwo classification tasks are addressed:\n\n1. **Binary classification** — whether a comment is offensive or not.\n2. **Multi-label classification** — which type(s) of offense a comment carries, including\n   hate-speech categories.\n\n## Corpus\n\nThe target corpus, **ITED-BR**, is a reference set of **3,400 tweets** manually annotated\nfor this research, drawn from the [Interfaces Twitter Elections Dataset](https://doi.org/10.1371/journal.pone.0317289)\n(Iasulaitis et al., 2025) — a collection of ~228 million tweets from the 2022 Brazilian\npresidential elections. The 3,400 tweets relate to the **second round** of the election\n(1,700 mentioning Bolsonaro and 1,700 mentioning Lula), published between October 2 and 30,\n2022.\n\nAnnotation was carried out by nine researchers in two stages: first labeling each comment\nas offensive/non-offensive, then assigning one or more offense types to offensive comments.\nInter-annotator agreement (Krippendorff's Alpha) reached **0.68** (Lula set) and **0.72**\n(Bolsonaro set).\n\n**Class distribution (ITED-BR).** Offense-type percentages are computed over the offensive\ncomments of each set; a comment may carry more than one type.\n\n| Offense | Bolsonaro set | Lula set | Total |\n|---|---|---|---|\n| Insult | 1,188 (95.4%) | 1,036 (96.2%) | 2,224 (95.8%) |\n| Ideology | 244 (20.5%) | 193 (18.0%) | 437 (18.8%) |\n| Obscene | 106 (8.9%) | 70 (6.5%) | 176 (7.6%) |\n| Others | 24 (1.4%) | 14 (0.8%) | 38 (1.1%) |\n| Xenophobia | 12 (1.1%) | 5 (0.4%) | 17 (0.7%) |\n| Sexism | 11 (0.9%) | 5 (0.4%) | 16 (0.7%) |\n| LGBTQphobia | 8 (0.7%) | 0 (0.0%) | 8 (0.3%) |\n| Racism | 3 (0.2%) | 1 (0.09%) | 4 (0.2%) |\n| **Offensive** | **1,245 (73.2%)** | **1,076 (63.3%)** | **2,321 (68.3%)** |\n| **Non-offensive** | **455 (26.8%)** | **624 (36.7%)** | **1,079 (31.7%)** |\n\n**Training corpora.** Models are trained on public Portuguese corpora before being applied\nto ITED-BR. The binary models use **Hate-BR** (Vargas et al., 2022; 7k Instagram comments).\nThe multi-label models use **Unified-Hate**, built by merging **TOLD-BR**, **OLID-BR** and\n**Hate-BR** under a common label scheme — **15,555 comments** after de-duplication.\n\n| Class | TOLD-BR | OLID-BR | Hate-BR | Total | Original classes |\n|---|---|---|---|---|---|\n| Insult | 4,371 | 5,589 | — | 9,864 (63.4%) | Insult |\n| Obscene | 6,639 | 2,311 | — | 8,819 (56.7%) | Obscene, Profanity |\n| Ideology | — | 1,221 | 496 | 1,704 (10.9%) | Ideology, Partyism |\n| LGBTQphobia | 344 | 375 | 17 | 731 (4.7%) | LGBTQ+phobia, Homophobia |\n| Racism | 138 | 92 | 8 | 236 (1.5%) | Racism |\n| Sexism | 463 | 461 | 97 | 1,011 (6.5%) | Misogyny, Sexism |\n| Xenophobia | 151 | 128 | 1 | 279 (1.8%) | Xenophobia |\n\n## Approaches Evaluated\n\n### AutoML (AutoGluon)\n\nTraditional machine-learning models trained with [AutoGluon](https://github.com/autogluon/autogluon),\ncombined with sentiment-analysis features. To measure each feature's contribution, an\n**incremental evaluation** is performed — starting from a feature-less *baseline* and adding\none attribute at a time, yielding six configurations:\n`Baseline → MOL → MOL+GE → MOL+GE+GEP → MOL+GE+GEP+Lex → MOL+GE+GEP+Lex+IP`.\n\nThe features are:\n\n- **MOL** — *Multilingual Offensive Lexicon* (Vargas et al., 2021): weighted score of\n  offensive terms found in the text.\n- **GE** — *GoEmotions* probability vector (28 emotions) from a BERTimbau model fine-tuned\n  for emotion analysis (Hammes \u0026 Freitas, 2021).\n- **GEP** — *GoEmotions polarity*: emotions above a 0.3 threshold mapped to +1/0/-1\n  (Assi et al., 2022) and summed.\n- **Lex** — proportions of positive/negative/neutral terms using three sentiment lexicons\n  (LIWC-Br, WordNetAffectBR, SentiLex-PT).\n- **IP** — a single *polarity index* combining the three Lex proportions via logistic\n  regression.\n\n### BERTimbau\n\nThe [BERTimbau Large](https://huggingface.co/neuralmind/bert-large-portuguese-cased)\nTransformer (Souza et al., 2020), fine-tuned separately for the binary and multi-label\ntasks. Hyperparameters: batch size 32, 3 epochs, learning rate 2e-5, weight decay 0.01,\nwarmup ratio 0.1, max sequence length 128, label smoothing 0.1, early-stopping patience 2.\n\n### Sabiá-3 (LLM)\n\nA generative large language model trained specifically for Brazilian Portuguese\n([Maritaca AI](https://www.maritaca.ai/)). It is prompted with task descriptions; a\n**few-shot** strategy outperformed zero-shot on both tasks and was the one adopted.\n\n### Perspective API\n\nThe [Perspective API](https://perspectiveapi.com/) (Jigsaw/Google), used **only for the\nbinary task** via its `TOXICITY` attribute. Comments scoring ≥ **0.4** (threshold defined\nempirically) are classified as offensive.\n\n## Results\n\nSummary on the **ITED-BR** corpus (Macro-F). Full per-class metrics and the training-stage\nresults are in **[results/README.md](results/README.md)**.\n\n| Approach | Binary Macro-F | Multi-label Macro-F | Compute cost |\n|---|---|---|---|\n| **BERTimbau** | **0.75** | 0.28 | High (GPU fine-tuning) |\n| **Sabiá-3 (LLM)** | 0.73 | **0.58** | API (paid inference) |\n| AutoML — best config | 0.69 | 0.55 | Low (CPU) |\n| AutoML — Baseline | 0.63 | 0.55 | Low (CPU) |\n| Perspective API | 0.56 | — | API |\n\n**Key findings.** In the **binary** task, the fine-tuned BERTimbau leads (0.75), followed\nclosely by the Sabiá-3 LLM (0.73); the much simpler and cheaper AutoML models reach 0.69,\ntrailing by only 4–6 points. In the **multi-label** task the LLM leads (0.58), with the\nAutoML Baseline right behind (0.55), while BERTimbau collapses to 0.28 — generalizing only\non the majority *insult* class. Sentiment-analysis features helped most in the binary task\n(+6 points over baseline), supporting the project's initial hypothesis. Rare classes\n(racism, xenophobia) remained the hardest to detect across all approaches.\n\n## Repository Structure\n\n```\n.\n├── data/\n│   ├── ited-br/              # target corpus: 3,400 annotated election tweets\n│   │   ├── processed/        # cleaned ited-br.csv\n│   │   └── features/         # mol, ge, gep, lex, ip feature tables\n│   ├── hate-br/              # training corpus — binary task\n│   ├── unified-hate/         # training corpus — multi-label task (TOLD+OLID+Hate)\n│   ├── told-br/, olid-br/    # source corpora for Unified-Hate\n│   └── lexicons/             # LIWC-Br, SentiLex-PT, WordNetAffectBR, MOL\n├── notebooks/\n│   ├── 1_analysis.ipynb              # exploratory data analysis\n│   ├── 2a_filtered_corpus.ipynb      # filter \u0026 sample raw election tweets\n│   ├── 2b_pre_processing.ipynb       # clean/normalize all corpora\n│   ├── 2c_unified_corpus.ipynb       # merge into Unified-Hate\n│   ├── 3a_feat_lex.ipynb             # Lex feature\n│   ├── 3b_feat_ge.ipynb              # GoEmotions (GE) feature\n│   ├── 3c_feat_gep.ipynb             # GoEmotions polarity (GEP) feature\n│   ├── 3d_feat_ip.ipynb              # polarity index (IP) feature\n│   ├── 3e_feat_mol.ipynb             # MOL feature\n│   ├── 4a_automl_binary.ipynb        # train AutoML — binary (Hate-BR)\n│   ├── 4a_automl_multilabel.ipynb    # train AutoML — multi-label (Unified-Hate)\n│   ├── 4b_bert.ipynb                 # fine-tune BERTimbau — both tasks\n│   ├── 5a_automl_inference.ipynb     # AutoML inference on ITED-BR\n│   ├── 5b_bert_inference.ipynb       # BERTimbau inference on ITED-BR\n│   ├── 5c_sabia.ipynb                # Sabiá-3 inference on ITED-BR\n│   ├── 5d_perspective.ipynb          # Perspective API inference on ITED-BR\n│   └── others/                       # auxiliary lexicon/feature ETL notebooks\n├── src/\n│   ├── pipeline/   # automl.py, bert.py, sabia.py, perspective.py\n│   ├── models/     # automl.py (MultilabelPredictor), bert.py\n│   └── utils/      # features, report, split, preprocessing, prompt, map, path\n├── models/         # trained AutoGluon and BERTimbau models\n├── results/        # detailed metrics (see results/README.md)\n├── requirements.txt\n└── README.md\n```\n\n## Installation\n\n```bash\ngit clone https://github.com/leomsfreitas/HateSpeech.git\ncd HateSpeech\npython -m venv .venv\nsource .venv/bin/activate\npip install -r requirements.txt\n```\n\n## How to Run\n\nThe notebooks are numbered in execution order. A typical end-to-end run is:\n\n1. **Pre-processing** — `2b_pre_processing.ipynb`, then `2c_unified_corpus.ipynb`.\n2. **Feature extraction** — `3a_feat_lex` → `3b_feat_ge` → `3c_feat_gep` → `3d_feat_ip` →\n   `3e_feat_mol`.\n3. **Training** — `4a_automl_binary`, `4a_automl_multilabel`, `4b_bert`.\n4. **Inference on ITED-BR** — `5a_automl_inference`, `5b_bert_inference`, `5c_sabia`,\n   `5d_perspective`.\n\n\u003e The Sabiá-3 and Perspective API notebooks require API keys (Maritaca AI and Jigsaw/Google,\n\u003e respectively).\n\n## Available Models\n\nThe trained models are too large to host on GitHub and are distributed separately. Only the\nfine-tuned BERTimbau models and the best-performing AutoML configuration per task are\nprovided.\n\n| Model | Task | Best config | ITED-BR Macro-F | Download |\n|---|---|---|---|---|\n| BERTimbau | Binary | — | 0.75 | [Google Drive](https://drive.google.com/file/d/1KI2kykgLwOw7YoiIKZfBjQVrFb8nUPQ0/view?usp=sharing) |\n| BERTimbau | Multi-label | — | 0.28 | [Google Drive](https://drive.google.com/file/d/1sHDnY9TAgEj8psYtASJr_B1mnVg4yZWT/view?usp=sharing) |\n| AutoML | Binary | MOL+GE | 0.69 | [Google Drive](https://drive.google.com/file/d/1EQYuoCvvHrF1YOOjTIWnEAambO51imna/view?usp=sharing) |\n| AutoML | Multi-label | Baseline | 0.55 | [Google Drive](https://drive.google.com/file/d/1UY6hwz3q5jR6CrOqqAo7tA7hEfB3VTlv/view?usp=sharing) |\n\n## Citation\n\n```bibtex\n@misc{freitas2026ofensivo,\n  title  = {Explorando Técnicas de Análise de Sentimento na Classificação de\n            Linguagem Ofensiva no Contexto da Política Brasileira},\n  author = {Freitas, Leo Marques Sabino de},\n  year   = {2026},\n  url    = {https://github.com/leomsfreitas/HateSpeech}\n}\n```\n\n## Authors\n\nLeo Marques Sabino de Freitas — [@leomsfreitas](https://github.com/leomsfreitas)\n\n## Acknowledgments\n\nThis research was developed within the **Interfaces** research group (UFSCar) under the\nsupervision of **Eloize Rossi Marques Seno**, as part of a regular **FAPESP** project\n(2022/03090-0). The author thanks **PACTEC/IFSP** for the research scholarship that made\nthis work possible, and **Maritaca AI** for the credits granted for the use of the Sabiá-3\nLLM.\n\n---\n\n\u003e The author declares the use of the **Opus 4.8** model to generate the README files of this\n\u003e project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleomsfreitas%2Fhatespeech","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleomsfreitas%2Fhatespeech","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleomsfreitas%2Fhatespeech/lists"}