{"id":20068350,"url":"https://github.com/pliang279/lm_bias","last_synced_at":"2025-05-05T19:31:15.264Z","repository":{"id":45058104,"uuid":"375889753","full_name":"pliang279/LM_bias","owner":"pliang279","description":"[ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models","archived":false,"fork":false,"pushed_at":"2022-11-02T13:14:06.000Z","size":65793,"stargazers_count":45,"open_issues_count":0,"forks_count":8,"subscribers_count":4,"default_branch":"main","last_synced_at":"2023-03-04T14:05:30.537Z","etag":null,"topics":["fairness-ai","language-model","machine-learning","natural-language-processing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pliang279.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-06-11T03:07:48.000Z","updated_at":"2023-02-21T14:23:07.000Z","dependencies_parsed_at":"2023-01-20T20:17:50.044Z","dependency_job_id":null,"html_url":"https://github.com/pliang279/LM_bias","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pliang279%2FLM_bias","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pliang279%2FLM_bias/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pliang279%2FLM_bias/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pliang279%2FLM_bias/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pliang279","download_url":"https://codeload.github.com/pliang279/LM_bias/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224461756,"owners_count":17315116,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fairness-ai","language-model","machine-learning","natural-language-processing"],"created_at":"2024-11-13T14:06:15.613Z","updated_at":"2024-11-13T14:06:16.327Z","avatar_url":"https://github.com/pliang279.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Towards Understanding and Mitigating Social Biases in Language Models\n\nThis repo contains code and data for evaluating and mitigating bias from generation models.\n\n\n## Paper\n\n[**Towards Understanding and Mitigating Social Biases in Language Models**](https://arxiv.org/pdf/2106.13219.pdf)\u003cbr\u003e\n[Paul Pu Liang](http://www.cs.cmu.edu/~pliang/), Chiyu Wu, [Louis-Philippe Morency](https://www.cs.cmu.edu/~morency/), and [Ruslan Salakhutdinov](https://www.cs.cmu.edu/~rsalakhu/)\u003cbr\u003e\nICML 2021\n\nIf you find this repository useful, please cite our paper:\n```\n@inproceedings{liang2021towards,\n  title={Towards Understanding and Mitigating Social Biases in Language Models},\n  author={Liang, Paul Pu and Wu, Chiyu and Morency, Louis-Philippe and Salakhutdinov, Ruslan},\n  booktitle={International Conference on Machine Learning},\n  pages={6565--6576},\n  year={2021},\n  organization={PMLR}\n}\n```\n\n### 1. Identify bias-sensitive tokens, obtain bias subspace and create the dataset to train the bias classifier\n```python\npython data_preprocess.py --embed_source glove --by_pca True --num_components 5 --save_subspace False\n```\n\nGlove embedding and gpt2 embedding are large files, you can download or extract them by yourself. We also provide the [google drive link](https://drive.google.com/drive/folders/1up_8TC3_RxyDcmTrm9GKk1rU3dAt76ND?usp=sharing).\n\n### 2. Train the bias classifier and learn the projection matrix P\n```python\npython context_nullspace_projection.py\n```\nThe code of nullspace projection is from [INLP](https://github.com/shauli-ravfogel/nullspace_projection). Thanks for their great work!\n\nTo run the INLP experiments, you need to git clone https://github.com/shauli-ravfogel/nullspace_projection first, and put it under the root directory of this repo.\n\n### 3. Evaluate Bias existing in the gpt2\n#### Local Bias\n```python\ncd src/local_bias\npython measure_local_bias.py\n```\n\nIt will take long time to run the evaluation script on the full data. Here we provide the subset of our evaluation data now. Full data is available via [google drive](https://drive.google.com/drive/folders/1TNCuWDm9gY0i-_ulfqmbPIOSh1Sx2Kyb). Note that when evaluating over full data, you may encounter numerical problems on some sentences, you can simply discard these samples.\n\n#### Global Bias\n\nWe use the regard score difference as the metric for global bias. The evaluation code is from https://github.com/ewsheng/nlg-bias. Thanks for their great work!\n\n```python\ngit clone https://github.com/ewsheng/nlg-bias.git\ncd src/global_bias\npython generate_full_sentence.py --algorithm INLP\n```\n\nAfter full sentences are generated, you need to use the regard classifier to measure the global bias. \n\nTo reproduce the result in our paper, we also provide the projection matrix P for the gender bias test in `data/saved_P/P_gender_test_79.npy`\n\n## Acknowledgements\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpliang279%2Flm_bias","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpliang279%2Flm_bias","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpliang279%2Flm_bias/lists"}