{"id":31812920,"url":"https://github.com/tslmy/politeness-estimator","last_synced_at":"2025-10-11T07:45:13.387Z","repository":{"id":46196355,"uuid":"289580533","full_name":"tslmy/politeness-estimator","owner":"tslmy","description":"A set of pre-trained machine-learning models that predict (im-)politeness scores in texts.","archived":false,"fork":false,"pushed_at":"2025-01-02T07:48:50.000Z","size":391,"stargazers_count":15,"open_issues_count":10,"forks_count":7,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-01-02T08:30:57.979Z","etag":null,"topics":["computational-linguistics","machine-learning","social-psychology"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tslmy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-22T23:03:59.000Z","updated_at":"2025-01-02T07:48:53.000Z","dependencies_parsed_at":"2023-02-01T01:30:33.501Z","dependency_job_id":null,"html_url":"https://github.com/tslmy/politeness-estimator","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tslmy/politeness-estimator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tslmy%2Fpoliteness-estimator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tslmy%2Fpoliteness-estimator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tslmy%2Fpoliteness-estimator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tslmy%2Fpoliteness-estimator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tslmy","download_url":"https://codeload.github.com/tslmy/politeness-estimator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tslmy%2Fpoliteness-estimator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006630,"owners_count":26084132,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computational-linguistics","machine-learning","social-psychology"],"created_at":"2025-10-11T07:45:12.328Z","updated_at":"2025-10-11T07:45:13.379Z","avatar_url":"https://github.com/tslmy.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Politeness Estimator for Microblogs (Pem)\n\n[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tslmy/politeness-estimator/HEAD?labpath=demo.ipynb)\n\nPem is a set of pre-trained machine-learning models that predict (im-)politeness scores in texts. The models are trained on annotated microblogs and packaged into a simple Python class for easy use. Currently, two language-microblog pairs are supported:\n\n- English Twitter\n- Mandarin Chinese Weibo\n\nThe description of the data and training process is part of our paper published at CSCW 2020. \nPreprint: https://arxiv.org/pdf/2008.02449.pdf \n\n**Looking for source code of analyses, and/or PoliteLex itself? See `src/`.**\n\n**The annotated corpora is available upon request.**\n\n## Installation\n\n1. Install requirements: `pip install -r requirements.txt`\n2. If you wish to use LIWC (highly recommended for increased accuracy), \n   1. Put your English LIWC `.dic` file to the same directory as `LIWC2015_Dictionary.dic`.\n   2. Convert LIWC dictionary to long form `liwc15.csv` by running `python prepare_liwc.py`.\n   3. Rename `liwc15.csv` to `english_liwc15.csv` and repeat this for other languages, if needed.\n3. Prepare also the EmoLex lexicon by `python prepare_emolex.py`. Both `english_emolex.csv` and `chinese_emolex.csv` should be generated.\n\nNotice that, due to license concerns, we are unable to provide LIWC in this repository.\n\n## Usage\n\n1. Put microblogs in a CSV file as a column `text`. I have included two toy examples, `tweets.csv` for English Twitter and `weibo.csv` for Mandarin Weibo. **If Chinese, please pre-segment/pre-tokenize posts by whitespace.**\n2. In your code:\n\n   ```python\n    from pem import Pem # `Pem` is short for \"Politeness Estimator for Microblogs\".\n    pem = Pem(\n        liwc_path           ='english_liwc15.csv', # or '' if LIWC is unavailable\n        emolex_path         ='english_emolex.csv', \n        estimator_path      ='english_twitter_politeness_estimator.joblib', # or 'english_twitter_politeness_estimator_noLiwc.joblib' if LIWC is unavailable\n        feature_defn_path   ='english_twitter_additional_features.pickle')\n    pem.load('tweets.csv')\n    pem.tokenize()\n    pem.vectorize()\n    print(pem.predict())\n   ```\n   \n   or, for Mandarin Weibo posts:\n   \n   ```python\n    from pem import Pem # `Pem` is short for \"Politeness Estimator for Microblogs\".\n    pem = Pem(\n        liwc_path           ='chinese_liwc15.csv', # or '' if LIWC is unavailable\n        emolex_path         ='chinese_emolex.csv', \n        estimator_path      ='chinese_weibo_politeness_estimator.joblib', # or 'chinese_weibo_politeness_estimator_noLiwc.joblib' if LIWC is unavailable\n        feature_defn_path   ='chinese_weibo_additional_features.pickle')\n    pem.load('weibo.csv')\n    pem.tokenize()\n    pem.vectorize()\n    print(pem.predict())\n   ```\n\nWe are actively working on understanding sources of bias in classifiers and currently, estimates between -0.5 and 0.5 are treated as neutral. Would love your feedback on how to make this classifer better. Reach out at myli at alumni dot upenn dot edu or sharathg at cis dot upenn dot edu.\n\n## Citation\n\n```\n@article{li2020cscw,\n  title={Studying Politeness across Cultures Using English Twitter and Mandarin Weibo},\n  author={Li, Mingyang and Hickman, Louis and Tay, Louis and Ungar, Lyle and Guntuku, Sharath Chandra},\n  journal={Proceedings of the ACM on Human-Computer Interaction},\n  number={CSCW},  \n  year={2020}\n}\n```\n\n```\nAPA\nLi, M., Hickman, L., Tay, L., Ungar, L., \u0026 Guntuku, S. C. (2020). Studying Politeness across Cultures Using English Twitter and Mandarin Weibo. Proceedings of the ACM on Human-Computer Interaction (CSCW)\n```\n \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftslmy%2Fpoliteness-estimator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftslmy%2Fpoliteness-estimator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftslmy%2Fpoliteness-estimator/lists"}