https://github.com/mit-ccc/message-translation
An assistive writing tool to construct community-centric messages
https://github.com/mit-ccc/message-translation
Last synced: 11 months ago
JSON representation
An assistive writing tool to construct community-centric messages
- Host: GitHub
- URL: https://github.com/mit-ccc/message-translation
- Owner: mit-ccc
- License: cc0-1.0
- Created: 2021-12-11T01:33:33.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2021-12-22T18:26:40.000Z (over 4 years ago)
- Last Synced: 2025-05-11T07:35:37.714Z (about 1 year ago)
- Language: Jupyter Notebook
- Size: 18 MB
- Stars: 1
- Watchers: 8
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# message-translation
[](https://creativecommons.org/licenses/by-nc/4.0/)
An assistive writing tool to analyze linguistic and cultural variation across communities
## Environment Setup
Please run the following:
```
conda create -n message python=3.8
pip install -r requirements.txt
```
## Dataset
You can follow the instructions from the public [BLM Twitter dataset](https://github.com/sjgiorgi/blm_twitter_corpus) to download tweets using our filtered tweetid to generate a smaller dataset which contains ~200K pro-BLM tweets and ~100K anti-BLM tweets. The preprocessing code and data are [here](https://github.com/social-machines/message-translation/tree/main/preprocessing). After that, move the dataset to `./data/blm_alm/raw/` such that you have the following two files: `pro_blm_200k.txt` and `anti_blm_100k.txt`.
## Semantic Shift Analysis
```
cd semantic_shift
# download BERTweet to your local machine
python download_bertweet.py
sh ./bash_scripts/compute_semantic_shifts.sh
```
Check the [notebook](https://github.com/social-machines/message-translation/blob/main/semantic_shift/analysis/blm-semantic-change-analysis.ipynb) to see the analysis.
## Cultural and Ideological Analysis
```
cd ideology-alignment
sh train_script.sh
```
Check the [notebook](https://github.com/social-machines/message-translation/blob/main/ideology-alignment/src/analysis/Misalignments.ipynb) to see the analysis.
## Acknowledgement
This github is developed on the basis of [UiO-UvA at SemEval-2020 Task 1](https://github.com/akutuzov/semeval2020) and [Aligning Multidimensional Worldviews and Discovering Ideological Differences](https://github.com/jmilbauer/worldview-ideology).