https://github.com/ecrows/l2-reddit-experiment
Code for the IEEE MLSP paper "Towards Ethical Content-Based Detection of Online Influence Campaigns".
https://github.com/ecrows/l2-reddit-experiment
Last synced: 8 months ago
JSON representation
Code for the IEEE MLSP paper "Towards Ethical Content-Based Detection of Online Influence Campaigns".
- Host: GitHub
- URL: https://github.com/ecrows/l2-reddit-experiment
- Owner: ecrows
- Created: 2019-06-10T19:36:10.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2019-08-16T03:45:09.000Z (almost 7 years ago)
- Last Synced: 2025-09-12T01:15:55.629Z (10 months ago)
- Language: Python
- Homepage:
- Size: 62.5 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# l2-reddit-experiment
Code for the submitted paper *"Towards Ethical Content-Based Detection of Online Influence Campaigns"* by Crothers et al. (2019).
Based off of the [official BERT experiment code](https://github.com/google-research/bert).
**Abstract**
The detection of clandestine efforts to influence users in online communities is a challenging problem with significant active development. We demonstrate that features derived from the text of user comments are useful for identifying suspect activity, but lead to increased erroneous identifications (false positive classifications) when keywords over-represented in past influence campaigns are present. Drawing on research in native language identification (NLI), we use "named entity masking" (NEM) to create sentence features robust to this shortcoming, while maintaining comparable classification accuracy. We demonstrate that while NEM consistently reduces false positives when key named entities are mentioned, both masked and unmasked models exhibit increased false positive rates on English sentences by Russian native speakers, raising ethical considerations that should be addressed in future research.
**Instructions**
Install dependencies using:
`pip3 install -r requirements.txt`
Code can then be executed using the provided helper scripts, or the run_experiment.py file can be called directly.
**Requirements**
This code is written to be run on a TPU in Google Cloud, but can also be run by CPU/GPU machines (however, this will likely require minor modifications to use the BERT-Base model to fit memory limits).