https://github.com/bloomberg/fast-noise-aware-topic-clustering
Research code and scripts used in the Silburt et al. (2021) EMNLP 2021 paper 'FANATIC: FAst Noise-Aware TopIc Clustering'
https://github.com/bloomberg/fast-noise-aware-topic-clustering
clustering emnlp emnlp2021 machine-learning nlp python topic-noise
Last synced: 8 months ago
JSON representation
Research code and scripts used in the Silburt et al. (2021) EMNLP 2021 paper 'FANATIC: FAst Noise-Aware TopIc Clustering'
- Host: GitHub
- URL: https://github.com/bloomberg/fast-noise-aware-topic-clustering
- Owner: bloomberg
- License: apache-2.0
- Created: 2021-10-01T14:07:26.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2023-07-06T22:49:30.000Z (about 3 years ago)
- Last Synced: 2025-04-19T23:31:57.816Z (about 1 year ago)
- Topics: clustering, emnlp, emnlp2021, machine-learning, nlp, python, topic-noise
- Language: Python
- Homepage:
- Size: 113 KB
- Stars: 10
- Watchers: 3
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# FANATIC: FAst Noise-Aware TopIc Clustering
Authors: Ari Silburt, Anja Subasic, Evan Thompson, Carmeline Dsilva, Tarec Fares
## General
This repo contains the research code and scripts used in the Silburt et al. (2021) paper "FANATIC: FAst Noise-Aware TopIc Clustering", and provides a basic overview of the code structure and its major components. For more questions, please directly contact the authors.
In particular, this repo allows a user to:
- Download the reddit data
- Train a word2vec embedding model
- Use `FANATIC` to cluster the reddit data and dump results for downstream analysis.
Note that the original paper results used an in-house preprocessor, but a very similar open-source one has been provided (see `fanatic/preprocess/nltk_preprocessor.py`).
## License
Please read the `LICENSE`.
## How-to
### Setup
It is recommended to create a fresh python virual environment and run the following commands from the base repo:
```
pip install --upgrade pip
pip install -r requirements.txt
```
And then in a python shell do:
```python
import nltk
nltk.download('stopwords')
```
This repo has been tested against `python3.7`.
### Download the Reddit Data
Data can be downloaded from [pushshift](https://files.pushshift.io/reddit/submissions/) using `wget`, e.g. `wget https://files.pushshift.io/reddit/submissions/RS_2017-11.zst`. If data files are downloaded to the `data/` directory, subsequent scripts are already set up to look there.
### Training a Word2vec Embedding
A new word2vec model can be trained using `embedding_driver.py`, and it is recommended to carefully inspect the arguments before running. In particular, a Reddit data file(s) must first be downloaded and specified in the `--data-files` argument.
### Cluster via FANATIC
Once the data has been downloaded and word2vec model trained, a clustering run can be performed using the `clustering_driver.py` script. All input arguments are specified in the `parse_args` function of `fanatic/arguments.py` including data, label, preprocessing, clustering algorithm and output arguments.
See the Silburt et al. (2021) paper for a detailed explanation of FANATIC's hyperparameters.
#### Clustering labels
By default, the data is clustered against the `data/subreddit_labels.json` labels file, which indicates:
- what subreddits are considered for clustering (all other subreddits are discarded).
- whether the subreddit is a "coherent" or "noise" topic, where all noise topics are assigned the same `NOISE` label.
The `--subreddit-noise-percentage` argument sets the fraction of documents that come from noise subreddits. In the case where `--num-docs-read` and `--subreddit-noise-percentage` are incompatible with each other, honouring `--subreddit-noise-percentage` is prioritized. If `--subreddit-noise-percentage` is set to `None`, the noise percentage is set by the natural data distribution.
The `data/subreddit_labels.json` labels file can be substituted for a different one, or ignored entirely by setting `--subreddit-labels-file None`. When `--subreddit-labels-file` is set to `None` all encountered subreddits are used, each subreddit becomes its own coherent topic, and the concept of "topic noise" disappears. Thus, the `--subreddit-noise-percentage` argument becomes irrelevant.
#### Clustering Outputs
After a successful clustering run, the files that are output are:
- `fanatic___labels_and_assignments.json`: this file is generated for each seed run and contains, for each document-id, the *assignment* (what cluster the document ended up in) and *label* (the label associated with the document). Therefore, the full clustering result is contained within this file for downstream analysis. Document-ids can be mapped back to the original datafile should additional metadata be desired.
- `fanatic___sample_clusters.txt` - this file is generated for each seed run and contains the first 10 documents from each cluster and associated label. Format is ` -> `. This gives the user a qualitative sense of what each cluster contains.
- `fanatic___summary.txt` - this file is generated for each seed run and contains all input parameters and clustering stats/metrics. It is effectively a summary of the entire clustering run, allowing you to quickly parse results and/or recreate the job if needed. It can be consumed by [configparser](https://docs.python.org/3.7/library/configparser.html).
- `fanatic__summary_averaged.txt` - this file is generated once for a dataset-id and contains the input arguments and *averaged* clustering stats/metrics across the seed runs.
In addition, the full clustering model can be dumped via pickle for deeper investigation by adding `--flag-save-clusteringmodel`. *Warning*: this file can become large, especially for big datasets.
### Custom Preprocessor / Featurizer
Results from the paper were generated using an in-house preprocessor that is not available to the public. Using nltk we created a very similar preprocessor, located at `fanatic/preprocess/nltk_preprocessor.py`, and inherits from `fanatic/preprocess/generic_preprocessor.py`. Users are free to create their own custom preprocessors that also inherit from `generic_preprocessor.py` and experiment with more sophisticated features (e.g. BERT embeddings). See `generic_preprocessor.py` for additional documentation and requirements.
### Non-Reddit Datasets
Users are encouraged to substitute or modify `fanatic/preprocess/read_data.py` to read in different kinds of data. In particular, `DATASET_INPUT_FIELD`, `DATASET_LABEL_FIELD` and `DATASET_ID_FIELD` must be changed to extract the relevant content from the new dataset for downstream preprocessing and clustering.
## Tests
Some basic unit tests can be run from the home directory with `python3.7 -m pytest tests/unit/`