https://github.com/dimits-ts/synthetic_moderation_experiments
Experiments relating to synthetic LLM user-agents and LLM facilitators in online discussions
https://github.com/dimits-ts/synthetic_moderation_experiments
data-analysis dataset-generation llms llms-reasoning nlp
Last synced: 2 months ago
JSON representation
Experiments relating to synthetic LLM user-agents and LLM facilitators in online discussions
- Host: GitHub
- URL: https://github.com/dimits-ts/synthetic_moderation_experiments
- Owner: dimits-ts
- Created: 2024-11-01T11:25:40.000Z (9 months ago)
- Default Branch: master
- Last Pushed: 2025-05-12T11:57:51.000Z (2 months ago)
- Last Synced: 2025-05-12T19:25:39.574Z (2 months ago)
- Topics: data-analysis, dataset-generation, llms, llms-reasoning, nlp
- Language: Jupyter Notebook
- Homepage:
- Size: 169 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Evaluating Online Moderation Strategies Through Synthetic Discussion Generation
Synthetic dataset generation using the [SynDisco](https://github.com/dimits-ts/synthetic_discussion_framework) library. Experiments exploring the effect of various LLM moderation strategies in online conversations.
## Project Structure
* [`run.py`](run.py) Execution script
* [`data/`](data/) Input and output data of the experiments
* [`data/annotation_input/`](data/annotation_output/) Configuration files for the synthetic annotation jobs
* [`data/annotation_output/`](data/annotation_output/) Annotation output (JSON format)
* [`data/discussions_input/`](data/discussions_input/) Configuration files for the synthetic discussion jobs
* [`data/discussions_output/`](`data/discussions_output/`) Logs of the executed synthetic discussions (JSON format)
* [`data/datasets/`](data/datasets) Exported CSV datasets (original + ablation)
* [`data/run_configs/`](data/run_configs/) YAML configuration files for running the experiments
* [`notebooks/`](`notebooks/`) Analyzing the experiments
* [`notebooks/tasks/`](notebooks/tasks/) Shared notebook modules
* [`notebooks/ablation.ipynb`](notebooks/ablation.ipynb) Ablation study
* [`notebooks/moderation.ipynb`](notebooks/moderation.ipynb) Moderator intervention analysis
* [`notebooks/timeseries.ipynb`](notebooks/timeseries.ipynb) Timeseries analysis of synthetic discussions
* [`notebooks/toxicity_aq.ipynb`](notebooks/toxicity_aq.ipynb) General discussion quality analysis
* [`graphs/`](graphs/) Exported graphs used in the paper