https://github.com/deezer/synthetic_lyrics_detection
Code used for the paper "Synthetic Lyrics Detection Across Languages and Genres" (NAACL 2025 Workshop TrustNLP), co-authored by Yanis Labrak, Markus Frohmann, Gabriel Meseguer-Brocal, and Elena V. Epure.
https://github.com/deezer/synthetic_lyrics_detection
Last synced: 5 months ago
JSON representation
Code used for the paper "Synthetic Lyrics Detection Across Languages and Genres" (NAACL 2025 Workshop TrustNLP), co-authored by Yanis Labrak, Markus Frohmann, Gabriel Meseguer-Brocal, and Elena V. Epure.
- Host: GitHub
- URL: https://github.com/deezer/synthetic_lyrics_detection
- Owner: deezer
- License: other
- Created: 2025-03-06T14:22:32.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-05-21T11:53:10.000Z (about 1 year ago)
- Last Synced: 2025-05-21T12:49:03.818Z (about 1 year ago)
- Homepage:
- Size: 7.81 KB
- Stars: 0
- Watchers: 5
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# synthetic_lyrics_detection
This repository provides Python code to reproduce the experiments from the article [**Synthetic Lyrics Detection Across Languages and Genres**](https://aclanthology.org/2025.trustnlp-main.34/), accepted for publication to [**NAACL 2025 Workshop TrustNLP**](https://trustnlpworkshop.github.io/).
## Installation
```sh
git clone https://github.com/deezer/synthetic_lyrics_detection.git
cd synthetic_lyrics_detection
```
## Build and Run the Docker Image
Build the Docker image and run it in a container with an interactive bash session.
> **Note:** The current Docker image requires a **CUDA-capable GPU**.
```sh
make build
make run-bash
```
## Data Generation Pipeline
Install Ollama and pull the required models:
```sh
curl -fsSL https://ollama.com/install.sh | sh
ollama serve&
ollama pull mistral && ollama pull tinyllama && ollama pull wizardlm2
```
Run the data generation pipeline:
```sh
python3 data_pipeline/run_pipeline.py output/
```
> **Note:** Replace **** with the path to your JSON file containing human-written lyrics.
## Synthetic Lyrics Detection
Please refer to [this repository](https://github.com/deezer/robust-AI-lyrics-detection) which contains the detectors and scripts needed to run the experiments.
## Paper
Please cite our paper if you use this data or code in your work:
```
@inproceedings{labrak2024detecting,
author = {Labrak, Yanis and
Frohmann, Markus and
Meseguer-Brocal, Gabriel and
Epure, Elena V.},
booktitle = {Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025)},
editor = {Cao, Trista and
Das, Anubrata and
Kumarage, Tharindu and
Wan, Yixin and
Krishna, Satyapriya and
Mehrabi, Ninareh and
Dhamala, Jwala and
Ramakrishna, Anil and
Galystan, Aram and
Kumar, Anoop and
Gupta, Rahul and
Chang, Kai-Wei},
isbn = {979-8-89176-233-6},
month = may,
pages = {524--541},
publisher = {Association for Computational Linguistics},
title = {Synthetic Lyrics Detection Across Languages and Genres},
url = {https://aclanthology.org/2025.trustnlp-main.34/},
year = {2025},
address = {Albuquerque, New Mexico},
abstract = {In recent years, the use of large language models (LLMs) to generate music content, particularly lyrics, has gained in popularity. These advances provide valuable tools for artists and enhance their creative processes, but they also raise concerns about copyright violations, consumer satisfaction, and content spamming. Previous research has explored content detection in various domains. However, no work has focused on the text modality, lyrics, in music. To address this gap, we curated a diverse dataset of real and synthetic lyrics from multiple languages, music genres, and artists. The generation pipeline was validated using both humans and automated methods. We performed a thorough evaluation of existing synthetic text detection approaches on lyrics, a previously unexplored data type. We also investigated methods to adapt the best-performing features to lyrics through unsupervised domain adaptation. Following both music and industrial constraints, we examined how well these approaches generalize across languages, scale with data availability, handle multilingual language content, and perform on novel genres in few-shot settings. Our findings show promising results that could inform policy decisions around AI-generated music and enhance transparency for users.}
}
```