https://github.com/camel-lab/barec_analyzer
https://github.com/camel-lab/barec_analyzer
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/camel-lab/barec_analyzer
- Owner: CAMeL-Lab
- License: mit
- Created: 2025-06-04T16:17:22.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2025-07-10T16:20:18.000Z (11 months ago)
- Last Synced: 2025-09-09T22:06:16.051Z (9 months ago)
- Language: Python
- Size: 24.4 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# BAREC Analyzer
This repository contains scripts for preprocessing, training, and evaluating the models in our paper [A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment](https://arxiv.org/abs/2502.13520).
The BAREC corpus is available on [Hugging Face](https://huggingface.co/datasets/CAMeL-Lab/BAREC-Corpus-v1.0).
## Repository Structure
- `scripts/preprocess.py`: Processes raw texts into our tokenized input variants (`Word`, `D3Tok`, `Lex`, and `D3Lex`). You **DO NOT** need to run this script to process the BAREC corpus as we already provide these input variants for the full corpus.
- `scripts/train.py`: Script for fine-tuning pre-trained models using the BAREC corpus. The script supports different loss functions and input variants. It also generates results and saves trained models.
- `scripts/collect_results.py`: Aggregates evaluation results from multiple trained models and exports them as CSV files for further analysis.
## Setup
### Install Dependencies
To run `scripts/preprocess.py`, you need to install [CAMeL Tools](https://github.com/CAMeL-Lab/camel_tools) and get the CAMeLBERT MSA morphosyntactic tagger from `camel_data`:
```sh
git clone https://github.com/CAMeL-Lab/camel_tools.git
cd camel_tools
conda create -n camel_tools python=3.9
conda activate camel_tools
pip install -e .
camel_data -i disambig-bert-unfactored-msa
```
To run `scripts/train.py` and `scripts/collect_results.py`:
```sh
git clone https://github.com/CAMeL-Lab/barec_analyzer.git
cd barec_analyzer
conda create -n barec python=3.9
conda activate barec
pip install -r requirements.txt
```
## Usage
### Preprocessing
Preprocess raw text to different input variants.
You **DO NOT** need this script if you want to train on the BAREC corpus as we already provide these input variants for the full corpus.
```sh
python scripts/preprocess.py \
--input \
--input_var \
--db \
--output
```
- `--input`: Path to input text file containing raw text data
- `--input_var`: Input variant (`Word`, `D3Tok`, `Lex`, or `D3Lex`)
- `--db` (**optional**): Path to morphological database to use for processing
- `--output`: Path to output file to save processed text data
**Important Note**: The default morphological analyzer used in the preprocessing script is not the same as the one in the paper, which is licensed by LDC. To download the same morphological analyzer, you need to:
1. Obtain the morphological analyzer from LDC ([LDC2010L01](https://catalog.ldc.upenn.edu/LDC2010L01)).
2. Download the muddled version of the analyzer from [here](https://github.com/CAMeL-Lab/CAMeLBERT_morphosyntactic_tagger/releases/download/v0.0.1/analyzer-msa.muddle).
3. Install [Muddler](https://github.com/CAMeL-Lab/muddler), a tool for sharing derived data, and use it to unmuddle the encrypted file.
```sh
pip install muddler
muddler unmuddle -s /PATH/TO/LDC2010L01.tgz -m /PATH/TO/analyzer-msa.muddle /PATH/TO/almor-s31.db.utf8
```
4. To use this analyzer in `scripts/preprocess.py`, pass it as a parameter (`--db "/PATH/TO/almor-s31.db.utf8"`).
### Training a Model
Run the training script on the BAREC corpus with configurable parameters:
```sh
python scripts/train.py \
--loss \
--model \
--input_var \
--save_dir \
--output_path
```
- `--loss`: Loss function (e.g., `CE`, `EMD`, `OLL1`, etc.)
- `--model`: Model checkpoint (e.g., HuggingFace model name or path)
- `--input_var`: Input variant (`Word`, `D3Tok`, `Lex`, or `D3Lex`)
- `--save_dir`: Base directory for saving trained model folders
- `--output_path`: Directory to save output XLSX files
### Collecting Results
After training multiple models, aggregate their results:
```sh
python scripts/collect_results.py \
--models_path \
--output_path
```
- `--models_path`: Directory containing all trained model folders
- `--output_path`: Directory to save the aggregated CSV files
## Citation
```
@inproceedings{elmadani-etal-2025-readability,
title = "A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment",
author = "Elmadani, Khalid N. and
Habash, Nizar and
Taha-Thomure, Hanada",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics"
}
```
## License
See the `LICENSE` file for license information.