https://github.com/yuvalpinter/rationale_analysis
https://github.com/yuvalpinter/rationale_analysis
Last synced: over 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/yuvalpinter/rationale_analysis
- Owner: yuvalpinter
- License: mit
- Created: 2019-11-15T20:48:41.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2019-11-27T14:54:30.000Z (over 6 years ago)
- Last Synced: 2025-02-07T18:16:12.455Z (over 1 year ago)
- Language: Python
- Size: 14.6 MB
- Stars: 0
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Suggested use is to use Anaconda and do `pip install -r requirements.txt` .
1. `data` : Folder to store datasets
2. `Rationale_Analysis/models` : Folder to store allennlp models
1. `classifiers` : Models that do actually learning
2. `saliency_scorer` : Takes a trained model and return saliency scorers for inputs
3. `rationale_extractors` : Models that take saliency scores and generate rationales in form readable by `rationale_reader.py`
4. `base_predictor.py` : Simple predictor to use with allennlp predict command as needed
3. `Rationale_Analysis/subcommands` : Subcommands to run saliency and rationale extractors since allennlp existingcommand semantics doesn't map quite as well to what we wanna do.
4. `Rationale_Analysis/training_config` : Contains jsonnet training configs to use with allennlp for each of the three types of models above.
5. `Rationale_Analysis/commands` : Actual bash scripts to run stuff.
6. `Rationale_Analysis/data/dataset_readers` : Contains dataset readers to work with Allennlp.
1. `rationale_reader.py` : Code to load actual datasets (jsonl with 3 fields - document, query, label)
2. `saliency_reader.py` : Read output of Saliency scorer to pass into rationale extractors.
Experimental Setup
==================
1. Train any model
```bash
CUDA_DEVICE=0 \
DATASET_NAME= \
CLASSIFIER= \
DATA_BASE_PATH= \
EXP_NAME= \
bash Rationale_Analysis/commands/model_train_script.sh
```
You `path to data` folder should contain three files - {train/dev/test}.jsonl . Each should be a list of dicts containing atleast two fields -
```python
{
"document" : str,
"label" : str,
"query" : Optional[str]
}
```
Output generated in `outputs////` .
2. Generate saliency scores
```bash
CUDA_DEVICE=0 \
DATASET_NAME=SST \
CLASSIFIER= \
DATA_BASE_PATH=Datasets/SST/data \
EXP_NAME= \
SALIENCY= \
bash Rationale_Analysis/commands/saliency_script.sh
```
Output generate in `outputs//SST//_saliency` .
3. Extract rationales from saliency
```bash
CUDA_DEVICE=0 \
DATASET_NAME=SST \
CLASSIFIER= \
EXP_NAME= \
SALIENCY= \
RATIONALE= \
RATIONALE_EXP_NAME= \
bash Rationale_Analysis/commands/rationale_extractor_script.sh
```
Output generated in `outputs//SST//_saliency/_rationale/`.
4. Extract Rationales and train model b.
```bash
CUDA_DEVICE=0 \
DATASET_NAME=SST \
CLASSIFIER= \
EXP_NAME= \
SALIENCY= \
RATIONALE= \
RATIONALE_EXP_NAME= \
bash Rationale_Analysis/commands/rationale_and_train_model_b_script.sh
```
Allowed values for now
1. saliency name - file names in training_config/saliency_scorers
2. classifier type - "bert_classification"
3. rationale extraction type - filenames in training_config/rationale_extractors . Note each rationale extractor may also need some hyperparameter setting through env variables.