https://github.com/idiap/sentence-planner
https://github.com/idiap/sentence-planner
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/idiap/sentence-planner
- Owner: idiap
- License: other
- Created: 2021-09-30T13:05:45.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2022-04-06T12:33:31.000Z (about 4 years ago)
- Last Synced: 2025-03-23T01:01:56.118Z (about 1 year ago)
- Language: Python
- Size: 12.7 MB
- Stars: 6
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# sentence-planner
This is the code for the paper [Sentence-level Planning for Especially Abstractive Summarization](https://aclanthology.org/2021.newsum-1.1.pdf) presented at the *New Frontiers in Summarization* workshop at EMNLP 2021.
The repository is a fork of the [PreSumm](https://github.com/nlpyang/PreSumm) repository. Changed and added source files are marked in their header comment, others are left untouched.
## Contents
1. [Data](#data)
2. [Installation](#installation)
3. [Training](#training)
4. [Evaluation](#evaluation)
5. [Citation](#citation)
## Data
For CNN/DM, download the preprocessed data or follow the instructions in the [PreSumm](https://github.com/nlpyang/PreSumm) repository.
For Curation Corpus, follow the instructions over at the [Curation Corpus](https://github.com/CurationCorp/curation-corpus) repository to download the articles. Then follow the instructions in Appendix B of our paper for the preprocessing. If you have trouble reconstructing the dataset, do not hesitate to contact us.
## Installation
First, get a working installation of conda, e.g. [Miniconda](https://docs.conda.io/en/latest/miniconda.html).
Then, recreate the environment from inside this directory by running:
```shell
conda env create -f environment.yml
conda activate presumm
```
## Training
The model can be trained with:
```shell
python src/train.py \
--model sentsumm \
--sentplan_loss_weight 1 \
--mode train \
--bert_data_path /curation \
--result_path /curation \
--model_path \
--log_file train.log
```
A few things to note:
* Data paths consist of the path to the data directory and the prefix of the preprocessed data files (in our case, this is "cnndm" or "curation").
* The same holds for result paths.
## Evaluation
### Validation
Similar to training, the model is validated as follows:
```shell
python src/train.py \
--model sentsumm \
--sentplan_loss_weight 1 \
--mode validate \
--bert_data_path /curation \
--result_path /curation \
--model_path \
--log_file validate.log \
--test_all
```
By looking at the results in the log file, you can select the best checkpoint to keep and discard the rest.
Validation also generates candidate-reference summary pairs, each in their respective files in the results directory. These are used in the evaluations below.
### ROUGE
The official Perl implementation can be called with a helper script like:
```shell
python src/perl_rouge.py \
--candidates_path / \
--references_path /
```
Installation instructions for the Perl ROUGE script can be found at the [PreSumm](https://github.com/nlpyang/PreSumm) repository.
### Novel bigrams, sentences, words
There is a short evaluation script for each of these, and they all work the same:
```shell
python src/eval/novel_bigrams.py --eval_dir
```
### Attribution
To compute the attribution to the sentence representation (Section 4.2), use the `src/gxa.py` for the Integrated Gradients algorithm, or `src/conductance.py` for the Conductance algorithm.
Example usage:
```shell
python src/gxa.py \
--model_path \
--result_path \
--bert_data_path /curation \
--share_word_enc \
--use_dec_src_attn \
--use_sent_cond \
--see_past_sents \
--num_examples 100 \
--num_ig_steps 50 \
--baseline zero
```
### Corefs
The Corefs evaluation tests the number of coreference links across sentence boundaries. The evaluation comes with a separate conda environment. Additionally, you have to download a spacy model, in our case `en_core_web_lg`.
Run it with:
```shell
conda env create -f coref.yml
conda activate coref
python -m spacy download en_core_web_lg
python src/eval/coref.py --eval_dir
```
## Citation
If you use this code, please cite us.
```
@inproceedings{marfurt-henderson-2021-sentence,
title = "Sentence-level Planning for Especially Abstractive Summarization",
author = "Marfurt, Andreas and
Henderson, James",
booktitle = "Proceedings of the Third Workshop on New Frontiers in Summarization",
month = nov,
year = "2021",
address = "Online and in Dominican Republic",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.newsum-1.1",
doi = "10.18653/v1/2021.newsum-1.1",
pages = "1--14",
}
```