https://github.com/soskek/arxiv_leaks
Whisper of the arxiv: read comments in tex of papers
https://github.com/soskek/arxiv_leaks
arxiv arxiv-analytics deep-learning scraper
Last synced: about 1 year ago
JSON representation
Whisper of the arxiv: read comments in tex of papers
- Host: GitHub
- URL: https://github.com/soskek/arxiv_leaks
- Owner: soskek
- Created: 2017-09-19T08:10:03.000Z (almost 9 years ago)
- Default Branch: master
- Last Pushed: 2018-05-16T02:42:29.000Z (about 8 years ago)
- Last Synced: 2025-03-28T20:21:17.503Z (over 1 year ago)
- Topics: arxiv, arxiv-analytics, deep-learning, scraper
- Language: Python
- Homepage:
- Size: 3.91 KB
- Stars: 31
- Watchers: 3
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# ArxivLeaks
Most of papers on arxiv have latex files, which often contain much comment out. Dig up the valuable comments!
For example, you can extract a secret comment from "[Attention Is All You Need](https://arxiv.org/abs/1706.03762)", as below:
>\\paragraph{Symbol Dropout} In the source and target embedding layers, we replace a random subset of the token ids with a sentinel id. For the base model, we use a rate of $symbol\\_dropout\\_rate=0.1$. Note that this applies only to the auto-regressive use of the target ids - not their use in the cross-entropy loss.
We found "Symbol Dropout", which do not appear in the paper (pdf).
## Run
Feed a text/page containing arxiv URLs by `-t`.
```
python -u run.py -t deepmind.html -s arxiv_dir
```
To test, run `sh test.sh`. This pre-downloads a publication page of deepmind.
You can also read only selected papers by `-i`, feeding their arxiv ids.
```
python -u run.py -i 1709.04905 1706.03762 -s arxiv_dir
```
- `-s`: Downloaded arxiv pages and files are stored into this directory.
- `-o`: Output is printed and saved as a json file with this file path. Default is `./comments.json`.
## Requirement
- requests
- lxml
## For Writers
You can remove `%`-comments from your file as follows:
```
perl -pe 's/(^|[^\\])%.*/\1%/' < old.tex > new.tex
```
This one line command is given from [arxiv](https://arxiv.org/help/faq/whytex).