{"id":20623240,"url":"https://github.com/soskek/arxiv_leaks","last_synced_at":"2025-04-15T12:36:52.331Z","repository":{"id":113372080,"uuid":"104046088","full_name":"soskek/arxiv_leaks","owner":"soskek","description":"Whisper of the arxiv: read comments in tex of papers","archived":false,"fork":false,"pushed_at":"2018-05-16T02:42:29.000Z","size":4,"stargazers_count":31,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-28T20:21:17.503Z","etag":null,"topics":["arxiv","arxiv-analytics","deep-learning","scraper"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/soskek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-19T08:10:03.000Z","updated_at":"2024-05-07T18:19:09.000Z","dependencies_parsed_at":"2023-06-15T11:30:47.618Z","dependency_job_id":null,"html_url":"https://github.com/soskek/arxiv_leaks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soskek%2Farxiv_leaks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soskek%2Farxiv_leaks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soskek%2Farxiv_leaks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soskek%2Farxiv_leaks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/soskek","download_url":"https://codeload.github.com/soskek/arxiv_leaks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249072826,"owners_count":21208253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arxiv","arxiv-analytics","deep-learning","scraper"],"created_at":"2024-11-16T12:26:26.534Z","updated_at":"2025-04-15T12:36:52.319Z","avatar_url":"https://github.com/soskek.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ArxivLeaks\n\nMost of papers on arxiv have latex files, which often contain much comment out. Dig up the valuable comments!\n\nFor example, you can extract a secret comment from \"[Attention Is All You Need](https://arxiv.org/abs/1706.03762)\", as below:\n\n\u003e\\\\paragraph{Symbol Dropout} In the source and target embedding layers, we replace a random subset of the token ids with a sentinel id.  For the base model, we use a rate of $symbol\\\\_dropout\\\\_rate=0.1$.  Note that this applies only to the auto-regressive use of the target ids - not their use in the cross-entropy loss.\n\nWe found \"Symbol Dropout\", which do not appear in the paper (pdf).\n\n\n## Run\n\nFeed a text/page containing arxiv URLs by `-t`.\n```\npython -u run.py -t deepmind.html -s arxiv_dir\n```\nTo test, run `sh test.sh`. This pre-downloads a publication page of deepmind.\n\nYou can also read only selected papers by `-i`, feeding their arxiv ids.\n```\npython -u run.py -i 1709.04905 1706.03762 -s arxiv_dir\n```\n\n- `-s`: Downloaded arxiv pages and files are stored into this directory.\n- `-o`: Output is printed and saved as a json file with this file path. Default is `./comments.json`.\n\n\n## Requirement\n\n- requests\n- lxml\n\n\n## For Writers\n\nYou can remove `%`-comments from your file as follows:\n\n```\nperl -pe 's/(^|[^\\\\])%.*/\\1%/' \u003c old.tex \u003e new.tex\n```\n\nThis one line command is given from [arxiv](https://arxiv.org/help/faq/whytex).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoskek%2Farxiv_leaks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoskek%2Farxiv_leaks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoskek%2Farxiv_leaks/lists"}