{"id":24809983,"url":"https://github.com/sushantnair/arxiv_extractor","last_synced_at":"2025-07-03T12:05:32.338Z","repository":{"id":272968363,"uuid":"918330636","full_name":"sushantnair/arxiv_extractor","owner":"sushantnair","description":"This code can effectively convert PDF Research Papers to clean Text files, avoiding images and tables.","archived":false,"fork":false,"pushed_at":"2025-01-18T16:05:22.000Z","size":8,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-25T12:23:50.792Z","etag":null,"topics":["arxiv","arxiv-papers","good-project","mozilla-firefox","pdf","pdf-to-text","research-paper","text"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sushantnair.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-17T17:54:49.000Z","updated_at":"2025-01-26T11:37:21.000Z","dependencies_parsed_at":"2025-01-18T17:20:04.933Z","dependency_job_id":null,"html_url":"https://github.com/sushantnair/arxiv_extractor","commit_stats":null,"previous_names":["sushantnair/arxiv-pdf-to-text","sushantnair/arxiv_extractor"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sushantnair/arxiv_extractor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sushantnair%2Farxiv_extractor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sushantnair%2Farxiv_extractor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sushantnair%2Farxiv_extractor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sushantnair%2Farxiv_extractor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sushantnair","download_url":"https://codeload.github.com/sushantnair/arxiv_extractor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sushantnair%2Farxiv_extractor/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263322792,"owners_count":23448714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arxiv","arxiv-papers","good-project","mozilla-firefox","pdf","pdf-to-text","research-paper","text"],"created_at":"2025-01-30T11:16:29.297Z","updated_at":"2025-07-03T12:05:32.308Z","avatar_url":"https://github.com/sushantnair.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# arXiv Extractor\n#### This code (for now) can effectively convert PDF Research Papers to clean Text files, avoiding images and tables.\n# Requirements\n#### It requires Playwright. \n```pip install playwright```\n#### It also requires firefox to be installed in Playwright environment.\n```playwright install firefox```\n# Compatible Softwares\n#### Currently, it works only for Mozilla Firefox Browsers.\nThis is because of the way Firefox renders the PDFs. It does in a way quite different than other browsers. This made it advantageous and easier. \n# Features to Add\n1. Conversion from PDF to other formats as well\n2. More field-specific cleaning (not just CS papers)\n# FAQ\n#### 1. Why I did not use the ArXiv API?\n\u003cem\u003e\"This url calls the api, which returns the results in the Atom 1.0 format.\"\u003c/em\u003e\nTo know more, click \u003ca href=\"https://info.arxiv.org/help/api/basics.html\"\u003ehere\u003c/a\u003e for the official documentation.\n#### 2. Will additional browser support be added?\nWell, I think it is easier to just install Firefox! You can open up a PDF of a research paper in other browsers vs. Firefox and understand the notable difference in the way the information is presented.\n#### 3. Does it work for all OS?\nActually at the time of release it worked fine on Windows. I am not sure about other OS.\n#### 4. Is there any scope for contributions?\nAbsolutely! If you can find a way to extract information from the unhelpful way it is displayed in other Browsers, or if you can extend support for other OS, or if you think there is a way you can further improve the quality of the extracted text, then you are welcome! Just submit a PR.\n# Gratitude\n#### I am grateful for the help I got from Mistral's Le Chat. It helped me overcome significant challenges.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsushantnair%2Farxiv_extractor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsushantnair%2Farxiv_extractor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsushantnair%2Farxiv_extractor/lists"}