{"id":17030717,"url":"https://github.com/vsoch/arxiv-catalog","last_synced_at":"2025-03-22T20:28:31.995Z","repository":{"id":141667620,"uuid":"158029275","full_name":"vsoch/arxiv-catalog","owner":"vsoch","description":"Let's go shopping! Or let's not, that's a terrible idea. But how about for equations? I like that much better.","archived":false,"fork":false,"pushed_at":"2018-11-20T16:38:52.000Z","size":424,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-28T00:30:59.732Z","etag":null,"topics":["arxiv","equations","mathjax"],"latest_commit_sha":null,"homepage":"https://vsoch.github.io/arxiv-catalog/","language":"CSS","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vsoch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-17T22:12:21.000Z","updated_at":"2020-08-17T05:22:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"804824ce-8880-42dc-abd4-4030f2251003","html_url":"https://github.com/vsoch/arxiv-catalog","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vsoch%2Farxiv-catalog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vsoch%2Farxiv-catalog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vsoch%2Farxiv-catalog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vsoch%2Farxiv-catalog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vsoch","download_url":"https://codeload.github.com/vsoch/arxiv-catalog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245018283,"owners_count":20547966,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arxiv","equations","mathjax"],"created_at":"2024-10-14T08:08:00.995Z","updated_at":"2025-03-22T20:28:31.970Z","avatar_url":"https://github.com/vsoch.png","language":"CSS","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The Arxiv Catalog\n\nAre you shopping for papers, based on their equations? This is a fun project\nrelated to [arxiv-equations](https://www.github.com/vsoch/arxiv-equations)\nto generate a rendering of the equations that are used in a research article.\nFor this first step, we just want to visualize summary metrics based on\ncategories, along with the rendered equations. For some future version,\nwe will go one step further and make these equations into pretty images\nusing [penrose](https://www.github.com/penrose/penrose).\n\n## Catalog Cover Page\nIt might be interesting to show some animation of [equation families over time](https://codepen.io/golle404/pen/jWZpoy).\n\n## Collections\n\nEach summary page I will represent as a collection, in the folder topics.\nThe input data to generate a table of summary metrics, organized with category in\nrows and metric in columns, was created [by this script](), and then\nthe script to generate the yaml front end matter to parse into the collection\nis [here]().\n\n## Item Pages\n\nThe remaining content (the posts) includes all (N=?) of the individual articles in\nthe arxiv dump, each of which falls into one of the collections above. The script\n[Generate](generatePage.py) was used to generate the yaml front end matter that\nrenders into the html/css page for a given article.\n\n## Generation\n\n### 1. Single Article Pages\n\nEach paper from Arxiv was downloaded via the [bulk data access](https://arxiv.org/help/bulk_data), meaning upper level tars and then extracted .tar.gz for each article. This work was done in the [arxiv-equations](https://www.github.com/vsoch/arxiv-equations) repository to generate a pickle of each paper. The pickle includes a dictionary data structure that has the following:\n\n - tex: the raw latex, as a string\n - metadata: metadata retrieved from the arxiv api, by using the [arxiv](https://github.com/lukasschwab/arxiv.py) python wrapper. Added is a length (the length of the latex).\n - equations: is a complete listing of equations (and latex symbols) from the article, extracted with the regular expression `\\\\$.*?(?\u003c!\\\\\\\\)\\\\$` run against the raw latex.\n\nUsing the script [generatePage.py](generatePage.py) we extract metadata and the \nequations to generate a markdown file with frontend matter including all of the \nabove (except for the tex). Instead of the tex, in the content area we put a \nsummary of the article. Each of these is saved into the [_posts](_posts) folder,\nand will be rendered into a signle article template, the goal being to visualize\nthe equations in an article.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvsoch%2Farxiv-catalog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvsoch%2Farxiv-catalog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvsoch%2Farxiv-catalog/lists"}