{"id":13714225,"url":"https://github.com/qs956/Latex_OCR_Pytorch","last_synced_at":"2025-05-07T01:32:52.587Z","repository":{"id":38535359,"uuid":"243772326","full_name":"qs956/Latex_OCR_Pytorch","owner":"qs956","description":"基于Pytorch实现的End-to-End图像Latex公式识别 inspire by LinXueyuanStdio/LaTeX_OCR_PRO","archived":false,"fork":false,"pushed_at":"2020-04-06T03:20:26.000Z","size":195,"stargazers_count":158,"open_issues_count":8,"forks_count":24,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-08-03T23:29:59.542Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qs956.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-28T13:46:26.000Z","updated_at":"2024-06-28T03:32:13.000Z","dependencies_parsed_at":"2022-08-09T04:31:48.731Z","dependency_job_id":null,"html_url":"https://github.com/qs956/Latex_OCR_Pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qs956%2FLatex_OCR_Pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qs956%2FLatex_OCR_Pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qs956%2FLatex_OCR_Pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qs956%2FLatex_OCR_Pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qs956","download_url":"https://codeload.github.com/qs956/Latex_OCR_Pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224551184,"owners_count":17330093,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T23:01:55.138Z","updated_at":"2024-11-14T01:30:50.500Z","avatar_url":"https://github.com/qs956.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"Latex_OCR_Pytorch\n\n主要是这个版本的Pytorch实现:\n\n[LinXueyuanStdio/LaTeX_OCR_PRO](https://github.com/LinXueyuanStdio/LaTeX_OCR_PRO)\n\n感谢@LinXueyuanStdio 的工作以及指导.本项目与上述项目思路一致，但在实现上修改了一些地方:\n\n* 数据集的重新定义,但使用原有类似的预处理方式\n* 代码简化，目前仅保留主要部分，命令行控制等在后续补充\n* 内存优化，相对较少的内存需求，支持较大批量的训练。但批大小一样的情况下实测速度提高不大\n* 使用Checkpoint特性，在编码过程中出现OOM则自动进行分段计算\n* 在训练时候采用贪婪策略，Beam Search仅在推断时候采用\n* Scheduled Sampling策略\n\nFollow these paper:\n\n1. [Show, Attend and Tell(Kelvin Xu...)](https://arxiv.org/abs/1502.03044)\n2. [Harvard's paper and dataset](http://lstm.seas.harvard.edu/latex/)\n\nFollow these tutorial:\n\n1. [Seq2Seq for LaTeX generation](https://guillaumegenthial.github.io/image-to-latex.html).\n2. [a PyTorch Tutorial to Image Captioning](https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning).\n\n## 环境\n1. Python \u003e= 3.6\n\n2. Pytorch \u003e= 1.2\n\n## 数据\n使用[LinXueyuanStdio/Data-for-LaTeX_OCR](https://github.com/LinXueyuanStdio/Data-for-LaTeX_OCR) 数据集,原仓库较大,后续提供打包下载.\n\n已包括上述仓库中small数据集\n印刷体数据全集[百度云](https://pan.baidu.com/s/1xIsgHDhVu85L8cGdqqG7kw ) 提取码：tapj [Google Drive](https://drive.google.com/open?id=1THp_O7uwavcjsnQXsxx_JPvYn9-gml7T)\n自己划分的混合CROHME2011,2012数据集[Google Drive](https://drive.google.com/open?id=1KgpAzA7k8ayjPTstin6M8ykGsW8GR9bu)\n\n\n## 训练模型\n在自己划分CROHME2011,2012数据集上使用以下参数的训练模型[Google Drive](https://drive.google.com/open?id=1_geqm9a86TJKK9RpZ39d9X5655s4NXa9)\nemb_dim = 30 \nattention_dim = 128\ndecoder_dim = 128\n后续补充模型测试结果以及colab\n\n## 数据格式\n\n数据集文件生成参考[utils.py](./model/utils.py)的get_latex_ocrdata\n\n数据集文件json格式,包括训练集文件,验证集文件,字典文件.\n\n字典格式:\n\npython字典(符号——编号)的json储存\n\n数据集格式:\n\n```\n​```shell\n训练/验证数据集\n├── file_name1 图片文件名 str\n│   ├── img_path:文件路径(到文件名,含后缀) str\n│   ├── size:图片尺寸 [长,宽] list\n│   ├── caption:图片代表的公式,各个符号之间必须要空格分隔 str\n│   └── caption_len:len(caption.split()) int\n|   ...\neg:\n{\n\"0.png\":\n    {\n    \"img_path\":\"./mydata/0.png\",\n    \"size\":[442,62],\n    \"caption\":\"\\frac { a + b } { 2 }\",\n    \"caption_len\":9,\n    }\n\"2.png\":...\n}\n\n​```\n```\n\n图片预处理\n\n参考dataloader/data_turn主要进行以下操作\n\n1. 灰度化\n2. 裁剪公式部分\n3. 上下左右各padding 8个像素\n4. `[可选]`下采样\n\n\n## To do\n\n- [ ]  推断部分\n- [ ] Attention层的可视化\n- [x] 预训练模型\n- [x] 打包的训练数据\n- [ ] perplexity指标","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqs956%2FLatex_OCR_Pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqs956%2FLatex_OCR_Pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqs956%2FLatex_OCR_Pytorch/lists"}