{"id":34680879,"url":"https://github.com/us/trocr_mathwriting","last_synced_at":"2026-05-17T16:38:41.973Z","repository":{"id":239496150,"uuid":"799670772","full_name":"us/trocr_mathwriting","owner":"us","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-10T00:34:24.000Z","size":468,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-10T01:36:34.269Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/us.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-12T20:23:32.000Z","updated_at":"2024-08-10T00:34:28.000Z","dependencies_parsed_at":"2024-05-15T13:37:24.170Z","dependency_job_id":null,"html_url":"https://github.com/us/trocr_mathwriting","commit_stats":null,"previous_names":["us/trocr_mathwriting"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/us/trocr_mathwriting","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/us%2Ftrocr_mathwriting","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/us%2Ftrocr_mathwriting/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/us%2Ftrocr_mathwriting/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/us%2Ftrocr_mathwriting/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/us","download_url":"https://codeload.github.com/us/trocr_mathwriting/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/us%2Ftrocr_mathwriting/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28008469,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-24T02:00:07.193Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-24T21:08:41.955Z","updated_at":"2025-12-24T21:08:42.831Z","avatar_url":"https://github.com/us.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TrOCR on MathWriting\n\n## Overview\n\n**TrOCR on MathWriting** is an innovative project that leverages the power of the TrOCR model developed by Microsoft to convert images of mathematical handwriting into machine-readable LaTeX text. This project is uniquely tailored to work with the MathWriting dataset, offering a comprehensive solution for recognizing handwritten mathematical expressions. Explore more about the model on our [Hugging Face repository](https://huggingface.co/us4/trocr-mathwriting).\n\nYou can access the pretrained model from https://huggingface.co/us4/trocr-mathwriting .\n\n\n### About the MathWriting Dataset\n\nMathWriting is the largest online dataset of handwritten mathematical expressions (HME) available today. It contains over **230,000** human-written samples and an additional **400,000** synthetic samples, making it the ideal dataset for both online and offline HME recognition tasks. MathWriting surpasses the size of all existing offline HME datasets like IM2LATEX-100K, providing unparalleled resources for researchers and developers.\n\n## Hugging Face Model Usage\n\nLeverage the pre-trained models with Hugging Face:\n\n```python\nfrom transformers import TrOCRProcessor, VisionEncoderDecoderModel\n\n# Load the processor and model\nprocessor = TrOCRProcessor.from_pretrained(\"microsoft/trocr-large-handwritten\")\nmodel = VisionEncoderDecoderModel.from_pretrained(\"us4/trocr-mathwriting\")\n```\n\n## Installation\n\nTo set up the **TrOCR on MathWriting** project, follow these steps:\n\n```bash\ngit clone https://github.com/us/trocr_mathwriting.git\ncd trocr_mathwriting\npip install -r requirements.txt\n```\n\n## Dataset Preparation\n\nPrepare your dataset using our conversion script:\n\n- **inkml_to_image.py**: This script converts `.inkml` files into images and creates a `metadata.csv` file containing `file_name` and `label`, which are essential for model training.\n\n  Usage:\n\n  ```bash\n  python inkml_to_image.py --help\n  \n  usage: inkml_to_image.py [-h] --input INPUT --output OUTPUT\n                           [--size WIDTH HEIGHT]\n\n  Process InkML files and generate images and annotations.\n\n  options:\n    -h, --help            show this help message and exit\n    --input INPUT, -i INPUT\n                          Directory containing InkML files\n    --output OUTPUT, -o OUTPUT\n                          Output directory for images and CSV annotations\n    --size WIDTH HEIGHT, -s WIDTH HEIGHT\n                          Dimensions of output images\n  ```\n\n## Usage\n\nThe workflow for using this project is as follows:\n\n1. **Prepare the Dataset**: Convert the MathWriting dataset (in INKML format) to images using the `inkml_to_image.py` script, and save these images along with their annotations in the `mw_images` directory. We use the default TrOCR input size `384x384` for width and height.\n\n   ```bash\n   python inkml_to_image.py -i mw/train -o mw_images/train\n   python inkml_to_image.py -i mw/test -o mw_images/test\n   python inkml_to_image.py -i mw/symbols -o mw_images/symbols\n   python inkml_to_image.py -i mw/synthetic -o mw_images/synthetic\n   python inkml_to_image.py -i mw/valid -o mw_images/valid\n   ```\n\n2. **Fine-Tuning the TrOCR Model**: Once the images are prepared, use the `Fine_tune_TrOCR_on_MathWriting_Database_using_Seq2SeqTrainer.ipynb` notebook to fine-tune the TrOCR model on the MathWriting images.\n\n   **Note**: Ensure that you have downloaded the MathWriting dataset, renamed it as `mw`, and have created the `mw_images` dataset according to the preparation step.\n\n3. **Prediction**\n\n   Use the `predict.py` script to process images and output results:\n\n   ```bash\n   python predict.py --help\n   usage: predict.py [-h] --input_dir INPUT_DIR --output_file OUTPUT_FILE\n                     --checkpoint_path CHECKPOINT_PATH\n\n   Process images and output results to a CSV file.\n\n   options:\n     -h, --help            show this help message and exit\n     --input_dir INPUT_DIR\n                           Directory containing images to process\n     --output_file OUTPUT_FILE\n                           Output CSV file to write results\n     --checkpoint_path CHECKPOINT_PATH\n                           Path to the model checkpoint\n   ```\n\n\n## Contributing\n\nContributions to **TrOCR on MathWriting** are welcome. Please ensure to follow the existing coding standards and include tests with your code if applicable.\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n---\n\nDiscover the future of handwritten mathematical expression recognition with TrOCR on MathWriting. Visit our [Hugging Face repository](https://huggingface.co/us4/trocr-mathwriting) to explore more and get started today!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fus%2Ftrocr_mathwriting","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fus%2Ftrocr_mathwriting","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fus%2Ftrocr_mathwriting/lists"}