{"id":21602080,"url":"https://github.com/cosmic-heart/Benetech-Chart-Derendering","last_synced_at":"2025-07-17T04:31:12.679Z","repository":{"id":236595582,"uuid":"647131210","full_name":"cosmic-heart/Benetech-Chart-Derendering","owner":"cosmic-heart","description":"Benetech Kaggle Competition Work. Fine Tuning Matcha (Multi Modal Transformer) on Line, Scatter, Dot, Horizontal and Vertical bar dataset.","archived":false,"fork":false,"pushed_at":"2023-06-20T03:32:24.000Z","size":9308,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-11T02:43:32.157Z","etag":null,"topics":["benetech","chart-derendering","deplot","image-to-text","matcha","transformers"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cosmic-heart.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-30T06:05:00.000Z","updated_at":"2024-02-21T03:21:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"41b24d33-8b12-43e2-bcb7-517e64b1a0d6","html_url":"https://github.com/cosmic-heart/Benetech-Chart-Derendering","commit_stats":null,"previous_names":["navinkumarmnk/benetech-chart-derendering","cosmic-heart/benetech-chart-derendering"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cosmic-heart/Benetech-Chart-Derendering","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-heart%2FBenetech-Chart-Derendering","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-heart%2FBenetech-Chart-Derendering/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-heart%2FBenetech-Chart-Derendering/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-heart%2FBenetech-Chart-Derendering/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cosmic-heart","download_url":"https://codeload.github.com/cosmic-heart/Benetech-Chart-Derendering/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-heart%2FBenetech-Chart-Derendering/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265564846,"owners_count":23788926,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benetech","chart-derendering","deplot","image-to-text","matcha","transformers"],"created_at":"2024-11-24T19:11:56.745Z","updated_at":"2025-07-17T04:31:12.667Z","avatar_url":"https://github.com/cosmic-heart.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BeneTech - Making Graphs Accessible \n\n![Alt text](\u003cassets/benetech.png\u003e)\n\n## Matcha \n\n\u003e MatCha is a model that is trained using the Pix2Struct architecture1. It is a Visual Question Answering subset of the Pix2Struct architecture. It renders the input question on the image and predicts the answer.\n\n\u003e MatCha is a model that enhances visual language models’ capabilities in jointly modeling charts/plots and language data. It is trained using the Pix2Struct architecture and performs several pretraining tasks that cover plot deconstruction and numerical reasoning, which are key capabilities in visual language modeling.\n\n\u003e On standard benchmarks such as PlotQA and ChartQA, the MatCha model outperforms state-of-the-art methods by as much as nearly 20%1. This is a significant improvement over other VQA models. MatCha pretraining also transfers well to domains such as screenshots, textbook diagrams, and document figures, verifying the usefulness of MatCha pretraining on broader visual language tasks\n\n![Alt text](\u003cassets/matcha.png\u003e)\n## Procedure \n\n\u003e Matcha (Pix2Struct) : To Derender Line, Vertical Bar, Horizontal Bar \n\n\u003e For Dot \u0026 Scatter Plot : icevision Faster RCNN Model \n\n## Datasets Used\n\n\u003e Official Kaggle Competition Data : https://www.kaggle.com/competitions/benetech-making-graphs-accessible/data\n\n\u003e Extra Generated Data : https://www.kaggle.com/datasets/brendanartley/benetech-extra-generated-data \u003cbr\u003e\n    - Deplot Training Format\n\n\u003e IDCAR 2023 : https://www.kaggle.com/datasets/sambitmukherjee/b-m-g-a-extra-data \u003cbr\u003e \n    - Additional images have been picked from the ICDAR 2023 Competition on Harvesting Answers and Raw Tables from Infographics (CHART-Infographics) website: https://chartinfo.github.io/ \u003cbr\u003e\n    - Annotations have been created using Make Sense (makesense.ai) \u0026 Roboflow, and through exploratory data analysis of the Benetech - Making Graphs Accessible Kaggle competition data \u0026 the ICDAR 2023 competition data.\n\n\n\n## Model Weights\n\n\u003e https://www.kaggle.com/datasets/navinkumarmnk/benetech-matcha-models\n \n## Results\n![Alt text](assets/results_matcha.png?raw=true \"Results\")\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcosmic-heart%2FBenetech-Chart-Derendering","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcosmic-heart%2FBenetech-Chart-Derendering","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcosmic-heart%2FBenetech-Chart-Derendering/lists"}