{"id":18322443,"url":"https://github.com/tencentarc/plot2code","last_synced_at":"2026-03-01T12:32:27.272Z","repository":{"id":239879583,"uuid":"799530252","full_name":"TencentARC/Plot2Code","owner":"TencentARC","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-17T11:49:13.000Z","size":112,"stargazers_count":18,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-05T23:31:46.654Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TencentARC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-12T12:25:31.000Z","updated_at":"2025-03-06T09:06:49.000Z","dependencies_parsed_at":"2024-05-15T22:22:57.081Z","dependency_job_id":"19ac25bd-e82c-4179-bfeb-223d75a398c8","html_url":"https://github.com/TencentARC/Plot2Code","commit_stats":null,"previous_names":["tencentarc/plot2code"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TencentARC/Plot2Code","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FPlot2Code","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FPlot2Code/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FPlot2Code/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FPlot2Code/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TencentARC","download_url":"https://codeload.github.com/TencentARC/Plot2Code/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FPlot2Code/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29969243,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T11:43:06.159Z","status":"ssl_error","status_checked_at":"2026-03-01T11:43:03.887Z","response_time":124,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T18:24:39.481Z","updated_at":"2026-03-01T12:32:27.253Z","avatar_url":"https://github.com/TencentARC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Plot2Code Benchmark\n\nPlot2Code benchmark is now open-sourced at [huggingface (ARC Lab)](https://huggingface.co/TencentARC) and [GitHub](https://github.com/TencentARC/Plot2Code). More information can be found in our [paper](https://arxiv.org/abs/2405.07990). \n\n\nThis repository contains the code for an evaluation pipeline that generates Python code from reference plots, executes the generated code to draw plots, and then calculates various evaluation metrics to assess the quality of the generated code.\n\n## Why we need [Plot2Code](https://huggingface.co/datasets/TencentARC/Plot2Code)?\n* 🧐 While MLLMs have demonstrated potential in visual contexts, their capabilities in visual coding tasks have not been thoroughly evaluated. Plot2Code offers a platform for comprehensive assessment of these models.\n\n* 🤗 To enable individuals to ascertain the proficiency of AI assistants in generating code that renders into plots given reference plots, we initiated the Plot2Code project. This ensures evaluations are pertinent to real-world applications.\n\n* 💻 Plot2Code accommodates all modalities (text and images) for both input and output, facilitating an exploration of the influence of each modality.\n\n## Supported Tasks\n\nPlot2Code is primarily designed as a benchmark for code generation from scientific plots. Specifically, it supports the following settings:\n\n* Text2Image: We provide instructions to the assistant, requesting it to generate pyplot code and subsequently render the plots.\n* Image2Image: Referred to as the Direct Asking setting in our paper, we input the reference plot directly and ask the assistant to generate pyplot code to render similar plots.\n* I+T 2 Image: Combining both instructions and reference plots as input, this is called the Conditional Asking setting in our paper.\n\nBy employing these settings, we can investigate the impact of each input modality on the quality of the final rendered plots.\n\n## Requirements\n\n- NumPy\n- Matplotlib==3.8.4\n- Pillow\n- Levenshtein\n- openai\u003e1.12.0\n\nYou can install the required packages using the following command:\n\n```bash\npip install -r requirements.txt\n```\n\n## How to Download\nYou can use following codes to download the dataset：\n```shell\ngit lfs install\nmkdir data\ncd data\ngit clone https://huggingface.co/datasets/TencentARC/Plot2Code\n```\n\n## Usage\n\n1. Generate code from reference plots. Add --instruction for the conditional setting.\n``` bash\nexport OPENAI_API_KEY=[API_KEY]\nexport OPENAI_API_BASE=[API_BASE]\n\n# GPT-4V generate code (direct asking)\npython -m plot2code.gpt4v_generate_code --prompt_strategy default\n\n# GPT-4V generate code (conditional asking)\npython -m plot2code.gpt4v_generate_code --prompt_strategy default --instruct\n\n# GPT-4V generate code (conditional asking with CoT)\npython -m plot2code.gpt4v_generate_code --prompt_strategy CoT --instruct\n```\n2. Execute the generated code to render the plots.\n``` bash\npython -m plot2code.execute_generated_code --model_name \"$model_name\" --prompt_strategy $prompt_strategy\n\n```\n3. Evaluate the similarity between the generated plots and the grond truth plots.\n\n``` bash\necho \"Calculating text match score...\"\npython -m plot2code.eval.text_match_score  --model_name \"$model_name\"  --prompt_strategy $prompt_strategy\n\necho \"Calculating gpt-4v evaluation score...\"\npython -m plot2code.eval.gpt4v_evaluations_score  --model_name \"$model_name\"  --prompt_strategy $prompt_strategy\n\necho \"Combining evaluation results...\"\npython -m plot2code.eval.combine_evaluation_results  --model_name \"$model_name\"  --prompt_strategy $prompt_strategy\n```\n\nSee [scripts](scripts) for more details.\n\n# News\n* 🔥[2024/08] We futther update the Python and R's plotly plot-code pairs with instruction for evaluation!🔥\n* 🔥[2024/05] We open source the [Plot2Code benchmark](https://huggingface.co/datasets/TencentARC/Plot2Code).\nStay tuned for this project! 😆\n\n# License\n\nIn this study, we crawled every website link listed in the Matplotlib gallery and Plotly documentation to collect data for our analysis. Both Matplotlib and Plotly libraries are distributed under permissive open-source licenses. We have taken the following steps to ensure compliance with the respective license terms:\n\n1. Acknowledgment of Licenses: We acknowledge that the Matplotlib library and its gallery are distributed under the BSD 3-Clause License, and the Plotly library and its documentation are distributed under the MIT License.\n2. Retention of Copyright Notices: We have retained all copyright notices and license information from the original Matplotlib gallery content and Plotly documentation, as required by their respective licenses.\n3. Usage and Distribution: Our use of the Matplotlib gallery and Plotly documentation content is solely for academic and research purposes. We have not modified the original content from the Matplotlib gallery or Plotly documentation, and any distribution of our work will include proper attribution to the Matplotlib and Plotly projects.\n\nBy adhering to these guidelines, we ensure that our use of the Matplotlib and Plotly content is fully compliant with their respective licenses.\n\nThis project is open-sourced under the [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0). These evaluation code and datasets are fully open for academic research and can be used for commercial purposes with official written permission. Check our [dataset sheet](dataset_sheet.pdf) for more information.\n\n# Citation\nThe code and model in this repository is mostly developed for or derived from the paper below. Please cite it if you find the repository helpful.\n```\n@misc{wu2024plot2code,\n      title={Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots}, \n      author={Chengyue Wu and Yixiao Ge and Qiushan Guo and Jiahao Wang and Zhixuan Liang and Zeyu Lu and Ying Shan and Ping Luo},\n      year={2024},\n      eprint={2405.07990},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fplot2code","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftencentarc%2Fplot2code","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fplot2code/lists"}