{"id":20106319,"url":"https://github.com/hiroakimikami/mlprogram","last_synced_at":"2025-05-06T09:31:53.515Z","repository":{"id":53500216,"uuid":"155963033","full_name":"HiroakiMikami/mlprogram","owner":"HiroakiMikami","description":"PyTorch library for synthesizing programs from natural language","archived":false,"fork":false,"pushed_at":"2021-03-28T05:44:44.000Z","size":9475,"stargazers_count":17,"open_issues_count":28,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-08-07T00:36:50.224Z","etag":null,"topics":["deep-learning","deep-neural-network","deeplearning","natural-language-understanding","nl2code","program-synthesis","pytorch","treegen"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HiroakiMikami.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-03T08:05:26.000Z","updated_at":"2023-03-04T21:40:18.000Z","dependencies_parsed_at":"2022-08-25T01:01:38.039Z","dependency_job_id":null,"html_url":"https://github.com/HiroakiMikami/mlprogram","commit_stats":null,"previous_names":[],"tags_count":2,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HiroakiMikami%2Fmlprogram","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HiroakiMikami%2Fmlprogram/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HiroakiMikami%2Fmlprogram/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HiroakiMikami%2Fmlprogram/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HiroakiMikami","download_url":"https://codeload.github.com/HiroakiMikami/mlprogram/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224499934,"owners_count":17321602,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-neural-network","deeplearning","natural-language-understanding","nl2code","program-synthesis","pytorch","treegen"],"created_at":"2024-11-13T17:49:46.424Z","updated_at":"2024-11-13T17:49:46.975Z","avatar_url":"https://github.com/HiroakiMikami.png","language":"Python","readme":"mlprogram\n===\n\nA Library of Deep Learning (Machine Learning) for Programming Tasks.\nIt provides a toolbox for implementing and evaluating deep learning methods related to programming.\n\nPurpose\n---\n\nThe main purpose of this repository is making my experiments easy. Recently, many papers proposed deep learning methods for programming, such as programming by example and auto reparing. But many of them requires complex and error-prone implementations. For example, beam search decoding with a programming language grammar is complex and unique to this task. I want to create and maintain well-tested implementations of such algorithms.\n\n### Focuses\n\n* Library for handling programming languages in deep learning tasks\n* Utilities for benchmark datasets of various tasks\n* Simple baseline solution for program generation\n\nNow I do not place value on re-implementing exsiting papers.\nThe machine learning for programming field is still immature. There are no de-fact benchmark tasks in this field (such as image classification w/ ImageNet and object detection w/ COCO in the image). Also, there are no de-fact model (such as ResNet in the image). \n\n\nFeature Lists and Plans\n---\n\n* Benchmark dataset\n    * Auto Reparing\n        * DeepFix: [the official repository](https://bitbucket.org/iiscseal/deepfix/src/master/)\n    * Program Synthesis from Natural Language\n        * Hearthstone: [Latent Predictor Networks for Code Generation](https://arxiv.org/abs/1603.06744)\n        * Django: [Learning to Generate Pseudo-code from Source Code Using Statistical Machine Translation, ACE2-15](https://ieeexplore.ieee.org/document/7372045)\n        * NL2Bash: [nl2bash](https://github.com/TellinaTool/nl2bash)\n        * (TODO) Spider: [Spider 1.0 Yale Semantic Parsing and Text-to-SQL Challenge](https://yale-lily.github.io/spider)\n    * Programming by Examples\n        * 2D CSG\n        * (TODO) DeepCoder\n        * (TODO) ShapeNet\n* Deep Learning Models\n    * Attention Based LSTM\n    * AST LSTM (based on NL2Code)\n* ProgramSynthesis Methods\n    * supervised training\n    * reinforcement learning for programming by example\n    * (TODO) interpreter arppoximated by DNN\n* Other Papers\n    * [NL2Code](https://arxiv.org/abs/1704.01696): [the official repository](https://github.com/pcyin/NL2code/)\n    * [TreeGen](https://arxiv.org/abs/1911.09983): [the official repository](https://github.com/zysszy/TreeGen)\n    * [PbE with REPL](http://arxiv.org/abs/1906.04604): [the official repository](https://github.com/flxsosa/ProgramSearch)\n\n\nBenchmark\n---\n\n### NL2Prog (Hearthstone)\n\n|Method|#params [MiB]|training time [min]|max time per example [sec]|BLEU@top1|config name|\n:-----|-------------:|------------------:|-------------------------:|--------:|:----------|\n|tree LSTM|       7.7|                 92|                        15|  0.75020|`hearthstone/baseline_evaluate_short`|\n|tree LSTM|       7.7|                 92|                       180|  0.76540|`hearthstone/baseline_evaluate_long`|\n\n\n### Programming by Example without Inputs (CSG)\n\n|Method                          |#params [MiB]|training time [min]|max time per example [sec]|generation rate|config file|\n|:-------------------------------|------------:|------------------:|-------------------------:|---------------:|:----------|\n|tree LSTM                       |16           |75                 |30                        |18/30|`csg/baseline_evaluate_short`|\n|tree LSTM                       |16           |75                 |360                       |22/30|`csg/baseline_evaluate_long`|\n|tree LSTM + REINFORCESynthesizer|16           |75                 |30                        |18/30|`csg/baseline_evaluate_rl_synthesizer_short`|\n|tree LSTM + REINFORCESynthesizer|16           |75                 |360                       |22/30|`csg/baseline_evaluate_rl_synthesizer_short`|\n\n\n### Auto Repair\n\nTODO\n\nUsage Examples\n---\n\n`tools/launch.py` is the launcher script and `configs` directory contains the examples.\n\n### Train/Evaluate NL2Code with Hearthstone Dataset\n\nIt requires CUDA enabled GPU.\n\n```bash\n$ python tools/launch.py --config configs/nl2code/nl2code_train.py\n$ python tools/launch.py --config configs/nl2code/nl2code_evaluate.py\n```\n\n\nWarning\n---\n\n* The implementation is highly experimental, and I may change it significantly.\n* The reproduced algorithms may be different from the authors' implementations. For example, the original implementation of NL2Code uses the grammar of Python 2.7.x while this repository uses the grammar of running Python version.\n\n\n\n\u003c!--\nImplemented Papers\n---\n\n### NL2Code\n\n[NL2Code](https://arxiv.org/abs/1704.01696) creates programs (ASTs) from natural language descriptions. It encodes the structure of ASTs.\nThere is no big known issue.\n\n#### 1. version of Python\nThe original implementation used Python 2.x, but this repository uses Python 3.x (I tested with the Python 3.7.4).\n\n#### 2. definitions of action sequence\nThe original implementation added the cast action and did not support actions that have variadic children. This implementation omits the cast action and supports actions with variadic children.\n\n#### 3. implementation of Dropout for LSTM\nDropout used in this repository and the original one is different. I tested dropout of the original repository, but it caused significant performance degradation (about x2 slower). So I decided to use more simple dropout implementations.\n\n#### 4. maximum length of query and action sequences\nThe original implementation limits the length of the query and action sequence because Theano employes Define-and-Run style.\nThis implementation does not set the maximum length because of PyTorch Define-by-Run style.\n\n\n### TreeGen\n\n[TreeGen](https://arxiv.org/abs/1911.09983) creates programs (ASTs) from natural language descriptions. It uses the Transformer as a DNN model.\nThere are the two known issues:\n\n* The loss sometimes becomes NaN in the final phase of the training\n* The result when using Hearthstone dataset is worse than the reported value.\n\n\n### Write, Execute, Assess: Program Synthesis with a REPL\n\n[PbE with REPL](http://arxiv.org/abs/1906.04604) creates programs (ASTs) from input/output examples. It uses reinforcement learning for better generalization.\nThere are many known issues in the implementation. The most crucial one is that RL training is very unstable. I cannot improve model performance by using RL.\n--\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhiroakimikami%2Fmlprogram","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhiroakimikami%2Fmlprogram","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhiroakimikami%2Fmlprogram/lists"}