{"id":13485354,"url":"https://github.com/tqchen/tinyflow","last_synced_at":"2025-09-27T10:31:17.264Z","repository":{"id":38361691,"uuid":"67749622","full_name":"tqchen/tinyflow","owner":"tqchen","description":"Tutorial code on how to build your own Deep Learning System in 2k Lines","archived":true,"fork":false,"pushed_at":"2018-10-04T18:34:44.000Z","size":93,"stargazers_count":2002,"open_issues_count":2,"forks_count":371,"subscribers_count":83,"default_branch":"master","last_synced_at":"2024-09-26T23:01:43.193Z","etag":null,"topics":["computational-graphs","deep-learning","nnvm"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tqchen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-09-09T00:04:31.000Z","updated_at":"2024-09-24T08:43:52.000Z","dependencies_parsed_at":"2022-08-25T05:01:29.430Z","dependency_job_id":null,"html_url":"https://github.com/tqchen/tinyflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tqchen%2Ftinyflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tqchen%2Ftinyflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tqchen%2Ftinyflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tqchen%2Ftinyflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tqchen","download_url":"https://codeload.github.com/tqchen/tinyflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234426421,"owners_count":18830900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computational-graphs","deep-learning","nnvm"],"created_at":"2024-07-31T18:00:20.064Z","updated_at":"2025-09-27T10:31:11.992Z","avatar_url":"https://github.com/tqchen.png","language":"C++","readme":"# TinyFlow: Build Your Own DL System in 2K Lines\n\nTinyFlow is  \"example code\" for [NNVM](https://github.com/dmlc/nnvm/).\n\nIt demonstrates how can we build a clean, minimum and powerful computational\ngraph based deep learning system with same API as TensorFlow.\nThe operator code are implemented with [Torch7](https://github.com/torch/torch7) to reduce the effort to write operators while still demonstrating the concepts of the system (and Embedding Lua in C++ is kinda of fun:).\n\nTinyFlow is a real deep learning system that can run on GPU and CPUs.\nTo support the examples, it takes.\n- 927 lines code for operators\n- 734 lines of code for execution runtime\n- 71 lines of code for API glue\n- 233 lines of code for front-end\n\nNote that more code in operators can easily be added to make it as\nfeature complete as most existing deep learning systems.\n\n\n\n## What is it for\nAs explained in the goal of [NNVM](https://github.com/dmlc/nnvm/),\nit is important to make modular and reusable components for to enable us to build\ncustomized learning system easily.\n\n- Course Material for teaching DL system. TinyFlow can be used to teach student the concepts in deep learning systems.\n  - e.g. design homeworks on implementing symbolic differentiation, memory allocation, operator fusion.\n- Experiment bed for learning system researchers. TinyFlow allows easy addition with new system features with\n  the modular design being portable to other system that reuses NNVM.\n- Showcase of intermediate representation usecase. It demonstrates how intermediate representation like NNVM to be able to\n  target multiple front-ends(TF, MXNet) and backends(Torch7, MXNet) with common set of optimizations.\n- Test bed on common reusable modules for DL system. TinyFlow, together with other systems(e.g. MXNet) can be used as testbed on the\n  common reusable modules in deep learning to encourage front-end, optimization module and backends\n  that are shared across frameworks.\n- Just for fun :)\n\n\nWe believe the Unix Philosophy can building learning system more fun and everyone can be able to build\nand understand learning system better.\n\n## Deep Learning System Course\nIf you are interested in learning how to build deep learning system from scratch, checkout [CSE 599G1: Deep Learning System](http://dlsys.cs.washington.edu/) from University of Washington. \n\n## The Design\n- The graph construction API is automatically reused from NNVM\n- We choose Torch7 as the default operator execution backend.\n  - So TinyFlow can also be called \"TorchFlow\" since it is literally TensorFlow on top of Torch:)\n  - This allows us to quickly implement the operators and focus code on the system part.\n- We intentionally choose to avoid using [MXNet](https://github.com/dmlc/mxnet) as front or backend,\n  since MXNet already uses NNVM as intermediate layer, and it would be more fun to try something different.\n\nAlthough it is minimum. TinyFlow still comes with many advanced design concepts in Deep Learning system.\n- Automatic differentiation.\n- Shape/type inference.\n- Static memory allocation for graph for memory efficient training/inference.\n\nThe operator implementation is easy Thanks to Torch7. More fun demonstrations will be added to the project.\n\n## Dependencies\nMost of TinyFlow's code is self-contained.\n- TinyFlow depend on Torch7 for operator supports with minimum code.\n  - We use a lightweight lua bridge code from dmlc-core/dmlc/lua.h\n- NNVM is used for graph representation and optimizations\n\n## Build\n- Install Torch7\n  - For OSX User, please install Torch with Lua 5.1 instead of LuaJIT,\n    i.e. ```TORCH_LUA_VERSION=LUA51 ./install.sh```\n- Set up environment variable ```TORCH_HOME``` to root of torch\n- Type ```make```\n- Setup python path to include tinyflow and nnvm\n```bash\nexport PYTHONPATH=${PYTHONPATH}:/path/to/tinyflow/python:/path/to/tinyflow/nnvm/python\n```\n- Try example program ```python example/mnist_softmax.py```\n\n## Enable Fusion in TinyFlow\n- Build NNVM with Fusion: uncomment fusion plugin part in config.mk, then `make`\n- Build TinyFlow: enable `USE_FUSION` in Makefile, then `make`\n- Try Example program `example/mnist_lenet.py`, change the config of session from `tf.Session(config='gpu')` to `tf.Session(config='gpu fusion')`\n","funding_links":[],"categories":["Uncategorized","C++","Machine Learning Tutorials","Course"],"sub_categories":["Uncategorized","Data Management","Conference"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftqchen%2Ftinyflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftqchen%2Ftinyflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftqchen%2Ftinyflow/lists"}