{"id":15020447,"url":"https://github.com/WeltXing/PyDyNet","last_synced_at":"2025-09-01T22:31:25.321Z","repository":{"id":48150157,"uuid":"489314626","full_name":"WeltXing/PyDyNet","owner":"WeltXing","description":"NumPy实现类PyTorch的动态计算图和神经网络框架(MLP, CNN, RNN, Transformer)","archived":false,"fork":false,"pushed_at":"2025-08-19T14:13:10.000Z","size":4815,"stargazers_count":83,"open_issues_count":0,"forks_count":16,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-19T16:16:28.062Z","etag":null,"topics":["autograd","cnn","cuda","cupy","deep-learning-framework","numpy","python","pytorch","pytorch-implementation","rnn","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WeltXing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-05-06T10:38:06.000Z","updated_at":"2025-08-19T14:13:13.000Z","dependencies_parsed_at":"2024-12-15T07:05:30.033Z","dependency_job_id":"ceea8d52-1dcb-4fb6-836b-6f0e36feca10","html_url":"https://github.com/WeltXing/PyDyNet","commit_stats":{"total_commits":46,"total_committers":2,"mean_commits":23.0,"dds":0.4130434782608695,"last_synced_commit":"7f0865a58595d69e7493191912bc10f8172e9dc2"},"previous_names":["weltxing/pydynet","kaslanarian/pydynet"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/WeltXing/PyDyNet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2FPyDyNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2FPyDyNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2FPyDyNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2FPyDyNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WeltXing","download_url":"https://codeload.github.com/WeltXing/PyDyNet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2FPyDyNet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273202256,"owners_count":25063272,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-01T02:00:09.058Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","cnn","cuda","cupy","deep-learning-framework","numpy","python","pytorch","pytorch-implementation","rnn","transformer"],"created_at":"2024-09-24T19:55:06.020Z","updated_at":"2025-09-01T22:31:25.304Z","avatar_url":"https://github.com/WeltXing.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PyDyNet：NumPy-based Dynamic Deep Learning Framework\n\nChinese README: [cnREADME.md](./cnREADME.md)\n\n[![Downloads](https://pepy.tech/badge/pydynet)](https://pepy.tech/project/pydynet)\n[![Downloads](https://static.pepy.tech/personalized-badge/pydynet?period=month\u0026units=international_system\u0026left_color=grey\u0026right_color=orange\u0026left_text=downloads/month)](https://pepy.tech/project/pydynet)\n![x](https://img.shields.io/pypi/l/pydynet)\n![x](https://img.shields.io/pypi/implementation/numpy)\n![x](https://img.shields.io/github/stars/Kaslanarian/PyDyNet?style=social)\n![x](https://img.shields.io/github/forks/Kaslanarian/PyDyNet?style=social)\n\n## Towards Large Language Model\n\n**In the summer of 2025, I restart the development of PyDyNet after two years.** PyDyNet implemented a pure inference version of Llama3 (6-layer Transformer, vocab-size=32000). The implementation is inspired by the NumPy version and dataset available [here](https://github.com/likejazz/llama3.np). To run it, download the dataset into the `llm/llama` folder and execute:\n\n```bash\n\u003e\u003e\u003e python -m llm.llama.infer\n\nThere was a boy named Timmy. He loved to play with hi toy and run around outside. One day, Timmy' mom asked him to help her with the laundry. Timmy didn't want to help because he wanted to play. But hi mom said, \"Timmy, you need to help me. It' important to help out.\"\nTimmy didn't want to help, but he knew he had to. So, he put on hi shoe and went outside to help hi mom. A they were folding the clothe, Timmy saw a big pile of laundry on the floor. He wanted to help, so he started to pick it up. But then, he accidentally knocked over a pile of clothe and they fell on him. Timmy wa okay, but he felt bad.\nHi mom saw what happened and said, \"Timmy, you need to be more careful. You could have hurt yourself.\" Timmy felt bad and said sorry. Hi mom hugged him and said, \"It' okay, accident happen. Let' clean up the laundry together.\" Timmy learned that it' important to be careful and help out when you need it.\n\nToken count: 262, elapsed: 0.87s, 300 tokens/s\n```\n\nWe also implemented a pure inference version of CLIP, inspired by the NumPy version and dataset available [NPCLIP](https://github.com/99991/NPCLIP). To run it, imigrate `data` folder of `MPCLIP` into `llm/clip` folder and execute: \n\n```bash\n\u003e\u003e\u003e python -m llm.clip.infer\nLabel probs: [0.000953   0.48176003 0.51728696]\n```\n\nfor the following image and query [\"a fish\", \"a dog\", \"a cat\"]\n\n\u003cimg src=\"llm/clip/picture.png\" alt=\"cat_dog\" width=\"400px\" /\u003e\n\n## Overview\n\nPyDyNet is a neural network framework implemented entirely in NumPy (with CuPy support since version 0.0.7, using the same API). Its syntax is inspired by PyTorch, and its structure is as follows:\n\n```mermaid\ngraph LR\n   N(numpy/cupy.ndarray)--Backend--\u003e A(Tensor) --\u003e ds(Dataset) ---\u003e Data(DataLoader)---\u003e Mission\n   A  --Eager execution--\u003e B(Basic operators:\u003cbr\u003e add, exp, etc)\n   B -.Autograd-.-\u003e A\n\n   B --\u003e CO(Complex\u003cbr\u003eoperators)\n   --\u003e f(Function:\u003cbr\u003eimg2col, etc) \n   --\u003e M(Basic Module:\u003cbr\u003eLinear, etc)\n   --\u003e CM(Advanced Module: CNN, RNN, Transformer, etc)\n   --\u003e Mission(Learning task)\n   A --\u003e GD(Optimizer:\u003cbr\u003e SGD, Adam, etc) ---\u003e LS(lr_scheduler: \u003cbr\u003eStepLR, etc)---\u003e Mission\n```\n\nDashed lines indicate that users can disable automatic differentiation using `no_grad`.\n\n## Install\n\nJust\n\n```bash\npip install pydynet\n```\n\nor\n\n```bash\ngit clone https://github.com/Kaslanarian/PyDyNet\ncd PyDyNet\npython setup.py install\n```\n\n## Example\n\nExamples can be found in the [examples/pydynet](./examples/pydynet) directory, with equivalent PyTorch implementations in [examples/pytorch](./examples/pytorch). To run an example, use:\n\n```bash\npython -m examples.pydynet.xxx\n```\n\n### Automatic Differentiation\n\nThe example [autodiff1d.py](examples/pydynet/autodiff1d.py) demonstrates automatic differentiation by performing gradient descent on a one-dimensional convex function:\n\n\u003cimg src=\"imgs/ad1d.png\" alt=\"ad1\" style=\"zoom:67%;\" /\u003e\n\nA multi-variable convex function example is provided in [autodiff2d.py](examples/pydynet/autodiff2d.py):\n\n\u003cimg src=\"imgs/ad2d.png\" alt=\"ad2\" style=\"zoom:67%;\" /\u003e\n\n### MLP \u0026 LeNet\n\nThe example [mlp_cnn.py](examples/pydynet/mnist.py) uses MLP and LeNet to classify MNIST digits. The training and testing accuracies are shown below:\n\n\u003cimg src=\"imgs/mlp_cnn.png\" alt=\"dnn\" style=\"zoom:67%;\" /\u003e\n\n### Dropout \u0026 Batch Normalization\n\nThe example [mlp_dropout_bn.py](examples/pydynet/dropout_bn.py) compares the performance of three networks on the `fetch_olivetti_faces` dataset (64×64 pixel images):\n\n1. Three-layer MLP;\n2. Three-layer MLP with Dropout;\n3. Three-layer MLP with Batch Normalization.\n\n\u003cimg src=\"imgs/dropout_bn.png\" alt=\"cnn\" style=\"zoom:67%;\" /\u003e\n\n### Recurrent Neural Network (RNN)\n\nThe example [ts_prediction.py](examples/pydynet/ts_prediction.py) demonstrates time series prediction using a GRU:\n\n\u003cimg src=\"imgs/rnn.png\" alt=\"RNN\" style=\"zoom:67%;\" /\u003e\n\n### Transformer\n\nThe example [transformer.py](examples/pydynet/transformer.py) shows how to train a text classification model using a Transformer. The training results are as follows:\n\n\u003cimg src=\"imgs/transformer.png\" alt=\"transformer\" style=\"zoom:67%;\" /\u003e\n\n\u003e Dataset (CoLA) link: \u003chttps://nyu-mll.github.io/CoLA/cola_public_1.1.zip\u003e\n\n## Cuda Acceleration\n\nPyDyNet supports CUDA acceleration through CuPy. To use it, simply install CuPy and use the same API as NumPy. We compare the performance of PyDyNet with CuPy and NumPy as follows on **Nvidia GeForce RTX 4090**:\n\n|      Network structure         |      Dataset      | CPU time (s) per epoch | GPU time (s) per epoch |\n| :-----------------: | :---------------: | :--------------------: | :--------------------: |\n|    3-layer MLP     | MNIST (80000×574) |      7.256±0.138      |       1.203±.0181       |\n|        LeNet        | MNIST (80000×574) |     239.664±2.108      |      2.841±0.026      |\n| 1-layer Transformer (dim=512, head=4) | CoLA (8551×45×64) |      17.503±0.251      |      1.075±0.002       |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FWeltXing%2FPyDyNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FWeltXing%2FPyDyNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FWeltXing%2FPyDyNet/lists"}