{"id":17164812,"url":"https://github.com/tcapelle/moving_mnist","last_synced_at":"2025-10-15T15:32:09.924Z","repository":{"id":40053076,"uuid":"278576153","full_name":"tcapelle/moving_mnist","owner":"tcapelle","description":"Exploring Moving Mnist dataset with forecasting algorithms","archived":false,"fork":false,"pushed_at":"2023-04-12T05:47:06.000Z","size":15702,"stargazers_count":31,"open_issues_count":4,"forks_count":4,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-04-13T13:55:51.984Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://tcapelle.github.io/moving_mnist/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tcapelle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-07-10T08:07:13.000Z","updated_at":"2024-03-27T12:44:34.000Z","dependencies_parsed_at":"2025-04-13T13:43:29.899Z","dependency_job_id":null,"html_url":"https://github.com/tcapelle/moving_mnist","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":"fastai/nbdev_template","purl":"pkg:github/tcapelle/moving_mnist","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcapelle%2Fmoving_mnist","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcapelle%2Fmoving_mnist/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcapelle%2Fmoving_mnist/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcapelle%2Fmoving_mnist/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tcapelle","download_url":"https://codeload.github.com/tcapelle/moving_mnist/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcapelle%2Fmoving_mnist/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279087276,"owners_count":26100357,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-15T02:00:07.814Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T22:52:50.672Z","updated_at":"2025-10-15T15:32:09.903Z","avatar_url":"https://github.com/tcapelle.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Moving MNIST forecasting\n\u003e A little experiment using Convolutional RNNs to forecast moving MNIST digits.\n\n\n```python\nfrom fastai.vision.all import *\nfrom moving_mnist.models.conv_rnn import *\nfrom moving_mnist.data import *\n```\n\n```python\nif torch.cuda.is_available():\n    torch.cuda.set_device(0)\n    print(torch.cuda.get_device_name())\n```\n\n    Quadro RTX 8000\n\n\n## Install\n\nIt only uses fastai (version 2) as dependency. Check how to install at https://github.com/fastai/fastai2\n\n## Example:\n\nWe wil predict:\n- `n_in`: 5 images\n- `n_out`: 5 images  \n- `n_obj`: 3 objects\n\n```python\nds = MovingMNIST(DATA_PATH, n_in=5, n_out=5, n_obj=[1,2,3])\n```\n\n```python\ntrain_tl = TfmdLists(range(500), ImageTupleTransform(ds))\nvalid_tl = TfmdLists(range(100), ImageTupleTransform(ds))\n```\n\n```python\ndls = DataLoaders.from_dsets(train_tl, valid_tl, bs=8,\n                             after_batch=[Normalize.from_stats(*mnist_stats)]).cuda()\n```\n\nLeft: Input, Right: Target\n\n```python\ndls.show_batch()\n```\n\n\n![png](docs/images/output_10_0.png)\n\n\n`StackUnstack` takes cares of stacking the list of images into a fat tensor, and unstacking them at the end, we will need to modify our loss function to take a list of tensors as input and target.\n\n```python\nmodel = StackUnstack(SimpleModel())\n```\n\nAs the `ImageSeq` is a `tuple` of images, we will need to stack them to compute loss.\n\n```python\nloss_func = StackLoss(MSELossFlat())\n```\n\n```python\nlearn = Learner(dls, model, loss_func=loss_func, cbs=[])\n```\n\n```python\nlearn.lr_find()\n```\n\n\n\n\n\n\n\n\n    SuggestedLRs(lr_min=0.005754399299621582, lr_steep=3.0199516913853586e-05)\n\n\n\n\n![png](docs/images/output_16_2.png)\n\n\n```python\nlearn.fit_one_cycle(4, 1e-4)\n```\n\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: left;\"\u003e\n      \u003cth\u003eepoch\u003c/th\u003e\n      \u003cth\u003etrain_loss\u003c/th\u003e\n      \u003cth\u003evalid_loss\u003c/th\u003e\n      \u003cth\u003etime\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003e0.915238\u003c/td\u003e\n      \u003ctd\u003e0.619522\u003c/td\u003e\n      \u003ctd\u003e00:12\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n      \u003ctd\u003e0.669368\u003c/td\u003e\n      \u003ctd\u003e0.608123\u003c/td\u003e\n      \u003ctd\u003e00:12\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e2\u003c/td\u003e\n      \u003ctd\u003e0.570026\u003c/td\u003e\n      \u003ctd\u003e0.559723\u003c/td\u003e\n      \u003ctd\u003e00:12\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e3\u003c/td\u003e\n      \u003ctd\u003e0.528593\u003c/td\u003e\n      \u003ctd\u003e0.532774\u003c/td\u003e\n      \u003ctd\u003e00:12\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n\n```python\np,t = learn.get_preds()\n```\n\n\n\n\n\nAs you can see, the results is a list of 5 tensors with 100 samples each.\n\n```python\nlen(p), p[0].shape\n```\n\n\n\n\n    (5, torch.Size([100, 1, 64, 64]))\n\n\n\n```python\ndef show_res(t, idx):\n    im_seq = ImageSeq.create([t[i][idx] for i in range(5)])\n    im_seq.show(figsize=(8,4));\n```\n\n```python\nk = random.randint(0,100)\nshow_res(t,k)\nshow_res(p,k)\n```\n\n\n![png](docs/images/output_22_0.png)\n\n\n\n![png](docs/images/output_22_1.png)\n\n\n## Training Example:\n- ConvGRU with attention and blur upsampling: [01_train_example.ipynb](01_train_example.ipynb)\n- ConvGRU trained with Cross Entropy instead of MSE: [02_train_cross_entropy.ipynb](02_train_cross_entropy.ipynb)\n- Seq2seq model trianed with MSE [03_trainseq2seq.ipynb](03_trainseq2seq.ipynb)\n- PhyDNet ported to fastai[04_train_phydnet.ipynb](04_train_phydnet.ipynb)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftcapelle%2Fmoving_mnist","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftcapelle%2Fmoving_mnist","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftcapelle%2Fmoving_mnist/lists"}