{"id":28451861,"url":"https://github.com/hoya012/fast-style-transfer-tutorial-pytorch","last_synced_at":"2025-07-02T03:31:17.893Z","repository":{"id":106548282,"uuid":"173872580","full_name":"hoya012/fast-style-transfer-tutorial-pytorch","owner":"hoya012","description":"Simple Tutorials \u0026 Code Implementation of fast-style-transfer(Perceptual Losses for Real-Time Style Transfer and Super-Resolution, 2016 ECCV) using PyTorch.","archived":false,"fork":false,"pushed_at":"2019-03-05T05:04:57.000Z","size":19627,"stargazers_count":20,"open_issues_count":0,"forks_count":12,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-06T17:09:17.239Z","etag":null,"topics":["deep-learning","deep-style-transfer","fast-style-transfer","machine-learning","pytorch","pytorch-tutorial","style-transfer"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hoya012.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-03-05T04:21:30.000Z","updated_at":"2025-04-02T13:15:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"f7766f9c-15b0-453c-a7ca-df5ff97f4915","html_url":"https://github.com/hoya012/fast-style-transfer-tutorial-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hoya012/fast-style-transfer-tutorial-pytorch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoya012%2Ffast-style-transfer-tutorial-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoya012%2Ffast-style-transfer-tutorial-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoya012%2Ffast-style-transfer-tutorial-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoya012%2Ffast-style-transfer-tutorial-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hoya012","download_url":"https://codeload.github.com/hoya012/fast-style-transfer-tutorial-pytorch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoya012%2Ffast-style-transfer-tutorial-pytorch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263069135,"owners_count":23408901,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-style-transfer","fast-style-transfer","machine-learning","pytorch","pytorch-tutorial","style-transfer"],"created_at":"2025-06-06T17:09:17.562Z","updated_at":"2025-07-02T03:31:17.857Z","avatar_url":"https://github.com/hoya012.png","language":"Jupyter Notebook","funding_links":[],"categories":["Datasets"],"sub_categories":["GANs, augmentation, etc."],"readme":"# fast-style-transfer-tutorial-pytorch\nSimple Tutorials \u0026amp; Code Implementation of fast-style-transfer(Perceptual Losses for Real-Time Style Transfer and Super-Resolution, 2016 ECCV) using PyTorch. This code is based on [pytorch example codes](https://github.com/pytorch/examples/tree/master/fast_neural_style)\n\n### Style Image from Battle Ground Game\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"700\" src=\"https://github.com/hoya012/hoya012.github.io/blob/master/assets/img/fast_style_transfer/3.PNG\"\u003e\n\u003c/p\u003e\n\n### Style Transfer Demo video (Left: original / Right: output)\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"700\" src=\"https://github.com/hoya012/hoya012.github.io/blob/master/assets/img/fast_style_transfer/mirama_demo.gif\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"700\" src=\"https://github.com/hoya012/hoya012.github.io/blob/master/assets/img/fast_style_transfer/sanok_demo.gif\"\u003e\n\u003c/p\u003e\n\nFor simplicity, i write codes in `ipynb`. So, you can easliy test my code.\n\n*Last update : 2019/03/05*\n\n## Contributor\n* hoya012\n\n## 0. Requirements\n```python\npython=3.5\nnumpy\nmatplotlib\ntorch=1.0.0\ntorchvision\ntorchsummary\nopencv-python\n```\n\nIf you use google colab, you don't need to set up. Just run and run!! \n\n##\n\n## 1. Usage\nYou only run `Fast-Style-Transfer-PyTorch.ipynb`. \n\nOr you can use Google Colab for free!! This is [colab link](https://colab.research.google).\n\nAfter downloading ipynb, just upload to your google drive. and run!\n\n##\n\n## 2. Tutorial \u0026 Code implementation Blog Posting (Korean Only)\n[“Fast Style Transfer PyTorch Tutorial”](https://hoya012.github.io/blog/Fast-Style-Transfer-Tutorial/)  \n\n##\n\n## 3. Dataset download \nFor simplicty, i use **COCO 2017 validation set** instead of **COCO 2014 training set**.\n\n- COCO 2014 training: about 80000 images / 13GB\n- COCO 2017 validation: about 5000 images / 1GB –\u003e i will use training epoch multiplied by 16 times\n\nYou can download COCO 2017 validation dataset in [this link](http://images.cocodataset.org/zips/val2017.zip)\n\n##\n\n## 4. Link to google drive and upload files to google drive\nIf you use colab, you can simply link ipynb to google drive.\n\n```python\nfrom google.colab import drive\ndrive.mount(\"/content/gdrive\")\n```\n\nUpload COCO dataset \u0026 Style Image \u0026 Test Image or Videos to Your Google Drive.\n\nYou can use google drive location in ipynb like this codes.\n\n```python\nstyle_image_location = \"/content/gdrive/My Drive/Colab_Notebooks/data/vikendi.jpg\"\n\nstyle_image_sample = Image.open(style_image_location, 'r')\ndisplay(style_image_sample)\n```\n\n##\n\n## 5. Transfer learning, inference from checkpoint.\nSince google colab only uses the GPU for 8 hours, we need to restart it from where it stopped.\n\nTo do this, the model can be saved as a checkpoint during training, and then the learning can be done.\nAlso, you can also use trained checkpoints for inferencing.\n\n```python\ntransfer_learning = False # inference or training first --\u003e False / Transfer learning --\u003e True\nckpt_model_path = os.path.join(checkpoint_dir, \"ckpt_epoch_63_batch_id_500.pth\")\n\nif transfer_learning:\n  checkpoint = torch.load(ckpt_model_path, map_location=device)\n  transformer.load_state_dict(checkpoint['model_state_dict'])\n  transformer.to(device)\n```\n\n##\n\n## 6. Training phase\n\n```python\nif running_option == \"training\":\n  if transfer_learning:\n      transfer_learning_epoch = checkpoint['epoch'] \n  else:\n      transfer_learning_epoch = 0\n\n  for epoch in range(transfer_learning_epoch, num_epochs):\n        transformer.train()\n        agg_content_loss = 0.\n        agg_style_loss = 0.\n        count = 0\n\n        for batch_id, (x, _) in enumerate(train_loader):\n            n_batch = len(x)\n            count += n_batch\n            optimizer.zero_grad()\n\n            x = x.to(device)\n            y = transformer(x)\n\n            y = normalize_batch(y)\n            x = normalize_batch(x)\n\n            features_y = vgg(y)\n            features_x = vgg(x)\n\n            content_loss = content_weight * mse_loss(features_y.relu2_2, features_x.relu2_2)\n\n            style_loss = 0.\n            for ft_y, gm_s in zip(features_y, gram_style):\n                gm_y = gram_matrix(ft_y)\n                style_loss += mse_loss(gm_y, gm_s[:n_batch, :, :])\n            style_loss *= style_weight\n\n            total_loss = content_loss + style_loss\n            total_loss.backward()\n            optimizer.step()\n\n            agg_content_loss += content_loss.item()\n            agg_style_loss += style_loss.item()\n\n            if (batch_id + 1) % log_interval == 0:\n                mesg = \"{}\\tEpoch {}:\\t[{}/{}]\\tcontent: {:.6f}\\tstyle: {:.6f}\\ttotal: {:.6f}\".format(\n                    time.ctime(), epoch + 1, count, len(train_dataset),\n                                  agg_content_loss / (batch_id + 1),\n                                  agg_style_loss / (batch_id + 1),\n                                  (agg_content_loss + agg_style_loss) / (batch_id + 1)\n                )\n                print(mesg)\n\n            if checkpoint_dir is not None and (batch_id + 1) % checkpoint_interval == 0:\n                transformer.eval().cpu()\n                ckpt_model_filename = \"ckpt_epoch_\" + str(epoch) + \"_batch_id_\" + str(batch_id + 1) + \".pth\"\n                print(str(epoch), \"th checkpoint is saved!\")\n                ckpt_model_path = os.path.join(checkpoint_dir, ckpt_model_filename)\n                torch.save({\n                'epoch': epoch,\n                'model_state_dict': transformer.state_dict(),\n                'optimizer_state_dict': optimizer.state_dict(),\n                'loss': total_loss\n                }, ckpt_model_path)\n\n                transformer.to(device).train()  \n```\n\n##\n\n## 7. Test(Inference) Phase\nI use video for demo. But you can use only single image. ( `running_option == \"test\"` )\nThe code below shows how to apply a style transfer with video as input and save the video as output.\n\nIf you download [trained weight](https://github.com/hoya012/fast-style-transfer-tutorial-pytorch/blob/master/ckpt_epoch_63_batch_id_500.pth), you can test without any training!\n\n```python\nif running_option == \"test_video\":\n    \n    with torch.no_grad():\n        style_model = TransformerNet()\n\n        ckpt_model_path = os.path.join(checkpoint_dir, \"ckpt_epoch_63_batch_id_500.pth\")\n        checkpoint = torch.load(ckpt_model_path, map_location=device)\n\n        # remove saved deprecated running_* keys in InstanceNorm from the checkpoint\n        for k in list(checkpoint.keys()):\n            if re.search(r'in\\d+\\.running_(mean|var)$', k):\n                del checkpoint[k]\n\n        style_model.load_state_dict(checkpoint['model_state_dict'])\n        style_model.to(device)\n\n        cap = cv2.VideoCapture(\"/content/gdrive/My Drive/Colab_Notebooks/data/mirama_demo.mp4\")\n\n        frame_cnt = 0\n        \n        fourcc = cv2.VideoWriter_fourcc(*'XVID') #cv2.VideoWriter_fourcc(*'MP42')\n        out = cv2.VideoWriter('/content/gdrive/My Drive/Colab_Notebooks/data/mirama_demo_result.avi', fourcc, 60.0, (1920,1080))\n\n        while(cap.isOpened()):\n            ret, frame = cap.read()\n            \n            try:\n              frame = frame[:,:,::-1] - np.zeros_like(frame)\n            except:\n              break\n              \n            print(frame_cnt, \"th frame is loaded!\")\n\n            content_image = frame\n            content_transform = transforms.Compose([\n                transforms.ToTensor(),\n                transforms.Lambda(lambda x: x.mul(255))\n            ])\n            content_image = content_transform(content_image)\n            content_image = content_image.unsqueeze(0).to(device)\n\n            output = style_model(content_image).cpu()\n            #save_image(\"/content/gdrive/My Drive/Colab_Notebooks/data/vikendi_video_result/\" + str(frame_cnt) +\".png\", output[0])\n            out.write(post_process_image(output[0]))\n            frame_cnt += 1\n            \n        cap.release()\n        out.release()\n        cv2.destroyAllWindows()\n```\n\n##\n\n## Reference\n- \u003ca href=\"https://github.com/pytorch/examples/tree/master/fast_neural_style\" target=\"_blank\"\u003e pytorch example code \u003c/a\u003e\n- \u003ca href=\"https://ezgif.com/video-to-gif\" target=\"_blank\"\u003e avi --\u003e gif for demo \u003c/a\u003e\n- \u003ca href=\"https://colab.research.google.com/\" target=\"_blank\"\u003e google colaboratory \u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoya012%2Ffast-style-transfer-tutorial-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhoya012%2Ffast-style-transfer-tutorial-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoya012%2Ffast-style-transfer-tutorial-pytorch/lists"}