{"id":14959568,"url":"https://github.com/ahmedbesbes/cartoonify","last_synced_at":"2025-04-05T20:06:00.160Z","repository":{"id":45760879,"uuid":"312701821","full_name":"ahmedbesbes/cartoonify","owner":"ahmedbesbes","description":"Deploy and scale serverless machine learning app - in 4 steps.","archived":false,"fork":false,"pushed_at":"2023-03-30T16:42:38.000Z","size":30844,"stargazers_count":409,"open_issues_count":4,"forks_count":84,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-03-29T19:03:04.216Z","etag":null,"topics":["aws","aws-lambda","cartoongan","cartoonify","computer-vision","deployment","gan","machine-learning-production","netlify","netlify-deployment","production","pytorch","react","scalable-applications","serverless","serverless-framework","streamlit","style-transfer"],"latest_commit_sha":null,"homepage":"https://cartoonify.netlify.app/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ahmedbesbes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-13T22:56:19.000Z","updated_at":"2025-03-16T03:25:24.000Z","dependencies_parsed_at":"2024-11-12T17:01:43.887Z","dependency_job_id":"3fa72de9-0653-4e5d-ae98-19cd1da75f81","html_url":"https://github.com/ahmedbesbes/cartoonify","commit_stats":{"total_commits":32,"total_committers":2,"mean_commits":16.0,"dds":0.03125,"last_synced_commit":"a37167c5cda7a56362395271d30aecd5098bbbcd"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fcartoonify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fcartoonify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fcartoonify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fcartoonify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ahmedbesbes","download_url":"https://codeload.github.com/ahmedbesbes/cartoonify/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393568,"owners_count":20931812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-lambda","cartoongan","cartoonify","computer-vision","deployment","gan","machine-learning-production","netlify","netlify-deployment","production","pytorch","react","scalable-applications","serverless","serverless-framework","streamlit","style-transfer"],"created_at":"2024-09-24T13:20:01.506Z","updated_at":"2025-04-05T20:06:00.140Z","avatar_url":"https://github.com/ahmedbesbes.png","language":"Jupyter Notebook","funding_links":["https://www.buymeacoffee.com/ahmedbesbes"],"categories":[],"sub_categories":[],"readme":"# Build and Deploy Cartoonify: a Serverless Machine Learning App\n\n\u003ca href=\"https://www.buymeacoffee.com/ahmedbesbes\" target=\"_blank\"\u003e\u003cimg src=\"https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png\" alt=\"Buy Me A Coffee\" style=\"height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;\" \u003e\u003c/a\u003e\n\nThis repo contains all the code needed to run, build and deploy Cartoonify: a toy app I made from scratch to turn your pictures into cartoons.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./images/demo_cartoonify.gif\" width=\"40%\"\u003e\n\u003c/p\u003e\n\n**Here's what motivated me in starting this project:**\n\n- Give GANs a try. I've been fascinated by these models lately. Trying the CartoonGAN model to turn your face into a cartoon seemed like real fun\n\n- Learn about deploying an application on a serverless architecture using different services of AWS (Lambda, API Gateway, S3, etc.)\n\n- Practice my React skills. I was so damn bored of Plotly, Dash and Streamlit. I wanted, for once, to build something custom and less mainstream\n\n- Use Netlify to deploy this React app. I saw demos of how easy this process was and I wanted to try it to convince myself\n\nIf you're interested in this project, here's a short introduction 🎥\n\n[![](https://res.cloudinary.com/marcomontalbano/image/upload/v1605735276/video_to_markdown/images/youtube--U3UjaRVRtWQ-c05b58ac6eb4c4700831b2b3070cd403.jpg)](https://youtu.be/U3UjaRVRtWQ)\n\n# 0. Some prerequisites to build and deploy Cartoonify 🛠\n\nIf you want to run and deploy Cartoonify, here are some prerequisites first:\n\n- An AWS account (don't worry, deploying this app will cost you almost **nothing**)\n- A free account on Netlify\n- Docker installed on your machine\n- node and npm (preferably the latest versions) installed on your machine\n- torch and torchvision to test CartoonGAN locally (optional)\n\nAll set? you're now ready to go.\n\n### Testing cartoonGAN on Google colab\n\ncheck out `cartoongan/notebooks/standalone_cartoonify.ipynb`\n\nor online on [Colab](https://colab.research.google.com/drive/1Lq8mACJKP8TBxY-TJcsdyo_qul-OVdYc#scrollTo=ug2o6M2IqyAu)\n\nPlease follow these four steps:\n\n# 1. Test CartoonGAN locally\n\nSome parts of the CartoonGan code as well as the pretrained models are borrowed from this [repo](https://github.com/Yijunmaverick/CartoonGAN-Test-Pytorch-Torch). A shout out to them for the great work!\n\nThis is more of an exploratory step where you get to play with the pretrained models and try them (**so inference only**) on some sample images.\n\nIf you're interested in the training procedure, have a look at the CartoonGAN [paper](https://openaccess.thecvf.com/content_cvpr_2018/papers/Chen_CartoonGAN_Generative_Adversarial_CVPR_2018_paper.pdf)\n\n- Download the four pretrained models first. These weights will be loaded inside the Generator model defined in **`cartoongan/network/Transformer.py`**\n\n```bash\ncd cartoongan\nbash download_pth.sh\n```\n\n- To test one of the four models, head over the notebook **`cartoongan/notebooks/CartoonGAN.ipynb`** and change the input image path to your test image.\n  This notebook calls **`cartoongan/test_from_code.py`** script to make the transformation.\n\n```bash\ncd cartoongan/notebooks\njupyter notebook\n```\n\n![](./images/demo_cartoongan.png)\n\n_You can watch this section on Youtube to learn more about GANs and the CartoonGAN model_\n\n\u003cp align=\"center\"\u003e\n\n[![](https://res.cloudinary.com/marcomontalbano/image/upload/v1605445418/video_to_markdown/images/youtube--R86zP6Hf4Hk-c05b58ac6eb4c4700831b2b3070cd403.jpg)](https://youtu.be/R86zP6Hf4Hk)\n\n\u003c/p\u003e\n\n# 2. Deploy CartoonGAN on a serverless API using AWS Lambda\n\nThe goal of this section is to deploy the CartoonGAN model on a serverless architecture so that it can be requested through an API endpoint ... from the internet :computer:\n\n## Why does a serverless architecture matter?\n\nIn a serverless architecture using Lambda functions, for example, you don't have to provision servers yourself. Roughly speaking, you only write the code that'll be execuded and list its dependencies and AWS will manage the servers for you automatically and take care of the infrastructure.\n\nThis has a lot of benefits:\n\n1. Cost efficiency: you don't have to pay for a serverless architecture when you don't use it. On the opposite, when you have an EC2 machine running and not processing any request, you still pay for it.\n\n2. Scalability: if a serverless application starts having a lot of requests at the same time, AWS will scale it by allocating more power to manage the load. If you had the manage the load by yourself using EC2 instances, you would do this by manually allocating more machines and creating a load balancer.\n\nOf course, Serverless architectures cannot be a perfect fit for any use-case. In some situations, they are not practical at all (need for real-time or quick responses, use of WebSocket, heavy processing, etc.).\n\nSince I frequently build machine learning models and integrate them into web applications, I found that a serverless architecture was interesting in these specific use-cases. Of course, here the models are used in **inference only** :warning:\n\n## Cartoonify workflow\n\nHere's the architecture of the app:\n\n\u003cp align=\"center\"\u003e\n \u003cimg src=\"./images/infrastructure.png\" width=\"75%\" \u003e\n\u003c/p\u003e\n\n- On the right side, we have a frontend interface in React and on the left side, we have a backend deployed on a serverless AWS architecture.\n- The backend and the frontend communicate with each other over HTTP requests. Here is the workflow:\n\n  - An image is sent from the client through a POST request\n  - The image is then received via API Gateway\n  - API Gateway triggers a Lambda function to execute and passes the image to it\n  - The Lambda function starts running: it first fetches the pretrained models from S3 and then applies the style transformation on it\n  - Once the Lambda function is done running, it sends the transformed image back to the client through API Gateway.\n\n## Deploy using the Serverless framework\n\nWe are going to define and deploy this architecture by writing it as a Yaml file using the [Serverless](https://www.serverless.com/) framework: an open-source tool to automate deployment to AWS, Azure, Google Cloud, etc.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./images/serverless.png\" width=\"75%\"\u003e\n\u003c/p\u003e\n\nHere are the steps to follow:\n\n1.  Install the serverless framework on your machine\n\n```bash\nnpm install -g serverless\n```\n\n2.  Create an IAM user on AWS with administrator access and name it **cartoonify**.\n    Then configure serverless with this user's credentials:\n\n```bash\nserverless config credentials --provider aws \\\n                              --key \u003cACCESS_KEY\u003e \\\n                              --secret \u003cSECRET_KEY\u003e \\\n                              --profile cartoonify\n```\n\n3.  bootstrap a serverless project with a python template at the root of this project\n\n```bash\nserverless create --template aws-python --path backend\n```\n\nFrom now on, you can either follow the steps from 4 to 10 to understand what happens, or run the code you just cloned to deploy the app.\n\n_If you're in hurry, just run these two commands:_\n\n```bash\ncd backend/\nnpm install\nsls deploy\n```\n\n4.  install two Serverless plugins:\n\n```bash\nsls plugin install -n serverless-python-requirements\nnpm install --save-dev serverless-plugin-warmup\n```\n\n5.  Create a folder called `network` inside `backend` and put the following two files in it:\n\n    - Transformer.py: a script that holds the architecture of the generator model.\n    - A blank \\_\\_init\\_\\_.py\n\n6.  Modify the serverless.yml file with the following sections:\n\n```yaml\n# The provider section where we setup the provider, the runtime and the permissions:\n\nprovider:\n  name: aws\n  runtime: python3.7\n  profile: cartoonify\n  region: eu-west-3\n  timeout: 60\n  iamRoleStatements:\n      - Effect: Allow\n      Action:\n          - s3:getObject\n      Resource: arn:aws:s3:::cartoongan/models/*\n      - Effect: Allow\n      Action:\n          - \"lambda:InvokeFunction\"\n      Resource: \"*\"\n```\n\n---\n\n```yaml\n# The custom section where we configure the plugins:\ncustom:\n  pythonRequirements:\n  dockerizePip: true\n  zip: true\n  slim: true\n  strip: false\n  noDeploy:\n    - docutils\n    - jmespath\n    - pip\n    - python-dateutil\n    - setuptools\n    - six\n    - tensorboard\n  useStaticCache: true\n  useDownloadCache: true\n  cacheLocation: \"./cache\"\n  warmup:\n  events:\n    - schedule: \"rate(5 minutes)\"\n  timeout: 50\n```\n\n---\n\n```yaml\n# The package section where we exclude folders from production\npackage:\n  individually: false\n  exclude:\n    - package.json\n    - package-log.json\n    - node_modules/**\n    - cache/**\n    - test/**\n    - __pycache__/**\n    - .pytest_cache/**\n    - model/pytorch_model.bin\n    - raw/**\n    - .vscode/**\n    - .ipynb_checkpoints/**\n```\n\n---\n\n```yaml\n# The functions section where we create the Lambda function and define the events that invoke it:\nfunctions:\n  transformImage:\n    handler: src/handler.lambda_handler\n    memorySize: 3008\n    timeout: 300\n    events:\n      - http:\n          path: transform\n          method: post\n          cors: true\n    warmup: true\n```\n\n---\n\n```yaml\n# and finally the plugins section:\nplugins:\n  - serverless-python-requirements\n  - serverless-plugin-warmup\n```\n\n---\n\n7. List the dependencies inside requirements.txt\n\n```bash\nhttps://download.pytorch.org/whl/cpu/torch-1.1.0-cp37-cp37m-linux_x86_64.whl\nhttps://download.pytorch.org/whl/cpu/torchvision-0.3.0-cp37-cp37m-linux_x86_64.whl\nPillow==6.2.1\n```\n\n8. Create an `src` folder inside `backend` and put handler.py in it to define the lambda function. Then modify handler.py\n\n```python\n# Define imports\ntry:\n    import unzip_requirements\nexcept ImportError:\n    pass\n\nimport json\nfrom io import BytesIO\nimport time\nimport os\nimport base64\n\nimport boto3\nimport numpy as np\nfrom PIL import Image\n\nimport torch\nimport torchvision.transforms as transforms\nfrom torch.autograd import Variable\nimport torchvision.utils as vutils\nfrom network.Transformer import Transformer\n```\n\n---\n\n```python\n# Define two functions inside handler.py: img_to_base64_str to\n# convert binary images to base64 format and load_models to\n# load the four pretrained model inside a dictionary and then\n# keep them in memory\n\ndef img_to_base64_str(img):\n    buffered = BytesIO()\n    img.save(buffered, format=\"PNG\")\n    buffered.seek(0)\n    img_byte = buffered.getvalue()\n    img_str = \"data:image/png;base64,\" + base64.b64encode(img_byte).decode()\n    return img_str\n\n\ndef load_models(s3, bucket):\n    styles = [\"Hosoda\", \"Hayao\", \"Shinkai\", \"Paprika\"]\n    models = {}\n\n    for style in styles:\n        model = Transformer()\n        response = s3.get_object(\n            Bucket=bucket, Key=f\"models/{style}_net_G_float.pth\")\n        state = torch.load(BytesIO(response[\"Body\"].read()))\n        model.load_state_dict(state)\n        model.eval()\n        models[style] = model\n\n    return models\n```\n\n---\n\n```python\ndef lambda_handler(event, context):\n  \"\"\"\n  lambda handler to execute the image transformation\n  \"\"\"\n  # warming up the lambda\n  if event.get(\"source\") in [\"aws.events\", \"serverless-plugin-warmup\"]:\n      print('Lambda is warm!')\n      return {}\n\n  # extracting the image form the payload and converting it to PIL format\n  data = json.loads(event[\"body\"])\n  print(\"data keys :\", data.keys())\n  image = data[\"image\"]\n  image = image[image.find(\",\")+1:]\n  dec = base64.b64decode(image + \"===\")\n  image = Image.open(BytesIO(dec))\n  image = image.convert(\"RGB\")\n\n  # loading the model with the selected style based on the model_id payload\n  model_id = int(data[\"model_id\"])\n  style = mapping_id_to_style[model_id]\n  model = models[style]\n\n  # resize the image based on the load_size payload\n  load_size = int(data[\"load_size\"])\n\n  h = image.size[0]\n  w = image.size[1]\n  ratio = h * 1.0 / w\n  if ratio \u003e 1:\n      h = load_size\n      w = int(h*1.0 / ratio)\n  else:\n      w = load_size\n      h = int(w * ratio)\n\n  image = image.resize((h, w), Image.BICUBIC)\n  image = np.asarray(image)\n\n  # convert PIL image from  RGB to BGR\n  image = image[:, :, [2, 1, 0]]\n  image = transforms.ToTensor()(image).unsqueeze(0)\n\n  # transform values to (-1, 1)\n  image = -1 + 2 * image\n  if gpu \u003e -1:\n      image = Variable(image, volatile=True).cuda()\n  else:\n      image = image.float()\n\n  # style transformation\n  with torch.no_grad():\n      output_image = model(image)\n      output_image = output_image[0]\n\n  # convert PIL image from BGR back to RGB\n  output_image = output_image[[2, 1, 0], :, :]\n\n  # transform values back to (0, 1)\n  output_image = output_image.data.cpu().float() * 0.5 + 0.5\n\n  # convert the transformed tensor to a PIL image\n  output_image = output_image.numpy()\n  output_image = np.uint8(output_image.transpose(1, 2, 0) * 255)\n  output_image = Image.fromarray(output_image)\n\n  # convert the PIL image to base64\n  result = {\n      \"output\": img_to_base64_str(output_image)\n  }\n\n  # send the result back to the client inside the body field\n  return {\n      \"statusCode\": 200,\n      \"body\": json.dumps(result),\n      \"headers\": {\n          'Content-Type': 'application/json',\n          'Access-Control-Allow-Origin': '*'\n      }\n  }\n```\n\n9. Start docker\n\n10. Deploy :rocket:\n\n    ```bash\n    cd backend/\n    sls deploy\n    ```\n\nDeployment may take up to 5 - 8 minutes, so go grab a :coffee:.\n\nOnce the lambda function deployed, **you'll be prompted a URL of the API**. Go to jupyter notebook to test it:\n\n![](./images/demo_api.png)\n\n_You can watch this section on Youtube to get every detail of it._\n\n[![](https://res.cloudinary.com/marcomontalbano/image/upload/v1605734471/video_to_markdown/images/youtube--palz3TLB6TE-c05b58ac6eb4c4700831b2b3070cd403.jpg)](https://youtu.be/palz3TLB6TE)\n\n# 3. Build a React interface\n\n- Before running the React app and building it, you'll have to specify the API url of the model you just deployed.\n  Go inside `fontend/src/api.js` and change the value of **baseUrl**\n\n- To run the React app locally:\n\n```bash\ncd frontend/\nyarn install\nyarn start\n```\n\nThis will start it at: http://localhost:3000\n\n- To build the app before deploying it to Netlify\n\n```bash\nyarn build\n```\n\nThis will create a `build/` folder that contains a build of the application to be served on Netlify.\n\n_You can watch this section on Youtube to understand how the code is structured._\n\n[![](https://res.cloudinary.com/marcomontalbano/image/upload/v1605870233/video_to_markdown/images/youtube--APiwsholTMQ-c05b58ac6eb4c4700831b2b3070cd403.jpg)](https://youtu.be/APiwsholTMQ)\n\n# 4. Deploy the React app on Netlify\n\n- To be able to deploy on Netlify you'll need an account. It's free, head over this [link](https://app.netlify.com/) to sign up.\n\n- Then you'll need to install **netlify-cli**\n\n```bash\nnpm install netlify-cli -g\n```\n\n- Authenticate the Netlify client with your account\n\n```bash\nnetlify login\n```\n\n- Deploy :rocket:\n\n```bash\ncd app/\nnetlify deploy\n```\n\n_You can watch this section on Youtube to understand how easy the deployment on Netlify can be._\n\n[![](https://res.cloudinary.com/marcomontalbano/image/upload/v1605870264/video_to_markdown/images/youtube--kk_5BAVQVfM-c05b58ac6eb4c4700831b2b3070cd403.jpg)](https://youtu.be/kk_5BAVQVfM)\n\n# 5. Want to contribute ? :grin:\n\nIf you've made this far, I sincerely thank you for your time!\n\nIf you liked this project and want to improve it, be my guest: I'm open to pull requests.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedbesbes%2Fcartoonify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fahmedbesbes%2Fcartoonify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedbesbes%2Fcartoonify/lists"}