{"id":19866708,"url":"https://github.com/sorenlind/transferit","last_synced_at":"2025-07-05T16:32:58.226Z","repository":{"id":95735653,"uuid":"361758978","full_name":"sorenlind/transferit","owner":"sorenlind","description":"Python package that can help you train an image classification model using transfer learning and serve the model with TensorFlow Serving and Docker.","archived":false,"fork":false,"pushed_at":"2021-04-28T20:09:16.000Z","size":18,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-11T15:48:57.492Z","etag":null,"topics":["ml","tf-serving","transfer-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sorenlind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-26T13:24:04.000Z","updated_at":"2021-04-28T20:09:18.000Z","dependencies_parsed_at":"2023-06-25T21:29:12.810Z","dependency_job_id":null,"html_url":"https://github.com/sorenlind/transferit","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorenlind%2Ftransferit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorenlind%2Ftransferit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorenlind%2Ftransferit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorenlind%2Ftransferit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sorenlind","download_url":"https://codeload.github.com/sorenlind/transferit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241278161,"owners_count":19937893,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ml","tf-serving","transfer-learning"],"created_at":"2024-11-12T15:27:01.262Z","updated_at":"2025-02-28T23:43:27.310Z","avatar_url":"https://github.com/sorenlind.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# transferit\n\nThis repository contains a Python package that can help you train an image\nclassification model using transfer learning and serve the model with TensorFlow Serving\nand Docker.\n\nThe repo contains sample images you can use to train the model to tell a certain kind of\ncollectible playing card, namely _Magic: The Gathering_ cards from other objects. These,\nhowever, only serve as an example. You can easily use the transferit package to train\nmodels on your own data.\n\n## Installation\n\nInstall from PyPI:\n\n```shell\npip3 install transferit\n```\n\nTo get the example data and run the examples, you will need to check out the repository:\n\n```shell\ngit clone git@github.com:sorenlind/transferit.git\n```\n\n## Quick start\n\n1. Clone the repo\n2. Install the package either from source or from PyPI\n3. Download the [Caltech 256\n   dataset](http://www.vision.caltech.edu/Image_Datasets/Caltech256/) and store the\n   `.tar` file inside the `data/raw` folder.\n4. Run the `01 Create model.ipynb` notebook in the `notebooks` folder.\n5. Build the docker image: `docker build -t transferit .`\n6. Start the container: `docker run -t --rm -p 8501:8501 transferit`\n7. Run the `02 API Example usage.ipynb` notebook in the `notebooks` folder.\n\n## Introduction\n\nThe package provides a command line application, `transferit` which can help you prepare\ndate for training and evaluation as well as training the model. Finally, it can also\nwrap or package the trained model in way that makes it compatible with TensorFlow\nServing. The `transferit` command has four sub commands as briefly explained below:\n\n1. `create-class`: Copy and resize images in a specified folder (and its subfolders) to\n   another folder. This is handy if, for example you are training a binary image\n   classifier and you have a library of various kinds of images which you will use for\n   the negative class and a smaller set of custom images that you will use for the\n   positive class. Running this command twice (once for the positive and once for the\n   negative) class can create a complete data set for you.\n2. `split`: Creates a train / dev split using a dataset already prepared using the\n   `create-class` sub command.\n3. `train`: Train the actual model using the training and dev data created using the\n   `split` sub command.\n4. `wrap`: Wrap a trained model to make it compatible with TensorFlow Serving and ready\n   to be copied to a docker image.\n\nIn addition to running the command line application you can also call the relevant\nfunctions from your Python code such as a Jupyter notebook. The `notebooks` folder\ncontains a notebook `01 Create model.ipynb` which runs through the entire process of\npreparing data, training a model and wrapping it for serving. Note that before you can\nrun the notebook, you will have to download the [Caltech 256\ndataset](http://www.vision.caltech.edu/Image_Datasets/Caltech256/) and store the `.tar`\nfile inside the `data/raw` folder.\n\n### Preparing images\n\n```shell\ntransferit create-class ./raw/256_ObjectCategories/ ./prepared/full/negative --n-max 3000\ntransferit create-class ./raw/cards/ ./prepared/full/positive\n```\n\n### Creating train / dev split\n\n```shell\ntransferit split ./prepared/full/ ./prepared/\n```\n\n### Training model\n\n```shell\n transferit train ./prepared/train/ ./prepared/dev/ ./models/naked/\n```\n\n### Wrapping up model for TF Serving\n\n```shell\ntransferit wrap models/naked/models_best_loss.hdf5 ./models/wrapped/00000001/ -c Negative Positive\n```\n\n### Creating Dockerfile from the template\n\nThe repository contains a template for a Dockerfile called `Dockerfile.template`. You\ncan create a copy of this simply called `Dockerfile` and edit it to match your setup. If\nyou have been running the Jupyter notebook to train and wrap a model, you do not need to\nmake any changes to the Dockerfile.\n\n### Serving model locally using Docker\n\nBuild the image:\n\n```shell\ndocker build -t transferit .\n```\n\nOnce you have built the image, you can serve the model in a container as follows:\n\n```\ndocker run -t --rm -p 8501:8501 transferit\n```\n\nOnce the container is running, you can access it as shown in the example below. The\n`notebooks` folder contains a notebook called `02 API Example usage.ipynb` which has\nsimilar code and classifies two images from the dev dataset.\n\n```python\nimport base64\nimport json\nimport requests\n\nURL = \"http://localhost:8501/v1/models/transferit:classify\"\nHEADERS = {\"content-type\": \"application/json\"}\n\nwith open(img_filename, mode=\"rb\") as file:\n    img = file.read()\njpeg_bytes = base64.b64encode(img).decode(\"utf-8\")\n\nbody = {\n    \"signature_name\": \"serving_default\",\n    \"examples\": [\n        {\n            \"x\": {\"b64\": jpeg_bytes},\n        }\n    ],\n}\n\njson_response = requests.post(URL, data=json.dumps(body), headers=HEADERS)\njson_response.status_code\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsorenlind%2Ftransferit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsorenlind%2Ftransferit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsorenlind%2Ftransferit/lists"}