https://github.com/asad-ismail/smn

Scaleable multitask Network
https://github.com/asad-ismail/smn

deep-learning image-processing instance-segmentation keypoint-detection multitask-learning phenomics phenotyping pytorch vegetable

Last synced: 12 months ago
JSON representation

Scaleable multitask Network

• Host: GitHub
• URL: https://github.com/asad-ismail/smn
• Owner: Asad-Ismail
• License: mit
• Created: 2021-11-22T12:54:36.000Z (over 4 years ago)
• Default Branch: main
• Last Pushed: 2022-05-26T22:38:05.000Z (almost 4 years ago)
• Last Synced: 2025-02-16T21:42:25.737Z (about 1 year ago)
• Topics: deep-learning, image-processing, instance-segmentation, keypoint-detection, multitask-learning, phenomics, phenotyping, pytorch, vegetable
• Language: Python
• Homepage:
• Size: 61.5 MB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
## All with One Image Net (SMN)

## [![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/Asad-Ismail/SMN/issues) [![Hits](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FAsad-Ismail%2FSMN&count_bg=%2379C83D&title_bg=%23555555&icon=&icon_color=%23E7E7E7&title=hits&edge_flat=false)](https://hits.seeyoufarm.com)

A Scaleable MultiTask Network Framework for image-based Object detection, Instance segmenation, Keypoint detection, Classification and Regression.

Our purpose is to solve two problems while using deep neural network in production

1. Design scaleable multitask network with shared bakbone

2. Handle dataset with sparse supervision

### Example Multi Task 

In an image we want to detect cucumbers, color board segment fruits, segment their backbone, and also classify there neck and overall rating

  


    

  

  

### Example Sparse Supervision Data


  


    

  

  

## Training MultiTask Network for two tasks using Sparse Data

  


    

  


### Training and Predictions

1) Install requirements using pip install -r requirements.txt

2) The netowrk can be seamless extended to handle more tasks by changing the config file

  

3) To train the netowrk run train_vegnet.py  

4) To predcit run predict_vegnet.py 

## Background

* Pipeiline of an Instance segmentation network like MaskRCNN with Keypoint Detector followed by seperate CNNs for per fruit classification/regression

 

First CNN network is for detection, segmentation, and keypoint detection like MaskRCNN which extracts the fruit patches or ROIs which are then fed to another network for multilabel classification and regression of each fruit. Figures below show this pipeline

  


    

  

  

  

  


    

  

  

  


    

  


* The second way is multi task learning. We propose here a multitask network with configureable inputs and that can be exapnded to do N segmentaiton,N classification and N regression tasks where N can be specified using a simple config file. The idea is to use shared features between task and use ROIs generated by RPN network to narrow the area of classification and regression. Multitask networks in addition to providing better generalizations aslo can be paralleleized and be computed very efficently.