https://github.com/robi56/faceemr
Facial Expression Recognition in android where the predictive model built in tensorflow using convolutional neural network
https://github.com/robi56/faceemr
android convolutional-neural-networks deep-learning face-recognition facial-expression-recognition image-processing java python tensorflow
Last synced: about 1 year ago
JSON representation
Facial Expression Recognition in android where the predictive model built in tensorflow using convolutional neural network
- Host: GitHub
- URL: https://github.com/robi56/faceemr
- Owner: robi56
- Created: 2018-04-26T14:29:58.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2018-04-27T21:08:04.000Z (about 8 years ago)
- Last Synced: 2025-05-07T22:03:29.310Z (about 1 year ago)
- Topics: android, convolutional-neural-networks, deep-learning, face-recognition, facial-expression-recognition, image-processing, java, python, tensorflow
- Language: Java
- Homepage:
- Size: 51.1 MB
- Stars: 32
- Watchers: 2
- Forks: 15
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# FaceEMR
## Emotion Recognition from Facial Expression
The project aims to train a model using tensorflow for facial emotion detection and used the trained model
as predictor in android facial expression recongnition app.
The model is trained using tensorflow python framework and used in android application where the basic langauge is java.
Basically tensorflow provides a c++ api, that can be used in android application. The trained model by python langauge can be integrated with android project after inclduing tensorflow c++ framework dependencies and using native interface the model can be loaded and called in java class. This is the whole thing.
The total work of this project is divided into two parts
1) Devlop a model in tensoflow using python langauge
**KaggleFaceEmotionRecognition** folder contains the work
2) Devlop an android appication for facial expression recongtion
**FaceEMR** folder contains the work
### Part 1. Facial Expression Recongition Model developed in Tensorflow
In this work , I have used a simple Convolutional Neural Network Architecture to train a facial expression dataset.
**1. DataSet:** The dataset is collected from Facial xpression recognition challenge in kaggle
The challenge link https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/
The data consists of 48x48 pixel grayscale images of faces.The dataset contains facial expression of seven categories (0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral
**2. Model:**
In this work I have used the below CNN model
input_image->conv2d->pooling->conv2d->pooling->conv2d->pooling->dropout->softmax
The code fragment
```
x_image = tf.reshape(x, [-1, 48, 48, 1])
#48*48*1
conv1 = tf.layers.conv2d(x_image, 64, 3, 1, 'same', activation=tf.nn.relu)
#48*48*64
pool1 = tf.layers.max_pooling2d(conv1, 2, 2, 'same')
#24*24*64
conv2 = tf.layers.conv2d(pool1, 128, 3, 1, 'same', activation=tf.nn.relu)
#24*24*128
pool2 = tf.layers.max_pooling2d(conv2, 2, 2, 'same')
#12*12*128
conv3 = tf.layers.conv2d(pool2, 256, 3, 1, 'same', activation=tf.nn.relu)
#12*12*256
pool3 = tf.layers.max_pooling2d(conv3, 2, 2, 'same')
#6*6*256
flatten = tf.reshape(pool3, [-1, 6*6*256])
fc = tf.layers.dense(flatten, 1536, activation=tf.nn.relu)
dropout = tf.nn.dropout(fc, keep_prob)
logits = tf.layers.dense(dropout, 7)
outputs = tf.nn.softmax(logits, name=output_node_name)
```
**3. Result:** I have used 5000 iterations with batch size 100 and restore the model in protocal buffer file
### Part 2. Facial Expression Recongition Application in Android
I have used Android Studio for this application.
Integrating tensorflow dependency in android is really a tedious thing. the good news is that the latest news that android studio manages all dependencis related to tensorflow after adding the dependencies in *build.gradle(Module:app)* file
```
dependencies {
compile 'org.tensorflow:tensorflow-android:+'
}
```
The final dependency part looks like
```
dependencies {
implementation fileTree(dir: 'libs', include: ['*.jar'])
implementation 'com.android.support:appcompat-v7:26.1.0'
implementation 'com.android.support.constraint:constraint-layout:1.1.0'
testImplementation 'junit:junit:4.12'
androidTestImplementation 'com.android.support.test:runner:1.0.1'
androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.1'
compile 'org.tensorflow:tensorflow-android:+'
}
```
**1. Designing the UI Components**
Home Screen look like this
After taking a picture
And the final result
**2. Interacting with the Tensorflow Native Api**
The *org.tensorflow.contrib.android.TensorFlowInferenceInterface* handles all necessary operation to interact with native api. See more details in https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/android
**3. Finalizing the work**
There are lots of options for improvement.
1. Adding some cool features and increasing model performance using another models
2. Tuning the hyperparameters of used cnn model