https://github.com/sharpiless/pacman-with-paddlepaddle-gesture-control
pacman with paddlepaddle gesture control，手势识别用于吃豆人小游戏
https://github.com/sharpiless/pacman-with-paddlepaddle-gesture-control
Last synced: 7 months ago
JSON representation
pacman with paddlepaddle gesture control，手势识别用于吃豆人小游戏
- Host: GitHub
- URL: https://github.com/sharpiless/pacman-with-paddlepaddle-gesture-control
- Owner: Sharpiless
- Created: 2020-12-09T03:29:01.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2021-03-09T03:16:21.000Z (almost 5 years ago)
- Last Synced: 2025-07-05T05:37:14.384Z (7 months ago)
- Language: Python
- Size: 95.8 MB
- Stars: 16
- Watchers: 1
- Forks: 3
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# PaddlePaddle实现手势识别玩转吃豆豆！
## 文章目录：
### 1. 手势数据采集
### 2. PaddleX训练模型
### 3. 测试手势识别模型
### 4. 测试游戏种手势控制
### 5. 大功告成~
```python
# 解压代码
!unzip /home/aistudio/data/data41298/code.zip -d /home/aistudio/work/
```
```python
# !pip install paddlex
# !pip install imgaug
```
拳头表示向下走：
手掌表示向上走：
下面两个分别是向左和向右：
空白表示按位不动：
```python
# 设置工作路径
import os
os.chdir('/home/aistudio/work/Pacman-master/')
```
## 1. 手势数据采集：
这一步需要在本地运行collect文件夹下PalmTracker.py文件进行手势数据采集；
运行该程序时会打开摄像头，在指定区域做出手势，按s保存；
```python
# !python collect/PalmTracker.py
```
collect data game.py pacman.py test.jpg utils.py
config.py demo.py images src tools weights
## 2. PaddleX训练模型
这一步使用PaddleX提供的ResNet18进行训练；
预训练模型使用在'IMAGENET'上训练的权重，PaddleX选择参数 pretrain_weights='IMAGENET' 即可；
我这里每种手势共收集了40张左右，训练结果准确率在93%以上；
### 2.1 定义数据集
```python
from paddlex.cls import transforms
import os
import cv2
import numpy as np
import paddlex as pdx
import imgaug.augmenters as iaa
base = './data'
with open(os.path.join('train_list.txt'), 'w') as f:
for i, cls_fold in enumerate(os.listdir(base)):
cls_base = os.path.join(base, cls_fold)
files = os.listdir(cls_base)
print('{} train num:'.format(cls_fold), len(files))
for pt in files:
img = os.path.join(cls_fold, pt)
info = img + ' ' + str(i) + '\n'
f.write(info)
with open(os.path.join('labels.txt'), 'w') as f:
for i, cls_fold in enumerate(os.listdir(base)):
f.write(cls_fold+'\n')
train_transforms = transforms.Compose([
transforms.ResizeByShort(short_size=224),
iaa.AddToBrightness((-30, 30)),
iaa.pillike.EnhanceSharpness(),
iaa.LinearContrast((0.8, 1.2)),
transforms.RandomRotate(rotate_range=30, prob=0.5),
iaa.Dropout(p=(0, 0.1)),
transforms.Normalize()
])
train_dataset = pdx.datasets.ImageNet(
data_dir=base,
file_list='train_list.txt',
label_list='labels.txt',
transforms=train_transforms,
shuffle=True)
```
data41298 train num: 1
2020-07-03 10:19:56 [INFO] Starting to read file list from dataset...
2020-07-03 10:19:56 [INFO] 0 samples in file train_list.txt
### 2.2 使用ResNet18训练模型
此处训练20个epoch，初始学习率为2e-2
```python
num_classes = len(train_dataset.labels)
model = pdx.cls.ResNet18(num_classes=num_classes)
model.train(num_epochs=20,
train_dataset=train_dataset,
train_batch_size=32,
lr_decay_epochs=[5, 10, 15],
learning_rate=2e-2,
save_dir='w',
log_interval_steps=5,
save_interval_epochs=4)
```
## 3 测试手势识别模型：
```python
from paddlex.cls import transforms
import matplotlib.pyplot as plt
import paddlex
import cv2
import warnings
warnings.filterwarnings('ignore')
train_transforms = transforms.Compose([
transforms.RandomCrop(crop_size=224),
transforms.Normalize()
])
model = paddlex.load_model('weights/final')
im = cv2.imread('test.jpg')
result = model.predict(im, topk=1)
print("Predict Result:", result)
%matplotlib inline
plt.imshow(im)
plt.show()
```
2020-06-23 09:27:29 [INFO] Model[ResNet18] loaded.
Predict Result: [{'category_id': 1, 'category': 'left', 'score': 0.9999609}]
## 4. 测试游戏中手势控制：
本地运行demo.py即可；
```python
!python demo.py
```
## 5. 大功告成
然后将该控制嵌入到游戏中即可~
游戏代码来自：https://github.com/hbokmann/Pacman
```python
!python game.py
```
### 演示视频我放到Youtube了（因为B站审核太慢了，，，）
链接地址：[https://youtu.be/tlZT2WeaK1U](https://youtu.be/tlZT2WeaK1U)
## 更新，B站审核通过啦！
链接地址：[https://www.bilibili.com/video/BV1xa4y1Y7Mb/](https://www.bilibili.com/video/BV1xa4y1Y7Mb/)
## 关于作者：
> 北京理工大学 大二在读
> 感兴趣的方向为：目标检测、人脸识别、EEG识别等
> 将会定期分享一些小项目，感兴趣的朋友可以互相关注一下：[主页链接](http://aistudio.baidu.com/aistudio/personalcenter/thirdview/67156)
> 也欢迎大家fork、评论交流
> 作者博客主页：[https://blog.csdn.net/weixin_44936889](https://blog.csdn.net/weixin_44936889)
## 联系作者~
