{"id":28518490,"url":"https://github.com/sharpiless/pacman-with-paddlepaddle-gesture-control","last_synced_at":"2025-07-16T08:09:16.152Z","repository":{"id":54121230,"uuid":"319830883","full_name":"Sharpiless/pacman-with-paddlepaddle-gesture-control","owner":"Sharpiless","description":"pacman with paddlepaddle gesture control，手势识别用于吃豆人小游戏","archived":false,"fork":false,"pushed_at":"2021-03-09T03:16:21.000Z","size":100477,"stargazers_count":16,"open_issues_count":3,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-05T05:37:14.384Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Sharpiless.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-09T03:29:01.000Z","updated_at":"2025-06-17T06:39:11.000Z","dependencies_parsed_at":"2022-08-13T07:00:48.135Z","dependency_job_id":null,"html_url":"https://github.com/Sharpiless/pacman-with-paddlepaddle-gesture-control","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Sharpiless/pacman-with-paddlepaddle-gesture-control","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fpacman-with-paddlepaddle-gesture-control","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fpacman-with-paddlepaddle-gesture-control/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fpacman-with-paddlepaddle-gesture-control/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fpacman-with-paddlepaddle-gesture-control/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Sharpiless","download_url":"https://codeload.github.com/Sharpiless/pacman-with-paddlepaddle-gesture-control/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fpacman-with-paddlepaddle-gesture-control/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265495388,"owners_count":23776622,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-09T05:37:16.839Z","updated_at":"2025-07-16T08:09:16.140Z","avatar_url":"https://github.com/Sharpiless.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PaddlePaddle实现手势识别玩转吃豆豆！\n\n![](https://ai-studio-static-online.cdn.bcebos.com/2207b0c19f2944cd997f003f3ddb8b8fc735d852ba584d1dadb12bb5c94d0697)\n\n\u003ciframe src=\"//player.bilibili.com/player.html?aid=668543771\u0026bvid=BV1xa4y1Y7Mb\u0026cid=204795622\u0026page=1\" scrolling=\"no\" border=\"0\" frameborder=\"no\" framespacing=\"0\" allowfullscreen=\"true\"\u003e \u003c/iframe\u003e\n\n## 文章目录：\n\n### 1. 手势数据采集\n### 2. PaddleX训练模型\n### 3. 测试手势识别模型\n### 4. 测试游戏种手势控制\n### 5. 大功告成~\n\n\n```python\n# 解压代码\n!unzip /home/aistudio/data/data41298/code.zip -d /home/aistudio/work/\n```\n\n\n```python\n# !pip install paddlex\n# !pip install imgaug\n```\n\n拳头表示向下走：\n\n![](https://ai-studio-static-online.cdn.bcebos.com/968be5f9b32f4840a0c3d7d4fd477359a6a41afec8cb400f81557827ae65b5de)\n\n手掌表示向上走：\n\n![](https://ai-studio-static-online.cdn.bcebos.com/a380aea8ff944a9fbc6f3677fb90b418aa8e05fdf6c1419495e1ad8cd49aa613)\n\n下面两个分别是向左和向右：\n\n![](https://ai-studio-static-online.cdn.bcebos.com/3b1a1c71eb9941e7ba41cc11cc3444f3994c17b8373f4d81b7ec7d8b18013e8f)\n![](https://ai-studio-static-online.cdn.bcebos.com/0155857100374323bf167f0fdb90973e1ef9b0f4880d408080e5e83a1d740397)\n\n空白表示按位不动：\n\n![](https://ai-studio-static-online.cdn.bcebos.com/f0d252786c2a428abc721c786b53d0244ca803b928f541dc8b2cbb5085f204b2)\n\n\n\n```python\n# 设置工作路径\nimport os\nos.chdir('/home/aistudio/work/Pacman-master/')\n```\n\n## 1. 手势数据采集：\n\n这一步需要在本地运行collect文件夹下PalmTracker.py文件进行手势数据采集；\n\n运行该程序时会打开摄像头，在指定区域做出手势，按s保存；\n\n![](https://ai-studio-static-online.cdn.bcebos.com/06c664b20f794a4da1d3cffc5c57aad616dc037a5c8542fa802c879bb67fdac9)\n\n\n\n```python\n# !python collect/PalmTracker.py\n```\n\n    collect    data     game.py  pacman.py\ttest.jpg  utils.py\n    config.py  demo.py  images   src\ttools\t  weights\n\n\n## 2. PaddleX训练模型\n\n这一步使用PaddleX提供的ResNet18进行训练；\n\n预训练模型使用在'IMAGENET'上训练的权重，PaddleX选择参数 pretrain_weights='IMAGENET' 即可；\n\n我这里每种手势共收集了40张左右，训练结果准确率在93%以上；\n\n### 2.1 定义数据集\n\n\n```python\nfrom paddlex.cls import transforms\nimport os\nimport cv2\nimport numpy as np\nimport paddlex as pdx\nimport imgaug.augmenters as iaa\n\nbase = './data'\n\nwith open(os.path.join('train_list.txt'), 'w') as f:\n    for i, cls_fold in enumerate(os.listdir(base)):\n        cls_base = os.path.join(base, cls_fold)\n        files = os.listdir(cls_base)\n        print('{} train num:'.format(cls_fold), len(files))\n        for pt in files:\n            img = os.path.join(cls_fold, pt)\n            info = img + ' ' + str(i) + '\\n'\n            f.write(info)\n\nwith open(os.path.join('labels.txt'), 'w') as f:\n    for i, cls_fold in enumerate(os.listdir(base)):\n        f.write(cls_fold+'\\n')\n\ntrain_transforms = transforms.Compose([\n    transforms.ResizeByShort(short_size=224),\n    iaa.AddToBrightness((-30, 30)),\n    iaa.pillike.EnhanceSharpness(),\n    iaa.LinearContrast((0.8, 1.2)),\n    transforms.RandomRotate(rotate_range=30, prob=0.5),\n    iaa.Dropout(p=(0, 0.1)),\n    transforms.Normalize()\n])\n\ntrain_dataset = pdx.datasets.ImageNet(\n    data_dir=base,\n    file_list='train_list.txt',\n    label_list='labels.txt',\n    transforms=train_transforms,\n    shuffle=True)\n```\n\n    data41298 train num: 1\n    2020-07-03 10:19:56 [INFO]\tStarting to read file list from dataset...\n    2020-07-03 10:19:56 [INFO]\t0 samples in file train_list.txt\n\n\n### 2.2 使用ResNet18训练模型\n\n此处训练20个epoch，初始学习率为2e-2\n\n\n```python\nnum_classes = len(train_dataset.labels)\nmodel = pdx.cls.ResNet18(num_classes=num_classes)\nmodel.train(num_epochs=20,\n            train_dataset=train_dataset,\n            train_batch_size=32,\n            lr_decay_epochs=[5, 10, 15],\n            learning_rate=2e-2,\n            save_dir='w',\n            log_interval_steps=5,\n            save_interval_epochs=4)\n            \n```\n\n## 3 测试手势识别模型：\n\n\n```python\nfrom paddlex.cls import transforms\nimport matplotlib.pyplot as plt\nimport paddlex\nimport cv2\nimport warnings\n\nwarnings.filterwarnings('ignore')\n\ntrain_transforms = transforms.Compose([\n    transforms.RandomCrop(crop_size=224),\n    transforms.Normalize()\n])\n\nmodel = paddlex.load_model('weights/final')\nim = cv2.imread('test.jpg')\nresult = model.predict(im, topk=1)\nprint(\"Predict Result:\", result)\n\n%matplotlib inline\nplt.imshow(im)\nplt.show()\n```\n\n    2020-06-23 09:27:29 [INFO]\tModel[ResNet18] loaded.\n    Predict Result: [{'category_id': 1, 'category': 'left', 'score': 0.9999609}]\n\n\n\n![png](output_13_1.png)\n\n\n## 4. 测试游戏中手势控制：\n\n本地运行demo.py即可；\n\n![](https://ai-studio-static-online.cdn.bcebos.com/39bb2143fff544e0b553e5499ec7f3f49346affa73b94f42902d2314b9d9c47d)\n\n\n\n```python\n!python demo.py\n```\n\n## 5. 大功告成\n\n然后将该控制嵌入到游戏中即可~\n\n游戏代码来自：https://github.com/hbokmann/Pacman\n\n\n```python\n!python game.py\n```\n\n![](https://ai-studio-static-online.cdn.bcebos.com/0f06ca2879024729a5f3411a4c7fac5d370e1b7ce4754833b084b624de9187c2)\n\n\n### 演示视频我放到Youtube了（因为B站审核太慢了，，，）\n\n链接地址：[https://youtu.be/tlZT2WeaK1U](https://youtu.be/tlZT2WeaK1U)\n\n## 更新，B站审核通过啦！\n\n链接地址：[https://www.bilibili.com/video/BV1xa4y1Y7Mb/](https://www.bilibili.com/video/BV1xa4y1Y7Mb/)\n\n## 关于作者：\n\u003e 北京理工大学 大二在读\n\n\u003e 感兴趣的方向为：目标检测、人脸识别、EEG识别等\n\n\u003e 将会定期分享一些小项目，感兴趣的朋友可以互相关注一下：[主页链接](http://aistudio.baidu.com/aistudio/personalcenter/thirdview/67156)\n\n\u003e 也欢迎大家fork、评论交流\n\n\u003e 作者博客主页：[https://blog.csdn.net/weixin_44936889](https://blog.csdn.net/weixin_44936889)\n\n\n## 联系作者~\n![](https://ai-studio-static-online.cdn.bcebos.com/8d46082a0464465da53ba213bd875537b94dc8c7de4543e9b0fc5c9e39404bc0)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharpiless%2Fpacman-with-paddlepaddle-gesture-control","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsharpiless%2Fpacman-with-paddlepaddle-gesture-control","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharpiless%2Fpacman-with-paddlepaddle-gesture-control/lists"}