{"id":16352700,"url":"https://github.com/cswellessun/embeddedsystemproject","last_synced_at":"2025-06-20T11:10:34.055Z","repository":{"id":176444987,"uuid":"649233764","full_name":"CSWellesSun/EmbeddedSystemProject","owner":"CSWellesSun","description":"Project for ZJU Embedded System. A face recognition and chatgpt project based on RK3568 and RKNN.","archived":false,"fork":false,"pushed_at":"2023-06-24T08:53:57.000Z","size":42382,"stargazers_count":5,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-20T05:09:20.714Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CSWellesSun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-04T08:02:50.000Z","updated_at":"2025-05-16T07:09:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"7307c3f5-2fe4-472a-a3b0-2bb99ef1c5ec","html_url":"https://github.com/CSWellesSun/EmbeddedSystemProject","commit_stats":null,"previous_names":["cswellessun/embeddedsystemproject"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CSWellesSun/EmbeddedSystemProject","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSWellesSun%2FEmbeddedSystemProject","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSWellesSun%2FEmbeddedSystemProject/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSWellesSun%2FEmbeddedSystemProject/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSWellesSun%2FEmbeddedSystemProject/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CSWellesSun","download_url":"https://codeload.github.com/CSWellesSun/EmbeddedSystemProject/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSWellesSun%2FEmbeddedSystemProject/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260933626,"owners_count":23084960,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T01:27:14.464Z","updated_at":"2025-06-20T11:10:29.040Z","avatar_url":"https://github.com/CSWellesSun.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Embedded System Project\n\n## 环境\n\n基本必须使用如下环境才能运行。\n\n### RKNN-Toolkit2 (PC) \n\n我们已经提供了 `resnet.rknn` 模型，这一步可以跳过，如果要自己生成，具体方法如下：\n\n使用 `Ubuntu18.04`，`python-3.6`。`RKNN-Toolkit2`的版本是 1.4.0，首先执行 `RKNN-Toolkit2` 所需要的所有环境，特别是 `torch==1.10.1` 和 `torchvision==0.11.2`。\n\n然后安装 `facenet-pytorch`的环境，由于RKNN不支持 `ReduceL1 / ReduceL2`等算子，而 `InceptionResnetV1` 里存在 `F.normalize(x, p=2)` 无法被转换成 RKNN，所以需要进入 `facenet-pytorch` 源码修改 `InceptionResnetV1`，将 `forward` 最后一层的 `normalize` 删除。\n\n最后执行 `python save_model.py` 就能导出 `model.pt`，然后再执行 `python model.py` 就能得到 `resnet.rknn` 模型。\n\n### RKNN-Toolkit-Lite2 (板子)\n\n我们使用的嵌入式设备是 [Firefly RK3568](https://wiki.t-firefly.com/zh_CN/ROC-RK3568-PC/)。`RKNN` 在 `RK3568` 上使用 NPU 必须要用 `Debain` 系统！系统使用官网提供的 [Debain10](https://www.t-firefly.com/doc/download/107.html)（该系统限制极大！），`RKNN-Toolkit2`的版本是 1.4.0，我们只需要安装 `RKNN-Toolkit-Lite2` 即可（安装其中的 `packages`）。\n\n然后极力推荐使用 [MiniConda](https://docs.conda.io/en/latest/miniconda.html) 以及 [whl官方下载网站](https://download.pytorch.org/whl/torch_stable.html)。默认的 `python3` 和 `pip3` 极难下载到正确的环境！\n\n我们使用的是 `python3.9`，具体的环境参考 `requirements.txt`，其中依旧是 `torch` 和 `torchvision` 的版本最为重要！其他基本不冲突即可。\n\n## 外设\n\n### 摄像头\n\n使用 USB 摄像头，只需要该摄像头能支持 `OpenCV` 即可。注意将 `camera.py` 中的 `camera` 变量改成具体的 `ID`。如果不知道 `ID` 值，可以执行 `python camera.py` 来得到获取 `ID` 值。\n\n## 使用\n\n### 执行\n\n`python system.py` 即可。一开始有可能出现 `torchvision` 中导入 `io.image` 失败的 `warning`，但是对系统执行不影响。\n\n### 配置\n\n首先需要将 `config.example.ini` 中的 `ChatGPT` 接口进行修改，目前仅支持 `Azure OpenAI` 接口（学生可免费申请），注意使用 `Azure OpenAI` 接口不需要代理。\n\n### RKNN-Toolkit-Lite2 源码修改\n\n在执行过程中可能会出现 `ToolKit-Lite2` 的报错，大意是 `inputs[0]` 类型不是 `numpy`，该类型为 `tensor`,只需要加上一行 `inputs = [inputs[i].numpy() for i in range(len(inputs))]` 即可。 \n\n### 注册\n\n当使用 `FaceRecoginition` 的 `register` 功能时，会在 `user` 目录下创建一个新的用户 `u\u003cid\u003e`（例如 `u0 / u1 / u2` ），其中会保存图像处理之后的图片 `img\u003cid\u003e.png` 、经过 `facenet` 抽取的特征 `embed.npy` 以及用户的姓名年龄信息 `info.txt`。\n\n使用 `MobaXterm` ssh 连接到板子之后就可以打开摄像头窗口，按下 `s` 键即可保存图片并抽取特征。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcswellessun%2Fembeddedsystemproject","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcswellessun%2Fembeddedsystemproject","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcswellessun%2Fembeddedsystemproject/lists"}