{"id":13411819,"url":"https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX","last_synced_at":"2025-03-14T17:31:10.952Z","repository":{"id":45171812,"uuid":"396474980","full_name":"AsuharietYgvar/AppleNeuralHash2ONNX","owner":"AsuharietYgvar","description":"Convert Apple NeuralHash model for CSAM Detection to ONNX.","archived":false,"fork":false,"pushed_at":"2021-08-21T07:27:26.000Z","size":15,"stargazers_count":1531,"open_issues_count":6,"forks_count":131,"subscribers_count":38,"default_branch":"master","last_synced_at":"2024-10-29T18:08:17.525Z","etag":null,"topics":["apple","machine-learning","neuralhash","reverse-engineering"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AsuharietYgvar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-15T19:52:47.000Z","updated_at":"2024-10-28T12:44:00.000Z","dependencies_parsed_at":"2022-07-13T18:21:41.701Z","dependency_job_id":null,"html_url":"https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AsuharietYgvar%2FAppleNeuralHash2ONNX","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AsuharietYgvar%2FAppleNeuralHash2ONNX/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AsuharietYgvar%2FAppleNeuralHash2ONNX/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AsuharietYgvar%2FAppleNeuralHash2ONNX/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AsuharietYgvar","download_url":"https://codeload.github.com/AsuharietYgvar/AppleNeuralHash2ONNX/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243618670,"owners_count":20320274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apple","machine-learning","neuralhash","reverse-engineering"],"created_at":"2024-07-30T20:01:17.238Z","updated_at":"2025-03-14T17:31:09.631Z","avatar_url":"https://github.com/AsuharietYgvar.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# AppleNeuralHash2ONNX\n\nConvert Apple NeuralHash model for [CSAM Detection](https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf) to [ONNX](https://github.com/onnx/onnx).\n\n## Intro\n\nApple NeuralHash is a [perceptual hashing](https://en.wikipedia.org/wiki/Perceptual_hashing) method for images based on neural networks. It can tolerate image resize and compression. The steps of hashing is as the following:\n1. Convert image to RGB.\n2. Resize image to `360x360`.\n3. Normalize RGB values to `[-1, 1]` range.\n4. Perform inference on the NeuralHash model.\n5. Calculate dot product of a `96x128` matrix with the resulting vector of 128 floats.\n6. Apply [binary step](https://en.wikipedia.org/wiki/Heaviside_step_function) to the resulting 96 float vector.\n7. Convert the vector of 1.0 and 0.0 to bits, resulting in 96-bit binary data.\n\nIn this project, we convert Apple's NeuralHash model to ONNX format. A demo script for testing the model is also included.\n\n## Prerequisite\n\n### OS\n\nBoth macOS and Linux will work. In the following sections Debian is used for Linux example.\n\n### LZFSE decoder\n\n- macOS: Install by running `brew install lzfse`.\n- Linux: Build and install from [lzfse](https://github.com/lzfse/lzfse) source.\n\n### Python\n\nPython 3.6 and above should work. Install the following dependencies:\n```bash\npip install onnx coremltools\n```\n\n## Conversion Guide\n\n### Step 1: Get NeuralHash model\n\nYou will need 4 files from a recent macOS or iOS build:\n- neuralhash_128x96_seed1.dat\n- NeuralHashv3b-current.espresso.net\n- NeuralHashv3b-current.espresso.shape\n- NeuralHashv3b-current.espresso.weights\n\n**Option 1: From macOS or jailbroken iOS device (Recommended)**\n\nIf you have a recent version of macOS (11.4+) or jailbroken iOS (14.7+) installed, simply grab these files from `/System/Library/Frameworks/Vision.framework/Resources/` (on macOS) or `/System/Library/Frameworks/Vision.framework/` (on iOS).\n\n\u003cdetails\u003e\n  \u003csummary\u003eOption 2: From iOS IPSW (click to reveal)\u003c/summary\u003e\n\n1. Download any `.ipsw` of a recent iOS build (14.7+) from [ipsw.me](https://ipsw.me/).\n2. Unpack the file:\n```bash\ncd /path/to/ipsw/file\nmkdir unpacked_ipsw\ncd unpacked_ipsw\nunzip ../*.ipsw\n```\n3. Locate system image:\n```bash\nls -lh\n```\nWhat you need is the largest `.dmg` file, for example `018-63036-003.dmg`.\n\n4. Mount system image. On macOS simply open the file in Finder. On Linux run the following commands:\n```bash\n# Build and install apfs-fuse\nsudo apt install fuse libfuse3-dev bzip2 libbz2-dev cmake g++ git libattr1-dev zlib1g-dev\ngit clone https://github.com/sgan81/apfs-fuse.git\ncd apfs-fuse\ngit submodule init\ngit submodule update\nmkdir build\ncd build\ncmake ..\nmake\nsudo make install\nsudo ln -s /bin/fusermount /bin/fusermount3\n# Mount image\nmkdir rootfs\napfs-fuse 018-63036-003.dmg rootfs\n```\nRequired files are under `/System/Library/Frameworks/Vision.framework/` in mounted path.\n\n\u003c/details\u003e\n\nPut them under the same directory:\n```bash\nmkdir NeuralHash\ncd NeuralHash\ncp /System/Library/Frameworks/Vision.framework/Resources/NeuralHashv3b-current.espresso.* .\ncp /System/Library/Frameworks/Vision.framework/Resources/neuralhash_128x96_seed1.dat .\n```\n\n### Step 2: Decode model structure and shapes\n\nNormally compiled Core ML models store structure in `model.espresso.net` and shapes in `model.espresso.shape`, both in JSON. It's the same for NeuralHash model but compressed with [LZFSE](https://en.wikipedia.org/wiki/LZFSE).\n\n```bash\ndd if=NeuralHashv3b-current.espresso.net bs=4 skip=7 | lzfse -decode -o model.espresso.net\ndd if=NeuralHashv3b-current.espresso.shape bs=4 skip=7 | lzfse -decode -o model.espresso.shape\ncp NeuralHashv3b-current.espresso.weights model.espresso.weights\n```\n\n### Step 3: Convert model to ONNX\n\n```bash\ncd ..\ngit clone https://github.com/AsuharietYgvar/TNN.git\ncd TNN\npython3 tools/onnx2tnn/onnx-coreml/coreml2onnx.py ../NeuralHash\n```\n\nThe resulting model is `NeuralHash/model.onnx`.\n\n## Usage\n\n### Inspect model\n\n[Netron](https://github.com/lutzroeder/netron) is a perfect tool for this purpose.\n\n### Calculate neural hash with [onnxruntime](https://github.com/microsoft/onnxruntime)\n\n1. Install required libraries:\n```bash\npip install onnxruntime pillow\n```\n2. Run `nnhash.py` on an image:\n```bash\npython3 nnhash.py /path/to/model.onnx /path/to/neuralhash_128x96_seed1.dat image.jpg\n```\n\nExample output:\n```\nab14febaa837b6c1484c35e6\n```\n\n**Note:** Neural hash generated here might be a few bits off from one generated on an iOS device. This is expected since different iOS devices generate slightly different hashes anyway. The reason is that neural networks are based on floating-point calculations. The accuracy is highly dependent on the hardware. For smaller networks it won't make any difference. But NeuralHash has 200+ layers, resulting in significant cumulative errors.\n\n|Device|Hash|\n|---|---|\n|iPad Pro 10.5-inch|`2b186faa6b36ffcc4c4635e1`|\n|M1 Mac|`2b5c6faa6bb7bdcc4c4731a1`|\n|iOS Simulator|`2b5c6faa6bb6bdcc4c4731a1`|\n|ONNX Runtime|`2b5c6faa6bb6bdcc4c4735a1`|\n\n## Credits\n\n- [nhcalc](https://github.com/KhaosT/nhcalc) for uncovering NeuralHash private API.\n- [TNN](https://github.com/Tencent/TNN) for compiled Core ML to ONNX script.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAsuharietYgvar%2FAppleNeuralHash2ONNX","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAsuharietYgvar%2FAppleNeuralHash2ONNX","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAsuharietYgvar%2FAppleNeuralHash2ONNX/lists"}