{"id":19228136,"url":"https://github.com/unit-mesh/android-semantic-search-kit","last_synced_at":"2025-04-21T01:32:01.618Z","repository":{"id":205707092,"uuid":"714650304","full_name":"unit-mesh/android-semantic-search-kit","owner":"unit-mesh","description":"A poc of ML/LLM/Embedding run in classic Android OS","archived":false,"fork":false,"pushed_at":"2023-11-15T01:12:23.000Z","size":3702,"stargazers_count":11,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-01T07:54:10.994Z","etag":null,"topics":["android","genai","llm","os"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/unit-mesh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-05T13:57:55.000Z","updated_at":"2025-02-27T05:37:05.000Z","dependencies_parsed_at":"2024-11-09T15:28:35.254Z","dependency_job_id":null,"html_url":"https://github.com/unit-mesh/android-semantic-search-kit","commit_stats":null,"previous_names":["unit-mesh/android-ml-poc"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unit-mesh%2Fandroid-semantic-search-kit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unit-mesh%2Fandroid-semantic-search-kit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unit-mesh%2Fandroid-semantic-search-kit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unit-mesh%2Fandroid-semantic-search-kit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/unit-mesh","download_url":"https://codeload.github.com/unit-mesh/android-semantic-search-kit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249982561,"owners_count":21355719,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","genai","llm","os"],"created_at":"2024-11-09T15:26:52.388Z","updated_at":"2025-04-21T01:31:59.843Z","avatar_url":"https://github.com/unit-mesh.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Run LLM/Embedding on Android\n\n[![Android](https://github.com/unit-mesh/run-llm-on-android/actions/workflows/ci.yml/badge.svg)](https://github.com/unit-mesh/run-llm-on-android/actions/workflows/ci.yml)\n[![Powered By](https://img.shields.io/badge/powered_by-chocolate_factory-blue?logo=kotlin\u0026logoColor=fff)](https://github.com/unit-mesh/chocolate-factory)\n![Android](https://img.shields.io/badge/Android-3DDC84?logo=android\u0026logoColor=white)\n\n## Screenshots\n\nEmbedding Demo:\n\n\u003cimg src=\"https://unitmesh.cc/android-llm/llm-embedding.png\" width=\"300\" /\u003e\n\n## Quick Start and Swift Retries\n\n参考：[GitHub Workflows](.github/workflows/ci.yml)\n\n### 本地构建（暂时不保证成功）\n\n```bash\ngit submodule update --init\n```\n\n1.setup NDK\n\n将 NDK 换成你的本地 NDK 路径和版本（我的本地 macOS 示例）\n\n```bash\nexport ANDROID_HOME=$HOME/Library/Android/sdk\nexport ANDROID_NDK_HOME=$HOME/Library/Android/sdk/ndk/26.1.10909125\n\nexport TOOLCHAIN=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/darwin-x86_64\nexport TARGET=aarch64-linux-android\nexport API=34\n\nexport AR=$TOOLCHAIN/bin/llvm-ar\nexport CC=$TOOLCHAIN/bin/$TARGET$API-clang\nexport AS=$CC\nexport CXX=$TOOLCHAIN/bin/$TARGET$API-clang++\nexport LD=$TOOLCHAIN/bin/ld\nexport RANLIB=$TOOLCHAIN/bin/llvm-ranlib\nexport STRIP=$TOOLCHAIN/bin/llvm-strip\n\nexport PATH=$PATH:$ANDROID_HOME/cmdline-tools/latest/bin\nexport PATH=$PATH:$TOOLCHAIN/bin\n```\n\n2.setup Rust target\n\n```bash\nrustup target add armv7-linux-androideabi i686-linux-android arm-linux-androideabi x86_64-linux-android aarch64-linux-android\n```\n\n3.config linker\n\nAfter that add the following like to `$HOME/.cargo/config` (make the config file if it doesn't exist):\n\nFollow: [https://doc.rust-lang.org/beta/rustc/platform-support.html](https://doc.rust-lang.org/beta/rustc/platform-support.html)\n\nPS: remember to change the path to your NDK\n\n```toml\n[target.i686-linux-android]\nlinker = \"/Users/phodal/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/i686-linux-android34-clang\"\n\n[target.arm-linux-android]\nlinker = \"/Users/phodal/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/arm-linux-android34-clang\"\n\n[target.x86_64-linux-android]\nlinker = \"/Users/phodal/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/x86_64-linux-android34-clang\"\n\n[target.aarch64-linux-android]\nlinker = \"/Users/phodal/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android34-clang\"\n```\n\n4.build FFI\n\n```\n./gradlew cargoBuild\n```\n\nfor iOS (sample, not working)\n\n```\nrustup target add aarch64-apple-ios\ncargo build --target=x86_64-apple-ios --release\n```\n\n5.run app\n\n## Model for Testing\n\n### Convert Model to OnnX\n\nTools: [https://github.com/huggingface/optimum](https://github.com/huggingface/optimum)\n\n1.setup\n\n```bash\npip install optimum[exporters,onnxruntime]\n```\n\n2.convert onnx\n\n```bash\noptimum-cli export onnx -m Helsinki-NLP/opus-mt-zh-en --optimize O2 optus-mt-zh-en-onnx\n```\n\n3.quantize Onnx\n\n```bash\noptimum-cli onnxruntime quantize \\\n  --avx512 \\\n  --onnx_model bert-tiny-onnx \\\n  -o quantized_bert-tiny-onnx\n```\n\nExamples: [https://github.com/unit-mesh/onnx-convert-test](https://github.com/unit-mesh/onnx-convert-test)\n\n### Mini Embedding Model spikes\n\nOther Testing Models:\n\n- [https://github.com/unit-mesh/testing-onnx-models](https://github.com/unit-mesh/testing-onnx-models)\n\nIntelliJ IDEA Search Everywhere Model:\n\n- semantic-text-search-0.0.1.jar,\n  88.4M - [multi-qa-MiniLM-L6-cos-v1](https://packages.jetbrains.team/maven/p/ml-search-everywhere/local-models/org/jetbrains/intellij/searcheverywhereMl/semantics/semantic-text-search/0.0.1/semantic-text-search-0.0.1.jar)\n- semantic-text-search-0.0.3.jar,\n  9.6M - [dan-bert-tiny](https://packages.jetbrains.team/maven/p/ml-search-everywhere/local-models/org/jetbrains/intellij/searcheverywhereMl/semantics/semantic-text-search/0.0.3/semantic-text-search-0.0.3.jar)\n\nBloop Model:\n\n- [all-MiniLM-L6-v2](https://github.com/BloopAI/bloop/tree/95559bf47dbe40497f01665184d194726378e800/apps/desktop/src-tauri/model),\n  21.9M\n\n## FAQ\n\n### Failed to load Huggingface native library.\n\n```\nCaused by: ai.djl.engine.EngineException: Failed to load Huggingface native library.\n    at ai.djl.huggingface.tokenizers.jni.LibUtils.\u003cclinit\u003e(LibUtils.java:43)\n    at ai.djl.huggingface.tokenizers.jni.LibUtils.checkStatus(LibUtils.java:50)\n    at ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.newInstance(HuggingFaceTokenizer.java:170)\n    at org.unitmesh.llmpoc.embedding.STSemantic$Companion.create(STSemantic.kt:78)\n    at org.unitmesh.llmpoc.MainActivity.onCreate(MainActivity.kt:33)\n    at android.app.Activity.performCreate(Activity.java:8143)\n    at android.app.Activity.performCreate(Activity.java:8114)\n    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1310)\n    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3513)\n    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3700) \n    at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85) \n    at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135) \n    at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95) \n    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2136) \n    at android.os.Handler.dispatchMessage(Handler.java:106) \n    at android.os.Looper.loop(Looper.java:236) \n    at android.app.ActivityThread.main(ActivityThread.java:8060) \n    at java.lang.reflect.Method.invoke(Native Method) \n    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656) \n    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967) \nCaused by: java.lang.IllegalStateException: Cannot copy jni files\n    at ai.djl.huggingface.tokenizers.jni.LibUtils.copyJniLibraryFromClasspath(LibUtils.java:108)\n    at ai.djl.huggingface.tokenizers.jni.LibUtils.loadLibrary(LibUtils.java:66)\n    at ai.djl.huggingface.tokenizers.jni.LibUtils.\u003cclinit\u003e(LibUtils.java:41)\n    at ai.djl.huggingface.tokenizers.jni.LibUtils.checkStatus(LibUtils.java:50) \n    at ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.newInstance(HuggingFaceTokenizer.java:170) \n    at org.unitmesh.llmpoc.embedding.STSemantic$Companion.create(STSemantic.kt:78) \n    at org.unitmesh.llmpoc.MainActivity.onCreate(MainActivity.kt:33) \n    at android.app.Activity.performCreate(Activity.java:8143) \n    at android.app.Activity.performCreate(Activity.java:8114) \n    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1310) \n    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3513) \n    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3700) \n    at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85) \n    at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135) \n    at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95) \n    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2136) \n    at android.os.Handler.dispatchMessage(Handler.java:106) \n    at android.os.Looper.loop(Looper.java:236) \n    at android.app.ActivityThread.main(ActivityThread.java:8060) \n    at java.lang.reflect.Method.invoke(Native Method) \n    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656) \n    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967) \n```\n\nIssue: [2170](https://github.com/deepjavalibrary/djl/issues/2170)\n\nSolution: [Rust Android Gradle Plugin](https://github.com/mozilla/rust-android-gradle)\n\n### arm-linux-androideabi-ranlib not found\n\nError building Rust project for Android (Flutter): arm-linux-androideabi-ranlib not found for OpenSSL compilation\n\nhttps://stackoverflow.com/questions/75943717/error-building-rust-project-for-android-flutter-arm-linux-androideabi-ranlib\n\n### Failed to load model with error\n\n需要更新 Onnx Runtime 的版本\n\nthread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: CreateSession(Msg(\"Failed to\nload model with error: /Users/runner/work/1/s/onnxruntime/core/graph/model_load_utils.h:56 void onnxruntime::\nmodel_load_utils::ValidateOpsetForDomain(const std::unordered_map\u003cstd::string, int\u003e \u0026, const logging::Logger \u0026, bool,\nconst std::string \u0026, int) ONNX Runtime only *guarantees* support for models stamped with official released onnx opset\nversions. Opset 19 is under development and support for this is limited. The operator schemas and or other functionality\nmay change before next ONNX release and in this case ONNX Runtime will not guarantee backward compatibility. Current\nofficial support for domain com.ms.internal.nhwc is till opset 18.\\n\"))',\n/Volumes/source/ai/Inference/inference_core/src/embed.rs:49:18\n\n其它问题： [use v1.9.0 to load ONNX Opset 15 failed](https://github.com/microsoft/onnxruntime/issues/9418)\n\n### JNI 函数名不对\n\n加载函数：`System.LoadLibrary(\"tokenizers\");`\n\nNo implementation found for long\norg.unitmesh.tokenizer.huggingface.tokenizers.jni.TokenizersLibrary.createTokenizerFromString(java.lang.String) (tried\nJava_org_unitmesh_tokenizer_huggingface_tokenizers_jni_TokenizersLibrary_createTokenizerFromString and\nJava_org_unitmesh_tokenizer_huggingface_tokenizers_jni_TokenizersLibrary_createTokenizerFromString__Ljava_lang_String_2) -\nis the library loaded, e.g. System.loadLibrary?","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funit-mesh%2Fandroid-semantic-search-kit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funit-mesh%2Fandroid-semantic-search-kit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funit-mesh%2Fandroid-semantic-search-kit/lists"}