https://github.com/iakashpaul/portal
Android app for the Hole in your Palm project, making LLMs accessible on-device!
https://github.com/iakashpaul/portal
android large-language-model large-language-models llama llm mistral
Last synced: about 1 year ago
JSON representation
Android app for the Hole in your Palm project, making LLMs accessible on-device!
- Host: GitHub
- URL: https://github.com/iakashpaul/portal
- Owner: iakashpaul
- Created: 2024-01-18T06:53:11.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-03T10:55:45.000Z (about 2 years ago)
- Last Synced: 2025-04-30T02:02:42.812Z (about 1 year ago)
- Topics: android, large-language-model, large-language-models, llama, llm, mistral
- Language: Kotlin
- Homepage: https://akashpaul.com/portal-app
- Size: 19.5 MB
- Stars: 17
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-mobile-llm - iAkashPaul/Portal
README
# Portal: Hole in your Palm
>Theme : AI for India [Hasgeek Hackathon Submission](http://has.gy/W7bW)
Android app for the ```Hole in your Palm``` project, making LLMs accessible on-device using the [🦙 Llama.cpp project](https://github.com/ggerganov/llama.cpp)
## Android App
[APK](https://drive.google.com/file/d/1NDtp3siDcvRpAMn4kyIDt8QNzBNq0GC_/view?usp=drivesdk)
Coming soon to the Play Store & Indus AppStore
## Video
[Youtube](https://youtu.be/EY1_139uccc)
## Slides
[PPT](https://docs.google.com/presentation/d/1vd319CIffL_MaXt-FbTPDNMprASr_-WR/edit?usp=sharing&ouid=116547059781397399357&rtpof=true&sd=true)
## Background
> Through this initiative I’d like to revisit the almost 20 year old experiment of [‘The Hole in the Wall’](https://en.wikipedia.org/wiki/Minimally_invasive_education) project by Sugata Mitra, which incorporated minimally invasive education by having a computer kiosk embedded in a wall at less fortunate locations across India.
Given the increased prevalence of smartphones throughout the country but with often limited internet/network coverage in far flung locations or restricted internet access. This along with families deciding on activating/deactivating internet plans to avoid paying high fees/rental plans leads to the friction of thinking about cost before getting answers to a question/query at any point of time.
I propose creating an Android app which can incorporate few of the smaller LLMs for tasks like general QnA, coding help, translation etc. Thereby making educational resources more accessible & available to users without depending on costly cloud GPUs or active internet connections. Adding modalities like STT/TTS on device along with possible image generation could be extensions to this app down the road.
## Roadmap
* Extend llama.cpp's example Android app with OpenCL support to make use CPU & GPU, already verified 1.6b, 3b & 7b models running at decent tok/s at Q4/Q8 quantization on CPU via termux & this app
* Add support to incorporate custom data & vector-db for RAG based QnA on-device & toggling to code assistance tasks
* Fix bugs with message history & unloading models completely
## Screenshots
### Long term goals
* Tool/Function calling via grammar & file output generation/sharing
* Look at incorporating on-device STT or Whisper based STT model for translation into English from regional language directly & respond via TTS with VAD based interruptibility
* Android Assistant integration, widget on home/lock screens & notification shade