https://github.com/cloneofsimo/adaptive-icl-labeler
https://github.com/cloneofsimo/adaptive-icl-labeler
Last synced: over 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/cloneofsimo/adaptive-icl-labeler
- Owner: cloneofsimo
- Created: 2024-01-21T15:49:08.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2024-01-21T16:07:35.000Z (over 2 years ago)
- Last Synced: 2025-01-21T06:43:24.192Z (over 1 year ago)
- Language: JavaScript
- Size: 3.02 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Adaptive In-Context-Labeler
*Using In-Context Learning Capability, LMQL, and multi-armed bandit examplars optimization for fast labeling pipeline*
Currently WIP, but feel free to try out.
Built with react, LMQL, fastap, replicate
## Demo
1. First, run frontend and backend server via
```bash
cd frontend
npm run start
cd ../backend
uvicorn main:app --reload
```
2. Run local server or use replicate backend. ("replicate:charles-dyfis-net/llama-2-13b-hf--lmtp-8bit")
* For local server, for example, run `lmql serve-model meta-llama/Llama-2-13b-chat-hf --port 8010 --cuda --load_in_4bit True`
* For replicate backend, don't run anything, instead, set osenv variables including [api token](https://replicate.com/account/api-tokens)
```
export REPLICATE_API_TOKEN= ... # YOUR API TOKEN
export MODEL_ID="meta-llama/Llama-2-13b-chat-hf"
export ENDPOINT="replicate:charles-dyfis-net/llama-2-13b-hf--lmtp-8bit"
```
3. Use the app at `http://localhost:3000/`. You would have to have your data in backend/data.csv file, with `item_id,text,processed_value,is_processed` columns. You can use `backend/seed.py` as an example dataset creation.
---
TODOS:
- [ ] Keep track of preference statistics per examplars, and use UCB based multi-armed bandit to select examplars
- [ ] Add more flexibility for saving - loading current state, maybe like json file or something
- [ ] Upload data CSV and have option to do everything in browser maybe?
- [ ] Able to edit examplars after creation
- [ ] Abstract out LMQL so that non-tech people can use it
- [ ] Put nice documentations