https://github.com/sdsc-ordes/sdschat

https://github.com/sdsc-ordes/sdschat

ai containers

Last synced: about 15 hours ago
JSON representation

• Host: GitHub
• URL: https://github.com/sdsc-ordes/sdschat
• Owner: sdsc-ordes
• License: apache-2.0
• Created: 2023-06-27T14:38:10.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-08-08T09:10:40.000Z (over 1 year ago)
• Last Synced: 2023-12-14T09:49:21.773Z (11 months ago)
• Topics: ai, containers
• Language: Shell
• Homepage:
• Size: 18.6 KB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# sdschat

Fastchat implementation for SDSC. 

## How to run this app in SDSC

```

runai submit sdschat-$USER -i caviri/sdschat:latest --service-type=portforward --port 8000:8000 --port 7860:7860 --attach --interactive --node-type "A100" -g 0.5

```

## How to build this image?

```

docker build -t caviri/sdschat:latest . 

```

```

docker push caviri/sdschat:latest

```

```

docker run -it --rm -p 7860:7860 -p 8000:8000 --gpus all caviri/sdschat:latest 

```

Add as many huggingface models as you want to use in `MODELS`. Separate each model by a comma. 

```

docker run -it --rm -p 7860:7860 -p 8000:8000 -e MODELS="lmsys/fastchat-t5-3b-v1.0,lmsys/vicuna-7b-v1.3" --gpus all caviri/sdschat:root 

```

If you want to use the OPENAI API. Use the variable: `-e OPENAIMODEL="nomic-ai/gpt4all-13b-snoozy"`