https://github.com/oaklight/xinference-singularity
scripts to run xinference via singularity
https://github.com/oaklight/xinference-singularity
Last synced: about 2 months ago
JSON representation
scripts to run xinference via singularity
- Host: GitHub
- URL: https://github.com/oaklight/xinference-singularity
- Owner: Oaklight
- Created: 2025-03-21T18:07:37.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2025-06-03T21:11:17.000Z (about 1 year ago)
- Last Synced: 2025-09-18T14:13:29.664Z (9 months ago)
- Language: Shell
- Size: 10.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# XInference Singularity
[中文版本](README_zh.md)
XInference Singularity is a repository containing a set of scripts designed to manage inference workflows using Singularity containers. The repository allows you to start and stop the inference server, download models or datasets from Hugging Face, and update the inference container.
## Files
- start-xinference.sh
Launches the inference container using Singularity with a default version (v1.2.2) and configured local directory.
- stop-xinference.sh
Stops the inference server by terminating the process listening on port 9997.
- hfd.sh
Downloads models or datasets from Hugging Face using aria2c or wget with support for authentication and resuming downloads.
- start-xinference-instance.sh
Starts a Singularity instance and runs the xinference-local command within the instance, logging output to a designated log file.
- update-xinference.sh
Updates the inference container by pulling a Docker image (default version v1.2.0) and converting it for use with Singularity.
## Usage
### Starting the Inference Server
Run:
```bash
./start-xinference.sh
```
Or, to run within an instance:
```bash
./start-xinference-instance.sh
```
### Stopping the Inference Server
Run:
```bash
./stop-xinference.sh
```
### Downloading Models/Datasets
Run:
```bash
./hfd.sh [options]
```
For detailed options, run:
```bash
./hfd.sh -h
```
### Updating the Inference Container
Run:
```bash
./update-xinference.sh
```
_Note: The update script uses a default version of v1.2.0 for updating the container image. Adjust the LATEST_VERSION environment variable as needed._
## Prerequisites
Ensure that Singularity is installed. Additionally, the following dependencies must be available on your system:
- aria2c or wget
- curl
- git and git-lfs
Environment variables can be used to customize paths, instance names, version numbers, and other configurations.