https://github.com/freinold/gliner-api
A minimal FastAPI app serving GLiNER models
https://github.com/freinold/gliner-api
entity-detection fastapi gliner ner
Last synced: 4 months ago
JSON representation
A minimal FastAPI app serving GLiNER models
- Host: GitHub
- URL: https://github.com/freinold/gliner-api
- Owner: freinold
- License: mit
- Created: 2025-05-25T08:56:37.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-17T15:30:07.000Z (about 1 year ago)
- Last Synced: 2025-06-17T15:31:09.034Z (about 1 year ago)
- Topics: entity-detection, fastapi, gliner, ner
- Language: Python
- Homepage:
- Size: 82 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# gliner-api
## Easily configurable API & frontend providing simple access to dynamic NER models
[](https://huggingface.co/spaces/bigscience/bloom-demo)
[](.python-version)
[](https://github.com/urchade/GLiNER)
[](https://github.com/freinold/gliner-api/blob/main/LICENSE)
[](https://github.com/freinold/gliner-api/actions/workflows/github-code-scanning/codeql)
[](https://github.com/freinold/gliner-api/actions/workflows/docker-release.yml)
[](https://scorecard.dev/viewer/?uri=github.com/freinold/gliner-api)
## Features
- **FastAPI** backend for serving GLiNER models (NER).
- **Gradio** frontend (optional) for interactive use.
- **Prometheus** metrics endpoint (`/metrics`).
- **Configurable** via YAML, CLI, or environment variables.
- **Docker** and **Docker Compose** support.
- **ONNX** inference support (including quantized models).
- **API key** authentication (optional).
- **Custom metrics port** and enable/disable option for Prometheus metrics.
---
## Documentation
For detailed documentation, [see DeepWiki](https://deepwiki.com/freinold/GLiNER-API) (⚠️ AI-generated)
---
## Live Demo
You can try the live demo of the GLiNER API container in it's Huggingface Space: [GLiNER API Demo](https://huggingface.co/spaces/freinold/gliner-api).
It uses a minimally changed image to make it work in the Huggingface Space environment.
---
## Usage
### Run with Docker
You can either build the container yourself or use a prebuilt image from GitHub Container Registry.
#### Run prebuilt container (recommended)
**CPU version:**
```bash
docker run \
-p 8080:8080 \
-p 9090:9090 \
-v $(pwd)/config.yaml:/app/config.yaml \
-v $HOME/.cache/huggingface:/app/huggingface \
ghcr.io/freinold/gliner-api:latest
```
**GPU version:**
```bash
docker run \
--gpus all \
-p 8080:8080 \
-p 9090:9090 \
-v $(pwd)/config.yaml:/app/config.yaml \
-v $HOME/.cache/huggingface:/app/huggingface \
ghcr.io/freinold/gliner-api-gpu:latest
```
**Mounting volumes:**
- `-v $(pwd)/config.yaml:/app/config.yaml` mounts your config file (edit as needed)
- `-v $HOME/.cache/huggingface:/app/huggingface` mounts your Huggingface cache for faster model loading
#### Build and run locally (CPU version)
```bash
docker build \
-f cpu.Dockerfile \
--build-arg IMAGE_CREATED="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--build-arg IMAGE_REVISION="$(git rev-parse HEAD)" \
--build-arg IMAGE_VERSION="$(git describe --tags --always)" \
-t gliner-api .
docker run --rm \
-p 8080:8080 \
-p 9090:9090 \
-v $(pwd)/example_configs/general.yaml:/app/config.yaml \
-v $HOME/.cache/huggingface:/app/huggingface \
gliner-api
```
PowerShell version
```powershell
docker build `
-f cpu.Dockerfile `
--build-arg IMAGE_CREATED="$(Get-Date -Format 'yyyy-MM-ddTHH:mm:ssZ')" `
--build-arg IMAGE_REVISION="$(git rev-parse HEAD)" `
--build-arg IMAGE_VERSION="$(git describe --tags --always)" `
-t gliner-api .
docker run --rm `
-p 8080:8080 `
-p 9090:9090 `
-v "$PWD/example_configs/general.yaml:/app/config.yaml" `
-v "$HOME/.cache/huggingface:/app/huggingface" `
gliner-api
```
#### Build and run locally (GPU version)
```bash
docker build \
-f gpu.Dockerfile \
--build-arg IMAGE_CREATED="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--build-arg IMAGE_REVISION="$(git rev-parse HEAD)" \
--build-arg IMAGE_VERSION="$(git describe --tags --always)" \
-t gliner-api-gpu .
docker run --rm \
--gpus all \
-p 8080:8080 \
-p 9090:9090 \
-v $(pwd)/example_configs/general.yaml:/app/config.yaml \
-v $HOME/.cache/huggingface:/app/huggingface \
gliner-api-gpu
```
PowerShell version
```powershell
docker build `
-f gpu.Dockerfile `
--build-arg IMAGE_CREATED="$(Get-Date -Format 'yyyy-MM-ddTHH:mm:ssZ')" `
--build-arg IMAGE_REVISION="$(git rev-parse HEAD)" `
--build-arg IMAGE_VERSION="$(git describe --tags --always)" `
-t gliner-api-gpu .
docker run --rm `
--gpus all `
-p 8080:8080 `
-p 9090:9090 `
-v "$PWD/example_configs/general.yaml:/app/config.yaml" `
-v "$HOME/.cache/huggingface:/app/huggingface" `
gliner-api-gpu
```
---
### Run with Docker Compose
Edit [`cpu.compose.yaml`](cpu.compose.yaml) / [`gpu.compose.yaml`](gpu.compose.yaml) to select the config you want (see [`example_configs`](example_configs/)).
Then run:
```bash
# For CPU version
docker compose -f cpu.compose.yaml up
# For GPU version
docker compose -f gpu.compose.yaml up
```
---
### Run the app directly
Be sure to check the [installation instructions](#installation) first.
```bash
uv run main.py [OPTIONS]
```
Or with FastAPI CLI:
```bash
fastapi run main.py --host localhost
```
#### Run options
```bash
uv run main.py --help
```
| Option | Description | Default |
| ----------------------- | --------------------------------------------------------- | ------------------------------------------------ |
| `--use-case` / `--name` | Use case for the GLiNER model (application/domain) | `general` |
| `--model-id` | Huggingface model ID ([browse models][gliner-models]) | `knowledgator/gliner-x-base` |
| `--onnx-enabled` | Use ONNX for inference | `False` |
| `--onnx-model-path` | Path to ONNX model file | `model.onnx` |
| `--default-entities` | Default entities to detect | `['person', 'organization', 'location', 'date']` |
| `--default-threshold` | Default detection threshold | `0.5` |
| `--api-key` | API key for authentication (if set, required in requests) | `null` |
| `--host` | Host address | `""` (bind to all interfaces) |
| `--port` | Port | `8080` |
| `--metrics-enabled` | Enable Prometheus metrics endpoint | `True` |
| `--metrics-port` | Port for Prometheus metrics endpoint | `9090` |
| `--frontend-enabled` | Enable Gradio frontend | `True` |
[gliner-models]: https://huggingface.co/models?library=gliner&sort=trending
---
## API & Frontend Endpoints
| Description | Path | Demo Link |
| ---------------------------- | ---------- | ------------------------------------------------------------- |
| Gradio Frontend (if enabled) | `/` | [Frontend](https://freinold-gliner-api.hf.space/) |
| API Docs (Swagger) | `/docs` | [Swagger UI](https://freinold-gliner-api.hf.space/docs) |
| API Docs (ReDoc) | `/redoc` | [ReDoc](https://freinold-gliner-api.hf.space/redoc) |
| Prometheus Metrics | `/metrics` | _(no public demo link; available on metrics port if enabled)_ |
---
## Example Request
```bash
curl -X POST "http://localhost:8080/api/invoke" -H "Content-Type: application/json" -d '{"text": "Steve Jobs founded Apple in Cupertino."}'
```
---
## Installation
**Prerequisites:**
- Python 3.13.9
- [uv](https://github.com/astral-sh/uv) (for dependency management)
**Install dependencies:**
```bash
# CPU version
uv sync --extra cpu [--extra frontend]
# GPU version
uv sync --extra gpu [--extra frontend]
```
The frontend is optional, but encouraged for interactive use.
**Install from source:**
```bash
git clone https://github.com/freinold/gliner-api.git
cd gliner-api
uv sync --extra cpu # or --extra gpu
```
---
## Configuration
You can configure the app via:
- `config.yaml` (default, see `example_configs/`)
- CLI options (see above)
- Environment variables (prefix: `GLINER_API_`)
Example configs:
- `example_configs/general.yaml` (default NER)
- `example_configs/pii.yaml` (PII detection)
- `example_configs/medical.yaml` (medical NER)
- `example_configs/general_onnx.yaml` (ONNX inference)
- `example_configs/general_onnx_quantized.yaml` (quantized ONNX)
---
## Used Frameworks & Libraries
- [FastAPI](https://fastapi.tiangolo.com/) (API backend)
- [Gradio](https://gradio.app/) (optional frontend)
- [Uvicorn](https://www.uvicorn.org/) (ASGI server)
- [Prometheus Client](https://github.com/prometheus/client_python) (metrics)
- [Huggingface Hub](https://huggingface.co/docs/huggingface_hub) (model loading)
- [PyTorch](https://pytorch.org/) (CPU/GPU inference)
- [ONNX](https://onnx.ai/) (optional, for ONNX models)
- [uv](https://github.com/astral-sh/uv) (dependency management)
---
## License
See [LICENSE](LICENSE).
---