Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/rapidai/rapidocrvl

Inference Service for PaddleOCR VL 0.9B
https://github.com/rapidai/rapidocrvl

Last synced: 13 days ago
JSON representation

Inference Service for PaddleOCR VL 0.9B

Awesome Lists containing this project

README 

          
# PaddleOCR-VL 0.9B Pure Go Runtime



Experimental pure Go inference runtime for the PaddleOCR-VL 0.9B Hugging Face

checkpoint.



This project is pure Go at runtime: no Python, no PaddlePaddle, no PyTorch, no

Transformers, and no external inference subprocess.



Current scope:



- reads `config.json`

- reads Hugging Face `model.safetensors` and sharded

  `model.safetensors.index.json`

- supports BF16/F16/F32 tensor conversion

- implements the ERNIE text decoder path in Go

- implements the PaddleOCR-VL image preprocessing path in Go

- implements the vision Transformer encoder path in Go

- projects visual features into text hidden states and replaces image tokens

- provides a token-id based greedy generation CLI



Not complete yet:



- full SentencePiece parity

- exact bicubic resize parity with Pillow

- GPU/SIMD kernels



## Usage



Download the model files:



```powershell

go run ./cmd/paddleocrvl-download D:\models\PaddleOCR-VL

```



Downloader also supports `-out`, `-base-url`, `-timeout`, and `-json` for

mirrors and automated setup. JSON output includes per-file bytes and SHA256.



Convert safetensors to GGUF manually:



```powershell

go run ./cmd/paddleocrvl-convert -model-dir D:\models\PaddleOCR-VL

```



The converter logs progress by default; use `-progress=false` for quiet runs

and `-json` for a machine-readable conversion summary. Conversion summaries

include output path, bytes, SHA256, source, F32, and quantized tensor counts.

Use `-gomaxprocs` and

`-gc-percent` during large conversions to tune CPU and GC behavior.

Custom `-out` parent directories are created automatically.



Build a quantized GGUF for faster startup and smaller text weights:



```powershell

go run ./cmd/paddleocrvl-convert -model-dir D:\models\PaddleOCR-VL -quant q8

go run ./cmd/paddleocrvl-convert -model-dir D:\models\PaddleOCR-VL -quant q6

go run ./cmd/paddleocrvl-convert -model-dir D:\models\PaddleOCR-VL -quant q4

```



All runtime loaders prefer GGUF. If it is missing and safetensors weights exist

as either `model.safetensors` or `model.safetensors.index.json` shards, the

loader converts to GGUF automatically before inference. With `-quant q8`,

`-quant q6`, or `-quant q4`, loaders prefer `model-q8.gguf`, `model-q6.gguf`,

or `model-q4.gguf`; if missing, the loader converts directly to a quantized

GGUF before inference.



CLI, server, and local benchmark loaders log auto-conversion and text-weight

loading progress during first load.



Inspect model metadata without loading all weights:



```powershell

go run ./cmd/paddleocrvl-inspect D:\models\PaddleOCR-VL

go run ./cmd/paddleocrvl-inspect -json D:\models\PaddleOCR-VL

```



Inspect output reports the active weight format, quantization, path, and file

size directly. JSON and text output also include per-weight-file SHA256 hashes.



Then run token-id inference:



```powershell

go run ./cmd/paddleocrvl-go `

  -model-dir D:\models\PaddleOCR-VL `

  -tokens 100273,1234,5678 `

  -max-new-tokens 16

```



Image run:



```powershell

go run ./cmd/paddleocrvl-go `

  -model-dir D:\models\PaddleOCR-VL `

  -image D:\docs\page.png `

  -tokens 100273,101305,100295,101306,1234 `

  -max-new-tokens 64

```



When `-image` is set, a single `100295` image placeholder is expanded to the

number of projected visual tokens for that image.



Official-style task prompt:



```powershell

go run ./cmd/paddleocrvl-go `

  -model-dir D:\models\PaddleOCR-VL `

  -image D:\docs\page.png `

  -task ocr `

  -max-new-tokens 1024 `

  -decode-generated-only `

  -skip-special

```



`-task` accepts `ocr`, `table`, `formula`, and `chart`.



HTTP inference service:



```powershell

go run ./cmd/paddleocrvl-server -model-dir D:\models\PaddleOCR-VL -addr 127.0.0.1:8080

```



Admin console and API documentation:



- open `http://127.0.0.1:8080/admin` for first-run admin initialization,

  login, model-path settings, API key issuance, quota management, and service

  overview.

- open `http://127.0.0.1:8080/doc` for human-readable API documentation.

- open `http://127.0.0.1:8080/doc/openapi.json` for machine-readable OpenAPI

  3.1, useful for other AI agents or HTTP clients. Operations include stable

  `operationId` values, `status`/`inference` tags, and request examples for

  OCR, JSON generation, and batch calls, plus success/error response examples.

- open `http://127.0.0.1:8080/doc/llms.txt` for a concise plaintext integration

  guide intended for AI agents.



The first admin initialization creates a default API key and shows the key only

once. Additional keys are generated by the server in the admin console. Keys can

be named or renamed, disabled, deleted, assigned a request quota, and reset for a new quota

period. Keys can also have a per-minute rate limit to protect local inference

capacity. Keys can be rotated when leaked; the old key is revoked immediately

and the new plaintext key is shown once. The console shows recent usage time and

client IP per key. A quota or rate limit of `0` means unlimited. Inference

endpoints require:



```text

Authorization: Bearer 

```



The admin console keeps an in-memory recent-call audit log with API key, client

IP, path, HTTP status, latency, authentication/quota errors, and JSON error

messages returned by inference handlers. It stores request metadata only, not

uploaded images or prompts. The recent audit log can be exported as CSV or

cleared from the admin console before a focused integration test.



Admin configuration can be exported and restored from the Settings page. Backups

include admin password hashes, API key hashes/limits, model path settings, and

post-processing path settings, but never plaintext API keys or passwords.

Admin pages and admin JSON endpoints use `no-store`, frame-deny, MIME nosniff,

same-origin referrer, and a restrictive content security policy.

Admin login applies a short failed-attempt lockout per client IP.

Admin write endpoints reject cross-origin `Origin`/`Referer` headers.



Useful service flags:



- `-admin-config paddleocrvl-admin.json` sets the admin console config path.

- `-timeout 10m` sets a per-request timeout.

- `-shutdown-timeout 30s` controls graceful shutdown after SIGINT/SIGTERM.

- `-request-limit 134217728` caps request body size.

- `-multipart-memory 33554432` caps memory used while parsing multipart forms.

- `-max-new-limit 4096` caps generated tokens per request.

- `-max-input-tokens 0` caps prompt/input tokens per request; `0` disables the cap.

- `-max-batch-size 0` caps `/v1/batch` item count; `0` disables the cap.

- generation options reject negative `max_new_tokens`, `temperature`, and `top_k`.

- `-concurrency 1` controls concurrent inference slots.

- `-gomaxprocs 0` controls Go CPU worker threads; `0` keeps the current value.

- `-gc-percent 0` controls Go GC target; `0` keeps current value and `-1` disables GC.

- `-preload-vision` loads vision weights during startup.

- `-warmup` runs one text-token warmup during startup.

- `-quant q8` enables row-wise int8 text-weight quantization.

- `-quant q6` enables row-wise int6 text-weight quantization.

- `-quant q4` enables row-wise int4 text-weight quantization.

- `-quant auto` picks an existing `model-q4.gguf`, then `model-q6.gguf`,

  then `model-q8.gguf`, then `model.gguf`; if only safetensors exists, it

  builds `model-q6.gguf`.



Windows service control:



```powershell

paddleocrvl-server.exe service install -model-dir "C:\ProgramData\PaddleOCRVL\models" -admin-config "C:\ProgramData\PaddleOCRVL\paddleocrvl-admin.json" -addr 127.0.0.1:8080

paddleocrvl-server.exe service start

paddleocrvl-server.exe service stop

paddleocrvl-server.exe service uninstall

```



## Installers



GitHub Actions release workflow:



- `.github/workflows/release.yml` runs tests, then builds Windows NSIS,

  macOS PKG, and Linux AppImage artifacts.

- Windows builds `amd64` and `arm64` installers.

- macOS builds one universal PKG containing x86_64 and arm64 binaries.

- Linux builds AppImage artifacts for `x86_64` and `aarch64`.

- Push a tag like `v1.0.0`, or run the workflow manually with a version input.



Windows NSIS installer:



```powershell

.\packaging\windows\build-nsis.ps1 -Version 1.0.0

```



The installer packages `paddleocrvl-client.exe` and `paddleocrvl-server.exe`.

On install it registers `PaddleOCRVLService` as an automatic NT service and

starts it. On uninstall it stops and removes the service before deleting files.

The service uses `C:\ProgramData\PaddleOCRVL\models` as its default model

directory and `C:\ProgramData\PaddleOCRVL\paddleocrvl-admin.json` for admin

state. Put the model files in the default model directory before installing, or

the installer will fail when it verifies that the service can start.



macOS universal PKG installer:



```sh

VERSION=1.0.0 sh ./packaging/macos/build-pkg.sh

```



Linux AppImage:



```sh

VERSION=1.0.0 ARCH=x86_64 sh ./packaging/linux/build-appimage.sh

VERSION=1.0.0 ARCH=aarch64 sh ./packaging/linux/build-appimage.sh

```



The Linux script uses `linuxdeploy` with the GTK plugin so the Wails/WebKitGTK

client gets a relocatable AppDir before AppImage output.



The PKG contains a universal Wails client app, a universal

`paddleocrvl-server`, and a LaunchDaemon named

`com.znsoft.paddleocrvl.service`. Post-install scripts load and kickstart the

service. The default model directory is

`/Library/Application Support/PaddleOCRVL/models`; it must contain model files

before installation so the LaunchDaemon can start successfully. The package

also installs `/usr/local/paddleocrvl/uninstall.sh`, which removes the

LaunchDaemon, server binary, and client app while keeping model/admin data.

- `-quant auto-fast` prefers/builds Q4 for speed and size.

- `-quant auto-quality` prefers/builds Q8 for quality.

- `-backend cpu|auto|vulkan` selects compute backend. `vulkan` is strict for

  loader/interface probing and fails startup if Vulkan is unavailable; `auto`

  falls back to CPU. The current pure-Go Vulkan layer exposes loader/device/driver

  status plus registered matvec/QKV/SwiGLU compute-kernel plans and model-shaped

  dispatch grids/summaries/stage execution graph in `/stats`. Kernel metadata

  includes descriptor bindings, push-constant ABI, and comparable pipeline cache

  keys. Command plans also expose per-dispatch resource bindings, pipeline

  layout plans, shader-module plans, descriptor-set layout/index plans,

  push-constant payloads, descriptor write plans, command-buffer recording

  plans, dispatch-batch bind-reuse plans, buffer barrier plans, buffer

  allocation plans, host/device transfer plans, byte ranges, and aligned

  storage-buffer allocation sizes plus

  descriptor-pool, command-pool,

  queue-submit, timeline, fence, pipeline-cache, pipeline-lifecycle, and

  validation plans for future descriptor-set writes. `/stats` and `-stats-only

  -json` expose `vulkan_command_plan_valid` and `vulkan_command_plan_error`.

  It reports CPU as the active tensor backend until GPU command submission is

  enabled.



Health and stats:



```powershell

curl http://127.0.0.1:8080/health

curl http://127.0.0.1:8080/ready

curl http://127.0.0.1:8080/stats

```



`/ready` reports whether the model, tokenizer, and inference slots are

initialized. `/stats` includes uptime, in-flight requests, request counters, failures,

cancel count, generated token count, average latency, average queue wait, last error, requested/effective

quantization, loaded `weight_path`, `weight_sha256`, Go memory stats, and model

dimensions. `weight_source` is `existing_gguf` when loading a GGUF file directly

and `converted_safetensors` when the loader converted original safetensors.

`load_stats` reports milliseconds spent in weight open/auto-convert, text preload,

runtime quantization, and total load. The `cache` section includes reusable task

prompts, tokenizer cache stats, and runtime vision position-table cache stats.



Batch JSON inference:



```powershell

curl -X POST http://127.0.0.1:8080/v1/batch ^

  -H "Authorization: Bearer " ^

  -H "Content-Type: application/json" ^

  -d "{\"requests\":[{\"prompt\":\"<|begin_of_sentence|>hello\",\"max_new_tokens\":1},{\"task\":\"ocr\",\"image_path\":\"D:\\docs\\page.png\",\"decode\":true,\"decode_generated_only\":true,\"skip_special\":true}]}"

```



Batch responses include `items`, aggregate `generated_tokens`, and per-request

`responses`. Batch items run concurrently up to the server `-concurrency`

slot limit while preserving response order.



HTTP benchmark:



```powershell

go run ./cmd/paddleocrvl-bench `

  -url http://127.0.0.1:8080/v1/generate `

  -n 20 `

  -c 1 `

  -prompt "<|begin_of_sentence|>hello" `

  -max-new-tokens 1 `

  -temperature 0 `

  -top-k 0 `

  -batch-size 1

```



Local runtime benchmark, bypassing HTTP:



```powershell

go run ./cmd/paddleocrvl-bench `

  -mode local `

  -model-dir D:\models\PaddleOCR-VL `

  -n 5 `

  -c 1 `

  -prompt "<|begin_of_sentence|>hello" `

  -max-new-tokens 1

```



Kernel microbenchmarks:



```powershell

go test ./internal/tensor -bench "MatVec|Fused|Quantize" -run "^$"

go test ./internal/model -bench "Sample|TopK" -run "^$"

go test ./internal/tokenizer -bench Encode -run "^$"

```



Set `-batch-size` above `1` to benchmark `/v1/batch`; the default `/v1/generate`

URL is automatically rewritten to `/v1/batch`. Benchmark output includes

request, item, and generated token throughput, plus `last_error` when failures

occur. Add `-json` for machine-readable benchmark results with CPU and memory

snapshots plus mode/backend/quantization/weight path/source context.



JSON request with an image path:



```powershell

curl -X POST http://127.0.0.1:8080/v1/generate ^

  -H "Content-Type: application/json" ^

  -d "{\"task\":\"ocr\",\"image_path\":\"D:\\docs\\page.png\",\"max_new_tokens\":1024,\"decode\":true,\"decode_generated_only\":true,\"skip_special\":true}"

```



JSON request with base64 image data:



```powershell

curl -X POST http://127.0.0.1:8080/v1/generate ^

  -H "Content-Type: application/json" ^

  -d "{\"task\":\"ocr\",\"image_base64\":\"\",\"max_new_tokens\":1024,\"eos_token_ids\":[2],\"decode\":true,\"decode_generated_only\":true,\"skip_special\":true}"

```



Base64 and multipart image requests are decoded in memory and passed directly

to the Go image preprocessing path. JSON generation responses include

`tokens`, `prompt_tokens`, and `generated_tokens`; `text` is included when

decoding is requested.



Multipart upload:



```powershell

curl -X POST http://127.0.0.1:8080/v1/ocr ^

  -F "task=ocr" ^

  -F "image=@D:\docs\page.png"

```



Desktop client:



```powershell

cd cmd\paddleocrvl-client

wails dev

```



The Wails client lets you set API URL and API Key, check `/ready`, open `/doc`,

choose and preview one or more images, manage the selected image queue with

per-image status, upload them to the multipart OCR endpoint, optionally continue a batch after per-image

errors, copy or save results, export batch results as JSON, set a request timeout, cancel in-flight

requests, inspect each batch result row, view the decoded text plus raw JSON, and reopen the last 10 local

runs from history. Client settings can be imported/exported as JSON. If the API

URL has no path, the client appends `/v1/ocr`. API Key is sent as both

`Authorization: Bearer ` and `X-API-Key`.



Prompt inference:



```powershell

go run ./cmd/paddleocrvl-go `

  -model-dir D:\models\PaddleOCR-VL `

  -prompt "<|begin_of_sentence|>hello" `

  -max-new-tokens 16 `

  -decode-generated-only

```



Load, convert, quantize, and print memory stats without generating:



```powershell

go run ./cmd/paddleocrvl-go -model-dir D:\models\PaddleOCR-VL -quant auto -stats-only

go run ./cmd/paddleocrvl-go -model-dir D:\models\PaddleOCR-VL -quant auto-fast -verify-only -verify-vision

```



Add `-json` to `paddleocrvl-go` for machine-readable stats, verification, or

generation output.

Stats and verification output include the loaded `weight_path` and

`weight_source`. Stats output also includes `load_stats`.



`-stats-only` also prints CPU features and backend details. On Linux, Vulkan

backend details include discovered ICD manifests and driver API versions; on

Windows, the loader reports the Vulkan instance API version when available.

CPU details include `num_cpu` and `gomaxprocs` for throughput tuning. Memory

details include heap, system allocation, object, and GC counters.

`-verify-only` exits after text weights load; add `-verify-vision` to force

vision weight loading too.



Tokenizer support is based on `tokenizer.json`: added special tokens, BPE merge

ranks, byte fallback, and the model's space replacement decoder are implemented.



The runtime uses Go CPU parallelism for large linear layers and batched vision

projections. Set `GOMAXPROCS` to control CPU worker count.



On Windows, if a default `go build ./cmd/...` output executable is locked by a

running process, build to an explicit path:



```powershell

go build -o .\.gocache\bin\paddleocrvl-server.exe ./cmd/paddleocrvl-server

```



Generation defaults to greedy decoding. Set `-temperature` above `0` to sample;

combine with `-top-k` and `-seed` for reproducible sampled runs.



Acceleration work in tree:



- row-wise int8 quantized text projection/MLP path (`-quant q8`)

- row-wise int6 quantized text projection/MLP path (`-quant q6`)

- row-wise int4 quantized text projection/MLP path (`-quant q4`)

- quantized GGUF conversion/loading path (`model.safetensors` -> `model-q8.gguf` / `model-q6.gguf` / `model-q4.gguf`)

- row-streamed GGUF quantized conversion to reduce peak memory during first load

- reusable safetensors row buffers during GGUF quantized conversion to avoid

  per-tensor block reallocations

- GGUF quantized conversion reuses scale and quantized-row buffers across

  tensors to reduce multi-tensor conversion churn

- reusable GGUF F32 row buffers during runtime Q8/Q6/Q4 quantization from

  `model.gguf`

- lower-peak GGUF quantized tensor loading

- single-read GGUF Q8/Q6/Q4 tensor loading with shared scale/data backing

- existing GGUF load path opens candidate files directly and skips a separate

  pre-open stat call

- GGUF metadata open reuses shape backing storage and zero-copy string views to

  reduce first-load allocation count

- safetensors fallback probing is lazy and cached during weight selection

- row-streamed F32 GGUF -> Q8/Q6/Q4 runtime quantization when only `model.gguf`

  is available

- pre-sized runtime text/vision weight maps to reduce loader rehash churn

- release text-weight map entries after caching layer pointers to reduce

  runtime map scanning and retained references

- release vision-weight map entries after caching vision layer pointers for the

  same reason

- unrolled safetensors BF16/F16 decode paths for first-load conversion

- wider safetensors F16 decode loop for lookup-table conversion

- fused Q8/Q6/Q4 SwiGLU MLP path

- fused Q/K/V attention projection path for F32/Q8/Q6/Q4 weights

- fused residual-add + RMSNorm path in the text decoder, including next-layer

  pre-normalization after MLP residuals

- vision residual LayerNorm path uses the faster two-pass AddInPlace+LayerNorm

  sequence on Go CPU kernels

- per-token text RoPE table reuse across decoder layers to avoid repeated

  sin/cos table builds

- `head_dim=128` and `head_dim=64` attention-score dot-product fast paths

- single-token `head_dim=128` and `head_dim=64` text attention-score fast paths

- fused text KV-cache score + Softmax + value path for stable 2-token decoding

- `head_dim=128` and `head_dim=64` attention value aggregation fast paths for

  text KV cache and vision attention

- short-context text KV and vision value aggregation fast paths for

  `head_dim=64`

- unrolled RMSNorm hot path

- unrolled greedy Argmax path

- unrolled Softmax path

- specialized length-5/6/7/8 Softmax paths for early attention steps

- wider Q4/Q6 dot-product decode loops

- Q8/Q6/Q4 decode lookup tables for quantized dot products

- single-pass Q8 triplet dot product for fused Q/K/V projection

- wider Q8/Q6/Q4 row quantization loops for safetensors -> GGUF conversion

- wider F32/Q8 dot-product loops

- parallel GELU over large vision MLP row batches

- no-sort full-vocab sampling path and unsorted heap-backed top-k sampling

- full-vocab sampling max pass avoids per-logit temperature multiplication

- full-vocab `temperature=1` sampling fast path skips per-logit temperature

  scaling

- top-k sampling scan skips eight low-score logits per branch before heap work

- sampled-token weighted pick loop checks eight weights per iteration

- short EOS-id lists use direct comparisons in the generation loop

- greedy decoding skips RNG initialization when sampling is disabled

- zero-token text generation returns before allocating KV/scratch state

- zero-token image generation returns before image decode/vision encoding

- multimodal RoPE position construction tracks the current maximum position

  incrementally instead of rescanning previous tokens for every image block

- multimodal RoPE position buffers are reused from generation scratch during

  image generation to avoid per-request position slice allocation

- server greedy requests skip random seed generation when sampling is disabled

- CLI greedy requests skip random seed generation when sampling is disabled

- single-item batch requests avoid unused response/error slice allocation

- health, ready, and stats endpoints use fixed response structs instead of

  dynamic maps for lower monitoring overhead

- short server error responses use a stack buffer before falling back to heap

- read-only tokenizer encode cache path for server/CLI prompts to avoid cached

  slice copies

- tokenizer special-token matching keeps ordered token/id entries to avoid map

  lookups on prefix matches

- empty tokenizer encode/decode inputs return before cache locks or builders

- shared RGBA resize/preprocess bilinear-index backing to reduce hot-path

  allocations

- exact-size RGBA preprocessing skips bilinear resize and extracts patches

  directly in parallel

- base 27x27 vision position tables reuse a cached row view instead of

  interpolating or allocating

- vision position and RoPE cache hits use read locks so concurrent image

  requests do not serialize on monitoring/cache reads

- BF16/F16 safetensors -> quantized GGUF conversion reuses a raw decode buffer

- direct BF16/F16 safetensors row quantization during runtime load reuses raw

  decode storage across tensors

- safetensors model selection opens candidate files directly and preserves bad

  single-file errors instead of probing with extra stat calls

- incremental vision projection block indexing to reduce per-row integer

  division/modulo work

- pooled attention score buffers and lower-overhead KV cache appends

- packed per-layer text attention score buffers to reduce first-request and

  long-context allocation count

- generation scratch and KV cache getters have direct fallback allocation paths

  for pool-miss robustness without changing the pooled hot path

- pre-sized tokenizer decode buffers

- tokenizer byte-fallback decode fast paths for pure byte streams and single

  byte tokens

- mixed tokenizer byte-fallback decode uses one pass with lazy string builder

- tokenizer Unicode byte fallback encodes UTF-8 into a stack buffer instead of

  allocating `[]byte` per unknown rune

- pooled vision projection scratch buffers and compact vision scratch row storage

- on-demand pooled vision embedding buffer in the image encoder to avoid a

  separate large embedding allocation during `EncodeImage`

- vision embedding rows share the main vision scratch backing block to reduce

  first image encode allocation count

- cached vision RoPE tables by image grid and head dimension to avoid repeated

  per-image table allocations

- compact vision RoPE table storage with one backing block per axis table

- fused RGBA resize + patch extraction precompute tables to cut image

  preprocessing allocations

- current-goroutine first worker for RGBA resize and patch extraction to reduce

  scheduler allocation overhead

- adaptive vision projection worker limits to reduce scheduler overhead on small

  image grids while keeping full parallelism for large projections

- adaptive batched row-projection worker limits for small per-patch vision

  embeddings

- batch endpoint holds one inference slot across batch execution to reduce scheduler churn

- CPU parallel matrix-vector/matrix-row kernels with unrolled dot products

- CPU feature reporting in `/stats` (`arm64` reports NEON-capable target)

- Vulkan loader probing on Windows and Linux exposed in `/stats`

- Vulkan operator registry for f32/Q8/Q6/Q4 matvec and fused QKV/SwiGLU kernel

  plans with 256-thread reductions

- current-model Vulkan dispatch plans and expanded dispatch/weight-byte summaries

  exposed in CLI/server JSON stats

- current-model Vulkan execution graph groups text and vision stages for future

  command-buffer submission

- Vulkan kernel ABI metadata exposes descriptor bindings, push constants, and

  activation input/output byte estimates

- Vulkan pipeline cache keys and unique pipeline counts are exposed for future

  layout/pipeline reuse

- Vulkan pipeline precreation plans expose unique pipeline keys, stage, reference

  counts, and expanded dispatch counts

- Vulkan pipeline plans include layout indices built from descriptor signatures

  and push-constant sizes for future pipeline-layout reuse

- Vulkan pipeline plans include shader-module indices and shader module plans

  with entry point, source hash, local size, tile columns, specialization

  constants, and pipeline refs

- Vulkan command plans map model ops to pipeline slots with dispatch grids and

  repeat counts for future command-buffer recording

- Vulkan command plans include per-command input/weight/scale/output resources,

  descriptor binding indices, access modes, and byte sizes for future

  descriptor-set writes

- Vulkan command plans include per-command descriptor-set layout/index plans and

  concrete push-constant payloads for rows/columns

- Vulkan command plans include storage-buffer descriptor write records and

  resource/write counts for each model-shaped dispatch plan

- Vulkan command plans expose unique pipeline-layout plans with storage-buffer

  bindings and pipeline reference counts

- Vulkan command plans emit future command-buffer recording records for

  bind-pipeline, bind-descriptor-set, push-constants, and dispatch steps

- Vulkan command plans include dispatch-batch grouping for consecutive commands

  that can reuse pipeline binding while updating descriptor sets and push

  constants

- Vulkan command plans include buffer memory barrier plans for host-to-compute

  reads and compute-write-to-compute-read handoff around planned dispatches

- Vulkan command plans include per-resource buffer usage, memory property, and

  aligned total buffer-byte allocation plans for future device-memory binding

- Vulkan command plans include host-to-device upload and device-to-host readback

  transfer plans with byte totals for future staging-buffer execution

- Vulkan command plans include descriptor-pool sizing, compute command-pool

  sizing, and single-submit queue plans for future queue submission

- Vulkan command plans include timeline semaphore values and fence reset/wait

  plans for future queue completion tracking

- Vulkan command plans include compute pipeline cache keys, reuse counts, and

  pipeline create/destroy lifecycle plans for future cache-backed pipeline setup

- Vulkan command plans have a pure-Go validator for pipeline/layout/shader,

  descriptor/resource, dispatch, sync, and lifecycle consistency

- Vulkan command-plan validation status is exposed as

  `vulkan_command_plan_valid`/`vulkan_command_plan_error` in server and CLI JSON

  stats

- Linux Vulkan ICD manifest discovery exposed in `/stats` and `-stats-only`

- Linux Vulkan ICD relative `library_path` entries are resolved against the

  manifest directory for clearer driver reporting

- GGUF conversion/loading path (`model.safetensors` -> `model.gguf`)