{"id":50567928,"url":"https://github.com/nimblemarkets/ds4go","last_synced_at":"2026-06-04T16:01:23.975Z","repository":{"id":358533042,"uuid":"1239938339","full_name":"NimbleMarkets/ds4go","owner":"NimbleMarkets","description":"Golang wrapper for DwarfStar4 (ds4)","archived":false,"fork":false,"pushed_at":"2026-05-28T03:34:33.000Z","size":302,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-28T05:15:41.855Z","etag":null,"topics":["deepseek-v4-flash","ds4"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NimbleMarkets.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-05-15T15:45:19.000Z","updated_at":"2026-05-28T03:34:24.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/NimbleMarkets/ds4go","commit_stats":null,"previous_names":["nimblemarkets/ds4go"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/NimbleMarkets/ds4go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fds4go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fds4go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fds4go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fds4go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NimbleMarkets","download_url":"https://codeload.github.com/NimbleMarkets/ds4go/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NimbleMarkets%2Fds4go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33912343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-04T02:00:06.755Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deepseek-v4-flash","ds4"],"created_at":"2026-06-04T16:01:21.899Z","updated_at":"2026-06-04T16:01:23.920Z","avatar_url":"https://github.com/NimbleMarkets.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ds4go\n\n\u003cp\u003e\n    \u003ca href=\"https://github.com/NimbleMarkets/ds4go/tags\"\u003e\u003cimg src=\"https://img.shields.io/github/tag/NimbleMarkets/ds4go.svg\" alt=\"Latest Release\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pkg.go.dev/github.com/NimbleMarkets/ds4go?tab=doc\"\u003e\u003cimg src=\"https://pkg.go.dev/badge/github.com/NimbleMarkets/ds4go?utm_source=godoc\" alt=\"GoDoc\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/NimbleMarkets/ds4go/blob/main/CODE_OF_CONDUCT.md\"\u003e\u003cimg src=\"https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg\"  alt=\"Code Of Conduct\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\n`ds4go` is a zero-CGO Go wrapper for the [`ds4` inference engine](https://github.com/antirez/ds4). Applications using `ds4go` loads a pre-built `libds4` shared library at runtime with [`github.com/ebitengine/purego`](https://github.com/ebitengine/purego).  The shared library owns hardware acceleration. Use a Metal, CUDA, or CPU build of ds4 that matches your machine and model.\n\n[`ds4`](https://github.com/antirez/ds4) itself is an inference engine focused on the [*DeepSeek v4 Flash* model](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) targeting machines with 96G or more of GPU-accessible RAM.  \n\nWe try to maintain parity with the upstream `ds4` library, wrapping its C API.  We build slightly-opinionated tools to facilitate using `ds4`.\n\n## Motivation\n\n`C` is a wonderful language for low-level, high-performance, portable code; a clean C API can be wrapped and used by other laguages.    `Golang` is a wonderful language for systems and tools development, and generally more friendly for developers, esepecially when creating networked applications.  LLMs are great at programming both.   We take the high-performance `C` engine of `ds4` and allow Golang to directly utilize it, simplifying local LLM application development.\n\n## Install\n\nInstall the `ds4go` CLI with the quick-install script, Homebrew, or the Go toolchain:\n\n```sh\n# Quick install script (Linux/macOS)\ncurl -fsSL https://nimblemarkets.github.io/ds4go/install.sh | sh\n\n# Homebrew (macOS/Linux)\nbrew install --cask nimblemarkets/tap/ds4go\n\n# or with the Go toolchain\ngo install github.com/NimbleMarkets/ds4go/cmd/ds4go@latest\n```\n\nTo use ds4go as a library:\n\n```sh\ngo get github.com/NimbleMarkets/ds4go\n```\n\nOnce the CLI is installed, fetch a prebuilt native `libds4` from GitHub Releases:\n\n```sh\nds4go install --backend auto\n```\n\nThe installer downloads from `github.com/NimbleMarkets/ds4` by default. Use\n`--repo`, `--version`, `--backend`, or `--url` to select a fork, release, build,\nor direct archive. It installs into `$DS4_DIR/lib`, defaulting to `~/.ds4/lib`.\n`--backend auto` selects `metal` on macOS arm64, `cuda` on Linux, and `cpu` elsewhere.\nIf the library is already installed and up-to-date, the installer exits successfully\nwithout re-downloading. If a different version is present, it will prompt to replace it\n(or require `--force` in non-interactive environments).\n\n`DS4_DIR` is the ds4 home directory used by ds4go tooling:\n\n```text\n$DS4_DIR/lib/      native shared libraries\n$DS4_DIR/models/   GGUF model files\n```\n\nManage curated DeepSeek V4 Flash models with:\n\n```sh\nds4go model list\nds4go model download q2-imatrix\nds4go model set q2-imatrix\n```\n\nThe default model path for commands and examples is\n`$DS4_DIR/models/ds4flash.gguf`.\n\nPlace the shared library in `~/.ds4/lib/`, `$DS4_DIR/lib/`, next to your\nexecutable, or in a `lib/` directory next to your executable. You can also point\nat it explicitly. The current working directory and the repository root are not\nsearched, to avoid loading a planted library:\n\n```sh\nexport DS4_LIB=/absolute/path/to/libds4.dylib\n# or\nexport DS4_DIR=/opt/ds4\n```\n\nPlatform defaults are:\n\n| Platform | Library |\n| --- | --- |\n| macOS | `libds4.dylib` |\n| Linux | `libds4.so` |\n| Windows | `libds4.dll` |\n\n## Usage\n\n```go\nimport ds4 \"github.com/NimbleMarkets/ds4go\"\n\nengine, err := ds4.NewEngine(ds4.EngineOptions{\n    ModelPath: \"/models/ds4flash.gguf\",\n    Backend:   ds4.BackendMetal,\n})\nif err != nil {\n    panic(err)\n}\ndefer engine.Close()\n\nsession, err := engine.NewSession(32768)\nif err != nil {\n    panic(err)\n}\ndefer session.Close()\n\nprompt, err := engine.EncodeChatPrompt(\"\", \"Explain Redis streams briefly.\", ds4.ThinkHigh)\nif err != nil {\n    panic(err)\n}\ndefer prompt.Free()\n\n_, err = ds4.Generator{Engine: engine, Session: session}.GenerateTokens(prompt, ds4.GenerateOptions{\n    MaxTokens: 128,\n    StopOnEOS: true,\n    OnToken: func(token int) {\n        text, _ := engine.TokenText(token)\n        fmt.Print(text)\n    },\n})\n```\n\n## CLI\n\n```sh\ngo run ./cmd/ds4go prompt --model ./ds4flash.gguf -p \"Explain Redis streams in one paragraph.\"\ngo run ./cmd/ds4go prompt --model ./ds4flash.gguf\n```\n\n`cmd/ds4go prompt` and the examples accept the **same arguments as the upstream `ds4` C programs**, parsed with [`pflag`](https://github.com/spf13/pflag) so options take the `--option` form. `cmd/ds4go prompt`, `examples/simple`, and `examples/chat` mirror the `ds4` CLI (`ds4_cli.c`); `examples/openai-compatible` mirrors `ds4-server` (`ds4_server.c`). Run any of them with `--help` for the full list.\n\nThe only addition with no C equivalent is `--lib`, which points at the `libds4` shared library the pure-Go wrapper loads at runtime. When empty, ds4go searches `DS4_LIB`, `$DS4_DIR/lib` (or `~/.ds4/lib`), executable-local paths, and then the platform loader path.\n\n```text\n$ ds4go help cheat\nds4go — command cheat sheet\n\n  ├── completion      Generate the autocompletion script for the specified shell\n  │   ├── bash        Generate the autocompletion script for bash\n  │   ├── fish        Generate the autocompletion script for fish\n  │   ├── powershell  Generate the autocompletion script for powershell\n  │   └── zsh         Generate the autocompletion script for zsh\n  │\n  ├── install  Download a prebuilt libds4 shared library\n  │\n  ├── model         Browse, download, and manage curated ds4 models\n  │   ├── delete    Delete a downloaded model from disk\n  │   ├── download  Download a curated model from Hugging Face\n  │   ├── info      Show details for a curated model\n  │   ├── list      List installed and available models\n  │   └── set       Set the default chat model\n  │\n  ├── prompt  Run prompt or interactive chat inference\n  │\n  ├── status  Find processes holding or using the libds4 shared library\n  │\n  ├── uninstall  Uninstall the installed libds4 shared library\n  │\n  ├── validate  Validate the installed libds4 shared library\n  │\n  └── web         Test browser-backed web tools\n      ├── search  Execute Google search and print Markdown links\n      └── visit   Visit a web page and print extracted Markdown\n\nRun 'ds4go help \u003ccommand\u003e' for detailed usage.\n```\n\n## Examples\n\n```sh\ngo run ./examples/simple --model ./ds4flash.gguf\ngo run ./examples/chat --model ./ds4flash.gguf\ngo run ./examples/toolloop --mock\ngo run ./examples/toolloop --model ./ds4flash.gguf --nothink --tokens 512\ngo run ./examples/openai-compatible --model ./ds4flash.gguf --host 127.0.0.1 --port 8000\n```\n\nThe `toolloop` example registers a Go `add` tool and exercises DSML tool-call parsing, tool dispatch, tool-result rendering, and exact replay. Use `--mock` for a no-model smoke test. The OpenAI-compatible example exposes `POST /v1/chat/completions` for a minimal local test server.\n\n## API Coverage\n\nMost users should import the root package `ds4` from `github.com/NimbleMarkets/ds4go`. It provides Go-native runtime policy and convenience helpers on top of the raw API.\n\nThe strict binding layer lives in package `ds4api`, imported as `github.com/NimbleMarkets/ds4go/ds4api`. It mirrors the public `ds4.h` API: engines, sessions, token vectors, chat prompt rendering, tokenization, logprob helpers, MTP metadata, directional steering options, snapshot/payload save-load, and DS4 context-memory helpers. APIs that take `FILE *` use the package's opaque `ds4api.File` wrapper around a C `FILE*`.\n\n`ds4_log` is exposed as `LogString`, which safely calls it with a fixed `\"%s\"` format. Arbitrary C varargs are intentionally not surfaced as a Go variadic API. `SetLogFunc` redirects libds4 diagnostics that flow through `ds4_log_set` into a Go callback, including Metal/CUDA backend messages routed through `ds4_gpu_log`. `SetAbortFunc` exposes libds4's fatal-invariant hook, which fires immediately before libds4 aborts the process.\n\n## Native stderr\n\nRecent `libds4` builds expose `ds4_log_set`, and ds4go wraps it as\n`SetLogFunc`. The root package also provides `SetLogOutput` for the common\n`io.Writer` case and `DiscardLogs` for quiet embedders. Use them to route libds4\ndiagnostics, including Metal/CUDA backend diagnostics, into your application's\nlogger or to discard them:\n\n```go\nerr := ds4.SetLogOutput(log.Writer())\nerr = ds4.DiscardLogs()\n```\n\nThe logger is process-global inside libds4, not per engine, so install it once\nduring startup. The callback may be invoked from native worker threads; keep it\nconcurrency-safe and quick. Install it before `NewEngine`, or immediately after\nan explicit `Load` and before `Library.NewEngine`, if you want to capture\nstructured model-load and metadata-validation failures from libds4.\n\nMost engine and GPU backend diagnostics now route through the callback. Some\nnative code paths may still write directly to `stderr` until upstream ds4\nconverts them to `ds4_log` or `ds4_gpu_log`. For CLI use, redirect stderr with\nyour shell:\n\n```sh\nds4go prompt ... 2\u003eds4.log\nds4go prompt ... 2\u003e/dev/null\n```\n\nFor Go applications, assigning `os.Stderr` only affects Go code that writes\nthrough `os.Stderr`; it does not reliably capture C `fprintf(stderr, ...)` from\nthe loaded shared library. Capturing direct native stderr inside one process\nrequires process-wide file-descriptor redirection, which can interfere with\nother goroutines, libraries, and concurrent engines. Prefer `SetLogFunc` for\nrouted libds4 diagnostics, shell redirection for CLI runs, or running the model\nworker as a subprocess with `exec.Cmd.Stderr`.\n\n## Fatal abort hook\n\nRecent `libds4` builds expose `ds4_abort_set`, and ds4go wraps it as\n`SetAbortFunc`. This is a last-chance fatal-invariant hook: libds4 calls it\nafter logging the fatal message at `LogError` and immediately before native\n`abort()`.\n\n```go\nerr := ds4.SetAbortFunc(func(msg string) {\n    crashReporter.Record(\"libds4 fatal invariant\", msg)\n})\n```\n\nReturning from the callback does not recover the engine. The native library\nstill calls `abort()` because the invariant is already broken. Use the hook for\ncrash telemetry, flushing logs, or deliberate process termination. Do not call\nback into ds4go/libds4 from the callback; it can run from native worker threads\nwhile an FFI call is active.\n\n## Signal Safety\n\n**Do not use `signal.NotifyContext` around C FFI calls.** `SIGINT` (Ctrl+C) can be delivered to any OS thread, including C worker threads inside `libds4` (Metal, CUDA, or CPU). When that happens the C runtime aborts and the process segfaults.\n\nSafe cancellation is **programmatic only** — pass a `context.Context` to `GenerateOptions.Context` and cancel it from Go code. The generator checks `ctx.Done()` between tokens, so cancellation never interrupts an active FFI call:\n\n```go\nctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)\ndefer cancel()\n\n_, err = ds4.Generator{Engine: engine, Session: session}.GenerateTokens(prompt, ds4.GenerateOptions{\n    MaxTokens: 128,\n    Context:   ctx,\n    OnToken: func(token int) {\n        text, _ := engine.TokenText(token)\n        fmt.Print(text)\n    },\n})\n```\n\nThis is exactly how `examples/openai-compatible` handles client disconnects — it wires `r.Context()` into generation so the engine stops cleanly when the HTTP connection drops.\n\n## Notes\n\nBindings are generated by hand against the public ds4 header at [`https://github.com/antirez/ds4/blob/main/ds4.h`](https://github.com/antirez/ds4/blob/main/ds4.h).\n\nInference runs in-process. The Golang wrapper adds FFI calls but does not proxy tokens through a server or copy model weights. Prefill, generation, Metal/CUDA/CPU execution, MTP, KV reuse, and disk KV payload serialization are all handled by the loaded `ds4` shared library.\n\n## Open Collaboration\n\nWe welcome contributions and feedback.  Please adhere to our [Code of Conduct](./CODE_OF_CONDUCT.md) when engaging our community.\n\n * [GitHub Issues](https://github.com/NimbleMarkets/ds4go/issues)\n * [GitHub Pull Requests](https://github.com/NimbleMarkets/ds4go/pulls)\n\n## Acknowledgements\n\nThanks to [@antirez](https://github.com/antirez) for his work on [`ds4`](https://github.com/antirez/ds4) and for his local-LLM advocacy.  Thanks to [DeepSeek](https://www.deepseek.com/) for their public contributions.\n\n## License\n\nReleased under the [MIT License](https://en.wikipedia.org/wiki/MIT_License), see [LICENSE.txt](./LICENSE.txt).\n\nCopyright (c) 2026 [Neomantra Corp](https://www.neomantra.com).   \n\n----\nMade with :heart: and :fire: by the team behind [Nimble.Markets](https://nimble.markets).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnimblemarkets%2Fds4go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnimblemarkets%2Fds4go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnimblemarkets%2Fds4go/lists"}