https://github.com/truefoundry/tfy-gateway-nemotron-example
This project demonstrates how to configure nvidia nemotron with Truefoundry's AI Gateway
https://github.com/truefoundry/tfy-gateway-nemotron-example
Last synced: 3 months ago
JSON representation
This project demonstrates how to configure nvidia nemotron with Truefoundry's AI Gateway
- Host: GitHub
- URL: https://github.com/truefoundry/tfy-gateway-nemotron-example
- Owner: truefoundry
- Created: 2026-03-11T18:47:36.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2026-03-12T00:35:39.000Z (4 months ago)
- Last Synced: 2026-03-28T17:32:46.687Z (3 months ago)
- Language: Python
- Size: 12.7 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Nemotron 3 on TrueFoundry AI Gateway
[Nemotron 3 Super](https://research.nvidia.com/labs/nemotron/Nemotron-3/) is NVIDIA's latest open model — 120B parameters, 12B active at inference via a hybrid Mamba-Transformer MoE. 5x higher throughput than the previous generation.
🚀 [Sign up for TrueFoundry](https://www.truefoundry.com/register) · 🔑 [Get NVIDIA API key](https://build.nvidia.com)
---
## Option 1: UI
Add Nemotron to the gateway directly from the TrueFoundry dashboard — no code needed.
📺 [Watch the walkthrough](https://www.loom.com/share/5342a8be7e62444f810dcb9e2144c0c7)
---
## Option 2: CLI
**1. Store your NVIDIA API key as a TFY secret**
Dashboard → Secrets → create a group → add `NVIDIA-API-KEY = nvapi-...` — [docs](https://docs.truefoundry.com/docs/manage-secrets)
Update `bearer_token` in `gateway.yaml` with your secret reference.
**2. Apply**
```bash
export TFY_HOST=https://.truefoundry.cloud
export TFY_API_KEY=
tfy apply -f gateway.yaml
```
**3. Call the model**
Set `TFY_BASE_URL` and `TFY_API_KEY` in `example.py` and run it.
---
## Option 3: Claude + TrueFoundry Skills
Install [TrueFoundry Agent Skills](https://github.com/truefoundry/tfy-agent-skills), set these env vars:
```bash
TFY_HOST=https://.truefoundry.cloud
TFY_BASE_URL=https://.truefoundry.cloud
TFY_API_KEY=
```
Then ask Claude:
> "Deploy Nemotron 3 thinking model to TrueFoundry"