{"id":24769216,"url":"https://github.com/inference-gateway/typescript-sdk","last_synced_at":"2026-04-01T20:43:44.031Z","repository":{"id":273790701,"uuid":"919883068","full_name":"inference-gateway/typescript-sdk","owner":"inference-gateway","description":"An SDK written in Typescript for the Inference Gateway","archived":false,"fork":false,"pushed_at":"2025-11-24T13:06:12.000Z","size":376,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-28T01:09:46.762Z","etag":null,"topics":["inference-gateway","sdk","typescript-sdk"],"latest_commit_sha":null,"homepage":"https://github.com/inference-gateway/inference-gateway","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/inference-gateway.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-01-21T07:26:16.000Z","updated_at":"2025-11-24T12:59:27.000Z","dependencies_parsed_at":"2025-01-23T01:23:55.045Z","dependency_job_id":"f2b65be4-4042-4410-bdb8-751e90dfe514","html_url":"https://github.com/inference-gateway/typescript-sdk","commit_stats":null,"previous_names":["inference-gateway/typescript-sdk"],"tags_count":29,"template":false,"template_full_name":null,"purl":"pkg:github/inference-gateway/typescript-sdk","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inference-gateway%2Ftypescript-sdk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inference-gateway%2Ftypescript-sdk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inference-gateway%2Ftypescript-sdk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inference-gateway%2Ftypescript-sdk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/inference-gateway","download_url":"https://codeload.github.com/inference-gateway/typescript-sdk/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inference-gateway%2Ftypescript-sdk/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31291780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["inference-gateway","sdk","typescript-sdk"],"created_at":"2025-01-29T02:47:53.279Z","updated_at":"2026-04-01T20:43:44.022Z","avatar_url":"https://github.com/inference-gateway.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Inference Gateway TypeScript SDK\n\nAn SDK written in TypeScript for the [Inference Gateway](https://github.com/edenreich/inference-gateway).\n\n- [Inference Gateway TypeScript SDK](#inference-gateway-typescript-sdk)\n  - [Installation](#installation)\n  - [Usage](#usage)\n    - [Creating a Client](#creating-a-client)\n    - [Listing Models](#listing-models)\n    - [Listing MCP Tools](#listing-mcp-tools)\n    - [Creating Chat Completions](#creating-chat-completions)\n    - [Streaming Chat Completions](#streaming-chat-completions)\n    - [Tool Calls](#tool-calls)\n    - [Proxying Requests](#proxying-requests)\n    - [Health Check](#health-check)\n    - [Creating a Client with Custom Options](#creating-a-client-with-custom-options)\n    - [Examples](#examples)\n  - [Contributing](#contributing)\n  - [License](#license)\n\n## Installation\n\nRun `npm i @inference-gateway/sdk`.\n\n## Usage\n\n### Creating a Client\n\n```typescript\nimport { InferenceGatewayClient } from '@inference-gateway/sdk';\n\n// Create a client with default options\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080/v1',\n  apiKey: 'your-api-key', // Optional\n});\n```\n\n### Listing Models\n\nTo list all available models:\n\n```typescript\nimport { InferenceGatewayClient, Provider } from '@inference-gateway/sdk';\n\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080/v1',\n});\n\ntry {\n  // List all models\n  const models = await client.listModels();\n  console.log('All models:', models);\n\n  // List models from a specific provider\n  const openaiModels = await client.listModels(Provider.openai);\n  console.log('OpenAI models:', openaiModels);\n} catch (error) {\n  console.error('Error:', error);\n}\n```\n\n### Listing MCP Tools\n\nTo list available Model Context Protocol (MCP) tools (only available when\nEXPOSE_MCP is enabled):\n\n```typescript\nimport { InferenceGatewayClient } from '@inference-gateway/sdk';\n\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080/v1',\n});\n\ntry {\n  const tools = await client.listTools();\n  console.log('Available MCP tools:', tools.data);\n\n  // Each tool has: name, description, server, and optional input_schema\n  tools.data.forEach((tool) =\u003e {\n    console.log(`Tool: ${tool.name}`);\n    console.log(`Description: ${tool.description}`);\n    console.log(`Server: ${tool.server}`);\n  });\n} catch (error) {\n  console.error('Error:', error);\n}\n```\n\n### Creating Chat Completions\n\nTo generate content using a model:\n\n```typescript\nimport {\n  InferenceGatewayClient,\n  MessageRole,\n  Provider,\n} from '@inference-gateway/sdk';\n\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080/v1',\n});\n\ntry {\n  const response = await client.createChatCompletion(\n    {\n      model: 'gpt-4o',\n      messages: [\n        {\n          role: MessageRole.System,\n          content: 'You are a helpful assistant',\n        },\n        {\n          role: MessageRole.User,\n          content: 'Tell me a joke',\n        },\n      ],\n    },\n    Provider.openai\n  ); // Provider is optional\n\n  console.log('Response:', response.choices[0].message.content);\n} catch (error) {\n  console.error('Error:', error);\n}\n```\n\n### Streaming Chat Completions\n\nTo stream content from a model:\n\n```typescript\nimport {\n  InferenceGatewayClient,\n  MessageRole,\n  Provider,\n} from '@inference-gateway/sdk';\n\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080/v1',\n});\n\ntry {\n  await client.streamChatCompletion(\n    {\n      model: 'llama-3.3-70b-versatile',\n      messages: [\n        {\n          role: MessageRole.User,\n          content: 'Tell me a story',\n        },\n      ],\n    },\n    {\n      onOpen: () =\u003e console.log('Stream opened'),\n      onContent: (content) =\u003e process.stdout.write(content),\n      onChunk: (chunk) =\u003e console.log('Received chunk:', chunk.id),\n      onUsageMetrics: (metrics) =\u003e console.log('Usage metrics:', metrics),\n      onFinish: () =\u003e console.log('\\nStream completed'),\n      onError: (error) =\u003e console.error('Stream error:', error),\n    },\n    Provider.groq // Provider is optional\n  );\n} catch (error) {\n  console.error('Error:', error);\n}\n```\n\n### Tool Calls\n\nTo use tool calls with models that support them:\n\n```typescript\nimport {\n  InferenceGatewayClient,\n  MessageRole,\n  Provider,\n} from '@inference-gateway/sdk';\n\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080/v1',\n});\n\ntry {\n  await client.streamChatCompletion(\n    {\n      model: 'openai/gpt-4o',\n      messages: [\n        {\n          role: MessageRole.User,\n          content: \"What's the weather in San Francisco?\",\n        },\n      ],\n      tools: [\n        {\n          type: 'function',\n          function: {\n            name: 'get_weather',\n            parameters: {\n              type: 'object',\n              properties: {\n                location: {\n                  type: 'string',\n                  description: 'The city and state, e.g. San Francisco, CA',\n                },\n              },\n              required: ['location'],\n            },\n          },\n        },\n      ],\n    },\n    {\n      onTool: (toolCall) =\u003e {\n        console.log('Tool call:', toolCall.function.name);\n        console.log('Arguments:', toolCall.function.arguments);\n      },\n      onReasoning: (reasoning) =\u003e {\n        console.log('Reasoning:', reasoning);\n      },\n      onContent: (content) =\u003e {\n        console.log('Content:', content);\n      },\n      onFinish: () =\u003e console.log('\\nStream completed'),\n    }\n  );\n} catch (error) {\n  console.error('Error:', error);\n}\n```\n\n### Proxying Requests\n\nTo proxy requests directly to a provider:\n\n```typescript\nimport { InferenceGatewayClient, Provider } from '@inference-gateway/sdk';\n\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080',\n});\n\ntry {\n  const response = await client.proxy(Provider.openai, 'embeddings', {\n    method: 'POST',\n    body: JSON.stringify({\n      model: 'text-embedding-ada-002',\n      input: 'Hello world',\n    }),\n  });\n\n  console.log('Embeddings:', response);\n} catch (error) {\n  console.error('Error:', error);\n}\n```\n\n### Health Check\n\nTo check if the Inference Gateway is running:\n\n```typescript\nimport { InferenceGatewayClient } from '@inference-gateway/sdk';\n\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080',\n});\n\ntry {\n  const isHealthy = await client.healthCheck();\n  console.log('API is healthy:', isHealthy);\n} catch (error) {\n  console.error('Error:', error);\n}\n```\n\n### Creating a Client with Custom Options\n\nYou can create a new client with custom options using the `withOptions` method:\n\n```typescript\nimport { InferenceGatewayClient } from '@inference-gateway/sdk';\n\nconst client = new InferenceGatewayClient({\n  baseURL: 'http://localhost:8080/v1',\n});\n\n// Create a new client with custom headers\nconst clientWithHeaders = client.withOptions({\n  defaultHeaders: {\n    'X-Custom-Header': 'value',\n  },\n  timeout: 60000, // 60 seconds\n});\n```\n\n### Examples\n\nFor more examples, check the [examples directory](./examples).\n\n## Contributing\n\nPlease refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for information\nabout how to get involved. We welcome issues, questions, and pull requests.\n\n## License\n\nThis SDK is distributed under the MIT License, see [LICENSE](LICENSE) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finference-gateway%2Ftypescript-sdk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finference-gateway%2Ftypescript-sdk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finference-gateway%2Ftypescript-sdk/lists"}