{"id":29194773,"url":"https://github.com/paradite/send-prompt","last_synced_at":"2025-07-02T04:38:07.418Z","repository":{"id":293102919,"uuid":"981013869","full_name":"paradite/send-prompt","owner":"paradite","description":"A unified TypeScript library for AI model interactions with standardized interfaces and function calling.","archived":false,"fork":false,"pushed_at":"2025-05-28T07:37:25.000Z","size":172,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-28T07:53:23.942Z","etag":null,"topics":["llm","model","prompt"],"latest_commit_sha":null,"homepage":"https://eval.16x.engineer/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paradite.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-10T06:27:58.000Z","updated_at":"2025-05-28T07:37:28.000Z","dependencies_parsed_at":"2025-05-13T18:08:10.253Z","dependency_job_id":null,"html_url":"https://github.com/paradite/send-prompt","commit_stats":null,"previous_names":["paradite/send-prompt"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/paradite/send-prompt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fsend-prompt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fsend-prompt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fsend-prompt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fsend-prompt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paradite","download_url":"https://codeload.github.com/paradite/send-prompt/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fsend-prompt/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263076973,"owners_count":23410164,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","model","prompt"],"created_at":"2025-07-02T04:38:04.573Z","updated_at":"2025-07-02T04:38:07.407Z","avatar_url":"https://github.com/paradite.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# send-prompt\n\n[![NPM version](https://img.shields.io/npm/v/send-prompt)](https://www.npmjs.com/package/send-prompt)\n\nA TypeScript library for interacting with models across providers with standardized interfaces and tool calling.\n\nRelated projects:\n\n- [llm-info](https://github.com/paradite/llm-info): Information on LLM models, context window token limit, output token limit, pricing and more.\n- [ai-file-edit](https://github.com/paradite/ai-file-edit): A library for editing files using AI models.\n- [model-quirks](https://github.com/paradite/model-quirks): Quirks, edge cases, and interesting bits of various models.\n- [16x Eval](https://eval.16x.engineer): Your personal workspace for prompt engineering and model evaluation.\n\n## Features\n\n- 🔄 Unified interface with comprehensive model support:\n  - First-party providers (OpenAI, Anthropic, Google, DeepSeek)\n  - Third-party providers (Vertex AI, Azure OpenAI, OpenRouter, Fireworks)\n  - Custom providers with OpenAI-compatible API\n- 🔧 Supports function calling and system prompt\n- 📝 Standardized message format and response structure\n- 🛠️ Full TypeScript support for type safety\n- 🎯 No additional dependencies for each provider\n- 🛡️ Handles all edge cases (message format, function calling, multi-round conversations)\n- 🎨 Provider specific options (headers, reasoning extraction)\n- 🖼️ Support for image input in messages (base64 and URL formats)\n- ⚡ Streaming support for real-time responses across all providers\n\n## Quick Demo\n\n```typescript\nimport { sendPrompt } from \"send-prompt\";\nimport { AI_PROVIDERS, ModelEnum } from \"llm-info\";\n\nconst response = await sendPrompt(\n  {\n    messages: [\n      { role: \"user\", content: \"What's the weather like in Singapore?\" },\n    ],\n    tools: [weatherTool],\n  },\n  {\n    model: ModelEnum[\"gpt-4.1\"],\n    // model: ModelEnum[\"claude-3-7-sonnet-20250219\"],\n    // model: ModelEnum[\"gemini-2.5-pro-exp-03-25\"],\n    provider: AI_PROVIDERS.OPENAI,\n    // provider: AI_PROVIDERS.ANTHROPIC,\n    // provider: AI_PROVIDERS.GOOGLE,\n    apiKey: process.env.API_KEY,\n  }\n);\n\nconsole.log(response.message.content);\n```\n\n## Installation\n\n```bash\n# Install llm-info to get the model and provider information\nnpm install llm-info\n# Install send-prompt to send prompt to models\nnpm install send-prompt\n```\n\n## Usage\n\n### Basic Usage\n\nThe same function `sendPrompt` works across all providers:\n\n```typescript\nimport { sendPrompt } from \"send-prompt\";\nimport { AI_PROVIDERS, ModelEnum } from \"llm-info\";\n\n// OpenAI\nconst openaiResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n    systemPrompt: \"You are a helpful assistant.\",\n  },\n  {\n    model: ModelEnum[\"gpt-4.1\"],\n    provider: AI_PROVIDERS.OPENAI,\n    apiKey: \"your-openai-api-key\",\n  }\n);\n\n// Anthropic\nconst anthropicResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n    systemPrompt: \"You are a helpful assistant.\",\n  },\n  {\n    model: ModelEnum[\"claude-3-7-sonnet-20250219\"],\n    provider: AI_PROVIDERS.ANTHROPIC,\n    apiKey: \"your-anthropic-api-key\",\n  }\n);\n\n// Google\nconst googleResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n    systemPrompt: \"You are a helpful assistant.\",\n  },\n  {\n    model: ModelEnum[\"gemini-2.5-pro-exp-03-25\"],\n    provider: AI_PROVIDERS.GOOGLE,\n    apiKey: \"your-google-api-key\",\n  }\n);\n\n// Google Vertex AI (requires gcloud CLI authentication)\n// https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-multimodal\nconst googleVertexResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n    systemPrompt: \"You are a helpful assistant.\",\n  },\n  {\n    model: ModelEnum[\"gemini-2.5-pro-exp-03-25\"],\n    provider: AI_PROVIDERS.GOOGLE_VERTEX_AI,\n    vertexai: true,\n    project: process.env.GOOGLE_CLOUD_PROJECT!, // Your Google Cloud project ID\n    location: process.env.GOOGLE_CLOUD_LOCATION!, // Your Google Cloud location (e.g., \"us-central1\")\n  }\n);\n\n// OpenRouter\nconst openrouterResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n    systemPrompt: \"You are a helpful assistant.\",\n  },\n  {\n    customModel: \"meta-llama/llama-4-scout:free\",\n    provider: AI_PROVIDERS.OPENROUTER,\n    apiKey: \"your-openrouter-api-key\",\n    headers: {\n      \"HTTP-Referer\": \"https://eval.16x.engineer/\",\n      \"X-Title\": \"16x Eval\",\n    },\n  }\n);\n\n// Fireworks\nconst fireworksResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n    systemPrompt: \"You are a helpful assistant.\",\n  },\n  {\n    customModel: \"accounts/fireworks/models/deepseek-v3-0324\",\n    provider: AI_PROVIDERS.FIREWORKS,\n    apiKey: \"your-fireworks-api-key\",\n  }\n);\n\n// DeepSeek\nconst deepseekResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n    systemPrompt: \"You are a helpful assistant.\",\n  },\n  {\n    customModel: \"deepseek-chat\",\n    provider: AI_PROVIDERS.DEEPSEEK,\n    apiKey: \"your-deepseek-api-key\",\n  }\n);\n\n// Custom Provider\nconst customResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n    systemPrompt: \"You are a helpful assistant.\",\n  },\n  {\n    customModel: \"custom-model\",\n    provider: \"custom\",\n    baseURL: \"https://your-custom-api.com/v1\",\n    apiKey: \"your-custom-api-key\",\n  }\n);\n\n// All responses have the same structure\nconsole.log(openaiResponse.message.content);\n```\n\n### Custom Models for First Party Providers\n\nThe `model` field is an enum of all models supported by the library, this is useful to avoid typos and to get the correct model information.\n\nIn case you want to use a model that is not yet available in the enum, you can use `customModel` field instead. This is supported for all first party providers (OpenAI, Anthropic, Google).\n\n```typescript\n// Using custom model string for new models\nconst response = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello, who are you?\" }],\n  },\n  {\n    customModel: \"gpt-4o-mini\", // Custom model string\n    provider: AI_PROVIDERS.OPENAI,\n    apiKey: \"your-openai-api-key\",\n  }\n);\n```\n\nNote that the `model` and `customModel` fields are mutually exclusive.\n\n### Image Input\n\nYou can send images to models that support vision capabilities:\n\n```typescript\nconst imageResponse = await sendPrompt(\n  {\n    messages: [\n      {\n        role: \"user\",\n        content: [\n          { type: \"text\", text: \"What's in this image?\" },\n          {\n            type: \"image_url\",\n            image_url: {\n              url: \"data:image/jpeg;base64,/9j/4AAQSkZJRg...\", // base64 image data\n            },\n          },\n        ],\n      },\n    ],\n  },\n  {\n    model: ModelEnum[\"gpt-4.1\"],\n    provider: AI_PROVIDERS.OPENAI,\n    apiKey: \"your-openai-api-key\",\n  }\n);\n```\n\n### Function Calling\n\n```typescript\n// Define your tool\nconst calculatorTool = {\n  type: \"function\",\n  function: {\n    name: \"calculator\",\n    description: \"Perform basic arithmetic calculations\",\n    parameters: {\n      type: \"object\",\n      properties: {\n        operation: {\n          type: \"string\",\n          enum: [\"add\", \"subtract\", \"multiply\", \"divide\"],\n          description: \"The arithmetic operation to perform\",\n        },\n        a: {\n          type: \"number\",\n          description: \"First number\",\n        },\n        b: {\n          type: \"number\",\n          description: \"Second number\",\n        },\n      },\n      required: [\"operation\", \"a\", \"b\"],\n      additionalProperties: false,\n    },\n    strict: true,\n  },\n};\n\nconst response = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"What is 5 plus 3?\" }],\n    tools: [calculatorTool],\n  },\n  {\n    model: ModelEnum[\"gpt-4.1\"],\n    provider: AI_PROVIDERS.OPENAI,\n    apiKey: \"your-openai-api-key\",\n  }\n);\n\n// Expected response structure:\n// {\n//   tool_calls: [\n//     {\n//       id: \"call_123\",\n//       type: \"function\",\n//       function: {\n//         name: \"calculator\",\n//         arguments: '{\"operation\":\"add\",\"a\":5,\"b\":3}'\n//       }\n//     }\n//   ]\n// }\n\n// Handle the function call\nif (response.tool_calls) {\n  const toolCall = response.tool_calls[0];\n  console.log(\"Tool called:\", toolCall.function.name);\n  console.log(\"Arguments:\", JSON.parse(toolCall.function.arguments));\n}\n```\n\n### Provider Options\n\n#### Headers\n\nYou can pass custom headers to providers using the `headers` option:\n\n```typescript\nconst response = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Hello\" }],\n  },\n  {\n    model: ModelEnum[\"gpt-4.1\"],\n    provider: AI_PROVIDERS.OPENAI,\n    apiKey: \"your-api-key\",\n    headers: {\n      \"Custom-Header\": \"value\",\n      \"X-Title\": \"My App\",\n    },\n  }\n);\n```\n\n#### Temperature\n\nYou can control the randomness of the model's responses using the `temperature` parameter. Temperature ranges from 0 to 2, where lower values make the output more focused and deterministic, while higher values make it more random and creative:\n\n```typescript\nconst response = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Write a creative story\" }],\n    temperature: 0.8, // More creative and random\n  },\n  {\n    model: ModelEnum[\"gpt-4.1\"],\n    provider: AI_PROVIDERS.OPENAI,\n    apiKey: \"your-api-key\",\n  }\n);\n\n// For more deterministic responses\nconst deterministicResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"What is 2 + 2?\" }],\n    temperature: 0.1, // More focused and deterministic\n  },\n  {\n    model: ModelEnum[\"claude-3-7-sonnet-20250219\"],\n    provider: AI_PROVIDERS.ANTHROPIC,\n    apiKey: \"your-api-key\",\n  }\n);\n```\n\nThe temperature parameter is supported across all providers (OpenAI, Anthropic, Google, OpenRouter, Fireworks, DeepSeek, Azure OpenAI, and custom providers). If not specified, each provider will use its default temperature value.\n\n#### Anthropic Max Tokens\n\nFor Anthropic models, you can control the maximum number of tokens in the response using the `anthropicMaxTokens` option:\n\n```typescript\nconst response = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Write a long story\" }],\n    anthropicMaxTokens: 2000, // Limit response to 2000 tokens\n  },\n  {\n    model: ModelEnum[\"claude-3-7-sonnet-20250219\"],\n    provider: AI_PROVIDERS.ANTHROPIC,\n    apiKey: \"your-anthropic-api-key\",\n  }\n);\n```\n\nIf not specified, it will use the model's default output token limit or 4096 tokens, whichever is smaller. When using function calling, it will default to 4096 tokens.\n\n#### Reasoning Extraction\n\nFor providers that support it (like DeepSeek), you can extract the model's reasoning from the response:\n\n```typescript\nconst response = await sendPrompt(\n  {\n    messages: [\n      { role: \"user\", content: \"Solve this math problem: 2x + 5 = 15\" },\n    ],\n  },\n  {\n    model: ModelEnum[\"deepseek-reasoner\"],\n    provider: AI_PROVIDERS.DEEPSEEK,\n    apiKey: \"your-api-key\",\n  }\n);\n\nif (response.reasoning) {\n  console.log(\"Model's reasoning:\", response.reasoning);\n}\n```\n\n### Streaming\n\nYou can stream responses from supported providers to get real-time content as it's generated. Streaming is supported for all providers.\n\n```typescript\nconst response = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"Write a short story about a robot\" }],\n    stream: true,\n    onStreamingContent: (content: string) =\u003e {\n      // This callback is called for each chunk of content\n      process.stdout.write(content);\n    },\n  },\n  {\n    model: ModelEnum[\"gpt-4o-mini\"],\n    provider: AI_PROVIDERS.OPENAI,\n    apiKey: \"your-openai-api-key\",\n  }\n);\n\n// The function still returns the complete response at the end\nconsole.log(\"\\n\\nComplete response:\", response.message.content);\nconsole.log(\"Duration:\", response.durationMs, \"ms\");\nif (response.usage) {\n  console.log(\"Token usage:\", response.usage);\n}\n```\n\n**Streaming Limitations:**\n\n- Cannot be used with function calling (`tools` parameter)\n\n### Response Format\n\nThe response from `sendPrompt` follows a standardized format across all providers:\n\n- `message`: The main response content\n- `tool_calls`: Any function calls made by the model\n- `reasoning`: The model's reasoning process (if available)\n- `usage`: Token usage information\n  - `promptTokens`: Number of tokens in the input messages\n  - `thoughtsTokens`: Number of tokens used for reasoning (if available)\n  - `completionTokens`: Number of tokens in the model's response (includes thoughts tokens)\n  - `totalTokens`: Total tokens used (includes thoughts tokens)\n- `durationMs`: The time taken by the API call in milliseconds\n\nExample response:\n\n```typescript\n{\n  message: {\n    role: \"assistant\",\n    content: \"I am a helpful assistant.\"\n  },\n  usage: {\n    completionTokens: 10,\n    promptTokens: 20,\n    totalTokens: 30,\n    thoughtsTokens: 0\n  },\n  durationMs: 1234\n}\n```\n\n### Multi-round Tool Calls (Google)\n\nFor Google's Gemini models, you can handle multi-round tool calls by including function call and response messages in the conversation:\n\n```typescript\n// First round - model makes a function call\nconst firstResponse = await sendPrompt(\n  {\n    messages: [{ role: \"user\", content: \"What is 15 plus 32?\" }],\n    tools: [calculatorTool],\n    toolCallMode: \"AUTO\",\n  },\n  {\n    model: ModelEnum[\"gemini-2.5-pro-exp-03-25\"],\n    provider: AI_PROVIDERS.GOOGLE,\n    apiKey: \"your-google-api-key\",\n  }\n);\n\n// Handle the function call and get the result\nif (firstResponse.tool_calls) {\n  const toolCall = firstResponse.tool_calls[0];\n  const args = JSON.parse(toolCall.function.arguments);\n  const result = calculate(args.operation, args.a, args.b); // Your calculation function\n\n  // Second round - include function call and response in messages\n  const secondResponse = await sendPrompt(\n    {\n      messages: [\n        { role: \"user\", content: \"What is 15 plus 32?\" },\n        {\n          role: \"google_function_call\",\n          id: toolCall.id,\n          name: toolCall.function.name,\n          args: args,\n        },\n        {\n          role: \"google_function_response\",\n          id: toolCall.id,\n          name: toolCall.function.name,\n          response: { result },\n        },\n      ],\n      tools: [calculatorTool],\n      toolCallMode: \"AUTO\",\n    },\n    {\n      model: ModelEnum[\"gemini-2.5-pro-exp-03-25\"],\n      provider: AI_PROVIDERS.GOOGLE,\n      apiKey: \"your-google-api-key\",\n    }\n  );\n\n  // The model will now respond with the final answer\n  console.log(\"Final response:\", secondResponse.message.content);\n}\n```\n\nThe multi-round tool calling process involves:\n\n1. First round: Model makes a function call\n2. Your code executes the function and gets the result\n3. Second round: Include both the function call and its response in the messages\n4. Model provides the final response using the function result\n\n## Roadmap\n\n- [x] Support for DeepSeek\n- [x] Support for image input\n- [x] Support for streaming\n- [ ] Better error handling\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparadite%2Fsend-prompt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparadite%2Fsend-prompt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparadite%2Fsend-prompt/lists"}