{"id":27236662,"url":"https://github.com/langtail/langtail-node","last_synced_at":"2025-04-10T17:45:42.670Z","repository":{"id":232610818,"uuid":"773819402","full_name":"langtail/langtail-node","owner":"langtail","description":"Langtail TypeScript SDK","archived":false,"fork":false,"pushed_at":"2025-04-08T08:46:37.000Z","size":607,"stargazers_count":11,"open_issues_count":2,"forks_count":4,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-08T09:44:45.286Z","etag":null,"topics":["llm","llmops","llms"],"latest_commit_sha":null,"homepage":"https://langtail.com","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/langtail.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-18T13:06:55.000Z","updated_at":"2025-04-08T08:46:34.000Z","dependencies_parsed_at":"2024-10-28T13:12:46.262Z","dependency_job_id":"d321c483-d116-4a59-9963-3146036e4a73","html_url":"https://github.com/langtail/langtail-node","commit_stats":null,"previous_names":["langtail/langtail-node"],"tags_count":50,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langtail%2Flangtail-node","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langtail%2Flangtail-node/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langtail%2Flangtail-node/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langtail%2Flangtail-node/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/langtail","download_url":"https://codeload.github.com/langtail/langtail-node/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248262343,"owners_count":21074291,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","llmops","llms"],"created_at":"2025-04-10T17:45:41.507Z","updated_at":"2025-04-10T17:45:42.664Z","avatar_url":"https://github.com/langtail.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Langtail SDK\n\nTypescript SDK for [Langtail](https://langtail.com/).\n\n[![CI check](https://github.com/langtail/langtail-node/workflows/CI%20check/badge.svg)](https://github.com/langtail/langtail-node/actions?query=workflow:\"CI+check\")\n[![GitHub tag](https://img.shields.io/github/tag/langtail/langtail-node?include_prereleases=\u0026sort=semver\u0026color=blue)](https://github.com/langtail/langtail-node/releases/)\n[![License](https://img.shields.io/badge/License-MIT-blue)](#license)\n\n## Install\n\n```bash\nnpm i langtail\n```\n\n## Usage\n\n### OpenAI chat completion\n\nbasic completion without any prompt. This just wraps openAI api and adds a few extra parameters you can use to affect how the request gets logged in langtail.\n\n```ts\nimport OpenAI from \"openai\"\nimport { createOpenAIProxy } from \"langtail/openai\"\n\nconst openai = new OpenAI({\n  apiKey: \"\u003cLANGTAIL_API_KEY\u003e\",\n})\nconst lt = createOpenAIProxy(openai)\n\nconst rawCompletion = await lt.chat.completions.create({\n  // Required\n  messages: [{ role: \"system\", content: \"You are a helpful assistant.\" }],\n  model: \"gpt-3.5-turbo\",\n  // Optional:\n  // All OpenAI fields (temperature, top_p, tools,...)\n  prompt: \"\u003cprompt-slug\u003e\",\n  doNotRecord: false, // false will ensure logs do not contain any info about payloads. You can still see the request in the logs, but you cannot see the variables etc.\n  metadata: {\n    \"custom-field\": \"1\",\n  },\n})\n```\n\n### Deployed prompts\n\nCompletion from a deployed prompt can be called with `lt.prompts.invoke`:\n\n```ts\nconst deployedPromptCompletion = await lt.prompts.invoke({\n  prompt: \"\u003cPROMPT_SLUG\u003e\", // required\n  environment: \"staging\",\n  variables: {\n    about: \"cowboy Bebop\",\n  },\n}) // results in an openAI ChatCompletion\n```\n\nOf course this assumes that you have already deployed your prompt to `staging` environment. If not, you will get an error thrown an error: `Error: Failed to fetch prompt: 404 {\"error\":\"Prompt deployment not found\"}`\n\n## LangtailPrompts\n\nIn case you only need deployed prompts, you can import just `LangtailPrompts` like this:\n\n```ts\nimport { LangtailPrompts } from \"langtail\"\n\nconst lt = new LangtailPrompts({\n  apiKey: \"\u003cLANGTAIL_API_KEY\u003e\",\n})\n// usage\nconst deployedPromptCompletion = await lt.invoke({\n  prompt: \"\u003cPROMPT_SLUG\u003e\",\n  environment: \"staging\",\n  variables: {\n    about: \"cowboy Bebop\",\n  },\n})\n```\n\nYou can initialize LangtailPrompts with workspace and project slugs like so:\n\n```ts\nimport { Langtail } from \"langtail\"\n\nconst lt = new Langtail({\n  apiKey: \"\u003cLANGTAIL_API_KEY\u003e\",\n  workspace: \"\u003cWORKSPACE_SLUG\u003e\",\n  project: \"\u003cPROJECT_SLUG\u003e\",\n})\n```\n\nwhich is necessary if your API key is workspace wide. For a project api key this is not necessary.\n\n## Streaming responses\n\nboth chat.prompts.create and prompts.invoke support streaming responses. All you need to enable it is `{ stream: true }` flag like this:\n\n```ts\nconst deployedPromptCompletion = await lt.prompts.invoke({\n  prompt: \"\u003cPROMPT_SLUG\u003e\",\n  environment: \"staging\",\n  stream: true, // changes result to be a streaming OpenAI response\n}) // results in an openAI Stream\u003cChatCompletionChunk\u003e\n```\n\nFull API reference is in [API.md](API.md)\n\nWe support the same [runtimes as OpenAI](https://github.com/openai/openai-node?tab=readme-ov-file#requirements).\n\n### Proxyless usage\n\nYou can avoid langtail API all together by constructing your prompt locally and calling your provider like openAI directly.\n\nlet's suppose you have a prompt called `joke-teller` deployed on staging in langtail. You can `get` it's template and all the playground config by calling `get` method like this:\n\n```ts\nimport { LangtailPrompts } from \"langtail\"\n\nconst lt = new LangtailPrompts({\n  apiKey: \"\u003cLANGTAIL_API_KEY\u003e\",\n})\n\nconst playgroundState = await lt.get({\n  prompt: \"\u003cPROMPT_SLUG\u003e\",\n  environment: \"preview\",\n  version: \"\u003cPROMPT_VERSION\u003e\", // optional\n})\n```\n\n`get` will return something like this depending on how your prompt configured when it was deployed:\n\n```\n          {\n            \"chatInput\": {\n              \"optionalExtra\": \"\",\n            },\n            \"state\": {\n              \"args\": {\n                \"frequency_penalty\": 0,\n                \"jsonmode\": false,\n                \"max_tokens\": 800,\n                \"model\": \"gpt-3.5-turbo\",\n                \"presence_penalty\": 0,\n                \"stop\": [],\n                \"stream\": true,\n                \"temperature\": 0.5,\n                \"top_p\": 1,\n              },\n              \"functions\": [],\n              \"template\": [\n                {\n                  \"content\": \"I want you to tell me a joke. Topic of the joke: {{topic}}\",\n                  \"role\": \"system\",\n                },\n              ],\n              \"tools\": [],\n              \"type\": \"chat\",\n            },\n          }\n```\n\nrender your template and builds the final open AI compatible payload:\n\n```ts\nimport { getOpenAIBody } from \"langtail/getOpenAIBody\"\n\nconst openAiBody = getOpenAIBody(playgroundState, {\n  stream: true,\n  variables: {\n    topic: \"iron man\",\n  },\n})\n```\n\nopenAiBody now contains this object:\n\n```js\n{\n            \"frequency_penalty\": 0,\n            \"max_tokens\": 800,\n            \"messages\": [\n              {\n                \"content\": \"I want you to tell me a joke. Topic of the joke: iron man\",\n                \"role\": \"system\",\n              },\n            ],\n            \"model\": \"gpt-3.5-turbo\",\n            \"presence_penalty\": 0,\n            \"temperature\": 0.5,\n            \"top_p\": 1,\n          }\n```\n\nNotice that your langtail template was replaced with a variable passed in. You can directly call openAI SDK with this object:\n\n```ts\nimport OpenAI from \"openai\"\n\nconst openai = new OpenAI()\n\nconst joke = await openai.chat.completions.create(openAiBody)\n```\n\nThis way you are still using langtail prompts without exposing potentially sensitive data in your variables.\n\n## Typed inputs\n\nYou can override input types to improve IntelliSense for the `prompt`, `environment`, `version` and `variables` when calling a prompt. Use the command `npx langtail generate-types`.\n\n## Vercel AI provider\n\nYou can use Langtail with [Vercel AI SDK](https://github.com/vercel/ai).\nImport `langtail` from `langtail/vercel-ai` and provide your prompt slug as an argument.\n```typescript\nimport { generateText } from 'ai'\nimport { langtail } from 'langtail/vercel-ai'\n\nasync function main() {\n  const result = await generateText({\n    // API key is loaded from env variable LANGTAIL_API_KEY\n    model: langtail('stock-simple', {\n      // Optional Langtail options:\n      variables: { 'ticker': 'TSLA' },\n      environment: \"production\",\n      version: \"2\",\n      doNotRecord: false,\n      metadata: {},\n    }),\n    // Optional LLM options:\n    prompt: 'show me the price',\n    temperature: 0,  // overrides setting in Langtail\n  })\n\n  console.log(result.text)\n}\n\nmain().catch(console.error);\n```\n\nYou can also use `aiBridge` from `langtail/vercel-ai` to use already existing Langtail instance:\n```typescript\nconst langtail = new Langtail({ apiKey })\nconst lt = aiBridge(langtail)\n\nconst result = await generateText({\n  model: lt('stock-simple', {\n    variables: { 'ticker': 'TSLA' },\n  }),\n  prompt: 'show me the price',\n})\n```\n\n### Using tools from Langtail\n\nIf your prompts in Langtail contain tools, you can generate a file containing tool parameters for every prompt deployment in your project. Run `npx langtail generate-tools --out [output_filepath]` to generate the file. For typings of the `tools` helper to work correctly, you also need to [generate types](#typed-inputs).\n\nAfter the file is generated, you can provide the Langtail tools to AI SDK like this:\n```typescript\nimport { generateText } from 'ai'\nimport { langtail } from 'langtail/vercel-ai'\nimport tools from './langtailTools';  // generated langtailTools.ts file\n\nconst ltModel = langtail('stock-simple',\n  {\n    environment: \"production\",\n    version: \"3\"  // pinning the version is recommended\n  }\n);\nconst result = await generateText({\n  model: ltModel,\n  prompt: 'Show me the current price!',\n  tools: tools(ltModel),  // loads all the tools for the specified prompt version\n});\n```\n\nYou can also define custom execute functions for your tools as follows:\n```typescript\ntools(ltModel, {\n  get_current_stock_price: {\n    execute: async ({ ticker }) =\u003e {\n      return ({\n        ticker,\n        price: 200 + Math.floor(Math.random() * 50),\n      });\n    },\n  },\n})\n```\n\n## Stream helpers\n\nThe AI streams are delivered as JSON objects, which are split into chunks. This can pose a challenge because JSON objects might be distributed across multiple chunks. We have provide you with helper functions to manage these JSON streams more effectively.\n\nHere's an example:\n\n```ts\nimport {\n  chatStreamToRunner,\n  type ChatCompletionStream,\n} from \"langtail/stream\"\n\nconst stream = await fetch(`/api/langtail`, {\n  method: \"POST\",\n  body: JSON.stringify({ messages: localMessages }),\n  headers: {\n    \"Content-Type\": \"application/json\",\n  },\n}).then((res) =\u003e res.body)\n\n// NOTE: await res.body =\u003e ReadableStream\nconst runner = chatStreamToRunner(stream)\n\nrunner.on(\"message\", (messageDelta: string) =\u003e {\n  // NOTE: this is a string delta directly from the AI you can put together\n  console.log(messageDelta)\n})\n\nrunner.on(\"chunk\", (chunk: ChatCompletionChunk) =\u003e {\n  // NOTE: chunk here is always a proper JSON even with parts of the message\n})\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flangtail%2Flangtail-node","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flangtail%2Flangtail-node","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flangtail%2Flangtail-node/lists"}