{"id":16024176,"url":"https://github.com/aaronfriel/fn-stream","last_synced_at":"2025-03-18T03:32:02.819Z","repository":{"id":215042131,"uuid":"737975629","full_name":"AaronFriel/fn-stream","owner":"AaronFriel","description":"Create interactive, powerful AI applications with streaming, structured responses from language models","archived":false,"fork":false,"pushed_at":"2024-01-16T00:37:29.000Z","size":163,"stargazers_count":20,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-28T06:50:37.235Z","etag":null,"topics":["ai","artificial-intelligence","javascript","llm","nodejs","openai","streaming","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AaronFriel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-02T05:39:33.000Z","updated_at":"2024-06-21T06:45:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"0cea0559-d5aa-4f1b-bd5f-65d941bb1c91","html_url":"https://github.com/AaronFriel/fn-stream","commit_stats":{"total_commits":5,"total_committers":1,"mean_commits":5.0,"dds":0.0,"last_synced_commit":"a208bf2f5dd686cf4e6e4f5c78d8e45e445daa7f"},"previous_names":["aaronfriel/fn-stream"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaronFriel%2Ffn-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaronFriel%2Ffn-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaronFriel%2Ffn-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AaronFriel%2Ffn-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AaronFriel","download_url":"https://codeload.github.com/AaronFriel/fn-stream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243901006,"owners_count":20366250,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","javascript","llm","nodejs","openai","streaming","typescript"],"created_at":"2024-10-08T19:05:48.711Z","updated_at":"2025-03-18T03:32:02.502Z","avatar_url":"https://github.com/AaronFriel.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fn-stream\n\n`fn-stream` is a library that makes it possible to combine generative AI (large language models) with tools in interactive, low-latency applications.\n\nTypical generative AI tools require users to choose between **streaming responses**, and **structured data** - called \"tools\".\n\nWith `fn-stream`, you can have both: structured data, streamed and handled however it makes sense for _your_ application.\n\n## How it Works\n\nThe key components of the `fn-stream` library are:\n\n* The online `StreamingParser` that can parse JSON output from a large language model token-by-token or character-by-character.\n* The format and type of `ParseEvents` to take the guesswork out of working with partially parsed objects.\n* The [`ts-pattern` package](https://github.com/gvergnaud/ts-pattern) which makes it possible to correctly pattern match on the parse events.\n* Adapters for defining tools for large language model APIs, such as the [`openai` package](https://www.npmjs.com/package/openai), making it easy to define a tool and write code against it with a minimum of guesswork.\n  * The OpenAI adapter uses the wonderful [`json-schema-to-ts` package](https://github.com/ThomasAribart/json-schema-to-ts) to reduce boilerplate.\n\nThis library is written in TypeScript but does not require it!\n\n## Examples\n\n### CLI Example with OpenAI\n\nThis example, from [examples/openai/index.js](./examples/openai/index.js), shows how the fn-stream tool can be used to incrementally stream parts of a response to the user in a CLI program.\n\nIn this example we prompt the model to generate code in a number of languages, using a tool definition to separate the code from the conversation. The tool's schema matches this TypeScript type:\n\n```typescript\ntype ToolArguments {\n  responses: {\n    preamble?: string;\n    language: string;\n    code: string;\n    postscript?: string;\n  }[]\n}\n```\n\nThe `fn-stream` library allows us to extract and summarize the code, here just the length, without requiring additional LLM API calls, finnicky parsing logic, or long delays.\n\n\u003cdetails\u003e\n\u003csummary\u003eRunning this example\n\u003c/summary\u003e\n\n1. Run `pnpm install` (see: [`pnpm`` installation](https://pnpm.io/installation)) in the root directory to install `fn-stream` in the workspace.\n\n2. Set the OPENAI_API_KEY environment variable, you may need to [sign up for an OpenAI API account](https://platform.openai.com/signup):\n\n3. In the `examples/openai` directory, run the example:\n\n   ```bash\n   node ./index.js\n   ```\n\n4. You should see output streaming after the call.\n\n5. Try running with the `FILTER_PART` environment variable to filter the output to just one of the **preamble** (message before code), **language**, **code**, or **postscript** (message after code).\n\n   ```bash\n   FILTER_PART=code node ./index.js\n   ```\n\u003c/details\u003e\n\n#### Full output\n\nIn this example, we prompt the model for code and the tool's schema limits the languages to the arguments after the prompt: JavaScript, Python, and Rust. We print the conversation, colorized by which field is rendered.\n\nThere is no Markdown parsing behind this, just streaming, structured output! The code fence backticks are inserted by the program when receiving a \"language\" field, and the closing backticks are inserted when the \"code\" field is complete.\n\n![A screen recording of the example. The program emits programs in Python, JavaScript, and Rust, and the text and code are rendered using different colors in the terminal.](examples/openai/demo.svg)\n\n#### Field extraction\n\nIn this example, we run the program with the default prompt and languages. This prompts the model to write \"Hello World\" in three languages. The `FILTER_PART` environment variable is used to filter the result to just the code. Again, the language model is not emitting unstructured text. The model is still producing the preamble, language, and postscript fields, they are filtered out of the text response.\n\nBy using `fn-stream`, we can prompt a model to generate multimodal output, text, code, structured data, and operate on or stream only what the application needs.\n\n![A screen recording of the example. The program emits *just* code, with no other content.](examples/openai/helloworld.svg)\n\n### More examples\n\nThese tests showcase using `fn-stream` in a more client and server style:\n* [OpenAI with separate server and client state, with hand-written reducers](./packages/fn-stream/examples/openai-reducer.spec.ts)\n* [OpenAI with separate server and client state, using immer to simplify synchronizing state](./packages/fn-stream/examples/openai-immer.spec.ts)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaronfriel%2Ffn-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faaronfriel%2Ffn-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaronfriel%2Ffn-stream/lists"}