{"id":14905605,"url":"https://github.com/fxnai/fxn-llm-js","last_synced_at":"2025-05-12T07:32:46.748Z","repository":{"id":254463073,"uuid":"843172834","full_name":"fxnai/fxn-llm-js","owner":"fxnai","description":"Use local LLMs in your browser and Node.js apps. Register at https://fxn.ai","archived":false,"fork":false,"pushed_at":"2024-09-19T19:33:15.000Z","size":766,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-09-23T14:04:53.512Z","etag":null,"topics":["embeddings","fxn","llm","nomic","on-device","openai-embeddings","retrieval","retrieval-augmented-generation"],"latest_commit_sha":null,"homepage":"https://fxn-llm-js.vercel.app","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fxnai.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-16T00:00:14.000Z","updated_at":"2024-09-21T08:08:16.000Z","dependencies_parsed_at":"2024-09-22T14:05:31.070Z","dependency_job_id":"331b8c9d-21f2-43f0-ba82-186d673688d9","html_url":"https://github.com/fxnai/fxn-llm-js","commit_stats":null,"previous_names":["fxnai/fxn-llm-js"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fxnai%2Ffxn-llm-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fxnai%2Ffxn-llm-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fxnai%2Ffxn-llm-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fxnai%2Ffxn-llm-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fxnai","download_url":"https://codeload.github.com/fxnai/fxn-llm-js/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225129418,"owners_count":17425323,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embeddings","fxn","llm","nomic","on-device","openai-embeddings","retrieval","retrieval-augmented-generation"],"created_at":"2024-09-22T14:01:35.713Z","updated_at":"2024-11-18T04:37:01.972Z","avatar_url":"https://github.com/fxnai.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"# Function LLM for JavaScript\n\nhttps://github.com/user-attachments/assets/86ae6012-264e-437f-9ab8-94408f4105ba\n\n[![Dynamic JSON Badge](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fdiscord.com%2Fapi%2Finvites%2Fy5vwgXkz2f%3Fwith_counts%3Dtrue\u0026query=%24.approximate_member_count\u0026logo=discord\u0026logoColor=white\u0026label=Function%20community)](https://discord.gg/fxn)\n[![X (formerly Twitter) Follow](https://img.shields.io/twitter/follow/fxnai)](https://twitter.com/fxnai)\n\nUse local LLMs in your browser and Node.js apps. This package is designed to patch `OpenAI` and `Anthropic` clients for running inference locally, using predictors hosted on [Function](https://fxn.ai/explore).\n\n\u003e [!TIP]\n\u003e We offer a similar package for use in Python. Check out [fxn-llm](https://github.com/fxnai/fxn-llm).\n\n\u003e [!IMPORTANT]\n\u003e This package is still a work-in-progress, so the API could change drastically between **all** releases.\n\n\u003e [!CAUTION]\n\u003e **Never embed access keys client-side (i.e. in the browser)**. Instead, [create a proxy URL](https://docs.fxn.ai/insiders/keys#in-the-browser) in your backend.\n\n## Installing Function LLM\nFunction LLM is distributed on NPM. Open a terminal and run the following command:\n```bash\n# Run this in Terminal\n$ npm install fxn-llm\n```\n\n\u003e [!IMPORTANT]\n\u003e Make sure to create an access key by signing onto [Function](https://fxn.ai/settings/developer). You'll need it to fetch the predictor at runtime.\n\n## Using the OpenAI Client Locally\nTo run text generation and embedding models locally using the OpenAI client, patch your `OpenAI` instance with the `locally` function:\n```ts\nimport { locally } from \"fxn-llm\"\nimport { OpenAI } from \"openai\"\n\n// 💥 Create your OpenAI client\nlet openai = new OpenAI({ apiKey: \"fxn\", dangerouslyAllowBrowser: true });\n\n// 🔥 Make it local\nopenai = locally(openai, {\n  accessKey: process.env.NEXT_PUBLIC_FXN_ACCESS_KEY\n});\n\n// 🚀 Generate embeddings\nconst embeddings = openai.embeddings.create({\n  model: \"@nomic/nomic-embed-text-v1.5-quant\",\n  input: \"search_query: Hello world!\"\n});\n```\n\n\u003e [!WARNING]\n\u003e Currently, only `openai.embeddings.create` is supported. Text generation is coming soon!\n\n___\n\n## Useful Links\n- [Discover predictors to use in your apps](https://fxn.ai/explore).\n- [Join our Discord community](https://discord.gg/fxn).\n- [Check out our docs](https://docs.fxn.ai).\n- Learn more about us [on our blog](https://blog.fxn.ai).\n- Reach out to us at [hi@fxn.ai](mailto:hi@fxn.ai).\n\nFunction is a product of [NatML Inc](https://github.com/natmlx).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffxnai%2Ffxn-llm-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffxnai%2Ffxn-llm-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffxnai%2Ffxn-llm-js/lists"}