{"id":28165839,"url":"https://github.com/sehugg/llmhelper","last_synced_at":"2025-07-05T05:07:21.790Z","repository":{"id":287901864,"uuid":"965864639","full_name":"sehugg/llmhelper","owner":"sehugg","description":"TypeScript library for interfacing with local and remote LLMs","archived":false,"fork":false,"pushed_at":"2025-05-09T21:08:56.000Z","size":497,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-09T21:27:22.139Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sehugg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-14T03:12:31.000Z","updated_at":"2025-05-09T21:08:59.000Z","dependencies_parsed_at":"2025-04-14T15:33:12.561Z","dependency_job_id":"53c28a55-5ff7-479c-bf33-3c7c3c757834","html_url":"https://github.com/sehugg/llmhelper","commit_stats":null,"previous_names":["sehugg/llmhelper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sehugg/llmhelper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sehugg%2Fllmhelper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sehugg%2Fllmhelper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sehugg%2Fllmhelper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sehugg%2Fllmhelper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sehugg","download_url":"https://codeload.github.com/sehugg/llmhelper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sehugg%2Fllmhelper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263687156,"owners_count":23496089,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-15T12:12:37.025Z","updated_at":"2025-07-05T05:07:21.783Z","avatar_url":"https://github.com/sehugg.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLMHelper: Leverage Language Models Helpfully Expediting Logic Processing and Easy Runtime\n\nTired of lang-chaining nodes together? Just want to write some code? LLMHelper helps you help yourself.\n\nKey features:\n\n* **Plain TypeScript Workflows:** No cryptic YAML or bizarre DSLs here. Just good ol' fluent TypeScript you actually understand.\n* **Structured Output:** Input a Zod schema, get a typed JSON result. We'll keep asking until the LLM gets it right.\n* **Restartable Workflows:** Hit Ctrl-C while generating that award-winning novel? No problem! LLMHelper can pick up where you left off.\n* **Run Tools via Docker:** Need access to a specific command-line tool? Just slap it into a Docker container \nand let LLMHelper handle the rest.\n* **Tree Search:** Don't settle for linear thinking! Explore possibilities with powerful tree \nsearch algorithms built right in.\n\n\n## Get Started\n\n### Installation\n\nYou need Node 22. Run `npm install` and create a config file:\n\n```bash\nnpm install\nnpx playwright install\ncp config.yml.example config.yml\n```\n\nEdit `config.yml` and replace the `apiKey` placeholders with your API keys for each service you want to use.\n\nYou can also set parameters like URL and temperature for some models.\n\n### Environment Setup\n\nAn Environment stores artifacts, e.g. content files + metadata.\nThis allows you to run workflows and automatically use previous results.\n\nFileSystemEnvironment stores artifacts in a directory tree, ideal for persistent or restartable workflows.\n```typescript\nconst env = new FileSystemEnvironment(\"./project_name\"); \nconst env = new TemporaryFileSystemEnvironment(); // creates a new folder in /tmp\n```\n\nMemoryStoreEnvironment stores artifacts in memory only:\n```typescript\nconst env = new MemoryStoreEnvironment();\n``` \n\nNullEnvironment is the default. It doesn't store anything.\n\n### Initialize LLMHelper\n\nSelect a model and optionally define its initial system message:\n\n```typescript\nconst llm = new LLMHelper(env).model('mini').system(\"You are a helpful assistant.\");\n```\n\nReplace `mini` with your desired language model or alias as defined in `config.yml` (e.g., `local`, `llama3`).\n\nThe default model is `default`, unless the `LLM_DEFAULT_MODEL` environment variable is set.\n\n### Run an Example\n\n```typescript\nnode --loader ts-node/esm examples/backronym.ts \"a new LLM framework\"\n```\n\n## Unstructured Output\n\nUse `run()` to generate text or Markdown:\n\n```typescript\nconst result = await llm\n    .system('Output everything in ALL CAPS.')\n    .prompt('Say \"hello\".')\n    .outputFile('sys.txt')\n    .run();\nconsole.log(result.output); // \"HELLO\"\n```\n\nThe framework will automatically add formatting prompts to the system message for text, Markdown, JSON, based on output file extension and whether a schema is present. Or use the `format()` method to set it manually.\n\n## Structured Output\n\nUse `generate()` to generate validated JSON objects via [Zod](https://zod.dev/) schemas:\n\n```typescript\nconst UserSchema = z.object({\n    name: z.string().describe(\"The user's full name including middle name\"),\n    age: z.number().int().positive(),\n    email: z.string().email().optional(),\n    tags: z.array(z.string()).min(1),\n    role: z.enum([\"admin\", \"user\", \"guest\"]),\n}).refine(data =\u003e data.name.split(\" \").length == 3, {\n    message: \"Name must include first, middle, and last name\",\n});\n\nconst user1 = await llm\n    .prompt('Give me an example output for a good superhero.')\n    .outputFile('good.json')\n    .generate(UserSchema);\nconsole.log(user1.output.age);\n```\n\n## Context\n\nLLMContext is a chain of message lists.\nYou can call `continue()` on a result to continue the conversation:\n\n```typescript\nconst user2 = await user1.continue()\n    .prompt('Now give me the evil twin with a similar name and same age')\n    .generate(UserSchema);\n```\n\nLLMHelper is immutable, so you have to store it to a mutable var if you extend it:\n\n```typescript\nlet llm = root_llm;\nllm = user1.continue();\n```\n\n## Overwrite Policy\n\nEach operation results in at least one file being written to the Environment.\nYou might want to restart a run, and not generate fresh results from the LLM.\nBut how do you know if you want to overwrite a file?\nUse `llm.overwrite(policy)`:\n\n* fail (default) - throw an error instead of overwriting an existing file\n* skip - quietly reuse the results of an existing file\n* force - always overwrite existing files\n* exact - only reuse if prompt is the same\n* timestamp - only reuse if input timestamps are later than output timestamp\n\n\n## Tool Use\n\nYou can use tools in a few different ways.\nUse `useTools()` to force output of raw tool results (the return type is an array of the union of all tool output types):\n\n```typescript\nconst tools = [new JSTool()] as const;\nconst result = await llm\n    .prompt('Compute the sieve of Eratosthenes up to 10000, and return the last prime number found.')\n    .useTools(tools);\nconsole.log(result[0].js_result); // 9973\n```\n\nOr call `addTools()` and let the LLM figure it out:\n\n```typescript\nconst result = await llm.prompt(`What is the 100,000th prime number? Use tools.`)\n    .addTools([new JSTool()])\n    .run();\n```\n\nThis may have mixed results, because not all LLMs are very good at tool usage.\nYou may have better luck getting structured outputs from the LLM, and calling tool methods directly from the code.\n\nTools available:\n* JavaScript execution\n* Docker image build/run\n* Web browsing\n* API use\n* Fetch tool (parses various kinds of web content)\n\nNOTE: Docker use may use lots of disk space.\nConsider running `docker container prune` and `docker image prune` to clean up.\n\nYou can add environment variables to your Docker images via your `config.yml`, for example to add a HTTP proxy:\n```yaml\ntools:\n  docker:\n    env:\n      http_proxy: http://192.168.0.123:3128/\n```\nIf you want to use tools to call APIs that pass secrets via query string (e.g. api key) you can add them to the secrets section of config.yml. They are identified by host or domain name, for example:\n```yaml\n  googleapis.com:\n    key: ...\n    cx: ...\n```\n\n## Benchmarking\n\n```typescript\nlet models = ['mini', 'gemini', 'llama3', 'phi3.5'];\nconst llm = rootllm.prompt(`What is the 100,000th prime number?`).addTools([new JSTool()]);\nconst results = await benchmarkModels(llm, models,\n    z.object({ answer: z.number() }),\n    (o) =\u003e o.answer == 1299709);\n```\n\n## Images\n\n```typescript\nconst out1 = await llm\n    .prompt('What is the make, model, and color of this car?')\n    .image(await loadImageToBase64Url('examples/car.jpeg'))\n    .run();\n```\n\n# TODO\n\n* Remove ```markdown blocks from Markdown files.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsehugg%2Fllmhelper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsehugg%2Fllmhelper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsehugg%2Fllmhelper/lists"}