{"id":47937606,"url":"https://github.com/jhsu/ai-rlm","last_synced_at":"2026-04-04T07:52:51.662Z","repository":{"id":338340076,"uuid":"1157544519","full_name":"jhsu/ai-rlm","owner":"jhsu","description":"A TypeScript implementation of the RLM (Recursive Language Model) inference strategy using the AI SDK","archived":false,"fork":false,"pushed_at":"2026-03-14T13:19:28.000Z","size":136,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-14T17:21:29.224Z","etag":null,"topics":["ai","ai-sdk","rlm"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/ai-rlm","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jhsu.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-14T00:08:57.000Z","updated_at":"2026-03-14T13:19:32.000Z","dependencies_parsed_at":"2026-02-14T07:05:45.367Z","dependency_job_id":null,"html_url":"https://github.com/jhsu/ai-rlm","commit_stats":null,"previous_names":["jhsu/ai-rlm"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/jhsu/ai-rlm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhsu%2Fai-rlm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhsu%2Fai-rlm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhsu%2Fai-rlm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhsu%2Fai-rlm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jhsu","download_url":"https://codeload.github.com/jhsu/ai-rlm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhsu%2Fai-rlm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31051896,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-27T16:55:14.406Z","status":"ssl_error","status_checked_at":"2026-03-27T16:55:07.885Z","response_time":164,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-sdk","rlm"],"created_at":"2026-04-04T07:52:51.191Z","updated_at":"2026-04-04T07:52:51.644Z","avatar_url":"https://github.com/jhsu.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ai-rlm\n\n[![npm](https://img.shields.io/npm/v/ai-rlm?style=for-the-bdage)](https://www.npmjs.com/package/ai-rlm)\n\nRLM (Recursive Language Model) provided via ai-sdk Agent or tool.\n\nBased on the paper \"Recursive Language Models\" by Zhang, Kraska, and Khattab (2025).\n\n## Overview\n\nRLM is an inference strategy where LLMs treat long contexts as part of an external environment rather than feeding them directly to the model. The LLM writes JavaScript code to programmatically examine, decompose, and recursively call sub-LLMs over snippets.\n\n### Key Features\n\n- **Iterative Code Execution**: The model writes JavaScript code, sees output, then writes more code\n- **Sub-LLM Queries**: Access to `llm_query()` and `llm_query_batched()` for semantic analysis\n- **Context Management**: Efficient handling of large contexts through chunking\n- **Sandboxed REPL**: JavaScript execution in a sandboxed QuickJS WebAssembly context\n- **Pluggable Sandbox Interface**: Swap the execution environment with your own sandbox implementation\n- **AI SDK Integration**: Works as an Agent or Tool with the Vercel AI SDK\n- **Multiple Usage Patterns**: Use as standalone agent or as a tool in larger workflows\n\n## Installation\n\n```bash\nnpm install ai-rlm ai zod @ai-sdk/openai\n```\n\n`ai` and `zod` are peer dependencies and must be installed in your project.\n\nThe `model` and `subModel` settings accept any AI SDK `LanguageModel` — use any provider ([OpenAI](https://sdk.vercel.ai/providers/ai-sdk-providers/openai), [Anthropic](https://sdk.vercel.ai/providers/ai-sdk-providers/anthropic), [Google](https://sdk.vercel.ai/providers/ai-sdk-providers/google-generative-ai), etc.).\n\n## Usage\n\n### As Agent (Recommended)\n\nThe **RLMAgent** class provides a clean, agent-based API that integrates seamlessly with the AI SDK:\n\n```typescript\nimport { RLMAgent } from 'ai-rlm';\nimport { openai } from '@ai-sdk/openai';\n\n// Create agent\nconst agent = new RLMAgent({\n  model: openai('gpt-4.1'),              // Root agent model\n  subModel: openai('gpt-4.1-mini'),      // Sub-LLM model for queries\n  maxIterations: 20,                      // Max REPL iterations\n  maxLLMCalls: 50,                        // Max sub-LLM calls\n});\n\n// Process a context\nconst context = `\n  The quick brown fox jumps over the lazy dog.\n  The magic number is 42.\n`;\n\nconst query = 'What is the magic number?';\n\nconst result = await agent.generate({\n  prompt: query,\n  options: { context },\n});\n\nconst rlmResult = result.output;\n\nconsole.log('Answer:', result.text);\nconsole.log('Iterations:', rlmResult.iterations);\nconsole.log('LLM Calls:', rlmResult.llmCallCount);\nconsole.log('Steps:', rlmResult.steps); // Full trajectory\n```\n\n### As Tool\n\nUse **createRLMTool** to create an AI SDK-compatible tool for use with `generateText` or `ToolLoopAgent`:\n\n```typescript\nimport { createRLMTool } from 'ai-rlm';\nimport { generateText } from 'ai';\nimport { openai } from '@ai-sdk/openai';\n\n// Create the tool\nconst rlmTool = createRLMTool({\n  model: openai('gpt-4.1'),\n  subModel: openai('gpt-4.1-mini'),\n});\n\n// Use in generateText\nconst result = await generateText({\n  model: openai('gpt-4.1'),\n  tools: { analyzeLargeContext: rlmTool },\n  prompt: 'Analyze this large codebase for security vulnerabilities',\n});\n```\n\n### With ToolLoopAgent\n\n```typescript\nimport { ToolLoopAgent } from 'ai';\nimport { createRLMTool } from 'ai-rlm';\nimport { openai } from '@ai-sdk/openai';\n\nconst agent = new ToolLoopAgent({\n  model: openai('gpt-4.1'),\n  tools: {\n    analyzeLargeContext: createRLMTool({\n      model: openai('gpt-4.1'),\n      subModel: openai('gpt-4.1-mini'),\n    }),\n    // ... other tools\n  },\n});\n\nconst result = await agent.generate({\n  prompt: 'Check this document for compliance issues',\n});\n```\n\n### Streaming Support\n\n```typescript\nconst stream = await agent.stream({\n  prompt: 'Analyze this',\n  options: { context: largeDocument },\n});\n\n// textStream emits the final text after generate() completes\nconst reader = stream.textStream.getReader();\nwhile (true) {\n  const { done, value } = await reader.read();\n  if (done) break;\n  process.stdout.write(value);\n}\n```\n\n## How It Works\n\nThe RLM agent writes JavaScript code to explore the context in an iterative loop:\n\n```javascript\n// First, explore the context\nconsole.log('Context length:', context.length);\nconsole.log('First 200 chars:', context.substring(0, 200));\n\n// Search for specific patterns\nconst lines = context.split('\\n');\nconst targetLine = lines.find(line =\u003e line.includes('magic number'));\nconsole.log('Found:', targetLine);\n\n// Store result for later\nconst answer = targetLine?.match(/magic number is (\\d+)/)?.[1];\n\n// Submit answer\nFINAL_VAR(answer)\n```\n\n1. **Context Loading**: The context is loaded into a sandboxed JavaScript REPL environment\n2. **Iterative Reasoning**: The root LLM writes JavaScript code to explore the context\n3. **Code Execution**: Code is executed in a QuickJS WebAssembly sandbox with a 30s timeout\n4. **Sub-LLM Queries**: For semantic analysis, `llm_query()` delegates to a sub-model\n5. **Result Accumulation**: The model iterates until it finds an answer\n6. **Final Answer**: The model submits an answer using `FINAL(answer)` or `FINAL_VAR(variable_name)`\n\n### System Prompt\n\nThe RLM system prompt instructs the model to:\n- EXPLORE FIRST - Look at data before processing\n- ITERATE - Write small code snippets, observe outputs\n- VERIFY BEFORE SUBMITTING - Check results are correct\n- USE llm_query FOR SEMANTICS - Code finds WHERE; LLM understands WHAT\n- CHUNK SMARTLY - Feed substantial chunks to sub-LLMs (~500K chars)\n\n## REPL Sandbox\n\nThe JavaScript REPL runs code in a QuickJS WebAssembly sandboxed context:\n\n### Available in the Sandbox:\n\n- **`context`**: The input context (string or object)\n- **`console.log()` / `console.error()`**: Output logging\n- **`llm_query(prompt)`**: Query a sub-LLM for semantic analysis\n- **`llm_query_batched(prompts)`**: Query multiple sub-LLMs\n- **`FINAL(answer)`**: Submit final answer directly\n- **`FINAL_VAR(varName)`**: Submit a variable from the REPL\n- **Standard JavaScript**: All ES6+ features, Array methods, String methods, Math, JSON, etc.\n\n### Security Features:\n\n- 30-second timeout on code execution\n- No access to Node.js built-in modules or file system\n- No network access\n- Sandboxed console output capture\n\n### Custom Sandbox Implementations\n\n`RLMAgent` supports user-defined sandboxes through `sandboxFactory`.\n\n```typescript\nimport {\n  RLMAgent,\n  createQuickJSSandbox,\n  type RLMSandbox,\n  type RLMSandboxFactoryOptions,\n} from 'ai-rlm';\nimport { openai } from '@ai-sdk/openai';\n\nconst sandboxFactory = (options: RLMSandboxFactoryOptions): RLMSandbox =\u003e {\n  // Wrap the default QuickJS sandbox, or return your own implementation.\n  return createQuickJSSandbox(options);\n};\n\nconst agent = new RLMAgent({\n  model: openai('gpt-4.1'),\n  subModel: openai('gpt-4.1-mini'),\n  sandboxFactory,\n});\n```\n\n### Logging\n\nLibrary diagnostics are silent by default. If you want internal agent logs, pass an explicit logger and log level:\n\n```typescript\nconst agent = new RLMAgent({\n  model: openai('gpt-4.1'),\n  subModel: openai('gpt-4.1-mini'),\n  logger: console,\n  logLevel: 'debug',\n});\n```\n\nUse this for local debugging. In application code, prefer wiring `logger` to your app's logging system rather than relying on `console`.\n\nYour sandbox must implement:\n\n```typescript\ninterface RLMSandbox {\n  loadContext(context: RLMContext): Promise\u003cvoid\u003e;\n  executeJavaScript(code: string): Promise\u003c{\n    stdout: string;\n    stderr: string;\n    error?: string;\n    result?: unknown;\n  }\u003e;\n  getVariable(name: string): unknown;\n  getLLMCallCount(): number;\n  getUsageSummary(): RLMUsageSummary;\n  cleanup(): void;\n}\n```\n\nCustom sandbox factories are also propagated to recursive `sub_rlm()` calls.\n\n## API Reference\n\n### RLMAgent\n\nThe primary class for using RLM as an agent.\n\n#### `constructor(settings: RLMAgentSettings)`\n\n```typescript\nimport type { LanguageModel } from 'ai';\n\ninterface RLMAgentSettings {\n  model: LanguageModel;     // Required: Root agent model\n  subModel?: LanguageModel; // Optional: Sub-LLM model (defaults to model)\n  maxIterations?: number;   // Max REPL iterations (default: 20)\n  maxLLMCalls?: number;     // Max sub-LLM calls (default: 50)\n  maxOutputChars?: number;  // Max REPL output chars (default: 100000)\n  maxHistoryPreview?: number; // Max output preview chars in model history (default: 500)\n  prepareIteration?: (ctx) =\u003e PrepareIterationResult | void | Promise\u003cPrepareIterationResult | void\u003e;\n  prepareSubAgent?: (ctx) =\u003e PrepareSubAgentResult | void | Promise\u003cPrepareSubAgentResult | void\u003e;\n  logger?: RLMLogger;       // Optional injected logger\n  logLevel?: RLMLogLevel;   // Log level for internal diagnostics (default: \"silent\")\n  sandboxFactory?: RLMSandboxFactory; // Optional custom sandbox factory\n}\n```\n\n#### `async generate(options): Promise\u003cRLMGenerateResult\u003e`\n\nGenerate an answer by iteratively analyzing the context.\n\n**Parameters:**\n```typescript\ninterface RLMAgentCallParameters {\n  context: RLMContext;                    // The large context to analyze\n  query: string;                          // The question or task\n  abortSignal?: AbortSignal;              // Optional abort signal\n  timeout?: number;                       // Optional timeout in ms\n  onStepFinish?: (step: REPLStep) =\u003e void; // Callback for each step\n}\n```\n\n**Returns:**\n```typescript\ninterface RLMGenerateResult {\n  text: string;             // The generated answer\n  steps: REPLStep[];        // Array of REPL steps taken\n  llmCallCount: number;     // Total LLM calls made\n  iterations: number;       // Total iterations performed\n  usage: RLMUsageSummary;   // Aggregated token usage across root + sub-calls\n}\n\ninterface REPLStep {\n  iteration: number;\n  reasoning: string;        // The model's reasoning before code\n  code: string;             // JavaScript code executed\n  output: string;           // Console output and results\n}\n```\n\n#### `async stream(options): Promise\u003cRLMStreamResult\u003e`\n\nRun `generate()` and emit AI SDK-style stream parts for iteration progress and final text output.\n\n**Returns:**\n```typescript\ninterface RLMStreamResult extends RLMGenerateResult {\n  textStream: ReadableStream\u003cstring\u003e;  // Emits text-delta content\n  fullStream: ReadableStream\u003cTextStreamPart\u003cToolSet\u003e\u003e; // Emits start/start-step/finish-step/text/finish events\n}\n```\n\n### createRLMTool\n\nFactory function to create RLM as an AI SDK-compatible tool.\n\n#### `createRLMTool(config?: RLMToolConfig)`\n\n```typescript\nimport type { LanguageModel } from 'ai';\n\nfunction createRLMTool(config?: {\n  model?: LanguageModel;    // Root agent model\n  subModel?: LanguageModel; // Sub-LLM model\n  maxIterations?: number;   // Max iterations (default: 20)\n  maxLLMCalls?: number;     // Max LLM calls (default: 50)\n  maxOutputChars?: number;  // Max output chars (default: 100000)\n  logger?: RLMLogger;       // Optional injected logger\n  logLevel?: RLMLogLevel;   // Log level for internal diagnostics\n}): Tool\n```\n\n**Tool Input Schema:**\n```typescript\n{\n  context: string | string[] | Record\u003cstring, unknown\u003e;\n  query: string;\n  maxIterations?: number;   // Optional override\n  maxLLMCalls?: number;     // Optional override\n}\n```\n\n**Tool Output:**\n```typescript\n{\n  answer: string;           // The generated answer\n  iterations: number;       // Number of iterations\n  stepsTaken: number;       // Number of steps executed\n}\n```\n\n### RLMContext\n\nContext can be any of these formats:\n```typescript\ntype RLMContext = string | string[] | Record\u003cstring, unknown\u003e;\n```\n\n- `string`: Raw text document\n- `string[]`: Array of lines or documents\n- `Record\u003cstring, unknown\u003e`: JSON/structured data\n\n## Architecture\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│                      RLMAgent Class                         │\n├─────────────────────────────────────────────────────────────┤\n│  ┌───────────────────────────────────────────────────────┐  │\n│  │              REPL Environment (QuickJS)               │  │\n│  │  - Sandboxed JavaScript execution                     │  │\n│  │  - llm_query() for sub-LLM semantic analysis          │  │\n│  │  - 30s timeout protection                             │  │\n│  └───────────────────────────────────────────────────────┘  │\n│                                                             │\n│  ┌───────────────────────────────────────────────────────┐  │\n│  │              generate() Method                        │  │\n│  │  1. Generate reasoning + JS code                      │  │\n│  │  2. Execute in sandboxed context                      │  │\n│  │  3. Process llm_query markers → real LLM calls        │  │\n│  │  4. Check for FINAL() answer                          │  │\n│  │  5. Repeat or return answer                           │  │\n│  └───────────────────────────────────────────────────────┘  │\n│                                                             │\n│  ┌───────────────────────────────────────────────────────┐  │\n│  │              stream() Method                          │  │\n│  │  - Delegates to generate()                            │  │\n│  │  - Emits start-step / finish-step progress events     │  │\n│  │  - Emits text-start / text-delta / text-end / finish  │  │\n│  └───────────────────────────────────────────────────────┘  │\n└─────────────────────────────────────────────────────────────┘\n                              │\n                              │ createRLMTool()\n                              ▼\n                    ┌──────────────────────┐\n                    │    AI SDK Tool        │\n                    │ - Tool interface      │\n                    │ - Input validation    │\n                    │ - Auto-execution      │\n                    └──────────────────────┘\n```\n\n## Examples\n\nRun the examples:\n\n```bash\n# Basic agent examples\nbun run examples/basic-usage.ts\n\n# Tool integration examples\nbun run examples/tool-usage.ts\n\n# Individual examples\nbun run -e \"import { example1SimpleTextSearch } from './examples/basic-usage.ts'; example1SimpleTextSearch()\"\n```\n\n## CLI Codebase Search\n\nThis repo includes a local CLI script for searching a codebase with `RLMAgent`.\n\nThe CLI now uses a `ToolLoopAgent` orchestrator with tools:\n- `list_files`\n- `search_files`\n- `read_file`\n- `analyze_with_rlm` (deep analysis on selected files)\n\nThis avoids preloading the entire repository into one context window.\n\n```bash\nnpm run code-search -- ./path/to/codebase \"Where is authentication handled?\"\n```\n\nYou can also run the bin directly:\n\n```bash\nnode ./bin/rlm-codebase-search.js ./path/to/codebase \"How are API routes defined?\"\n```\n\nRequired environment variable:\n\n```bash\nexport OPENAI_API_KEY=\"your_key_here\"\n```\n\n### Example Files\n\n- **`examples/basic-usage.ts`**: Agent API examples (generate, stream, callbacks)\n- **`examples/tool-usage.ts`**: Tool API examples (with generateText, ToolLoopAgent)\n- **`examples/document-comparison.ts`**: Document diffing example\n- **`examples/data-transformation.ts`**: Data extraction and transformation\n\n## License\n\nMIT\n\n## References\n\n- Paper: \"Recursive Language Models\" (Zhang, Kraska, Khattab, 2025)\n- AI SDK Documentation: https://sdk.vercel.ai/docs\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhsu%2Fai-rlm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjhsu%2Fai-rlm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhsu%2Fai-rlm/lists"}