{"id":35205585,"url":"https://github.com/sourcefuse/loopback4-llm-chat-extension","last_synced_at":"2026-04-07T15:32:12.291Z","repository":{"id":312244757,"uuid":"1035993541","full_name":"sourcefuse/loopback4-llm-chat-extension","owner":"sourcefuse","description":"A Loopack4 based component to integrate a basic Langgraph.js based endpoint in your application which can use any tool that you register using the provided decorator.","archived":false,"fork":false,"pushed_at":"2026-03-30T08:30:04.000Z","size":2431,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-30T10:19:11.895Z","etag":null,"topics":["loopback4-chatbot","loopback4-extension","loopback4-llm"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sourcefuse.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-11T11:59:38.000Z","updated_at":"2026-03-23T14:27:56.000Z","dependencies_parsed_at":"2025-12-29T16:07:48.008Z","dependency_job_id":null,"html_url":"https://github.com/sourcefuse/loopback4-llm-chat-extension","commit_stats":null,"previous_names":["sourcefuse/llm-chat-component","sourcefuse/loopback4-llm-chat-extension"],"tags_count":27,"template":false,"template_full_name":null,"purl":"pkg:github/sourcefuse/loopback4-llm-chat-extension","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Floopback4-llm-chat-extension","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Floopback4-llm-chat-extension/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Floopback4-llm-chat-extension/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Floopback4-llm-chat-extension/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sourcefuse","download_url":"https://codeload.github.com/sourcefuse/loopback4-llm-chat-extension/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Floopback4-llm-chat-extension/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31518499,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T03:10:19.677Z","status":"ssl_error","status_checked_at":"2026-04-07T03:10:13.982Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["loopback4-chatbot","loopback4-extension","loopback4-llm"],"created_at":"2025-12-29T13:44:40.338Z","updated_at":"2026-04-07T15:32:12.285Z","avatar_url":"https://github.com/sourcefuse.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://sourcefuse.github.io/arc-docs/arc-api-docs\" target=\"_blank\"\u003e\u003cimg src=\"https://github.com/sourcefuse/loopback4-microservice-catalog/blob/master/docs/assets/logo-dark-bg.png?raw=true\" alt=\"ARC By SourceFuse logo\" title=\"ARC By SourceFuse\" align=\"right\" width=\"150\" /\u003e\u003c/a\u003e\n\n# [A Loopback4 LLM Chat Extension](https://github.com/sourcefuse/loopback4-llm-chat-extension)\n\n\u003cp align=\"left\"\u003e\n\u003ca href=\"https://www.npmjs.com/package/lb4-llm-chat-component\"\u003e\n\u003cimg src=\"https://img.shields.io/npm/v/lb4-llm-chat-component.svg\" alt=\"npm version\" /\u003e\n\u003c/a\u003e\n\u003ca href=\"https://sonarcloud.io/summary/new_code?id=sourcefuse_llm-chat-component\" target=\"_blank\"\u003e\n\u003cimg alt=\"Sonar Quality Gate\" src=\"https://img.shields.io/sonar/quality_gate/sourcefuse_llm-chat-component?server=https%3A%2F%2Fsonarcloud.io\"\u003e\n\u003c/a\u003e\n\u003ca href=\"https://github.com/sourcefuse/loopback4-llm-chat-extension/graphs/contributors\" target=\"_blank\"\u003e\n\u003cimg alt=\"GitHub contributors\" src=\"https://img.shields.io/github/contributors/sourcefuse/loopback4-llm-chat-extension?\"\u003e\n\u003c/a\u003e\n\u003ca href=\"https://www.npmjs.com/package/loopback4-authentication\" target=\"_blank\"\u003e\n\u003cimg alt=\"downloads\" src=\"https://img.shields.io/npm/dw/loopback4-llm-chat-extension.svg\"\u003e\n\u003c/a\u003e\n\u003ca href=\"https://github.com/sourcefuse/loopback4-llm-chat-extension/blob/master/LICENSE\"\u003e\n\u003cimg src=\"https://img.shields.io/github/license/sourcefuse/loopback4-llm-chat-extension.svg\" alt=\"License\" /\u003e\n\u003c/a\u003e\n\u003ca href=\"https://loopback.io/\" target=\"_blank\"\u003e\n\u003cimg alt=\"Powered By LoopBack 4\" src=\"https://img.shields.io/badge/Powered%20by-LoopBack 4-brightgreen\" /\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n### Overview\n\nA Loopack4 based component to integrate a basic Langgraph.js based endpoint in your application which can use any tool that you register using the provided decorator.\n\n### Installation\n\nInstall AIIntegrationsComponent using `npm`;\n\n```sh\n$ [npm install | yarn add] lb4-llm-chat-component\n```\n\n### Basic Usage\n\nConfigure and load the AIIntegrations component in the application constructor\nas shown below.\n\n```ts\nimport {AiIntegrationsComponent} from 'lb4-llm-chat-component';\n// ...\nexport class MyApplication extends BootMixin(\n  ServiceMixin(RepositoryMixin(RestApplication)),\n) {\n  constructor(options: ApplicationConfig = {}) {\n    // could be any LLM provider or your own LangGraph supported LLM provider\n    // you can also have different LLM for different LLM type - cheap, smart and multimodal\n    this.bind(AiIntegrationBindings.CheapLLM).toProvider(Ollama);\n    this.bind(AiIntegrationBindings.SmartLLM).toProvider(Ollama);\n    this.bind(AiIntegrationBindings.FileLLM).toProvider(Ollama);\n    // configuration\n    this.bind(AiIntegrationBindings.Config).to({\n      // if not set to true, it will bind a ARC based sequence from @sourceloop/core with authentication and authorization\n      useCustomSequence: true,\n      // if not set to false, it will bind the core component from @sourceloop/core by default\n      mountCore: false\n      // if not set to false, it will bind @sourceloop/file-utils component with defaults config\n      mountFileUtils: false\n    });\n    this.component(AiIntegrationsComponent);\n\n    // ...\n  }\n  // ...\n}\n```\n\n## LLM Providers\n\n### Ollama\n\nTo need the `Ollama` based models, install the package - `@langchain/ollama` and update your application.ts -\n\n```ts\nthis.bind(AiIntegrationBindings.CheapLLM).toProvider(Ollama);\nthis.bind(AiIntegrationBindings.SmartLLM).toProvider(Ollama);\nthis.bind(AiIntegrationBindings.FileLLM).toProvider(Ollama);\n```\n\n### Gemini\n\nTo use the `Gemini` based models, install the package - `@google/generative-ai` and `@langchain/google-genai` and update your application.ts -\n\n```ts\nthis.bind(AiIntegrationBindings.CheapLLM).toProvider(Gemini);\nthis.bind(AiIntegrationBindings.SmartLLM).toProvider(Gemini);\nthis.bind(AiIntegrationBindings.FileLLM).toProvider(Gemini);\n```\n\n### Cerebras\n\nTo use the `Cerebras` based models, install the package - `@langchain/cerebras` and update your application.ts -\n\n```ts\nthis.bind(AiIntegrationBindings.CheapLLM).toProvider(Cerebras);\nthis.bind(AiIntegrationBindings.SmartLLM).toProvider(Cerebras);\nthis.bind(AiIntegrationBindings.FileLLM).toProvider(Cerebras);\n```\n\n### Anthropic\n\nTo use the `Anthropic` based models, install the package - `@langchain/anthropic` and update your application.ts -\n\n```ts\nthis.bind(AiIntegrationBindings.CheapLLM).toProvider(Anthropic);\nthis.bind(AiIntegrationBindings.SmartLLM).toProvider(Anthropic);\nthis.bind(AiIntegrationBindings.FileLLM).toProvider(Anthropic);\n```\n\n### OpenAI\n\nTo use the `OpenAI` models, install the package - `@langchain/openai` and update your application.ts -\n\n```ts\nthis.bind(AiIntegrationBindings.CheapLLM).toProvider(OpenAI);\nthis.bind(AiIntegrationBindings.SmartLLM).toProvider(OpenAI);\nthis.bind(AiIntegrationBindings.FileLLM).toProvider(OpenAI);\n```\n\n### Bedrock\n\nTo use the `Bedrock` based models, install the package - `@langchain/aws` and update your application.ts -\n\n```ts\nthis.bind(AiIntegrationBindings.CheapLLM).toProvider(Bedrock);\nthis.bind(AiIntegrationBindings.SmartLLM).toProvider(Bedrock);\nthis.bind(AiIntegrationBindings.FileLLM).toProvider(Bedrock);\n```\n\nThis binding would add an endpoint `/generate` in your service, that can answer user's query using the registered tools. By default, the module gives one set of tools through the `DbQueryComponent`\n\n## Limiters\n\nThe package provides a way to limit the usage of the LLM Chat functionality by binding a provider on the key `AiIntegrationBindings.LimitStrategy` that follows the interface - `ILimitStrategy`.\n\nThe packages comes with 3 strategies by default that are bound automatically on the basis of `AiIntegrationBindings.Config` -\n\n- **ChatCountStrategy** - Applies limits per user based on number of chats. It is used if only `chatLimit` and `period` is provided in `tokenCounterConfig`.\n- **TokenCountStrategy** - Applies a fixed limit per user based on number of tokens used. It is used if `tokenLimit` and `period` are provided with `bufferToken` as optional field that determines that how much buffer to keep while checking for token limit.\n- **TokenCountPerUserStrategy** - Applies token based limit similar to `TokenCountStrategy` except the number of tokens commes from user permission `TokenUsage:NUMBER` in the user's token. It applies if only `period` is set in `tokenCounterConfig`, it also works with `bufferToken` just like `TokenCountStrategy`.\n\n## DbQueryComponent\n\nThis component provides a set of pre-built tools that can be plugged into any Loopback4 application -\n\n- **generate-query** - this tool can be used by the LLM to generate a database query based on user's prompt. It will return a `DataSet` instead of the query directly to the LLM.\n- **improve-query** - this tool takes a `DataSet`'s id and feedback from the user, and uses it to modify the existing `DataSet` query. Users can also vote on datasets via the dataset actions endpoint.\n- **ask-about-dataset** - this tool takes a `DataSet`'s id and a user prompt, and tries to answer user's question about the database query. Note that it can not run the query.\n\n### Database Schema\n\nThe component uses a dedicated `chatbot` schema with the following tables:\n\n- **datasets** - Stores generated SQL queries with metadata including description, prompt, tables involved, and schema hash\n- **dataset_actions** - Tracks user actions on datasets (votes, comments, improvements)\n- **chats** - Stores chat sessions with metadata and token usage\n- **messages** - Stores individual messages within chats\n\n### Dataset Feedback System\n\nUsers can provide feedback on generated datasets through the dataset actions endpoint. Each dataset can receive votes and comments, which can be used to improve future query generation. The system tracks:\n\n- Vote count for each dataset\n- User comments and suggestions\n- Improvement history\n- Creation and modification timestamps\n\n### Query Generation Flow\n\n![alt text](https://raw.githubusercontent.com/sourcefuse/llm-chat-component/refs/heads/main/db-query-graph.png)\n\n## VisualizerComponent\n\nThe `VisualizerComponent` extends the LLM Chat functionality by providing intelligent data visualization capabilities. This component automatically generates charts and graphs based on database query results, making data insights more accessible and visually appealing.\n\n### Features\n\n- **Automatic Visualization Selection** - The system intelligently selects the most appropriate visualization type based on the data structure and user prompt\n- **Multiple Chart Types** - Supports bar charts, line charts, and pie charts out of the box\n- **LLM-Powered Configuration** - Uses AI to generate optimal chart configurations including axis selection, orientation, and styling\n- **Seamless Integration** - Works directly with datasets generated by the DbQueryComponent\n\n### Available Visualizers\n\nThe component includes three built-in visualizers:\n\n#### Bar Chart Visualizer (`src/components/visualization/visualizers/bar.visualizer.ts:13`)\n\n- **Best for**: Comparing values across different categories or showing trends over time\n- **Configuration**: Automatically determines category column (x-axis) and value column (y-axis)\n- **Options**: Supports both vertical and horizontal orientations\n\n#### Line Chart Visualizer (`src/components/visualization/visualizers/line.visualizer.ts`)\n\n- **Best for**: Displaying trends and changes over time\n- **Configuration**: Optimized for time-series data and continuous variables\n\n#### Pie Chart Visualizer (`src/components/visualization/visualizers/pie.visualizer.ts`)\n\n- **Best for**: Showing proportions and percentages of a whole\n- **Configuration**: Automatically identifies categorical data and corresponding values\n\n### Usage\n\n#### Basic Setup\n\n```ts\nimport {VisualizerComponent} from 'lb4-llm-chat-component';\n\nexport class MyApplication extends BootMixin(\n  ServiceMixin(RepositoryMixin(RestApplication)),\n) {\n  constructor(options: ApplicationConfig = {}) {\n    // Add the visualizer component\n    this.component(VisualizerComponent);\n\n    // ... other configuration\n  }\n}\n```\n\n#### Generate Visualization Tool\n\nThe component provides a `generate-visualization` tool that can be used by the LLM to create visualizations:\n\n- **Input**: Takes a user prompt and dataset ID from a previously generated query\n- **Process**: Automatically selects the best visualization type and generates optimal configuration\n- **Output**: Renders the visualization in the UI for the user\n\n#### Example Usage Flow\n\n1. User asks: \"Show me sales by region as a chart\"\n2. LLM uses `generate-query` tool to create a dataset with sales data by region\n3. LLM uses `generate-visualization` tool with the dataset ID\n4. System selects bar chart as the most appropriate visualization\n5. Chart is rendered with regions on x-axis and sales values on y-axis\n\n### Visualization Graph Flow\n\nThe visualization process follows a structured graph workflow (`src/components/visualization/visualization.graph.ts:9`):\n\n1. **Get Dataset Data** - Retrieves the dataset and query information\n2. **Select Visualization** - Chooses the most appropriate chart type based on data structure\n3. **Render Visualization** - Generates the final chart configuration and displays it\n\n### Creating Custom Visualizers\n\nYou can extend the system with custom visualizers by implementing the `IVisualizer` interface (`src/components/visualization/types.ts:4`):\n\n```ts\nimport {visualizer} from 'lb4-llm-chat-component';\nimport {IVisualizer, VisualizationGraphState} from 'lb4-llm-chat-component';\n\n@visualizer()\nexport class CustomVisualizer implements IVisualizer {\n  name = 'custom-chart';\n  description = 'Description of when to use this visualizer';\n\n  async getConfig(state: VisualizationGraphState): Promise\u003cAnyObject\u003e {\n    // Generate configuration based on the data and user prompt\n    return {\n      // your custom chart configuration\n    };\n  }\n}\n```\n\n### Configuration\n\nThe visualizer component automatically registers all available visualizers and makes them available to the LLM. No additional configuration is required for basic usage. You can register a new visualizer using the `@visualizer` decorator on a class following the IVisualizer interface.\n\n### Integration with DbQueryComponent\n\nThe visualizer component works seamlessly with the `DbQueryComponent`:\n\n1. Use database query tools to generate datasets\n2. The visualization tool automatically accesses dataset metadata including:\n   - SQL query structure\n   - Query description\n   - User's original prompt\n3. This context helps generate more accurate and relevant visualizations\n\n## Providing Context\n\nThere are two ways to provide context to the LLM -\n\n### Global Context\n\nGlobal context can be provided as an array of strings through a binding on key `DbQueryAIExtensionBindings.GlobalContext`. This binding can be a constant or come through a dynamic provider, something like this -\n\n```ts\nexport class ChecksProvider implements Provider\u003cstring[]\u003e {\n  constructor(\n    @repository(CurrencyRepository)\n    private readonly currencyRepository: CurrencyRepository,\n  ) {}\n  async value(): Promise\u003cstring[]\u003e {\n    return [`Current date is ${new Date().toISOString().split('T')[0]}`];\n  }\n}\n```\n\nin application.ts -\n\n```ts\n...\nthis.bind(DbQueryAIExtensionBindings.GlobalContext).toProvider(ChecksProvider);\n...\n```\n\n### Model Context\n\nEach model can have associated context in 3 ways -\n\n```ts\n@model({\n  name: 'employees', // Use plural form for table name\n  settings: {\n    description: 'Model representing an employee in the system.',\n    context: [\n      'employee salary must be converted to USD, using the currency_id column and the exchange rate table',\n    ],\n  },\n})\nexport class Employee extends Entity {\n  ...\n  @property({\n    type: 'string',\n    required: true,\n    description: 'Name of the employee',\n  })\n  name: string;\n\n  @property({\n    type: 'string',\n    required: true,\n    description: 'Unique code for the employee, used for identification',\n  })\n  code: string;\n\n  @property({\n    type: 'number',\n    required: true,\n    description:\n      'The salary of the employee in the currency stored in currency_id column',\n  })\n  salary: number;\n  ...\n}\n```\n\n- Model description - this is the primary description of the model, it is used to select model for generation, so it should only define the purpose of the model itself.\n- Model context - this is secondary information about the model, usually defining some specific details about the model that must be kept in mind while using it. NOTE - These values should always include the model name. This must be information that is applicable to overall model usage, or atleast to multiple columns, and not related to any single field of the model.\n- Property description - this is the description for a property of a model, providing context for the LLM on how to use and understand a particular property.\n\n## Usage\n\nYou just need to register your models in the configuration of the component, and if the Models have proper and detailed descriptions, the tools should be able to answer the user's prompts based on those descriptions.\n\n```ts\nthis.bind(DbQueryAIExtensionBindings.Config).to({\n  models: [\n    {\n      model: Employee, // A normal loopback4 model class with proper description\n      readPermissionKey: '1', // permission key used to check access for this particular model/table\n    },\n  ],\n  db: {\n    dialect: SupportedDBs.PostgreSQL, // dialect for which the SQL will be generated.\n    schema: 'public', // schema of the database in case of DBs like Postgresql\n    ignoredColumns: ['deleted'], // list of db column names that will be ignored for query generation (Do not use Loopback field names in this list)\n  },\n  readAccessForAI: false // give access of the query result to the llm\n  maxRowsForAI: 0 // number of rows from the result that are passed to the LLM\n  columnSelection: false // add a column selection step in generation in case you have tables with a lot of columns.\n});\n```\n\n## Connectors\n\nThe package comes with 3 connectors by default -\n\n- **PgConnector** - basic connector for PostgreSQL databases\n- **SqlLiteConnector** - basic connector SqlLite databases, can be used for testing\n- **PgWithRlsConnector** - Connector for PostgreSQL databases with support for [Row Security Policies](https://www.postgresql.org/docs/current/ddl-rowsecurity.html). Refer [`PgWithRlsConnector`](#pgwithrlsconnector) for more details.\n\nYou can write your own connector by following the `IDbConnector` interface and and binding it on `DbQueryAIExtensionBindings.Connector`.\n\nBy default, the package binds `PgWithRlsConnector` but if you are not planning to use row security policies or default conditions, you can bind `PgConnector` -\n\n```ts\n// application.ts\nthis.bind(DbQueryAIExtensionBindings.Connector).toClass(PgConnector);\n```\n\n## Default Conditions\n\nThe package allows binding an optional provider on key `DbQueryAIExtensionBindings.DefaultConditions` that are applied on every query generated by the LLM. **NOTE** This only works for connectors that support this option.\n\nAs of now, the only provider that supports this is `PgWithRlsConnector`.\n\n## PgWithRlsConnector\n\nYou can take advantage of the `DbQueryAIExtensionBindings.DefaultConditions` by using this connector with a PostgreSQL database. To use this you need to first setup your database to use [Row Security Policies](https://www.postgresql.org/docs/current/ddl-rowsecurity.html). You can use an SQL script that looks something like this to do this -\n\n```sql\nDO $$\nDECLARE\n    tbl text;\n    tables text[] := ARRAY[\n        'main.test-table',\n    ];\nBEGIN\n    FOREACH tbl IN ARRAY tables LOOP\n        -- Enable RLS\n        EXECUTE format('ALTER TABLE %I ENABLE ROW LEVEL SECURITY;', tbl);\n\n        -- Drop existing policy if it already exists (to avoid duplicates)\n        EXECUTE format('DROP POLICY IF EXISTS tenant_policy ON %I;', tbl);\n\n        -- Create policy (applies to SELECT, INSERT, UPDATE, DELETE)\n        EXECUTE format($f$\n            CREATE POLICY tenant_policy ON %I\n            -- conditions to apply by default are tenant id = some value and deleted = false\n            USING (tenant_id = current_setting('app.current_tenant')::int AND deleted = false)\n            WITH CHECK (false);\n        $f$, tbl);\n    END LOOP;\nEND$$;\n```\n\nOnce the policies are setup, you can bind the provider for `DbQueryAIExtensionBindings.DefaultConditions` -\n\n```ts\n// default-conditions.provider.ts\nimport {IAuthUserWithPermissions} from '@sourceloop/core';\nimport {AuthenticationBindings} from 'loopback4-authentication';\nimport {AnyObject} from '@loopback/repository';\nimport {Provider} from '@loopback/core';\n\nclass DefaultConditionsProvider implements Provider\u003cAnyObject\u003e {\n  constructor(\n    @inject(AuthenticationBindings.CURRENT_USER)\n    private readonly user: IAuthUserWithPermissions,\n  ) {}\n  value() {\n    return {\n      tenant_id: this.user.tenantId,\n    };\n  }\n}\n```\n\n```ts\n// application.ts\nthis.bind(DbQueryAIExtensionBindings.DefaultConditions).to(\n  DefaultConditionsProvider,\n);\n```\n\n## Writing Your Own Tool\n\nYou can register your own tools by simply using the `@graphTool()` decorator and implementing the `IGraphTool` interface. Any such class would be automatically registered with the `/generate` endpoint and the LLM would be able to use it as a tool.\n\n```ts\nimport {tool} from '@langchain/core/tools';\nimport z from 'zod';\nimport {graphTool, IGraphTool} from 'lb4-llm-chat-component';\n\n...\n@graphTool()\nexport class AddTool implements IGraphTool {\n  needsReview = false;\n\n  build() {\n    return tool((ob: {a: number, b: number}) =\u003e {\n        return ob.a + ob.b\n    },\n    {\n        name: 'add-tool',\n        description: 'a tool to add two numbers',\n        schema: z.object({\n            a: z.number(),\n            b: z.number()\n        })\n    });\n  }\n}\n```\n\n# Observability\n\n## With Langsmith\n\nYou can enable langsmith observability by simply adding the Langsmith env variables. Refer [this](https://docs.langchain.com/langsmith/observability-quickstart) for more details.\n\n## With Langfuse\n\nYou can enable Langfuse based tracing with the following steps -\n\n- install the following packages -\n\n```sh\nnpm i @langfuse/core @langfuse/langchain @langfuse/otel\n```\n\n- adding the following component in your LB4 application -\n\n```ts\nimport {LangfuseComponent} from 'lb4-llm-chat-component/langfuse';\n...\n\nthis.component(LangfuseComponent);\n```\n\n- set up langfuseSpanProcessor -\n\n```ts\nimport {LangfuseSpanProcessor} from '@langfuse/otel';\nimport {OTLPTraceExporter} from '@opentelemetry/exporter-trace-otlp-http';\nimport {DnsInstrumentation} from '@opentelemetry/instrumentation-dns';\nimport {HttpInstrumentation} from '@opentelemetry/instrumentation-http';\nimport {PgInstrumentation} from '@opentelemetry/instrumentation-pg';\nimport {\n  defaultResource,\n  resourceFromAttributes,\n} from '@opentelemetry/resources';\nimport {NodeSDK} from '@opentelemetry/sdk-node';\nimport {BatchSpanProcessor, SpanProcessor} from '@opentelemetry/sdk-trace-base';\nimport {\n  ATTR_SERVICE_NAME,\n  ATTR_SERVICE_VERSION,\n} from '@opentelemetry/semantic-conventions';\nimport * as dotenv from 'dotenv';\nimport * as dotenvExt from 'dotenv-extended';\n\ndotenv.config();\ndotenvExt.load({\n  schema: '.env.example',\n  errorOnMissing: true,\n  includeProcessEnv: true,\n});\n\nif (!!+(process.env.ENABLE_TRACING ?? 0)) {\n  const resource = defaultResource().merge(\n    resourceFromAttributes({\n      [ATTR_SERVICE_NAME]: process.env.SERVICE_NAME ?? 'reporting-service',\n      [ATTR_SERVICE_VERSION]: process.env.SERVICE_VERSION ?? '1.0.0',\n    }),\n  );\n\n  const spanProcessors: SpanProcessor[] = [];\n  const instrumentations = [];\n\n  // Add OTLP exporter if Jaeger endpoint is configured\n  if (process.env.OPENTELEMETRY_HOST \u0026\u0026 process.env.OPENTELEMETRY_PORT) {\n    const otlpExporter = new OTLPTraceExporter({\n      url: `http://${process.env.OPENTELEMETRY_HOST}:${process.env.OPENTELEMETRY_PORT}/v1/traces`,\n    });\n    spanProcessors.push(new BatchSpanProcessor(otlpExporter));\n    instrumentations.push(\n      new HttpInstrumentation(),\n      new DnsInstrumentation(),\n      new PgInstrumentation(),\n    );\n  }\n\n  if (process.env.LANGFUSE_BASE_URL) {\n    console.log('Langfuse tracing enabled');\n    spanProcessors.push(new LangfuseSpanProcessor());\n  }\n\n  const sdk = new NodeSDK({\n    resource: resource,\n    spanProcessors: spanProcessors,\n    instrumentations,\n  });\n\n  // Initialize the SDK\n  console.log('Starting OpenTelemetry SDK');\n  sdk.start();\n\n  // Graceful shutdown\n  process.on('SIGTERM', () =\u003e {\n    sdk\n      .shutdown()\n      .then(() =\u003e console.log('Tracing terminated'))\n      .catch(error =\u003e console.log('Error terminating tracing', error))\n      .finally(() =\u003e process.exit(0));\n  });\n}\n```\n\n- setup env -\n\n```sh\nexport ENABLE_TRACING=1\nexport LANGFUSE_SECRET_KEY=\"sk-lf-...\"\nexport LANGFUSE_PUBLIC_KEY=\"pk-lf-...\"\nexport LANGFUSE_BASE_URL=\"https://cloud.langfuse.com\"\n```\n\n# Testing\n\n## Generation Acceptance Builder\n\nThe `generation.acceptance.builder.ts` file provides a utility to run acceptance tests for the `llm-chat-component`. These tests validate the functionality of the `/reply` endpoint and ensure that the generated SQL queries and their results align with expectations.\n\n## Overview\n\nThis builder facilitates the execution of multiple test cases, each defined with specific prompts, expected results, and configurations. It also generates detailed reports to analyze the performance and correctness of the tests.\n\n## Key Features\n\n- **Dynamic Prompt Parsing**: Replaces placeholders in prompts with environment-specific values.\n- **Token Generation**: Creates JWT tokens with required permissions for test execution.\n- **Query Execution**: Executes the generated SQL queries and compares the results with expected outputs.\n- **Detailed Reporting**: Generates markdown reports with metrics such as success rates, token usage, and execution times.\n\n## Usage\n\n### Importing the Builder\n\n```typescript\nimport {generationAcceptanceBuilder} from './generation.acceptance.builder';\n```\n\n### Running Tests\n\nTo use the builder, define your test cases as an array of `GenerationAcceptanceTestCase` objects and pass them to the `generationAcceptanceBuilder` function along with the required parameters.\n\n#### Example\n\n```typescript\nconst testCases = [\n  {\n    case: 'Test Case 1',\n    prompt: 'Find all the active resources',\n    outputInstructions:\n      'The output should have a single column `resource_name` arranged in alphabetical order.',\n    resultQuery:\n      'SELECT name as resource_name FROM resource WHERE status = 1 ORDER BY name',\n    count: 1,\n  },\n];\n\nconst result = await generationAcceptanceBuilder(\n  testCases,\n  client,\n  app,\n  1,\n  true,\n);\nconsole.log(result);\n```\n\n### Parameters\n\n- `cases`: An array of test cases to execute.\n- `client`: The LoopBack test client.\n- `app`: The LoopBack application instance.\n- `countPerPrompt`: Number of iterations per test case (default: 1).\n- `writeReport`: Whether to generate a markdown report (default: false).\n\n### Test Case Structure\n\nEach test case should follow the `GenerationAcceptanceTestCase` interface:\n\n```typescript\ninterface GenerationAcceptanceTestCase {\n  case: string; // Name of the test case\n  prompt: string; // Prompt to send to the LLM\n  outputInstructions: string; // Additional instructions for the output\n  resultQuery: string; // Expected SQL query\n  count?: number; // Number of iterations (optional)\n  only?: boolean; // Run only this test case (optional)\n  skip?: boolean; // Skip this test case (optional)\n}\n```\n\n## Report Generation\n\nThe builder generates a markdown report summarizing the test results. The report includes:\n\n- Success metrics\n- Time metrics\n- Token usage metrics\n- Detailed results for each test case\n- Failed queries with actual and expected results\n\nThe report is saved in the `llm-reports` directory with a filename based on the model name.\n\n## Environment Variables\n\nThe builder relies on the following environment variables:\n\n- `SAMPLE_DEAL_NAME`: Default value for `\u003ctestDeal\u003e` placeholder.\n- `TEST_TENANT_ID`: Tenant ID for token generation.\n- `JWT_SECRET`: Secret key for signing JWT tokens.\n- `JWT_ISSUER`: Issuer for JWT tokens.\n\n## Dependencies\n\n- `@loopback/testlab`\n- `@loopback/core`\n- `@loopback/repository`\n- `@sourceloop/core`\n- `jsonwebtoken`\n- `crypto`\n- `fs`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcefuse%2Floopback4-llm-chat-extension","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsourcefuse%2Floopback4-llm-chat-extension","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcefuse%2Floopback4-llm-chat-extension/lists"}