{"id":15442343,"url":"https://github.com/langchain-ai/data-enrichment-js","last_synced_at":"2025-08-24T22:33:14.919Z","repository":{"id":256843047,"uuid":"844782545","full_name":"langchain-ai/data-enrichment-js","owner":"langchain-ai","description":null,"archived":false,"fork":false,"pushed_at":"2024-09-19T18:21:46.000Z","size":2626,"stargazers_count":4,"open_issues_count":1,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-15T08:23:50.614Z","etag":null,"topics":["langgraph","langgraph-js","langgraph-template"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/langchain-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-20T00:45:57.000Z","updated_at":"2024-11-27T02:26:17.000Z","dependencies_parsed_at":"2024-10-27T16:29:58.572Z","dependency_job_id":"fa838afe-e928-491d-9107-7617b3068b6c","html_url":"https://github.com/langchain-ai/data-enrichment-js","commit_stats":{"total_commits":16,"total_committers":3,"mean_commits":5.333333333333333,"dds":0.5,"last_synced_commit":"46a181b01cbf654c3d9e8f734e06a5b16a9aeb59"},"previous_names":["langchain-ai/data-enrichment-js"],"tags_count":0,"template":true,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langchain-ai%2Fdata-enrichment-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langchain-ai%2Fdata-enrichment-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langchain-ai%2Fdata-enrichment-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langchain-ai%2Fdata-enrichment-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/langchain-ai","download_url":"https://codeload.github.com/langchain-ai/data-enrichment-js/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230845680,"owners_count":18289042,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["langgraph","langgraph-js","langgraph-template"],"created_at":"2024-10-01T19:26:47.722Z","updated_at":"2024-12-22T15:08:34.806Z","avatar_url":"https://github.com/langchain-ai.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LangGraph Studio Data Enrichment Template\n\n[![CI](https://github.com/langchain-ai/data-enrichment-js/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/langchain-ai/data-enrichment-js/actions/workflows/unit-tests.yml)\n[![Integration Tests](https://github.com/langchain-ai/data-enrichment-js/actions/workflows/integration-tests.yml/badge.svg)](https://github.com/langchain-ai/data-enrichment-js/actions/workflows/integration-tests.yml)\n[![Open in - LangGraph Studio](https://img.shields.io/badge/Open_in-LangGraph_Studio-00324d.svg?logo=data:image/svg%2bxml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI4NS4zMzMiIGhlaWdodD0iODUuMzMzIiB2ZXJzaW9uPSIxLjAiIHZpZXdCb3g9IjAgMCA2NCA2NCI+PHBhdGggZD0iTTEzIDcuOGMtNi4zIDMuMS03LjEgNi4zLTYuOCAyNS43LjQgMjQuNi4zIDI0LjUgMjUuOSAyNC41QzU3LjUgNTggNTggNTcuNSA1OCAzMi4zIDU4IDcuMyA1Ni43IDYgMzIgNmMtMTIuOCAwLTE2LjEuMy0xOSAxLjhtMzcuNiAxNi42YzIuOCAyLjggMy40IDQuMiAzLjQgNy42cy0uNiA0LjgtMy40IDcuNkw0Ny4yIDQzSDE2LjhsLTMuNC0zLjRjLTQuOC00LjgtNC44LTEwLjQgMC0xNS4ybDMuNC0zLjRoMzAuNHoiLz48cGF0aCBkPSJNMTguOSAyNS42Yy0xLjEgMS4zLTEgMS43LjQgMi41LjkuNiAxLjcgMS44IDEuNyAyLjcgMCAxIC43IDIuOCAxLjYgNC4xIDEuNCAxLjkgMS40IDIuNS4zIDMuMi0xIC42LS42LjkgMS40LjkgMS41IDAgMi43LS41IDIuNy0xIDAtLjYgMS4xLS44IDIuNi0uNGwyLjYuNy0xLjgtMi45Yy01LjktOS4zLTkuNC0xMi4zLTExLjUtOS44TTM5IDI2YzAgMS4xLS45IDIuNS0yIDMuMi0yLjQgMS41LTIuNiAzLjQtLjUgNC4yLjguMyAyIDEuNyAyLjUgMy4xLjYgMS41IDEuNCAyLjMgMiAyIDEuNS0uOSAxLjItMy41LS40LTMuNS0yLjEgMC0yLjgtMi44LS44LTMuMyAxLjYtLjQgMS42LS41IDAtLjYtMS4xLS4xLTEuNS0uNi0xLjItMS42LjctMS43IDMuMy0yLjEgMy41LS41LjEuNS4yIDEuNi4zIDIuMiAwIC43LjkgMS40IDEuOSAxLjYgMi4xLjQgMi4zLTIuMy4yLTMuMi0uOC0uMy0yLTEuNy0yLjUtMy4xLTEuMS0zLTMtMy4zLTMtLjUiLz48L3N2Zz4=)](https://langgraph-studio.vercel.app/templates/open?githubUrl=https://github.com/langchain-ai/data-enrichment-js)\n\nProducing structured results (e.g., to populate a database or spreadsheet) from open-ended research (e.g., web research) is a common use case that LLM-powered agents are well-suited to handle. Here, we provide a general template for this kind of \"data enrichment agent\" agent using [LangGraph](https://github.com/langchain-ai/langgraph) in [LangGraph Studio](https://github.com/langchain-ai/langgraph-studio). It contains an example graph exported from `src/enrichment_agent/graph.ts` that implements a research assistant capable of automatically gathering information on various topics from the web and structuring the results into a user-defined JSON format.\n\n![Overview of agent](./static/overview.png)\n\n![](/static/studio.png)\n\n# What it does\n\nThe enrichment agent defined in `src/enrichment_agent/graph.ts` performs the following steps:\n\n1. Takes a research **topic** and requested **extractionSchema** as input.\n2. Searches the web for relevant information\n3. Reads and extracts key details from websites\n4. Organizes the findings into the requested structured format\n5. Validates the gathered information for completeness and accuracy\n\n![Graph view in LangGraph studio UI](./static/studio.png)\n\n## Getting Started\n\nYou will need the latest versions of `@langchain/langgraph` and `@langchain/core`. See these instructions for help upgrading an [existing project](https://langchain-ai.github.io/langgraphjs/how-tos/manage-ecosystem-dependencies/).\n\nAssuming you have already [installed LangGraph Studio](https://github.com/langchain-ai/langgraph-studio?tab=readme-ov-file#download), to set up:\n\n1. Create a `.env` file.\n\n```bash\ncp .env.example .env\n```\n\n2. Define required API keys in your `.env` file.\n\nThe primary [search tool](./src/enrichment_agent/tools.ts) [^1] used is [Tavily](https://tavily.com/). Create an API key [here](https://app.tavily.com/sign-in).\n\n\u003c!--\nSetup instruction auto-generated by `langgraph template lock`. DO NOT EDIT MANUALLY.\n--\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eSetup for `model`\u003c/summary\u003e\nThe `llm` configuration defaults are shown below:\n\n```yaml\nmodel: anthropic/claude-3-5-sonnet-20240620\n```\n\nFollow the instructions below to get set up, or pick one of the additional options.\n\n### Anthropic Chat Models\n\nTo use Anthropic's chat models:\n\n1. Sign up for an [Anthropic API key](https://console.anthropic.com/) if you haven't already.\n2. Once you have your API key, add it to your `.env` file:\n\n```\nANTHROPIC_API_KEY=your-api-key\n```\n\n### Fireworks Chat Models\n\nTo use Fireworks AI's chat models:\n\n1. Sign up for a [Fireworks AI account](https://app.fireworks.ai/signup) and obtain an API key.\n2. Add your Fireworks AI API key to your `.env` file:\n\n```\nFIREWORKS_API_KEY=your-api-key\n```\n\n#### OpenAI Chat Models\n\nTo use OpenAI's chat models:\n\n1. Sign up for an [OpenAI API key](https://platform.openai.com/signup).\n2. Once you have your API key, add it to your `.env` file:\n\n```\nOPENAI_API_KEY=your-api-key\n```\n\n\u003c/details\u003e\n\n\u003c!--\nEnd setup instructions\n--\u003e\n\n3. Consider a research topic and desired extraction schema.\n\nAs an example, here is a research topic we can consider:\n\n```\n\"Autonomous agents\"\n```\n\nWith an `extractionSchema` of:\n\n```json\n{\n  \"type\": \"object\",\n  \"properties\": {\n    \"facts\": {\n      \"type\": \"array\",\n      \"description\": \"An array of facts retrieved from the provided sources\",\n      \"items\": {\n        \"type\": \"string\"\n      }\n    }\n  },\n  \"required\": [\"facts\"]\n}\n```\n\nAnother example topic with a more complex schema is:\n\n```\n\"Top 5 chip providers for LLM Training\"\n```\n\nAnd here is a desired `extractionSchema`:\n\n```json\n{\n  \"type\": \"object\",\n  \"properties\": {\n    \"companies\": {\n      \"type\": \"array\",\n      \"items\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"name\": {\n            \"type\": \"string\",\n            \"description\": \"Company name\"\n          },\n          \"technologies\": {\n            \"type\": \"string\",\n            \"description\": \"Brief summary of key technologies used by the company\"\n          },\n          \"market_share\": {\n            \"type\": \"string\",\n            \"description\": \"Overview of market share for this company\"\n          },\n          \"future_outlook\": {\n            \"type\": \"string\",\n            \"description\": \"Brief summary of future prospects and developments in the field for this company\"\n          },\n          \"key_powers\": {\n            \"type\": \"string\",\n            \"description\": \"Which of the 7 Powers (Scale Economies, Network Economies, Counter Positioning, Switching Costs, Branding, Cornered Resource, Process Power) best describe this company's competitive advantage\"\n          }\n        },\n        \"required\": [\"name\", \"technologies\", \"market_share\", \"future_outlook\"]\n      },\n      \"description\": \"List of companies\"\n    }\n  },\n  \"required\": [\"companies\"]\n}\n```\n\n4. Open the folder LangGraph Studio, and input `topic` and `extractionSchema`.\n\n## How to customize\n\n1. **Customize research targets**: Provide a custom JSON `extractionSchema` when calling the graph to gather different types of information.\n2. **Select a different model**: We default to anthropic (`claude-3-5-sonnet-20240620`). You can select a compatible chat model using `provider/model-name` via configuration. Example: `openai/gpt-4o-mini`.\n3. **Customize the prompt**: We provide a default prompt in [src/enrichment_agent/prompts.ts](./src/enrichment_agent/prompts.ts). You can easily update this via configuration.\n\nFor quick prototyping, these configurations can be set in the studio UI.\n\n![Config In Studio](./static/config.png)\n\nYou can also quickly extend this template by:\n\n- Adding new tools and API connections in [src/enrichment_agent/tools.ts](./src/enrichment_agent/tools.ts). These are just any TypeScript functions.\n- Adding additional steps in [src/enrichment_agent/graph.ts](./src/enrichment_agent/graph.ts).\n\n## Development\n\nWhile iterating on your graph, you can edit past state and rerun your app from past states to debug specific nodes. Local changes will be automatically applied via hot reload. Try adding an interrupt before the agent calls tools, updating the default system message in [src/enrichment_agent/utils.ts](./src/enrichment_agent/utils.ts) to take on a persona, or adding additional nodes and edges!\n\nFollow up requests will be appended to the same thread. You can create an entirely new thread, clearing previous history, using the `+` button in the top right.\n\nYou can find the latest (under construction) docs on [LangGraph.js](https://langchain-ai.github.io/langgraphjs/) here, including examples and other references. Using those guides can help you pick the right patterns to adapt here for your use case.\n\nLangGraph Studio also integrates with [LangSmith](https://smith.langchain.com/) for more in-depth tracing and collaboration with teammates.\n\n[^1]: https://js.langchain.com/docs/concepts#tools\n\n\u003c!--\nConfiguration auto-generated by `langgraph template lock`. DO NOT EDIT MANUALLY.\n{\n  \"config_schemas\": {\n    \"agent\": {\n      \"type\": \"object\",\n      \"properties\": {\n        \"model\": {\n          \"type\": \"string\",\n          \"default\": \"anthropic/claude-3-5-sonnet-20240620\",\n          \"description\": \"The name of the language model to use for the agent. Should be in the form: provider/model-name.\",\n          \"environment\": [\n            {\n              \"value\": \"anthropic/claude-1.2\",\n              \"variables\": \"ANTHROPIC_API_KEY\"\n            },\n            {\n              \"value\": \"anthropic/claude-2.0\",\n              \"variables\": \"ANTHROPIC_API_KEY\"\n            },\n            {\n              \"value\": \"anthropic/claude-2.1\",\n              \"variables\": \"ANTHROPIC_API_KEY\"\n            },\n            {\n              \"value\": \"anthropic/claude-3-5-sonnet-20240620\",\n              \"variables\": \"ANTHROPIC_API_KEY\"\n            },\n            {\n              \"value\": \"anthropic/claude-3-haiku-20240307\",\n              \"variables\": \"ANTHROPIC_API_KEY\"\n            },\n            {\n              \"value\": \"anthropic/claude-3-opus-20240229\",\n              \"variables\": \"ANTHROPIC_API_KEY\"\n            },\n            {\n              \"value\": \"anthropic/claude-3-sonnet-20240229\",\n              \"variables\": \"ANTHROPIC_API_KEY\"\n            },\n            {\n              \"value\": \"anthropic/claude-instant-1.2\",\n              \"variables\": \"ANTHROPIC_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-3.5-turbo\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-3.5-turbo-0125\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-3.5-turbo-0301\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-3.5-turbo-0613\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-3.5-turbo-1106\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-3.5-turbo-16k\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-3.5-turbo-16k-0613\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-0125-preview\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-0314\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-0613\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-1106-preview\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-32k\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-32k-0314\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-32k-0613\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-turbo\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-turbo-preview\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4-vision-preview\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4o\",\n              \"variables\": \"OPENAI_API_KEY\"\n            },\n            {\n              \"value\": \"openai/gpt-4o-mini\",\n              \"variables\": \"OPENAI_API_KEY\"\n            }\n          ]\n        }\n      },\n      \"environment\": [\n        \"TAVILY_API_KEY\"\n      ]\n    }\n  }\n}\n--\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flangchain-ai%2Fdata-enrichment-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flangchain-ai%2Fdata-enrichment-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flangchain-ai%2Fdata-enrichment-js/lists"}