{"id":18395853,"url":"https://github.com/deepset-ai/haystack-sagemaker","last_synced_at":"2025-07-23T11:05:00.290Z","repository":{"id":176931992,"uuid":"656258501","full_name":"deepset-ai/haystack-sagemaker","owner":"deepset-ai","description":"🚀 This repo is a showcase of how you can use models deployed on AWS SageMaker in your Haystack Retrieval Augmented Generative AI pipelines","archived":false,"fork":false,"pushed_at":"2023-07-27T20:42:43.000Z","size":1107,"stargazers_count":13,"open_issues_count":2,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-07T03:36:09.578Z","etag":null,"topics":["aws","haystack","llm","nlp","opensearch","sagemaker"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepset-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-20T15:15:49.000Z","updated_at":"2024-01-26T07:25:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"a7f0bb32-3660-41e8-8304-f108e1808303","html_url":"https://github.com/deepset-ai/haystack-sagemaker","commit_stats":null,"previous_names":["deepset-ai/haystack-sagemaker"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/deepset-ai/haystack-sagemaker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Fhaystack-sagemaker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Fhaystack-sagemaker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Fhaystack-sagemaker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Fhaystack-sagemaker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepset-ai","download_url":"https://codeload.github.com/deepset-ai/haystack-sagemaker/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Fhaystack-sagemaker/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266665785,"owners_count":23964973,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","haystack","llm","nlp","opensearch","sagemaker"],"created_at":"2024-11-06T02:12:10.091Z","updated_at":"2025-07-23T11:05:00.262Z","avatar_url":"https://github.com/deepset-ai.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Haystack Retrieval-Augmented Generative QA Pipelines with SageMaker JumpStart\nThis repo is a showcase of how you can use models deployed on [SageMaker JumpStart](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html) in your Haystack Retrieval Augmented Generative AI pipelines.\n\n**Instructions:**  \n- [Starting an OpenSearch service](#starting-an-opensearch-service)  \n- [Indexing Documents to OpenSearch](#the-indexing-pipeline-write-documents-to-opensearch)   \n- [The RAG Pipeline](#the-rag-pipeline)   \n\n**The Repo Structure**  \nThis repository contains 2 runnable Python scripts for indexing and the retrieval augmented pipeline respectively,  with instructions on how to run them below:\n\n`opensearch_indexing_pipeline.py`\n\n`rag_pipeline.py`\n\n We've also included notebooks for them both in `notebooks/` which you can optionally use to create and run each pipeline step by step.\n\n**Prerequisites**  \nTo run the Haystack pipelines and use the models from SageMaker in this repository, you need an AWS account, and we suggest setting up [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) on your machine.\n\n\n## The Data\nThis showcase includes some documents we've crawled from the OpenSearch website and documentation pages. \nYou can index these into your own `OpenSearchDocumentStore` using `opensearch_indexing_pipeline.py` or `notebooks/opensearch_indexing_pipeline.ipynb`.\n\n## The Model\nFor this demo, we deployed the `falcon-40b-instruct` model on SageMaker Jumpstart. Once deployed, you can use your own credentials in the `PromptNode`.\nTo deploy a model on JumpStart, simply log in to your AWS account and go to the Studio on SageMaker. \nNavigate to JumpStart and deploy `falcon-40b-instruct`. This may take a few minutes:\n\u003cimg width=\"949\" alt=\"image\" src=\"https://github.com/deepset-ai/haystack-sagemaker/assets/15802862/b7a1adee-eb9c-4258-b3e0-bf5942f9c960\"\u003e\n\n## Starting an OpenSearch service\n### Option 1: OpenSearch service on AWS\n**Requirements:** An AWS account and AWS CLI\n\nYou can use the provided CloudFormation template `opsearch-index.yaml` to deploy an OpenSearch service on AWS.\n\nSet the `--stack-name` and `OSPassword` to your preferred values and run the following.\nYou may also change the default `OSDomainName` and `OSUsername` values, set to `opensearch-haystack-domain` and `admin` respectively, in `opensearch-index.yaml`\n\n```bash\naws cloudformation create-stack --stack-name HaystackOpensearch --template-body file://cloudformation/opensearch-index.yaml --parameters ParameterKey=InstanceType,ParameterValue=r5.large.search ParameterKey=InstanceCount,ParameterValue=3 ParameterKey=OSPassword,ParameterValue=Password123!\n```\nYou can then retrieve your OpenSearch host required to [Write documents](#writing-documents) by running:\n```bash\naws cloudformation describe-stacks --stack-name HaystackOpensearch --query \"Stacks[0].Outputs[?OutputKey=='OpenSearchEndpoint'].OutputValue\" --output text\n```\n### Option 2: Local OpenSearch service\n**Requirements:** Docker\n\nAnother option is to have a local OpenSearch service. For this, you may simply run:\n```python\nfrom haystack.utils import launch_opensearch\n\nlaunch_opensearch()\n```\nThis will start an OpenSearch service on `localhost:9200`\n\n## The Indexing Pipeline: Write Documents to OpenSearch\nTo run the scripts and notebooks provided here, first clone the repo and install the requirements.\n```bash\ngit clone git@github.com:deepset-ai/haystack-sagemaker.git\ncd haystack-sagemaker\npip install -r requirements.txt\n```\n\n### Writing documents\nYou can use a Haystack indexing pipeline to prepare and write documents to an `OpenSearchDocumentStore`.\n1. Set your environment variables:\n```bash\nexport OPENSEARCH_HOST='your_opensearch_host'\nexport OPENSEARCH_PORT='your_opensearch_port'\nexport OPENSEARCH_USERNAME='your_opensearch_username'\nexport OPENSEARCH_PASSWORD='your_opensearch_password'\n```\n2. Use the indexing pipeline to write the preprocessed documents to your OpenSearch index:\n#### Option 1:\nFor this demo, we've prepared documents which have been crawled from the OpenSearch documentation and website. As an example of how you may use an S3 bucket, we've also made them available [here](https://haystack-public-demo-files.s3.eu-central-1.amazonaws.com/haystack-sagemaker-demo/opensearch-documentation-2.7.json) and [here](https://haystack-public-demo-files.s3.eu-central-1.amazonaws.com/haystack-sagemaker-demo/opensearch-website.json)\n\nRun `python opensearch_indexing_pipeline.py --fetch-files` to fetch these 2 files from S3 or modify the source code in `opensearch_indexing_pipeline.py` to fetch your own files from an S3 bucket. This will fetch the specified files from the S3 bucket, and put them in `data/`. The script will then preprocess and prepare `Documents` from these files, and write them to your `OpenSearchDocumentStore`.\n\n#### Option 2:\nRun `python opensearch_indexing_pipeline.py`\n\nThis will write the same files, already available in `data/`, to your `OpenSearchDocumentStore`\n\n\n## The RAG Pipeline\n\nAn indexing pipeline prepares and writes documents to a `DocumentStore` so that they are in a format which is useable by your choice of NLP pipeline and language models.\n\nOn the other hand, a query pipeline is any combination of Haystack nodes that may consume a user query and result in a response.\nHere, you will find a retrieval augmented question answering pipeine in `rag_pipeline.py`.\n\n```bash\nexport SAGEMAKER_MODEL_ENDPOINT=your_falcon_40b_instruc_endpoint\nexport AWS_PROFILE_NAME=your_aws_profile\nexport AWS_REGION_NAME=your_aws_region\n```\n\nRunning the following will start a retrieval augmented QA pipeline with the prompt defined in the `PromptTemplate`. Feel free to modify this template or even use one of our prompts from the [PromptHub](https://prompthub.deepset.ai) to experiment with different instructions.\n\n```bash\npython rag_pipeline.py\n```\n\nThen, ask some questions about OpenSearch 🥳 👇\n\nhttps://github.com/deepset-ai/haystack-sagemaker/assets/15802862/40563962-2d75-415b-bac4-b25eaa5798e5\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepset-ai%2Fhaystack-sagemaker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepset-ai%2Fhaystack-sagemaker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepset-ai%2Fhaystack-sagemaker/lists"}