{"id":41343020,"url":"https://github.com/rodekruis/hia-search","last_synced_at":"2026-01-23T06:53:10.470Z","repository":{"id":276980553,"uuid":"930870705","full_name":"rodekruis/hia-search","owner":"rodekruis","description":"HIA's search and chat engine","archived":false,"fork":false,"pushed_at":"2025-12-30T14:44:31.000Z","size":825,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-01-03T08:19:07.818Z","etag":null,"topics":["cea"],"latest_commit_sha":null,"homepage":"https://hia-search.azurewebsites.net","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rodekruis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-02-11T10:52:00.000Z","updated_at":"2025-12-30T14:44:35.000Z","dependencies_parsed_at":"2025-02-11T14:23:22.969Z","dependency_job_id":"155aa755-034e-4f27-a952-493a30719dfe","html_url":"https://github.com/rodekruis/hia-search","commit_stats":null,"previous_names":["rodekruis/hia-search"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rodekruis/hia-search","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodekruis%2Fhia-search","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodekruis%2Fhia-search/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodekruis%2Fhia-search/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodekruis%2Fhia-search/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rodekruis","download_url":"https://codeload.github.com/rodekruis/hia-search/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodekruis%2Fhia-search/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28682263,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-23T05:48:07.525Z","status":"ssl_error","status_checked_at":"2026-01-23T05:48:07.129Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cea"],"created_at":"2026-01-23T06:53:09.736Z","updated_at":"2026-01-23T06:53:10.458Z","avatar_url":"https://github.com/rodekruis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `hia-search`\n\nSearch and chat with [HIA](https://github.com/rodekruis/helpful-information).\n\n## Description\n\nSynopsis: a [dockerized](https://www.docker.com/) [Python](https://www.python.org/) API to search and chat with [HIA](https://github.com/rodekruis/helpful-information).\n\nBased on [LangChain](https://github.com/langchain-ai/langchain), powered by language models. Uses [Poetry](https://python-poetry.org/) for dependency management and [Azure AI Search](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search) for indexing and searching.\n\nInspired by [`knowledge-enriched-chatbot`](https://github.com/deloitte-nl/knowledge-enriched-chatbot) and [`HIA-search-engine`](https://github.com/PetrovViktor/HIA-search-engine), kudos to the authors.\n\n## Usage\n\n### 1. Set up HIA\n\n[Set up a HIA instance](https://github.com/rodekruis/helpful-information/blob/main/docs/Guide-How_to_set_up_an_instance.md) and populate its content.\n\n### 2. Set up the search service\n\n  - Find the item called `HIA Search API-key(s) [production]` in Bitwarden and copy the value of `API_KEY`\n  - Go to the HIA repository's \"**Settings**\" \u003e \"**Secret and variables**\" \u003e \"**Actions**\" \u003e \"**New repository secret**\"\n  - Name the secret: `SEARCH_API_KEY`and insert the API Key as value\n  - In the HIA repository, go to the file: `.github/workflows/deploy-github-pages.yml` and click \"**Edit this file**\" (pencil icon)\n  - Under `jobs` \u003e `deploy` \u003e `steps` \u003e `Build` \u003e `env` add the two variables `SEARCH_API` and `SEARCH_API_KEY` as follows\n\n  ```yaml\n        - name: Build\n        working-directory: 'helpful-information'\n        # NOTE: When the instance will be run on a custom (sub-)domain, remove the `--base-href`-flag+value.\n        run: 'npm run build:production -- --output-path=../www --base-href=/${GITHUB_REPOSITORY#*/}/'\n        env:\n          # See all variables: https://github.com/rodekruis/helpful-information/blob/main/.env.example\n          NG_PRODUCTION: 'true'\n          GOOGLE_SHEETS_API_KEY: ${{ secrets.GOOGLE_SHEETS_API_KEY }}\n          GOOGLE_SHEETS_API_URL: 'https://sheets.googleapis.com/v4/spreadsheets'\n          SEARCH_API: 'https://hia-search.azurewebsites.net/search'\n          SEARCH_API_KEY: ${{ secrets.SEARCH_API_KEY }}\n  ```\n  - Redeploy HIA by triggering the deployment workflow: `Actions` \u003e `Deploy to GitHub Pages` \u003e `Run workflow`\n  \n\n### 2. Set up the chat service\n\nThe chat service is based on [Twilio incoming Messaging Webhooks](https://www.twilio.com/docs/usage/webhooks/messaging-webhooks#incoming-message-webhook). You need an active Twilio account and an active phone or WhatsApp number, see [how to buy one](https://help.twilio.com/articles/223135247-How-to-Search-for-and-Buy-a-Twilio-Phone-Number-from-Console).\n\nSelect the active number in Twilio and [configure the webhook](https://www.twilio.com/docs/messaging/tutorials/how-to-receive-and-reply/python#configure-your-webhook-url) using the `/chat-twilio-webhook` endpoint, with a `googleSheetId` as query parameter. Example:\n```\nhttps://hia-search.azurewebsites.net/chat-twilio-webhook?googleSheetId=14NZwDa8DNmH1q2Rxt-ojP9MZhJ-2GlOIyN8RF19iF04\n```\nThe answer will be sent via message directly to the user. The phone number of the user will be used as thread ID, for the chat model to remember the conversation.\n\n\u003e[!NOTE]\n\u003eThe instructions that the chatbot will follow are by default [these ones](config/rag_agent_prompt.txt). If you want to customize them, create a new sheet named `Chat` in your HIA Google Sheet file following [this template](https://docs.google.com/spreadsheets/d/1op6Ouyxtwv4f8GAEAMSn5PVzcXtfZuftMiLYWsX0pbs/edit?pli=1\u0026gid=1707339525#gid=1707339525), then insert the desired instructions under `#VALUE`, cell `B2`. Make sure to follow [best practices in prompt engineering](https://www.promptingguide.ai/introduction/tips); if it's the first time you do this, make sure the CEA Data Specialist reviews what you wrote.\n\n\n### 3. Keep your data up to date\n\nBoth the chat and search services need HIA content to be transformed into _embeddings_, i.e. numerical representations of text that capture semantic meaning in a high-dimensional vector space. Embeddings are stored in dedicated databases called _vector stores_. When searching or chatting with HIA, the user query \n\nGenerating and storing the embeddings of your specific HIA instance is done automatically the first time you call the chat or search endpoints, if the HIA Google Sheet file is publicly accessible. If the Google Sheet file is not publicly accessible or you need to update it, you can use the  `/create-vector-store` endpoint, which you can call directly from [the swagger UI](https://hia-search.azurewebsites.net). You will need to authenticate with `API_KEY_WRITE`, which you find in Bitwarden. See API reference below.\n\n\u003e[!NOTE]\n\u003e Currently, each search or chat service is linked to only one Google Sheets file, a.k.a. _region_ in the HIA terminology. If it is not clear what this means, see [how HIA works](https://github.com/rodekruis/helpful-information?tab=readme-ov-file#how-it-works).\n\n## API Reference\n\nIf not here, see the [`/docs`](https://hia-search.azurewebsites.net/docs).\n\n### `/create-vector-store`\n\nThe `/create-vector-store` endpoint accepts a `googleSheetId` body parameter, fetches all data from the Q\u0026A sheet, and creates an [index](https://learn.microsoft.com/en-us/azure/search/search-what-is-an-index) in Azure AI Search, Azure's native vector database. If the index already exists, its content will be updated.\n\nIf the Q\u0026A sheet is not publicly accessible, you can pass its content under the `data` body parameter. The content must be a valid JSON object structured as the [test-data from the `helpful-information`-app](https://github.com/rodekruis/helpful-information/blob/main/data/test-sheet-id-1/values/Q%26As.json).\n\n🔐 This endpoint is protected with the API_KEY_WRITE environment-variable, to prevent unauthorized users from modifying the index.\n\n### `/search`\n\nThe `/search` endpoint accepts three parameters:\n* `query`: the search query\n* `googleSheetId`: the Google Sheet ID\n* `lang`: the language of the search query; results will be translated to this language\n* `k`: the number of results to return\n\nand returns a list of relevant questions and answers, in this format:\n\n```json\n{\"results\":\n    [\n        {\n            \"categoryID\": 8,\n            \"subcategoryID\": 38,\n            \"slug\": \"disabilities\",\n            \"question\": \"Where can I go when I have special care needs?\",\n            \"answer\": \"To receive specialist care, you often first need a referral from your General Practitioner (GP).\",\n            \"score\": 0.3011241257190705,\n            \"children\": [\n                {\n                    \"categoryID\": 8,\n                    \"subcategoryID\": 38,\n                    \"question\": \"Social Security Act (WMO)\",\n                    \"answer\": \"Support for persons with disabilities is provided through the Social Security Act (WMO).\",\n                    \"score\": 0\n                }\n            ]\n        }\n    ]\n}\n ```\n\n This endpoint is protected with the `API_KEY` environment variable. As this key will be stored by the client-application in plain-text, visible in the browser, it should be considered public. Its main purpose is to prevent abuse of the API by unauthorized users (with possible future measures against it).\n\n## Configuration\n\n```sh\ncp example.env .env\n```\n\nEdit the provided [ENV-variables](./example.env) accordingly.\n\n### Run locally\n\n```sh\npip install poetry\npoetry install --no-root\npython -m spacy download en_core_web_sm\nuvicorn main:app --reload\n```\n\n### Run with Docker\n\n```sh\ndocker build -t hia-search .\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frodekruis%2Fhia-search","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frodekruis%2Fhia-search","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frodekruis%2Fhia-search/lists"}