{"id":19809556,"url":"https://github.com/didier-durand/qstensils","last_synced_at":"2025-05-01T08:30:51.534Z","repository":{"id":223872612,"uuid":"761580218","full_name":"didier-durand/qstensils","owner":"didier-durand","description":"Tools and scripts for Amazon Q operations and explorations","archived":false,"fork":false,"pushed_at":"2024-04-11T04:07:46.000Z","size":141,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-04-11T05:49:18.106Z","etag":null,"topics":["amazon","amazonq","aws","generative-ai","qbusiness","retrieval-augmented-generation"],"latest_commit_sha":null,"homepage":"https://aws.amazon.com/q/business-expert/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/didier-durand.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-22T05:08:02.000Z","updated_at":"2024-03-31T18:48:53.000Z","dependencies_parsed_at":"2024-03-14T16:41:33.452Z","dependency_job_id":"42992bc8-aa56-4baa-b649-4c880b904270","html_url":"https://github.com/didier-durand/qstensils","commit_stats":null,"previous_names":["didier-durand/qstensils"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/didier-durand%2Fqstensils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/didier-durand%2Fqstensils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/didier-durand%2Fqstensils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/didier-durand%2Fqstensils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/didier-durand","download_url":"https://codeload.github.com/didier-durand/qstensils/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224245661,"owners_count":17279649,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon","amazonq","aws","generative-ai","qbusiness","retrieval-augmented-generation"],"created_at":"2024-11-12T09:17:20.380Z","updated_at":"2024-11-12T09:17:21.083Z","avatar_url":"https://github.com/didier-durand.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"img/q-logo.png\" height=\"120\" alt=\"Q logo\"\u003e\n\n## qstensils: tools and utility scripts for Amazon Q\n\n![AWS](https://img.shields.io/badge/AWS-%23FF9900.svg?style=for-the-badge\u0026logo=amazon-aws\u0026logoColor=white)\n![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge\u0026logo=python\u0026logoColor=ffdd54)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Codacy Badge](https://app.codacy.com/project/badge/Grade/1c826b70f5dd4b45b350c0337f75075d)](https://app.codacy.com/gh/didier-durand/qstensils/dashboard?utm_source=gh\u0026utm_medium=referral\u0026utm_content=\u0026utm_campaign=Badge_grade)\n[![build workflow](https://github.com/didier-durand/qstensils/actions/workflows/build.yml/badge.svg)](https://github.com/didier-durand/qstensils/actions)\n\n * [Provided Tools ](#provided-tools)\n * [About Amazon Q](#about-amazon-q) \n * [Usage](#usage)\n * [Security](#security)\n\n### Provided Tools\n\n**Disclaimer**: _the scripts provided in this repository reflect the state of service Amazon Q for Business as it was launched \nin December 2023 by AWS in Preview mode. Due to this Preview mode, the features of Amazon Q and their \nimplementation can change at any time during Preview and for General Availability. Such changes may require updates to this repository._\n\nThis project gathers diverse tools and utility scripts to explore and operate Amazon Q for Business. \nWe will add new scripts based on your demand: feel free to cut a ticket\n[here](https://github.com/didier-durand/qstensils/issues) if you have a need or idea!\n\nWe currently provide the following utilities:\n1. [q_list_applications](doc/q_list_applications.md) to inventory the applications existing in a given region of an AWS account. The returned \njson structure details the various components (indices, data source, retrievers, etc.) of those Amazon Q \napplications.\n2. [q_list_data_source_sync_jobs](doc/q_list_data_source_sync_jobs.md) to list the history of index synchronization \njobs executed on a given Q data source. This script adds additional metric like total job duration, document scan rate \nand average scan duration per document.\n3. [q_list_documents](doc/q_list_documents.md) to list all the documents of an Amazon Q index and get all their associated metadata,\nin particular their status. The returned list can be filtered (via inclusion or exclusion) to return \nonly a fraction of those documents for example based on their indexing status.\n4. [q_list_conversations](doc/q_list_conversations.md) to obtain all list of all past conversations between \na given application and a user as remembered by Amazon Q.\n5. [q_chat](doc/q_chat.md) to be able to script conversations (based on single or multiple messages) with the \nassistant of Amazon Q.\n\nAll those scripts return json structures that can be further processed in [shell pipelines](https://en.wikipedia.org/wiki/Pipeline_(Unix)) with various utilities \nlike jq, sed, awk, etc.\n\nThose scripts rely on the Python AWS SDK. All APIs related to Q for business are described in details in the \n[SDK boto3 public documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/qbusiness.html).\n\n### About Amazon Q\n\n[![IMAGE ALT TEXT](http://img.youtube.com/vi/bZsIPinetV4/0.jpg)](http://www.youtube.com/watch?v=bZsIPinetV4 \"Amazon Q\")\n\nAmazon Q is a fully managed, generative-AI powered assistant that can be configured to answer questions, \nprovide summaries, generate content, and complete tasks based on data in your enterprise. Amazon Q \nprovides immediate and relevant information to its users, and helps streamline tasks and \naccelerate problem-solving.\n\nAn Amazon Q application relies on a corpus of documents to build its specific Q index. This corpus of documents is \nstored in one or more document repositories (S3, Jira, Quip, etc.) called Q data sources. The answers to user questions \nby the assistant will be prepared through the leverage of RAG technology. \n\n[Retrieval-Augmented Generation](https://www.promptingguide.ai/techniques/rag) (RAG) is a natural language processing (NLP) technique. It is composed of a \nlanguage model-based system, usually a [Large Language Model](https://en.wikipedia.org/wiki/Large_language_model) (LLM), that accesses external knowledge sources \nto complete tasks. This enables more contextuality, factual consistency, improves reliability of the generated \nresponses, and helps to mitigate the problem of \"hallucinations\".\n\n### Security\n\nThe scripts of this project assume that the AWS user reflected by environment variables named `AWS_ACCESS_KEY_ID` and \n`AWS_SECRET_ACCESS_KEY` has proper [IAM credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) in terms of authorizations \nto access the APIs of Amazon Q in the AWS account. See \n[IAM policy examples](https://docs.aws.amazon.com/amazonq/latest/business-use-dg/security_iam_id-based-policy-examples.html) \nin the [Security section](https://docs.aws.amazon.com/amazonq/latest/business-use-dg/security-iam.html) of Q Documentation for all details.\n\n### Usage\n\nfrom the /src directory of this project, the following commands can be used to get all command options\n\nfor list_applications.py\n\n```\n%python3 q_list_applications.py -h\nusage: q_list_applications.py [-h] [-v]\n\nlist applications, indexes, retrievers, web experiences, plugins, etc. running in Amazon Q for business\n\noptions:\n  -h, --help     show this help message and exit\n  -v, --verbose  verbose mode\n```\n\nq_list_data_source_sync_jobs\n\n```\n % python3 q_list_data_source_sync_jobs.py -h\nusage: q_list_data_source_sync_jobs.py [-h] [-a APP_ID] [-i IDX_ID] [-d DS_ID] [-v]\n\nlist synchronization jobs executed for a given data source of an Amazon Q application\n\noptions:\n  -h, --help            show this help message and exit\n  -a APP_ID, --app_id APP_ID\n                        Q application id\n  -i IDX_ID, --idx_id IDX_ID\n                        Q data source id\n  -d DS_ID, --ds_id DS_ID\n                        Q data source id\n  -v, --verbose         verbose mode\n```\n\nfor q_list_documents.py\n\n```\n% python3 q_list_documents.py  -h                                                                                                   \nusage: q_list_documents.py [-h] [-a APP_ID] [-i IDX_ID] [-incl INCLUDE] [-excl EXCLUDE] [-inv] [-v]\n\nlist documents indexed by Amazon Q\n\noptions:\n  -h, --help            show this help message and exit\n  -a APP_ID, --app_id APP_ID\n                        Q application id\n  -i IDX_ID, --idx_id IDX_ID\n                        Q index id\n  -incl INCLUDE, --include INCLUDE\n                        comma-separated list of status to include\n  -excl EXCLUDE, --exclude EXCLUDE\n                        comma-separated list of status to exclude\n  -inv, --inventory     with document inventory\n  -v, --verbose         verbose mode\n```\n\nfor q_list_conversations.py\n\n```\n% python3 q_list_conversations.py -h\nusage: q_list_conversations.py [-h] [-a APP_ID] [-u USR_ID] [-v]\n\nlist documents indexed by Amazon Q\n\noptions:\n  -h, --help            show this help message and exit\n  -a APP_ID, --app_id APP_ID\n                        Q application id\n  -u USR_ID, --usr_id USR_ID\n                        Q user id\n  -v, --verbose         verbose mode\n```\nfor q_chat.py\n\n```\n% python3 q_chat.py  -h\nusage: q_chat.py [-h] [-a APP_ID] [-u USR_ID] [-p PROMPT] [-f FILE] [-c CNV_ID] [-m MSG_ID] [-d] [-v]\n\nask a question to a Q application and get answer\n\noptions:\n  -h, --help            show this help message and exit\n  -a APP_ID, --app_id APP_ID\n                        Q application id\n  -u USR_ID, --usr_id USR_ID\n                        Q index id\n  -p PROMPT, --prompt PROMPT\n                        question prompt or path to file with list of prompts\n  -f FILE, --file FILE  path to attachment file\n  -c CNV_ID, --cnv_id CNV_ID\n                        Q conversation id (only to continue an existing conversation)\n  -m MSG_ID, --msg_id MSG_ID\n                        Q parent message id (only to continue an existing conversation)\n  -d, --details         full conversation details\n  -v, --verbose         verbose mode\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdidier-durand%2Fqstensils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdidier-durand%2Fqstensils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdidier-durand%2Fqstensils/lists"}