{"id":37071441,"url":"https://github.com/helecloud/redshift-query","last_synced_at":"2026-01-14T08:22:58.311Z","repository":{"id":46082837,"uuid":"276961247","full_name":"helecloud/redshift-query","owner":"helecloud","description":"This is a very simple library that gets credentials of a cluster via redshift.GetClusterCredentials API call and then makes a connection to the cluster and runs the provided SQL statements.","archived":false,"fork":false,"pushed_at":"2022-12-26T21:32:18.000Z","size":52,"stargazers_count":5,"open_issues_count":2,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-10-26T21:09:13.646Z","etag":null,"topics":["etl","redshift","serverless"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/helecloud.png","metadata":{"files":{"readme":"README.md","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-03T18:11:47.000Z","updated_at":"2024-10-30T13:49:17.000Z","dependencies_parsed_at":"2023-01-31T01:45:36.259Z","dependency_job_id":null,"html_url":"https://github.com/helecloud/redshift-query","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/helecloud/redshift-query","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helecloud%2Fredshift-query","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helecloud%2Fredshift-query/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helecloud%2Fredshift-query/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helecloud%2Fredshift-query/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/helecloud","download_url":"https://codeload.github.com/helecloud/redshift-query/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helecloud%2Fredshift-query/sbom","scorecard":{"id":460031,"data":{"date":"2025-08-11","repo":{"name":"github.com/helecloud/redshift-query","commit":"391ab4dd98e594c35facbdf49162f30ed8c271bd"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.7,"checks":[{"name":"Code-Review","score":0,"reason":"Found 1/28 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2021-437 / GHSA-5xp3-jfq3-5q8x","Warn: Project is vulnerable to: PYSEC-2023-228 / GHSA-mq26-g339-26xf","Warn: Project is vulnerable to: PYSEC-2022-43017 / GHSA-qwmp-2cf2-g9g6"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 3 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-19T10:51:48.569Z","repository_id":46082837,"created_at":"2025-08-19T10:51:48.569Z","updated_at":"2025-08-19T10:51:48.569Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28413827,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:16:59.381Z","status":"ssl_error","status_checked_at":"2026-01-14T08:13:45.490Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["etl","redshift","serverless"],"created_at":"2026-01-14T08:22:57.503Z","updated_at":"2026-01-14T08:22:58.302Z","avatar_url":"https://github.com/helecloud.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# redshift-query\n\n[![image](https://img.shields.io/pypi/v/redshift-query.svg)](https://pypi.python.org/pypi/redshift-query)\n[![image](https://img.shields.io/travis/helecloud/redshift-query.svg)](https://travis-ci.com/helecloud/redshift-query)\n[![image](https://readthedocs.org/projects/redshift-query/badge/?version=latest)](https://redshift-query.readthedocs.io/en/latest/?badge=latest)\n\n\n## Introduction\n\nThis is a very simple library that gets credentials of a cluster via\n[redshift.GetClusterCredentials](https://docs.aws.amazon.com/redshift/latest/APIReference/API_GetClusterCredentials.html)\nAPI call and then makes a connection to the cluster and runs the provided SQL statements, once done it will close the connection and return the results.\n\nThis is useful for when you want to run queries in CLIs or based on events for example on AWS Lambdas, or on a regular\nbasis on AWS Glue Python Shell jobs.\n\n## Usage\n\nWhile redshift_query could be used as a library it or it's own standalone script.\n\nIt requires the following parameters supplied either as environmental variables or keys in the parameters dict passed to the function:\n\n- DB_NAME: The database name. You can also provide this via db_name parameter to the function.\n- DB_USER: The user to request credentials from. You can optionally provide this via db_user parameter to the function.\n- CLUSTER_HOST: The host name of the redshift cluster. You can optionally provide this via cluster_host parameter to the function.\n- CLUSTER_ID: The id of the cluster. You can optionally provide this via cluster_id parameter to the function.\n- SQL_STATEMENTS: The SQL statements to run. You can optionally provide this via sql_statements parameter to the function.\nThis parameter is going to be formatted(via [string.format](https://docs.python.org/3/library/string.html)) with the event object that is passed to it.\nThis way you can have the SQL statement to be based of the event that's passed to your function, which is useful\nfor when you have a Lambda and it is called by an event source(S3 for example).\n- REDSHIFT_QUERY_LOG_LEVEL: By default set to ERROR, which logs nothing. Normally errors are not logged and bubbled up instead so they crash the script.\nIf set to INFO, it will log the result of queries and if set to DEBUG it will log every thing that happens which is good for debugging why it is stuck.\n\n\n### Deploying it via a glue job\n\nYou can use the [redshift-query-glue-job](https://serverlessrepo.aws.amazon.com/applications/eu-west-1/708984232979/redshift-query-glue-job) application from AWS Serverless Application Repository.\n\n### Deploying via AWS SAM \u0026 Lambda\n\nHere's an example that copies to a table when new manifests in S3 are written:\n\n```yaml\nAWSTemplateFormatVersion: '2010-09-09'\nTransform: AWS::Serverless-2016-10-31\nResources:\n  Function:\n    Type: AWS::Serverless::Function\n    Properties:\n      CodeUri: ./redshift_query/redshift_query.py\n      Handler: redshift_query.query\n      Runtime: python3.8\n      VpcConfig:\n        SecurityGroupIds: 'sg-12312312'\n        SubnetIds: 'sub-123123'\n      Policies:\n        - Version: '2012-10-17'\n          Statement:\n            - Effect: \"Allow\"\n              Action: \"redshift:GetClusterCredentials\"\n              Resource: \"arn:aws:redshift:eu-west-1:123123123:dbuser:test/master\" # https://docs.aws.amazon.com/redshift/latest/mgmt/generating-iam-credentials-role-permissions.html\n      Events:\n        S3:\n          Type: Schedule\n          Properties:\n            Bucket: mybucket\n            Events: s3:ObjectCreated:*\n            Filter:\n              S3Key:\n                Rules:\n                  - Name: suffix\n                    Value: .manifest\n\n      Environment:\n        Variables:\n          CLUSTER_ID: 'test'\n          CLUSTER_HOST: 'test.j242tj1qkjuz.eu-west-1.redshift.amazonaws.com'\n          DB_NAME: 'test'\n          DB_USER: 'master'\n          SQL_STATEMENTS: |-\n            copy customer\n              from {Records[0]['s3']['object']['key']}\n            iam_role 'arn:aws:iam::0123456789012:role/MyRedshiftRole'\n            manifest\n          LOG_LEVEL: DEBUG\n```\n\n### Usage as a Library\n\nTo use the library directly you can simply provide the parameters in snake_case format:\n```python\nimport redshift_query\n\nresults = redshift_query.query({\n    'db_name': 'test',\n    'db_user': 'master',\n    'cluster_host': 'test.j242tj1qkjuz.eu-west-1.redshift.amazonaws.com',\n    'cluster_id': 'test',\n    'sql_statements': '''\n        copy customer\n            from 's3://mybucket/cust.manifest'\n            iam_role 'arn:aws:iam::0123456789012:role/MyRedshiftRole'\n            manifest;\n        select * from customer;\n     '''\n})\n\n[copy_results, select_results] = results\n\nprint('select results', select_results)\n```\n\nIf you use redshift_query.query multiple times in your code you can use redshift_query.set_config to set the static configuration once:\n\n```python\nimport redshift_query\n\nredshift_query.set_config({\n    'db_name': 'test',\n    'db_user': 'master',\n    'cluster_host': 'test.j242tj1qkjuz.eu-west-1.redshift.amazonaws.com',\n    'cluster_id': 'test'\n})\n\nredshift_query.query({\n    'sql_statements': '''\n        copy customer\n            from 's3://mybucket/cust.manifest'\n            iam_role 'arn:aws:iam::0123456789012:role/MyRedshiftRole'\n            manifest;\n     '''\n})\n```\n\nOptionally you can also provide boto_session, if you want to assume role or provide your own credentials:\n\n```python\nimport redshift_query\nimport boto3\n\nredshift_query.query({\n    'boto_session': boto3.session.Session(profile_name='production'),\n    'sql_statements': 'select 1;'\n})\n```\n\n### As a CLI\n\n```\n$ DB_USER=neshat DB_NAME=test CLUSTER_ID=test CLUSTER_HOST=test.3j232jjqji21.eu-west-1.redshift.amazonaws.com SQL_STATEMENTS='select 1;' REDSHIFT_QUERY_LOG_LEVEL=DEBUG redshift_query\nDEBUG:redshift_query:Passed Event: {}\nDEBUG:redshift_query:Passed this_config with added defaults: {'db_name': 'test', 'db_user': 'neshat', 'cluster_host': 'test.3j232jjqji21.eu-west-1.redshift.amazonaws.com', 'cluster_id': 'test', 'sql_statements': 'select 1;', 'boto_session': None}\nDEBUG:redshift_query:Received Credentials\nDEBUG:redshift_query:Connected\nDEBUG:redshift_query:Running select 1\nINFO:redshift_query:Statement \"?column?\n--------\n       1\n(1 row)\" result:\n```\n\nCredits\n-------\n\nThis package was created with\n[Cookiecutter](https://github.com/audreyr/cookiecutter) and the\n[audreyr/cookiecutter-pypackage](https://github.com/audreyr/cookiecutter-pypackage)\nproject template.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhelecloud%2Fredshift-query","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhelecloud%2Fredshift-query","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhelecloud%2Fredshift-query/lists"}