{"id":14065364,"url":"https://github.com/mazen160/tfquery","last_synced_at":"2025-04-05T23:05:34.833Z","repository":{"id":43068271,"uuid":"362387791","full_name":"mazen160/tfquery","owner":"mazen160","description":"tfquery: Run SQL queries on your Terraform infrastructure.  Query resources and analyze its configuration using a SQL-powered framework.","archived":false,"fork":false,"pushed_at":"2022-10-27T11:10:22.000Z","size":96,"stargazers_count":322,"open_issues_count":0,"forks_count":18,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-29T22:04:40.685Z","etag":null,"topics":["aws","azure","cloud-security","devsecops","gcp","terraform"],"latest_commit_sha":null,"homepage":"https://mazinahmed.net","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mazen160.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-04-28T08:06:33.000Z","updated_at":"2024-11-28T16:34:59.000Z","dependencies_parsed_at":"2022-09-10T04:03:24.390Z","dependency_job_id":null,"html_url":"https://github.com/mazen160/tfquery","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazen160%2Ftfquery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazen160%2Ftfquery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazen160%2Ftfquery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazen160%2Ftfquery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mazen160","download_url":"https://codeload.github.com/mazen160/tfquery/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247411226,"owners_count":20934653,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","azure","cloud-security","devsecops","gcp","terraform"],"created_at":"2024-08-13T07:04:27.179Z","updated_at":"2025-04-05T23:05:34.815Z","avatar_url":"https://github.com/mazen160.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# 🌩️ tfquery 🌩️\n\n## Run SQL queries on your Terraform infrastructure. Ask questions that are hard to answer\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mazen160/public/master/static/images/tfquery-demo.png\" /\u003e\n\u003c/p\u003e\n\n---\n\n# 🚀 What is tfquery?\n\ntfquery is a framework that allows running SQL queries on Terraform code. It's made to analyze your Terraform infrastructure, locate resources, run security compliance checks, spot misconfigured resources, develop CI benchmarks, and much more.\n\ntfquery is made to help in answering questions that are hard to answer about your infrastructure-as-code. It allows querying resources and analyzing its configuration using a SQL-powered framework.\n\n# Why?\n\ninfrastructure-as-code is the de-facto today for documenting and deploying infrastructure on cloud providers. As the organization grows, it becomes really hard to understand and analyze the deployed infrastructure. Grepping and searching for resources in Terraform state files is not enough. Terraform Modules are automating processes dynamically for infrastructure deployment, so searching for static resources is also not feasible for good visibility.\n\nWith tfquery, you can run SQL queries on Terraform state files, and gain the best possible visibility.\n\n---\n\n# 💡 Use tfquery to\n\n- Have full coverage of your infrastructure, without being locked on a specific provider, including Amazon AWS, Microsoft Azure, Google Cloud Platform, Alibaba Cloud, IBM Cloud, Oracle Cloud, and many others.\n\n- Analyze deployed resources configuration.\n\n- Develop CI and monitoring checks for cloud infrastructure.\n\n- Write custom queries to scan Terraform resources.\n\n- Scan current Terraform plan for new risky changes.\n\n---\n\n# tfquery vs. Cloud-specific SQL engines?\n\nThere are cloud-specific SQL engines that allow you to run SQL queries to understand resources on their infrastructure, both are covered as provided service by the cloud provider, or given as an open-source tool or a product. The main difference tfquery brings:\n\n- **Maintainability**: Cloud-specific SQL engines require maintenance in case of new services or breaking changes to existing ones. tfquery make use of Terraform schemas as a standard. tfquery will work on all given services, without the need to continuously update it with new API specs.\n\n- **Coverage**: tfquery covers all the cloud providers that Terraform supports out of the box (thanks to Terraform Providers).\n\n---\n\n# 📖 Usage\n\n\n## tfstate\n\n#### Run SQL query on Terraform states\n\n```python\n\u003e\u003e\u003e import tfquery\n\u003e\u003e\u003e\n\u003e\u003e\u003e result = tfquery.tfstate.run_query(\"terraform.tfstate\", \"select count(*) from resources\")\n\u003e\u003e print(result)\n[{'count(*)': 86}]\n```\n\n#### Parse all resources from a Terraform state file\n\n```python\n\u003e\u003e\u003e import tfquery\n\u003e\u003e\u003e\n\u003e\u003e\u003e resources = tfquery.tfstate.parse_resources(\"terraform.tfstate\")\n\u003e\u003e\u003e print(f\"[i] Resources Count: {len(resources)}\")\n[i] Resources Count: 1475\n```\n\n\n## Advanced Usage\n\n### Migrate Version 3 to Version 4 Terraform states\n\nThis is a parsing library to migrate the older Version 3 Terraform states to a Version 4 state. This is made to add backward compatibility for Terraform states that is made for releases older than `Terraform v0.11`.\n\n```python\n\u003e\u003e\u003e import tfquery\n\u003e\u003e\u003e\n\u003e\u003e\u003e tfstate_v3 = tfquery.tfstate.load_file(\"terraform.tfstate\")\n\u003e\u003e\u003e tfstate_v4 = tfquery.tfstate_v3_migration.upgrade_v3_tfstate(tfstate)\n\n```\n\n## tfplan\n\n#### Run SQL queries on Terraform changes through Terraform Plan (tfplan)\n\n1) Save the Terraform plan\n\n```\n$ terraform plan -out saved.plan\n$ terraform show -json saved.json \u003e current.tfplan\n```\n\n2) Run tfquery on the Terraform Plan\n\n```python\n\u003e\u003e\u003e import tfquery\n\n\u003e\u003e\u003e result = tfquery.tfplan.run_query(\"current.tfplan\", \"select count(*) from changes\")\n\u003e\u003e print(result)\n[{'count(*)': 10}]\n```\n\n\n## 🖲️ Command-Line (`tfquery`)\n\nTFquery is also available as a CLI tool. It can be used to run SQL queries directly on Terraform states, and for importing resources into persistent storage.\n\n```shell\nmazin@hackbox$\u003e tfquery -h\nusage: tfquery [-h] [--tfstate TFSTATE] [--tfplan TFPLAN] [--tfstate-dir TFSTATE_DIR] [--query QUERY] [--db DB_PATH] [--interactive] [--import] [--include-tfplan-no-op]\n\ntfquery-cli: Run SQL queries on your Terraform infrastructure.\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --tfstate TFSTATE     Terraform .tfstate file.\n  --tfplan TFPLAN       Terraform tfplan JSON file.\n  --tfstate-dir TFSTATE_DIR\n                        Directory of Terraform .tfstate files, for running queries on environments.\n  --query QUERY, -q QUERY\n                        SQL query to execute.\n  --db DB_PATH          DB path (optional. default: temporarily-generated database).\n  --interactive, -i     Interactive mode.\n  --import              Import tfstate and tfplan into database.\n  --include-tfplan-no-op\n                        Include tfplan no-op actions.\n\n```\n\n### Examples\n\n- **Run SQL query for a directory of multiple Terraform states (for multiple workspaces).**\n\n```python\n$ tfquery -q 'select count(*) as count from resources;'  --tfstate-dir /path/to/terraform-states\n[i] DB Path: tfstate.db\n[+] Imported 4203 resources from ./prod.tfstate.\n[i] DB Path: tfstate.db\n[+] Imported 3675 resources from ./nonprod.tfstate.\n[i] DB Path: tfstate.db\n[+] Imported 463 resources from ./qa.tfstate.\n```\n\n- **Import Terraform states into Database.**:\n\n```python\n$ python3 tfquery --tfstate /path/to/terraform.state --db tfstate.db --import\n[i] DB Path: tfstate.db\n[+] Imported 386 resources from terraform.tfstate.\n```\n\n- **Run queries on imported resources in a database**\n\n```python\n$ tfquery --db tfstate.db -q 'select count(*) as count from resources;'\n[\n    {\n        \"count\": 386\n    }\n]\n```\n- **Run Tfplan queries on current changes**\n\n```python\n$ terraform plan -out saved.plan\n$ terraform show -json saved.json \u003e current.tfplan\n\n$ tfquery --tfplan current.tfplan -q \"select count(*) as count from changes\"\n\n[\n    {\n        \"count\": 10\n    }\n]\n```\n\n---\n\n# 💭 Awesome Queries \u0026 Scripts\n\n**Find all AWS S3 buckets without versioning being enabled**\n\n```python\nimport tfquery, sys\nresults = tfquery.tfstate.run_query(sys.argv[1], \"select * from resources where type = 'aws_s3_bucket'\")\nfor result in results:\n    attributes = result[\"attributes\"]\n    if 'versioning' not in attributes or len(attributes[\"versioning\"]) == 0:\n        # print(result)\n        continue\n    for versioning in attributes[\"versioning\"]:\n        if versioning[\"enabled\"] is False:\n            # print(result)\n            pass\n```\n\n**Find all AWS IAM users, and print their ARNs**\n\n```python\nimport tfquery, sys\nresults = tfquery.tfstate.run_query(sys.argv[1], \"select json_extract(attributes, '$.arn') as arn from resources where type = 'aws_iam_user';\")\nfor result in results:\n    print(result[\"arn\"])\n```\n\nor\n\n```python\nimport tfquery, sys\nresults = tfquery.tfstate.run_query(sys.argv[1], \"select attributes from resources where type = 'aws_iam_user';\")\nfor result in results:\n    print(result[\"attributes\"][\"arn\"])\n```\n\n**Find all resources in the environment, and show how many instances were deployed**\n\n```python\nimport tfquery\nresults = tfquery.tfstate.run_query(\"terraform.tfstate\", \"select type, count(*) as count from resources group by type order BY count desc;\")\nprint(results)\n```\n\n---\n\n# ✨ Interested in tfquery?\n\n1. **Post a Tweet about the project and tag [`@mazen160`](https://twitter.com/mazen160) 🙏**\n\n2. **🌟 Star it on Github 🌟**\n\n3. **Create a PR for a new awesome feature 💛**\n\n4. **Would like to sponsor the project? Contact me on email!**\n\n---\n\n# 💻 Contribution\n\nContribution is always welcome! Please feel free to report issues on Github and create PRs for new features.\n\n## 📌 Ideas to Start on\n\nWould like to contribute to tfquery? Here are some ideas that you may start with:\n\n- Better documentation: would be great to enhance the documentation with additional examples and queries.\n\n- CI: Implement CI along with test terraform states for Terragoat.\n\n- Support dependencies for resources lookup: Create a new table called \"dependencies\", parse V4 Terraform states, and implement a many-to-one relation for dependencies of resources.\n\n- More V3 --\u003e V4 migration support: currently V3 resources migrations are supported. Dependencies are not migrated to the new V4 state. It will be great to continue on V3--\u003e V4 support for Terraform states.\n\n- General validation of Terraform states parser implementation: Validate current implementation of the parser, and enhance it where possible.\n\n- [x] ~~Connect resources with terraform state base name: For environments with many workspaces, each workspace can have a different name, it would be nice to add a column for terraform state file base name, to help in querying across different workspaces.~~\n\n- [x] ~~tfplan parsing: Allow parsing of tfplan files. This can be an opening addition for implementing a new CI security scanner for Terraform deployments.~~\n\n- Logo design: a logo design would be great.\n\n- Web interface representation with [coleifer/sqlite-web](https://github.com/coleifer/sqlite-web) - Thanks [@securityfu](https://twitter.com/securityfu/) for the idea!\n\n- Add a managed repository of pre-defined queries to enable teams to be able to query Terraform for different compliance and security controls.\n\n\n### As you can see, there are many ways to support. Please help us make the project bigger for everyone!\n\n---\n\n# Installation\n\n```shell\n\nmazin@hackbox$\u003e git clone https://github.com/mazen160/tfquery.git\nmazin@hackbox$\u003e cd tfquery\nmazin@hackbox$\u003e python3 setup.py install\n\n```\n\nor\n\n```shell\nmazin@hackbox$\u003e pip install git+https://github.com/mazen160/tfquery\n```\n\n---\n\n# 📄 License\n\nThe project is licensed under MIT License.\n\n# 💚 Author\n\n**Mazin Ahmed**\n\n- **Website**: [https://mazinahmed.net](https://mazinahmed.net)\n- **Email**: `mazin [at] mazinahmed [dot] net`\n- **Twitter**: [https://twitter.com/mazen160](https://twitter.com/mazen160)\n- **Linkedin**: [http://linkedin.com/in/infosecmazinahmed](http://linkedin.com/in/infosecmazinahmed)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazen160%2Ftfquery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmazen160%2Ftfquery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazen160%2Ftfquery/lists"}