{"id":26316846,"url":"https://github.com/nathanthorell/sql-data-compare","last_synced_at":"2025-06-27T18:08:55.514Z","repository":{"id":276056969,"uuid":"800096548","full_name":"nathanthorell/sql-data-compare","owner":"nathanthorell","description":"A Python tool for comparing SQL query results across different data sources. It executes queries against two databases and compares their result sets, making it useful for data validation and migration verification.","archived":false,"fork":false,"pushed_at":"2025-02-06T02:35:22.000Z","size":19,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-06T03:29:33.230Z","etag":null,"topics":["mssql","sql"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nathanthorell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-13T17:37:03.000Z","updated_at":"2025-02-06T02:42:06.000Z","dependencies_parsed_at":"2025-02-06T03:39:36.845Z","dependency_job_id":null,"html_url":"https://github.com/nathanthorell/sql-data-compare","commit_stats":null,"previous_names":["nathanthorell/sql-data-compare"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nathanthorell%2Fsql-data-compare","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nathanthorell%2Fsql-data-compare/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nathanthorell%2Fsql-data-compare/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nathanthorell%2Fsql-data-compare/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nathanthorell","download_url":"https://codeload.github.com/nathanthorell/sql-data-compare/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243732223,"owners_count":20338839,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mssql","sql"],"created_at":"2025-03-15T13:16:59.047Z","updated_at":"2025-06-27T18:08:55.481Z","avatar_url":"https://github.com/nathanthorell.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sql-data-compare\n\nA Python tool for comparing SQL query results across different data sources. It executes queries against two databases and compares their result sets, making it useful for data validation and migration verification.\n\nSupports both Microsoft SQL Server and PostgreSQL databases, with a flexible configuration system that allows comparing data across different database types.\n\n## Local Installation\n\n```bash\n# Create and activate virtual environment\npython -m venv .venv\nsource .venv/bin/activate\n\n# Install package with development dependencies\nmake install\n```\n\n## Configuration\n\n### Database Support\n\nThis tool supports both Microsoft SQL Server and PostgreSQL databases:\n\nMicrosoft SQL Server: Tested with ODBC Driver 18. Connection strings are automatically formatted for SQL Server.\nPostgreSQL: Implemented using the psycopg2 driver.\n\nTo specify which database type to use:\n\nIn your comparison configuration, set left_db_type and right_db_type to either \"mssql\" or \"pg\"\nEnsure you have the appropriate environment variables configured for each database type\n\n### SQL Queries\n\nStore your SQL queries in the `sql/` directory at the project root. Each comparison can reference two different query files.\n\n### Config File\n\nCreate a config.json file at the project root:\n\n```json\n{\n  \"compare_list\": [\n    {\n      \"name\": \"example compare\",\n      \"left_db_type\": \"mssql\",\n      \"left_query_file\": \"old_query.sql\",\n      \"right_db_type\": \"pg\",\n      \"right_query_file\": \"new_query.sql\"\n    }\n  ]\n}\n```\n\nYou can mix and match database types for your comparisons as needed. SQL syntax differences between database engines should be handled in your query files.\n\n## Environment Variables\n\nRequired environment variables can be set directly or via a .env file in your working directory:\n\n```text\nDB_DRIVER=\"ODBC Driver 18 for SQL Server\"\nDB_ENCRYPT=no\n\n# Left database connection\nLEFT_DB_HOST=\nLEFT_DB_NAME=\nLEFT_DB_PORT=\nLEFT_DB_USER=\nLEFT_DB_PASS=\n\n# Right database connection\nRIGHT_DB_HOST=\nRIGHT_DB_NAME=\nRIGHT_DB_PORT=\nRIGHT_DB_USER=\nRIGHT_DB_PASS=\n```\n\n## Usage\n\n```bash\n# Run comparisons\nmake run\n\n# Development commands\nmake lint        # Run ruff and mypy\nmake test        # Run pytest\nmake coverage    # Run tests with coverage\nmake format      # Format code with ruff\nmake clean       # Clean build artifacts\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnathanthorell%2Fsql-data-compare","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnathanthorell%2Fsql-data-compare","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnathanthorell%2Fsql-data-compare/lists"}