Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/sodadata/soda-sql

Soda SQL and Soda Spark have been deprecated and replaced by Soda Core. docs.soda.io/soda-core/overview.html
https://github.com/sodadata/soda-sql

Last synced: 3 months ago
JSON representation

Soda SQL and Soda Spark have been deprecated and replaced by Soda Core. docs.soda.io/soda-core/overview.html

Host: GitHub
URL: https://github.com/sodadata/soda-sql
Owner: sodadata
License: apache-2.0
Archived: true
Created: 2022-03-22T08:33:35.000Z (over 2 years ago)
Default Branch: main
Last Pushed: 2022-11-30T20:20:48.000Z (over 1 year ago)
Last Synced: 2024-03-19T08:22:39.467Z (3 months ago)
Language: Python
Homepage: https://docs.soda.io/
Size: 27 MB
Stars: 60
Watchers: 8
Forks: 15
Open Issues: 126
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE

Lists

data-tools - Soda SQL - accessible data (Not exhaustive and personnal list of "modern" Data Tools and Projects)
awesome-dataops - SodaSQL - Data profiling, testing, and monitoring for SQL accessible data. (Data Quality)
awesome-observability - Soda Sql - Data profiling, testing, and monitoring for SQL accessible data. (10. Application Performance Monitoring Solutions (APM) / Anomalies Detection)
awesome - soda-sql - Data profiling, testing, and monitoring for SQL accessible data. (Python)
awesome-observability - Soda Sql - Data profiling, testing, and monitoring for SQL accessible data. (10. Application Performance Monitoring Solutions (APM) / Anomalies Detection)
awesome-profiling - soda-sql - Data testing, monitoring, and profiling for SQL-accessible data. (4. Database / Node.js)

README

        
# Soda SQL and Soda Spark have become Soda Core

Soda SQL and Soda Spark are deprecated in favor of Soda Core and the Soda Checks Language. If you are new to Soda, start with Soda Core!



Data testing, monitoring, and profiling for SQL-accessible data.




  

  

  

  

  



 


 


✔  [Install Soda SQL](/docs/soda-sql/installation.md) from the command-line


✔  Access comprehensive [Soda SQL documentation](/docs/soda-sql/overview.md)


✔  Compatible with [Snowflake, Amazon Redshift, BigQuery](/docs/soda-sql/installation.md#compatibility), and more


✔  [Write tests](/docs/soda-sql/tests.md) in a YAML file


✔  [Run programmatic scans](/docs/soda-sql/programmatic_scan.md) to test data quality





#### Example scan YAML file (Deprecated)

```yaml

table_name: breakdowns

metrics:

  - row_count

  - missing_count

  - missing_percentage

...

# Validates that a table has rows

tests:

  - row_count > 0

# Tests that numbers in the column are entered in a valid format as whole numbers

columns:

  incident_number:

    valid_format: number_whole

    tests:

      - invalid_percentage == 0

# Tests that no values in the column are missing

  school_year:

    tests:

      - missing_count == 0

# Tests for duplicates in a column

  bus_no:

    tests:

      - duplicate_count == 0

# Compares row count between datasets

sql_metric: 

  sql: |

    SELECT COUNT(*) as other_row_count

    FROM other_table

  tests:

    - row_count == other_row_count

```

## Install (Deprecated)

* [Install Soda SQL](/docs/soda-sql/installation.md)

* [Quick start for Soda SQL](/docs/soda/quick-start-soda-sql.md)

## Contributors ✨

Thanks goes to these wonderful people! ([emoji key](https://allcontributors.org/docs/en/emoji-key))

  

    
_{Vijay Kiran}
💻

    
_{abhishek khare}
💻

    
_{Jelte Hoekstra}
💻 📖

    
_Cor
💻 📖

    
_{Milan Aleksić}
🚇

    
_{Ayoub Fakir}
💻

    
_{Alex Tonkonozhenko}
💻

  

  

    
_{Todd de Quincey}
💻

    
_{Antonin Jousson}
💻

    
_Jonas
🚇

    
_cwouter
💻

    
_{Janet R}
📖

    
_{Bastien Boutonnet}
💻

    
_{Tom Baeyens}
💻

  

  

    
_{AlessandroLollo}
💻

    
_mmigdiso
💻

    
_ericmuijs
💻

    
_{Lieven Govaerts}
💻

    
_{Milan Lukac}
💻

    
_{Sebastián Villarroel}
💻

    
_{Benjamin Berriot}
💻

  

  

    
_{Alexey Minakov}
💻

  

This project followed the [all-contributors](https://github.com/all-contributors/all-contributors) specification. 

### Open Telemetry Tracking (Deprecated)

Soda SQL collects statistical usage and performance information via the [Open Telemetry framework](https://opentelemetry.io). This information helps the Soda Core developer team proactively track performance issues and understand how users interact with the tool.

The collected information is strictly limited to usage and performance and does not contain Personal Identifying Information (PII). It is used for internal purposes only. Soda keeps the data in its raw form for a maximum of five years. If some information needs to be kept for longer, it will only be kept in aggregated form.

Read more about the information Soda tracks, and learn how to opt-out of tracking by consulting the [Anonymous usage statistics](/docs/soda-sql/global-configuration.md) documentation.