{"id":21510556,"url":"https://github.com/elkronos/sql_portfolio","last_synced_at":"2025-03-17T15:20:36.916Z","repository":{"id":209252352,"uuid":"723583552","full_name":"elkronos/SQL_Portfolio","owner":"elkronos","description":"Examples of SQL scripts. Includes templates for admin (SQL server) and data cleaning, as well as scripts designed for creating a mock database, illustrating triggers, common table expressions, analysis, and ETL.","archived":false,"fork":false,"pushed_at":"2023-12-10T23:21:51.000Z","size":50,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-24T01:36:49.581Z","etag":null,"topics":["analysis","cte","database","database-management","etl","feature-engineering","sql","sql-server","trigger-events"],"latest_commit_sha":null,"homepage":"","language":"TSQL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elkronos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-11-26T05:49:50.000Z","updated_at":"2024-09-06T17:22:51.000Z","dependencies_parsed_at":"2023-12-10T07:20:30.269Z","dependency_job_id":null,"html_url":"https://github.com/elkronos/SQL_Portfolio","commit_stats":null,"previous_names":["elkronos/sql_portfolio"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkronos%2FSQL_Portfolio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkronos%2FSQL_Portfolio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkronos%2FSQL_Portfolio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkronos%2FSQL_Portfolio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elkronos","download_url":"https://codeload.github.com/elkronos/SQL_Portfolio/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244056428,"owners_count":20390720,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analysis","cte","database","database-management","etl","feature-engineering","sql","sql-server","trigger-events"],"created_at":"2024-11-23T21:49:22.266Z","updated_at":"2025-03-17T15:20:36.894Z","avatar_url":"https://github.com/elkronos.png","language":"TSQL","readme":"# SQL_Portfolio\nExamples of SQL scripts. Includes scripts designed for creating a data base, common table expressions, analyzing data, ETL, and admin.\n\n## Admin\nScripts which illustrate basic admin commands using SQL Server.\n- `Backup` -  Perform a full backup of the database.\n- `Deadlock` - Retrieves all the current lock information for database objects from the system dynamic management view.\n- `MonitorPerformance` - Lists SQL Server queries that have been running for longer than 5 minutes, showing details like the query text, session ID, status, command type, CPU time, and total elapsed time, ordered by the longest running queries first.\n- `Restore` - Restores database from a backup file.\n\n## Data Cleaning\nScripts which illustrate how to clean data using SQL.\n- `ConvertCase` - Format each entry so that the first letter is uppercase and all subsequent letters are lowercase.\n- `CorrectingNulls` - Replaces any NULL values in that column with a 'Default Value'.\n- `DeleteOprhans` - Deletes rows from a child table (ChildTable) where values in a specified column (ChildColumn) do not have corresponding entries in a column (ParentColumn) of a parent table (ParentTable).\n- `RemoveDuplicates` - Removes duplicate rows from a specified table based on a specific column, keeping the row with the higher ID value and deleting the others.\n- `StandardizingDates` - Updates a date column in a specified table by converting date strings from a DD/MM/YYYY format to a standard date format, applying this change only to those entries that match the DD/MM/YYYY pattern.\n- `StandardizingPhoneNumbers` - Updates a phone number column in a specified table by removing any non-numeric characters from the phone numbers, applying this change only to entries that contain non-numeric characters.\n- `TrimeWhiteSpace` - Removes leading and trailing white spaces from specified columns applying this operation only to entries in those columns where leading or trailing spaces are detected.\n- `UpdateValues` - Replaces a user specified value with a different user specified value.\n\n## Dynamic Templates\nScripts designed to work across different tables without hardcoding table or variable names.\n- `DescriptiveStatistics` - Generate descriptive statistics (frequency and percent of null, range, if numeric mean, standard deviation etc) for all columns in a table.\n- `FuzzyJoin` - Demonstration of `CHARINDEX` fuzzy join in SQL server. Illustrates using interim tables and saves to optimize performance.\n- `KeyFinder` - Look for possible keys based on column names across tables. If no exact matches are found, uses soundex to find approximate matches.\n- `TableProfiler` - Analyze structure of tables using metadata.\n\n## Mock Database\nScripts which illustrate the set-up, use, and maintence for a fake (mock) database. These scripts are organized into the following directories.\n\n- `Analyze Database` - Scripts that demonstrate how to analyze the mock database. Calculate outcomes such as sales trends over time, sales proportion by product category, average purchase per user, and more.\n- `CTEs` - Common table expresion scripts; Illustrating how to delete, join, query recent orders, recursive joining, updating, analyzing records, and more.\n- `Create Database` - Scripts used to construct a mock database (see schema for database below).\n- `Data Migrations` - Script used to show how to migrate old users to new users table with adjustments.\n- `Data Validation` - Scripts designed to show how data validation can be automated.\n- `ETL` - How to stage, create, ad insert a product sales table for reproducible reporting.\n- `Feature Engineering` - How to econde, normalize, and extract features from tables for establishing pipelines.\n- `Roles` - Create and manage role and user role information.\n- `Security` - Scripts designed with security in mind. Illustrates encryption, validation, authentication, hash passwords etc.\n- `Triggers` - Demonstrating how to automatically update tables based on event occurences. Includes scripts such as audit logging, email change notifications, stock management, etc.\n\n\nA schema of the mock database is listed below.\n\n### Users Table\n- `UserID` (Primary Key, Auto Increment)\n- `Username` (Varchar(50), Not Null)\n- `Email` (Varchar(100), Not Null, Unique)\n- `PasswordHash` (Varchar(255), Not Null)\n- `DateJoined` (Datetime, Not Null)\n- `LastLogin` (Datetime)\n\n**Relationships**:\n- Has multiple `Orders` via `UserID`.\n\n\n### OrderDetails Table\n- `OrderDetailID` (Primary Key, Auto Increment)\n- `OrderID` (Foreign Key referencing `Orders.OrderID`)\n- `ProductID` (Foreign Key referencing `Products.ProductID`)\n- `Quantity` (Integer, Not Null, Check Constraint: Quantity \u003e 0)\n- `PriceAtTimeOfOrder` (Decimal(10, 2), Not Null)\n\n**Relationships**:\n- Belongs to an `Order` via `OrderID`.\n- References a `Product` via `ProductID`.\n\n\n### Orders Table\n- `OrderID` (Primary Key, Auto Increment)\n- `UserID` (Foreign Key referencing `Users.UserID`)\n- `OrderDate` (Datetime, Not Null)\n- `TotalAmount` (Decimal(10, 2), Not Null, Check Constraint: TotalAmount \u003e= 0)\n- `Status` (Varchar(50), Not Null)\n\n**Relationships**:\n- Belongs to a `User` via `UserID`.\n- Contains multiple `OrderDetails` via `OrderID`.\n\n\n### Products Table\n- `ProductID` (Primary Key, Auto Increment)\n- `ProductName` (Varchar(100), Not Null)\n- `Description` (Text)\n- `Price` (Decimal(10, 2), Not Null, Check Constraint: Price \u003e= 0)\n- `Stock` (Integer, Not Null, Check Constraint: Stock \u003e= 0)\n- `DateAdded` (Datetime, Not Null)\n\n**Relationships**: Can be related to `OrderDetails` as needed.\n\n## Schemas\nScripts which illustrate setting up schemas for multiples users in AWS Redshift.\n- `00-master_schema` - Create master table with log.\n- `01-client_schema_procedure` - Set up a schema for each client.\n- `02-grant_permissions` - Grant access to users.\n- `03-apply_schema_change` - Apply a change to schemas.\n- `RedshiftClientListUpdater` - Python functions to load client IDs.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felkronos%2Fsql_portfolio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felkronos%2Fsql_portfolio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felkronos%2Fsql_portfolio/lists"}