{"id":14286486,"url":"https://github.com/dfinke/PSDuckDB","last_synced_at":"2025-08-15T07:31:21.292Z","repository":{"id":247128389,"uuid":"824945177","full_name":"dfinke/PSDuckDB","owner":"dfinke","description":"PSDuckDB is a PowerShell module that provides seamless integration with DuckDB, enabling efficient execution of analytical SQL queries directly from the PowerShell environment.","archived":false,"fork":false,"pushed_at":"2024-09-10T00:07:32.000Z","size":17573,"stargazers_count":44,"open_issues_count":3,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-14T23:22:13.767Z","etag":null,"topics":["data-analysis","data-science","duckdb","powershell","sql"],"latest_commit_sha":null,"homepage":"","language":"PowerShell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dfinke.png","metadata":{"files":{"readme":"README.md","changelog":"changelog.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-06T11:09:04.000Z","updated_at":"2024-12-14T08:08:41.000Z","dependencies_parsed_at":"2024-09-10T02:53:17.207Z","dependency_job_id":"272dc7a8-6000-45d4-a804-0ec9197ec7c7","html_url":"https://github.com/dfinke/PSDuckDB","commit_stats":null,"previous_names":["dfinke/psduckdb"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfinke%2FPSDuckDB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfinke%2FPSDuckDB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfinke%2FPSDuckDB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfinke%2FPSDuckDB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dfinke","download_url":"https://codeload.github.com/dfinke/PSDuckDB/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229899524,"owners_count":18141525,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","data-science","duckdb","powershell","sql"],"created_at":"2024-08-23T17:00:52.669Z","updated_at":"2024-12-16T02:30:20.908Z","avatar_url":"https://github.com/dfinke.png","language":"PowerShell","readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"PSDuckDB.png\" width=200/\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://x.com/dfinke\"\u003e\n    \u003cimg src=\"https://img.shields.io/twitter/follow/dfinke.svg?style=social\u0026label=Follow%20%40dfinke\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://youtube.com/@dougfinke\"\u003e\n    \u003cimg src=\"https://img.shields.io/youtube/channel/subscribers/UCP47ZkO5EDkoI2sr-3P4ShQ\"\u003e\u003c/a\u003e\n\u003c/p\u003e    \n\n\u003cp align=\"center\"\u003e\n\n  \u003ca href=\"https://www.powershellgallery.com/packages/PSDuckDB/\"\u003e\n      \u003cimg src=\"https://img.shields.io/powershellgallery/v/PSDuckDB.svg\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://www.powershellgallery.com/packages/PSDuckDB/\"\u003e\u003cimg src=\"https://img.shields.io/powershellgallery/dt/PSDuckDB.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n# PSDuckDB\nPSDuckDB is a PowerShell module that provides seamless integration with DuckDB, enabling efficient execution of analytical SQL queries directly from the PowerShell environment.\n\nhttps://duckdb.org/why_duckdb\n\n![alt text](DuckDBAtAGlance.png)\n\n## Installation\n\n```powershell\nInstall-Module PSDuckDB \n```\n\n## The Video\nPowerShell Meets DuckDB Integrate, Query, and Analyze Data Seamlessly\n\n\u003ca href=\"https://youtu.be/jCnRXp2z4Uc\"\u003e\u003cimg src=\"https://img.youtube.com/vi/jCnRXp2z4Uc/0.jpg\" width=\"475\"\u003e\n\n## Why DuckDB\nThere are many database management systems (DBMS) out there. But there is no one-size-fits all database system. All take different trade-offs to better adjust to specific use cases. DuckDB is no different.\n\nTo start with, DuckDB is a relational (table-oriented) DBMS that supports the Structured Query Language (SQL).\n\n## Simple\nSQLite is the world's most widely deployed DBMS. Simplicity in installation, and embedded in-process operation are central to its success. DuckDB adopts these ideas of simplicity and embedded operation.\n\n## Portable\nThanks to having no dependencies, DuckDB is extremely portable.\n\n## Feature-Rich\nDuckDB provides serious data management features.\n\n## Fast\nDuckDB is designed to support analytical query workloads, also known as [online analytical processing (OLAP)](https://en.wikipedia.org/wiki/Online_analytical_processing). \n\n## Free\nDuckDB's development started while the main developers were public servants in the Netherlands. \n\n## Use PSDuckDB - ADO.NET Style\n\nIn our ever-evolving world of data manipulation, the ability to efficiently pivot datasets can reveal new insights and trends. This example leverages PowerShell to dynamically transform city population data from a CSV file, enabling a year-over-year comparison. Let's dive into the process of creating a pivot table using PowerShell and DuckDB.\n\nFirst, we establish a connection to DuckDB and create a table from our CSV dataset. This forms the foundation for our pivot operation, setting the stage for extracting valuable insights from raw data.\n\n```powershell\n$dataset = \"cities.csv\"\n\n$db = New-DuckDBConnection\n\n$db.sql(\"CREATE TABLE Cities AS SELECT * FROM '$dataset';\")\n\n$db.sql(@\"\nPIVOT Cities\nON Year\nUSING sum(Population);\n\"@) | Format-Table\n\n$db.CloseDB()\n```\n\nWith the table in place, the next step is to execute a pivot operation. This script restructures the data, allowing for a concise year-over-year comparison of city populations, revealing trends that may not be apparent in the raw data.\n\n```\nCountry Name          2000 2010 2020\n------- ----          ---- ---- ----\nUS      New York City 8015 8175 8772\nUS      Seattle        564  608  738\nNL      Amsterdam     1005 1065 115\n```\n\nBy seamlessly integrating DuckDB with PowerShell, we unlock powerful data transformation capabilities. This example demonstrates how simple scripts can convert raw data into actionable insights, essential for informed decision-making. Explore these techniques further to harness the full potential of your datasets.\n\nSee [pivot.ps1](examples-sql-method/07-pivot.ps1)\n\n## Examples\n\nCheck out the `SQL Introduction` and details for `DuckDB` https://duckdb.org/docs/sql/introduction\n\nRefer to the [Data Import](https://duckdb.org/docs/data/overview) section for more information.\n\n### Basic \n[basic.ps1](examples/01-basic.ps1)\n\n```powershell\npsduckdb -command \"select 10+2\"\n```\n\n```ps\n(10 + 2)\n--------\n      12\n```\n\n### Read CSV\n[read-csv.p1](examples/02-read-csv.ps1)\n\n```powershell\npsduckdb -command \"SELECT * FROM '$PSScriptRoot\\..\\data\\csv\\sample\\sales1.csv';\" | Format-Table\n```\n\n```powershell\nRegion Tag    Units  Price State     Source\n------ ---    -----  ----- -----     ------\nWest   Sales1   927 923.71 Texas     csv\nNorth  Sales1   466 770.67 Tennessee csv\nEast   Sales1   520 458.68 Florida   csv\nEast   Sales1   828 661.24 Maine     csv\n```\n\n### Read from Repo\n[read-csv-from-repo](examples/06-read-csv-from-repo.ps1)\n\n```powershell\n$path = 'https://raw.githubusercontent.com/dfinke/ImportExcel/master/Examples/PassThru/sales.csv'\n\n$stmt = @\"\nSELECT Item, SUM(UnitSold) AS Total_Sold\nFROM '$path'\nGroup By Item\nOrder By Item;\n\"@\n\npsduckdb -command $stmt | Format-Table\n```\n\n```powershell\nTotal_Sold Item\n---------- ----\n       272 Apple\n       416 Banana\n       234 Kale\n       165 Pear\n       137 Potato\n```\n\n### Pivot the Data\n[pivot.ps1](examples/07-pivot.ps1)\n\n```powershell\n$dataset = \"$psscriptroot/../data/otherData/cities.csv\"\n\n$stmt = @\"\nCREATE TABLE Cities AS SELECT * FROM '$dataset';\n\nPIVOT Cities\nON Year\nUSING sum(Population);\n\"@\n\npsduckdb -command $stmt | Format-Table\n```\n\n```powershell\nName          Country 2000 2010 2020\n----          ------- ---- ---- ----\nAmsterdam     NL      1005 1065 1158\nNew York City US      8015 8175 8772\nSeattle       US       564  608  738\n```\n\n### Import Parquet Files\n[read-parquet](examples/08-read-parquet.ps1)\n\n```powershell\npsduckdb -command \"SELECT * FROM '$PSScriptRoot\\..\\data\\parquet\\sales.parquet';\" | Format-Table\n```\n\n```powershell\nPrice   Region State        Units\n-----   ------ -----        -----\n923.71  West   Texas          927\n770.67  North  Tennessee      466\n458.68w East   Florida        520\n661.24  East   Maine          828\n53.58   West   Virginia       465\n235.67  North  Missouri       436\n992.47  South  Kansas         214\n640.72  North  North Dakota   789\n508.55  South  Delaware       712\n```\n\n## CLI API\nThe PSDuckDB CLI (Command Line Interface) is simple cli interface. \n\n```powershell\nPS C:\\\u003e psduckdb\n```\n\n```ps\nWelcome to PSDuckDB! 07/06/2024 13:03:42\nConnected to a transient in-memory database\n\nPSDuckDB:\n```\n\n### Examples\n\nSelect a string.\n```powershell\nPSDuckDB: select 'quack' as my_col\n\nmy_col\n------\nquack\n```\n\nGenerate a series of numbers from 0 to 5.\n```powershell\nPSDuckDB: SELECT * FROM generate_series(5);\n\ngenerate_series\n---------------\n              0\n              1\n              2\n              3\n              4\n              5\n```\n\nRead a parquet file from a local path.\n\n```powershell\nPSDuckDB: select * from 'D:\\yourpath\\sales.parquet'\n\nUnits Region Price   State\n----- ------ -----   -----\n  927 West   923.71  Texas\n  466 North  770.67  Tennessee\n  520 East   458.68w Florida\n  828 East   661.24  Maine\n  465 West   53.58   Virginia\n  436 North  235.67  Missouri\n  214 South  992.47  Kansas\n  789 North  640.72  North Dakota\n  712 South  508.55  Delaware\n```\n\n## DuckDB.NET\n\nPSDuckDB leverages the DuckDB.NET bindings, developed by Giorgi. You can explore this work on GitHub at https://github.com/Giorgi/DuckDB.NET.\n\n## Development\n\nPlease also refer the the [Contribution Guide](CONTRIBUTING.md).\n\n\n","funding_links":[],"categories":["Client APIs"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfinke%2FPSDuckDB","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdfinke%2FPSDuckDB","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfinke%2FPSDuckDB/lists"}