{"id":19248962,"url":"https://github.com/skar-software/duckdb-py-lambda","last_synced_at":"2026-03-16T22:01:40.286Z","repository":{"id":240854372,"uuid":"802515213","full_name":"skar-software/DuckDB-Py-Lambda","owner":"skar-software","description":"DuckDB Python runtime working in AWS Lambda","archived":false,"fork":false,"pushed_at":"2024-05-21T04:25:27.000Z","size":6,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-08T14:41:34.235Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/skar-software.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-18T14:07:20.000Z","updated_at":"2024-06-01T14:06:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"0f91ea1e-907e-4e89-95f5-b4b0fcaac3c7","html_url":"https://github.com/skar-software/DuckDB-Py-Lambda","commit_stats":null,"previous_names":["skarcapital/duckdb-py-lambda","skar-software/duckdb-py-lambda"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skar-software%2FDuckDB-Py-Lambda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skar-software%2FDuckDB-Py-Lambda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skar-software%2FDuckDB-Py-Lambda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skar-software%2FDuckDB-Py-Lambda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/skar-software","download_url":"https://codeload.github.com/skar-software/DuckDB-Py-Lambda/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243278332,"owners_count":20265529,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T18:11:41.703Z","updated_at":"2025-12-28T02:23:44.040Z","avatar_url":"https://github.com/skar-software.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Usage\njust pass your python code into the \"pandas_code\" key in the input event json of the lambda like below\n```json\n{\n  \"pandas_code\": \"import duckdb\\nimport pandas as pd\\nimport numpy as np\\ndf = pd.DataFrame({\\n'A': ['foo', 'bar', 'foo', 'bar', 'foo', 'bar', 'foo', 'foo'],\\n'B': ['one', 'one', 'two', 'three', 'two', 'two', 'one', 'three'],\\n'C': np.random.randn(8),\\n'D': np.random.randn(8)\\n})\\nresult = duckdb.query('SELECT A, AVG(D) FROM df GROUP BY A').to_df()\\nprint(result)\"\n}\n```\nif you require to use double quotes in the query then it must be escaped while its being passed into the event json\n\n# Example queries\nfetch count of records present in example csv\n\n**Input Code**\n```python\nimport duckdb\nimport pandas as pd\nimport numpy as np\n\n# Create a Pandas DataFrame\ndf = pd.DataFrame({\n   'A': ['foo', 'bar', 'foo', 'bar', 'foo', 'bar', 'foo', 'foo'],\n   'B': ['one', 'one', 'two', 'three', 'two', 'two', 'one', 'three'],\n   'C': np.random.randn(8),\n   'D': np.random.randn(8)\n})\n \n# Use DuckDB to run a SQL query on the DataFrame\nresult = duckdb.query(\"SELECT A, AVG(D) FROM df GROUP BY A\").to_df()\n```\n**Output**\n```\nA     avg(D)\n0 foo 0.468670\n1 bar 0.399205\n```\n\n\n# Option 1 : Run EXE file (pre-compiled)\n- Go DuckDB in Lambda EXE file is avalaible under the releases tab. You can download the zip and directly upload it to a AWS lambda and test it out your self.\n  https://github.com/skarcapital/DuckDB-Py-Lambda/releases/tag/v1\n- since the zip which contains the program and requries dependencies is \u003e 50MB we have to upload it to AWS S3 and configure our lambda to utilize that\n- If you have questions, post it in the issues.\n\n# Option 2: Compile steps\n- the lambda/lambda_function.py file is the main program\n- inside this folder install all required dependencies locally\n`pip install duckdb pandas -t .`\n- zip the folder that has the lambda_function.py file and the dependencies\n- since the zip which contains the program and requries dependencies is \u003e 50MB we have to upload it to AWS S3 and configure our lambda to utilize that\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskar-software%2Fduckdb-py-lambda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskar-software%2Fduckdb-py-lambda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskar-software%2Fduckdb-py-lambda/lists"}