{"id":16851613,"url":"https://github.com/machow/dbcooper-py","last_synced_at":"2025-04-11T06:55:34.465Z","repository":{"id":37893340,"uuid":"466578478","full_name":"machow/dbcooper-py","owner":"machow","description":"Quickly access and tab-complete the tables in your database.","archived":false,"fork":false,"pushed_at":"2023-03-27T19:53:08.000Z","size":63,"stargazers_count":11,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-10T14:35:08.778Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/machow.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-05T22:08:35.000Z","updated_at":"2023-04-24T17:05:03.000Z","dependencies_parsed_at":"2023-02-15T12:05:38.317Z","dependency_job_id":null,"html_url":"https://github.com/machow/dbcooper-py","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":"machow/template-python-pkg","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/machow%2Fdbcooper-py","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/machow%2Fdbcooper-py/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/machow%2Fdbcooper-py/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/machow%2Fdbcooper-py/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/machow","download_url":"https://codeload.github.com/machow/dbcooper-py/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248358555,"owners_count":21090402,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T13:29:11.927Z","updated_at":"2025-04-11T06:55:34.446Z","avatar_url":"https://github.com/machow.png","language":"Python","readme":"```{python tags=c(\"hide-cell\")}\n# TODO: is there a way to get it so dbc.list() does not show 1 item per line?\n\n# this keeps the pandas dataframe repr from spitting out scoped style tags\n# which don't render on github\nimport pandas as pd\npd.set_option(\"display.notebook_repr_html\", False)\n```\n\n# dbcooper-py\n\n[![CI](https://github.com/machow/dbcooper-py/actions/workflows/ci.yml/badge.svg)](https://github.com/machow/dbcooper-py/actions/workflows/ci.yml)\n[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/machow/dbcooper-py/HEAD)\n\nThe dbcooper package turns a database connection into a collection of functions,\nhandling logic for keeping track of connections and letting you take advantage of\nautocompletion when exploring a database.\n\nIt's especially helpful to use when authoring database-specific Python packages,\nfor instance in an internal company package or one wrapping a public data source.\n\nFor the R version see [dgrtwo/dbcooper](https://github.com/dgrtwo/dbcooper).\n\n## Installation\n\n```\npip install dbcooper\n```\n\n## Example\n\n### Initializing the functions\n\nThe dbcooper package asks you to create the connection first.\nAs an example, we'll use the Lahman baseball database package (`lahman`).\n\n```{python}\nfrom sqlalchemy import create_engine\nfrom dbcooper.data import lahman_sqlite\n\n# connect to sqlite\nengine = create_engine(\"sqlite://\")\n\n# load the lahman data into the \"lahman\" schema\nlahman_sqlite(engine)\n```\n\nNext we'll set up dbcooper\n\n```{python}\nfrom dbcooper import DbCooper\n\ndbc = DbCooper(engine)\n```\n\nThe `DbCooper` object contains two important things:\n\n* Accessors to fetch specific tables.\n* Functions for interacting with the underlying database.\n\n### Using table accessors\n\nIn the example below, we'll use the `\"Lahman\".\"Salaries\"` table as an example.\nBy default, dbcooper makes this accessible as `.lahman_salaries`.\n\n**Plain** `.lahman_salaries` prints out table and column info, including types and descriptions.\n\n```{python}\n# show table and column descriptions\ndbc.lahman_salaries\n```\n\nNote that sqlite doesn't support table and columnn descriptions, so these sections\nare empty.\n\n**Calling** `.lahman_salaries()` fetches a lazy version of the data.\n\n\n```{python}\ndbc.lahman_salaries()\n```\n\nNote that this data is a siuba `LazyTbl` object, which you can use to analyze the data.\n\n```{python}\nfrom siuba import _, count\n\ndbc.lahman_salaries() \u003e\u003e count(over_100k = _.salary \u003e 100_000)\n```\n\n### Using database functions\n\n* `.list()`: Get a list of tables\n* `.tbl()`: Access a table that can be worked with using `siuba`.\n* `.query()`: Perform a SQL query and work with the result.\n* `._engine`: Get the underlying sqlalchemy engine.\n\nFor instance, we could start by finding the names of the tables in the Lahman database.\n\n```{python}\ndbc.list()\n```\n\nWe can access one of these tables with `dbc.tbl()`, then put it through any kind\nof siuba operation.\n\n```{python}\ndbc.tbl(\"Salaries\")\n```\n\n```{python}\nfrom siuba import _, count\ndbc.tbl(\"Salaries\") \u003e\u003e count(_.yearID, sort=True)\n```\n\nIf you'd rather start from a SQL query, use the `.query()` method.\n\n```{python}\ndbc.query(\"\"\"\n    SELECT\n        playerID,\n        sum(AB) as AB\n    FROM Batting\n    GROUP BY playerID\n\"\"\")\n```\n\nFor anything else you might want to do, the sqlalchemy Engine object is available.\nFor example, the code below shows how you can set its `.echo` attribute, which\ntells sqlalchemy to provide useful logs.\n\n```{python}\ndbc._engine.echo = True\ntable_names = dbc.list()\n```\n\nNote that the log messages above show that the `.list()` method executed two queries:\nOne to list tables in the \"main\" schema (which is empty), and one to list tables\nin the \"lahman\" schema.\n\n\n## Advanced Configuration\n\n\u003e ⚠️: These behaviors are well tested, but dbcooper's internals and API may change.\n\ndbcooper can be configured in three ways, each corresponding to a class interface:\n\n* **TableFinder**: Which tables will be used by `dbcooper`.\n* **AccessorBuilder**: How table names are turned into accessors.\n* **DbcDocumentedTable**: The class that defines what an accessor will return.\n\n```{python}\nfrom sqlalchemy import create_engine\nfrom dbcooper.data import lahman_sqlite\nfrom dbcooper import DbCooper, AccessorBuilder\n\nengine = create_engine(\"sqlite://\")\nlahman_sqlite(engine)\n```\n\n### Excluding a schema\n\n```{python}\nfrom dbcooper import TableFinder\n\nfinder = TableFinder(exclude_schemas=[\"lahman\"])\ndbc_no_lahman = DbCooper(engine, table_finder=finder)\ndbc_no_lahman.list()\n```\n\n\n### Formatting table names\n\n```{python}\nfrom dbcooper import AccessorBuilder\n\n# omits schema, and keeps only table name\n# e.g. `salaries`, rather than `lahman_salaries`\nbuilder = AccessorBuilder(format_from_part=\"table\")\n\ntbl_flat = DbCooper(engine, accessor_builder=builder)\ntbl_flat.salaries()\n```\n\n### Grouping tables by schema\n\n```{python}\nfrom dbcooper import AccessorHierarchyBuilder\n\ntbl_nested = DbCooper(engine, accessor_builder=AccessorHierarchyBuilder())\n\n# note the form: \u003cschema\u003e.\u003ctable\u003e\ntbl_nested.lahman.salaries()\n```\n\n### Don't show table documentation\n\n```{python}\nfrom dbcooper import DbcSimpleTable\n\ndbc_no_doc = DbCooper(engine, table_factory=DbcSimpleTable)\ndbc_no_doc.lahman_salaries\n```\n\nNote that sqlalchemy dialects like `snowflake-sqlalchemy` cannot look up things\nlike table and column descriptions as well as other dialects, so `DbcSimpleTable`\nmay be needed to connect to snowflake (see [this issue](https://github.com/snowflakedb/snowflake-sqlalchemy/issues/276)).\n\n\n## Developing\n\n```shell\n# install with development dependencies\npip install -e .[dev]\n\n# or install from requirements file\npip install -r requirements/dev.txt\n```\n\n### Test\n\n```shell\n# run all tests, see pytest section of pyproject.toml\npytest\n\n# run specific backends\npytest -m 'not snowflake and not bigquery'\n\n# stop on first failure, drop into debugger\npytest -x --pdb\n```\n\n\n### Release\n\n```shell\n# set version number\ngit tag v0.0.1\n\n# (optional) push to github\ngit push origin --tags\n\n# check version\npython -m setuptools_scm\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmachow%2Fdbcooper-py","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmachow%2Fdbcooper-py","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmachow%2Fdbcooper-py/lists"}