{"id":13468745,"url":"https://github.com/ptmcg/littletable","last_synced_at":"2025-05-16T07:05:13.195Z","repository":{"id":28844531,"uuid":"119497348","full_name":"ptmcg/littletable","owner":"ptmcg","description":"An in-memory database of Python objects, searchable using quasi-SQL API","archived":false,"fork":false,"pushed_at":"2025-04-18T05:59:30.000Z","size":9593,"stargazers_count":166,"open_issues_count":2,"forks_count":6,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-18T18:44:18.772Z","etag":null,"topics":["data-analysis-python","database","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ptmcg.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-30T07:14:31.000Z","updated_at":"2025-04-18T05:59:33.000Z","dependencies_parsed_at":"2024-01-16T09:00:55.260Z","dependency_job_id":"283f1252-de0c-425b-8cad-c677bca0cf3d","html_url":"https://github.com/ptmcg/littletable","commit_stats":{"total_commits":443,"total_committers":4,"mean_commits":110.75,"dds":"0.12189616252821667","last_synced_commit":"ff3d7c66924a416f1331cee25b630a6fcab0c106"},"previous_names":[],"tags_count":36,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ptmcg%2Flittletable","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ptmcg%2Flittletable/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ptmcg%2Flittletable/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ptmcg%2Flittletable/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ptmcg","download_url":"https://codeload.github.com/ptmcg/littletable/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254485057,"owners_count":22078767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis-python","database","python"],"created_at":"2024-07-31T15:01:18.102Z","updated_at":"2025-05-16T07:05:08.180Z","avatar_url":"https://github.com/ptmcg.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# littletable - a Python module to give ORM-like access to a collection of objects\n[![Build Status](https://travis-ci.org/ptmcg/littletable.svg?branch=master)](https://travis-ci.org/ptmcg/littletable) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/ptmcg/littletable/master)\n\n- [Introduction](#introduction)\n- [Optional dependencies](#optional-dependencies)\n- [Importing data from CSV files](#importing-data-from-csv-files)\n- [Tabular output](#tabular-output)\n- [For More Info](#for-more-info)\n- [Sample Demo](#sample-demo)\n\nIntroduction\n------------\nThe `littletable` module provides a low-overhead, schema-less, in-memory database access to a collection \nof user objects. `littletable` Tables will accept Python `dict`s or any user-defined object type, including:\n\n- `namedtuples` and `typing.NamedTuples`\n- `dataclasses`\n- `types.SimpleNamespaces`\n- `attrs` classes\n- `PyDantic` data models\n- `traitlets`\n\n`littletable` infers the Table's \"columns\" from those objects' `__dict__`, `__slots__`, or `_fields` mappings to access\nobject attributes. \n\nIf populated with Python `dict`s, they get stored as `SimpleNamespace`s.\n\nIn addition to basic ORM-style insert/remove/query/delete access to the contents of a `Table`, `littletable` offers:\n* simple indexing for improved retrieval performance, and optional enforcing key uniqueness \n* access to objects using indexed attributes\n* direct import/export to CSV, TSV, JSON, and Excel .xlsx files\n* clean tabular output for data presentation\n* simplified joins using `\"+\"` operator syntax between annotated `Table`s \n* the result of any query or join is a new first-class `littletable` `Table` \n* simple full-text search against multi-word text attributes\n* access like a standard Python list to the records in a `Table`, including indexing/slicing, `iter`, `zip`, `len`, `groupby`, etc.\n* access like a standard Python `dict` to attributes with a unique index, or like a standard Python `defaultdict(list)` to attributes with a non-unique index\n\n`littletable` `Table`s do not require an upfront schema definition, but simply work off of the attributes in \nthe stored values, and those referenced in any query parameters.\n\n\nOptional dependencies\n---------------------\nThe base `littletable` code has no dependencies outside of the Python stdlib. However, some operations\nrequire additional package installs:\n\n| operation                   | additional install required                                        |\n|-----------------------------|--------------------------------------------------------------------|\n| `Table.present`             | `rich`                                                             |\n| `Table.excel_import/export` | `openpyxl` (plus `defusedxml` or `lxml`, `defusedxml` recommended) |\n| `Table.as_dataframe`        | `pandas`                                                           |\n\n\nImporting data from CSV files\n-----------------------------\nYou can easily import a CSV file into a `Table` using `Table.csv_import()`:\n\n```python\nimport littletable as lt\nt = lt.Table().csv_import(\"my_data.csv\")\n# or\nt = lt.csv_import(\"my_data.csv\")\n```\n\nIn place of a local file name, you can also specify an HTTP url:\n\n```python\nurl = \"https://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv\"\nnames = [\"sepal-length\", \"sepal-width\", \"petal-length\", \"petal-width\", \"class\"]\niris_table = Table('iris').csv_import(url, fieldnames=names)\n```\n\nYou can also directly import CSV data as a string:\n\n```python\ncatalog = Table(\"catalog\")\n\ncatalog_data = \"\"\"\\\nsku,description,unitofmeas,unitprice\nBRDSD-001,Bird seed,LB,3\nBBS-001,Steel BB's,LB,5\nMGNT-001,Magnet,EA,8\"\"\"\n\ncatalog.csv_import(catalog_data, transforms={'unitprice': int})\n```\n\nData can also be directly imported from compressed .zip, .gz, and .xz files.\n\nFiles containing JSON-formatted records can be similarly imported using `json_import()`.\n\n\nTabular output\n--------------\nTo produce a nice tabular output for a table, you can use the embedded support for\nthe [rich](https://github.com/willmcgugan/rich) module, `as_html()` in [Jupyter Notebook](https://jupyter.org/),\nor the [tabulate](https://github.com/astanin/python-tabulate) module:\n\nUsing `table.present()` (implemented using `rich`; `present()` accepts `rich` `Table` keyword args):\n\n```python\ntable(title_str).present(fields=[\"col1\", \"col2\", \"col3\"])\n    or\ntable.select(\"col1 col2 col3\")(title_str).present(caption=\"caption text\", \n                                                  caption_justify=\"right\")\n```\n\nUsing `Jupyter Notebook`:\n\n```python\nfrom IPython.display import HTML, display\ndisplay(HTML(table.as_html()))\n```\n\nUsing `tabulate`:\n\n```python\nfrom tabulate import tabulate\nprint(tabulate((vars(rec) for rec in table), headers=\"keys\"))\n```\n\nFor More Info\n-------------\nExtended \"getting started\" notes at [how_to_use_littletable.md](https://github.com/ptmcg/littletable/blob/master/how_to_use_littletable.md).\n\nSample Demo\n-----------\nHere is a simple `littletable` data storage/retrieval example:\n\n```python\nfrom littletable import Table\n\ncustomers = Table('customers')\ncustomers.create_index(\"id\", unique=True)\ncustomers.csv_import(\"\"\"\\\nid,name\n0010,George Jetson\n0020,Wile E. Coyote\n0030,Jonny Quest\n\"\"\")\n\ncatalog = Table('catalog')\ncatalog.create_index(\"sku\", unique=True)\ncatalog.insert({\"sku\": \"ANVIL-001\", \"descr\": \"1000lb anvil\", \"unitofmeas\": \"EA\",\"unitprice\": 100})\ncatalog.insert({\"sku\": \"BRDSD-001\", \"descr\": \"Bird seed\", \"unitofmeas\": \"LB\",\"unitprice\": 3})\ncatalog.insert({\"sku\": \"MAGNT-001\", \"descr\": \"Magnet\", \"unitofmeas\": \"EA\",\"unitprice\": 8})\ncatalog.insert({\"sku\": \"MAGLS-001\", \"descr\": \"Magnifying glass\", \"unitofmeas\": \"EA\",\"unitprice\": 12})\n\nwishitems = Table('wishitems')\nwishitems.create_index(\"custid\")\nwishitems.create_index(\"sku\")\n\n# easy to import CSV data from a string or file\nwishitems.csv_import(\"\"\"\\\ncustid,sku\n0020,ANVIL-001\n0020,BRDSD-001\n0020,MAGNT-001\n0030,MAGNT-001\n0030,MAGLS-001\n\"\"\")\n\n# print a particular customer name\n# (unique indexes will return a single item; non-unique\n# indexes will return a new Table of all matching items)\nprint(customers.by.id[\"0030\"].name)\n\n# see all customer names\nfor name in customers.all.name:\n    print(name)\n\n# print all items sold by the pound\nfor item in catalog.where(unitofmeas=\"LB\"):\n    print(item.sku, item.descr)\n\n# print all items that cost more than 10\nfor item in catalog.where(lambda o: o.unitprice \u003e 10):\n    print(item.sku, item.descr, item.unitprice)\n\n# join tables to create queryable wishlists collection\nwishlists = customers.join_on(\"id\") + wishitems.join_on(\"custid\") + catalog.join_on(\"sku\")\n\n# print all wishlist items with price \u003e 10 (can use Table.gt comparator instead of lambda)\nbigticketitems = wishlists().where(unitprice=Table.gt(10))\nfor item in bigticketitems:\n    print(item)\n\n# list all wishlist items in descending order by price\nfor item in wishlists().sort(\"unitprice desc\"):\n    print(item)\n\n# print output as a nicely-formatted table\nwishlists().sort(\"unitprice desc\")(\"Wishlists\").present()\n\n# print output as an HTML table\nprint(wishlists().sort(\"unitprice desc\")(\"Wishlists\").as_html())\n\n# print output as a Markdown table\nprint(wishlists().sort(\"unitprice desc\")(\"Wishlists\").as_markdown())\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fptmcg%2Flittletable","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fptmcg%2Flittletable","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fptmcg%2Flittletable/lists"}