{"id":13936856,"url":"https://github.com/frictionlessdata/tableschema-pandas-py","last_synced_at":"2025-10-03T15:07:09.662Z","repository":{"id":57473094,"uuid":"58187088","full_name":"frictionlessdata/tableschema-pandas-py","owner":"frictionlessdata","description":"Generate Pandas frames, load and extract data, based on JSON Table Schema descriptors.","archived":false,"fork":false,"pushed_at":"2021-06-01T12:56:17.000Z","size":69,"stargazers_count":51,"open_issues_count":0,"forks_count":8,"subscribers_count":9,"default_branch":"main","last_synced_at":"2024-05-03T06:22:28.923Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frictionlessdata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-05-06T06:50:34.000Z","updated_at":"2024-01-19T10:24:04.000Z","dependencies_parsed_at":"2022-09-19T09:30:27.470Z","dependency_job_id":null,"html_url":"https://github.com/frictionlessdata/tableschema-pandas-py","commit_stats":null,"previous_names":["frictionlessdata/jsontableschema-pandas-py"],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-pandas-py","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-pandas-py/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-pandas-py/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frictionlessdata%2Ftableschema-pandas-py/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frictionlessdata","download_url":"https://codeload.github.com/frictionlessdata/tableschema-pandas-py/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226693902,"owners_count":17667757,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-07T23:03:03.665Z","updated_at":"2025-10-03T15:07:03.371Z","avatar_url":"https://github.com/frictionlessdata.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# tableschema-pandas-py\n\n[![Travis](https://img.shields.io/travis/frictionlessdata/tableschema-pandas-py/master.svg)](https://travis-ci.org/frictionlessdata/tableschema-pandas-py)\n[![Coveralls](http://img.shields.io/coveralls/frictionlessdata/tableschema-pandas-py.svg?branch=master)](https://coveralls.io/r/frictionlessdata/tableschema-pandas-py?branch=master)\n[![PyPi](https://img.shields.io/pypi/v/tableschema-pandas.svg)](https://pypi.python.org/pypi/tableschema-pandas)\n[![Github](https://img.shields.io/badge/github-master-brightgreen)](https://github.com/frictionlessdata/tableschema-pandas-py)\n[![Gitter](https://img.shields.io/gitter/room/frictionlessdata/chat.svg)](https://gitter.im/frictionlessdata/chat)\n\nGenerate and load Pandas data frames [Table Schema](http://specs.frictionlessdata.io/table-schema/) descriptors.\n\n## Features\n\n- implements `tableschema.Storage` interface\n\n## Contents\n\n\u003c!--TOC--\u003e\n\n  - [Getting Started](#getting-started)\n    - [Installation](#installation)\n  - [Documentation](#documentation)\n  - [API Reference](#api-reference)\n    - [`Storage`](#storage)\n  - [Contributing](#contributing)\n  - [Changelog](#changelog)\n\n\u003c!--TOC--\u003e\n\n## Getting Started\n\n### Installation\n\nThe package use semantic versioning. It means that major versions  could include breaking changes. It's highly recommended to specify `package` version range in your `setup/requirements` file e.g. `package\u003e=1.0,\u003c2.0`.\n\n```\n$ pip install tableschema-pandas\n```\n\n## Documentation\n\n```python\n# pip install datapackage tableschema-pandas\nfrom datapackage import Package\n\n# Save to Pandas\n\npackage = Package('http://data.okfn.org/data/core/country-list/datapackage.json')\nstorage = package.save(storage='pandas')\n\nprint(type(storage['data']))\n#  \u003cclass 'pandas.core.frame.DataFrame'\u003e\n\nprint(storage['data'].head())\n#               Name   Code\n#  0     Afghanistan   AF\n#  1   Åland Islands   AX\n#  2         Albania   AL\n#  3         Algeria   DZ\n#  4  American Samoa   AS\n\n# Load from Pandas\n\npackage = Package(storage=storage)\nprint(package.descriptor)\nprint(package.resources[0].read())\n```\n\nStorage works as a container for Pandas data frames. You can define new data frame inside storage using `storage.create` method:\n\n```python\n\u003e\u003e\u003e from tableschema_pandas import Storage\n\n\u003e\u003e\u003e storage = Storage()\n```\n\n```python\n\u003e\u003e\u003e storage.create('data', {\n...     'primaryKey': 'id',\n...     'fields': [\n...         {'name': 'id', 'type': 'integer'},\n...         {'name': 'comment', 'type': 'string'},\n...     ]\n... })\n\n\u003e\u003e\u003e storage.buckets\n['data']\n\n\u003e\u003e\u003e storage['data'].shape\n(0, 0)\n```\n\nUse `storage.write` to populate data frame with data:\n\n```python\n\u003e\u003e\u003e storage.write('data', [(1, 'a'), (2, 'b')])\n\n\u003e\u003e\u003e storage['data']\nid comment\n1        a\n2        b\n```\n\nAlso you can use [tabulator](https://github.com/frictionlessdata/tabulator-py) to populate data frame from external data file. As you see, subsequent writes simply appends new data on top of existing ones:\n\n```python\n\u003e\u003e\u003e import tabulator\n\n\u003e\u003e\u003e with tabulator.Stream('data/comments.csv', headers=1) as stream:\n...     storage.write('data', stream)\n\n\u003e\u003e\u003e storage['data']\nid comment\n1        a\n2        b\n1     good\n```\n\n## API Reference\n\n### `Storage`\n```python\nStorage(self, dataframes=None)\n```\nPandas storage\n\nPackage implements\n[Tabular Storage](https://github.com/frictionlessdata/tableschema-py#storage)\ninterface (see full documentation on the link):\n\n![Storage](https://i.imgur.com/RQgrxqp.png)\n\n\u003e Only additional API is documented\n\n__Arguments__\n- __dataframes (object[])__: list of storage dataframes\n\n\n## Contributing\n\n\u003e The project follows the [Open Knowledge International coding standards](https://github.com/okfn/coding-standards).\n\nRecommended way to get started is to create and activate a project virtual environment.\nTo install package and development dependencies into active environment:\n\n```bash\n$ make install\n```\n\nTo run tests with linting and coverage:\n\n```bash\n$ make test\n```\n\n## Changelog\n\nHere described only breaking and the most important changes. The full changelog and documentation for all released versions could be found in nicely formatted [commit history](https://github.com/frictionlessdata/tableschema-pandas-py/commits/master).\n\n#### v1.1\n\n- Added support for composite primary keys (loading to pandas)\n\n#### v1.0\n\n- Initial driver implementation\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrictionlessdata%2Ftableschema-pandas-py","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrictionlessdata%2Ftableschema-pandas-py","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrictionlessdata%2Ftableschema-pandas-py/lists"}