{"id":35749361,"url":"https://github.com/lincc-frameworks/nested-pandas","last_synced_at":"2026-01-21T19:03:05.108Z","repository":{"id":230968151,"uuid":"780600289","full_name":"lincc-frameworks/nested-pandas","owner":"lincc-frameworks","description":"Efficient Pandas representation for nested associated datasets.","archived":false,"fork":false,"pushed_at":"2026-01-12T20:46:11.000Z","size":18742,"stargazers_count":18,"open_issues_count":49,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-01-12T21:29:13.413Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://nested-pandas.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lincc-frameworks.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-04-01T20:08:32.000Z","updated_at":"2026-01-12T20:43:00.000Z","dependencies_parsed_at":"2026-01-06T13:09:05.898Z","dependency_job_id":null,"html_url":"https://github.com/lincc-frameworks/nested-pandas","commit_stats":null,"previous_names":["lincc-frameworks/nested-pandas"],"tags_count":35,"template":false,"template_full_name":null,"purl":"pkg:github/lincc-frameworks/nested-pandas","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lincc-frameworks%2Fnested-pandas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lincc-frameworks%2Fnested-pandas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lincc-frameworks%2Fnested-pandas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lincc-frameworks%2Fnested-pandas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lincc-frameworks","download_url":"https://codeload.github.com/lincc-frameworks/nested-pandas/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lincc-frameworks%2Fnested-pandas/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28640166,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T18:04:35.752Z","status":"ssl_error","status_checked_at":"2026-01-21T18:03:55.054Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-06T19:09:58.692Z","updated_at":"2026-01-21T19:03:05.097Z","avatar_url":"https://github.com/lincc-frameworks.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nested-pandas\n\n[![Template](https://img.shields.io/badge/Template-LINCC%20Frameworks%20Python%20Project%20Template-brightgreen)](https://lincc-ppt.readthedocs.io/en/latest/)\n\n[![PyPI](https://img.shields.io/pypi/v/nested-pandas?color=blue\u0026logo=pypi\u0026logoColor=white)](https://pypi.org/project/nested-pandas/)\n[![Conda](https://img.shields.io/conda/vn/conda-forge/nested-pandas.svg?color=blue\u0026logo=condaforge\u0026logoColor=white)](https://anaconda.org/conda-forge/nested-pandas)\n\n[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/lincc-frameworks/nested-pandas/smoke-test.yml)](https://github.com/lincc-frameworks/nested-pandas/actions/workflows/smoke-test.yml)\n[![codecov](https://codecov.io/gh/lincc-frameworks/nested-pandas/branch/main/graph/badge.svg)](https://codecov.io/gh/lincc-frameworks/nested-pandas)\n[![Read the Docs](https://img.shields.io/readthedocs/nested-pandas)](https://nested-pandas.readthedocs.io/)\n[![benchmarks](https://img.shields.io/github/actions/workflow/status/lincc-frameworks/nested-pandas/asv-main.yml?label=benchmarks)](https://lincc-frameworks.github.io/nested-pandas/)\n\nAn extension of pandas for efficient representation of nested\nassociated datasets.\n\nNested-Pandas extends the [pandas](https://pandas.pydata.org/) package with \ntooling and support for nested dataframes packed into values of top-level \ndataframe columns. [Pyarrow](https://arrow.apache.org/docs/python/index.html) \nis used internally to aid in scalability and performance.\n\nNested-Pandas allows data like this:\n\n\u003cp align=\"left\"\u003e\n    \u003cimg src=\"https://github.com/lincc-frameworks/nested-pandas/raw/refs/heads/main/docs/intro_images/pandas_dfs.png\" alt=\"pandas dataframes\" width=\"400\"/\u003e\n\u003c/p\u003e\n\nTo instead be represented like this:\n\n\u003cp align=\"left\"\u003e\n    \u003cimg src=\"https://github.com/lincc-frameworks/nested-pandas/raw/refs/heads/main/docs/intro_images/nestedframe_example.png\" alt=\"nestedframe\" width=\"300\"/\u003e\n\u003c/p\u003e\n\nWhere the nested data is represented as nested dataframes:\n\n```python\n   # Each row of \"object_nf\" now has it's own sub-dataframe of matched rows from \"source_df\"\n   object_nf.loc[0][\"nested_sources\"]\n```\n\n\u003cp align=\"left\"\u003e\n    \u003cimg src=\"https://github.com/lincc-frameworks/nested-pandas/raw/refs/heads/main/docs/intro_images/loc_into_nested.png\" alt=\"sub-dataframe\" width=\"225\"/\u003e\n\u003c/p\u003e\n\nAllowing powerful and straightforward operations, like:\n\n```python\n   # Compute the mean flux for each row of \"object_nf\"\n   import numpy as np\n\n   def mean_flux(row):\n   \"\"\"Calculates the mean flux for each object\"\"\"\n       return np.mean(row[\"nested_sources.flux\"])\n\n   object_nf.map_rows(mean_flux, output_names=\"mean_flux\")\n```\n\n\u003cp align=\"left\"\u003e\n    \u003cimg src=\"https://github.com/lincc-frameworks/nested-pandas/raw/refs/heads/main/docs/intro_images/reduce.png\" alt=\"using reduce\" width=\"150\"/\u003e\n\u003c/p\u003e\n\nNested-Pandas is motivated by time-domain astronomy use cases, where we see\ntypically two levels of information, information about astronomical objects and\nthen an associated set of `N` measurements of those objects. Nested-Pandas offers\na performant and memory-efficient package for working with these types of datasets. \n\nCore advantages being:\n* hierarchical column access\n* efficient packing of nested information into inputs to custom user functions\n* avoiding costly groupby operations\n\n\n\nThis is a LINCC Frameworks project - find more information about LINCC Frameworks [here](https://lsstdiscoveryalliance.org/programs/lincc-frameworks/).\n\n\n\n## Acknowledgements\n\nThis project is supported by Schmidt Sciences.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flincc-frameworks%2Fnested-pandas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flincc-frameworks%2Fnested-pandas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flincc-frameworks%2Fnested-pandas/lists"}