{"id":13541203,"url":"https://github.com/fdncred/nu_plugin_parquet","last_synced_at":"2025-03-21T22:32:13.974Z","repository":{"id":54819851,"uuid":"522557084","full_name":"fdncred/nu_plugin_parquet","owner":"fdncred","description":"A nushell plugin to read and write parquet files.","archived":false,"fork":false,"pushed_at":"2025-02-10T15:49:37.000Z","size":234,"stargazers_count":9,"open_issues_count":1,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-18T05:51:15.858Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fdncred.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-08T13:18:04.000Z","updated_at":"2025-02-10T15:49:19.000Z","dependencies_parsed_at":"2023-02-10T09:45:51.100Z","dependency_job_id":"4c4c975e-2240-42d5-a98d-376271fd05ee","html_url":"https://github.com/fdncred/nu_plugin_parquet","commit_stats":null,"previous_names":["fdncred/nu_plugin_parquet"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fdncred%2Fnu_plugin_parquet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fdncred%2Fnu_plugin_parquet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fdncred%2Fnu_plugin_parquet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fdncred%2Fnu_plugin_parquet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fdncred","download_url":"https://codeload.github.com/fdncred/nu_plugin_parquet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244880449,"owners_count":20525507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T10:00:41.323Z","updated_at":"2025-03-21T22:32:13.610Z","avatar_url":"https://github.com/fdncred.png","language":"Rust","funding_links":[],"categories":["Plugins"],"sub_categories":[],"readme":"# From jakeswenson\n\nThis repo is a copy from jakeswenson's repo and updated to support nushell v0.60+ by @flying-sheep. See the original here https://github.com/jakeswenson/nu_plugin_from_parquet. Asked permission to fork, update, and add license here https://github.com/jakeswenson/nu_plugin_from_parquet/issues/4\n\n# nu_plugin_parquet\n\n[nushell]: https://www.nushell.sh/\n[plugin]: https://www.nushell.sh/book/plugins.html#adding-a-plugin\n[structured types]: https://www.nushell.sh/book/types_of_data.html\n\nThis is a [nushell] [plugin] to add parquet compatibility with `nu` structured types. It can read parquet files to `nu` tables, or write tables to parquet files.\n\n\n# Installing\n\n[add the plugin]: https://www.nushell.sh/book/plugins.html#adding-a-plugin\n\nTo [add the plugin] permanently, just install it and call `plugin add` on it:\n\n## Using Cargo\n\n```nushell\ncargo install --path .\nplugin add ~/.cargo/bin/nu_plugin_parquet\nplugin use ~/.cargo/bin/nu_plugin_parquet # required if you don't want to quit out and restart nushell\n```\n\n## Usage\n### Reading\n\n```nushell\nopen -r sample.parquet | from parquet | first 10\n```\nor\n```nushell\nopen sample.parquet | first 10\n```\n\n```nushell\n╭───┬───────────────┬────┬────────────┬───────────┬──────────────┬────────┬──────────────┬──────────────┬──────────────┬────────────┬───────────┬──────────────┬──────────╮\n│ # │ registration… │ id │ first_name │ last_name │    email     │ gender │  ip_address  │      cc      │   country    │ birthdate  │  salary   │    title     │ comments │\n├───┼───────────────┼────┼────────────┼───────────┼──────────────┼────────┼──────────────┼──────────────┼──────────────┼────────────┼───────────┼──────────────┼──────────┤\n│ 0 │ 6 years ago   │  1 │ Amanda     │ Jordan    │ ajordan0@co… │ Female │ 1.197.201.2  │ 67595218649… │ Indonesia    │ 3/8/1971   │  49756.53 │ Internal Au… │ 1E+02    │\n│ 1 │ 6 years ago   │  2 │ Albert     │ Freeman   │ afreeman1@i… │ Male   │ 218.111.175… │              │ Canada       │ 1/16/1968  │ 150280.17 │ Accountant … │          │\n│ 2 │ 6 years ago   │  3 │ Evelyn     │ Morgan    │ emorgan2@al… │ Female │ 7.161.136.94 │ 67671190719… │ Russia       │ 2/1/1960   │ 144972.51 │ Structural … │          │\n│ 3 │ 6 years ago   │  4 │ Denise     │ Riley     │ driley3@gmp… │ Female │ 140.35.109.… │ 35760315989… │ China        │ 4/8/1997   │  90263.05 │ Senior Cost… │          │\n│ 4 │ 6 years ago   │  5 │ Carlos     │ Burns     │ cburns4@mii… │        │ 169.113.235… │ 56022562552… │ South Africa │            │           │              │          │\n│ 5 │ 6 years ago   │  6 │ Kathryn    │ White     │ kwhite5@goo… │ Female │ 195.131.81.… │ 35831363260… │ Indonesia    │ 2/25/1983  │  69227.11 │ Account Exe… │          │\n│ 6 │ 6 years ago   │  7 │ Samuel     │ Holmes    │ sholmes6@fo… │ Male   │ 232.234.81.… │ 35826413669… │ Portugal     │ 12/18/1987 │  14247.62 │ Senior Fina… │          │\n│ 7 │ 6 years ago   │  8 │ Harry      │ Howell    │ hhowell7@ee… │ Male   │ 91.235.51.73 │              │ Bosnia and … │ 3/1/1962   │ 186469.43 │ Web Develop… │          │\n│ 8 │ 6 years ago   │  9 │ Jose       │ Foster    │ jfoster8@ye… │ Male   │ 132.31.53.61 │              │ South Korea  │ 3/27/1992  │ 231067.84 │ Software Te… │ 1E+02    │\n│ 9 │ 6 years ago   │ 10 │ Emily      │ Stewart   │ estewart9@o… │ Female │ 143.28.251.… │ 35742541103… │ Nigeria      │ 1/28/1997  │  27234.28 │ Health Coac… │          │\n├───┼───────────────┼────┼────────────┼───────────┼──────────────┼────────┼──────────────┼──────────────┼──────────────┼────────────┼───────────┼──────────────┼──────────┤\n│ # │ registration… │ id │ first_name │ last_name │    email     │ gender │  ip_address  │      cc      │   country    │ birthdate  │  salary   │    title     │ comments │\n╰───┴───────────────┴────┴────────────┴───────────┴──────────────┴────────┴──────────────┴──────────────┴──────────────┴────────────┴───────────┴──────────────┴──────────╯\n```\n\n### Displaying Metadata\n\nDisplay metadata, instead of data, from the parquet file by passing the `--metadata, -m` flag to `from parquet`:\n\n```nushell\nopen -r sample.parquet | from parquet --metadata  | table -e\n```\n\n```nushell\n╭────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────╮\n│ version    │ 1                                                                                                   │\n│ creator    │ parquet-mr version 1.8.1 (build 4aba4dae7bb0d4edbcf7923ae1339f28fd3f7fcf)                           │\n│ num_rows   │ 1000                                                                                                │\n│ key_values │ [list 0 items]                                                                                      │\n│            │ ╭─────────────┬───────────────────────────────────────────────────────────────────────────────────╮ │\n│ schema     │ │ name        │ hive_schema                                                                       │ │\n│            │ │ num_columns │ 13                                                                                │ │\n│            │ │             │ ╭────┬───────────────────┬────────────┬────────────┬─────────────┬──────────────╮ │ │\n│            │ │ schema      │ │  # │       name        │ repetition │    type    │ type_length │ logical_type │ │ │\n│            │ │             │ ├────┼───────────────────┼────────────┼────────────┼─────────────┼──────────────┤ │ │\n│            │ │             │ │  0 │ registration_dttm │ OPTIONAL   │ INT96      │             │              │ │ │\n│            │ │             │ │  1 │ id                │ OPTIONAL   │ INT32      │             │              │ │ │\n│            │ │             │ │  2 │ first_name        │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │  3 │ last_name         │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │  4 │ email             │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │  5 │ gender            │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │  6 │ ip_address        │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │  7 │ cc                │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │  8 │ country           │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │  9 │ birthdate         │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │ 10 │ salary            │ OPTIONAL   │ DOUBLE     │             │              │ │ │\n│            │ │             │ │ 11 │ title             │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ │ 12 │ comments          │ OPTIONAL   │ BYTE_ARRAY │          -1 │ UTF8         │ │ │\n│            │ │             │ ╰────┴───────────────────┴────────────┴────────────┴─────────────┴──────────────╯ │ │\n│            │ ╰─────────────┴───────────────────────────────────────────────────────────────────────────────────╯ │\n│            │ ╭───┬──────────┬─────────────────╮                                                                  │\n│ row_groups │ │ # │ num_rows │ total_byte_size │                                                                  │\n│            │ ├───┼──────────┼─────────────────┤                                                                  │\n│            │ │ 0 │     1000 │          112492 │                                                                  │\n│            │ ╰───┴──────────┴─────────────────╯                                                                  │\n╰────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────╯\n```\n\n### Writing\n\n```nushell\n[{a:1, b:3}, {a: 2, b:4}] | save example.parquet\n```\n\nOr, to save all running processes: \n```nushell\nps | save example.parquet\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffdncred%2Fnu_plugin_parquet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffdncred%2Fnu_plugin_parquet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffdncred%2Fnu_plugin_parquet/lists"}