{"id":17552573,"url":"https://github.com/iloveitaly/funcy-pipe","last_synced_at":"2025-04-24T01:41:52.013Z","repository":{"id":188978751,"uuid":"679277722","full_name":"iloveitaly/funcy-pipe","owner":"iloveitaly","description":"If Funcy and Pipe had a baby. Decorates all Funcy methods with Pipe superpowers.","archived":false,"fork":false,"pushed_at":"2025-04-14T12:00:48.000Z","size":167,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-14T13:22:03.387Z","etag":null,"topics":["data-manipulation","functional-programming","funcy","pipe","python"],"latest_commit_sha":null,"homepage":"https://github.com/iloveitaly/funcy-pipe","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iloveitaly.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["iloveitaly"]}},"created_at":"2023-08-16T13:36:43.000Z","updated_at":"2025-04-14T12:00:52.000Z","dependencies_parsed_at":"2023-12-19T08:01:04.641Z","dependency_job_id":"7e250965-e476-4647-b38e-f10c4c8439d3","html_url":"https://github.com/iloveitaly/funcy-pipe","commit_stats":{"total_commits":28,"total_committers":3,"mean_commits":9.333333333333334,"dds":0.2857142857142857,"last_synced_commit":"b48d5d021cb682d500f60b4163758d081437c47b"},"previous_names":["iloveitaly/funcy-pipe"],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Ffuncy-pipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Ffuncy-pipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Ffuncy-pipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iloveitaly%2Ffuncy-pipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iloveitaly","download_url":"https://codeload.github.com/iloveitaly/funcy-pipe/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250545653,"owners_count":21448246,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-manipulation","functional-programming","funcy","pipe","python"],"created_at":"2024-10-21T05:43:14.300Z","updated_at":"2025-04-24T01:41:51.986Z","avatar_url":"https://github.com/iloveitaly.png","language":"Python","funding_links":["https://github.com/sponsors/iloveitaly"],"categories":[],"sub_categories":[],"readme":"[![Release Notes](https://img.shields.io/github/release/iloveitaly/funcy-pipe)](https://github.com/iloveitaly/funcy-pipe/releases) [![Downloads](https://static.pepy.tech/badge/funcy-pipe/month)](https://pepy.tech/project/funcy-pipe) [![Python Versions](https://img.shields.io/pypi/pyversions/funcy-pipe)](https://pypi.org/project/funcy-pipe) ![GitHub CI Status](https://github.com/iloveitaly/funcy-pipe/actions/workflows/build_and_publish.yml/badge.svg) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n# Funcy with pipeline-based operators\n\nIf [Funcy](https://github.com/Suor/funcy) and [Pipe](https://github.com/JulienPalard/Pipe) had a baby. Deal with data transformation in python in a sane way.\n\nI love Ruby. It's a great language and one of the things they got right was pipelined data transformation. Elixir got this\neven more right with the explicit pipeline operator `|\u003e`.\n\nHowever, Python is the way of the future. As I worked more with Python, it was driving me nuts that the\ndata transformation options were not chainable.\n\nThis project fixes this pet peeve.\n\n## Installation\n\n```shell\npip install funcy-pipe\n```\n\nOr, if you are using poetry:\n\n```shell\npoetry add funcy-pipe\n```\n\n## Examples\n\nExtract a couple key values from a sql alchemy model:\n\n```python notest\nimport funcy_pipe as fp\n\nentities_from_sql_alchemy\n  | fp.lmap(lambda r: r.to_dict())\n  | fp.lmap(lambda r: r | fp.omit([\"id\", \"created_at\", \"updated_at\"]))\n  | fp.to_list\n```\n\nOr, you can be more fancy and use [whatever](https://github.com/Suor/whatever) and `pmap`:\n\n```python notest\nimport funcy_pipe as f\nimport whatever as _\n\nentities_from_sql_alchemy\n  | fp.lmap(_.to_dict)\n  | fp.pmap(fp.omit([\"id\", \"created_at\", \"updated_at\"]))\n  | fp.to_list\n```\n\nCreate a map from an array of objects, ensuring the key is always an `int`:\n\n```python notest\nsection_map = api.get_sections() | fp.group_by(f.compose(int, that.id))\n```\n\nGrab the ID of a specific user:\n\n```python notest\nfilter_user_id = (\n  collaborator_map().values()\n  | fp.where(email=target_user)\n  | fp.pluck(\"id\")\n  | fp.first()\n)\n```\n\nGet distinct values from a list (in this case, github events):\n\n```python\nevents = [\n  {\n    \"type\": \"PushEvent\"\n  },\n  {\n    \"type\": \"CommentEvent\"\n  }\n]\n\nresult = events | fp.pluck(\"type\") | fp.distinct() | fp.to_list()\n\nassert [\"PushEvent\", \"CommentEvent\"] == result\n```\n\nWhat if the objects are not dicts?\n\n```python notest\nfilter_user_id = (\n  collaborator_map().values()\n  | fp.where_attr(email=target_user)\n  | fp.pluck_attr(\"id\")\n  | fp.first()\n)\n```\n\nHow about creating a dict where each value is sorted:\n\n```python notest\ndata\n  # each element is a dict of city information, let's group by state\n  | fp.group_by(itemgetter(\"state_name\"))\n  # now let's sort each value by population, which is stored as a string\n  | fp.walk_values(\n    f.partial(sorted, reverse=True, key=lambda c: int(c[\"population\"])),\n  )\n```\n\nA more complicated example ([lifted from this project](https://github.com/iloveitaly/todoist-digest/blob/2f893709da2cf3a0f715125053af705fc3adbc4c/run.py#L151-L166)):\n\n```python notest\ncomments = (\n    # tasks are pulled from the todoist api\n    tasks\n    # get all comments for each relevant task\n    | fp.lmap(lambda task: api.get_comments(task_id=task.id))\n    # each task's comments are returned as an array, let's flatten this\n    | fp.flatten()\n    # dates are returned as strings, let's convert them to datetime objects\n    | fp.lmap(enrich_date)\n    # no date filter is applied by default, we don't want all comments\n    | fp.lfilter(lambda comment: comment[\"posted_at_date\"] \u003e last_synced_date)\n    # comments do not come with who created the comment by default, we need to hit a separate API to add this to the comment\n    | fp.lmap(enrich_comment)\n    # only select the comments posted by our target user\n    | fp.lfilter(lambda comment: comment[\"posted_by_user_id\"] == filter_user_id)\n    # there is no `sort` in the funcy library, so we reexport the sort built-in so it's pipe-able\n    | fp.sort(key=\"posted_at_date\")\n    # create a dictionary of task_id =\u003e [comments]\n    | fp.group_by(lambda comment: comment[\"task_id\"])\n)\n```\n\nWant to grab the values of a list of dict keys?\n\n```python\ndef add_field_name(input: dict, keys: list[str]) -\u003e dict:\n    return input | {\n        \"field_name\": (\n            keys\n            # this is a sneaky trick: if we reference the objects method, when it's called it will contain a reference\n            # to the object\n            | fp.map(input.get)\n            | fp.compact\n            | fp.join_str(\"_\")\n        )\n    }\n\nresult = [{ \"category\": \"python\", \"header\": \"functional\"}] | fp.map(fp.rpartial(add_field_name, [\"category\", \"header\"])) | fp.to_list\nassert result == [{'category': 'python', 'header': 'functional', 'field_name': 'python_functional'}]\n```\n\nYou can also easily test multiple conditions across API data ([extracted from this project](https://github.com/iloveitaly/github-overlord/blob/a3c0e5d0765b3748747e6721e602c0021be0c8e1/github_overlord/__init__.py#L66-L71))\n\n```python notest\nall_checks_successful = (\n    last_commit.get_check_runs()\n    | fp.pluck_attr(\"conclusion\")\n    # if you pass a set into `all` each element of the set is used to build a predicate\n    # this condition tests if the \"conclusion\" attribute is either \"success\" or \"skipped\"\n    | fp.all({\"success\", \"skipped\"})\n)\n```\n\nWant to grab the values of a list of dict keys?\n\n```python\ndef add_field_name(input: dict, keys: list[str]) -\u003e dict:\n    return input | {\n        \"field_name\": (\n            keys\n            # this is a sneaky trick: if we reference the objects method, when it's called it will contain a reference\n            # to the object\n            | fp.map(input.get)\n            | fp.compact\n            | fp.join_str(\"_\")\n        )\n    }\n\nresult = [{ \"category\": \"python\", \"header\": \"functional\"}] | fp.map(fp.rpartial(add_field_name, [\"category\", \"header\"])) | fp.to_list\nassert result == [{'category': 'python', 'header': 'functional', 'field_name': 'python_functional'}]\n```\n\nYou can also easily group dictionaries by a key (or arbitrary function):\n\n```python\nimport operator\n\nresult = [{\"age\": 10, \"name\": \"Alice\"}, {\"age\": 12, \"name\": \"Bob\"}] | fp.group_by(operator.itemgetter(\"age\"))\nassert result == {10: [{'age': 10, 'name': 'Alice'}], 12: [{'age': 12, 'name': 'Bob'}]}\n```\n\n## Extras\n\n* to_list\n* log\n* bp. run `breakpoint()` on the input value\n* sort\n* exactly_one. Throw an error if the input is not exactly one element\n* reduce\n* pmap. Pass each element of a sequence into a pipe'd function\n\n## Extensions\n\nThere [are some functions](funcy_pipe/funcy_extensions.py) which are not yet merged upstream into funcy, and may never be. You can patch `funcy` to add them using:\n\n```python\nimport funcy_pipe\nfuncy_pipe.patch()\n```\n\n## Coming From Ruby?\n\n* uniq =\u003e distinct\n* detect =\u003e `where(some=\"Condition\") | first` or `where_attr(some=\"Condition\") | first`\n* inverse =\u003e complement\n* times =\u003e repeatedly\n\n### Module Alias\n\nCreate a module alias for `funcy-pipe` to make things clean (`import *` always irks me):\n\n```python notest\n# fp.py\nfrom funcy_pipe import *\n\n# code py\nimport fp\n```\n\n# Inspiration\n\n* Elixir's pipe operator. `array |\u003e map(fn) |\u003e filter(fn)`\n* Ruby's enumerable library. `array.map(\u0026:fn).filter(\u0026:fn)`\n* https://pypi.org/project/funcy-chain\n* https://github.com/JulienPalard/Pipe\n\n# Related Projects\n\n* https://pydash.readthedocs.io/en/latest/\n\n# TODO\n\n- [ ] tests\n- [ ] docs for additional utils\n- [ ] fix typing threading\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filoveitaly%2Ffuncy-pipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Filoveitaly%2Ffuncy-pipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filoveitaly%2Ffuncy-pipe/lists"}