{"id":13470377,"url":"https://github.com/linkedin/parseq","last_synced_at":"2025-05-15T06:05:17.263Z","repository":{"id":4711591,"uuid":"5859444","full_name":"linkedin/parseq","owner":"linkedin","description":"Asynchronous Java made easier","archived":false,"fork":false,"pushed_at":"2024-06-11T00:23:49.000Z","size":4927,"stargazers_count":1167,"open_issues_count":54,"forks_count":266,"subscribers_count":133,"default_branch":"master","last_synced_at":"2025-04-11T15:57:08.876Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/linkedin.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-09-18T16:43:59.000Z","updated_at":"2025-04-08T06:35:12.000Z","dependencies_parsed_at":"2023-12-13T07:31:28.515Z","dependency_job_id":"385f45c6-b3cc-4a38-94de-b702859d9963","html_url":"https://github.com/linkedin/parseq","commit_stats":{"total_commits":736,"total_committers":45,"mean_commits":"16.355555555555554","dds":"0.39538043478260865","last_synced_commit":"3e6e50b73e069f31233f1bc4cd9cc3370f0090c0"},"previous_names":[],"tags_count":118,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fparseq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fparseq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fparseq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fparseq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/linkedin","download_url":"https://codeload.github.com/linkedin/parseq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254283339,"owners_count":22045140,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T16:00:29.404Z","updated_at":"2025-05-15T06:05:17.239Z","avatar_url":"https://github.com/linkedin.png","language":"Java","funding_links":[],"categories":["Java","并发编程"],"sub_categories":[],"readme":"# ParSeq\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Build Status](https://secure.travis-ci.org/linkedin/parseq.png?branch=master)](http://travis-ci.org/linkedin/parseq)\n\nParSeq is a framework that makes it easier to write asynchronous code in Java.\n\nSome of the key benefits of ParSeq include:\n\n* [Parallelization of asynchronous operations (such as IO)](https://github.com/linkedin/parseq/wiki/User%27s-Guide#parallel-composition)\n* [Serialized execution for non-blocking computation](https://github.com/linkedin/parseq/wiki/User%27s-Guide#transforming-tasks)\n* [Code reuse via task composition](https://github.com/linkedin/parseq/wiki/User%27s-Guide#composiing-tasks)\n* [Simple error propagation and recovery](https://github.com/linkedin/parseq/wiki/User%27s-Guide#handling-errors)\n* [Execution tracing and visualization](https://github.com/linkedin/parseq/wiki/Tracing)\n* [Batching of asynchronous operations](https://github.com/linkedin/parseq/tree/master/subprojects/parseq-batching)\n* [Tasks with retry policy](https://github.com/linkedin/parseq/wiki/User%27s-Guide#retrying)\n\n[Our Wiki](https://github.com/linkedin/parseq/wiki) includes an introductory example, a User's Guide, javadoc, and more.\n\nSee [CHANGELOG](https://github.com/linkedin/parseq/blob/master/CHANGELOG.md) for list of changes.\n\n## Introductory Example\n\nIn this example we show how to fetch several pages in parallel and how to combine them once they've all been retrieved.\n\nYou can find source code here: [IntroductoryExample](https://github.com/linkedin/parseq/tree/master/subprojects/parseq-examples/src/main/java/com/linkedin/parseq/example/introduction/IntroductoryExample.java).\n\nFirst we can retrieve a single page using an [asynchronous HTTP client](https://github.com/linkedin/parseq/tree/master/subprojects/parseq-http-client) as follows:\n\n```java\n    final Task\u003cResponse\u003e google = HttpClient.get(\"http://www.google.com\").task();\n    engine.run(google);\n    google.await();\n    System.out.println(\"Google Page: \" + google.get().getResponseBody());\n```\n\nThis will print:\n\n```\nGoogle Page: \u003c!doctype html\u003e\u003chtml\u003e...\n```\n\nIn this code snippet we don't really get any benefit from ParSeq. Essentially we create a task that can be run asynchronously, but then we block for completion using `google.await()`. In this case, the code is more complicated than issuing a simple synchronous call. We can improve this by making it asynchronous:\n\n```java\n    final Task\u003cString\u003e google = HttpClient.get(\"http://www.google.com\").task()\n        .map(Response::getResponseBody)\n        .andThen(body -\u003e System.out.println(\"Google Page: \" + body));\n\n    engine.run(google);\n```\n\nWe used `map` method to transform `Response` into the `String` and `andThen` method to print out result.\nNow, let's expand the example so that we can fetch a few more pages in parallel.\nFirst, let's create a helper method that creates a task responsible for fetching page body given a URL.\n\n```java\n    private Task\u003cString\u003e fetchBody(String url) {\n      return HttpClient.get(url).task().map(Response::getResponseBody);\n    }\n```\n\nNext, we will compose tasks to run in parallel using `Task.par`.\n\n```java\n    final Task\u003cString\u003e google = fetchBody(\"http://www.google.com\");\n    final Task\u003cString\u003e yahoo = fetchBody(\"http://www.yahoo.com\");\n    final Task\u003cString\u003e bing = fetchBody(\"http://www.bing.com\");\n\n    final Task\u003cString\u003e plan = Task.par(google, yahoo, bing)\n        .map((g, y, b) -\u003e \"Google Page: \" + g +\" \\n\" +\n                          \"Yahoo Page: \" + y + \"\\n\" +\n                          \"Bing Page: \" + b + \"\\n\")\n        .andThen(System.out::println);\n\n    engine.run(plan);\n```\n\nThis example is fully asynchronous. The home pages for Google, Yahoo, and Bing are all fetched in parallel while the original thread has returned to the calling code. We used `Tasks.par` to tell the engine to parallelize these HTTP requests. Once all of the responses have been retrieved they are transformed into a `String` that is finally printed out.\n\nWe can do various transforms on the data we retrieved. Here's a very simple transform that sums the length of the 3 pages that were fetched:\n\n```java\n    final Task\u003cInteger\u003e sumLengths =\n        Task.par(google.map(String::length),\n                 yahoo.map(String::length),\n                 bing.map(String::length))\n             .map(\"sum\", (g, y, b) -\u003e g + y + b);\n```\n\nThe `sumLengths` task can be given to an engine for execution and its result value will be set to the sum of the length of the 3 fetched pages.\n\nNotice that we added descriptions to tasks. e.g. `map(\"sum\", (g, y, b) -\u003e g + y + b)`. Using ParSeq's [trace visualization tools](https://github.com/linkedin/parseq/wiki/Tracing) we can visualize execution of the plan.\nWaterfall graph shows tasks execution in time (notice how all GET requests are executed in parallel):\n\n![sum-lengths-waterfall-example.png](images/sum-lengths-waterfall-example.png)\n\nGraphviz diagram best describes relationships between tasks:\n\n![sum-lengths-graphviz-example.png](images/sum-lengths-graphviz-example.png)\n\nFor more in-depth description of ParSeq please visit [User's Guide](https://github.com/linkedin/parseq/wiki/User's-Guide).\n\nFor many more examples, please see the [parseq-examples](https://github.com/linkedin/parseq/tree/master/subprojects/parseq-examples) contrib project in the source code.\n\n## Build\n\nBuild and test whole parseq code `./gradlew clean build`\n\nBuild ParSeq subproject(modules) instead of the whole project:\n`./gradlew :\u003cmodule_name\u003e:build`\n\nBuilding on MacOS Catalina (\u003e=10.15): Follow [this guide](https://github.com/nodejs/node-gyp/blob/master/macOS_Catalina.md) to install the required Xcode Command Line Tools and add below environment variables\n```shell script\nexport LDFLAGS=\"-mmacosx-version-min=10.13\"\nexport CXXFLAGS=\"-mmacosx-version-min=10.13\"\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinkedin%2Fparseq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinkedin%2Fparseq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinkedin%2Fparseq/lists"}