{"id":21444132,"url":"https://github.com/relvaner/nodes4j-core","last_synced_at":"2025-07-14T18:31:13.085Z","repository":{"id":42781822,"uuid":"156108579","full_name":"relvaner/nodes4j-core","owner":"relvaner","description":"Framework for parallel processing based on Actor4j. Useful for data analysis.","archived":false,"fork":false,"pushed_at":"2023-12-07T20:04:41.000Z","size":286,"stargazers_count":4,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-04-16T19:10:24.575Z","etag":null,"topics":["actor","actor-model","actor4j","actors","batch-processing","data-analysis","graph-processing","java","java-17","message-passing","parallelization","reactive-system","stream-processing"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/relvaner.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-04T17:45:37.000Z","updated_at":"2024-04-16T19:10:24.576Z","dependencies_parsed_at":"2022-08-22T17:51:08.972Z","dependency_job_id":null,"html_url":"https://github.com/relvaner/nodes4j-core","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/relvaner%2Fnodes4j-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/relvaner%2Fnodes4j-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/relvaner%2Fnodes4j-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/relvaner%2Fnodes4j-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/relvaner","download_url":"https://codeload.github.com/relvaner/nodes4j-core/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225990432,"owners_count":17556155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actor","actor-model","actor4j","actors","batch-processing","data-analysis","graph-processing","java","java-17","message-passing","parallelization","reactive-system","stream-processing"],"created_at":"2024-11-23T02:16:15.077Z","updated_at":"2025-07-14T18:31:13.074Z","avatar_url":"https://github.com/relvaner.png","language":"Java","readme":"[![Build Status](https://travis-ci.org/relvaner/nodes4j-core.svg?branch=master)](https://travis-ci.org/relvaner/nodes4j-core)\r\n[![Coverage Status](https://coveralls.io/repos/github/relvaner/nodes4j-core/badge.svg?branch=master)](https://coveralls.io/github/relvaner/nodes4j-core?branch=master)\r\n[![License](https://img.shields.io/hexpm/l/plug.svg)](https://www.apache.org/licenses/)\r\n\r\n\u003e This library has been **moved to [Actor4j - Streams](https://github.com/relvaner/actor4j/tree/master/actor4j-streams)**.\r\n\r\n## Nodes4j ##\r\n\r\nAt `nodes4j` nodes correspond to the processes, following the process algebra [[1](#1)]. Several processes can be executed both sequentially and in parallel. Figure 1 shows the general workflow of a process. The incoming data of the process P1 is first split evenly and then mapped accordingly. Then the results are merged (Reduce) and sent to the processes P2, P3 and P4 (Hub). The `MapReduce` process is executed in parallel. The advantage of this approach lies in the loose coupling of the nodes or processes. They can be easily exchanged and replaced by others.\r\n\r\n\u003cimg src=\"doc/images/workflow.jpg\" alt=\"Schematic representation of the workflow of nodes4j\" width=\"450\" height=\"255\"/\u003e\r\n\r\nFig. 1: Schematic representation of the workflow of `nodes4j`\r\n\r\nIf several processes are connected in sequence at `nodes4j`, this is a linear pipeline. When arranged in parallel around a concurrent pipeline (nonlinear pipeline [[2](#2)]). The process could be optimized by a vertical scaling, if appropriate hardware would be available. This means that the corresponding pipeline layout would need to be duplicated repeatedly in order to be able to scale it vertically. The processes can be arranged to a directional acyclic graph (Figure 2).\r\n\r\n\u003cimg src=\"doc/images/dag.jpg\" alt=\"Directional acyclic graph (DAG)\" width=\"329\" height=\"295\"/\u003e\r\n\r\nFig. 2: Directional acyclic graph (DAG)\r\n\r\n## Implementation ##\r\n\r\nThe core components of `nodes4j` are `Process`, `NodeActor` and `TaskActor`. A process instantiates a `NodeActor` at startup. The `NodeActor` is the receiving point of the data to be processed. `TaskActors` then process the data in parallel. `Process` provides a variety of operation to apply to the data.\r\n\r\n### NodeActor ###\r\n\r\nIf child nodes are present, they are initialized accordingly. This automatically creates the structure of a directed graph. \r\n- Upon receipt of the `DATA` message, the list of data is evenly distributed to the newly generated `TaskActors` for parallel processing. \r\n- If the `NodeActor` is at a final end point of the graph, the result of the processing is stored under its current `UUID` or alias. \r\n- When all final endpoints of the graph have completed their calculations, the `ActorSystem` is currently automatically shutdown.\r\n\r\n### TaskActor ###\r\n\r\n- First, on the partial data, the registered node operations (filtering, mapping, etc.) are performed.\r\n- In the second step, the reduce operation is initiated, which may include a binary operation (e.g., new result of p0 x p1 -\u003e p0*). See the lower tree-like communication structure during the reduce operation of the participating `TaskActors`. The communication is asynchronous. The structure of the merge process (see Figure 3) enables an optimal load distribution to the involved `TaskActors`.\r\n- The result may be passed on to successor nodes.\r\n\r\n\u003cimg src=\"doc/images/tree.jpg\" alt=\"Representation of the tree-like communication structure during the reduction process.\" width=\"366\" height=\"334\"/\u003e\r\n\r\nFig. 3: Representation of the tree-like communication structure during the reduction process\r\n\r\n## Internal DSL ##\r\n\r\n```java\r\nprocess1\r\n    // intra-process operation\r\n    .data(...)\r\n    .filter(...)\r\n    .map(...)\r\n    .forEach(...)\r\n    .reduce(…)\r\n    \r\n    // or\r\n    .data(...)\r\n    .flatMap(...)\r\n    .reduce(…)\r\n    \r\n    // or with Java Stream API\r\n    .data(...)\r\n    .stream(s -\u003e s.filter(...).map(...))\r\n    .reduce(…)\r\n    \r\n    // or with RxJava 2\r\n    .data(...)\r\n    .streamRx(o -\u003e o.filter(...).map(...))\r\n    .reduce(…)\r\n    \r\n    // sorting as inter-process operation\r\n    .sortedASC() or .sortedDESC()\r\n\r\n    // inter-process operation\r\n    .sequence(process2, process3.parallel(process5, process6, process7));\r\n        \r\nprocess8.merge(process6, process7);\r\n```\r\n## Example ##\r\n\r\n```java\r\nProcess\u003cInteger, Integer\u003e process_main = new Process\u003c\u003e(\"process_main\");\r\nprocess_main\r\n\t.data(List.of(14, 31, 34, 45, 78, 99, 123, 9257));\r\n\t\t\r\nProcess\u003cInteger, Integer\u003e process_a = new Process\u003c\u003e(\"process_a\");\r\nprocess_a\r\n\t.filter((v) -\u003e v\u003e50 \u0026\u0026 v\u003c100)\r\n\t.map((v) -\u003e v+2);\r\nProcess\u003cInteger, Integer\u003e process_b = new Process\u003c\u003e(\"process_b\");\r\nprocess_b\r\n\t.filter((v) -\u003e v\u003e0 \u0026\u0026 v\u003c=50)\r\n\t.map((v) -\u003e v+1);\r\nProcess\u003cInteger, Integer\u003e process_sort_asc = new SortProcess\u003cInteger\u003e(\"process_sort_asc\",\r\n\tSortType.SORT_ASCENDING);\t\t\r\n\t\t\r\nprocess_main.parallel(process_a, process_b);\r\nprocess_sort_asc.merge(process_a, process_b);\r\n\t\t\r\nProcessManager manager = new ProcessManager(true);\r\nmanager\r\n\t.onTermination(() -\u003e { \r\n\t\tlogger().debug(\"Data (process_a): \"+manager.getData(\"process_a\")); \r\n\t\tlogger().debug(\"Data (process_a): \"+process_a.getData()); \r\n\t\tlogger().debug(\"Data (process_b): \"+manager.getData(\"process_b\")); \r\n\t\tlogger().debug(\"Data (process_sort_asc): \"+manager.getData(\"process_sort_asc\")); \r\n\t\tlogger().debug(\"Result (process_sort_asc): \"+manager.getResult(\"process_sort_asc\")); \r\n\t})\r\n\t.start(process_main);\r\n```\r\n\r\n## License ##\r\nThis framework is released under an open source Apache 2.0 license.\r\n\r\n## Announcement ##\r\nThis software framework is currently under an prototype state.\r\n\r\n## References ##\r\n[1]\u003ca name=\"1\"/\u003e Baeten, J.C.M., Basten, T. \u0026 Reniers, M.A (2009). Process Algebra. Equational Theories of Communicating Processes. Volume 50. Cambridge University Press.  \r\n[2]\u003ca name=\"2\"/\u003e Mattson, Timothy G., Snaders, Beverly A. \u0026 Massingill, Berna L. (2004). A Pattern Language for Parallel Programming. Addison Wesley. http://www.cise.ufl.edu/research/ParallelPatterns/PatternLanguage/AlgorithmStructure/Pipeline.htm  \r\n\r\nPage to be updated 11/09/2018\r\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frelvaner%2Fnodes4j-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frelvaner%2Fnodes4j-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frelvaner%2Fnodes4j-core/lists"}