{"id":32388740,"url":"https://github.com/data-integrations/repartitioner","last_synced_at":"2025-10-25T03:55:36.480Z","repository":{"id":56463125,"uuid":"86523542","full_name":"data-integrations/repartitioner","owner":"data-integrations","description":"Repartitions a spark RDD","archived":false,"fork":false,"pushed_at":"2023-09-07T23:14:32.000Z","size":71,"stargazers_count":0,"open_issues_count":4,"forks_count":2,"subscribers_count":6,"default_branch":"develop","last_synced_at":"2024-04-16T07:44:25.337Z","etag":null,"topics":["cask-marketplace","cdap","cdap-plugin","rdd","repartition","spark","spark-streaming"],"latest_commit_sha":null,"homepage":"http://docs.cask.co/cdap","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/data-integrations.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null}},"created_at":"2017-03-29T01:15:58.000Z","updated_at":"2022-12-21T09:23:23.000Z","dependencies_parsed_at":"2023-01-30T03:15:44.813Z","dependency_job_id":null,"html_url":"https://github.com/data-integrations/repartitioner","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/data-integrations/repartitioner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-integrations%2Frepartitioner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-integrations%2Frepartitioner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-integrations%2Frepartitioner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-integrations%2Frepartitioner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/data-integrations","download_url":"https://codeload.github.com/data-integrations/repartitioner/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/data-integrations%2Frepartitioner/sbom","scorecard":{"id":324100,"data":{"date":"2025-08-11","repo":{"name":"github.com/data-integrations/repartitioner","commit":"71cdf45a79569f40cb6d72ccd2a9b166dc4f6bad"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":5.2,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":5,"reason":"Found 6/12 approved changesets -- score normalized to 5","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 17 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-18T02:04:31.424Z","repository_id":56463125,"created_at":"2025-08-18T02:04:31.424Z","updated_at":"2025-08-18T02:04:31.424Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280901444,"owners_count":26410586,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-25T02:00:06.499Z","response_time":81,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cask-marketplace","cdap","cdap-plugin","rdd","repartition","spark","spark-streaming"],"created_at":"2025-10-25T03:55:19.120Z","updated_at":"2025-10-25T03:55:36.476Z","avatar_url":"https://github.com/data-integrations.png","language":"Java","readme":"[![Build Status](https://travis-ci.org/hydrator/repartitioner.svg?branch=develop)](https://travis-ci.org/hydrator/repartitioner) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n=======\n\u003ca href=\"https://cdap-users.herokuapp.com/\"\u003e\u003cimg alt=\"Join CDAP community\" src=\"https://cdap-users.herokuapp.com/badge.svg?t=repartitioner\"/\u003e\u003c/a\u003e [![Build Status](https://travis-ci.org/hydrator/repartitioner.svg?branch=develop)](https://travis-ci.org/hydrator/repartitioner) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![cm-available](https://cdap-users.herokuapp.com/assets/cm-available.svg)](https://docs.cask.co/cdap/current/en/integrations/cask-market.html) \u003cimg  alt=\"CDAP Spark Compute\" src=\"https://cdap-users.herokuapp.com/assets/cdap-sparkcompute.svg\"/\u003e\n\n# Repartitioner\n\nRepartitioner partitions a RDD and define how it gets spread out over a cluster. \n\n## Plugin Configuration\n\n| Configuration | Required | Default | Description |\n| :------------ | :------: | :----- | :---------- |\n| **Partitions** | **N** | 1 | Specifies the number of partitions or level of parallelism to be set for the RDD emitted out of this plugin. |\n| **Shuffle Data** | **N** | False | Specifies whether the data need to be shuffled during repartitioning of RDD. |\n\n## Usage Notes\n\nBy default the behavior of repartitioning is similar to ```coalesce``` in which case, you decrease number of partition of RDD without shuffling data over the network. You can use this plugin to reduce or increase partitions of an RDD. If you are decreasing partition it's a good practise not to shuffle the data -- in doing so, it will keep the data on the number of nodes and pull in the remaining data from other nodes. \n\nIf you are using this plugin to increase the partitions it's a good practise to shuffle the data, there is a cost associated with shuffling, but evening out data over the partitions helps improve the performance. \n\n## Clone this repo\nClone the this repo to your local environment\n\n```\n  git clone https://github.com/hydrator/repartitioner.git repartitioner\n```\n\n# Build\n\n## Clone this repo\nClone the this repo to your local environment\n\n```\n  git clone https://github.com/hydrator/repartitioner.git repartitioner\n```\n\n## Build\n\nTo build your plugins:\n\n    mvn clean package -DskipTests\n\nThe build will create a .jar and .json file under the ``target`` directory.\nThese files can be used to deploy your plugins.\n\n## Deployment\nYou can deploy your plugins using the CDAP CLI:\n\n    \u003e load artifact \u003ctarget/repartitioner-\u003cversion\u003e.jar\u003e config-file \u003ctarget/repartitioner-\u003cversion\u003e.json\u003e\n\nFor example, if your artifact is named 'repartitioner-\u003cversion\u003e:\n\n    \u003e load artifact target/repartitioner-\u003cversion\u003e.jar config-file target/repartitioner-\u003cversion\u003e.json\n\n# Mailing Lists\n\nCDAP User Group and Development Discussions:\n\n- `cdap-user@googlegroups.com \u003chttps://groups.google.com/d/forum/cdap-user\u003e`__\n\nThe *cdap-user* mailing list is primarily for users using the product to develop\napplications or building plugins for appplications. You can expect questions from \nusers, release announcements, and any other discussions that we think will be helpful \nto the users.\n\n\n# License and Trademarks\n\nCopyright © 2016-2019 Cask Data, Inc.\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except\nin compliance with the License. You may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under the \nLicense is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, \neither express or implied. See the License for the specific language governing permissions \nand limitations under the License.\n\nCask is a trademark of Cask Data, Inc. All rights reserved.\n\nApache, Apache HBase, and HBase are trademarks of The Apache Software Foundation. Used with\npermission. No endorsement by The Apache Software Foundation is implied by the use of these marks.\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdata-integrations%2Frepartitioner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdata-integrations%2Frepartitioner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdata-integrations%2Frepartitioner/lists"}