{"id":18810389,"url":"https://github.com/absaoss/spark-launcher-supervisor","last_synced_at":"2026-07-06T00:31:40.358Z","repository":{"id":168049107,"uuid":"643327852","full_name":"AbsaOSS/spark-launcher-supervisor","owner":"AbsaOSS","description":"A Java agent that oversees a Spark launcher and check/enforce settings","archived":false,"fork":false,"pushed_at":"2023-05-21T22:27:43.000Z","size":16,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-07-31T18:34:18.362Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AbsaOSS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-20T20:08:23.000Z","updated_at":"2023-05-21T22:05:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"50d97891-e165-43a1-bcd4-b6a72238acc0","html_url":"https://github.com/AbsaOSS/spark-launcher-supervisor","commit_stats":null,"previous_names":["absaoss/spark-launcher-supervisor"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/AbsaOSS/spark-launcher-supervisor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbsaOSS%2Fspark-launcher-supervisor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbsaOSS%2Fspark-launcher-supervisor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbsaOSS%2Fspark-launcher-supervisor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbsaOSS%2Fspark-launcher-supervisor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AbsaOSS","download_url":"https://codeload.github.com/AbsaOSS/spark-launcher-supervisor/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbsaOSS%2Fspark-launcher-supervisor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35174071,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-05T02:00:06.290Z","response_time":100,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T23:20:03.150Z","updated_at":"2026-07-06T00:31:40.243Z","avatar_url":"https://github.com/AbsaOSS.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# spark-launcher-supervisor\nA Java agent that oversees a Spark launcher and checks/enforces settings\n\nThis is a quick PoC of a JVM agent that is aimed to be globally installed on edge nodes\nto make sure Spark jobs are submitted with correct settings.\n\nThe current version of agent simply checks the following Spark settings and prints a warning message to the console if they don't match the expected values:\n1. `master` to be `yarn`\n2. `deploy-mode` to be `cluster`\n\n## Supported Spark version\nAny\n\n## Todo:\n- Flexible definition of rules/checks that can be either passed as an agent parameter, config file, or even managed centrally (remotely)\n- Send notifications\n- Option to enforce settings\n- Shade Byte-Buddy to avoid potential classpath conflicts\n- Build RPM\n- Improve logging\n\n## Building\n```shell\nmvn install\n```\n\n## Usage\n```shell\nexport JAVA_TOOL_OPTIONS=\"$JAVA_TOOL_OPTIONS -javaagent:agent/target/agent-0.1.0-SNAPSHOT.jar\"\n\n# run Spark as usual\nspark-submit ...\npyspark ...\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabsaoss%2Fspark-launcher-supervisor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabsaoss%2Fspark-launcher-supervisor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabsaoss%2Fspark-launcher-supervisor/lists"}