{"id":29230510,"url":"https://github.com/underpostnet/spark-template.g8","last_synced_at":"2026-06-23T03:31:38.188Z","repository":{"id":301597358,"uuid":"1008812758","full_name":"underpostnet/spark-template.g8","owner":"underpostnet","description":"Spark template runner suite","archived":false,"fork":false,"pushed_at":"2025-11-04T20:32:13.000Z","size":93,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-11-04T22:19:21.352Z","etag":null,"topics":["gpu-acceleration","kubeflow","ml-ops","spark"],"latest_commit_sha":null,"homepage":"https://www.nexodev.org","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/underpostnet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-26T06:15:10.000Z","updated_at":"2025-11-04T20:32:16.000Z","dependencies_parsed_at":"2025-07-01T13:47:50.950Z","dependency_job_id":null,"html_url":"https://github.com/underpostnet/spark-template.g8","commit_stats":null,"previous_names":["underpostnet/spark-template","underpostnet/spark-template.g8"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/underpostnet/spark-template.g8","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underpostnet%2Fspark-template.g8","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underpostnet%2Fspark-template.g8/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underpostnet%2Fspark-template.g8/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underpostnet%2Fspark-template.g8/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/underpostnet","download_url":"https://codeload.github.com/underpostnet/spark-template.g8/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/underpostnet%2Fspark-template.g8/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34674702,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-23T02:00:07.161Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gpu-acceleration","kubeflow","ml-ops","spark"],"created_at":"2025-07-03T14:00:50.639Z","updated_at":"2026-06-23T03:31:38.179Z","avatar_url":"https://github.com/underpostnet.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Spark on Kubernetes Template\n\n\u003cimg alt='apache_spark' src='https://img.shields.io/badge/apache_spark%20v3.5.5-100000?style=flat\u0026logo=apache-spark\u0026logoColor=FFFFFF\u0026labelColor=C76E00\u0026color=727273'/\u003e \u003cimg alt='scala' src='https://img.shields.io/badge/scala%20v2.12-100000?style=flat\u0026logo=scala\u0026logoColor=FFFFFF\u0026labelColor=8b0000\u0026color=727273'\u003e\n\n[![Spark CI](https://github.com/underpostnet/spark-template-demo/actions/workflows/docker-image.yml/badge.svg)](https://github.com/underpostnet/spark-template-demo/actions/workflows/docker-image.yml)\n\nThis project provides template for building, testing, and deploying Scala Spark applications on Kubernetes using the Spark Operator.\n\n### Core Features\n\n- **Scala \u0026 sbt**: Leverages `sbt` for a standard Scala project structure, dependency management, and build automation.\n- **Containerized Workflows**: A multi-stage `Dockerfile` ensures a lean and optimized Docker image for your Spark application, based on official Apache Spark images.\n- **Kubernetes-Native Deployment**: Designed for seamless deployment and management via the Spark on Kubernetes Operator, utilizing `SparkApplication` custom resources.\n- **Integrated Testing**:\n  - Supports unit and integration testing with `scalatest` and `spark-fast-tests`.\n  - Includes a `TestRunner` that enables **in-cluster testing**, allowing your test suites to execute directly on the Spark cluster in an environment identical to production.\n- **GPU Acceleration Ready**: Pre-configured with RAPIDS Accelerator for Apache Spark, enabling GPU-accelerated Spark SQL and DataFrame operations.\n- **Pre-configured RBAC**: Includes necessary Kubernetes Role-Based Access Control (`spark-rbac.yaml`) to grant the Spark driver the permissions required to create and manage its executor pods.\n- **Ephemeral Storage Configuration**: Demonstrates how to configure ephemeral storage requests and limits for Spark driver and executor pods.\n- **ConfigMap Integration**: Utilizes Kubernetes ConfigMaps (`spark-driver-pod-config`, `spark-executor-pod-config`) to inject pod spec fragments, allowing for flexible and advanced pod configurations.\n\n### How to Use This Template\n\nThis project is designed as a [Giter8 template](https://www.foundweekends.org/giter8/index.html), making it easy to generate new Spark projects with all the pre-configured settings.\n\n#### 1. Install sbt\n\nIf you don't have sbt installed, follow the instructions on the [official sbt website](https://www.scala-sbt.org/download.html). Ensure you have sbt launcher version 0.13.13 or above.\n\n#### 2. Create a New Project from the Template\n\nOpen your terminal and run the following command.\n\n```bash\nsbt new underpostnet/spark-template.g8\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funderpostnet%2Fspark-template.g8","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funderpostnet%2Fspark-template.g8","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funderpostnet%2Fspark-template.g8/lists"}