{"id":18368681,"url":"https://github.com/radanalyticsio/oshinko-s2i","last_synced_at":"2026-03-12T12:40:39.445Z","repository":{"id":72027118,"uuid":"62732902","full_name":"radanalyticsio/oshinko-s2i","owner":"radanalyticsio","description":"This is a place to put s2i images and utilities for spark application builders for openshift","archived":false,"fork":false,"pushed_at":"2021-04-28T12:35:09.000Z","size":73127,"stargazers_count":15,"open_issues_count":24,"forks_count":27,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-03-22T04:02:05.478Z","etag":null,"topics":["java","openshift","oshinko-s2i","pyspark","s2i-image","scala","spark"],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radanalyticsio.png","metadata":{"files":{"readme":"README.md","changelog":"change-yaml.sh","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-07-06T15:39:43.000Z","updated_at":"2023-11-20T14:33:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"6c0f4dab-d396-4b8a-b587-1d7b41b6f339","html_url":"https://github.com/radanalyticsio/oshinko-s2i","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radanalyticsio%2Foshinko-s2i","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radanalyticsio%2Foshinko-s2i/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radanalyticsio%2Foshinko-s2i/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radanalyticsio%2Foshinko-s2i/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radanalyticsio","download_url":"https://codeload.github.com/radanalyticsio/oshinko-s2i/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247522407,"owners_count":20952542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["java","openshift","oshinko-s2i","pyspark","s2i-image","scala","spark"],"created_at":"2024-11-05T23:26:58.058Z","updated_at":"2026-03-12T12:40:39.412Z","avatar_url":"https://github.com/radanalyticsio.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/radanalyticsio/oshinko-s2i.svg?branch=master)](https://travis-ci.org/radanalyticsio/oshinko-s2i)\n[![Docker python build](https://img.shields.io/docker/automated/radanalyticsio/radanalytics-pyspark.svg)](https://hub.docker.com/r/radanalyticsio/radanalytics-pyspark)\n[![Docker java build](https://img.shields.io/docker/automated/radanalyticsio/radanalytics-java-spark.svg)](https://hub.docker.com/r/radanalyticsio/radanalytics-java-spark)\n[![Docker scala build](https://img.shields.io/docker/automated/radanalyticsio/radanalytics-scala-spark.svg)](https://hub.docker.com/r/radanalyticsio/radanalytics-scala-spark)\n\n# oshinko-s2i #\nThis is a place to put s2i images and utilities for Apache Spark application builders for OpenShift.\n\n## Complete versus incomplete images ##\n\nThere are two types of images that can be built from this repository, `complete` and `incomplete`\n\nComplete images include a pre-selected Apache Spark distribution that is installed when the\nimage is built.\n\nIncomplete images contain radanalytics.io tooling but do not include a Spark distribution. With these\nimages, users can perform s2i builds and produce images with Spark distributions of\ntheir choosing. See below for information on how to use the incomplete images.\n\n## Building the s2i images ##\n\nThe easiest way to build the s2i images is to use the makefiles provided:\n\n    # To build all images\n    $ make\n\n    # To build images individually\n    $ make -f Makefile.pyspark\n    $ make -f Makefile.java\n    $ make -f Makefile.scala\n    $ make -f Makefile.pyspark-inc\n    $ make -f Makefile.java-inc\n    $ make -f Makefile.scala-inc\n\nThe default repository for the image can be overridden with the `LOCAL_IMAGE` var:\n\n    $ LOCAL_IMAGE=myimage make -f Makefile.pyspark\n\n## Modifying dependencies in the image yaml files\n\nThe cekit tool generates the image context directories\nbased on the content of the image.*.yaml files. Please\ntry to use the same version of cekit as noted in the\ncurrent set of checked in Dockerfiles (currently 2.2.7)\n\nA script has been provided to make altering the image.*.yaml files\nsimpler. It handles modifying the specified versions of oshinko and Spark.\nRun this for more details\n\n    $ change-yaml.sh -h\n\n## Remaking image context directories when things change\n\nThe image context directories are generated with the cekit tool and contain\nthe artifacts needed to build the images. They are:\n\n    * pyspark-build\n    * java-build\n    * scala-build\n    * pyspark-build-inc\n    * java-build-inc\n    * scala-build-inc\n\nIf the yaml files used by cekit change (ie image.*.yaml) or the content\nincluded in an image changes (essentially anything under modules/), the\nimage context directories need to be rebuilt.\n\n### Rebuilding context directories for an upstream pull request\n\nIf the changes being made are part of a PR to github.com/radanalyticsio/oshinko-s2i\nthen all of the build directories should be generated from scratch.\nThe best way to do this is with the make-build-dirs.sh script\n\n    $ make-build-dirs.sh\n\nThis will recreate the context directories starting from a clean environment,\nmake sure any tarballs are truncated for github, and add all of the changes\nto the commit.\n\n### Rebuilding a particular context directory for testing/development\n\nIf the actual components specified in an image.*.yaml file have changed\nas opposed to only the _contents_ of existing modules, then the `target`\ndirectory should be cleaned before generating the context directory\n\n    $ make clean-target\n\nTo regenerate a particular context directory, like pyspark-build, do this\n\n    $ make -f Makefile.pyspark clean-context context\n\nTo regenerate the context directory and also build the image, do this\n\n    $ make -f Makefile.pyspark clean build\n\n## Using incomplete images to choose an Apache Spark distribution\n\n[This link](https://radanalytics.io/howdoi/choose-my-spark-distribution) explains how\nto use radanalytics.io with a specific Spark distribution. It's a must read if you\nare not familiar with the incomplete images and how to use them.\n\n### Getting `rad-image` with R support enabled\n\nThe version of `rad-image` from radanalytics.io has generation of the R image\ndisabled. Do this to download `rad-image` and enable R support\n\n    $ ./get-rad-image.sh\n\n### Modifying templates/* with `templates-is.sh`\n\nIn addtion to `resources.yaml` from radanalytics.io, the templates in this repository\ncan be used with the imagestreams created by `rad-image` with a few changes.\nUse the `templates-is.sh` script to generate modified templates in `templates-is/`\n\n    $ ./templates-is.sh\n\nRun `./templates-is.sh -h` for more information\n\n## Git pre-commit hook\n\nThe `hooks/pre-commit` hook can be installed in a local repo to\nprevent commits with non-zero length tarballs in the image build\ndirectories or to warn when changes have been made to yaml files or\nscripts but the image build directories have not changed.\nTo install the hook locally do something like this:\n\n    $ cd .git/hooks\n    $ ln -s ../../hooks/pre-commit pre-commit\n\nThis is recommended, since the CI tests will reject a pull request\nwith non-zero length tarballs anyway. Save some time, install the hook.\n\n## Using `release-templates.sh` ##\n\nThe templates included in this repository always reference the latest\n[s2i images](https://hub.docker.com/u/radanalyticsio/). Those images may\nchange during the normal course of development.\n\nThe `release-templates.sh` script can be used to create local versions of\nthe templates that reference s2i images from a particular oshinko release.\nYou may want to use this script to guarantee that you are using a stable image.\nFor example:\n\n    $ ./release-templates.sh v0.5.6\n\n    Successfully wrote templates to release_templates/ with version tag v0.5.6\n\n    grep radanalyticsio/radanalytics-.*:v0.5.6 *\n\n    release_templates/javabuilddc.json:                     \"name\": \"radanalyticsio/radanalytics-java-spark:v0.5.6\"\n    release_templates/javabuild.json:                     \"name\": \"radanalyticsio/radanalytics-java-spark:v0.5.6\"\n    release_templates/pythonbuilddc.json:                     \"name\": \"radanalyticsio/radanalytics-pyspark:v0.5.6\"\n    release_templates/pythonbuild.json:                     \"name\": \"radanalyticsio/radanalytics-pyspark:v0.5.6\"\n    release_templates/scalabuilddc.json:                     \"name\": \"radanalyticsio/radanalytics-scala-spark:v0.5.6\"\n    release_templates/scalabuild.json:                     \"name\": \"radanalyticsio/radanalytics-scala-spark:v0.5.6\"\n\n    tar -czf oshinko_s2i_v0.5.6.tar.gz release_templates\n\n    $ oc create -f release_templates/pythonbuilddc.json\n\n## MacOS Tips\n\nFor MacOS you will also need to download these tools: gsed and truncate.\nYou can install these using homebrew and these commands:\n\n```\nbrew install truncate\n```\n\n```\nbrew install gnu-sed\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradanalyticsio%2Foshinko-s2i","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradanalyticsio%2Foshinko-s2i","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradanalyticsio%2Foshinko-s2i/lists"}