{"id":18289748,"url":"https://github.com/spark-jobserver/chef-spark-jobserver","last_synced_at":"2025-07-16T05:32:49.410Z","repository":{"id":20504379,"uuid":"23782911","full_name":"spark-jobserver/chef-spark-jobserver","owner":"spark-jobserver","description":"Chef recipe for Spark JobServer","archived":false,"fork":false,"pushed_at":"2014-10-13T17:29:59.000Z","size":248,"stargazers_count":6,"open_issues_count":1,"forks_count":6,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-05T09:34:49.456Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spark-jobserver.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-09-08T07:47:52.000Z","updated_at":"2021-11-22T06:14:36.000Z","dependencies_parsed_at":"2022-09-09T11:52:18.485Z","dependency_job_id":null,"html_url":"https://github.com/spark-jobserver/chef-spark-jobserver","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/spark-jobserver/chef-spark-jobserver","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spark-jobserver%2Fchef-spark-jobserver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spark-jobserver%2Fchef-spark-jobserver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spark-jobserver%2Fchef-spark-jobserver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spark-jobserver%2Fchef-spark-jobserver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spark-jobserver","download_url":"https://codeload.github.com/spark-jobserver/chef-spark-jobserver/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spark-jobserver%2Fchef-spark-jobserver/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265484796,"owners_count":23774398,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T14:07:56.965Z","updated_at":"2025-07-16T05:32:49.379Z","avatar_url":"https://github.com/spark-jobserver.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# spark-jobserver cookbook\n\nInstalls the [Spark Jobserver](../spark-jobserver/spark-jobserver).\n\nDev quickstart:\n\n```ruby\n# minimal required configuration; these values have no defaults and must be provided\nchef.json = {\n  spark: {\n    jobserver: {\n      # path to your spark install\n      spark_home: '/usr/local/spark',\n      spark_conf_dir: '/usr/local/spark/conf',\n      # you need to provide the jobserver jar; this gets downloaded, but can be a `file://` uri\n      jar_url: 'https://domain.com/some/path/spark-job-server.jar'\n    }\n  }\n}\n\ninclude_recipe 'spark-jobserver::default'\n```\n\nTo use the provided [Vagrantfile](Vagrantfile) to spin up a demo instance:\n\n1. Copy the Spark distribution binary into the repo root. By default this is\n   assumed to be `spark-1.1.0-bin-hadoop2.4.tgz`. See below if you wish to use\n   a different Spark distro.\n2. Copy the jobserver jar into the repo root. By default this is assumed to be\n   `spark-job-server.jar`. Building the jobserver itself is easy: `pushd\n   .../spark-jobserver; sbt clean assembly`.\n3. Spin up the Vagrant instance: `vagrant up`\n  - If you want to use a different Spark distro, set the `SPARK_DIST`\n    environment variable (_without_ the extension):\n    `SPARK_DIST=spark-1.1.0-bin-cdh4 vagrant up`\n4. Now you can browse the Jobserver UI at \u003chttp://33.33.33.123:8090\u003e and go\n   through the [jobserver\n   examples](https://github.com/spark-jobserver/spark-jobserver#wordcountexample-walk-through).\n\nNote that this repo assumes RVM; YMMV with other installations such as ChefDK.\nThe Vagrant demo requires following plugins:\n\n- [`vagrant-berkshelf`](https://github.com/berkshelf/vagrant-berkshelf)\n- [`vagrant-omnibus`](https://github.com/opscode/vagrant-omnibus)\n\n## Supported Platforms\n\nChefspec [tested on](spec/default_helper.rb):\n\n```ruby\n{\n  'ubuntu' =\u003e [ '12.04', '13.04', '13.10', '14.04' ],\n  'debian' =\u003e [ '6.0.5' ],\n  'centos' =\u003e [ '5.8', '6.0', '6.3' ],\n  'redhat' =\u003e [ '5.8', '6.3' ],\n}\n```\n\nConfirmed to work on:\n\n- Ubuntu 12.04\n- Ubuntu 14.04\n\nWith Chef 11.8+\n\n## Attributes\n\nSee [attributes](attributes/default.rb) for all options.\n\n## Usage\n\nThe only cookbook dependency is runit; java is assumed to be installed somehow.\n\nFollow the appropriate guides for Spark\n[standalone](http://spark.apache.org/docs/latest/spark-standalone.html) or\n[Mesos](http://spark.apache.org/docs/latest/running-on-mesos.html) deployment.\nThe relevant attributes are:\n\n- `node.spark.jobserver.master_type`\n- `node.spark.jobserver.master_url`\n\n### spark-jobserver::default\n\nInclude `spark-jobserver::default` in your node's `run_list`:\n\n```json\n{\n  \"run_list\": [\n    \"recipe[spark-jobserver::default]\"\n  ]\n}\n```\n\n## Contributing\n\n1. Fork the repository on Github\n2. Create a named feature branch (i.e. `add-new-recipe`)\n3. Write your change\n4. Write tests for your change (if applicable)\n  - At least a Chefspec proving the fix; Test Kitchen coming later\n5. Run the tests, ensuring they all pass\n6. Submit a Pull Request\n\n## License and Authors\n\nAuthor:: Matt Chu (matt.chu@gmail.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspark-jobserver%2Fchef-spark-jobserver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspark-jobserver%2Fchef-spark-jobserver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspark-jobserver%2Fchef-spark-jobserver/lists"}