{"id":13527785,"url":"https://github.com/EclairJS/eclairjs-node","last_synced_at":"2025-04-01T10:30:28.316Z","repository":{"id":81168000,"uuid":"45638708","full_name":"EclairJS/eclairjs-node","owner":"EclairJS","description":"Node.js API for Apache Spark with Remote Client","archived":false,"fork":false,"pushed_at":"2016-10-12T15:51:26.000Z","size":69994,"stargazers_count":356,"open_issues_count":21,"forks_count":38,"subscribers_count":32,"default_branch":"master","last_synced_at":"2024-11-02T12:35:06.325Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EclairJS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-11-05T20:54:16.000Z","updated_at":"2024-10-16T19:10:09.000Z","dependencies_parsed_at":"2023-07-22T22:30:13.015Z","dependency_job_id":null,"html_url":"https://github.com/EclairJS/eclairjs-node","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EclairJS%2Feclairjs-node","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EclairJS%2Feclairjs-node/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EclairJS%2Feclairjs-node/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EclairJS%2Feclairjs-node/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EclairJS","download_url":"https://codeload.github.com/EclairJS/eclairjs-node/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246623702,"owners_count":20807391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T06:02:01.688Z","updated_at":"2025-04-01T10:30:27.540Z","avatar_url":"https://github.com/EclairJS.png","language":"JavaScript","readme":"The eclairjs-node and eclairjs-nashorn repositories have been merged into a new one called [eclairjs](https://github.com/EclairJS/eclairjs) and all future work is done there.\n\nEclairJS Node\n===================\nEclairJS Node provides Node.js language bindings for [Apache Spark](https://spark.apache.org).\n\nLearn more about the larger [EclairJS project](http://www.eclairjs.org).\n\n## Installation\n\n```bash\n$ npm install eclairjs\n```\n\nEclairJS Node requires Node 0.12 or higher and also requires a running instance of [EclairJS Nashorn](https://github.com/EclairJS/eclairjs-nashorn/).\n\nSupported Spark versions can be found in the [Versions](#version) section below.\n\n## Example\nEclairJS Node's api mirrors the Spark api.  Here is the classic word count example:\n\n```node\nvar eclairjs = require('eclairjs');\n\nvar spark = new eclairjs();\n\nvar sc = new spark.SparkContext(\"local[*]\", \"Simple Word Count\");\n\nvar textFile = sc.textFile('foo.txt');\n\nvar words = textFile.flatMap(function(sentence) {\n  return sentence.split(\" \");\n});\n\nvar wordsWithCount = words.mapToPair(function(word, Tuple2) {\n  return new Tuple2(word, 1);\n}, [eclairjs.Tuple2]);\n\nvar reducedWordsWithCount = wordsWithCount.reduceByKey(function(value1, value2) {\n  return value1 + value2;\n});\n\nreducedWordsWithCount.collect().then(function(results) {\n  console.log('Word Count:', results);\n  sc.stop();\n});\n```\n\n## Try It\nEclairJS Node provides a Docker container that contains all of its dependencies on [Dockerhub](https://hub.docker.com/r/eclairjs/minimal-gateway/).\n\nThe Docker image supports the latest released version of EclairJS Node and may not work with `master`.   You can simply check out the appropriate branch (` git checkout branch-0.7` for example).\n\n```bash\ndocker pull eclairjs/minimal-gateway\ndocker run -p 8888:8888 eclairjs/minimal-gateway\n```\n\nAfter retrieving Docker's IP address (`docker-machine ip`), you will need to set two environment variables:\n\n```bash\nexport JUPYTER_HOST=??.??.??.?? (your docker ip)\nexport JUPYTER_PORT=8888\n```\n\nNow you can run the Word count example:\n\n```bash\nnode --harmony examples/rddtop10.js ./dream.txt\n```\n\nYou can learn more about the Docker container [here](https://github.com/EclairJS/eclairjs-node/wikis/Using-the-Docker-Container).\nYou can also try out EclairJS in Jupyter notebooks running under the [IBM Bluemix Cloud](https://github.com/EclairJS/eclairjs-node/wikis/EclairJS-with-IBM-Bluemix).\n\n## Documentation\n* [Developing with EclairJS](https://github.com/EclairJS/eclairjs-node/wiki/Developing-With-EclairJS)\n* [API Docs](https://github.com/EclairJS/eclairjs-node/wiki/API-Documentation)\n* [Wiki](https://github.com/EclairJS/eclairjs-node/wiki)\n* [Presentations](https://github.com/EclairJS/eclairjs-node/wiki/Project-and-Community#presentations)\n* [API Examples](https://github.com/EclairJS/eclairjs-node/tree/master/examples)\n* [Example Applications](https://github.com/EclairJS/eclairjs-examples)\n\n## Community\n* [EclairJS Project](http://eclairjs.org/)\n* [Google Group](https://groups.google.com/forum/#!forum/eclairjs)\n* [Slack](https://eclairjs.slack.com)\n\n## Deploy\nYou can choose to either deploy using Docker ([Using the Docker Container](https://github.com/EclairJS/eclairjs-node/wikis/Using-the-Docker-Container)) \nor manually build and setup your own environment ([Build and Package](https://github.com/EclairJS/eclairjs-node/wikis/Build-and-Package)).\n\n## Progress\n\n|Spark Feature    |EclairJS Node Status|\n|-----------------|--------------------|\n|RDD              | Partial Support    |\n|SQL/DataFrames   | Partial Support    |\n|Streaming        | Partial Support    |\n|ml               | Partial Support    |\n|mllib            | Partial Support    |\n|GraphX           | Unsupported        |\n\nRefer to the [API Documentation](https://github.com/EclairJS/eclairjs-node/wikis/API-Documentation) for a list of what is currently implemented.  Please note as new APIs are implemented for EclairJS Node they will be added to the master branch.\n\nContributions are always welcome via pull requests.\n\n## Versions\nOur goal is to keep the EclairJS master branch up to date with the latest version of Spark. When new versions of Spark require code changes, we create a separate branch. The table below shows what is available now.\n\n|EclairJS Version/Tag | Apache Spark Version |\n| ---------------| ----- |\n| 0.1            | 1.5.1 |\n| 0.2 - 0.7      | 1.6.0 |\n| 0.8  (master)  | 2.0.0 |\n","funding_links":[],"categories":["JavaScript","Data Processing"],"sub_categories":["Torrent"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEclairJS%2Feclairjs-node","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FEclairJS%2Feclairjs-node","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEclairJS%2Feclairjs-node/lists"}