{"id":20380157,"url":"https://github.com/ammahmoudi/mapreduce-examples","last_synced_at":"2026-04-16T17:02:01.828Z","repository":{"id":145078474,"uuid":"510439369","full_name":"ammahmoudi/MapReduce-examples","owner":"ammahmoudi","description":"Map Rduce Examples  using pure Scala and Then using Spark","archived":false,"fork":false,"pushed_at":"2023-11-03T09:46:10.000Z","size":203,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-15T07:31:25.833Z","etag":null,"topics":["map-reduce","mapreduce","scala","spark","spark-mapreduce"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ammahmoudi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-07-04T16:59:16.000Z","updated_at":"2023-11-03T08:47:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"b9a93d0a-e20b-46a4-b11e-883e53b31666","html_url":"https://github.com/ammahmoudi/MapReduce-examples","commit_stats":null,"previous_names":["ammahmoudi/mapreduce-examples"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ammahmoudi%2FMapReduce-examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ammahmoudi%2FMapReduce-examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ammahmoudi%2FMapReduce-examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ammahmoudi%2FMapReduce-examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ammahmoudi","download_url":"https://codeload.github.com/ammahmoudi/MapReduce-examples/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241929594,"owners_count":20044004,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["map-reduce","mapreduce","scala","spark","spark-mapreduce"],"created_at":"2024-11-15T02:06:17.967Z","updated_at":"2026-04-16T17:02:01.732Z","avatar_url":"https://github.com/ammahmoudi.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MapReduce-examples\nMapReduce Examples using pure Scala and Then using Spark.\n\n![spark scala](/cover.jpg)\n\nThe repository has two folders: scala and spark. The scala folder contains pure Scala implementations of some common MapReduce algorithms, such as word count, inverted index, and matrix multiplication. The spark folder contains Spark implementations of the same algorithms, using Spark's built-in APIs, such as RDDs, DataFrames, and Datasets.\n\nTo run the examples, you need to have Scala and Spark installed on your machine. You also need to have SBT, a build tool for Scala projects. You can use the following commands to compile and run the examples:\n\n- To compile the Scala examples, go to the scala folder and run `sbt compile`.\n- To run the Scala examples, go to the scala folder and run `sbt run`. You will see a list of options to choose which example to run. Enter the number of the example and press enter.\n- To compile the Spark examples, go to the spark folder and run `sbt package`. This will create a JAR file in the target folder.\n- To run the Spark examples, go to the spark folder and run `spark-submit --class \u003cclass name\u003e target/scala-2.12/mapreduce-examples_2.12-0.1.jar \u003cinput file\u003e \u003coutput folder\u003e`. Replace `\u003cclass name\u003e` with the name of the example class, such as `WordCount` or `InvertedIndex`. Replace `\u003cinput file\u003e` with the path to the input file, such as `data/words.txt` or `data/documents.txt`. Replace `\u003coutput folder\u003e` with the path to the output folder, such as `output/wordcount` or `output/invertedindex`.\n\nThe input and output files are in the data and output folders, respectively. You can also use your own files as input and output. The output files are in text format, with each line containing a key-value pair separated by a tab. You can view the output files using any text editor or command-line tool, such as `cat` or `less`.\n\nThe repository also contains some test cases for the examples, written using ScalaTest, a testing framework for Scala. You can run the test cases using `sbt test` in the scala or spark folder. The test cases will compare the output files with the expected files in the test folder and report any differences or errors.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fammahmoudi%2Fmapreduce-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fammahmoudi%2Fmapreduce-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fammahmoudi%2Fmapreduce-examples/lists"}