{"id":15352224,"url":"https://github.com/xd-deng/spark-ml-intro","last_synced_at":"2025-04-15T05:51:52.823Z","repository":{"id":45575441,"uuid":"48379357","full_name":"XD-DENG/Spark-ML-Intro","owner":"XD-DENG","description":"PySpark Machine Learning Examples","archived":false,"fork":false,"pushed_at":"2018-03-08T09:21:56.000Z","size":138,"stargazers_count":45,"open_issues_count":0,"forks_count":29,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-15T05:51:47.562Z","etag":null,"topics":["machine-learning","spark"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/XD-DENG.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-12-21T15:32:57.000Z","updated_at":"2025-03-08T05:51:30.000Z","dependencies_parsed_at":"2022-08-04T16:00:20.252Z","dependency_job_id":null,"html_url":"https://github.com/XD-DENG/Spark-ML-Intro","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XD-DENG%2FSpark-ML-Intro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XD-DENG%2FSpark-ML-Intro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XD-DENG%2FSpark-ML-Intro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XD-DENG%2FSpark-ML-Intro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/XD-DENG","download_url":"https://codeload.github.com/XD-DENG/Spark-ML-Intro/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249016317,"owners_count":21198832,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","spark"],"created_at":"2024-10-01T12:09:02.197Z","updated_at":"2025-04-15T05:51:52.801Z","avatar_url":"https://github.com/XD-DENG.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Spark Machine Learning Introduction\n\n**NOTE: the methods introduced here are all based on RDD-based API. As of Spark 2.0, the RDD-based APIs in the `spark.mllib` package have entered maintenance mode. The primary Machine Learning API for Spark is now the DataFrame-based API in the `spark.ml` package. I would strongly suggest NOT use this repo for your learning anymore (please refer to https://spark.apache.org/docs/2.1.0/ml-guide.html).**\n\nIn this repo, I try to introduce some basic machine learning usages of *PySpark*. The contents I'm going to cover would be quite simple. But I guess it would be helpful for some people since I would cover some questions I encountered myself from the perspective of a person who's used to more \"normal\" ML settings (like R language). \n\nFor the basic PySpark operations (Tranformations and Actions), you may refer to my another GitHub repo, [Spark Practice](https://github.com/XD-DENG/Spark-practice).\n\nSome of the examples are from the official examples given by Spark. But I will give more details.\n\n- [Random Forest](https://github.com/XD-DENG/Spark-ML-Intro/tree/master/chapters/random_forest.md)\n- [Regression](https://github.com/XD-DENG/Spark-ML-Intro/tree/master/chapters/regression.md)\n- [K-means](https://github.com/XD-DENG/Spark-ML-Intro/tree/master/chapters/k_means.md)\n- [References](https://github.com/XD-DENG/Spark-ML-Intro/tree/master/chapters/references.md)\n\n## License\nPlease note this repostory is under the Creative Commons Attribution-ShareAlike License[https://creativecommons.org/licenses/by-sa/3.0/].\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxd-deng%2Fspark-ml-intro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxd-deng%2Fspark-ml-intro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxd-deng%2Fspark-ml-intro/lists"}