{"id":21458534,"url":"https://github.com/hibayesian/spark-fm","last_synced_at":"2025-10-16T03:21:15.496Z","repository":{"id":85150688,"uuid":"91775658","full_name":"hibayesian/spark-fm","owner":"hibayesian","description":"A parallel implementation of factorization machines based on Spark","archived":false,"fork":false,"pushed_at":"2020-06-28T08:01:12.000Z","size":444,"stargazers_count":74,"open_issues_count":4,"forks_count":31,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-02T13:49:49.211Z","etag":null,"topics":["factorization-machines","machine-learning","spark"],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hibayesian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-19T06:59:17.000Z","updated_at":"2025-04-24T12:36:19.000Z","dependencies_parsed_at":"2023-03-04T20:15:45.779Z","dependency_job_id":null,"html_url":"https://github.com/hibayesian/spark-fm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hibayesian/spark-fm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hibayesian%2Fspark-fm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hibayesian%2Fspark-fm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hibayesian%2Fspark-fm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hibayesian%2Fspark-fm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hibayesian","download_url":"https://codeload.github.com/hibayesian/spark-fm/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hibayesian%2Fspark-fm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265385758,"owners_count":23756728,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["factorization-machines","machine-learning","spark"],"created_at":"2024-11-23T06:23:09.171Z","updated_at":"2025-10-16T03:21:10.463Z","avatar_url":"https://github.com/hibayesian.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Spark-FM\nFactorization Machines is a general predictor like SVMs but is also able to estimate reliable parameters under very high sparsity. However, they are costly to scale to large amounts of data and large numbers of features. Spark-FM is a parallel implementation of factorization machines based on Spark. It aims to utilize Spark's in-memory computing to address above problems.\n\n# Highlight\nIn order to meet users' demands, Spark-FM supports various of optimization methods to train the model as follows.\n + Mini-batch Stochastic Gradient Descent (MLlib)\n + L-BFGS (MLlib)\n + Parallel Stochastic Gradient Descent ([spark-optim](https://github.com/hibayesian/spark-optim))\n + Parallel Ftrl ([spark-optim](https://github.com/hibayesian/spark-optim))\n\n\n# Examples\n## Scala API\n```scala\nval spark = SparkSession\n  .builder()\n  .appName(\"FactorizationMachinesExample\")\n  .master(\"local[*]\")\n  .getOrCreate()\n\nval train = spark.read.format(\"libsvm\").load(\"data/a9a.tr\")\nval test = spark.read.format(\"libsvm\").load(\"data/a9a.te\")\n\nval trainer = new FactorizationMachines()\n  .setAlgo(Algo.fromString(\"binary classification\"))\n  .setSolver(Solver.fromString(\"pftrl\"))\n  .setDim((1, 1, 8))\n  .setReParamsL1((0.1, 0.1, 0.1))\n  .setRegParamsL2((0.01, 0.01, 0.01))\n  .setAlpha((0.1, 0.1, 0.1))\n  .setBeta((1.0, 1.0, 1.0))\n  .setInitStdev(0.01)\n  // .setStepSize(0.1)\n  .setTol(0.001)\n  .setMaxIter(1)\n  .setThreshold(0.5)\n  // .setMiniBatchFraction(0.5)\n  .setNumPartitions(4)\n\nval model = trainer.fit(train)\nval result = model.transform(test)\nval predictionAndLabel = result.select(\"prediction\", \"label\")\nval evaluator = new MulticlassClassificationEvaluator().setMetricName(\"accuracy\")\nprintln(\"Accuracy: \" + evaluator.evaluate(predictionAndLabel))\nspark.stop()\n```\n\n# Requirements\nSpark-FM is built against Spark 2.1.1.\n\n# Build From Source\n```scala\nsbt package\n```\n\n# Licenses\nSpark-FM is available under Apache Licenses 2.0.\n\n# Contact \u0026 Feedback\nIf you encounter bugs, feel free to submit an issue or pull request. Also you can mail to:\n+ hibayesian (hibayesian@gmail.com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhibayesian%2Fspark-fm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhibayesian%2Fspark-fm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhibayesian%2Fspark-fm/lists"}