{"id":18300478,"url":"https://github.com/comet-ml/scala-example","last_synced_at":"2025-04-09T09:29:40.305Z","repository":{"id":40331336,"uuid":"399949783","full_name":"comet-ml/scala-example","owner":"comet-ml","description":null,"archived":false,"fork":false,"pushed_at":"2022-05-16T11:01:45.000Z","size":34,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-03T02:54:46.632Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/comet-ml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-25T20:31:20.000Z","updated_at":"2021-10-22T13:32:19.000Z","dependencies_parsed_at":"2022-08-09T17:30:50.345Z","dependency_job_id":null,"html_url":"https://github.com/comet-ml/scala-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fscala-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fscala-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fscala-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comet-ml%2Fscala-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/comet-ml","download_url":"https://codeload.github.com/comet-ml/scala-example/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248011995,"owners_count":21033112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T15:12:35.032Z","updated_at":"2025-04-09T09:29:40.282Z","avatar_url":"https://github.com/comet-ml.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Comet Java SDK Scala Example\nCollection of examples demonstrating how to use [Comet Java SDK](https://github.com/comet-ml/comet-java-sdk) \nwith [Apache Spark](https://spark.apache.org) using [Deeplearning4j on Spark](https://deeplearning4j.konduit.ai/spark/tutorials/dl4j-on-spark-quickstart)\nand [Scala](https://scala-lang.org) programming language.\n\n# System Requirements\n\nThe Scala experiments defined in this repository use **Scala 2.11.12** and intended to be run\nusing **Java 8**.\n\nYou need to have **Java 8** installed and configured as default on your system in order to\nrun Scala examples from the command line as described later. If you are using IDE, make \nsure to select Java 8 SDK for building and running.\n\nIf you have higher Java version in your system, please make sure to change Scala version\ndefined in the project's [POM](./pom.xml) to meet your JDK version. For version compatibility please\nrefer to [Scala version compatibility table](https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html).\n\n# Configuration and Setup\n\nPlease make sure that you have account at [Comet.ml](https://www.comet.ml) and got your Comet API key.\nFor more details please refer to [Quick Start Guide](https://www.comet.ml/docs/quick-start/).\n\n## Providing configuration for Comet experiment\n\nThere are a few ways to properly configure your Comet experiment:\n\n* Using environment variables\n* Using configuration file\n* Using Experiment builder API\n\nNext, we describe all of them.\n\n### Environment variables\n\nThe configuration can be done using the following environment variables that you should set\nbefore running the Comet experiment:\n\n* **COMET_API_KEY** - the Comet API key you got from your Comet cabinet\n* **COMET_WORKSPACE_NAME** - the name of workspace where your experiments will be saved at Comet.ml\n* **COMET_PROJECT_NAME** - the name of project within workspace for your experiments\n\nYou can set mentioned environment variables via command line when running experiment. \nWe discuss this method later.\n\n### Configuration file\n\nAdditionally, you can use experiment configuration file to provide necessary configuration options.\nThis can be useful if you plan to run your experiment multiple times. The configuration file you can\nfind at [application.conf](./src/main/resources/application.conf).\n\nIt has the following records which you should update by uncommenting and providing valid values:\n\n```text\ncomet {\n    # baseUrl = \"https://www.comet.ml\"\n    # apiKey = \"XXXX\"\n    # project = \"java-sdk\"\n    # workspace = \"my-team\"\n}\n```\n\nFor extra options please refer to [CometConfig.java](https://github.com/comet-ml/comet-java-sdk/blob/master/comet-java-client/src/main/java/ml/comet/experiment/impl/config/CometConfig.java)\n\n### Using Experiment builder API\n\nAlso, it is possible to configure Comet Experiment using experiment builder API.\nThe following code snippet demonstrate how to do this:\n\n```text\nval experiment = ExperimentBuilder.OnlineExperiment\n      .interceptStdout\n      .withWorkspace(\"my-team\")\n      .withProjectName(\"java-sdk\")\n      .withApiKey(\"XXXX\")\n      .build\n```\n\nor \n\n```text\nval experiment = ExperimentBuilder.OnlineExperiment\n      .interceptStdout\n      .withApiKey(\"XXXX\").asInstanceOf[OnlineExperimentBuilder] // need to have explicit type casting here\n      .withWorkspace(\"my-team\")\n      .withProjectName(\"java-sdk\")\n      .build\n```\n\nFor extra configuration options please refer to [OnlineExperimentBuilder](https://github.com/comet-ml/comet-java-sdk/blob/master/comet-java-client/src/main/java/ml/comet/experiment/builder/OnlineExperimentBuilder.java).\n\n# Running Comet Experiment example\n\nWe provide two examples of running the classic MNIST experiment:\n\n* Using Deeplearning4j on Spark and Scala\n* Using Deeplearning4j and Java\n\nNext, we discuss how to run each of them.\n\n## Scala with Deeplearning4j on Spark\n\nThe source code of this experiment available at [MNistExampleSpark](./src/main/scala/ml/comet/examples/MNistExampleSpark.scala)\n\nYou can run the experiment with the following command using your favourite terminal application.\n\n```text\nCOMET_API_KEY=your_api_key \\\nCOMET_WORKSPACE_NAME=your_workspace \\\nCOMET_PROJECT_NAME=your_project_name \\\nmvn package exec:java -Dexec.mainClass=\"ml.comet.examples.MNistExampleSpark\" -Dexec.args=\"--numEpochs 2\"\n```\n\nWhere:\n* **numEpochs** - allows you to define number of training epochs [**default: 10**]\n* **useSparkLocal** - allows to use spark local (helper for testing/running without spark submit) [**default: true**]\n\nMake sure to provide correct values of the environment variables.\n\n## Java with Deeplearning4j\n\nAs reference, we also provide Java implementation of the MNIST experiment with source code available \nat [MNistExampleJava](./src/main/scala/ml/comet/examples/MNistExampleJava.java)\n\nYou can run the experiment with the following command.\n\n```text\nCOMET_API_KEY=your_api_key \\\nCOMET_WORKSPACE_NAME=your_workspace \\\nCOMET_PROJECT_NAME=your_project_name \\\nmvn exec:java -Dexec.mainClass=\"ml.comet.examples.MNistExampleJava\" -Dexec.args=\"--epochs 2\"\n```\n\nWhere:\n* **epochs** - allows you to define the number of training epochs [default: 10]\n\nMake sure to provide correct values of the environment variables.\n\n# References\n\n1. [Comet Java SDK](https://github.com/comet-ml/comet-java-sdk)\n2. [Deeplearning4j on Spark](https://deeplearning4j.konduit.ai/spark/tutorials/dl4j-on-spark-quickstart)\n3. [Apache Spark](https://spark.apache.org)\n4. [Comet Quick Start Guide](https://www.comet.ml/docs/quick-start/)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomet-ml%2Fscala-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcomet-ml%2Fscala-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomet-ml%2Fscala-example/lists"}