{"id":17047798,"url":"https://github.com/residentmario/comet-test","last_synced_at":"2025-03-23T03:44:54.179Z","repository":{"id":66134300,"uuid":"234671841","full_name":"ResidentMario/comet-test","owner":"ResidentMario","description":null,"archived":false,"fork":false,"pushed_at":"2020-01-18T02:44:47.000Z","size":1,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-28T10:26:13.255Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ResidentMario.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-18T02:44:03.000Z","updated_at":"2020-01-18T02:44:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"b524754b-9f07-4aff-9f92-54f7324fed25","html_url":"https://github.com/ResidentMario/comet-test","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fcomet-test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fcomet-test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fcomet-test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fcomet-test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ResidentMario","download_url":"https://codeload.github.com/ResidentMario/comet-test/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245052639,"owners_count":20553163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T09:50:21.182Z","updated_at":"2025-03-23T03:44:54.159Z","avatar_url":"https://github.com/ResidentMario.png","language":null,"readme":"# comet-test\n\nThis repo tests out some CometML features.\n\n## Experiments\n\n* The unit of value in CometML is an **experiment**. An experiment is a single algorithm run.\n* Strictly speaking there are four experiment classes in the Python SDK: `Experiment`, `ExistingExperiment` (for connecting to, you know), `OfflineExperiment` (which logs to a file instead of streaming to the Internet), and `APIExperiment` (used for querying an existing experiment instead of streaming to/from it).\n* Experiments may be symlinked to multiple projects via `create_symlink`.\n* You can IFrame an experiment in a notebook using `display`.\n* Metrics are logged via monkey patching the requisite libraries. A `disable_mp` method is provided for disabling this behavior, which is recommended for non-trivial use, as the monkey patching is a go-to-market strategy.\n* It's good practice to `end` an experiment at completion.\n* Experiment logs are available via various `get_*` and `log_*` methods. There are a lot of different loggable things, so much so in fact that the UI is very confusing.\n* `send_notification` is interesting, it's a hook to an email notification system built into Comet.\n* The UI has a reproduce button, which points to some logabble values. In the case of a run this is Environment Information, Git Information, and Run Command. In the case of a notebook the Git Information and Run Command options seem to be replaced with Notebook Information, which is an IPYNB download. Overall this feature is not that useful in my opinion, a README does much better for this function.\n* Offline experiments are uploaded to the cloud using a Python lib utility script or a CLI wrapper thereof.\n* The notebook write-out is kind of janky, albeit for the right reason. The cells are automatically flattened out into a script. Any cells that get run more than once will be included more than once. This output format is reproducible, however, which is why they do it this way.\n\n## Experiment UI\n\n* The experiment web UI is only controllable in the browser, e.g. it's not possible to adjust it using libraries.\n* Experiments are bucketed into projects. Experiments can appear in more than one project via symlinks (see previous section).\n* Experiments in a project are subselected using views. Views are combinations of query builder that lets you subselect experiments using tag boolean logic, and a panel of model metric visualizations and assets that you headline the page with.\n* The UI is the works, there's a ton you can do with it. The charting can be custom-emplaced, experiments can be archived, the table components are resizable...\n\n## Hyperparameter search\n\n* The optimizer is configured using a JSON fragment with a well-defined schema. The fragment basically states what the goals are, e.g. minimize this loss using these parameters.\n* The resulting `Optimizer` object acts as a source for model parameters and (passthrough to the web UI) sink for model losses.\n* If used via the CLI, the optimizer supports threads via the `-j` parameter.\n* Search is supported via grid, random (preferable to grid, see my notes), and Bayesian search (via a specific algorithm).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresidentmario%2Fcomet-test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fresidentmario%2Fcomet-test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresidentmario%2Fcomet-test/lists"}