{"id":13567240,"url":"https://github.com/clearml/clearml-server","last_synced_at":"2025-06-17T04:09:11.944Z","repository":{"id":39593479,"uuid":"191134809","full_name":"clearml/clearml-server","owner":"clearml","description":"ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling \u0026 Serving in one MLOps/LLMOps solution","archived":false,"fork":false,"pushed_at":"2025-06-04T11:47:17.000Z","size":2178,"stargazers_count":415,"open_issues_count":112,"forks_count":145,"subscribers_count":29,"default_branch":"master","last_synced_at":"2025-06-11T19:36:53.296Z","etag":null,"topics":["ai","clearml-agent","control","deep-learning","deeplearning","experiment","experiment-manager","k8s","kubernetes","machine-learning","machinelearning","trains-agent","version","version-control"],"latest_commit_sha":null,"homepage":"https://clear.ml/docs","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clearml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-10T09:11:19.000Z","updated_at":"2025-06-05T10:42:20.000Z","dependencies_parsed_at":"2024-07-07T11:43:32.313Z","dependency_job_id":"c01da1cc-266c-4e25-81cd-9bb79cb940e3","html_url":"https://github.com/clearml/clearml-server","commit_stats":null,"previous_names":["clearml/clearml-server","allegroai/clearml-server"],"tags_count":46,"template":false,"template_full_name":null,"purl":"pkg:github/clearml/clearml-server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clearml%2Fclearml-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clearml%2Fclearml-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clearml%2Fclearml-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clearml%2Fclearml-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clearml","download_url":"https://codeload.github.com/clearml/clearml-server/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clearml%2Fclearml-server/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260288443,"owners_count":22986668,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","clearml-agent","control","deep-learning","deeplearning","experiment","experiment-manager","k8s","kubernetes","machine-learning","machinelearning","trains-agent","version","version-control"],"created_at":"2024-08-01T13:02:26.558Z","updated_at":"2025-06-17T04:09:11.896Z","avatar_url":"https://github.com/clearml.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"docs/clearml_server_logo.png\" width=\"250px\"\u003e\n\n**ClearML - Auto-Magical Suite of tools to streamline your ML workflow \n\u003c/br\u003eExperiment Manager, ML-Ops and Data-Management**\n\n[![GitHub license](https://img.shields.io/badge/license-SSPL-green.svg)](https://img.shields.io/badge/license-SSPL-green.svg)\n[![Python versions](https://img.shields.io/badge/python-3.9-blue.svg)](https://img.shields.io/badge/python-3.9-blue.svg)\n[![GitHub version](https://img.shields.io/github/release-pre/clearml/clearml-server.svg)](https://img.shields.io/github/release-pre/clearml/clearml-server.svg)\n[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/clearml)](https://artifacthub.io/packages/search?repo=clearml)\n\n\u003c/div\u003e\n\n## ClearML Server\n#### *Formerly known as Trains Server*\n\nThe **ClearML Server** is the backend service infrastructure for [ClearML](https://github.com/clearml/clearml).\nIt allows multiple users to collaborate and manage their experiments.\n**ClearML** offers a [free hosted service](https://app.clear.ml/), which is maintained by **ClearML** and open to anyone.\nIn order to host your own server, you will need to launch the **ClearML Server** and point **ClearML** to it.\n\nThe **ClearML Server** contains the following components:\n\n* The **ClearML** Web-App, a single-page UI for experiment management and browsing\n* RESTful API for:\n    * Documenting and logging experiment information, statistics and results\n    * Querying experiments history, logs and results\n* Locally-hosted file server for storing images and models making them easily accessible using the Web-App\n\nYou can quickly [deploy](#launching-the-clearml-server)  your **ClearML Server** using Docker, AWS EC2 AMI, or Kubernetes. \n\n## System design\n\n\n![Alt Text](docs/ClearML_Server_Diagram.png)\n\nThe **ClearML Server** has two supported configurations:\n- Single IP (domain) with the following open ports\n    - Web application on port 8080\n    - API service on port 8008\n    - File storage service on port 8081\n\n- Sub-Domain configuration with default http/s ports (80 or 443)\n    - Web application on sub-domain: app.\\*.\\*\n    - API service on sub-domain: api.\\*.\\*\n    - File storage service on sub-domain: files.\\*.\\*\n    \n## Launching The ClearML Server\n\n### Prerequisites\n\nThe ports 8080/8081/8008 must be available for the **ClearML Server** services.\n   \nFor example, to see if port `8080` is in use:\n\n* Linux or macOS: \n   \n        sudo lsof -Pn -i4 | grep :8080 | grep LISTEN\n\n* Windows:\n\n        netstat -an |find /i \"8080\"\n   \n### Launching   \n    \nLaunch The **ClearML Server** in any of the following formats:\n\n- Pre-built [AWS EC2 AMI](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_aws_ec2_ami)\n- Pre-built [GCP Custom Image](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_gcp)\n- Pre-built Docker Image\n    - [Linux](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_linux_mac)\n    - [macOS](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_linux_mac)\n    - [Windows 10](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_win)\n- Kubernetes    \n    - [Kubernetes Helm](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_kubernetes_helm)\n    - Manual [Kubernetes installation](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_kubernetes)\n\n## Connecting ClearML to your ClearML Server\n\nIn order to set up the **ClearML** client to work with your **ClearML Server**:\n- Run the `clearml-init` command for an interactive setup.\n- Or manually edit `~/clearml.conf` file, making sure the server settings (`api_server`, `web_server`, `file_server`) are configured correctly, for example:\n\n        api {\n            # API server on port 8008\n            api_server: \"http://localhost:8008\"\n\n            # web_server on port 8080\n            web_server: \"http://localhost:8080\"\n\n            # file server on port 8081\n            files_server: \"http://localhost:8081\"\n        }\n\n**Note**: If you have set up your **ClearML Server** in a sub-domain configuration, then there is no need to specify a port number,\nit will be inferred from the http/s scheme.\n\nAfter launching the **ClearML Server** and configuring the **ClearML** client to use the **ClearML Server**,\nyou can [use](https://github.com/clearml/clearml) **ClearML** in your experiments and view them in your **ClearML Server** web server,\nfor example http://localhost:8080.  \nFor more information about the ClearML client, see [**ClearML**](https://github.com/clearml/clearml).\n\n## ClearML-Agent Services  \u003ca name=\"services\"\u003e\u003c/a\u003e \n\nAs of version 0.15 of **ClearML Server**, dockerized deployment includes a **ClearML-Agent Services** container running as \npart of the docker container collection.\n\nClearML-Agent Services is an extension of ClearML-Agent that provides the ability to launch long-lasting jobs \nthat previously had to be executed on local / dedicated machines. It allows a single agent to \nlaunch multiple dockers (Tasks) for different use cases. To name a few use cases, auto-scaler service (spinning instances \nwhen the need arises and the budget allows), Controllers (Implementing pipelines and more sophisticated DevOps logic),\nOptimizer (such as Hyper-parameter Optimization or sweeping), and Application (such as interactive Bokeh apps for \nincreased data transparency)\n\nClearML-Agent Services container will spin **any** task enqueued into the dedicated `services` queue. \nEvery task launched by ClearML-Agent Services  will be registered as a new node in the system, \nproviding tracking and transparency capabilities.  \nYou can also run the ClearML-Agent Services manually, see details in [ClearML-agent services mode](https://github.com/clearml/clearml-agent#clearml-agent-services-mode-)\n\n**Note**: It is the user's responsibility to make sure the proper tasks are pushed into the `services` queue. \nDo not enqueue training / inference tasks into the `services` queue, as it will put unnecessary load on the server.\n\n## Advanced Functionality\n\nThe **ClearML Server** provides a few additional useful features, which can be manually enabled:\n \n* [Web login authentication](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config#web-login-authentication)\n* [Non-responsive experiments watchdog](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config#non-responsive-task-watchdog)  \n\n## Restarting ClearML Server\n\nTo restart the **ClearML Server**, you must first stop the containers, and then restart them.\n\n   ```bash\n   docker-compose down\n   docker-compose -f docker-compose.yml up\n   ```\n\n## Upgrading \u003ca name=\"upgrade\"\u003e\u003c/a\u003e\n\n**ClearML Server** releases are also reflected in the [docker compose configuration file](https://github.com/clearml/clearml-server/blob/master/docker/docker-compose.yml).  \nWe strongly encourage you to keep your **ClearML Server** up to date, by keeping up with the current release.\n\n**Note**: The following upgrade instructions use the Linux OS as an example.\n\nTo upgrade your existing **ClearML Server** deployment:\n\n1. Shut down the docker containers\n   ```bash\n   docker-compose down\n   ```\n\n1. We highly recommend backing up your data directory before upgrading.\n\n   Assuming your data directory is `/opt/clearml`, to archive all data into `~/clearml_backup.tgz` execute:\n\n   ```bash\n   sudo tar czvf ~/clearml_backup.tgz /opt/clearml/data\n   ```    \n\n   \u003cdetails\u003e\n   \u003csummary\u003eRestore instructions:\u003c/summary\u003e\n\n   To restore this example backup, execute:\n   ```bash\n   sudo rm -R /opt/clearml/data\n   sudo tar -xzf ~/clearml_backup.tgz -C /opt/clearml/data\n   ```\n   \u003c/details\u003e\n\n1. Download the latest `docker-compose.yml` file.\n\n   ```bash\n   curl https://raw.githubusercontent.com/clearml/clearml-server/master/docker/docker-compose.yml -o docker-compose.yml \n   ```\n\n1. Configure the ClearML-Agent Services (not supported on Windows installation). \n   If `CLEARML_HOST_IP` is not provided, ClearML-Agent Services will use the external \n   public address of the **ClearML Server**. If `CLEARML_AGENT_GIT_USER` / `CLEARML_AGENT_GIT_PASS` are not provided, \n   the ClearML-Agent Services will not be able to access any private repositories for running service tasks.\n   \n   ```bash\n   export CLEARML_HOST_IP=server_host_ip_here\n   export CLEARML_AGENT_GIT_USER=git_username_here\n   export CLEARML_AGENT_GIT_PASS=git_password_here\n   ```\n\n1. Spin up the docker containers, it will automatically pull the latest **ClearML Server** build    \n   ```bash\n   docker-compose -f docker-compose.yml pull\n   docker-compose -f docker-compose.yml up\n   ```\n\n**\\* If something went wrong along the way, check our FAQ: [Common Docker Upgrade Errors](https://clear.ml/docs/latest/docs/faq/).**\n\n\n## Community \u0026 Support\n\nIf you have any questions, look to the ClearML [FAQ](https://clear.ml/docs/latest/docs/faq), or\ntag your questions on [stackoverflow](https://stackoverflow.com/questions/tagged/clearml) with '**clearml**' tag.\n\nFor feature requests or bug reports, please use [GitHub issues](https://github.com/clearml/clearml-server/issues).\n\nAdditionally, you can always find us at *clearml@allegro.ai*\n\n## License\n\n[Server Side Public License v1.0](https://github.com/mongodb/mongo/blob/master/LICENSE-Community.txt)\n\nThe **ClearML Server** relies on both [MongoDB](https://github.com/mongodb/mongo) and [ElasticSearch](https://github.com/elastic/elasticsearch).\nWith the recent changes in both MongoDB's and ElasticSearch's OSS license, we feel it is our responsibility as a\nmember of the community to support the projects we love and cherish.\nWe believe the cause for the license change in both cases is more than just,\nand chose [SSPL](https://www.mongodb.com/licensing/server-side-public-license) because it is the more general and flexible of the two licenses.\n\nThis is our way to say - we support you guys!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclearml%2Fclearml-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclearml%2Fclearml-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclearml%2Fclearml-server/lists"}