{"id":19733962,"url":"https://github.com/scaleoutsystems/fedn-ultralytics-tutorial","last_synced_at":"2025-05-08T17:00:03.847Z","repository":{"id":258160398,"uuid":"871183382","full_name":"scaleoutsystems/fedn-ultralytics-tutorial","owner":"scaleoutsystems","description":"A tutorial of how to implement ultralytics models in a federated setting using FEDn","archived":false,"fork":false,"pushed_at":"2025-02-07T10:57:40.000Z","size":1879,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-08T16:59:45.643Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scaleoutsystems.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-11T12:41:55.000Z","updated_at":"2025-02-20T16:07:06.000Z","dependencies_parsed_at":"2025-01-22T10:36:39.516Z","dependency_job_id":null,"html_url":"https://github.com/scaleoutsystems/fedn-ultralytics-tutorial","commit_stats":null,"previous_names":["scaleoutsystems/ultralytics-implementation-in-fedn-tutorial"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scaleoutsystems%2Ffedn-ultralytics-tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scaleoutsystems%2Ffedn-ultralytics-tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scaleoutsystems%2Ffedn-ultralytics-tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scaleoutsystems%2Ffedn-ultralytics-tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scaleoutsystems","download_url":"https://codeload.github.com/scaleoutsystems/fedn-ultralytics-tutorial/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253112071,"owners_count":21856070,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T00:34:47.992Z","updated_at":"2025-05-08T17:00:03.840Z","avatar_url":"https://github.com/scaleoutsystems.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FEDn Ultralytics Tutorial\n\n## Introduction\nThis tutorial will guide you through the process of implementing a federated learning setup using the [FEDn] framework in combination with [Ultralytics] YOLOv8 models. Federated learning allows multiple clients to collaboratively train a global model without sharing their local datasets, ensuring data privacy and security. This tutorial is designed for users familiar with machine learning and federated learning concepts and will provide step-by-step instructions on configuring and running a complete federated learning workflow.\n\nBy the end of this tutorial, you will have built a distributed training environment where clients independently train local models, and a global model is aggregated on the server. You will also learn how to use Ultralytics YOLOv8 models for object detection tasks in this federated setting.\n\n1.\tStarting the server in FEDn – Learn how to initiate the central server that coordinates federated learning activities.\n2.\tCloning the repository – Set up the project by cloning the necessary repository for configuration and deployment.\n3.\tInstalling prerequisites – Install all required dependencies for the client environments.\n4.\tSetting up the dataset – Properly structure and configure your dataset to be used with Ultralytics models.\n5.\tSetting up configurations – Configure the model, such as defining the number of classes for the YOLOv8 model.\n6.\tBuilding the compute package – Build the compute package to prepare for the training process.\n7.\tInitializing the seed model – Generate the initial model to start the training process.\n8.\tInitializing the server-side – Set up the server-side of the federated learning system.\n9.\tConnecting and starting the clients – Connect the clients that will participate in the federated training.\n10.\tTraining the global model – Observe how the global model improves through the aggregation of client updates and monitor training progress.\n\nBy following these steps, you will not only gain hands-on experience with the FEDn platform but also learn how to integrate object detection tasks with YOLOv8 in a federated learning environment.\n\n## Prerequisites\n-  `Python \u003e=3.9, \u003c=3.12` \u003chttps://www.python.org/downloads\u003e\n\n## Step 1: Starting the server in FEDn\nFirstly, create an account on the FEDn studio platform \u003chttps://fedn.scaleoutsystems.com/signup\u003e.\nOnce you are logged in, you need to start a new project by clicking on the `New project` button.\nThis initializes the server which later will be used to run the federated learning process.\n\n## Step 2: Cloning the repository\nNext, you need to clone the repository:\n```bash\ngit clone https://github.com/scaleoutsystems/fedn-ultralytics-tutorial\n```\nThen navigate into the repository:\n```bash\ncd fedn-ultralytics-tutorial\n```\nThis repository contains all the necessary files and configurations for the federated learning setup.\n\n## Step 3: Installing prerequisites\nNext, you need to install the prerequisites. You can install everything using pip:\n```bash\npip3 install -r requirements.txt\n```\nThis is recommended to be done in a virtual environment to avoid conflicts with other packages.\n\n## Step 4: Setting up the dataset\nStart setting up the dataset by creating a directory named 'datasets' inside the repository:\n```bash\nmkdir datasets\n```\nThen copy the path to the 'datasets' folder by running the following command:\n```bash\necho \"$(pwd)/datasets\" | pbcopy\n```\nUltralytics uses a specific directory to store datasets, which you can configure using the datasets_dir option in YOLO settings. To set this up to the 'datasets' directory you previously created, run the following command, replacing \u003cpath_to_dataset\u003e with the path you just copied:\n```bash\nyolo settings datasets_dir=\u003cpath_to_dataset\u003e\n```\nAfter setting the dataset directory, you’ll need to organize your data into a folder named 'fed_dataset' inside the datasets directory. Your final folder structure should look like this:\n```bash\ndatasets/\n  fed_dataset/\n    train/\n      images/\n        image1.jpg\n        image2.jpg\n        ...\n      labels/\n        image1.txt\n        image2.txt\n        ...\n    valid/\n      images/\n        image1.jpg\n        image2.jpg\n        ...\n      labels/\n        image1.txt\n        image2.txt\n        ...\n```\n\nEach label file should correspond to an image file, and the format of each label should be:\n\n```bash\n\u003cclass\u003e \u003cx_center\u003e \u003cy_center\u003e \u003cwidth\u003e \u003cheight\u003e\n\u003cclass\u003e \u003cx_center\u003e \u003cy_center\u003e \u003cwidth\u003e \u003cheight\u003e\n...\n```\nEach line corresponds to one bounding box in the image.\n\nFor further details on how to prepare your dataset, you can visit \u003chttps://docs.ultralytics.com/datasets/\u003e.\n\nFor getting started quickly, you can navigate into the `examples` repository to download and partition a sample dataset.\n\n## Step 5: Setting up configurations\n\n### Global configurations\nTo set up your YOLOv8 model, you need to configure the global_config.yaml inside the 'client' folder. Here you choose the number of classes for the YOLOv8 model by setting the `num_classes` parameter, and respective class names. You also choose which YOLOv8 model to use by setting the `model_size` parameter. \n\n### Local client configurations\nEach client can set different training configurations in the `client_config.yaml` file. This file contains the configurations for the client environments, such as the number of local epochs, and batch size. You can adjust these configurations to suit each client's hardware and training requirements.\n\n##  Step 6: Building the compute package\nOnce you’ve completed all the configurations, you can build the compute package by running the following command:\n```bash\npython3 client/setup.py\nfedn package create -p client\n```\nThis creates he compute package `package.tgz` which contains all the necessary files and configurations for the client environments.\nIf you make any changes to the global_config.yaml, you’ll need to rebuild (Step 6) and reupload (Step 8) the compute package to apply the updates. For changes in the `client_config.yaml`, you don't need to rebuild the compute package.\n\n## Step 7: Initializing the seed model\nTo initialize the seed model, run the following command:\n```bash\nfedn run build -p client\n```\nThis command will generate the seed model that will be used as the starting point for the federated learning process.\n\n## Step 8: Initializing the server-side\nThe next step is to initialize the server side. This is done by uploading the compute package and the seed model to the FEDn studio platform.\nThis is done by pressing the Sessions button and then the \"New session\" button. Here you can upload the compute package `package.tgz` and the seed model `seed.npz`.\nOnce the compute package and seed model are uploaded, you can create the session by pressing the \"Create session\" button. Here you also configure the total number of rounds and aggregator function for the federated learning process.\n\nBefore starting the training process, you need to connect the clients to the server which is done in the next step.\n\n## Step 9: Connecting and starting the clients\nTo connect a client to the server, you need to hand each client a client.yaml file. This file contains the necessary configurations for the client to connect to the server and gain access to the compute package. Connect clients by pressing the \"Clients\" button on the left hand side. Here you can download a client.yaml file for each client.\nPlace the client.yaml file in the repository and start the client by running following command:\n```bash\nfedn client start -in client.yaml\n```\nThis starts the client and connects it to the server. Repeat this process for each client you want to connect to the server.\n\n\n## Step 10: Training the global model\nOnce the clients are running, you can start the global training by pressing the Start session button in FEDn studio. This will initiate the federated learning process, where the global model is trained by aggregating the updates from the clients. Now you will see things happening on both the server and client side. You can monitor the training progress on the FEDn studio platform where metrics such as recall, precision, and mAP scores are shown.\n\n\u003cimg src=\"figs/global_convergence.png\" width=80% height=80%\u003e\n\n\nOnce training is completed, you can download any model from a certain round in the session. The model can be used for inference on new data.\n\n## Conclusion\nIn this tutorial, you have learned how to implement Ultralytics YOLOv8 models in a federated learning setting using the FEDn platform. By following the steps outlined in this tutorial, you have successfully set up a distributed training environment where clients independently train local models, and a global model is aggregated on the server. You have also learned how to configure the dataset, set up the model configurations, build the compute package, and start the federated learning process. By completing this tutorial, you have gained hands-on experience with federated learning and object detection tasks using Ultralytics models.\n\n# Note\nSteps 1, 2, 5, 6, 7 and 8 only need to be done once to set up the federated learning environment in FEDn.\n\nTo connect a new client, the only steps that needs to be followed are step 2, 3, 4 and 9.\nEach client can have different configurations in the client_config.yaml file to account for different hardware and training requirements.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscaleoutsystems%2Ffedn-ultralytics-tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscaleoutsystems%2Ffedn-ultralytics-tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscaleoutsystems%2Ffedn-ultralytics-tutorial/lists"}