{"id":19068708,"url":"https://github.com/machine-learning-tokyo/edgeaicontest3","last_synced_at":"2025-04-28T13:40:39.196Z","repository":{"id":96775643,"uuid":"263310768","full_name":"Machine-Learning-Tokyo/EdgeAIContest3","owner":"Machine-Learning-Tokyo","description":"This repository present MLT Team solution for the The 3rd AI Edge Contest.","archived":false,"fork":false,"pushed_at":"2020-09-19T08:47:33.000Z","size":59749,"stargazers_count":53,"open_issues_count":1,"forks_count":11,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-18T16:26:02.611Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Machine-Learning-Tokyo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-05-12T10:55:35.000Z","updated_at":"2024-12-08T12:39:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"56349a4e-819f-4368-9f1e-b68db7bc2de3","html_url":"https://github.com/Machine-Learning-Tokyo/EdgeAIContest3","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Machine-Learning-Tokyo%2FEdgeAIContest3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Machine-Learning-Tokyo%2FEdgeAIContest3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Machine-Learning-Tokyo%2FEdgeAIContest3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Machine-Learning-Tokyo%2FEdgeAIContest3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Machine-Learning-Tokyo","download_url":"https://codeload.github.com/Machine-Learning-Tokyo/EdgeAIContest3/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251321170,"owners_count":21570689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T01:11:29.962Z","updated_at":"2025-04-28T13:40:39.152Z","avatar_url":"https://github.com/Machine-Learning-Tokyo.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The 3rd AI Edge Contest (Algorithm Contest 2)\nThis repository present **MLT Team** solution for the  [The 3rd AI Edge Contest](https://signate.jp/competitions/256).\n\n![object_tracking](https://github.com/Machine-Learning-Tokyo/EdgeAIContest3/blob/master/imgs/object_tracking.gif)\n\n## Introduction\nFirst of all thank you to [Signate](https://signate.jp/) for hosting this exciting competition.\nOur team MLT is based in Tokyo and we are really interested in the Edge devices and AI applications at the edge.\nHaving the opportunity to work on **“a made in” Tokyo dataset** is really motivating and gives the feeling to work on a real world project compared to other competitions.\nOn top of that, the field of [MoT](https://en.wikipedia.org/wiki/Multiple_object_tracking) is still a hot research topic, and certainly very challenging by it’s complexity and application.\nAlso the sense of working together with one aim to get a good rank was a continuous source of motivation.\n\nWe are planning to continue this project and actually deploying the code on an edge device.\n\n## Structure\n    .\n    ├── src             # Source files (submission file / src module / training)\n    ├── evaluation      # Testing files (small unit tests etc.)\n    ├── model           # Models (binary/json model files)\n    ├── data            # Data (augmented/raw/processed)\n    ├── notebook        # Jupyter Notebooks (Data exploration/ Data preparation)\n    ├── LICENSE\n    └── README.md\n\n## Solution overview\n\nOur solution is composed of two sequential modules: Object detection and Tracking.\n\n![overview](https://github.com/Machine-Learning-Tokyo/EdgeAIContest3/blob/master/notebook/overview.png)\n\n### 1] Object Detection\nThe first step consists of detecting objects in the frame.\nWe selected RetinaNet with ResNet 101 as CNN backbone as our model architecture because it is really efficient with dense and small scale objects while assuring fast inference for edge device application as being a single stage detector. Please have a look at [the annotated paper](https://github.com/Machine-Learning-Tokyo/papers-with-annotations/blob/master/object-detection/RetinaNet.pdf) from one of our team member to get a detailed understanding of this particular model.\n\nOne specification was to adopt a **batching augmentation**, that consist of inferring the detection on augmented image, and finally applying Non-Maximum-Suppression on all the detected objects. We tried out several augmentations (Dark-Bright / Crop Side / ...), but based on experiment our final submission only use *flip Right-Left* augmentation.\n\nFinally, we did some heuristics bounding box filtering based on our dataset exploration (final submission use filtering based on image position.)\n\nAll our training related information is summarized in [ObjectDetectionTraining.md](src/ObjectDetectionTraining.md).\n\n### 2] Tracking\n\nWe formalized the tracking problem as a maximum weighted matching problem for objects in two adjacent frames and solved it using [Hungarian Algorithm](https://en.wikipedia.org/wiki/Hungarian_algorithm#:~:text=Hungarian%20algorithm%20%2D%20Wikipedia-,Hungarian%20algorithm,anticipated%20later%20primal%2Ddual%20methods.).\n\nFor matching costs, we utilized several features as:\n- position\n- size\n- image similarity (histogram)\n\nOur tracker keeps all the history of object tracking and estimates the next position of each object in the next frame by linear or quadratic regression.\n\nThen, the tracker tries to match objects with close position, similar size and similar image as much as possible.\nDuring object matching, the tracker also takes into account object appearance and disappearance.\n\nWe added some virtual objects where the objects matched to these virtual objects are regarded as newly appeared or disappeared.\n\nThe disappeared objects are also kept in the tracker for a while and can be matched to some objects in the subsequent frames.\n\n## Improvement and Lesson learned\n\n### Improvements\nThere are lot of room for improvement:\n- Better tunned or new batch augmentation.\n- Pedestrian classifier to reduce FP. *(developed but not tuned enough to be used for the submission)*\n- Automation pipeline for parameters tunning.\n\n### What did we learn\n- Public score does not always reflect the private score, and those make the tunning hard.\n- Running dummy inference in the load_model method allows us to reduce inference time.\n- Heuristics are very valuable to increase the score\n- We should have spent some resource on cleaning the dataset\n- When you think you're done with data exploration, you're not: **Make summary and clean report.**\n\n## Data\nPlease download data from the competition into the ```data/``` folder:\n\n**Note that Signate may not open the dataset.**\n\nAfter setting the [signate CLI link](https://pypi.org/project/signate/)\n```\ncd data/\nsignate download --competition-id=256\n```\n\n## Notebook\nThis folder contain our **DataExploration notebook**:\n\n![overview](https://github.com/Machine-Learning-Tokyo/EdgeAIContest3/blob/master/notebook/all_video.png)\n\n## Source code\nPlease refer to [src/README.md](src/README.md) for explanation of our source code.\n\n## Submission - Evaluation\n\nEvaluation contains Signate code to run local evaluation.\n\nRun the following command to generate the sample_submit folder.\n```\nbash generate_mlt_submission.sh\n```\n\nIn order to test submission instance run following:\n```\nbash test_submit.sh\ncd sample_submit; pwd\npython src/main.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmachine-learning-tokyo%2Fedgeaicontest3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmachine-learning-tokyo%2Fedgeaicontest3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmachine-learning-tokyo%2Fedgeaicontest3/lists"}