{"id":20041731,"url":"https://github.com/ajithvcoder/multivideo_objectdetection_mlops_project","last_synced_at":"2026-04-28T16:38:17.490Z","repository":{"id":220759190,"uuid":"752518912","full_name":"ajithvcoder/Multivideo_Objectdetection_MLOPS_Project","owner":"ajithvcoder","description":"Multi video offline streaming and Yolov8 object detection with Kafka, torch serve, AWS EC2","archived":false,"fork":false,"pushed_at":"2024-02-25T15:26:45.000Z","size":97488,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-02T07:14:51.811Z","etag":null,"topics":["aws-ec2","confluent-kafka","flask","flask-socketio","mongodb","onnx","streaming","torchserve","yolov8"],"latest_commit_sha":null,"homepage":"https://www.youtube.com/playlist?list=PLaiyLZ9Xs1bzsHfMhjTSJD9Jfj2VwBXZU","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ajithvcoder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-04T04:26:01.000Z","updated_at":"2024-08-06T06:41:49.000Z","dependencies_parsed_at":"2024-02-25T16:32:35.056Z","dependency_job_id":null,"html_url":"https://github.com/ajithvcoder/Multivideo_Objectdetection_MLOPS_Project","commit_stats":null,"previous_names":["ajithvcoder/multivideo_objectdetection_mlops_project"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ajithvcoder/Multivideo_Objectdetection_MLOPS_Project","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2FMultivideo_Objectdetection_MLOPS_Project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2FMultivideo_Objectdetection_MLOPS_Project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2FMultivideo_Objectdetection_MLOPS_Project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2FMultivideo_Objectdetection_MLOPS_Project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ajithvcoder","download_url":"https://codeload.github.com/ajithvcoder/Multivideo_Objectdetection_MLOPS_Project/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2FMultivideo_Objectdetection_MLOPS_Project/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32390053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-28T14:34:11.604Z","status":"ssl_error","status_checked_at":"2026-04-28T14:32:37.009Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-ec2","confluent-kafka","flask","flask-socketio","mongodb","onnx","streaming","torchserve","yolov8"],"created_at":"2024-11-13T10:47:38.439Z","updated_at":"2026-04-28T16:38:17.458Z","avatar_url":"https://github.com/ajithvcoder.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multivideo_Objectdetection_MLOPS_Project\n\n**Youtube Playlist** : [Link here](https://www.youtube.com/playlist?list=PLaiyLZ9Xs1bzsHfMhjTSJD9Jfj2VwBXZU)\n\n\u003cimg src=\"./assets/snap_multivideostreaming.png\" width=\"500\" height=\"300\" title=\"Object detection\"\u003e\n\n**Description** \n\nI am having a set of videos and kafka producer is taking and passing to kafka server then consumer consumes it. The consumer has torchserve part and then objects are detected by it and then stored in mongo db.\n\n**its offline streaming project not real time streaming.** but can be made as real time streaming with modifications in producer part\n\n### Procedure\n    docker network create kafka-network\n\n    docker compose -f kafka/docker-compose.yml build --no-cache\n\n    docker compose -f kafka/docker-compose.yml up\n\n    docker compose -f docker-compose.yml build --no-cache\n\n    docker compose -f docker-compose.yml up\n\n    Inference at http://localhost:5000\n\n    Check if torchserve is enabled http://localhost:8081/models\n\nTo delete the containers \n\n    docker compose -f kafka/docker-compose.yml down\n    docker compose -f docker-compose.yml down\n\nEC2 docker installation\n\n    sudo apt-get update -y\n\n    sudo apt-get upgrade\n\n    curl -fsSL https://get.docker.com -o get-docker.sh\n\n    sudo sh get-docker.sh\n\n    sudo usermod -aG docker ubuntu\n\n    newgrp docker\n\n## Note \u0026 Value added\n\nif you use a better model with good dataset it can predict quality will be increased. I worked on integrating following parts in code as its a MLOPs project.\n\n- Confluent Kafka(base) - docker and python based confluent kafka (i didnt use other features of confluent kafka)\n- TorchServe(Onnx)\n- Flask\n- Flask socketio\n- From data ingestion to evaluation stage\n- dvc pipeline\n- docker-compose\n- Mongodb\n- AWS EC2\n\n- Kindly change the epochs in params.yaml file to 60 and then replace \"trained_model_60epoch.onnx\" with \"detection.onnx\" in Dockerfile for  using the model training with dvc repro. I dont have a gpu so i trained in colab and placed \"trained_model_60epoch.onnx\" file in git already.\n\n- Kafka part can also be improved \n\n- In comming days i will try to create and explain part by part example how kafka producer and consumer commincates, how torchserve serves, etc in detail\n\n## Github CI/CD\n- it seems that docker compose with github CI/CD is not possible but you can always create more docker containers with [this type](https://github.com/ajithvcoder/Spaceship_Titanic_MLOps_Project/blob/main/.github/workflows/main.yaml) of CI/CD pipeline feel free to explore it \n\n### References\n\n1. https://github.com/wingedrasengan927/Distributed-Multi-Video-Streaming-and-Processing-with-Kafka/tree/master\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajithvcoder%2Fmultivideo_objectdetection_mlops_project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fajithvcoder%2Fmultivideo_objectdetection_mlops_project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajithvcoder%2Fmultivideo_objectdetection_mlops_project/lists"}