{"id":24955207,"url":"https://github.com/pedrala/disaster_bot","last_synced_at":"2026-04-19T03:02:23.992Z","repository":{"id":270464297,"uuid":"910461271","full_name":"pedrala/disaster_bot","owner":"pedrala","description":"This program receives images from the Turtlebot4 camera in a ROS2 environment, detects objects (fire extinguishers and people) in real time, estimates the locations of the objects, and publishes them as ROS Marker publisher.","archived":false,"fork":false,"pushed_at":"2025-01-05T08:41:09.000Z","size":53927,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-31T06:12:40.020Z","etag":null,"topics":["calibration","flann-matchers","gazebo","homography","multithreading","opencv","orb","perspective-n-point","rviz2","slam","undistort"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pedrala.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-12-31T10:36:59.000Z","updated_at":"2025-01-05T08:41:13.000Z","dependencies_parsed_at":"2025-10-18T16:13:53.368Z","dependency_job_id":null,"html_url":"https://github.com/pedrala/disaster_bot","commit_stats":null,"previous_names":["pedrala/disaster_bot"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pedrala/disaster_bot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrala%2Fdisaster_bot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrala%2Fdisaster_bot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrala%2Fdisaster_bot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrala%2Fdisaster_bot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pedrala","download_url":"https://codeload.github.com/pedrala/disaster_bot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrala%2Fdisaster_bot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31992822,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T20:23:30.271Z","status":"online","status_checked_at":"2026-04-19T02:00:07.110Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["calibration","flann-matchers","gazebo","homography","multithreading","opencv","orb","perspective-n-point","rviz2","slam","undistort"],"created_at":"2025-02-03T05:40:23.899Z","updated_at":"2026-04-19T03:02:23.974Z","avatar_url":"https://github.com/pedrala.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Disaster Bot Project\n\n## Overview\nThis program receives images from the Turtlebot4 camera in a ROS2 environment, detects objects (fire extinguishers and people) in real time, estimates the location of the objects, and publishes them as ROS Markers. After extracting feature points and descriptors from the images of fire extinguishers and people using the ORB algorithm, it matches the real-time images of the Turtlebot4 camera with the feature points of the previously extracted images using the FLANN matcher, and if there are more than 5 matching points, it is judged that the object has been recognized and displays the visual results through OpenCV. It estimates the location of the image using the homography and PnP algorithms, and if you add a marker in RVIZ, you can check the location of the image (fire extinguisher, person).\n\nIn addition, the parameters of the yaml file were adjusted for auto-mapping without the map saving process after SLAM.\n\n\n## Capture\n\n### Pre-prepared fire extinguisher and human images for feature extraction \n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"disaster_bot/images/ext_orig.png\" alt=\"Fire Extinguisher Detection\" width=\"300\"\u003e\n  \u003cimg src=\"disaster_bot/images/man_orig.png\" alt=\"Fallen Man Detection\" width=\"300\"\u003e\n\u003c/p\u003e\n\n### Real-time image and feature point matching using the ORB algorithm\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"capture/orb_recog_firex.png\" alt=\"Fire Extinguisher Detection\" width=\"500\"\u003e\n\u003c/p\u003e\n\n### Indoor SLAM track\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"capture/ttb4_playground.jpeg\" alt=\"ttb4_playground\" width=\"1000\"\u003e\n\u003c/p\u003e\n\n\n## Project Demo\n[turtlebot4 automapping video](capture/ttb4_automapping_navigation.mp4)\n\n[ttb4_automapping_in_rviz_video](capture/ttb4_automapping_navigation_rviz_short_480p.mp4)\n\n[image recoognition and displaying location markers in rviz2 video](capture/demo.gif)\n\n\n### FLANN Matcher, Homography and Perspective-n-Point algorithm\n\n```python\n# 각 참조 이미지에 대해 매칭 및 탐지 수행\nfor ref in self.reference_data:\n    name = ref['name']\n    keypoints_ref = ref['keypoints_ref']\n    descriptors_ref = ref['descriptors_ref']\n    object_points_3d = ref['object_points_3d']\n    w_ref, h_ref = ref['image_size']\n\n    # 참조 이미지(descriptors_ref)와 실시간 이미지(descriptors_frame)의 특징 매칭\n    matches = self.match_features(descriptors_ref, descriptors_frame)\n\n    if len(matches) \u003e 5:  # 매칭 조건을 10개에서 5개로 낮춤\n        self.get_logger().info(f\"{name}: {len(matches)}개의 좋은 매칭점을 발견했습니다.\")\n        matches = matches[:50]\n\n        pts_ref = np.float32([keypoints_ref[m.queryIdx].pt for m in matches]).reshape(-1, 1, 2)\n        pts_frame = np.float32([keypoints_frame[m.trainIdx].pt for m in matches]).reshape(-1, 1, 2)\n\n        H, mask = cv2.findHomography(pts_ref, pts_frame, cv2.RANSAC, 5.0)\n\n        if H is not None:\n            corners = np.float32([[0, 0], [w_ref, 0], [w_ref, h_ref], [0, h_ref]]).reshape(-1, 1, 2)\n            transformed_corners = cv2.perspectiveTransform(corners, H)\n            image_points = transformed_corners.reshape(-1, 2)\n\n            camera_matrix = np.array(self.camera_info.k).reshape(3, 3)\n            dist_coeffs = np.array(self.camera_info.d)\n            \n            # PnP 알고리즘을 사용하여 위치 및 방향 추정\n            success, rotation_vector, translation_vector = cv2.solvePnP(\n                object_points_3d, # 객체의 3D 좌표\n                image_points,  # 이미지에서의 변환된 2D 좌표\n                camera_matrix,  # 카메라 행렬\n                dist_coeffs,  # 왜곡 계수\n                flags=cv2.SOLVEPNP_ITERATIVE  # 반복 알고리즘 사용\n            )                \n\n            if success:\n                self.publish_marker(translation_vector, rotation_vector, name, ref['marker_scale'])\n                object_detections[name]['corners'] = transformed_corners\n                object_detections[name]['matches'] = matches\n                object_detections[name]['detected'] = True\n                #self.save_matched_image(frame, name)  # 매칭 성공 시 이미지 저장\n            else:\n                self.get_logger().warn(f\"{name}: PnP 알고리즘 실패\")\n        else:\n            self.get_logger().warn(f\"{name}: 호모그래피 계산 실패\")\n    else:\n        self.get_logger().warn(f\"{name}: 충분한 매칭점({len(matches)}개)이 없습니다.\")\n```\n\nHow to execute test\n===============================\nYou can test object recognition by placing an image of a fire extinguisher or a person printed on A4 paper in front of your laptop's webcam.\n\n```console\nrviz2\nros2 run disaster_bot test\n```\n\nHow to execute real\n===============================\nDuring TurtleBot4 SLAM, the real-time images subscribed to by the TurtleBot4 camera are matched with existing images using the ORB algorithm, and then the location is estimated using the homography-PnP algorithm.\n\n```console\nrviz2\nros2 run disaster_bot dibot\n```\n\nHow to execute launch file\n=============================\nRun auto-mapping launch file by applying yaml file\n\n```console\nros2 launch disaster_bot auto_mapping_launch.py\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpedrala%2Fdisaster_bot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpedrala%2Fdisaster_bot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpedrala%2Fdisaster_bot/lists"}