{"id":25080827,"url":"https://github.com/opencv/bpc","last_synced_at":"2025-04-05T17:02:55.981Z","repository":{"id":272232092,"uuid":"915552282","full_name":"opencv/bpc","owner":"opencv","description":"Codebase for Perception Challenge For Bin-Picking","archived":false,"fork":false,"pushed_at":"2025-03-28T00:33:18.000Z","size":5302,"stargazers_count":83,"open_issues_count":4,"forks_count":44,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-03-29T16:03:12.891Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://bpc.opencv.org/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opencv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-12T06:37:04.000Z","updated_at":"2025-03-26T16:31:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"194f2cd1-7fae-42c3-99a2-23fbf5c939b9","html_url":"https://github.com/opencv/bpc","commit_stats":null,"previous_names":["yadunund/ibpc","yadunund/bpc","opencv/bpc"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencv%2Fbpc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencv%2Fbpc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencv%2Fbpc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencv%2Fbpc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opencv","download_url":"https://codeload.github.com/opencv/bpc/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247369950,"owners_count":20927928,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-07T04:26:41.934Z","updated_at":"2025-04-05T17:02:55.972Z","avatar_url":"https://github.com/opencv.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Perception Challenge For Bin-Picking\n\n[![build_packages](https://github.com/opencv/bpc/actions/workflows/build.yaml/badge.svg?branch=main)](https://github.com/opencv/bpc/actions/workflows/build.yaml)\n[![style](https://github.com/opencv/bpc/actions/workflows/style.yaml/badge.svg?branch=main)](https://github.com/opencv/bpc/actions/workflows/style.yaml)\n[![test validation](https://github.com/opencv/bpc/actions/workflows/test.yaml/badge.svg?branch=main)](https://github.com/opencv/bpc/actions/workflows/test.yaml)\n\nChallenge [official website](https://bpc.opencv.org/).\n\n![](../media/bpc.gif)\n\n## Key Steps for Participation\n1. [Set up the local environment](#settingup)\n2. [Download training and validation/testing data](#download_train_data)\n3. Prepare submission:\n    1. [Check the baseline solution for input/output examples](#baseline_solution)\n    2. [Build your custom image with your solution and test it locally](#build_custom_bpc_pe)\n4. [Submit your solution](#submission)\n\n## Overview\n\nThis repository contains the sample submission code, ROS interfaces, and evaluation service for the Perception Challenge For Bin-Picking. The reason we openly share the tester code here is to give participants a chance to validate their submissions before submitting.\n\n- **Estimator:**\n  The estimator code represents the sample submission. Participants need to implement their solution by editing the placeholder code in the function `get_pose_estimates` in `ibpc_pose_estimator.py`. The tester will invoke the participant's solution via a ROS 2 service call over the `/get_pose_estimates` endpoint.\n\n- **Tester:**\n  The tester code serves as the evaluation service. A copy of this code will be running on the evaluation server and is provided for reference only. It loads the test dataset, prepares image inputs, invokes the estimator service repeatedly, collects the results, and submits for further evaluation.\n\n- **ROS Interface:**\n  The API for the challenge is a ROS service, [GetPoseEstimates](ibpc_interfaces/srv/GetPoseEstimates.srv), over `/get_pose_estimates`. Participants implement the service callback on a dedicated ROS node (commonly referred to as the PoseEstimatorNode) which processes the input data (images and metadata) and returns pose estimation results.\n\nIn addition, we provide the [ibpc_py tool](https://github.com/opencv/bpc/tree/main/ibpc_py) which facilitates downloading the challenge data and performing various related tasks. You can find the installation guide and examples of its usage below. \n\n## Design\n\n### ROS-based Framework\n\nThe core architecture of the challenge is based on ROS 2. Participants are required to respond to a ROS 2 Service request with pose estimation results. The key elements of the architecture are:\n\n- **Service API:**\n  The ROS service interface (defined in the [GetPoseEstimates](ibpc_interfaces/srv/GetPoseEstimates.srv) file) acts as the API for the challenge.\n\n- **PoseEstimatorNode:**\n  Participants are provided with Python templates for the PoseEstimatorNode. Your task is to implement the callback function (e.g., `get_pose_estimates`) that performs the required computation. Since the API is simply a ROS endpoint, you can use any of the available [ROS 2 client libraries](https://docs.ros.org/en/jazzy/Concepts/Basic/About-Client-Libraries.html#client-libraries) including C++, Python, Rust, Node.js, or C#. Please use [ROS 2 Jazzy Jalisco](https://docs.ros.org/en/jazzy/index.html).\n\n- **TesterNode:**\n  A fully implemented TesterNode is provided that:\n  - Uses the bop_toolkit_lib to load the test dataset and prepare image inputs.\n  - Repeatedly calls the PoseEstimatorNode service over the `/get_pose_estimates` endpoint.\n  - Collects and combines results from multiple service calls.\n  - Saves the compiled results to disk in CSV format.\n\n### Containerization\n\nTo simplify the evaluation process, Dockerfiles are provided to generate container images for both the PoseEstimatorNode and the TesterNode. This ensures that users can run their models without having to configure a dedicated ROS environment manually.\n\n## Submission Instructions \u003ca name=\"submission\"\u003e\u003c/a\u003e\n\nParticipants are expected to modify the estimator code to implement their solution. Once completed, your custom estimator should be containerized using Docker and submitted according to the challenge requirements. You can find detailed submission instructions [here](https://bpc.opencv.org/web/challenges/challenge-page/1/submission). Please make sure to register a team to get access to the submission instructions. \n\n## Setting up \u003ca name=\"settingup\"\u003e\u003c/a\u003e\n\nThe following instructions will guide you through the process of validating your submission locally before official submission.\n\n#### Requirements\n\n- [Docker](https://docs.docker.com/) installed with the user in docker group for passwordless invocations. Ensure Docker Buildx is installed (`docker buildx version`). If not, install it with `apt install docker-buildx-plugin` or `apt install docker-buildx` (distribution-dependent). \n- 7z -- `apt install p7zip-full`\n- Python3 with virtualenv  -- `apt install python3-virtualenv`\n- The `ibpc` and `rocker` CLI tools are tested on Linux-based machines. Due to known Windows issues, we recommend Windows users develop using [WSL](https://learn.microsoft.com/en-us/windows/wsl/about).\n\n\u003e Note: Participants are expected to submit Docker containers, so all development workflows are designed with this in mind.\n\n#### Setup a workspace\n```bash\nmkdir -p ~/bpc_ws\n```\n\n#### Create a virtual environment \n\n📄 If you're already working in some form of virtualenv you can continue to use that and install `bpc` in that instead of making a new one. \n\n```bash\npython3 -m venv ~/bpc_ws/bpc_env\n```\n\n#### Activate that virtual env\n\n```bash\nsource ~/bpc_ws/bpc_env/bin/activate\n```\n\nFor any new shell interacting with the `bpc` command you will have to rerun this source command.\n\n#### Install bpc \n\nInstall the bpc command from the ibpc pypi package. (bpc was already taken :-( )\n\n```bash\npip install ibpc\n```\n\n#### Fetch the source repository\n\n```bash\ncd ~/bpc_ws\ngit clone https://github.com/opencv/bpc.git\n```\n\n#### Fetch the dataset\n\n```bash\ncd ~/bpc_ws/bpc\nbpc fetch ipd\n```\nThis will download the ipd_base.zip, ipd_models.zip, and ipd_val.zip (approximately 6GB combined). The dataset is also available for manual download on [Hugging Face](https://huggingface.co/datasets/bop-benchmark/ipd).\n\n#### Quickstart with prebuilt images\n```bash\nbpc test ghcr.io/opencv/bpc/bpc_pose_estimator:example ipd\n```\nThis will download the prebuilt zenoh, tester, and pose_estimator images and run containers based on them. The pose_estimator image contains an empty get_pose_estimates function. After the containers start, you should see the following in your terminal:\n```\n[INFO] [1740003838.048516355] [bpc_pose_estimator]: Starting bpc_pose_estimator...\n[INFO] [1740003838.049547292] [bpc_pose_estimator]: Model directory set to /opt/ros/underlay/install/models.\n[INFO] [1740003838.050190130] [bpc_pose_estimator]: Pose estimates can be queried over srv /get_pose_estimates.\n```\n### Build and test custom bpc_pose_estimator image \u003ca name=\"build_custom_bpc_pe\"\u003e\u003c/a\u003e\n\nYou can then build custom bpc_pose_estimator image with your updated get_pose_estimates function\n\n```bash\ncd ~/bpc_ws/bpc\ndocker buildx build -t \u003cPOSE_ESTIMATOR_DOCKER_TAG\u003e \\\n    --file ./Dockerfile.estimator \\\n    --build-arg=\"MODEL_DIR=models\" \\\n    .\n```\n\nand run it with the following command\n```bash\nbpc test \u003cPOSE_ESTIMATOR_DOCKER_TAG\u003e ipd\n```\n\nFor example: \n```bash\ncd ~/bpc_ws/bpc\ndocker buildx build -t bpc_pose_estimator:example \\\n    --file ./Dockerfile.estimator \\\n    --build-arg=\"MODEL_DIR=models\" \\\n    .\nbpc test bpc_pose_estimator:example ipd\n```\nThis will validate your pose_estimator image against the local copy of validation dataset.\nWhen you build a new image you rerun this test.\n\nThe console output will show the system getting started and then the output of the estimator. \n\nIf you would like to interact with the estimator and run alternative commands or anything else in the container you can invoke it with `--debug`\n\nThe tester console output will be streamed to the file `ibpc_test_output.log` Use this to see it\n\n```bash\ntail -f ibpc_test_output.log\n```\n\nThe results will come out as `submission.csv` when the tester is complete.\n\n### Baseline Solution \u003ca name=\"baseline_solution\"\u003e\u003c/a\u003e\n\nWe provide a simple baseline solution as a reference for implementing the solution in `ibpc_pose_estimator_py`. Please refer to the [baseline_solution](https://github.com/opencv/bpc/tree/baseline_solution) branch and follow the instructions there.\n\n\n### Tips\n\n🐌 **If you are iterating a lot of times with the validation and are frustrated by how long the cuda installation is, you can add it to your Dockerfile as below.**\nIt will make the image significantly larger, but faster to iterate if you put it higher in the dockerfile. We can't include it in the published image because the image gets too big for hosting and pulling easily.\n\n```\nRUN apt-get update \u0026\u0026 apt-get install -y --no-install-recommends \\\n    wget software-properties-common gnupg2 \\\n    \u0026\u0026 rm -rf /var/lib/apt/lists/*\n\nRUN \\\n  wget -q https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb \u0026\u0026 \\\n  dpkg -i cuda-keyring_1.1-1_all.deb \u0026\u0026 \\\n  rm cuda-keyring_1.1-1_all.deb \u0026\u0026 \\\n  \\\n  apt-get update \u0026\u0026 \\\n  apt-get -y install cuda-toolkit \u0026\u0026 \\\n  rm -rf /var/lib/apt/lists/*\n```\n\n## Further Details\n\nThe above is enough to get you going.\nHowever we want to be open about what else were doing.\nYou can see the source of the tester and build your own version as follows if you'd like. \n\n### If you would like the training data and test data \u003ca name=\"download_train_data\"\u003e\u003c/a\u003e\n\nUse the command:\n```bash\nbpc fetch ipd_all\n```\nThe dataset is also available for manual download on [Hugging Face](https://huggingface.co/datasets/bop-benchmark/ipd).\n\n### Manually Run components \n\nIt is possible to manually run the components.\n`bpc` shows what it is running on the console output.\nOr you can run as outlined below. \n\n\n#### Start the Zenoh router\n\n```bash\ndocker run --init --rm --net host eclipse/zenoh:1.2.1 --no-multicast-scouting\n```\n\n#### Run the pose estimator\nWe use [rocker](https://github.com/osrf/rocker) to add GPU support to Docker containers. To install rocker, run `pip install rocker` on the host machine.\n```bash\nrocker --nvidia --cuda --network=host --volume \u003cPATH_TO_DATASET\u003e/models:/opt/ros/underlay/install/3d_models -- bpc_pose_estimator:example\n```\n\n#### Run the tester\n\n\u003e Note: Substitute the \u003cPATH_TO_DATASET\u003e with the directory that contains the [ipd](https://huggingface.co/datasets/bop-benchmark/ipd/tree/main) dataset. Similarly, substitute \u003cPATH_TO_OUTPUT_DIR\u003e with the directory that should contain the results from the pose estimator. By default, the results will be saved as a `submission.csv` file but this filename can be updated by setting the `OUTPUT_FILENAME` environment variable.\n\n```bash\ndocker run --network=host -e BOP_PATH=/opt/ros/underlay/install/datasets -e SPLIT_TYPE=val -v\u003cPATH_TO_DATASET\u003e:/opt/ros/underlay/install/datasets -v\u003cPATH_TO_OUTPUT_DIR\u003e:/submission -it bpc_tester:latest\n```\n\n### Build the bpc_tester image\nGenerally not required, but to build the tester image, run the following command:\n```bash\ncd ~/bpc_ws/bpc\ndocker buildx build -t bpc_tester:latest \\\n    --file ./Dockerfile.tester \\\n    .\n```\nYou can then use your tester image with the bpc tool, as shown in the example below:\n```bash\nbpc test bpc_pose_estimator:example ipd --tester-image bpc_tester:latest\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencv%2Fbpc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencv%2Fbpc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencv%2Fbpc/lists"}