{"id":23891624,"url":"https://github.com/stephendade/aprilmav","last_synced_at":"2025-04-10T11:23:46.463Z","repository":{"id":53896268,"uuid":"213301097","full_name":"stephendade/aprilmav","owner":"stephendade","description":"Indoor navigation via Apriltags over MAVLink","archived":false,"fork":false,"pushed_at":"2025-04-08T07:56:12.000Z","size":56322,"stargazers_count":23,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-08T08:45:03.080Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stephendade.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-07T05:19:39.000Z","updated_at":"2025-04-08T07:56:13.000Z","dependencies_parsed_at":"2025-03-17T09:34:23.971Z","dependency_job_id":"6e18cd94-bf5a-47f7-9b40-90741be56bec","html_url":"https://github.com/stephendade/aprilmav","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stephendade%2Faprilmav","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stephendade%2Faprilmav/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stephendade%2Faprilmav/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stephendade%2Faprilmav/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stephendade","download_url":"https://codeload.github.com/stephendade/aprilmav/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248208609,"owners_count":21065203,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-04T13:00:20.047Z","updated_at":"2025-04-10T11:23:46.455Z","avatar_url":"https://github.com/stephendade.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# aprilmav - Indoor navigation via Apriltags over MAVLink\n\nThis library is a work-in-progress to provide accurate (cm-level)\nindoor navigation for MAVlink (APM, etc) vehicles via Apriltags (\ndeveloped by [AprilRobotics](https://april.eecs.umich.edu/))\n\nThe advantage of this method is that it does not require any expensive\nor complicated equipment. All you need is:\n- Printed April tags on a ~A4 paper mounted around the areas of travel\n- Small embedded computer (Raspberry Pi or similar) with camera (Global shutter preferred)\n\nThis library uses the the [Apriltags Python bindings](https://github.com/WillB97/pyapriltags).\n\n## How to get started\n\nNote for the ArduCam, use ``sudo apt install i2c-tools`` and ensure ``dtparam=i2c_vc=on`` is in ``/boot/config.txt``.\n\nNote for using libcamera, ensure that the picamera2 package is installed via ``sudo apt install -y python3-picamera2 --no-install-recommends``\nbefore creating the virtual environment.\n\nSet up the virtual environment and install the required packages:\n\n```\npython -m venv --system-site-packages .venv\nsource .venv/bin/activate\npip install -r requirements.txt\n```\n\nUsers running in a headless environment may need to also ``sudo apt install ffmpeg libsm6 libxext6``.\n\nPrint the Apriltags (``tagStandard41h12.pdf``) and place on the ceiling of the travel area of the robot. A density of 2-4 tags per m^2 is recommended, the idea being that the camera is able to see 3+ tags at any one time.\n\nThe camera should be mounted such that it has an unobstructed view of the tags on the ceiling.\n\nIf you are not using one of the currently supported cameras, you will need to calibrate the camera (see Adding a New Camera below).\n\nRun the ``capture_test.py`` to check the image quality, particularly when the vehicle is moving and turning. ``process_test.py`` can then be used to confirm if the Apriltags are detectable and system performance.\n\n### Note for the Raspberry Pi\n\nThe Raspberry Pi 4 is *just* powerful enough to run aprilmav in realtime (~10fps). It is highly recommended\nto use the Raspberry Pi 5 instead, which can give ~30fps with a 1.5MP camera\n\nIn some cases, the default OpenCV for the Raspberry Pi 4 may not process frames fast enough.\n\nIn general, 5+ FPS is enough for a live feed to ArduPilot.\n\nSee https://qengineering.eu/install-opencv-4.4-on-raspberry-pi-4.html for building an optimised\nversion of OpenCV.\n\n## Performance Testing\n\nThere are several scripts for testing the performance - both accuracy and\ndetected speed:\n\nSpeed of camera capture. Useful for confirming if the camera itself is a system\nbottleneck. Use ``--outputFolder=xxx`` to save the images to a specific folder, which is\nuseful for playing back datasets in the other scripts:\n\n```\n$ capture_test.py\n```\n\nSpeed of camera capture and Apriltag detection. Useful for confirming the camera\ncalibration via the Apriltag distances from the camera:\n\n```\n$ process_test.py\n```\n\nLocalisation test. Estimates the vehicles current position and velocity based on Apriltags and \noutputs to csv. Useful for confirming the camera calibration and data quality. \nUses many of the same options as ``aprilmav.py``\n\n```\n$ geo_test.py\n```\n[Use the --gui option to get a plot of the tag and vehicle locations]\n\n## Connecting to ArduPilot\n\n```\n$ aprilmav.py\n```\n\n- ``--tagSize``       Apriltag size in mm\n- ``--camera``        Camera profile in camera.yaml\n- ``--maxError``      Maximum pose error to use, in n*E-8 units\n- ``--outFile``       Output position data to this file in csv format\n- ``--device``        MAVLink connection string\n- ``--baud``          MAVLink baud rate, if using serial port in ``--device``\n- ``--source-system`` MAVLink Source system\n- ``--outputFolder``  Save processed images to this folder\n- ``--video``         Output an RTP H264 video to this IP:Port, 0 to disable\n- ``--decimation``    Apriltag decimation. Tradeoff against detection speed and distance.\n- ``--extraOpt``      Optimise detected position better. Uses a lot of extra CPU.\n- ``--outliers=N``    Reject any outlier positions (more than 3 std dev above mean), based on last N frames.\n- ``--cuda``          Use OpenCV CUDA Extensions. Only applies to Gstreamer, GenericUSB and File camera drivers.\n- ``--tagFamily``     Use this Apriltag family. Defaults to ``tagStandard41h12``\n- ``--opencv``        Use OpenCV instead of pyapriltag for decoding. Only works for tag family tagStandard31h11\n  \nCaptures, processes and localises vehicle position and orientation. Sends this in MAVLink format\nto a connected ArduPilot.\n\nIt will send a heartbeat, plus the VISION_POSITION_DELTA and VISION_POSITION_ESTIMATE messages. By default, these messages will be sent to ``udp:127.0.0.1:14550``, but can be changed via the ``--device`` argument.\n\nThe following parameters will need to be set in ArduPilot:\n\n```\nVISO_TYPE        1\nVISO_ORIENT      0\nEK3_SRC1_POSXY   6\nEK3_SRC1_POSZ    1\nEK3_SRC1_VELXY   6\nEK3_SRC1_VELZ    6\nEK3_SRC1_YAW     6\n```\n\nNote that coordinate frame conversion from the camera (see [here](https://github.com/AprilRobotics/apriltag#coordinate-system)) to \nvehicle (NED) frames takes place in aprilmav. Use the ``rotationRelVehicle`` in ``camera.yaml`` to define the camera-to-vehicle \nconversion. Thus the ``VISO_ORIENT`` should be 0 in ArduPilot. An example of this is a ``rotationRelVehicle = !!python/tuple [0, 180, 90]`` for a upwards facing camera, with the bottom of the image towards the pack of the vehicle.\n\nThe ``VISO_DELAY_MS`` should be set to 1000/framerate (ie 7fps gives a ``VISO_DELAY_MS`` of 142).\n\nThe ``--video`` and ``--imageFolder`` options will have a performance impact. It is recommended not to use these options unless for debugging purposes.\n\nNote if you are using the ``--video`` option, you will likely require to build OpenCV from source, as most pre-built\npackages do not include GStreamer support. See https://linuxize.com/post/how-to-install-opencv-on-ubuntu-20-04/ for\ndetails.\n\nTo check if your OpenCV has GStreamer support, see https://learnopencv.com/get-opencv-build-information-getbuildinformation/\n\n## Camera compatibility\n\nThe following cameras have been tested as compatible with Aprilmav:\n\n- Raspberry Pi Camera V2 (needs to be used in a very well lit room. Is not accurate in Apriltag pose detection). Use ``--camera=PiCamV2FullFoVHD``\n- Raspberry Pi Camera GS (IMX296). The 6mm lens is quite narrow, so will need a greater density of Apriltags. Use ``--camera=imx296-6mmlens``. \n- [ArduCam UC-717 (IMX296)](https://www.arducam.com/product/1-58mp-imx296-color-global-shutter-camera-module-with-m12-lens-for-raspberry-pi/). The default 6mm lens is too narrow. For a 2.7mm lens use ``--camera=imx296-2.7mmlens``\n- Arducam 1MP OV9281 (works quite well, as it has a global shutter). Requires specific libraries. Run ``./lib/getArduCamfiles.sh`` to download the files. Use ``--camera=ArduCamUC580``\n- Generic USB webcams. Will need a specific calibration and settings file for each camera model. Use ``--camera=GenericUSB``\n- IMX219 Cameras on the Nvidia Jetson at a variety of resolutions: ``--camera=JetsonCameraIMX219-8MP``, ``--camera=JetsonCameraIMX219-6MP`` and ``--camera=JetsonCameraIMX219-2MP``.\n- IMX678 Camera (Arducam B0497 USB camera). Works quite well. Requires a high-end platform (Jetson or Laptop) to process in realtime. Use ``--camera=Arducam_B0497``\n\nCameras with global shutters are preferred, as they give much more accurate position solutions.\n\n### Adding a new camera\n\nTo add a new camera, follow the following steps:\n\n- Create a camera driver in the ``./drivers`` directory. Look at ``cameraGenericUSB.py`` as an example.\n- Add a camera profile in ``camera.yaml``\n- Run ``capture_test.py`` whilst showing a chessboard (https://github.com/opencv/opencv/blob/master/doc/pattern.png) in a variety of orientations and distances\n- Run ``cameracal.py`` using the above images and put the resultant camera parameters in in ``camera.yaml``\n\nNote a separate profile will be required for a specific lens and resolution combination, and camera rotation/position.\n\n### Accuracy and Performance\n\nArduPilot requires a good velocity estimate from AprilMAV. This can be graphed via the xxx MAVLink messages.\n\nIn general, 10+ FPS is required for a stable calculation of the pose and velocity. Raising ``--decimation`` will increase the framerate, at the cost of lowering the maximum detection distance. It does not affect the accuracy of the tag pose estimate.\n\n``process_test.py`` can used in a static scence to confirm the detected distance is correct and the distance stability.\n\n``geo_test.py`` can be used to output a csv file showing the postion and velocity values for analysis. This analysis is best when with an image capture set of the vehicle moving at a constant velocity in 1 direction.\n\n``geo_test.py`` will output the timing statistics for various stages of the pipeline. This is useful for performance tuning.\n\nIf the velocity numbers are too noisy, the following options will help:\n- Ensure the camera's focus is as sharp as possible for the typical Apriltag distances\n- Decrease exposure time as much as possible to reduce motion blur during sharp turns (\u003c5ms preferred)\n- Decrease camera gain to reduce any noise in the images. Apriltags are capabile of being detected in quite low-light environments\n- A good camera calibration (if not using one of the supplied calibrations) is essential\n- Use the ``--averaging=N`` option to average over N frames. N should be a maximum of camera fps/2\n- Ensure at least 3 Apriltags are visible at all times\n  \n### Running Simulations\n\nA simulated camera is included to test the code. It is contained within ``./drivers/cameraSim.py`` and\nproduces a series of (noisy) Apriltag images forming a box track over 80 frames.\n\nIt can be used via the following arguments in ``process_test.py`` and ``geo_test.py``:\n\n``--camera=SimCamera-720p --loop=80 --tagFamily=tag36h11``\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstephendade%2Faprilmav","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstephendade%2Faprilmav","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstephendade%2Faprilmav/lists"}