https://github.com/leggedrobotics/semantic_front_end_filter
https://github.com/leggedrobotics/semantic_front_end_filter
Last synced: 6 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/leggedrobotics/semantic_front_end_filter
- Owner: leggedrobotics
- License: gpl-3.0
- Created: 2022-03-22T10:16:05.000Z (over 4 years ago)
- Default Branch: public
- Last Pushed: 2025-01-29T13:37:40.000Z (over 1 year ago)
- Last Synced: 2025-12-10T09:57:11.906Z (7 months ago)
- Language: Python
- Size: 94.5 MB
- Stars: 7
- Watchers: 3
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
Semantic Pointcloud Filter
[Project page]
[Paper]
[Data]
[Video]
Official implementation of paper **Seeing Through the Grass: Semantic Pointcloud Filter for Support Surface Learning**. Here you can find the code for SPF training and the self-supervised label genration.
## Installation
### Dependencies
create and activate conda environment
```bash
conda env create -f environment.yaml
conda activate spf_venv
```
If you want to visulize the reconstruced support surface created by Gaussian Process, you also need to install the [msgpack-c](https://github.com/msgpack/msgpack-c/tree/cpp_master). And add the [grid_map](https://github.com/ANYbotics/grid_map) packages into your catkin workspace.
### Install SPF
```bash
cd semantic_front_end_filter
pip install -e .
```
## Getting started
### Self-supervised label generation
For the details of how to reconstruct the support surface from the robot feet trajectories, see [here](https://github.com/leggedrobotics/semantic_front_end_filter/tree/public/semantic_front_end_filter/utils/labelling).
### Dataset structure
You can download our training data from [here](https://drive.google.com/drive/folders/1tRlrYeos8YdGmtDGacB-2Bt_fNqFKyHx), which we build on data collected from Perugia, Italy.
This dataset contains six trajectories: three for training and three for evaluation. To visualize a single data point from any trajectory, use the following command:
```bash
python semantic_front_end_filter/scripts/dataset_vis.py --data_path
```
This will display an image like the one shown below:
- **Left column**: Model input for the SPF network.
- **Right column**: Ground truth labels needed for training.
- SSDE Label: Support Surface Depth Estimation (mean and variance)
- SSSeg Label: Support Surface Semantic Segmentation (Obstacles vs. Support Surface)
### Train model
For your own robot, you need to raycast the reconstructed support surface in the camera point of view to get the supervison label for depth estimation.
To starting training, run
```bash
python semantic_front_end_filter/scripts/train.py --data_path
```
To validate the trained model, run
```bash
python semantic_front_end_filter/scripts/eval.py --model --outdir --data_path
```
Our trained model can be downloaded [here](https://drive.google.com/drive/folders/1N1dHvVLqcGxWwW_Jcsr1a0xgog58q9Xg?usp). Please remember to download the whole folder.
## Citing this work
```bibtex
@ARTICLE{qiao23spf,
author={Li, Anqiao and Yang, Chenyu and Frey, Jonas and Lee, Joonho and Cadena, Cesar and Hutter, Marco},
journal={IEEE Robotics and Automation Letters},
title={Seeing Through the Grass: Semantic Pointcloud Filter for Support Surface Learning},
year={2023},
volume={8},
number={11},
pages={7687-7694},
doi={10.1109/LRA.2023.3320016}
}