{"id":41438908,"url":"https://github.com/nesl/gdtm_tracking_anonymized","last_synced_at":"2026-01-23T14:52:52.673Z","repository":{"id":208136681,"uuid":"719825733","full_name":"nesl/GDTM_Tracking_Anonymized","owner":"nesl","description":null,"archived":false,"fork":false,"pushed_at":"2023-11-19T23:13:11.000Z","size":54027,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-04T19:41:39.494Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nesl.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-11-17T01:20:23.000Z","updated_at":"2023-11-17T01:23:16.000Z","dependencies_parsed_at":"2023-11-20T00:31:17.018Z","dependency_job_id":null,"html_url":"https://github.com/nesl/GDTM_Tracking_Anonymized","commit_stats":null,"previous_names":["nesl/gdtm_tracking_anonymized"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nesl/GDTM_Tracking_Anonymized","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nesl%2FGDTM_Tracking_Anonymized","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nesl%2FGDTM_Tracking_Anonymized/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nesl%2FGDTM_Tracking_Anonymized/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nesl%2FGDTM_Tracking_Anonymized/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nesl","download_url":"https://codeload.github.com/nesl/GDTM_Tracking_Anonymized/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nesl%2FGDTM_Tracking_Anonymized/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28694457,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-23T14:15:13.573Z","status":"ssl_error","status_checked_at":"2026-01-23T14:09:05.534Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-23T14:52:50.931Z","updated_at":"2026-01-23T14:52:52.665Z","avatar_url":"https://github.com/nesl.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n# GDTM-Tracking\n\n**GDTM** is a new multi-hour dataset collected with a network of multimodal sensors for the indoor geospatial tracking problem. It features time-synchronized steoreo-vision camera, LiDAR camera, mmWave radar, and microphone arrays, as well as ground truth data containing the position and orientations of the sensing target (remote controlled cars on a indoor race track) and the sensor nodes. For details of the dataset please refer to [GitHub](https://anonymous.4open.science/r/GDTM_Anonymized-4469/README.md) and PDF (still under review).\n\nThis repository contains our baseline applications described in PDF (still under review) built to use GTDM data. It features two architectures (early fusion and late fusion and two choices of sensor sets (camera only and all-modalities) to track the locations of a target RC car.\n\n**Note** for dataset documentation and pre-processing, please refer to [GitHub](https://anonymous.4open.science/r/GDTM_Anonymized-4469/README.md).\n\n\n\n## Installation Instuctions\n\n### Environment\nThe code is tested with:\nUbuntu 20.04\nAnaconda 22.9.0 (for virtual python environment)\nNVIDIA-driver 525.105.17\nThe code should be compatible with most Anaconda, NVIDIA-driver, and Ubuntu versions available around 2023/06.\n\n### Code Repository Structure\nWe only release the early fusion, all modalities version of the model. Further variants will be released upon acceptance. Details are described in **Baseline 1** section of PDF (Still under review).\n\n\n\nAs step one, please clone the desired branch using terminal. It is not possible to clone the anonymous repo, and these instructions will be updated before the camera-ready. \n```\ncd ~/Desktop\ngit clone https://anonymous.4open.science/r/GDTM_Anonymized-4469.git\n```\nor\n```\ncd ~/Desktop\ngit clone --branch \u003cbranchname\u003e https://anonymous.4open.science/r/GDTM_Anonymized-4469.git\n```\n\n### Install Dependencies\nFirst, place the repository folder on Desktop and rename it \"mmtracking\".\n```\nmv \u003cpath-to-cloned-repository\u003e ~/Desktop/mmtracking\n```\nCreate a new conda environment using\n```\ncd ~/Desktop/mmtracking\nconda create -n iobt python=3.9\nconda activate iobt\n```\nInstall a few torch and mmcv using pip:\n```\npip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113\npip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.12.0/index.html\n```\nInstall other dependencies:\n```\npip  install  -r  requirements/build.txt\n```\nInstall a few local packages (the terminal should still be in ~/Desktop/mmtracking):\n```\npip install -e src/cad\npip install -e src/mmdetection\npip install -e src/TrackEval\npip install -e src/mmclassification\npip install -e src/resource_constrained_tracking\npip install -e src/yolov7\npip install  -v  -e  .\n```\n\n## Data Preparation\n#### Sample dataset\nPlease visit the [data repository](https://drive.google.com/drive/folders/1ccdCBq1Xh9tW6CHoGbzizIo79T18nMfo?usp=drive_link) for sample data to test this repository. Due to constraints of uploading data to an anonymous google drive, we have only provided two instances of the data, good lighting (view 3) and poor lighting (view 6) under single-view and all modality conditions, for only the test data. \n\n#### Full dataset\nWe are going to release the full dataset on a later date. Check for updates at [GitHub](https://anonymous.4open.science/r/GDTM_Anonymized-4469/README.md).\n\n#### Unzip the data\n\nPlease unzip the data, rename it to **\"mcp-sample-dataset/\"** and put it on the Desktop.\nThe final data structure should be like following:\n```\n└── Desktop/mcp-sample-dataset/\n\t├── test/\n\t│   ├── node1/       \n\t│   │   ├── mmwave.hdf5\n\t│   │   ├── realsense.hdf5\n\t│   │   ├── respeaker.hdf5\n\t│   │   └── zed.hdf5\n\t│   ├── node2/\n\t│   │   └── same as node 1\n\t│   ├── node3/\n\t│   │   └── same as node 1  \n\t│   └── mocap.hdf5\n\t├── train/\n\t│   └── same as test\n\t└── val/\n\t    └── same as test\n```\nNote that you only need test/ if you are running test from checkpoints only.\n#### Specify Filepath\nOpen mmtracking/configs/\\_base\\_/datasets/one_car_early_fusion.py\nIn Line 75, Line 114, and Line 153, change the data_root to absolute path:\ne.g. ~/Desktop/... -\u003e /home/USER_NAME/Desktop/...\n\n## Code Usage\n### Make Inference Using A Checkpoint (Testing)\nPlease download the pretrained checkpoints [here](https://drive.google.com/drive/folders/17RXrXc4Qd0yKhGVMyyZxVxF9NrwolRgn?usp=sharing).\n\nNote that for single-view case (Baseline 1 in the paper), please make sure to use the checkpoints corresponding to the code and data of your choice. \n\nFor example, if we use view 3 data (single view, good lighting condition) and master branch code (single view, early fusion, all modalities), we should download \"dataset_singleview3.zip\". \n\nAfter downloading the checkpoint, please rename it to **logs/** and put it under \"mmtracking\" folder using this hierachy. \n\n```\n└── Desktop/mmtracking/\n    └── logs/\n        └── early_fusion_zed_mmwave_audio/\n            ├── val\n            ├── epoch_xx.pth\n            └── latest.pth (to be created)\n```\nwhere the \"latest.pth\" above is created by (in a terminal in early\\_fusion\\_zed\\_mmwave\\_audio\\/):\n```\nln -s epoch_40.pth latest.pth\n```\n\nThen, you could run the evaluations by running (still in terminal under ~/Desktop/mmtracking, make sure you have used \"conda activate iobt\")\n```\nbash ./tools/test_from_config_nll_local.sh ./configs/mocap/early_fusion_zed_mmwave_audio.py 1\n```\n---\n**Warning**: This script will cache the dataset in system memory (/dev/shm)\nIf the dataset loading operation was not successful, or you have changed the dataset in \"~/Desktop/mcp-sample-dataset\", please make sure to run this line **before** the \"test_from_config_nll_local.sh\" above:\n```\nrm -r /dev/shm/cache_*\n```\n---\n\n\nThe visualization results will apprear in \n```\nmmtracking/logs/early_fusion_early_fusion_zed_mmwave_audio/test_nll/latest_vid.mp4\n```\nand numerical results appears at the last two lines of \n```\nmmtracking/logs/early_fusion_early_fusion_zed_mmwave_audio/test_nll/mean.txt\n```\n\nIf you would like to train a model from scratch instead , please refer to the “training” and “scaling” sections down below.\n\n### Training\nSet up the data as instructed by previous sections, and run\n```\nbash ./tools/train_from_config_local.sh ./configs/mocap/early_fusion_zed_mmwave_audio.py 1\n```\nwhere the last digit indicate the number of GPU you have for training.\n\n### Scaling\nAfter training, some additional data is required to perform a post-hoc model recalibration as described in the paper to better capture model prediction uncertainties. More specifically, We apply an affine transformation Σ′ = aΣ + bI to the output covariance matrix Σ with parameters a and b that minimize the calibration data’s NLL. \n\nInstructions for scaling:\n```\nbash ./tools/val_from_config_local.sh ./configs/mocap/early_fusion_zed_mmwave_audio.py 1\n```\nThe last digit must be \"1\". Scaling with multiple GPU will cause an error.\n\n\n## Troubleshooting\n\nHere we list a few files to change in case some error happens during your configurations.\n#### Data not found error\nThis is where the filepath are stored\nmmtracking/configs/\\_base\\_/datasets/one_car_early_fusion.py\n\nDon't forget to do \"rm -r /dev/shm/cache_*\" after you fix this error. Otherwise a \"List out of range\" error will pop up.\n\n#### GPU OOM Error, Number of Epoches, Inteval of checkpoints\nmmtracking/configs/mocap/early_fusion_zed_mmwave_audio.py\nReduce \"samples_per_gpu\" in Line 127 helps with OOM error.\nLine 169-187 changes the training configurations.\n\nThis configuration also defines (1) the valid modalities (2) backbone, adapter, and output head architecture hyperparameters\n\n#### Something wrong with dataset caching\nmmtracking/mmtrack/datasets/mocap/cacher.py\n\n#### Something wrong with model training/inferences\nmmtracking/mmtrack/models/mocap/early_fusion.py\nFunction forward_train() for training\nFuction forward_track() for testing\n\n#### Something wrong with final visualzations\nmmtracking/mmtrack/datasets/mocap/hdf5_dataset.py\nin function write_videos()\n\n#### Backbone definitions\nmmtracking/mmtrack/models/backbones/tv_r50.py\n\n## Citation and Acknowledgements\n\n@misc{mmtrack2020,\n    title={{MMTracking: OpenMMLab} video perception toolbox and benchmark},\n    author={MMTracking Contributors},\n    howpublished = {\\url{https://github.com/open-mmlab/mmtracking}},\n    year={2020}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnesl%2Fgdtm_tracking_anonymized","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnesl%2Fgdtm_tracking_anonymized","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnesl%2Fgdtm_tracking_anonymized/lists"}