{"id":18993478,"url":"https://github.com/owen-liuyuxuan/visionfactory","last_synced_at":"2025-04-22T12:41:20.224Z","repository":{"id":197732735,"uuid":"698797889","full_name":"Owen-Liuyuxuan/visionfactory","owner":"Owen-Liuyuxuan","description":"Open source training framework for vision tasks. Scales up on data and scales up on tasks. Official Implementation for https://arxiv.org/abs/2310.00920","archived":false,"fork":false,"pushed_at":"2024-02-22T05:38:19.000Z","size":21280,"stargazers_count":38,"open_issues_count":0,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-17T01:35:39.830Z","etag":null,"topics":["3d-object-detection","bdd100k","cityscapes","kitti","monodepth","nuscenes","object-detection"],"latest_commit_sha":null,"homepage":"https://sites.google.com/view/fmaafmono3d","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Owen-Liuyuxuan.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-10-01T02:11:45.000Z","updated_at":"2025-04-10T12:42:58.000Z","dependencies_parsed_at":null,"dependency_job_id":"891d7224-aa4d-4c5b-9643-dca983ccfba1","html_url":"https://github.com/Owen-Liuyuxuan/visionfactory","commit_stats":null,"previous_names":["owen-liuyuxuan/visionfactory"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Owen-Liuyuxuan%2Fvisionfactory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Owen-Liuyuxuan%2Fvisionfactory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Owen-Liuyuxuan%2Fvisionfactory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Owen-Liuyuxuan%2Fvisionfactory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Owen-Liuyuxuan","download_url":"https://codeload.github.com/Owen-Liuyuxuan/visionfactory/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250242840,"owners_count":21398207,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-object-detection","bdd100k","cityscapes","kitti","monodepth","nuscenes","object-detection"],"created_at":"2024-11-08T17:21:33.406Z","updated_at":"2025-04-22T12:41:20.160Z","avatar_url":"https://github.com/Owen-Liuyuxuan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vision Factory\n\nThis is a general repo for training and testing vision tasks (monocular 3D detection, segmentation, monodepth and more). \n\n![image](./docs/vision_base/kitti360_all_on_gif_demo.gif)\n\nThis repo contains the training/inference/export codes to obtain the three independent network models trained with completely different strategies (Notice: ROS Nodes and Dataset bridges are in other repos). Demonstration-Ready Models are in [release page](https://github.com/Owen-Liuyuxuan/visionfactory/releases/tag/v1.0).\n\nThe general starting points/runtime backbone will be in the \"scripts\", common modules, runtime plugins and helper functions will be in \"vision_base\". \n\n**For different tasks, we may meed to overwrite parts of the dataloader / model / evaluator. Checkout [mono3d], [segmentation], [monodepth] for their settings and module overview.**\n\n**Notice For FSNet Users:** The training/testing scripts, vision_base and even the monodepth folder are basically the same as [FSNet](https://github.com/Owen-Liuyuxuan/FSNet). For monodepth tasks, for now it is completely fine to directly transferring configs/checkpoints across this two repos. We try to guarantee the backward compliance, especially in the scripts and vision_base package.\n\n**Notice For VisualDet3d Users:** This repo mainly extracts the MonoFlex model from [visualDet3D](https://github.com/Owen-Liuyuxuan/visualDet3D). However after a complete redesign in the data section for more general usage and works in the new paper, we sadly determined not to follow the original design and merge with the [FSNet](https://github.com/Owen-Liuyuxuan/FSNet) standard.\n\nIn this main page, we will focus on the general usage of vision_base.\n\n## (New) Launching Web Demo\n\nWe introduced ways to demonstrate the result of the pretrained ONNX models. (So that we could check results with any data at hand)\n\nThe onnxmodels are created with the export scripts. And we provide pretrained models in [release page of inference](https://github.com/Owen-Liuyuxuan/ros2_vision_inference/releases).\n\nInstall `streamlit` and run the following command from the base directory:\n```\npython3 -m streamlit run demos/web_demo.py --server.fileWatcherType none\n```\n\n![Alt text](docs/web_demo_image.png)\n\nWe could upload any images and check the visualized results. If we do not indicate the camera parameters, the web-page will make a pseudo camera parameter for the demo usage.\n\nThe pretrained model is trained on road scenes so we could expect the potential and limitation of the models.\n\n## Launching ROS Demo\n\nCheck [this repo](https://github.com/Owen-Liuyuxuan/ros2_vision_inference) for launching inference demo in ROS1/ROS2.\n\n## Pre-Use\n\nWe recommend torch \u003e 1.10 or torch 2.0 with the corresponding CUDA versions. run \n\n```\n./make.sh\n```\nThis operation will build the dcn module inplace and does not install anything.\n\n**Common Notice**: Make sure the nvcc version (```nvcc --version```) and the pytorch cuda version (```torch.version.cuda```) aligned.\n\n## General Usage\n\nStart **Everything** in the base directory of this repo. Please check out each tasks([mono3d]/[segmentation]/[monodepth]) for dataset preparation.\n\nTraining:\n\n```\n### multi-gpu training\n./launcher/train.sh $CONFIG_PATH $COMMA_GPULIST $EXPERIMENT_LOGGING_NAME\n\n### single-gpu training / debugging\npython3 scripts/train.py $CONFIG_PATH $EXPERIMENT_LOGGING_NAME\n```\n\nSingle testing:\n```\npython3 scripts/test.py $CONFIG_PATH $SINGLE_GPU $CHECKPOINT\n```\n\n## Check the Logging\n \nBy default, the train scripts and vision_base will create a detailed tensorflow report besides the logging from specific packages. The logging will be in ```./workdirs/\\$NAME_IN_CONFIG/log/${EXPERIMENT_LOGGING_NAME}config=${CONFIG_NAME}```, we suggest tensorboard open in ```workdirs/$NAME_IN_CONFIG/log/``` or ```./workdirs/```.\n\nThe tensorboard report will include the **full expanded** config file \n\n![image](./docs/vision_base/tensorboard_config.png)\n\nand the git changes / git commits\n\n![image](./docs/vision_base/tensorboard_git.png)\n\nwith these info, it is easier to reproduce the experiments when we are tuning parameters in config or modifying codes.\n\n## Tips for Developing\n\nIn general, we recommend not making breaking change in vision_base (adding more functionalities is generally fine, and extrely welcome for features/bugs report).\n\nIf we are starting a new project with new data settings / new models / new strategies, it is recommended to start a new folder and overwrite existing classes, and write config files pointing to new modules. This is achieved by [builder](./vision_base/utils/builder.py) and [find_object](./vision_base/utils/utils.py).\n\nCheck the codes in [segmenatation](./segmentation/) for how to start a simple new project.\n\nCheck the codes in [mono3d](./mono3d/) for how to overwrite data pipeline for complex tasks.\n\n## Further Info and Bug Issues\n\n1. Open issues on the repo if you meet troubles or find a bug or have some suggestions.\n2. Directly Email to yliuhb@connect.ust.hk because the guy may not see email in the github-linked mailbox.\n\n[mono3d]:docs/mono3d/readme.md\n[segmentation]:docs/segmentation/readme.md\n[monodepth]:docs/monodepth/readme.md","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fowen-liuyuxuan%2Fvisionfactory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fowen-liuyuxuan%2Fvisionfactory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fowen-liuyuxuan%2Fvisionfactory/lists"}