{"id":22869846,"url":"https://github.com/choyingw/meta-initialized-depth","last_synced_at":"2026-04-28T11:32:06.867Z","repository":{"id":266956029,"uuid":"899864584","full_name":"choyingw/Meta-Initialized-Depth","owner":"choyingw","description":"[IROS 2024] Boosting Generalizability towards Zero-Shot Cross-Dataset Single-Image Indoor Depth by Meta-Initialization","archived":false,"fork":false,"pushed_at":"2024-12-07T08:16:33.000Z","size":3,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-06T16:57:18.196Z","etag":null,"topics":["3d","computer-vision","deep-learning","depth-estimation","iros","machine-learning","meta-learning","navigation","robotics"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/choyingw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-07T08:07:23.000Z","updated_at":"2024-12-17T08:55:49.000Z","dependencies_parsed_at":"2024-12-07T09:19:20.492Z","dependency_job_id":"c61f23ce-b236-43aa-8891-fcd478a4b731","html_url":"https://github.com/choyingw/Meta-Initialized-Depth","commit_stats":null,"previous_names":["choyingw/meta-initialized-depth"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FMeta-Initialized-Depth","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FMeta-Initialized-Depth/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FMeta-Initialized-Depth/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FMeta-Initialized-Depth/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/choyingw","download_url":"https://codeload.github.com/choyingw/Meta-Initialized-Depth/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246459899,"owners_count":20781009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","computer-vision","deep-learning","depth-estimation","iros","machine-learning","meta-learning","navigation","robotics"],"created_at":"2024-12-13T13:12:03.194Z","updated_at":"2026-04-28T11:32:01.837Z","avatar_url":"https://github.com/choyingw.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# [IROS 2024] Meta-Initialized-Depth\n[IROS 2024] Boosting Generalizability towards Zero-Shot Cross-Dataset Single-Image Indoor Depth by Meta-Initialization\n\n## :dolphin: The following shows the main recipe for training meta-initialization for depth estimation\n\n```python\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nimport copy\nfrom timm.models import create_model\nimport argparse\n\n\nclass SmoothLoss(nn.Module):\n    def __init__(self):\n        super(SmoothLoss, self).__init__()\n\n    def forward(self, pred_map):\n        def gradient(pred):\n            D_dy = pred[:, :, 1:] - pred[:, :, :-1]\n            D_dx = pred[:, :, :, 1:] - pred[:, :, :, :-1]\n            return D_dx, D_dy\n\n        if type(pred_map) not in [tuple, list]:\n            pred_map = [pred_map]\n\n        loss = 0\n        weight = 1.\n\n        for scaled_map in pred_map:\n            dx, dy = gradient(scaled_map)\n            dx2, dxdy = gradient(dx)\n            dydx, dy2 = gradient(dy)\n            loss += (dx2.abs().mean() + dxdy.abs().mean() + dydx.abs().mean() + dy2.abs().mean())*weight\n            weight /= 2.3  # don't ask me why it works better\n        return loss\n\nclass Trainer:\n    def __init__(self, options):        \n        self.opt = options\n        self.models[\"encoder\"] = create_model('convnext_base', pretrained=True)\n        self.models[\"encoder\"] = create_model('convnext_small', pretrained=True)\n        self.models[\"encoder\"].to(self.device)\n        self.parameters_to_train += list(self.models[\"encoder\"].parameters())\n\n        self.models[\"decoder\"] = DepthDecoder(...)\n        self.models[\"decoder\"].to(self.device)\n        self.parameters_to_train += list(self.models[\"depth\"].parameters())\n        self.model_optimizer = optim.SGD(self.parameters_to_train, self.opt.learning_rate)\n        self.sup_model_optimizer = optim.Adam(self.parameters_to_train, self.opt.sup_learning_rate)\n        self.sup_model_lr_scheduler = optim.lr_scheduler.CosineAnnealingLR(\n            self.sup_model_optimizer, self.opt.iterations, eta_min=1e-6) \n        self.mse_loss = torch.nn.MSELoss()\n        self.smoothness_loss = SmoothLoss()\n\n    def inner_loop(self, inner_encoder, inner_depth, optim, inner_steps, imgs=None, gt=None):\n        \"\"\"\n        train the inner model for a specified number of iterations\n        \"\"\"\n\n        for step in range(inner_steps):\n            # Your dataloader\n            inputs = next(self.train_dataloader_iter)\n            imgs = inputs[\"image\"].cuda()\n            gt = inputs[\"depth_gt\"].cuda()\n\n            optim.zero_grad()\n            features = inner_encoder.forward_features(imgs)\n            depth = inner_depth(features)\n            loss = self.mse_loss(depth, gt) + 0.001 * self.smoothness_loss(depth)\n            loss.backward()\n            optim.step()\n\n    def run_epoch_reptile(self):\n        self.model_optimizer.zero_grad()\n        inner_encoder = copy.deepcopy(self.models[\"encoder\"])\n        inner_depth = copy.deepcopy(self.models[\"decoder\"])\n        inner_optim = torch.optim.SGD(list(inner_encoder.parameters())+list(inner_depth.parameters()), self.opt.inner_lr)\n        torch.nn.utils.clip_grad_value_(list(inner_encoder.parameters())+list(inner_depth.parameters()), 1.0)\n        self.inner_loop(inner_encoder, inner_depth, inner_optim, self.opt.inner_steps)\n\n        with torch.no_grad():\n            for meta_param_enc, inner_param_enc in zip(self.models[\"encoder\"].parameters(), inner_encoder.parameters()):\n                meta_param_enc.grad = meta_param_enc - inner_param_enc\n            for meta_param_dec, inner_param_dec in zip(self.models[\"decoder\"].parameters(), inner_depth.parameters()):\n                meta_param_dec.grad = meta_param_dec - inner_param_dec\n\n        self.model_optimizer.step()\n\n    def train(self):\n        # meta-initialization\n        self.run_epoch_reptile()\n        # regular supervised learning\n        self.run_epoch_supervise()\n\nif __name__ == \"__main__\":\n    parser = argparse.ArgumentParser(description=\"Meta-Initialization\")\n    parser.add_argument(\"--learning_rate\",\n                                type=float,\n                                help=\"learning rate\",\n                                default=1e-1)\n    parser.add_argument(\"--inner_lr\",\n                                type=float,\n                                help=\"inner learning rate\",\n                                default=1e-3)\n    parser.add_argument(\"--inner_steps\",\n                                type=int,\n                                help=\"inner_steps\",\n                                default=4)\n    parser.add_argument(\"--sup_learning_rate\",\n                                 type=float,\n                                 help=\"learning rate\",\n                                 default=5e-4)\n    # ...... (Other options for your training recipe) \n    opts = parser.parse()\n\n    trainer = Trainer(opts)\n    trainer.train()\n\n```\n\n## \u003cdiv align=\"\"\u003eCitation\u003c/div\u003e\n\n    @inproceedings{wu2024boosting,\n        title={Boosting Generalizability towards Zero-Shot Cross-Dataset Single-Image Indoor Depth by Meta-Initialization},\n        author={Wu, Cho-Ying and Zhong, Yiqi and Wang, Junying and Neumann, Ulrich},\n        booktitle={2024 IEEE/RSJ international conference on    intelligent robots and systems (IROS)},\n        year={2024},\n        organization={IEEE}\n        }","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchoyingw%2Fmeta-initialized-depth","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchoyingw%2Fmeta-initialized-depth","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchoyingw%2Fmeta-initialized-depth/lists"}