{"id":25452621,"url":"https://github.com/ruipreis/terrain-dreamer","last_synced_at":"2025-11-04T21:03:57.047Z","repository":{"id":240204349,"uuid":"609299452","full_name":"ruipreis/terrain-dreamer","owner":"ruipreis","description":"Package for seamless 3D terrain generation using inpainting models, GANs, DEMs, and RGB satellite imagery.","archived":false,"fork":false,"pushed_at":"2024-05-17T07:55:23.000Z","size":59959,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-17T23:41:38.537Z","etag":null,"topics":["digital-elevation-model","gans","image-inpainting","procedural-generation","qdrant","satellite-imagery","terrain-generation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ruipreis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-03T20:05:18.000Z","updated_at":"2024-11-27T16:13:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"1e017f9d-e919-48b0-b091-c95649020095","html_url":"https://github.com/ruipreis/terrain-dreamer","commit_stats":null,"previous_names":["ruipreis/terrain-dreamer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruipreis%2Fterrain-dreamer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruipreis%2Fterrain-dreamer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruipreis%2Fterrain-dreamer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruipreis%2Fterrain-dreamer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ruipreis","download_url":"https://codeload.github.com/ruipreis/terrain-dreamer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254535825,"owners_count":22087399,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["digital-elevation-model","gans","image-inpainting","procedural-generation","qdrant","satellite-imagery","terrain-generation"],"created_at":"2025-02-17T23:40:54.525Z","updated_at":"2025-11-04T21:03:57.004Z","avatar_url":"https://github.com/ruipreis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Terrain-Dreamer: Procedural Terrain Generation\n\n\n![banner](docs/banner.jpg)\n\n**Terrain-Dreamer** is a novel approach to procedural terrain generation that places key tiles on a tilemap and fills in the gaps using inpainting models. This allows for smooth transitions between different biomes.\n\n## 🛠️ Requirements\n\n- Clone the repository and install it as a package:\n  ```bash\n  git clone https://github.com/ruipreis/terrain-dreamer.git\n  cd terrain-dreamer\n  pip install .\n  ```\n- Download the available checkpoints if you want to experiment without training the models from scratch.\n- For manually downloading and processing the original dataset, ensure you have access to a service account with Google Earth API access.\n\n## 🌍 Dataset\n\nThe ALOS World 3D - 30m (AW3D30) is a global Digital Elevation Model (DEM) produced by the Japan Aerospace Exploration Agency (JAXA), offering 30-meter resolution digital surface and terrain models worldwide.\n\n### Downloading the Dataset\n\nYou can download the dataset from the original source or use our prepared dataset with satellite and depth data in NPZ files, split into training and testing sets. The prepared dataset is available [here](https://storage.googleapis.com/terrain-generation-models/AW3D30.zip).\n\n### Preparing the Dataset\n\nTo download the dataset from the official source:\n- Visit the [AW3D30 data page](https://www.eorc.jaxa.jp/ALOS/en/aw3d30/data/index.htm).\n- Example download link for the area between N075W030 and N080W025: [N075W030_N080W025.zip](https://www.eorc.jaxa.jp/ALOS/aw3d30/data/release_v2012/N075W030_N080W025.zip).\n\nAlternatively, use the following script to automatically download AW3D30 data and obtain corresponding satellite imagery from the Google Earth API:\n1. Download the AW3D30 data:\n    ```bash\n    python terrdreamer/dataset/aw3d30.py\n    ```\n2. Obtain matching satellite imagery:\n    ```bash\n    python terrdreamer/dataset/gearth.py\n    ```\n\n### Documentation\n\nFor more information about the AW3D30 dataset, refer to the official [documentation](https://www.eorc.jaxa.jp/ALOS/en/aw3d30/aw3d30v3.2_product_e_e1.0.pdf).\n\n\n## 🧠 Models\n\nWe use four models to achieve our results:\n- **image-to-dem**: A [pix2pix](https://arxiv.org/abs/1611.07004) Conditional GAN that converts RGB data to a DEM.\n- **dem-to-image**: A pix2pix model that converts DEM data to RGB data.\n- **image generation**: A [ProGAN](https://arxiv.org/abs/1710.10196) to generate novel satellite imagery.\n- **inpainting**: An inpainting model based on [Generative Image Inpainting with Contextual Attention](https://arxiv.org/abs/1801.07892), used to predict masked parts of satellite data.\n\n## 🎓 Training\n\nTraining scripts use `wandb` for experiment tracking. Ensure `wandb` is installed and configured.\n\n### Inpainting Model Training\n\nTo train the inpainting model:\n```bash\npython terrdreamer/models/infinity_grid/train.py \\\n    --train-dataset aw3d30/train \\\n    --test-dataset aw3d30/test \\\n    --limit 10000 \\\n    --save-model-path checkpoints/inpainting \\\n    --wandb-project inpainting\n```\n\n![inpainting](docs/epoch_200.png)\n\n### Checkpoints\n\nDownload checkpoints for all models:\n```bash\nmkdir -p checkpoints\nwget https://storage.googleapis.com/terrain-generation-models/checkpoints.zip -P checkpoints\nunzip checkpoints/checkpoints.zip -d checkpoints\n```\n\n## 🖼️ Inference\n\nTo generate a tile map:\n1. Initialize a Qdrant container:\n    ```bash\n    docker pull qdrant/qdrant\n    docker run -p 6333:6333 qdrant/qdrant\n    ```\n2. Populate the Qdrant vector database with randomly generated satellite imagery:\n    ```bash\n    python terrdreamer/grid/__init__.py\n    ```\n3. Create real and fake tiles:\n    ```bash\n    python terrdreamer/grid/generate.py --height 10 --width 50\n    ```\n4. Interpolate between placed tiles:\n    ```bash\n    python terrdreamer/grid/interpolation.py --height 10 --width 50\n    ```\n5. Fill gaps with inpainting:\n    ```bash\n    python terrdreamer/grid/filling.py --height 10 --width 50\n    ```\n6. Estimate depth for image tiles:\n    ```bash\n    python terrdreamer/grid/depth.py\n    ```\n7. Convert to image format:\n    ```bash\n    python terrdreamer/grid/to_image.py\n    ```\n\n## 🌟 Inspiration\n\nThis project is inspired by the following works by [Emmanouil Panagiotou](https://github.com/Panagiotou):\n- **image-to-dem**: [Generating Elevation Surface from a Single RGB Remotely Sensed Image Using Deep Learning](https://github.com/Panagiotou/ImageToDEM).\n- **dem-to-image**: [Procedural 3D Terrain Generation using Generative Adversarial Networks](https://github.com/Panagiotou/Procedural3DTerrain).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruipreis%2Fterrain-dreamer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fruipreis%2Fterrain-dreamer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruipreis%2Fterrain-dreamer/lists"}