https://github.com/sshh12/terrain-diffusion-app
An infinite collaborative inpainter which allows users to dynamic generate satellite-realistic map tiles.
https://github.com/sshh12/terrain-diffusion-app
ably image-editor inpainting procedural-generation sentinel-2 stable-diffusion world-building
Last synced: about 1 month ago
JSON representation
An infinite collaborative inpainter which allows users to dynamic generate satellite-realistic map tiles.
- Host: GitHub
- URL: https://github.com/sshh12/terrain-diffusion-app
- Owner: sshh12
- License: mit
- Created: 2023-08-22T00:33:06.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-12-07T19:03:52.000Z (10 months ago)
- Last Synced: 2025-09-08T00:02:12.988Z (about 1 month ago)
- Topics: ably, image-editor, inpainting, procedural-generation, sentinel-2, stable-diffusion, world-building
- Language: JavaScript
- Homepage: https://terrain.sshh.io
- Size: 370 KB
- Stars: 6
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# terrain-diffusion-app
> AKA This Map Does Not Exist, is an infinite collaborative inpainter which allows users to dynamic generate satellite-realistic map tiles.
## How it works
The app is pretty much serverless with [Ably](https://ably.io/) doing most of the live networking and communication between the client and the GPU worker. The GPU worker writes directly to AWS s3 and then notifies the client to re-download them. Netlify is mainly there for static hosting but a cloud function is used to init the Ably websocket.
A slight later modification was to replace the GPU worker with a serverless modal app.
#### The GPU Endpoint
The endpoint listens for generation requests and then:
1. Translates an x, y coordinate into the (up to 4) 512x512 tiles that will need to be modified
2. Download these tiles from s3 (if s3 doesn't have it just generate a blank image)
3. Generate a mask based on any areas that are blank
4. Run this through the stable diffusion inpainting model
5. Update all the tiles and re-upload to s3
6. Broadcast which tiles were updated
### The Canvas
The canvas was written from scratch to support mobile and desktop without too much weirdness. `react-canvas-draw` was used heavily as a reference. It includes minimal optimizations for processing tiles besides lazy loading based on the client's viewport and background image rendering.
## Model
For more information on training the model see https://github.com/sshh12/terrain-diffusion.
## Self-Hosting
At a high level:
1. Create a netlify static app from this repo
2. Create an app on https://ably.io/ (free tier works) and set the environment variable `ABLY_API_KEY`
3. Create an https://modal.com/ app and `cd modal && modal deploy modal_app`
Feel free to create an issue if you want help setting this up. This app should work fairly seamlessly for any [diffusers](https://huggingface.co/docs/diffusers/index) model.