{"id":21229130,"url":"https://github.com/marian42/shapegan","last_synced_at":"2025-08-01T19:05:35.124Z","repository":{"id":48162225,"uuid":"181573173","full_name":"marian42/shapegan","owner":"marian42","description":"Generative Adversarial Networks and Autoencoders for 3D Shapes","archived":false,"fork":false,"pushed_at":"2021-10-09T08:12:37.000Z","size":26930,"stargazers_count":319,"open_issues_count":16,"forks_count":56,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-05-19T21:05:20.882Z","etag":null,"topics":["3d","autoencoder","gan","generative-adversarial-network","machine-learning","pytorch","sdf"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2002.00349","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marian42.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-15T22:15:19.000Z","updated_at":"2025-04-24T00:50:45.000Z","dependencies_parsed_at":"2022-08-30T07:50:33.668Z","dependency_job_id":null,"html_url":"https://github.com/marian42/shapegan","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/marian42/shapegan","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marian42%2Fshapegan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marian42%2Fshapegan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marian42%2Fshapegan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marian42%2Fshapegan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marian42","download_url":"https://codeload.github.com/marian42/shapegan/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marian42%2Fshapegan/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268281821,"owners_count":24225159,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-01T02:00:08.611Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","autoencoder","gan","generative-adversarial-network","machine-learning","pytorch","sdf"],"created_at":"2024-11-20T23:24:51.311Z","updated_at":"2025-08-01T19:05:35.088Z","avatar_url":"https://github.com/marian42.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Generative Adversarial Networks and Autoencoders for 3D Shapes\n\n![Shapes generated with our propsed GAN architecture and reconstructed using Marching Cubes](https://raw.githubusercontent.com/marian42/shapegan/master/examples/gan_shapes.png)\n\nThis repository provides code for the paper \"[Adversarial Generation of Continuous Implicit Shape\nRepresentations](https://arxiv.org/abs/2002.00349)\" and for my master thesis about generative machine learning models for 3D shapes.\nIt contains:\n\n- the networks proposed in the paper (GANs with a DeepSDF network as the generator and a 3D CNN or Pointnet as discriminator)\n- an autoencoder, variational autoencoder and GANs for SDF voxel volumes using [3D CNNs](http://papers.nips.cc/paper/6096-learning-a-probabilistic-latent-space-of-object-shapes-via-3d-generative-adversarial-modeling.pdf)\n- an implementation of [the DeepSDF autodecoder](https://arxiv.org/pdf/1901.05103.pdf) that learns implicit function representations of 3D shapes\n- a GAN that uses a DeepSDF network as the generator and a 3D CNN as the discriminator (\"Hybrid GAN\", as proposed in the paper, but without progressive growing and without gradient penalty)\n- a data prepration pipeline that can prepare SDF datasets from triangle meshes, such as the [Shapenet dataset](https://www.shapenet.org/) (based on my [mesh_to_sdf](https://github.com/marian42/mesh_to_sdf) project)\n- a [ray marching](http://jamie-wong.com/2016/07/15/ray-marching-signed-distance-functions/) renderer to render signed distance fields given by a neural network, as well as a classic rasterized renderer to render triangle meshes reconstructed with Marching Cubes\n- tools to visualize the results\n\nNote that although the code provided here works, most of the scripts need some configuration to work for a specific task.\n\nThis project uses two different ways to represent 3D shapes.\nThese representations are *voxel volumes* and *implicit functions*.\nBoth use [signed distances](https://en.wikipedia.org/wiki/Signed_distance_function).\n\nFor both representations, there are networks that learn latent embeddings and then reconstruct objects from latent codes.\nThese are the *autoencoder* and *variational autoencoder* for voxel volumes and the [*autodecoder* for the DeepSDF network](https://arxiv.org/pdf/1901.05103.pdf).\n\nIn addition, for both representations, there are *generative adversarial networks* that learn to generate novel objects from random latent codes.\nThe GANs come in a classic and a Wasserstein flavor.\n\n\n# Reproducing the paper\n\nThis section explains how to reproduce the paper \"Generative Adversarial Networks and Autoencoders for 3D Shapes\".\n\n## Data preparation\n\nTo train the model, the meshes in the Shapenet dataset need to be voxelized for the voxel-based approach and converted to SDF point clouds for the point based approach.\n\nWe provide readily prepared datasets for the Chairs, Airplanes and Sofas categories of Shapenet as a [download](https://ls7-data.cs.tu-dortmund.de/shape_net/ShapeNet_SDF.tar.gz).\nThe size of that dataset is 71 GB.\n\nTo prepare the data yourself, follow these steps:\n\n1. install the `mesh_to_sdf` pip module.\n2. Download the Shapenet files to the `data/shapenet/` directory or create an equivalent symlink.\n3. Review the settings at the top of `prepare_shapenet_dataset.py`.\nThe default settings are configured for reproducing the GAN paper, so you shouldn't need to change anything.\nYou can change the dataset category that will be prepared, the default is the chairs category.\nYou can disable preparation of either the voxel or point datasets if you don't need both.\n4. Run `prepare_shapenet_dataset.py`.\nYou can stop and resume this script and it will continue where it left off.\n\n## Training\n\n### Voxel-based discriminator\n\nTo train the GAN with the 3D CNN discriminator, run\n\n    python3 train_hybrid_progressive_gan.py iteration=0\n    python3 train_hybrid_progressive_gan.py iteration=1\n    python3 train_hybrid_progressive_gan.py iteration=2\n    python3 train_hybrid_progressive_gan.py iteration=3\n\nThis runs the four steps of progressive growing.\nEach iteration will start with the result of the previous iteration or the most recent result of the current iteration if the \"continue\" parameter is supplied.\nAdd the `nogui` parameter to disable the model viewer during training.\nThis parameter should be used when the script is run remotely.\n\n### Point-based discriminator\n\nTODO\n\nNote that the pointnet-based approach currently has a separate implementation of the generator and doesn't work with the visualization scripts provided here.\nThe two implementations will be merged soon so that the demos work.\n\n## Use pretrained generator models\n\nIn the `examples` directory, you find network parameters for the GAN generators trained on chairs, airplanes and sofas with the 3D CNN discriminator.\nYou can use these by loading the generator from these files, i.e. in `demo_sdf_net.py` you can change `sdf_net.filename` accordingly.\n\nTODO: Examples for the pointnet-based GANs will be added soon.\n\n# Running other 3D deep learning models\n\n## Data preparation\n\nTwo data preparation scripts are available, `prepare_shapenet_dataset.py` is configured to work specifically with the Shapenet dataset.\n`prepare_data.py` can be used with any folder of 3D meshes.\nBoth need to be configured depending on what data you want to prepare.\nMost of the time, not all types of data need to be prepared.\nFor the DeepSDF network, you need SDF clouds.\nFor the remaining networks, you need voxels of resolution 32.\nThe \"uniform\" and \"surface\" datasets, as well as the voxels of other resolutions are only needed for the GAN paper (see the section above).\n\n## Training\n\nRun any of the scripts that start with `train_` to train the networks.\nThe `train_autoencoder.py` trains the variational autoencoder, unless the `classic` argument is supplied.\nAll training scripts take these command line arguments:\n- `continue` to load existing parameters\n- `nogui`  to not show the model viewer, which is useful for VMs\n- `show_slice` to show a text representation of the learned shape\n\nProgress is saved after each epoch.\nThere is no stopping criterion.\nThe longer you train, the better the result.\nYou should have at least 8GB of GPU RAM available.\nUse a datacenter GPU, training on a desktop GPU will take several days to get good results.\nThe classifiers take the least time to train and the GANs take the most time.\n\n## Visualization\n\nTo visualize the results, run any of the scripts starting with `demo_`.\nThey might need to be configured depending on the model that was trained and the visualizations needed.\nThe `create_plot.py` contains code to generate figures for my thesis.\n\n## Using the pretrained DeepSDF model and recreating the latent space traversal animation\n\nThis section explains how get a DeepSDF network model that was pre-trained on the Shapenet dataset and how to use it to recreate [this latent space traversal animation](https://twitter.com/marian42_/status/1188969971898048512).\n\nSince the model was trained, some network parameters have changed.\nIf you're training a new model, you can use the parameters on the master branch and it will work as well.\nTo be compatible with the pretrained model, you'll need the changes in the [`pretrained-deepsdf-shapenet`](https://github.com/marian42/shapegan/tree/pretrained-deepsdf-shapenet) branch.\n\nTo generate the latent space animation, follow these steps:\n\n1. Switch to the the [`pretrained-deepsdf-shapenet`](https://github.com/marian42/shapegan/tree/pretrained-deepsdf-shapenet) branch.\n\n2. Move the contents of the `examples/deepsdf-shapenet-pretrained` directory to the project root directory.\nThe scripts will look for the .to files in `/models` and `/data` relative to the project root.\n\n3. Run `python3 demo_latent_space.py`.\nThis takes about 40 minutes on my machine.\nTo make it faster, you can lower the values of `SAMPLE_COUNT` and `TRANSITION_FRAMES` in `demo_latent_space.py`.\n\n4. To render a video file from the frames, run `ffmpeg -framerate 30 -i images/frame-%05d.png -c:v libx264 -profile:v high -crf 19 -pix_fmt yuv420p video.mp4`.\n\nNote that after completing steps 1 and 2, you can run `python3 demo_sdf_net.py` to show a realtime latent space interpolation.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarian42%2Fshapegan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarian42%2Fshapegan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarian42%2Fshapegan/lists"}