{"id":20401937,"url":"https://github.com/jmmanley/forms","last_synced_at":"2025-07-22T18:35:16.659Z","repository":{"id":111674709,"uuid":"124268486","full_name":"jmmanley/forms","owner":"jmmanley","description":"Flexible Object Reconstruction from Multiple Silhouettes. Cashman \u0026 Fitzgibbon, 'What Shape are Dolphins? Building 3D Morphable Models from 2D Images', 2012. Updated 2018, Jason Manley.","archived":false,"fork":false,"pushed_at":"2018-03-07T17:16:11.000Z","size":1360,"stargazers_count":7,"open_issues_count":0,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-05T01:43:51.359Z","etag":null,"topics":["3d-models","3d-reconstruction","dolphin","morphable-model","silhouette-reconstruction"],"latest_commit_sha":null,"homepage":"","language":"Matlab","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jmmanley.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-03-07T17:08:38.000Z","updated_at":"2022-04-07T17:15:25.000Z","dependencies_parsed_at":"2023-05-25T10:00:22.267Z","dependency_job_id":null,"html_url":"https://github.com/jmmanley/forms","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jmmanley/forms","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmmanley%2Fforms","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmmanley%2Fforms/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmmanley%2Fforms/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmmanley%2Fforms/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jmmanley","download_url":"https://codeload.github.com/jmmanley/forms/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmmanley%2Fforms/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266552738,"owners_count":23947183,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-models","3d-reconstruction","dolphin","morphable-model","silhouette-reconstruction"],"created_at":"2024-11-15T04:52:28.511Z","updated_at":"2025-07-22T18:35:16.631Z","avatar_url":"https://github.com/jmmanley.png","language":"Matlab","readme":"# Forms: Flexible Object Reconstruction from Multiple Silhouettes\n\n### Thomas J. Cashman, University of Lugano\n### Andrew W. Fitzgibbon, Microsoft Research, Cambridge\n\n### Updates by Jason M. Manley, Rockefeller University\n### Questions? jmanley@rockefeller.edu\n\nSee the bottom for the original documentation provided by Cashman \u0026 Fitzgibbon\nto reproduce the results from their 2012 paper (http://ieeexplore.ieee.org/document/6165306/).\nThe dolphin model produced by running the code described in their README.txt is given in _fitzgibbonModel.mat_.\n\nThis code was written and tested in *MATLAB R2017a* and requires the *Optimization Toolbox*.\n\nOriginal source code retrieved from https://archive.codeplex.com/?p=forms.\n\n\n## What do I do?\n\n*Forms* uses a procedure developed by Cashman \u0026 Fitzgibbon to fit a 3D morphable\nmodel to a collection of 2D images showing some type of object in its various poses\nand conformations. This fitting is formulated as an energy minimization problem,\nand the model represents the object as a linear combination of subdivision surfaces.\n\n\n## Summary of updates from the original source code\n\n- _set_initial_conditions.m_ GUI for easily inputting constraints and initial orientations for new datasets.\n- Reformatting the _forms.m_ output as a structure (instead of a vector with very careful indexing).\n- Once a model has been built, _fit_model_to_image.m_ allows fitting a model to another image outside the original input set.\n- _calculate_shapediff.m_ provides a potential metric for how well the model has fit the image's silhouette.\n\n\n## Example scripts\n\nSee _run_forms_bananas_dolphins.m_ for fitting the model to the example data provided by Cashman \u0026 Fitzgibbon.\n\nSee _run_forms_new_images.m_ for curating new images and fitting the model to a new dataset.\n\n\n## How to structure input data\n\nThis algorithm requires a set of 2D images, in which the desired object has\nbeen segmented. While Cashman \u0026 Fitzgibbon stored their data in .ply and .fpj\nfiles (as seen in forms/projects), we have opted to simply store our data in\nstructure arrays within .mat files. This is referred to as a Forms 'project'.\n\n\n#### Each project should contain the following structure fields:\n\n_project.vertices_    -\u003e a Nx3 matrix specifying the N vertices of the pre-defined\n                         mesh model\n\n_project.mesh_        -\u003e a meshtri (see Cashman \u0026 Fitzgibbon's class definition)\n\n_project.faces_       -\u003e a Nfx3 matrix specifying the indices of vertices which\n                         form the Nf triangular faces of the pre-defined mesh model\n\n_project.cand_ixs_    -\u003e\n\n_project.cand_uvs_    -\u003e\n\n_project.cand_limits_ -\u003e\n\n_project.cand_dists_  -\u003e\n\n_project.cand_derivs_ -\u003e\n\n\nFor our analyses to date, the above fields have been the same as those utilized\nin Cashman \u0026 Fitzgibbon's dolphin analysis (see _fitzgibbonModel.mat_). The following\nfields should be specified for future analyses of new dolphins datasets:\n\n_project.images_      -\u003e a Nx1 structure array containing the input dataset for\n                         N input images\n\n_project.parameters_  -\u003e a structure array containing the parameters for running\n                         the model fitting in forms.m. See makeModelParameters.m.\n\n\n_project.images_ should contain the following fields for the i-th image project.images(i):\n\n\n#### The following fields must be directly specified by the user.\n\n_image_            -\u003e the mxn(x3) original image\n\n_silhouette_       -\u003e a Nx2 matrix containing the 2D location of N points along\n                      the object's silhouette\n\n_frame_            -\u003e the frame index at which this image occurs in the movie\n\n_movie_            -\u003e the path to this frame's original movie\n\n\n#### The following fields will be found using _set_initial_conditions.m_.\n\n_constraints3d_    -\u003e a Nx1 vector containing the indices of the N contraint points\n                      on the 3D mesh model. This index refers to an actual 3D location\n                      given in project.vertices.\n\n_constraints2d_    -\u003e a Nx2 matrix containing the 2D locations of the N constraint\n                      points on the image, such that constraints3d(i) corresponds to\n                      the location of constraints2d(i,:) on the model.\n\n_constraintsonsil_ -\u003e a Nx1 binary vector describing whether is constraint point is\n                      on (1) or off (0) the silhouette.\n\n_normalsLeft_      -\u003e a Nx1 binary vector describing whether the normal vectors\n                      should be flipped (0) or not (1).\n\n_rotate_           -\u003e a rotation matrix describing the approximate orientation\n                      of the dolphin for the model's initial conditions. This is\n                      specified by hand using set_initial_conditions.m.\n\n\n#### The following fields will be automatically found using the pipeline below.\n\n_translate_        -\u003e a translation matrix for aligning the 3D model with the\n                      2D silhouette, found using align_project.m.\n\n_scale_            -\u003e a dilation matrix for aligning the 3D model with the\n                      2D silhouette, found using align_project.m.\n\n_transform_        -\u003e the full transformation matrix, such that\n                      transform = rotate * translate * scale.\n\n_points_           -\u003e a 4x(N/4)x2 matrix containing a reshaped version of the\n                      silhouette, in order to be compatible with forms.m. % Why?\n\n\n## How to fit a model to the input data\n\nOnce you have initialized a project with a mesh structure, images, and silhouettes,\nfollow the pipeline below to find a 3D morphable model for your object.\n\nFirstly, the initial conditions must be specified by running the following\nfor each of the images i=1:length(project.images):\n\n\u003e set_initial_conditions(project,i)\n\nThis will open a figure and guide you through setting the initial orientation\nof the object (using the angle sliders), selecting the constraint points, and\nproperly orienting the silhouette normal vectors. Note: this also automatically\nruns the alignment code \u003e align_project(project).\n\nIf you choose to use the default parameters, the following creates the\nparameter structure:\n\n\u003e parameters = makeModelParameters;   % see makeModelParameters or Cashman \u0026 Fitzgibbon's original paper for a description of these parameters\n\nThe model can then by found by running:\n\n\u003e model = forms(project, parameters);\n\nThe fit of the model for image i can the be visualized by running:\n\n\u003e plot_modelfit(project, i, model);\n\nThe norm of the ith basis shape can be plotted on the 0th mode by running:\n\n\u003e plot_basisshapenorm(project, model, i);\n\n\nSee Cashman \u0026 Fitzgibbon's original README below. Note that the forms\nfunction has been modified in order to store the model in a structure, rather\nthan in a carefully indexed single vector. The function _archive/convert_modelVec_to_modelStruct_\ncan be utilized to convert the vector version of a model to the structure\nversion of a model that is utilized here. In addition, the parameters inputs\nhave been modified to be stored in a structure.\n\n\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nForms: Flexible Object Reconstruction from Multiple Silhouettes\n\nThomas J. Cashman          Andrew W. Fitzgibbon\nUniversity of Lugano       Microsoft Research, Cambridge\n\n\nThis MATLAB code gives the implementation for our paper 'What Shape are\nDolphins? Building 3D Morphable Models from 2D Images'. You will need the\nMATLAB Optimization Toolbox to run it.\n\n\nTo try the code on one of our datasets, load a Forms 'project', and then use the\nfunction 'forms' to run our optimization. For example:\n\n\u003e\u003e bananas = read_project('.\\projects\\bananas.fpj');\n\u003e\u003e banana_model = forms(bananas, 2);\n\nYou can then compare the fit of the model before and after optimization, by\nusing\n\n\u003e\u003e plot_modelfit(bananas, 1);                % Show banana 1 before optimization\n\u003e\u003e plot_modelfit(bananas, 1, banana_model);  % Show banana 1 after optimization\n\nOr take a look at the basis shapes plotted as a colour map by using\n\n\u003e\u003e plot_basisshapenorm(bananas, banana_model, 1);  % First basis shape norm\n\u003e\u003e plot_basisshapenorm(bananas, banana_model, 2);  % Second basis shape norm\n\n\nTo reproduce our results, use the weights and normal noise estimates described\nin the paper, and use the function 'align_project' to find the camera\ntranslation and scale parameters automatically, i.e.\n\n\u003e\u003e bananas = read_project('.\\projects\\bananas.fpj');\n\u003e\u003e bananas = align_project(bananas);\n\u003e\u003e banana_model = forms(bananas, 2, 0.5, 0.25, 4);\n\n\u003e\u003e pigeons = read_project('.\\projects\\pigeons.fpj');\n\u003e\u003e pigeons = align_project(pigeons);\n\u003e\u003e pigeon_model = forms(pigeons, 7, 0.25, 0.05, 5);\n\n\u003e\u003e bears = read_project('.\\projects\\bears.fpj');\n\u003e\u003e bears = align_project(bears);\n\u003e\u003e bear_model = forms(bears, 10, 0.25, 0.25, 4);\n\nNote that you should expect some of these optimizations to take a long time.\nThis is particularly true for the dolphins project, which can be used to build a\na model from 32 dolphin instances:\n\n\u003e\u003e dolphins = read_project('.\\projects\\dolphins.fpj');\n\u003e\u003e dolphins = align_project(dolphins);\n\u003e\u003e dolphin_model = forms(dolphins, 8, 0.5, 0.25, 10);\n\nThe dolphin dataset is available here with manual segmentations, rather than the\ngrab cut silhouettes from Powerpoint 2010. This is the reason we suggest\nsigma_norm = 10 above, rather than the value 40 / 3 that appears in the paper:\nthe manual segmentations have more reliable normals, so we can use this slightly\nlower noise estimate. However, there is no significant difference between the\nmanual and automatic segmentations; the results are very similar.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmmanley%2Fforms","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjmmanley%2Fforms","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmmanley%2Fforms/lists"}