Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/timvanscherpenzeel/gltf-to-usdz-research

Research and proof of concept of converting glTF to USDZ for AR Quick Look (iOS 12+).
https://github.com/timvanscherpenzeel/gltf-to-usdz-research
ar cli converter gltf ios pbr pixar proof-of-concept quicklook usd usdz xcode
Last synced: about 1 month ago
JSON representation
Research and proof of concept of converting glTF to USDZ for AR Quick Look (iOS 12+).
Host: GitHub
URL: https://github.com/timvanscherpenzeel/gltf-to-usdz-research
Owner: TimvanScherpenzeel
License: mit
Created: 2018-06-14T13:19:34.000Z (over 6 years ago)
Default Branch: master
Last Pushed: 2018-09-13T13:57:50.000Z (over 6 years ago)
Last Synced: 2024-12-10T12:43:12.476Z (about 1 month ago)
Topics: ar, cli, converter, gltf, ios, pbr, pixar, proof-of-concept, quicklook, usd, usdz, xcode
Language: JavaScript
Homepage:
Size: 21.8 MB
Stars: 187
Watchers: 15
Forks: 19
Open Issues: 1
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project

README

        # glTF to USDZ research

Research and proof of concept of converting glTF to USDZ for AR Quick Look (iOS 12+).

## Reasoning

Even though I think the intentions of Apple / Pixar are great with the open source [USD](https://github.com/PixarAnimationStudios/USD) pipeline I think we as an industry should be relying more on truly open formats that are not controlled by a single entity. Installing `USD` is cumbersome, requires a lot of disk space and is completely overkill for most situations (if your goal is to convert some 3D models to USDZ and show them using `AR Quick Look`). I'm hoping that as we get closer to the public release of `iOS 12` more tools will pop up that directly convert to `USDZ` so that we can get cross-platform support.

In order to move away from using the [USD](https://github.com/PixarAnimationStudios/USD) pipeline solution offered by Pixar I think it would be wise to try and manipulate the intermediary readeable `USDA` format. Unfortunately there are very little examples available of `USDA` files.

The general idea is to dynamically generate / manipulate the intermediary a general `USDA` file-structure and pass that to the `usdz-converter` to handle the further conversion to `USDZ`.

After some work, debugging and friendly help from [@domenicopanacea](https://twitter.com/domenicopanacea/status/1011577221310447618) and [@virakri](https://twitter.com/virakri/status/1010356558742589440) I got it to work. Most of the findings on `OBJ` support come from `@trayser` who posted details regarding `OBJ` to `USDZ` conversion on [developers.apple.com](https://forums.developer.apple.com/thread/104042). The `OBJ` files that are generated by `gltf-to-usdz` only use the supported tags as mentioned by `@trayser`.

Please note that this is just an experimental setup and should be seen as a proof of concept. As it stands I've only tested it with the `DamagedHelmet` glTF asset included in `/assets/`. You will very likely run into issues with the current code.

My goal of this research is to inspire others to make proper converters that handle all the complexities.

## Live demo

[Live demo](https://timvanscherpenzeel.github.io/gltf-to-usdz-research/)

![Screenshot](/assets/helmet_including_preview.jpg?raw=true)

## Limitations

- `.glb` files are not yet accepted. Please use `glTF` files with seperate textures.

- Only the first mesh in the `glTF` file will be parsed and exported. Make sure you merge all meshes.

- Only `glTF` files with external textures are currently handled (as opposed to embedded base64 encoded textures).

- Animations from the glTF are not yet transferred, it seems possible to transfer skeleton animation and regular transformations. Unfortunately due to the shadow being baked on the first frame of the animation you can't really do large moving animations. Besides that the filesize increases quite dramatically. It is not recommended to have animations that move a character away from the origin as it results in tracking problems. Your best bet are micro-animations that enhance a character or model instead of large animations.

## To do

- ~~Convert the example `USDZ` examples to `USDA` structures by converting `USDC` to `USDA`. Unfortunately I think this requires the installation of the [USD pipeline](https://github.com/PixarAnimationStudios/USD) and the use of [usdcat](https://github.com/PixarAnimationStudios/USD/blob/e6ce9e884a65e7d6acd762e9dbc961dcf9aa36bb/pxr/usd/bin/usdcat/usdcat.py).~~

  Thanks to [@wave-electron](https://github.com/wave-electron) the `USDC`'s of the examples of the AR Quick Look gallery have been converted to `USDA`. Due to copyright concerns I unfortunately won't be able to upload them to this repo. I've had a look at them and for the most part look very similar compared to the USDA file generated by `gltf-to-usdz`. I've since also been able to convert the examples myself using `usdcat`.

## Installation

- Make sure you have [Node.js](http://nodejs.org/) installed

- Upgrade your operating system to macOS High Sierra 10.13.4 or newer

- Download Xcode 10 beta and put it in `/Applications/`

```

https://developer.apple.com/download/

```

- Link to the beta version instead of the normal version

```

sudo xcode-select --switch /Applications/Xcode-beta.app

```

- Construct a new `.usda` using `gltf-to-usdz` and run it through the `usdz_converter`

```

node ./bin/gltf-to-usdz.js -i ./assets/DamagedHelmet/DamagedHelmet.gltf -o ./assets/DamagedHelmet.usda -s 10.0 && xcrun usdz_converter ./assets/DamagedHelmet.usda ./assets/DamagedHelmet-`date +"%H-%M-%S"`.usdz

```

- On succes the following should be outputted to the console

```

2018-06-20 17:21:23.364 usdz_converter[82749:13335731]

Converting asset file 'DamagedHelmet.usda' ...

```

In order to see the contents of the outputted `USDZ` change the extension to `.zip` and unzip it. You will see a `.usdc` and several textures (if the original glTF file had textures).

## Development

In order to install [USD](https://github.com/PixarAnimationStudios/USD) on MacOS please follow the following instructions:

- Upgrade your operating system to macOS High Sierra 10.13.4 or newer

- Download Xcode 10 beta and put it in `/Applications/`

```

https://developer.apple.com/download/

```

- Link to the beta version instead of the normal version

```

sudo xcode-select --switch /Applications/Xcode-beta.app

```

- Install Cmake and QT. If you get any [issues](https://github.com/TimvanScherpenzeel/gltf-to-usdz-research/issues/1) during compiling or installing you might have to install QT5.

```

brew install cmake

brew install qt@4

```

- Install PyOpenGL

```

pip install PyOpenGL

```

- Install PySide2

```

pip install --index-url=http://download.qt.io/snapshots/ci/pyside/5.9/latest/ pyside2 --trusted-host download.qt.io

```

- Update OpenImageIO release version from `Release-1.7.14.zip` to `Release-1.8.12.zip` and add the following section in `build_scripts/build_usd.py` as mentioned as a [solution](https://github.com/PixarAnimationStudios/USD/issues/19#issuecomment-399918369) by `@robpieke` to an earlier Python crash I was experiencing.

```diff

diff --git a/build_scripts/build_usd.py b/build_scripts/build_usd.py

index 37fb2a5e..3c940b3b 100644

--- a/build_scripts/build_usd.py

+++ b/build_scripts/build_usd.py

@@ -692,7 +692,7 @@ PTEX = Dependency("Ptex", InstallPtex, "include/PtexVersion.h")

 ############################################################

 # OpenImageIO

-OIIO_URL = "https://github.com/OpenImageIO/oiio/archive/Release-1.7.14.zip"

+OIIO_URL = "https://github.com/OpenImageIO/oiio/archive/Release-1.8.12.zip"

 def InstallOpenImageIO(context, force):

     with CurrentWorkingDirectory(DownloadURL(OIIO_URL, context, force)):

@@ -855,6 +855,12 @@ def InstallUSD(context):

         if context.buildPython:

             extraArgs.append('-DPXR_ENABLE_PYTHON_SUPPORT=ON')

+            if MacOS():

+                import distutils.sysconfig

+                pyLibPath = distutils.sysconfig.get_config_var('LIBDIR')

+                pyIncPath = distutils.sysconfig.get_config_var('INCLUDEPY')

+                extraArgs.append('-DPYTHON_LIBRARY=' + pyLibPath + '/libpython2.7.dylib')

+                extraArgs.append('-DPYTHON_INCLUDE_DIR=' + pyIncPath)

         else:

             extraArgs.append('-DPXR_ENABLE_PYTHON_SUPPORT=OFF')

```

- Run `python USD/build_scripts/build_usd.py BUILD`, it will take roughly 1 hour the first time you build because of all the dependencies that need to be compiled, resulting in the following output if succesfully installed:

```

➜  pixar python USD/build_scripts/build_usd.py BUILD

Building with settings:

  USD source directory          /Users/timvanscherpenzeel/Projects/pixar/USD

  USD install directory         /Users/timvanscherpenzeel/Projects/pixar/BUILD

  3rd-party source directory    /Users/timvanscherpenzeel/Projects/pixar/BUILD/src

  3rd-party install directory   /Users/timvanscherpenzeel/Projects/pixar/BUILD

  Build directory               /Users/timvanscherpenzeel/Projects/pixar/BUILD/build

  CMake generator               Default

  Downloader                    curl

  Building                      Shared libraries

    Imaging                     On

      Ptex support:             Off

    UsdImaging                  On

    Python support              On

    Documentation               Off

    Tests                       Off

    Alembic Plugin              Off

      HDF5 support:             Off

    Maya Plugin                 Off

    Katana Plugin               Off

    Houdini Plugin              Off

    Dependencies                zlib, boost, TBB, JPEG, TIFF, PNG, OpenEXR, GLEW, OpenImageIO, OpenSubdiv

STATUS: Installing zlib...

STATUS: Installing boost...

STATUS: Installing TBB...

STATUS: Installing JPEG...

STATUS: Installing TIFF...

STATUS: Installing PNG...

STATUS: Installing OpenEXR...

STATUS: Installing GLEW...

STATUS: Installing OpenImageIO...

STATUS: Installing OpenSubdiv...

STATUS: Installing USD...

Success! To use USD, please ensure that you have:

    The following in your PYTHONPATH environment variable:

    /Users/timvanscherpenzeel/Projects/pixar/BUILD/lib/python

    The following in your PATH environment variable:

    /Users/timvanscherpenzeel/Projects/pixar/BUILD/bin

```

## Taking apart a USDZ file

To take apart a `USDZ` file one should follow the following steps:

1.  Follow the steps mentioned in `Development` to install USD

2.  Change the extension of the `usdz` file from `.usdz` to `.zip`

3.  Unzip the file using `Archive Utility` (comes with MacOS) or likely any other unzipping tool

The folder will contain the included textures seperated out and a `.udsc` file.

4.  Now use `usdcat` to convert the `USDC` file to readeable `USDA`: `usdcat input.usdc -o output.usda`

The resulting `USDA` file will contain the mesh data.

## Resources

- https://graphics.pixar.com/usd/docs/Converting-Between-Layer-Formats.html

- https://forums.developer.apple.com/thread/104042

- https://graphics.pixar.com/usd/docs/UsdPreviewSurface-Proposal.html

- https://developer.apple.com/videos/play/wwdc2018/603/

- https://devstreaming-cdn.apple.com/videos/wwdc/2018/603augiuv41xoowslk8/603/603_integrating_apps_and_content_with_ar_quick_look.pdf?dl=1

## Contributing

As it is my goal to create a community project I welcome you to submit any pull requests or open a ticket if you spot an issue. Please be aware that as of now the project is in a very experimental state and only serves as a proof of concept so there are probably lots of things wrong and the code is far from clean.

## Licence

`gltf-to-usdz` is released under the [MIT licence](https://raw.githubusercontent.com/TimvanScherpenzeel/gltf-to-usdz-research/master/LICENSE).