Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/isarandi/bodycompress

Compress a stream of nonparametric 3D human mesh estimates
https://github.com/isarandi/bodycompress

3d-human-mesh 3d-human-shape-and-pose-estimation compression smpl smplx

Last synced: about 1 month ago
JSON representation

Compress a stream of nonparametric 3D human mesh estimates

Awesome Lists containing this project

README 

        
# BodyCompress



This library compresses and serializes the output of nonparametric 3D human mesh estimators such as 

[Neural Localizer Fields (NLF)](https://virtualhumans.mpi-inf.mpg.de/nlf) to disk.



Without compression, a sequence of 3D human meshes extracted from a video can take up huge amounts

of disk space, as

we need to store the coordinates for thousands of vertices in every frame. At 30 fps and 6890

vertices (like SMPL), this amounts to almost 9 GB/person/hour. If you want to save the estimation

result for a multi-person video, it will be proportionally more.



This library achieves a **compression ratio of over 8x** on temporal human mesh data, with minimal

loss in

information. It consists of the following steps:



1. **Quantization**: The floating-point coordinates of the vertices are quantized at 0.5 mm

   resolution.

2. **Differential Encoding**: The quantized coordinates are differentially encoded in the order of

   the vertices as listed in the template. In SMPL, this order is not random but has locality, so

   the

   differences between adjacent vertices tend to be small.

3. **Serialization**: `msgpack-numpy` is used to serialize the NumPy arrays to a byte stream.

3. **LZMA Compression**: The serialized byte stream is compressed using the lossless

   Lempel–Ziv–Markov chain algorithm with the `xz` binary tool. This can run in multi-threaded

   parallel mode and is reasonably fast at compression level 5. (The Python `lzma` module does not

   have multi-threading support and is too slow for our use case.)



The format supports storing additional metadata in the header, and several per-frame pieces of

information, such as vertices, joints, uncertainties, and camera parameters, compressing it all into

one sequentially readable file.



## Installation



```bash

pip install git+https://github.com/isarandi/bodycompress.git

```



## Usage



Use the `BodyCompressor` and `BodyDecompressor` classes to compress and decompress the data.

Both should be used as context managers. The compressor has an `append` method which should be

called with keyword arguments. The decompressor is an iterator over dictionaries with the same keys.



Note that seeking is not supported, the stream is compressed as a whole to achieve the best

compression ratio.



### Compression



```python

from bodycompress import BodyCompressor



with BodyCompressor('out.xz', metadata={'whatever': 'you want'}) as bcompr:

    for frame in frames:

        vertices, joints = estimate(frame)

        bcompr.append(vertices=vertices, joints=joints)

```



Any keyword arguments can be passed to `append` that are nested dicts/lists/tuples of primitive

types or NumPy arrays.

However the following keywords are handled specially:



* `vertices`: a `(..., num_verts, 3)` NumPy array of vertex coordinates (in millimeters)

* `joints`: a `(..., num_joints, 3)` NumPy array of joint coordinates (in millimeters)

* `vertex_uncertainties`: a `(..., num_verts)` NumPy array of vertex uncertainties (in meters)'

* `joint_uncertainties`: a `(..., num_verts)` NumPy array of joint uncertainties (in meters)'

* `camera`: a `cameralib.Camera` object



### Decompression



```python

from bodycompress import BodyDecompressor



with BodyDecompressor('out.xz') as bdecompr:

    print(bdecompr.metadata)  # {'whatever': 'you want'}

    for data in bdecompr:

        render(data['vertices'], data['joints'])

```