Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/text2motion/blender-integration
Integrate Blender with the Text2Motion platform to generate 3D animations from text prompts using Generative AI.
https://github.com/text2motion/blender-integration
3d animation blender blender-addon free generative-ai skeletal-animation text-to-animation
Last synced: 29 days ago
JSON representation
Integrate Blender with the Text2Motion platform to generate 3D animations from text prompts using Generative AI.
- Host: GitHub
- URL: https://github.com/text2motion/blender-integration
- Owner: text2motion
- License: mit
- Created: 2024-10-17T21:10:27.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2024-12-12T22:18:17.000Z (2 months ago)
- Last Synced: 2024-12-12T23:19:58.508Z (2 months ago)
- Topics: 3d, animation, blender, blender-addon, free, generative-ai, skeletal-animation, text-to-animation
- Language: Python
- Homepage: https://www.text2motion.ai
- Size: 44.9 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE_(OPTION1)_MIT.txt
Awesome Lists containing this project
README
# Blender Integration
> [!NOTE]
> This package is currently in early preview and is subject to changes at any time.
Text2Motion's Blender integration client, can be use to make request to Text2Motion's API for generating Animation on a given Mixamo model.
Currently, the animation can only be generated for character with Mixamo rig name and structure. Also only model imported as GLB are supported, FBX currently does not work. This repo contains an example Mixamo model in GLB that can be downloaded and imported to Blender to test out the package. Support for retargeting to other humanoid rig structure may be introduced in the future.
## Table of Contents
1. [Getting Started](#getting-started)
1. [Obtaining the API Key](#obtaining-the-api-key)
2. [Blender installation](#blender-installation)
3. [Installing the extension](#installing-the-extension)
4. [Animation Generation](#animation-generation)
2. [Development](#development)
1. [Building the extension](#building-the-extension)
2. [Blender Extension Review](#extension-review)
3. [Features](#features)
1. [Apply Root Motion](#apply-root-motion)
2. [Configure Animation Duration](#configure-animation-duration)
## Getting Started
### Obtaining the API Key
In order to use Text2Motion, you must sign up for an account and obtain the API Key via the developer portal.
1. Go to [https://developer.text2motion.ai/](https://developer.text2motion.ai/)
2. Click on Sign in on the upper right corner
3. Click on **LOGIN WITH SAML**
4. Choose either **Continue with Google** for Google SSO, or **Sign Up** with any email you own
5. Click on your username on the upper right corner of the site
6. Click on **Apps**
7. Click on **+ NEW APP** on the right side of the screen
8. Put in an **App Name**, it could be `Blender Client` or whatever you want
9. Click **Enable** for **Text2motion Free Tier**
10. Click **Save**
11. You should now have an API Key. This will be needed later
### Blender installation
The minimum supported Blender version is `4.2.0`. This package was only tested for Blender version `4.2.3`. Other version have not been tested.
### Installing the extension
1. Download the extension from the [Releases](https://github.com/text2motion/blender-integration/releases) page.
2. Go to **Edit > Preferences...**
3. Go to **Extensions**, click on the down arrow on the right and Select **Install from Disk**
4. Select the downloaded extension zip file and click **Install from Disk**
5. After that, the extension should be installed
### Animation Generation
> [!NOTE]
> All generated animations are licensed under [CC-by-4.0](https://creativecommons.org/licenses/by/4.0/)
This instruction uses an example Mixamo Rig provided in the repository. You may try using your own model as long as it is using Mixamo Rig.
1. Download [Y Bot.glb](https://github.com/text2motion/blender-integration/raw/refs/tags/releases/0.1.0/assets/Y%20Bot.glb?download=)
2. In Blender, go to **File > Import > glTF 2.0 (.glb/gltf)**
3. Select the downloaded `Y Bot.glb` and click **Import glTF 2.0**
4. Go to the **Animation** workspace
5. In the **3D View port**, locate the left arrow on the upper right corner of the screen, expand it and select the **Animation** tab
6. Fill in the key you obtained from [Obtaining the API Key](#obtaining-the-api-key) here
7. After that, you should be able to generate animation by providing a prompt
## Development
We recommend using Visual Studio Code for developing this package. This package comes with a devcontainer with Blender installed and uses the [Blender Developer](https://marketplace.visualstudio.com/items?itemName=JacquesLucke.blender-development) Visual Studio Code extension.
To get started:
1. Install [Visual Studio Code](https://code.visualstudio.com/)
2. Install [Dev Container extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
3. Press `Ctrl+shift+P` and select `Dev Containers: Reopen in Container`
4. From here, you can launch Blender with the extension installed by pressing `Ctrl+shift+P` and select `Blender: Build and Start`. To reload the changes, select `Blender: Reload Addons`
### Extension Review
Make sure changes follows the [Extension Moderation Guideline](https://developer.blender.org/docs/features/extensions/moderation/guidelines/) before submitting the new revision.
Run a linter on all scripts:
```bash
find "." -iname "*.py" | xargs ruff check --output-format=concise --ignore=E402,E741,E731
```
*While this doesn't need to pass without warnings: large numbers of unused imports, undefined variables or invalid syntax are a bad sign and might hint at inclusion of outdated/unused code.*
### Troubleshooting
#### Blender crashes on Preferences > add-on
Sometimes Blender would get in a state such that it crashes when you open Preferences > Add-on or Extension. To recover from that issue, comment out `wheels` in `blender_manifest.tom.template`, start Blender, open Preference and attempt to install Text2Motion. From here you should get an error saying the dependencies are missing, but it won't crash. After that, you can uncomment `wheels` and it should go back to normal again.
### Building the extension
To build the extension, press `Ctrl+shift+P`, select `Tasks: Run Task` then select `Build Extension`. The resulting extension will be located under `build/` at the root of the package.
## Features
### Apply Root Motion
By default, Text2Motion apply root motion to the generated animation. This means the resulting animation can leave the initial position. If you want to disable root motion and have the model animated in place, you can uncheck **Apply Root Motion** under **Advanced Options**
Note that with **Apply Root Motion** unchecked, there will be no keyframe for the root bone. If the initial position of the model is away from the origin when the animation is generated, it will render from where the last position was instead of from the origin. You can t-pose the model to move it back to the origin by pressing `A`, `alt+G`, `alt+R`, `alt+S`.
### Configure Animation Duration
By default, Text2Motion will generate animation with estimated duration decided by the model. This means the animation duration could varies from request to request. If you want to control the duration of the animation, you can uncheck **Auto** and enter the durations you want for the generated animation.
Note that since the server only accept duration in seconds, the frames option is rounded to the nearest seconds based on the frame-per-second configuration. By default it is `24`, frames can only be incremented by multiple of `24`.
You can also change the Animation timeline's unit from frames to seconds by pressing `Ctrl+T`