Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/charisma-ai/charisma-sdk-unreal
Charisma SDK for Unreal Engine 4.
https://github.com/charisma-ai/charisma-sdk-unreal
Last synced: 4 months ago
JSON representation
Charisma SDK for Unreal Engine 4.
- Host: GitHub
- URL: https://github.com/charisma-ai/charisma-sdk-unreal
- Owner: charisma-ai
- License: mit
- Created: 2020-10-09T12:44:38.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-12-22T16:51:02.000Z (6 months ago)
- Last Synced: 2023-12-22T17:29:23.785Z (6 months ago)
- Language: C++
- Size: 1.68 MB
- Stars: 14
- Watchers: 2
- Forks: 5
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Lists
- awesome-unreal - Charisma SDK - A tool for building conversational characters into your game. (Awesome Unreal Repositories / AI)
- awesome-unreal - charisma-sdk-unreal - Charisma SDK for Unreal Engine 4. (AI)
README
# Charisma SDK for Unreal Engine
This Unreal Engine plugin is verified to work with C++ projects using engine versions `4.26`-`4.27` and `5.0`-`5.2` only. If you find the plugin also works in another version, feel free to submit a pull request to update this!
If you have any questions or need a hand, please reach out to us at [[email protected]](mailto:[email protected])!
## Getting started
**Important:** Before setting up the Charisma SDK for Unreal Engine, you’ll need to have created a Pro story (not a Starter story!), which requires a Charisma licence. Please visit the [licencing docs on our website](https://charisma.ai/docs/licencing) for more info!
1. If you haven’t already, go ahead and create an Unreal **C++** project. Make sure you are using a version of the engine that is supported by this plugin.
2. Head to the [Releases page](https://github.com/charisma-ai/charisma-sdk-unreal/releases) of this GitHub repository and download the latest release. Unzip the contents into the `/Plugins` folder located at the root of your project. If this folder doesn’t exist, create it!
3. Close the Unreal Engine editor and rebuild the project in Visual Studio.
4. When you next open the editor, verify the plugin loaded correctly by navigating to `Settings -> Plugins` and locating the Charisma plugin. It should look something like this:
5. You can now start using Charisma in Unreal Engine! 🎉
## Usage
To create playthrough tokens, you’ll first need to find out your story ID, and optionally an API key and version of the story you want to play.
### Story ID
The `StoryId` is the unique ID of the story that you want to play. To find this, navigate to your story on the Charisma website, and copy the ID number after `/stories` in the URL.
### Story version
- To play the latest published version, pass `0`.
- To play a specific, published version of your story, set the number to that particular version.
- To play the draft version of a story, pass `-1` and supply an `ApiKey` to the `Charisma` object before calling the `CreatePlaythroughToken` method.
### API key
To create a playthrough that references the draft version of your story, you also need to provide an `ApiKey`. This can be found on the 'Story Overview' page.
**Important:** Please make sure to not share the API key with anyone you do not trust, and strip the API key from any public builds.
## Quick start
This SDK provides a template Game Mode Blueprint which can help you get started quickly with Charisma. The template performs some basic steps, such as creating a playthrough token, conversation, and connecting to Charisma. It can then be customised further to suit your own project's needs!
1. To use the template, open up "Window" -> "World Settings" and change the "GameMode Override" to `BP_CharismaGameModeTemplate`.
2. Open up the `BP_CharismaGameModeTemplate` Blueprint, and head to the Event Graph. To connect Unreal Engine with your story in Charisma, find the "Create Playthrough Token" node and enter the "Story Id", "Story Version" and "API Key" (as described above).
3. _(Only if you are using a game story, not a web story)_, in the "Start" node on the far-right, enter a "Start Graph Reference Id" corresponding to the graph in Charisma you want to start from. The reference ID can be found by clicking the three dots next to the subplot's name in the Charisma editor, and selecting "Rename".
4. If you play the scene now, you should find that the messages in your story start coming through as on-screen debug messages!
Next, we can place some actors in the scene, one for each character in your story, and use the "Charisma Actor Component" to connect them to the active playthrough in the Game Mode. This enables actors to automatically play any voice clips that are received from Charisma.
The SDK provides a template actor to do this:
1. Open up the content browser, and make sure "Show plugin content" is selected in the "View Options" menu in the bottom-right corner.
2. Select the "charisma-unreal Content" folder and drop the template actor inside the scene.
3. In the "Details" panel, select the "CharismaActor" component, and enter the "Character Name" as it is in the Charisma editor. Then check "Play audio".
4. Now, when you play, the actor (character) will play all of its relevant lines on its own audio component!
This should be all you need to get started with Charisma. For further customisation of the Charisma integration in your experience, please see the documentation below!
## Concepts
### Starting
To start playing a particular story or graph, use the `Start` node, which accepts the following parameters:
- `ConversationUuid`: The specific conversation you wish to start. This comes from the output pin of a `CreateConversation` node.
- `SceneIndex`: If you are referencing a non-game story, you will want to enter the index of the scene you want to start here. Otherwise leave this at `0`.
- `StartGraphReferenceId`: If you are referencing a game story you can use the `StartGraphReferenceId` to reference the scene you want to start. Unlike `StartGraphId`, the `StartGraphReferenceId` is the same across the draft and published versions of your story. To find the `StartGraphReferenceId` of the particular graph you want to start, locate the Subplot in the `Subplots` section of your story page and select rename. This will bring up a textbox containing your `StartGraphReferenceId`. If you wish to use another method for referencing your graph, leave this field empty.
- `UseSpeech`: This is a `boolean` letting Charisma know whether or not you want to include the character voice data of the node in the messages you receive.
### Receiving messages
The `Charisma Message Event` contains all information about the particular node in your graph that fired the `OnMessage` event. What you choose to do with the data is up to you!
At this point you can start interacting with the graph by sending replies or actions based on input and events that take place in Unreal Engine.
### Replying
When you hit a player node in your graph you’ll want the player to be able to send a reply. To do this, call the `Reply` function on the Charisma object.
The `Reply` function accepts two parameters:
- `ConversationUuid`: A `String`. The UUID of the specific conversation to reply to.
- `Message`: A `String`. The text we want to to send in our reply to Charisma. For example, this message could come from a Unreal Engine text input box or from a speech-to-text service such as 'Google Speech To Text'.
### Actions
You can use action nodes in your graph to respond to actions the player takes in the game world.
The `Action` function accepts two parameters:
- `ConversationUuid`: A `String`. The UUID of the specific conversation to reply to.
- `ActionName`: A `String`. The name of the action.
For example the `open-door` action in this graph;
would be triggered like this from Unreal Engine:
### Character voices
You may have assigned voices to your characters that you wish to play in-engine. To do this you will want to drag out on the `Message` pin on the `Charisma Message Event` and select `Break Message`.
From the `Message` node you will want to drag out on the `Speech` pin and select `Break Speech` from the context menu.
Drag out on the byte array called `Audio` and search for `Create Sound From Bytes` in the context menu. Select `Create Sound From Bytes`. This function returns a `Sound Wave` that you can play from any audio source in Unreal Engine.
### Setting memories
You can set memories in your playthrough straight from within Unreal. When setting memories we want to make sure that we pass the correct `RecallValue` to the `SetMemory` function. To find a memories `RecallValue` select the memory on your stories page and look for the field called `Memory Recall Value`.
The `Set Memory` function accepts three parameters:
- `Token`: A `String`. Token of the playthrough to set the memory on.
- `RecallValue`: A `String`. The recall value of the memory.
- `SaveValue`: A `String`. The value we wish to set the memory to.
## Reference
All methods and events have a C++ and corresponding Blueprint event or function.
For methods relating to the Charisma HTTP API, consult the [APIBlueprints](/Source/CharismaModule/Public/APIBlueprints) folder. This currently includes:
- `CreateConversation`
- `CreatePlaythroughToken`
- `GetMessageHistory`
- `GetPlaythroughInfo`
- `RestartFromEventId`
- `SetMemory`
For most other methods relating to the Charisma play API (runtime chat events), consult the [Playthrough.h](/Source/CharismaModule/Public/Playthrough.h) file.
## Using Speech Recognition
To use the `Start Speech Recognition` and `Stop Speech Recognition` Blueprint nodes, you first need to add the following config to `Config/DefaultEngine.ini`:
```ini
[Voice]
bEnabled=True
[SystemSettings]
voice.SilenceDetectionThreshold=-1
voice.MicNoiseGateThreshold=-1
```
This will enable the voice subsystem and ensure that audio data is always picked up and forwarded to the speech recognition system, regardless of how quiet or noisy it is.
Additionally, double-check that you have "Is speech to text enabled?" on your Story Overview page. Speech recognition is a premium feature that uses credits.