LipSync Documentationbeta

Welcome to the new online documentation for LipSync Pro and Lite.
This area will be updated with new information and tutorials when available.



Menu

Blend Systems

Blend Systems are an important component of LipSync. They are responsible for updating the visual representation of the character in the scene.

Overview

Blend Systems are classes that inherit from the built-in RogoDigital.Lipsync.BlendSystem class. They exist as hidden components on a GameObject that has LipSync or Eye Controller added to it, and they are managed automatically.
The purpose of Blend Systems is to allow LipSync to be as flexible as possible by not actually doing the animation itself. In terms of the Blend Shape Blend System, instead of LipSync directly storing references to Skinned Mesh Renderers or setting the values of Blend Shapes, it simply passes the blendable's index and desired value to the Blend System to handle.

Blendables

A Blendable is any aspect of the Character's representation that the Blend System can alter. In the Blend Shape Blend System they are Blend Shapes, in the Sprite Blend System they are layer/sprite combinations stored in the Sprite Manager component.

Included Blend Systems

Both versions of LipSync include a few Blend Systems out of the box. They are as follows:
  • Blend Shape Blend System
    This is the most common Blend System. It uses Blend Shapes on Skinned Mesh Renderers.
    It requires one main Mesh Renderer, and also supports any number of additional ones so long as they have the same set of Blend Shapes.
  • Sprite Blend System
    This Blend System is designed to work with 2D Sprites. When selected, a Sprite Manager component will also be added to the GameObject. You can use this component to define one or more 'layers' (Sprite Renderers) and as many different sprites as you wish. You can then select layer/sprite combinations to use in poses. Each layer will display the sprite with the highest blendable value, and will simply swap sprites when one value becomes greater.
    For example, you could use this functionality to create one foreground layer for the mouth and a second layer for the rest of the face. Emotion poses could then use the background layer and Phoneme poses would use the foreground one, to allow both to display independently.
  • Bones Only Blend System
    This is more of a pseudo Blend System, as it doesn't actually do anything. It simply disables all blendable related functions to make the UI clearer if you only need to use bone transforms.
Additionally, there are a number of others available to download from the Extensions window.

Writing New Blend Systems

Part of the power of Blend Systems is the fact that anyone can write their own. This allows LipSync to work with almost any character imaginable - anything from basic sprites and 3D models, to crazy light or particle-based systems.
In order to function as a Blend System, your script needs to inherit from RogoDigital.Lipsync.BlendSystem. This base class has a number of virtual methods you can override in order to implement your Blend System's functionality. While you can write all this yourself, the simplest way to get started is to use the provided boilerplate code.
Just right-click somewhere in the Project panel, and choose Create > LipSync Pro > Empty BlendSystem.

The most important parts of a Blend System class are as follows:

OnEnable()

Unlike a standard MonoBehaviour, you need to override OnEnable like so:
public override void OnEnable () {

	base.OnEnable();

}
In this method, you can do any initialization your Blend System requires. You should also set the value of blendableDisplayName, blendableDisplayNamePlural, noBlendablesMessage , and notReadyMessage here. The two DisplayName variables are used in place of 'Blendable'/'Blendables' in the inspector, e.g. 'Blend Shape' or 'Sprite'.
You can also optionally set the values of blendRangeLow & blendRangeHigh to re-map the minimum and maximum values on the blendable sliders in the Pose Editor. These are set to 0 and 100 respectively by default.
It is important to call base.OnEnable() after setting these variables to ensure that the Blend System works correctly.


GetBlendables()

This method is used to poll a Blend System for the blendables that can be selected in the editor. It should return a string[] containing the names of each blendable in its corresponding index. By convention, the index itself is appended in parentheses, though this is not strictly required.
In this method, you should get a list of available blendables, register them internally using AddBlendable(int index, float startingValue) if necessary and then return the list.

Below is the GetBlendables() method from BlendShapeBlendSystem.cs for reference:
public override string[] GetBlendables () {

if (!isReady || characterMesh == null)

	return null;



	bool setInternal = false;

	string[] blendShapes = new string[characterMesh.sharedMesh.blendShapeCount];

	if (blendableCount == 0) setInternal = true;



	for (int a = 0; a < blendShapes.Length; a++) {

		blendShapes[a] = characterMesh.sharedMesh.GetBlendShapeName(a) + " (" + a.ToString() + ")";

		if (setInternal) AddBlendable(a, characterMesh.GetBlendShapeWeight(a));

	}



	return blendShapes;

}

SetBlendableValue(int blendable, float value)

This method is called whenenver a blendable's value needs updating. E.G. in the editor when a slider in the pose editor is moved, or during an animation at runtime.
It should do whatever is required to actually update the character's visuals, and also call SetInternalValue(int index, float value).
It's important to keep the internal values synchronised with the real ones, as this is the value that will be reported when GetBlendableValue(int blendable) is called. This helps with preventing conflicts when multiple scripts attempt to change the same value.

Again, here is what this method looks like in the Blend Shape Blend System:
public override void SetBlendableValue (int blendable, float value) {

if (!isReady || characterMesh == null)

	return;



	characterMesh.SetBlendShapeWeight(blendable, value);

	SetInternalValue(blendable, value);

	foreach (SkinnedMeshRenderer renderer in optionalOtherMeshes) {

		if (blendable < renderer.sharedMesh.blendShapeCount) renderer.SetBlendShapeWeight(blendable, value);

	}

}

Menu

Contents

Tutorials

Miscellaneous

Components

Editor Windows & Inspectors

  • Clip Editor
    • AutoSync Window Pro
    • Clip Settings Window
    • Marker Settings Window
    • Emotion Mixer Window
  • LipSync Inspector
    • Phonemes & Emotions
    • Gestures
    • Pose Extractor Wizard
    • Gesture Setup Wizard Pro
  • Project Settings
  • Extensions Window
  • Blend Shape Baker Pro