Plugins | Unreal Engine | Gesture Control
Note: This article applies to both Prime 3 Haptic XR Gloves and Prime 3 Mocap Gloves
Note: This article applies to both Quantum Mocap Metagloves and Quantum XR Metagloves.
This guide uses Unreal 5.2 with the Right-Hand Pawn blueprint which can be found as a preset in our plugin. This guide also applies to Unreal version 4.27, 5.0 and 5.1.
Make sure to download and use MANUS Core 2.2 for gesture recognition in Unreal Engine.
Introduction
To avoid any interference issues, please make sure to use the correct model setup including the correct physics layout.
You can find the necessary files in the Unreal Live Link Plugin, available from the MANUS Resource Center.
Retrieve gestures
First fill the Get Gestures function list. This needs to be done only once at startup.
When just starting up your system, the gesture list mightbe empty. You can retry every frame until the list is filled.
If you want to add or remove gestures during runtime, you need to update the gesture list which you can do at any time.
Detect gesture threshold
Inthis example we will use one of the default gestures; “Grab”.
TheId of the gesture is retrieved from the previously setup gesture landscape.
Thenmatching the Glove Id and Gesture Id, a check is done if the threshold has been reached with a value ranging from 0-1 (0% match to 100% match).
Inthis example it has been set to 0.8, so if a value is detected above 0.8, thereis a high confidence in the gesture being correctly detected. A value of 0.8 isusually accurate enough for most interactions.
A value of 1 is not advised, as glove gestures between users can vary enough to never match 100%.
Act on gesture with grab
MANUS uses Unreal’s “TryGrabbing” system which has been reworked to fit our grabbing use case.
Once a change in gesture recognition has been detected and the previously set threshold is reached, the TryGrabbing function is called.
If one or more objects are close enough, they will be handled by Unreal’s grab system.
Objects will then be placed and oriented in alignment with the glove.
To finetune or offset the position of the object you can add a translation transform to the grabbed object.
