Predictive Human Motion Using Physics-Based Avatars for Unreal Engine 4

T. Klopfenstein, R. Bhatt, K. Malek
University of Iowa Technology Institute,
United States

Keywords: virtual humans, predictive human motion, predictive human behavior, physics based modeling, human digital twins


The Virtual Soldier Research (VSR) Program at the University of Iowa Technology Institute conducts basic and applied research in predictive, physics-based human modeling and simulation. Over the past 20 years, the VSR team has delivered major products to the US Military, including Santos®, a comprehensive platform for human simulation, human systems integration, and human performance. The VSR team comprises experts in robotics, optimization, physiology, AI, ergonomics, and other relevant subject-matter areas. The digital avatar developed at VSR, also named Santos, is very high fidelity with 215 degrees-of-freedom (dofs) to model full-body skeletal motion. Santos also surpasses currently available tools with respect to predictive capabilities, meaning posture and motion are not simulated using pre-recorded data; they are simulated using validated mathematical models that can be re-run easily with changes in problem parameters. Consequently, Santos AI predicts how people strike various poses and how they move. VSR worked a project with Epic Games to bring these predictive capabilities of Santos software into the Unreal Engine. Work under this project focused on two main areas: 1) Creating a map between the two different skeletons, DigitalHuman-based single -axis per joint Santos skeleton and the multi-axis joint Unreal skeleton; and 2) Ability to import Metahumans within Santos framework and utilize the photo-realistic Metahuman avatars with all Santos’ predictive tools. The Santos avatar, along with its skin and skeleton, has been imported into Unreal engine. While the import of the avatar was relatively easy task, the major effort was focused on mapping the correct degrees of freedoms and orientation. The first phase of development was to create a one-way map that allows a developer to select an animation created for Santos and replay the motion in Unreal environment on the UE mannequin. A MetaHuman is a high-quality digital character that is created in the Unreal Engine online application MetaHuman Creator. MetaHumans created there can be downloaded through Quixel Bridge, directly into Unreal Engine, with only a few clicks. Previous techniques for creating compatible avatars of variable anthropometry were process intensive, taking anywhere from days to weeks to complete, even for a graphics expert. MetaHumans are believed to simplify the creation process and enable developers to start generating content quickly. Santos can leverage these novel capabilities to enable more realistic digital twin simulations, and to enahance MetaHuman avatars to create a specific avatar with desired anthropometry, range of motion, mass and inertia properties, clothing, loading, etc for engineering use.