Ultimate Guide to MetaHuman Creation & Facial Animation | Unreal Engine 5 Tutorial

0:00Anyone can now make metahumans that look

0:02this real and animate them with Unreal

0:05Engine with full customization like

0:08never before. Different faces, different

0:10hairstyles, thin, fat, tall, short,

0:14muscular, feminine, and whatever this

0:17is. Making a metahuman takes less than

0:1910 minutes, and it's easy and free.

0:22>> But I don't know how to animate their

0:24face. It must be hard and expensive. In

0:27this video, we are going to give you the

0:29ultimate guide of creating metahumans

0:32and animating their face. We'll teach

0:34you how to build your metahumans from

0:36scratch [music] and walk you through all

0:38the customization. Then we are going to

0:40teach you all three different methods

0:42you can use to animate their face. It is

0:45so easy that you can do it [music] with

0:47anything that records video or audio.

0:50You can use a webcam to animate the

0:52face. You can use a phone to animate the

0:55face. iPhone or Android, doesn't matter.

0:57>> You can use a DSLR camera. You can even

1:00just use your voice alone to animate the

1:03face.

1:03>> Yeah, that's now possible.

1:05>> And by the way, if you have a phone that

1:07has a depth camera, then we're also

1:09going to teach you how to create your

1:11own custom Metahuman that looks just

1:13like you. Actually, Farad, since we have

1:16our Metahumans, why don't we just use

1:17them to teach the video? Why are we

1:20wasting our time here? Okay.

1:21>> Ow.

1:25Never start with the head. The victim

1:27gets all fuzzy.

1:29>> Yeah,

1:30>> but I need you to talk.

1:32>> Talk about what? There is a new guy in

1:36town.

1:38Getting access into banks with

1:40metahumans that look so photorealistic

1:43that even our security system can't tell

1:46them apart from real people.

1:48animated with perfect lip sync and

1:51facial movement.

1:52>> What makes you think I want to help you?

1:55>> Because he's using your name and he has

1:57already stolen $50 million.

2:02That's more than you stole from us.

2:05Clearly, he is better than you.

2:10No way. He's just a phony. Can't do what

2:14I can do.

2:16Do you have [music] any surveillance

2:18footage?

2:19>> We want to know exactly how he's making

2:22[music] these metahumans

2:24and how he's animating them so

2:26realistically.

2:28I'll help you under one condition.

2:33And what's [clears throat] that?

2:35You're going to walk to Dairy King. Get

2:39me a large extra swirl vanilla chocolate

2:44ice cream.

2:46Dipped in dark chocolate, so thick it

2:51cracks when I bite it. Rolled in exactly

2:55three layers of toasted coconut flakes.

3:01Not two,

3:03not four,

3:05three. Deal. Now show me how you do it.

3:10>> Okay, so first you need a metahuman.

3:14When you download your Unreal Engine

3:15version, you'll have to enable in the

3:17options the Metahuman creator core data.

3:20Then inside the engine under edit

3:22plugins, you must ensure to have all

3:25metahuman related options enabled as

3:27well. Once Unreal Engine is restarted,

3:30now we can get into business. First,

3:33let's get into our content browser with

3:35control space. Let's create a new folder

3:38for our Metahuman. Right click in an

3:40empty space. Hover over the metahuman

3:42category and from there click on

3:44metahuman character. Now let's get

3:47inside. This is where we can fully

3:49design and customize our character.

3:52There are some crazy new upgrades and

3:54features that were never available

3:57before 5.6. On the left side of this

3:59window, you'll see several categories

4:02like presets, body, head, materials,

4:07hair and clothing, assembly. We're going

4:09to go through them one by one from top

4:11to bottom to fully customize the

4:14metahuman. Let's start with the presets

4:17here. You can select a pre-made

4:18metahuman and be done with it or you can

4:21select one and customize on top of it. I

4:25won't be using the presets here, but

4:27we're going to make our own custom

4:28metahuman from scratch. So, let's move

4:31to the body. On the left panel, click on

4:33body. You'll now see your metahuman

4:35standing in the middle of the screen.

4:37And on the side, you'll see multiple

4:39options like blend, conform, and model.

4:43We can use the blend mode to select

4:45multiple bodies of pre-made characters

4:48and start defining the overall shape and

4:50proportions of our character by blending

4:53in between them. Using the conform

4:55option, we can import a pre-made body of

4:58our own made in other 3D softwares and

5:01transform the body to it. There are two

5:04options here. Metahuman DNA or use a

5:07template. If you've made a custom body

5:10shape in other 3D software that is not a

5:12metahuman and you want to bring it in,

5:14you can use the template option. If

5:17you've made custom metahuman bodies in

5:19previous projects and want to transfer

5:21it, you will use the DNA option. Next is

5:25model. In this window, let's select the

5:28parametric option. Here you can adjust

5:30pretty much every part of the body

5:33separate from each other with full

5:35flexibility using simple sliders. This

5:38allows you to create any body type you

5:41desire. This level of customization

5:44never existed before the 5.6 update. To

5:48give you an idea of how powerful this

5:50new update is, if we click on the fix

5:53tab, these are the older body types that

5:56were the only options available to use

5:58for customization.

6:00Once you're satisfied with a body, we'll

6:02move on to the head. On the left panel,

6:05click on head. There are a lot of

6:07familiar options here, similar to what

6:09we had for the body. In the blend

6:11option, you can mix and match the

6:13presets to get the face that you want

6:16exactly like the body. In the conform

6:19option, you have the option to import a

6:21DNA file from a previous metahuman

6:23you've made. The other option is to use

6:26a metahuman identity. A metahuman

6:28identity is a file that contains the

6:30data of a custom 3D face we've made,

6:34either using another 3D software or

6:37using a 3D scan of a face. Regardless,

6:40these 3D faces have to be processed by

6:42Metahuman Identity before being used

6:45here, which means they have to be a

6:47human in shape and not a creature or a

6:50monster. The process is simple, and I'll

6:53show you how to use it. The template

6:55option is where you can be super

6:57creative and flexible with the design.

6:59If you've made a custom head shape in

7:01other 3D software that is a nonhuman

7:03character, imagine a shark face, you can

7:06use the template option to import the

7:08mesh and transform your metahuman. This

7:11is a slightly more complex option as it

7:14requires you to wrap the metahuman mesh

7:16for it to work properly.

7:19In the transform tab, you can literally

7:21move, scale, and rotate different parts

7:24of the face for finer controls. You can

7:27disable symmetry if you want to adjust

7:30only one side of the face without

7:32mirroring the changes to the other side.

7:35And if you need to, you can also

7:38increase the head scale from this panel.

7:40If you select the sculpt option, you'll

7:42see a ton more control points directly

7:45on the face. You can click on these

7:47points around the eyes, nose, mouth,

7:50jaw, and so on, and drag them gently to

7:54further sculpt the shape of the face.

7:57And the last one is teeth and eyelashes.

8:00Here you can fine-tune the shape and

8:03look up the teeth. As for eyelashes,

8:06we're going to ignore them now since we

8:07can change them later in the hair tab.

8:09Under teeth, you can use jaw open slider

8:13to open the mouth before you make

8:15adjustments. You can use variation to

8:18add an overall variation to the look of

8:20every tooth. And finally, hovering over

8:24and leftclick dragging on each of these

8:26arrows will allow us to change a

8:28specific parameter of the teeth. When

8:31you're happy with the face structure, we

8:34move on to the materials.

8:36This section controls the surface and

8:39appearance of your Metahuman, the color

8:42and texture of the skin, eyes, teeth,

8:45and so on. Let's start with the skin.

8:48You can adjust the skin tone here. You

8:50also have the options like skin

8:52roughness or small details such as

8:54freckles.

8:56One thing that will make the biggest

8:57difference is the face texture index.

9:00This is where you can select between 152

9:04different facial wrinkles, pores,

9:07redness, and details. Previously, you

9:11only had a few tiny options. You can

9:14also do the same for the body using the

9:16body texture index. If you scroll down,

9:18you also have the option to add custom

9:20textures on the Metahuman if you've

9:22created your textures in other 3D

9:24software or want to import them from a

9:26real face scan. Next, let's go to the

9:30eyes. Here you can start with a preset

9:32and then customize on top. Under iris,

9:35you can adjust things like the iris

9:37texture, color, rotation, pupil size,

9:40and more. You also have the options such

9:42as pupil dilation, cornea size, and

9:45scara rotation. Then, let's move to the

9:48teeth and eyelashes. For teeth, choose a

9:51natural looking shade. You want them

9:54clean, but not glowing unnaturally

9:56white. So, you're probably better off

9:58with a yellowish plaque color and a tiny

10:01plaque amount for eyelashes. We'll

10:03adjust it later with all the hair.

10:05Finally, we also have an option if you

10:08want to add makeup for your character.

10:10Now, let's give our Metahuman some style

10:12with hair and clothing. On the left

10:15side, click on hair and clothing. Here,

10:18you can choose all the groom and

10:19wardrobe elements: hair, eyebrows,

10:22mustache, beard, eyelashes, and the

10:24actual outfit. First, let's pick the

10:26hair. Browse through the hair presets

10:29and select a style that matches your

10:31character. Once you click on the style,

10:33it will appear directly on the

10:35character. You can double click the same

10:36style to remove it. After that, check

10:39the eyebrows, eyelashes, mustache, and

10:42beard. They all work in the exact same

10:45way as the hair.

10:48Now, let's go to the outfit/clog

10:52and pick the only outfit we have

10:53available. Here, we can either import

10:56our own clothing asset we've made or

10:58drag in the purchased assets we can get

11:01from the Fab Marketplace, and there's a

11:04ton of them. The awesome thing about the

11:06new parametric body shapes and wardrobes

11:08is that new clothing assets can

11:11automatically fit onto any body type.

11:14Now, let's click on the detail panel.

11:17This is where you can start adjusting

11:18the materials and colors for all the

11:21hair and clothing. We can adjust details

11:23like roughness, melanin, highlights, and

11:26more to achieve the kind of look that we

11:28want. Once you're happy with everything,

11:31it's time to assemble the character and

11:33set its quality. Now, we're going to go

11:36to assembly and prepare the metahuman

11:38for use. This step is necessary because

11:41the metahuman you're viewing is only for

11:43display purposes and not tuned to your

11:46specific use case. On the left, click on

11:49assembly. This is where you choose the

11:52quality level of the Metahuman between

11:55complete, high, medium, low, or UEFN for

12:00Fortnite or DCC to export to other 3D

12:04softwares. We can also give the

12:06character a name. For quality, we're

12:08going to select UE cinematic. UEIE

12:11Cinematic will give you the highest

12:12quality Metahuman with very detailed

12:15textures and features, which is ideal if

12:18you're working on cinematics or close-up

12:20shots. Just remember, higher quality

12:23also means heavier performance cost and

12:26larger file sizes. If you're working on

12:29a big game with many characters, you

12:31might later choose lower quality

12:33options. As for the name, type in a

12:35clear name so you can recognize it

12:37later. for example, suspect metahuman.

12:40Once everything is set and ready, on the

12:42top left, we'll create the full rig by

12:45clicking the button. This will literally

12:47create the bone structure, an armature

12:49for the metahuman, so we can animate it.

12:51Once a full rig creation is done, we're

12:53going to download the texture source.

12:56You'll be asked to choose the texture

12:57quality. The rule here is simple. Higher

13:00quality equals more detailed skin, but

13:03also more VRAMm usage, more disk space

13:07usage, and more loading times. Pick the

13:10quality that makes sense for you. Once

13:12everything is done, your assembly button

13:14might be gray or maybe missing. Just

13:17click on another tab and then come back

13:19into assembly and it should be ready to

13:21use. Click on the green assemble button

13:24to finalize your metahuman. This step

13:27packages everything together and sends

13:29the fully built Metahuman into your

13:31project as a usable character inside

13:34your content browser. Once the assembly

13:36finishes, you can save everything and

13:38close the Metahuman window. Go back to

13:41the content browser at the bottom of the

13:43screen. Navigate to your Metahumans

13:45folder or the folder where Unreal stores

13:47metahumans in your project. You should

13:49now see the metahuman you just created

13:51with the name you gave it earlier. From

13:54here, you can drag it into your level

13:56and begin animating.

13:58>> Okay, so that's how he makes his

14:01metahumans. But how [snorts] does he

14:03animate them?

14:05>> So, there are two possible ways he could

14:07have pulled this off. First one, he

14:11probably used any consumer grade camera,

14:14literally anything, lying around in his

14:16house, like um a webcam or a DSLR to

14:20record its face and drive the metahuman.

14:22Wait, I thought you could only use

14:25iPhones.

14:26>> New update, dummy. You can use any

14:29camera. Look here. Now that we have our

14:33Metahuman, let's try animating it with a

14:35random camera like this webcam. First,

14:38we'll record a video with the camera and

14:40then bring it into Unreal to process it

14:42and drive the Metahuman animation.

14:44Impressive, isn't it? So, to do that,

14:48first we need some plugins. Go to the

14:50top left of Unreal Engine and click on

14:52edit and select the last option,

14:54plugins. In the search bar, type live

14:57link. Make sure you enable live link,

15:00live link control rig, and live link

15:02hub. Once you've done that, Unreal may

15:05ask you to restart the software. Go

15:07ahead and restart. Now that you're back,

15:10go to the top left again and click on

15:12tools. Under the virtual production

15:14category, click on live link hub. This

15:17will open a new application for LiveLink

15:19Hub. Give it a second to load. Inside

15:23Live LinkHub, on the top left, click on

15:26live data dropdown and switch it to

15:28capture manager. Once you're in capture

15:31manager, you need to add a device. Click

15:33on add device and choose mono video

15:36ingest. This is where we'll input our

15:39videos. After you've added mono video

15:41ingest, click on it. In the device

15:43details below, look for take directory

15:46and click the three dots beside it. This

15:49directory is the folder where you have

15:51saved the video recordings of you

15:52talking. Select the correct folder that

15:55contains a video you want to use and

15:57click select folder. You can ingest many

16:00videos at the same time. Once that is

16:03done, you should be able to see your

16:05videos listed under take browser in the

16:07middle section. Now select the video

16:11that you want Unreal Engine to import.

16:13Once you click on your video, go to the

16:15top right and click add to Q. After

16:19adding it, you should see the name of

16:20your video in the Q list. Before we

16:23start importing, click on the video in

16:24the list and scroll down. If the

16:26orientation of your video is wrong, go

16:29to the output section, find image

16:31rotation, and fix it by rotating the

16:33video. Once that's done, go back up and

16:36select all your videos in the queue and

16:38click start to import everything into

16:40Unreal. Once the process is complete, in

16:42your content browser, you now have a new

16:44folder called capture manager. Click on

16:47the capture manager folder. Inside it,

16:49go into import, mono video ingest, and

16:52look for your video name. Unreal has

16:54processed your video in three different

16:56assets. Your audio and image media

16:59source and the capture data. These are

17:01the files that we'll be using to drive

17:03the metahuman animation. Now, how do we

17:06process the video and track the face to

17:08create facial animation data for the

17:10metahuman? Every time we want to do

17:12that, we'll need something called a

17:14metahuman performance. In the quantum

17:16browser, right click, hover over the

17:18metahuman category, click on the

17:20metahuman performance,

17:22and once the metahuman performance asset

17:24is created, double click to open it.

17:27Here is where we can see the real-time

17:30processing that happens when we feed

17:31videos to turn into metahuman facial

17:34animation. In the middle is our viewport

17:36where we can bring a face to visualize

17:38things. On the right are all of our

17:41settings where we can input footages and

17:43on the bottom is our timeline that will

17:45show us the full animation. On the right

17:47side, switch the footage mode from depth

17:50footage to moninocular footage. In the

17:52footage capture data field, select the

17:54capture data asset from the video we

17:56just processed.

17:58Now on the left side under image view,

18:00you should be able to see your video

18:02you've recorded and you can scroll

18:03through it with the timeline below. You

18:06can also drag the green and red sliders

18:08to set a start and end time for when you

18:11want the animation to be processed. Now

18:14in the details panel, look for

18:16visualization mesh. search for and

18:18select SKM

18:21face, which is going to load in the

18:23default Metahuman face mesh so we can

18:25see what our animation looks like when

18:27it's being processed. You can also load

18:29any other Metahuman face mesh you may

18:30have. When you do this, you'll get a

18:32popup asking use audio clock source.

18:36Click yes so it syncs the animation with

18:38the audio properly. Before we start the

18:41process, instead of putting the head

18:43movement mode on the transform track, we

18:45switch it to control rig. This is a

18:48better choice most of the time because

18:50we want the neck and head movement to

18:51come from the body motion capture rather

18:54than the facial motion capture. Now

18:56we're ready. So on the top left of the

18:58Medigum performance window, click

19:00process to begin the tracking. And

19:03voila, it is done and it's fast.

19:07Now, click the play button to preview

19:09the metahuman face getting animated

19:10based on the video. Isn't it insane?

19:13This was only possible with phones

19:15before, but now anyone can use any

19:19camera to make facial animations this

19:21fast. Now that you have your Metahuman

19:24face animation ready, let's export it.

19:26Go to the top of the Metahuman

19:28performance window and click on export

19:30animation. Choose a new folder for it.

19:33We'll create a folder called animations.

19:35Inside that, we'll name our performance

19:37video animation. Naming it clearly makes

19:40it easier to recognize it later. Click

19:42save. In this tab, disable head

19:45movement. If your body animation is

19:47going to drive the head, then click

19:49create. There you go. Done. Your

19:52animation is now exported. And we can

19:53view it by double clicking the animation

19:55sequence where we saved it. Okay, but

19:58how do we actually use this on our

20:00metahuman? Let's go into the level

20:02sequence, which is technically our

20:04timeline where we can animate objects in

20:05Unreal. Top left corner, find a clapper

20:08icon and click on it and choose add

20:10level sequence. Name it whatever you

20:12want. Now, if you drag the bottom bar

20:14up, you can see your sequencer. Drag and

20:17drop your management character from the

20:18outlier into the left side of the

20:20timeline. You'll now see multiple tracks

20:22such as body, medium, and control rig,

20:24face, control board, control rig. Remove

20:28the existing control rigs by clicking

20:29them and hitting backspace on your

20:31keyboard. If you keep them, your new

20:33animations would not work. Now, click on

20:35the plus icon on the face track so we

20:37can add our animations. Hover over

20:39animation. Now, you'll see a list of

20:42animations you can choose from. Look for

20:44the animation that we've just created.

20:46For us, we named it video animation.

20:49We're going to select it under the face

20:50track. You should now see the full bar

20:53that represents the facial animation. If

20:55it is longer than our current playhead,

20:57we're going to go ahead and drag the end

20:59slider to match our animation. Press the

21:01play button and see your animation come

21:04to life.

21:07So, now let's cover method two, which is

21:10using a phone to animate the metahuman.

21:12Here's the cool bit. If you have a phone

21:14with a depth camera, not only do we

21:16achieve a much higher fidelity of facial

21:18animation, we can also use the depth

21:20footage to map out the face of the

21:22person in the video to then create a

21:25metahuman digital double of them, a

21:28custom metahuman. So, let's get started.

21:31Download the iPhone or Android Live Link

21:34application using the app store and open

21:36it. Once you open it, go on the left

21:38side and tap on settings. There you will

21:40see certain options. Under mode, make

21:43sure it is set to the Metahuman

21:45Animator. Select video. And here you can

21:48see if your phone is capable of

21:49recording depth footage or not. If it

21:52is, enable capture depth. Regardless the

21:56depth, you always want to select the

21:58highest target frame rate for the best

21:59animation quality. For us, that is 60.

22:03Now, let's go out and make sure head

22:05stabilization is enabled. Scroll down

22:07and find video quality and set it to

22:09high. Now that everything is set up, we

22:12need to connect the phone to Unreal

22:14Engine. Go back to the top of the

22:16settings and look for your Wi-Fi

22:18address, which is showing your IP

22:20address. We need this to connect the

22:22phone to the computer. First, ensure

22:25that your phone and computer are on the

22:27same Wi-Fi for this to work properly.

22:30Now, before we begin, we first required

22:33to install a new plugin from the Fab

22:35Marketplace.

22:37Let's first save and close our Unreal

22:39Engine and go to Epic Games Launcher and

22:42click on the Fab section and search for

22:44Metahuman Animator Depth Processing. Add

22:47it to the library. Now, once added, go

22:50to the Unreal Engine tab. Scroll down to

22:53the Fab library and refresh it.

22:57Search Metahuman Animator Depth

22:59Processing and install it to your

23:01current engine version. Now open Unreal

23:04Engine again and go to the top left edit

23:07plugin and search for metahuman

23:09animation depth processing. Enable it

23:12and restart your Unreal Engine. This is

23:15going to allow us to do depth processing

23:17for our character. Inside Unreal Engine,

23:20let's open the live link hub once again.

23:23on the top left. Go to tools. Hover over

23:26the virtual production category. Click

23:28on the live link hub. Go to the top

23:31left. Switch to capture manager. In the

23:34capture manager, click on add device.

23:36This time we're going to add the live

23:39link face device to connect to the live

23:41link app on our phone. Once you add the

23:44live link face and click it, on the

23:46bottom you will see the IP address

23:48field. Here you'll need to input the

23:50same Wi-Fi address we found earlier on

23:53our LiveLink app. Once that is done,

23:56click on connect. If the setup is

23:58correct and you're on the same Wi-Fi

24:00network, you should be able to see the

24:02option for us to disconnect at the

24:04bottom.

24:06Now that it's connected, I can hold my

24:08phone and start to record my video by

24:11clicking on the red button in the app.

24:13When I'm done, I can click on the same

24:15red button to stop recording. And if the

24:17connection is right, within a few

24:19seconds to a minute, you should see the

24:21videos pop up in the middle of LiveLink

24:24Hub under take browser. Never close the

24:26app on your phone until we're done

24:28transferring the files. It's always a

24:30good practice to have a neutral pose of

24:32the face without expressions before we

24:34start recording our performance. It

24:37becomes handy later on, trust me. Also,

24:40ensure that your phone is stable and

24:42static. Once you're done recording your

24:45take, just like our previous method,

24:47click on the take that you want to use

24:48and on the top, click on add to queue.

24:52Once you add it to the queue, it will

24:54appear on the right side in the job

24:55list. When it's there in the job list,

24:58you can click start process. It will

25:00take a few seconds to some minutes

25:01depending on the length of the

25:02recording. When it's done, we will once

25:05again have our footage inside our

25:07capture manager folder in the content

25:08browser under the live link section. Now

25:11that we have our footage, it is time to

25:13process it using our Metahuman

25:14performance we've created earlier. You

25:17can either add a new Metahuman

25:19performance asset or go back to the old

25:21one and change things. For now, we're

25:23going to add a new one. Right click in

25:25the content browser and create a new

25:27Metahuman performance. Now that the

25:29Metahuman performance asset is created,

25:31double click it and open it. on the

25:32right side. If you've recorded without

25:34depth footage, then you're going to

25:36choose moninocular footage once again

25:38for your input and literally follow the

25:41exact same steps as the previous method

25:43of using any camera to obtain your

25:45facial animation. But if you have

25:48recorded with depth footage, then set

25:50the footage type to depth footage. For

25:53the footage capture data, assign the

25:55footage we just processed from the live

25:56link hub. Now with depth footage, we now

25:59have a mandatory new item called

26:02metahuman identity. We need to select

26:04one. If not, we cannot process the

26:07animation. Now, why do we need to add a

26:10metahuman identity? And what is the

26:12benefit of it?

26:14Because we're using depth footage,

26:16Unreal wants to reconstruct your face in

26:193D to create a digital double of you so

26:22that it can create the highest fidelity

26:24of animation.

26:26So let's create the metahuman identity.

26:28Click on metahuman identity and under

26:31create new asset, choose metahuman

26:33identity. Select the folder where you

26:35want to save it. Once it is done, double

26:38click the metahuman identity asset to

26:40open it. Inside the metahuman identity

26:42window, go to the top left and click on

26:44create components from footage. And once

26:48you're done, select your live link

26:50footage that you recorded with your

26:51phone. Once the footage is selected on

26:54the left side, click on neutral pose

26:56because here we have to select one frame

26:59of the video in which our face is in

27:01neutral pose. Once you find that frame,

27:04while still having neutral pose

27:06selected, look for promote frame on the

27:08top and click it. A pop-up will appear

27:11to confirm your selection. Just press

27:13okay. What this will do is create

27:15neutral trackers on your face. If the

27:17trackers are incorrect, you can adjust

27:19them by clicking and dragging them. If

27:21the trackers did not appear at all for

27:23you, you can click on track markers at

27:25the top. Now that the markers are

27:27tracked, click on metahuman identity

27:29solve. It will take a minute to process.

27:32Since this is a depth footage, it will

27:34try to capture your facial data in 3D

27:36space so that it can reconstruct your

27:38exact face in 3D for the highest quality

27:40motion capture. Once the solve is done,

27:42if you click on mix mode on the top, the

27:44central window, you can witness the

27:47magic of Unreal Engine and its

27:48technology. Drag the slider in the

27:51center of the screen to compare how the

27:533D mesh looks versus the face of the

27:55person in the recording. Once you're

27:58happy with it, click on auto rig

28:00Metahuman identity on the top so Unreal

28:02Engine can add the bones and armature to

28:04the face. This will take a moment to

28:06load. If your Metahuman looked deformed

28:09and incorrect, there's a chance that

28:11doing the auto rig can also solve that

28:13problem for you. It did in our case. And

28:17now finally in order to use this

28:19identity we've created for our metahuman

28:21performance asset we'll need to click on

28:23prepare for performance. Give it some

28:25time. Okay. So we are done and now we

28:28have a metahuman identity. What's the

28:30added benefit of this you may ask? Once

28:33we create the Metahuman identity digital

28:36double view, if you remember, we can go

28:39to our Metahuman creator and use the

28:41conform tab and import an identity to

28:44then create a fully custom metahuman

28:47that looks just like you. That is insane

28:51to use a single video of you to create

28:54your custom metahuman. Previously, you'd

28:57have to do a full 3D scan of your face

28:59using professional equipment and

29:01specialized software, which would cost a

29:03lot more money or a lot more time. Now,

29:06let's go back to the animation.

29:08With a new Metahuman identity, we can go

29:11back to our Metahuman performance window

29:13and add it there. Make sure to set the

29:15head movement to control rig like before

29:17and add a visualization mesh. Here you

29:20can choose SKM face for a default

29:21metahuman face or search for the same

29:23name of your identity to have your own

29:25face mesh. Once it's done on the top

29:28left, click process. This will take a

29:31few minutes to process depending on how

29:32long the animation is, but this is going

29:34to be the highest fidelity animation you

29:37can get.

29:39Okay, great. Our animation is done

29:41processing. View it and enjoy it. Just

29:44like before, go to the top and click on

29:46export. Select a folder where you want

29:49to export your animation. Once you're

29:51done, name the animation asset and click

29:53save. Done.

29:56Now that it's done, we can go back to

29:58our sequencer where we have our

29:59metahuman. This time it is the phone

30:02animation we're looking for. Let's

30:04replace our method one animation by

30:06right-clicking on the track and hovering

30:07over properties and finding our new

30:09animation and changing it to the method

30:122 animation name. Now you can see we

30:15have our phone animation driving the

30:17Metahuman.

30:18>> What are you talking about?

30:19>> We're done.

30:20>> This is This is insane.

30:24I can't I can't do that.

30:29Wait. So just from a video on your

30:32phone, you can make a custom metahuman

30:35that looks just like you and animate it

30:38at the same time?

30:40>> Yeah. The new 5.6 6 update is insane.

30:43>> Then that's it. I don't think we need to

30:46see the CCTV footage we have of our guy.

30:49>> Hold up. There's a CCTV footage.

30:52>> Not important. Just the footage of the

30:55guy in the lobby. You can't even see him

30:58doing anything. He's probably just

31:00sending a voice message.

31:01>> You guys really suck at your job. This

31:05is literally the only evidence we

31:07needed.

31:08>> What do you mean? You guys really don't

31:10know anything about the new Unreal

31:12Engine updates, do you? There's even

31:14another way you can do facial animation,

31:18audio to animation. You feed it audio

31:22and then Unreal Engine automatically

31:24takes a sound information and translates

31:27it to facial animation in just a few

31:30seconds. You can even select the emotion

31:32of the animation. Happy, angry, sad, you

31:36name it, it's there. It's insane

31:39technology. Your guy is using audio to

31:43drive the facial animation.

31:45>> I don't understand. That can't be real.

31:48>> Okay. Okay. I'll show you the audio

31:50feature, but first you have to know

31:53there's a lot more. 5.6 was a huge

31:56upgrade for animation inside Unreal

31:58Engine.

32:00The curve editor has been completely

32:02redesigned with a new lattice tool,

32:04which allows for group adjustments when

32:07editing a bunch of key frames at the

32:09same time. We now have an improved twin

32:12tools and motion trails, so you can

32:15adjust the timing, spacing, and arcs

32:19much more easily without manually

32:22tweaking every single key frame. The

32:25sequencer even got an upgrade and is way

32:27more organized with a navigation tool

32:30when dealing with tons of data. And the

32:33curve editor now has better performance

32:35when dealing with tons of key frames.

32:38Yeah, that's much better. Didn't I

32:41mention we now have real time audio

32:43scrubbing? So lining up dialogue and lip

32:46sync is so much faster.

32:50We also get control rate physics which

32:53allows us to make complex procedural

32:55animations using nodes and add more

32:59natural secondary motion.

33:02Before we'd have to use third party

33:05tools to achieve all of this, but now we

33:08can do it inside Unreal itself.

33:11Okay, I may have sidetracked a little

33:13bit. Let's get back into the audio.

33:16Okay, so if the criminal was recording

33:18his voice, we have this final method

33:21where we don't need any cameras, no

33:23video, no depth footage, only audio.

33:27First, grab your phone or any microphone

33:29and record your voice. Record the full

33:32script or dialogue that you want your

33:33Metahuman to say. Once that's done,

33:36export the audio file and import it into

33:38your Unreal Engine project through the

33:40content browser. Once you've imported

33:42the audio file, right click in the

33:44content browser and add a new Metahuman

33:46performance asset. Open the Metahuman

33:48performance asset and on the top right,

33:51switch the input type to audio. After

33:54switching the input type to audio,

33:55select the audio sample that you want to

33:57use. Now, scroll down to the control rig

34:00section. For the visualization mesh,

34:02select the face mesh that you want. For

34:04us, we'll be using SKM face. Now, we've

34:07done this before. For the head movement,

34:09make sure it is driven by the control

34:11rig. Now, scroll down a little bit more

34:14and you should see a new section for the

34:16mood based on the audio. Here, you can

34:18select what matches your audio best.

34:21Neutral, happy, sad, disgust, anger,

34:26surprise, fear. For our sample, we're

34:30going to use anger. And that will

34:32naturally move the features of the face

34:34to make the character look angry.

34:37Go to the top left of the Metahuman

34:39performance window and click process.

34:41Wait for a moment while Unreal analyzes

34:44the audio and generates the facial

34:46animation.

34:47And that's it. The first time I saw

34:50this, it blew my mind to pieces. I mean,

34:53there's no video footage to look at

34:55here. And it was processed in seconds.

34:58And the animation looks insanely good.

35:01We're ready to export. Click on export

35:04animation. Choose the folder where you

35:06want to save your animation. Give it a

35:08clear name and then click save. Close

35:10the Metahuman performance window. Go

35:12back to your sequencer where you already

35:14have your Metahuman in the scene. On the

35:16face track, replace your animation with

35:18a new one. Your Metahuman should now be

35:20speaking with facial animation generated

35:23only from an audio source. I don't

35:25understand.

35:27What do you mean when you're saying that

35:30you stole my money? What are you talking

35:33about?

35:35This is my money, not yours.

35:43What's this? Our deal. I asked for a

35:46very specific kind of ice cream with

35:49toasted coconut flake. This is a

35:51cheesecake.

35:56If you like this video, subscribe