We Made a Better CGi Luke Skywalker

Those dead eyes, yeah, pretty good, there's a debate about how they

made

Luke Skywalker in the Mandalorian, so this is what we're going to do. Let's create a flawed Luke Skywalker. We'll show you how we do it. it literally asks: can we do

better

than a multi-billion dollar company called Disney? We might be able to do the hardest thing to do in visual effects is create a digital human being from scratch. Now spoiler alert if you're watching The Mandalorian and I haven't finished the second season maybe I'll stop watching this video okay spoiler alert Luke Skywalker came out pretty well he had a lot of mixed reviews about how he looked and a lot of controversy about how it was

made

because disney has been very secretive this is how they did it now in vfx artist reacts we do a lot of criticism but today we are going to put our money where our mouth is we are trying to create our own version of

luke

skywalker

it is more realistic more emotional and more vibrant and realistic that what disney has created we are going to make a big fake what you are about to see is not really tom cruise someone took the computer and forced him to say and do things that he would not say now look and remember this is not tom cru ready, ring the bell, yes, for a little subscription more action, yes, and I'm having the time of my life, it's a great honor, oh, it's an honor and a privilege, don't forget that now I can't exaggerate how.

It's a big deal if we pull this off because it represents a fundamental change in how visual effects will be done in the industry. We're getting to a point where artificial intelligence and machine learning can allow anyone to create a shot that's already been done before. It's just been impossible, so come and let's see if we can do it right, guys, we have some very important matters to discuss. Luke Skywalker was spoofed in the final episode of The Mandalorian. There is a debate about how they did it with Luke Skywalker. On The Mandalorian, when I first saw it, I was pretty sure it wasn't a deepfake because I looked more like Moff Tarkin or Princess Leia from Rogue One, kind of a very classic pipe cgi rather than an experimental person. who did it on YouTube. deep fake style stuff now the thing is to do it for a TV show during a 10 second scene at the end of an episode, it's pretty crazy because you would have to make 3D models, you would have to do facial tracking, it would take a lot of work to the animation.

Did you know that Disney actually developed their own defect method and they published an entire whitepaper and video about what they did for Luke? There seems to be some element of deepfake. The challenge they were facing is that the person playing Luke in the actual shot they have to film on camera is not Mark Hamill, so they need Mark Hamill's performance to be plastered on that person's face, so I think you're dealing with a double stack here you have an actor who then has a 3D like CG head or Mark Hamill's real face being tracked on top of that and then you have a defect applied on top of that now you're basically stuck with a double stack situation and that's where I wonder if we could overcome them.

You are literally asking if we can do

better

than a multi-billion dollar company called Disney. Maybe we can take the first step, so let's try it. To figure out how we can improve this shot, there are little things we can look for here. The shadow is a hand that passes through space. Does it look sharp or blurry? The lighting is very soft and a kind of omnidirectional light compared to all. The sharp, sharp lighting and the rest of the scenes make it feel flat and washed out, almost like it's airbrushed, and the way the light interacts with things is what makes it feel real, so if we can coming into that scene with a lot of directional light, it can really give things a sense of shape, we have flashing lights and moving light sources and all these things that can really help us sell that this is a real thing, look at how the light interacts with it, you can easily do this if it were fake.

Look for those dead eyes, yeah, pretty good, your eyes never stray, they always lock, lock, lock when he lifts his head, his eyes just gently move upward, like you're watching his mouth move, that shows. Feels like keyframe targets to me, doesn't it? A little bit, all of these heads have a very smooth movement, they're cute, they're framed, they have a curve that moves gently, whereas human heads like to shake vigorously sometimes and we have erratic movements in our heads. People's expressions are connected, like when I smile. This happens, this happens, you know, when I raise my eyebrows, my mouth twitches.

The most important thing about that scene is how rigid the acting is. I feel like they are very locked in a box and can't turn their heads much. or have them make big expressions because they're probably worried about how realistic it will look now if we can make our version of Luke emotional and show smiles and you know, vibrant eyes that look around, all of that will really help sell, this it is real. character because as a visual effects artist you know that half of your job is actually building these elements that are going to go into a shot, the other part is all the little tricks that are going to sell it as real, it's a classic visual trick if we can do it. everything we can do to improve Luke Skywalker we are going to use artificial intelligence we are going to use machine learning we are going to make a deep fake dpix that will allow us to put a Luke Skywalker mask on an actor it is basically a computer that practices to be a portrait artist look at the someone's face and paint a version of it, it looks a little blurry and weird, but if you practice it over and over and retain the memory of what worked well, then you end up with a computer that can draw someone's face pretty well and the magic of a deep fake comes from a computer that learns to draw one person's face, another person's face, and then you change the wires.

The nice thing about deep fig is everything I talked about about lighting expression in motion, even perspective, you get all that without having to build 3D models and deal with all the crazy physics simulations you need to do to make a face look realistic. The way we are going to do deepfake is well documented. You can search online for tutorials, let's do the same now we have a couple of tricks from our dream first nvidia 30 90 gpu incredibly fast incredibly powerful a lot of vram so you can get some higher resolution puget deep fakes systems has been building workstations here in the corridor and that will be our reference machine to do this deepfake.

It will allow us to get a higher quality version of Luke Skywalker and a higher resolution in a shorter period of time than any other computer here in this office. Here you will see a cutting edge deepfake, so the key ingredient for a deepfake when you draw a face is that you need to feed the computer with examples of the face, we need to inform the computer and how to draw Luke. Skywalker, so really the only version of Luke Skywalker we can use is the one we see in exchange for legitimacy, so I'm going to recruit Peter to be my teammate.

Together we will try to gather as many high-quality photographs of Luke Skywalker as possible. as much as possible, every angle, every pose and every lighting condition. Return of the Jedi was released in the 80s. It was still shot on film, which can be of great quality. Some of these shots are high quality, but many of them are blurry. causing problems because the images you input are the type of image you are going to get, then we will have a deep blurry fake, so now I am trying to use these different AI sharpening tools and some of them are giving some strange results, oh my god, there is a sharp blue here, so it's scary.

We've gone and found a ton of photos of Luke Skywalker, but now we need to find our actor who will wear the deep face that will play Luke Skywalker. because everything fake is a mask everything else we have to do for real even closer we can get to someone who looks like mark hamill from return of the jedi I better volunteer I don't want to be a part of this wait

luke

, but I was going to go to Tashi station to pick up some power converters. There is. I've been waiting my whole life for this moment.

Dean has the exact same chin. By the time we're done with you, they won't even recognize you. Man he was hoping to be a jetty, oh yeah dude I'm probably more like Baby Yoda. It's between you and Sam. Well, we find Luke Skywalker and Dean Hughes all the time. Who would have thought? Now we know it's Dean. We need to capture photos. of dean so the computer can practice recreating dean's face remember it's about being able to look at and draw dean's face look at and draw luke's face and then cross the wires in the middle we're going to film dean under a bunch of lighting conditions doing a bunch of expressions from a bunch of different angles I definitely have enough data to start this training this is really cool this is like a bunch of really good lighting samples so now we have our images of Dean and we have our images of Luke Skywalker and the computer is training those who practice drawing Dean practicing drawing Luke is practicing drawing Luke to Dean but this takes time and depends entirely on the quality of the images we have, so it is imperative that we have this defect working as much as possible. this workout on a 30-90 will probably still take us about 10 days, maybe a little less, everything needs to be set up to run all weekend because we can't afford to waste that time, check this out, check this out.

Dean Dean, you're starting to turn into Mark Hamill, man, look, don't look at that one, the most we can do now is step back and let it play out. Oh, hey, I don't know if you know. this, but the guys are in the studio right now and they're doing something that has to do with the second season of Mandalorian. I haven't seen it yet and I don't want to know anything about it, so I'm stuck. here in my attic with one purpose and one purpose only raid chat claims courtesy of today's sponsor that's right, one thing we like to do here at the runner is play little dungeons and dragons and if you like dungeons and dragons and you like mobile games, well Then you might also like big chat legends.

What I like most about this is that you can choose how you want to play. They have everything from arena battles to campaigns and dungeon battles where you can team up. with friends and actually come in with your friends, you don't need to wait for anyone to play, but you can also play with people whenever you want. Another thing you can do is summon different champions and level them up as you play now. The quickest way to level up a champion is by farming a campaign, you get XP and you get a lot of silver at the same time, picking champions is one of the most fun things I have in the game and I get it, hey, hey, little buddy, that? look at that guy over there, i mean, he's one of the champions over there, you can pick him, you can level him up, you can put him in an arena battle, you can put him in a campaign, it doesn't matter because wrath had a legend.

It's all true, they have fights with noble knights and dark wizards, they have difficult bosses, they have millions of players and best of all, they have years of content and regular updates so that when you jump in and play. The game, whether you downloaded it now or six months ago, there will always be something new for you. Recently, raid developers released the biggest update to date. Doom Tower. That's right, it's a giant tower with 120 floors and a ton of secret challenge rooms. and 12 really badass bosses to face, so if you're interested, now is a good time to try Raid Chat Legends, yeah, what do you think is good?

I don't know what to do with that. Do you live here? You live here, so if you're interested, click the link in the description to redeem your free void champion, an XP booster, 50 gems, some energy recharges, and an ancient shard. Also check out this champion's stronghold that you'll get for free. a big help probably in the tower of doom, keep going anyway watch the moment it's time to get back to work. I think the guys have gone through this Mandalorian spoilers phase. I'm going to consult with them. There are no Moors on the coast. Here, honey, yes, do you think Luke Skywalker knew that?

Damn, we've been talking a lot about what we're going to try to create. Here we are going to try to remake the Mandalorian scene. Now you may have seen it. like other deep fakes online where they defeat that scene but they are simply putting a deep fake on top of the scene that already exists but in this case we are starting from scratch we are going to have our own actor with his own hair. and makeup on a blue screen withour own cg background, it's all going to be us, so what we learned with our Tom Cruise deep fake is that acting posture, body language, all these things, your hair, they all matter, so Dean's not an actor, but He loves Star Wars, so I try to make my hair look a little nicer, okay, I'm George Lucas, I want to see you walk into this room and I want to see you seriously take off your hood, okay, he needs to be the puppeteer who directs the expression, you know, if Dean is smiling in a way that Luke Skywalker never smiles because Mark Campbell doesn't smile that way, no matter how good the t-pick is, it won't match, we're going to go into a coma .

Make this happen, if we make it, we have our chance. Okay, we're about to start filming our shots. We're filming on a green screen. We're setting up all the lighting like all the ways I talked about to be able to bring this to life. We'll put on a demon costume. I'll put it on stage. I am going to realize the dream of his life. Take your child to work. I made a complete shot list of all the lines. How I want to make a performance around them. I've been working on some effects with the lights. to simulate sparks and things like that, you know, at this point we just have to shoot and see what happens, okay, all mysterious, you just got into the smoke action, does it occur to you that Mark Hamill himself might see this?

Best moment of my life Dean is more of a puppeteer than an actor right now he needs to puppeteer our deep fake so in order to puppeteer effectively he needs to move his face like Mark Hamill does well, good job, good job , good. Acting, I have everything we need for this guy. It was the best day of my life. So I'm going back to work. Can I keep the costume? You can keep the costume in your shoe with Dean. It was great. It worked much better than me. i had some hope that it would work the green lightsaber in his face the flickering sparks everything worked it's really cool but we have a lot of problems oh look at that look that look that lighting look at that natural and realistic expression of Luke look at these dreams are coming true actually Luke's hair is definitely puffier even though Luke has his helmet on we have to shoot him again with the wig so it will be, no big deal Dean's neck is longer his jaw is a little wider his shoulders are a little wider slouched and Dean's eyebrows are a little bit lower than his eyes, so once you put Luke Skywalker on him, it looks like Luke is doing this all the time and just walks around with a neutral expression like he's frowning and there's no way we could have known any of that. this until we did it, clearly we need to reshoot it again, but we have a lot more knowledge about what will make this look realistic, we need to get the hair right, the hair doesn't work, you get the right posture when you get the right body type , but not the lighting, I'm going to do more of that, today we're going to take the photo to the next level, Jordan went and bought us some wigs, right, that wasn't my real hair, could you say this is what we have?

What we need to use if we ever want to make a fake nico is this damn presenter, yes the background looks great, it's too wavy, this is too short on the side, this part is not right, yes, it's not going to work I'm telling you, friend. The color is not that important, we can always alter the color. George is coming. I can literally show you this photo with that wig. I ask him, friend, can you cut this with that hairstyle? That's definitely the wig there, that's the wig. We are looking for something like this. Okay, totally by coincidence, today we were doing haircuts before work started because a lot of the guys wanted haircuts because they hadn't had them in a long time, so George is actually going to help out and cut. the closest Luke Skywalker style wig, you can close it really close, the colors are a little tricky but Dean needs to put on some masks and I think the only way to do that is to just ask him to do some push ups and then we stuff them with some rags and stuff, but I think you should like get the base layer until it fails until the failure gets bigger, okay, let's do this, let's put the cup full, it does it, it shortens my neck a little bit, it does it if you've done it .

I didn't lock it up I just want to see if you want to cut it it looks good it looks amazing the wig is a big help it's not perfect but it's much better Dean's performance much better much more lighting we have his stance he looks much more majestic you know we've reinforced those shoulders anyway, I'm very excited, but this is what we have to do now, the deep fake is doing its last last bit of final training that we need to compose in cg backgrounds that we are just ignoring as a challenge like that is a challenge, still using a good blue screen key and making cg backgrounds is still difficult, how are you doing here?

It's going great, man, it's Friday, this video is coming out on Sunday and we're still working on it, the reason we are. We are still working on it because we have seen the potential and we are doing everything we can to make this the best it can be. The defects must be deep. We're doing our composition, you know, taking our new Luke Skywalker and putting it into our new one. background that we've created as if assuming that we could, you know, have a 3D hallway behind Luke with flashing lights and all that kind of stuff, I'll find out later because you know I've done it before, so Peter's been working on it as fast as he can.

Yeah, just taking the original Mandalorian shot and building the geometry based on what was in that scene. I imported a frame from the program into this program called f-spy. Which is super useful. You like to specify some straight lines in the scene and then the software interprets them and gives you the exact camera position and you can just drop that into Blender and then you have the movie scene perfectly aligned with your 3D view. Then you can start placing blocks and panels and stuff and start lining it up to match up perfectly, so what you have to do for the hallway team channel is subscribe, if you don't subscribe my voice will go back to normal and I'll be very sad but if you subscribe I will continue to talk like this forever if you are wondering why I am doing this it is because I am going to make a video showing how many balloons it would be I really needed to lift the house from the top subscribe for that I have been training on this deep fake for two weeks , we have almost a million iterations, which means that Peter has looked at a face over a million times, you know, as we mark this here.

You also have to match the colors of the film, you have to match the other shots, like all the things that are not related to a deepfake, they are only related to producing normal visual effects and then there is a whole that is not He looks like Luke Skywalker. Let's see if it was worth it. I just want to like it at this point. I've been watching it forever, so yeah, let's go find everyone. Hey guys, do you want to see the mandalorian? But with our version of Luke Skywalker, yes, it's time. Alright guys, uh, we've all been working really hard on this, I hope it's great, we'll see you guys, tell me, are you a Jedi?

I am, he doesn't want to go with you, he wants his permission, he is strong with him. strength, but talent without training is nothing, may the force be with you, okay, here we go, we achieve your dreams. You know, the crazy thing about this is that it looks amazing, but there are like five and some shots, even like ten percent of Dean in that blue Skywalker. it's considered ten percent Mark Hamill if you're watching this you can have my body anytime the sparkles look incredibly sweet and if you look closely you actually see the reflections and the eyeballs move too it's crazy the resemblance still doesn't is one hundred percent, but it's much closer, much sharper and has many more emotional flaws, it's definitely a viable way to bring old characters back to life and if anyone is at Disney watching this like it's a very resume app stylish right now, we're all a little worn out.

It seems like we've been functioning with full focus not just today but all week. Okay, deep correction. Very well, we have done other experiments with defects in the channel. You should check out how to fix the worst visual effects shot of all time. Keanu Reeves stops. a robbery and the one we did with Tom Cruise, which is no longer accurate, but at the time we labeled it the best deep fake on the internet. I want to know what you think. Did we do better or worse? Was it more vibrant? really weird and a little creepy it's your guys turn to be the vfx artist react in vfx rs react and we're on display here so yeah I'm going to read all those comments it's one of my favorite things to do, yeah, give us some. breakdowns give us some reviews I want to see it subscribe