The genesis of Ang Lee’s high-concept GEMINI MAN stems from the 1997 reveal of the birth of Dolly the Sheep. Cloning, long a scientific fascination in literature, in film, and in actual science, proved to be a success with Dolly, the first mammal cloned using the process of nuclear transfer with adult cells. Dolly’s birth proved that an exact copy of the animal the cell came from was not only possible but successful. By 2017, Zhong Zhong and Hua Hua, longtail macaques (monkeys), were the first primates cloned. And primates are even closer to humans in the evolutionary order. But if cells could be used to duplicate and create exact copies, couldn’t they be manipulated to create stronger stock, more intelligent, faster, more intuitive, fatter stock, fluffier stock, disease-free stock, etc.? Enter director Ang Lee and Weta Digital with GEMINI MAN.
The story of a younger clone of a Special Forces veteran at the top of his game with military training, expertise, and marksmanship, hired as an assassin for government agencies, the twist with GEMINI MAN is that the clone has “manipulated” DNA, not to mention the love of his adoptive father, which allegedly makes him better than the original man from whom he was cloned. So what happens when they come face-to-face and the original man is now the target of his younger self? This is the ultimate “is it live or is it Memorex” scenario and one that must first look to technology to determine if this is even cinematically possible. And when it comes to any form of CGI, motion capture, de-aging or, most importantly, the creation of a 100% digital human character, that means a call to Weta Digital and VFX Supervisor GUY WILLIAMS, proven leaders in the VFX field as is readily seen with the most recent “Planet of the Apes” series” and the realism and emotionality brought to the screen with the apes, particularly Cesar. Weta is truly the gold standard thanks to its ongoing technological research and development, not to mention the passion of its master artisans.
There are three major components of GEMINI MAN. The first is the story itself. The second, Ang Lee’s decision for the film to be lensed with high frame rate stereoscopic technology aka HFR-3D. And the third, the de-aging/creation of a 100% digital 23-year old Will Smith. Each of these elements is interdependent upon the other starting with Lee’s idea to shoot in High Frame Resolution. HFR-3D is a digital format which boasts a frame rate of 120 frames per second from the standard 24. By projecting these 3D images at 60 frames per second, taken from a second master running 4K at 120 frames per second, the 3D renders the images so they appear as closely as possible to what the human eye actually sees. Thanks to the HFR-3D, the clarity and sharpness of the imagery is beyond striking, so much so it’s as if you can touch each blade of grass on which Smith’s “older” character of Henry Brogan is laying, feel the trigger of the firepower, not to mention seeing every pore of Brogan’s skin as if you were sitting six inches from him. However, developing a story within which this technology could be implemented while creating a 100% digital human was an even more important part of the puzzle. The question became, “Can we build him? And this is where GUY WILLIAMS and Weta Digital comes in.
GUY WILLIAMS has more than 20 years in the visual effects (VFX) industry, specializing in photorealistic effects for live-action films. He first came to Weta in 1999 to work on “The Lord of the Rings” trilogy, films which really kickstarted much of the VFX we see today in the MCU, the “Apes” series of films, and now GEMINI MAN. Since first dipping his toe into the VFX waters, Williams has gone on to become the Visual Effects Supervisor on “Avatar”, “X-Men: First Class”, “The Hunger Games: Catching Fire”, “Guardians of the Galaxy Vol. 2”, Spielberg’s “The BFG”, as well as “The Avengers” and “Iron Man 3”, both of which earned him Academy® Award nominations for Best Visual Effects.
As part and parcel to Williams’ work as VFX Supervisor, particularly with a film like GEMINI MAN, it falls to him to come into the project early and manage pre-production, move into the look development and pipeline setup phases and then onset or production VFX supervision, and ultimately to post-production and completion. No stranger to creating digital humans as they have been used for some time now in situations where actors who have passed away are digitally replaced, and then of course, with digital stunts, the key here was creating a 100% digital human to fill the entire film and be an emotional, grounded, solid character who can also carry the performance. And not just carry the performance, but go toe-to-toe, beat-for-beat with the 51-year old “real” Will Smith.
A true cinephile with a passion not only for his craft but storytelling, Williams is beyond affable and articulate, not to mention gracious and a true believer in cinematic collaboration amongst all the filmmaking parties involved in a production. During a quick trip to Los Angeles from Weta Digital in New Zealand, VFX Supervisor GUY WILLIAMS sat down with me and went in-depth into all of the VFX aspects of GEMINI MAN, including lighting, action, those 120 frames per second, “the eyes”, facial expressiveness and emotion, facial obstructions, working with water, bubbles and smoke, and above all, is there such a thing as “too much” Will Smith.
So the first thing I have to ask you Guy is, you get a script and see that Ang Lee wants two Will Smiths. What do you do? What is the first thing that you think of?
Ang and Jerry Bruckheimer came to us about two years ago and told us about the project. It’s been around Hollywood for a while, so we’ve known about the story. I personally have even been on it once before when it didn’t get made. But, they come to us with this idea that they’re going to make GEMINI MAN. They’re going to cast Will Smith into it. Do we think that it’s possible now? Do we think technology has gotten to the point now where we can actually pull off a believable character on screen? And it’s not just a digital human. We’ve been doing digital humans for a while. There’s been a few movies where they’ve been used to replace actors who tragically passed away. There have been lots of movies use them for digital stunts. But Ang was very specific. He’s directing this movie. It’s going to be a very emotional, grounded, solid character. And the digital version of it has to be able to carry the performance. And we said, “Yeah.” We kind of knew that this was the time to do it because of all the advances in technology, how good the facial animation systems have become. Also, even though we’ve never done this kind of thing, you just thirst after the challenge. You love when a director comes to you with something that hasn’t been done before and asks, “Can you do it?” It gives you a chance to try something new. There are those times where they ask you if you could do the impossible and you have to tell them no, but we felt that this was the time that we could do it.
When did you find out that Ang wanted to shoot the High Film Rate Stereoscopic 3D because that has to play a part in what you guys were able to do. 120 frames is just insane!
Exactly! We’ve been aware of it for a while. We had helped Ang on a project before GEMINI MAN and he was passionate about the high frame rate, the stereo, the larger aspect of image. When he came to us and said they had been greenlit on this that we were going to do the high frame rate, the high resolution, the stereo, it’s terrifying. To some degree it’s a mechanical plus. In other words, what I mean by “plus” is an increase. You can quantify it very readily. You know exactly 120 frames is five times the frame rate and 4K is four times the resolution and stereo’s twice the number of images. So you know that it’s a 40 to one bump on the pixels that you have to deal with. But you can’t just turn it around and say, “Well it’s four times as much information. We’re going to charge you 40 times as much money.” First of all, no, that would never work. Second, it’s not fair because we know that it’s not going to take 40 times the amount of work. Some departments are linear like paint and roto and you know that that’s going to be a significant increase in costs. Whereas, tasks like animation shouldn’t increase that much. It shouldn’t change that much. So you do the best passive planning for it and you set about doing it.
It was a big enough challenge for us that we did what we called an “end-to-end test” where we synthesized 120 frames per second shot. We actually had shot some high frame rate stuff for body kinematic motion studies a few years back. So we just took one of those plates, did some tracking, did some roto, did some paint, put a CG thing in there, rendered it comped it. Basically tried to skip across every single department in the company to make sure that everybody was capable of working at 120. And of course, it was hard! That first test shot was really hard. Took us a lot of effort. A lot of departments stumbled upon things that they found that they just couldn’t deal with this kind of information, but the thing is we did that six months before the first turnover. So that by the time we started getting turnovers of shots, the pipeline had been sort of polished back down so that we were ready to take in the data. So it’s about considerate planning and early planning.
What I find striking here is this is such an action-heavy film. So you’re shooting 120 frames action-heavy. And by “action”, this is not somebody falling down the stairs a few times. This is insanity from the guys at Stunts Unlimited pulling out all the stops here. And then you’ve got so many sequences where it’s all Henry and Junior, Henry and Junior. With this amount of action, is that more difficult to bring to life than the quiet or more emotional moments where you have a still frame on the character?
Definitely the quieter moments are the hardest. Action is very forgiving. Ergo, we’ve been doing digitals, humans and action sequences for a while now. We call them digital stunts. So it’s easy enough to make a digital human work at a certain distance, especially if what he’s doing he’s doing with his body, not with his face. We’re so adroit at picking up subtlety. So at a distance, you want to see the clothes moving. You want to make sure that when they land down hard, you feel the clothes flock a little bit, and you want to make sure that the lighting is right. And we can pull all that off easily. It’s when you get into a closeup shot on a digital human and the actor is giving you such a nuanced performance. My favorite kind of acting is when an actor can say 1000 things without saying a single word. It’s basically when you could feel all the emotion and all the thought process behind the face through their visible emotion. That to me is amazing and that’s the harder stuff to do because even dialogue is harder mechanically because it takes more time in animation to get all the shapes right, but it’s a little bit more forgiving because he’s talking, he’s moving a lot. It’s when he’s sitting there in those still frames and you could see him internalizing something, thinking about something, that we have to get all of that back ends of the digital construct, and that takes a lot of just a lot of attention to detail.
That’s one of the really interesting things here because Will Smith over the decades has gotten very good at internalizing that and having the smallest nuance. He has learned how to be very telling with his eyes, with stillness. But then you’ve got the character of Junior who cannot have that exact same expressiveness that time has given the seasoned, older version. Where does that come in for you guys when you’re trying to create Junior? You’re doing your mo-cap (motion capture) with Will to get the facial nuance and expressiveness when you’re doing your flip flops, but how do you then tone it back so that Junior behaves like a Junior to some degree?
That’s a very good question. And Ang, because he’s a great director, he’s obviously begging the question long before anybody else is thinking of it. It was the main thing he was pushing on during prep. He actually considered having another person act the role of Junior just because he wanted to get that 20-year-old naivete. We kept on advising against that because as beautiful of a naive and innocent performance as that would be, it wouldn’t feel like Will Smith. So, then Ang worked really hard with Will. Will is an amazing actor and he rises to every challenge, but he definitely rose to this one as well. He constantly talks about how Ang would come to him and say, “That was great, but just do it again and just a little bit worse, not so good,” because Ang kept on trying to pull that innocent performance out of him, pull that naivete out of him. Will totally talks about his thought process. He would be a great person to ask that question because he had this entire way of tackling that challenge, and the performance he gives is rich with that lack of gravity. I t still has its own kind of gravity, but it just doesn’t have an old man’s wisdom. It has more of a young man’s vibrant, naive, corrupt view on the world, if you will.
That, to me, is the most striking is because you can see this is the younger version, but the wisdom isn’t there. And something that you do so beautifully is with the wisdom of age, you see it in the eyes. The eyes is something that Weta has developed so far above everybody else. We really saw it come to life in the last “Apes” movie with the eyes, and we see that here. What I find so striking with Junior’s eyes is that the spark is there, but the wisdom is not there. The play of light that you have going – how difficult is it with the eyes because it’s not an ape? This is actually a person, and it’s a person we’re looking at and we have all watched him over the past 40 years. How do you then scale that back, scale back what Will naturally brings in his own eyes to scale it back in the creation of Junior?
There’s two sides to that equation. There’s the realism of the eyes and trying to make it feel like it’s a living, breathing thing. And then there’s that sort of youth and that naivete. Will brought the youth. He brought the naivete. His performance had that innocence in it, which is a testament to him as an actor that he could go out of his comfort zone back to the past and portray innocence. But for us, it was about making sure that once he had successfully done that, that we portray it properly, that we get it back onto the screen in such a way that all the depth of his innocence comes back out through the digital character. That’s once again just a matter of making sure that you get every shape of the eyes exactly right; that as he’s giving that performance, you’re matching it to the half pixel. You have to get super accurate with i. We have this one presentation piece where we’d shown a shot at Junior looking down at his … not to spoil the movie for anybody who hasn’t seen it … but his other clone. And you see this look of almost horror and fear in his face. And the first time we showed it looked pretty good and it was definitely bang-on to what Will had done, but it had this just tinge of disgust. Like he was disgusted at the clone. And looking at Will, he didn’t have that. So we changed the shape of the eye. If you see this clip, you could see the changes happen. But if you toggle back and forth, you don’t see a different shape, you just feel the different emotion. The changes have to be so insanely subtle. We’re talking like a 10th of a millimeter difference in your eye. Is that disgust? So we move the eyes just a fraction and the disgust disappears and it goes back to being horror and fear at seeing himself yet again.
The eyes are what really captivated me watching the film and how you really had that differential. That’s so far beyond even what I saw in “Apes” that Joe [Letteri] had done.
We put a lot of work into the eyes for this film. And do not get me wrong. The work that Joe and Dan did on the “Apes” films is phenomenal. Those guys were three times robbed for an Oscar that they deserved. But we knew going into this one that we didn’t have the luxury of it not being a human being. The eyes had to be absolutely spot on, bang-on perfect. And we do what we do, whether we dove deep down, hard into the tech, made sure that we understood what it is about an eye that makes it look right. You do render after render after render. You put everything in there, you polish it all up, and you render it and if you still don’t like it you try to figure out. People from the outside say you just look at the corner and say, “The corner’s wrong.” It’s not that easy. You look at it and you just say, “Why does it not feel right?”, because you’re talking about the subtlest of changes that you have to make and you chase those changes. You chase those areas of the eye that just don’t quite feel right to you. And you look for advice in science. What is it that a real eye does that maybe we’re not doing that would help us here? There were a few major changes that we made to the eyes post the “Apes” tech. We did a lot of 8K photographs of the bottom of an eyelid just to make sure that we had the shape of the eyelid going onto the eye. We changed the shaders of the eyelids so that it integrates with the eye properly. Because your eyelid almost disappears into your eye at times, and we had to make sure that we were getting that but we didn’t want to just blur that line. We wanted to make sure we understand why it occurs and do it properly because it’s not always there. Sometimes it’s there, sometimes it’s not. If you start guessing at it, then you’re just heading down a different path for heading down a different path’s sake. You’re not necessarily solving the problem. We realize that the eye is not a perfect sphere, that it’s relatively a sphere, but then it gets mushed inside the eye socket. It’s larger than the eye socket so it gets squished a little bit to the shape. So the corners of the eye is actually pushed out a little bit more. We also modeled the layer, and I think that we’ll probably do a lot more of this in the future, but we modeled the layer called the conjunctiva, which is this transparent membrane that sits on top of your eye. It’s in between the air and what we call the sclera. Well, what is the sclera, but the conjunctiva is what gives your eye the slight discoloration at the corners of the eye. So all of these things we got in there and we just kept on banging on trying to figure out why it just wasn’t right yet. The main thing I kept on saying is that as good as the eyes are, there’s still the chance that it could look like a doll’s eye. To me, that’s the number one giveaway. When you’re watching a digital human, they’ll look like their eyes, just feel just a little bit magical for some reason, like a doll’s eyes. Worst case scenario, is it looks like a doll’s eye. Best case scenario in some of the previous attempts, it’ll look like “Stargate.” Remember the movie “Stargate” when the ghoul would use their powers, their scleras would get just a little bit brighter? It will look like a cheap ghoul effect, but it never looked like a real eye. So we put a lot of effort in the eyes trying to make sure that they just always radiated warmth and humanity.
How closely did you work with your cinematographer Dion [Beebe]? There was so much here, especially when we get down in the catacombs and we’ve got the two Wills fighting each other and then we’d go into the water. So you’re adding water on top of the digital Will Smith. And lighting is so important in that entire sequence. So I’m curious how closely you worked with Dion as then you’ve got to worry about refraction and reflection on top of a 100% CGI human.
So let me start by saying that Dion is one of the nicest guys I’ve worked with. Anybody you talk to in an interview is going to say that. But this is a true case. Dion, Ang, Will. They’re all just genuinely great human beings. Dion, even though he’s the DP, an Oscar-winning DP, he’s still approachable. You can still go and talk to him. You can still say, “Hey, can I ask you a question about this? Or can we talk about this?” But having said all that, as easy as it was to work with him, one of the things that we strive to do is not be another footprint on the set. We try not to be a burden. We try not to be some other part of the day that slows the shooting day down. So to a large degree, we’re there to witness and to help move things along and to answer questions and educate the process. We don’t go in and say, “Hey, we need to move the camera over here because it works better for visual effects.” We’ll figure out how to make the visual effects work. We want Dion to be 100% free to shoot the movie the way he thinks he wants to shoot it so it’s Dion’s movie. It’s not Dion’s movie accommodating for visual effects. It’s Dion’s movie. So to a large degree we make sure that we can make it work with what he’s shooting, but at the same time we try not to at all have to ask him to change it.
Were there any instances in this film where you had to ask for a lighting tweak or a lensing or an angle shift?
No, we never had to ask. There are times when you’re on a film set and you’re trying to figure out the best way to shoot a shot. There’s some creative conversations that happen. You might chip in at that point, but for the most part, once again, they hired Dion because he’s so darn good. I don’t think I’m better than him. So if the man’s going to shoot the film a certain way, I know it’s going to be gorgeous. So I’m going to support him the best I can.
Now what about the water sequences? Is it just a matter of layering or how does that process work of having Junior in the water? And we’re still seeing everything come through, but we have all that reflection and refraction.
You’d be surprised that the reflections and the refractions aren’t too terribly bad just because the way the lens works. They actually try to cut down on lensing so that the water doesn’t become yet another contributor. It actually just becomes a 20% increase in the focal length. So for the most part you can account for that during the camera tracking stage. The hard part with water is the bubbles. I mean, as far as the diffusion through water and just the cloudiness of the water, that’s not very much different than when Will Smith is lying on the bones in the cave with all the smoke drifting across his face. It’s still a lot of stuff in front of his face, a lot of color contamination back onto him. Color contamination happens in every single shot in the movie. There’s a lot of tricks that we do to account for that. In the good old days, you just looked at the picture and said, “I think it’s not yellow enough,” and you dialed more yellow in. Now we do a lot of photography of the set, a lot of photography in the proper lighting to make sure that if he’s next to a bone that’s got a bright yellow light on it, that that bone’s pushing just a little bit of the yellow light back into his face. We actually model that all onto the lighting. So all that’s par for the course for every single shot that we’re doing. When we get to the water, the only thing that’s really challenging are the bubbles. Anything that goes in front of a digital character’s face, you have to recreate and put it back on top or else lift out of the plate, put it back on top. So smoke going across his face – you can’t roto smoke. So you end up finding smoke that looks similar and you time it and position it so it feels like it’s the same kind of smoke. It can be a little bit off as it crosses the line. No one will ever know. But bubbles are very specific, and you’ll see bubbles changing positions because they’re such sharp little things. So you end up having to roto or find comp tricks to try to lift them back out of the plane. If you don’t get that right, it starts to sizzle, it starts to look like half the bubbles disappear. You start to get into these artifacts and that’s a testament to the comp team at Weta. They just keep working at it until it looks like it’s the right amount of bubbles coming back on top.
That entire catacombs sequence is probably my favorite sequence in the entire film. Because you go from the room of all the bones, and you’ve got bones flying everywhere and you’ve got the two of them in intense hand-to-hand combat, and then you have some very pensive moments in there where they stop. And they are just thinking or listening. And then we’re going into water and we’ve got light and we got bubbles and we got everything happening. And I just think that is such a technically magnificent sequence.
It was a fantastic set. Built on two levels in Budapest. The entire walk down to the Court’s Chamber, which is where Dani is tied up into what we call the Donut Room, which is the round room, and then through the door, that’s all one stage built on the floor of the stage that has rolling elevations. But then once they fall off that thing and fall down the three stories into the water, that’s actually this giant three-story set built inside of the soundstage that the stuntees jumped off the top and plunged down into the water. And Dani jumped into it too. It was a beautiful, beautiful set and once again, good planning. Dion and everybody else got gorgeous lighting in there without ever actually seeing the lights. It just had a nice feel to it. So I would love to take credit for that but the truth is that’s fantastic art direction, production design, and cinematography.
Wow! Now is it a help or a hindrance for you with this film, because there is so much, so many decades of Will Smith on film, on video, in photos, does that help you or hinder you when you’re creating Junior?
Both. It helps you in that it gives you an incredibly rich library with which to go study. So if you’re looking for something, you can find it. It hurts you because everybody else has seen that library. So everybody else has their preconceived notion of what Will should be. The single biggest challenge we had in that regard on the show was the fact thatin every film that Will did when he was young, he was a kind of a quirky, humorous action hero. So his motion is constrained to a certain series of emotions. He laughs, he smirks, he gets a little bit upset, but it’s more comedic upset. He’s intense and stern and action-oriented. And then we get to this movie and the way Ang’s designed the character, he’s very introverted. He’s very quiet, thoughtful, paced out, moody. So we found it very hard to go back and try to find versions of Will that match this.
Did you find any?
Two. Because the other thing is that everybody remembers Will, so when you look at the digital Will, it’s so easy to say, “That’s not how I remember him.” And we would have to go and find an example that matched it. And because we’re creating a Will that doesn’t match what everybody remembers, a lot of time people are just like, “That doesn’t look Will.” It’s like, “Well, it does.” And they’re like, “No, I’ve never seen Will do that.” It’s like, “Yeah, you haven’t seen him do that because he’s never acted like that when he was 23,” but you’d find the one frame in a movie where he did that kind of emotion. You put it back up and they’d be like, “Oh, okay. I could see it.” So that was a help and a hindrance at the same time.
I never questioned for a minute that it was 23-year-old Will. Watching all of his antics on film over the years … even though Junior is a quieter, more pensive and it’s because he’s in this role where he has been trained … if Will Smith had done a role like this at 23, trained to be an assassin with control and all, I firmly believe this is exactly what we would have seen.
Totally. And that that was our sort of stance on it was as much as everybody wants it to look like Will, they don’t want it to look like Will. This is not “Men in Black 6” or “Bad Boys 4” or whatever. It’s not “Fresh Prince 2.” This is Will being directed by Ang Lee to give a very thoughtful performance about the existential crisis that exists when you find out that you’re not who you think you are. It’s a beautiful performance and we want to honor that performance, but it means that we’re taking the character in a direction that’s new, but by definition, that’s kind of where we wanted to take the character.
And that’s what an actor is supposed to do with every role – take their performance in a new direction.
And that’s what Will thrives on and does. Once again, I cannot tell you how much of the success of this character comes from Will’s performance. It’s fantastic.
Favorite moment in the film or the proudest achievement, technical achievement in this film for you?
There’s two great scenes. The scene in Verris’ office where Junior’s confronting Verris [Clive Owen] about being a clone, at the end when he just breaks down and starts crying. The humanity and the emotion on his face is amazing. Paul Story and Sheldon Stopsack, the team on that scene, did a fantastic job of capturing that emotion, of just polishing and polishing and polishing it because Will gave such a good performance, and they got every single drop of it back into the digital. The other favorite one I have is in the catacombs where Henry is having the long sort of monologue with Junior. He’s like, “Let me explain who this man is that is telling you to pull the trigger.” I love that scene just because Junior says hardly any lines in that entire exchange, but Will’s giving all the performance. So, even though Henry is sitting there telling all the lines in the background and you have moments of performance from Henry, for the most part, it’s all about looking at Junior compartmentalizing all this information on a board, truly being challenged. To me that’s the whole crux of the existential crisis. This is the moment where he starts to wonder, “What am I? Who am I? Is this possible? Could I be a clone?” By the end of it, the fight happens, not because he’s some badass who has to find his way out of a bad situation, but because it’s easier to fight and maybe get killed than it is to accept the fact that maybe you’re a clone. And all that comes through in Will’s performance, just that beautiful sort of being challenged, having all of your beliefs in life challenged and undermined and then trying to fight back against it and having them rebut it yet again. And starting to understand that maybe, just maybe, the guy’s telling the truth and what are you going to do about it? Then just seeing the brain not be able to deal with that and just say, “Fight. That’s what I’m going to do about it. I’m going to get out of here and run away.” It’s a fantastic scene play.
Everything that you did in this and that the team did in this film is just amazing Guy. When I reviewed the film, I said that there were three reasons to see this: (1) for the 120 frames per second, (2) Weta Digital’s work in creating this 100% CGI human, and then (3) for Will’s performance. I think it’s masterful and as much as I love what was done in that climactic fight scene in “Avengers: Endgame” when it was every Avenger in the world on screen at one time, what you and the team at Weta achieved with this film surpasses that.
That’s very kind for you to say. We were incredibly lucky to have some of the most talented people I’ve ever worked with working on this project. From our facial team to our animation supervisor, Paul Story, to just a constellation of luminaries that were on this project with us that all rose to the challenge and gave it their all. I was blown away.
by debbie elias, exclusive interview 12/18/2019
