Sponsor Session – PRG

4:10pm in Studio A

This is the Way: Our Journey Navigating Real-Time Pre-Viz on Set

Join us for a panel delving into a journey filled with unique challenges and remarkable solutions. We’ll take you on an adventure where we blend the magic of live action and CG content, no matter the environment or gear (just like our favorite bounty hunter). Our path involves the seamless integration of camera tracking, Unreal engine, and cutting-edge servers, all working in harmony. With the skill of a Jedi, we achieve faster and more precise set-ups, as if guided by the Force itself.

Drawing from our experiences on set, we’ll share insights on effective communication and the perils of assumptions in the virtual production realm. This technology evolves at a rapid pace, constantly reshaping what we know. Get ready to embark on a journey where even the most standard positions take on new meaning in this virtual galaxy far, far away.

Download Transcript

Jeroen Hallaert  00:12

Hello, everybody. Thank you for joining us here today. Apologies for the switcheroo earlier. There’s a little bit of Murphy right there for you. I’m sure you’re all familiar with that. First of all, thank you, Laura and JT for having us. And not to forget the people all the way in the back of the room who are making this into a flawless production. Good job, guys. My name is Jeroen Hallaert vice president production at PRG. For those not familiar with PRG. We are a global company, not just a gear provider, we’re also providing services and people some of our gear services and people one of the three or combined show up on on your productions across the globe and in North America. My team specifically deals with challenging and not really off the shelf serviceable productions. Today here, we got Ben and Derek, I’m going to introduce to you in a minute. With us here today. And we we wanted to talk we heard today actually a few people talking about communication and education. I think Nicole did an excellent job earlier this afternoon giving us some key insights about about communication. What we will do, we’re going to do that as well as effective communication on set, but also explore a little bit the perils of assumptions in the virtual production realm. It’s a Technology that evolves in an amazing space, constantly reshaping what we know when you and the last five years for me personally thank you pandemic, I’ve moved more and more with my team into virtual production. And the challenges that come with that. One of my favorite toolboxes I call this toolbox in the company is a simulcam solution. And that’s what we’re going to have here on the stage, they’re going to show you a little bit of that simulcam or as one DP once called it the fancy cam. The reason why we’re trying to push this on to more and more productions on set is the question we’re asking DPS and directors like why would you not want to shoot your movie, you’ve done all the work all the time spent all the money in previous you want to shoot it, you want it all on set, and you want to shoot it as you envisioned it as you programmed it. So without further ado, here with me today that a Call Ben Whittington they both work, both as a project manager in virtual production, but also as on set simu. Cam. He doesn’t like the word gurus, Derek give it

Derek Cole  02:51

as there it is. I assume a lot of you already know what simulcam is. Yes, maybe. simulcam is basically just a compositing your previous live on set. So and what we’ve adopted since 2016, is the end cam solution for our Camera tracking. And I forgot what I was gonna say.

Jeroen Hallaert  03:24

Income is a markerless Camera tracking system. It basically builds a point clouds from natural features in its environment, instantaneously. And the reason why Derek and Ben like to use it on set is because it’s the only system that can be rolled outside and be ready in 15 minutes, except when you’re in XR studios. Without having anything basically, pre rigged on set. We use it on productions that would simply not have simulcam. If this wasn’t the case, it’s not a matter of choosing the best solution for the job that’s already in the bank. Getting the job at all is dependent on having the right solution. And Derek was actually on the first Star Wars Show within the big universe of Star Wars, which was book of Boba Fett. And the team, the VFX team there on set made it very, very clear that the system being modeled markerless was non negotiable because of all the challenges that were on on set difficult environments, outside environments inside environments. So we’re gonna ask Derek again, with the with the solution of simulcam at hand. What are some of the big obstacles and challenges? Is it just a tool to be used on satellite and expendable or does it give actually the production something useful? Further down the production pipeline?

Ben Withington  04:52

One major so with n cam, and outdoor prevas we found under that it is the like he said that the only solution. But that doesn’t mean it doesn’t have difficulties, you’ll find yourself in situations where the, you know, no other solution would address your problems, but you’re still going to have, you know, complications. So on Mandalorian season three, we had a shot of this giant turtle that ended up being like an alligator when it actually got to final. But on site, there was this big lake. And all the water was moving on the bottom, all the top of it was sky, so you have moving bottom, no tracking on the sky. So we had to go off of just geometry in the horizon. So you can imagine there’s not really anywhere to put stickers in the water or in the sky. There’s not really any kind of, you know, Wi Fi solution when you’re out in the field. So it was really the only thing we could do. But it still takes effort. And that was a major challenge for us. But we still pulled off the shot by managing when to use visual tracking when to go off the I am us thankfully at any time, you can actually choose when to shut it off and focus on which and so we could restart, realign, go until we knew something was off, go off of visual tracking go off of I am us here. So it was an actually have a shot that worked. So it’s something that’s still very challenging, but also the only solution there.

Jeroen Hallaert  06:32

On a set like this, there’s obviously a lot of people working together. Communication is not always happening the way we think it is. And that’s where assumptions come into play. That will end up bringing some some disaster to the to the actual shot. You guys have a beautiful example or a story to tell about the recent show, it’s going to air August 28 On Disney plus, which is a soaker, where at one point, you had a standing, framing the shot basically and then as soon as Rosario walked on set you want.

Derek Cole  07:15

So when during Ahsoka, the stand-in was about 20 feet away from the Camera. And then when the actual Rosario came walking, and she actually stood right in front of the Camera, and since this is a visual Camera tracking system, it blocked my view, which then I wasn’t tracking the set. So quickly, we had to try to work as a team as best that we could, by having my tech run out and move this move the N cam bar. So it was looking off to the like to the left. And then I was doing the witness to main while the unreal operator was doing the lineup, but we had to move fast. And that’s why we split up the duties while on set.

Jeroen Hallaert  08:01

talk a lot about educating clients in virtual production and educating the creams in virtual production. But there’s a huge difference between educating and arguing. Camera crew are not in a hurry to adopt new responsibilities. And as Technology improves, and foot in filmmaking goes faster production simply scheduled more in less time. And I think, Ben, you had some good experiences with that about communication on a recent production we’ve done in Atlanta.

Ben Withington  08:37

Yeah, I mean, I that’s the thing that we found the most important is that, like a lot of these productions that are on traditional formats, you know, like when we’re doing green screen, they’re, they’re interested in the speed and servicing the shoot. And so they’re Yeah, like the local 600 is not interested in taking on a bunch of extra tasks when they’re busy as Oh, you know, so the way to get accommodation from them is to show accommodation be prepared. And, and also not feel entitled to certain things. We expect skepticism, we welcome it. And so we keep a lot of tools on hand to prove things. If people are mean it’s like virtual production, even though we’ve had the big boom, there’s still a lot of veterans that are questioning it and are skeptical, and I think that’s fine. I think that if anything that you should be prepared to prove things to them just prove your claims, then you’ll grant gain trust. We had a example on a recent movie that we did in Atlanta, this Christmas movie there’s this giant palace in the sky, so it was just Palace stars and then it would come down and then you’d see actors and when they would just see the palace and the stars they thought the tracking was broken. But if there was a misunderstanding about the nature of power of parallax, because if you know if a palace tower is so far away and then it stars you don’t get parallax by moving 10 feet And so we just had to quickly throw in an object in between there so they could see the parallax happening. Instead of arguing with them. We’re just like here. How’s this? This this look? Right? Okay, that’s it. And we do that the same with measurement scale, anything. If they want something proven to them, we prove it to them. And and that gains their trust. And that gains the respect not arguing about how things should look or how they should work.

Jeroen Hallaert  10:24

And it’s still hard to employ improvise, even with a simulcam on set.

Ben Withington  10:30

Oh, yeah. No, it’s very hard to improvise a lot of a lot of VFX Supes have a bad taste in their mouth, from teams that are not willing to try anything, if it has wasn’t been pre approved, because it is so hard to improvise on set for simulcam. Because with the wall, a Content is not getting shot, unless tracking and animation is working on a green screen, they can justify moving on in their minds, even if it’s going to cost more in the long run. So we we always make a measure of like not saying necessarily, yes, but we always we always say and they’re always very receptive to can’t promise but we’ll try, we’ll give it we’ll give it our best. And when you do deliver something, they’re ecstatic it no one’s no one’s upset by that statement. That’s what they’re doing a lot of times, and they just expect the same from you. We had a really egregious example of that one day on a Ahsoka, after 14 hour day, we got asked for a crazy Camera trick shot. Derek, can you go over that?

Derek Cole  11:25

The Camera trick shot was basically Ahsoka needed to look like she was standing at kind of an angle, but we’re shooting on a completely flat ground. So we had to, we had to stop the rotational value of the movement. So that way, she was connected to wires. So in the Camera with a Dutch angle, it looked like she was standing on the side. And then we would turn the Camera and look like she would do a flip. But yeah, the plane would look like it never moved. So she looked like she did this massive flip from one side to the other. And that was essentially the the Camera trick.

Ben Withington  12:01

And the big thing about that is that that was asked at the end of the day, that blueprint had not been written, that was not a feature. Like today we’re using pixoto was not a feature of pixoto. The Blueprint was written to freeze that tracking on that roll there at that time. And again, within the frame of no promises, but let’s give it a shot. See what happens. And they love that.

Jeroen Hallaert  12:20

Great. Well, we’re gonna give it a little test. Is it up in the house? No. Then I’m going to Chinese volunteer Sarah behind the cameras. She wants to come up on stage. We don’t not only have our two d two here in the in the on stage, we also have all the way in the back of the room, an ad and D.

Ben Withington  12:44

Well, yeah, it’s um, I guess one of the things that this illustrates is how hard it is to estimate scale. So I guess the idea is like if something was 75 feet tall, and like 40 feet away around in this direction, where do you think its head would be? Where you think it would be around there. So well short. Pretty close. So you can imagine if you have a three Camera shoot, and you’re Miss estimating scale from three different angles, and now you have to fix all that. It compounds.


That’s the whole that’s the whole idea. Like, why wouldn’t we want to see what you’re shooting? Right?

Ben Withington  13:53

It’s like, how would you get that with a tennis ball? That’s something we always we always think about.

Jeroen Hallaert  14:02

All right. I think we’re getting close to the end of our 15 minutes here. One last thing, Ben, you and I talk a lot about is what’s the demand for video production concerning chroma key sets.

Ben Withington  14:22

Yeah. So with demand of chroma key. Not all the demand comes from skepticism and low budgets. Like we work with ILM a lot. ILM is not just a has not just invested in LED. They’re pioneers of it. They’ve they’ve really developed a lot and they schedule a profound amount of chroma key. I mean, we’re always busy on their shows, and we’re always shooting on green screens, blue screens. And I think that shows that like while that demand might change as Technology progresses the demand that’s there is rational. It’s not it’s not just based on the skepticism people that develop the Technology are saying I want this on, on a on a chroma key. And you’ll see very specific situations, kind of like this, where you have a certain amount of scale, a certain amount of set. And like we had a, we had a, we had a set that was a giant forest that was twice the size of Ireland’s biggest volume, and the shots and vaulted ceiling down a ship hovering eyelines. There’s there’s just no way to do that right now. And ILM knew that because they have a huge volume. And they were just like, right, this needs to be scheduled over here. And so it there’s still a gap to be filled. Because the problems that led solves for those shoots doesn’t go away, they still have those visualization problems. And so even though you can’t fix, you can’t close the gap to the point of final pixel. You can close the gap more and make a lot of catch a lot of problems, a lot of scale problems a lot of and then also just increase morale because I mean, who wants to look at, you know, green screen for six months. You know, it’s when you’re actually seeing Content. That just helps your mood.

Jeroen Hallaert  16:03

Great. Thank you, Ben direct. Thank you everybody here. This concludes our a little presentation here today. Thank you


production, tracking, shoot, shot, set, derek, solution, camera, moving, challenges, ilm, cam, ahsoka, green screen, give, ben, demand, feet, scale, little bit


Derek Cole, Ben Withington, Jeroen Hallaert