Film sets have always been a place of creativity, collaboration, and magic. But what happens when that magic is replaced by LED screens? How does the art of filmmaking change when the set isn’t real? Join us in a conversation with Sam Nicholson, founder and CEO of Stargate Studios as we explore the world of virtual film sets.

Sam started Stargate Studios in 1989 and has worked on some of the most well-known tv shows and movies of all time. From his first experience as a Visual Effects Supervisor on Star Trek I and the ground-breaking Max Headroom to his Emmy Awards for Best Visual Effects for TNT’s Battleground and NBC’s Heroes, Sam has built Stargate Studios into an international leader in digital imaging for high concept television, feature films, and commercials.

In the past 30-plus years, Stargate has experienced the whole arc from film to digital and now into virtual production. Now, the studio is taking virtual production techniques to all new heights with real-time rendering in Unreal Engine and creating final-pixel VFX shots in-camera, on set.

We sat down with Sam to learn more about this innovative technology and where he sees the future of filmmaking.

“Don’t get too fixed in yesterday’s applications, because the technology is changing so fast that the real challenge is going to be our creative ability to use these technologies when they arrive.” – Sam Nicholson

What’s unique about shooting on a virtual stage that would make a production want to shoot with Stargate versus shooting practically?

Well, I think we’ve just scratched the surface of how virtual production can really be used to save time and money and increase the creative options for filmmakers. We did a sequence with Netflix for White Noise. The scene that we were shooting was a group of kids in the rain at night. Kids obviously can’t work at night, so we had to recreate the environment and shoot it on a stage.

So, we did everything — created everything in a virtual set — so that we could work onstage during the day. But, of course, in the movie, it’s at night, and it’s in the rain… The stage is much more controllable. And it’s a lot safer to work on a stage. And a lot easier, and predictable to work on stage.

As we say in virtual production here, Magic Hour lasts forever. You want the sun to go up again? Go ahead, you can, you know, you can control the weather. You can control the time of day, you can control the location.

You know, if you have a whole film that’s set in Morocco, go to Morocco. But if you only have two pages in Morocco, you’re either going to tear those pages out, or you’re going to do it virtually.

So we can go, photographically, and bring Morocco to Los Angeles. It’s much better. And COVID has accelerated that tremendously.

With that said, I am definitely a traditionalist, if it exists in the real world, then shoot it. Because the real world has all sorts of mistakes and things that aren’t perfect. If a virtual set can be an extension, fantastic. We went to New York virtualized New York for Ugly Betty and went to Seattle virtualized Seattle for Grey’s Anatomy. And now we have those as libraries. For an ongoing show, virtual production is incredibly valuable because you can recycle your locations and your sets.

To create those virtual sets you need high-quality videos. Can we talk a little about equipment? There is always this temptation to make it quick and let’s say use Insta360 Titan.

Yes, it’s always very tempting to shoot with something like Insta360 Titan. All in one, boom, hit a button, and you’re done. Right? But, when you look at any specific angle, you’re only getting 2k. Because it’s maybe 12k all the way around.

For Our Flag Means Death, we used a 20k wall. But when you look at it with a 50mm lens, you’re looking at something maybe 4k, so your overall resolution is critical. So for virtual production, I’m a big believer that you need to maximize the resolution and the quality of each individual image.

So at that point, what is one Sony α1 8k seeing? How does it perform at night? How does it perform during the day? What’s the ease of use? But every project is different, so there is no single perfect answer.

For Our Flag Means Death we’re shooting for large LED screens, and we need the resolution. Our circular resolution instead of 12k is 64k. Any single image that you look at is a full 8k. You will get a better-looking image. It’s closer to reality, and we’re really trying to recreate reality.

What is your go-to memory card?

Lately, we’ve been shooting on CFexpress type A cards. The workflow is very important for us. You generally don’t have a lot of time. There’s never enough time, never enough money, you know all those things. So it has to be economical. Memory cards as a rule gotta be fast, reliable, and dependable. And it’s got to be affordable.

Your mission is to find 100% dependable tools — they have to be bulletproof. Because no matter what the price is, if your tool is not dependable it should not even be a consideration. So the dependability of the ProGrade cards is incredible. We’re recording in all sorts of conditions. Night, day, hot, cold, wet, dry. And the memory card is going to take a lot of abuse and we’re going to recycle it a lot.

I think one of the most remarkable things about the ProGrade product is that it is incredibly dependable, and it’s affordable. Also, the ProGrade ecosystem is really great. Because there are also very dependable, affordable readers that stack. And then you’ve got a fantastic memory that is dependable and affordable, that drops into it.

And, you know, when you’re building out your kit, dependability is number one to us. If it’s not dependable, forget it. When we first tried ProGrade cards we got a batch of them and tried them and they were bulletproof. And I’m looking forward to building out an ecosystem of the ProGrade products because the readers are the same way — they’re really solid. And so, you know, we’re gonna use a lot. We’re gonna build it into all our productions.

So you guys chose ProGrade CFexpress Type A 160GB cards specifically because those are the highest data that you can fill up on one card?

We do use large broadcast cameras. But by and large, if you’re multiplying everything times nine, when you’re shooting circle vision, cost can become an issue. But regardless of how you construct your system, how you handle the data is incredibly important.

If you shoot large amounts of data 9 or 10 times, every time you hit the button, you’re generating a terrific amount of data — up to 300 terabytes from one shot. And we’ve done that. We’ve handled it on a 20k and 165-foot big video wall that looks beautiful. You can actually shoot heavy data, which is really why I love the Sony Alpha One using the ProGrade CFexpress Type A cards. There are 160 gigabytes on a card. So you can load up nine of them, and you can have a terabyte in your hand.

It sounds like you’ve got your data management system down to a science.

Yes. CFexpress cards from ProGrade, the 160 gigabytes times two in each camera. So we have 18 cards loaded at once. It’s really efficient. And you don’t have the unpredictability of outboard memory and all these things floating around.

And when you download the cards, you can pull 18 cards out and put them into your readers. Then you can download that whole thing in about an hour. Out in the field, that’s essential, because we’re rotating memory, you know, and you want to load it off as quickly as you can.

So with a laptop and the ProGrade readers stacked, we can take 18 160 GB cards and load them all into a Mac, BAM, they’re downloaded and you’re rolling simultaneously while that’s happening. So data management is an absolutely critical part of picking your camera technology.

What comes out of the camera, how you get it copied, backed up, and safely distributed to the client as fast as possible, and the dependability — all of that is huge. Because if one camera has a failure on one take, it’s okay, you have nine other cameras to capture it, reducing your failure rate.

So what you don’t want are a whole bunch of cable holes and things and “oh, maybe this thing connected, maybe it didn’t, or that one was a little loose or whatever.” So the internal memory in the Sony Alpha One, not to mention the quality of the camera in the build, is a beautiful thing.

But from a memory standpoint, it’s fantastic to be able to slide a couple of ProGrade CFExpress 160 GB in there, boom, lock it, good to go. And there’s no doubt that you’re not going to have a loose connection, you’re not going to have some wire on a boat that’s going like this, or a car that’s bouncing. You know, we’ve thought through all those things. And we’ve gotten to this point by looking at dependability, number one.

As someone who has been in the industry for decades and has been at the forefront of innovation, are there any new innovations or technologies that get you excited about the future of filmmaking?

I’m looking forward to the evolution of the LED wall technology. We need big images that are very lightweight. We specialize in mobile applications and big fixed installations. The kind that you can brag about. But the ones that I love are the tools that are incredibly diverse, multi-use, economical, simple, and affordable.

So the LED wall right now is RGBW. But lights like the ProLight, where we have six light engines in every pixel, and it has full rec 2020 color space, we’re using that in terms of volumetric lighting to enhance and go beyond what the LED can put out. The LED light is pretty dirty. I mean, it’s very low bandwidth. It’s 10-bit, you know, it’s not pretty light, necessarily.

So we have to augment it with a light that has a much more dynamic range. So, with the convergence of the chip technologies in the camera, the LED technologies are getting lighter with carbon fiber so that one person can pick up a 20-foot screen. Right now, if you put a 20-foot screen over a car you have to get a 10-ton forklift to do it. It’s heavy and dangerous, and there’s a ton of power on it. It’s going to change. That is exciting. And as that changes, the real challenge is our ability to change ahead of our preconceived notions of what is possible.

In other words, as technology progresses, the challenge is to progress along with it?

Advanced technology is converging from any number of directions, the chip technology in the cameras, the memory, the readers, the computers, the LED walls themselves, the displays, the laser, projectors, you name it, they’re all progressing, and they’re all intersecting.

And when they intersect, it creates, like a hyper nexus of possibilities, and the old rules that we were obeying six months ago, don’t necessarily apply six months from now. So you can’t have a fixed mindset. And so the real challenge to filmmakers, I think, is to change with the technology and potentially ahead of it, your attitudes, you know, the creative designs, what is possible, you might have said, six months ago, “oh, that’s impossible.” Well technology may have advanced since then and now suddenly, it’s possible.

Don’t get too fixated on yesterday’s applications, because the technology is changing so fast that the real challenge is going to be our creative ability to use these technologies when they arrive.