Virtual production: ILMxLAB and NVIDIA on Stars Wars, immersive entertainment, and real-time ray tracing

It’s Star Wars in real time! In the latest podcast in our Visual Disruptors series, host Mike Seymour chats with Mohen Leo from ILMxLAB, Lucasfilm’s immersive entertainment division, about the virtual production techniques that led to VR experiences like Star Wars: Secrets of the Empire. He also talks with Rick Grady at NVIDIA about the groundbreaking RTX technology behind the Reflections video, which showcased real-time ray tracing on three Star Wars characters.

Scene from “Star Wars: Secrets of the Empire”

Leo, Creative Director and Visual Effects Supervisor at ILMxLAB, explains that ILM’s interest in virtual production is what led his group to develop experiences like Star Wars: Secrets of the Empire. “ILM as a whole realized that if you can use these techniques to let directors interact with the world of Star Wars, then audiences would want to interact with the world of Star Wars,” he says. “We’re trying to basically have the end result be something that’s actually available to consumers.”

Mohen Leo of ILMxLAB

While embracing real-time rendering, ILMxLAB is ever-mindful of maintaining the visual standards ILM is known for. They chose location-based entertainment, as opposed to home VR, as the vehicle for their VR experiences because they felt this would provide the highest-quality sensory experience. With location-based entertainment, participants visit a fixed location set up specifically for the VR experience, such as an amusement park or museum, where the hardware can include enough computing power to ensure the visual fidelity meets the ILMxLAB standard.

Scene from “Star Wars: Secrets of the Empire”
 

As a result of this exploration into virtual production and real-time rendering, ILMxLAB teamed up with THE VOID to produce the immersive VR experience Star Wars: Secrets of the Empire, where up to four players can enter VR together as avatars to meet a challenge, interacting with physical objects and even feeling wind on their skin. 

“Unreal Engine and NVIDIA have been helping us because they get the tools that we use,” he says. “You don’t have to apply a whole different mindset to trying to get a realistic-looking image in real time anymore.”

Together, ILMxLAB and THE VOID also produced Ralph Breaks VR, an immersive world where players, as cartoon avatars, join familiar characters from the Wreck-It Ralph film series in an animated adventure. 

ILMxLAB worked toward making sure each of these experiences “isn’t just something you watch, but you can actually step into and become part of,” says Leo. And they’ve got a whole lot more coming, including a VR experience based on a Marvel series.

In the second part of the podcast, Rick Grady, Senior Solutions Architect at NVIDIA, explains the RTX technology that makes real-time ray tracing possible.

NVIDIA stunned the VFX community at GDC 2018 when, in conjunction with Epic Games and ILMxLAB, they used RTX to showcase real-time ray tracing in Unreal Engine on some very shiny Star Wars characters in the video Reflections. After that, “pretty much every studio was calling us asking how they get their hands on it—now!” laughs Grady.

He explains that RTX architecture introduces two new types of cores for distributing rendering tasks, Tensor and RT, with RT devoted to accelerating ray tracing. Grady also explains how NVIDIA’s denoising techniques reduce render time for reflections and shadows while keeping visual quality sharp.

Ray tracing with RTX isn’t yet part of Unreal Engine, but NVIDIA continues to work with Epic to push the envelope and advance this technology, as evidenced by the Speed of Light demo at SIGGRAPH 2018. 

“You have to present an entire platform in order for you to get widespread adoption,” Grady says. “Everyone wants RTX. Now we’re mainly supporting the software vendors to help them get the software out to market, so people can actually utilize this within their production workflows.” 

There’s a whole lot more packed into this podcast, so be sure to listen to all of it! And visit our Virtual Production hub to get more videos, podcasts, and insights into this emerging field.

Virtual production: ILMxLAB and NVIDIA on Star Wars, immersive entertainment, and real-time ray tracing

It’s Star Wars in real time! In the latest podcast in our Visual Disruptors series, host Mike Seymour chats with Mohen Leo from ILMxLAB, Lucasfilm’s immersive entertainment division, about the virtual production techniques that led to VR experiences like Star Wars: Secrets of the Empire. He also talks with Rick Grandy at NVIDIA about the groundbreaking RTX technology behind the Reflections demonstration, which showcased real-time ray tracing on three Star Wars characters.

Scene from “Star Wars: Secrets of the Empire”

Leo, Creative Director and Visual Effects Supervisor at ILMxLAB, explains that ILM’s interest in virtual production is what led his group to develop experiences like Star Wars: Secrets of the Empire. “ILM as a whole realized that if you can use these techniques to let directors interact with the world of Star Wars, then audiences would want to interact with the world of Star Wars,” he says. “We’re trying to basically have the end result be something that’s actually available to consumers.”

Mohen Leo of ILMxLAB

While embracing real-time rendering, ILMxLAB is ever-mindful of maintaining the visual standards ILM is known for. They chose location-based entertainment, as opposed to home VR, as the vehicle for their VR experiences because they felt this would provide the highest-quality sensory experience. With location-based entertainment, participants visit a fixed location set up specifically for the VR experience, such as an amusement park or museum, where the hardware can include enough computing power to ensure the visual fidelity meets the ILMxLAB standard.

Scene from “Star Wars: Secrets of the Empire”
 

As a result of this exploration into virtual production and real-time rendering, ILMxLAB teamed up with The VOID to produce the immersive VR experience Star Wars: Secrets of the Empire, where up to four players can enter VR together as avatars to meet a challenge, interacting with physical objects and even feeling wind on their skin. 

“Unreal Engine and NVIDIA have been helping us because they get the tools that we use,” he says. “You don’t have to apply a whole different mindset to trying to get a realistic-looking image in real time anymore.”

Together, ILMxLAB and The VOID also produced Ralph Breaks VR, an immersive world where players, as cartoon avatars, join familiar characters from the Wreck-It Ralph film series in an animated adventure. 

ILMxLAB worked toward making sure each of these experiences “isn’t just something you watch, but you can actually step into and become part of,” says Leo. And they’ve got a whole lot more coming, including a VR experience based on a Marvel series.

In the second part of the podcast, Rick Grandy, Senior Solutions Architect at NVIDIA, explains the RTX technology that makes real-time ray tracing possible.

NVIDIA stunned the VFX community at GDC 2018 when, in conjunction with Epic Games and ILMxLAB, they used RTX to showcase real-time ray tracing in Unreal Engine on some very shiny Star Wars characters in the Reflections cinematic. After that, “pretty much every studio was calling us asking how they get their hands on it—now!” laughs Grandy.

He explains that RTX architecture introduces two new types of cores for distributing rendering tasks, Tensor and RT, with RT devoted to accelerating ray tracing. Grandy also explains how NVIDIA’s denoising techniques reduce render time for reflections and shadows while keeping visual quality sharp.

Ray tracing with RTX isn’t yet part of Unreal Engine, but NVIDIA continues to work with Epic to push the envelope and advance this technology, as evidenced by The Speed of Light demo at SIGGRAPH 2018. 

“You have to present an entire platform in order for you to get widespread adoption,” Grandy says. “Everyone wants RTX. Now we’re mainly supporting the software vendors to help them get the software out to market, so people can actually utilize this within their production workflows.” 

There’s a whole lot more packed into this podcast, so be sure to listen to all of it! And visit our virtual production hub to get more videos, podcasts, and insights into this emerging field.

Mood Lighting: How Metro Exodus Uses Ray Tracing to Heart-Pounding Effect

The Star Wars-themed “Reflections” demo is so visually stunning, it’s natural to assume it’s a film clip. I felt that way the first time I saw this real-time ray-tracing showcase, at GDC 2018, and so have the hundreds of people I’ve shown it to since. Hitting a hotkey to reveal the wire-frame leaves you floored. Read article >

The post Mood Lighting: How Metro Exodus Uses Ray Tracing to Heart-Pounding Effect appeared first on The Official NVIDIA Blog.

It’s ON! Putting Next Gen in the Hands of Tens of Millions of Gamers

Gaming is thriving. An entire generation is growing up gaming. Two billion gamers and counting. They play at home, on the go and on multiple platforms, and by far the most vibrant, innovative and open is the PC. The forces driving PC gaming are staggering in scale. Take the battle royale genre — think Fortnite, Read article >

The post It’s ON! Putting Next Gen in the Hands of Tens of Millions of Gamers appeared first on The Official NVIDIA Blog.

NVIDIA Teams with Autodesk, RED to Bring 3D Animation, 6K Video Editing to Laptops

Whether you create complex 3D animations or edit high resolution video, you will soon be able to do your best work on the go. This week at the Consumer Electronics Show (CES) in Las Vegas, we’re demonstrating how we’re bringing hardware-accelerated ray tracing, AI-enhanced graphics, and advanced video processing to our new, thin and light Read article >

The post NVIDIA Teams with Autodesk, RED to Bring 3D Animation, 6K Video Editing to Laptops appeared first on The Official NVIDIA Blog.

Doing Reflections and Shadows Justice: Ray Tracing, DLSS to Transform MMO Games

A row of red lanterns reflected in a warrior’s shining scales of armor. Shadows cast by children running down the paved pathway of an open-air market. A lit carriage reflected onto colorful ceramic vases that line the street. These stunning ray-traced reflections and shadows will soon enhance the game environment of Justice, one of China’s Read article >

The post Doing Reflections and Shadows Justice: Ray Tracing, DLSS to Transform MMO Games appeared first on The Official NVIDIA Blog.

Prepare for Future of Design: Introducing the Quadro RTX 4000

NVIDIA today introduced the Quadro RTX 4000 graphics card — the company’s first midrange professional GPU powered by the NVIDIA Turing architecture and the NVIDIA RTX platform. Unveiled at the annual Autodesk University Conference in Las Vegas, the Quadro RTX 4000 puts real-time ray tracing within reach of a wider range of developers, designers and Read article >

The post Prepare for Future of Design: Introducing the Quadro RTX 4000 appeared first on The Official NVIDIA Blog.

NVIDIA, Adobe to Bring Interactive, Photo-Real Ray Tracing to Millions of Graphic Designers

Oil and canvas. Thunder and lightning. Salt and pepper. Some things just go together — like Adobe Dimension CC and NVIDIA RTX ray tracing, which are poised to revolutionize the work of graphic designers and artists. Adobe Dimension CC makes it easy for graphic designers to create high-quality, photorealistic 3D images, whether for package design, Read article >

The post NVIDIA, Adobe to Bring Interactive, Photo-Real Ray Tracing to Millions of Graphic Designers appeared first on The Official NVIDIA Blog.

By the Light of the Moon: Turing Recreates Scene of Iconic Lunar Landing

If you’re going to fake a moon landing, you’re going to need the world’s most advanced GPUs. Four years ago, our demo team used GPUs to debunk the myth that the Apollo 11 moon landing was a hoax. So thoroughly, in fact, that it’s become a bit of a joke that the best way to Read article >

The post By the Light of the Moon: Turing Recreates Scene of Iconic Lunar Landing appeared first on The Official NVIDIA Blog.