Netflix claims to have developed a process that improves on the long-used green screen technique.
Netflix researchers stated in a post published on the public server arXiv that they have created a new technique dubbed Magenta Green Screen (MGS) that offers more accuracy and faster results.
The term comes from the lighting technique they employ. Actors receive light from the front with red and blue LEDs against a backdrop lit with brilliant green LEDs. The red and blue lighting gives the actors a magenta shine.
Because digital cameras record color values along red, green, and blue tracks, one track catches the backdrop (the green screen channel) with the foreground completely black, while the red and blue (magenta) channels record the foreground with the background completely black.
Netflix uses AI technologies to remove magenta colors from action movies by comparing images of performers in natural lighting to action scenes with hues.
“Our technique yields high-quality compositing results when implemented on a modern LED virtual production stage,” says Netflix senior researcher Dmitriy Smirnov.
According to Smirnov, the method may capture activity with filled-in virtual backdrops in real-time. There is no need for specialized camera equipment or manual color changes.
They tackled the issue of pixels at the perimeter of things that might be part of the object or backdrop.
“Partial coverage at edges, wispy and transparent structures, defocused and motion-blurred areas all exhibit partial transparency,” Smirnov said.
“Determining the RGB color of the foreground element at a pixel, as well as the pixel’s transparency, is … tricky.”
The researchers improved on a process known as triangulation, which involves filming stationary objects in front of several backdrops to generate very accurate definitions of the items.
“Our technique appears to outperform the matting that which can be obtained with either traditional chromakey green screen or time-multiplexed techniques,” Smirnov said.
“Computers already have provided such powerful tools to make a lot of stuff easier,” said Paul Debevec, another member of the research team. As a result, “the talented artists that we have can focus on the artistry, actually making things look better.”
Green screen technology has existed for almost a century. When “The Wizard of Oz” appeared in theaters in 1939, it captivated 45 million people with its extraordinary effects. The wonderful moments in Emerald City were the first to have actors performing in front of a green screen. They replaced it with footage of fanciful scenes shot at a separate time and location.
However, the technique is not flawless. For the most realistic effects, proper, well-balanced lighting is essential. Color constancy takes a hit as green leaks from the backdrop onto adjacent objects. Color fidelity also suffers from shadows on the green screen. And the mobility of tiny things, such as hair strands, can create disturbing visual artifacts.
Manual intervention can fix these difficulties, but it is time-consuming and does not always produce flawless outcomes.