LED Wall Evokes Early-Cinema Effects for Poor Things

This article was originally published on theasc.com  by Noah Kadner.

Most scenes in Yorgos Lanthimos’ highly stylized dark comedy Poor Things were shot on traditional sets, but key portions of the steamship journey that Bella (Emma Stone) embarks on with Duncan (Mark Ruffalo) called for virtual production on an LED volume — a first for both Lanthimos and cinematographer Robbie Ryan, BSC, ISC. (Our complete story with them here.)

The team led by Simon Hughes, creative director at London VFX house Union, included visual-effects supervisor Tim Barter and virtual-production supervisor Adrian Weber.

Surreal Effects

Weber recalls that he was initially brought onboard to advise Ryan about the technical considerations involved in shooting on the volume, where surreal skies and seascapes would be displayed behind the actors on deck and, in a separate setup, behind a miniature of the ship.

“Robbie was shooting on film and looking for someone to review the details and help him understand everything necessary to shoot on the volume,” says Weber. “The main reason they decided to shoot with virtual production and not greenscreen was that the ship had lots of glass and other reflective surfaces. They knew an LED volume would give them an excellent visual starting point they could extend out with additional effects, as needed.”

Lanthimos wanted to evoke early-cinema visual effects such as rear projection and cycloramas, so realism was decidedly not the goal.

“We were intentionally trying to mimic a filming style of early cinematography, where a model looks like a model,” says Barter.

“The water needed to look somewhat surreal — not completely realistic, but not modern CG water, either. The art department referenced the fluid-effect artwork of photographer Chris Parks, who does these slightly wispy, ethereal, evolving image pieces, so the style was a little like lava lamps or ink undulating in water. We combined that with mood boards and other photographic references and eventually developed a style for each scene that worked for everyone.

Ultimately, we created nine different LED skies: six settings, three with nighttime variations.”

Bespoke Solution in Budapest

To provide the filmmakers with maximum flexibility, the production created a bespoke volume at Origo Studios in Budapest, Hungary, where the production was based. ICT AG and Halostage constructed and prepared the volume over approximately five weeks.

The volume comprised 2,400 Infiled 2.6mm LED panels arranged in a 197’x33′ semicircle, with NovaStar processors fed by four Pixera One servers.

The wall resolution was 23,040×3,840 pixels running at 10-bit color and 24 fps. Instead of LED panels overhead, gaffer Andy Cole and crew installed approximately 150 Arri SkyPanel S60s for fuller-spectrum cinema lighting. Meanwhile, four Arri 18K HMIs simulated sunlight in the scenes.

Ryan shot the volume material on film with an Arricam ST camera — as he did for much of the main production — using primarily Kodak’s Ektachrome 100D 5294 stock for the top-deck work and Vision3 500T 5219 for interiors.

To sync the film camera’s shutter speed to the LED wall’s frame rate, Weber notes, “The sync signal was generated by a Rosendahl Nanosyncs HD, converted to analog, and it was processed in the camera via the Arricam “SCB” Speed Control Box — which originally was designed to match the recording frame rate to old TVs — which also supported sync-shift to match the image playback.

Rewriting to Render

“Because the environments were so big at 23K, our initial challenge was figuring out how to play back everything smoothly,” says Weber. “Union’s engineers had to rewrite their entire render pipeline to output the backgrounds in sections. That worked out to four chunks, one per Pixera processor.”

2D Plates, 3D Assists

The Union team created background sea/sky plates as 6,144×3,840-pixel NotchLC files, four per environment, played back in sync on the whole volume. Because the horizon was reasonably far from the actors, the production eschewed 3D environments and mostly shot fixed 2D background plates without camera tracking.

“The main factor in going 2D was Yorgos’ wishes,” Barter says. “Evoking early cinema’s rear-projection and cyclorama effects was a creative choice that resulted in a slightly askew perspective on the horizon. Of course, the assets going into those 2D plates included 3D elements such as the water, which we created in [SideFX] Houdini and [Autodesk] Maya, and composited with [Foundry] Nuke.

“Though our created background sea/sky plates were 2D, we did, in fact, utilize Unreal Engine to help review how the plates would look on the full-size live-action ship set,” he continues.

“As part of their design process, the Poor Things art department had 3D realizations of the ship’s deck and interior in Unreal Engine. So, [I] proposed bringing into Unreal Engine lower-resolution iterations of Union’s plates.” The plates were then projection-mapped to a 2D “surface” that displayed behind the Unreal Engine ship — which mimicked how the LED wall’s imagery would appear behind the actual miniature ship.

“This meant that we were able to see how the horizon line of our plates responded to different camera positions, and to assess the speed of the ocean as it passed beside the ship, as seen from every camera viewpoint. Although further adjustments would go on to be made to the shots during post, this pre-shoot enabled Lanthimos [and I] to review and assess how the LED content looked in relation to the set, well before the LED screen and set were finished being built.”

Manual Operation

The original plan was to synchronize the video content with the lighting board via DMX programming, but ultimately this essentially wasn’t necessary. “We gave the lighting team the environments,” says Weber.

“They were working with the GrandMA, and we had the idea to take a low-res version of the plates to use as a base for the ‘image-based lighting’ approach and sync the start/stop of the timelines between Pixera and the light board. But [because] there weren’t any sudden changes in light, and it was easier to trigger the loops independently, for the most part they operated the lights over the volume manually.

Miniatures Against the Wall

In addition to overseeing this relatively traditional volume work, Barter worked on the miniature shoot, which included a depiction of the ship’s progress shot in front of a much smaller wall. The footage was again shot on the Arricam ST — primarily on 5219 stock, with some use of Ektachrome 5294 as well.

“We had three days to capture all the model work, which was shot by Tristan Oliver [BSC],” says Barter. “The miniature ship was mounted on tracks in order that it could glide slowly through the shot against the LED-wall skies. The [water] and ship’s wake were added during postproduction, utilizing the [CG] seas created for the live-action LED ship shoot.”

The production initially considered shooting the miniature ship in real water, but this would have required high-speed filming to adjust the scale of the water so the movement of the waves would appear full size — and given the limitations of the LED wall’s refresh rate, shooting at high speed was not an option.

Other miniature work that required LED-wall skies included the city of Alexandria with its dilapidated slum area, and the mansion and grounds of Alfie (Christopher Abbott), who enters Bella’s life toward the end of the film. “These were both shot on raised platforms,” Barter says.

Parting Shots

Some angles that Lanthimos composed captured frames beyond the physical boundaries of the LED wall, which was a “predetermined directorial choice,” Barter says.

“A key part of Yorgos’ style is wide-angle shots, which could be slightly exasperating because we had this giant screen and were still managing to shoot off it! But we were expecting that and prepared to do digital set extensions in every instance as needed. We even changed some of the skies and parts of the water in postproduction as the edit evolved. That said, one of the great boons of using LED volumes is the lighting and reflections you get from the live environments. Even when you alter the backgrounds in post, it doesn’t change the overall bakedin grade reflected onto the actors, the sets and the models. It all works out very nicely.”