Four Fundamentals for Shooting XR
As an XR technician, I love solving problems so myself and others can be confidently creative, and creatively confident. There’s no definitive playbook yet for shooting on an XR stage, and nor should there be. All stages are different, and are built for different things! One of the most fulfilling parts of my job is to guide director’s and DOPs on what it takes to shoot on XR, and what tools that XR provides that help them clarify and accelerate their vision. This blog post will ask and address some important questions that a director, DOP, or producer can expect while planning an XR shoot.
Framing Reality
The main principle of shooting XR is having your scene background be an image on a LED wall, that you record in camera. You’ll likely have some midground set decoration like furniture, plants, a car, or anything else, but your background should fill your camera frame edge to edge. Obviously, you don’t want to shoot off the edge of the led wall. Here, its important for the DOP to step on stage, ideally with camera, and get a sense of the space. Where can you shoot? How wide? How much can I tilt up or down? How far can talent walk and still fill the camera frame with LED background? This becomes very important, unlike shooting on chroma green which has a bit more flexibility in this space. When in doubt about framing and lensing, the best answer is to tech scout the stage. A stage’s XR technicians can also provide scale 3D models of the stage and use other virtual pre-production techniques to test shots if necessary.
When in doubt, step on stage and look at it through camera.
Screen Size and Real World Scale
At IR Studios XR, we had an automotive commercial director come in for a tech scout. She wanted to see how wide a lens she could shoot with, so she came to us. Keep in mind, most of our car commercials before this had fairly tight shots of talent behind the wheel. The director wanted to show big grand shots of an expansive truck interior and we worked with her to test our limits. We had a similarly sized truck on stage and did camera tests against the led wall. The director’s time on stage helped her not only visualize the shots possible, but get a feel for the space and light that emits from the led wall. It’s a magical moment when you can see in- camera the fundamentals of your composed shot, and know “yeah, this is going to work“.
Video plates should be captured to display at the correct scale.
Unreal Engine and Video Plates
There are generally two flavors of XR we do at IR. One is using video plates on the led wall, and the other is rendering an environment with Unreal Engine in real time. These different methods are worth having their own blog posts, but for now, the main difference is how content is loaded onto our render server and adjusted onscreen.
Video plates are captured or otherwise provided by the client, and then layered and arranged on the led screen behind the subject. This arrangement happens after camera is set so the background is essentially “moved into place” behind the subject. When the camera changes position for a new shot, the XR team moves the background to match. One important thing is that video plates will not change perspective when the camera changes position. Video plates are great for locked camera positions or nodal pans and tilts, but trucking or dollying too much will break the illusion.
With Unreal Engine content (and a 3D camera tracker), the position of the camera actively changes the perspective of the image on the LED wall in real time. Camera moves create accurate parallax between subject and background. Unreal also gives far greater control over the content on screen. Its exciting to have this kind of control on set, and the real time parallax effect is a sight to behold.
Prep Content in Advance!
With each workflow, content for the LED wall must be ready before day 1 of principal photography. A common question for us is “how much time before production do you need the video or Unreal project files?” The answer is- it depends on the state of the files themselves! We have a list of requirements for LED content, and once we can ingest them to test on our stage, we can give an estimate on what it takes to get them ready. As XR technicians, getting files to us sooner is always better. We always reserve time for edits after the initial test on stage, and before day one of production. We always have a full day where we have final content on the led wall for production to approve.
In my experience, a good day is when XR has the full list of backgrounds prepped and ready for each shot on the shot list before that day. Any “day of” changes for LED content are usually simple color correction to help the background look good in the lighting conditions on set, or minor scaling, placement, or layering solid colors for light sources. Like other departments, the most important creative decisions are made in pre production, and executed in production. Once there’s a good flow of communication between production and the XR team, scene changes become fast and seamless!
Your background is “ready” when it looks good in camera.
At the end of the day, the most important question to ask is “does it look good in camera?”. This is the joy of shooting XR as opposed to chroma green. Once you have a background visible in camera, shot composition, blocking, and performances become much more intuitive. Cast and crew sync up when they see the same space they’re working in. I’ve had crew tell me they prefer shooting on an XR stage than traditional production! Shooting XR is a rewarding production experience, and with the right technical expertise, it’s intuitive and accessible to any director, DOP, or producer.
