Lighting hasn't really changed in the past few decades. Sure, new fixtures, faster cameras and lenses, and shooting styles may change, but the essence of lighting really hasn't changed. The look of the scene is still driven by the DP and still created by the lighting designer; it is still all about ratios, and no matter how technology advances, lighting is a powerful tool for putting realism and drama on screen.
What Is Image-Based Lighting?
Simply put, image-based lighting (IBL) is using lighting fixtures to mimic the effects of natural surroundings in a virtual stage. Conceptually, this isn't all that different from blasting a light through a window from outside in a traditional set to mimic sunlight streaming through a window or backlighting a character who is standing near a practical. With IBL, you are creating nuanced light effects on the talent and props in a virtual production environment, selling the set as if it was a real location. Using the fixtures, much as you would in a real set, but you also have the potential for creating not only naturalistic lighting effects, but dramatic and powerful lighting as well.
Working in a Virtual Production Stage
The virtual production stage is a combination of LED volume and LED lighting units controlled and powered by an array of servers and computers to produce background images and lighting effects in real time. Unlike a simple video wall or digital signage that is playing back images, the images on the LED volume are synched with your camera via software, allowing both to work in concert, adjusting the background in depth, adjusting parallax as the position of the camera changes.
What Is the Point of Image-Based Lighting?
OSVP (on screen virtual production) is far more than just a virtual set or blue/green screen that uses software to combine your foreground and background in post. The video wall component is more like an interactive rear screen projection, so the camera captures the complete image foreground and virtual background without post processing. Throw image-based lighting, which uses LED lighting fixtures that provide pixel level control, similar to the LED volume/wall, into the mix, and you can have video playing on the LED fixture, helping to light the scene, create the mood, and sell the effect of being in a real location.
Take, for example, a "moving car" shot. You can go shoot on a process vehicle in the real world, having to deal with traffic and sound issues. Resetting back to the beginning of the shot will entail either starting over from wherever the car is when you start over, thus creating background continuity issues, or taking a fair amount of time to get back to your starting point. Again, you must be concerned about changing lighting conditions. Or you can shoot in a studio, with more control, and lots of gag (supplementary) lighting. However, image-based lighting (IBL), whether as part of a virtual production set, or even with the IBL lighting units on their own, allows you more consistency and control. In the driving shots, imagine a gathering storm as the background playing on the LED volume. You can also play that video on lighting fixtures that are illuminating your virtual production set. Using OSVP, as your car "drives" and the background changes, the units lighting your characters will show the video as well, providing an in-sync change to the light hitting your talent. Selling the effect, making it appear more natural, and really putting your talent within the virtual environment. Add lightning pulses synched to the virtual storm approaching your talent, and the lightning pulses can be synched to the video, and always strike at the same time, no matter how many takes you do.
Another advantage of shooting with virtual production, aside from avoiding the costs of shooting on location, is that the light won't change as your day progresses. For example, if you are shooting a morning scene and it takes 8 hours to shoot, well the sun moves throughout the day and what started off as a low angle key from the left with luscious long shadows, soon turns into an overhead lit scene with short shadows, and finally at the end of the day, with the sun now coming from the other side, casting shadows in the wrong direction, it becomes a lighting continuity nightmare. Let's not mention the change in the color temperature as the sun moves through the sky, which entails more time and money spent in post. Shooting on a virtual production set and using image-based lighting can make your production (and postproduction) life simpler.
Why Not Just Use a Video Wall to Light Your Talent?
Simply put, video walls are not cheap, and while video walls are useful for looking directly at an image, they do tend to have narrow color reproduction, so while it may look good as you look at the screen, the equivalent CRI of the video wall as a light source is limited. This means any light from the video wall reflecting off your subject, which is what a lighting fixture provides, will be lacking in color reproduction making for a poor image. Additionally, since the wall is meant to be looked at, as compared to a lighting fixture, the brightness of the output will be limited as well. Lighting fixtures are designed to illuminate, while LED walls are designed to be looked at. On top of that, light spill from LED panels mixed with your IBL talent lights will make the final result appear more natural and realistic.
Of course, all this must be controlled via your system. Sometimes it takes multiple systems to control the image playback and coordinate the image-based lighting as well as control your camera. While DMX control of lighting units has been around for a significant amount of time, as far as image-based lighting goes, being able to map each pixel of the fixture with an image and brightness and have it update in real time at 60, 120, 240, and even faster fps creates a whole range of possibilities.
Not Just Another Light Fixture
But not just any lighting fixture makes it as an image-based lighting unit. An IBL fixture works best when your control/image playback software can control each pixel on the fixture, such as with the KINO FLO MIMIK, Creamsource Vortex4 and Vortex8, or a bank of Quasar Rainbow 4-Bank Light Kit. This allows you to create an overall color cast, adjust the overall brightness, and have the OSVP software adjust and control each and every pixel of your light fixture using software such as from Unreal engine, Assimilate, Aperture, and Light Gear, which can pixel map video on the light unit.
Breaking Down Some of the Lighting Units That Work for IBL
With the Creamsource Vortex4 and Vortex8, these units do not completely recreate the video exactly as if you are watching the video on a screen, instead transforming it into a blocky recreation with powerful output, allowing for movement to be expressed.
Similar to the Vortex units, the Quasar Double Rainbow 2 is a linear tube, and creating a bank of these tubes provides a less defined representation of your video, providing a soft output that captures the feel of the lighting effects without creating a sharp video representation. But again, the Vortex and Quasar are being used to project the ambience of what would be naturally occurring in the environment that you are virtually recreating—now that's a mouthful.
Kino Flo MIMIK creates a much more realistic representation, and it is great to use not just for creating realistic light effects on your talent, but also for creating reflections visible in the scene, be it on an actors eyeglasses, or say a drinking glass. Similar to your video wall or volume, it displays the video you send to it, so it is ideal not only for lighting and effects, but for generating the live reflections that our brain would miss if the reflections weren't there.
Conclusion
One thing to remember is that image-based lighting can be used in standard production environments in addition to virtual production. So, for example, consider the idea of a fire effect: with image-based lighting, gone are the days of someone playing with a dimmer to adjust your light source's brightness, which is doubling for a real fire. Likewise, those wonderful rigs that we used to make using several layers of different colored gels that you wiggle in front of your light for making for random light flickers are now easily replicated and improved on by using the recorded image of fire itself as your light source, with the advantage of no gel noise and a very organic and natural feel to the fire's ebb and flow.
I hope this article has been helpful as you are considering moving into virtual production. To see these concepts put to use, check out B&H Photo's video on IBL hosted by Steve Giralt.
