If you’re a regular reader of The Focus, you’re already aware that movies from Technicolor’s award-winning visual effects studios contain a fair number of digital elements, from the computer-generated (CG) Sonic the Hedgehog to the seamless trench and village extensions of 1917.
Like any of their practical counterparts, these assets get reviewed by the show’s VFX supervisor and by the filmmaker. Before or during a live action shoot, they’d be looking to see how these digital elements fit with the practical set. If the asset is an environment, they’d want to virtually visit it. And if the scene is fully CG, then they’d even lay down some virtual cameras and test some angles. The latter process, common in filmmaking in general, is called set scouting, and game engines have particularly facilitated its porting to virtual shoots.
Sitting at the computer, the director/cinematographer/supervisor would click and move their viewpoint in the engine’s 3D viewer. Fancier: they can put a virtual reality headset on, walk around their 3D assets and contextually interact with them. But what if you’re the type of filmmaker who’s always on the go and would prefer a lightweight (literally) implementation of this service? Well, in that case, VPad might work for you.
VPad is a plugin for Unreal Engine (UE) 4 that can turn one level of your Unreal project into a standalone mobile scouting app. Once installed and launched, the app makes your phone or tablet the viewfinder through the which you can view your level, carry out all sorts of actions and make artistic decisions.
If you’re familiar with UE’s virtual production toolkit, you may have come across a built-in plugin called Virtual Camera or VCam. Although VCam was also made to allow you to use a mobile device as a viewfinder, it requires a permanent network connection between the said device and an Unreal workstation. Indeed, with VCam, the computer remains the place where the level is stored and streams, in real time, renders of it to the screen of the mobile device that acts like a controller in this case. Therefore, in addition to showing the scene with computer-like quality, VCam instantaneously propagates to the project all of the actions carried out on the device.
If VPad sacrifices these two benefits, it is to deliver its unique advantage: operating independently from a computer and thus, being more easily available to the user when they need it.
This straightforwardness also shows in the user interface of a VPad application. When started, the app doesn’t display a start menu but, instead, immediately drops you into your level in a first-person view. You can then look around by orienting the device itself and move around by touching two joystick-shaped buttons on the right and left sides of the screen.
Alternatively, if your device is equipped with augmented reality (AR) tracking technology (all of the Apple phones and tablets of the last 4 years do), you can simply walk and your viewpoint in the app will move with you. You can also double tap anywhere on the screen to teleport to that location. Right on top of the two joysticks are two arrows that change the field of view of the virtual camera. The rest of the controls surrounding the screen allow you to mark certain locations, teleport to those marks, hide specific objects, change the sun’s position or playback preloaded animation.
Most of them can be configured in the Unreal editor prior to building the app to decide, for example, which fields of view and animation sequences will be available or which objects can be hidden. Objects can also be repositioned in the app if labelled as such in the editor.
Finally, since you’re operating the device in your hands like a viewfinder all along, it is also possible to record camera animation tracks or video clips in the app. Camera tracks can be played back in the app itself or later manually sent to the Unreal project on the computer. Levels edits, in contrast, cannot be exported at this time.
(Main screen of the VPad app)
We’ve mentioned that VPad can take advantage of the device’s AR capabilities to replicate the user’s movements. But it can also use AR to integrate the 3D content with the live feed captured by the device’s video camera. A first mode puts the video feed as the background of the level, provided the said level has been built with a maskable skybox or no skybox at all in UE.
(Thanks to AR, static or animated assets can be viewed inside the real world)
Conversely, there’s a chroma key functionality that will insert your 3D content in place of any plain coloured (usually green or blue) surface you have on your set. Either modes can be activated from the Settings menu, which, by the way, offers even more customisation options for the app.
(With good lighting conditions, you can replace any plain coloured surface with a digital background)
The VPad plugin is maintained by MPC’s software department and is available to all of Technicolor’s VFX supervisors. Thanks to the early interest expressed by many, it has quickly joined the panoply of MPC Film's Previz department. Nevertheless, as the user base expands to more shows and creative studios, there is, of course, still room for refining the tool. – thefocus.com
Work with this kind of technology for award-winning projects at one of Technicolor's studios by applying through The Focus.