What's being projected is (mostly) CG, it's just not done through post production techniques. Â The system knowing the position and orientation of the robotic arm at all times relative to the camera POV (like how the realtime line of scrimmage and yards-to-go lines are created for live sporting event broadcast) makes the orientation of projection surfaces easier to derive compared to having to track them in post.Â
Â
You could, additionally, sync a second motion controlled camera to photograph a completely different live scene, instead of using pre-rendered or on-the-fly CGI, to project onto the surfaces. Â After 5th Element we built a system within Houdini to interface to the Digital Domain motion-control stage that allowed animators to design physically accurate moves for motion control miniatures rather than the screen-based methodology that was standard to the industry that often introduced "skating" and other kinds of motion or performance artifacts that have plagued flying miniatures for decades (though miniatures were phased out shortly afterwards so it was a neat proof of concept that never really got used). Â This adds a projection component to that sort of realtime telemetry and view-dependent spacial awareness.
Â
I didn't see specific info on the website or VIMEO but I'm wondering if the projector is on-axis with the composite camera via beam splitter or if they're pre-distorting the projection so that the projector can be fixed, up out of frame and not attached to the composite camera. Â
Â
It's neat looking, however it's done.



