Andy Serkis is changing the face of acting—literally. His breakthrough-role as Gollum, the five hundred year old schizophrenic hobbit in Peter Jackson’s Lord of the Rings movie trilogy, led Serkis to realize how performance-capture technology allows actors to play non-human roles with incredible emotional depth and lifelike visuals. The Imaginarium, his studio that was inspired by his work as an actor, is at the forefront of CGI and performance capture technology.
Serkis, alongside independent film producer Jonathan Cavendish, co-founded The Imaginarium, an effects studio based within Ealing Studios in west London in 2011. They describe it as a laboratory for the development and exploration of the performance capture (or motion capture) art of characters, from Hollywood and independent films, television, video games, to theatres that perform for the virtual world.
The interior of The Imaginarium is unlike any other studio in the world—as you walk in, an empty arena (known as The Volume), is surrounded by 48 LED infrared cameras. On the floor are tape markings showing where the cameras move attached to a metal frame work. The tape also indicates the positioning of actors and actresses who perform in black suits dotted with tiny Styrofoam balls covered in reflected tape. The performers’ recorded image is translated into an avatar, which is captured by a technician that wears a pair of virtual goggles. The avatar, a roughly rendered character in grayscale, is viewable in real time.
Motion capture, which has rapidly developed over the past decade since Gollum first appeared on the silver screen, involves filming a real-life actor and then turning their recorded movements as an avatar into a CGI character. But transforming the avatar into a convincing photo-real character with complex emotions and realistic movements is a challenge.
A strong pre-production is the first and very important step. Preparations must be made such that the intricate shooting stage will run smoothly and be a dynamic, interactive environment for the actor to give their best performance, which requires an impressive amount of skill in itself given the artificial environment. High-quality data is of utmost importance.
In post-processing, the data must be reconstructed, a process that involves taking the optical data (the dots from the Styrofoam balls on the actor’s black suit) and converting it into continuous 3D trajectories. At first, the dots are merely a cloud of 3D data and each individual piece must be labeled to give the markers an identity that is associated with the underlying skeleton of the human body.
A motion capture system has a 360 degree view of the marked performer. This is possible by having at least three cameras tracking a 2D view. The data collected from the three (or more) cameras contribute to the making of the 3D data. More cameras mean better vision and overall high quality data. But certain stunts or character-to-character interactions can hide the visibility of some cameras. The holes in visibility must be filled in to avoid sporadic key frames.
The end result is the “solve” of the performer’s skeleton with accurately labeled trajectories and is essential for the authenticity of an actor’s performance to be translated into the CGI character.
For Serkis, performance capture is simply digital make-up. As cosmetics and costumes enhance the performance of traditional actors and actresses, performance capture technology enhances performance, but with the added edge of bringing to life non-human characters.
Laymen audiences oftentimes mistake performance capture with voice-acting, not realizing that the performer is responsible for the whole package: motion, expression, voice. But that view is quickly changing. Serkis has proved, with his roles as LOTR’s Gollum, Caesar in Planet of the Apes, Peter Jackson’s King Kong, and Steven Spielberg’s Captain Haddock in The Adventures of Tintin, that it’s possible to create a strong emotional character that carries a story’s narrative despite being digitally generated.
Technical and creative teams work together to combine technology, storytelling, and character development innovation pioneered by Serkis. There are no visual-effects secrets for his performances. Such a multi-dimensional creative driving force is at the heart of The Imaginarium.