The Projection Mapping in Girls and Boys

From the 8th of February to the 17th of March at the Royal Court Theatre in London, the Royal Court had the play Girls and Boys being performed in the Jerwood Theatre Downstairs. Girls and Boys was written by Dennis Kelly, directed by Lyndsey Turn, and starred Carey Mulligan. One of the major set pieces was the whole of the stage itself, as it was designed to be both a blank box from which the character portrayed by Carey Mulligan would directly address the audience, and then the back of the box would rise to reveal a fully furnished apartment painted a vibrant turquoise. What made both this set and the overall show unique was the implementation of video mapping and projection over the whole of the apartment set, strategically and specifically transforming it from the turquoise blue from the start of the show to a fully colored and realistic looking apartment by the show’s closing moments. Not only did this lighting method demonstrate the advances in technology that can be incorporated into an artistic medium, but more specifically served a narrative function as a means of subtly conveying the overall story of the play throughout the show’s runtime.

Projection mapping is the use of projection technology in tandem with everyday objects to transform irregularly shaped and off-white objects into a place on which an image can be projected. One of the earliest documented cases of public projection mapping was developed and used by Disney in 1969 within the Haunted Mansion ride at Disneyland. The ride has singing bust statues, and the projection was done by filming the signers singing via a head shot and then projected onto the stationary busts within the ride to give them the appearance of being animated and alive. Projection mapping began to gain more traction when it was pursued in academia.

Spatial Augmented Reality was a paper written and published in 1998 at UNC Chapel Hill by Ramesh Raskar, Greg Welch, Henry Fuchs and Deepak Bandyopadhyay et al, which spoke about how SAR could potentially expand the modern day workspace by incorporating augmented reality into the workforce, bridging the gap between various places and people. It was published alongside a paper called The Office of the Future. The Office of the Future envisioned a world where instead of staring at a small computer monitor, projectors could cover any surface allowing the user to be able to experience and work within augmented reality from their desk. Proposed benefits of this technology was the ability to have conferences via the use of multiple cameras and projections with life-size versions of office mates, view life-size virtual 3D models projected onto the workspace, amongst others. This was proposed as a mean to create an effective illusion of virtual objects coexisting with the real world, enhancing the user's view at work with images of virtual objects via the use of augmented reality techniques and technology. It was proposed to be done by introducing virtual objects that were rendered directly within or on the user's physical space. A key benefit of the proposed technology was that the user did not need to wear a head-mounted display. Instead, with the use of projection mapping onto specific displays and fixtures, wide field of view and possibly high-resolution images of virtual objects could have been integrated directly into the environment. It has since developed, both in academia and as an artistic medium as a means of turning anything from drapes to cars to entire buildings into screens on which someone could project and manipulate the environment.

By using specialized software, any 2- or 3D object is spatially mapped within the virtual program which mimics the space that the image is to be projected on. This technique is used by both artists and professional businesses to add extra dimensions, optical illusions, and notions of movement onto previously static objects. Upon choosing the object which will be projected onto, mapping software is used to map the corners of the video to the surfaces within the 3d space of the program itself. First, the image or video wishing to be projected is chosen and imported into the software. Then, the image is placed onto the model of the object in its desired outcome. When mapping a projection onto a 3D object, the coordinates of the image being projected need to be defined for where the object is placed in relation to the projector, the XYZ orientation of both the object and image within the real world, position, and lens specification of the projector in order to create the desired result of the projection. Following that, the final step in the process is what is called 'masking,' which means distorting the image’s and object’s edges via the use of opacity filters and templates to disguise the exact shapes and positions of the different elements of the projection and blend both image and object together in the real world.

The apartment set and the accompanying video mapping was the fruit of the collaboration between the show’s set, lighting, and video designers: Es Delvin, Oliver Fenwick, and Luke Halls and his video design studio respectively. After building the set out of real household furniture and objects, set designer Es Devlin then 3D-scanned each item and surface. Once all the surface data had been recorded, Devlin and her team then painted the entire set turquoise. Video designer Luke Halls then used the surface data provided to create a virtual version of the set with all its original colours, which was then intermittently and precisely mapped back onto each surface throughout different points in the play, with the final scene of the play being the apartment returning to its former state, as the original colors have all been mapped onto the stage, replacing the turquoise blue with the real world.

Universal Pixels was approached by Luke Halls’ own studio to provide aid with the projections and masking to return Es Devlins’ set into its original state. Luke Halls states “ ‘a LIDAR scan of the set was required to ensure a very accurate detailed map could be built. Once this had been interpreted, and loaded into the disguise 3D environment, it allowed us to really accurately highlight elements of the set at a moments notice… A combination of Epson LB1755 & LB1505 projectors were used.’ ” Oliver Luff, who works for Universal Pixels, said “ ‘Epson units were specified due to their minimal noise levels which were critical for the auditorium placement of the projectors and the nature of a theatre show.’ ”

Narratively speaking, the use of projection mapping within Girls and Boys serves the narrative function of cluing the audience into the fact that Carey Mulligan’s character is going through the process of erasing her husband from her memories as a means of reclaiming those happy memories for herself. Es Devlin states “ ‘The key to the design is the 'hinge' between the states of direct engagement with audience and interaction with remembered space and children… Each time the character engages in a memory sequence, a normal looking room gets instantly fogged and muffled in pastel turquoise matt colour… Following this, each time the play transitions from a section of monologue to an enacted memory, the black screen disappears and the virtual 3D scan is projection mapped precisely onto every object and surface in the room, momentarily returning it to its original full colours.’ ” During the play, Mulligan's character switches between delivering monologues directly to the audience in the blue box, and acting out memories of her two invisible children in the monochrome turquoise living room setting when the back of the box is raised. It becomes clear that she is reenacting and telling the audience members these moments in her life in an attempt to rewrite her own memories, erasing her husband from her memories after the tragic loss of their children. This is where the specific use of projection mapping comes into play. During the scenes in the living room in which she is reliving the memories of her children, while subtle at first, certain objects in the background of the set start to stand out as they are shown with their original color projected onto them. This is done to clue the audience into the fact that the rewriting of her memoires is working, as the solid visual indicator of a colored object within the sea of monochromatic blue demonstrates a key change in the zeitgeist of the world she is living in.