3D Art
CG Character / Prop Fabrication
3D Art
CG Character / Prop Fabrication
3D Art
CG Character / Prop Fabrication
3D Art
CG Character / Prop Fabrication
Santiago: A Multi-sensory Art Installation
Brief
*When there are no quotes (Santiago) I'm referring to the character/sculpture and when there are quotes ("Santiago") I'm referring to the experience itself.
​
WHAT
"Santiago" is a VR installation that features a physical sculpture that players can interact with to create visual and auditory feedback in a virtual environment. The experience revolves around Santiago, an ancient fish god who transforms into a living, breathing musical instrument that can be played through touch. The music created by the player affects objects and visualizations in the virtual environment, giving each player the agency to shape his or her experience. The piece is built for the HTC Vive and uses Leap Motion technology, allowing users to move freely around the sculpture and physically interact with Santiago.
​
WHY
This project started as an experiment in tactile VR. I was curious to see if touching a physical object while seeing its virtual counterpart would enhance a VR experience or instead break immersion. When I discovered that touching a physical object was in fact compelling, I expanded the experience into a full-blown art piece that incorporates a variety of mediums including 2D and 3D animation, music composition, sculpture, and painting.
WHO
My role as the creative director and technical artist included defining the artistic style, interaction design, world building, and fabrication. I led an amazing team of 11 programmers, designers, animators, musicians, and artists (credited below) and together we got the amazing opportunity to exhibit and share our experience with the public both domestically and internationally (listed below).
Trailer edited by Yimin Zhang. Shot by Nicholas Pudjarminta and Yimin Zhang. Music by David Deedwania.
User Experience
​
GOAL
To create a funky and fun escape into a vibrant world of wonder.
CORE INTERACTION: TOUCH
"Santiago" is fundamentally a tactile experience so touching the sculpture should be intuitive and seamless. I designed Santiago's eyes to resemble buttons -- objects meant to be touched, and made the core interaction tapping -- an action people are accustomed to and often do absentmindedly. Interacting with Santiago is much like playing the drums.
​
Players learn how to interact with Santiago as the experience progresses. The experience starts with Santiago frozen in marble form begging the player to touch him to reawaken his godly powers. He continues to beg until the player complies, and when they do, his sculptural form glows and he wiggles to life. This initial interaction with Santiago establishes an association between touch and auditory + visual reward. After the first, all subsequent touches trigger new musical notes and animation.
Players interacting with Santiago. Footage from WIGI E3 Afterparty and Visual Reality showcase.
FEEDBACK LOOP: TOUCH TRIGGERS MUSIC + ANIMATION
Every touch of Santiago's tongue or eyeballs triggers an auditory and visual response, creating a feedback loop that encourages players to continue interacting with the sculpture. Santiago's tongue and each eyeball all produce different notes and animations to allow the player creative freedom to craft their own personalized music and experience.
This gif shows in-game footage from Stage 1: When you touch an eyeball it lights up and turns on a spotlight (also plays a note)
DIRECTING ATTENTION
The visuals triggered by each eyeball are strategically placed in the scene to catch the attention of the player and direct their gaze. The animations associated with each eyeball are positioned in front of where the player will most likely be standing when touching that particular eyeball.
​
In early playtests we noticed that people would often be so focused on interacting with Santiago that they forget to explore the world around them. Santiago is the main character in this experience, but there are so many story elements and details in the environment that should not be ignored. To fixed this issue, we rearranged the visual cues in the surrounding scene so that they catch the player's eye and momentarily steer their attention away from Santiago.
​
We also used light to direct the player's gaze. For example, in Stage 1, spotlights (triggered through touch) illuminate sections of a dark cave to reveal little pockets of bustling sea life. The patch of light is framed by darkness and easily becomes the center of attention.
This gif illustrates that animated elements and visual cues occur directly in front of the eye that has been touched.
SECONDARY INTERACTION: DANCE
When players step away from Santiago he wiggles and dances. This interaction is super simple, but we believe it adds life to Santiago and makes the experience much more fun.
​
Santiago starts off as a a marble sculpture, stone cold and still. During the initial and subsequent touches, the player will still be standing relatively close to Santiago so he will remain motionless. At this rate, our players usually assume that Santiago doesn't move during the experience. Therefore, when they step away to explore the environment and see Santiago dance, it often comes as a surprise. So far always a happy surprise as we've witnessed at our playtests and exhibits. It's even initiated some dance parties!
This gif shows that when players are far from Santiago he starts dancing, and when they are within range of touching him he stops.
LIMITING THE PLAYSPACE WITHOUT BREAKING IMMERSION
I'm personally not a fan of the caged boundary lines that pop up in VR when you're at the edge of a playspace because they're so jarring and break immersion. I understand that they are necessary for safety, but I think there are less invasive methods to indicate when a player is out of bounds. I wanted to explore this issue and see if I could find a solution.
For "Santiago" we had a 12 feet by 12 feet moveable area using the HTC Vive. I came up with the idea to create a circular platform of the same size in VR and raise it up in the air so that the player would feel as if they were floating over the scene. Most people are afraid or at least uncomfortable with heights so most players tend to stay away from the edge of the platform in fear of "falling off."
Santiago's position at the center of the raised platform also helps to indicate that he is the main focus and what the players are meant to interact with. We also played with the perspective of the scene surrounding the platform by making it appear farther and larger to so that it would seem unattainable. Keeping the focus on Santiago and establishing a limited ground plane helped to keep the player in place.
Raised platform
The platform defines the playable area
Credits + Exhibitions
​
TEAM
Creative Director: Brenda Chen
Effects Artist: Allison Comrie
Interaction Design: Awu Chen
Lead Programmer: Drew Okenfuss
Technical Artist: Sagar Ramesh
Animator and Rigger: Yimin Zhang
Music and Sound Design: David Deedwania
Concept and Environment Artist: Blake Weber
Texture Artist and Animator: Erika Gomi
Concept Artist and Animator: Hannah Bosnian
Concept Artist and Modeler: Tammu Do
​
From left to right: Brenda, Drew, Sagar, Awu
EXHIBITIONS
-
Infinity Film Festival (Los Angeles, CA): Novemmber 1 - November 2, 2018
-
J. Paul Getty Museum (Los Angeles, CA): April 16, 2018
-
Visual Reality (Los Angeles, CA): January 13, 2018
-
Indiecade (Los Angeles, CA): October 5 – October 7, 2017
-
Tokyo University of the Arts Games Exhibition (Tokyo, Japan): July 20-31, 2017
-
Women in Gaming International (WIGI) E3 Afterparty (Los Angeles, CA): June 14, 2017
-
Rhythms + Visions (Los Angeles, CA): April 28, 2017
-
VRSC Student Festival (Los Angeles, CA): April 21, 2017 – Best Interactive Award
-
USC Undergraduate Research Symposium (Los Angeles, CA): March 12, 2017 – Arts Honorable Mention
-
Invited to VRTO in Toronto, Canada
-
Invited to Indiecade Europe in Paris, France
​
If you would like to learn more about "Santiago" please click here to visit our website.