Kaleidosense

Kaleidosense

An interactive artwork for all five senses.

Year

2023

Role

Interactive Engineer /Technical Artist

↓ Scroll to continue
0:000:00
View credits (3)

Dannie Leigh (袁愷葳)

Director

Directed the project and led its visual design. The concept grew out of Dannie’s research into the human senses.

David Kuo (郭岱維)

Interactive Engineer / Technical Artist

Built the interactive system in Unity, integrating the Xbox Kinect, Leap Motion, and Arduino weight sensors, and wiring the fragrance diffusers to trigger on cue.

Hsu, Shao-Hsuan (許少軒)

Technical Specialist

Supported the on-site installation of the exhibition.

About the project

Explore the human senses with "Kaleidosense," an interactive artwork that goes beyond traditional boundaries. Incorporating 3D animation, microcontrollers, and motion sensing devices, it provides a dynamic and multisensory journey through sight, hearing, touch, smell, and taste. The project aims to redefine how we engage with and experience art, inviting the audience to actively participate in unraveling the narrative of human senses in novel and imaginative ways.

Tools & Software

UnityC#BlenderArduinoLeap MotionKinect

Categories

InteractiveUnity

Interactive Design

Sight

Waving arms alters the colors across all scenes.

Touch

Changing the textures across all scenes by touching different surfaces.

Smell and Taste

Grabbing taste symbols causes them to melt. Grabbing scent symbols makes them diffuse.

Technical Detail

Xbox One Kinect & Arduino

In Sight, the Xbox One Kinect is employed to detect the audience's arm movements, enabling interaction with the visual component of the screen. In Touch, under each material, weight sensors transmit touch signals from the audience to Arduino, controlling the presentation of materials on the screen.

Kaleidosense — Xbox One Kinect & Arduino 1

Leap Motion & fragrance dispenser

Using Leap Motion sensors, capture the audience's hand movements to interact with objects related to taste and smell on the screen. Upon successful interaction, release corresponding scents through a fragrance dispenser.

Kaleidosense — Leap Motion & fragrance dispenser 1
Kaleidosense — Leap Motion & fragrance dispenser 3

Exhibition

The finished installation: five screens, three interaction stations, and visitors moving freely between sight, touch, smell, and taste.

Development Process

01 · Touch

From pressure to weight

We started with pressure sensors. They were sensitive, but only within a narrow range: past a certain force the reading simply maxed out, and the touchable area was too small for a hand resting on a material. Weight sensors gave us the range and the surface we needed.

02 · Smell

Scent on cue

Next we paired Leap Motion with an off-the-shelf fragrance diffuser. We rewired the diffuser’s circuit to the Arduino so that every time the signal goes high, it releases a single puff, turning a gesture on screen into a scent in the room.

03 · Integration

Every input, one scene

Finally we tested the Xbox Kinect and sent every input (Kinect, Leap Motion, and the Arduino sensors) into Unity at the same time, making sure each one ran independently without interfering with the others.

Every input, one scene

04 · On site

Five screens, one line

Moving into the space, we used a laser level to set all five screens at exactly the same height and depth, so the room reads as one continuous piece.

Five screens, one line
Five screens, one line
Five screens, one line

05 · Plinths

Plinths that hide the machinery

We had custom wooden plinths made, filled and painted them white, and left enough room inside to hide the diffusers’ wiring, the computer, and every other cable.

Plinths that hide the machinery
Plinths that hide the machinery

06 · Cabling

Final cabling

With the screens linked together, the last step was routing and tidying every cable, so when visitors walk in, all that’s left is the experience.

Final cabling
Mid-installation
Final cabling
Finished

Thank you to Dannie Leigh for inviting me to join Kaleidosense. I’m grateful to have been part of the team and to have helped bring this project to life.