Universitario

research highlights
d oi:10.1145/ 1610252.16 1 0 2 77

ThinSight: A Thin Form-Factor
Interactive Surface Technology
By Shahram Izadi, Steve Hodges, Alex Butler, Darren West, Alban Rrustemi, Mike Molloy and William Buxton

abstract
ThinSight is a thin form-factor interactive surface technology based on optical sensors embedded inside a regular
liquid crystal display (LCD). These augmentthe display
with the ability to sense a variety of objects near the surface, including fingertips and hands, to enable multitouch
interaction. Optical sensing also allows other physical items
to be detected, allowing interactions using various tangible objects. A major advantage of ThinSight over existing
camera and projector-based systems is its compact formfactor, making it easier to deployin a variety of settings. We
describe how the ThinSight hardware is embedded behind
a regular LCD, allowing sensing without degradation of display capability, and illustrate the capabilities of our system
through a number of proof-of-concept hardware prototypes
and applications.
1. introduction
Touch input using a single point of contact with a display
is a natural and established techniquefor human computer
interaction. Research over the past decades,3 and more
recently products such as the iPhone and Microsoft Surface,
have shown the novel and exciting interaction techniques
and applications possible if multiple simultaneous touch
points can be detected.
Various technologies have been proposed for multitouch
sensing in this way, some of which extend to detection ofphysical objects in addition to fingertips. Systems based
on optical sensing have proven to be particularly powerful
in the richness of data captured and the flexibility they can
provide. As yet, however, such optical systems have predominately been based on cameras and projectors and require
a large optical path in front of or behind the display. This
typically results in relatively bulkysystems—something that
can impact adoption in many real-world scenarios. While
capacitive overlay technologies, such as those in the iPhone
and the Dell XT Tablet PC, can support thin form-factor multitouch, they are limited to sensing only fingertips.
ThinSight is a novel interactive surface technology which
is based on optical sensors integrated into a thin form-factor
LCD. It is capable of imagingmultiple fingertips, whole
hands, and other objects near the display surface as shown
in Figure 1. The system is based upon custom hardware
embedded behind an LCD, and uses infrared (IR) light for
sensing without degradation of display capability.
In this article we describe the ThinSight electronics and the
modified LCD construction which results. We present two prototype systems we havedeveloped: a multitouch laptop and a
touch-and-tangible tabletop (both shown in Figure 1). These
90

c ommunicatio ns o f th e acm

| d ec e m be r 2 0 0 9 | v o l . 5 2 | n o. 1 2

systems generate rich sensor data which can be processed
using established computer vision techniques to prototype a
wide range of interactive surface applications.
As shown in Figure 1, the shapes of manyphysical objects,
including fingers, brushes, dials, and so forth, can be “seen”
when they are near the display, allowing them to enhance
multitouch interactions. Furthermore, ThinSight allows
interactions close-up or at a distance using active IR pointing devices, such as styluses, and enables IR-based communication through the display with other electronic devices.
We believe that ThinSightprovides a glimpse of a future
where display technologies such as LCDs and organic light
emitting diodes (OLEDs) will cheaply incorporate optical
sensing pixels alongside red, green and blue (RGB) pixels in
figure 1. thinsight brings the novel capabilities of surface computing
to thin displays. top left: photo manipulation using multiple fingers on
a laptop prototype (note the screen has...