Friday, June 12, 2009

It's happened: You can now wave your hands in the air and do something useful

You can tell if a technology is cool if it gets uber-geeks all hot under the collar. Well, I find this particular experimental foray pretty darned cool; and also, I believe it soon will be hot. This thing's like the Nintendo Wii, except you might be able to do just about everything with it.

The PhonePoint Pen project underway at Duke University under the guidance of Prof. Romit Roy Choudhury is an application that runs on mobile phones that includean accelerometer to detect motion: the Apple iPhone, Palm Pre, Blackberry Storm and some high-end Nokia phones are among such devices. The application allows one to create messages by literally writing in the air while holding the phone; the phone senses the movement of your hand and deciphers the letters and words you are trying to form. It records the message and then sends them off. To a lot of people this might appear a silly thing to do: now who would want to do such a thing other than people in cultures where flailing one's arms about, gesticulatingly wildly while speakng is de rigueur, like in India and Italy? Well, PhonePoint Pen should be treated as merely a demonstration of what's possible; like all great new ideas, it is the long-term, downstream implications and not the initial expressions of the idea that demonstrate its power. Electricity was merely a toy concept in Benjamin Franklin's outdoor demonstration; it took another century before the everyday use of electricity began to spread and today life is unthinkable without it.

The Nintendo Wii was perhaps the first device to enter the popular domain that relied primarily on motion detection for its effective use. Advanced techologies sometimes enter everyday life in this manner -- for entertainment and use by children and those who are capable of still thinking and acting like children. Introducing new ideas that don't yet fit into the context of human society in this manner is an excellent strategy since children's minds are unencumbered by preconceived notions in the manner adult minds are. Children have fresh, untainted visions. By observing how they use, appropriate and adapt new ideas, we may gain new insights which could help advance the state of research and development in regard to the ideas.

The Wii, however, is a special purpose device designed primarily for play. Incorporating similar motion detection technologies in a ubiquitous, common place device such as the mobile phone (or the emergent Convergent Device) opens up entirely new vistas of use. The imagination boggles at the moment, but there ought to more, not less research into the uses of motion detection. I can easily see how my mother, for instance would find such devices enhancing the usefulness of mobile phones -- indeed, the elderly would be excellent subjects for research into the benefits of the technology. Like all elderly folks (and I'm definitely getting there, having to reach for my glasses all the time) she is unable to read the tiny, low contrast LCD screen on the mobile phone, nor is she able to find the buttons and press them in the appropriate sequence. A simple sequence of *non-jerky* motions would remove the need to look at the screen, or for the fine motor control needed for pressing buttons. Heck, I would love such a feature myself.

Related research projects:
Most modern digital devices - including mobile phones - have a many features, far more than most people will ever get down to using. This situation creates unnecessary complexity when all that someone typically wants is to perform a couple of elementary functions. Using such a device -- even a humble remote control -- demands careful attention, including the need to look squarely at the device, selecting from among many buttons and performing operations that are often non-trivial for small children and the elderly - and even for many technically unsavvy adults. A device that doesn't require visual attention and inspection and which can be operated entirely through kinesthetic means may not be appropriate for people of all ages, but for a significant fraction of them it would be a dream. If nothing else, motion detection provides an alternative means for interaction that would be most welcome at least in some contexts.

The iPhone provides a platform for such exploration. As a first step, Apple eliminated all but one button from the face of device, replacing them with virtual buttons that appear or disappear as needed. One the one hand, Apple designers recognized the complexity issue by eliminating buttons, but on the other continued to support the button metaphor through visual means, even while creating the potential for alternative interaction metaphors through the incorporation of an accelerometer in the device. In theory, then, it may be possible to operate iPhone applications either through on-screen buttons and widgets, or alternatively, through motion sensing. It can therefore cater to two widely divergent usage styles. The iPhone is built to be a transitional device -- it supports the Old Order while also subtly suggesting a New Order. In my view, this is an example of masterful design that is also innovative.

I expect the use of gesture recognition only to grow. More and more digital devices, especially small, hand-held ones, are likely to incorporate accelerometers. Indeed, practically any situation that involves motion might be a candidate for using motion-sensing applications.

No comments: