Tag Archives: gestural

Gestural interface: like a Wacom tablet, just without the plastic bits

Via SlashDot, here’s a project from Potsdam University in which the clever boffins have built a user interface that requires only hand gestures as input:

We present Imaginary Interfaces, screen-less devices that allow users to perform spatial interaction with empty hands and without visual feedback. Unlike projection-based solutions, such as Sixth Sense, all “feedback” takes place in the user’s imagination. Users define the origin of an imaginary space by forming an L-shaped coordinate cross with their non-dominant hand. Users then point and draw with their dominant hand in the resulting space. The interaction is tracked by a tiny camera device clipped to the user’s clothing and pointed at the user’s hands.

A bit rough and ready, sure, but it’s early days. Bolt this onto AR (they both need similar face-mounted hardware, so convergence is pretty inevitable), and stuff gets weird real quick. Cities full of people wandering around, seemingly talking to themselves and waving their hands in gnomic gestures… it’d look like a city of mad magicians.

Or, y’know, like Burning Man or Glastonbury at 5am on a Saturday. 🙂

Gesturing toward tomorrow: gestural UIs, hardware hacking and rise of the makers

Suddenly, touchscreen devices seem to be everywhere, changing the ways in which we interact with our phones, computers and tablet devices. But the next user interface revolution is already waiting in the wings – gestural interfaces will complete the user-interface paradigm shift that touchscreens have started. So says Stowe Boyd:

Gestural UI, or ‘hand jive’ as I call it, once deployed as a built in aspect of future computers, like touchpads and mouses are today, will set the stage for a rethink about user experience.

First we will see hand jive as a way to manipulate the gears of now-tradition windowed UIs: pulling down a menu in an app, moving windows around, dragging a file to the trash.

In the future, we’ll have real Minority Report stuff, without the enormous touch screens: we’ll also see the emergence of augmented reality goggles — Terminator goggles — where we can toggle back and forth between 100% computer screen sorts of display to 100% augmented reality. And the goggles — as an integrated part of the computing device — will be watching our hands for commands, and watching the world for reality to augment.

The combination of these trends will make computing primarily mobile: we’ll have an iPhone sized device we carry all the time, which will be a phone and a PC. We will be free of LCD screens — in general — courtesy of our goggles, and free of keyboards, courtesy of hand jive. A keyboard can be imaged on any flat surface by the goggles, and we can type without a physical keyboard because the gestural system is watching our fingers in 3D. And of course, a lot of things could be done without typing, especially once kids start using sign language and voice to communicate with computers. (I say kids because that’s who start first.)

While we’re waiting for that revolution to arrive, the inclusion of accelerometers in mobile hardware offers some avenues for interfacing with your phone without mashing the keypad or fingering the screen. Anyone who’s ever found themselves with a pocket full of unsolicited novelty ring-tone in a crowded cinema will probably appreciate the opportunity to silence their phone with a few well-timed slaps of the hand through their clothes:

With the right software installed, it may one day be possible to cut a call by “whacking” the phone in a particular pattern while it’s still in your pocket.


The team developed a simple vocabulary of “whack gestures” designed to rapidly communicate simple commands such as silencing the phone. To help the device distinguish the gestures from background bumps, each begins and ends with a firm “whack”.

The biggest roadblock for gestural UI will probably be the software houses, however. Keith Stuart of the Guardian Games Blog wonders whether anyone will actually bother using Microsoft’s Project Natal motion sensor device for truly new gaming experiences, or whether they’ll all play it safe with re-runs of what has gone before:

For a start, publishers are massively, obsessively risk averse. If there’s any way of leveraging market pre-awareness into a new product they’ll leap at it. So even if these companies are developing titles that work only on Natal – not just new games with vaguely specified Natal-support – it’s unlikely that they’ll do this without recourse to familiar brands and gameplay experiences. In other words, we may get a dedicated Natal version of, say, Mass Effect 3, but it’ll still be Mass Effect, it’s just that you’ll act out those in-depth personal relationships with aliens rather than just talking and watching the cut-scenes.

A little pessimistic, perhaps, but given the enduring tightness of the global economy, playing it safe is likely to be the order of the day for those with the most to lose. But we shouldn’t discount the independent hardware hackers, who the Electronic Frontier Foundation suggests will be a growing cultural force in the year to come. Every day my RSS feeds are full of ordinary geeks doing amazing things with off-the-shelf devices and a handful of cheap parts, and despite the best efforts of easily-riled device manufacturers and their copyright lawyers, it’s getting harder and harder to keep the details of mods, hacks and retrofits a secret.

Cory Doctorow’s latest novel Makers (which I still haven’t had the time to sit down and read beyond the tenth instalment or so) posits a near-future economy where the agile and frugal make-do mind-set of hackers and makers changes the way the world does business for ever. With 3D printing showing every sign of maturing to the affordable “prosumer” level in the next year or so, and landfills across the planet still inhaling mountains of obsolete consumer electronics and tcotchkes, it’s far from being the most implausible future I’ve read about lately. [image by See-ming Lee]

What do you think – will our recent economic woes push us toward reuse and repurposing, or will we wander slowly but surely back to corporate-capitalist business-as-usual?