Duck, dive, scribble, spray – now gestural interfaces are within everyone’s reach

Lower down this post, you’ll probably find some high-flown stuff about gestural user interfaces going mainstream, but in all honesty the thread that joins together the following two-and-a-half things is that they’ve all left me grinning like a fool. A hand-waving grinning fool. And a bobbing my head up and down like Churchill the nodding dog grinning fool.

Thing 1 – Season’s greetings from friends and former colleagues at Common Agency, in the form of Snowballed – and yes, I know it’s a Flash gimmick from a design agency, but stay with me, please. What makes Snowballed stand out from the crowd is the way it’s controlled using your PC’s video camera. As Karl Marx and Sigmund Freud pop up and hurl snowballs at you (no really), the object is to dodge them just by moving about. Squint carefully at the image below and my face is visible direct from my laptop’s webcam.

Move left, and the snowman moves left, move right, and you get the idea.

Now I know this interface isn’t brand new – take Eyetoy for example. What is different is that this just works on any PC with a webcam and Flash installed – no specialist kit required. When HP specced my laptop with a camera built-in, I guess they were thinking of video calling or whatever. I’ve never used my laptop for video calling, but now I have used it to dodge snowballs thrown by dead white men with beards.

Thing 2 – hacking the Wiimote’s built-in IR camera to make  FriiSpray – Open source Infra-red graffiti. From the project site:

FriiSpray is a project thought up by three heads in the North of England, based at the Innovation North co-working space in the Old Broadcasting House, Leeds. The project is based around the Wiimote Whiteboard software, built by Johnny Chung Lee – have a look at his stuff here. We thought that it would be a great idea to take this interface between the Wiimote and the computer and adapt it to allow people to create digital, or virtual grafitti as an interactive media installation.

It’s a fun experience to take round events, and the team already have one booking. They’ll also be presenting the work at the forthcoming O’Reilly Ignite event in Leeds. Again, a great hack of cheaply obtainable stuff to do something wholly different from the original purpose.

Incidentally, Friispray also made me aware of a bit of misdirection by Nintendo: the Wii comes with a “sensor bar” which sits by the TV and interacts with the Wiimote. But the sensor bar, does not sense, it emits two points of infra red light, which the remote senses with its IR camera. At Stuart Childs‘ suggestion I tried pointing the Wiimote at other IR light sources, and it works. I wonder at what point in the product design and marketing process, it was decided that it would be easier to explain this as doing the opposite to how it actually works?

Thing 2.5 – Crayon Physics Deluxe, just because it made me smile and now I can’t wait to test it out on my children. Crayon Physics works a treat on a Tablet PC, and I’ve also seen good reviews by users of Wacom graphics tablets and the like.  In its own words it is:

a 2D physics puzzle / sandbox game, in which you get to experience what it would be like if your drawings would be magically transformed into real physical objects. Solve puzzles with your artistic vision and creative use of physics.

Draw stuff with a pen on a computer screen and they come to life. Brilliant! Video demo here.

Here comes the science bit.

I think in the future using a mouse will feel a bit like painting with your fist. Typing on a keyboard may have more longevity, but is still not so many steps removed in sophistication from writing a ransom note by pasting cutout newspaper letters onto a sheet of paper. Gestural interfaces have been around for a while, and are slowing making headway into mobile devices as well.

What marks out the stuff I’m writing about here is how accessible and natural it can be. Got a webcam? You’ve got motion control. Got £20 for a Wiimote? You’ve got an IR camera. There are so many ways to control our computers, and I sense that this year is the year that some of them will go mainstream. I’ll be grinning from ear to ear.