Future interfaces

Pro

From touch screens to voice control and gesture commands, human computer interface technologies have come a long way in the last five years.

While there’s no sign of the keyboard and mouse combination being put into mothballs just yet, there is reason to believe that the way we interact with technology could change significantly in the next decade. Already this change has made itself felt in the way touchscreen technology has been adopted in mobile computing, and it seems likely that next big leap in the way human beings interact with technology will be in the area of voice and gesture.

Neither are particularly new technologies, but what is new is a degree of reliability and usability. Microsoft recently launched a version of its Kinect motion sensor control system for Windows, taking it out of the arena of gaming and into the enterprise environment. First launched in November 2010, Kinect has become a major hit for the tech giant, selling 8 million units in its first 60 days.

Motion capture
Last month the first commercial software development kit (SDK) for Windows was released, allowing third party developers to create their own applications. Microsoft’s Kinect team leader Alex Kipman has been quoted as saying "I can guarantee you that 12 months from now, educational, academic and commercial applications will look nothing like what they are today."

Whether he is right remains to be seen-the central question when competing technologies appear tends to be which one will emerge as the standard. But in the case of interface technologies, industry observers suggest that this may be one of the few situations where different standards may peacefully co-exist, based on best situational fit rather than universal appeal.

"Interface technologies are where a lot of really exciting things are happening right now," said David Wortley, author of the new book "Gadgets to God", documenting the ways in which people relate to the technology around them.

"There are loads of interesting developments in this area, not just in terms of the multi touch technologies that you get on devices like Apple’s iPad, but also with technologies like voice and gesture recognition, movement capture and even the use of brainwaves for controlling computer technology. All of these have a huge amount of potential."

"Right now, the most proven of these is touch, along with some of the technologies found on games consoles like Nintendo’s Wii, Sony’s Playstation Move and Microsoft’s Kinect," he said.

Wortley is a consultant and founding director of the Serious Games Institute, a think tank based at Coventry University that studies the use of gaming technology, virtual worlds and immersive technologies.

Stability in control
"The most stable and most usable of the so called next generation interface technologies are probably those found in the iPad. They represent the first major advance on the mouse in that they are mechanical processes that work the same for everybody. They’re intuitive and give control over the technology in a way that anyone can understand."

"The problem with some of the others, motion capture, gesture recognition and brainwave technologies, is that they are much more complex. They are very much dependent on individual characteristics of the user whereas touch works the same for everybody, whether they’re aged 3 or 90, with the same level of reliability."

According to Wortley, a characteristic of successful next generation interfaces will be that will be geared towards the individual rather than being equally accessible to all. It’s an interesting idea, that instead of us having to adapt to the idiosyncrasies of a particular interface technology, they will adapt to our personal quirks.

Learning patterns
"They won’t operate the same way for everybody. Instead they’ll look at people’s usage patterns and learn from them. At the moment, most of these interfaces aren’t that reliable or easy to use, at least until you get the hang of them. But if we look at some examples from the gaming industry, we can see some signposts pointing the way," he said.

"Artificial intelligence can be used to fix these problems, being able to detect the personal characteristics of an individual and adjust the way it responds according to them. When I use a Microsoft Kinect and I want my PC to do something, the software needs to be able to recognise that I may do that in a slightly different way to someone else, and still give the right response."

"It’s a much more complex process so it will be a little while until some of these technologies break out to the main stream. Over the next ten years, I think computers will learn how better to understand context and individuality as they interact with people."

Touching technology
Touch is the most obvious example of the mainstream evolution of interface technology, currently found on smart phones and tablets and soon to become a mainstream technology in desktop PCs. However touch has been around for 20 years, first finding favour in the form of the interactive white board.

"David Martin in Smart Technologies first introduced that product in 1992 but he initially tried to introduce it to the corporate market and it was by accident that a school teacher happened to see the interface and comment that it looked and functioned like a chalkboard. From there Smart started to develop software around the idea of enabling touch as an input that was capable of delivering educational material in a meaningful way," said Greg Tierney of Steljes.

"Touch was originally based on resistant membrane technology, with two layers of membrane behind a board. When a point on the board was pressed, it created an XY coordinate that could be read like a mouse command. Since then there are new technologies that have appeared to build on that, but capacitive touch is still quite relevant."

Today’s versions of those first interactive white boards are far more sophisticated, understanding the difference between a screen being touched by a finger, a stylus, the side of a person’s hand and capable of reacting differently to each.

"They can be used much more intuitively. You can write with a plastic pen, pick up the text and move it around and use the side of your hand to erase what you’ve written."

Projection moves
According to Tierney, it’s now possible to create new kinds of interactive screens purely through projection and there are many new and interesting applications to these technologies in the area of virtual conferencing.

"Smart has done some very interesting work in creating new interactive surfaces using camera technology to move away from capacitive touch-allowing for interactive video walls to be projected onto any surface. These technologies will become, in our view, key in the next generation of meeting room technology."

"Video conferencing has been around for years, and while it’s great for holding conversations, it’s not very good when it comes to actually getting work done. For collaborative working, we can do a lot better now. You can add touch based technologies to the video conferencing facilities to create a really powerful working environment. Gesture control is opening up a lot of doors in this area," he said.

An increasing number of smartphones and tablets come with touch technology as standard, and as devices such as Apple’s iPad become more commonplace there is broadening acceptance of new ways of interacting with technology.

"People are becoming more comfortable with and more used to using gesture and touch technologies so they intuitively know what to do when they encounter these technologies in other contexts," said Tierney.

"At the same time, there needs to be some improvements to step acceptance up a notch in the corporate environment. For example, we’ve noticed that one impediment to using these technologies in meeting rooms is that often executives don’t want to get up and write on the board because they’re ashamed of their handwriting."

"To help with that, we’ve developed a calligraphic pen that converts handwriting to more aesthetically pleasing text on the board, and that’s turned into a real success for us," he said.

Beyond touch
The future of touch technology as an avenue of human computer interaction is interesting to consider. While touch is fairly well realised, interesting work is being done in development circles to advance it further.

"There are several areas in which we expect improvements and innovations to take-off, particularly with things like haptics, proximity and stylus. A little further out, we expect things like 3D gesturing and hovering to become standard, so that users can get additional functionality from their devices," said Patrick Hanley, product marketing manager for the US-based Atmel. The company specialises in interface technologies and has a research and development site near Drogheda.

"Touch will continue to evolve, possibly in the same way Moore’s Law affects memory. There will be continuous improvements made. There is always room for improvement and that is what makes the hi-tech industry so exciting."

For those unfamiliar with the terminology, haptics refers to tactile feedback technology where vibration and force feedback stimuli give extra information to someone using a touch screen, in much the same way that video game controllers often vibrate in response to players actions within a game.

3D gesturing refers to combining touch technology with an interface that can tell the location of a pointer or finger above the screen allowing the user to, for example, select one software function by hovering over the screen and a second by actually touching it.

"Haptic and gesture integration is certainly something that’s on the horizon with touch-based interface technology. 3D gesturing in particular has lots and lots of applications as diverse as in remote controls, stereos, video games, lights, you name it," said Hanley.

While there are many applications for these new and improved interface technologies in the general world of IT, their direct applicability to the enterprise is often less clear.

Voice over
"Voice control is interesting and has a lot to offer depending on the need of the individual using it, but I don’t think the keyboard and mouse is dead just yet," said Greg Tierney of Steljes.

"I think people will become a bit more adaptive to using different technologies then they currently are. Onscreen or touch based keyboards will become more common, depending on the need of the application. Sometimes change is a little difficult for people, but it depends on what you’re doing and I think we’ll become more used to using different means to enter data."

Hanley of Atmel suggests that the most significant development on the immediate horizon is basically more of the same, with touchscreen technology becoming more common and more deeply embedded in all kinds of devices.

"Touchscreens are already used in industrial, automotive, and networking applications today, I firmly expect for this to just continue to flourish in this manner across many different industries and applications," he said.

"Specifically with touchscreens, as tablets become more and more common in everyday life and as a potential laptop CPU replacement, this will become much more a business tool as much as a personal tool."

Business impact
Wortley feels there are many potential benefits for businesses in adopting these technologies, particularly given that it seems certain that in future, there will be even more information about us as individuals available to businesses.

"In ten to 15 years’ time, the IT infrastructure that underpins so much of our society will know so much about people entering the work force, about their capabilities and characteristics that their careers and lives will be very different from ours. These kinds of interface technologies will have huge implications for how businesses operate, particularly in a few key areas."

"The first is customer relationship management (CRM). These will be technologies that not only engage your customers and help retain them, but also let you build a much more personal relationship with them," he said.

"However you contact your customer, whether through the web or through a phone call, there will be a lot more personal CRM data made available to the company to allow it to deliver a much more personal service to the customer."

Wortley suggests that a second area of importance has to do with human resource management, career assessment and development.

"These technologies will also be used to assess people’s competencies and interests, and ultimately will be used to attract the right people to the organisation. They will also make sure that career development and training needs are much more geared towards to the capabilities and interests of the individual."

Read More: hardware