New reality: mixed up and augmented
1 August 2017 | 0
One could hardly exist in today’s business environment without having at least heard of augmented, mixed or virtual reality (AR, MR, VR) technologies.
Interest and adoption of the technologies, across consumer and business, is growing, and market analysis reflects this. Analyst IDC has projected worldwide revenue from AR/VR would grow from $5.2 billion (€4.55 billion) in 2016 to $162 billion (€142 billion) by 2020.
Venture capital and corporate investment in AR/VR start-ups totalled $2.3 billion (€2 billion) in 2016, according to Digi-Capital and Deloitte, an increase of 230% over the previous year.
New AR/VR hardware from at least five companies, including Microsoft and HTC, hit the market in the last year, reported TechRepublic, with some of these products specifically targeting enterprise.
So what are these technologies, and what is the difference between augmented, mixed and virtual?
Virtual reality is probably the most familiar of the trio, insofar as it has been about for quite some time. Everyone has probably by now seen someone with a helmet or enclosing headset attached to a PC, enjoying the likes of a rollercoaster, simulated ride or even a journey through the planets. A paper from Deloitte University press described VR as “a technology that allows a user to perceive and interact with a simulated environment. The environment may be either realistic or fanciful.”
Augmented reality is a newer development that relies more on integrating, or overlaying information or enhancements on the surrounding environment, with the recent Pokémon Go craze as a good example. Again, the above paper describes AR as a technology that “overlays information onto a user’s field of view as she perceives the real world. The information typically informs the user about an object or place at which she is looking.”
Mixed reality is perhaps the newest variety of reality experience and is described as a subset of AR. The Deloitte University paper says, “this technology enables the perceived merging of real and virtual worlds to produce new environments and visualisations in which simulated digital objects and information and real objects can co-exist and interact with one another.”
The technologies to deliver these realities vary greatly, from consumer headsets and hefty PCs to mobile phones in cardboard cases and dedicated helmets and headsets. Samsung, HTC, Microsoft, Google, Oculus and DAQRI are just some of the names likely to be encountered when exploring a tech show floor.
Despite what may be thought, A/M/VR technologies are actually well established in the business world, with design, modelling, manufacturing, training and skill extension all seeing usage. In terms of verticals, the aerospace and automotive industry, retail, healthcare and tourism also make use. However, there is little in the way of standardisation, off-the-shelf options or even base requirements set out for an organisation looking to leverage these exciting technologies. All of which, however, have failed to stunt growth or adoption, as Deloitte has reported that more than 150 companies in multiple industries, including 52 of the Fortune 500, are testing or have deployed AR/VR solutions.
The AREA.org, which stands for Augmented Reality for Enterprise Alliance, is a “global, member-driven trade organisation focusing on reducing barriers to, and accelerating the smooth introduction and widespread adoption of Augmented Reality by and for professionals”.
AR has long been thought to be of more immediate benefit for enterprise, and Mark Sage, executive director, The AREA.org, says there is a lot of work to be done to establish a set of requirements and goals to help adoption of these technologies.
Sage said the organisation is building the ecosystem for AR in enterprise, for content, technology and platforms.
The organisation has been working with enterprise to develop a set of requirements, with initiatives including such companies as Caterpillar, Proctor and Gamble, and Lockheed Martin.
With regard to the technology in particular, there are issues to deal with.
“It turns out that the enterprise requirements were a lot higher than what the vendors can provide at the moment,” said Sage.
For example, enterprise requirements include such items as an 8-hour battery life for devices, a 65-degree field of view, and around 125 g in weight.
What we found, said Sage, was that what enterprises needed to use this technology normally in their environment is not quite there yet.
However, Sage is optimistic.
The one thing we know with all of this, he asserts, is that the technology will get better.
“It will always move forward, so although it is not going to happen straight away and there will be some bumps along the way, ultimately, the technology will get better, but it depends how long it takes,” said Sage.
“I strongly believe that AR has more potential in the enterprise space than what VR will.”
Design in VR, build in AR
Among the use cases of AR in enterprise currently, said Sage, sees organisations do their design work in VR, and then build in AR.
This can be particularly useful for very large projects, he said, such as shipbuilding where rather than a job that is done regularly, some major projects might be done only once or twice in a decade. AR can help visualise structures and fit out in these instances, allowing technicians and engineers to literally see where equipment and machinery will go, its configuration and even operation.
The DAQRI Smart Helmet is one such device. It is a self-contained AR system that can overlay the likes of building infrastructure management (BMI) systems over the surrounding environment, for construction or maintenance purposes, allowing personnel to see high quality, accurate rending of layered information.
This kind of technology is now moving beyond mere BMI into what IBM is calling Cognitive Buildings.
The idea behind cognitive buildings is that with a combination of Internet of Things (IoT) sensors, networks, software and visualisations, building management can move beyond its current state, to give “real estate and facilities management professionals the tools necessary to better manage energy use, optimise space, reduce operating costs, and simplify real estate planning and management,” says IBM.
According to the company, employing a cognitive buildings strategy can help reduce energy use up to 50% and increase facilities utilisation by up to 85%. In the US alone, reports IBM, commercial real estate accounts for 12 billion square feet of covered area, but only 67% is utilised. In a world where employees are spending less time in the office and more time in alternative workplaces, a cognitive buildings strategy can also enhance employee productivity up to 18%, IBM argues.
Dr Joern Ploennigs is a research scientist and manager of Cognitive IoT for Buildings and Environment Ireland Research Lab with IBM. He believes that data visualisation is a key capability of A/M/V R technologies, and will become increasingly important as they mature.
Real time delivery
“Both VR and AR can help deliver real time system data,” said Dr Ploennigs. “With VR, it makes sense to show things that are hidden, such as [the] temperature of [a] device or the flow of water or some infrastructure underground in a city, like gas or electricity pipes or cables.”
“The data visualisation may also be a piece of a building/product that is not there yet, like with a prototype. Here you can use virtual reality (VR) as it creates a totally artificial environment.”
“For buildings, it may also be one that is not there and in plan or it may be one that is not there anymore, such as an ancient castle or the ancient Temple in Jerusalem—there a lot of opportunity to recreate our cities in this manner when combined with the real world on the spot environments.”
“In my own research area of Cognitive Buildings,” said Dr Ploennigs, “we can see it being applied by architects and industry designers, whereby they can develop a site design or product using VR instead of using the actual drawings or plans to create a prototype.”
Retail has been another key area for use of these technologies, and has many lessons for enterprise, not least of which is user acceptance on several fronts.
VR has been used successfully to allow those who could not attend the ability to see catwalk models walk past, and savour the atmosphere of the likes of Paris, Milan and London fashion shows. Samsung made a special effort his year at the spring New York Fashion Week, partnering with fashion production firm, FTL Moda, to use its VR technology.
These projects do vital work in normalising the technologies, but also ironing out some of the kinks. Car maker Audi has been something of a pioneer in this field too, developing a virtual showroom for its prestige cars.
Using commercially available virtual reality headsets, such as the Oculus Rift and HTC Vive, the company has developed a complete virtual showroom experience. Using a development of the Unity game engine, the company has built upon gaming development for a 90-frames per second, 6 million polygon rendering capability to bring a cinematic experience to the potential buyer.
According to Marcus Kühne, digital retail solutions, Audi AG, every configurable detail for a car can now be visualised and manipulated. From a Parisian backdrop, to the haunting landscapes of Iceland, the car can be visualised anywhere to give the user a personalised experience.
The development effort was significant, said Kühne. Starting more than four years ago, a team of up to 30 specialists, from six different suppliers, worked to achieve a lofty set of specifications to ensure a high-quality experience, and one that would not make the user motion sick, says Kühne. While it may sound high, the 90 frames a second rate was discovered to be the optimal delivery rate, as anything less, despite the details levels, could have the aforementioned undesirable effect.
Despite the technical challenges in developing the virtual showroom, Kühne said that the ability to see the car in every detail is an important marketing tool.
With AR and VR now firmly in place in various markets, Microsoft is ploughing its own furrow with mixed reality with its HoloLens device, that is one of the likely bridges between the more niche applications and the mainstream of business usage.
“With AR, the user sees a layer or screen of data that overlays the real world. While this data can be contextual to the user’s location, or where the device’s camera is pointed, it is not the same as being able to see holographic objects pinned, or anchored, to specific physical locations or objects in the real world,” said Shirley Finnerty, Windows Business Group lead, Microsoft Ireland.
“With VR,” said Finnerty, “the user is completely immersed in a computer-generated reality, or virtual world. At Microsoft, we are focused on mixed reality which is a brand new holographic category. MR provides endless opportunities for developers to improve how people communicate, create and explore by mixing the physical and virtual world together.”
Describing HoloLens as “the world’s first self-contained holographic computer,” the device runs Windows 10 and is completely untethered, meaning there are no wires, phones, or connection to a PC needed.
However, despite being a ground-breaking device, HoloLens still suffers from some of the previously described issues. Firstly, its battery life is a mere 5.5 hours, or 2.5 if used under ‘heavy load’, and at 579 g is several times the ideal of 125 g.
As TheAREA’s Sage pointed out, the technology will only improve, and one can be guaranteed that Hololens II is already in development and heading toward, if not exceeding, the industry ideals.
Microsoft sees MR as a key development in several areas. In creation and design, it delivers the ability to see and manipulate 3D designs in real time, in a real-world setting, at true size and scale. For assembly and manufacturing, there is the ability to synthesise complex data sets and provide visual displays that aid in navigation and operations. In training and development, there is now the capability to explain complex scenarios visually, with movement and in three dimensions. And in the area of communication and understanding, there is the ability to provide remote expert assistance. Seeing through the eyes of a remote user and provide annotations that appear as holographic content through HoloLens.
All the experts agreed that this is one of the key benefits of augmented and mixed reality, the ability to remotely assist a technician or operative by having the benefit of a master technician or engineer, augmenting the skills on the ground, seeing through their eyes, guiding the work on site.
However, most of our experts also agreed that security is still an issue.
Sage said that base levels of security need to be established for the full range of these technologies, but especially when used in public, or as Dr Ploennigs put it “outside the showroom”.
As was seen with the mild backlash against the use of devices like Google Glass, Dr Ploennigs said the camera is the security issue.
“For enterprises, it’s when/where the data these devices are scanning is the issue, and for many consumers it is an invasion of privacy,” he said.
While Microsoft’s Finnerty is confident of the abilities conferred by Windows 10 on its Hololens device, it is not really intended for untethered use outside of the business environment, and so circumvents at least some of these issues.
Despite the possible user acceptance and security issues, the potential for data visualisation, collaboration and unified communications in enterprise is undeniable, and so we are likely to see these technologies creep into the mainstream in the near future. Sage’s optimism, in terms of development, appears to be well founded and so expect major jumps in capability, as well the issues addressed by the time you are expected to don a smart bit of headwear at the boardroom table, the project room or to join a meeting from home.