Introduction to Extended Reality
Extended reality (XR) is a term that encompasses real and virtual environments that are generated by wearable devices or computer technology to provide an immersive experience. It can also be described as a collection of all immersive technologies that combine real and virtual worlds. <!--more-->
What is Extended Reality (XR)?
Both the number of XR applications and advancements with those applications is growing. This technology is still evolving because of the ongoing development in human-machine interactions.
Pillars of Extended Reality
Extended reality encompasses three main pillars of immersive technology: Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR).
Virtual Reality (VR)
This immersive technology employs a head mounted display (HMD) to provide an immersive expression through a simulation (or simulated environment). The simulated digital environment makes users feel as if they are there.
For example, VR developers may create an artificial world that makes users feel as if they are walking around on the moon. The first industries to adopt this kind of technology include the gaming and entertainment industries.
Augmented Reality (AR)
In this component of XR, smart devices are used to overlay sounds and digital graphics in the real world. Some of the digital details overlaid into the real-world environment include images, animation, and texts. This enhanced experience can be achieved using AR screens, tablets, smartphones or screens.
Snapchat filters and Pokémon Go are good examples of this AR technology. In the game of Pokémon Go, digital creatures are overlaid onto the real world (via a phone screen). Snapchat filters involve putting digital objects such as glasses and hats (overlaid) onto the user’s head to provide unique experiences.
Mixed Reality (MR)
Mixed Reality encompasses the co-existence of both real-world and digital objects. There's an interaction between these objects in real time. This is an advanced immersive technology that needs an MR headset and higher processing power than virtual reality and augmented reality. A good example of mixed reality is Microsoft's HoloLens, which allows users to interact with digital objects that are placed in a room.
How Extended Reality Works
From a developer perspective, the extended reality is achieved by designing immersive algorithms. Software developers and data scientists should design clever and efficient algorithms to produce interesting XR features that offer these unique user experiences.
Other important concepts that need to be incorporated by developers of immersive technologies include 3D modeling and advanced computer vision. Artificially intelligent programs are made using machine learning, biomechanical modeling, and motion tracking.
Some of the programming languages used in the design of extended reality include C, C++, Java, JavaScript, Python, and Swift. A full description of these languages and their application in AR can be found here.
Developers that may be interested in applying these programming languages in the development of VR can get more information here as well.
From a user perspective, the three main components of XR perform specific roles to provide an immersive experience. within AR solutions, smartphone cameras are used to capture a section of the environment. These solutions then scan the captured environment to identify a specific point where additional information will be overlaid.
Computer-generated content or digital information is then overlaid onto the captured environment using markers. Trackers such as laser, GPS, infrared, and other sensors can also be used to overlay information in the aforementioned environment. MR works similarly to AR, although the former provides a more advanced immersive experience than the latter, by allowing the mixed interaction of virtual objects with the real world.
Virtual reality works by using a person's senses to provide a uniquely immersive experience. The main senses used in VR include touch, sight, and hearing. It also uses the power of the mind. Various features of VR provide users with unique experiences that reconfigure the mind. In most cases, a headset is used to provide this kind of experience because of its ability to influence the senses of the user.
The five senses utilized or influenced in immersive technologies are as follows:
-
Sight: In extended reality devices, a computer-created reality is seen by the users, which gives them an immersive experience. This occurs because the computer-generated reality is laid in over a real-world environment. This visual experience is mostly offered in video games, especially on PlayStation and Nintendo gaming platforms. In some museums, 360 degrees cameras have been mounted on XR headsets to provide the users with a unique visual experience.
-
Hearing: Sounds in XR devices can be heard from all directions. These devices enable users to perceive sound as they would in real situations. For example, within a football match in a video game, users will hear the sound of football fans, which would also happen in real life.
-
Touch: The sense of touch is also integrated into extended reality to improve user experience. In XR devices, gloves and bodysuits are used to give users a sense of touch within the simulation. Wearable devices use haptic technology, that employs a feedback system that helps with the expression of vibrations and motions.
-
Taste and Smell: These are new fields in extended reality that are undergoing research and development. Scent cartridges are used to provide a sense of smell. These cartridges produce pleasant, neutral, and unpleasant scents. Electrodes can be placed on a user's tongue to provide a simulated sense of taste.
Applications for Extended Reality
Extended reality has a wide range of applications in the modern world. Some of its applications include entertainment, healthcare, education, real estate, and marketing.
Entertainment
Extended reality has played a great role in providing immersive technologies in the entertainment world. Users can now get virtual experiences of sports and music events using their VR headsets.
In movie theatres, virtual reality has been used to provide audiences with 3D experiences. In the acting world, real-time rendering and camera tracking provide immersive experiences to actors, which makes them feel as if they really within the scene.
Extended reality is also increasingly being used in video games. There is an improved user experience because of the enhanced comprehensive participation effect.
Healthcare
In the healthcare system, extended reality is being used by medical practitioners to visualize complex medical procedures. It also assists surgeons to plan for complicated surgeries.
Image Source: University of Virginia
Education and Training
Extended reality can help in the training of people that may work in high-risk conditions. It could be dangerous to teach new aviation students to fly with real planes.
With digital simulations that offer immersive experiences, aviation students can learn how to fly a plane effectively without any potential risk of an accident.
Medical students are also using immersive technologies to learn important skills. A good example is gaining hands-on experience with virtual patients.
Real Estate
XR can be employed in the real estate sector by offering virtual experiences to potential buyers or tenants. In some cases, tenants or buyers may be far from properties that have been listed for sale. These properties can be viewed virtually to enable users to get a better digital view of the assets.
Marketing
XR has enabled companies to offer immersive ways in which customers can interact with new products. Some companies have also used immersive technologies to allow customers to try products before buying.
A good example is Cathay Pacific, which has offered its customers a 360-degree video that allowed users to experience its brand firsthand. This strategy enabled the company to increase brand favorability and customer awareness.
Current Limitations of XR
XR developers are facing certain challenges that are limiting their capabilities. Developing these technologies requires the collection and use of huge amounts of personal data (which needs to be protected).
This spread of data may make companies vulnerable to cyber-attacks. Implementing immersive technology is costly, which has barred many companies from investing in extended reality.
Companies are required to develop wearable devices that are comfortable and fashionable. Before implementing immersive technologies, developers have to deal with technical issues such as power, connectivity, motion tracking, and common illumination.
The Future of XR
Extended reality may shape the future of machine-human interaction and digital transformations. The future will include many advanced and complex immersive technologies that will solve many global problems. We may also get to experience immersive technologies that use the sense of taste and smell. These are technologies that are currently under extensive research.
With the introduction of 5G technology, there will be increased adoption of extended reality in the future. This is due in part to the better sustained network performance and uniform experience.
Finally, XR could greatly leverage mobile computing because of its increasing ability to revolutionize mobile experiences for users. It wouldn’t be surprising to find smartphones becoming some type of XR wearable.
Resources
Peer Review Contributions by: Lalithnarayan C
