What Is Extended Reality?

Extended reality (XR) uses technological innovations to move beyond the natural environment and into one that allows for computer-generated, 3D experiences.

Though it seems like a relatively new technology, the concept of XR emerged over 150 years ago.

According to Smithsonian Magazine, in 1838, British scientist Charles Wheatstone discovered that “if you drew two pictures of something—say, a cube, or a tree—from two slightly different perspectives, and then viewed each one through a different eye, your brain would assemble them into a three-dimensional view.” Thus, Wheatstone invented the world’s first stereoscope, which created an impression of depth and immersion unimaginable at that time.

With the advent of the photograph around the same time, along with a more refined version of the stereoscope created by scientist David Brewster a decade later, the device’s popularity soared throughout Europe. These 19th-century inventions paved the way for the eventual rise of the extended reality technologies being explored today.

By the mid-20th century, more instances of virtual and augmented reality technologies emerged. In 1956, the first VR machine was invented by Morton Heilig. This device blended 3D color video with audio, smells and a vibrating chair to give users a fully immersive viewing experience. The inventor also patented the first head-mounted display four years later.

In the 1960s, VR headsets with motion tracking capabilities were invented for military use, and by the 1970s, MIT developed the Aspen Movie Map, enabling users to explore the streets of Aspen via a computer-generated tour.

Additional XR wearables, such as goggles and gloves, came along in the 1980s, and the term “virtual reality” was coined in 1987. The first VR arcade machines were installed in the early 1990s and more affordable headsets emerged later that decade.

Since 2010, extended reality technologies have gained significant momentum. The Oculus Rift VR headset prototype was developed in 2010, providing users with a 90-degree field of vision of a simulated world. Tech leaders from Sony, Samsung and Google released their own VR devices in 2014. By 2016, Microsoft released the HoloLens headset, giving rise to even more interactive augmented reality experiences. In the years since, XR technologies have continued to expand in both innovation and accessibility.

Augmented Reality

Augmented reality (AR) is a type of XR technology in which virtual information and objects are superimposed onto natural surroundings. In other words, AR allows for real-world enhancements by adding images, text or animation to the existing environment.

Examples of AR technology in use today include Snapchat interactive filters, Pokemon Go! and IKEA’s augmented reality home design experience, all of which are discussed in greater detail below.

Virtual Reality

Unlike augmented reality, virtual reality involves fully immersing users in a simulated 3D virtual environment. VR experiences require users to wear a headset to experience a 360-degree view of a digital world. This exciting segment of XR technology enables users to see and interact with objects and places beyond the real world.

While VR technology caught on early in the gaming and entertainment industries, it has gained steam in other industries such as healthcare, construction and education in recent years. Current uses of this technology include gamers exploring virtual worlds with the Meta Quest VR headset and surgeons using VR models to identify the exact position of tumors.

Mixed Reality

Bridging the gap between AR and VR capabilities is mixed reality (MR) technology. Like AR experiences, MR enables digital and real-world objects to exist alongside each other. However, this hybrid environment takes things one step further by enabling digital and real-world objects to interact in real time. And like VR, MR utilizes wearables to help integrate impressive holographic representations of people and objects within real-world surroundings.

One example of mixed reality technology in use today is Microsoft’s HoloLens headset, which includes a see-through display that enables the user to perceive digital objects as if they exist in a real-world environment.

Current Uses of XR Technology

XR technology is already being used in transformative ways across a range of industries, including entertainment, education and training, healthcare, manufacturing and retail. Whether utilizing virtual, augmented or mixed reality components, extended reality offers great promise for enhancing capabilities within these sectors and beyond.

Entertainment

The gaming industry has been a significant beneficiary of extended reality technology for some time. One example of virtual reality in gaming being used today is the Meta Quest 2, formerly known as the Oculus Quest. This VR headset enables players to fully immerse themselves in video game realms, where they can explore fantasy worlds and complete missions without ever leaving their homes.

Games have also begun to incorporate augmented reality capabilities into their user experiences. One example of AR in action is the popular game Pokemon Go!, in which users catch digital Pokemon characters using the camera function on their mobile devices. Instead of seeing Pikachu in his own virtual world, he shows up on your couch, in your yard or at a nearby, real-world location.

Beyond gaming, XR’s fingerprints can be seen all over the media and entertainment industries, from film production to broadcast news to social media. On the social media front, another instance of AR use is the integration of filters on photo and video sharing apps like Snapchat. In a single swipe, users can add virtual elements to real-life images and videos, from exotic backgrounds to face morphing that makes you look like a cat, a dog or even a younger version of yourself.

Education and Training

No longer do educational experiences have to live within the walls of a school building. With extended reality, students can be immersed in their subject matter like never before. The Information Technology and Innovation Foundation finds that “Immersive technologies allow students to passively participate in remote learning experiences and engage with instructors and peers in real-time using shared virtual elements.”

Additionally, XR technology is particularly promising in offering students with learning disabilities educational alternatives that go beyond physical tools like textbooks and flash cards.

While extended reality is an important emerging tool within K–12 school systems, its educational capabilities go beyond the traditional classroom. Many businesses are exploring the possibilities of virtual reality training environments for new employees. And with working from home becoming more mainstream, there is increasing demand for virtual workplaces that keep remote workers engaged and included.

Healthcare

In the healthcare industry, extended reality technologies offer a range of possibilities and implications for the future of medicine and medical education. Since beginning its partnership with Microsoft in 2015, Case Western Reserve University in Cleveland has been leading the way in developing educational uses for the HoloLens via the HoloAnatomy app. Students at the school use this mixed reality application to “study human anatomy using a life-sized, standing 3D image, which they can walk around and manipulate, dissect with hand gestures, and examine from unobstructed points of view.”

Additionally, hospital systems such as Stanford Medicine are using extended reality software to combine images from MRIs, CT scans and angiograms to “create a three-dimensional model that physicians and patients can see and manipulate — just like a virtual reality game.”

Manufacturing

XR technology is already in place in the manufacturing world, where many companies are using VR capabilities to help cut costs and improve processes. Boeing and Airbus are currently utilizing simulated digital environments to design and test new aircraft features and models.

By eliminating the need for full-scale prototypes, virtual reality can help companies explore early concepts in a safer and more cost-effective way.

Retail

Imagine being able to test drive a car, try on clothes or set up and explore your dream house from the comfort of your home. With extended reality, not only are these experiences possible, in some cases, they already exist.

Furniture retailer IKEA currently uses XR technology to improve their customers’ shopping experiences. The industry giant provides a virtual showroom experience where customers can explore more than 50 furnished or empty spaces.

IKEA also uses mixed reality capabilities that enable clients to preview what products would look like in their specific real-world environment. In this scenario, shoppers first scan their own living spaces and then add 3D products to their homes to find the perfect setup. When the shopper is satisfied with their design, they simply add the items to their cart and check out, without ever having to visit the store.

Challenges of Utilizing XR Technology

With great potential comes great challenges. Although XR technology is poised to continue its exponential growth into the future, there are some obstacles to overcome before extended reality goes completely mainstream. These challenges include: cost, comfort, connectivity, and security and privacy.

Cost

One of the most significant challenges of integrating extended reality is its high cost, especially when it comes to the latest and most advanced tools. Many companies are currently priced out of utilizing XR technology to its fullest extent due to expensive hardware requirements. Additional innovations are needed in order to make these costs more manageable for small businesses and consumers.

Comfort

Comfort level is another area that poses a challenge for the wide-scale adoption of extended reality technologies, particularly among virtual reality experiences. Current VR devices are bulky and uncomfortable, which may be off-putting or inconvenient for some users. A clunky wearable can also limit the immersiveness desired. To reach more users, XR technologies need to be more fashionable and comfortable.

Connectivity

Few things can interrupt the magic of an immersive experience like a spotty connection. Technical issues with connectivity are another challenge facing the extended reality sector.

In order to have seamless experiences with minimal lag and latency, XR users require reliable access to the fastest possible connections. 5G offers a potential solution to this challenge.

Security and Privacy

The success of XR technology is dependent on the secure transfer of data and information between platforms. To keep user information safe, security and privacy concerns will need to be prioritized and vulnerabilities addressed.

XR Trends

Despite the challenges extended reality technologies face when it comes to widespread adoption, industry trends are promising.

Not only are XR capabilities expected to grow in the years to come, but the technological innovations that fuel XR optimization are also advancing rapidly.

Projected Growth and Adoption

Between 2022 and 2027, the extended reality market is expected to achieve a compound annual growth rate of 57.91%.

According to Mordor Intelligence, the COVID-19 pandemic and its resulting lockdowns have played a role in driving XR technology development, with businesses worldwide “trying to find ways to get their internal and wider geographically dispersed teams to communicate, collaborate, and find a path forward during the crisis.”

Furthermore, the pandemic has highlighted the potential of immersive technology to address pandemic-related challenges, with possibilities like recreating travel experiences, shopping from home and enhancing remote learning and working.

Virtual reality is expected to lead the way over the next few years, having already carved out spaces within the gaming, education and healthcare industries. However, as technologies continue to emerge, AR and MR applications are poised to become more widespread as well.

Advances in Hardware

One way to address some of the challenges facing XR technology integration is with hardware solutions. As technology advances, hardware becomes smaller and more powerful, a helpful adaptation when it comes to XR wearables.

According to Forbes, hardware innovations will allow for smaller and lighter headsets, enhanced digital displays and illumination, as well as important features like eye-tracking, motion detection and haptic feedback.

The Metaverse

The Metaverse is a shared virtual space, which Vice describes as an “all-encompassing digital world that exists parallel to the real world.” Also referred to as Web 3.0, the Metaverse consists of three key components: 3D environments, avatars and gamification.

Users within the Metaverse can interact with places, objects and other people in a completely virtual world, where users can “attend” live concerts, earn virtual currency and carry out many of the same functions of daily life in the physical world. This extended reality universe is expected to grow rapidly, reaching an $800 billion market valuation by 2024.

5G Connectivity

Extended reality technologies require the quick transfer of large data volumes in order to operate efficiently. 5G connectivity allows for speeds up to 20 times faster than existing mobile networks, and these capabilities help make wireless and cloud-based VR and AR possible.

XR Today reports that 5G is “particularly crucial to the future of extended reality, ensuring users can access wireless content wherever they are for the ultimate immersive experience.” Furthermore, 5G will help expand access to XR technologies by eliminating the need for expensive computers and enabling businesses to integrate XR capabilities while avoiding hefty infrastructure investments.

Remote Collaboration

As working from home has become more prevalent over the past few years, the need for productivity and engagement solutions has become more apparent. Extended reality can improve remote collaboration and help solve problems related to worker disengagement by enabling remote workers to connect virtually with colleagues in a way that makes them feel as if they are in the same office space.

While it may require an upfront investment by businesses, applications of XR in remote work settings have the potential for a significant return on investment and employee satisfaction over time.

Virtual Production

Virtual production is gaining steam in the entertainment and media industries for its seamless integration between real world and virtual environments. It combines physical and digital elements using XR technologies and software tools to deliver virtual sets that are cost-effective and captivating.

Virtual backgrounds and AR elements superimposed onto physical sets can place broadcasters or actors anywhere in the world. Unlike green screens, where surroundings are masked and have to be imagined, virtual production enables performers to see the computer-generated world on the screen in real time. Additionally, XR technology enables the content on the screen to be rendered dynamically from the camera’s point of view, meaning that content and camera move together. A popular example of virtual production is image-based tracking of live sporting events.