What is XR?

Introduction to XR

For XR or extended reality, it encompasses everything between actual reality and a completely virtual reality. It spans your real world to a fully digital world. When it comes to your actual reality, take a step outside and look around you. The Earth is vast and beautiful with millions of experiences to enjoy across the realm. Trust your eyes. Many of us are already in the extended reality (XR) realm without even realizing it.

You’ve only taken a small step into an intriguing and magical journey that might shape how you think for years to come. For the world that you live in, anything that is digitally added to it shifts your version of actual reality. Your phone, TV, devices, billboards, etc. all shape that reality. If you don’t believe me, truly disconnect from all digital devices for a week and see what happens. What wonderful benefits could there be from fully disconnecting from the matrix?

But you're not here to enjoy beautiful pictures of our realm. How could any of this be connected to extended reality technologies? Well, you might want to know where you are in order to know where you're going.

Common XR Interfaces

Your current devices typically provide a 2D interface for your 3D world. When you realize how reality is being adapted to or changed, you might be able to imagine how a 3D interface of your 3D world could look like. That realization alone can truly build incredible dreams and worlds.

With 2D interfaces, some of your common interactions in XR include, maps, reviews, recommendations for online shopping, or other fun apps that change how you see the world. When you look at a maps application, it gives you information of where to go or what to do. You might consider what to eat based on reviews without ever actually trying the place. You’re provided an illusion of choice when you’re scrolling through a list of applications. And while using specific applications, you can look up to find constellations to witness an amazing sky moving over our heads. This application projects a digital image onto our real world. This takes us further away from the real world and deeper into extended reality.

As the hardware changes, that version of your altered reality can become much more immersive. Now, you can start interacting with and engaging with others in this digital space. Current applications with your phone allow you move around in your real world to play games with digital assets or to place and view digital objects in your room. This helps you live a more active lifestyle or expedites buying decisions. How many current apps on your phone have such capabilities?

With other augmented reality devices, the 3D digital objects can guide you to acquire skills that can help you fix objects, design structures, or connect with people. Think about the number of times a 2D video or experience has helped you take action. Imagine what could be achieved when you add a third dimension to our experiences. Microsoft's HoloLens 2 gives a preview of these futuristic experiences.

As you proceed forward across the spectrum you start having larger and more obvious pieces of hardware.  Frequently, these experiences can be much more immersive and engaging. For example, a 360 photo or video allows you to look around you from a stationary location. You are now immersed in the 3D space to see things in ways you might not have before. 360 cameras provide a different perspective and freedom of movement within a photo or picture. What are some locations that you would want to view with a 360 camera?

Common XR Accessories

Then, you move along the spectrum into the more interactive hardware and worlds where you have controllers, hand tracking, eye tracking, body tracking, speech recognition, or some combination of these tools. These tools allow you to control digital objects in ways that you normally couldn't in your real world. In the virtual reality space, your controllers enable you to grab, activate, interact with, and engage with digital objects. Each of these digital objects tend to have responses or effects that indicate to you that an action took place to simulate that reality. After this point for experiences, you end up in a fully digital world.

XR Spectrum

And you might be wondering, "what 'spectrum' is being referenced?" Based on the descriptions above, extended reality spans your real world to a fully digital world. What other additions would you add to this spectrum of realities?

Areas of Focus

The key areas that will be explored from this point on for the extended reality spectrum are augmented reality, mixed reality, and virtual reality experiences. Each of these areas have key attributes that set them apart from each other. By the end of this journey, you will be able to determine which extended reality technology is the best to explore for your dreams and purposes.

Augmented Reality

AUGMENTED REALITY (Merriam-Webster): an enhanced version of reality created by the use of technology to overlay digital information on an image of something being viewed through a device (such as a smartphone camera).

Mixed Reality

MIXED REALITY (Microsoft): a blend of physical and digital worlds, unlocking natural and intuitive 3D human, computer, and environment interactions. This new reality is based on advancements in computer vision, graphical processing, display technologies, input systems, and cloud computing.

Virtual Reality

VIRTUAL REALITY (Merriam-Webster): an artificial environment, which is experienced through sensory stimuli (such as sights and sounds) provided by a computer and in which one's actions partially determine what happens in the environment.

Degrees of Freedom

With a virtual environment, you have varying degrees of freedom. With 3-DoF (three degrees of freedom), you can make rotational movements with your device known as the roll, pitch, and yaw. With 6-DoF (six degrees of freedom), you can also make translational movements along the X, Y, and Z axes. Below are some examples to visualize different degrees of freedom in XR experiences.

3-DoF in VR

This person remains stationary and rotating her head while attempting to reach for digital objects in her line of sight. 360 videos, VR films, and low-interaction experiences thrive in this mode.

3-DoF in AR

This person makes translational movements through a warehouse with augmented reality guidance on his tablet. The visual cues are shown on a 2D view while he is free to move in different directions.

6-DoF in VR

This person has rotational and translation movements in this virtual reality experience. Highly immersive games, training, demos, and scenarios excel with six degrees of freedom for the user.

At six degrees of freedom, experiences can be quite magical. You can move your head to look left and right, up and down, and side to side while being able to physically move forward and backward, step left and right, and raise or lower your elevation. One of the things to consider when creating such experiences is the type of space that you would have for six degrees of freedom. If you are using an augmented reality headset, the experience can stay very self-contained on a table, floor, or on another type of object. At that point, the object remains motionless, but you can can move around to look at the object.

These XR experiences are only limited by your imagination. There are countless environments and dreams that can be created in the digital world. Knowing the hardware that you want to invest can help you determine which experiences to build first. The hardware discussion will be entirely separate post for the future. What does the hardware need to be like for you to consider using these XR experiences on a regular basis?

Interactions

When it comes to XR interactions, it ranges from using controllers, your own hands, or an interface that displays digital information. In augmented reality, you will most likely be interacting with digital objects on a 2D screen like a tablet or phone. As you move into mixed reality, you can start to engage with the digital objects with your hands or controllers that mimic your hands' movements. You will be able to touch, grab, rotate, throw, and many more actions.

Over time, developers created more interactions for the digital objects to provide you with a realistic sensation of what is happening in your reality. Sight and touch are critical components of XR experiences. The next sense that is just as critical: hearing. When a user engages with a digital object, auditory feedback at the appropriate distance and direction can strongly indicate that an event took place. Common examples include pushing a button, moving an object, or even movement from other digital objects. An additional type of interaction for you to experience is haptic feedback. These are tactile sensations for your hands or other parts of your body that indicate actions are taking place in a digital world. With controllers or other accessories, you can receive some haptic feedback once you've engaged with an object. What are some other realistic feedback mechanisms that could help with immersion?

And its these interactions that sets XR experiences apart in the digital world. When it comes to engagement and user experience, you're able to gather insights about them that might not be previously available in a 2D space. You can see what they're looking at, how long they're looking at it, and where they're looking after a prompt or a piece of information is given. You can see how they engage with their hands and how they move to determine the accuracy of their actions. XR experiences allow users a safe space to practice skills for a wide range of industries. New applications include virtual cameras to replay the experience, so the user can observe their own actions. With these insights, you can always improve future experiences or adapt them to the user.

Industry Applications

While this list is not complete, there continues to be a growing list of applications for XR experiences. The only limit is your imagination. XR enables you to add digital components to normal experiences in ways that you couldn't have in the past.

Construction

The construction industry provides a transformative impact everywhere that you look. Many of the skills require operators, licensing, specialists, and engineers that are highly-skilled in specific areas. Practice with these skills in a digital environment will elevate their confidence, expertise, and safety for ongoing and future projects. Tools, machinery, and environments need to be mapped to scale to simulate the environments that these professionals will be in.

Supply Chain

This industry moves everything across Earth. With the amount of supply being moved on a daily basis, teams start to automate their logistics strategy with robotics and more advanced equipment. Very specific and expensive machinery can be demonstrated for new and seasoned professionals to ensure that the supply chain runs smoothly. Highly technical equipment can be trained upon in a digital environment so individuals can acquire new skills quickly, increase efficiency in processes, and adapt to future situations.

Medical

Medical professionals range from patient care, researchers, frontline workers, and more. Each role might need to learn and use highly specialized equipment to make a difference. Digital practice environments expedite the onboarding process while lowering job anxiety and increasing safety.

Manufacturing

Experts in the manufacturing industry create a wide range of specific and detailed creations that have to be precise and accurate. Many environments have varying conditions with technical tools. Being able to simulate any of these experiences can make a difference with onboarding and best practices.

Create Your First Experience

With all of these details in mind, you're ready to make your dreams a reality. Remember, XR includes everything between your real world and a fully digital world. Creating your first XR experience is magical, exhilarating, and freeing. As a human in your current reality, don't be limited by the artificial constructs that limit your imagination. Your reality can be created, transformed, and even shattered when you dare chase your dreams and aspirations. XR is only limited by your imagination. 

Dare to dream.
Dare to ask questions.
And dare to unlearn everything that you've known in your current reality.

The XR space is an exceptional medium anyone can venture into, and I look forward to hearing about your ideas to see what we can create together.