XR Technologies – Bold steps to new realities

The simulation of reality has developed vastly in the last decade, and the pace of the development is in­creasing. Virtual and augmented reality have become buzzwords and artificial intelligence (AI) is shaping up to change mankind for better or worse. Applica­tions can be found in industry as well as in the gaming world. Where is simulation going and how will it shape our world?

Faster, better, and simpler are words that drive industry. Instant gratification and consumption are the order of the day. The more one knows about the world, the more one becomes aware of experiences that are missing. But what is an experience? What makes a trip more real than a simulation of the same trip? What is missing?

The birth of different realities

X Reality (cross reality / XR) encompasses multiple forms of artificial realities such as augmented reality, virtual reality and mixed reality.

Augmented reality, as the name implies, seeks to add ar­tificial entities such as data, text, and objects to a physical environment. The entities are visible through a camera that recognises physical trackers and places the entities accord­ing to the location of these trackers.

Virtual reality is an artificial environment created with the intention to simulate self-presence in an environment with the use of a head-mounted display (HMD) that tracks head movements and position thanks to cameras or light­house boxes.

Mixed reality is a combination of both technologies. In mixed reality us­ers experience a virtual environment, yet some of its components are an­chored to real-life physical objects.

A collective interest

Communication is incredibly impor­tant for all living beings; visual inputs, sounds, smells, tastes and sensations are ways of gathering information from the environment, thereby increasing human chances of adapting and sur­viving. There is, naturally, no longer the need to survive in the wild. Humans are, nevertheless, highly dependent on information. It is not always important how the information is obtained, but rather how easily it is understood.

Information has become much more readily available to everyone – from gossip to messengers, newspa­pers, news channels, and the Internet. Basic knowledge is no longer a luxury and even academic diplomas are no longer an advantage in many fields in the same way that experience is.

Simulators have long been used as tools to gauge performance. The line between simulation and reality is shrinking quickly as new tools and software are developed. If simulations are combined with AI, completely dif­ferent scenarios to test users can be developed. This allows learning in a more involved manner than with a textbook.

Room for both the old and the new

It always takes some time for new tech­nologies to be adopted. The number of detractors usually peaks when the technology is new and untested; many fear the consequences and are blinded from the benefits. History has shown, however, that it is always a matter of time: vehicles did not make human legs redundant, emails are extremely convenient, but people can still write letters if they wish. Cell phones re­placed many technologies, but spe­cialised tools are still sold.

Standards are difficult to predict, be­fore the Internet was invented, it was difficult to imagine it. VR and AI togeth­er will not replace the cinema or drivers and it will take time to reach conclusive opinions about it. Perhaps it is the fear of no longer needing human imagina­tion or of losing more control to tech­nology that makes some people reluc­tant to adapt.

Some doubt the viability of VR mov­ies, claiming that it is impossible to keep an audience focused with such freedom. The theory is that they will likely miss important details of the sto­ry if they are easily distracted. However, these kinds of movies are possibly not meant to be seen once, or in a room with many people.

More than a game

There is still a stigma attached to games and the people who enjoy them. The biggest concerns are the in­ability to separate fiction and reality and the normalisation or celebration of actions that are not deemed accept­able in society.

The debate will probably rage for several more years, but attitudes are changing gradually. Games are not only the means to live a fantasy, but a means to engage with others, to test skills or develop them.
A game is a structured form of play: rules and boundaries are set, and an objective is explained. A well-crafted video game easily relays these rules and objectives to the user: how high one can jump, what can be interacted with, what rules end the game, and what is the objective.

Similarly, a well-crafted application that monitors health not only keeps track of numbers but can also chal­lenge the user to reach an objective. It can provide an environment with rules to follow as well as visually appealing functionalities to help the user focus on the objective.

When considering realities, it is im­portant to talk about immersion. Af­ter all, if one wants to build a machine, many factors such as the scale, parts, and procedure are important, but moving a machine is not always a cheap option, neither is having it maintained on site.

Nowadays, a virtual copy or digital twin of a machine can be created with the use of surrounding data to predict behaviours, or at the very least to gain a realistic and updated status of the machine.

A digital twin can bring many ben­efits to many industries; it is not always necessary to add artificial components to a machine as the data obtained is valuable itself. However, there is a branch that takes digital twins further: artificial Intelligence (AI), or more pre­cisely neural networking, which can assist in troubleshooting machine be­haviour by understanding patterns and suggesting solutions.

Patterns everywhere

The term artificial intelligence was coined back in 1950, but the idea of artificial beings developing intelli­gence has been around for centuries. But even with the immeasurable ad­vantage machines have over humans, in some fields the biggest disadvan­tage they have is the lack of pattern recognition.

Neural networking algorithms seek to emulate human brain-pattern rec­ognition based on the comparison of data. How does a human recognise 1 as 1? What about if it was written as I? Effective algorithms do not provide a single answer to the above-mentioned questions, but a set of answers with a conclusion.

Everyone is an architect

Game engines have come a long way. Not only are there multiple options to create applications, but there is a vast number of sources to learn and im­prove the game skillset as well as tools to ease development.

Anyone can engage in creating their own environments or team up to tackle larger projects. It takes a few hours to grasp the basics, and minutes to publish even an empty space with a cube on multiple platforms, from a cell phone to a PC as standalone or virtual reality enabled.

The biggest players right now are Unity3D and Unreal Engine. They are constantly pushing their respective en­gines to appeal to different kinds of de­velopers, including VR-focused ones.

As technology keeps evolving, it may no longer require hours to gener­ate content. Taking a trip to an exotic island with fantastical creatures may be as simple as ticking a few param­eters. In such a world, education will be both academic and empirical with­out costs and risks. In short, it will be a world where simulations can provide all the answers.

Universal simulation

Will perfect simulation be possible in the future? It is currently possible to simulate images, sounds, smells, and tastes. Haptic feedback is slowly ad­vancing, but why would one want to feel pain from a virtual source?

All available senses are important to humans and the joint perception makes experiences “real”. The fear of pain from haptic technology is un­derstandable, but not much different to the fear of hearing a deafening or screeching sound or that of food that is met with immediate rejection. It is natural to try to avoid these sensations, but musicians and chefs need to be exposed to both good and bad music and food.

AI that learns from human feedback can provide unexpected, yet enjoyable or challenging scenarios that allows all senses to take part in experiences.

This article was first published in Elomatic’s Top Engineer magazine 2/2019