As mixed-reality environments continue rising in popularity, researchers are working to develop safeguards to protect users from nefarious hackers.   

Kennesaw State University assistant professor of computer science Rifatul Islam and a team of researchers from around the country are working to develop these safeguards, aiming to protect against cognitive attacks on mixed-reality systems, which fuse real-world environments with virtual ones.

Islam is collaborating with researchers from higher-education institutions Pennsylvania State University, George Washington University, Northeastern University, the University of Southern California, in addition to industry partner Design Interactive

The three-year, $8.5 million project, Verified Probabilistic Cognitive Reasoning for Tactical Mixed Reality Systems, or VeriPro, is funded by the Defense Advanced Research Project Agency, a research and development agency within the U.S. Department of Defense.

“We are looking for ways to protect the end user, particularly in the military, where augmented and mixed-reality applications are used for training for different types of combat,” Islam said.

Mixed-reality environments are commonly employed by the military, which uses these systems for training purposes, allowing users to grow accustomed to potential environments they might face during service.

While mixed reality is a useful training tool, it can be vulnerable to cognitive attacks, a type of warfare that seeks to alter people’s thoughts and actions into ones that favor the attacker’s objectives, which can also cause cybersickness, a set of symptoms that are similar to that of motion sickness.

The primary objective of this project is to construct and validate cognitive models to protect users against cognitive attacks during tactical mixed-reality experiences. These models will be used to automatically verify critical mixed-reality system properties, including but not limited to cybersickness, information flow and the prevention of user interface. If a system fails to meet a property, the system will provide diagnostic feedback and suggest modifications.

“One example of something that could occur during an attack is that the user’s perception could be changed. If an object in the real world is 10 meters away, an attacker could make it look like its only 5 meters away, with the change in perspective being very jarring to the user,” Islam said.

Cybersickness can cause nausea, dizziness, headaches, and other symptoms, but it is caused by time spent using a screen, such as a smartphone or a computer, rather than movement. When a person in mixed reality experiences cybersickness, it can keep them from being able to make decisions in their optimal cognitive state.

Islam’s role on the team stems from his six years of work on cybersickness, during which he has studied how to detect cybersickness in real time and how to mitigate the symptoms.

“I’ve conducted a lot of different types of research around cybersickness and how to ensure user safety and comfort when they are using a mixed-reality device,” Islam said.

Islam’s expertise in cybersickness will help the team achieve their main goal of this project: to understand when a system has been compromised under different attack models and ultimately safeguard the user by observing the risks of mixed reality systems and creating solutions.