Using a new $2.8 million grant from the Defense Advanced Research Projects Agency (DARPA), researchers in the University of Florida Department of Computer & Information Science & Engineering (CISE) will work to augment human cognition by providing task guidance through augmented reality (AR) headset technology in extreme environments, including high hazard and risky operations.
The project titled, “ENKIx: Enabling Knowledgeable Task Guidance In the Extremes,” will use commercial smartwatches, head-mounted equipment and eye-tracking glasses to guide the operator of a specific task. The development of ENKIx will enable the multidisciplinary team, led by principal investigator Jaime Ruiz, Ph.D., a CISE associate professor, to make advancements to task guidance in multiple domains, including cognitive modeling, AI and human-computer interaction.
“This project highlights that with the advancement of AI, we move toward a future where technology is no longer a tool but a partner able to collaborate with us as we perform tasks,” Dr. Ruiz said.
The system is being designed to help both civilian operators and Department of Defense warfighters take on high-complexity tasks such as airplane repair and field medicine. Both activities tend to occur in adversarial environments. The team is developing open-source software for user-agent interactions and reasoning over complex tasks. The system is being prototyped with commercial VR/AR technology so that it can be readily transitioned to applied research programs at the end of the project.
“In a world where technology so often makes your life harder because of increasing complexity, our research looks to develop technology that makes your work easier,” said James Fairbanks, Ph.D., a CISE assistant professor and one of the project’s co-PIs. “It should provide you the right information at the right moment, with the right context to be helpful without distracting. Technology should help, not hurt, people trying to achieve their goals.”
Along with Dr. Ruiz and Dr. Fairbanks, the research team includes co-principal investigators Lisa Anthony, Ph.D., CISE associate professor; David Kaber, Ph.D., chair of the UF Department of Industrial and Systems Engineering; Maryam Zahabi, Ph.D., assistant professor in the Texas A&M Department of Industrial and Systems Engineering; and Evan Patterson, Ph.D., a research scientist at the Topos Institute.
“This project will provide many advancements in helping us understand how to develop and design intelligent systems that can understand the user, the state of the world, and a user’s task,” Dr. Ruiz said.
DARPA developed the Perceptually enabled Task Guidance (PTG) program to explore the development of methods, techniques, and technology for AI assistants capable of helping users perform complex physical tasks. The grant will last for four years.
Allison Logan
Marketing & Communications Specialist
Herbert Wertheim College of Engineering