Machines will Adapt to Humans
What still sounds like science fiction today may soon become reality: intelligent XR agents that intuitively guide humans through complex working environments. DASHH alumna Ke Li is already developing and exploring these technologies at particle accelerators and advanced laser laboratories.
Ke Li, you work on intelligent virtual agents and mixed reality systems for complex research infrastructures. What exactly do you do, and why should people outside academia care about it?
My research is about creating genuine synergy between humans, AI, and robots, and I am convinced that mixed and extended reality is the key technology to mediate that collaboration. I study these interactions under some of the most demanding conditions imaginable: particle accelerators, advanced laser laboratories, and other large-scale research infrastructures. The reason this matters far beyond academia is simple: if a system works reliably in environments where radiation, high-energy beams, or remote operation make a single mistake costly, the same principles transfer naturally to industrial settings such as manufacturing, energy, healthcare, and any workplace where humans and intelligent machines need to cooperate safely and efficiently.
Machines should not only process language or text, but also be able to interpret gestures, eye behavior, and biometric signals such as brain activity.
Dr. Ke Le
In your PhD, you developed the idea of a "particle accelerator metaverse." What does that mean in practice?
In popular media, the "metaverse" is mostly described as an immersive virtual space for social interaction. In my dissertation, I redefine it as something quite different: an interconnected digital space in which collaboration takes place not only between humans, but also between humans and robots, and between humans and AI. The "particle accelerator metaverse" is therefore a working environment, a place where scientists, autonomous systems, and intelligent agents jointly operate, monitor, and maintain one of the most complex machines ever built.
What becomes possible in research through such a metaverse that was not possible before?
It opens up access to spaces that were previously off-limits. Researchers can virtually enter a particle accelerator tunnel during operation through photorealistic digital twins, and teleoperate robots inside it with a level of spatial awareness that flat screens simply cannot offer. In hazardous laser laboratories, where a scientist's hands and eyes are constantly occupied, they can speak naturally to a companion agent, saying something like “show me the beam profile”, and the visualization appears, anchored directly on the physical detector, seamlessly integrated into the 3D environment. What used to require switching between instruments, screens, and protective equipment becomes a single, fluid interaction.
How does the interaction between humans and intelligent virtual agents change the way humans work with machines?
Human communication has evolved over millennia to be deeply multimodal: we combine speech, gesture, gaze, posture, and even subtle physiological cues. I believe that the most intuitive human-computer interfaces are the ones that meet us on those terms. That means machines should not only parse speech or text, but also reason about human intent through gesture, gaze, and biometric signals such as brain activity. The shift this enables is fundamental: instead of humans adapting to machines, machines begin to adapt to humans, leading to safer, more accurate, and more efficient task execution while protecting the user's well-being. At advanced laser laboratories, for example, a mixed reality head-mounted display is not a gadget; it is a critical safety device for protecting researchers' eyes and, at the same time, a scientific instrument in its own right.
With the rapid progress of AI, what once looked like decades of work is becoming achievable within a single research career, and that conviction continues to drive me.
Dr. Ke LI
You were a PhD fellow at the DASHH graduate school. What from that time has most strongly shaped your current research?
What shaped me most was being trusted with a genuinely critical and forward-looking research project from the very beginning. DASHH placed me at the intersection of accelerator physics and human-computer interaction at a moment when both fields were ready for that conversation. The combination of an ambitious vision, demanding scientific standards, and the freedom to define my own path taught me to think long-term, work across disciplines, and take ownership of research questions that had no established playbook.
Your work combines physics, computer science, and XR technologies. How did DASHH prepare you to work across these disciplines?
DASHH gave me everything genuinely interdisciplinary research requires. My supervisors, internationally recognized across both physics and computer science, modeled what it means to think rigorously in more than one discipline at once. The infrastructure was equally exceptional: access to the particle accelerators at DESY on one side, and to the human-computer interaction and XR laboratories at the University of Hamburg on the other. The structured curriculum, soft-skills training, and the community of fellow doctoral researchers across the natural sciences turned interdisciplinary work from an aspiration into a daily practice.
Was there a moment during your PhD that significantly influenced your direction, either scientifically or personally?
The defining moment came at the very start of my PhD, when my collaborators Ara and Tino from the advanced laser group at DESY approached me to develop a mixed reality solution for a long-standing laser safety problem. In that conversation I saw immediately both the real-world impact such a system could have and the size of the research gap that still needed to be closed. A second formative moment came in 2022, at my first international conference, IEEE ISMAR in Singapore. An emeritus professor in our field predicted that mixed reality would need at least twenty more years of development before everyday adoption. I left that talk convinced of the opposite: a twenty-year gap does not require twenty years of research to bridge. With the rapid progress of AI, what once looked like decades of work is becoming achievable within a single research career, and that conviction continues to drive me.
You now lead a work package in an EU project. How did your PhD prepare you for this level of responsibility?
From very early on, my supervisors, Prof. Frank Steinicke, Prof. Wim Leemans, and Dr. Reinhard Bacher, gave me the freedom and the opportunities to act independently and to think boldly. They consistently encouraged me to take initiative, propose directions, and represent the work externally, all of which trained me to think like a leader rather than only as an executor. DASHH also embedded me in a network of peers conducting world-class research, and that environment made curiosity, ambition, and continuous learning feel completely natural, exactly what is needed to lead a work package in a multi-partner European project.
How will we interact with complex machines or virtual agents in ten years, and what role do you want to play in shaping that future?
While today everyone is talking about AI agents, I believe that in ten years we will be working with what I call "X-RAI agents", eXtended Reality-mediated Artificial Intelligence agents. Think of Jarvis from Iron Man: an intelligent system that fluidly moves across 2D desktops, immersive displays, and holographic interfaces, responding to high-level human input, anticipating intent, and creating context-aware visualizations and actions in the physical world. My ambition is to help drive this development by grounding it in real scientific and industrial challenges, by ensuring that these agents remain safe, transparent, and human-centered, and by training the next generation of researchers who will build them.
