— The explanatory text is provided here in English only. However, the thesis may also be written in German. —
Context & Motivation
As the demand for care robotics grows, the ability of robots to safely and intelligently assist humans with Activities of Daily Living (ADLs) is becoming crucial. Preparing a cup of tea is a classic example of a complex, multi-step everyday task. The task requires coordination, timing (boiling water, steeping tea, adding sugar), and an understanding of human preferences.
The goal of the thesis will be to be to develop a proactive, assistive agent that anticipates human needs and behaviour. By predicting the next steps and recognizing potential hazards before/when they happen (e.g., spilling boiling water), the agent will provide seamless and safe support in a shared workspace.
There is already a fully developed Unity VR environment (it might need to be ported to a newer Unity version) and a rich multimodal dataset collected from 45 participants during tea preparation. This dataset includes:
- Full kinematics (Head and hand positions/rotations)
- Eye-tracking (Per-eye rotation, gaze position, and gazed objects)
- Interaction data (Grabbed objects and task progress)
Using this foundation, the task will be to integrate and program a proactive robotic assistant within the VR simulation. The core challenges will include:
- Integrating a Robotic Arm: Implementing a virtual robotic arm designed to physically assist the user (e.g., stabilizing a mug, proactively fetching out-of-reach items like sugar or a tea bag).
- Safety & Recommendation: Developing an algorithm to analyse the user’s real-time kinematics and gaze to detect dangerous situations before they occur, and to provide contextual recommendations for the next step.
Methodology
The architecture of the Human-Aware agent must remain transparent and explainable. Thus, an approach utilizing ACT-R (Adaptive Control of Thought—Rational) is preferred.
Additional Resources
C. Hao, N. Russwinkel, D. F. B. Haeufle and P. Beckerle, “Conceptualizing the Integration of Cognitive and Physical Models to Enable Continuous Human-Robot Interaction,” 2025 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), Duisburg, Germany, 2025, pp. 111-118, doi: 10.1109/CogSIMA64436.2025.11079490.
Hao, C., Russwinkel, N., Haeufle, D. F. B., & Beckerle, P. (2023). A Commentary on Towards autonomous artificial agents with an active self: Modeling sense of control in situated action. Cognitive Systems Research, 79, 1–3. doi:10.1016/j.cogsys.2022.12.006
What you should bring with you
- Prior experience with Unity and ACT-R or a willingness to learn
- Interest in relevant factors of underlying principles of human behaviour
- Be able to work on the thesis in Lübeck, as the VR headset can only be used at the university
Sound like your kind of role? Let’s talk.
If you are interested, please contact Timon (timon.dohnke@uni-luebeck.de) with the following information:
- Who are you, what are you studying and what stage are you at (which semester)?
- What is your background? Have you already taken any of our modules (e.g. HMI, Intelligent Agents, CCS)? (recommended)
- When would you like to start and what are your temporal requirements?
- What interests you about this topic? Which aspects are particularly relevant to you?
- Do you have any preferences regarding what you would like to learn during your thesis or how you would like our collaboration to be structured?