Motivation
In incidents involving a mass accumulation of victims — such as natural disasters — human rescue personnel are a scarce resource, and rescuers may put themselves at risk when reaching victims in hazardous environments, such as partially collapsed buildings. As a result, victims may not receive help in time. Rescue robotics aims to close this gap by supporting human responders: robots can be deployed in larger numbers and, unlike humans, replaced if damaged, without risking additional lives. One-way mobile robots can help is by locating victims and assessing their condition in the field, so that help can be directed efficiently and risk to on-site rescuers minimized.
Today, several vital signs can already be measured contactless: heart rate via remote photoplethysmography (skin-reflection changes, primarily on the face) and consciousness level via the upper end of the Glasgow Coma Scale, which requires detecting blinks and eye openness. Both measurements depend on a clear, unobstructed view of the face — but how close a robot can approach a victim is limited by the physical dimensions of the robot and the surroundings (e.g., rubble). As a result, in many real-world scenarios the victim’s face is too far away or at too extreme an angle for reliable measurement.
Objective
The goal of this thesis/project is to build a system that automatically pans, tilts, and zooms a camera onto a human face, using an RGB-D camera (Intel RealSense) as the reference sensor. Concretely, the work involves:
- Selecting or building a pan-tilt-zoom (PTZ) camera.
- Spatially calibrating the PTZ camera into the coordinate frame of the RGB-D camera.
- Detecting and tracking a face in real time and using the calibration to continuously steer the PTZ camera so the face remains centered and in focus in its video feed.
The final system should demonstrably improve the range and reliability at which facial vital-sign measurements can be taken, compared to using the RGB-D camera’s own (fixed, non-zooming) image.
Required Skills
- Knowledge in programming (python, C++ or C)
- Basic Understanding of ROS2
- Understanding in robotics and image processing
- Understanding of CAD design with programs like Solidworks
- Interest in hardware design in general
Language
- German or English
The scope of the project may be adjusted depending on whether it is undertaken as a bachelor’s thesis or a project. Interested individuals are encouraged to reach out for further information.