Robotics issues on vision-based human motion analysis

Citation

Liu, H. (2009), "Robotics issues on vision-based human motion analysis", Industrial Robot, Vol. 36 No. 2. https://doi.org/10.1108/ir.2009.04936baa.002

Robotics issues on vision-based human motion analysis

Article Type: Viewpoint From: Industrial Robot: An International Journal, Volume 36, Issue 2

Keywords: Robotics, Intelligent sensors

Robotics research involves a wide spectrum from the first industrial manipulator to Mars rovers, and from surgery robotics to cognitive robotics, industrial and real-world applications are the force to drive the research frontier forward; the multidisciplinary nature of research nowadays is pushing the boundaries and conventional robotics of both theory and lessons learned. Both academic and industrial applications have been required to change their forms and adapt the variants into the context of intelligent robotics, e.g. vision-based human motion analysis. It is expected that analyzing human motion behaviour, especially recognizing suspicious behaviour or identifying unhealthy motion behaviour, could play a crucial role in applications in crime and healthcare industry. Recently, a report by the British Government also warned that billions of pounds of investment in closed-circuit TV technology has failed to cut UK crime, though Britain had more cameras than any other European country, in that “no thought has gone into how the police are going to use the images and how they will be used in court”.

It is technically evident that the state-of-the-art in vision sensing hardware sufficiently meets the application requirements for the purposes of public safety and healthcare, the hardware that CCTV-based human motion analysis involves are hundreds of thousands of networked robot cameras, organized in certain regions and locations. It seems that the principle learned from system integration industry has been encouraged to solve the hardware-related CCTV problems, that is to say that embedding robot motion control and calibration, etc. into field cameras instead of adding up the whole workload into a central computation of CCTV-version distributed control systems. On the other hand, developing algorithms capable of analyzing human behaviour in certain constraints is the major bottleneck; it holds the key to transfer research findings into real-world applications. A typical software package of such a system usually consists of motion segmentation, object classification, action recognition and behaviour description. It is anticipated that an ideal human motion analysis system automatically generates either a quantitative or symbolic description for understanding the involved human motion behaviour given a set of visual sequences.

A large amount of research has been conducted and contributed to human behaviour understanding at different systematic levels in the past two decades, view-invariant human motion analysis and computational complexity, however, remain most problematic; making a human motion analysis system infeasible to real-world applications. In general, a practical scenario always offers partial view/image sequences, it is crucial that an ideal human motion monitoring system be able to analyze and reason from human behaviour with partial information, e.g. 3D pose estimation from monocular sequences. On the other hand, it is demonstrated, without consideration of computational cost constraints, that existing tracking and object recognition algorithms provide arguably acceptable results in that a large number of parameters need to be estimated and that perspective projection makes the recovered poses ambiguous. Hence, trade-offs have to be balanced between computational cost and recognition accuracy. Behaviour understanding is complex in that the same behaviour might have several different meanings depending upon the scene and task context in which it is performed. It is argued that more efforts might be diverted to the intelligent connection between preprocessed image sequences and a semantic human motion description which could link the former with domain-dependent scenarios. From the perspective of robotics, a human motion skeleton can be constructed in terms of robot kinematics models of body segments; it then converts human motion analysis into a conventional robotics problem. For instance, a technique called qualitative normalized templates integrates a variant of robot kinematics and template matching, it achieves the same motion recognition rate and requires less than one-fourth of the computational cost required by the dominating methods of HMM and its variants. It is demonstrated that combining robotics and algorithmic mechanisms might achieve a feasible solution to describing human behaviour in a more natural way.

Like it or not, we will live in CCTV camera-controlled environments sooner or later, to which the contribution of intelligent robotics will be increasingly evident and recognized.

Honghai LiuInstitute of Industrial Research, University of Portsmouth, Portsmouth, UK