3D Modeling of Human Motion from 2D Video Images by AI

Contributed Talk | Day 2 | 15:25:00 | 20 Minute Duration | GG-C
  • Yunju Lee
    Grand Valley State University Assistant Professor

Did you attend? Rate this session

3D Modeling of Human Motion from 2D Video Images by AI

Contributed Talk | Day 2 | 15:25:00 | 20 Minute Duration | GG-C

Technology has enhanced the ability to study human movement and provide accurate measurement and real-time analysis in sports, medical, rehabilitation, and forensic fields. However, there are limitations in the acquisition of accurate 3D positions using the current motion capture system because the laboratory has to be equipped with a 3D motion capture system that imposes space and time constraints. Current 3D motion study needs markers attached to the human body and these markers limit motion range and total body movements. In many ways, the current system is not useful to apply in the real world outside of the lab. Is it possible to accurately estimate the human body’s 3D motion from 2D video images without any markers? If this is feasible, this provides ample freedom in human movement research in a variety of areas including forensic and clinical gait analyses and human action classification. 

The goal of the current project is to develop a high-quality 3D motion estimator using the artificial neural network (ANN). To do this, it is necessary to have high-volume and highly-accurate 3D motion datasets that are currently being collected in the Biomechanics and Motor Performance Laboratory (BMPL) at the Grand Valley State University. Establishing high-quality 3D datasets, itself, could be a valuable resource to contribute in the research field. The project includes a data acquisition system providing exact 3D body joint kinematics from a 3D motion capture system as ground truth, which will serve as control data to be used to validate 2D data. At the same time, three 2D video cameras will capture human motion images as the input of a 2D estimator. The acquired data will be used to train a Deep Neural Network to estimate 3D human body joint positions accurately.

Once a high-accuracy estimation in 3D joint positions is achieved then, we can use the estimator to analyze properties of human movements, such as gait features to see changes in hip extension and knee flexion angles in the initial contact phase, or leg inclination of terminal stance phase. For example, we can identify human behaviors and/or analyze and further diagnosis abnormality in gait practically by taking only 2D video images without markers or complicated systems. Furthermore, in a practical manner, we can apply the 3D estimator to identify and study concussion for football players in the field without markers or complicated systems.