Markerless gait analysis through a single camera and computer vision

Walking is one of the most common locomotion in daily living activities and provides vital information about an individual’s health status (Camomilla et al., 2017). Through gait analysis, physicians or therapists can systematically diagnose gait-related disorders and injuries, such as ankle sprains, knee osteoarthritis or hip joint impairments (Baker, 2006, Fritz et al., 2014, Gu et al., 2018). Kinematic gait analysis, a subcategory of gait analysis, involves measuring a range of spatiotemporal and kinematic variables, such as step cycle time, stride length, speed, and lower extremity joint angles, to quantitatively evaluate the degree of abnormality in walking (Chen et al., 2016). Quantitative evaluations of gait performance are more reliable than visual observations alone due to their repeatability and objectivity (Leardini et al., 2017).

To date, motion capture methods used in gait analysis can be classified into two categories: marker-based methods and markerless methods. Marker-based methods, such as motion tracking systems, measure joint kinematic parameters by directly attaching markers to anatomical landmarks on human body (Reissner et al., 2019). Although these motion tracking systems can track body motion with great precision and accuracy, they have several limitations, such as being time-consuming for experiment setup and data post-processing, requiring expertise for correct marker placement, and potentially altering one’s natural body movement patterns (Carse et al., 2013). In a few prior studies, Inertial Measurement Units (IMUs) have also been adopted for the measurement of lower extremity kinematics (Beravs et al., 2011). These IMU sensors are portable, lightweight, less obtrusive, and do not require a controlled environment. Yet, gyroscope-based orientation updates are prone to errors caused by gyro integration drifts, and these errors tend to accumulate over time (Fan et al., 2017).

Recent advancements in deep neural networks have led to the emergence of computer vision-based human pose estimation algorithms as a promising solution for markerless gait analysis. These techniques can automatically extract subjects’ kinematic parameters from red-green-blue (RGB) images and reduce the dependency on observers (Cronin, 2021). For instance, a previous study used OpenPose library to analyze the gait trajectories of patients with cerebral palsy by estimating body keypoints and joint angle variations in the 2D image plane (Kidziński et al., 2020). Although this approach could potentially assess early symptoms of neurological disorders, it was challenging to evaluate the spatiotemporal parameters in 3D space due to the exclusion of depth information in 2D images. To overcome this limitation, several recent studies have studied 3D pose estimation algorithms by adopting multiple cameras to perform markerless gait analysis (Kanko et al., 2021a, Kanko et al., 2021b, Moro et al., 2022, Vafadar et al., 2022). For example, in Kanko et al. (2021a) and Kanko et al. (2021b), researchers assessed the spatiotemporal and kinematic performance of treadmill walking based on a markerless motion capture software and eight synchronized webcams. Vafadar et al. (2021) evaluated a four-camera markerless system in terms of kinematics and spatiotemporal parameters by processing walking trials from 41 asymptomatic and pathological participants. These multi-camera markerless methods yielded comparable results against marker-based motion capture systems. However, calibrating multiple webcams remains a time-consuming and expertise-required procedure for researchers and clinicians.

Given that many common gait-related disorders and injuries manifest in the lower extremities, our research aimed to provide insights for diagnosing and treating conditions that primarily affect the lower limbs. Therefore, this study mainly investigated the feasibility of employing a single-camera-based computer-vision method in gait analysis and we primarily focused on the lower limb kinematics. In particular, an open-source markerless pose estimation model named Human Motion and Shape Prior (HuMoR) was adopted to estimate the 3D positions of 22 body keypoints, including hips, knees, ankles, and toes, by processing RGB images collected by a single camera (Rempe et al., 2021). To our knowledge, the HuMoR model has not been previously utilized in other studies for 3D gait analysis or validated against a marker-based motion capture system. We compared the gait parameters regarding lower extremity kinematics and spatiotemporal characteristics between the marker-based method and the markerless method across 14 participants. Furthermore, this study also investigated the effect of camera viewing angle and viewing distance on the accuracy of the measured gait parameters yielded from the markerless method.

Several recent studies have demonstrated the potential of single RGB-D or RGB cameras in 3D gait analysis and lower extremity angle measurement (Guo et al., 2019, Hatamzadeh et al., 2022, Balta et al., 2023, Gu et al., 2018, Liang et al., 2022, Zhu et al., 2023).

Guo et al. (2019) developed a mobile system using a single RGB-D camera (Microsoft Kinect V2) for human gait tracking and analysis in canonical coordinates. The system integrated depth and 2-D RGB image data for 3-D lower limb pose estimation and employed simultaneous localization and mapping for real-time limb and camera tracking. Enhanced by a mask-based strategy for dynamic environments, the system utilized a support vector machine and bidirectional long-short term memory network to detect abnormal gait from extracted features. Hatamzadeh et al. (2022) introduced a kinematic-geometric model for spatiotemporal gait analysis, relying on ankle depth trajectories in the frontal plane and distance-to-camera data. Their approach involved identifying gait patterns, modeling them with parameterized curves, and then using this model to develop a fitting algorithm for calculating spatiotemporal gait parameters. Balta et al. (2023) proposed and validated an innovative clinical gait analysis protocol tailored for cerebral palsy (CP) patients, which employed a markerless system using a single RGB-D camera, aiming to reduce the setup time and patient discomfort. Their study included 18 CP patients and assessed the accuracy and reliability of spatial–temporal parameters and sagittal lower limb joint kinematics, comparing them with those derived from a conventional 3D marker-based clinical gait analysis protocol.

Gu et al. (2018) introduced a method utilizing a single RGB camera along with OpenPose (Cao et al., 2021) and the GrabCut algorithm (Rother et al., 2023) to track the 2D joint coordinates and foot orientation. They applied the sparse representation of an active shape model to reconstruct 3D trajectories and estimated gait angular features under various walking conditions. Liang et al. (2022) developed a 3D markerless pose estimation system based on OpenPose and 3DPoseNet algorithms (Moon et al., 2019). They adopted sample entropy to analyze dynamic signal irregularity degree for gait parameters among elderly and young groups. Zhu et al. (2023) designed a markerless human gait assessment system employing a monocular camera, integrated with BlazePose (Bazarevsky et al., 2020) for pose estimation and an additional post-processing filter. Their system can process video input into the gait signal (e.g., hip angles, knee angles) and then decode the gait parameters to obtain a visualized gait analysis report. To the best of our knowledge, no other study has investigated the impact of a single camera’s viewing angles and distances on spatiotemporal gait parameter estimation and lower extremity angular variations.

留言 (0)

沒有登入
gif