Validity and Reliability of Movement Analysis Methods Used in HumanTrak

Case StudiesAug 23, 2024Last Updated: Aug 23, 2024

Study Information

Collings, T. J., Devaprakash, D., Pizzolato, C., Lloyd, D. G., Barrett, R. S., Lenton, G. K., Thomeer, L. T., & Bourne, M. N. (2024). Inclusion of a skeletal model partly improves reliability of lower limb joint angles derived from a markerless depth camera. Journal of Biomechanics, 170, 112160.

https://doi.org/10.1016/j.jbiomech.2024.112160

Purpose of the Study

The study aimed to assess the validity and reliability of lower limb joint angle measurements using a single markerless depth camera, specifically the Azure Kinect, in various movement assessments. It focused on comparing direct kinematic calculations to inverse kinematic methods that incorporate a skeletal model to determine which approach provides more reliable data for clinical and athletic applications. The Azure Kinect is the same camera used by HumanTrak.

Primary Objective: Compare joint angles and between-day reliability of direct and inverse kinematic methods recorded using a single markerless depth camera against gold standard references for:

Treadmill walking
Treadmill running
Jumps
Squats

Secondary Objective: Identify the minimum number of trials required to maximize reliability for each task and kinematic method, ensuring that the data collection process is both efficient and effective.

Methods

Eighteen healthy adults (5 females and 13 males; age = 28.1 ± 3.9 years, mass = 79.7 ± 15.9kg, height = 176.5 ± 11.0cm) participated in the study. Each participant attended two 60-minute testing sessions at a biomechanics laboratory approximately one week apart. Data were collected using an Azure Kinect depth camera (Microsoft, Washington, US) and a 12-camera marker-based optical system (Vicon, Oxford, UK). The depth camera was positioned two to three meters in front of the participant, who faced the camera. They performed a series of movement tasks, including:

Treadmill walking
Treadmill running
Drop vertical jumps
Single-leg Countermovement Jump (CMJ)
Bilateral CMJ
Single-leg squat

Five trials of each jumping task and 35 seconds of walking and running were recorded, with 30-60 seconds of rest between trials. Key point positions were obtained from the Azure Kinect and processed to determine lower limb joint angles using direct kinematics and inverse kinematics with a skeletal model in OpenSim. Direct kinematics calculated joint angles based on key point-defined segments, while inverse kinematics used a full-body musculoskeletal model constrained by physiological joint movements. Marker-based data from the Vicon system served as a “gold standard” reference.

Statistical analyses compared lower limb joint angles from the two methods using one-dimensional Statistical Parametric Mapping (SPM) to identify significant differences. Between-day reliability was assessed with Intraclass Correlation Coefficients (ICCs), Mean Absolute Error (MAE) and Minimal Detectable Change (MDC).

The number of trials required to maximize reliability was determined by calculating Intraclass Correlation Coefficient (ICC) values using one to five trials per participant, identifying the minimum number of trials needed to achieve an ICC within 0.025 of the maximum ICC.

Key Findings

…direct measurements from the Azure depth camera provided high between-day reliability for measuring lower limb joint angles across various movement tasks…[with] ICCs ranging from 0.80 to 0.93…

Validity

Azure Kinect vs. Azure Kinect with Inverse Kinematics vs. Vicon

The results indicate that direct measurements from the Azure Kinect camera closely align with data filtered using inverse kinematics and Vicon data across most movements and joint angles, suggesting high accuracy and reliability (Figure 1).

The consistent overlap between Azure Kinect angles and Vicon angles, coupled with low variability, highlights the precision of the Azure Kinect camera. This method demonstrates robust performance in capturing joint movements, making it a valuable tool for clinicians and researchers in motion analysis. Overall, angles measured using the Azure Kinect camera provide a reliable and accurate approach for analyzing lower limb movements.

Figure 1: Validity of joint angle measurements, including hip flexion-extension, hip adduction-abduction, knee flexion-extension and ankle dorsiflexion-plantarflexion, across various tasks such as walking, running, single-leg squats, single-leg CMJ, bilateral CMJ and drop vertical jumps. Green lines and shaded areas represent direct measurements from the Azure Kinect camera, orange lines and shaded areas represent Azure Kinect measurements processed with inverse kinematics and dashed lines represent Vicon measurements processed with inverse kinematics. P-values indicate statistically significant differences between methods.

Reliability

Azure Kinect vs. Azure Kinect with Inverse Kinematics vs. Vicon

The Azure Kinect method performs particularly well in this comparison, consistently showing lower MAE for most movements and joints, indicating high reliability. It excels in knee flexion/extension during walking and hip adduction/abduction during single-leg squats, where it has the lowest error compared to other methods. The Azure Kinect method generally exhibits high ICC across movements and joints, particularly for knee flexion/extension during single-leg CMJ, demonstrating strong reliability.

Additionally, the Azure Kinect method shows excellent sensitivity with low MDC values for hip adduction/abduction during walking and knee flexion/extension during bilateral CMJ. Overall, the Azure Kinect method stands out in terms of accuracy, consistency and sensitivity, making it a reliable and precise tool for motion analysis in clinical and research settings.

Comparison of three different calculations for joint angles (hip flexion-extension, hip adduction-abduction, knee flexion-extension and ankle dorsiflexion-plantarflexion) across multiple trials for various movements, including walking, running, single-leg squats, single-leg CMJ, bilateral CMJ and drop vertical jumps. Columns represent MAE, ICC and MDC, respectively. Rows correspond to different joints, with sub-rows representing different test types. Subplots display how MAE, ICC and MDC values change over five trials for each movement, with higher ICC values indicating greater consistency and lower MAE and MDC values suggesting improved accuracy and sensitivity.

Azure Kinect Camera: Session 1 vs. Session 2

The study found that direct measurements from the Azure depth camera provided high between-day reliability for measuring lower limb joint angles across various movement tasks. This method demonstrated ICCs ranging from 0.80 to 0.93, indicating strong reliability.

MAE for joint angles was approximately 4.8° and MDC ranged widely but showed improvements as the task became less dynamic. Azure Kinect showed a high degree of repeatability in walking and running tasks, with ICCs close to those obtained from the Vicon system. Although there were differences in specific joint angle measurements, the Azure Kinect method provided reliable data suitable for tracking changes in biomechanics over time.

For the secondary objective, the study determined that performing two to four trials was sufficient to achieve reliable measurements for most tasks. This efficiency in data collection is crucial for practical application, making the Azure Kinect method a viable and reliable option for biomechanical assessments in the real world.

The chart shows how reliability improves with an increasing number of trials for each joint angle and movement type, where higher ICC values indicate greater consistency. It compares the consistency (ICC) of Azure Kinect and filtering (top row) and Azure Kinect (bottom row) across five trials for tasks such as walking, running, single-leg squats, single-leg CMJ, bilateral CMJ and drop vertical jumps. The movements analyzed within each task include hip flexion-extension, hip adduction-abduction, knee flexion-extension and ankle dorsiflexion-plantarflexion.

VALD’s Solution

Azure Kinect depth camera, leveraged by HumanTrak, demonstrated robust reliability for clinical and athletic assessments. These findings support using VALD's system in various settings, offering an efficient, reliable and non-invasive solution for capturing precise biomechanical data.

The Azure Kinect’s validated reliability and minimal trial requirements support HumanTrak’s practicality and effectiveness as a 3D motion capture for the real world. HumanTrak is an ideal tool for both clinical applications and athletic performance monitoring, providing reliable data that professionals can trust.