eCite Digital Repository

Moving towards intelligent telemedicine: Computer vision measurement of human movement


Li, R and St George, RJ and Wang, X and Lawler, K and Hill, E and Garg, S and Williams, S and Relton, S and Hogg, D and Bai, Q and Alty, J, Moving towards intelligent telemedicine: Computer vision measurement of human movement, Computers in Biology and Medicine, 147 Article 105776. ISSN 0010-4825 (2022) [Refereed Article]

Copyright Statement

Copyright 2022 Elsevier Ltd

DOI: doi:10.1016/j.compbiomed.2022.105776


Background: Telemedicine video consultations are rapidly increasing globally, accelerated by the COVID-19 pandemic. This presents opportunities to use computer vision technologies to augment clinician visual judgement because video cameras are so ubiquitous in personal devices and new techniques, such as DeepLabCut (DLC) can precisely measure human movement from smartphone videos. However, the accuracy of DLC to track human movements in videos obtained from laptop cameras, which have a much lower FPS, has never been investigated; this is a critical gap because patients use laptops for most telemedicine consultations.

Objectives: To determine the validity and reliability of DLC applied to laptop videos to measure finger tapping, a validated test of human movement.

Method: Sixteen adults completed finger-tapping tests at 0.5 Hz, 1 Hz, 2 Hz, 3 Hz and at maximal speed. Hand movements were recorded simultaneously by a laptop camera at 30 frames per second (FPS) and by Optotrak, a 3D motion analysis system at 250 FPS. Eight DLC neural network architectures (ResNet50, ResNet101, ResNet152, MobileNetV1, MobileNetV2, EfficientNetB0, EfficientNetB3, EfficientNetB6) were applied to the laptop video and extracted movement features were compared to the ground truth Optotrak motion tracking.

Results: Over 96% (529/552) of DLC measures were within 0.5 Hz of the Optotrak measures. At tapping frequencies 4 Hz, there was progressive decline in accuracy, attributed to motion blur associated with the laptop camera’s low FPS. Computer vision methods hold potential for moving us towards intelligent telemedicine by providing human movement analysis during consultations. However, further developments are required to accurately measure the fastest movements.

Item Details

Item Type:Refereed Article
Keywords:telemedicine, DeepLabCut, finger tapping, motor control, computer vision
Research Division:Biomedical and Clinical Sciences
Research Group:Neurosciences
Research Field:Neurology and neuromuscular diseases
Objective Division:Information and Communication Services
Objective Group:Information systems, technologies and services
Objective Field:Artificial intelligence
UTAS Author:Li, R (Mr Renjie Li)
UTAS Author:St George, RJ (Dr Rebecca St George)
UTAS Author:Wang, X (Miss Xinyi Wang)
UTAS Author:Lawler, K (Dr Katherine Lawler)
UTAS Author:Hill, E (Dr Eddy Roccati)
UTAS Author:Garg, S (Dr Saurabh Garg)
UTAS Author:Bai, Q (Dr Quan Bai)
UTAS Author:Alty, J (Associate Professor Jane Alty)
ID Code:150619
Year Published:2022
Web of Science® Times Cited:1
Deposited By:Wicking Dementia Research and Education Centre
Deposited On:2022-06-22
Last Modified:2022-12-09
Downloads:4 View Download Statistics

Repository Staff Only: item control page