eCite Digital Repository

Moving towards intelligent telemedicine: Computer vision measurement of human movement


Li, R and St George, RJ and Wang, X and Lawler, K and Hill, E and Garg, S and Williams, S and Relton, S and Hogg, D and Bai, Q and Alty, J, Moving towards intelligent telemedicine: Computer vision measurement of human movement, Computers in Biology and Medicine pp. 105776. ISSN 0010-4825 (In Press) [Refereed Article]

PDF (In press version)
Pending copyright assessment - Request a copy

DOI: doi:10.1016/j.compbiomed.2022.105776


Background The use of telemedicine consultations in healthcare systems is rapidly increasing around the world, accelerated by the global COVID-19 pandemic. Digital cameras integrated in laptops, as well as smartphones and tablet computers, have provided an accessible method for patients to consult with clinicians. However, telemedicine consultations remain severely limited compared to standard faceto-face consultations, because clinicians cannot accurately examine patients remotely. This limitation, combined with the swift uptake of telemedicine globally, results in significant risks to patient safety. Thus, there is a growing and urgent need to extend the capabilities of technologies so that telemedicine consultations can better meet the needs of patients and clinicians. In all areas of healthcare, clinicians assess human movement throughout the life course of their patients as a fundamental part of clinical assessment. Currently, there are no methods to do this objectively via telemedicine.

Objectives Laptops are one of the most common devices used by patients for telemedicine consultations, but it is unclear whether videos from laptop cameras (with relatively low frame rate) can be used to accurately measure human movements at a range of speeds. The objectives of this study were to determine the validity and reliability of deep learning computer vision methods applied to video collected via a laptop camera to measure finger tapping, a well validated test of human movement.

Method Sixteen healthy adults (9 female, mean age 34.5 years; range 24-52) completed finger-tapping tests internally-paced ‘as big and fast as possible’ and externally-paced by an auditory metronome at frequencies of 0.5Hz, 1Hz, 2Hz and 3Hz. Hand movements were recorded simultaneously by a standard laptop camera at 30 frames per second (FPS) and by Optotrak, a high-speed 3D motion analysis system sampling at 250 FPS. Three DeepLabCut (deep learning-based artificial neural network) architectures were applied to the laptop video to track thumb-tip and index fingertip position and the extracted movement features of each method were compared to the ground truth Optotrak motion tracking.

Results The computer vision methods showed excellent validity and reliability compared to the Optotrak between 0.5Hz and 4Hz tapping frequency. Over 97% (552/538) of the computer vision measures were within +/-0.5Hz of the Optotrak measures. At higher tapping frequencies, there was a progressive decline in accuracy, attributed to motion blur associated with the laptop camera’s low FPS. This study shows that deep learning computer vision methods using laptop cameras hold strong potential for providing objective measurements of human movements during telemedicine consultations. This technology could substantially augment remote clinical assessments in healthcare systems but further developments will be required to accurately measure the fastest movements.

Item Details

Item Type:Refereed Article
Keywords:telemedicine, DeepLabCut, finger tapping, motor control, computer vision
Research Division:Biomedical and Clinical Sciences
Research Group:Neurosciences
Research Field:Neurology and neuromuscular diseases
Objective Division:Information and Communication Services
Objective Group:Information systems, technologies and services
Objective Field:Artificial intelligence
UTAS Author:Li, R (Mr Renjie Li)
UTAS Author:St George, RJ (Dr Rebecca St George)
UTAS Author:Wang, X (Miss Xinyi Wang)
UTAS Author:Lawler, K (Dr Katherine Lawler)
UTAS Author:Hill, E (Dr Edward Hill)
UTAS Author:Garg, S (Dr Saurabh Garg)
UTAS Author:Bai, Q (Dr Quan Bai)
UTAS Author:Alty, J (Associate Professor Jane Alty)
ID Code:150619
Year Published:In Press
Deposited By:Wicking Dementia Research and Education Centre
Deposited On:2022-06-22
Last Modified:2022-06-26
Downloads:4 View Download Statistics

Repository Staff Only: item control page