Li, R and St George, RJ and Wang, X and Lawler, K and Hill, E and Garg, S and Williams, S and Relton, S and Hogg, D and Bai, Q and Alty, J, Moving towards intelligent telemedicine: Computer vision measurement of human movement, Computers in Biology and Medicine pp. 105776. ISSN 0010-4825 (In Press) [Refereed Article]
Background The use of telemedicine consultations in healthcare systems is rapidly increasing around the world, accelerated by the global COVID-19 pandemic. Digital cameras integrated in laptops, as well as smartphones and tablet computers, have provided an accessible method for patients to consult with clinicians. However, telemedicine consultations remain severely limited compared to standard faceto-face consultations, because clinicians cannot accurately examine patients remotely. This limitation, combined with the swift uptake of telemedicine globally, results in significant risks to patient safety. Thus, there is a growing and urgent need to extend the capabilities of technologies so that telemedicine consultations can better meet the needs of patients and clinicians. In all areas of healthcare, clinicians assess human movement throughout the life course of their patients as a fundamental part of clinical assessment. Currently, there are no methods to do this objectively via telemedicine.
Objectives Laptops are one of the most common devices used by patients for telemedicine consultations, but it is unclear whether videos from laptop cameras (with relatively low frame rate) can be used to accurately measure human movements at a range of speeds. The objectives of this study were to determine the validity and reliability of deep learning computer vision methods applied to video collected via a laptop camera to measure finger tapping, a well validated test of human movement.
Method Sixteen healthy adults (9 female, mean age 34.5 years; range 24-52) completed finger-tapping tests internally-paced ‘as big and fast as possible’ and externally-paced by an auditory metronome at frequencies of 0.5Hz, 1Hz, 2Hz and 3Hz. Hand movements were recorded simultaneously by a standard laptop camera at 30 frames per second (FPS) and by Optotrak, a high-speed 3D motion analysis system sampling at 250 FPS. Three DeepLabCut (deep learning-based artificial neural network) architectures were applied to the laptop video to track thumb-tip and index fingertip position and the extracted movement features of each method were compared to the ground truth Optotrak motion tracking.
Results The computer vision methods showed excellent validity and reliability compared to the Optotrak between 0.5Hz and 4Hz tapping frequency. Over 97% (552/538) of the computer vision measures were within +/-0.5Hz of the Optotrak measures. At higher tapping frequencies, there was a progressive decline in accuracy, attributed to motion blur associated with the laptop camera’s low FPS. This study shows that deep learning computer vision methods using laptop cameras hold strong potential for providing objective measurements of human movements during telemedicine consultations. This technology could substantially augment remote clinical assessments in healthcare systems but further developments will be required to accurately measure the fastest movements.
|Item Type:||Refereed Article|
|Keywords:||telemedicine, DeepLabCut, finger tapping, motor control, computer vision|
|Research Division:||Biomedical and Clinical Sciences|
|Research Field:||Neurology and neuromuscular diseases|
|Objective Division:||Information and Communication Services|
|Objective Group:||Information systems, technologies and services|
|Objective Field:||Artificial intelligence|
|UTAS Author:||Li, R (Mr Renjie Li)|
|UTAS Author:||St George, RJ (Dr Rebecca St George)|
|UTAS Author:||Wang, X (Miss Xinyi Wang)|
|UTAS Author:||Lawler, K (Dr Katherine Lawler)|
|UTAS Author:||Hill, E (Dr Edward Hill)|
|UTAS Author:||Garg, S (Dr Saurabh Garg)|
|UTAS Author:||Bai, Q (Dr Quan Bai)|
|UTAS Author:||Alty, J (Associate Professor Jane Alty)|
|Year Published:||In Press|
|Deposited By:||Wicking Dementia Research and Education Centre|
|Downloads:||4 View Download Statistics|
Repository Staff Only: item control page