eCite Digital Repository

Human behavior analysis by means of multimodal context mining


Banos, O and Villalonga, C and Bang, J and Hur, T and Kang, D and Park, S and Huynh-The, T and Le-Ba, V and Amin, MB and Razzaq, MA and Khan, WA and Hong, CS and Lee, S, Human behavior analysis by means of multimodal context mining, Sensors, 16, (8) Article 1264. ISSN 1424-8220 (2016) [Refereed Article]

PDF (Published version)

Copyright Statement

c 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (

DOI: doi:10.3390/s16081264


There is sufficient evidence proving the impact that negative lifestyle choices have on peopleís health and wellness. Changing unhealthy behaviours requires raising peopleís self-awareness and also providing healthcare experts with a thorough and continuous description of the userís conduct. Several monitoring techniques have been proposed in the past to track usersí behaviour; however, these approaches are either subjective and prone to misreporting, such as questionnaires, or only focus on a specific component of context, such as activity counters. This work presents an innovative multimodal context mining framework to inspect and infer human behaviour in a more holistic fashion. The proposed approach extends beyond the state-of-the-art, since it not only explores a sole type of context, but also combines diverse levels of context in an integral manner. Namely, low-level contexts, including activities, emotions and locations, are identified from heterogeneous sensory data through machine learning techniques. Low-level contexts are combined using ontological mechanisms to derive a more abstract representation of the userís context, here referred to as high-level context. An initial implementation of the proposed framework supporting real-time context identification is also presented. The developed system is evaluated for various realistic scenarios making use of a novel multimodal context open dataset and data on-the-go, demonstrating prominent context-aware capabilities at both low and high levels.

Item Details

Item Type:Refereed Article
Keywords:human behaviour, context awareness, activity recognition, location tracking, emotion identification, machine learning, ontologies
Research Division:Information and Computing Sciences
Research Group:Artificial intelligence
Research Field:Knowledge representation and reasoning
Objective Division:Information and Communication Services
Objective Group:Information systems, technologies and services
Objective Field:Human-computer interaction
UTAS Author:Amin, MB (Dr Muhammad Bilal Amin)
ID Code:143603
Year Published:2016
Web of Science® Times Cited:21
Deposited By:Information and Communication Technology
Deposited On:2021-03-25
Last Modified:2021-05-26
Downloads:14 View Download Statistics

Repository Staff Only: item control page