eCite Digital Repository

Proper comparison among methods using a confusion matrix


Salmon, BP and Kleynhans, W and Schwegmann, CP and Olivier, JC, Proper comparison among methods using a confusion matrix, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 26-31 July 2015, Milan, Italy, pp. 3057-3060. ISBN 978-1-4799-7929-5 (2015) [Refereed Conference Paper]

Restricted - Request a copy

Copyright Statement

Copyright unknown

Official URL:

DOI: doi:10.1109/IGARSS.2015.7326461


An important aspect of research in the remote sensing field is to objectively compare different classifiers. This is the foundation of hundreds of research projects and in this paper we will address some raising concerns when evaluating solutions for classification of data sets with skewed class distributions. The quality of assessment is based on the problem specified by the user and the corresponding hypothesis defined. This hypothesis will determine how two or more classifiers are scored to determine which one is better for a particular application. In this paper we present two experiments that illustrate how, if unaware and misunderstood, statistical measurements can be misleading. One experiment is based on a Synthetic Aperture Radar image with a highly skewed class distribution and the second experiment is based on a Landsat image with a minor skewed distribution. From both experiments it can be seen that ill-defining the problem, can lead to false statements and the reporting of statistically invalid conclusions.

Item Details

Item Type:Refereed Conference Paper
Keywords:image classification, probability distribution, remote sensing, satellites, statistics
Research Division:Engineering
Research Group:Geomatic engineering
Research Field:Photogrammetry and remote sensing
Objective Division:Expanding Knowledge
Objective Group:Expanding knowledge
Objective Field:Expanding knowledge in engineering
UTAS Author:Salmon, BP (Dr Brian Salmon)
UTAS Author:Olivier, JC (Professor JC Olivier)
ID Code:103120
Year Published:2015
Deposited By:Engineering
Deposited On:2015-09-22
Last Modified:2016-07-19

Repository Staff Only: item control page