Comparing the predictive validity of behavioral codings and behavioral ratings in a working-dog breeding program

Most working-dog breeding programs have a substantial interest in using behavioral assessments of their young dogs to predict their subsequent success. Different methods of measuring behavior may capture different aspects of behavior yet working-dog programs typically use only a single measurement method. Thus, the primary aim of this study was to test whether two different measurement methods (ratings or codings) would differ in their predictive validity with respect to working-dog selection outcomes. Rating methods require observers to intuitively aggregate their observations into a single rating and in doing so may reduce error variance in measurement, resulting in improved validity. Coding methods on the other hand do not demand so much judgment on the part of the observer so may be less influenced by observer biases. Here we analyzed the two methods with respect to their ability to predict selection for training in a sample of odor-detection dogs bred at the U.S. Transportation Security Administration Canine Breeding and Development Center. Behaviors observed in two standardized tests (search & retrieve and environment) at four different time points across the first year of life were measured using nine ratings and 23 codings. Data reduction techniques identified two underlying dimensions in ratings (environmental stability and hunt drive) and nine in codings (confidence, anxiety, exploration, excitability, search performance, dominant possession, independent possession, energy management, and search aptitude). There were no differences in predictive validity between the two methods; both ratings and codings correctly classified a high percentage of dogs that were/were not selected for training at 12 months of age (84.6–88.5%). In the search & retrieve test, codings and ratings appeared to be measuring the same construct. In the environment test the only significant coding predictor of training selection (confidence) was strongly related to the single rating predictor (environmental stability). Rating methods tended to capture behavior that was more consistent, while coding methods tended to capture behavior that was more situation-specific. Our mixed-models approach also allowed us to discriminate between average behavior (between-individual variation) and behavioral change through time (within-individual variation); such findings emphasize different aspects of development that may need to be monitored during rearing. Our results suggest that, in some cases, the use of ratings versus codings may be inconsequential from the standpoint of predicting which dogs get selected for training. Virtually all research on animal behavior assesses behavior via coding or rating methods; further work is needed to verify these results.