Background: Online health education and other electronic health improvement strategies are developing rapidly, highlighting the growing need for valid scales to assess health literacy (HL). One comprehensive HL scale is the Health Literacy Questionnaire (HLQ), but little is known about its measurement properties in online health education cohorts.

Objective: The purpose of this study was to determine if the multidimensional HLQ is an appropriate tool to measure HL in a cohort of Understanding Multiple Sclerosis (MS) online course enrollees.

Methods: Participants who enrolled in the first two open enrollments of the Understanding MS online course completed the HLQ (N = 1,182) in an online survey prior to beginning course materials. We used Rasch analysis to assess the measurement properties of the HLQ.

Key results: The nine Domains of the HLQ each had ordered category function and a good fit with the Rasch model. Each domain was one-dimensional and exhibited good internal consistency and reliability. None of the 44 individual items of the HLQ demonstrated item bias or local dependency. However, while the overall fit was good, few measurement gaps were identified in this cohort for participants in each of the nine Domains, meaning that the HLQ may have low measurement precision in some participants.

Conclusion: Our analysis of the HLQ indicated acceptable measurement properties in a cohort of Understanding MS online course enrollees. Although reliable information on nine separate constructs of HL was obtainable in the current study indicating that the HLQ can be used in similar cohorts, its limitations must be also considered.