Developing assessment items to measure tertiary students' reasoning about explained and unexplained variability

As one of the principle forms of fundamental statistical thinking, consideration of variation impacts on all aspects of statistics. There has been extensive research about students' reasoning about variation and numerous assessment items have been developed to assess students' reasoning about many aspects of variability in data. However, there has been little research focusing on helping tertiary students model variability as a combination of explained and unexplained variability, and few items have been developed to assess this construct. An instrument is needed to assess this reasoning, before and after learning experiences, so that the change in level of reasoning can be measured. This can then inform curriculum development and the design of learning experiences. This paper presents a measurement instrument that was developed to assess tertiary students' intuitive reasoning about variation, focusing on explained and unexplained variability. The theoretical framework that provided the structure for the development of assessment items is discussed, followed by presentation of the results from a pilot study designed to test newly-developed items, and lastly the inclusion of previously-developed items is justified. Analysis of the responses led to clarification of those items that were misinterpreted by students or did not measure the relevant construct. The refined assessment items will be incorporated with previously-developed items in a measurement instrument to be administered to tertiary students enrolled in an introductory service statistics course. The direction for future research is also presented.