Searching questions, informal modelling, and massively multiple choice

The Intelligent Book project aims to improve online education by designing materials that can model the subject matter they teach, in the manner of a Reactive Learning Environment. In earlier work, we developed mathematical proof exercises that used an automated theorem prover to model the student's proof. By observing human tutorial sessions, however, we find that this level of formal modelling is not necessary for many proof questions. In this paper, we investigate whether less formal modelling can still provide a useful Reactive Learning Environment. We constructed a system that uses search and informal reasoning about prewritten statements to ask questions for an undergraduate Discrete Mathematics course. When generalised and simplified, we find these search-based questions can also be used as a replacement for multiple choice questions, or to provide "massively multiple choice" questions.