This chapter explores three broad principles of user-centered design methodologies, including participatory design, iteration, and usability considerations. We discuss characteristics of teachers as an important type of ITS end user, including barriers teachers face as users and their role in educational technology design. To exemplify key points, we draw upon our own experiences in developing an ITS for writing strategies (i.e., the Writing Pal). We conclude by offering a tentative design approach—the Design Implementation Framework (DIF)—that builds upon existing cyclical design methods but with some tailoring to ITS and educational technology contexts.
TopIntroduction
With each new school year, the list of available educational technologies expands dramatically, along with more widespread integration of these technologies into classrooms. Educators at all levels increasingly rely on a variety of technologies to engage students in learning complex material across domains. These technologies vary in form and function, such as assistive technologies (e.g., text-to-speech software and interactive whiteboards) that facilitate or transform teaching, and instructive technologies that directly teach concepts, train skills, scaffold practice, and provide feedback. Across these formats, however, educational technologies are beholden to similar questions and constraints regarding design, usability, and implementation. How can these tools be developed and deployed in ways that consider teachers’ and learners’ true needs in school environments?
One widely used type of educational technology is the intelligent tutoring system (ITS). Briefly, ITSs are computer programs that model expert human tutors by instructing students on a specific topic or skill, providing feedback on responses, and strategically facilitating student performance and understanding through adaptive instruction (Anderson, Boyle & Reiser, 1985; Burns, Luckhardt, Parlett & Redfield, 2014; VanLehn, 2006). One-on-one human tutoring has been shown to be a highly effective mode of instruction; however, it is severely limited by the availability of expert human tutors (Muldner, Lam, & Chi, 2014). ITSs have been developed in response to this problem. These systems rely on artificial intelligence-based computer systems and agents to provide students with personalized educational experiences that model human tutoring.
By various measures, these efforts have been successful. For example, Ma, Adesope, Nesbit, and Liu, (2014) conducted a meta-analysis of effect sizes from 107 ITS studies that incorporated 14,321 participants from K-12 and college populations and spanned eight different content domains. The study found that, across various uses (i.e., the primary means driving instruction, a supplement to instruction, a component of instruction, and an aid to homework), ITSs were associated with superior performance in comparison to large-group instruction (g = 0.42), non-ITS computer-based instruction (g = 0.57) and instruction via textbooks (g = 0.35). Importantly, the meta-analysis revealed no significant differences when comparing ITS use to human tutoring or small-group instruction. This finding provides support for a previous review (VanLehn, 2011), which showed that ITSs provided nearly the same learning gains in STEM topics as one-on-one human tutors. Support for the effectiveness of these systems has been replicated in studies across a variety of domains, such as in K-12 pupils’ mathematical learning (Steenbergen-Hu & Cooper, 2013), and physics and computer literacy (Nye, Graesser & Hu, 2014). These positive findings have spurred the popularity of ITSs in classrooms, as these systems continue to influence the educational landscape.