Research on Reliability and Validity of Mobile Networks-Based Automated Writing Evaluation

Research on Reliability and Validity of Mobile Networks-Based Automated Writing Evaluation

Fei Lang, Siyan Li, Siwen Zhang
DOI: 10.4018/IJMCMC.2019010102
OnDemand:
(Individual Articles)
Available
$37.50
No Current Special Offers
TOTAL SAVINGS: $37.50

Abstract

Pigaiwang, an automated writing evaluation (AWE) online platform, has been increasingly applied into English writing scoring in China. This article conducted research to empirically assess the reliability and validity of Pigaiwang under mobile networks. It introduces the background and previous works for AWE. Then, an experiment was implemented to evaluate the reliability and validity of Pigaiwang automated evaluation by comparing with the results of human raters and corpus-based assessment. The results show that Pigaiwang alone is not an adequate aid for English writing evaluation. It suggests that combining Pigaiwang, human raters, and corpus might be a beneficial trial for improving English writing mobile education.
Article Preview
Top

Introduction

As electronic technology and mobile networks are increasingly developing, mobile learning, a new way of learning, has attracted more attentions from experts and educators all around the world. Mobile learning (M-learning) refers to the way of applying mobile devices into learning through social communications and under various backgrounds (Crompton, 2013). M-learning is a form of distance education, which enables M-learners to start their learning flexibly at anytime and anywhere aided by mobile devices and related educational technology. Its portability, immediacy, interactivity, flexibility, and personalization will make it possible to bring future learning into the era of mobile learning.

M-learning “features learners engaged in educational activities, using technology as a mediating tool for learning via mobile devices accessing data and communicating with others through wireless technology.” (Wu et al., 2012, p. 818) It lays emphasis on the flexibility of learning locations and time and the mobility of virtual educational environment. M-learners have access to M-learning more easily than traditional distance education, and moreover its features result in the appearance of learning system with quick feedback. In other words, M-learners take advantage of this flexible way of learning. Using mobile devices like mobile phones and tablets offered by M-learning technology, M-learners have increasing opportunities of personalized learning without being fixed in physical classrooms or labs (Kukulska-Hulme & Shield, 2008). Additionally, with the developing technology of mobile phone, Wi-Fi and social networking applications, M-learning even brings about great influence on the decreasing usage of textbooks and notebooks (Wikipedia, n.d.).

Evaluation is an important part of college English writing teaching. A comprehensive, objective, and accurate evaluation system is crucial to achieving the goal of mobile learning and future development. This study aims to explore the combination of mobile networks and AWE, and then promote the development of teaching evaluation, to provide a brand-new attempt to promote mobile education and college English writing teaching evaluation.

AWE (automated writing evaluation) is defined in Wikipedia as a network aided teaching system to evaluate and score compositions. It has become more and more completed since it came out in 1966. PEG (Project Essay Grade), IEA (Intelligent Essay Assessor) and E-rater are the most representative systems abroad at present. Especially E-rater, developed by ETS (Educational Testing Service), has been officially used in large-scale language tests like GMAT.

Contrasted to M-learning, AWE, however, is neither a brand-new concept, nor a fresh form of technology, which can be traced the origins since the work 1960s. It was firstly introduced to aid human raters to evaluate students’ essays, which aimed to remove the heavy burden of grading essays and save time of correcting (Warschauer & Ware, 2006). Original AWE systems, known as Project Essay Grade, was developed by a US universities network and designed to evaluate writing levels by analyzing linguistics features (Page, 2003). Since then, with the development of computer technology, AWE systems have been promoted (Chen & Cheng, 2008).

Although AWE tools have enjoyed a history of over 50 years in western countries, they have only gained popularity in China in recent years. Liang (2011) is a pioneer in construction module researches of AWE in domestic. His research team in developing EEE1.0 (EFL essay evaluator) has contributed valuable data to the actual design and development of China’s own AWE tool. In domestic, His new model has achieved some results, which fits Chinese students better. Some Internet technology companies is developing AWE independently, like Bingo (http://www.pigai.org).

Complete Article List

Search this Journal:
Reset
Volume 15: 1 Issue (2024)
Volume 14: 1 Issue (2023)
Volume 13: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 12: 4 Issues (2021)
Volume 11: 4 Issues (2020)
Volume 10: 4 Issues (2019)
Volume 9: 4 Issues (2018)
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing