Presentations

Back

The validity of automated scoring software and its application in various elt contexts

Timothy L Farnsworth

One of the most interesting new developments in language assessment this decade has been the rapid progress of automated scoring software for both second language writing and speech assessments. Technologies such as Pearson's Versant family of assessments and ETS' e-rater software have the potential to make scoring of performance tasks much more practical and reliable. Indeed, these technologies are already beginning to be widely used around the world. But what is this software actually measuring? Are automated scoring procedures consistent with current views on communicative competence? Are the constructs the same or different than the constructs that humans measure when they examine an essay or an oral interview? If the constructs are different, does it matter in the end, as long as the scores are similar? How should teachers respond to these new assessments? In this presentation, research evidence will be reviewed relating to the validity of automated scoring software and its application in various ELT contexts, including exam preparation classes. An argument will be made that this technology, if applied widely, may have wide ranging impacts on how we teach in the future, with unpredictable consequences.