Volume 5, Issue 1: JANUARY 2012
Big Rubrics and Weird Genres: The Futility of Using Generic Assessment Tools Across Diverse Instructional Contexts
(NOTE: The order of authorship is purely alphabetical.)
Interest in "all-purpose" assessment of students’ writing and/or speaking appeals to many teachers and administrators because it seems simple and efficient, offers a single set of standards that can inform pedagogy, and serves as a benchmark for institutional improvement. This essay argues, however, that such generalized standards are unproductive and theoretically misguided. Drawing on situated approaches to the assessment of writing and speaking, as well as many years of collective experience working with faculty, administrators, and students on communication instruction in highly specific curricular contexts, we demonstrate the advantages of shaping assessment around local conditions, including discipline-based genres and contexts, specific and varied communicative goals, and the embeddedness of communication instruction in particular "ways of knowing" within disciplines and subdisciplines. By sharing analyses of unique genres of writing and speaking at our institutions, and the processes that faculty and administrators have used to create assessment protocols for those genres, we support contextually-based approaches to assessment and argue for the abandonment of generic rubrics.
An Annotated Bibliography of Writing Assessment: Machine Scoring and Evaluation of Essay-length Writing
This installment of the JWA annotated bibliography focuses on the phenomenon of machine scoring of whole essays composed by students and others. “Machine scoring” is defined as the rating of extended or essay writing by means of automated, computerized technology. Excluded is scoring of paragraph-sized free responses of the sort that occur in academic course examinations. Also excluded is software that checks only grammar, style, and spelling. Included, however, is software that provides other kinds of evaluative or diagnostic feedback along with a holistic score. While some entries in this bibliography describe, validate, and critique the ways computers “read” texts and generate scores and feedback, other sources critically examine how these results are used. The topic is timely, since the use of machine scoring of essays is rapidly growing in standardized testing, sorting of job and college applicants, admission to college, placement into and exit out of writing courses, content tests in academic courses, and value-added study of learning outcomes.
Response Rethought…Again: Exploring Recorded Comments and the Teacher-Student Bond
Responding to Student Writing as “Conversation”