(Caution: I'm playing around with Scalar's settings for this post, just to see what different settings do.)


This image reminded me of my week's reading challenge assignment this week (New Learning Chapter 4), especially as it references the as yet "undiscovered country" of the New Learning paradigm and the classrooms practicing it. Our readings this week seemed to revolve around that future paradigm in what is hoped for, but seemed to especially hit hard on the conventions of the other two paradigms: didactic and (less so) authentic.

“NCTE Position Statement on Machine Scoring.” NCTE Task Force on Writing Assessment. 15 Apr. 2013.

This NCTE statement outlines their stance on the increasing call for / use of machine scoring of student writing, as well as the concerns that such assessment practices diminish or fail to validate that “writing is a highly complex ability developed over years of practice.” Essentially, the machine-based assessment trend promoted by concerns other than teachers (“various consortia, private corporations, and testing agencies”) threatens the value of human feedback as a key part of the assessment practices that are such integral parts of writing process instruction.

The statement presents a series of concerns that point out what is lost when machine scoring is institutionalized. In answer to this, it ends with a statement offering a list of “research-based guidelines for effective teaching and assessment of writing” – listing the standards we’ve already reviewed. The authors argue that such an emphasis on assessment would promote student engagement in literacy, reflective practices, as well demonstrate how complex literacy is.  They further call (like one of our earlier readings) for more involvement by teaching professionals in the development of evaluation / assessment standards and practices. Their #1 suggestion is portfolio assessment, which they point out would “allow students to learn as a result of engaging in the assessment process.” In other words, increased student agency (authentic learning). The annotated bibliography of current research, dating from 1995 to 2012, largely indicates agreement within the field on this point.

Purdy, James P. “Anxiety and the Archive: Understanding Plagiarism Detection Services as Digital Archives.” Computers and Composition 26 (2009): 65-77.

Coming from an institution that uses Turnitin for all writing courses, this duet of articles came at an interesting time (I’m deep into grading 80 student papers at the moment, all of which have been submitted to Turnitin). Purdy’s approach is, I think, one I’ve not seen before. Other articles object to Turnitin on largely ethical and pedagogical issues. Purdy examines Turnitin (and other similar products) from the perspective of the system’s archival function and how we must approach that from a rhetorical angle.

One of his main points is that the regulatory / “gatekeeping” (67) powers held by the Turnitin company in setting up this archive constitutes a value-laden evaluation system (i.e., it’s not really “just” an archive). In doing so, he points to some of the scholarship in our field that emphasizes the many ethical and pedagogical issues with such a system. I appreciated his approach, as this provides an alternative angle to this still-debated assessment / institutional quality control mechanism. His criticism touches on the element of “centralized proprietary control” that serves an “evaluative role … with which to determine the suitability of texts” (67). Such value practices reminded me of our earlier readings from New Learning (specifically, the didactic paradigm) still maintaining a foothold in the classroom practices of composition teachers. These observations of value also reminded me of the institutional and faculty assessment texts, in which value (68) is determined by paradigms and practitioners that seem to distance the act of teaching as the ultimate concern.

Of most interest to me is his discussion of the rhetoric of the Turnitin site, its functions, and its contracts. He points out the language used as “evaluative” rather than “pedagogical,” creating a sense of the didactic – of knowledge and authority seated in the institution (and the marketing), not the student’s learning (68). These observations are echoed in the NCTE objections to machine scoring as well, especially in the potential for error in a system that frames student writers in terms of right/wrong, honest/dishonest, correct/incorrect (69).  One of his final objections has to do with the purpose for the program (reminding me of Selfe and Selfe’s comments in our Week 3 readings on the commodification of knowledge) – emphasizing “efficiency” over pedagogy (70).

(800 level) Glassick, James E., Mary Taylor Huber, and Gene I. Maeroff. Scholarship Assessed: Evaluation of the Professoriate. San Francisco: Jossey-Bass, 1997.

These final chapters continue with the suggestions for revolutionizing the assessment of faculty, which for chapter 3 focuses on the use of documentation. The authors point out that even here, the “types and sources of materials” selected remain a point of contention (37).  They observe the value of “multiple data sources” as evidence, particularly favoring the portfolio system (39).

They create a list of potentially useful questions which might be used to evaluate such documentation: (1) statement of responsibility, (2) biography (like teaching reviews), (3) samples (like syllabi and assignments), (4) student reviews, (5) reviews by colleagues, and (6) reflective statements. Combined, they argue, such documents create a richer sense of the scholarship involved.

In chapter 4, the authors discuss the impact played by “trusting the process” (50). All parties, they assert, would benefit from a “commitment … to a conscientious application of the standards” as “key to the process” of assessment (50). This process emphasizes again the same six factors laid out in earlier chapters for setting standards of quality work (clear goals, etc.). But they observe that there is a problem of “mixed messages” of expectations – between institutions and practitioners – which inevitably lead to a lack of trust in the process (51). Part of the solution is to see this process as “a community project,” one in which mentoring plays an important role (54).

Finally, in chapter 5, the authors refer to the “qualities of a scholar”: integrity, perseverance, and courage (64-66). They assert that assessment must “begin with ideals,” not formula (67). This chapter especially brought to mind the non-neutrality of technologies – even when those technologies are methods of assessment.

This page is a tag of:

Join this page's discussion (1 comment)