The
Action Research Laboratory
Page Thirteen
Back
to Table of Contents
Assessing the Assessments – What really works?
An Action Research Report
Ann Nick
Summation of Student Responses to Survey: the Primary
Sources Speak!
Findings
Implications for Practice: "To Do" Notes to
Myself
Which leads to Further Questions
Table One: Talley of Student Responses to Survey given
12/04/00
Table Two: Data Analysis for "Further Question 1
Have you
ever wondered if all that time spent grading student writing is worth it? As a
high school history teacher, I am passionately committed to assisting students
to become powerful analytic writers. Constructing a thesis and persuasively
supporting it with historical and logical evidence does more than force
students to wrestle with essential questions and demonstrate mastery of
content. In the long run, practice in
effective use of language shapes thought processes, shapes how one defines and
reacts to issues and strengthens the ability to shape the views and actions of
others. So, like many of you, I spend countless (and sometimes overwhelming)
hours assessing student papers, giving what I hope is insightful, clear and
constructive coaching, encouragement and fair evaluation. While struggling
through the piles of essays, some questions kept recurring. Why do some
students begin and end the year writing shallowly while others develop their
skills? Why is most communication about writing limited to
“student-to-teacher-to student”? How
can students best learn from each other as well? How can students get an
objective view of their own work’s strengths and weaknesses? Honestly now, is it really worth the time to
carefully construct rubrics? Do students actually read and apply my feedback to
future work? In short, I wanted to know that the time I spend assessing
student work was being used as effectively as possible. For that, I needed to
ask how various modes of assessment impact student learning.
This reality check demanded that I find out what the students themselves had to say about the assessment strategies already in use. These assessment tools include checks in the margins of essays to note appropriate, effective use of facts and logic, specific comments and editor’s marks that question and coach, an extensive rubric for analytic writing and particular standards for evaluation for specific writing assignments. I wanted to ask what works for them, and why, and to solicit their input and suggestions. With the assistance of learning team colleagues experienced in Action Research, a survey was written to elicit student opinions and experiences. Selecting questions was a process in itself. I needed to be honest in asking only what I really wanted to know and addressing areas on which I would be willing to take action. It was key to keep the survey short enough to be doable and yet complete enough to be useful. Questions were phrased consistently and neutrally in order to avoid leading the responses. It was administered to twenty-eight AP Modern European History students, including both sophomores and seniors. Their responses were tallied and comments compiled. General findings were discussed in class, and students confirmed my understandings of their responses. (Survey questions and data follow this reflection in Table 1.) One striking finding was that, while students are mostly skeptical yet somewhat open to peer assessment, most are quite resistant to self-assessment. Why? Could this gap be due to differences in experience with peer and self-assessment? The data was then analyzed to see if there was any correlation between practice with and perceived value of peer and self-assessments. (Table 2).
Summation of Student Responses to Survey: the Primary Sources Speak!
The survey
asked for reactions to this teacher’s specific assessment techniques.
Students find that the checks in the margins of their papers identify what they are doing right and justify the grade. While some students feel the checks would be even more helpful if it were always clear to what a particular check referred, others sort that out for themselves by looking for the content in the indicated line of their own prose. Checks make some students feel good about their work and have more confidence in the teacher. Teacher comments help identify strengths and areas to improve; the more specific, frequent, and legible the better. Students do seem to read them! This is the most useful form of feedback for most students. Standards for Evaluation provide a checklist as students write, clarify expectations, help students to focus, and justify scoring. On the other hand, reactions to Rubrics are mixed. Some like the clear standards but many find them too rigid and/or too thorough. Some students find rubrics too vague to be of help in improving writing. Students may feel they are following the rubric well but then teacher circles a “lower” box, and students are frustrated as to how to match their own work to the rubric and/or how to improve.
The survey asked what students do with the feedback and if they learned from others’ work.
Many students find that reviewing feedback on prior work can often identify bad habits to break as well as good skills to build on. Not all assignments are similar in nature, however; so prior assessment doesn’t always transfer to new challenges.
Models of other student’s work can help some to see how an assignment can be well done, and expose students to fine writing. On the other hand, many expressed that good examples can intimidate students who compare their own work unfavorably to the model. There is skepticism that one student’s style can help another improve. Models of work in which the teacher has annotated strengths and weaknesses are seen as most helpful.
The survey addressed both peer and self-assessment.
A few students see peer
assessment as constructive, for peers are “in the same place”. Many more
however, lack confidence in their peers’ ability to judge accurately or to be
honest. Students are afraid to hurt others’ feelings. Teacher feedback is
deemed more valuable. A few just don’t take the task or the feedback seriously.
What experience there is in peer assessment comes in English &
History classes or as a Writing Lab Tutor.
In some cases students feel that peer assessment affects grades, which makes some uncomfortable that personal issues could impinge on fairness. Thus some students are more comfortable when peer assessment is used solely for suggestions, not for grading. Several comments indicate a desire to avoid this responsibility and a lack of confidence in knowing how to do it well.
Comments on self-assessment were widely mixed. Some find it helps a teacher understand what students are thinking, and that it helps students be more objective. Most dislike self-assessment. Students expressed fears that dishonesty, bias or grade strategy can affect their scores. “As a teenager, I think it’s safe to say my perception of myself and my work are skewed.” “People generally feel what they wrote is right; if not they wouldn’t have written it.” Many of these comments equate self-assessment with the resulting grade. On the whole, students submit work that they feel fulfills the standards well; if they knew how it could be better, they would have made it so. Students have widely varying experiences with self-assessment at HPHS. It seems to depend on which courses and teachers a student encounters.
The survey solicited suggestions for other assessment
techniques and any general comments.
Some additional suggestions for assessment techniques include: personal conferences, a whole class oral assessment, a simple generic checklist, a general assessment of trends in a student’s writing, requiring student written reactions to feedback before revealing grades and rewarding just doing the reading. As supplemental to my feedback, it was suggested that we use last year’s students and other teachers for assessment. There were also requests for even more written comments, more self-assessment for small assignments, appreciation for current practices and an acknowledgement that there’s not enough time to do everything.
The bottom line of these messages from students is that the more personal, specific and complete the assessment tool, the more helpful it is to them. Students perceive teacher feedback as the most valuable, the more individualized the better. They find Standards for Evaluation very useful, and Rubrics are also useful but have inherent problems. Students are somewhat uncomfortable with peer assessment as a learning tool. They often do not feel competent to do accurate assessment, nor do they trust their peers’ judgment as much as they trust the teacher’s judgment. Students are even more frustrated with self-assessment. The most common refrain is that if they knew what was wrong, they would have fixed it. Here, equating assessment with grades is also unsettling to students. Self-assessment appears to be practiced throughout our school, but very inconsistently with a range of formality and effectiveness. There seems to be a weak indication of a correlation between depth of experience and perceived usefulness with peer assessment. There is no clear correlation between experience and perceived usefulness of self-assessment. There is a reassuring amount of feedback that the most heavily used forms of assessment are very effective.
Implications for Practice: “To Do”Notes to Myself
The implications for practice are both reassuring and
challenging.
I’m glad to know that it is worth the time for the teacher to write as many comments as possible, noting both strengths and weaknesses in writing and analysis. Not everything works, at least not at first. In an attempt to be very specific in both expectations and feedback, I wrote an extensive new rubric for writing assignments. It added so much time to assessment that it was only used once by teacher as a model and once by students. Perhaps it needs more frequent practice to be user-friendly. Frustratingly, I find that using rubrics takes much more time than writing specific comments, as I have to think through the same feedback in order to use the rubric anyway. Students have said that they need comments to understand why rubrics are marked as they are. Is it worth the time to create and use rubrics when students value the individual comments more? One possible approach is to number each section of the rubric, and then annotate students’ work with those numbers. In this way, students might track where each rubric element is accomplished to what degree in their own writing.
Stubbornly, I still believe that the teacher need not be the only source of feedback, so tactics in peer assessment were successfully adjusted to address the interpersonal issues that create mistrust. Peer assessment was made anonymous and guided with even more specific guidelines. Students each read and assessed two pieces of work, so that they received feedback from more than one source. This worked well with a particularly challenging writing task called the Document Based Question, which requires a specific approach to analysis. Students made up code names for themselves, and did a very objective, accurate job assessing each other’s work. It was clear that the more specific the guidelines for peer assessment, the better job the students do in giving each other accurate and constructive feedback. Next year, there should be more practice with peer-assessment on smaller assignments as well as in-depth work.
Finally, in a follow-up class discussion, students insisted that they definitely want to self-assess their class participation (an established and effective practice), but are very reluctant to pursue it for written work. Perhaps they expect that the teacher really does know better because she has much more experience and that it is her job to show the way. To greater or lesser degrees, they would be right. Is it even fair to expect adolescents to keep their own egos and concerns about grades from clouding self-assessment of their own written work when most adults (including this one) are far too attached to our own words? The most productive form of self assessment may be for students to compare pieces of their own writing in similar tasks over time, when they can look at different concrete examples of their own writing in view of specific criteria. Then they could say that this thesis is clearer than that, or that this paragraph uses more evidence than the earlier piece did. Identifying their own strengths based on the teacher’s feedback may provide a stepping-stone to objectivity about their own work.
Which leads to Further Questions:
This Action Research has provided clarity and direction to the assessment process. There is still much to consider. Is formal self-assessment essential for our students, and if so, what would be the steps to learning to objectively apply standards to one’s own work? (i.e. How can students accurately match their own work with the rubric’s descriptive boxes?) Would this involve the same steps as peer-assessment? Why do students seem able to be more objective with others’ work than with their own? Are there maturational or developmental processes at work? The ability to step outside one’s own work to view it objectively requires some confidence that one is personally “OK” and that mistakes are acceptable; that teachers expect growth, not perfection. How can we instill that trust in oneself, that ability to risk constructive criticism and the ability to view one’s own efforts as we do the work of others? Is there a correlation between classroom community and objective, effective peer-assessment? The reality of the calendar taunts me: how can I add my students’ suggestions for other forms of assessments into the already very full school year? Teaching is indeed a learning process. As questions lead to questions, a little practical wisdom is gleaned along the way.
TABLE ONE: Talley of Student Responses to Survey given 12/04/00
STUDENT FEEDBACK on ASSESSMENT TECHNIQUES: What Works for
You?
* Denotes responses that fell in between choices!
1. How helpful to you are teacher’s Ös in
margins of your written work?
a. very helpful b. somewhat helpful c. of little use d. I never read them
17 2* 7 2 0
2. How helpful to you are teacher’s comments on your
written work?
a. very helpful b. somewhat helpful c.
of little use d. I never read
them
20 7 1 0
3. Do you ever look back at prior assignments and/or papers to identify how to improve the ones you are working on?
a. Yes, all the time b. Yes, sometimes c. Rarely d. Not at all
4 13 1* 7 0
4. How effective are Standards for Evaluation with Point
Values (e.g. for formal papers) in helping you to improve your writing?
a. very effective b. somewhat effective c. of little use d. I never read them
12 9 2 0
5. How effective are Rubric Charts with descriptions of
excellent, acceptable, etc., work in helping you to improve your writing?
a. very effective b. somewhat effective c. of little use d. I never read them
7 13 8 0
6. How helpful is it to have models of other students’
work?
a. very helpful b. somewhat helpful c. of little use d. I never read them
11 9 1* 6 1
7. How helpful is peer feedback?
a. very helpful b. somewhat helpful c. of little use d. useless
4 13 2* 8 1
8. How much
experience do you have with peer assessment in general?
a. very experienced b. some experience c. a little experience d. only this class
4 2* 15 1* 6 0
9. How helpful is self-assessment?
a. very helpful b. somewhat helpful c. of little use d. useless
4 1* 8 1* 9 4
10. How much experience do you have with self-assessment
in general?
a. very experienced b. some experience c. a little experience d. only this class
6 1* 11 6 3
no response: 1
TABLE TWO: Data Analysis for “Further Question 1” :
Question: Is there a correlation between perceived usefulness of peer and self-assessment and the depth of student experience with these types of assessment?
Findings:
Key: A=very
experienced
B=somewhat
experienced
C=a
little experience
D= only
this class
PEER ASSESSMENT
Students who marked very helpful also marked depth of experience in following numbers:
A: 2 B: 2 C: 0 D: 0
Students who marked somewhat helpful also marked depth of experience in following numbers:
A: 3 B: 9 C: 3 D: 0
Students who marked of little use also marked depth of experience in following numbers:
A: 1 B: 5 C: 2 D: 0
Students who marked useless also marked depth of experience in following numbers:
A: 0 B: 0 C: 1 D: 0
SELF ASSESSMENT
Students who marked very helpful also marked depth of experience in following numbers:
A: 2 B: 1 C: 2 D: 0
Students who marked somewhat helpful also marked depth of experience in following numbers:
A: 2 B: 5 C: 1 D: 0
Students who marked of little use also marked depth of experience in following numbers:
A: 2 B: 4 C: 2 D: 1
Students who marked useless also marked depth of experience in following numbers:
A: 1 B: 1 C: 1 D: 0
