Difference between revisions of "Rubrics"

From English Wiki
Jump to navigation Jump to search
m
 
(13 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
   
 
   
  
Grading a large number of students or assignments can be challenging. One efficient method can be a grading bank or feedback database. While grading, an instructor can save comments that are regularly used for common issues in a text document or spreadsheet, and then the comments can be copied and pasted into feedback forms for other students, and then modified or personalized for the individual student.  
+
Grading a large number of students or assignments can be challenging. One efficient method can be a grading bank or [[feedback database]]. While grading, an instructor can save comments that are regularly used for common issues in a text document or spreadsheet, and then the comments can be copied and pasted into feedback forms for other students, and then modified or personalized for the individual student.  
  
 
Rubrics are a tool for criterion-based assessment, i.e., assessment is based on specified criteria or standards. This contrasts with normative assessment, where performance is ranked according to a statistical norm or standard like a bell curve. A rubric is a scoring guide based on specific criteria or categories for assessing students’ performance on a task or assignment. A rubric is a set of grading criteria for different areas of performance. It is often in table format, and it tells the students exactly what you expect. This can be provided when you give the assignment, and/or when you grade it. Instructors can mark on a paper or electronic form to indicate criteria that students have met well or have not achieved. This allows for more efficient grading for teachers, and clearer evaluations for students.  
 
Rubrics are a tool for criterion-based assessment, i.e., assessment is based on specified criteria or standards. This contrasts with normative assessment, where performance is ranked according to a statistical norm or standard like a bell curve. A rubric is a scoring guide based on specific criteria or categories for assessing students’ performance on a task or assignment. A rubric is a set of grading criteria for different areas of performance. It is often in table format, and it tells the students exactly what you expect. This can be provided when you give the assignment, and/or when you grade it. Instructors can mark on a paper or electronic form to indicate criteria that students have met well or have not achieved. This allows for more efficient grading for teachers, and clearer evaluations for students.  
Line 14: Line 14:
 
Traditional grading methods tend to follow the following approaches.  
 
Traditional grading methods tend to follow the following approaches.  
  
Holistic. Each student is graded holistically, without a clear breakdown of criteria or their importance. For students, this can seem subjective, and the rationale for a grade may seem unclear, or open to interpretation.  
+
;Holistic: Each student is graded holistically, without a clear breakdown of criteria or their importance. For students, this can seem subjective, and the rationale for a grade may seem unclear, or open to interpretation.  
  
Comparative. Students are compared with each other and ranked. It may seem relatively easy to sort assignments into different piles or groups based on holistic quality comparisons. But this can also seem subjective or open to interpretation.  
+
;Comparative: Students are compared with each other and ranked. It may seem relatively easy to sort assignments into different piles or groups based on holistic quality comparisons. But this can also seem subjective or open to interpretation.  
  
Objective (close-ended). Students are graded on responses to simple questions (multiple choice, binary choices like true/false) and passive recognition of contents. While this may be easy to grade, students do not engage with the materials and concepts, and merely demonstrate passive learning. The assignments are thus more about regurgitating knowledge, when more engaging assignments could be given to promote deeper thinking.  
+
;Objective (close-ended): Students are graded on responses to simple questions (multiple choice, binary choices like true/false) and passive recognition of contents. While this may be easy to grade, students do not engage with the materials and concepts, and merely demonstrate passive learning. The assignments are thus more about regurgitating knowledge, when more engaging assignments could be given to promote deeper thinking.
 +
 
 +
;Open-ended tasks: Assignments with no clear answers can be more challenging, and can promote conceptual learning and engagement with the course contents. Thus, open-ended discussion questions, essay questions, written assignments, essays, essay exams, presentations, and discussion activities can be better tools for learning as well as assessment.  
  
Open-ended tasks. Assignments with no clear answers can be more challenging, and can promote conceptual learning and engagement with the course contents. Thus, open-ended discussion questions, essay questions, written assignments, essays, essay exams, presentations, and discussion activities can be better tools for learning as well as assessment.
 
  
 
Grading major projects, essays, presentations, and other major assignments can be difficult, especially for more larger classes. Students expect and deserve fairly detailed feedback and justification for the grades they receive. Feedback should describe the various strengths and weaknesses of a student’s performance – exactly what s/he did well, and where s/he lacks or needs improvement. Feedback is also an opportunity to provide students specific advice on how to improve.  
 
Grading major projects, essays, presentations, and other major assignments can be difficult, especially for more larger classes. Students expect and deserve fairly detailed feedback and justification for the grades they receive. Feedback should describe the various strengths and weaknesses of a student’s performance – exactly what s/he did well, and where s/he lacks or needs improvement. Feedback is also an opportunity to provide students specific advice on how to improve.  
Line 48: Line 49:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-  
 
|-  
| Content-focused
+
| '''Content-focused'''
 
| Defines the desired content <br> Indicates what characteristics / content has more / less importance
 
| Defines the desired content <br> Indicates what characteristics / content has more / less importance
 
|-  
 
|-  
| Clear
+
| '''Clear'''
 
| Communicates expectations to students
 
| Communicates expectations to students
 
|-  
 
|-  
| Practical
+
| '''Practical'''
| Students can understand areas of comprehension & improvement; students can self-assess
+
| Students can understand areas of comprehension & improvement; students can self-assess; teachers can easily grade with the rubric
 
|-  
 
|-  
|| Fair
+
|| '''Fair'''
 
|| Teachers consistently apply valid, reliable assessment criteria
 
|| Teachers consistently apply valid, reliable assessment criteria
  
 
Avoiding bias
 
Avoiding bias
 
|-  
 
|-  
|| Goals
+
|| '''Specific goals'''
 
|| Rubric criteria are based on specific learning goals, not vaguely defined outcomes.  
 
|| Rubric criteria are based on specific learning goals, not vaguely defined outcomes.  
 
|-
 
|-
Line 88: Line 89:
 
* Undermining transparency, reliability, & validity
 
* Undermining transparency, reliability, & validity
 
* Introducing bias  
 
* Introducing bias  
* Undermining, hingering, or not reflecting learning goals  
+
* Undermining, hindering, or not reflecting learning goals  
  
  
Line 98: Line 99:
 
* how students evaluate their own knowledge
 
* how students evaluate their own knowledge
  
 
+
==Types of rubrics==
==Types of rubrices==
 
  
 
=== Simple scalar rubrics ===
 
=== Simple scalar rubrics ===
Line 149: Line 149:
 
=== Holistic rubrics ===
 
=== Holistic rubrics ===
  
Holistic rubrics can be ideal for minor assignments, for assignments where minor errors can be tolerated, and especially for tasks that are open-ended, for example where there is no definite correct answer.  
+
A holistic rubric does not have categories for different aspects of an assignment or for achievement. Instead, it simply describes different achievement levels for a task, for example, the criteria for a perfect score, for a high pass, for an average score, and such. Holistic rubrics can be ideal for minor assignments, for assignments where minor errors can be tolerated, and especially for tasks that are open-ended, for example where there is no definite correct answer.  
  
  
Line 159: Line 159:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-  
 
|-  
|| Points  
+
! Points !!  Criterion !! Specific criteria  
|| Criterion
 
|| Specific criteria  
 
 
|-  
 
|-  
 
|| 2  
 
|| 2  
Line 185: Line 183:
 
|}
 
|}
  
 +
 +
====Just-in-time teaching (pre-class task) ====
 +
A technique known by the awkward moniker of just-in-time teaching (JiTT)<ref>''Just in Time Teaching.'' (2009). Simkins, Scott, and Maier, Mark (Eds.). Sterling, VA: Stylus Publishing. </ref> can be used with holistic or qualitative rubrics like the one above.  A better term would be a pre-class mini-quiz or task. This method is used sometimes in content-area courses, and less commonly in language courses. Students are required to read a textbook chapter before the class lecture on the topic, and then respond to a question via an online form, such as a Google Form or an assignment in a university's course management system. It is usually one conceptual question or a few short questions, focusing on concepts to be discussed in class, so that students are primed and ready for more discussion in class. The students' responses can be open-ended, such as writing a few sentences; or it can be an objective question, followed preferably by a follow-up question in which students explain or justify their answers in one or two sentences. These can be graded as a minor assignment, and since the material has not been explained in lecture, it is better to evaluate the response in a way that gives as much weight to effort as well as accuracy. The above holistic rubric is based on this type of evaluation for pre-class tasks.
 +
 +
 +
See also: ''Just in Time Teaching.'' (2009). Simkins, Scott, and Maier, Mark (Eds.). Sterling, VA: Stylus Publishing.
 +
 +
  
 
==== Participation grades ====
 
==== Participation grades ====
Line 216: Line 222:
  
  
=== Analytical rubrics ===
+
=== Analytic rubrics ===
  
 
Analytic rubrics provide more detailed feedback on different aspects and different categories of evaluation for an assignment. They are thus are more appropriate for more detailed assignments and major assignments.  
 
Analytic rubrics provide more detailed feedback on different aspects and different categories of evaluation for an assignment. They are thus are more appropriate for more detailed assignments and major assignments.  
Line 226: Line 232:
 
;Film presentations:  
 
;Film presentations:  
 
The following rubric is for presentations on film interpretation, from Stevens and Levi (2012).<ref>Stevens, Dannelle D., & Levi, Antonia J. ''Introduction to Rubrics'', 2nd Ed. (2012).  Stylus Publishing</ref>
 
The following rubric is for presentations on film interpretation, from Stevens and Levi (2012).<ref>Stevens, Dannelle D., & Levi, Antonia J. ''Introduction to Rubrics'', 2nd Ed. (2012).  Stylus Publishing</ref>
 
  
 
{| class="wikitable"
 
{| class="wikitable"
Line 264: Line 269:
  
  
;Sociology presentations:  
+
;Content area presentations:  
 
The following rubric is for oral presentations in senior sociology classes, from Stevens and Levi (2012).<ref>Stevens, Dannelle D., & Levi, Antonia J. ''Introduction to Rubrics'', 2nd Ed. (2012).  Stylus Publishing</ref>
 
The following rubric is for oral presentations in senior sociology classes, from Stevens and Levi (2012).<ref>Stevens, Dannelle D., & Levi, Antonia J. ''Introduction to Rubrics'', 2nd Ed. (2012).  Stylus Publishing</ref>
  
Line 362: Line 367:
 
The above rubric is well detailed, and makes it possible to assign a grade by assessing it against various criteria. If a project consistently fits the criteria for ‘good’, then you can give it a certain grade. Or each category could be worth, say, 4 points, and you total up the points for each category. In this case, the maximum is 16 points, which could be scaled up to a score of 100 (e.g., 14/16=88).
 
The above rubric is well detailed, and makes it possible to assign a grade by assessing it against various criteria. If a project consistently fits the criteria for ‘good’, then you can give it a certain grade. Or each category could be worth, say, 4 points, and you total up the points for each category. In this case, the maximum is 16 points, which could be scaled up to a score of 100 (e.g., 14/16=88).
  
However, the criteria for ‘length of presentation’ seem poorly written – no one should be allowed to exceed the allocated time by four to ten minutes; the time limits should be more strictly enforced. These could be better written, e.g., according to how well the presenters are able to cover the materials within the allocated time, how well they give adequate time to each major point or section, whether they finish too early, whether they have to end abruptly because they have run out of time, etc.
+
However, the criteria for ‘length of presentation’ seem unclearly written – no one should be allowed to exceed the allocated time by four to ten minutes; the time limits should be more strictly enforced. These could be better written, e.g., according to how well the presenters are able to cover the materials within the allocated time, how well they give adequate time to each major point or section, whether they finish too early, whether they have to end abruptly because they have run out of time, etc.
  
 
+
===Weighted rubrics===
 
 
====Weighted rubrics====
 
  
 
In grading this way, you might find not want to give as much importance to some criteria as others. In the example above, ‘contents’ would be quite important, ‘speaking skills’ seems important but not as much as contents, but ‘length’ seems less important than the others. We can solve this by assigning different value or weights to different categories. Maybe for your projects, you want to evaluate students according to English ability (style, vocabulary, grammar, intelligibility), but in fairness to students in many fields of study, this should not be a major concern, certainly not as important as contents and coherence. So we might grade the rubric like this, where some categories have more points – more weight or value.  
 
In grading this way, you might find not want to give as much importance to some criteria as others. In the example above, ‘contents’ would be quite important, ‘speaking skills’ seems important but not as much as contents, but ‘length’ seems less important than the others. We can solve this by assigning different value or weights to different categories. Maybe for your projects, you want to evaluate students according to English ability (style, vocabulary, grammar, intelligibility), but in fairness to students in many fields of study, this should not be a major concern, certainly not as important as contents and coherence. So we might grade the rubric like this, where some categories have more points – more weight or value.  
Line 375: Line 378:
 
! Criteria !! Poor – 2!! Satisfactory – 4 !! Good – 6 !! Excellent – 8 !! Score   
 
! Criteria !! Poor – 2!! Satisfactory – 4 !! Good – 6 !! Excellent – 8 !! Score   
 
|-
 
|-
| '''Content'''
+
| '''Content''' || || || || ||
|
 
|
 
|
 
|
 
|
 
 
|-
 
|-
| '''Coherence'''
+
| '''Coherence''' || || || || ||
|
 
|
 
|
 
|
 
|
 
 
|-
 
|-
|  
+
| || '''Poor – 1''' || '''Satisfactory – 2''' || '''Good – 3 ''' |'''Excellent – 4 ''' ||  '''Score '''
| '''Poor – 1'''
 
| '''Satisfactory – 2'''
 
| '''Good – 3 '''
 
| '''Excellent – 4 '''
 
|  '''Score '''
 
 
|-
 
|-
| '''Speaking skills'''
+
| '''Speaking skills''' || || || || ||
|
 
|  
 
|  
 
|
 
|
 
 
|-
 
|-
| '''Length'''
+
| '''Length''' || || || || ||
|
 
|
 
|
 
|
 
|
 
 
|-
 
|-
 
|}
 
|}
Line 454: Line 432:
 
|  81
 
|  81
 
|-
 
|-
 +
|}
 +
 +
 +
It is best to use weighted rubrics only if the criteria, weights, and relative importance of categories would be clear to students. In some cases, such a rubric may be harder for students to interpret easily. One good rationale would be if the assignment focuses on a particular skill. In the example below for short writing assignments, support for arguments or topic sentences is weighted more. This would make sense if the assignment emphasizes support, after teaching a lesson on support and evidence.
 +
 +
{| class="wikitable"
 +
! Points !! Criterion
 +
|-
 +
| 10 || Organization
 +
|-
 +
| 10 || Format
 +
|-
 +
| 10 || Argumentation
 +
|-
 +
| 20 || Support
 +
|-
 +
| 10 || Source citations
 
|}
 
|}
  
Line 459: Line 454:
  
 
== Sample rubrics for different assignment types ==
 
== Sample rubrics for different assignment types ==
 +
 +
Instructors can take the following examples as templates, and can adapt these to fit their needs.
 +
  
 
=== Presentations ===
 
=== Presentations ===
Line 509: Line 507:
 
|-  
 
|-  
 
| 1.  
 
| 1.  
| '''Rationale, goals, objectives '''–''' '''Clear rationale & explanation for the topic or project; clear objectives / goals and benefits
+
| style="width:80%;"| '''Rationale, goals, objectives '''–''' '''Clear rationale & explanation for the topic or project; clear objectives / goals and benefits
 
|  1 2 3 4 5 6 7 8 9 10
 
|  1 2 3 4 5 6 7 8 9 10
 
|-  
 
|-  
Line 517: Line 515:
 
|-  
 
|-  
 
| 3.  
 
| 3.  
| '''Support''' – Sufficient evidence, details & explanations of main points  
+
| '''Support / details''' – Sufficient evidence, details & explanations of main points  
 
|  1 2 3 4 5 6 7 8 9 10
 
|  1 2 3 4 5 6 7 8 9 10
 
|-  
 
|-  
Line 557: Line 555:
 
|-
 
|-
 
|}
 
|}
 +
 +
 +
====Group presentations / projects====
 +
The above rubric is actually for a group project. Depending on the context, one might want to give the same grade to all students using such a rubric, or grading can be individualized. The above rubric allows for this easily. Many of the above categories can be assessed for the entire group. Then in some categories, an individual score can be assigned to each student as needed, such as the categories of '''Speaking / vocal delivery, Interaction, Clarity, and Support / Details'''. 
 +
 +
  
 
===Composition evaluation rubrics ===
 
===Composition evaluation rubrics ===
Line 579: Line 583:
 
|  
 
|  
 
|-  
 
|-  
| 3. General contents
+
| 3. '''General contents'''
 
* Good development of ideas; Clear, specific information and details; appropriate detail and coverage of contents
 
* Good development of ideas; Clear, specific information and details; appropriate detail and coverage of contents
 
* Good examples and/or detailed explanations  
 
* Good examples and/or detailed explanations  
Line 732: Line 736:
 
|-
 
|-
 
|}
 
|}
 +
 +
 +
 +
===EAP / ESP tasks===
 +
The following rubrics are for language focused tasks in English for academic / special purposes, such as business English.
 +
 +
 +
====Job interview task====
 +
The following is for mock job interviews in an EFL/ESL course, where of course language ability and performance are being evaluated. The following rubric is designed for students to evaluate other students. Instructors can use this rubric, or they can combine parts of this with the presentation / discussion task rubrics above.
 +
 +
 +
{| class="wikitable"
 +
|+ Rate each candidate on his/her performance. For each criterion (standard), rank him/her on a scale of 1-10. You can also write comments below about each applicant. For the last section of this form, tell me which persons you might or might not hire, and why.
 +
! # !! Criteria !! Description !! Rating
 +
|-
 +
| 1. || '''Objectives:'''
 +
| Clear goals / objectives as a job applicant ||
 +
|-
 +
| 2. || '''General contents:'''
 +
| Sufficient answers; answers are sufficiently informative,  of sufficient length to give satisfying answers to the questions;  not too short (but not too long or unnecessarily wordy) ||
 +
|-
 +
| 3. || '''Specific  contents:'''
 +
| Answers contain specific and sufficient details ||
 +
|-
 +
| 4. || '''Clarity:''' 
 +
| Answers are clear, understandable, logical, logically organized, realistic, and relevant to the questions ||
 +
|-
 +
| 5. || '''Speaking:'''
 +
| Speaking and vocal delivery: Clear, audible voice; good vocal intonation ||
 +
|-
 +
| 6. || '''Interaction:'''
 +
| Audience interaction: Good poise / posture, confidence, eye contact, body  language ||
 +
|-
 +
| 7. || '''Persuasive:'''
 +
| Effectiveness: Persuasive and informative answers ||
 +
|-
 +
|  || '''Comments:''' || (optional) ||
 +
|-
 +
| || '''Score''' || ||
 +
|-
 +
|}
 +
 +
 +
====Résumés, CVs, statements of purpose====
 +
The following is a checklist for résumé / CV writing assignments, or for grading sample statements of purpose for graduate school applications.
 +
 +
{| class="wikitable"
 +
|+ Resumé / SOP project
 +
|-
 +
| '''Cover letter'''
 +
<nowiki>[ ]</nowiki>  Clear, appropriate introductory paragraph; interesting; good self-intro
 +
<br> <nowiki>[ ]</nowiki>  Explains well why you want to apply for the particular job & company / school
 +
<br> <nowiki>[ ]</nowiki>  Shows familiarity with company / school
 +
<br> <nowiki>[ ]</nowiki>  Persuasive explanation of abilities, strengths,
 +
<br> <nowiki>[ ]</nowiki>  Persuasive explanation of why you’d be a valuable employee, and your future potential
 +
<br> <nowiki>[ ]</nowiki>  Good discussion of your educational, work, or other relevant background
 +
<br> <nowiki>[ ]</nowiki>  Good conclusion
 +
<br> <nowiki>[ ]</nowiki>  Appropriate length, detail, amount of content
 +
<br> <nowiki>[ ]</nowiki>  other:
 +
|-
 +
| '''Résumé or CV'''
 +
<nowiki>[ ]</nowiki>  Good, clear, professional looking format; visually well organized
 +
<br> <nowiki>[ ]</nowiki>  Sufficient detail of educational & work history
 +
<br> <nowiki>[ ]</nowiki>  Indicates good background and qualifications
 +
<br> <nowiki>[ ]</nowiki>  Other sections are clear, seem appropriate, and convey good information about you
 +
<br> <nowiki>[ ]</nowiki>  Appropriate length, detail, amount of content
 +
<br> <nowiki>[ ]</nowiki>  other:
 +
|-
 +
| '''Statement of purpose (SOP)'''
 +
<nowiki>[ ]</nowiki>  Clear, appropriate introductory paragraph
 +
<br> <nowiki>[ ]</nowiki>  Effective and interesting self-intro
 +
<br> <nowiki>[ ]</nowiki>  Explains well why you want to apply for the particular department & school
 +
<br> <nowiki>[ ]</nowiki>  Good discussion of your educational, work, or other relevant background
 +
<br> <nowiki>[ ]</nowiki>  Persuasive explanation of abilities, strengths
 +
<br> <nowiki>[ ]</nowiki>  Persuasive explanation of why you’d be a valuable employee, and your future potential
 +
<br> <nowiki>[ ]</nowiki>  Shows familiarity with dept. & school
 +
<br> <nowiki>[ ]</nowiki>  Good conclusion
 +
<br> <nowiki>[ ]</nowiki>  Appropriate length, detail, amount of content
 +
<br> <nowiki>[ ]</nowiki>  other:
 +
|-
 +
|}
 +
 +
 +
===Major projects===
 +
Major projects can be graded with a more complex rubric. In this example, students in a linguistics course are to create a fictional language for a TV series. Students have the option of doing this alone or in groups, and the same rubric applies to both cases. If students do it individually, they have to create somewhat more content than if working in a group, as the workload is shared somewhat in a group. The rubric has 15 categories.
 +
 +
;Project rubric:
 +
Your paper will be graded based on the following criteria.
 +
# Focus, e.g., clear focus on the assignment 
 +
# Organization
 +
# Contents (general): Sufficient contents, overall
 +
# Contents (specific details): Good detailed development of ideas and contents; developing all the necessary elements of the language, such as:
 +
#* Phonology / phonetics
 +
#* Morphology & lexical classes
 +
#* Syntax
 +
#* Semantics
 +
#* Lexicon / vocabulary
 +
# Explanation: Sufficient explanation of the grammar and other aspects of the language
 +
# Good examples
 +
# Clarity, e.g., clear explanations & descriptions 
 +
# Originality: creativity, and uniqueness; plausible and interesting as an alien language
 +
# Learnability (especially for actors) 
 +
# Format & procedures: (1) Equal participation and effort of all members; (2) Suitable document format, layout, etc.; (3) Turned in on time
 +
  
  
 
==Rubrics with feedback==
 
==Rubrics with feedback==
Additional feedback can be provided in various ways. A hard copy rubric form with descriptions can be handed back, with key items on the descriptions of the criteria highlighted or underlined to flag problematic areas or strengths of the student's performance. More efficient would be doing rubrics in a spreadsheet, with a column or columns for additional comments. These can be emailed to students, with rubric scores and additional comments; see the page on [[spreadsheet tips for grading]]. Also, students can submit work via Google Forms, which the instructor can comment on, and a rubric evaluation can be appended from another rubric template file. Or a Google Form rubric can simply be provided as feedback, like the example below.
+
Additional feedback can be provided in various ways. A hard copy rubric form with descriptions can be handed back, with key items on the descriptions of the criteria highlighted or underlined to flag problematic areas or strengths of the student's performance. More efficient would be doing rubrics in a spreadsheet, with a column or columns for additional comments. These can be emailed to students, with rubric scores and additional comments; see the page on [[spreadsheet tips for grading]]. Also, students can submit work via Google Forms, which the instructor can comment on, and a rubric evaluation can be appended from another rubric template file. Or a Google Doc rubric can simply be provided as feedback, like the example below. This example shows a rubric evaluation supplemented with comments from a [[feedback database]].
 +
 
  
 
{| class="wikitable"
 
{| class="wikitable"
Line 741: Line 850:
 
! # !! Criteria (My comments are in blue) !! Score (1-10)
 
! # !! Criteria (My comments are in blue) !! Score (1-10)
 
|-
 
|-
| 01 || '''Rationale, goals, objectives''' – Clear rationale & explanation of why the project is needed, goals, and its benefits <br> <div style="color:blue;">
+
| 01  
 +
| style="width:85%;" | '''Rationale, goals, objectives''' – Clear rationale & explanation of why the project is needed, goals, and its benefits <br> <div style="color:blue;">
 
Which group / people / areas are you targeting, specifically? What kinds of areas are you priority? Since the Lifestraw and Waka are already existing technologies, what exactly are you proposing that is novel or inventive? </div>  
 
Which group / people / areas are you targeting, specifically? What kinds of areas are you priority? Since the Lifestraw and Waka are already existing technologies, what exactly are you proposing that is novel or inventive? </div>  
 
| 5
 
| 5
Line 778: Line 888:
 
|5
 
|5
 
|-
 
|-
| || Grade || 70
+
| || '''Grade''' || 70
 
|}
 
|}
  
  
See also:
+
 
* [[Spreadsheet tips for grading]]
+
===Rubric + grading bank ===
 +
 
 +
Finally, a grading bank or [[feedback database]] can also be used in conjunction with grading rubrics. A rubric can be done in a spreadsheet, along with a column for comments for each student. The spreadsheet can be used for emailing students feedback on rubric categories and comments via email merge. Similarly, a Google Form can be constructed based on rubric categories, and checkboxes for common issues or comments. Each rubric category corresponds to an item in the online form, with checkboxes; each checkbox contains a commonly used comment. The instructor can fill out a form for each student, tick the appropriate checkboxes, and submit the form, which goes into a spreadsheet, which the instructor can then use for grade records. From the spreadsheet, the instructor can create an email merge to send each student feedback for rubric criteria and comments. See the following page for more on spreadsheets and email merging for feedback.
 +
 
 +
;See also:  
 +
* [[Spreadsheet tips for grading]]  
  
  
Line 827: Line 942:
 
# Various tasks: Effort. This is the students’ apparent effort in preparation, rehearsal, revision, attention to detail, research, etc.  
 
# Various tasks: Effort. This is the students’ apparent effort in preparation, rehearsal, revision, attention to detail, research, etc.  
  
 
;Rubric + grading bank:
 
Finally, grading banks or feedback databases can also be used in conjunction with grading rubrics. A rubric can be done in a spreadsheet, along with a column for comments for each student. The spreadsheet can be used for emailing students feedback on rubric categories and comments via email merge. Similarly, a Google Form can be constructed based on rubric categories, and checkboxes for common issues or comments. Each rubric category corresponds to an item in the online form, with checkboxes; each checkbox contains a commonly used comment. The instructor can fill out a form for each student, tick the appropriate checkboxes, and submit the form, which goes into a spreadsheet, which the instructor can then use for grade records. From the spreadsheet, the instructor can create an email merge to send each student feedback for rubric criteria and comments.
 
  
  
Line 839: Line 951:
  
 
===Resources===
 
===Resources===
Stevens, Dannelle D., & Levi, Antonia J. ''Introduction to Rubrics'', 2nd Ed. (2012). Stylus Publishing
+
* ''Just in Time Teaching''. (2009). Simkins, Scott, and Maier, Mark (Eds.). Sterling, VA: Stylus Publishing.
 +
* Stevens, Dannelle D., & Levi, Antonia J. ''Introduction to Rubrics'', 2nd Ed. (2012). Stylus Publishing
 +
* [[Spreadsheet tips for grading]]
 +
* [[Feedback database]]
 +
 
  
  
 
[[Category:Assessment]]
 
[[Category:Assessment]]

Latest revision as of 11:12, 12 April 2020


Grading a large number of students or assignments can be challenging. One efficient method can be a grading bank or feedback database. While grading, an instructor can save comments that are regularly used for common issues in a text document or spreadsheet, and then the comments can be copied and pasted into feedback forms for other students, and then modified or personalized for the individual student.

Rubrics are a tool for criterion-based assessment, i.e., assessment is based on specified criteria or standards. This contrasts with normative assessment, where performance is ranked according to a statistical norm or standard like a bell curve. A rubric is a scoring guide based on specific criteria or categories for assessing students’ performance on a task or assignment. A rubric is a set of grading criteria for different areas of performance. It is often in table format, and it tells the students exactly what you expect. This can be provided when you give the assignment, and/or when you grade it. Instructors can mark on a paper or electronic form to indicate criteria that students have met well or have not achieved. This allows for more efficient grading for teachers, and clearer evaluations for students.

Note
For my KOTESOL presentation on rubrics, the slides can be accessed at this link.


1 Overview

1.1 Rationale

Traditional grading methods tend to follow the following approaches.

Holistic
Each student is graded holistically, without a clear breakdown of criteria or their importance. For students, this can seem subjective, and the rationale for a grade may seem unclear, or open to interpretation.
Comparative
Students are compared with each other and ranked. It may seem relatively easy to sort assignments into different piles or groups based on holistic quality comparisons. But this can also seem subjective or open to interpretation.
Objective (close-ended)
Students are graded on responses to simple questions (multiple choice, binary choices like true/false) and passive recognition of contents. While this may be easy to grade, students do not engage with the materials and concepts, and merely demonstrate passive learning. The assignments are thus more about regurgitating knowledge, when more engaging assignments could be given to promote deeper thinking.
Open-ended tasks
Assignments with no clear answers can be more challenging, and can promote conceptual learning and engagement with the course contents. Thus, open-ended discussion questions, essay questions, written assignments, essays, essay exams, presentations, and discussion activities can be better tools for learning as well as assessment.


Grading major projects, essays, presentations, and other major assignments can be difficult, especially for more larger classes. Students expect and deserve fairly detailed feedback and justification for the grades they receive. Feedback should describe the various strengths and weaknesses of a student’s performance – exactly what s/he did well, and where s/he lacks or needs improvement. Feedback is also an opportunity to provide students specific advice on how to improve.

For complex assignments – semester projects, group projects, presentations, essays, and such, a well designed rubric can make grading more efficient, fair, and transparent.


1.2 General features

Well designed rubrics are those that meet the following criteria. # Measure stated objectives

  1. Use scales to evaluate and score
  2. Explanations for each category / scale dimension
  3. Work is evaluated according to how well standard is met


They will thus have the following outcomes, if designed properly.

  1. Objective standards or criteria
  2. Reducing grade competition
  3. Transparency & validity
  4. Clear expectations & learning objectives
  5. Grading efficiency
  6. Clearer, more meaningful feedback


Thus, rubrics should follow the following criteria for good design. This is essentially a rubric for a good rubric.

Content-focused Defines the desired content
Indicates what characteristics / content has more / less importance
Clear Communicates expectations to students
Practical Students can understand areas of comprehension & improvement; students can self-assess; teachers can easily grade with the rubric
Fair Teachers consistently apply valid, reliable assessment criteria

Avoiding bias

Specific goals Rubric criteria are based on specific learning goals, not vaguely defined outcomes.


The goals of an assignment should be specific and measurable. Rather than vague outcomes like “students will understand X,” specific, measurable goals are preferred. These would be specific, concrete goals, such as:

  • Students can ask for directions in a new city.
  • Students can do a job interview in English.
  • Students can critique a new conspiracy theory, and know how to go about debunking it.
  • Students can write a critical essay about a novella.
  • Students can write a critical response to an article.


1.3 Potential pitfalls

Poorly designed rubric items can lead to the following problems[1].

  • Can lead to “teaching to the test” or too much focus on “correct answers”
  • Overly specific criteria can discourage students from thinking creatively
  • Feedback at the end of project without guidance during the project can lead to (1) missed learning opportunities for students, and (2) teachers not being able to evaluate the effectiveness of their teaching


Vague or inconsistent criteria can have the following negative effects.

  • Undermining transparency, reliability, & validity
  • Introducing bias
  • Undermining, hindering, or not reflecting learning goals


Rubrics, if not well designed, may not provide info about:

  • advanced comprehension
  • application of knowledge
  • what students would create outside of the rubric guidelines
  • how students analyze
  • how students evaluate their own knowledge

2 Types of rubrics

2.1 Simple scalar rubrics

A very simple kind of rubric lists basic categories for evaluation, and the instructor can grade it in various ways:

  • Along a 10-point scale
  • A 5-point or 7-point Likert scale, which has the advantage of providing a meaningful “average” midpoint
  • Verbal designations like “poor/missing, average, satisfactory, good, excellent”
  • It need not total 100 points for scoring / grading purposes


This simple kind of rubric might be better suited for shorter assignments, where less detailed feedback would be sufficient. Students can see how they place along the scale for each category, and the instructor can write short comments on the rubric form as well. If the points do not add up to 100, they can be scaled up, so for the sample below, a total of 50 points × 2 = 100. Any point value can be scaled up to 100 points (e.g., 35/40 = 87.5). Or you can have cumulative points, e.g., 20 points per assignment × 5 assignments = 100 points.


2.1.1 Rubric for short paper assignments

Category Rating Subtotal
Introduction & thesis 1 2 3 4 5 6 7 8 9 10
Organization, transitions 1 2 3 4 5 6 7 8 9 10
Evidence, support 1 2 3 4 5 6 7 8 9 10
Use of sources 1 2 3 4 5 6 7 8 9 10
Style, vocabulary, grammar 1 2 3 4 5 6 7 8 9 10
TOTAL


2.2 Holistic rubrics

A holistic rubric does not have categories for different aspects of an assignment or for achievement. Instead, it simply describes different achievement levels for a task, for example, the criteria for a perfect score, for a high pass, for an average score, and such. Holistic rubrics can be ideal for minor assignments, for assignments where minor errors can be tolerated, and especially for tasks that are open-ended, for example where there is no definite correct answer.


2.2.1 Minor assignment rubric

Shorter writing assignments, quizzes, and in-class tasks might be graded along a 10-point scale like this (or the equivalent for whatever the total point value is).


Points Criterion Specific criteria
2 minimal effort The student does not try to answer, indicates that s/he does not know, or offers minimal or no response.
4 incorrect answer; low effort The student tries to answer but shows no evidence of making effort; may show serious misconceptions; does not use any information from readings or lectures (or from previous courses, knowledge, or experience) to formulate the response.
6 partially correct answer, but still incomplete; medium effort Student shows some prior knowledge and uses some correct terminology, but does not provide a complete explanation for the answer. Student does not use appropriate information from the readings or lectures (or prior knowledge). Little evidence of original thought or analysis.
8 nearly correct; good effort Student answers the question with few mistakes and with a complete explanation. Student incorporates information from the lectures and readings, and shows original thought or analysis.
10 correct; very high effort Student provides a very detailed explanation, with information from outside the course materials, e.g., has obtained and incorporated more information from outside sources, and/or shows great creativity, original thought, or critical thinking skills.


2.2.2 Just-in-time teaching (pre-class task)

A technique known by the awkward moniker of just-in-time teaching (JiTT)[2] can be used with holistic or qualitative rubrics like the one above. A better term would be a pre-class mini-quiz or task. This method is used sometimes in content-area courses, and less commonly in language courses. Students are required to read a textbook chapter before the class lecture on the topic, and then respond to a question via an online form, such as a Google Form or an assignment in a university's course management system. It is usually one conceptual question or a few short questions, focusing on concepts to be discussed in class, so that students are primed and ready for more discussion in class. The students' responses can be open-ended, such as writing a few sentences; or it can be an objective question, followed preferably by a follow-up question in which students explain or justify their answers in one or two sentences. These can be graded as a minor assignment, and since the material has not been explained in lecture, it is better to evaluate the response in a way that gives as much weight to effort as well as accuracy. The above holistic rubric is based on this type of evaluation for pre-class tasks.


See also: Just in Time Teaching. (2009). Simkins, Scott, and Maier, Mark (Eds.). Sterling, VA: Stylus Publishing.


2.2.3 Participation grades

The following is designed as a rubric for participation grades. Such grades are given in some fields to encourage students to participate in class discussion or to exercise their language skills. For example, 10% of a students’ grade in the course might be based on participation. However, grading participation can be rather subjective. To explain participation grades, and to mitigate against the possibility of students taking issue with their participation grades, the following criteria can be used.


Criterion poor (C+/C or worse) excellent (A/A+)
Attendance Skipping class; often late; lacking or not providing a valid reason for absence or lateness; seemingly contrived or artificial excuses for absences or tardiness; overburdens prof. with questions about missed work or contents; fails to make up work in time Always in class and on time; contacts prof. about legitimate reasons for repeated lateness or absence; finds out from fellow students about missed work and contents, contacting prof. when necessary; takes care of missed work responsibly
Attentiveness Does not seem to pay attention to lectures; seems to be using devices or materials for non-class-related purposes; falls asleep in class; ... Usually focused on the lecture, discussion and class activities; well prepared
Active participation Not participating in group & class discussions or class activities; not answering questions or raising relevant questions in class; never talks to prof. after / outside of class about difficulties; or may try to dominate discussions unfairly, not allowing others a chance to participate Regularly participates in class discussion and activities; asks and/or responds to questions in class; sees prof. about questions or difficulties after class; does not try to dominate discussions
Quality of contributions to class & group discussions Likes to say things that are not relevant, tangential, or self-focused; no intelligent or insightful contributions; says little beyond what is obvious; shows little sign of critical thinking Has intelligent, specific, insightful, focused comments or questions; comments or questions demonstrate critical thinking skills and creativity


2.3 Analytic rubrics

Analytic rubrics provide more detailed feedback on different aspects and different categories of evaluation for an assignment. They are thus are more appropriate for more detailed assignments and major assignments.

A point-scale based rubric like the one below might work for shorter assignments with simpler criteria, like the one below. The following sample provides specific criteria and feedback for students, and ratings for each category and/or comments can be provided in the right column. For larger assignments, a more detailed scheme is needed, with more specific criteria, which can simplify grading. The first example of a simple rubric (from Stevens & Levi, 2005, p. 40) is designed for class presentations in a film interpretation course.


2.3.1 Rubrics for presentations

Film presentations

The following rubric is for presentations on film interpretation, from Stevens and Levi (2012).[3]

Category Criteria Comments
Introduction The introduction tells the audience exactly what to expect in terms of how the speaker feels about the movie, what theories and theoretical framework(s) s/he will introduce, and what conclusions s/he will draw.
Organization The presentation is organized to create a logical argument and so that topics that need to be discussed together are presented together.
Context The presenter discusses the main historical issues raised by the film and how other film scholars and historians have dealt with these issues, both with regard to this film and in general. The presenter explains where s/he stands on these issues, which theories s/he finds most useful, and why.
Evidence, support The presenter includes sufficient, detailed examples from the film and other sources to support her/his analyses.
Analysis The presenter uses her/his evidence to support a consistent, coherent analysis of how the film does or does not contribute to our understanding of the time period.
Presentation The presenter spoke clearly, slowly, loudly enough to be heard, but not too loudly; used appropriate, effective gestures and body language; and maintained eye contact with the class. Audio-visual aids, if used, are technically sound (to prevent fumbling with equipment), appropriate, and referenced in the presentation.


One could grade this with a point-scale system, e.g., with comments and a numerical ranking for each category in the right column cells. One could also assign a global, overall, holistic grade based on overall quality, rather than assigning points for each category, and simply use the rubric for feedback and general justification of the grade.

Instead of using a point scale, one could develop a more detailed rubric with criteria for different levels of performance. The rubric will define, e.g., what constitutes an A, A-, B+, B, B-, C+, or C quality paper. On the next page is another sample rubric (by D. Stevens, 2010, from a conference handout). Evaluate the rubric in terms of the relevance and value of the criteria, clarity, fairness, and practicality in grading.


Content area presentations

The following rubric is for oral presentations in senior sociology classes, from Stevens and Levi (2012).[4]

Category Poor – 1 Satisfactory – 2 Good – 3 Excellent – 4 Score
Content Student does not have grasp of material;

information included does not support thesis in any way;

near absence of supporting materials

There is a great deal of information that is not clearly connected to the subject of the presentation;

very little use of supporting materials

Sufficient information that relates to subject of presentation;

many good points made, but there is an uneven balance;

a variety of supporting materials

An abundance of material clearly related to subject of presentation;

points are clearly made and all evidence supports subject;

a variety of supporting materials

Coherence & organization Presentation is disjointed and choppy; does not flow; development of thesis is vague; no apparent logical order of presentation Concept and ideas are loosely connected;

lacks clear transitions;

organization and flow are choppy

Most information presented in a logical sequence;

generally well organized, but needs better transitions between ideas, and between media

Subject is clearly stated and developed;

specific examples are appropriate;

conclusion is clear;

flows well with good transitions;

succinct and well organized

Speaking skills Inaudible or too loud;

no eye contact;

rate too slow or too fast;

speaker seemed uninterested;

monotonous delivery

Some mumbling;

incorrect use or mispronunciation of key terms;

little eye contact;

uneven rate of delivery;

little or no expression;

difficult to hear presentation

Clear articulation; voice is clear;

uneven eye contact;

even rate of delivery;

most audience members can hear presentation

Clear articulation and precise pronunciation of terms;

proper volume; steady rate of delivery;

good posture and eye contact; enthusiastic and confident

Length of presentation Presentation much too long or short;

ten or more minutes above or below the allocated time

Presentation is within six minutes of allocated time Presentation is within four minutes of allocated time Presentation is within two minutes of allocated time


The above rubric is well detailed, and makes it possible to assign a grade by assessing it against various criteria. If a project consistently fits the criteria for ‘good’, then you can give it a certain grade. Or each category could be worth, say, 4 points, and you total up the points for each category. In this case, the maximum is 16 points, which could be scaled up to a score of 100 (e.g., 14/16=88).

However, the criteria for ‘length of presentation’ seem unclearly written – no one should be allowed to exceed the allocated time by four to ten minutes; the time limits should be more strictly enforced. These could be better written, e.g., according to how well the presenters are able to cover the materials within the allocated time, how well they give adequate time to each major point or section, whether they finish too early, whether they have to end abruptly because they have run out of time, etc.

2.4 Weighted rubrics

In grading this way, you might find not want to give as much importance to some criteria as others. In the example above, ‘contents’ would be quite important, ‘speaking skills’ seems important but not as much as contents, but ‘length’ seems less important than the others. We can solve this by assigning different value or weights to different categories. Maybe for your projects, you want to evaluate students according to English ability (style, vocabulary, grammar, intelligibility), but in fairness to students in many fields of study, this should not be a major concern, certainly not as important as contents and coherence. So we might grade the rubric like this, where some categories have more points – more weight or value.


Criteria Poor – 2 Satisfactory – 4 Good – 6 Excellent – 8 Score
Content
Coherence
Poor – 1 Satisfactory – 2 Good – 3 Excellent – 4 Score
Speaking skills
Length


Or in a spreadsheet, you could use cell formulas to easily assign weights and calculate scores. Here, each category is ranked on a 10-point scale, and is then weighted.


Category Weight Rank (max.=10) Calculation
Contents 20% of grade 8 .2*8
Coherence 20% 9 .2*9
Evidence, support 20% 7 .2*7
Argumentation 20% 7 .2*7
Speaking skills 10% 9 .1*9
Time management 10% 10 .1*10
Total score 81


It is best to use weighted rubrics only if the criteria, weights, and relative importance of categories would be clear to students. In some cases, such a rubric may be harder for students to interpret easily. One good rationale would be if the assignment focuses on a particular skill. In the example below for short writing assignments, support for arguments or topic sentences is weighted more. This would make sense if the assignment emphasizes support, after teaching a lesson on support and evidence.

Points Criterion
10 Organization
10 Format
10 Argumentation
20 Support
10 Source citations


3 Sample rubrics for different assignment types

Instructors can take the following examples as templates, and can adapt these to fit their needs.


3.1 Presentations

3.1.1 Minor or short presentations

These criteria will vary depending on the particular assignment.


Criteria Score
1. Creativity, originality
2. Clear explanations
3. Sufficient detail & contents
4. Clear speaking & vocal delivery (including body language)
5. Equal work & contributions by each member
6. Media: good use of PPT, whiteboard, handouts, or other aids
7. Feasibility or practicality of ideas
8. Appropriate use of sources
Total score


3.1.2 Major presentation / discussion tasks

These criteria will vary depending on the particular assignment; some of these may not apply.

# Criteria Score
1. Rationale, goals, objectives Clear rationale & explanation for the topic or project; clear objectives / goals and benefits 1 2 3 4 5 6 7 8 9 10
2. General contents – Sufficient contents & preparation 1 2 3 4 5 6 7 8 9 10
3. Support / details – Sufficient evidence, details & explanations of main points 1 2 3 4 5 6 7 8 9 10
4. Effectiveness – Good, insightful analysis of the issue; good, unique, and well-thought ideas & explanations of how to deal with the problem; realistic, feasible proposals for the problem 1 2 3 4 5 6 7 8 9 10
5. Clarity – Clear explanations, easy to understand follow; clear wording & vocabulary 1 2 3 4 5 6 7 8 9 10
6. Organization – Well-organized and structured; concise, clear intro; good flow; flow indicated by intro, transition words & expressions; clear conclusion 1 2 3 4 5 6 7 8 9 10
7. Speaking & vocal delivery – Clear, audible voices; clear speaking & delivery; good vocal volume & intonation; the presentation quality indicates adequate rehearsal and preparation; no excessive fillers, pauses (uh, um), pauses, unfinished sentences; good pace 1 2 3 4 5 6 7 8 9 10
8. Interaction with audience – Eye contact, body language, etc. 1 2 3 4 5 6 7 8 9 10
9. Visual aids – Good use of whiteboard, graphics, handouts, Prezi, or PPT 1 2 3 4 5 6 7 8 9 10
10. Equal participation – Each group member participates equally 1 2 3 4 5 6 7 8 9 10
11. Value – Overall social, academic, practical, , and/or scientific value & benefit; creativity and originality 1 2 3 4 5 6 7 8 9 10
12. Reception / effectiveness – Effective, informative, and persuasive presentation; clear and coherent logical arguments and development of arguments; positive reception from audience members 1 2 3 4 5 6 7 8 9 10


3.1.3 Group presentations / projects

The above rubric is actually for a group project. Depending on the context, one might want to give the same grade to all students using such a rubric, or grading can be individualized. The above rubric allows for this easily. Many of the above categories can be assessed for the entire group. Then in some categories, an individual score can be assigned to each student as needed, such as the categories of Speaking / vocal delivery, Interaction, Clarity, and Support / Details.


3.2 Composition evaluation rubrics

This rubric may be helpful for evaluating and assessing essays.


3.2.1 Minor writing assignments

These criteria will vary depending on the particular assignment.

Criteria Score
1. Focus
  • Clear focus throughout the paper
 
2. Organization and structure
  • Well organized paragraphs and essay
  • Well developed sentences & paragraphs (¶s); not too short and choppy, nor excessively long
3. General contents
  • Good development of ideas; Clear, specific information and details; appropriate detail and coverage of contents
  • Good examples and/or detailed explanations
4. Coherence
  • Clear logical flow of ideas
  • Ideas / topics are developed before moving on; No info dump w/o enough discussion of ideas
  • No unnecessary or unrelated information
5. Clarity
  • Good, clear explanations, examples, and/or discussion of ideas
  • Clear wording, style and expressions
6. Effort / other issues
  • The writer has made appropriate effort in writing a full paper
  • Original ideas & thinking
  • Appropriate format
  • Appropriate paraphrasing, source use
Score


3.2.2 Major written assignments

These criteria will vary depending on the particular assignment.

# Criteria Score (1-10)
1. Essay topic & general focus
  • Good, specific, well-defined topic
  • Appropriate focus
2. Intro & conclusion
  • Clear, specific, intro; effective lead-in to the thesis
  • Main idea(s) of the paper are specific and clear (thesis)
  • Good summary / conclusion
3. Organization
  • Well organized paragraphs; Clear, logical flow of paragraphs
  • Clear, specific main points (topic sentences) in the body paragraphs
  • Well developed paragraphs (¶s); not too short and choppy, nor excessively long
  • Clear focus throughout the paper, without unnecessary or unrelated information
4. Contents (general)
  • Sufficient contents
  • Overall good development of ideas; sufficient explanation of thesis / main idea
  • Paragraphs are well developed
5. Support & specific details
  • Effective support, e.g., clear & detailed explanations, examples, and/or discussion of ideas
  • Specific ideas / proposals are developed with enough detail
  • Clear, specific information; sufficient details;
  • Appropriate detail and coverage of all supporting claims / topic sentences
6. Coherence
  • Clear logical flow of ideas
  • Ideas / topics are developed before moving on
  • Logical flow of sentences, ideas, and paragraphs
7. Style & wording
  • Clear wording & style
  • Appropriate academic style and tone
  • No serious wording, style or grammar problems that make the paper hard to understand
  • Well developed sentences; not too short and choppy, nor excessively long
8. Format
  • Proper paper / document layout
  • Standard text & ¶ format; correct punctuation, spelling, etc.
  • Proper file format
  • Proper format for in-text citations, and end references / works cited entries
9. Overall effectiveness
  • Overall effectiveness
  • Sufficiently informative
  • Overall persuasiveness; logical, coherent arguments
  • Shows appropriate effort
10. Source use
  • Appropriate use of sources in the paper; appropriate sources
  • Appropriate summarizing / paraphrasing
  • Appropriate use of in-text citation style
  • Appropriate end references
Score


3.2.3 Detailed analytical rubric

The following is a rather detailed rubric for written assignments. This may be cumbersome for online or spreadsheet grading, but it can be photocopied and given to each student. Notes can be added, and key phrases can be marked by pen or highlighter to draw attention to problematic or noteworthy areas.[5]

Features low fail: D narrow fail: C adequate: B good pass: B+ high pass: A
Organization:

Degree to which logical flow of ideas and explicitness of the plan are clear and connected

No plan, insufficient length to ascertain organization Attempted plan is noticeable, inadequate paragraphing Plan is clear; some cohesion and coherence. Plan is clear, most points connected; coherent; various uses of cohesive devices Excellent us of format elements with all points connected and signified with transitions and/or other cohesive devices
Content:

Degree to which the main points/ elements are elaborated and/ or explained by evidence and detailed reasons

No support; insufficient length and/or extensive plagiarized passages, too short to evaluate Attempted elaboration; maybe a list of related specifics; some plagiarism, inadequate length Some points elaborated; some acknowledgement of sources Most points elaborated; argument is developed in a clear and logical manner, characterized by adequate generalizations All major points elaborated; argument is developed in a clear and logical manner with proper use of sources and leads to sophisticated generalizations and conclusions
Conventions:

Use of grammatical conventions of standard English (usage, sentence construction, spelling, and ¶ format).

Many errors, almost unreadable, confused meaning; problems with sentence construction; insufficient length to evaluate Some major and many minor errors; confusion; sentence construction not mastered. Developed; few major errors, some minor ones, meaning unimpaired; mastery of sentence construction A few minor errors, but no more than one major error No major errors, one or two minor errors; more sophisticated sentence constructions
Vocabulary, style:

Degree to which the student has mastered register, sentence variation, and word/idiom use.

Very poor, essential translation, no sentence variety Limited range; frequent errors of word/idiom choice; form and usage that confuse meaning; some sentence variety. Adequate range of word/idiom choice and sentence variety, some errors that do not obscure meaning Few word form errors, occasionally misused words/idioms; appropriate register (tone, style); varied sentence structure Sophisticated range; effective word/idiom choice and usage; word form mastery; appropriate register; varied sentence structure


3.3 EAP / ESP tasks

The following rubrics are for language focused tasks in English for academic / special purposes, such as business English.


3.3.1 Job interview task

The following is for mock job interviews in an EFL/ESL course, where of course language ability and performance are being evaluated. The following rubric is designed for students to evaluate other students. Instructors can use this rubric, or they can combine parts of this with the presentation / discussion task rubrics above.


Rate each candidate on his/her performance. For each criterion (standard), rank him/her on a scale of 1-10. You can also write comments below about each applicant. For the last section of this form, tell me which persons you might or might not hire, and why.
# Criteria Description Rating
1. Objectives: Clear goals / objectives as a job applicant
2. General contents: Sufficient answers; answers are sufficiently informative, of sufficient length to give satisfying answers to the questions; not too short (but not too long or unnecessarily wordy)
3. Specific contents: Answers contain specific and sufficient details
4. Clarity: Answers are clear, understandable, logical, logically organized, realistic, and relevant to the questions
5. Speaking: Speaking and vocal delivery: Clear, audible voice; good vocal intonation
6. Interaction: Audience interaction: Good poise / posture, confidence, eye contact, body language
7. Persuasive: Effectiveness: Persuasive and informative answers
Comments: (optional)
Score


3.3.2 Résumés, CVs, statements of purpose

The following is a checklist for résumé / CV writing assignments, or for grading sample statements of purpose for graduate school applications.

Resumé / SOP project
Cover letter

[ ] Clear, appropriate introductory paragraph; interesting; good self-intro
[ ] Explains well why you want to apply for the particular job & company / school
[ ] Shows familiarity with company / school
[ ] Persuasive explanation of abilities, strengths,
[ ] Persuasive explanation of why you’d be a valuable employee, and your future potential
[ ] Good discussion of your educational, work, or other relevant background
[ ] Good conclusion
[ ] Appropriate length, detail, amount of content
[ ] other:

Résumé or CV

[ ] Good, clear, professional looking format; visually well organized
[ ] Sufficient detail of educational & work history
[ ] Indicates good background and qualifications
[ ] Other sections are clear, seem appropriate, and convey good information about you
[ ] Appropriate length, detail, amount of content
[ ] other:

Statement of purpose (SOP)

[ ] Clear, appropriate introductory paragraph
[ ] Effective and interesting self-intro
[ ] Explains well why you want to apply for the particular department & school
[ ] Good discussion of your educational, work, or other relevant background
[ ] Persuasive explanation of abilities, strengths
[ ] Persuasive explanation of why you’d be a valuable employee, and your future potential
[ ] Shows familiarity with dept. & school
[ ] Good conclusion
[ ] Appropriate length, detail, amount of content
[ ] other:


3.4 Major projects

Major projects can be graded with a more complex rubric. In this example, students in a linguistics course are to create a fictional language for a TV series. Students have the option of doing this alone or in groups, and the same rubric applies to both cases. If students do it individually, they have to create somewhat more content than if working in a group, as the workload is shared somewhat in a group. The rubric has 15 categories.

Project rubric

Your paper will be graded based on the following criteria.

  1. Focus, e.g., clear focus on the assignment
  2. Organization
  3. Contents (general): Sufficient contents, overall
  4. Contents (specific details): Good detailed development of ideas and contents; developing all the necessary elements of the language, such as:
    • Phonology / phonetics
    • Morphology & lexical classes
    • Syntax
    • Semantics
    • Lexicon / vocabulary
  5. Explanation: Sufficient explanation of the grammar and other aspects of the language
  6. Good examples
  7. Clarity, e.g., clear explanations & descriptions
  8. Originality: creativity, and uniqueness; plausible and interesting as an alien language
  9. Learnability (especially for actors)
  10. Format & procedures: (1) Equal participation and effort of all members; (2) Suitable document format, layout, etc.; (3) Turned in on time


4 Rubrics with feedback

Additional feedback can be provided in various ways. A hard copy rubric form with descriptions can be handed back, with key items on the descriptions of the criteria highlighted or underlined to flag problematic areas or strengths of the student's performance. More efficient would be doing rubrics in a spreadsheet, with a column or columns for additional comments. These can be emailed to students, with rubric scores and additional comments; see the page on spreadsheet tips for grading. Also, students can submit work via Google Forms, which the instructor can comment on, and a rubric evaluation can be appended from another rubric template file. Or a Google Doc rubric can simply be provided as feedback, like the example below. This example shows a rubric evaluation supplemented with comments from a feedback database.


Community service project proposal: Evaluation
Group: Alice Cooper, Tom Petty, Iggy Pop, Patti Smith; Project: Lifestraw
# Criteria (My comments are in blue) Score (1-10)
01 Rationale, goals, objectives – Clear rationale & explanation of why the project is needed, goals, and its benefits
Which group / people / areas are you targeting, specifically? What kinds of areas are you priority? Since the Lifestraw and Waka are already existing technologies, what exactly are you proposing that is novel or inventive?
5
02 Contents – Sufficient contents & explanations
What aspects of your concept are novel, creative, innovative…?
7
03 Project details – Sufficient details & explanations
Where do you want to carry out the project, and how are these devices to work together in your concept?
7
04 Other details – Timeline, budget, participants, etc., as necessary
Few details here.
8
05 Explanations – Clear explanations, easy to understand follow;
It’s not clear what new ideas are presented here.
8
06 Clarity & expression - Clear expressions & style; clear sentences; writing that shows sufficient work, revision, & proofreading 8
07 Organization - Well organized; clear intro & conclusion
Weak conclusion
8
08 Format - Readable; appropriate formatting. Also, if used, suitable and well-done graphics, tables, charts, graphs, etc. (not required)
Long paragraphs; hard to follow.
7
09 Value – Overall social, practical and scientific value & benefit; creativity and originality
7
10 Persuasiveness - Overall, the proposal would be persuasive and interesting to a professional audience 5
Grade 70


4.1 Rubric + grading bank

Finally, a grading bank or feedback database can also be used in conjunction with grading rubrics. A rubric can be done in a spreadsheet, along with a column for comments for each student. The spreadsheet can be used for emailing students feedback on rubric categories and comments via email merge. Similarly, a Google Form can be constructed based on rubric categories, and checkboxes for common issues or comments. Each rubric category corresponds to an item in the online form, with checkboxes; each checkbox contains a commonly used comment. The instructor can fill out a form for each student, tick the appropriate checkboxes, and submit the form, which goes into a spreadsheet, which the instructor can then use for grade records. From the spreadsheet, the instructor can create an email merge to send each student feedback for rubric criteria and comments. See the following page for more on spreadsheets and email merging for feedback.

See also


5 Developing and using rubrics

Instructors can create new rubrics in several ways. At first, this takes time, but over time, they can save a great deal of time in grading. The following are some useful steps for creating rubrics.

  1. Define specific learning outcomes (not general objectives)
  2. Define categories / criteria based on past feedback & issues
  3. Draw from feedback databases
  4. Work collaboratively on rubric design and implementation
  5. Focus on conceptual understanding, higher order learning


First, specific, measurable learning outcomes should be defined. Instead of a vague objective (“students will understand X”), more specific, concrete outcomes would be better, e.g.: Students can ask for directions in a new city; Students can do a job interview in English; or Students can write a critical response to an article.

The categories and criteria for a rubric and be developed based on past feedback given, issues observed with previous assignments, or typical issues that students have. The categories and criteria can also come a qualitative analysis or examination of past feedback given to students, especially from an instructor’s general feedback database or grading bank. Also, teachers can work collaboratively on brainstorming rubric categories and criteria. They can not only develop a rubric together, but they can try it out by evaluating some sample assignments, and then discussing how they rate the assignments with the new rubric. They can not only calibrate how they grade assignments, but they can also see the strengths and weaknesses of their rubric and then make improvements before using it on actual assignments.


Finally, assessments should emphasize conceptual knowledge and application of knowledge, not simply recalling facts, i.e., knowledge recall. More important is their comprehension of the materials, and it is even more important that students be able to do something with their comprehension, such as somehow applying, problem-solving, analyzing, synthesizing, critiquing, and evaluating in ways that reflect what they will do in their future studies, their careers, or their lives.


5.1 Further tips

De-emphasize grammar and mechanics

For ESL/EFL learners, grading criteria like grammar and mechanics can seem intimidating, especially if they have experienced a good deal of stress in the past from standardized exam stress, prescriptive grammar instruction, grammar-translation instruction, and traditional style learning and grading. A rubric can de-emphasize grammar and mechanics by using a slightly more general criterion that sounds less intimidating, such as: * Style

  • Expression / expressiveness
  • Clarity
  • Coherence / flow


Split up larger categories

Instead of a weighted rubric (one criterion worth more than others), more important criteria can be split up into smaller rubric categories. In the examples in the previous section, the following were split up into two categories, so that all items in the rubric were of equal value.

  • Presentations: Speaking → (1) Vocal delivery, and (2) Audience interaction
  • Presentations: Contents → (1) General contents, and (2) Project details
  • Essays: Contents → (1) General contents, and (2) Support & details


Flexible categories

For major assignments, consider having one category that allows for some flexibility. It is possible for an assignment to excel in all areas, but suffer in one area that the rubric did not anticipate; or conversely, a student might excel in an unusual aspect that is not included in the rubric. Some leeway is desirable for such situations. The examples above have the following flexible categories.

  1. Presentations: Reception & effectiveness. This includes how well the presentation was received by audience members (as students fill out short evaluation forms of other student presenters), its persuasiveness and informativeness, and its overall effectiveness.
  2. Essays: Effectiveness. This includes the essay’s persuasiveness and informativeness, logic, and overall effectiveness.
  3. Various tasks: Effort. This is the students’ apparent effort in preparation, rehearsal, revision, attention to detail, research, etc.



6 See also

  1. Adapted from http://educ6040fall10.wikispaces.com (now defunct).
  2. Just in Time Teaching. (2009). Simkins, Scott, and Maier, Mark (Eds.). Sterling, VA: Stylus Publishing.
  3. Stevens, Dannelle D., & Levi, Antonia J. Introduction to Rubrics, 2nd Ed. (2012). Stylus Publishing
  4. Stevens, Dannelle D., & Levi, Antonia J. Introduction to Rubrics, 2nd Ed. (2012). Stylus Publishing
  5. Source unknown.


6.1 Resources

  • Just in Time Teaching. (2009). Simkins, Scott, and Maier, Mark (Eds.). Sterling, VA: Stylus Publishing.
  • Stevens, Dannelle D., & Levi, Antonia J. Introduction to Rubrics, 2nd Ed. (2012). Stylus Publishing
  • Spreadsheet tips for grading
  • Feedback database