Anonymous marking and moderation - how to do it?

Options
Hi everyone. At my institution, we have recently implemented anonymous marking. We are now struggling with how a moderation process would work. Traditionally, the first marker would provide the feedback in the annotations tool, and in the feedback box as usual. The second marker (moderator) would then add a comment in the Grades tool, where the comment can be private, to indicate that they were happy with the judgment. An external examiner would then review the feedback and this comment. However, because we now use anonymous marking, that process can't take place unless the grades are first published (and therefore de-anonymised). Does anybody know how we might best handle this workflow so that all markers will see anonymised work? Thanks!

Answers

  • Megan.Shuker78
    Megan.Shuker78 Posts: 9
    edited November 2022

    Hi Olly - which person would be responsible for publishing the evaluations so that the students can see them?

  • Olly.Fayers41
    Olly.Fayers41 Posts: 31 🌱
    edited November 2022

    Good question. I think that would typically be the first marker or an academic administrator.

  • Sophie.McGown62
    Sophie.McGown62 Posts: 85
    edited November 2022

    HI @Olly Fayers​ ,

    So we don't have a perfect solution to this one yet (working on it), but i can give you a bit of guidance on some options/caveats:

    Tip: Do not go down the route of just hiding the grade item i.e. publishing but hiding it. It's still visible in class progress when published no matter if you've hidden the grade item.

    Option 1

    1. Steve from Huddersfield has written a script which essentially surfaces draft assignment grades in a sortable list/exportable excel. This means the marker fills in all details as normal, and then the script sits as an option called 'assignment reports' on the navbar.
    2. The moderator can export that list, and pop in another column to indicate Y/N agreement, and if you want that to be as quick and easy for them as possible, you can import just that column into the gradebook into a hidden item (instructions https://documentation.brightspace.com/EN/semester_start/-/instructor/import_grades.htm ).
    3. Here's the script - he's kindly shared it on here: https://community.brightspace.com/s/article/Developer-Spotlight-Using-a-script-to-surface-draft-assignment-grades

    = trail of moderation in gradebook, no need to publish grades before this moderation happens.

    Option 2: Rubrics

    You can create a second marker rubric that's hidden permanently from students. Have 1 criteria 2 levels that says agree or disagree which the moderator can fill in.

    Only issue here is that you'd have to go into each assignment to see if the moderator had agreed/disagreed - if the moderation process halts marking/publishing (which i expect it does), then this might not be the best solution.

     

    Hope that helps?

    Sophie

     

  • Olly.Fayers41
    Olly.Fayers41 Posts: 31 🌱
    edited November 2022

    Sophie, that is a wonderful answer, thank you. I did say to a colleague yesterday that I'd bet money that @Steve Bentley​  would have done something clever with APIs that deals with this. I shall pass this on and see what we'd like to do with it!