Overview

Rubrics are the foundation of accurate and consistent scoring in Feedback Aide. A well-designed rubric ensures that the engine grades responses the way a human grader would—fair, reliable, and aligned with expectations.

Refer to Feedback Aide Rubrics for an overview of the basics.

While Feedback Aide’s rubrics are designed to align with common standards and mark schemes, following these guidelines will lead to more accurate and consistent marking.

Designing effective Feedback Aide rubrics

Short response rubrics

Short responses usually require a fact, concept, or brief reasoning. For AI graders, ambiguity in rubrics can lead to inconsistent or incorrect scoring, because AI cannot infer intent the way a human might.

Avoid Ambiguity (be clear and unambiguous)

Consider the following rubric for the question: “Explain what is meant by the term periodicity.”

• Original rubric:

  
  Ambiguous rubric

  Correct Response: repeating trends/patterns seen in elements across a period
  Problem: The use of the slash (“trends/patterns”) creates ambiguity. Should learners mention both, or is one sufficient? Feedback Aide (AI) grades inconsistently because of this unclear instruction.

• Improved rubric:

  Correct Response: Periodicity refers to the recurring patterns that are seen in the elements across a period
  Alternate Response: Periodicity refers to the trends in the element properties across a period

Benefit: By explicitly breaking responses into Correct and Alternate, ambiguity is removed. Both human graders and AI graders can now consistently award credit as long as learners mention either trends or patterns.

Add specificity (state expected details)

Now consider this key points rubric for the question: “Read the passage below and explain how Maya responds to losing. In your answer, describe what her actions show about her as a person and why that’s important.”

• Original rubric:

  The rubric asked for three key ideas – identify the response, name a character trait, and explain why the trait is important
  Problem: Without explicitly stating the expected answer, any response, trait, or explanation could be accepted, making grading inconsistent.

• Improved rubric:

  The optimized rubric specifies:

  • Identifies Maya’s response (e.g., she stayed calm, asked for help)
  • Names character traits shown (e.g., determined, resilient, open to learning)
  • Explains why that trait is important (e.g., helps her grow as a person, not just win next time)

Benefit: By giving details in the rubric, graders (human or AI) can evaluate both the skill and the correctness of the response. Learners don’t need to use the exact words, but their answers must align with these clearly defined expectations.

Make distribution of marks explicit

Question: "The nth term of a sequence is 4n – 3. Is 75 a term of this sequence? Give a reason for your answer."

• Original rubric:

  
  Rubric with incomplete justification

  Key point 1 - Correct answer: 75 is not a term in the sequence
  Key point 2 - Correct reasoning: There is no integer solution OR 75 is between the 19th and 20th term
  Problem: Consider the learner response "75 is not a term", which would earn a score of 1 based on the rubric above. However, in the evaluation setting in this example, human graders only wanted to award points if the learner made an attempt at reasoning. In other words, they expected the response "75 is not a term" to score 0 points.

  This is an example of an unspoken expectation – the “hidden rubric.” It’s important to draw these implications out and make them explicit in the rubric. 

• Improved rubric with explicit justification:

  To achieve the expected behaviour, the rubric must make justification part of the scoring rules:

  • Key point 1 - Identifies the correct answer (75 is not a term of the sequence) AND makes some attempt to justify this.
  • Key point 2 - Has correct reasoning, either: there is no integer solution, OR 75 is between the 19th term (73) and the 20th term (77).

Benefit: Now, the response “75 is not a term” scores 0 points because reasoning is missing. The response “75 is not a term because when I add the numbers together, the sum is odd” scores 1 point for attempting reasoning.

Ensure mutual exclusivity

In rubrics, like “X from Y” rubrics, learners are asked to provide a certain number of valid points (e.g., 3 from a list of 8).

If valid points overlap too much, a single learner response might fit multiple categories, leading to double credit or inconsistent scoring.

Ensure valid answers are mutually exclusive, meaning each response aligns clearly with only one rubric category.

Essay Rubrics

Draw out the hidden rubric

Human graders often apply expectations intuitively, but AI requires every detail to be written out. For instance, if a learner is required to integrate three sources in order to receive full credit, this requirement must be included in the rubric. Similarly, if learners need to demonstrate correct use of a grammatical concept, that too should be clearly stated. Without these details, AI graders may award credit inconsistently, even if teachers would expect otherwise.

Optimize the language used

Some rubrics attempt to create custom definitions, such as defining “Line of Reasoning” as “an arrangement of claims and evidence that makes a point.” While this may appear convenient, it reduces consistency. Learners, teachers, and AI graders may all interpret such specialized definitions differently, which leads to variation in scoring.

The more effective approach is to use terms in their normal, widely accepted definitions. This ensures shared understanding and consistency across graders and systems. In the same way, it is important to avoid overly academic or lengthy phrasing. The simpler and clearer the rubric language, the more effective and reliable it will be.

Using sources

A Source is the text or material that the learner is responding to, and it can be passed to Feedback Aide to provide additional context. This may include literature, informational texts, historical documents, or even visual materials such as political cartoons or charts (in which case only the alternative text is interpreted by Feedback Aide).

By including the source, Feedback Aide is able to more accurately identify correct responses and align grading with the rubric.

Use sources when:

• A learner is responding directly to a piece of content.
• A learner is asked to provide evidence from a given text.

For example, if a learner is tasked with explaining how a character responds to losing in a passage, the source allows Feedback Aide to check the learner’s response against the actual text, ensuring that credit is awarded only when the evidence aligns with what is written.

Using Additional Grading Guidance

The Additional Grading Guidance field allows authors to clarify nuanced aspects of the marking instructions that may not be fully captured in the rubric criteria alone. Refer to the Additional Grading Guidance section within the Feedback Aide Rubrics article for more information.

Suppose a rubric requires learners to reference a well-vetted source. On its own, the rubric might accept any reference. But using Additional Grading Guidance, you can specify: “General reference guides such as encyclopedias and dictionaries do not fulfill the requirement for a well-vetted source.”

This guidance ensures that learners are graded fairly and consistently, and that AI applies the same professional judgment a teacher would.

Use additional context:

• To clarify exceptions or edge cases.
• To provide examples of borderline answers.
• To highlight partial credit rules.
• To add grading requirements not captured in the rubric itself.