Overview
A key component of Feedback Aide is the Rubric. The rubric is the scoring criteria definition, that a human or AI can use to evaluate and provide a score and relevant feedback to the learner.
The rubric format is designed to allow existing rubrics that are currently in use to be easily migrated to the Feedback Aide Rubric format.
Feedback Aide Rubric Types
Feedback Aide currently supports analytic and holistic rubrics with discrete and multi-discrete scoring.
Analytic
The following is a discrete analytic rubric which has four traits and four levels. In a discrete rubric, there is a fixed score for each level.
The rubric above is represented by the following JSON data:
{
"title": "Narrative Writing Rubric",
"type": "analytic",
"criteria": [
{
"title": "Ideas",
"type": "discrete",
"levels": [
{
"score": 3,
"description": "Tells a story with ideas that are clearly focused on the topic and are thoroughly developed with specific, relevant details."
},
{
"score": 2,
"description": "Tells a story with ideas that are somewhat focused on the topic and are developed with a mix of specific and/or general details."
},
{
"score": 1,
"description": "Tells a story with ideas that are minimally focused on the topic and developed with limited and/or general details."
},
{
"score": 0,
"description": "Ideas are not focused on the task and/or are undeveloped."
}
]
},
{
"title": "Organization",
"type": "discrete",
"levels": [
{
"score": 3,
"description": "Organization and connections between ideas and/or events are clear and logically sequenced."
},
{
"score": 2,
"description": "Organization and connections between ideas and/or events are logically sequenced."
},
{
"score": 1,
"description": "Organization and connections between ideas and/or events are weak."
},
{
"score": 0,
"description": "No organization evident."
}
]
},
{
"title": "Style",
"type": "discrete",
"levels": [
{
"score": 3,
"description": "Command of language, including effective and compelling word choice and varied sentence structure, clearly supports the writer's purpose and audience."
},
{
"score": 2,
"description": "Adequate command of language, including effective word choice and clear sentences, supports the writer's purpose and audience."
},
{
"score": 1,
"description": "Limited use of language, including lack of variety in word choice and sentences, may hinder support for the writer's purpose and audience."
},
{
"score": 0,
"description": "Ineffective use of language for the writer's purpose and audience."
}
]
},
{
"title": "Conventions",
"type": "discrete",
"levels": [
{
"score": 3,
"description": "Consistent, appropriate use of conventions of Standard English for grammar, usage, spelling, capitalization, and punctuation for the grade level."
},
{
"score": 2,
"description": "Adequate use of conventions of Standard English for grammar, usage, spelling, capitalization, and punctuation for the grade level."
},
{
"score": 1,
"description": "Limited use of conventions of Standard English for grammar, usage, spelling, capitalization, and punctuation for the grade level."
},
{
"score": 0,
"description": "Ineffective use of conventions of Standard English for grammar, usage, spelling, capitalization, and punctuation."
}
]
}
]
}
The below example is a multi-discrete rubric (or "range rubric") with five criteria and five levels. Instead of assigning a fixed score for each level, the grader can choose from a range of possible points within that level, giving the grader more precision in scoring.
Holistic
The following is a holistic discrete rubric. Holistic rubrics provide a single score, capturing the general quality and impact of the work as a whole. Holistic rubrics streamline grading and provide students with a big-picture view. We also support multi-discrete (range) holistic rubrics.
A holistic rubric can be selected by specifying the global type as 'holistic.'
"type": "holistic"
Rubric JSON Schema
The Feedback Aide rubric format is defined in the following JSON schema. This can be used to validate schemas for use with Feedback Aide.
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://feedbackaide.learnosity.com/docs/rubric.schema.json",
"title": "Learnosity Rubric v0.1",
"description": "Rubric definition for essay scoring",
"type": "object",
"required": ["title", "type", "criteria"],
"additionalProperties": false,
"properties": {
"title": {
"description": "The title of the rubric",
"type": "string",
"minLength": 0,
"maxLength": 60
},
"type": {
"description": "Type of rubric - holistic or analytic",
"type": "string",
"enum": ["holistic", "analytic"]
},
"criteria":{
"description": "Array of Criteria for scoring",
"type": "array",
"minItems": 1,
"maxItems": 10,
"items":{
"type": "object",
"required": ["title", "type", "levels"],
"additionalProperties": false,
"properties": {
"title": {
"description": "The title of the criteria",
"type": "string",
"minLength": 0,
"maxLength": 60
},
"description": {
"description": "Detailed description of the criteria",
"type": "string",
"minLength": 0,
"maxLength": 500
},
"type": {
"description": "Type of criteria - discrete or discrete_multi",
"type": "string",
"enum": ["discrete", "discrete_multi"]
},
"levels": {
"description": "Array of levels for the criteria",
"type": "array",
"minItems": 1,
"maxItems": 6,
"items": {
"type": "object",
"required": ["description"],
"additionalProperties": false,
"properties": {
"title": {
"description": "The title of the rubric",
"type": "string",
"minLength": 0,
"maxLength": 60
},
"description": {
"description": "The full description of the rubric",
"type": "string",
"minLength": 0,
"maxLength": 500
},
"score": {
"description": "The score of the level - only for discrete type criteria",
"type": "integer",
"minimum": 0,
"maximum": 100
},
"scores": {
"description": "Array of possible scores at this level - only for discrete_multi type criteria",
"type": "array",
"minItems": 1,
"maxItems": 6,
"items": {
"type": "integer",
"minimum": 0,
"maximum": 100
}
}
}
}
}
},
"$comment": "Conditional validation based on type of criteria - TODO investigate if can be done with to support more than 2 types of criteria",
"if": {
"properties": {
"type": {
"const": "discrete"
}
}
},
"then": {
"properties": {
"levels": {
"type": "array",
"items": {
"type": "object",
"required": ["score"]
}
}
}
},
"else": {
"properties": {
"levels": {
"type": "array",
"items": {
"type": "object",
"required": ["scores"]
}
}
}
}
}
}
}
}