Randomizing content in Learnosity assessments

When creating Learnosity assessments, you can choose to have randomization of the content. There are a number of different modes of operation and approaches for this.

Modes of operation for random content in Learnosity assessments

The following modes of operation are available for content randomization in Learnosity assessments:

1. Random Activities (delivering random Items from groups of Items)
 Random Activities allows you to create Item groups from which you can configure a to subset be delivered randomly at runtime. You can select a mixture of item groups where items can be delivered sequentially, or in a random order.  

2. Shuffle options (shuffle the display order of potential answers)
The answer choices displayed on screen can be shuffled (in the context of a single Question).

3. Shuffle Items (shuffling the order of Items in the assessment)
Shuffling the order in which the Items are displayed (in the context of an assessment).

4. Dynamic content (drawing random choices from a large set of possible data)
Selecting data for Questions or Items randomly from preset data tables.

5. "Try Again" shuffling (Item content shuffled on demand)
This shuffles the Item content when the student clicks "try again", generating a new Item from tabular data.

6. Math Question Generator (generating different combinations of Item data)
Generating random mixtures of content from a data table into Items (for example, a range of different number values for arithmetic addition drills).

7. Adaptive assessment (responding to student skill level to choose the next Item)
Adaptive assessment responds to the student's knowledge level and shows different follow-on Items, based on the student's performance. This requires a configured branching structure, or configuration of algorithmic behaviour based on Item difficulty values. 

Each approach is summarised below, with instructions on how to enable it.

 

1. Random Activities (Items API, back end)

In Random Activities, Items for delivery are generated at run-time based on the Item groups configuration when the assessment is initialized in the initial mode. Once generated, the same set of Items are presented during the resume mode. Items are generated starting with the first Item group's configuration, moving to the next group's configuration (and so on) until the Items for the last Item group have been generated. Each Item group within the Random Activity can have its own configuration.

Turning on Random Activities

To switch on Random Activities, set the add_random init option to true in the Activity list’s toolbar configuration:

"activity_list": {
    "toolbar": {
        "add_random": true
    }
}

Code example 1: setting add_random to true.

 

When the add_random initialization option has been set when calling the Author API, new random Activity options will appear. 

In the Activity Editor view, you have a new drop-down menu where you can choose to add a "Random Item group" to the Activity, which will randomly choose Items from that Item group at assessment time, and show them to the learner. You can also combine these "random Item" groups with "fixed Item" groups. You can choose to include or exclude any of the Items from these groups. The groups can also be renamed to any name you desire.

Further Reading

See the article Authoring Random Activities for more information.

 

2. Shuffle Options (Questions API, front end)

This shuffles the display of the available answers in a Question. For example, in the multiple choice Question type, if the default order of available answers for a given Question is authored with the order 1-2-3-4, with shuffle options enabled, it might show the answers in the order 3-1-2-4, instead.

Turning on shuffle options

In the multiple choice Question authoring, you have a check box at the bottom of the authoring screen for turning on "shuffle options". You can also configure it in the Question JSON.

In JSON: set shuffle_options: true.

{
    "stimulus": "Which of these colors has the smallest wavelength?",
    "options": [
        {
            "value": "0",
            "label": "Red"
        },
        {
            "value": "1",
            "label": "Violet"
        },
        {
            "value": "2",
            "label": "Blue"
        },
        {
            "value": "3",
            "label": "Orange"
        }
    ],
    "type": "mcq",
    "validation": {
        "scoring_type": "exactMatch",
        "valid_response": {
            "value": [
                "1"
            ]
        },
        "shuffle_options": true
    }
}

Code example 2: setting shuffle_options to true.

 

Note: by default, for the random seed, the response id will be used. This means that every student will see a different order of shuffling in the possible answers shown on the screen (because each response has a unique id). If you want every student to see the same order of shuffling, you can set your own seed value.

Setting a seed value for shuffle options

The seed value acts as a starting value for the randomization. You can change this by providing a different seed value to the default.

To set a seed value: 

In JSON: set "shuffle_options": "seed-value-here"
Note: The seed value can be any string. 

Turning off shuffle options

In JSON: set "shuffle_options: false"

Note: "false" is the default setting.

Further reading

See more about authoring options for the Multiple Choice Question type.

 

3. Shuffle Items (Items API, Assess API, back-end)

Shuffle Items will randomize the order of Items in an assessment. For example, if the default order of Items in a given assessment is authored with this order:

  • Item 1
  • Item 2
  • Item 3
  • Item 4

With shuffle Items enabled, the assessment might show the order of Items as follows:

  • Item 2
  • Item 1
  • Item 4
  • Item 3

Turning on shuffle Items

1. To switch on shuffle Items, add the following to your configuration JSON:

{
    "config": {
        "configuration": {
            "shuffle_items": "true"
        }
    }
}

Code example 3: setting 'shuffle_items' to true.

 

It's best to set this configuration by taking steps in the following order:

a. Configure the assessment,

b. Set the order of Items, 

c. Apply the shuffle setting,

d. Make no further changes to the Item order.

Note: In this case, when no random seed value is assigned, the session ID will be used as the random seed. This means that every student will see a different order of shuffling, because they each have a unique session ID.

Setting a seed value for shuffle Items

2. Switch it on and set a seed value:

{
    "config": {
        "configuration": {
            "shuffle_items": "seed-value-here"
        }
    }
}

Code example 4: activating 'shuffle_items' and setting a seed value.

 

Note: when setting a specific seed value, all students will see the same order of Item shuffling.

 

Turning off shuffle Items

{
    "config": {
        "configuration": {
            "shuffle_items": "false"
        }
    }
}

Code example 5: deactivating 'shuffle_items'.

 

The default setting for shuffle_items is "false".

 

Note: After setting the shuffle_items option, you should not make any further changes to the original order of Items in the Activity, because this may cause students to see unpredictable content when resuming a session. 

Further reading

See the syntax for the shuffle_items setting in the Assess API developer reference documentation.

 

4. Dynamic Content (Items API, back end)

You can also use the dynamic content feature to randomize the content delivery. Using this, the Item content is set up by the author as a CSV data table. Then, the dynamic content system will draw from that to choose one of many possible choices for the Item data.

Turning on dynamic content
Dynamic Content is set up in the authoring interface, and the method for doing this varies, depending on which Question type you are using. The first step is to set the dynamic_content initialization option to true (under config >item_edit). See the article What is Dynamic Content? for more information. 

Setting up a data table for dynamic content

Next, you must create a data table for the dynamic content. See the article Setting up a Data Table and Inserting Dynamic Content into a Question. 

Setting a random seed value for dynamic content

By default, the order of shuffling is set by the response ID. To configure the seed so that all students see the same order of shuffling, you can set it by adding the following to the init options when calling the API:

"dynamic_items": {
        "data_table_seed": "SOME_SEED", // optional

Code example 6: setting a seed value for dynamic content

 

Turning off dynamic content

To turn off dynamic content, you can simply remove the data table from the Item(s) where it was in use and replace that content with static content. You could also disable dynamic content from the Author API configuration options (under config >item_edit).

Further reading

See the knowledge base article on Using Dynamic Content and Try Again in your Assessments.

 

5. "Try Again" Shuffling (Assess API/Questions API front end)

You can configure "Try Again" for the local practice assessment scenario. This means that when the student clicks the "Try Again" button, the content is shuffled on the front end, to show a different combination of possible Items and answers without starting a new session. This is a form of dynamic content. This useful for formative assessments, where students are learning and reinforcing their knowledge. 

Turning on "Try Again"
Using "Try Again" requires setting up dynamic content, so this is your first step. Dynamic Content is set up in the authoring interface, and the method for doing this varies, depending on which Question type you are using. The first step is to set the dynamic_content initialization option to true (under config >item_edit). See the article What is Dynamic Content? for more information. 

Setting a seed value for "Try Again" options

By default, the order of shuffling is set by the response id. To configure the seed so that all students see the same order of shuffling, you can set it by adding the following to the init options when calling the API:

"dynamic_items": {
        "data_table_seed": "SOME_SEED", // optional

Code example 7: setting a seed value for dynamic content

 

Configuring options and adding the "Try Again" button

You must set some specific options and enable the button for "Try Again". See the knowledge base article on Using Dynamic Content and Try Again in your Assessments.

Turning off "Try Again"

You can simply remove the "Try Again" button, or roll back the configuration options. See the knowledge base article on Using Dynamic Content and Try Again in your Assessments.

Further reading

See the knowledge base article on Using Dynamic Content and Try Again in your Assessments.

 

6. Math Question Generator (Items API, back end)

You can also use the Math Question Generator to randomize content delivery. The Math Question Generator draws from a data table to create different combinations of stimulus Item values and possible answers shown on the screen. In this way, many different Item and answer combinations can be automatically generated, and shown to students.

Using the Math Question Generator
To create a Math Question Generator Question, in the Learnosity authoring interface, add a new Item and select the Math Question Generator tile from the math section in the tile view. The interactive wizard will guide you through the steps required.

For more information, see the knowledge base article on Using the Math Question Generator.

Further reading

See the knowledge base article on Using the Math Question Generator.

 

7. Adaptive assessment (Items API, back end)

In this scenario, the students are shown different Questions during the assessment, depending on whether they answer correctly or incorrectly at each Item. For example, if a student is answering correctly each time, the system will then show them a harder, or more difficult Item. If a student is answering incorrectly each time, the system will then show them an easier, or more fundamental Item. In this way, the test should adapt to the abilities of the student in order to give them a challenge that matches their knowledge level. These modes of operation are not random, but the journey that students take through the list of Items could vary widely, based on their answers.

Turning on adaptive assessment
There are four kinds of adaptive assessment:

  1. Item Branching.
  2. Item Adaptive.
  3. Testlet Adaptive.
  4. Self-hosted.

Each of these requires a different configuration. See the article Using Adaptive Assessments in Learnosity for more information.

Further reading

See the knowledge base article on Using Adaptive Assessments in Learnosity.

 

 

 

Learn more

 

Was this article helpful?

Related articles


Did you arrive here by accident? If so, learn more about Learnosity.