Submit a request
Submit a request

Configuring Item or Testlet Adaptive Assessments

{
    "adaptive": {
description

Type of adaptive behaviour.

In itemadaptive, items are selected based on responses to previous items.

In branching, activities (testlets) are selected instead of items.

The same algorithm is used to select either items or activities. The algorithm can be customized using the options on this page.

In the interests of simplicity this page discusses items only, but all the options apply to branching mode in the same way at the activity level.

type string
options itemadaptivebranching
mandatory parameters Either termination_criteria or sequence
    "type": "itemadaptive",
description

The initial ability measure for the user. This will be used as the target difficulty for the first item.

type float (logits)
default 0
    "initial_ability": 0,
description

The size of the range of difficulties from which an item should be randomly selected.

For example, if the target difficulty is 0 logits, a tolerance of 1 will select an item at random from all items with a difficulty between -0.5 and 0.5 logits.

If no items are found in that range, an item with difficulty between 0.5 and 1.5will be selected, followed by the range -1.5 to -0.5, and so on.

type float (logits)
default 1
minimum 0.1
    "item_difficulty_tolerance": 1,
description

An offset to apply to each difficulty target. A positive value will cause more difficult items to be selected.

type float (logits)
default 0
    "item_difficulty_offset": -0.3,
description

When an ability estimate for the user is not yet available, this value is added or subtracted (depending on the user's response) from the difficulty of the previous item to determine the difficulty target for the next item.

type float (logits)
default 0.7
    "max_difficulty_change": 0.7,
description

The minimum difficulty of all available items.

type float (logits)
default -9
    "min_item_difficulty": -9,
description

The maximum difficulty of all available items.

type float (logits)
default 9
    "max_item_difficulty": 9,
 

The following methods are available for calculating ability estimates. They can be used for interim, subscore, and final ability estimates.

mle Maximum-Likelihood Estimation - this method requires a mixed response pattern for the items seen i.e. at least one correct response and one incorrect response.
eap Expected A Posteriori (EAP) - this method requires eap to be configured (see below).
empirical-eap Empirical Expected A Posteriori (EAP) - as above, but the mean used for the calculation is the final ability estimate for the overall session. empirical_eap can optionally be configured (see below).
score-lookup Use the raw score for the session (the sum of correct items) to select an ability estimate from a lookup table - this method requires score_lookup to be configured (see below).
description

Ordered list of the preferred estimation methods to use during the session i.e. to determine the target difficulty for the next item.

type Array
default ["mle", "eap"]
    "interim_ability_estimation": ["mle", "eap"],
description

Ordered list of the preferred estimation methods to use for the final ability estimate calculated at the end of the session.

type Array
default ["mle", "score-lookup", "eap"]
    "final_ability_estimation": ["mle", "score-lookup", "eap"],
description

Ordered list of the preferred estimation methods to use for subscores.

type Array
default ["empirical-eap", "mle"]
    "subscore_ability_estimation": ["empirical-eap", "mle"],
description

Information used to calculate EAP ability estimates.

type Object
    "eap": {
        "mean": -0.25,
        "standard_deviation": 0.95,
        "theta_min": -3,
        "theta_max": 2.25,
        "num_points": 45
    },
description

Information used to calculate empirical-eap estimates. These values override those in eap.

type Object
    "empirical_eap": {
        "standard_deviation": 1
    },
description

The lookup table used for score-lookup estimates. min and max are mapped to the lowest and highest possible scores, respectively.

type Object
    "score_lookup": {
        "min": -3.5,
        "max": 3.5
    },
description

The criteria used to determine when to finish a user's session.

type Object
attributes By order of precedence:
  • min_items minimum number (> 1) of items to present. The session will not finish until at least that many items have been presented.
  • max_items maximum number (> 1) of items to present. The session will finish as soon as that many items have been presented.
  • error_below threshold on the standard error of measurement of the user's ability estimate. The session will finish when the estimate's standard error falls below this threshold. This value should be set in consultation with psychometricians calibrating the items' difficulty.
mandatory parameters Either error_below or max_items
    "termination_criteria": {
        "max_items": 45,
        "min_items": 20,
        "error_below": 0.65
    },
description

The tags from which items must be selected. Where multiple tags of the same type are specified, the algorithm will alternate between each tag.

By default, the tags will be distributed equally, but the weighting can be customised by providing an object (instead of a string) with the tag's name and desired weight.

Using this JSON example, all items in the session would be tagged with subject:Math.

Half of the items would be tagged with technology-enhanced-item:Yes, and the other half with technology-enhanced-item:No.

60% of the items would be tagged with chapter:1 - The Number System, and 40% would be tagged with chapter:2 - Statistics and Probability.

type array of Objects
    "required_tags": {
        "subject": [
            "Math"
        ],
        "technology-enhanced-item": [
            "Yes",
            "No"
        ],
        "chapter": [
            {
                "name": "1 - The Number System",
                "weight": 3
            },
            {
                "name": "2 - Statistics and Probability",
                "weight": 2
            }
        ]
    },
description

The tags from which items must not be selected.

type array of Objects
    "blacklist_tags": {
        "unit": [
            "2.7 - Conditional statements"
        ]
    },
description

Instructions for how seeding items should be indentified and included in the session for calibration purposes. These items are not expected to have difficulty measures, and responses to these items have no effect on the user's ability estimate for the session.

tagtype and tagname define the specific tag that identifies items for seeding.

weight defines the proportion of seed items that should be selected in the session - 0.1 or 10% in this example.

type Object
    "seeding": {
        "tagtype": "lifecycle",
        "tagname": "seeding",
        "weight": 0.1,
        "earliest_seed_item": 3,
        "latest_seed_item": 3 // i.e. 3rd last
    },
description

Items which should not be selected in the session. type can be hard for items that should never be selected, or soft for items that should only be selected if no similar items are available.

type Object
    "exclude_items": {
        "type": "hard",
        "references": [
            "item-A",
            "item-B",
            "item-C"
        ]
    }
description

Specific requirements for each individual item in the session, allowing full control of the profile of every item. The session will end when the end of this sequence is reached.

required_tags are the tags from which the item must be selected.

blacklist_tags are the tags from which the item must not be selected.

In the example to the right, the first item in the session must be from Unit 2.7, and must not be technology enhanced.

The second item must be tagged with lifecycle:seeding. As this tag matches the seeding tag defined above, it will be treated as a seed item i.e. it will not be expected to have a difficulty, and responses to it will not be used when calculating ability estimates.

type array of Objects
    "sequence": [
        {
            "required_tags": {
                "unit": [
                    "2.7 - Conditional statements"
                ]
            },
            "blacklist_tags": {
                "technology-enhanced": [
                    "Yes"
                ]
            }
        },
        {
            "required_tags": {
                "lifecycle": [
                    "seeding"
                ]
            }
        }
    ],


    }
}
Was this article helpful?
0 out of 0 found this helpful

Did you arrive here by accident? If so, learn more about Learnosity by clicking here.