Submit a request
Submit a request

Introducing Item Analysis Datasets via Data API

Learnosity's Item analysis reporting capabilities allow you to perform detailed analysis on your Item bank content. This can range from large scale analysis of your whole Item bank, and all of your sessions, to more focused analysis of a single Activity, or specific selection of sessions. Our solution aims to provide insight in the following areas:

  • Performance of individual Questions - difficulty (p-value) and discrimination index.
  • Identification of misleading or misconfigured Questions.
  • Usage of and engagement with Questions across an Item bank.

 

Learnosity's Item analysis is implemented in Data API using datasets. There are two different dataset types used for Item analysis:

  • ibk-analysis-by-question

    Item bank scale analysis across all Questions and sessions.

  • activity-analysis-by-question

    Analysis of a specific set of identical sessions. All sessions being analyzed must contain exactly the same Questions (in any order). This allows the overall session scores to be compared, which is required for certain measures e.g. discrimination index.

See the available fields for a list of Question-level and summary fields available in each dataset type.

 

For performance and scalability, Item analysis reports are generated asynchronously via Data API datasets. See the datasets implementation guide for details.

These are sample requests made to Data API's SET reports/datasets endpoint.

  • Calculate p_value for all Questions across all sessions, sorted by p_value.

     

    {
        "dataset_type": "ibk-analysis-by-question",
        "organisation_id": 1,
        "file_count": 0,
        "options": {
            "default_sort_field": "p_value",
            "default_sort_order": "asc",
            "fields": [
                "organisation_id",
                "item_reference",
                "question_number",
                "question_reference",
                "count_sessions",
                "count_attempted",
                "p_value",
                "stddev_p_value"
            ],
            "summary_fields": [
                "count_questions"
            ]
        }
    }

     

  • Calculate discrimination_index for Questions in sessions with a specific Activity ID, and within a date range.

     

    {
        "dataset_type": "activity-analysis-by-question",
        "filters": {
            "activity_id": ["ELA_comprehension"],
            "maxtime": "2017-07-31T23:59:59Z",
            "mintime": "2017-07-01T00:00:00Z"
        },
        "file_count": 0,
        "options": {
            "default_sort_field": "item_position",
            "default_sort_order": "asc",
            "fields": [
                "organisation_id",
                "item_position",
                "item_reference",
                "question_number",
                "question_reference",
                "count_sessions",
                "count_attempted",
                "p_value",
                "stddev_p_value",
                "discrimination_index"
            ],
            "summary_fields": [
                "count_questions",
                "count_sessions",
                "count_sessions_discarded",
                "count_sessions_analyzed"
            ]
        }
    }

     

  • Calculate discrimination_index for a specific list of sessions. The session IDs are uploaded in a separate data file, as indicated by file_count. See step 2 of the implementation guide for further details.

     

    {
        "dataset_type": "activity-analysis-by-question",
        "file_count": 1,
        "options": {
            "default_sort_field": "item_position",
            "default_sort_order": "asc",
            "fields": [
                "organisation_id",
                "item_position",
                "item_reference",
                "question_number",
                "question_reference",
                "count_attempted",
                "p_value",
                "stddev_p_value",
                "discrimination_index"
            ],
            "summary_fields": [
                "count_questions",
                "count_sessions",
                "count_sessions_discarded",
                "count_sessions_analyzed"
            ]
        }
    }
Was this article helpful?
2 out of 2 found this helpful

Did you arrive here by accident? If so, learn more about Learnosity by clicking here.