Tyrell Corporation provides a service for the assessment of English language proficiency. They offer branded testing services to universities. These services are used to test a university applicant's ability to speak English as part of their application process.

The existing version of Tyrell's assessment system uses auto-scored multiple-choice and fill-in-the-blanks questions. In the next iteration, they want to introduce audio and essays as they built a proprietary essay scoring system and established a partnership with a third-party company that specializes in analysing and understanding audio. They would like to:

1. Have a single authoring environment where they can create all three groups of Questions; multiple-choice, essay and audio
2. Present these Questions in a white-labelled assessment player which they can personalize for each university
3. Have a robust and reliable cross-platform means of capturing audio
4. Auto-grade the multiple-choice and fill-in-the-blanks Question types
5. Have a means of storing all grades in one place so they can generate a single unified report

Tyrell can meet all of their requirements using the Learnosity APIs and hosted services. We will walk through an example where Tyrell has used:

Author Site

• This Learnosity-hosted authoring environment will allow Tyrell to author Questions in a user-friendly instinctual manner
• Requires zero development time, just log in and start creating Items straight away
• Provides the ability to author 65 different Question types including essay and audio
• Provides the ability to create Questions where the stimulus and response are both audio
• Provides the ability to create timed guided workflows with instructions presented as pre-recorded auto-playing audio
• Saves all content to Tyrell’s Learnosity-hosted Item bank

Items API

• Used to present the assessment to the student
• The Items API pulls Items from Tyrell's Learnosity-hosted Item bank
• Along with CSS styling and other customisations, will render a branded version of the assessment in the client's web page
• Saves the student's work, and submits it to the Learnosity servers for scoring

Data API

• A server to server API which gives the client access to all of their raw data
• Used to push scores from the Audio and Essay questions in to Learnosity servers to facilitate unified reporting

Reports API

• Automatically builds a series of standard report types based on the results of Tyrell’s auto-scored assessments
• Reports will be augmented with data from the Audio and Essay Question types
• Embed these reports in Tyrell's system with just a few lines of code

Authoring of content (Items) can be done using the Learnosity Author Site. This requires no development work whatsoever. Users simply log in and start creating Items.

Tyrell staff can create Items using 65 different Question types. They can then tag those Items. Tagging facilitates complex actionable reporting and data analysis. All of this Item data is stored in Tyrell's Learnosity-hosted Item bank. Note that all Question types except Audio and Essay are auto-scored by Learnosity.

Authors can create Items using audio as the stimulus and the response, this is very useful for language training. The student will hear a Question, will be given a predefined amount of time to think about it and will then be asked to record their answer. An automated, timed workflow can be created to guide the student through the assessment.

The authoring tools can be used to create a series of ‘Activities’, an Activity is a sequence of Items that are grouped together for the purpose of presenting them to the students as an assessment. Tyrell will create three Activities, one for each of the three sections to their exam. All of this is handled by Learnosity, Tyrell only need to store references to those Activities in their database.

 

This completes Tyrell Corporation's authoring process.

The assessment experience is presented on a website hosted on Tyrell's servers. Each client gets their own subdomain, such as nyu.tyrell.com. On this website, university applicants can log in and take the tests. The assessment is handled by the Learnosity Items API.

User management and user authentication is handled by Tyrell. When the logged-in users are ready to begin their assessment they will click a link which will take them to a web page where the Learnosity Items API displays the appropriate Activity for them. It will take the ID of the Activity and the ID of the logged in user as two of its parameters. No personally identifying information is used.

As the student works their way through the three Activities, their student responses are saved by Learnosity. The data, including essay answers, are stored in the Learnosity Student Response database, while mp3 audio files are stored in one of our Amazon S3 storage buckets.

The User ID, Session ID and Activity ID can later be used to generate a range of reports for individuals or groups.

Now it is time to consider the integration of Tyrell's in-house essay scoring and 3rd-party audio scoring. This is done in a back-end processing system that Tyrell will code as part of this integration project. The Learnosity Data API will be used to retrieve the student's essay responses from the Learnosity Student Response Database, these will be scored in Tyrell's system and the scores will be pushed back to Learnosity so they can be used in reports.

The Learnosity Data API is also used to retrieve the URLs of the MP3 audio recordings of the students responses and pass these to the third party audio scoring service. When scored they are sent back to Tyrell and then are written to the Learnosity database, again using the Data API.

Let's recap what has been created so far. Tyrell has used Learnosity's Author Site to create and store all the Items required for their language assessments. They just have to store the Activity References on their side. When it's time to present those activities to their users, they just need to pass a few relevant IDs from their side along with the Activity Reference for Learnosity to render the assessment experience.

When all three parts of the assessment are completed a Tyrell back-end process will find scores for the essay type Questions and the audio Questions and will then write those scores into the Learnosity Student Response Database using our Data API.

 

The IDs referred to above are the User ID, Session ID and Activity ID which were described earlier.

 

Finally, we will deal with the analytics part. Tyrell currently provides a simple report showing the percentage grade for the multiple choice assessment. They want to maintain this and have it include all of the essay types and audio Questions. This can be done with Learnosity's Reports API. Like the current report this will be made available via a web page on the Tyrell system.

The IDs which Tyrell saved during the Assessment experience, can now be passed to the Reports API.

 

The IDs referred to above are the User ID, Session ID and Activity ID which were described earlier.

 

 

The User ID can be used to get reports on individual students, while the Session and Activity IDs identify different sets of Activities taken.

 

In this final diagram we see the entire system:

By using this workflow, Tyrell Corporation was able to create 65 different Question types including audio workflows, render them as a branded assessment for each client, integrate third-party scoring and produce detailed unified reporting.

Here are the steps Tyrell needs to follow to integrate this plan:

1. Create the Items needed for their assessments. Assemble these into three ‘Activities’. Learnosity Authoring team will provide training for this
2. Add a web page where their logged-in users can take an assessment. On this page initialize the Items API using the required IDs, which includes User ID and Activity ID from the Tyrell database
3. Build a backend processing system to perform these 4 steps:
   • Use the Data API to retrieve Student Essays from Learnosity
   • Score the essays using their in-house scoring engine and write the score back to Learnosity
   • Use the Data API to retrieve Student MP3 recordings from Learnosity
   • Score the MP3s using their third-party scoring engine and write the score back in to Learnosity
4. Add web pages where the teacher can see reports.

Return to the index of technical solutions.