Scores Integrity Report

  Previous topic Next topic JavaScript is required for the print function Mail us feedback on this topic! Mail us feedback on this topic!  

 

ScoresIntegrity

Printed Scores Integrity report

 

The Scores Integrity Report should be run at least once a week, generally at the end of a research cycle. It can be run as part of the Weekly Wizard procedures, or directly from the Analysis / Scores Management menu.

 

IntegrityFilter

Scores Integrity report filter

 

When you run the Scores Integrity Report, you specify the start and end dates of the period you desire, as well as whether to tag incomplete respondents for callback in XX number of days.  You can also dictate whether they must have XX number of scores before they're tagged for callback.

 

When the report is run, if a respondent did not finish the test, ComQuest will automatically add a callback date of however many days after they last took the test that you indicate on this screen.  If you call them back within the current test week, they will only have to score the songs they didn't complete.  In other words, if you're testing 30 songs, and a respondent scored 25 of them, but then hung up because they got a call on the other line, they'll only have to score the 5 songs they missed the first time; not all 30 songs over again.  Generally, we recommend calling back incompletes in 1-2 days after they last took the test.  (If you set this value at 0, no callback date will be assigned to these incomplete respondents.  Zero, in this case, means off.)

 

Be sure that you want to call back all Take Test respondents before using this feature.  For example, some stations use screening montages before the actual song hook test.  Many respondents remain a Take Test simply because they did not qualify for the test, and as such never qualified to become a complete.  In this case, you would probably not want to call back Take Tests in a few days, but rather call them again in another month or so, if at all.

 

In addition to dictating how soon these incompletes get called back, you can also define how many songs they must have completed before you call them back.  Some people set this to 0, meaning all incompletes, no matter how many or few songs they scored, will get called back again in XX number of days.  Other people however, prefer to set this to a threshold such as 10 or 15.  In other words, don't bother calling someone back to finish the test if they only scored 1 or 2 songs; but if they scored 15 out of 30, then definitely call them back to finish.

 

HandWhen these respondents are tagged for callback, and they are displayed on the interviewer's Workstation in Make Calls at the appointed time, the script will indicate to the interviewer that this is a non-complete they're calling back, as well as the date/time they last took the test.

 

This will help prepare the interviewer to engage the respondent in such a way that they can be successfully persuaded to complete the music test this time.

 

HandIf you don't want to set call backs for any of the respondents, simply set both the days and scores to 0 (zero).

 

ComQuest will initially display the values for days and scores that were last set on this computer when the Scores Integrity Report was run (these values are stored locally on each computer; not globally in the data files, like most other ComQuest settings).

 

The actual Scores Integrity Report is broken into four sections.

 

The top section includes any respondents who started the test, and scored at least the first question or song you were testing, but didn't finish.  Their respondent ID number, as well as the number of songs they scored, the date they took the test, the interviewer that transferred them, the respondent's name and phone number and their status and tier are all displayed.  Their status will be something like Take Test/3, which means this is the 3rd time they took the test.  The number at the far right of each line in brackets, such as [1], indicates which tier of the test they took.  All stations have at least 1 tier; if you're testing music for multiple radio stations from the same database, you might have two, three or even four tiers on your system.

 

The second section of the report will include any respondents for whom there are no scores.  Especially in the first couple of weeks of using ComQuest, this might be the result of a bad transfer, where the interviewer actually hangs up on the respondent when attempting to flash the phone prior to transferring.  Generally, if there are no scores from a respondent, it's a result of being improperly transferred, although it is possible the respondent may have hung up or answered another call using Call Waiting before responding to even the first question or song.

 

The third section includes those respondents that have multiple test scores for at least one question or song during this research cycle.  This might be the result of an interviewer transferring a respondent to the Fileserver to take the test, and for some reason inadvertently calling them back and re-testing them again (perhaps because they mistakenly believed they didn't get transferred properly, etc.)  This situation should never occur, but just in case it does, it will be reflected in the middle section of the Scores Integrity report.

 

The fourth section details missing Perceptual Data for Completes, based on the current Perceptual Survey and Questions you are testing for this tier (each Quota Tier can have its own active Perceptual Survey).

 

IntegrityPerceptual

Missing Perceptual Data in Scores Integrity Report

 

HandTo reduce the likelihood of interviewers skipping over collecting answers for all Perceptual Questions, we recommend turning on the Override Setting that disables the Cancel/Quit buttons.

 

You can later delete any scores from incomplete respondents, as well as any duplicate scores from respondents, by using the Delete Incomplete Scores procedure under the Analysis/Scores Management menu.

 

HandThe number of songs and questions used by the Scores Integrity Report is derived from the number of active songs you currently have on your Song and Question Test Lists.

 

Thus, if you are testing 30 songs this week, but you tested 29 songs last week, you can not now reliably run a Scores Integrity report for last week.  All completes from last week would be represented as having incomplete tests, because their 29 scores from last week (which made them a complete last week), would fall short of the 30 scores required to be considered as a complete this week.  So, you can really only run the Scores Integrity Report for a current research cycle, unless you never change the number of songs and interactive questions you test each week. ComQuest doesn't keep a record of how many songs/questions and which ones you tested in prior weeks.  It is only able to calculate the number of songs/questions being currently tested.  Thus, running the Scores Integrity Report for a prior test week (after that test cycle has been completed), may produce unexpected results, especially if the number of songs and/or questions being tested has changed since that previous test cycle.

 

See also

Weekly Wizard

Delete Incomplete Scores

Override Settings