Inter-Rater Reliability Clinical Data Review



As part of our commitment to data validity and reliability, MSQC conducts inter-rater reliability (IRR) clinical data review at each participating hospital.

The IRR is an important tool to validate the reliability of the data collected which informs various processes across the MSQC. It provides an opportunity for collaborative learning and educational processes for both the MSQC site and Coordinating Center.
Each year MSQC will audit a pre-determined list of sites and their data.  MSQC will prioritize sites based upon if/when a previous IRR was completed and SCQR status. For new hospitals, following at least 6 months of data submission, MSQC may request an on-site audit of the data. MSQC reserves the right to delay, add or remove sites as needed.

In brief, the process for the IRR is as follows:


  • Complete this form to provide contact information for the MSQC Clinical Site Coordinators to be granted remote access to the site’s EMR for the IRR.
  • 2 weeks prior to the IRR, the site will be notified by the MSQC Coordinating Center of when the IRR will begin at their site along with the 15 case numbers that have been chosen at random for potential audit.
  • A minimum of 4 out of the 15 cases will be reviewed by at least two MSQC Clinical Site Coordinators.
  • Following the review, the Coordinating Center will return IRR results and summary to the site for review.
  • After the results and summary are reviewed by the site, a mutually convenient date and time for a web-conference or site visit will be held to allow for discussion of the IRR results between the site and the MSQC Coordinating Center.
  • Following this discussion, a final report will be forwarded to the SCQR, Supervisor, and Surgeon Champion at the site.

The following sites are expected to undergo Inter-Rater Reliability Clinical Data Review in 2017:


Beaumont Hospital - Taylor
Beaumont Hospital - Trenton
Dickinson County Healthcare System
Garden City Hospital
Hillsdale Hospital
Holland Hospital
Hurley Medical Center
Huron Medical Center
McLaren - Flint
Memorial Healthcare - Owosso
Mercy Health – Hackley
MidMichigan Medical Center - Alpena
MidMichigan Medical Center - Clare                               
MidMichigan Medical Center - Gratiot
MidMichigan Medical Center - Midland
Munson HealthCare Grayling Hospital 
Oaklawn Hospital
ProMedica Monroe Regional Hospital
Sparrow - Carson Hospital
Spectrum Health Big Rapids Hospital
Spectrum Health United Hospital
St. John River District Hospital
St. Joseph Mercy Chelsea
St. Joseph Mercy Livingston
West Branch Regional Medical Center


Linked below are a number of resources to help you prepare for your IRR. Please contact the MSQC Clinical Support Team if you have any questions or concerns.