Inter-Rater Reliability Clinical Data Review



As part of our commitment to data validity and reliability, MSQC conducts inter-rater reliability (IRR) clinical data review at each participating hospital.

The IRR is an important tool to validate the reliability of the data collected which informs various processes across the MSQC. It provides an opportunity for collaborative learning and educational processes for both the MSQC site and Coordinating Center.
Each year MSQC will audit a pre-determined list of sites and their data.  MSQC will prioritize sites based upon if/when a previous IRR was completed and SCQR status. For new SCQRs, following at least 6 months of data submission, MSQC may request an on-site or remote audit of the data. MSQC reserves the right to delay, add or remove sites as needed. 

In brief, the process for the IRR is as follows: 

  • If requested, complete this form to provide contact information for the MSQC Clinical Site Coordinators to be granted remote access to the site’s EMR for the IRR.
  • 2 weeks prior to the IRR, the site will be notified by the MSQC Coordinating Center of when the IRR will begin at their site along with the 15 case numbers that have been chosen at random for potential audit.
  • A minimum of 4 out of the 15 cases will be reviewed by at least two MSQC Clinical Site Coordinators.
  • Approximately 2 weeks following the review, the Coordinating Center will return IRR results and summary to the site for review.
  • After the results and summary are reviewed by the site, a mutually convenient date and time for a web-conference or site visit may be held to allow for discussion of the IRR results between the site and the MSQC Coordinating Center.
  • Following this discussion, a final report will be forwarded to the SCQR, Supervisor, and Surgeon Champion at the site.

The following sites are expected to undergo Inter-Rater Reliability Clinical Data Review in 2018: 

Ascension Crittenton Hospital
Bronson Battle Creek Hospital
Bronson Methodist Hospital
Henry Ford Allegiance Health
Henry Ford Hospital
Henry Ford Macomb Hospital
Henry Ford West Bloomfield Hospital
Henry Ford Wyandotte Hospital
McLaren - Central Michigan
McLaren - Flint
McLaren - Lapeer Region
McLaren - Macomb

McLaren - Northern Michigan
McLaren - Oakland
McLaren - Port Huron
Mercy Health Saint Mary's
Metro Health Hospital
Michigan Medicine
Munson HealthCare Cadillac Hospital
Munson Medical Center
Sparrow Carson Hospital
Sparrow Hospital
St. Mary Mercy Hospital
St. Mary's of Michigan

The following sites are expected to undergo Inter-Rater Reliability Clinical Data Review in 2019: 

Ascension Borgess Hospital
Ascension Genesys Hospital
Ascension Macomb Oakland Hospital
Ascension Providence Hospital - Southfield and Novi
Ascension River District Hospital
Ascension St. John Hospital
Beaumont Hospital - Dearborn
Beaumont Hospital - Farmington Hills
Beaumont Hospital - Grosse Pointe
Beaumont Hospital - Royal Oak
Beaumont Hospital - Troy
Beaumont Hospital - Wayne


Covenant HealthCare
Detroit Receiving Hospital
Harper University Hospital
Huron Valley Sinai Hospital
Lake Huron Medical Center
Lakeland Healthcare
McLaren - Bay Region
ProMedica Bixby Hospital
Sinai-Grace Hospital
Spectrum Health Butterworth Hospital
St. Joseph Mercy Ann Arbor
St. Joseph Mercy Oakland
UP Health System - Marquette 


Linked below are a number of resources to help you prepare for your IRR. Please contact the MSQC Clinical Support Team if you have any questions or concerns.