IRB Performance & Metrics
The PAR program supports continuous quality improvement through evaluation, assessment, and action.
PAR quality improvement activities associated with the Institutional Review Board (IRB) performance are conducted in accordance with key program initiatives and Association for the Accreditation of Human Research Protection Programs (AAHRPP) standard I-5.
Standard I-5: The organization measures and improves, when necessary, compliance with organizational policies and procedures and applicable laws, regulations, codes, and guidance. The organization also measures and improves, when necessary, the quality, effectiveness, and efficiency of the Human Research Protection Program.
- Active Studies
- Monthly Submissions
- IRB Review
- Turnaround Time
- Median IRB Duration
- Research Community Feedback
- Reliance Agreements
The University of Minnesota Human Research Protection Program (HRPP) publishes metrics related to IRB activities. These metrics are an important element of tracking and monitoring IRB performance and implementing quality improvement initiatives.
Many program improvements are in progress, including implementation of the HRPP Toolkit and the Ethical Oversight Submission System (ETHOS), an electronic IRB submission system. The metrics provided below will be used to measure and report progress toward our goals of enhancing the HRPP program and improving the IRB experience for researchers.
As of January 1, 2018, the IRB has 4,190 active [non-exempt] studies in its records:
While the number of active studies in IRB records is a helpful measure, it does not account for the multiple submissions often associated with a single active IRB study.
Total IRB submission volumes is another useful metric, as it accounts for all categories of submissions associated with a study and enables HRPP staff to evaluate and track the IRB’s total workload on a continual basis. IRB submissions can be divided into the following categories:
- New studies (NEW STUDY)
- Modifications to an existing study (MOD)
- Continuing review (CR)
- Minor modifications - add/change personnel (MINOR MOD)
- Reportable new information (RNI)
- Continuing Review with Modification (MOD CR)
Notes about submission volume metrics and transition from RSPPi to ETHOS:
- Average monthly submission volumes include only the original submission of an item to the IRB. These figures do not include investigator responses to stipulations, responses to deferral, or responses to pre-review clarifications requested. The IRB typically receives an additional 200–300 of these investigator responses each month.
- In ETHOS, investigators no longer categorize submissions as “Exempt” or “Determination Form”. These designations are assigned by IRB staff at the time of the review determination.
- In RSPPi, MODS that included a personnel change were counted as two separate submissions. In ETHOS, a MOD that includes a personnel change is counted as one submission.
- MOD/CR is a new submission type that did not exist in RSPPi. In RSPPi, investigators would have submitted two separate items.
Most submissions to the IRB are reviewed via “Committee Review” or “Non-Committee Review” procedures. A smaller percentage of submissions are reviewed via an external IRB or are determined to be “not human research”. A brief description of each review category is provided below.
- Full Committee Review: Greater than minimal risk study. Review is conducted by the full board at a convened IRB meeting.
- Exempt Review: Meets categories of exemption from IRB review. Review is conducted by experienced HRPP staff.
- Expedited Review: Not greater than minimal risk study. Review is conducted by at least one IRB committee member.
- An IRB external to the University of Minnesota conducts the regulatory review and oversight of a study.
Not Human Research Determinations
- The IRB determines activities do not meet the regulatory definition of “Human Research” under any of the following:
- U.S. Department of Health and Human Services (HHS) Regulations
- U.S. Food & Drug Administration (FDA) Regulations
HRPP has adopted a number of new and enhanced approaches to the review of human research. Key milestones and activities of the “IRB Renew Project” are illustrated in the graphic below. For more detailed information about the HRPP’s current initiatives, read the Research Community Newsletter.
Full migration to the new online submission system, ETHOS, will be completed in fall 2018. While we anticipate steady improvement in IRB turnaround times, we expect to realize the full benefit of these changes only after full migration to ETHOS.
In the meantime, we may experience slightly increased turnaround times as all users are adapting to the new system and processes. HRPP staff will monitor turnaround times and feedback from the research community to identify and address potential sources of delay as quickly as possible.
Using data available from ETHOS, the following metrics for IRB turnaround times are provided:
- Median number of calendar days from entered IRB (e.g., submission) to exempt determination.
- Median number of calendar days from pre-review complete to IRB final determination or final approval recorded, including the percentage of time all studies in the review category were with the IRB versus not with the IRB.
- This turnaround time measure, starting at pre-review complete, is consistent with turnaround time metrics guidelines and definitions published by a) AAHRPP and b) the National Center for Advancing Translational Sciences (NCATS) Common Metrics Initiative (IRB Duration metric).
- Median number of calendar days from entered IRB (e.g., submitted) to IRB final determination or final approval recorded, including the percentage of time all studies in the review category were with the IRB versus not with the IRB.
The National Center for Advancing Translational Sciences (NCATS) initiated the Common Metrics Initiative as part of its effort to accelerate and improve clinical and translational research. The NCATS Common Metrics Initiative uses a specified set of metrics as the basis for collaborative strategic management around key topic areas. One of the topic areas is median IRB duration.
HRPP has been collaborating with the University of Minnesota Clinical and Translational Science Institute (CTSI) on data collection and strategic management around median IRB duration. The chart below shows the University of Minnesota’s median IRB duration, as reported to NCATS for the Common Metrics Initiative.
Key assumptions for the NCATS median IRB duration metric:
- Includes only studies reviewed by the convened IRB (“full committee review”); excludes studies reviewed via non-committee review procedures (expedited review or exempt);
- Includes only initial study submissions; excludes modification (MOD) and renewal (CR) submissions;
- Includes biomedical and social behavioral protocols;
- Includes only studies that received IRB final approval during the calendar year noted;
- Median IRB duration (as defined in the CTSA Program Common Metrics Operational Guideline for IRB Duration) is calculated as the median number of calendar days between the following time points:
- Receipt date (START): The receipt date is the actual date that the IRB office initially received an application for IRB review in their office or in an electronic inbox for review. For submissions to IRBs that perform triage or pre-review, the receipt date is the date the protocol is received back at the IRB office subsequent to such triage or pre-review;
- Final Approval Date (END): The final approval date is the date that the IRB determined that the protocol was approved with no IRB-related contingencies or stipulations remaining, so that, from the perspective of human subjects, the research can commence.
The IRB asks researchers to provide feedback about their experience at the time they receive final approval for a new study. The IRB Research Community Feedback Survey provides HRPP staff with valuable information about ways in which we can improve our performance and better serve the University research community. The information below summarizes recent feedback we have received from researchers via the survey.
IRB reliance agreements refer to the use of an IRB external to the University of Minnesota to conduct the regulatory review and oversight of a study. HRPP is responsible for negotiating and coordinating the execution of IRB reliance agreements.
The figures below provide an overview of recent reliance agreement activity and the current scope of University research ceded to an external IRB.
Note: Updated IRB reliance data is not yet available for all metrics. New reporting in ETHOS under development.