The Center for Research & Innovation in Patient Care

Robert Wood Johnson Rapid Response Team Initiative Evaluation Project

Nancy E. Donaldson RN, DNSc., Principal Investigator

Introduction

Robert Wood Johnson Foundation has launched an ambitious initiative that provides support to nine health care systems/associations to establish learning networks to assist and accelerate the efforts of their member hospitals to implement and evaluate the impact of rapid response team (RRTs) interventions with the aim of improving patient outcomes and the work environment of nurses. The literature suggests that RRTs mobilize clinical expert responders to manage emergent changes in patient condition which are potentially life threatening and may be related to clinician or system errors and omissions.

RRTs, also known as Medical Emergency Teams (METs), have been adopted as one of the IHI 100,000 Lives Campaign innovations based on preliminary evidence that they are a low risk, high benefit intervention that engages professionals in early rescue of patients when direct care providers note triggers in patient status that suggest possible deteriorating, perhaps life threatening, condition changes. Several published reports note that nurses, in the vast majority of instances, sound the RRT alarm, summoning the team to the patient’s bedside, thus it is posited that the implementation of RRTs may ease the burden of nurse confronting worsening patient condition. It is also possible that variation in the deployment and expertise of nurses at the bedside may influence the effectiveness of the RRT, for example, by delaying the RRT call. Retrospective chart reviews have revealed a pattern in which one or more signs of patient deterioration were present by not apparently noted by nurses prior to cardiac or pulmonary arrest, two common emergent conditions the RRT is intended to reduce.

In advancing RRTs as a patient safety innovation ready for implementation, IHI proposed three core measures and offered a preliminary operational measurement approach. In responding to the RWJ RFP, successful grantees (8 out of 9) included the IHI measures in their evaluation approach, although the specifications for the measures were not fully presented and may vary widely across grantees and their hospital sites.

The UCSF RWJ RRT Initiative Evaluation Project is designed to conduct a formal evaluation of the RWJ RRT implementation impacts by working with grantees to standardize their measures, develop coding guidelines for data capture, provide training for data collectors/coders, as well as aggregate and analyze data submitted by grantees to trace the impacts of the RRT Initiative at baseline, mid point and final points in time. In addition, we will integrate qualitative measures which explore important contextual factors related to “best practices” in RRT impacts and factors that emerge as significant barriers to progress. Finally, we will work with one or more grantees who have access to nurse staffing data on the RRT pilot test units, and engage them in a descriptive analysis that examines the association between key staff variables (hours of care, skill mix, ratios, use of contract/agency workers, years of experience) to permit exploration of the potential contribution of these variables to understanding variation in the observed impacts of RRTs on patient outcomes.

Project Principal Objectives and Target Audience

The principal objectives of this two year, descriptive evaluation project are to, in collaboration with RWJ Project Staff and Grantees:

1. Establish standardized metrics that build on IHI and Grantee proposed measures to establish a matrix of baseline, formative and summative measures that are feasible to collect and aggregate across sites and provide insight into the impacts of RRT implementation overtime in the diverse RWJ RRT grantee hospital settings.
2. .Identify, explore, validate and describe contextual factors, including variation in nurse staffing, that are associated with comparatively “best” implementation of RRTs, and factors that emerge as barriers to effective RRT implementation.
3. Develop and deploy educational and consultative assistance to grantees to build their capacity for evaluation data capture and optimize the integrity of the evaluation data they submit.
4. Conduct baseline, formative and summative point-in-time aggregation and analysis of “core” quantitative variables.

The UCSF Team expects to work in a highly collaborative relationship with grantees, to optimize their evaluation data capture and participation in the evaluation enterprise. It is our goal to provide evaluation data to the Foundation, RRT grantees and the “community-at-large” that advances understanding of RRT impacts, outcomes and effective implementation.

Project Rationale and Approach

The proposed evaluation project is especially timely. In addition to IHI and RWJ, the Gordon and Betty Moore Foundation is supporting efforts to accelerate development and implementation of RRTs in Bay Area hospitals in California. These efforts cannot, however, rely on a robust knowledge base with well developed measures. Metrics have varied widely across studies and much of the foundational work has been studies in the United Kingdom and Australia. The literature suggests much of the research related to RRT/MET implementation is preliminary. As a result, there is not consensus on measures, rates and consensus on standards for judging the effectiveness of RRTs has not been achieved.

Our proposed approach builds on the preliminary evaluation approaches proposed by RWJ RRT grantees summarized in Table 1. Proposed Grantee RRT Learning Network interventions vary widely in their content, contacts and delivery methods. While a pattern of proposed evaluation measures is noted, grantees varied in their metric foci and specifications. It is likely that the educational and supportive interventions proposed will vary greatly in their “effect power” and that the ultimately impacts over the 18 months of the project may be preliminary. We therefore envision an approach that is iterative and integrates valid and reliable hospital level core metrics that are standardized, from the National Registry for Cardiopulmonary Resuscitation (NRCPR), including CPR immediate and final survival rates (more sensitive than gross measures of mortality). We will develop additional metrics to describe the number of RRT calls. Working with grantees, we will then identify contextual factors which may confound outcomes, for example what percent of RRT calls result in Do Not Resuscitate (DNR) action, which may be more an indicator of how advanced directives and DNR processes are managed in that setting, than RRT impacts. In collaboration with grantees and gleaned from a sample of selected site visits, we expect to explore contextual factors that will contribute to interpreting the implementation outcomes, including how variation in the deployment of the nursing workforce may impact RRT calls and patient survival. We also expect to work with one or more grantees to systematically examine how standardized measures of nurse staffing are associated with RRT clinical outcomes. Nurse staffing measures will be standardized based on the measures now used by the California Nursing Outcomes Coalition, which are consistent with the National Database for Nursing Quality and the VA and Military nursing outcomes datasets.

To optimize the integrity of the data collection process within grantee systems and across grantees, we will create and distribute evaluation data collection guidelines and tools, as well as provide grantees with conference call and web-based training and technical support as they begin baseline data collection. Using email and conference calls, grantees will have opportunities to receive supportive technical assistance from the UCSF Team and forge connections with one another that may enhance their efforts.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -