DOT and NHTSA Logo

Methods


The Methods part of the report is divided into four distinct sections. The first three sections describe the three components of the STAF intervention: Standards of Care, Training and Feedback. In the fourth section, we describe the methods used for the project evaluation.

I. STANDARDS OF CARE

Development

For the standards of care, the main project staff (Principal Investigator and Project Coordinator) drafted trauma care guidelines. Two different sets of guidelines, one for prehospital personnel and one for ED personnel, were drafted. Specific deficits in care identified by the 1992 RPMS were addressed by these guidelines.

In drafting the guidelines, we made sure we adhered to principles in the following references: 1) Advanced Trauma Life Support Program for Doctors (American College of Surgeons, 1997); 2) Basic Trauma Life Support for Paramedics and Advanced EMS Providers (Campbell, 1995); 3) Guidelines for the Management of Severe Head Trauma (Brain Trauma Foundation, 1995); and 4) Trauma Nurse Core Course Instructor Manual (Emergency Nurses Association, 1995).

Approval

Initial drafts of the guidelines were presented to the Eastern Regional Trauma Coalition for feedback. This Coalition (now called the Eastern Regional Advisory Council) is composed of hospital administrators, trauma surgeons, emergency physicians, prehospital personnel, and emergency department personnel representing the 29 county referral region. Feedback from this group was incorporated into the final trauma care guidelines, which the Coalition approved in May 1997.

The guidelines approved by the Coalition and used as the Standards for the project are included in Appendix I.

Dissemination

Once the guidelines were approved by the Coalition, copies of the guidelines for both prehospital and emergency department personnel were mailed to medical directors (18), emergency medical services (EMS) directors (30), and EMS squads (141) in the region.

The REACT Coordinator met with the ED managers of the 20 hospitals in the region and members of EastCare critical care transport service. (EastCare is a service of Pitt County Memorial Hospital (PCMH), the hospital that houses the trauma center at which the REACT project was based.) The purpose of these meetings was to explain the project in detail and deliver copies of both sets of guidelines. These meetings were conducted from April through June 1997.

II. TRAINING

The Coalition-approved trauma care guidelines provided the foundation for the training of prehospital providers and ED personnel. EMS educators with EMS experience were hired to assist the REACT Coordinator with providing education to prehospital personnel.

Training Materials

The REACT Coordinator and the main EMS educator developed training materials based on the Coalition-approved trauma care guidelines. Materials included background information from the 1992 RPMS, a description of the REACT project, copies of the trauma care guidelines, and trauma case scenarios to use in practice.

Training Sessions for Prehospital Providers

In an attempt to reach as many rescue squads and EMS personnel as possible, contact was made with the individual responsible for continuing education for county EMS agencies as well as squad captains in each of 29 counties in the region for the project. The 29 North Carolina counties for the project included Bertie, Beaufort, Camden, Carteret, Chowan, Craven, Currituck, Dare, Duplin, Edgecombe, Gates, Greene, Halifax, Hertford, Hyde, Jones, Lenoir, Martin, Nash, Northhampton, Onslow, Pamlico, Pasquotank, Perquimons, Pitt, Tyrrell, Washington, Wayne and Wilson.

The majority of the educational sessions were conducted at squad meetings since we found that continuing education meetings were unavailable because they had been scheduled two years in advance.

We established agreements with continuing education personnel or committees in each county so that continuing education hours could be awarded to squad members who attended REACT training sessions. During the sessions, laminated copies of the guidelines were passed out to squad members. Seventy-six training sessions were conducted in the 29 county region between July 1997 and June 1998. A total of 700 EMS personnel attended these training sessions.

Training of Hospital ED Providers

The REACT Coordinator carried out educational training sessions for the ED staff from regional hospitals at their staff meetings between July 1997 and June 1998. The REACT Coordinator offered to attend staff meetings at each hospital to conduct these sessions. However, it was impossible for her to attend every hospital ED staff meeting. When this was the case, ED managers presented the material after being briefed by the REACT Coordinator. The same material presented at EMS sessions was presented at the hospital ED sessions but with greater emphasis on the ED guidelines.

The REACT Coordinator and EMS educators also conducted four educational sessions for the EastCare critical care transport service. At these sessions, they presented the same material presented at the ED training sessions.

Since the trauma care principles from the Trauma Nurse Core Course were used in the development of the trauma care guidelines, the REACT project also sponsored the attendance of 20 ED staff nurses from regional hospitals at a Trauma Nurse Core Course held at PCMH. This course was developed by the Emergency Nurses Association to educate ED staff in the basics of trauma care.

Regional Workshops

In addition to training sessions, two regional workshops were held for EMS and ED personnel. At the first workshop, attended by 60 persons from throughout the region, the REACT project was discussed, guidelines were distributed, and education based on the guidelines was presented. The second workshop, attended by 90 persons, consisted of presentations of actual patient scenarios from the region and discussion of the appropriateness of the patient’s trauma care. (See Appendix II for copies of the Workshop Brochures.)

Newsletters

Quarterly newsletters were mailed to all EMS squads, medical directors, and EDs in the referral region. The purpose of the newsletter was to keep all project participants informed of the progress of the project. Educational articles were also included in the newsletters. Attempts were made to involve EMS personnel and hospitals in the writing of the newsletters by asking for interesting cases and by asking ED managers to submit a brief article on their hospital. (See Appendix III for copies of Quarterly Newsletters)

III. FEEDBACK

The intent of the feedback component of the STAF model was to provide information on adherence to the trauma care guidelines. Providing feedback was the most challenging portion of the REACT project. In order to provide feedback, data collection forms based on the trauma guidelines were developed and disseminated.

Data Collection Forms

Two data collection forms were developed: a form for the evaluation of prehospital care and one to evaluate care at EDs that referred patients to the PCMH Trauma Center. The ED staff at participating regional hospitals completed the prehospital form. For patients transferred to our trauma center, either the EastCare transport personnel or PCMH ED staff completed the form.

Both forms listed interventions based on the trauma care guidelines and provided space for the data collector to indicate whether the intervention was needed and done; needed but not done; or not needed. There was also space for comments. The prehospital form also requested information on the level of certification of the EMS providers. On the ED care form, additional information included any other interventions done for the patient prior to transfer. On the back of the forms we placed instructions for use of the form and, on the prehospital form, criteria for selecting patients for data collection. The criteria were taken from Advanced Trauma Life Support Program for Doctors (American College of Surgeons, 1997). Copies of the prehospital and ED Data Collection Forms are included in Appendix IV.

Form Approval

These data collection forms were presented to the Eastern Regional Trauma Coalition for their feedback and approval. Feedback from this group was incorporated into the final data collection forms, which the Coalition approved in May 1997.

Form Dissemination

Data collection forms for both prehospital and ED care were included in the initial mailings of the guidelines to the medical directors, EMS directors, and prehospital providers. The data collection forms (prehospital and ED) were shared at the initial meetings with the ED managers. A contact person was designated by each institution and the EastCare transport service to ensure completion of the data forms and to mail or fax completed forms to the REACT coordinator.

Providing Feedback

Emergency departments at each of the referring hospitals designated an individual to receive feedback based on the data collected. In most cases, the nurse manager received both the prehospital and ED feedback. The ED manager shared feedback on prehospital care with the appropriate EMS personnel.

The initial plan was to provide feedback by telephone to regional hospitals within 72 hours of patient admission to the trauma center. The REACT coordinator attempted to meet this goal initially; however, difficulties were frequently encountered in reaching the contact person at referring hospitals by telephone within this time frame. Additionally, data forms often were not faxed or mailed to the REACT coordinator until several days after patient presentation. Many referring hospitals found it easier to have one person collect the data during chart reviews and mail or fax the information in bulk. The EastCare transport staff also found this method easier and usually provided data collection forms once per month.

The REACT coordinator reviewed completed data collection forms. Any major issues identified were discussed with the principal investigator and attempts were made to contact the appropriate hospital by telephone. Feedback on routine matters was provided in writing to each of the 20 referring EDs each quarter of the intervention year. The REACT coordinator met with the contact person at each of the 20 institutions at least once (in addition to the initial visits) during the intervention phase to provide as well as receive feedback. Information on compliance with the trauma care guidelines was provided to each hospital using percentages (percent needed and done; percent needed but not done; and percent not needed) for each item on the data collection forms. Hospital-specific data for prehospital care and ED care was provided. Information on patient outcome data was also provided to the hospitals. In addition, regional data for each point on the prehospital and ED forms were included to aid referring facilities in comparing themselves to the region.

Attempts were made to provide feedback directly to prehospital personnel on prehospital care; however, numerous obstacles were encountered. Often the names of specific squads were omitted from the prehospital forms. In many cases, volunteers staffed the squads and there was no one to receive the feedback by telephone. Specifically, the project coordinator placed 10 calls to prehospital providers to provide feedback and in almost every case a message was left and the call was not returned.

The project coordinator provided written feedback via letter to several squads on the care they delivered in the same format used for sending feedback to the regional hospitals. About 25 letters were sent to local rescue squads throughout the region with feedback. However, some of the squad members indicated they would prefer to receive feedback in person rather than by letter. At that point a decision was made to provide general feedback during training classes. Feedback was delivered during the remaining 45 training classes that took place from November 1997 through June 1998.

IV. EVALUATION

The Evaluation section begins with a brief description of our method for evaluating compliance with the trauma care guidelines. That is followed by a description of the preventable mortality study. The data for the methods described below are provided in the Results section of the Report.

Guideline Compliance

As described in the feedback section, during the intervention year we asked participating hospitals to complete data collection forms on prehospital trauma patients. In addition, personnel of the PCMH trauma center and the EastCare transport service completed the data collection forms for “scene run” and transferred trauma patients.

Preventable Mortality Study (PMS)

In order to evaluate the effectiveness of the STAF model in improving the trauma preventable mortality rate, a rural preventable mortality study was conducted. To allow comparison of the results to the prior PMS, the 1992 PMS methods were replicated (Cunningham, 1995).

Selection of PMS Patients

The Office of the Chief Medical Examiner (OCME) for North Carolina was asked to provide a list of trauma deaths for the region during year two (intervention year) of the project using the same criteria as the 1992 study. The main criteria consisted of a list of external cause of injury codes (e-codes) for mechanical trauma deaths. As in the study of 1992 deaths, we also excluded suicides (since they are almost always gunshot wounds and result in immediate death) and deaths which occurred at the scene prior to any medical care. A list of the e-codes used for patient selection is in Appendix V. The 29 North Carolina counties for the PMS were the same as for the project (see training section above). The names of the North Carolina counties for the PMS were provided to the OCME.

The goal was to have 150 deaths to study; 75 from each half of the intervention year. A total of 134 records were obtained for review using the following process:

The list of deaths from the OCME was examined to ensure that the patients met inclusion and exclusion criteria. The final working OCME list was comprised of 104 names from July 1-December 31, 1997 and 66 names from January 1 - June 30,1998. Seventy-five records were selected at random from the 1997 list for initial review. Medical records personnel at each hospital were contacted and asked to provide copies of the medical records for patients seen at their institution. In many cases, the records were unavailable or were incomplete and could not be used. Once the list of 75 records was exhausted, the working list was again consulted and the decision was made to request the remainder of the records on the list. Seventy three complete records were obtained for review from 1997.

The initial 1998 OCME list did not include all trauma deaths due to the amount of time allowed for reporting deaths to the OCME. The OCME was contacted later in the evaluation and asked to provide another list of deaths from January 1, 1998 through June 30, 1998. At about that time the OCME was changing databases and was unable to provide a list in a timely manner due to glitches with their new software. Therefore, all 66 records on the OCME list were requested from the involved hospitals. Again, there were several records that local hospitals were unable to locate or were so incomplete they were not useful. In an effort to identify additional trauma deaths that met inclusion criteria, the regional medical examiner’s office was contacted. They provided a list of eight additional names that met the criteria. The Trauma Service at Pitt County Memorial was also contacted and provided a list of trauma deaths during the designated time period. Two names from that list were used. A total of 61 complete records were available and selected for review for 1998.

Preparation of Records

As with the 1992 study, the medical records were redacted to insure patient confidentiality. Two copies of each were made. Each copy was divided into sections and separated by tabs: prehospital; ED 1; prehospital 2; ED 2; OR; ICU/Floor; and medical examiner’s (ME) report.

The information for each PMS patient was compiled from the following sources: prehospital call reports (available for 86% of the PMS patients); hospital medical records (100% available); medical examiner reports (99% available, two deaths were not reviewed by the medical examiner); and autopsy reports (conducted on 64%).

Record Summaries

The REACT coordinator summarized records. Included in the summaries were the following data from all institutions that provided care for the patient: age; mechanism of injury; chief complaint; safety information if available; prehospital care; prehospital arrival, scene, and transport times; prehospital level of certification; ED care; care during transfer; trauma center care; OR care; and ICU care.

Review Panel

The review panel consisted of three trauma surgeons, two ED physicians, two prehospital personnel, an anesthesiologist, and a forensic pathologist. Attempts were made to recruit a nurse panel member without success. All members participated in the 1992 project as reviewers. With the exception of the regional medical examiner, all were from outside of the project region. Here is a list of the panel members:

Review Checklist

Panel members used a review checklist similar to the one used for the 1992 study to record their interpretations. A few minor revisions were made in the organization of the form with the data points remaining the same. (A copy of the Review Checklist is included in Appendix VI.)

Review Sessions

Four meetings were conducted to review PMS records. All were held in the central part of North Carolina. The dates for the meetings were: October 15, 1998; December 3, 1998; February 25, 1999; and May 6, 1999. At each session, all panel members were given binders containing the summaries of all cases to be reviewed during the session. The first half of each meeting was devoted to individual record review: each complete record was reviewed separately by two panel members who recorded their findings on the data forms. In addition, each panel member read the summaries for all the cases. Each completed data form (two for each case) was copied onto a transparency to be projected via overhead projector for discussion by the group. Both data forms for the case under review were projected simultaneously and panel members asked to discuss their observations. A consensus was then reached by the panel on the preventability or non-preventability of the death as well as the appropriateness of care delivered at each level. The method of review differed from the 1992 RPMS in which two panel members worked together to review their assigned cases rather than individually as in the REACT PMS.

Continue to Results  |  Return to Contents