Model Driver Screening and Evaluation Program
Volume II: Maryland Pilot Older Driver Study

 

Chapter 2: 
Pilot Study Development

Research Design Issues

The research activities conducted as part of the Maryland Pilot Older Driver Study were designed to improve the state-of-the-knowledge in two broad areas: (1) the validity of functional tests as predictors of driving impairment associated with crashes and other safety outcomes, and (2) the administrative feasibility of delivering screening and evaluation services in a driver licensing setting and/or in other settings in the community. Specific questions addressed within each of these areas during development of the Pilot Study, and their impact on the research design, are briefly discussed in the following four pages of the report.

Validating Functional Tests as Predictors of Driving Impairment

A test or procedure to detect declines in the functional capabilities needed to drive safely must possess a certain degree of validity to merit application in the licensing or relicensing process. One goal of the Maryland Pilot Older Driver Study was to provide, to as large an extent as possible, the data needed to validate the application of functional performance measures to account for differences in crash and violation experience. With success in meeting this general study objective, evidence supporting more specific conclusions regarding the preliminary identification of cutpoints, or pass/fail criteria for individual screening procedures, could be sought. The research design guiding data collection and analysis in the Pilot Study incorporated a number of key assumptions about test validation, as elaborated below.

Single Variable Versus Multiple or Combination Variable Predictors. It is an understandable desire of State agencies to identify a screening protocol that can yield the most information about the risk of driving impairment, in the shortest time. This desire suggested, as one option, that data collection and/or analysis should be structured in a stepwise fashion. In this approach, the measure indicated by prior research to be the strongest predictor of driving impairment might be obtained first; then, if performance was below some threshold, another measure would be obtained, and so on. Or, even if all measures were obtained, a stepwise analysis technique might be employed where one variable would be entered into a regression equation first, followed in turn by other variables that were weighted less strongly, until bringing additional variables into the equation no longer produced any gain in explaining differences in the outcome measure. Either course could lead to a relatively more rigid model for driver screening and evaluation, where predictors of driving impairment are formally linked (failure results from this score on measure A and that score on measure B or this score on measure C, etc.), and there is less reliance on the role of clinical judgment in reaching decisions about fitness to drive.

This approach was rejected for several reasons. First, the interrelationships between different broad domains of functional ability important for safe driving--i.e., physical, mental, and visual abilities--are not well defined; nor, in many cases, have the relationships within domains been reliably quantified, especially with regard to the array of perceptual-cognitive (mental) abilities of interest in the Pilot Study. It was a fundamental assumption in this research that a gross deficit in any of the targeted aspects of functional performance could result in a significant increase in the risk of driving impairment. In addition, to combine measures inevitably results in a loss of information. A clinician may, after many evaluations, choose to group certain indicators of functional status together to reach a decision about fitness to drive. But this preserves access to all available information regarding an individual's functional status, and allows the physician, occupational therapist or other professional greater flexibility in applying clinical judgment in determining a person's overall driving health.

Accordingly, in the Maryland Pilot Older Driver Study all included functional measures were obtained for all research participants, within the limits of what was technically and logistically possible, and analyses of the relationships between functional status and crash and violation experience were performed on an individual measure-by-individual measure basis.

Accounting for Potential Selection Biases Yielding an Unrepresentative Test Sample. One of the most common deficiencies in the design of traffic safety research projects, and greatest threats to the validity of a study's outcomes, is collecting data from a test sample that results in an unrepresentative, or biased, estimator of the performance of the broad population of interest. The population of interest for products of this research includes all older persons who wish to retain driving privileges. It was therefore crucial to understand--and hopefully preclude--any systematic differences between the obtained sample and a completely random sample in carrying out measures of functional status in the Pilot Study.

This is not to say that the performance of identified subgroups of older persons was not of interest in this research. In particular, data were desired to describe functional abilities among a group of presumed "superfit" or well elderly who live in a residential community or continuing care retirement community (CCRC), and also for older persons at the other end of the spectrum, i.e., those who have been referred for evaluation specifically because of a suspected medical problem or condition. But neither of these groups is representative of the broad population of normally aging drivers, and thus cannot serve as the primary source of data for analyses to establish relationships between functional status and crash and violation experience that are applicable for driver screening.

The preferred design for the Pilot Study dictated a purely random selection of drivers who would be compelled, for research purposes, to undergo functional screening. Unfortunately, this proved to be not feasible under existing statutes in Maryland.

Accordingly, sample selection in the Maryland Pilot Older Driver Study incorporated random in-person contacts, carefully stipulating that study participation would have no impact on license status. Drivers contacted could refuse to participate, however. Documentation of those who accept versus those who decline, with subsequent analyses to test for differences indicating a bias in the likelihood of causing crashes or committing violations, was adopted as the methodology for this research. Samples of convenience among retirement community residents and the population of medically-referred drivers were also obtained.

Criteria for Judging the Significance of Research Results. One measure of the validity of a screening technique for predicting driving impairments associated with increased crash risk is the level of statistical significance that can be demonstrated when accepted analysis techniques are applied to test the strength of such relationships. It can be further argued that the best choice for the cutpoint in a given functional measure is the score where the strongest predictor-outcome relationship, quantified in terms of statistical significance, is obtained. When research results are to be applied in real world settings, however, the significance of a study's findings may be gauged as much or more by an entirely different set of criteria. In one example, a statistically significant test result may be obtained when there are very small differences in measurements, given a sufficiently large number of observations, but have no operational significance whatsoever. Conversely, when criterion events are rare, as in the case of motor vehicle crashes, a difference that fails to reach statistical significance could still have a major impact on an administrator's decisions about program content or resource allocation.

Even more confusing can be the application of composite indices of the strength of relationship between a predictor and an outcome, where concurrent changes in several different variables contributing to the overall test statistic value can obscure the meaning of a change in the composite measure. This problem was anticipated in the design of this research because of plans to use calculated "odds ratio" values to help identify the most promising screening tools. Consistent with recent trends in research evaluating interventions for preventable injuries--including motor vehicle crashes (e.g. Diller, Cook, Leonard, Reading, Dean, and Vernon, 1999; Vernon, Diller, Cook, Reading, and Dean, 2001)--this statistic in its planned application in the Pilot Study expresses the odds of being in a crash if you fail a test compared to the odds of being in a crash if you pass. As discussed later in more detail, odds ratio calculations involve four separate quantities that are combined multiplicatively, such that a higher overall odds ratio value does not necessarily mean that a test was more effective in detecting impaired (i.e., crash involved) drivers.

Finally, it has long been emphasized that random and uncontrollable factors account for substantial variance in the incidence of motor vehicle crashes (see Peck, McBride, and Coppin, 1971). And in addition, even those drivers who are at greater risk of crashing due to functional impairment may be affected by a diminished capability other than the one a specific test is designed to detect.

Accordingly, analysis and interpretation of data in the Maryland Pilot Older Driver Study was geared to the search for patterns and trends with overarching significance for the validation of functional testing in the detection of impaired drivers. This was a descriptive exercise designed to supplement, not to replace, the statistical tests and techniques designed to quantify the strength of relationship between specific predictors and crash and violation outcomes. In particular, evidence was sought to validate the application of functional testing through its ability to disaggregate crash-involved drivers into separate and discriminable groups: those who are at increased risk because of a specific functional ability being measured, and those who have been involved in a crash because of other factors.

Administrative Feasibility of Delivering Screening Services

State-level involvement in driver screening and evaluation activities will be guided, inevitably, by their feasibility of implementation. Given procedures deemed valid and that also are accepted by the public, an agency may calculate projected program costs based on the equipment, materials, and staff needed to administer them. These costs in turn will be driven by the time to complete screening procedures for each driver; the level and qualifications of the staff who conduct screening; the amount of training required by test administrators; the facilities and physical infrastructure necessary to support testing; the specific hardware and software components of the test protocol; and any supplemental expenses associated with specialists such as occupational therapists who may be desired on site to provide education and counseling services to drivers in conjunction with screening. On the other side of the equation are savings relative to existing program activities due to, for example, a reduction in the number of more costly interventions once a screening program for early detection of impaired drivers is in place.

In this research, the administrative costs were documented as closely as possible, and otherwise estimated, by MVA staff providing oversight to data collection activities in the Pilot Study. An estimation of cost savings produced by having functional screening information available, to help resolve cases where an on-road examination would otherwise be required to make a fitness-to-drive determination, also was developed by the MVA during the course of this research.

Prior to embarking on full-scale implementation of the screening activities--initially in three MVA field offices and eventually involving facilities and personnel statewide--data from a "pre-pilot" feasibility study were analyzed to refine functional test procedures. The "pre-pilot" study was conducted in a storefront office location in Salisbury, MD, by ophthalmic technicians employed by Johns Hopkins University (JHU) to perform data collection and interviews with an established test sample as part of the longitudinal Salisbury Eye Exam study; they were not otherwise affiliated with this project. The JHU technicians were trained in the use of candidate driver screening procedures by members of the research team. Goals of the "pre-pilot" study included documentation of problems in administering screening tests and identification of enhancements to the data collection protocol. A target test length of 15 minutes or less for the Gross Impairments Screening (GRIMPS) protocol was desired.

The "pre-pilot" study was performed over a four-month interval from April to July 1998. During this period, 363 older persons with valid licenses who reported themselves to be active drivers were screened by the JHU technicians, using a candidate test battery and data collection protocol. Sample demographics were distributed as follows: 54 percent were male, and 46 percent were female; 82 percent were Caucasian, and 18 percent were other races; the age of those tested ranged from 68 to 88, with a mean age of 75.7 and a standard deviation of 4.9 years.

The "pre-pilot" study results defined the measures to be included in the Maryland Pilot Older Driver Study. Modifications of selected test methods and improvements to instructions and scoring procedures were suggested by the JHU technicians; when implemented, a 15-minute test length for the battery of functional ability measures comprising GRIMPS was achieved. These measures are described in detail later in the report.

Test Site and Sample Selection

The selection of test sites and recruitment of samples for the Maryland Pilot Older Driver Study proceeded in tandem. The research design initially called for four sites/samples in this study. But as described below, data collection at one site type--Senior Center--was discontinued due to practical considerations and only three of the sites/samples--License Renewal, Residential Community, and Medical Referral--yielded data that were subsequently analyzed to examine the relationships of interest in this research. Materials used to recruit test subjects for the Pilot Study are presented in appendix B.

License Renewal Sample

The largest and most critical sample of drivers tested in this research was obtained in field offices of the Maryland Motor Vehicle Administration (MVA). By design, a random sample of older drivers was sought to yield reliable population estimates of performance distributions on each of the functional measures of interest in this research, and to define relationships of functional ability with crash involvement. Without a random sample, there was concern that selection bias could restrict the ranges and/or skew the distributions of the measured functional abilities. In particular, a sample bias in favor of those with lower crash experience could potentially distort analysis outcomes, such that obtained relationships between functional ability and crash risk would appear unrealistically weak.

Unfortunately, candidate study participants could only be asked, not compelled, to join this research effort, thus ruling out a purely random sample and opening the door to potential selection bias as noted above. Several steps were taken during sample recruitment to mitigate against this threat. First, all older persons appearing at MVA field offices on randomly selected days were approached by project staff with a request for study participation, but only after completing license renewal; each individual already had a new, valid license in his/her possession, and was assured, in writing, that participation in the research activities would not affect license status. In addition, the license numbers (Soundex numbers) for all persons approached--"accepters" and "decliners" alike--were recorded to permit later analyses of any differences between these groups that could suggest a lack of representativeness of the obtained test sample. Key comparisons between those accepting and declining participation in the study are reported below, before presenting a detailed description of the License Renewal Sample.

First, the age distributions of the drivers who accepted and who declined when contacted in a MVA office with a request to participate in the study are presented in table 2. As shown, the mean, median, and standard deviation values were nearly identical. A t-test confirmed that there was not a significant age difference between these distributions (t = 1.24, p < 0.22).

Next, the crash and violation experience of the drivers accepting and declining participation in the study was examined. Six event types were included, connoting varying levels of safety threat. All crashes (excluding alcohol-related events) were counted; at-fault crashes (as per police report) were counted; and an intermediate category of unknown fault crashes was also counted, where the driver was potentially at fault but there was insufficient evidence to confirm fault status in the opinion of the investigating officer. Convictions for all moving violations, for moving violations excluding speeding, and for moving violations excluding speeding and occupant restraint violations were counted, for the respective groups.

Table 2.
Age Comparison for Groups of Drivers 
Who Accepted and Declined to be Screened
Statistic Declined Screening Accepted Screening
N of cases20981876
Minimum Age 55.00 55.00
Maximum Age 90.00 96.00
Median Age 68.00 68.00
Mean Age 68.59 68.28
Standard Deviation 7.95 7.92

The analysis interval was keyed to the date of contact for each individual. It extended one year prior to this date, retrospectively, based on the desire to capture as many crashes as possible for analysis, coupled with a clinical judgment1 that a period of relative stability of functional status for to 12 months into the past could be assumed for most people. The analysis interval also extended 2 to 3 years into the future from the date of contact. The variable analysis interval for prospective data resulted from the fact drivers were contacted regarding study participation over a period of more than a year, while a common "end date" at which analyses were begun was applied to everyone. In summary, the analysis period for each driver bracketed his or her date of contact, with prospective experience accounting for approximately twice as much exposure as retrospective experience.

The relative experience of the comparison groups for each event type is presented in table 3. As shown in this table, those who accepted the request to be screened, though slightly fewer in number, actually demonstrated higher group counts in every crash category analyzed. Using chi-square tests, this difference was found to be statistically significant for all crashes (X2 = 4.79, p<.03) and for the event category including at-fault plus unknown fault crashes (X2 = 5.14, p<.02). The respective groups were not significantly different for any of the other measures; although, the drivers accepting screening were convicted of fewer moving violations than those who declined during the analysis interval.

Table 3. 
Event Counts for Groups of Drivers 
Who Accepted and Declined to be Screened
Event Type Declined Screening Accepted Screening
All crashes (except alcohol related) 93 111
Unknown fault and at-fault crashes (except alcohol related) 50 67
At-fault crashes (except alcohol related) 39 43
All moving violations 197 196
All moving violations (except speeding) 146 102
All moving violations 
(except speeding and occupant restraint)
46 31

These data were taken as satisfactory evidence that, at least with respect to crash experience, there was no basis upon which to infer a "volunteer bias" such that more at-risk individuals were avoiding screening. Thus, the drivers who accepted the request for study participation are hereafter described as the License Renewal Sample, and conclusions drawn from analyses of their functional performance and crash and violation experience serve as the basis for generalizations regarding implications of project findings to the entire (older) population of interest.

The 1,876 drivers in the License Renewal Sample consisted of 1,027 males and 849 females. A more detailed breakdown of drivers by 10-year age group and gender are presented in table 4.

Table 4.
Detailed Age and Gender Breakdown
for Drivers in the License Renewal Sample
Age GroupMalesFemalesTotal
55-64 352 310 662
65-74 426 354 780
75-84 231 174 405
85-94 18 10 28
95+ 0 1 1

The geographic distribution of the License Renewal Sample was dictated by the location of the particular MVA field office in which a given driver was recruited into the study. Drivers were recruited to participate in the screening activities, after completing their license renewal or other business, from November, 1998 to October, 1999. Through their transactions with the MVA, the age of potential study participants was typically revealed to study recruiters; thus, recruitment efforts could be focused on individuals 55 or older.

Driver contacts were made in three office locations which, based on census data, were classified by the research team as representative of relatively more rural, suburban, and urban driving environments. These were, respectively, the Bel Air office, Harford County, MD; Annapolis office, Anne Arundel County, MD; and Glen Burnie office, just outside the City of Baltimore, MD. Demographic information provided by drivers indicated that the areas in which 95 percent of the License Renewal Sample lived and originated their travel by personal vehicles could be accounted for as follows: Harford County, 39 percent; Anne Arundel County, 30 percent; and Baltimore City and County, 26 percent.

Residential Community Sample

One potentially important setting with regard to the implementation of screening activities, from the standpoints of both personal mobility and public health and safety, are residential communities comprised mostly or entirely of older persons. Accordingly, a sample of drivers for the Maryland Pilot Older Driver Study was obtained at the Leisure World facility in suburban Montgomery County, MD.

Leisure World is one of the largest senior independent living communities on the East coast of the U.S. Geared to the "well elderly," its residents live in 4,600 homes, apartments, and condominiums whose prices range from the $150,000's to the $300,000's, with monthly fees averaging $600. This community was also therefore assumed to represent a sample of drivers who were likely to be more fit and socially active than the overall population, for its age cohort. Vehicles are registered to 6,500 of its approximately 8,000 residents.

To facilitate recruitment at Leisure World, the MVA proposed sending a mobile office to the community on a monthly basis. This would provide a convenient service to residents, who could transact business that they would otherwise have to travel to a fixed office location to conduct. Leisure World's Executive Board approved the proposal, further agreeing to a quid pro quo: residents who availed themselves of the vehicle registration, titling, license renewal, and related services now provided on-site by the MVA must agree to participate in the research project, completing driver screening activities, counseling about functional ability and driving health, plus follow up data collection to document changes in driving habits.

The resulting Residential Community Sample recruited in this fashion consisted of 266 drivers, 102 males and 164 females, ranging in age from 56 to 92. The mean age of this sample was 77.1, with a standard deviation of 6.8. A more detailed breakdown of drivers by 10-year age group and gender for this sample is presented in table 5.

Table 5.
Detailed Age and Gender Breakdown
for Drivers in the Residential Community Sample
Age GroupMalesFemalesTotal
55-64 3 8 11
65-74 23 57 80
75-84 62 83 145
85-94 14 16 30
95+ 0 0 0

Medical Referral Sample

A sample of drivers referred to the MVA for medical evaluation by the MAB was also included in the study. In addition to providing information about the relationships of interest in this research from a group of drivers who a priori could be assumed to evidence a higher incidence of impairing conditions, including these drivers permitted an evaluation of the added value of functional status data in reaching clinical judgments about fitness to drive relative to conventional medical review procedures.

All drivers age 55 and older who were referred from any source between October 2000, and October 2001, for MAB evaluation were candidates for this study. For this group, screening could be performed on a compulsory basis. Excluding alcohol offenders, 530 individuals were referred for suspected medical impairment during the specified period; 59 drivers or 11 percent failed to appear, and another 105 were not in the desired age range. As a result, 366 people were selected into the Medical Referral Sample for this study. This total included 209 males, 154 females, and 3 individuals for whom gender was not coded. Driver ages in this sample ranged from 55 to 95, with a mean age of 76.8 and a standard deviation of 9.4 years. A more detailed breakdown of the sample by 10-year age group and gender is presented in table 6.

Table 6.
Detailed Age and Gender Breakdown
for Drivers in the Medical Referral Sample
Age GroupMalesFemalesTotal
55-64 30 21 52
65-74 42 30 72
75-84 94 65 160
85-94 42 38 81
95+ 1 0 1

Drivers in this sample were referred from a variety of sources. The largest share of the sample (35%) was "self-referred," inasmuch as they checked one or more boxes on their renewal forms indicating a medical condition or symptom that is a basis for evaluation in Maryland. Almost as many drivers, 33 percent of the sample, were referred by police. Sixteen percent of referrals came from health care professionals (12% from physicians and 4% from occupational therapists). Family members and friends together were the source of 7 percent of referrals. Other citizens-whose complaints were authenticated before the MAB required a driver to undergo medical evaluation-were the referral source for 4 percent of the sample. One percent of the sample was court-referred. The remaining 14 drivers, or 4 percent of the Medical Referral Sample, were obtained from miscellaneous sources apart from the categories listed above.

Senior Center Sample

The research design for the Pilot Study also included data collection in a Senior Center. This venue was desired to examine the feasibility of combining screening and counseling activities in a familiar and supportive setting that was accessible to the general public, and which did not include any direct involvement by the Motor Vehicle Administration that might, for some people, raise concerns about restriction or loss of driving privileges. The Howard County, Maryland, Office on Aging (HCOA) subsequently agreed to serve as a Pilot Study site. As detailed below, early experience at this site determined that driver functional screening as required to meet the objectives of this research could not feasibly be completed. Data collection activities were subsequently curtailed, and no performance data obtained in the Howard County Senior Center were included in the later analyses of functional status and crash/violation involvement. However, because the experience with data collection at the Senior Center factored into project conclusions about screening program feasibility, a summary of this experience is provided below.

The HCOA, in collaboration with project staff, made a decision to offer a service to its customers titled "Getting Around - Seniors Safely on the Go." By design, customers would participate only on a voluntary basis. Senior Centers have an established relationship of trust with the senior community, and are therefore well positioned to provide a non-threatening site for older drivers to learn about the relationship between functional changes and driving ability, while becoming better informed about transportation alternatives in their community. Keeping seniors connected to the community, regardless of the mode of transportation, was the central theme of the program. All prospective program participants were explicitly told that their names and license numbers would be held in confidence, i.e., regardless of screening outcome, this information would not be shared with the MVA. The older drivers choosing to participate in the program received feedback regarding their performance on the functional screening measures, including the provision of counseling about how to maintain or improve driving skills; information describing alternative transportation resources; and recommendations for further consultations with other health professionals or driving specialists if screening results indicated probable driving impairment.

A sample size of 650 drivers over age 65 was the targeted level of involvement for the Senior Center in this research. Other goals included the use of "peer screeners," older persons who would be trained in the administration of functional tests and would, in turn, perform the actual data collection with customers who volunteered to be program participants. In addition, the feasibility of using local occupational therapists (OT's) to provide feedback and counseling on-site to older drivers after they completed screening was to be determined.

The HCOA embarked on an ambitious marketing plan to attract customers to participate in screening and counseling activities. Press releases, advertisements, and direct mailings were used to recruit participants, and articles were published in a number of newspapers-the Senior Connection (an HCOA newspaper with 6,000 readers), the Baltimore Sun, the Washington Post, and ZIP Publications which publishes three Howard County community newspapers with a high senior readership. Cable spots were aired on GTV, the Howard County Government channel. Fliers describing the program were distributed on an ongoing basis at social and cultural events in the county and surrounding areas. The HCOA Administrator made numerous personal appearances, speaking reassuringly about the anticipated benefits of the program to older drivers and reiterating that license status would in no way be affected by program participation. Finally, HCOA staff engaged in outreach activities to potential referral sources, including area police departments and health care providers.

After several months HCOA staff could confirm that significant portions of the senior community were aware of the program; but, their customers did not believe that participation would not affect their driver's licenses or insurance. The following concerns figured most prominently in forums which afforded the possibility of feedback from the target population2.

During a 10-month interval, from March 1999 to January 2000, 113 drivers (73 females and 40 males) between the ages of 51 and 92 chose to participate in screening and counseling activities in a Senior Center in Howard County. The mean age of the participants was 72.9 years, with a standard deviation of 7.3 years. The small projected size coupled with highly self-selected nature of the sample that could be anticipated through the planned period of data collection did not support a decision to proceed with continued, aggressive marketing activities; also, staff time and costs associated with program administration, including a $40 per hour consulting fee for the included occupational therapist services, were difficult to justify. Thus, despite an enduring commitment at HCOA to the program goals of "Getting Around - Seniors Safely on the Go," the formal involvement of the Senior Centers in Howard County, Maryland, as test sites in this research was discontinued.

Test Sites

The sites at which data were collected in the Pilot Study varied according to test sample, with some sites serving multiple samples. This section identifies the locations and describes characteristics of the sites used for the respective study samples. An overview of all locations serving as data collection sites is presented in figure 1.

License Renewal Sample. Screening for the License Renewal sample was conducted in three of the MVA full-service field offices: Bel Air, Harford County; and Glen Burnie and Annapolis, Anne Arundel County (see figure 1). At each site, a private testing room was provided at least 3.6 m by 3.6 m (12 ft by 12 ft) in size. The testing room contained two desks and two chairs, two full sets of all materials needed to conduct the functional screening, and two computers suitable to administer the Useful Field of View Subtest 2 and the Dynamic Trails test. Each testing room was illuminated by overhead fluorescent fixtures; there were no windows in any of the testing rooms.

At the Bel Air test site, a room located behind the photo counter that was off limits to the public, was used to conduct screening. This room contained two doors that could be closed for privacy. Although the room was closed off from the counter activities, counter personnel were allowed to enter and pass through the room, if necessary, to perform other tasks unrelated to the screening data collection in progress. At the Glen Burnie and Annapolis field offices, a conference room was dedicated to functional screening activities. There were no interruptions from MVA personnel at these sites.

Residential Community Sample. Data collection for the Residential Community sample was performed at Leisure World in Montgomery County, Maryland. A large conference room was provided at one of the "activity buildings" within the facility to conduct the functional screening measures on those days when the MVA mobile office was scheduled for a visit. The room accommodated a waiting area, and three screening stations including computer facilities. Two counseling stations were also provided where individuals received feedback on their functional status and its implications for driving after completing the screening battery. Temporary partitions were used to divide the large room into separate areas for the screening stations and the counseling stations.

Medical Referral Sample. Functional screening for the Medical Referral sample was conducted in 11 of the MVA full-service offices located across the state. At each site, a private conference/training room was dedicated to screening activities. The test site locations included: Bel Air , Harford County; Cumberland, Allegany County; Easton, Talbot County; Elkton, Cecil County; Frederick, Frederick County; Gaithersburg, Montgomery County; Glen Burnie, Anne Arundel County; Hagerstown, Washington County; Largo, Prince George's County; Salisbury, Wicomico County; and Waldorf, Charles County (see figure 1).

Figure 1. 
Location of Test Sites in Maryland Pilot Study

Map of Figure 1. Location of Test Sites in Maryland Pilot Studyd

The Bel Air and Glen Burnie sites were the same used for screening the License Renewal sample in these locations. These testing environments were described above. The sizes and characteristics of the rooms used for testing in the remaining nine MVA offices were also consistent with these locations. In all cases except Bel Air, the testing rooms were restricted to screening only, without any potential for interruption of data collection activities.

Training of Data Collection Personnel

The same personnel collected functional screening data for the License Renewal and Residential Community samples in this research. These were "line personnel" selected by the Driver Safety Research office of the Maryland MVA. In contrast, the functional screening measures were administered to the Medical Referral sample by MVA employees designated as "driver license examiners" by the organization. The training provided to each set of data collectors is described below.

First, ten employees were selected to perform data collection for the License Renewal sample. These prospective screeners were chosen based on the following criteria: they were judged by their supervisors to be highly motivated; and, they expressed interested in participating in the project. It was explained by their supervisors that, if selected, this would become a regular duty for 3 of their 5 days of work each week for the duration of the study. The Principal Investigator requested individuals with good "people skills;" however, less than half were involved in a current position that involved interaction with the public, with only two serving as counter staff. All personnel selected were full-time employees at the MVA.

The candidate test administrators were trained by project staff in a group setting, over a 2-day period in November 1998. During the first day, an overview of the project was provided by the Chief of the Maryland Medical Advisory Board (MAB). Next, a videotape produced by project staff was shown. The videotape showed a 10-minute functional screening protocol being conducted, and then broke each test down into segments to further describe materials needed, script to be used to deliver instructions, proper procedures for conducting the test, and scoring procedures. During the second day, screeners were paired to practice the functional screening procedures on each other. Project staff provided constructive criticism with encouragement to perform all included measures in a consistent and uniform manner, as per the instructions delivered earlier.

Test administrators were observed in their field test sites during the first two days of actual data collection, to provide one-on-one, hands-on training to ensure that the procedures were conducted in accordance with the required protocol. The functional screening materials distributed to each field office contained a videotape showing how each procedure should be conducted and scored, as well as an instruction sheet containing the test set up, the script, and scoring instructions that each test administrator was to follow exactly.

Random, periodic visits were conducted during subsequent weeks to monitor data collection procedures. Some variability between test administrators in the conduct of the screening measures was observed during these visits, and one procedure in particular showed variability from one test to the next by the same administrators. Accordingly, refresher training sessions were conducted in March 1999, at each test site, to reinforce the standardization of procedures used during screening, and to introduce a change in the problematic procedure that was successful in eliminating within-screener variability in test administration technique.

When screening for the Residential Community sample was begun in July 1999, it was determined by MVA officials that a subset of the test administrators performing screening for the License Renewal sample would also conduct these procedures. Thus, a brief period of familiarization with the new setting was required, to adapt procedures previously performed in a small, private room to the test stations separated by dividers in the large, activity room provided for this purpose by Leisure World. No additional training specific to the materials, instructions, administration or scoring of the functional measures was provided, however.

Project staff, in consultation with MVA officials, reviewed the performance of the line personnel at the conclusion of data collection for the License Renewal and Residential Community samples. At this time an issue was raised as to whether better accuracy and consistency in test administration could be achieved by employees already experienced in driver evaluation activities. A decision was made to conduct data collection for the Medical Referral sample--which followed the other samples--using driver license examiners at the MVA, to inform project conclusions regarding not only the effectiveness but also the cost and feasibility of alternate staffing approaches to implement a screening program. The training provided to the MVA driver license examiners participating in the pilot study is described below.

Functional screening for the Medical Referral sample was performed by driver license examiners in all 11 full-service Maryland MVA field offices identified earlier (see figure 1). These individuals were MVA personnel who otherwise conduct closed-course and on-road driving exams in Maryland. In September 2000, full-day training sessions were conducted for 2 groups of 15 examiners in the Glen Burnie headquarters office, one session for each group on successive days. In this training format, the morning consisted of an overview of the project's history and goals, and an orientation to functional abilities as they relate to safe driving ability. Based on feedback provided to project staff, such background information aids test administrators when they must explain the relevance of the screens to inquiring drivers, while also providing justification for their efforts to the examiners, themselves. Concluding the morning segment, a 10-minute videotape produced for the project was shown. Introduced by the MAB Chief, the tape showed a demonstration of the functional screening protocol being conducted on a fit older person by a project staff member also serving as the trainer. This individual then demonstrated the full battery of screening measures on one examiner, breaking down each procedure into its specific components: set-up, materials needed, exact script to use in delivering instructions, and how to score driver performance.

The afternoon segment of the training provided to the driver license examiners was devoted to one-on-one training, practice, and feedback. Examiners paired off in groups of two, allowing one examiner to practice administering the test to the other, with observation and feedback by the trainer.

After this group of driver license examiners returned to their respective field office locations, the trainer made a site visit to each office. During this visit a half-day, follow-up training session was conducted for the 2-3 individuals serving as test administrators at that site.


1 Robert Raleigh, M.D., Chief, Medical Advisory Board, Maryland Motor Vehicle Administration.
2 Pers. comm., Ms. Phyllis Madachy, Administrator, Howard County, Maryland, Office on Aging, (5/12/1999).

BACK  |  CONTENTS  |  NEXT