Expert panel members convened for a meeting lasting one and a half days. The purpose of this meeting was to thoroughly discuss ways to determine the effects of BAC levels on motorcycle operator impairment. Prior to the meeting, members received a document containing background information on the issues to be discussed. This document was generated from the literature review and has been incorporated into Volume II of this report. Workshop participants reviewed the document before the panel meeting. The meeting began with introductions of panel and project team members followed by a description of background, prior research, and project history by the contractor. Subsequent panel discussion included methodologies and elements of methodologies described in the background information, plus any alternative methodologies suggested by panel members. One alternative methodology (the cohort study) that was not considered in the background report was added to the discussion. Also discussed were strengths and weaknesses for each methodology, procedures which would be used for each methodology, measures of effectiveness, data analysis techniques, and promising research sites. Discussions were largely open-ended though discussion was guided by a moderator. After the main discussion ended on the second day, panel members gave their opinions on the relative priorities of the various methodologies for future research. Detailed notes of the discussions were recorded and used to generate this report. A draft of the report was sent to panel members for review. Changes suggested by panel members were incorporated where deemed appropriate.
The following section discusses the various methodologies under consideration (Table 2) and summarizes the expert panel’s review of the methodologies including their primary advantages, disadvantages, and an estimate of general cost category. Cost categories used were: estimates of less than $250,000; between $250,000 and $500,000; and over $500,000.
Driving simulation has been used for many years in research studies on driving behavior for automobiles, trucks, and other vehicles. There are many simulators available for these vehicles at various levels of cost, complexity, and fidelity; however, very few motorcycle simulators are known to the project staff and panel members. Most motorcycle simulators are not of sufficient fidelity to be useful for a study of the type proposed. The National Advanced Driving Simulator (NADS) does not have a motorcycle “cab” simulator. Dynamic Research, Inc., in Torrance, California, has a relatively high-fidelity driving simulator with a motorcycle and scooter cabs and associated hardware and software.1
There are two important aspects of fidelity or realism of the simulator: cue fidelity and dynamics and control fidelity.
Cue fidelity pertains to the display the operator interacts with, including visual, motion, and sound. If they are suitably designed, motorcycle simulators can more readily achieve higher cue fidelity regarding their ability to simulate the riding task faithfully, in comparison with automobile simulators. This is largely because real motorcycles, like aircraft, are “bank-to-turn” vehicles, which means that the operator experiences and senses mainly visual cues, rather than a combination of visual and motion cues that car drivers experience. Both pilots and riders must see a “horizon” (either real or artificial) to control and stabilize the vehicle; if they loose that visual reference, they can be essentially unaware (via motion cues) of whether they are turning or level, because the turns are “coordinated” and for the most part cannot be sensed by the body’s motion-sensing apparatus (tactile, proprioceptive, vestibular, kinesthetic, etc.). Consequently, high-fidelity motorcycle simulators can be less expensive than high-fidelity automobile simulators, because motorcycle simulators can function well with much simpler motion systems. The four-wheeler simulators need more elaborate motion systems, which are an expensive part of the system, to induce realistic operator responses and delays.
Dynamics and control fidelity pertain to the simulated vehicle response to various kinds of operator inputs. For a motorcycle, the vehicle response and operator inputs are considerably more complex than for a car. For a motorcycle, the responses include motorcycle roll, yaw, and pitch angles and rates, and forward and lateral translation. The operator inputs include steer torque and steer angle; lateral and fore-aft upper-body lean angle; and braking, throttle, and gearshift control inputs. The dynamics and control mathematical relationships have been well established since about 1970, and NHTSA-sponsored research (e.g., Weir et al., 1979) experimentally measured and verified these dynamics and control relationships. A motorcycle simulator needs (a) to include these dynamics and control “equations of motion,” and (b) measure the relevant operator control inputs.
In contrast to cue fidelity, motorcycle dynamics and control fidelity is relatively more complex and difficult to provide than it is for cars and other four-wheelers. There are more motion degrees-of-freedom, stronger interactions among them, and more operator inputs to measure. Therefore, for motorcycle simulators, the overall complexity required to provide a suitable vehicle dynamic model and responses to operator inputs, in addition to high-fidelity visual cues, can be substantially greater than it is for four-wheeler simulators. For example, riding a motorcycle through a curve requires that the motorcycle be leaned with respect to the horizon. Opposing physical forces allow the motorcycle to operate leaned over without falling down. This type of phenomenon can be accounted for in creating the simulator, which requires that the simulated cues be different from those for typical four-wheeled-vehicle simulators. As noted, this does not mean that motorcycle simulators are necessarily more expensive; to the contrary, their hardware can sometimes be less costly than comparable automotive simulators. Rather, they are technologically much more complex to develop from a dynamics and control viewpoint, and therefore much less common than four-wheel simulators.
It may be advantageous to create scenarios using real-world situations known to have resulted in motorcycle crashes. It may also be possible to increase the validity of a simulator study by anchoring it to the real world. Examples of this include selecting crash scenarios and impairment measures that, when run with cars and drivers, provide impairment curves mirroring the Borkenstein (1974) and Compton (2002) relative risk curves for drivers; or calibrating the simulation by duplicating the low-speed closed-course test described subsequently, collecting impaired versus unimpaired riding data from a sample of riders on both closed-course and simulated versions of the test, and comparing results.
For the findings of a simulator motorcycle study to be accepted as relevant, they must be validated using real-world information. This issue is discussed in greater detail under the closed-course study.
If the simulation was intended to duplicate reality, to the extent that crash likelihood is similar to actual riding in traffic, it would be necessary to expose subjects to many hours of riding before a crash might be seen. This is not to say that the “many hours” need to be concurrent or that fatigue is a main cause of crashes. The Hurt report (1981) found the bulk of crashes occurred within a few minutes after the start of a trip. However, hundreds of trips may have been completed successfully before the crash. To expose subjects to hundreds of hours of simulation would be prohibitive. More likely, researchers would create a simulation with a higher frequency of potential hazards. This would increase the likelihood of a crash or some other measure of impaired operation.
The ability to record very precise data about the operation of the simulator (e.g., control inputs, lane position, responses to other vehicles) makes it possible to use measures other than crashes to identify impaired operation.
Generally, a simulator study with dosed subjects exposes each subject to a series of different scenarios, each at a different BAC level. This approach requires that the sequence of scenarios be randomized such that each subject faces each one once, and across all scenarios each one is seen an equal number of times at each BAC level. This is done to avoid order effects and scenario effects. By one panel member’s estimation, at least 10 subjects per scenario/BAC combination would be needed to get reliable data. The most typical protocol as used in car alcohol research would be to administer alcohol in a controlled and monitored way, such that crash avoidance/involvement would be measured at multiple (e.g., four) BAC levels, the maximum being just above the typical current legal limit (e.g., .10 g/dL) [See Moskowitz et al., 2000]. Such protocols are well established and have been approved in the past by various internal review boards.
Because dosed-subject simulator studies usually involve multiple trials over time as BAC rises and falls, it is important to make certain that subjects receive enough practice on the simulator to get past the learning curve before actual trials begin. Otherwise, the BAC effects will become confounded with learning effects.
The study design must acknowledge that there is a potential difference between effects of a given BAC while drinking, as BAC is going upward, versus the same BAC after drinking when the BAC is coming down. This would be done by continuing the experiment after maximum BAC had been reached and drinking had been halted, and by recording whether trials were recorded before or after this point.
There are many advantages to the use of simulation for impaired-operator research. The same subjects can be used for impaired and unimpaired conditions, removing inter-subject differences as a possible confounding variable. Differences in performance at varying BAC levels could also be examined for simulated car driving for the same subjects and compared to their motorcycle data. This would increase the validity of the findings compared to generating BAC and performance curves from a motorcycle-riding population, then comparing them to automobile-driving curves from a population that may be very different.
Another advantage of simulation is that the riding experience can be completely controlled so that all subjects are exposed to the same situations across all conditions. Riders can be subjected to simulated risks, in the form of dangerous situations and higher BACs, which could only safely be experienced using simulation. Measurements and data collection are relatively easy compared to that with instrumented real vehicles. A wealth of data can be collected on performance (e.g., speed, response latency, eye movements, close calls) rather than relying completely on crashes as a measure of performance. Differences in judgment might be measured. Data would have some face validity because it would come from actual operation of a (simulated) vehicle comparing impaired to unimpaired conditions in a typical (albeit only visual) crash scenario (e.g., based on a real crash reconstruction); yet it would be safer than having actual impaired subjects riding on a closed course or in traffic (cohort study). BAC levels could be controlled to a greater extent than could those collected in natural occurrences of crashes and other comparison cases found in the field. BAC measurements could be extremely accurate, both in terms of getting a good measure and getting it as close to the time of riding as possible. A videotape of the simulated ride would demonstrate the effects of alcohol on riding and would provide a good teaching tool.
Simulator studies have limited face validity. Although similar to actual riding, it is not actual riding. It will not necessarily be possible to relate simulated crash likelihood to actual crash likelihood, although relative comparisons are common in simulator studies. There will be problems motivating subjects to perform safely in the same way they would if the consequences were real injury. The number of simulator subjects in such a study would probably be far lower than the number of crashes that can be studied using archival data. Although safer than actual riding, there is still the potential for simulator motion sickness, as well as general risks related to dosing subjects. IRB approval is required for such a study. It has been theorized that crashes are more likely for motorcycles than for cars at a given BAC due to the greater skill necessary to operate a motorcycle. To the extent that this is true, it becomes extremely important that a simulated motorcycle operate as much as possible like a real one, especially under emergency conditions. This can be much more difficult with a motorcycle simulator than with an automobile simulator. It was suggested that a simulator study would be most useful if it could be done in conjunction with a low-speed closed-course study with dosed riders, because similarities in performance and impairment, between simulated and actual low-speed operation would tend to validate use of the simulator. However the closed-course study may be difficult to get approved by an IRB.
1 In the interest of full disclosure we note that John Zellner, who is a co-author of this report, is also president of Dynamic Research, Inc. (DRI), which maintains a motorcycle simulator described in some detail in this report. Research by project staff at PIRE and HPRL indicated that the DRI simulator was the only motorcycle simulator suitable for the type of research discussed in this report at the time the project was being conducted. The discussion of the DRI simulator is not intended to advertise the DRI product, or promote its use in research. Rather it has been contributed by Zellner in the interest of providing the reader with as much useful information on the subject of motorcycle simulation as possible.