Banner-- assortment of photos representing physicians, paper work, and handicap parking only sign

IN-DEPTH STUDY

RELATIVE VALUE ASSESSMENT EXERCISE

Method

Representatives from all 51 jurisdictions sampled earlier in this project were contacted again through AAMVA with a request to participate in the Relative Value Assessment (RVA) exercise. As the first step in developing the RVA, medical review program components were identified as viable candidates for driver medical review recommended strategies. This was done through review of the earlier deliverable submitted to NHTSA for this project, titled Summary of Medical Advisory Board Practices in the United States by the TransAnalytics principal investigator and senior analyst. Sixty-four candidate recommended strategies were thus identified.

A framework for the RVA data collection and analysis was defined by Sage (1977). Basically, experts knowledgeable in the field assigned weights among related groups of components, moving in a structured fashion from more general to more specific levels of organization in a hierarchy that encompasses the entire system of interest—in this case, a State's medical review program. Instead of asking survey respondents to rate the relative value of the 64 components with respect to each other all at once, respondents were asked to assign weights to 21 sets of medical review program components, 4 components at a time. Respondents were instructed to assign relative values adding up to 100 to each set of components to indicate how important each component would be to the success of a model medical evaluation program. Further, they were asked to complete this exercise without regard to current feasibility of implementation. The task was not intended as an information-gathering activity to determine how each jurisdiction currently treats each of these elements—this was obtained through the earlier survey activity; instead, it sought a synthesis of expert opinion about what would comprise an “ideal system.”

A worksheet was developed in which the 64 candidate best-practice components were organized into a hierarchical branching table, shown in table 1. The components are labeled alphabetically from A to CF, from top to bottom, and left to right on the page. At the highest level of the hierarchy, the left column lists four very general aspects of a medical review program (labeled A-D). The middle column shown in table 1 shows 16 components of increasing specificity, organized in sets of 4 that branch off of the components shown in the first column (labeled E-T). The right column shows 64 extremely detailed components in sets of 4 that branch off of the components shown in the middle column (labeled U-CF).

The instructions provided to respondents for this exercise explained that while they may consider all four components in a set to be important in a medical review program, there may be one or two that are relatively more important, in their opinion, compared to the other two or three. Or there might be one that is really not important at all. Respondents were asked to “weight” the relative importance of components by assigning numbers from 0 to 100 to show how important each component in each set of 4 would be—in a model program—in comparison to the other 3 components.

To eliminate confusion about which components should be considered in any given comparison, each set of four to-be-weighted components was presented on a separate page, with its own instructions. For example, for the page displaying components A-D, respondents were given the following instructions:

What is the relative importance of each of the following four broad categories (A, B, C, and D) as a potential influence on the effectiveness of a model medical review program for your jurisdiction? Please answer by assigning a number between 0 and 100 to each of the 4 choices (shown in bold type) below, such that the 4 numbers add up to 100.

Results

Forty-five jurisdictions completed and returned RVA exercises. An overview of the calculations performed on the subjective data follows, with the resulting relative values (weights) assigned to each candidate best practice.

The mean of the weights assigned by the 45 jurisdictions for each component is presented in table 2. This value shows the relative importance of the 4 components, in each of the groupings, at every level in the hierarchy. Inspection of table 2 reveals that the sum of the 4 mean values shown in column 1 totals 100. This allows for a direct comparison of how much more important one component is when compared with another component (in the same column).

Table 1. Relative Value Assessment Exercise Branching

Relative Value Assessment Branching Table
%
%
%

[A] Policies governing medical review activities
[E] Nature/extent of DMV Medical Advisors' mission
[U]Develop medical criteria/guidelines for licensing
[V] Review individual cases
[W] Hear appeals
[X] Develop report forms


100

[F] Comprehensiveness of criteria for licensure
[Y] Standards for vision
[Z] Standards for blackouts/seizures/losses of consciousness (includes mental disorders & dementia)
[AA] Standards for medical conditions affecting multiple body systems (e.g. for heart, lung, endocrine musculoskeletal, etc)
[AB] Standards for alcohol/substance abuse


100

[G] Due process for drivers referred for medical review
[AC] No anonymous reports
[AD] Follow up of reporting source to validate claim
[AE] Road test
[AF] Appeal of departmental action


100

[H] Physician reporting responsibilities and protections
[AG] Confidential
[AH] Protection from tort action/immunity for reporting
[AI] Mandated by law for specified medical conditions
[AJ] Sanctions for failure to report
100
100

[B] Process for identifying at-risk drivers
[I] Extent of DMV testing for license renewal
[AK] Vision
[AL] Knowledge
[AM] Road
[AN] Functional screening


100

[J] Use of internal triggers for medical reviews
[AO] Self reports
[AP] Observations by counter staff
[AQ] Driving history (points, crashes)
[AR] Age


100

[K] Use of external, non-medical triggers for medical reviews
[AS] Law enforcement/courts
[AT] Family
[AU] General public
[AV] Social services (includes geriatric evaluation)


100

[L] Use of external, medical triggers for medical reviews
[AW] Personal physician
[AX] Hospital discharge planners
[AY] OT/driving evaluators
[AZ] Vision care specialists
100
100

[C] Case review procedures

[M] Availability of options for preliminary disposition (determines path for evaluation)
[BA] Hearing officer interview with driver
[BB] Assignment by medical staff advisor (e.g., nurse case worker, on-staff or physician consultant)
[BC] Assignment by non-medical staff (administrative determination via procedure manual, checklist)
[BD] Voluntary surrender


100

[N] Extent of DMV evaluation procedures
[BE] Interview (in-person or video)
[BF] Request for and review of medical history
[BG] Functional screening
[BH] DMV examination (may include vision, knowledge, and/or road)


100

[O] Use of external evaluation procedures
[BI] Driving evaluation (driver rehabilitation or driver training specialist (OT/CDRS, driving school)
[BJ] Examination by personal physician
[BK] Examination by medical specialist (e.g., Neurologist)
[BL] Clinical/laboratory testing


100

[P] Composition of Medical Advisory Board
[BM] Full-time DMV staff physicians
[BN] Part-time DMV staff physicians
[BO] Paid consultants
[BP] Voluntary consultants
100
100

[D] Options supporting continuing safe mobility
[Q] Availability of restrictions for license "customization"
[BQ] Daylight/time of day
[BR] Geographical (e.g., radius of home, within city limits, not in city limits)
[BS] Specific routes or destinations
[BT] Road class exclusion (e.g., no freeways, no roads with speeds of 45 mph or greater)


100

[R] Type/extent of referrals for at-risk drivers
[BU] Counseling (for adjustment to change in license or functional status)
[BV] Remediation (to correct or ameliorate functional deficits)
[BW] Alternative transportation
[BX] Retraining/"skills refresher"


100

[S] Breadth of outreach activities by DMV
[BY] Physician education
[BZ] Public awareness/injury prevention
[CA] Law enforcement training in signs of impairment
[CB] Other agencies providing services to seniors


100

[T] Scope of DMV staff training
[CC] Counter staff ( to recognize signs of functional impairment)
[CD] License examiners ( to conduct functional screening)
[CE] License examiners (to conduct specialized road tests)
[CF] Sensitivity training for issues relating to senior drivers & drivers with disabilities
100
100
100


Table 2. Results of Relative Value Assessment Exercise

Mean Weights Across States (n=45)
%
%
%
29.8
[A] Policies governing medical review activities
25.6
[E] Nature/extent of DMV Medical Advisors' mission
40.6
[U]Develop medical criteria/guidelines for licensing
28.5
[V] Review individual cases
15.0
[W] Hear appeals
16.3
[X] Develop report forms


100

32.2 [F] Comprehensiveness of criteria for licensure 27.9 [Y] Standards for vision
28.4 [Z] Standards for blackouts/seizures/losses of consciousness (includes mental disorders & dementia)
24.8 [AA] Standards for medical conditions affecting multiple body systems (e.g. for heart, lung, endocrine musculoskeletal, etc)
19.2 [AB] Standards for alcohol/substance abuse


100
16.6 [G] Due process for drivers referred for medical review 23.5 [AC] No anonymous reports
23.3 [AD] Follow up of reporting source to validate claim
31.8 [AE] Road test
21.5 [AF] Appeal of departmental action


100
25.6 [H] Physician reporting responsibilities and protections 21.2 [AG] Confidential
35.4 [AH] Protection from tort action/immunity for reporting
28.6 [AI] Mandated by law for specified medical conditions
15.1 [AJ] Sanctions for failure to report
100
100
29.3 [B] Process for identifying at-risk drivers 25.4 [I] Extent of DMV testing for license renewal 29.6 [AK] Vision
18.8 [AL] Knowledge
28.6 [AM] Road
23.1 [AN] Functional screening


100
18.6 [J] Use of internal triggers for medical reviews 28.0 [AO] Self reports
28.8 [AP] Observations by counter staff
24.4 [AQ] Driving history (points, crashes)
19.2 [AR] Age


100
24.8 [K] Use of external, non-medical triggers for medical reviews 33.7 [AS] Law enforcement/courts
28.4 [AT] Family
14.4 [AU] General public
23.5 [AV] Social services (includes geriatric evaluation)


100
31.2 [L] Use of external, medical triggers for medical reviews 35.4 [AW] Personal physician
15.2 [AX] Hospital discharge planners
20.8 [AY] OT/driving evaluators
28.6 [AZ] Vision care specialists
100
100
23.5 [C] Case review procedures
19.8 [M] Availability of options for preliminary disposition (determines path for evaluation) 22.8 [BA] Hearing officer interview with driver
27.4 [BB] Assignment by medical staff advisor (e.g., nurse case worker, on-staff or physician consultant)
29.8 [BC] Assignment by non-medical staff (administrative determination via procedure manual, checklist)
20.2 [BD] Voluntary surrender


100
28.7 [N] Extent of DMV evaluation procedures 16.3 [BE] Interview (in-person or video)
13.2 [BF] Request for and review of medical history
21.0 [BG] Functional screening
31.6 [BH] DMV examination (may include vision, knowledge, and/or road)


100
32.2 [O] Use of external evaluation procedures 22.2 [BI] Driving evaluation (driver rehabilitation or driver training specialist (OT/CDRS, driving school)
32.8 [BJ] Examination by personal physician
30.7 [BK] Examination by medical specialist (e.g., Neurologist)
14.4 [BL] Clinical/laboratory testing


100
19.3 [P] Composition of Medical Advisory Board 23.4 [BM] Full-time DMV staff physicians
24.4 [BN] Part-time DMV staff physicians
27.0 [BO] Paid consultants
25.5 [BP] Voluntary consultants
100
100

[D] Options supporting continuing safe mobility 29.2 [Q] Availability of restrictions for license "customization" 34.3 [BQ] Daylight/time of day
24.8 [BR] Geographical (e.g., radius of home, within city limits, not in city limits)
16.5 [BS] Specific routes or destinations
24.6 [BT] Road class exclusion (e.g., no freeways, no roads with speeds of 45 mph or greater)


100
20.3 [R] Type/extent of referrals for at-risk drivers 21.5 [BU] Counseling (for adjustment to change in license or functional status)
24.7 [BV] Remediation (to correct or ameliorate functional deficits)
24.4 [BW] Alternative transportation
29.6 [BX] Retraining/"skills refresher"


100
21.7 [S] Breadth of outreach activities by DMV 29.5 [BY] Physician education
22.2 [BZ] Public awareness/injury prevention
26.0 [CA] Law enforcement training in signs of impairment
22.4 [CB] Other agencies providing services to seniors


100
28.7 [T] Scope of DMV staff training 24.9 [CC] Counter staff ( to recognize signs of functional impairment)
26.9 [CD] License examiners ( to conduct functional screening)
27.9 [CE] License examiners (to conduct specialized road tests)
20.4 [CF] Sensitivity training for issues relating to senior drivers & drivers with disabilities
100
100
100

The relative value of the 16 components in column 2 was indicated through an additional calculation. Specifically, using decimal equivalents instead of percentages, each of the 4 mean values in column 1 was multiplied by each of the 4 mean values in the group that branches from it in column 2. For example, the mean value for component A (.298) was multiplied by the mean value for component E (.256). The product was then multiplied by 100 to obtain the relative value of component E (7.63%). Because the sum of the 16 values in column 2 derived through this multiplication process equals 100, a direct comparison can be made regarding how much more important one component is than another, within the same column.

Finally, this procedure was extended to gauge the relative value of each of the 64 components in column 3: The mean weight calculated for the column 1 component was multiplied by the mean weights calculated for its branching components in column 2, and then by the mean weights of its branching components in column 3. For example, the relative weight of component U (3.10%) was obtained by multiplying the mean weight for component A (.298) by the mean weight for component E (.256), then multiplying that product by the mean weight for component U (.406) and then multiplying by 100. Again, the sum of the 64 multiplied values in column 3 equals 100, permitting a direct comparison of how much more important one component is when compared to any other component in the same column.

The resulting component scores (calculated weights) are presented in table 3. These scores have been sorted to present the components that ranked highest in importance within each grouping of 4 components. Table 4 presents the 16 components listed in column 2 only, in rank order from highest to lowest weight. Table 5 presents all 64 model program components included in this exercise, organized in descending order of importance in the RVA.

Back Next