Vol. XIV, No. 2 (1992) ISSN: 0732-2623 NATIONAL INSTITUTE ON DISABILITY AND REHABILITATION RESEARCH OFFICE OF SPECIAL EDUCATION AND REHABILITATIVE SERVICES DEPARTMENT OF EDUCATION * WASHINGTON, D.C. 20202 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Human Measurement In Rehabilitation The concepts of measurement, assessment, and evaluation are fundamental to rehabilitation. Consumers are measured to establish their eligibility for benefits or services, to determine which services are appropriate, to assess their needs, to ascertain their current level of functioning, and to estimate their potential. This Rehab BRIEF reviews a number of new measurement systems. They all aim for high levels of precision. Some are also designed to involve the evaluee as a co-evaluator. MEASURING INPATIENTS' FUNCTIONAL INDEPENDENCE Rehabilitation outcomes can be predicted more precisely by establishing the reliability and validity of functional assessment instruments used in impatient rehabilitation settings. More precise prediction of rehabilitation outcomes is desirable as a means of enhancing the cost-effectiveness of rehabilitation, assuring quality of care, and improving treatment planning. Rasch analysis (Wright & Masters, 1982) is a psychometric tool that promises to help achieve these goals. Functional assessment scales often rate behavior from worse to better, dependent to independent, or some variation on this theme; these are ordinal scales. In an ordinal scale the sequence of levels is known, but the distances between the levels are unknown and may be unequal. Rasch analysis transforms ordinal scales to interval scales. Interval measurement exists when the distances between levels of a variable have equal values. Measures made from an interval scale can be subjected to statistical analyses such as multiple regression that relate independent and dependent variables. The use of interval scales also enables assessment of small changes in a patient's status that could not be assessed with ordinal data. The National Institute on Disability and Rehabilitation Research (NIDRR) and the Centers for Disease Control (CDC) have underwritten a study, under the leadership of Allen W. Heinemann, of the scalability and metric characteristics of a widely used tool, the Functional Independence Measure (FIM). The FIM was developed by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation Task Force to Develop a National Uniform Data System for Medical Rehabilitation (Hamilton, Granger, Sherwin, Zielezny, & Tashman, 1987). The FIM was designed to document the severity of patient disability and the outcomes of medical rehabilitation. It assesses self-care, sphincter management, mobility, locomotion, communication, and social cognition on a 7-level scale. Data used in Heinemann's study consisted of patient records contributed by subscribers to the Uniform Data System for Medical Rehabilitation. Rasch analysis results demonstrated that the FIM contains two fundamental subsets of measures: one relating to motor and the other to cognitive function. Further analyses revealed that additional, and important, information is contained in the mode of locomotion, expres- sion, and comprehension. This information was used for subsequent analyses. The study also sought to describe the FIM's predictive validity. Multiple regression was used to predict eight types of outcome variables: motor and cognitive function at discharge, change in motor and cognitive function from admission to discharge, efficiency in motor and cognitive gains, length of stay, and changes in living arrangement from admission to discharge. Rasch FIM measures at admission and discharge were used in these analyses. The independent variables used were gender, race, number of interruptions of rehabilitation stay, payer, living arrangement at admission, marital status, and age at onset of impairment. The Uniform Data System patient sample fit the Rasch model well enough to yield interval measures of motor and cognitive function. Researchers were able to scale this aspect of human performance with a common unit. The validity of the FIM was supported by the patterns of difficulty of the items across several impairment groups. The frequency of misfit between patients and the scales used to measure performance varied across impairment groups but was deemed acceptable. Adequate precision of the FIM was demonstrated, although suggestions for improvement of the instrument emerged. Prediction of various indicators of rehabilitation outcome were also successful. MEASURING FUNCTION AFTER HEAD INJURY Many of the available systems designed for the rapid assessment of head-injured individuals provide only measures of general function. While useful for incidence and prevalence studies, these instruments are not sensitive enough to reflect the discrete information needed for rehabilitation planning and clinical research. The Comprehensive Head Injury Attainment and Outcome Scale (CHAOS) (Jacobs, 1991) is being developed to fill this need. CHAOS contains 70 scales derived by Factor and Guttman analyses from the findings of the more than 700 items of the Los Angeles Head Injury Survey. Only scales with probabilities of at least .85 are used, result- ing in a very high level of statistical control. CHAOS is compatible with other instruments in standard use in head injury and rehabilitation. The instrument can be used on a one-time basis or be re- administered to document changes over time. Items are grouped within skill area categories. Questions within each skill area are grouped from most complex to simplest, based on the abilities of previous subjects. In areas in which the individual is high functioning, only one or two questions will be needed to elicit a positive response; in areas of lower function, questions will continue until an affirmative answer is obtained. Categories include specific activities of daily living, locomotion, orientation, communication, social behavior, and so on. In addition to describing discrete skill levels in each domain, CHAOS identifies six general outcome or expectation levels: nursing home, home dependent, home sufficient, community sufficient, part-time work or school, and full-time work or school. When available, CHAOS promises to delineate a highly articulated profile of the individual's skills and deficits. A national content validation study is now being conducted on the instrument. MEASURING REACHING The capacity for accurate reaching while maintaining postural stability is a requisite for most activities of daily living and for at least 90 percent of all occupations. Although reaching and balance are functionally inseparable in motor performance, very little has been known about their interaction. Haley, Maulucci, Eckhouse, and Schoenberg (December 1990) developed a microcomputer workstation for the simultaneous assessment of upper extremity control and balance. The Rehabilitation Workstation uses a personal computer with a touchplate and a touch-sensitive screen for acquiring temporal and spatial parameters of the hand; two electrodes to sense the contraction of arm and leg muscles; and a seatplate (for seated trials), or two footplates for standing trials, to record pressure signals. To establish a baseline, evaluees are measured both sitting and standing while attending to a dot on the computer screen for 30 seconds. Evaluees are assessed while reaching for stationary and moving targets during sitting and standing. Each of the scored test trials is preceded by an unscored demonstration. The system integrates readings from the various instruments to provide a picture of spatial, temporal, muscle timing, and postural changes during each task. In a different type of task addressing training, an onscreen picture of the changing center of pressure during movement provides evaluees with performance feedback. Although no longitudinal studies have been conducted, initial performance changes showed that evaluees improved body symmetry when provided with this feedback. Future plans for the Rehabilitation Workstation include development of a training protocol. The system contains a library of elements to enable therapists to design new tests. If therapists can break down real world tasks into relevant components that are trainable at the workstation, the system will provide additional benefits. At its present stage of development, the system appears to be a very accurate and clinically feasible method for describing upper extremity control and postural stability. MEASURING JOB ACQUISITION AND RETENTION SKILLS Investigators at the Research and Training Center in Vocational Rehabilitation at the University of Arkansas (with funding from NIDRR) have produced an array of instruments to measure rehabilitation clients along dimensions critical to choosing, getting, and keeping employment. Choosing Selection of an appropriate vocational goal is one of the most significant steps in the rehabilitation process. Goal selection requires that clients and counselors match clients' abilities, interests, and values to the characteristics of occupations. The Occupational Report (OR) (Bolton, June 1987) is a computer-generated report that provides occupational information for planning vocational rehabilitation services. The OR processes raw scores from the USES Interest Inventory, and either the General Aptitude Test Battery or the Non-Reading Aptitude Test Battery, to generate lists of feasible occupational areas. The report contains an occupational interest profile with 12 interest areas ranked from high to low; an occupational aptitude profile with nine aptitudes ranked from high to low; a list of work groups for which the client has suitable aptitudes for success; a list of work groups for which the client has minimal aptitudes for success; and a list of supplementary work subgroups for which clients who function below minimal aptitude levels possess employ- ment potential. The OR uses the 66 work groups identified in the Guide for Occupational Exploration. Each group includes occupations characterized by similar interest preferences and minimal aptitude requirements. The Vocational Personality Report (VPR) (Bolton, March 1987) is a computer-generated assessment report for use in vocational rehabilitation counseling. It is based on the widely used Sixteen Personality Factor Questionnaire (16 PF). The VPR processes raw scores from the 16 PF-Form E to measure core aspects of rehabilitation clients' vocational personalities, including normal sphere traits, pathological tendencies, vocational interest patterns, and temperamental suitability for major occupational groups. Because this is a psychological assessment instrument, it should be used only by qualified professionals. The Employability Maturity Interview (EMI) (Roessler & Bolton, March 1987) is a 10-item structured interview developed to assess client readiness to participate in the vocational planning process. It is useful as a brief screening technique to identify clients who need additional vocational and employability services before selecting an occupational goal. Getting Another tool developed by the Arkansas RTC in Vocational Rehabilitation provides a systematic method of assessing clients' job-getting skills (Hinman, Means, Parkerson, & Odendahl, February 1988). The assessment includes administration and scoring of a standardized job application and a 5-minute interview. Applications are scored for completion time, accuracy, neatness, spelling, completeness, following directions, and so forth. The employment interview is scored for 19 content items including, for example, shaking hands on arriving, self-introduction, and stating specific job skills related to the position. Thirteen style components such as eye contact, gestures, facial expression, and grooming are also assessed. The evaluator also makes two global judgments about the interview: an overall rating of the client's interviewing competence (ranging from excellent to poor) and a hire/not hire decision. Keeping The Arkansas RTC has developed two instruments to measure clients' job-holding skills. The Work Personality Profile (WPP) (Bolton & Roessler, December 1986) is a work behavior rating instrument for use by work evaluators in situational assessments. It measures work behaviors, habits, values, and attitudes that are necessary to maintain a job. The instrument includes 58 items, each scored on a 4-point scale. The assessment takes only 5 to 10 minutes. The WPP yields a profile of strengths and weaknesses. When critical skill deficiencies are identified, targeted remediation can be provided. The WPP can then be re-administered at regular intervals to measure progress toward goal levels. Computer software is available to score and profile WPP ratings. The Work Performance Assessment (WPA) (Lewis, Roessler, & Hinman, 1988; Roessler, Hinman, & Lewis, May 1987) provides a framework for work evaluators to assess clients' work-related social and interpersonal skills. Clients are rated on their responses to four categories of job demands that represent common on-the-job situations: responding to supervision, completing job tasks, cooperating with other workers, and socializing on the job. The WPA script specifies the demand, the dialogue, the materials, and the rating criteria for each of 19 demands. Clients are evaluated two at a time, and administration takes one hour. The WPA profile shows employability strengths and weaknesses on over 80 behaviors. The authors recommend multiple administrations to enhance the reliability of ratings. FROM MEASUREMENT TO EMPOWERMENT Researchers at the Arkansas RTC (Farley, Bolton, & Little, 1990; Farley, Little, Bolton, & Chunn) have expounded a career development model that emphasizes the role of the consumer as developer and co-manager of his or her rehabilitation plan. Each phase of vocational development-vocational choice, job acquisition, and job retention and advancement-has specific task demands. They recommend using many of the instruments discussed above in an integrated Employability Assessment and Planning Program. To promote client involvement, they have developed the Know Thyself Intervention (Farley, Parkerson, Farley, & Martin, in press), a notebook with three sections. The sections are entitled Choose, Get, and Keep. Each section guides the consumer through a series of activities and briefly describes each anticipated assessment procedure to be conducted in that topic area. After each assessment, there are activities to share results and to provide summaries, interpretations, and conclusions. Activities are included to help consumers to personalize vocational information and to utilize the information in goal setting and developing action plans. IMPLICATIONS And I have known the eyes already, known them all -- The eyes that fix you in a formulated phrase, And when I am formulated, sprawling on a pin, When I am pinned and wriggling on the wall, Then how should I begin To spit out the butt-ends of my days and ways? And how should I presume? In "The Love Song of J. Alfred Prufrock," T.S. Eliot (1917) paints a picture of what it can feel like to be the object being measured. Vash (1984), in discussing evaluation from the client's point of view, has called attention to the frequent lapses in common courtesy that often contribute to clients' discomfort and anxiety. The measurers need always to remember to recognize the evaluee as a person, lest he or she feel like an exhibit. A number of the instruments discussed in this Rehab BRIEF go a long way toward making the client a partner in the measurement process. This involves not only giving clients data about themselves, but it also means letting the clients use that information to make important decisions about their own lives. Used in this way, measurement can lead to empowerment. * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * REHABILITATION ENGINEERING CENTERS FOR QUANTIFICATION OF HUMAN PERFORMANCE The National Institute on Disability and Rehabilitation Research has funded two Rehabilitation Engineering Centers to quantify human performance. The Harvard-Massachusetts Institute of Technology Rehabilitation Engineering Center (H-MIT REC) is developing computer-based instrumentation to quantify the effects of various surgical and therapeutic efforts. The H-MIT REC conducts seven interdisciplinary projects to evaluate the physical status of disabled persons and the effectiveness of remedial approaches. Two projects deal with lower extremity orthopedic surgery, two address upper limb function, two deal with tremor and its effect on performance, and one focuses on postural specifications. Different projects address medical interventions, including orthopedic surgery and physical therapy, and/or augmentative technology, including amputation prostheses, orthoses, and functional neuromuscular stimulation. A central theme is the use of human-interactive computer-based systems for assessment of disability status. Inherent in the approach is the conviction that mathematically expressible (and therefore, computer- manageable) models of aspects of the disabled human, of augmentative technology, and of character of disability and better evaluate proposed rehabilitation protocols and technology. When used diagnostically, these systems present performance criteria and outcome data via "friendly" computer interfaces to physicians, rehabilitationists, and/or patients. When the consequences of differential therapies are being evaluated, these systems emulate alternate assistive technologies using special- purpose hardware in conjunction with the computer system to achieve physical interaction with the disabled person. The computer-based systems provide quantitative measures that enhance the qualitative and subjective medical and rehabilitation assessments. The Rehabilitation Engineering Center for Quantification of Human Performance at Ohio State University is designed to develop innovative methods of applying technology to quantify human performance and to develop a system of technical and engineering information exchanges. The program builds on the force/motion measurement system that has been developed to assess gait. The program's interrelated activity areas include the development and improvement of instrumentation to assess physical function and the development of a data registry. * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * SOURCES Bolton, B. (I 987, March). Manual for the vocational personality report. Fayetteville, AR: Research and Training Center in Vocational Rehabilitation, University of Arkansas. Bolton, B. (1987, June). Manual for the occupational report. Fayetteville, AR: Arkansas Research and Training Center in Vocational Rehabilitation, University of Arkansas. Bolton, B., & Roessler, R. (1986, December). Manual for the work personality profile. Fayetteville, AR: Research and Training Center in Vocational Rehabilitation, University of Arkansas. Eliot, T. S. (I 952). The complete poems and plays. New York: Harcourt, Brace and Co., Inc. Farley, R.C., Bolton, B., & Little, N.D. (1990, Winter). Employability assessment and planning in rehabilitation and educational settings. Vocational Evaluation and Work Adjustment Bulletin, pp. 117-123. Farley, R.C., Little, N.D., Bolton, B., & Chunn, J. (undated). Employability assessment and planning in rehabilitation and educational settings. Fayetteville, AR: Research and Training Center in Vocational Rehabilitation, University of Arkansas. Farley, R.C., Parkerson, S., Farley, O.C., & Martin N. (in press). Know thyself. Fayetteville, AR: Research and Training Center in Vocational Rehabilitation, University of Arkansas. Haley, S.M., Maulucci, R.A., Eckhouse, R.H., & Schoenberg, S. (1990, December 31). Modification of the rehabilitation workstation for the clinical assessment and training of upper extremely control and balance: Final report. Boston: New England Medical Center Hospitals. Hamilton, B.B., Granger, C.V., Sherwin, F.S., Zielezny, M., & Tashman, J.S. (1987). A uniform national data system for medical rehabilitation. In M.J. Fuhrer (Ed.), Rehabilitation outcomes: Analysis and measurement. Baltimore: Paul H. Brookes. Heinemann, A. W. (I 99 1). Scaling of functional assessment instruments. Unpublished manuscript. Chicago: Rehabilitation Institute of Chicago. Hinman, S., Means, B., Parkerson, S., & Odendahl, B. (1988, February). Assessment of job application and employment interview skills for job seekers with disabilities: Assessor's manual. Fayetteville, AR: Research and Training Center in Vocational Rehabilitation, University of Arkansas. Jacobs, H.E. (1991). Comprehensive head-injury attainment and outcome scale. Unpublished manuscript. Los Angeles: Department of Psychiatry, University of California, Los Angeles. Lewis, F., Roessler, R., & Hinman, S. (1988). Assessing TBI clients' work adjustment skills: The work performance assessment. Rehabilitation Psychology, 33(4), 213-220. Mann, R.W. (1991). Resume on the Harvard-MIT Rehabilitation Engineering Center. Unpublished manuscript. Cambridge: Harvard University-Massachusetts Institute of Technology Rehabilitation Engineering Center. Roessler, R. (I 988). Work performance assessment script and rating form. Fayetteville, AR: Research and Training Center in Voca- tional Rehabilitation, University of Arkansas. Roessler, R., & Bolton, B. (1987). Manual for the employability interview. Fayetteville, AR: Research and Training Center in Vocational Rehabilitation, University of Arkansas. Roessler, R., & Bolton, B. (1987, March). Manual for the employability maturity interview. Fayetteville, AR: Research and Training Center in Vocational Rehabilitation, University of Arkansas. Roessler, R., Hinman, S., & Lewis, F. (1987, May). Manual for the work performance assessment. Fayetteville, AR: Research and Training Center in Vocational Rehabilitation, University of Arkansas. Vash, C.L. (1984). Evaluation from the client's point of view. In A.S. Halpern & M.J. Fuhrer (Eds.), Functional assessment in rehabilitation (pp. 253-267). Baltimore: Paul H. Brookes. Wright, B.D., & Masters, G. (1982). Rating scales analysis: Rash measurement. Chicago: MESA press. ------------------------------------------------------------- We welcome your comments on this BRIEF and on BRIEFS put out during the past year, as well as your suggestions for topics and for improving this publication of Conwal Incorporated. Prepared by Conwal Incorporated, 510 N. Washington St., Suite 200, Falls Church, VA 22046. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This document was scanned into electronic form for ABLE INFORM BBS: 301/589-3563 or FTS 301/427-0280 (data); Silver Spring, MD USA Internet telnet: fedworld.gov, then dd115 from the Top Menu Internet mail: naric@cap.gwu.edu Electronic release date: September 29, 1993 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-