Christine McDonough, PT, PhD, TREAT advisor in clinical and comparative effectiveness, shares the experience of innovating an efficient and accurate method of evaluating mental and physical health function for the United States Social Security Administration’s (SSA) claims handling process in collaboration with researchers at Boston University, the SSA, and the National Institutes of Health.
Work disability is a critical problem for countries worldwide, presenting challenges to health care, social support, and economic systems. The United States (US) faces unique challenges due to the volume of applications and beneficiaries to manage. For example, the US Social Security Association (SSA) handles 2-3 million disability applications per year. To apply for disability programs, an individual submits a claim to their local SSA field office or State agency (often called Disability Determination Services or DDS). Adjudicators guide the claim through the multi-step disability determination process, which includes collecting both information from the “claimant” and medical evidence for comparison to established criteria.
Because of the sheer size of the programs, it is challenging for the SSA to provide efficient, accurate determination of disability status for initial claims, and to identify important change that would alter disability status. National experts have been convened to identify ways to improve the process. One panel recommended incorporation of information about the applicant’s functioning. We focus here on self-reported claimant functioning information, which SSA collects, but not in a comprehensive, systematic manner.
The SSA, National Institutes of Health, and a research team at Boston University collaborated on a multi-year project to develop a measurement tool to rigorously quantify claimant functioning in areas of particular relevance to work. Their goal was to apply contemporary knowledge about disability and advances in measurement theory and methods to enable collection of standardized functional information with minimal burden on SSA. The team used contemporary measurement theory and methods to develop allow computer-adaptive administration. These approaches require developing a large set of items (questions) that represent low to high functioning, and then collecting data in large field studies to estimate the difficulty of each item on the scale (calibration). Once the items are calibrated, this information is used to allow the computer to select the item that best fits the person’s difficulty level based on their prior response. This computer-adaptive approach allows test scores to be estimated with sufficient precision with 6-8 questions per scale.
The team began by developing self-report questions to cover low to high functioning in physical and mental health domains and to represent the full continuum of functioning relevant to work. Creating these questions for administration to SSA claimants is a unique challenge. This is due to the fact that in order to file a claim, a claimant must have been out of work for at least 18 months. The team tackled this problem by conducting a comprehensive search of evaluation tools and questions of work capacity and disability measurement, as well as disability concepts (e.g. Department of Labor Dictionary of Occupational Titles, SSA advisory panel reports, work disability literature).
Next, contemporary frameworks of disability were used to assess content coverage, and to enable translation from work-specific to tasks and activities in everyday life. For this task, the team used the World Health Organization’s International Classification of Functioning, Disability and Health. Whereas medical evidence focuses on diagnostic and functioning at the level of Body Structures and Functions (e.g. organs and organ systems), the Work-Disability Functional Assessment Battery (WD-FAB) focused on how the person functions in their environment.
The WD-FAB was developed in three phases with multiple large samples, including 5,752 SSA claimants and 4,024 working-age US adults. The structure of the WD-FAB is summarized in Figures 1 and 2 below.
Figure 1. WD-FAB Physical Function Domain Structure
Figure 2. WD-FAB Mental Health Domain Structure
Because changes to the disability determination workflow can have massive consequences, the threshold for demonstrating reliability, validity and value is very high for this application. The WD-FAB covers important domains of work-relevant functioning using over 300 questions across 8 scales.
Research on the development data and additional studies have demonstrated promising reliability, validity, and the ability to compare scores to working-age adults. To date, 13 articles have been published on the WD-FAB and research is ongoing. Studies in work-disability and SSA claimant populations are underway, and there is considerable interest in international social security systems, with translation and pilot projects proceeding in Germany and Belgium.
Christine McDonough is Assistant Professor of Physical Therapy at the School of Health and Rehabilitation Sciences and in Orthopaedic Surgery at the University of Pittsburgh. She has a clinical background as a physical therapist focused in orthopedics, sports and geriatrics. Her mission is to enable optimal activity, participation and inclusion of people with health conditions and impairments. Her research includes clinical and health services research, and the application of contemporary disability and measurement theory and methods to the development and testing of patient-centered health outcome measures. She serves as Editor of Clinical Practice Guidelines for the Orthopaedic Section and Academy of Geriatric Physical Therapy of the American Physical Therapy Association.
Marfeo EE NP, Jette AM, et al. Development of an instrument to measure behavioral health function for work disability: item pool construction and factor analysis. Arch Phys Med Rehabil. 2013;94:1670-1678.
McDonough CM JA, Ni P, et al. Development of a self-report physical function instrument for work disability assessment: item pool construction and factor analysis. Arch Phys Med Rehabil. 2013;94:1653-1660.
Marfeo EE NP, Haley SM, et al. Scale refinement and initial evaluation of a behavioral health function measurement tool for work disability evaluation. Arch Phys Med Rehabil. 2013;94:1679-1686.
Marfeo EE NP, McDonough CM, et al. Improving assessment of work related mental health function using the Work Disability Functional Assessment Battery (WD-FAB). Occup Rehabil. 2018;28:190-199.
McDonough CM NP, Peterik K, et al. Improving measures of work-related physical functioning. Quality of LIfe Research. 2017;26:789-798.
Ni P MC, Jette AM, et al. Development of a computer-adaptive physical function instrument for Social Security Administration disability determination. Arch Phys Med Rehabil. 2013;94:1661-1669.
Supported by U.S. Social Security Administration-National Institutes of Health Interagency Agreements under the National Institutes of Health (contract nos. HHSN269200900004C, HHSN269201000011C, HHSN269201100009I, and HHSN269201200005C) and by the National Institutes of Health Intramural Research Program.