Report Card - Consumer/Survivor

MHSIP

U.S. Department of Health and Human Services
Substance Abuse and Mental Health Services Administration
Center for Mental Health Services

Consumer-Oriented Mental Health Report Card

The Final Report of the
Mental Health Statistics
Improvement Program (MHSIP)
Task Force on a Consumer-Oriented
Mental Health Report Card

This is a collaborative effort of consumers, the MHSIP community and the Center for Mental Health Services

April, 1996

OVERVIEW

BACKGROUND

A Focus on the Consumer
An Emphasis on Outcomes
A Focus on People with Serious Mental Illnesses
A Unique Document

THE MHSIP REPORT CARD

Uses of the MHSIP Report Card
A Comprehensive Approach
Concerns and Indicators Related to Access
Concerns and Indicators Related to Appropriateness
Concerns and Indicators Related to Outcomes
Concerns and Indicators Related to Prevention
Collecting Data and Interpreting the Results

DATA REQUIREMENTS AND METHODOLOGY

Recommended Data Sources
Methodological Concerns
A Work in Progress

RECOMMENDATIONS FOR PILOT-TESTING

SUMMARY

ENDNOTES

REFERENCES

ACKNOWLEDGMENTS

The Mental Health Statistics Improvement Program (MHSIP) Task Force to develop a consumer-oriented mental health report card gratefully acknowledges the many individuals whose hard work and expertise made this report possible. Members of the Task Force, led by Vijay Ganju, Ph.D., of the Texas Department of Mental Health and Mental Retardation, remained committed to the values and goals of appropriate treatment for people with mental illnesses through long months of research and discussion.

Several activities of the Task Force were supported by The Evaluation Center"HSRI (Human Services Research Institute of Cambridge, MA), a grant program of CMHS. The Task Force is especially grateful for the commitment and support of Task Force member Virginia Mulkern, Ph.D., Associate Director of The Evaluation Center, who coordinated the various HSRI/Task Force initiatives. These included the following:

The Evaluation Center provided the Task Force with a mini-grant to conduct a review of the literature related to performance indicator systems, published and unpublished literature on performance monitoring, and current report card initiatives. This project was conducted by Rex S. Green, Ph.D., and Frederick L. Newman, Ph.D.
With HSRI support, Task Force members Paul Sherman, Ph.D., and Caroline Kaufmann, Ph.D., conducted a review of published and unpublished literature, from the consumers' perspective, on what people with serious mental illnesses want from mental health services.
Also under the auspices of HSRI, Task Force members Caroline Kaufmann, Ph.D., and J. Rock Johnson, J.D., conducted a focus group in Washington, DC, at which consumers rated the concerns and indicators drafted by the Task Force.
In addition, The Evaluation Center"HSRI funded production of the final report, which was edited and designed by Susan Milstrey Wells of Waterford, NY.

The MHSIP Task Force also wishes to thank Task Force member Shula Minsky, Ed.D., who pilot-tested and analyzed the consumer self-report survey developed by the Task Force.

OVERVIEW

The rapid growth of healthcare reform at the state and local level is aimed at controlling costs and improving the quality of care. Healthcare purchasers have increasingly demanded a way to measure the effectiveness of these efforts.

In response, a number of healthcare organizations and government entities have begun to gather and publish data that allow corporate purchasers, state agencies, and consumers to compare the performance of competing health plans. These documents are commonly referred to as "report cards."

In 1993, the Mental Health Statistics Improvement Program (MHSIP) of the Center for Mental Health Services convened a Task Force to develop a prototype consumer-oriented report card to assess the quality and cost of mental health and substance abuse services. Task Force members include mental health consumers; representatives of federal, state, and local mental health and substance abuse agencies; advocacy groups; researchers; and policy analysts.

In Phase I, the conceptual phase of the project, Task Force members outlined the major issues involved in the design of a mental health report card and defined the critical domains such a document should contain. These include access, appropriateness, outcomes, consumer satisfaction, and prevention.

Actual development of the report card began with Phase II in 1995. The summary that follows is a progress report on Phase II activities, which included a review of the literature on performance measures related to mental health report cards, a review of the literature on consumer-based research, identification of concerns related to the various domains, and a consumer focus group to identify and prioritize concerns.

The result is a set of recommended indicators and measures for a mental health report card that is (1) consumer-oriented, (2) based on research and explicit values, (3) focused on, but not limited to, serious mental illness, (4) designed to emphasize the outcomes of mental health treatment, and (5) conscious of related costs and staff burden. An overview of this work is presented in the first section of this report.

The technical appendices include a more in-depth definition of the specific indicators and measures. In addition, they contain suggested data sources needed to complete the report card, including a consumer survey, clinician-administered instruments, and enrollment/encounter data requirements; tables relating the measures to these data sources and to the populations for which the measures are relevant; an analysis of the validity and reliability of the suggested measures; and a report on development of the consumer survey. Sample report card formats are also included in the appendix.

The final phase of the Task Force's work involves pilot-testing the recommended report card. However, this will not be the end of the process. Rather, it is the beginning of a national dialogue about the development of reliable, comparable, and relevant measures to determine the effectiveness of mental health and substance abuse services. When mental health consumers are empowered to choose services that provide the best value for their healthcare dollar, they will be true partners in the effort to improve healthcare in this country.

BACKGROUND

The idea of monitoring the quality of healthcare services is not new. Prior to the mid-1980s, however, quality assessment efforts focused largely on hospitals, and the results rarely were made available to the public.[1] With the widespread implementation of managed care, public oversight of healthcare services has become a critical issue. Various healthcare organizations and government entities have begun to develop report cards to help purchasers and consumers assess the cost and quality of care, but individual consumers and consumer groups have had limited involvement in these efforts.

This tendency was noted by the General Accounting Office (GAO) in a 1994 summary of report card initiatives.[2] "Individual consumers have had minimal input into selecting report card indicators, and little is known about their needs or interests, " the GAO reported. "As a result, their needs may not be met." The MHSIP Task Force recognizes the importance of including people who are receiving or have received mental health services in significant roles at all stages of report card development and implementation. Established with a specific mandate to construct a report card that addresses the needs of mental health consumers, especially adults with serious mental illnesses[3] and children with serious emotional disturbances[4], the Task Force included consumers as active participants in every step of the process.

Development of a report card designed to help mental health consumers make informed choices began with the discussion of national healthcare reform. Concerned that mental health and substance abuse services would not be included as part of the minimum benefit, the mental health community began to define the services consumers need and the outcomes they expect. With the demise of national reform efforts, and the subsequent increase in state and local healthcare initiatives, the need for a mental health report card that reflects consumer concerns has become even more critical.

A Focus on the Consumer

In the context of national healthcare reform, a primary use of the report card was to help consumers choose among various mental health services and systems. A consumer perspective on the critical elements such a document should contain was integral to this effort. Building on these earlier initiatives, the singular characteristic that defines the MHSIP report card is its emphasis on the consumer.

This focus on consumer needs goes beyond the inclusion of consumers in developing and evaluating the report card's indicators and measures. Indeed, the domains, concerns, indicators, and measures of the MHSIP report card are specifically designed to assess consumer concerns with various aspects of mental health treatment, not merely global satisfaction with mental health services. The report card's indicators include both objective measures of a provider's commitment to mental healthcare (e.g., the average resources expended on mental health services), and consumer assessment of the convenience, appropriateness, and outcomes of the services the system supports. The MHSIP report card is unique among similar documents in measuring those dimensions that matter most to mental health consumers.

An Emphasis on Outcomes

Implicit in the development of the MHSIP report card is the assumption that mental health systems exist to produce specific outcomes and that, to achieve these outcomes, certain attitudes, processes, and services need to be in place. For example, effective mental health treatment should reduce symptom distress and help an individual increase independent functioning, improve performance and productivity at work or school, develop a system of natural supports, and gain access to physical healthcare services. To achieve these outcomes, a mental health system must offer a wide range of service choices that are voluntary, easily accessible, culturally appropriate, focused on recovery (e.g., psychosocial rehabilitation and support services), and designed to promote consumer inclusion.

Many existing report cards shy away from including information on outcomes. Instead, they measure organizational arrangements (the structure of care) and/or provider activities (the process of care). While many experts believe that outcomes are the best measure of quality (GAO, 1994), outcome data can be difficult to interpret because they are affected by individual consumer characteristics. In addition, such evolving concepts as "personhood"[5] and "recovery,"[6] considered by consumers to be critical in any discussion of outcomes, do not have accepted operational definitions. Finally, because of the difficulty in measuring outcomes, most existing data bases do not include the needed information.

The Task Force acknowledged these difficulties but developed the report card based on considerations of what should be measured and not what is conveniently available or easy to measure. Members believe that outcome data will become more accurate as organizations are held accountable for reporting it. In addition, they feel that by defining more clearly the goals and values of the public mental health system, consumers will receive better quality of care, and the broader healthcare community will recognize the effectiveness of mental health and substance abuse treatment.

To minimize the costs and the burden involved in gathering outcome data, the Task Force attempted to define data requirements that could be addressed. In many cases, this meant choosing surrogate measures that reflect how well mental health systems incorporate certain concepts into their practices. For example, the proportion of annual resources a mental health provider expends on such services as supported housing, vocational rehabilitation, and consumer-run activities may be an indirect measure of the availability of services that promote recovery.

A Focus on People with Serious Mental Illnesses

Many of the current healthcare report card efforts focus on the general population; therefore, the concerns of people with serious mental illnesses are not addressed adequately. Adults with serious mental illnesses are a major focus of the MHSIP report card, and many of the indicators and measures have been designed with these individuals in mind.

However, the report card is a comprehensive document that also includes indicators and measures relevant to children with serious emotional disturbances, to adults and children with other mental disorders, and to adults with a dual diagnosis of a mental illness and substance use disorder. Though more work needs to be done to assess the appropriateness of mental health services to these individuals (e.g., the concerns of parents with children who have serious emotional disturbances need to be reflected), the MHSIP report card advances the state of knowledge about what can and should be measured for these groups.

A Unique Document

The result of the Task Force's deliberations is a report card that uniquely reflects the needs of mental health consumers, and that can be used by a broad constituency to determine whether a mental health system is meeting the needs of adults and children with mental disorders. Specific features of the report card and its recommended uses are outlined in the next section.

THE MHSIP REPORT CARD

With the rapidly increasing interest in assessing the quality and cost of healthcare services, several mental health report cards have emerged. Among these, the MHSIP report card is unique in a number of ways.

It is consumer-oriented.
The report card was developed with consumers and family members and reflects their concerns.
It is value-based.
The MHSIP report card explicitly addresses issues of consumer choice, empowerment, and involvement. Though concepts and measures of recovery, personhood, and self-management are evolving, the report card includes these as integral elements. In addition, the report card's concerns, indicators, and measures reflect expectations that appropriate services will be available, easily accessible, developed with and by consumers, and offered in the least restrictive setting.
It emphasizes concerns related to serious mental illness.
There is a clear emphasis on issues related to serious mental illnesses and serious emotional disturbances in the report card's indicators and measures. However, the document also is intended to address mental healthcare delivery to all people with mental health needs, both children and adults.
It includes outcomes.
As noted, many report cards avoid outcomes. While there may be additional burden or costs associated with obtaining such data, this is the critical element in determining the performance of a provider or system.
It is research-based.
The development of the report card included an extensive review of the literature on performance measures, outcomes, and report cards. Expert consultants were involved in this effort.
It is cost- and burden-conscious.
The Task Force went to considerable effort to minimize the cost and burden of obtaining the needed data.

Uses of the MHSIP Report Card

Most experts agree that report cards can be useful tools to educate stakeholders about the quality and cost of healthcare services. But depending on the context, and on the information presented, their uses can vary.

Mental health report cards are being developed for employers to evaluate and select managed behavioral healthcare organizations and provider systems. Providers are using report cards to monitor the performance of their systems over time. State agencies are being required to develop report cards to ensure desired outcomes.

A primary purpose for the development of the MHSIP report card is to ensure that there is a consumer-oriented report card that reflects the values and concerns of consumers, and that can provide a model and guidelines for the development of subsequent mental health report cards. In particular, the MHSIP report card can be used to help

consumers compare and evaluate alternative mental health service options;
advocacy groups promote better services;
healthcare purchasers evaluate managed care organizations or provider systems;
providers monitor the performance of their systems over time and implement continuous quality improvement efforts; and,
state mental health agencies and other funders monitor quality and desired outcomes across different provider systems.

A Comprehensive Approach

The MHSIP report card consists of domains, concerns, indicators, and measures. For example, under the broader domain of access, one concern is the convenience with which mental health consumers are able to enter services. The average length of time from a request for services to the first face-to-face meeting with a mental health professional is one indicator of the relative ease with which consumers can access services. The total time between a request for services and the first face-to-face contact with a mental health professional for new admissions during the year, divided by the total number of new admissions, is one way to measure this indicator. Each of these elements is described in more detail below.

Domains. The MHSIP report card is organized around the domains of access, appropriateness, outcomes, and prevention. These categories reflect broad areas included in proposed national healthcare reform legislation. Consumer satisfaction was dropped as a separate domain because Task Force members determined that specific elements of consumer satisfaction are, in effect, related directly to consumer assessment of concerns within each of the other domains. These have, therefore, been subsumed under those domains.

In a similar fashion, the Task Force did not explicitly develop indicators and measures for cost as a separate domain. While members felt that the Medicaid HEDIS (Health Plan Employer Data and Information Set) contains an adequate set of financial performance measures, they did identify the following major concerns related to cost, which are included in the access and appropriateness domains:

Adequate resources available for all mental health services
Identification of resources for important subpopulations, including children and adolescents with serious emotional disturbances, children and adolescents with other emotional disturbances, adults with serious mental illnesses, adults with other mental illnesses, and adults with a dual diagnosis of a mental illness and substance use disorder.[7]
Resources allocated for psychosocial rehabilitation and support services (e.g., supported housing, vocational services/supported employment, consumer-run services, family education, etc.).

Concerns. The Task Force conducted extensive research to develop a set of concerns or value statements related to each domain. The group reviewed published and unpublished literature on what people with serious mental illnesses want from mental health services, developed an initial set of priority concerns, and conducted a focus group with mental health consumers to assess these concerns. Task Force members found broad consensus in the mental health community about the shape and scope of mental health treatment for people with serious mental illnesses, as reflected in part by the Community Support Program (CSP) model.[8] As a result, the concerns in the MHSIP report card reflect consumer priorities and represent the service elements, processes, and outcomes that illustrate whether the key domains are successfully being addressed.

Indicators. An indicator is an operational specification of how well an organization is performing relative to each concern. To help choose appropriate indicators, the Task Force commissioned reviews of published and unpublished literature on performance indicators; studied current State, local, and private performance evaluation initiatives; reviewed existing report cards (a listing is included in the references); and asked the consumer focus group to comment on the proposed indicators. The resulting indicators reflect both objective appraisals of a mental health system's performance and consumer assessment of the value of services received.

Measures. A measure is the methodology used for deriving and calculating the indicator. In addition to reviewing existing reporting instruments and developing specific measures, Task Force members designed and pilot-tested a consumer survey to measure indicators not addressed in available standard instruments. The domains, concerns, and indicators of the MHSIP report card are outlined in brief below. More complete information, including specific measures, is included in the technical appendix.

Concerns and Indicators Related to Access

Access refers to the degree to which services are quickly and readily obtainable. This includes the responsiveness of the system to individual and cultural needs, and the availability of a wide array of relevant services.

The priority concerns related to access include the following:

Quick and convenient entry into services.
A full range of service options.
Cultural and linguistic access.
Financial barriers.

The indicators presented below are a representative set. The complete list for this domain, along with relevant measures, is presented in the technical appendix.

The average length of time from request for services to the first face-to-face meeting with a mental health professional.
The average resources expended on mental health services.
The proportion of resources expended on mental health services that are consumer-run.
The proportion of resources expended on mental health services provided in a natural setting (home, school, and work).
The percentage of people served in a year who had only one mental health contact.
The percentage of people receiving Supplemental Security Income or Social Security Disability Insurance benefits who received services.

Concerns and Indicators Related to Appropriateness

Measuring the appropriateness of mental health services is difficult. There is no widely accepted equation that automatically links assessment with a standardized treatment plan. Appropriate services are those that are individualized to address a consumer's strengths and weaknesses, cultural context, service preferences, and recovery goals.

The priority concerns related to appropriateness include the following:

Voluntary participation in services.
Services that promote recovery.
Services that maximize continuity of care.
Consumer involvement in policy development, planning, and quality assurance activities.
Adequate information to make informed choices.
Application of best-practice guidelines.

The indicators presented below are a representative set. The complete list for this domain, along with relevant measures, is presented in the technical appendix.

The percentage of consumers who actively participate in decisions concerning their treatment.
The proportion of resources expended on services that promote recovery.
The percentage of people discharged from inpatient services who receive ambulatory services within 7 days.
The percentage of people discharged from emergency care who receive ambulatory services within 3 days.
The percentage of service recipients who had a change in principal mental healthcare provider during the year or term of treatment.
The percentage of consumers who receive adequate information to make informed choices.
The percentage of service recipients whose treatment follows accepted, best-practice guidelines.

Concerns and Indicators Related to Outcomes

Outcomes are reflected by the extent to which services provided to individuals with emotional and behavioral disorders have a positive or negative effect on their well-being, life circumstances, and capacity for self-management and recovery.

The priority concerns related to outcomes include the following:

Increased access to general healthcare.
Minimal negative outcomes from treatment.
Reduced psychological distress.
Increased sense of personhood.
Reduced impairment from substance abuse.
Increase in productive activity.
Capacity for independent community living.
Increase in independent functioning.
Reduced involvement in the criminal justice system.
Participation in self-help activities.
Minimal recurrence of problems.
Positive changes (in areas for which treatment is sought).
Increased natural supports and social integration.

The indicators presented below are a representative set. The complete list for this domain, along with relevant measures, is presented in the technical appendix.

The percentage of people with mental illnesses who are connected to primary care.
The percentage of consumers who experience a decreased level of psychological distress.
The percentage of consumers who experience an increased sense of self-respect and dignity.
The average change in days of work lost.
The percentage of children with serious emotional disturbances placed outside the home for at least one month during the year.
The percentage of consumers who experience an increased level of functioning.
The percentage of consumers who are involved in self-help activities.
The percentage of inpatient readmissions that occur within 30 days of discharge.
The percentage of consumers who experience increased activities with family, friends, or social groups.

Concerns and Indicators Related to Prevention

Preventive activities are those that are designed to reduce the incidence of mental disorders through (1) early identification of risk factors or precursor signs and symptoms of disorders, and (2) interventions that increase social supports and coping skills in individuals who are at risk for developing mental disorders.

The priority concerns related to prevention include the following:

Information provided to reduce the risk of developing mental disorders.
Interventions designed to reduce the risk of developing mental disorders.

The indicators presented below are a representative set. The complete list for this domain, along with relevant measures, is presented in the technical appendix.

Expenditure per enrollee on dissemination of preventive information.
The percentage of enrollees participating in selected or indicated preventive programs.

Collecting Data and Interpreting the Results

Collecting the data that will allow healthcare providers to assess the performance of their systems vis-a-vis the domains and concerns outlined above requires accessible, reliable data. Suggested data sources that may be used to complete the report card with minimal cost and burden to providers are outlined in the next section. Methodological concerns are also discussed.

DATA REQUIREMENTS AND METHODOLOGY

Typically, data used to complete healthcare report cards are derived from several sources. These include enrollment/encounter and financial information contained in a provider's administrative database, information on individual patients gleaned from clinician assessments and medical records, and consumer surveys.

The Task Force recognized that few provider data systems would include all the elements that comprise this report card. Indeed, several indicators and measures were excluded because of this fact. In other cases, the group proposed alternative ways of obtaining information for some of the measures that are included. At the same time, the Task Force was aware that implementing the report card could be an expensive proposition. In choosing data sources for the MHSIP report card, the Task Force attempted to minimize the potential cost and burden of data collection by using items consistent with other reporting standards.

However, because this report card also covers the critical domain of outcomes, some additional types of information may need to be collected. Where appropriate, these additional data sources are included with specific measures in the technical appendix, along with the instruments and reporting forms recommended by the Task Force The actual costs and staff time involved in gathering the data and tabulating the results will be determined in the pilot-test phase.

Recommended Data Sources

A brief review of data sources required to complete the MHSIP report card is provided below.

Enrollment/encounter data. Enrollment/encounter data, sometimes referred to as "client/event" data, include demographic and other characteristics for individual enrollees and relate these to the types and amounts of services provided. Financial and claims information include the costs associated with services. The technical appendix presents the demographic and financial elements needed to complete the report card's measures.

Clinician survey/chart review. Instruments administered by clinicians when consumers enter services, three months after treatment begins, and/or at discharge will help determine the outcomes of services provided. The Task Force has suggested the following clinician- administered instruments, each of which is included in the technical appendix, along with a table highlighting the subpopulations from whom information for the report card measures will be obtained:

The Child and Adolescent Functional Assessment Scale for children and adolescents.
The Abnormal Involuntary Movement Scale for adults with serious mental illnesses.
The Clinician Alcohol and Drug Use Scale for all mental health service recipients.

To determine whether treatment is appropriate according to established best-practice guidelines, a sample number of patient charts will need to be reviewed.

Consumer self-report. (a) A consumer survey has been developed by the Task Force specifically for the MHSIP report card (see the technical appendix for the complete document). The pilot version contained 40 consecutively numbered questions, including those about general satisfaction, access to services, appropriateness of treatment, and outcomes of care.

Initial testing of the instrument by consumer volunteers in four states revealed that the survey is an appropriate length, easy to understand, and relevant to consumer needs. There was little redundancy in the items. A number of consumers reported that this was the first survey they were able to complete.

To meet the data requirements of the MHSIP report card, a representative sample of service recipients will need to complete this survey annually. In addition, consumers will complete a 15-item symptom distress scale (adapted from the SCL-90 and the BSI), and three items from the SF-36, at entry, three months after entry, and at discharge.

(b) An additional consumer report items form also has been included to consolidate the information from standardized instruments such as the SF-36 and Rosenberg's self-esteem scale that are needed as measures for the concerns. This form also includes such items as work history, involvement with the criminal justice system, and involvement with self- help activities.

Data collection. The Task Force did not address issues of data flow or training. As noted above, instruments completed and administered by clinicians and the consumer report items form will be administered at admission, three months after treatment begins (or at the end of treatment), and a year from admission for those still receiving services. The consumer survey will be administered at discharge and will be used in an annual survey of mental health service recipients.

Methodological Concerns

Providers, researchers, and other experts have raised a number of questions about the reliability and comparability of report card data. Several of these concerns are outlined below.

Survey administration. Information obtained from consumers will be a key element of the MHSIP report card. Based on initial analysis of the consumer survey, Task Force members are confident that high response rates can be achieved, but specific techniques for administering the survey will need to be investigated in the pilot phase. In recent efforts, high response rates have been achieved by including the consumer report as part of the assessment/treatment process, and by using new technologies, such as computer screens with a touch response and automated telephone surveys. The Task Force recommends that states or payers who are purchasing healthcare services be responsible for administering the consumer survey.

Risk adjustment. A problem that has plagued healthcare report cards is the issue of risk adjustment. Different plans may be serving consumers who vary significantly in their mental health status. Variations in outcomes may be related more to the initial health or mental health condition of the consumer than to the services the plan provides. In that situation, report cards would penalize plans that provide services to those most in need.

The MHSIP report card addresses these issues in part by defining measures for specific populations (e.g., children and adolescents with serious emotional disturbances, children and adolescents with other emotional disturbances, adults with serious mental illnesses, adults with other mental illnesses, and adults with a dual diagnosis of a mental illness and substance use disorder). A table that outlines report card indicators and measures by population is included in the technical appendix. In addition, the Task Force recommends that initial differences in the base information available in enrollment/encounter data systems be adjusted using appropriate analytic methods.

Data quality. Information contained in administrative databases and patient records may be incomplete, inaccurate, or misleading. Standardizing data across providers will require staff training and the implementation of quality assurance methods.

Report card format. The report card must be "user-friendly" and accessible to both consumers and professionals, and should include an explanation of variations and trends. Several suggested report card formats are included in the technical appendix. These are for illustrative purposes only; the final format of the report card has not been determined at this point.

A Work in Progress

Mindful that mental health report cards are in their infancy, the Task Force considers this document to be a work in progress. As it is refined through pilot-testing, it will become a keystone in the national dialogue about how to assess the quality and costs of behavioral healthcare services. Specific recommendations for pilot-testing are outlined in the final section.

RECOMMENDATIONS FOR PILOT-TESTING

The Task Force has issued the following recommendations for pilot- testing of the report card in Phase III of this project.

The Center for Mental Health Services should issue a specific Request for Application (RFA) for pilot sites to test the MHSIP report card.
The RFA should require that the pilot site include the consumer self-report survey, enrollment/encounter data, and the clinician- administered instruments the Task Force recommends.
The pilot site should comprise multiple providers and include children and adolescents with serious emotional disturbances, children and adolescents with other emotional disturbances, adults with serious mental illnesses, adults with other mental illnesses, and adults with a dual diagnosis of a mental illness and substance use disorder.
An implementation review and evaluation should be an integral component of the RFA. The evaluation will allow the report card to be streamlined and revised as needed.

SUMMARY

The need to monitor healthcare reforms is becoming increasingly critical. As managed care plans proliferate, corporate and public healthcare purchasers, health plan administrators and providers, and individual consumers want reliable methods to determine the relative cost and quality of services. Report cards are one way to meet this need.

Designed for and with mental health consumers, the MHSIP report card is uniquely suited to assess the outcomes of mental health and substance abuse treatment. It is predicated on the notion that certain attitudes, processes, and services must be in place in order to address such key concepts as recovery, personhood, and self-management. Though these concepts initially may be difficult to quantify, the Task Force believes that outcome data will become more accurate as organizations are held accountable for reporting it.

The MHSIP report card is a work in progress. As it is refined through pilot-testing, it will become a standard against which future mental health report cards are measured. When mental health consumers have a reliable way to choose services that best meet their needs, they will be true partners in efforts to reform healthcare in this country.

ENDNOTES

U.S. General Accounting Office. Health Care Reform: Report Cards Are Useful but Significant Issues Need to Be Addressed. Washington, DC: U.S. General Accounting Office, 1994.
Ibid., page 6.
Pursuant to section 1912(c) of the Public Health Service Act, as amended by Public Law 102-321, "adults with a serious mental illness" are persons:
- Age 18 and over,
- Who currently or at any time during the past year,
- Have had a diagnosable mental, behavioral, or emotional disorder of sufficient duration to meet diagnostic criteria specified within DSM- III-R,
- That has resulted in functional impairment which substantially interferes with or limits one or more major life activities.
Pursuant to section 1912(c) of the Public Health Service Act, as amended by Public Law 102-321, "children with a serious emotional disturbance" are persons:
- From birth up to age 18,
- Who currently or at any time during the past year,
- Have had a diagnosable mental, behavioral, or emotional disorder of sufficient duration to meet diagnostic criteria specified within DSM- III-R,
- That has resulted in functional impairment which substantially interferes with or limits the child's role or functioning in family, school, or community activities.
Personhood is emerging as a critical concept and goal in the treatment of serious mental illnesses. It subsumes dignity, self- respect, self-mastery, self-esteem, and self-worth. More work is needed to operationalize and measure this concept, but its fundamental nature should be reflected in any set of mental health outcomes being proposed.
The concept of recovery, though it has many different interpretations, has become integral to any discussion of serious mental illness. See, for example, Anthony, W.A., ARecovery from Mental Illness: The Guiding Vision of the Mental Health Service System in the 1990s," Psychosocial Rehabilitation Journal 16(4), 1993.

Recovery is important to consumers of mental health services because it underscores the fact that most people in treatment for psychiatric conditions do get better as a result of treatment. See, for example, Center for Mental Health Services Mental Health Statistics Improvement Project, Outcomes of Treatment for Mental Illnesses. Rockville, MD: Substance Abuse and Mental Health Services Administration, 1994.

Behavioral healthcare plans that stress recovery indicate a commitment to personal independence and participation in community life for people with psychiatric and substance abuse disorders. See, for example, Blanche, A., et al., "Consumer-practitioners and psychiatrists share insights about recovery and coping." Disability Studies Quarterly 13(2), 17-20.
This category may have to be broken down further into adults with serious mental illnesses and substance use disorders and adults with other mental illnesses and substance use disorders. viii. See, for example, Stroul, B, Community Support System for Persons with Long-Term Mental Illness: Questions and Answers. Rockville, MD: National Institute of Mental Health Community Support Program, 1988.
See, for example, Stroul, B, Community Support System for Persons with Long-Term Mental Illness: Questions and Answers. Rockville, MD: National Institute of Mental Health Community Support Program, 1988.

REFERENCES

Report cards, performance measurement, and monitoring systems surveyed.

Florida Dade County Status of Children.
Consortium Research on Indicators of System Performance Report.
HEDIS 2.5.
Kaiser Quality Report Card.
United Healthcare Performance Indicators for Choosing Managed Behavioral Health Care (United Behavioral Health).
New Hampshire Outcome Based Performance Indicators.
Vermont Key Performance Indicators.
Montana Regional Performance Data.
Colorado Incentive System.
Hoosier Assurance Plan--Provider Profile System.
Washington Regional Support Network MIS.
Texas Strategic Planning/Budgeting Performance Measures.
Hawaii 1994 Program Evaluation Data Set for CMHCs.
Minnesota Annual Performance Report.
Utah Annual Statistical Report.
California Medi-Cal Inpatient Consolidation Monitoring System.
California Adult Performance Outcome Survey.
Oklahoma Mental Health Information Systems.
North Carolina Division MH/DD/SA Outcomes.
Oregon Quality Assurance for Managed Care Indicators.
Rhode Island Division of Integrated Mental Health Systems Report Card.
Healthy People 2000.
Institute for Behavioral Healthcare Performance Standards.
AMBHA. Developing a Collaborative Report Card: The American Managed Behavioral Healthcare Association's Experience. Panzarino, P.
National Leadership Council Task Force. Industry Performance Standards and Monitoring: Preliminary Report.
Performance Indicators for Mental Health Services: Values, Accountability, Evaluation and Decision Support. Final Report of the Task Force on the Design of Performance Indicators Derived from the MHSIP Content. 1993.
Digital Equipment Corporation Performance Indicators for HMOs.
Group Health Association of America's Consumer Satisfaction Survey. A. Ross Davies and J.E. Ware.
Design of a Survey to Monitor Consumers' Access to Care, Use of Health Services, Health Outcomes, and Patient Satisfaction. Research Triangle Institute. January 1995.

Substance Abuse & Mental Health Services Administration