Skip Navigation
Text Size:

Using Data to Inform and Guide Services to Persons with Co-Occurring Disorders

April 12, 2011
2:00-3:30pm EST

Coordinator:   Welcome and thank you for standing by. At this time, all participants are in a listen only mode. During the question and answer session, please press Star 1 on your touchtone phone. Today’s conference is being recorded. If you have any objections, you may disconnect at this time. Now, I will turn the meeting over to your moderator, Ms. Beth Fraster. You may begin.

(Beth Fraster):   Good afternoon and welcome, consumers, family members, advocates, service providers, policymakers, administrators, and researchers. We have over 500 people with us today and we welcome all of you. Before we begin our presentation, we’d like you all to know, today’s event is being recorded and will be archived on SAMHSA Co-Occurring News Web site at Twenty four hours after this presentation, the presentation in its entirety can also be replayed on the Verizon site. This link will be sent to you in a follow-up email. Today’s Webinar, Using Data to Inform and Guide Services to Persons with Co-Occurring Disorders, is sponsored by the Substance Abuse and Mental Health Services Administration, co-occurring disorders, integration, and innovation, CODI. My name is Beth Fraster, and I work for Advocates for Human Potential. On behalf of SAMHSA and the CODI Project, it is my pleasure to serve as your moderator for today’s presentation.

This is the fifth in a series of CODI Webinars centered around SAMHSA’s eight strategic initiatives. The first Webinar in the series, Co-Occurring Disorders and Justice-Involve People, held on April 13 is archived on the GAIN Center Web site. The second Webinar, Healthcare Reform: Implications for Behavioral Health Providers, held on September 21, is archived on the CODI Web site and the National Council Web site, as well as on YouTube. The third Webinar in the series was Understanding and Addressing Trauma among People Receiving Behavioral Health Services. This was held on November 9, 2010 and is also archived on the CODI Web site. And the fourth Webinar, Integrating Behavioral Health into the Person Centered Healthcare Home was held on January 19, 2011 and is also archived on the CODI Web site and can be viewed on YouTube.

We have two guest speakers today who will bring a great wealth of information alive and provide concrete information to assist you in your work and support of people with co-occurring disorders. We have a full agenda, beginning with welcoming remarks from Rear Admiral Peter Delany, and we will follow with Dr. Mark McGovern. Then we will open the discussion to include you, our listening audience.

Before we begin, I’d like to draw your attention to some important logistics. You may submit questions at any time throughout the presentation using the dialogue box on the top of your screen. If you see the question and answer screen at the top, you just click there and you can write your questions at any time throughout the presentation. The presenters will respond to these questions during the question and answer period. All participants will receive a follow-up email with a link to an evaluation form. We appreciate everyone’s feedback and we’ll use it in developing future Webinars.

We will begin with comments and introductions from Rear Admiral Peter Delany. Rear Admiral Peter Delany is the Director for the Center for Behavioral Health Statistics and Quality at the Substance Abuse and Mental Health Services Administration. He provides direction to a diverse office of professionals, charged with the collection, analysis, and dissemination of critical behavioral health data. I will allow him to set the stage for this Webinar and introduce our main speaker, Dr. Mark McGovern.

Peter Delany:   Well, good afternoon everybody. It’s a pleasure to be here today and very weird. I’ve never done a Webinar, so this is my inaugural attempt, so it’s pretty cool. As Beth had noted, I’m the Director for the Center for Behavioral Health Statistics and Quality at SAMHSA and I am also serving as the Strategic Lead for the Data Outcomes and Quality Initiative. So obviously, I’m pretty biased that this is a critical topic that you’re bringing up today. For the many of you may know, that we had eight strategic initiatives that we’re working on. The primary one and the most important is the work on prevention of substance abuse and mental health - mental illness. That is the number one choice and it's probably also written into the Secretary’s ten top initiatives.

We’re also working on trauma and justice, military families, recovery, and support, health reform, health information technology, data outcomes and quality, and public awareness and support. So those last two really are initiatives that are meant to cut across every other initiative. They really underscore what we’re doing here at SAMHSA. So, if you’ve looked at the papers over the last years, it’s been pretty clear from this administration, as well as from other fields, that data has become a critical part of trying to figure out how we lead with data to help inform our policies, as well as to measure our successes.

So as health reform moves forward, it becomes very important and critical to begin to think about how we integrate behavioral health and primary care, but then how do we document both what we do, and then document our success, or document when we’re not being successful and then how do we change it to move it towards success?

So in this initiative, the one that I lead, Data Quality and Outcomes or Data Outcomes and Quality, we keep kind of shifting those. What we focus on is, first of all, identifying what the problem is, and then setting goals and then tracking results, so that we can improve the quality of care and the outcomes for the people that we serve.

The importance of that is that, at the end of the day, this is not so much about an initiative, as it is about making sure that the people that we serve, those who are often the most vulnerable in society, are actually getting high-quality care that leads to measurable differences in their lives. So, there are four primary objectives that we’re working on, or four primary goals. The first is to figure out how to develop an integrated approach for the collection and analysis and use of data. And what that means is that we’re focusing on bringing up, kind of parity, in our data collection efforts in terms of mental health and substance use, as well as in prevention, and bring those to parity, so that we have the full breadth of issues covered.

The second area is to begin to create a common set of standards to measure the quality of care, to measure outcomes, and to help the field do - have at least a constancy in figuring out how they can collect data and then use it to meet the needs of those they serve. The third goal, is really to look at improving how we evaluate our programs here, as well as the integration and development of a greater services research program.

Finally, we’re working on improving the quality and accessibility of our data and the information and putting that out in multiple different contexts, so that people can see, not only what we’re doing, but then they can understand how they’re doing. If we’re going to do this — if we’re going to make this work — we’re going to have to figure out better and more efficient approaches to integrating the data across service systems. Since the effectiveness and cost effectiveness of behavioral health services are sometimes realized more through reductions in service utilization rather than in improving of outcomes. I’m very happy that the co-occurring disorders integration and innovation project

The learning community that supports states using dual diagnosis capability in addiction treatment, dual diagnosis capability in mental health treatment, and dual diagnosis capability in health care settings, promotes a good use of these tools and helps states to use these tools, not only to measure organizational capability to provide services for individuals with co-occurring disorders, but also to measure and encourage progress. I understand that the COSIG learning community also is incorporating issues related to data integration in a series of issue briefs that it is developing, especially with respect to workforce issues, screening, and assessments.

Today you have a fantastic speaker who’s going to discuss the use of data, to assess the quality of services to persons with co-occurring mental health and substance use disorders, on three levels: the systems, the program and the patient level, and on the treatment process level.

Let me introduce real quickly Dr. Mark McGovern. He’s an Associate Professor of Psychiatry and Community and Family Medicine at Dartmouth Medical School, in New Hampshire. He’s a practicing clinician and a treatment services researcher with expertise in addiction and psychosocial interventions for co-occurring substance use and psychiatric disorders. Dr. McGovern is a primary developer of the dual diagnosis capability in addiction treatment index and he coordinates a multistate learning community. I’m going to turn it over to him now and I’m going to wish you well. I’m going to have to leave in a few minutes, because I have a budget meeting, a little pesky thing down on the Hill has kind of been keeping us all busy. So I wish you well for the rest of the day. Dr. McGovern.

Dr. Mark McGovern: Thank you Rear Admiral Delany. Good to hear you and good to hear your kind introduction. I also appreciate your leadership at SAMHSA and many of the goals you outlined, I think, are laudable goals for the field and, hopefully, fit very nicely with what treatment providers and policymakers and patients and consumers want to know about the services they receive.

So, good afternoon, good morning everybody. My name is Mark McGovern. I’m happy to be here. This is not my first Webinar, but I still experience them as rather strange in that it’s hard to know what the nonverbal cues are in terms of when I speak and the connectivity with the audience. So hopefully, I’ll be able to imagine people enthralled and being able to understand what it is that I’m speaking to and hopefully, if not, there will be plenty of time for questions at the end. And I do realize that what I’m talking about, and you’ll see this sooner rather than later, involves a certain amount of detail and, given the amount of time we have, I’m really going to ask you to feel free to contact me after the presentation directly if there are specific measures or information about measures that you’re curious about or you know, in the absence of a good search engine, really need help locating out there in the universe.

But, we now like the term “dashboard indicators” and I think it’s a very useful measure to talk about how important data can be for us as we’re driving a vehicle or, I suppose, if we’re pilots, it’s something we think about as being very important. You know, in terms of the metaphor, it helps us to steer, it helps us to make a decision whether to go faster or slower, stop, change direction and, as such, hopefully can guide decisions. And sometimes, instantaneously, has decisions that we might need to make. Our field as I think, Peter suggested, often lacked clear and consistent and common standards that things that could go on the dashboard. Some of us have used some gauges and dials, and others have used other gauges and dials, and some of us have really just been kind of looking out the windshield and hoping for the best, you know, maybe even not looking out the windshield,

So, I’m going to lay out what I think is a reasonable set of potential dashboard indicators and, as such, it’s — you know, it’s somewhat of a long list, but I think I’ll try to reduce it to a set of key factors that one might consider. At this point, I think the idea is, that we — you know, if we had these kind of indicators — we could evaluate our success, evaluate our outcomes, compare programs, compare and contrast interventions, evaluate the impact of training, evaluate the impact of evidence-based treatments that we’re advised to implement in our service systems.

However, I think I live enough in the real world to know that being able to operate with these sort of dashboard indicators is, at this point, probably more ideal than real but, hopefully, we’ll be giving you enough information along the way here that lets you think about things that you might consider implementing, if you’re not implementing already.

Ideally, the dashboard indicators involve at-a-glance information, so that you would know, how your system, your agency, or your clinicians are doing. Now, I probably could have put how consumers, patients, clients are doing, so that it’s the equivalent of a behavioral health vital sign, if you will. Information must be relevant, so that we’re not collecting data that, you know, are really not at all with clinical implications, that accurate, you know, that really is valid, that relates to the thing it is that we’re hoping to measure.

That’s timely, and I put, real time, because I think as close as you can get to real time, is optimal. However, based on the nature of data that we collect, sometimes you might get data at the end of the month or the end of a quarter or the end of the year.

And last, and I can’t under — I can’t overemphasize this, I probably could under emphasize it, but I think we want to keep it simple. And I might go on for the rest of the hour to contradict myself on this one, but to the extent that data are complex, that they’re hard to gather, that they’re hard to decipher, you’re probably not going to use them.

So I’m a big advocate for keeping things as short and sweet, but hopefully, reliable and valid as possible. So where there’s an opportunity for me to suggest something simple, I’m going to try to do that.

We’re here this morning, this afternoon, to talk about people with co-occurring disorders in particular and it’s really nice to hear someone from SAMHSA in a key leadership position talk about co-occurring disorders and behavioral health and integrated services and integrated treatments and, even more, integrated data.

As many of you out there know, if you’ve been around, there’s parallel systems that often collect different kinds of data and the thought of actually having a common database, a common dataset with uniform questions that cut across both systems of care, is awesome.

And I think that’s the kind of thing I’d like to spend some time talking about this afternoon.

Consumer-level outcomes are typically what people think of when they think about outcomes, right? Is the person better than they were when they started? Have we done them at least no harm, and preferably good? Have we helped them to heal? Have we helped them to recover? Have we helped them to feel safer? Have we helped them to lead a more productive life? Have we done so in a collaborative way that emphasizes their strengths? The interesting thing is, and I sort of get this question all the time, what’s a co-occurring outcome?

And here, unfortunately, there is no single co-occurring outcome. And typically what we collect are a range of outcomes and the first two probably make the most sense to focus on.

Substance use: you know; is a person using substances less than they were when they first started treatment? And then how do you know? And are there fewer consequences associated with their use if they are using?

Mental health symptoms; sort of the other side of the street. Are people less symptomatic? Are they more functional? Do they feel safe? Do they feel more hopeful?

So both substance use and mental health symptoms are two key outcome, primary outcome measures to be thinking of if one is contemplating integrative treatments, behavioral health, integrative function, or co-occurring outcomes.

Typically, we also like to look at life — other life areas, including legal issues. Are folks less involved with the criminal justice system, engaged in fewer illegal activities, social and family, how people are getting along in the world and key relationships, in terms of good citizenship, fitting in, contributing, feeling comfortable in their own skin.

Work and academic functioning, are people feeling productive? Are people productive? Are they engaged in employment with support or without support? Are they back in school?

And then, lastly, medical and health issues. So medical, typically we think of illnesses and managing disease and reducing physical consequences of mental health or substance use disorders.

Health is sort of the preventive side of that. So — how are — how’s our folk’s diet, exercise, activity levels? You know, body mass index; you know, things along those lines. So medical and physical health can be measured from a kind of pathanomic to a more positive continuum.

Treatment satisfaction is another key outcome. Do people value the services they receive? And I’ll talk in more detail about how treatment satisfaction has been measured.

The cost of providing the treatment, the cost offset of having received the treatment and, you know, as Peter mentioned at the outset of great interest is if people get good care, are there other health costs reduced further downstream?

Some studies have actually found that, some studies have found the opposite, where, if people get good care, often the idea that they need more care for other problems that have previously been neglected, sometimes involves at least in the short term, an increased cost.

So, being able to look at cost, the cost of providing the service, the cost offsets that having received the service, is you know, typically an issue that legislators and funders really want to know about.

Treatment utilization, this is an outcome that might be considered favorable if it’s outpatient or less intensive service utilization. It might be considered unfavorable if it’s service utilization along the lines of hospitalization or emergency department visits or even detoxification program recidivism. So treatment utilization is often used as a proxy for positive or less than positive outcomes.

Peer recovery support group involvement has been associated with recovery and good outcomes and stability of outcome. So, measuring the degree of involvement and participation and affiliation is often an indicator of how effective your program might be.

Quality of life, perhaps, you know, thrown into the middle here, it actually might and should be at the top of the list. So are people feeling fulfilled and happy and useful, free of symptoms, you know, free to make choices, unburdened, you know, like anyone else, so to speak.

Death is an outcome. And you know, those of you that work in settings where you deal with high-risk patients and high-risk situations, in terms of suicide or chronic disease, you know sometimes, that being able to, you know, to extend life is a favorable outcome. Death is an inevitable outcome, but hopefully, something that can be delayed with very good care.

Citizenship is an aspect of recovery; at least it has been highlighted in the addiction recovery movement, which obviously dates back decades, but the idea of service and giving back. And you might identify it as a part of a 12-step work, but not exclusively that, but the idea of being a contributor and a role model to others who share the same issues.

Recovery, it sounds like that is one of the SAMHSA’s eight key initiatives, which is good to hear. I think the idea here is that what we do is beyond symptom reduction, you know, beyond eliminating things that are negative and as much about creating an opportunity for people to lead a fulfilling positive life and not just involved in symptom relief.

So it’s a broader, and I think if I dare say, more holistic orientation to the services we provide.

So that’s a bit of a list of things that might be considered dashboard indicators. It’s an awful lot. So you probably think, well Mark, what are the few things we could look at that might be, you know, might be simple, like you said at the outset and I’m going to tell you to stay tuned. I’m going to get to that.

One other — a few other considerations in outcome is short versus long-term perspective. So you know, if you’re in the pharmaceutical industry, you might be looking at 12-week outcomes, you know, response to medications.

If you’re following people with chronic disease, it might years. If you’re an oncologist, you might be thinking about 5-year outcomes. And you know, beyond 5 years, obviously care and monitoring continues. So, for mental health and addiction treatment, it’s good to keep in mind what your perspective in terms of timeline might be. Unfortunately, we don’t have a lot of long-term perspective outcome studies.

You know, there are a few examples. George Valiant’s study of alcoholic men in Boston and studies of heroin-addicted folks in Los Angeles by the group at UCLA.

The group at Dartmouth has studied dual disorder patients, using integrated dual disorder treatment. So there’s really only a short — relatively short list and ’m leaving others out, but I see those as exemplar studies.

Other questions, are during treatment versus post treatment measures. Do we measure people at the outset and at the end of treatment and during treatment or do we measure them after they’ve left treatment? And I think “post” is in quotation marks because for the most part, the disorders and problems we treat aren’t cured. People will require some type of ongoing care, if not monitoring, for the foreseeable future. So post-treatment might mean at a 1-year followup or a 2-year followup or, if we go with a cancer analogy, a 5-year followup. Are you symptom free? Are you in remission? Are you leading a good quality life, etc.

So, really considering your opportunity and time for measurement is a good consideration.

This is a bit of a non-sequitor, but often we’re — we proceed without data. We have no map. We do our work with an end of one at a time, rather than an end of 100 or a 1,000 or 10,000.

The idea here is, that we’re encouraging all of us really, to begin to collect data so that we have an idea about where we are and where we’re headed.

Folks really like the idea of having information to share with patients or consumers at the outset of treatment to describe treatment options and likely benefits or, you know, no guarantees, but if you take this medication, your chances are this to have reduced symptoms.

Or if you elect to not take the medication and instead use the time with a therapist who’s going to be working with you in a cognizant behavior approach, your chances for symptom reduction are this.

We don’t do that. Wouldn’t it be great if we could do that? Lastly, many of us study interventions, which are relatively circumscribed approaches to address a problem.

In the real world, we typically use services or models — in other words, a group of interventions that are clustered together that hopefully, leverage positive change. Models are probably more realistic, harder to study. Interventions are — may be more simple to study — but probably less realistic, because they’re often embedded in a range of other kinds of services that are typically offered.

Program-level outcomes are probably — many of you listening today might be looking at program-level outcomes, if you’re collecting data across a range of services in group or a set of clinicians. So you might look at aggregate outcomes. So those patient-level outcomes added and

So what is the percentage of people who you treat that are absent at 1 year as a kind of an example of that? Or what is the percentage of people who’ve reduced substance use by 50 percent at the end of 3 months in outpatient treatment?

Process outcomes, and I’m going to talk about these later as Washington Circle indicators, other people have been involved in process outcomes more recently using the work of IATECH and that was for the improvement of addiction treatment.

And these are relatively easy indicators to collect and I’m an advocate for these indicators because I think they can give you some of what you need at a glance to learn about how your program’s doing.

Another program-level measure is called the Organizational Readiness for Change. You “Lord of the Ring” fans might be able to read that as ORC. In this case, ORC is a favorable group or a favorable thing to have in your tool box and it measures the degree to which your organization is ready to either implement evidence-based practices or stable in a kind of organizational and motivational sense to be able to adopt innovation. So it’s a really nice measure developed by folks at Texas Christian University, most notably Wayne Layman.

Another program-level outcome that you might already collect, although sometimes folks don’t add these events up, is adverse events and serious adverse events. So, a serious adverse event could be a death or a hospitalization. You know, an adverse event could be, you know, return to a previous pathological level of functioning.

So being able to count these events and develop a benchmark so that you have a sense of what these are over time, and be able to monitor a high and low periods, and connect those with changes in programming or staffing is a relatively straightforward thing to do.

Other program-level outcomes that people have been engaged in monitoring recently are the use of evidence-based treatment. So taking a look at the range of services that we offer, how many can we say, or what percentage or proportion of services that we offer can we say are evidence based?

And how do we verify that? Do we actually take a look at fidelity or integrity or adherence to the treatment as its — as it was developed, rather than just as ’s been named.

And then the last thing and this has been part of our work here at Dartmouth for the past, oh — I should say 20 years or so, is the issue of program level

And that’s essentially taking a look at the capability of a program to — from a policy and treatment and workforce prospective — how capable is the program in

Co-occurring capability, there are a number of different measures for that. One is the integrated dual disorder treatment, or IDDT fidelity scale, which really is designed to assess a treatment team within the context of a community mental health program.

The compass measures — and this was developed by Ken Minkoff and Christine Kline — the self assessment, and these are measures that can be used for either mental health or addiction treatment agencies and can completed by a staff — a number of staff within the agency and really serve to raise awareness about the current co-occurring capacity.

The three measures that we’ve been involved with are the dual diagnostic capability and addiction treatment, the dual diagnosis capability in mental health treatment and, more recently, the dual diagnosis capability in health care setting indexes or indexes, affectionately known as the DDCAT, DDCMHT, and the DDCHCS. So, if I use the shorthand version, I’m calling them those different animals there, the DDCAT, the DDCMHT, and the DDCHCS.

System-level outcomes are things that folks working at a state agency, or if you’re involved in a managed care organization, or if you’re involved in a uniform services group of behavioral health care providers or the Veteran’s Administration, you might be interested in aggregate patient-level outcomes.

You might be interested in those process outcomes like Washington Circle Indicators or NIATECHs indicators to see how they vary across programs under your aegis.

You might be interested if you’re a state agency in the percentage of evidence-based treatments that are offered by programs across your territory or region.

And lastly, you might be interested in the co-occurring capability of programs that you fund. And also, as a consumer, you might be interested in almost a consumer’s guide to the co-occurring capability of programs that you might have in your area to choose from.

So very — potentially very useful information if this sort of data — if these sort of data were posted, ala a consumer’s guide, it might really help a consumer make a choice about where they can obtain treatment and feel most comfortable with ’re making.

Of course, as a provider, having this sort of data available, is — makes us all a little squeamish, but I would argue for a transparency on these issues and, for the extent that we collect these data, I think we’re doing our field a world of

As I said earlier, we’ve been involved in collecting data using the three measures, the DDCAT, the DDCMHT, and the DDCHCS.

We have data across a number of states and this just shows a snapshot of some of the data that we’ve collected as a sort of system — almost a national system

The pie chart on the top left of your screen, the upper left hand corner, is a DDCAT, so these are addiction treatment programs across a number of states. And as you can see, about 80 percent of the programs fall into a category of addiction-only services, 20 percent or so are at least dual diagnosis capable or dual diagnosis enhanced.

So, what this means, and this is based on the American Society of Addiction Medicine Taxonomy of program services, that the majority of addiction programs tend to focus on people with primary substance use disorders and, to some extent, do not address mental health issues.

On the DDCMHT side, the top right hand corner, you can see that comparable mental health only services account for close to 91 percent of mental health programs. What this means is that the majority of treatment agencies that were assessed focus only on mental health issues, and do not address substance use issues, at least not in a consistent and systematic way.

The bottom pie chart or cheese chart, if you will, shows a small number of programs. This number is actually currently being increased and, if any of you are out there listening and would like to learn more about the DDCHCS and possibly help with some of the pilot testing of the newer version of the measure, please let me know.

But at this point, a small number of agencies were assessed and two out of three were health care only services. A third, one program, was dual diagnosis capable.

So what this says is that the majority of programs at baseline tend to focus on, you know, the particular disorder that they’re, you know, that they’re organized to treat and not offering integrated or dual diagnosis capable services.

On the other hand, and this is, I think, a large part to the SAMHSA-funded initiative through COSIG grant mechanisms that, if a change strategy is intentional — in other words, if people put their mind to it and their —they can become dual diagnosis capable with some effort.

So, we also have been engaged in a number of studies trying to improve services for people with co-occurring disorders, measuring these at the program level, and as this slide indicates, you know, through the process of program improvements, at least on addiction sides, the rates of dual diagnosis-capable programs have increased threefold on the mental health program side, close to eightfold, in terms of dual diagnosis capabilities.

So, we actually have more data on this and there’s a paper that has recently appeared in the Journal of Dual Diagnosis describing the study so, if you want to check that out or have trouble finding it, just let me know.

Data information options — and I think this is the part where it would really be great to have some consistency across our field — you know, much like health care has vital signs that look at pulse and heart rate and body temperature and blood pressure, our field has those kinds of key vital sign indicators, but our ways of going about measuring them vary considerably.

So we have few what you might call, gold standard indicators. And I believe our field has suffered from that kind of diversification and lack of standardization.

And I’m hopeful that, with leadership as Peter Delany offered, that might be something that is in our lifetime that can ultimately be addressed.

For substance use, you know, if you’re an addiction treatment researcher or you know a lot about substance use, you recognize that there’s a whole lot of different ways to measure substance use.

Frequency of use, for instance, the percentage of days used out of the past 30 or 90. There are different measures of the problem severity of substance use, often involving the consequences of use.

So, one as much of a gold standard measure as we have in the field, is the addiction severity index or ASI, which is really a problem composite related to drug and alcohol use.

There’s a proprietary measure called the GAIN, or the Global Appraisal of Individual Needs. Many of you who have probably been SAMHSA funded for adolescent treatment services have had some experience with the GAIN.

It’s also a good measure of substance use and the range of issues that come with substance use. There are a number of other measures of alcohol use disorders; the audit and the dudit are two relatively well known ones.

There are simple screening measures for alcohol and other drug abuse. These measures were included in the TIP 42, the co-occurring disorder TIP, published by SAMHSA, principally authored by Doctors Sax and Reeves, so the SSIAOD, the CAGE aide, the assist, are all measures and, for the most part, unless I refer otherwise, the measures I’m describing are public domain measures.

We like that. We like the public domain measures, because we, you know, we — we’re

Diagnosis is another measure of substance use. So do people meet criteria for substance dependence? Do they meet criteria for substance abuse, or do they not meet criteria for either of those disorders?

And then, last and probably not least, is our toxicology data. This could mean urine, breath, or even blood and hair data. And the idea here is that this is probably the best data that a person with a substance use disorder has to convey how they’re doing.

So, those of you in addiction treatment are very knowledgeable and comfortable with toxicology data for substance use. The fields of mental health providers and behavioral health providers are increasingly comfortable obtaining toxicology data around substance use issues.

For mental health symptoms, I — you know, the range is longer here or the list is longer here, so it goes from the modified mini-screen or MMS, which is a broad spectrum measure with several components, including a mood disorder component, an anxiety disorder component, including several items related to trauma and

And lastly, a sort of psychotic spectrum component. And if the measure yields three primary scores along those three domains, as well as a total score, it’s a very easy measure to complete. Measure symptoms or asked about symptoms over the past 2 weeks.

So it could conceivably be used as a baseline and followup measure. I referred early to the Addiction Severity Index, which is in many ways an integrated assessment measure, in that it assesses both substance use and mental health issues.

So, being able to use the Addiction Severity Index to assess psychiatric symptoms and problems is a really good measure to be able to employ.

Global Assessment Functioning, which is a single measure on a range from 0 to 100. Some people point to the fact that the GAF is somewhat unreliable, in that the same clinicians will vary by 20, 30, 40 points but, with good training those variances or that variation can be significantly reduced, so the GAF is actually a fairly good and short and sweet measure of functioning to GAIN.

I referred to that earlier, as a Global Appraisal of Individual Needs, has very excellent components that have to do with psychiatric disorders, for both adults and adolescents.

So it’s well deployed, using measures of externalizing or internalizing disorders among youth.

The Breeze Psychiatric Rating Scale assesses more severe psychiatric symptoms and is a clinician rating scale. It’s in some respect a gold standard, if you typically work with individuals with severe mental illnesses.

The last two measures, the Symptom Checklist 90 and the Breeze Symptom Inventory, which is a sort of a shorter version of the FCL 90, are proprietary measures, nine factors, meant to measure subjective distress around mental health symptoms and are very sensitive to change, a self-report measure, also a very good measure.

You might be interested in specific symptoms, so one of the items that SAMHSA has targeted as an initiative is trauma-informed and trauma-related services.

So being able to assess trauma in PTSD seems like a good idea, especially since we know for a person who has those symptoms, their chances to benefit from treatment are less than average.

So a PTSD checklist, or PCL, is a relatively short self-administered public domain inventory. There’s also the PTSD, primary care checklist, which is a four-item scale developed by the National Center for PTSD and for use in primary care settings is really a four-item screening measure for PTSD.

We know that there are many measures for depression, which is probably the most common mental health problem, mental health symptom, so the Beck Depression Inventory is one. The Zung Depression Inventory, the Hamilton, the PHQ9 are all excellent measures.

For those of you interested in social anxiety or social phobia, there’s a social interaction anxiety scale. And, again, many of these measures are in the public domain.

There’s an ADHD Rating Scale, which covers symptoms related to ADHD. And again, for mental health symptoms, there’s the issue of the presence or absence of a psychiatric diagnosis, which also might be considered an outcome indicator.

Life problems and functioning, the range of possibilities is fairly broad here: to some extent these things get a little bit more complicated to measure, but in terms of social and family functioning, the frequency of contact, the satisfaction with contact, the degree of conflict or perceived support, perceived closeness, legal issues, range from arrest and incarcerations to illegal activity and income, resulting from illegal activities, whether it be prostitution or drug trafficking.

Work and academic areas of functioning and medical areas of functioning obviously are very important in terms of outcomes.

Treatment satisfaction is an area that we typically find important. Some state agencies will require funded treatment programs to assess treatment satisfaction.

There’s a measure called the CSQ8, which is an eight-tem version of a longer version of the Client Satisfaction Questionnaire; available by Larson and colleagues, it is a nice public domain measure.

There’s also the Treatment Services Review that’s been developed by Arthur Alterman and John Casiola and Tom McClelland at Treatment Research Institute of the University of Pennsylvania. It's somewhat longer, but also assesses satisfaction with services. It turns out, that for treatment satisfaction, these two questions, “Will you come back?” “Would you recommend to a friend or family member,” are very, very predictive. So, if you have to ask two treatment satisfaction questions, these would be the ones.

Typically we put these things on five-point scales and we only hear back from 25 percent of those who we give them to and you know, quite naturally, there’s a favorable response bias. So, I guess Caveat Emptor when it comes to measuring treatment satisfaction.

Service utilization — I spoke earlier of being able to track the favorable and unfavorable indicators, so use of Outpatient Services, compliance with medications. Less favorable indicators are emergency room visits, inpatient hospitalization, detox services, and incarceration.

So things that cost a lot are typically viewed in a less favorable light. The cost of providing services, we spoke about these earlier, being able to measure these. These are typically hard variables to measure and often require a lot of effort to obtain.

And sometimes the help of a health care economist or behavioral health economist to help you plan this sort of information. But, if you have these data, you can make some great points to your legislature.

Process outcomes, I spoke about earlier, and I think we think about these as emanating from a few different sources — Washington Circle and NIATECHs and I’m just going to talk about a few. Access, so the percentage of consumers that bidded with co-occurring disorders. So is

So that’s an indicator that you might want to gather. How long does it take for people to get into services, from point of initial contact to point of first appointment? How long does it take for them to go from intake to their initiation of care?

So these are all data that might be easy to put on a dashboard indicator and typically involve an Excel Spreadsheet or an Access Database or even a “Blast from the Past,” a logbook that tracks this kind of information. And

Engagement. So the percentage of people, the rates of people who complete a second session or a third session or a fourth session if you’re an outpatient setting.

So we’d like these numbers to be 100 percent. We’re not curing people in four visits, so we like these numbers to be as close to 100 percent as they can be.

But what are they? What are these numbers in your program? If you’re an inpatient setting and you’re an acute care setting, this figure might not be that relevant. But, if you’re a longer term care setting or a residential addiction treatment program, being able to track retention or engagement in the short term to how many — what percentage of your patients make it 2 weeks or 3

Again, this probably would need to be adjusted based on the planned length of stay, but being able to track these data can be done in a very low tech way.

Retention, percent completing treatment or continuing at a predetermined level, so how many people have we retained at 3 months? If you’re a methadone provider, how many patients have you retained at 1 year or, if you want to be more liberal, how many patients that started your services have you retained at 6 months?

If you’re a residential or inpatient setting, we know that magic can happen in those settings, very critical levels of care; however, the rubber really meets the road when people transition from residential or inpatient to outpatient.

So how many make that connection? How many made that linkage and can you verify that? Can you track that? Can you improve that over time?

SAMHSA has helped us in, I think in many respects having integrated measures is a nice thing to hear and a goal that we heard is targeted.

So the Center for Mental Health Services, CMHS, has the NOMs. The CCAT has the GPRA.

So those are the two SAMHSA measures that many of you who are SAMHSA funded typically complete. The NOMs’ measures go from admission through 54 month self-report. It’s a lot of different, as you all know, variability in how these data are collected.

These are the kinds of things on your screen that are assessed and there’s quite a variation, as I said earlier, in the format of how these data are gathered.

The GPRA is typically done at intake. Again, it assesses a wide range of things and you might think that social connectiveness is a common denominator across both measures, but it’s actually different.

A nice factor, nice dimension, but different as it’s assessed from NOMs to GPRA.

Again, huge variation in whether these are self-report, interviewer, or administratively completed and, I think as people feel more comfortable with the data that they’re providing, they’ll feel better about providing the information.

What about required data collection? Well, I’m a clinician and often times I look at the data that I’m required to complete as a “necessary evil.” It keeps the doors open. It keeps my supervisor off my back. It keeps the, you know, the EMR

You know, it kind of, you know, it keeps that record from appearing again in my inbox.

So, but one of the issues there is that we often ask clinicians to complete data and complete forms and complete scales that are never really fed back. So, it’s one of those you know, people pointing fingers at one another.

So, if I’m collecting the data, if I’m asking you to collect the data, the burden is on me to use that data and present the information back to you as a clinician, so that’s useful to you and useful to your work.

If I’m a clinician, my job is to provide you with information that’s reliable and valid, because you’re going to be feeding it back to me in a way that could be useful or really make or break a clinical interaction.

Right now, unfortunately, I think there’s a bit of a stalemate between the data that are provided and the data that are gathered in many organizations. So, it’s really important that if you are going to be moving in the direction of dashboard indicators, that the way you do that does involve clinicians, and the way you do that does involve using the data that are collected, making it real.

There is a lot of variation and training expertise and interpretation of the questions for required data collection, so it might seem simple, it might seem straightforward what the questions are, but it’s really important not to side step sitting your people down in a room and being clear about how to interpret the questions.

And I put here, it has tremendous upside potential, because these are data that we’re already collecting and ,again, the burden is on us to make use of these data.

Most agencies, maybe I can’t say that, many agencies are in the direction of EMRs, at least the agencies that I come in contact with, and they present an excellent opportunity for aggregate standardized data collection.

Oftentimes, this information or this opportunity is missed. Again, I think it has to do with the fact that the data that are gathered are never really fed back and used in a way that makes them valuable to a clinician.

So, the clinician acts in a way that’s almost independent of the data that are gathered. And we want to change that.

And, again, I think if you use the concept of dashboard indicators to feed back the information as quickly as possible, the clinician can actually use that to guide the conversation with a consumer about treatment options, about treatment progress or the need to change a treatment course, because it’s working or it’s not working.

And having an EMR really sets the stage, I think, to be able to do that.

What about health insurance, health care reform? A great opportunity as we’ve heard to unify the kind of data that are collected, you know, to standardize the behavioral health vital signs.

It’s a fantastic opportunity. I hope I live long enough to see the idea that we duplicate services. We offer services that are competing, sending mixed messages to consumers and families, having common information systems much like we are more able to do with the pharmacy and medication dispensing. It really offers a tremendous promise, you know, downstream for us.

Being able to share data, obviously we have a great deal of concerns about confidentiality. Our issues seem to be dealt with greater sensitivity and concerns about privacy than other health care issues.

So, being able to, again, have those considerations and protect the welfare and the privacy of our patients, while at the same time, being able to communicate across service settings seems to be an important balance, but an important outcome for us to achieve.

Just in terms of discussion, I believe that there are a few interesting issues out here that I don’t have a resolution for, but I think it’s worth acknowledging.

The first is, that we don’t have good standardized measures. I think if you work in addiction treatment settings, you probably have more to choose from than if you work in a mental health setting, unfortunately.

If I visit ten mental health programs, there’s ten different measures used to assess functioning and symptom severity at baseline and followup.

If I visit ten addiction treatment programs, anywhere from three to six are likely to be using something like an Addiction Severity Index.

So at least, in addiction programs, there’s some standardization. Of course, there’s a huge variability in how those measures are used and how, again, the data are attended to and fed back. Well, all bets are off on that one.

Another question is, really, are any of these measures getting at, are you better? You know, it might not be as simple a question as this, but you know, are we really measuring what we’re supposed to be measuring and what about the true validity of —

Are things changing and are we measuring them the way, you know, the way we should? And, as a clinician, I’m often not sure about this. I think it’s more of a conversation than a yardstick, but we’re — the burden is on us to figure out, I believe, how to measure this in a way that makes sense across a variety of

When to assess and for how long? So FDA-approved medications, 12 weeks. It doesn’t sound very long. It’s not very long. What happens when people stop taking the medications?

Well, they typically — it’s not good. If we studied some of our folks for 12 weeks and submitted the article to study it to a journal, they would say, “Twelve weeks is not long enough. You need at least a 6-month followup.”

Well 6 months is probably not long enough either. What about 1 year? Is 1 year long enough? Well, probably not. Is 5 years long enough? Well, it might be, but that still might not be long enough.

You see, many of the problems of people that we treat ebb and flow and you get better and get worse and, you know, there’s a chronic cyclic and sort of slow and steady progressive course, sometimes for better and hopefully, rarely, for worse.

But, you know, being able to measure people over time during treatment, seems to me to be the best case scenario.

What about what happens to us along the way? This is — if you do research, always the head scratcher, because you will measure people at followup and it turns

So you know, they experienced the death of a spouse or a parent or a child, and, of course, their depression score will go down. Or, if they were a victim of domestic violence, running into the perpetrator, the spouse who was a perpetrator, just before the followup assessment or losing your job or, you know, being diagnosed with a serious medical problem and being assessed at followup.

So post-treatment stressors have been found to be very much associated with post-treatment functioning. In many ways, studies have found that post-treatment stressors are probably more important than any ingredient for the treatments that are offered.

So, what happens to folks along the way? It turns out to be critically important and often not the kind of things we ask about. And this probably is redundant from “Are we assessing the most important things?”

You know, assessing substance use and mental health symptoms does not really get at the kind of recovery positive psychology, quality of life issues that I think are critical for us to really be taking a look at.

Are people’s lives better than they were before? Has treatment been a good thing for them? How do we know? Is it their perspective? Is it their family’s perspective? Is it society’s perspective? Is it the police department’s perspective, etc.

So, I think these are still questions that we continue to wrestle with, but I think unfortunately we’re going to need to make some decisions about answering them, probably long before we resolve them.

Lastly, you know, people would say to me, we’d like to measure outcomes and it’s kind of like, you know, approaching an architect and saying, “Well, I’d like to build a house.”

Well, you know, “What kind of house do you want?” And, by the way, “The house you want is probably going to be more expensive than you think.”

So it’s really not to minimize the cost of doing good a outcome study, measuring things, having people whose jobs are dedicated to making sure things get measured and making sure that things get aggregated and added and averaged and tracked.

It’s a resource issue. And it’s not free. And the better you want to do this, the more it’s going to cost you to do this.

And I think oftentimes we overlook this and want something here that’s unrealistic, but it’s got to be factored into any plan to construct dashboard indicators, you know, for you or your agency.

I think this is how you would reach me. These are a couple of Web sites where some of the things, including the DDCAT, DDCMHT, and the DDCHCS have been described. They’re actually, I believe, some of the measures that I’ve referred to are also on some of these Web sites.

And, again, if you have any trouble tracking things down that I’ve referred to and you’ve given a search engine a run and still haven’t come up with them, please don’t hesitate to email me. I’m happy to respond to any of the questions that you have.

So at this point, I think I’m going to invite (Beth Fraster) who’s been gathering questions that may have come in and folks are still with us. I — this is where

(Beth Fraster):   Well —we’re still with you Mark.

Dr. Mark McGovern: Okay, good.

(Beth Fraster):   And I just wanted to remind people that people have been writing in questions and I have been gathering them.

And if you would like to continue to ask questions, I just want to remind you that at the top of the screen where it says “Q&A” you can click that and ask — and write in questions and we can continue to take the questions in the

And we can accept questions for Mark and, unfortunately, Rear Admiral Peter Delany is no longer with us because he had to leave for another meeting.

But, I’d like to get to some of the questions that did come in. One of the questions that came in, and I think you — right toward the end you did begin to address this — but a question came in about reconciling the need for just-in-time data with the long-term nature of mental health and addictions outcomes.

Can you address that issue of trying to come to terms with the just-in-time data and the need for real-time data, and yet the need for long-term outcomes with the people that we are working with and working on behalf of?

Dr. Mark McGovern: Yeah. No, I think that’s a great question and the answer to that is both. I think the real-time data help and can serve to guide a clinician and guide a consumer in terms of their immediate treatment needed and some assessment of their treatment response.

And then, at a program level, knowing that your rates of patient improvement are similar or at least competitive with other expected rates of improvement.

Now, an example of the former might be that, if you’re measuring, oh, anxiety symptoms and, you know, you’re offering consumers either a medication or a — let’s say —

And that person starts in on SSRI, those of you that are in the addiction field, were probably hoping I wouldn’t say Zanex or Zquantapin, but let’s say an SSRI for an anxiety problem and, at the same time, initiate a cognitive behavioral therapy to work on perhaps a social anxiety.

So, at the outset of treatment, a person completes a social interaction anxiety scale, gets a score of 25 and, after 2 weeks or the 2 weeks of cognitive behavioral therapy and initiation of medication, social interaction anxiety scale is re-administered and a person has made no change.

So you might say, in sitting down with the person, you know, you filled this out when you first got here. You filled it out this morning. I’m not seeing any change.

How are you doing with that? Are you feeling that you wouldn’t expect a change this quickly? Are you feeling that you’re a little impatient? Let’s talk about the fact that there’s been no change.

Or, you might say at the end of 30 days — and this could be more realistic — you would re-administer the social interaction anxiety scale and notice that there’s a very positive change and you can not only affirm that the treatment is working but you can refer to the information to share with the patient. And say, “Look, this is where you are now. This is where you were when you first

“This is something. This is good, right?”

So I think being able to use the information clinically, you know, to engage a collaborative venture or adventure together, being able to adjust treatment based on measurable treatment response or lack thereof, I think, is a fantastic opportunity that some of these measures give us.

In contrast, being able to look at, you know, the first 100 patients treated with a combined medication and cognitive behavioral therapy for social anxiety gives us an idea about, overall, how our services are doing or what happens to people that decide, they only want the medication, or what happens to people that decide that they only want cognitive behavioral therapy.

And I’m being simplistic here because I think, as a program, you might want to know overall how the program’s working. And I suppose you might want to say, “Well how’s therapist A doing versus therapist B?” You know, that gets into some different issues but, as a program you probably want to be able to collect these data, you know, as a baseline to see how you’re doing and then see how those baselines change over time.

I hope that answers that question. It’s a — it’s not an either or. It’s both in

(Beth Fraster):   Thank you. Thank you. We have a couple of questions that I’m putting together here and these pertain to a followup.

So we struggle with obtaining followup information about people who leave treatment, either with our or against our advice. And are there any proven ways to obtain good followup information? And there’s going to be a Part Two to this question, but I’ll let you answer Part One first. So this is about ways to obtain good followup information.

Dr. Mark McGovern: Okay. Yeah. No. Hopefully the Part Two’s not a setup. I’m going to walk into a Baptist — just kidding. No, I think that that’s a great question. You know, I think it’s typical if you’re in a residential program business or even if you’re in an outpatient program business. You obtain the address of the people who you’ve served and you’d like to send them, you know, a questionnaire, at you know, at a certain interval, whether it’s 1 month or 6 months or 1 year, and you want to find out how they’re doing.

Are they drinking? Are they using drugs? Are they depressed? Are they working?

But what happens is, that you might hear from 10 percent or, you know, in the best case scenario you hear from 25 percent. And, of course, you know, let’s say the 25 percent you hear from, they’re doing great and you start to think, well, does that mean the other 90 percent or 75 percent I haven’t heard from aren’t doing great, and do I need to count them as treatment failures or nonresponders or could I just pretend as if, well, you know, the 25 percent I’ve heard from are representative of everyone we’d treated? Well, probably not. So rates of followup are huge. If you’re involved in trying to get research grants, you always have to speak to what kind of followup you expect and what kind of followup rates have you demonstrated in your previous work? And, I think that there are some proven techniques, although I have to say that there’s a lot of room for improvement in these proven techniques.

One technique is being able to obtain multiple contact information. So, this might range from an address to a cell phone number, to an email address, to obtaining contact information about people who would know where the client might be and how to reach them. So this gets into who does the participant want you to be able to communicate with to find them later in the event that they’re not findable?

We’ve recently begun to experiment with the possibility of using Facebook, you know, being able to find people on Facebook and being able to reach out to them and have them connect with us on Facebook.

So, other folks have done things like give people cell phones, cheap cell phones with limited call time and add to the call time, based on participation and followup assessments.

One thing that does seem not be overlooked is the importance of the relationship. So, having a person make a contact who’s known to the individual, having them not wait to long too make those contacts, preparing them for when those contacts are likely to take place. So, for instance, you know you’re leaving here today, we’re going to call you in a month, and I’m going to be the one that calls you and how do you feel about me calling you? Can we set up an exact time that I can call you? Can I call you a week before? Can I send you a postcard a week before, so that, you know, we know we’re on the same page relative to the call.

So I think, you know, the kind of things I’m describing, aren’t, you know, aren’t really rocket science. They’re not magical. You know, there’s a certain amount of just common sense associated with it, but this gets back to the cost, the cost of collecting followup data; you know, it’s high.

And one of the things that people will also do is pay people or incentivize people for followup participation. So that could range from, you know, actual, you know, gift certificates, gift cards, cell phone minutes, to you know, to cash. So I think the more you’re able to incentivize, the literature will show the better the followup rates.

(Beth Fraster):   Thank you. You actually answer the Part Two, which people were really writing in asking for some of those techniques and ways of retaining and reaching out to

Another question just going in a different direction. Do you see any differences with measuring outcomes in rural versus very rural areas?

Dr. Mark McGovern: Well, I think we — you know, we have both. I guess it’s arguable if you — I live and work in Northern New England and a lot of folks consider that rural and I think, yeah, some of the outcome issues that might vary have to do with social connectedness. But, you know, I think that my answer, unless someone has a specific question, is probably no, that you know, the things should be the same from rural to extreme

(Beth Fraster):   Okay. Okay. And another question came in, thinking about the clinical administrator and how frequently should a clinical administrator look at outcome data? Should they do this annually, quarterly, monthly? How much time should they be putting into this activity? And I would imagine this has to do also with how much —how much should they be budgeting for this activity?

Dr. Mark McGovern: Yeah. No. I think, how often do you want to look at your dashboards, you know, a lot. So, you know, how often you want to look at it probably will dictate, you know, kind of what you need in place to be able to gather and aggregate those data. So, you know, I think what many people I know, agencies that I know who are doing this, will try to do is a quarterly, you know, quarterly sort of view of program-level data. That way it gives some data collection or IT people time to pull some things together. It’s multiple months, so it might not be as subject to, you know, what some agencies find the seasonal variation, you know, whether it’s summer or fall or beginning of the year or, you know, other kind of changes associated with that sort of stuff.

So, I think quarterly is something to aim for. Annually strikes me as too long in between. Monthly is great. It might be hard effort-wise to come up with data that can be enough to help you to draw any conclusions.

(Beth Fraster):   And just as a followup question to that, I would imagine that an organization or an agency would then begin to find some patterns over years and finding patterns, if they were doing it quarterly, that might be very influential in ’re doing their programming and information that they find out.

Dr. Mark McGovern: Yeah, you know, I think that’s just it, because sometimes, you know, you’ll hear a patient ask a question, like “What are my chances of getting better?” which is a really good question.

And I think it’s a question that as clinicians, you know, we ought to be able to answer based on information that we know. And you know, we might answer that question just based on, you know, the well spring of clinical experience we have. We might answer the question reflectively, something like, “Well, you know, it depends. It depends on, you know, things that come up for us. It depends how, you know, how much you work at it. You know, it depends on the different things that might happen in your life,” so, you know, that’s a good answer.

But it sort of ducks the question. So, wouldn’t it be great if we actually knew, you know, that of a hundred people who we treat, you know, that 30 percent at 6 months are substance free and, you know, 40 percent have reduced symptoms to the point of, you know, being, in a normal range. Now, I make these numbers up, because I think our field doesn’t really have good information on those numbers. You know, as a researcher, we’re frequently asked to estimate effect size.

You know, that really has to do with change, aggregate change, you know, with treatment, without treatment, with a certain treatment, compared to another treatment. And you know, it turns out there’s not a lot of good information, you know, to start with. You really have to accumulate that information over time to estimate what your expected effect size is.

So likewise I think, if you’re an agency, you probably will need to collect that information to start — and develop a baseline and maybe do different things over time to see if you can improve that and be able to refer to that baseline to show that improvement. So maybe that would be, you know, employing more evidence-based treatments.

But what if you employ evidence-based treatments and things — and your patients get worse, your outcomes aren’t as good? Well, sorry to say, that evidence-based treatment developers, but maybe that’s not the evidenced-based treatment you ought to be thinking about. And this would be one way of sort of demonstrating to you and to your staff that ’s not working and you might want to choose from a different list of evidence-based treatment.

So, we don’t think that would happen, but how would you know, unless you collect those data?

(Beth Fraster):   Right. Thank you. Thank you. Another question that just came in before we have to end here. “How might co-occurring disorder treatment providers get a better understanding of side effects, including addiction, dependence issues for ”

Dr. Mark McGovern: Okay. Great. Great question. Yeah, I think that some very knowledgeable people, and especially if you study the effects of certain medications over time, you do start to wonder, you know, are the benefits, the short-term benefits worth the long-term consequences?

So you know, we know the — particularly the psychotropic medications, the anti-psychotics have long-term deleterious effects on mortality and you know, other types of toxicity. Obviously, the idea there is that we, you know, we want to try to give people medicines that help them to function as best as possible for as long as possible, but I think, you know, more information about the, you know, liver and kidney toxicity is, you know, it’s important that people are made aware of those issues and, you know, it’s part of a shared decisionmaking process.

So I think the answer to that question is, keep track of it. You know, keep track of the side effects. Keep track of the medical problems that, you know, might be associated with medication. You know, be able to, you know, use that information to help your patients decide on what kind of approaches they’re, you know, they’re interested in.

So, you know, I think in addiction, we — you know, folks think about, what about addiction medications and, you know, what are the benefits of being on methadone or Pupornorvin and will that help with other drugs or will I need to get additional professional help to, you know, help me learn to live without cocaine or without marijuana or without alcohol. Then, what about methadone or what about Suboxo and Purpurnovin, are there things that I should be looking for that are consequences of that, that you know, are side effects of that.

And the answer is, well yes, and we want to keep track of those. So that might be, you know, part of — if you’re looking at things at a program level, an adverse event or a serious adverse event, but I think the question ultimately is keep track of it, keep — gather the data.

(Beth Fraster):   Thank you. And I think this is a great question to end on, because this will lead us all in the right direction, but what advice would you give an organization that is beginning performance and outcome issues? Where would you tell the organization to begin?

Dr. Mark McGovern: Well, I think, you know, you need to sit down with some key people. I would say the key people to sit down with, if you have these members of your team, would be some key clinicians who actually complete the forms, some maybe key front office people who answer the phone and complete forms and collect the forms that the clinicians complete.

Probably you would want program level directors or supervisors who monitor all of that. If you have a person who’s quality assurance or IT who’s knowledgeable about the systems that are in place and knows about how things are collected and how things are reported and the potential that might exist in software that the agency has or could liaison with some software people to know about the reports that can be generated.

An authority, so a clinical administrator, you know, someone in a leadership position who can talk about the importance of this, to lend vision but also, you know, talk about the, you know, really the short term objectives, to make, you know, to make all of this real, to make it important, to make it integrated in what we do on a daily basis. I think you need that leadership vision and buy in and involvement.

So I think members of that team. And then I would say start off with, what do we already collect? You know, what can we make sense of that we already collect and what would be the easiest stuff for us to collect, so that we could get information as quickly as possible and let’s start off with something easy like, you know, how many people continued to the second week.

Maybe we could start off with that. Let’s start figuring that out and taking a look at that at the end of the month.

So you know, keep it as is said many times, many ways, keep it simple.

(Beth Fraster):   That is good advice. And I’m sorry that we did not — we are unable to get to everybody’s questions and they were all good questions, but I know that Dr. McGovern is available other ways and he did — I will actually, why don’t I put his information back up for people to see again. As he mentioned, he is available and he will accept emails. For more information you can reach Mark McGovern. I want to thank Mark McGovern and Admiral — Rear Admiral Peter Delany for their time in hosting and being our guests today and sharing their knowledge with us.

I would like to also thank our SAMHSA Government Project Officers for their support in creating the series, (Charlene Le Fauve), (Tison Thomas), and (Deborah Stone) for their help and support and vision. And you will be receiving in the mail and email with a link to an evaluation form. We appreciate everyone’s feedback and will use it in developing future Webinars.

This Webinar can be reviewed in 48 hours on the Web site that will be sent to you with this evaluation form. And it will also be uploaded on the COCE Web site in the end of April.

We thank you for joining us and this concludes our call for today.

Thank you.