The Impact of Service Characteristics on Functional Outcomes From Community Support Programs for Persons With Schizophrenia
A Growth Curve Analysis
John S. Brekke
School of Social Work University of Southern California
Jeffrey D. Long
School of Social Work University of Southern California
Portals Mental Health Rehabilitation Agency
Department of Preventive Medicine University of Southern California
Reprint: Journal of Consulting and Clinical Psychology, June 1997 Vol. 65, No. 3, 464-475
A total of 172 individuals diagnosed with a schizophrenia spectrum disorder were followed for 36 months in 3 distinct models of community-based care. Functional outcome data gathered every 6 months were combined with service implementation data to test hypotheses concerning the impact of service characteristics on prospective client outcomes. The results using hierarchical linear modeling supported associations between the intensity, specificity, and longitudinality of services and improved client outcomes. Specifically, more intense services were associated with higher levels or rates of improvement on all indices of clinical and psychosocial functioning. The specificity results suggested that services needed to be targeted to specific areas of functioning in order for improvement to occur. The effect of longitudinality was contingent on the outcome domain examined.
Jeffrey D. Long is now at the Department of Psychology, St. John's University.
This research was supported by Grant MH 43640 from the National Institute of Mental Health.
Correspondence may be addressed to John S. Brekke, University Park MC-0411, University of Southern California, Los Angeles, California, 90089-0411.
Community Support Programs (CSPs) are community-based psychosocial rehabilitation programs for individuals with chronic mental illness (National Institute of Mental Health [NIMH], 1982 ; Turner & TenHoor, 1978 ). CSPs were designed to break the cycle of revolving door hospitalizations and to increase the psychosocial functioning and community integration of these vulnerable individuals. The programs have varied in the types, form, and organization of the services they offer. They have ranged from family-based psychoeducation or comprehensive psychosocial rehabilitation centers, to assertive community treatment (ACT) teams (Stroul, 1988 ). Controlled studies on community-based care have quite consistently found that CSPs reduce hospitalization rates when compared with usual community care (Bond, McGrew, & Fekete, 1995 ; Hargreaves & Shumway, 1989 ; Olfson, 1990 ; Scott & Dixon, 1995 ; Solomon, 1992 ; Test, 1984 , 1992 ). The findings on other psychosocial outcomes have not been as consistent with studies finding CSPs to be either superior or no better than usual care in terms of independent living, social or occupational functioning. In this regard, Olfson (1990) has stated that the challenge for future research on CSPs is to understand the conditions under which these programs achieve superior functional outcomes. A critical aspect of this effort concerns delineating the service delivery, or program process, variables that are related to client outcomes (Brekke, 1988 ; Brekke & Test, 1992 ; Solomon, 1992 ; Taube, Morlock, Burns, & Santos, 1990 ). However, in searching for the conditions under which functional outcomes can be maximized in CSPs, several issues are notable.
First, because most studies have compared CSPs with usual community care, there have been very few comparisons of different CSPs (Scott & Dixon, 1995 ; Solomon, 1992 ; Test, 1984 ). Second, it has been argued that successful community-based interventions should be organized according to several service principles, which will then be reflected in certain service characteristics (Bachrach, 1992 ; Test & Scott, 1990 ). Unfortunately, almost no studies have measured actual service implementation (Brekke & Test, 1992 ; McGrew, Bond, Dietzen, & Salyers, 1994 ). As a result, it has not been possible to empirically assess the characteristics of their service delivery. In general, the absence of both comparative outcome designs and program implementation data in research on CSPs has precluded tests of how particular service characteristics affect client outcomes. Nonetheless, there are several service characteristics that have been discussed as potentially important to effective CSPs. Three of them are intensity, specificity, and longitudinality (Bachrach, 1992 ; Brekke & Test, 1992 ; Test, 1981 ; Test & Scott, 1990 ; Torrey & Drake, 1994 ). Because these are emerging concepts in community-based care, there is little consensus as to their conceptual or operational boundaries (Tessler, Willis, & Gubman, 1986 ; Torrey & Drake, 1994 ). In this study they were defined as follows.
The notion of intensity suggests that more services are better than fewer services. This can be seen in both the actual quantity of services, as well as the array of service domains offered. Concerning the quantity of services, three studies using samples of individuals with persistent and severe mental illness are relevant. Snowden and Clancey (1990) found that clients who received more units of service at a community clinic showed higher levels of global functioning on the Global Assessment Scale (GAS). Sands and Canaan (1994) compared two ACT programs and found that the program with higher rates of client contact over 12 months had higher global outcome on the GAS, but that there were no differences in hospitalization outcomes. Dietzen and Bond (1993) found that a minimum frequency of service contact in ACT was related to lower rates of hospitalization. They concluded, however, that future research needs to focus on specific dimensions of outcome (rather than relying on global outcome assessments) and also target more service variables.
The principle of longitudinality suggests that, because these disorders are going to present clients with long-term challenges, the services must be available in an ongoing fashion (Bachrach, 1992 ; Test & Scott, 1990 ). Longitudinality has also been taken to imply that the termination of services will result in the loss of rehabilitative gains for the individual. In this regard, some studies found that once services ended, the functional gains made by chronically mentally ill individuals were lost (Scott & Dixon, 1995 ; Test & Stein, 1978 ).
Finally, specificity refers to the need for services to target the specific areas where functional change is desired. For example, if change in work functioning is desired, vocational services need to be offered. If gains in social functioning are targeted, then interventions tailored to that functional domain are necessary. This is based on the notion that the generalization of change across functional domains is not likely to occur during rehabilitation with this population (Bachrach, 1992 ; Test, 1981 ). However, we are not aware of any empirical study concerning the impact of specificity of services on client outcomes in CSPs.
As can be seen, there has been very little study of the association between service characteristics and functional outcomes in CSPs. This study sought to compare prospective client outcomes over 36 months from three CSPs that empirically varied on the intensity, specificity, and longitudinality of their services.
This study used a quasi-experimental follow-along design of patients admitted to three community-based programs in urban Los Angeles, California. The three programs are described in detail below. Briefly, two of the programs are psychosocial rehabilitation programs that target substantial change in certain functional domains. The third is a case management program connected with residential board and care homes that has no substantial rehabilitative focus. The sample consisted of consecutive admissions to the two rehabilitation programs, as well as case management patients selected from board and care homes that had reputations for providing high-quality care according to Los Angeles County Department of Mental Health criteria. Nearly all of the patients had received treatment in the publicly funded mental health system and were receiving public assistance.
There were four study admission criteria: (a) a diagnosis of schizophrenia or schizoaffective disorder; (b) residence in Los Angeles for at least 3 months before study admission; (c) 18—60 years of age; (d) neither a diagnosis of mental retardation or organic brain syndrome, nor a primary diagnosis of substance dependence. Measures of psychosocial functioning were administered in face-to-face interviews at baseline and every 6 months over a 3-year period. Diagnoses were established in a two-step process. First, an initial diagnostic screening for schizophrenia (using chart and interview data) was conducted by an admitting clinician at the program sites. Second, all patients who passed the first screen were subsequently diagnosed in a face-to-face interview by a licensed, doctorate-level clinician trained in the use of the Schedule for Affective Disorders and Schizophrenia (SADS; Endicott & Spitzer, 1978 ). Structured interview data and clinical records were used in determining the SADS diagnosis.
There was a 6-month replacement period, so that any patient who dropped out of the study in their initial 6 months was replaced in the sample by the next available patient at that site. The replaced patients were excluded from analyses presented in this study. There was an 18-month recruitment period to obtain the study sample. Patients were followed in the study protocol for 3 years whether they exited treatment or not.
It has been argued that service outcomes in schizophrenia need to be assessed along multiple dimensions (Rosenblatt & Attkisson, 1993 ). These typically include the functional domains of social, work, independent living, and hospitalization. To achieve this, we used two outcome measures in this study: the Role Functioning Scale (RFS; Goodman, Sewell, Cooley, & Leavitt, 1993 ; McPheeters, 1984 ), and the Strauss and Carpenter Outcome Scale (SCOS; Strauss & Carpenter, 1972 ). The SCOS has been widely used in schizophrenia outcomes research, and the RFS was selected as a scale of choice for this population by Green and Gracely (1987) . These scales allow for the assessment of outcome along four distinct dimensions. The four psychosocial outcome variables were the hospitalization and work items from the SCOS, and the independent living and social functioning items from the RFS. The scores on each item have anchors made up of clinical descriptors which will be used as a basis for discussing the clinical significance of findings in the four outcome domains.
The scale ratings were derived from a face-to-face interview instrument, the Community Adjustment Form (Test et al., 1991 ). The ratings were derived according to procedures outlined in Brekke (1992) . Interrater reliability using the intraclass correlation (ICC) was established during intensive rater training, and during booster rating assessments throughout the study period. The ICC on the four outcome items ranged from .75 to .98, with an average of .89.
Several other measures were used to assess the equivalence of the three groups at baseline (see Table 1 ). The current psychometric performance of these measures is detailed in Brekke, Levin, Wolkon, Sobol, and Slade (1993) . In brief, they showed high levels of interrater and interitem reliability.
Recent advances in longitudinal research suggest that modeling should begin on the individual patient level. This allows for the assessment of both intraindividual and interindividual differences in change (Rogosa, Brand, & Zimowski, 1982 ; Rogosa & Willett, 1985 ; Willett, 1988 ). The recommended approach is to model individual change across time and then examine the effects of covariates (e.g., gender or program type) to determine whether there are systematic differences in rate (i.e., slope) or type of change (i.e., linear, quadratic, cubic, etc.). This twofold aim can be accomplished using growth curves and hierarchical linear modeling (HLM; Bryk & Raudenbush, 1987 , 1992 ).
HLM involves modeling at two levels. At Level 1, a least-squares regression equation is fit to each individual's data across all time points (this equation is the growth curve). Each individual's scores on the criterion are regressed on time or a transformation of time. At Level 2, the Level 1 parameter estimates of the linear slope, the quadratic slope, and so forth, are treated as criterion scores and each is regressed on the covariate (in this study, program type). Final estimates of the growth curve parameters for each individual are derived through empirical Bayes estimation (Strenio, Weisberg, & Bryk, 1983 ). Empirical Bayes estimation provides a composite procedure that uses both the information from each patients' data and the information from the covariate in determining final parameter estimates; that is, each individual's growth curve parameters are estimated with a weighted combination of the Level 1 and Level 2 estimates (see Bryk & Raudenbush, 1987 ; Raudenbush & Bryk, 1985 ). After individual curve parameters are estimated, hypothesis testing can be used to assess the fit of the group linear and quadratic curves in the population, as well as the significance of the covariate interaction (see Bryk & Raudenbush, 1992 , for a detailed discussion).
In this study, there was interest in only one covariate, program type. The central question was whether this covariate was responsible for systematic differences in growth curve rate (i.e., the magnitude of the slopes), and type (i.e., whether the curve was linear or quadratic), or a combination of the two. Linear and quadratic curves were chosen for modeling for several reasons. First, these two types of curves are the ones most commonly fit in psychological research (Cliff, 1987 ). Second, we wanted to be as descriptive as possible by including straight-line (linear) change, and also nonlinear (quadratic) change for each individual. Third, it was also assumed that linear and quadratic curves would be interpretable in the context of clinical change; that is, a linear trend indicates that an individual's scores continually go up (or down) as a function of time. A quadratic trend indicates that an individual's scores go up and then down (a negative curve) or down and then up (a positive curve) as a function of time.
The linear and quadratic growth curves at Level 1 were regressed on orthogonal polynomial transformations of time (see Kirk, 1995 , p. 191). This was to ensure that the Level 1 "predictors" (i.e., time and time squared) were uncorrelated. The orthogonal polynomials were also centered at the first time point, so that the intercept reflected predicted status at Time 1. After Level 2 equations were computed, tests of significance for slope rate, type of curve, and program type were evaluated for each of the dependent variables related to the hypotheses below. All analyses were conducted with the HLM/2L software (Bryk, Raudenbush, & Congdon, 1994 ).
There were three programs in this study: Portals, the Community Living Program (CLP), and case management (CM). They were selected because of the degree to which they varied on certain service delivery characteristics. The programs are briefly described here and then compared on relevant service characteristics.
Portals is a psychosocial rehabilitation clubhouse. It targets rehabilitative change in social, vocational, and independent living functioning. It has service continuums in vocational and independent living areas. The social component of the program consists of an on-site daily socialization program, and evening activities at the clubhouse. Portals provides on-site psychiatric monitoring of medication and crisis management to prevent hospitalization. Services can last up to 2 years in each of the vocational and independent living continuums.
CLP is located in an apartment complex. Clients are admitted in cohorts of 8—10 for 3 months of intensive independent living training and group socialization. Ongoing supportive and rehabilitative services are available after the initial 3-month training. CLP is not designed to provide vocational services, but crisis contacts are provided to prevent hospitalization.
The CM program is run by the Los Angeles County Department of Mental Health. It is connected with residential board and care homes. As part of CM, the County provides fiscal incentives to the board and care homes for taking seriously disabled clients. The clients are required to be seen by their case manager once every 3 months and cannot be dropped from the caseload. The primary goals are to keep clients in the community, and to provide them with optimal levels of care and maintenance. Although the program has no specific rehabilitative focus, there is an intention to enrich the milieu for clients in the residential facilities.
Data from Brekke and Test (1992) , as well as county management information system data (Jordan, 1985 ), were used to compare the programs on the intensity and specificity of their services during this study period. While the Brekke and Test (1992) study compared Portals and CLP during the first 12 months of treatment using a subsample of the present study's patients, subsequent unpublished analyses have confirmed that the program differences were very similar for the entire sample. Concerning service intensity during the first 12 months of the program, CLP averaged over 35 hr/month of staff—client contact during its 3-month training period, which dropped to about 7 hr/month after that. Portals averaged about 12 hr/month of staff—client contact during the first 3 months and then dropped to about 5 hr/month after that. CM averaged approximately 1 hr of manager—client contact every two months. Clearly, Portals and CLP are more intense programs than CM.
Concerning specificity, Brekke and Test (1992) found that 27% of the staff—client contacts at Portals were in the vocational area, whereas less than 1% of the contacts at CLP were vocational, which was a statistically significant difference. In terms of contacts associated with daily and independent living skills, CLP had 39% of its contact, and Portals had 17% of its contact in this area, which was a statistically significant difference. These data suggest that Portals has a substantial focus in the vocational area, whereas CLP has almost none. Concerning independent living, although Portals has a notable emphasis in this area, CLP has a significantly greater independent living focus than Portals. Because CM was not a rehabilitative environment, it was not included in the analyses on specificity.
In terms of longitudinality, clients can exit both CLP and Portals at any time after admission. Because clients remained in the study protocol whether they exited the programs or not, we used county-based billing information to track the length of stay at CLP and Portals. From this, we indicated whether a client was in or out of Portals or CLP at any interview period, and this provided a basis for the analyses on longitudinality.
On the basis of the program implementation data and the definitions of the service characteristics provided above, the following hypotheses were tested.
The intensity hypothesis was that Portals and CLP would have higher rates of improvement than CM on hospitalization, independent living, work, and social functioning over time. The outcome curves for Portals and CLP were expected to be positive linear or quadratic, or a combination of the two; CM was expected to show minimal change in any outcome area.
There were two specificity hypotheses: (a) Portals would show greater positive linear or quadratic curves in terms of work functioning over time than CLP, and furthermore, CLP would show little vocational change over time; (b) in terms of independent living over time, both programs would show a positive linear or quadratic curve, but CLP would have a steeper slope than Portals.
The longitudinality hypothesis was that for Portals and CLP, in-treatment times would show positive linear or negative quadratic curves in all outcome domains; out-treatment times would show negative linear or positive quadratic curves in these domains.
The sample consisted of 172 individuals diagnosed with schizophrenia or schizoaffective disorder. The sample included 127 men (73.8%) and 45 women (26.2%); the average age was 33 years (SD = 7.3), and the average length of illness was 11.2 years (SD = 7). The ethnic composition of the sample was 87 Caucasians (50.6%), 50 African Americans (29.1%), 27 Latinos (15.7%), and 15 Asians or Native Americans (4.6%). The average BPRS score was 43.8 (SD = 13.4). All of the patients were living in community-based settings at study entry. Seventy patients were in Portals, 34 were in CLP, and 68 were in CM.
Equivalence of the Groups at Baseline
Because this was a quasi-experimental comparative outcome design, assessing the equivalence of the groups at baseline on demographic and functional variables was essential. Table 1 presents baseline data on 16 important variables. In general, the groups were equivalent at baseline on the variables examined. The significant differences indicated that the Portals group was somewhat younger than CLP, and Portals also showed greater social deficits and longer institutional time in the 6 months before baseline than the other two groups.
Two kinds of attrition were examined: study and treatment attrition. Study attrition concerned patients who dropped out of the study. At 12 and 18 months, 88% of the sample was retained. This dropped to 83% at 24 months, 80% at 30 months, and 72% at 36 months. There were no statistically significant differences in study attrition rates across the programs. We also compared study completers at 36 months (n = 123) to study dropouts ( n = 49) on gender, race, prognosis, age, length of illness, baseline symptoms, baseline role functioning, baseline substance use, and baseline self-esteem and satisfaction with life. None of the differences was statistically significant.
Treatment attrition concerned the patients who exited Portals or CLP but who remained in the study protocol. We did not assess whether these were planned or unplanned treatment exits. The exit rate at Portals was 19% at 6 months, 45% at 12 months, 76% at 18 months, 81% at 24 months, 90% at 30 months, and 91% at 36 months. At CLP, the exit rate was 21% at 6 months, 56% at 12 months, 65% at 18 months, 71% at 24 months, 74% at 30 months, and 79% at 36 months. There were no statistically significant differences in exit rates between the two programs. Given this treatment attrition, we examined the relationships between months in treatment and each of the client variables in Table 1 . The two statistically significant relationships indicated that being older and having poorer work functioning at baseline were associated with more months in treatment.
Functional Outcomes Across the Three Programs
The study hypotheses are addressed below. In each case, we tested the intercept (initial status), linear, and quadratic coefficients for statistical significance, and for group differences in change curves. When directional hypotheses were tested, one-tailed tests of statistical significance were reported. Effect size is also very relevant for assessing the practical implications of rejecting the null hypothesis (Friedman, 1968 ). A useful measure of effect size in psychological research is the product—moment correlation, r ( Rosenthal, 1993 ). Product—moment correlations were computed for each of the significant (i.e., p < .05) hypothesis tests using the formula, r = [t 2/(t 2+ df )]1/2(Rosenthal, 1993 ). Cohen (1988) suggests that in psychological research, r = .10 is a small effect size, r = .30 is a medium effect size, and r = .50 is a large effect size.
The intensity hypothesis was tested in two ways: first, by using the complete sample from the three programs regardless of whether patients had exited them; second, by using only those patients who remained in CM, CLP, or Portals at each time point.
Findings using the complete sample
Figure 1 presents the predicted growth curves for each group on hospitalization scores using the whole sample. There was a significant Group × Intercept (initial status) interaction, t (170) = 4.6, p < .001, with Portals and CLP having poorer scores at baseline than CM (higher scores indicate less hospitalization). The linear and quadratic slopes were significant. The linear slope was positive, t (169) = 1.9, p < .03, and the quadratic slope was negative, t (169) = -2.4, p < .01. There were also significant Group × Slope interactions for both the linear, t (170) = -2.1, p < .02, and the quadratic slopes, t (170) = 1.8, p < .04. The linear slope coefficients were .02 for Portals, .00 for CLP, and -.02 for CM. The quadratic coefficients were -.02 for Portals, -.01 for CLP, and .00 for CM. As shown in Figure 1 , the patients from CM declined in functioning over time, whereas those in Portals showed the most improvement.
Figure 1. Predicted hospitalization score as a function of interview period and program. Diamonds represent the Portals program; squares represent the Community Living Program; triangles represent case management.
Figure 2 shows the predicted growth curves for each group on work functioning using the whole sample. There was no Group × Intercept interaction, suggesting that the groups were equivalent at baseline. The linear coefficient was not significant, nor was there a Group × Linear Slope interaction. The quadratic slope was significant, t (169) = -2.8, p < .005, and there was a significant Slope × Group interaction, t (170) = 2.5, p < .01. The quadratic coefficients were -.033 for Portals, -.01 for CLP, and .01 for CM. These results indicated that, whereas CLP and CM had declining work functioning over time, Portals work scores improved for 2 years and then declined.
Figure 2. Predicted work score as a function of interview period and program. Diamonds represent the Portals program; squares represent the Community Living Program; triangles represent case management.
Figure 3 presents the predicted growth curves for each group on independent living. There was no Group × Intercept interaction. The linear term was significant, t (169) = 4.2, p < .001, and there was a Group × Linear Slope interaction, t (170) = 2.1, p < .02. The linear coefficients were .11 for Portals, .075 for CLP, and .04 for CM. The quadratic term was significant, t (169) = -2.6, p < .01, and the Group × Quadratic Slope interaction was a trend, t (170) = 1.5, p < .07. The quadratic coefficients were -.04 for Portals, -.02 for CLP, and 0 for CM. Figure 2 indicates that although all of the groups improved, Portals and CLP improved more than CM, with Portals showing the greatest increase in functioning over 3 years.
Figure 3. Predicted independent living score as a function of interview period and program. Diamonds represent the Portals program; squares represent the Community Living Program; triangles represent case management.
Figure 4 presents the predicted growth curves for each group on social functioning. There was a significant Group × Intercept interaction, suggesting that Portals was the lowest functioning group, t (170) = 2.8, p < .01. The linear term was significant, t (169) = 3.0, p < .002, and there was also a significant Group × Linear Slope interaction, t (170) = -1.7, p < .05. The linear coefficients were .13 for Portals, .09 for CLP, and .04 for CM. The quadratic term was also significant, t (169) = -2.3, p < .015, and there was a significant Quadratic Slope × Group interaction, t (170) = 2.0, p < .025. The quadratic coefficients were -.04 for Portals, -.01 for CLP, and .015 for CM. From these results, it can be seen that, although all of the groups improved, Portals showed the greatest improvement over the study period.
Figure 4. Predicted social functioning score as a function of interview period and program. Diamonds represent the Portals program; squares represent the Community Living Program; triangles represent case management.
In summary, the results using the whole sample largely supported the intensity hypotheses. The two more intense programs (Portals and CLP) showed higher rates of improvement in hospitalization, independent living, and social functioning than the less intense CM program. Even when CM showed positive change in social and independent living functioning, Portals and CLP improved more in these areas. In terms of work functioning, although both Portals and CLP maintained higher levels of work performance over 3 years than CM, Portals had a significantly higher rate of improvement than both CLP and CM. This differential impact in terms of work functioning is one focus of the specificity analyses.
The effect sizes of these findings ranged from .13 to .31, with an average across the four outcome domains of .19. The average effect size for hospitalization was .16; for work, it was .20; for independent living, it was .22; and for social functioning, it was .17.
Findings using only the in-treatment participants
The second set of intensity analyses used only the patients who remained in treatment at each time point. In terms of hospitalization, the findings on the in-treatment sample were essentially the same as for the whole sample but were statistically stronger. Concerning work functioning, all of the slope and Group × Slope interactions were significant at p < .005. The linear coefficients were .04 for Portals, -.015 for CLP, and -.07 for CM. The quadratic coefficients were -.04 for Portals, -.01 for CLP, and .02 for CM. Portals was the only group that showed improvement in work functioning and maintained positive change from baseline over 3 years. In terms of independent living, the linear, t (169) = 1.9, p < .04; quadratic, t (169) = -2.8, p < .01; and Group × Quadratic, t (170) = 1.9, p < .04, terms were significant. Concerning social functioning, the linear and Group × Linear coefficients were not significant, but the quadratic and Group × Quadratic coefficients remained significant, t (169) = -2.4, p < .01, and t (170) = 2.2, p < .05, respectively.
The effect sizes for these analyses ranged from .14 to .30, with an average across the four domains of .21. The average for hospitalization was .24; for work, it was .26; for independent living, it was .17; and for social functioning, it was .18.
This second set of intensity analyses, which used only the in-treatment patients at each time point, also provided support for the intensity hypothesis. In terms of hospitalization and work functioning, the findings are stronger and more consistent than those using the whole sample, which is notable given the treatment attrition rate over time. On the other hand, in terms of independent living and social functioning the superiority of the two rehabilitation programs was somewhat less evident in these analyses than when using the whole sample. This could suggest that exiting treatment has a greater impact on hospitalization and work functioning than on social and independent living. This is addressed again in the longitudinality analyses below.
Specificity of Services
The hypotheses on specificity were tested using data from Portals and CLP. As above, two sets of analyses were conducted. The first used the complete sample from the two groups. The second used only those patients who remained in treatment at each time point.
The hypothesis concerning work functioning was that Portals would have a significantly greater positive linear or quadratic function than CLP, and that CLP would show little or no change over time. The findings from the complete sample indicated that there was a significant quadratic slope, t (101) = -2.6, p < .005, and a significant Group × Slope interaction, t (102) = 2.0, p < .025. The quadratic coefficients were -.04 for Portals and .01 for CLP. Figure 5 presents the predicted work scores using only the in-treatment patients. The findings were similar and stronger than on the complete sample concerning the quadratic slope and the Group × Slope interaction, t (101) = -4.9, p < .001; t (102) = 3.5, p < .001, respectively; but there was also trend toward significance for the overall linear coefficient, t (101) = 1.4, p < .08. The linear coefficients were .03 for Portals, and 0 for CLP, and the quadratic coefficients were -.045 for Portals, and .01 for CLP.
Figure 5. Predicted work score as a function of interview period and program: in-treatment clients only. Diamonds represent the Portals program; squares represent the Community Living Program.
In terms of independent living, the hypothesis was that both groups would show improvement over time but that CLP would show significantly more improvement than Portals. Using the complete sample, we found that the overall linear slope was positive, but there was no Group × Slope interaction, and none of the parameters was statistically significant. Figure 6 shows the predicted curves using only the in-treatment patients. In this case, there was a trend for the Group × Linear interaction, t (101) = 1.5, p < .07. The linear coefficients were .07 for CLP and .035 for Portals.
The effect sizes for the findings on work functioning averaged .18 (range, .15—.20) for the whole sample and averaged .37 (range, .33—.41) for the in-treatment sample.
Figure 6. Predicted independent living score as a function of interview period and program: in-treatment clients only. Diamonds represent the Portals program; squares represent the Community Living Program.
In summary, there was some support for the specificity hypotheses. Concerning work functioning, as hypothesized, the Portals group showed significantly more improvement than the CLP group, which showed little change over time. In terms of independent living, there was only marginal support for the hypothesis. Although both groups improved over time, the finding was not statistically significant. Similarly, although CLP improved more than Portals, it was only a statistical trend when using the in-treatment patients.
Longitudinality of Services
The longitudinality hypothesis was that the change curves for the in-treatment periods at CLP and Portals would show positive linear or negative quadratic slopes in the functional domains, whereas the out-treatment times would show negative linear or positive quadratic slopes.
In terms of hospitalization, the in-treatment linear coefficient was positive, the quadratic coefficient was negative, and both were statistically significant, t (101) = 1.95, p < .05; and t (101) = -2.7, p < .005, respectively. For the out-treatment times, although the average linear slope was negative (-.02) and the average quadratic slope was positive (.02), which is in the hypothesized direction, neither was statistically significant. The lack of statistical significance for the out-treatment times was likely due to low statistical power because the reliability of the parameter estimates was low and the estimates of the variance components were not statistically significant. Nonetheless, it appears that the significant gains made during treatment attenuated once patients exited treatment.
Concerning work functioning during treatment, the linear slope was positive, and the quadratic slope was negative. The quadratic slope was statistically significant, t (101) = -4.5, p < .001, and the linear slope showed evidence of a trend, t (101) = 1.4, p < .09. The average linear slope during the out-treatment times was negative (-.09), and the average quadratic slope was positive (.02), and although these were in the hypothesized direction, neither was statistically significant. As was the case for the hospitalization scores, these results suggest that work functioning gains made in treatment began to attenuate once patients exited treatment.
In terms of independent living, the in-treatment and out-treatment average linear slopes were positive (.05 and .02), and both average quadratic slopes were negative (-.02, and -.001), although none of the coefficients was statistically significant. This suggests that, although both groups improved over time while in treatment, these gains were not lost once they exited treatment.
Turning to social functioning, the average in-treatment and out-treatment linear slopes were positive (.03 and .02). The in-treatment quadratic slope was negative (-.03) and the out-treatment slope was positive (.02). Although none of these slopes was statistically significant, they suggest that although social functioning improved during treatment, these gains were not lost once patients were out of treatment.
The effect sizes of the statistically significant findings on longitudinality ranged from .22 to .46. The average effect size was .33.
In summary, there was some support for the longitudinality hypothesis with regard to work and hospitalization. In other words, treatment gains in terms of increased work performance and lower rates of institutionalization attenuated once patients exited treatment. In terms of independent living and social functioning the evidence did not clearly support the longitudinal hypothesis. Specifically, although the results suggested that positive change occurred during treatment, the gains in social and independent living functioning were not necessarily lost once the patients exited treatment.
We gathered self-report data on the number of days on prescribed psychiatric medication during each interview period. There were no statistically significant differences across the programs on days medicated throughout the study period.
This study tested hypotheses on multiple client outcomes from three CSPs that varied in the intensity, longitudinality, and specificity of their services. We found that indicators of these service characteristics at the program level were associated with different patterns of change in client functional outcomes.
Concerning intensity, two previous studies found that the amount of contact was related to improvements in global outcome (Sands & Canaan, 1994 ; Snowden & Clancey, 1990 ), but the findings on reduced hospitalization have been contradictory (Dietzen & Bond, 1993 ). In the present study, we found that, over a 3-year period, the two intense programs had higher rates of improvement on social functioning, independent living, as well as time spent in institutional settings than the low-intensity program. The intense programs also had the highest levels of functioning over time in independent living. Concerning work functioning, although patients from both of the intense programs maintained the highest levels of work performance over 3 years, one of the intense programs had considerably higher rates of change and improvement than both of the other programs.
There are two implications of these findings. First, more intense CSPs yielded superior rates of change on a variety of functional outcomes. Put another way, more services were better than less in terms of improved client functioning. The second implication is that intensity alone was not sufficient. This can be seen in the area of work functioning where one of the intense programs showed the greatest vocational change over time, and the other showed almost no change. In this regard, we hypothesized that the specificity of services might be related to particular functional outcomes.
Concerning specificity, we tested two hypotheses that were based on differential program characteristics of the two most intense programs in the study. The hypothesis that the Portals program would show superior longitudinal work outcomes to CLP, and that CLP would show little or no change in vocational performance, was supported. The second hypothesis, also based on differential program characteristics, was that, although both programs would improve in independent living, CLP would improve more than Portals. There was also some support for this hypothesis, in the form of a trend toward statistical significance (p < .07). The implication of these findings is that intensive services might need to be focused on particular functional domains, such as work or independent living, to improve client functioning in those areas. This also supports the notion that targeted multimodal interventions will be the most efficacious with this population (Torrey & Drake, 1995).
Another service variable examined in this study was longitudinality. Previous literature has suggested that terminating CSP services resulted in a loss of gains made in all functional domains (Test, 1984 ). We found that the effect of longitudinality for intensive services was contingent on the outcome domain examined. Concerning hospitalization and work functioning, we found that when patients exited treatment, they began to lose the functional gains they had made. Therefore, it appears that CSPs need an ongoing service focus in these areas if functional change is to be maintained. It is also possible that clients had not improved enough in the work and hospitalization domains and that with more improvement the services could be faded without a loss of functional gain. On the other hand, in terms of social functioning and independent living, exiting treatment did not appear to result in the loss of treatment gains; therefore, maintaining change in these domains might not require ongoing intervention. As such, it is possible that, once change occurs in these areas, treatment could be faded or contacts could be shifted to support other areas where longitudinality is more necessary. This has implications for the design of program services and the arrangement of intervention components.
It should also be noted that some of the findings on longitudinality were based on trends in the data that were not statistically significant. However, corroboration of these conclusions came from the intensity analyses in which treatment effects in terms of work and hospitalization were stronger for those patients who remained in treatment over time. This same pattern was not found in the social and independent living domains.
Another important issue concerns the clinical significance of these findings. First, we argue that, given the difficulty this field has had in achieving functional outcomes with this population, the effect sizes that we reported (ranging from .13 to .46, with a study average of .26, which is near the "medium" range) are clinically notable and can provide a foundation for clinical decision making, as well as for future research in this area.
Second, the scales themselves provide a basis for assessing clinical significance in the four outcome domains. In this regard, each score has anchors consisting of clinical descriptors. As we illustrate here, a small change in values on the scale reflects a relatively large change in behavior. For example, concerning work functioning, the change in scores presented in this study represents movement in the range from no work to half-time work (i.e., scores of 0—2 on the item). This is clinically significant change for this population. Similarly, the change in scores on hospitalization is from no hospitalization to hospitalization for less than 6 weeks during a 6-month period (i.e., scores of 4 and 3), which can be considered practically significant, as well as having cost implications. On social functioning, a change from below 3 to above 4 represents movement from severely limited or conflicted friendships, sometimes with only mental health providers, to having one or two close and continuing peer friendships. For this population, this is also clinically meaningful change. The same is true for independent living where the achieved scores change from limited independent living skills to having achieved some marginal maintenance of independent living and self-care. As such, we argue that these results represent clinically significant rehabilitative change for this population of individuals with schizophrenia.
In summary, these results suggest that the intensity, specificity, and longitudinality of services in CSPs deserve attention as variables important to improving client functional outcomes. These are particularly important findings because although there is a growing body of literature on the effectiveness of CSPs (Olfson, 1990 ; Scott & Dixon, 1995 ; Solomon, 1992 ; Test, 1992 ), and considerable discussion about how these services should be organized to maximize client outcomes (e.g., Bachrach, 1992 ), there has been almost no study concerning the impact of service characteristics on functional outcomes from CSPs. These findings have clear implications for both clinicians and program administrators in CSPs. Clinicians can be trained to deliver interventions that reflect these service characteristics. Administrators can facilitate the organizational conditions that support the implementation of these service characteristics. Future research should continue to investigate the influence of these variables on client outcomes in various settings and with different client groups. More work is also needed to develop conceptual and operational consensus on these variables.
Several other issues deserve comment from these findings. First, the frequent significance of the negative quadratic curve coefficients suggests that the nature of functional change in CSPs might be nonlinear. More specifically, it appears that change begins in an accelerated fashion and then attenuates over time. This suggests that clinicians should not expect clients to maintain initial levels of improvement and that clients might need particularly intensive support after initial change begins to occur. It is also possible that these quadratic trends are related to a loss of sample size over time, especially in analyses using only the in-treatment patients.
Second, this was a diagnostically homogeneous sample of individuals in the schizophrenia spectrum. Because individuals with schizophrenia are commonly seen as more refractory to rehabilitative interventions than other diagnostic groups (Group for the Advancement of Psychiatry, 1992 ), the treatment effects we reported are notable. Third, this study investigated client outcomes over a 3-year period that offered a substantial window on the trends in outcome. Finally, previous investigators have noted the importance of gathering service implementation data (Brekke, 1988 ; McGrew et al., 1994 ). This study illustrates how combining program implementation data with client outcome data can address important questions about the effective ingredients of CSPs.
There are several methodological caveats to consider. First, in this study we did not use random assignment to conditions; therefore, differences in client characteristics across the conditions could confound the interpretation of differential program effects and the effects of the service variables. In general, however, the groups were equivalent at baseline on numerous variables that have been related to functional outcomes in schizophrenia. In addition, group differences on baseline levels of the hospitalization and social outcome variables suggested that the program that showed consistently high rates of change (Portals) also had patients that were somewhat lower functioning in these two areas at baseline. Although this could be due to statistical regression, it could suggest that this program worked well with the lowest functioning clients in the study. It should also be noted that the effect of the group covariate was generally the same in all functional outcome domains regardless of whether there were differences in initial status. Nonetheless, we cannot rule out the possibility that a selection factor affected our findings to some degree.
Second, there could be a variety of other service delivery variables not measured here that could account for some of the differences in client outcomes across the programs. This suggests that, although we found support for hypotheses about specific service characteristics, causal inferences should be drawn with caution. Third, the variations in service implementation in this study occurred at the program level. It should be noted that these findings might not generalize to service implementation effects at the individual client level. Fourth, although one of the strengths of this study was that we compared three CSPs that varied in their service implementation, the degree to which the findings are generalizable to other CSPs is unknown. Fifth, because we excluded patients diagnosed with comorbid substance dependence, the generalizability of these findings to a dually diagnosed population needs to be assessed in future studies.
Future research needs to examine other service variables such as individualization, comprehensiveness, or other aspects of longitudinality such as gaps in care (Brekke & Test, 1992 ). In this regard, although there has been increasing attention given to assessing program fidelity in research on CSPs (Brekke, 1987 ; McGrew et al., 1994 ; Scott & Dixon, 1995 ), of equal significance is the examination how specific service ingredients are related to functional outcomes. In this effort, a different methodological strategy than the one used in this study would be to examine the correlations between client scores on service variables and their prospective changes in psychosocial functioning. This is based on using the client as the unit of observation in gathering service data and will allow for the examination of the degree to which these individual-level service variables directly covary with client outcomes. Investigators should also note the utility of multidimensional functional outcome measures. Clearly, they allow for assessing the differential impact of service variables on distinct dimensions of psychosocial outcome. Finally, another critical area for study concerns individual client characteristics such as prognosis or functional level that could be related to the character of services that are necessary to achieve functional change.
ReferencesBachrach, L. L. (1992). Psychosocial rehabilitation and psychiatry in the care of long-term patients.( American Journal of Psychiatry, 149, 1455—1463.)
Bond, G. R., McGrew, J. H. & Fekete, D. M. (1995). Assertive outreach for frequent users of psychiatric hospitals: A meta-analysis.( Journal of Mental Health Administration, 22, 4—16.)
Brekke, J. (1987). The model-guided method for monitoring program implementation.( Evaluation Review, 11, 281—299.)Brekke, J. (1988). What do we really know about community support programs?(Strategies for better monitoring. Hospital and Community Psychiatry, 39, 946—953.)
Brekke, J. (1992). An examination of the relationships among three outcome scales in schizophrenia.( Journal of Nervous and Mental Disease, 180, 162—167.)
Brekke, J., Levin, S., Wolkon, G., Sobel, G. & Slade, B. (1993). Psychosocial functioning and subjective experience in schizophrenia.( Schizophrenia Bulletin, 19, 599—608.)
Brekke, J. & Test, M. A. (1992). A model for measuring the implementation of Community Support Programs: Results from three sites.( Community Mental Health Journal, 28, 227—247.)
Bryk, A. S. & Raudenbush, S. W. (1987). Application of hierarchical linear models to assessing change.( Psychological Bulletin, 101, 147—158.)
Bryk, A. S. & Raudenbush, S. W. (1992). Hierarchical linear models. (Newbury Park, CA: Sage)
Bryk, A. S., Raudenbush, S. W. & Congdon, R. T. (1994). HLM/2L Release 3.1. (Chicago: Scientific Software International)
Cliff, N. (1987). Analyzing multivariate data. (New York: Harcourt Brace Jovanovich)
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. (Hillsdale, NJ: Erlbaum)
Dietzen, L. L. & Bond, G. R. (1993). Relationship between case manager contact and outcome for frequently hospitalized psychiatric clients.( Hospital and Community Psychiatry, 44, 839—843.)
Endicott, J. & Spitzer, R. (1978). A diagnostic interview: The schedule of affective disorders and schizophrenia.( Archives of General Psychiatry, 35, 837—844.)
Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid estimation.( Psychological Bulletin, 70, 245—251.)
Goodman, S. H., Sewell, D. R., Cooley, E. L. & Leavitt, N. (1993). Assessing levels of adaptive functioning: The role functioning scale.( Community Mental Health Journal, 29, 119—131.)
Green, R. S. & Gracely, E. J. (1987). Selecting a rating scale for evaluating services to the chronically mentally ill.( Community Mental Health Journal, 23, 91—102.)
Group for the Advancement of Psychiatry.(1992). Beyond symptom suppression: Improving long-term outcomes of schizophrenia. (Washington, DC: American Psychiatric Press)
Hargreaves, W. A. & Shumway, M. (1989). Effectiveness of services for the severely mentally ill.(In C. A. Taube, D. Mechanic, & A. A. Hohmann (Eds.), The future of mental health services research (pp. 253—283). Washington, DC: U.S. Department of Health and Human Services.)
Heinrichs, D. W., Hanlon, T. E. & Carpenter, W. T. (1984). The Quality of Life Scale: An instrument for rating the schizophrenia deficit syndrome.( Schizophrenia Bulletin, 10, 388—396.)
Jordan, D. (1985). Impact of an alternatives program on client utilization of services in a county mental health system.( Hospital and Community Psychiatry, 36, 1291—1296.)
Kirk, R. E. (1995). Experimental Design. (Pacific Grove, CA: Brooks/Cole)
McGrew, J. H.,Bond, G. R., Dietzen, L. & Salyers, M. (1994). Measuring the fidelity of implementation of a mental health program model.( Journal of Consulting and Clinical Psychology, 62, 670—678.)
McPheeters, H. L. (1984). Statewide mental health outcome evaluation: A perspective of two southern states.( Community Mental Health Journal, 20, 44—55.)
National Institute of Mental Health.(1982). A network for caring: The Community Support Program of the National Institute of Mental Health ((Publication no.ADM-01-1063). Rockville, MD: Author)
Olfson, M. (1990). Assertive community treatment: An evaluation of the experimental evidence.( Hospital and Community Psychiatry, 41, 634—651.)
Raudenbush, S. W. & Bryk, A. S. (1985). Empirical Bayes meta-analysis.( Journal of Educational Statistics,10, 75—98.)
Rogosa, D. R., Brand, D. & Zimowski, M. (1982). A growth curve approach to the measurement of change.( Psychological Bulletin, 90, 726—748.)
Rogosa, D. R. & Willett, B. (1985). Understanding correlates of change by modeling individual differences in growth.( Psychometrika, 50, 203—228.)
Rosenblatt, A. & Attkisson, C. C. (1993). Assessing outcomes for sufferers of severe mental disorders: A conceptual framework and review.( Evaluation and Program Planning, 16, 347—363.)
Rosenthal, R. (1993). Cumulating evidence.(In G. Keren & C. Lewis (Eds.), A Handbook for data analysis in the behavioral sciences: Methodological issues (pp. 519—552). Hillsdale, NJ: Erlbaum.)
Sands, R. G. & Canaan, R. A. (1994). Two modes of case management: Assessing their impact.( Community Mental Health Journal, 30, 441—457.)
Scott, J. E. & Dixon, L. B. (1995). Assertive community treatment and case management for schizophrenia.( Schizophrenia Bulletin, 21, 657—668.)
Snowden, L. R. & Clancey, T. (1990). Service intensity and client improvement at a predominantly Black community mental health center.( Evaluation and Program Planning, 13, 205—210.)
Solomon, P. (1992). The efficacy of case management services for severely mentally diasbled clients.( Community Mental Health Journal, 28, 163—180.)
Strauss, J. S. & Carpenter, W. T. (1972). The prediction of outcome in schizophrenia: I. Characteristics of outcome.( Archives of General Psychiatry, 27, 739—746.)
Strauss, J. S. & Carpenter, W. T. (1974). The prediction of outcome in schizophrenia: II. Relationships between predictor and outcome variables.( Archives of General Psychiatry, 31, 159—163.)
Strenio, J. L., Weisberg, H. I. & Bryk, A. S. (1983). Empirical Bayes estimation of individual growth curve parameters and their relationship to covariates.( Biometrics, 39, 71—86.)
Stroul, B. (1989). Community support systems for person with long-term mental illness: A conceptual framework.( Psychosocial Rehabilitation Journal, 12, 9—26.)
Taube, C. A., Morlock, L., Burns, B. J. & Santos, A. B. (1990). New directions in research on assertive community treatment.( Hospital and Community Psychiatry, 41, 642—647.)
Tessler, R. C., Willis, G. & Gubman, G. D. (1986). Defining and measuring continuity of care.( Psychosocial Rehabilitation Journal, 10, 27—38.)
Test, M. A. (1981). Effective community treatment of the chronically mentally ill: What is necessary?( Journal of Social Issues, 37, 71—86.)
Test, M. A. (1984). Community support programs.(In A. S. Bellack (Ed.), Treatment and care for schizophrenia (pp. 347—373). New York: Grune and Straton.)
Test, M. A. (1992). Training in community living.(In R. P. Liberman (Ed.), Handbook of psychiatric rehabilitation (pp. 221—242). New York: Allyn and Bacon.)
Test, M. A., Knoedler, W. H., Allness, D. J., Burke, S. S., Brown, R. L. & Wallisch, L. S. (1991). Long-term community care through an assertive continuous treatment team.(In C.
Tamminga & S. Schultz (Eds.), Advances in neuropsychiatry and psychopharmacology, Vol. I: Schizophrenia research (pp. 239—246). New York: Raven Press.)
Test, M. A. & Scott, R. (1990). Theoretical and research bases of community care programs.(In I. M. Marks & R. A. Scott (Eds.), Mental health care delivery: Innovations, impediments and implementation (pp. 11—16). New York: Cambridge University Press.)
Test, M. A. & Stein, L. I. (1978). Community treatment of the chronic patient: Research overview.(Schizophrenia Bulletin, 4, 350—364.)
Torrey, W. C. & Drake, R. E. (1994). Current concepts in the treatment of schizophrenia.(Psychiatry, 57, 278—285.)
Turner, J. C. & TenHoor, W. J. (1978). The NIMH Community Support Program: Pilot approach to a needed social reform.( Schizophrenia Bulletin, 4, 319—349.)
Willett, J. B. (1988). Questions and answers in the measurement of change.(In E. Rothkopf (Ed.), Review of research in education 1988—89 Washington, DC: American Educational Research Association.)
Copyright © 2000, Southern Development Group,
S.A. All Rights Reserved.
A Private Non-Profit Agency for the good of all, published in the UK & Honduras