Abstract
We developed a 13-item scale to measure work and educational satisfaction and aspiration among mental health clients.  The scale was initially developed on a sample of 150 clients, and was validated on a sample of 358 clients.  The validation sample was 52% male, mean age was 48 years, 83% had no paid employment, and 79% were high school graduates.  Exploratory factor analysis indicated that the scale measures four distinct dimensions: work satisfaction (WS), educational satisfaction (ES), work aspiration (WA) and educational aspiration (EA).  The subscales are brief (3 to 4 items each) and demonstrate good psychometric properties.  Cronbach’s alpha ranged from .79 to .90 across the four subscales.  Criterion-related validity was demonstrated for each subscale.  WS was higher for employed than unemployed participants; ES was higher among more educated participants; WA was higher for clients who expressed a desire to start working or to move to more competitive levels of employment; EA was higher for clients who indicated a desire for more education.  The scale was created to support recovery-based mental health treatment, and is potentially useful in both individual treatment and program evaluation.

Keywords: educational, vocational, assessment, recovery modelMeasurement of Vocational and Educational Aspiration and Satisfaction among Mental Health Clients

Introduction:
Over the last half century, mental health treatment has increasingly emphasized integration of clients into the community.  Deinstitutionalization was implemented in the 1960s and 1970s.  This led to the concepts of community support and psychiatric rehabilitation in the 1980s.  The recovery model of mental health treatment began to take hold in 1990.  Central to this model is the idea that many of the limitations due to mental illness can be overcome, and that mental health clients can lead productive lives.  A recovery model promotes the development of meaning and purpose as one overcomes the catastrophic effects of severe mental illness (Anthony, 1993). 

Adoption of a recovery model may require new tools and new ways of conceptualizing client problems.  Ralph, Ridgway and Onken (2004) suggested that recovery programs need to offer replicable, measureable services, and that this requires the creation of new instruments to quantify recovery-based care.  Campbell-Orde, Chamberlin,  Carpenter,  and Leff (2005) echoed that sentiment when they stated, “As the current mental health system begins to undergo fundamental changes based on [recovery], research and evaluation activities are necessary to the successful development and continuous delivery of services that do indeed promote recovery” (p. 10).  

Securing employment is a key element of reintegration into society.  Marrone, Gandolfo, Gold and Hoff (1998) suggested that, “Any helper's belief system has to include a core value that people with mental illness in our society can, and should, work as part of their citizenship rights and responsibilities” (p. 37). Cook et al. (2005) conducted a randomized trial of supported employment interventions for individuals with severe mental illness.  They found that clients who participated in supported employment had better outcomes at 24-month follow-up, including greater likelihood of competitive employment and higher earnings.

The unemployment rate among the mentally ill is typically 90%, exceeding the rate of any other disabled group (New Freedom Commission on Mental Health, 2003).  This high rate is in spite of the finding that many mentally ill persons want to work, and could work with modest accommodations (Drake et al., 1999).

A number of existing tests measure vocational interests, attitudes, and job-seeking skills, but a review of the literature did not show any currently available instruments which could validly and reliably measure client desire for both employment and education.  The employment satisfaction scale of the Peer Outcome Protocol appears to come closest (Campbell, Einspahr, Evenson & Adkins, 2004).  However, this scale demonstrates relatively low internal consistency and test-retest reliability, with Cronbach’s alpha = .70, and two week test-retest reliability = .51.  It appears to merge several different constructs into a single score, including items which query satisfaction with employment status, desire to work, employment-related program resources, and self-rated ability to find and keep a job (Campbell et al, , 2004).   

The goal of the current study was to identify clients who were dissatisfied with their current employment and educational status, and who were seeking to advance in these areas.  This information can then be used for program planning purposes, to target resources, and as a basis for counseling.  To this end, we created four scales, measuring satisfaction and aspiration with regard to education and employment.

Instrument Creation
Item Creation
The authors developed 32 items, 8 per factor, covering four factors.  An ad hoc committee of mental health researchers reviewed the items and suggested modifications.  Each item was to be rated on a 4-point scale from 1/strongly disagree to 4/strongly agree.  The four scales are:
•    Work satisfaction (WS): Satisfaction with current work situation, whether employed or not.
•    Work aspiration (WA): Desire to work, or to improve one’s work situation.
•    Educational satisfaction (ES): Satisfaction with current level of educational attainment.
•    Educational aspiration (EA): Desire to obtain additional education.
Current living situation, current living situation satisfaction, current work situation, desired work situation, current educational level, and desired educational level also were assessed. Demographic and treatment program information, including age, gender and clinic name were recorded.

Data Collection
These surveys were distributed at the five Orange County Full Service Partnership (FSP) programs. FSPs are county outpatient programs, specified by the California Mental Health Services Act (MHSA, 2005), which provide a full spectrum of community services to clients with severe mental illness.  Eligible clients are functionally impaired and at risk for homelessness, criminal justice involvement, psychiatric hospitalization or emergency room use for mental health issues.  FSPs provide mental health services and supports, including standard, alternative, and culturally specific treatment.   Non-mental health services and supports include housing subsidies, food, clothing, and access to educational, social, and vocational rehabilitative services.  FSPs are an integral part in the MHSA vision of transforming the mental health system into one that is ever more responsive to the real needs of mental health consumers and one that can support their journey through the recovery process.

The surveys were distributed by program service coordinators.  They explained the nature of the assessment and obtained client consent.  The study was reviewed and approved by the Institutional Review Board of the Orange County Health Care Agency.  All clients attending an FSP during the two-week study period were offered a survey.  A total of 130 participants anonymously completed surveys; 119 of these surveys contained no more than 10 omitted items and were retained for this analysis. 
Sample Characteristics
This initial sample was 57.1% male with a mean age of 44.8 years (SD = 12.2 years; range 23-72 years).  Of the 112 participants who reported their employment status,73.2% were unemployed and 15.2% reported that they did volunteer work.  Out of 92 participants who reported their living situation, 47.8% had secured permanent housing and the remainder were in transitional or emergency housing.    Level of education was reported by 98 participants, with 75.5% being high school graduates and 11.8% being college graduates.

Item Selection
Because we wanted to create brief scales, we retained the fewest items possible while still maintaining adequate internal consistency (α > .70).  We eliminated items which sounded very similar to other items and items with significant amounts of missing data.  Using factor analytic techniques, we chose items so that four distinct factors would emerge, removing items which correlated with scales other than their own.  Thirteen items were retained for the final Instrument Validation

Method
The 13-item instrument was distributed to outpatient mental health clients in FSP programs.  A total of 358 anonymous surveys were returned. 
Sample description.  Of those reporting their gender (n = 298), 52.0% were male.  Age was reported by 350 participants, with a range of 19 – 79 years, mean of 48.2 years and standard deviation of 14.0 years.  Work situation was reported by 338 participants: 67.9% did not have paid or volunteer work, volunteer engagements were reported by 14.8%, part-time competitive employment by 8.7%, full-time competitive work by 1.7% and supported employment by 1.4%.  Supported employment is paid work with additional support services, such as such as job coaches, transportation, assistive technology, specialized job training, and individually tailored supervision.   Of those reporting their educational level (n = 340), 10.7% had a four-year college degree, and 78.8% had graduated high school.

Results
Scale characteristics.  Table 1 shows the characteristics of the four scales.  In spite of their brevity, they demonstrated adequate reliability, with Cronbach’s alpha ranging from .79 to .90.  Participants gave mean ratings to the work and educational aspiration scales which were near the “Agree” level.  Work and educational satisfaction, on the other hand, were between the “Agree” and “Disagree” ratings.

Table 1   Scale characteristics

Scale	N	N of items	Cronbach's alpha	Mean*	Std. Deviation
Work Aspiration (WA)	314	3	.80	2.96	0.74
Educational Aspiration (EA)	319	4	.90	2.89	0.92
Work Satisfaction (WS)	296	3	.79	2.41	0.87
Educational Satisfaction (ES)	324	3	.82	2.65	0.86
*Items are rated from 1 (Strongly disagree) to 4 (Strongly agree).  For easier interpretation and comparison, individuals' scale scores are represented as the mean of their item responses.

Table 2  Exploratory Factor Analysis Showing Item Factor Loadings

Item	EA	ES	WS	WA
WA1  I will work hard to improve my work situation	-.01	-.02	-.01	-.77
WA2  I am willing to put in effort to have a job I enjoy.	-.01	.00	.01	-.87
WA3  Having a good job is important to my sense of well-being.	.05	.00	-.08	-.67
EA1  I am interested in attending more school.	.90	-.02	-.04	.07
EA2  I am willing to work hard to get more education.	.82	-.03	-.04	-.07
EA3  One of my most important goals is to get more education.	.86	-.02	.02	-.05
EA4  I would put effort into a school or a training program if it would lead to a good job.	.51	.05	.08	-.37
WS1  I have work which is satisfying to me.	.02	-.08	-.75	-.01
WS2  I feel as though my work is a good fit for my skills and abilities.	.02	.03	-.82	.04
WS3  I do work that is interesting to me.	-.03	.11	-.68	-.08
ES1  I am happy with the amount of education I have completed.	-.20	.72	.02	-.02
ES2  I am satisfied with my educational accomplishments.	-.07	.86	-.05	.01
ES3  I am proud of my education.	.19	.73	-.03	.02

Note.  Factor loadings with absolute value > .50 are in boldface.  EA = Educational Aspiration, ES = Educational Satisfaction, WS = Work Satisfaction, WA = Work Aspiration.

 

Factor analysis.  An exploratory factor analysis was performed using principle axis factoring as the extraction method and direct oblimin as the rotation method.  Oblique rotation was chosen because it was anticipated that the factors would have some intercorrelation.  A scree test indicated that a four-factor solution was appropriate.  Table 2 shows the pattern matrix resulting from this analysis.  The scales demonstrated factor structure which was consistent with expectation.  All items had high loadings on their own scales, and loaded minimally on the other scales.

Table 3 Factor Intercorrelations

Measure	EA	ES	WS	WA
EA	--	-.17	-.16	-.59
ES		--	-.34	.01
WS			--	.30
WA				--

Validity.
Work aspiration (WA).  The WA scale measures the client’s desire to work.  It is expected that clients with an interest in working have negotiated a work recovery plan with their counselor.  The survey asked clients whether they had a work recovery plan, and they were given response options of “yes,” “no,” and “don’t know.”  As expected, clients who reported having a work recovery plan had a higher mean WA score than those who responded with “no” or “don’t know” (3.43 vs. 2.51, respectively; t = 13.75, df = 304, p < .001,  = .62). 

Clients were asked to describe their current work status and their desired work situation.  For both items, the options were 1:  not working, 2: volunteer, 3: supported employment, 4: part-time competitive employment, and 5: full-time competitive employment.  When clients indicated that their desired work situation was at a higher level than their current work status, they were coded as having a desire for work advancement.  For instance, a client who was volunteering but desired supported or competitive employment was coded as having a desire for work advancement; a client who was currently volunteering who would like to continue volunteering or who wished to stop working altogether was coded as not having a desire for work advancement.  As expected, clients with a desire for work advancement had a higher mean WA score than clients without a desire for work advancement (3.25 vs. 2.50, respectively; t = 9.43, df = 295, p < .001,  = .48). 
Educational aspiration (EA).  The EA scale measures the client’s desire for educational advancement.  It is expected that clients with an interest in working have negotiated an educational recovery plan with their counselor.  The survey asked clients whether they had an educational recovery plan, and they were given response options of “yes,” “no,” and “don’t know.”  As expected, clients who reported having an educational recovery plan obtained a higher mean EA score than clients who responded with “no” or “don’t know” (3.48 vs. 2.36, respectively, t = 13.26, df = 301, p < .001,  = .37). 

 Clients were asked to describe their current level of education and their desired level of education.  Clients were coded as either indicating or not indicating a desire for additional education.  As expected, clients with a desire for additional education scored higher on the EA scale than clients without a desire for additional education (3.25 vs. 2.06, t = 12.78, df = 302, p < .001,  = .59). 
Work satisfaction (WS).  Consistent with expectation, clients who were not working obtained a lower WS score than those who were engaged in volunteer or paid employment (2.21 vs. 2.84 respectively, t = 6.00, df = 284, p < .001,  = .34).

Educational satisfaction (ES).  Educational level was coded as 0: No high school diploma, 1: Diploma/GED, 2: Some college or vocational training, 3: Associate’s degree, 4: Bachelor’s degree, 5: Some graduate school, 6: Graduate degree.  As anticipated, ES was positively associated with educational level, rs = .26,  df  =  313, p <  .001. 

Discussion
The present study was undertaken to develop measures of vocational and educational satisfaction and aspiration for clients with severe and chronic mental illnesses.  Integration into educational and work settings is an important step for many mental health clients.  The present scales were seen as important both as clinical tools and a means to track program success in these areas.  The scales that were developed are brief and show strong psychometric qualities.  They measure four distinct dimensions, show respectable internal consistency, and exhibit criterion-related validity. 

These scales could provide a clinician with valuable clinical information.  The aspirational scales measure a client’s desire to pursue employment and education.  High work aspiration with low educational aspiration suggests a preference for work placements requiring little or no additional training.  The reverse, low work aspiration with high educational aspiration, could suggest the desire to return to school primarily for personal enrichment.  If a client scores low on both the work satisfaction and aspiration scales, this information could serve as a useful focus for therapeutic discussion.  This pattern would suggest a client who is unhappy with the current state of affairs, but with little motivation for change.  Many clients have become discouraged by what they perceive as limitations imposed by their illness.  A skillful therapist could help a discouraged client to explore vocational possibilities.  Increasing vocational and educational aspiration could then become a measureable therapeutic goal.

These scales also could be useful for program evaluation.  As clients progress through a program and their own recovery, programs would aspire to increase their vocational and educational satisfaction.  This could mean different things to different clients, and would not necessarily mean that all clients would be competitively employed or attain college degrees.  It would simply mean that they become comfortable with their vocational or avocational pursuits, and feel satisfied with their level of training or education.  Programs would also seek a general increase in aspiration, understanding that some clients may have made realistic and well-considered decisions not to move forward in these areas, and other clients may reduce their aspirational levels once they achieve their vocational and educational goals.
     
As the recovery model matures, there will be an ongoing need for instruments to measure, track and facilitate specific domains of client improvement.  Determining areas of recovery that could benefit from assessment and developing instruments to accomplish this could prove a fruitful area of research and would make a significant contribution to both the literature and the quality of clinical care.   An area for future research with these scales would be to look at their predictive utility, determining whether clients’ levels of satisfaction and aspiration tend to be predictive of the speed and extent of their recovery.   It would also be useful to determine whether scores on these tests can be increased through counseling and psychotherapeutic interventions, and whether these increases translate into integration into the workforce.

References

Anthony, W. A. (1993).  Recovery from mental illness: The guiding vision of the mental health service system in the 1990s.  Psychosocial Rehabilitation Journal, 16(4), 11–23.

Campbell-Orde, T., J., Chamberlin, J., Carpenter, J. C., & Leff, H. S.  (2005).   Measuring the Promise: A Compendium of Recovery Measures, Volume II.  Cambridge: Human Services Research Institute. 

Campbell, J., K., Einspahr, K., Evenson, R. & Adkins, R.  (2004).  Peer Outcomes Protocol (POP): Psychometric Properties of the POP.  Chicago: University of Illinois.  Available online at http://www.cmhsrp.uic.edu/download/POP.Psychometrics.pdf.

Compton, M. T.  (2007) Recovery: Patients, families, communities conference report.  Medscape Psychiatry & Mental Health, October 11-14, 2007.  Retrieved from http://www.medscape.com/viewarticle/565489_print.

Cook, J. A.,  Leff, H. S., Blyler, C.R., Gold, P. B., Goldberg, R. W., Mueser, K. T., … Burke-Miller, J.  (2005). Results of a multisite randomized trial of supported employment interventions for individuals with severe mental illness.  Archives of  Gen Psychiatry. 62:505-512. 

Drake, R.E., Becker, D.R., Clark, R.E., and Mueser, K.T. (1999). Research on the individual placement and support model of supported employment. Psychiatric Quarterly, 70(4), 289-301.

Marrone, J., Gandolfo, C., Gold, M., Hoff, D. (1998).  Just doing it: Helping people with mental illness get good jobs. Journal of Applied Rehabilitation Counseling, 29 (1), 37-48.

Mental Health Services Act, California Code of Regulations, Title 9, § 3200 (2005).

New Freedom Commission on Mental Health. (2003). Achieving the Promise: Transforming Mental Health Care in America. Final Report. DHHS Pub. No. SMA-03-3832. Rockville, MD.

Ralph, R. O.,  Ridgway, P. & Onken, S. J. (2004).  Tools in development: Measuring recovery at the individual, program, and system levels.  Retrieved from National Association of State Mental Health Directors website: http://www.nasmhpd.org/spec_e-report_fall04measures.cfm.