The International Journal of Psychosocial Rehabilitation

Preliminary Psychometrics of a New Scale:
 A Sense of Acceptance in Community Activities


Phyllis Solomon, PhD
Professor
 University of Pennsylvania School of Social Policy and Practice. Affiliated with UPENN Collaborative.

Sungkyu Lee, MSW
Doctoral Student
 University of Pennsylvania School of Social Policy and Practice.

Arpita Chatterjee, MSW
Doctoral Student
Rutgers University School of Social Work.

LaKeetra McClaine, BA, BSN, RN
Research Specialist
 UPENN Collaborative, University of Pennsylvania School of Medicine




Citation:
Solomon P, Lee S, Chatterjee A & McClaine L.(2010). Preliminary Psychometrics of a New Scale: A Sense of
Acceptance in Community Activities. International Journal of Psychosocial Rehabilitation. Vol 14(2) 113-122

---

Introduction
Community integration of adults with severe psychiatric disorders is a major policy goal in the United States as affirmed most recently by the landmark Olmstead Supreme Court decision in 1999. Yet, all too frequently, this population is physically in the community, but they do not have a psychological or social sense of belonging to the community (Dewees, Pulice, & McCormick, 1996; Granerud & Severinsson, 2006; Prince & Prince, 2002). Although loneliness and social isolation are highly prevalent among those with severe psychiatric disorders, they are frequently reluctant to participate in community resources and activities due to fear of rejection (Elisha, Castle, & Hocking, 2006; Perese & Wolf, 2005). Consequently, an essential goal of community mental health programs, particularly psychiatric rehabilitation programs, is to assist in facilitating greater community participation of their clients (Prince & Prince, 2002). As a part of such efforts in the current atmosphere of evidence based practice, there is a need to be able to measure these programs’ effectiveness in achieving community integration of their clients. However, most of the measures of community integration have focused on physical integration, (i.e., participating in community activities and use of community resources), neglecting social (i.e., social interactions and social networks); and psychological aspect of integration (i.e., a sense of belonging and acceptance) (Wong & Solomon, 2002). The present study addressed this gap by developing a measure of a sense of acceptance in community activities for persons with severe psychiatric disorders. In this article, we report on the psychometric properties of the measure Sense of Acceptance in Community Activities, specifically designed for the population served by community-based mental health programs.

Background
Psychological integration has to do with the perceptual aspects of feeling a part of the community or feeling as though one belongs to a group and is a valued and accepted member of the group. Therefore, it is not surprising that some have equated a sense of belonging with the concept of social integration (Hagerty & Patusky, 1995). However, there has been limited development of measures in the realm of psychological and social integration per se. Recently, there is an emerging mental health literature on a sense of belonging, which is an important component of psychological integration. The research in this area has focused on the general population, not specifically on persons with severe psychiatric disorders.
According to Baumeister and Leary (1995), belonging has been identified as a basic human motivation and an integral component of positive mental health. Hagerty and her colleagues (1992) have defined a sense of belonging as “the experience of personal involvement in a system or environment so that persons feel themselves to be an integral part of that system or environment” (p. 173). This concept has two delineated characteristics: “[1] the experience of being valued, needed, or important with respect to other people, groups, or environments, and [2] the experience of fitting in or being congruent with other people, groups, or environments through shared or complementary characteristics” (Hagerty, Williams, Coyne, & Early, 1996, p. 236). Hagerty and her colleagues (1996) note that in the absence of a sense of belonging individuals feel lonely and depressed. Their subsequent research concluded that those with a greater sense of belonging have a greater degree of social and psychological functioning. Deficits in a sense of belonging have been determined to be associated with increased feelings of loneliness (Hagerty et al., 1996), higher levels of depression (Cuijpers & Van Lameren, 1999), and diminished psychological well-being (Coyne & Downey, 1991).  

While the goal of community integration or social inclusion is to have persons with a psychiatric disorder feel a sense of belonging in their communities of choice, the first step toward achieving this goal is for them to feel comfortable and accepted when participating in community groups and social activities. Without this level of comfort and an absence of anxiety, it is unlikely that persons with severe psychiatric disorders will utilize or participate in the resources available to them as members of their communities. Consequently, we focused our efforts on developing a measure on a concept that we saw as a precursor to achieving this more ambitious goal, a sense of acceptance when participating in community activities. We defined a sense of acceptance as feeling socially and psychologically comfortable and welcomed when attending events and activities in the community. Given the high degree of stigma and discrimination that those with psychiatric disorder frequently experience, their fears of rejection and feeling of shame may inhibit them from taking advantage of opportunities available to them, including using a variety of community resources. Therefore, they may well avoid utilizing such resources even though they have the potential to reduce their social isolation and feelings of loneliness and increasing their sense of social inclusion. Ultimately, it is expected that such participation by persons with severe psychiatric disorders will likely lead to increasing the size of their social networks and the degree of their social support. We therefore hypothesized that measures of loneliness, stigma, and social inclusion would likely be related to the construct of social acceptance; and if so, would offer evidence of the construct validity of our measure. 

Method
Participants
Study participants were recruited from two Assertive Community Treatment (ACT) teams. The Sense of Acceptance in Community Activities (SACA) scale was part of a comprehensive interview schedule that was administered prior to clients entering an intervention study designed to increase their participation in community resources and activities, where clients from one team, which was randomly selected, would receive the intervention and clients from the other team would not. Team staff asked clients about their interest in participating in the study. If clients were interested, they were referred to research assistants who screened for study eligibility: their interest in increasing or maintaining their involvement in community activities and whether they had a severe psychiatric diagnosis (i.e., schizophrenia spectrum disorder or major affective disorder). If eligible and interested, their consent for the study was obtained. This study was reviewed and approved by the Institutional Review Board of University of Pennsylvania.

A total of 137 clients were approached for the study. Of the 137 clients, about 21% of clients (n = 29) refused to be screened and 17% of clients (n = 23) were ineligible for the study. After the screening and determined to be eligible, six clients (4%) refused to participate in the study and fifteen clients (11%) did not show for the interview. Finally, sixty-four clients (47%) were interviewed by trained interviewers. Among them, eight clients were terminated by either interviewers or participants themselves. One client who did not answer the items on the SACA measure was excluded from the present analysis.

The final sample of 55 clients consisted of 50.9 % male (n = 28) and 49.1 % female (n = 27), ages 23-71 with an average age of 43.33 years (SD = 11.64). The majority of the sample was white (50.9%), followed by black (43.6%), and had a diagnosis of schizophrenia or schizophrenia spectrum disorder (70.9 %) with 25.5% having a bipolar or major depressive disorder.

Measurements
A Sense of Acceptance in Community Activities (SACA)
A Sense of Acceptance in Community Activities (SACA) was originally an 8-item, self-report scale. The participants were asked if they feel a part of the community and feel comfortable when participating with strangers in community activities and groups. The items were generated by the first author based on a review of the literature, as well as clinical and research experiences. The items were then reviewed by three experts who had extensive research and/or practice experience with the population. Based on their input, the items were refined. The questions were measured on a four-point Likert scale, ranging from 1 (never) to 4 (often). Item 1, 2, 3, 6, and 7 were reverse-recoded so higher scores indicate higher levels of SACA (see Table 1 for listing of items).
In order to test the construct validity of the measure, the following scales were correlated with the SACA.

Social inclusion
Social inclusion scale assessed the extent to which the respondent had social contact with others. This instrument was a scale in Lehman’s Quality of Life Interview (Lehman, 1988) which was developed for this population and has good reported psychometrics. The social inclusion measure is a 6-item scale asking about the extent to which the person had contact with others using a five-point Likert scale ranging from 1 (not at all) to 5 (at least once a day), with a higher score indicating a greater degree of social inclusion. The internal consistency of the social inclusion scale was .68 in the present study.

Loneliness
Loneliness was measured by the UCLA Loneliness scale which was developed to assess subjective feelings of loneliness or social isolation (Russel, Peplau, & Cutrona, 1980). This 20-item measure has reported high internal consistency and good evidence of construct, concurrent, and discriminant validity (Hagerty et al., 1996; Russel et al., 1980). Items were assessed on a four-point Likert scale ranging from 1 (never) to 4 (always), with a higher score indicating a greater degree of loneliness. The internal consistency of the Loneliness scale was .86 in the current study.

Stigma
The Consumer Experience of Stigma Questionnaire (CESQ) was used to measure stigma and seems to have demonstrated validity (Dickerson, Sommerville, Origoni, Ringel, & Parente, 2002). This scale was comprised of two subscales, a 9-item stigma scale and a 12-item discrimination scale. Items were scored from 1 (Never) to 5 (Very Often), with a higher score indicating a higher level of stigma. The internal consistency of the stigma scale was .72. The discrimination scale had a low reliability (.45) in the present study and was therefore not included in further analysis.

Data analyses
Data analyses were designed to examine internal consistency, content validity, and construct validity of SACA. First, in order to examine the extent to which the items were internally consistent with the SACA scale, an item analysis was conducted. Second, to examine content validity, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were employed. As for EFA, principal component estimation was used for extracting factors and varimax with Kaiser Normalization technique was conducted as a rotation method. To verify the SACA factor structure, CFA was conducted using LISREL 8.80. Because the variables of SACA were measured using an ordinal scale, robust weighted least squares method was used, which is also more suitable for small sample sizes (Flora & Curran, 2004). Guided by Hu and Bentler (1999), the following model fit statistics were employed: Chi Square, root mean squared error of approximation (RMSEA; <.06, acceptable), non-normed fit index (NNFI; >.95, acceptable), and comparative fit index (CFI; >.95, acceptable). In order to examine the construct validity of the scale, correlation coefficients with measures theoretically expected to be related to SACA (i.e., social inclusion, loneliness, and stigma) were examined.

Results
Item Characteristics
Descriptive statistics of SACA are presented in Table 1. Means of each of the items ranged from 2.13 to 3.07. The mean of the total scale items was 21 with standard deviation of 4.73. For all items, the skewness was between –1 to +1, which indicated a normal distribution of the scores. Even though some of kurtosis values were slightly greater than –1, the value demonstrated a close to normal distribution for all items.

Table 1. Item characteristics (N=55)

Item  Number	Item	Mean (SD)	Min-Max	Skewness	Kurtosis
1	I don’t have many friends to do activities with.	2.40 (0.91)	1-4	.46	-.57
2	I don’t feel comfortable attending functions when I don’t know anyone.	2.13 (1.09)	1-4	.54	-1.0
3	I am very anxious going to new places alone.	2.22 (1.11)	1-4	.50	-1.04
4	When attending activities in the community,  I feel accepted.	2.93 (0.96)	1-4	-.50	-.69
5	When attending activities in the community,  I feel like I belong.	2.96 (0.96)	1-4	-.57	-.61
6	When attending activities in the community others make me feel unwelcome.	2.76 (0.94)	1-4	-.05	-1.06
7	When attending activities in the community,  I feel like I have nothing in common with anyone else present.	2.53 (0.96)	1-4	.18	-.92
8	When attending activities in the community, I think that I am no different from others in attendance.	3.07 (0.79)	1-4	-.60	.09
Total		21.00 (4.73)	11-32

Note. A higher score indicates a higher level of Sense of Acceptance in Community Activities (SACA).

Internal consistency
The item-scale correlations were examined using Cronbach’s alpha. The overall coefficient alpha for the SACA with eight-items was .76 and the values ranged from .24 to .64. As a result of the item analysis, one item (number = 8), which had a low item remainder coefficient (r = .24), was deleted and this resulted in a minimal increase of the overall correlation coefficient alpha for the scale from .76 to .77. Table 2 shows the internal consistency of SACA.

Table 2. Internal consistency among items (N=55)

Item	1	2	3	4	5	6	7	8
1
2	0.32*
3	0.33*	0.24
4	0.35**	0.38**	0.23
5	0.25	0.27	0.27*	0.62***
6	0.33*	0.46***	0.14	0.45***	0.22
7	0.11	0.45***	0.35**	0.38**	0.28*	0.30*
8	-0.14	0.18	0.19	0.25	0.13	0.22	0.27*

Note. ^* p < .05. ^** p < .01. ^*** p < .001.

Content validity
Exploratory factor analysis (EFA)
Table 3 presents the results of EFA. Based on the criterion of an eigenvalue greater than one (Kaiser, 1960), two factors were identified in the first EFA model. In terms of factor loadings, seven items had loadings from .53 to .78 with factor 1, and one item (#8) had a loading of .83 with factor 2. The first factor had a higher eigenvalue of 3.0, explaining 38.1 % of total variance where the second factor had an eigenvalue of 1.2, explaining 14.9 % of the total variance. However, even though the eigenvalue of second factor is greater than one, the examination of a scree plot suggests a one-factor solution because there is only one large “break” between component 1 and component 2 (Figure 1) (Hatcher & Stepanski, 1994).

Table 3. Results of the exploratory factor analysis (N=55)

Item number	Factor loading
	Model 1			Model 2
	Factor 1	Factor 2		Factor 1
1	.646	-.576		.573
2	.682	.144		.700
3	.527	.053		.523
4	.776	.113		.782
5	.653	.012		.653
6	.643	.117		.650
7	.576	.442		.635
8	.210	.829		N / A

Note. Item 8 was excluded in the model 2.




In order to verify the one-factor structure model, the second EFA model was conducted with seven-items. The results of the second EFA indicate all seven items loaded on a single factor, explaining 42.2 % of total variance. In addition, all seven items are moderately or highly correlated with the factor (r = .52 - .78). Therefore, based on the results from the item analysis and EFA, item 8 was excluded in the final model.

Confirmatory factor analysis (CFA)
To verify the one-factor structure with the seven-items derived from the EFA, CFA was conducted. Given the results from the EFA, it was assumed that seven items of SACA would load on a single factor. The results of CFA indicate that a one-factor structure model is acceptable based on a number of goodness of model fit statistics: X² = 15.62 (p = .34), RMSEA = .046, NNFI = .99, and CFI = .99.

Construct validity
In order to examine the construct validity of SACA, several measures that were hypothesized to be correlated with the SACA were employed. Based on the previous results of the item analysis and factor analyses, only seven-items of SACA were used to examine correlation coefficients. The results indicate that social inclusion is positively and moderately associated with SACA (r = .52, p<.0001), while loneliness is negatively and moderately related to SACA (r = -.51, p<.0001). Stigma has negative moderate correlation with SACA (r = -.35, p<.05).

Discussion
The present study was designed to develop a measure of a Sense of Acceptance in Community Activities (SACA). A feeling of acceptance when attending functions in the community was considered as the first step toward achieving a sense of belonging for people with severe psychiatric disorders, which is a major component of psychological integration. Consequently, the development of such a measure as the SACA was deemed as essential. Based on the results obtained, the SACA appears to be a promising measure with strong psychometric properties, having good reliability, content validity, and construct validity.

One item from the original measure was deleted. This item varied from other items in that it dealt with what the respondent thought rather than the respondents’ feelings. This finding may indicate a direction for the extension of the measure with additional items that deal with the respondents’ thoughts about community acceptance.
The stigma measure did not correlate as highly as social inclusion and loneliness with the SACA scale. This finding may be due to some of the items on the stigma measure tapped a broader context than the other measures (i.e., Have you seen or read things in the mass media about persons receiving psychiatric treatment and their psychiatric disorders which you found hurtful or offensive).

Other strengths of the SACA measure are its brevity, feasibility, and more importantly the fact that the items can be responded to with relative ease by the intended population for whom it was designed, adults with severe psychiatric disorders. Interviewers for the current study reported ease of administration of the measure and found that respondents had no problems responding to the items.

Given the brevity and ease of administration of the SACA, it can serve as an evaluation tool to assess outcomes for interventions designed to increase community integration of persons with severe psychiatric disorders. However, we do recognize that the scale was measured on a limited sample size who may not be representative of those served in community mental health programs. Therefore, there is a need for further research on this measure. Also, the measure in the current study was used in an interview format. It may be useful if this could be employed as a self administered measure. This too would require additional evaluation. It was designed for a low level of comprehension, due to some possible cognitive impairment of the population. But given the potential for a low literacy level of those with severe mental illness, the SACA scale may require further refinement to ensure that the reading level of the measure is consistent with the literacy level of the population.

This measure serves to begin to fill the gap in existing measures to assess community acceptance of persons with severe psychiatric disorders. Given the importance of this goal for psychiatric rehabilitation, more work is needed to develop appropriate measures for evaluating outcomes of programs targeted to increasing community participation. This is a small, but important, step toward meeting this need.

This measure may be a useful assessment tool for mental health providers working with this population. Since a major goal of mental health services and policies is to achieve greater community integration for this population, such a measure will offer direction for assisting clients who are having difficulty in feeling unaccepted when attending community activities. With this knowledge, mental health providers can work with their clients to overcome these barriers, rather than being discouraged from participation. With greater community participation clients will likely increase their social networks, and therefore, have more support resources available in times of crisis rather than totally relying on professional resources.