Vol. 7, No. 2: Winter 2002
Introduction to This Issue
This issue of The Behavioral Measurement Letter has three featured articles: one describes a process through which measurement instruments may be improved in various ways and outlines a model for revising instruments; another discusses self-estimated intelligence, gender differences in overestimations and underestimations of intelligence, and possible explanations for and consequences of these differences; the third piece presents ten very practical guidelines — a list of “do’s and don’ts” –for selecting measurement instruments that will best meet one’s needs.
In “Refinements to the Lubben Social Network Scale: The LSNS-R,” James Lubben, Melanie Gironda, and Alex Lee describe in detail the process of successive changes and analyses thereof through which they modified the Lubben Social Network Scale to improve it in various ways. The process involved performing principal component analysis to determine which items in the instrument contribute to underlying factors being measured by the instrument, and thus which items best measure what they want the instrument to measure. This analytical process resulted in a “cleaner and meaner” version of the LSNS, the LSNS-R. In addition, by detailing the process of successive modifications and mathematical analyses that produced the LSNS-R, the authors outline a model for instrument revision that is broadly applicable.
In a piece titled “Self-Estimated Intelligence,” Adrian Furnham from London University reviews literature pertaining to estimation of intelligence by oneself and others, and then compares, contrasts, and offers explanations for the reported findings. The literature he reviewed shows that there is a gender difference in intelligence estimation — females tend to underestimate their IQ and that of other females, while males, on the other hand, tend to overestimate their IQ and that of other males, and that this gender difference exists across cultures and nationalities, across socio-economic classes, and across age groups. He then explores possible explanations for the gender difference and suggests possible life consequences of over- and under-estimating one’s abilities.
Our regular contributor, Fred Bryant, presents a set of guidelines for choosing measurement tools, “Ten Commandments for Selecting Self- Report Instruments.” Some of the guidelines may seem to be common-sense. Many readers will be familiar with their basic content. Those experienced in instrument selection may see mistakes they’ve made in the past (and vowed never to make again). In any case, all of our readers, from neophytes to seasoned instrument users, will find value in the piece — if not something they hadn’t known previously, then, certainly, practical reminders of the numerous subtleties, cautions, and pitfalls that should be attended to while selecting a measurement instrument.
Address comments and suggestions to The Editor, The Behavioral Measurement Letter, Behavioral Measurement Database Services, PO Box 110287, Pittsburgh, PA 15232-0787. If warranted and as space permits, your communication may appear as a letter to the editor. Whether published or not, your feedback will be attended to and appreciated.
We also accept short manuscripts for The BML. Submit, at any time, a brief article, opinion piece or book review on a BML-relevant topic to The Editor at the above address. Each submission will be given careful consideration for possible publication.
HaPI reading…
Al K. DeRoy, Editor
Refinements to the Lubben Social Network Scale: The LSNS-R
James Lubben, Melanie Gironda, and Alex Lee
Increased interest in social support networks during the past decade spawned the development of many new social network scales. However, psychometric properties of these instruments are generally inadequately reported and there are few reported attempts to refine them. The work presented here addresses both of these concerns in examining the psychometric properties of the Lubben Social Network Scale (Lubben, 1988) and creating a revised version, the LSNS-R. In addition, the analytic plan of procedure used in this work is a model that could be employed to evaluate and refine other social and behavioral measures.
O’Reilly (1988) lamented that most social support network assessment instruments have inadequate clarity of definition and also lack reported reliability and validity statistics. Similarly, a common criticism of many studies that examine social support networks is that they employ instruments with unknown or unreported psychometric properties (Winemiller, Mitchell, Sutliff, & Cline, 1993). The work discussed below addresses these concerns by examining psychometric properties of the Lubben Social Network Scale (Lubben, 1988) and putting forth a refinement of the LSNS, the LSNS-R (i.e., the “revised” version of the LSNS). Given growing consensus on the importance of social support networks to health and well-being, developing and using consistent tools to conduct assessments of such networks are becoming ever more crucial to gerontological research and geriatric practice (House, Landis, & Umberson, 1988; Steiner, Raube, Stuck, Aronow, Draper, Rubenstein, & Beck, 1996; Glass, Mendes, de Leon, Seeman, & Berkman, 1997).
Lubben Social Network Scale
The Lubben Social Network Scale (LSNS; Lubben, 1988) has been used in a wide array of studies, and in both research and practice settings, since it was first reported more than a decade ago (Lubben; Lubben, Weiler, & Chi, 1989; Siegel, 1990; Mor-Barak & Miller, 1991; Mor-Barak, Miller, & Syme, 1991; Potts, Hurwicz, Goldstein, & Berkanovic, 1992; Hurwicz & Berkanovic, 1993; Rubenstein, L. et al., 1994; Rubenstein, R., Lubben, & Mintzer, 1994; Dorfman, Walters, Burke, Hardin, & Karanik, 1995; Luggen & Rini, 1995; Lubben & Gironda, 1997; Okwumabua, Baker, Wong, & Pilgrim, 1997; Mor-Barak, 1997; Gironda, Lubben, & Atchison, 1998; Chou & Chi, 1999; Martire, Schulz, Mittelmark, & Newsom, 1999; Mistry, Rosansky, McGuire, McDermott, & Jarvik, 2001). Further, the LSNS has been employed in a variety of ways, including use as a control variable as well as an outcome variable in health and social science studies. It also has been used as a screening tool for health risk appraisals and as a “gold” standard by which to evaluate other social network assessment instruments.
The LSNS was developed as an adaptation of the Berkman-Syme Social Network Index (BSSNI; Berkman & Syme, 1979). Whereas the LSNS was developed specifically for use among elderly populations, the BSSNI was initially developed for a study of an adult population that purposefully excluded older persons. The LSNS is based on items borrowed from questionnaires used in the original epidemiological study for which the BSSNI was constructed. However, the LSNS excluded BSSNI items dealing with secondary social relationships (viz., group and church membership) because these organizational participation items showed limited variance when used with older populations, especially those having large numbers of frail elderly persons (Lubben, 1988). In contrast, the LSNS elaborated on an array of items dealing with the nature of relationships with family and friends, in view of the growing body of empirical data suggesting that the structure and functions of kinship and friendship networks are particularly salient to the health and well-being of older persons.
The LSNS total scale score is computed by summing ten equally weighted items that quantify structural and functional aspects of primary social relationships. Scores for each LSNS item range from zero to five, with lower scores indicating smaller networks. The scale has been found to have relatively good internal consistency among a widely diverse set of study populations (α = 0.70). Factor analyses on the LSNS suggest that it measures three different types of social networks: family networks, friendship networks, and interdependent relationships (Lubben, 1988; Lubben & Gironda, 1997).
Methodology
The main purpose of the work presented here is to address deficits that became apparent in the original LSNS as it was used with diverse populations over the past decade. But because the LSNS was found to have relatively stable reliability and validity across this wide array of settings, any proposed modifications were not to jeopardize the relatively strong psychometric properties of the original LSNS.
Reliability is a fundamental issue in psychological measurement (Nunnally, 1978). One important type of measurement reliability is internal consistency, i.e., the extent to which items within a scale relate to the latent variable being measured (DeVellis, 1991; Streiner & Norman, 1995). Cronbach’s (1951) coefficient alpha was chosen to examine the internal consistency of the LSNS and modifications designed to improve upon the original version. The acceptable range of coefficient alpha values employed here was 0.70 to 0.90 (Nunnally; De Vellis) because assessment instruments with reliability scores higher than 0.90 are likely to suffer from excessive redundancy, whereas those with alpha less than 0.70 are likely to be unreliable (Streiner & Norman). A further test of item homogeneity used was the item-total test score correlation (De Vellis; Streiner & Norman). Here acceptable values of the item- total score correlation were 0.20 and greater (Streiner & Norman).
Principal component analysis looks for underlying (latent) components that account for most of the variance of a scale (Stevens, 1992). Principal component analysis with varimax rotation was used here to explore the component structure of various versions of the LSNS to see if the modified versions conformed in actuality to the hypothesized structure. Although more sophisticated methods exist to examine factor or latent variable structures, such as maximum likelihood factor analysis and confirmatory factor analysis, many scholars contend that principal component analysis is both adequate and yet more practical than more sophisticated techniques, for principal component analysis is mathematically easier to manage, easier to interpret, and yields results similar to those from maximum likelihood factor analysis (Nunnally, 1978; Stevens). The size of the sample used in the analyses discussed below is adequate to conduct principal component analysis according to general sample size guidelines (Stevens; Guadagnoli & Velicer, 1988).
Four Objectives Used in Refining the LSNS
The work to refine the LSNS has four principal objectives. One was to distinguish between and better specify the nature of family and friendship social networks. A second was to replace, where feasible, items in the original LSNS that have small statistical variance. A third objective was to disaggregate “double-barreled” questions. The fourth was to produce a parsimonious instrument to encourage and facilitate its use in research and practice settings where time constraints or other issues preclude using longer social support network instruments.
With regard to the first two objectives above, it should be noted that social and behavioral measures are purposely designed to discriminate among groups for a certain construct, and so lack of variation within a given item limits a scale’s ability to identify and discriminate among variations (DeVellis, 1991; Streiner & Norman, 1995). Thus, eliminating items with limited statistical variance generally increases a scale’s overall sensitivity and specificity, and that in turn improves its effectiveness in measuring constructs of interest (McDowell & Newell, 1987; De Vellis).
Double-barreled items are those in which two different questions are contained in one item. Such items often confuse respondents because they are not sure as to which aspect of the double-barreled question they should respond (De Vellis, 1991; Streiner & Norman, 1995). Disaggregating double-barreled questions as per the third objective not only helps respondents in answering, it allows researchers to determine the extent to which each part of the original question helps to define a particular construct.
Plan and Procedures
Production of the LSNS-R progressed along a series of four analytical steps to address the objectives stated above. In each step, alpha reliability coefficients and the results of a series of principal component factor analyses were examined to determine whether and the extent to which items corresponded to the latent structural components of family networks and friendship networks.
The four steps are summarized in Figure 1. In the first step, reliability statistics were obtained for the original LSNS administered to the sample described above. These values then served as reference points for comparison with values obtained for subsequent modifications.
In the second analytical step, two items from the original LSNS scale – L9 (“Helps others with various tasks“) and L10 (“Living arrangements“) – were dropped because they demonstrated limited response variation among a number of sample groups including the present one. Furthermore, neither of these items helps to distinguish between family networks and friendship networks better than the other items in the original LSNS.
The L9 item was originally included in the LSNS in part because social exchange theory suggests that a reciprocal social relationship is stronger than one that is unidirectional (Jung, 1990; Burgess & Huston, 1979). Thus, rather than only capturing what others do for the older person being assessed, it is desirable to include items that also assess what the older person does for other people, i.e., items should be included to assess reciprocity of social support within kinship and friendship networks. Moreover, in past studies L9 generally demonstrated insufficient item variance and thus was a good candidate for elimination or replacement.
The L10 item on living arrangements also had not worked out well over time. When the original LSNS was constructed, both living arrangements and marital status were common items included in measures of social support networks. It therefore seemed seemed entirely appropriate to include an item merging these two related constructs. However, the L10 item has been the worst performing item on the LSNS across different settings. Part of the problem has been scoring it, which is constrained by the limited number of response options available as well as by disagreements among scorers in assigning ordinal weights to specific response options. Perhaps most important, marital status and living arrangements are generally not malleable nor appropriate for intervention, so items concerning them should not be included in any case.
In the third analytical step, two “double- barreled” questions were disaggregated. The two items, L3 (relatives) and L4 (friends) ask, “How many (relatives) (friends) do you feel close to? That is, how many of them do you feel at ease with, can talk to about private matters, or call on for help?” In this step, L3 and L4 were each recast as two distinct questions. One asks, “How many (family members) (friends) do you feel at ease with such that you can talk to them about private matters?” whereas the other asks, “How many (family members) (friends) do you feel close to such that you can call on them for help?” The first of these substitute questions examines somewhat intangible or expressed support, whereas the other taps into more tangible support, such as help with running an errand. Both types of support have been suggested as important aspects of social support networks (Litwak, 1985; Sauer & Coward, 1985).
In the fourth step, items that identify both the targets and sources of respondents’ confidant relationships were constructed and tested. Here the two confidant relationship items in the original LSNS (L7 and L8) were recast to distinguish between confidant relationships with family members and those with friends. These changes recognize that confidant relationships with family members may serve different functions than confidant relationships with friends (Keith, Hill, Goudy, & Power, 1984). The final result of the four-step process is a revised version of the LSNS, the LSNS-R.
Plan of Analysis
Step 1: Analyze original LSNS
Original LSNS Items:
L1 Family: Number seen or heard from per month
L2 Family: Frequency of contact with family member most in contact
L3 Family: Number feel close to, talk about private matters, call on for help
L4 Friends: Number feel close to, talk about private matters, call on for help
L5 Friends: Number seen or heard from per month
L6 Friends: Frequency of contact with friend most in contact
L7 Confidant: Has someone to talk to when have important decision to make
L8 Confidant: Others talk to respondent when they have important decision to make
L9 Helps others
L10 Living arrangements
Step 2: Eliminate items with limited variation
Items eliminated: L9 and L10
Step 3: Uncouple double-barreled questions
Items modified: L3 and L4 each split into two separate questions
L3A Family: Number feel at ease with whom you can talk about private matters
L3B Family: Number feel close to whom you can call on for help
L4A Friends: Number feel at ease with whom you can talk about private matters
L4B Friends: Number feel close to whom you can call on for help
Step 4: Distinguish between source and target confidant relationships with family and friends
Items modified: L7 and L8 each split into separate questions for family and friends
L7A Family: Respondent functions as confidant to other family members
L7B Friends: Respondent functions as confidant to friends
L8A Family: Respondent has family confidant L8B Friends: Respondent has friend who is a confidant.
Data Source
The data are from a survey of older white, non- Hispanic Americans in Los Angeles County, California done between June and November 1993. A self-weighting, multistage probability sample was selected from 861 census tracts in the area in which the white, nonHispanic population exceeded any other single racial or ethnic group. This sampling strategy insured a high level of homogeneity in the sample.
The first