You are on page 1of 35

Theta Reliability and Factor Scaling

Author(s): David J. Armor


Source: Sociological Methodology, Vol. 5 (1973 - 1974), pp. 17-50
Published by: Wiley
Stable URL: http://www.jstor.org/stable/270831 .
Accessed: 10/02/2015 01:34

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Wiley is collaborating with JSTOR to digitize, preserve and extend access to Sociological Methodology.

http://www.jstor.org

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND
FACTOR SCALING

David3. Armor
THE RAND CORPORATION

The authorwishesto ex-


presshis gratitude
toAlbertE. Beatonforpointingouttherelationship be-
tweenthetaand maximumalpha. David Heise, Donald Olivier,and Fred
Mostellerreadearlierdraftsand providedhelpfulcriticisms
and suggestions
forthefinal manuscript.

Reliabilitymeasurementof compositevariables has attracted a


considerableamount of interestamong sociologistsin the last several
years.' Althoughsociologistshave always lamentedthe problemof reli-
ability in sociologicalmeasurement,until recentlylittle attentionhas
beenpaid to techniquesofreliabilityassessmentand improvementwithin
the sociologicalmethodologyliteratureitself.While much of the recent
1 Throughoutthis chapterthe term
reliabilityrefersto internal-consis-
tancy reliability-as opposed to stabilityreliability-unless otherwisestated
(Heise, 1969).

17

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
18 DAVID J. ARMOR

attentionis expositoryand tutorialin nature,progresshas been made in


conveyingbasic principles(Upshaw, 1968; Blalock, 1969), in connecting
reliabilityto path analysis(Siegeland Hodge, 1968; Heise, 1969), and in
providingsimplified computationalprocedures(Bohrnstedt,1969).
In spite of this activity,thereare still many technicalproblems
facingthe practitionerof reliabilityassessmentand improvementfor
compositemeasures.Most proceduresforreliabilityassessmentdepend
on the applicationof Cronbach's alpha (Cronbach, 1951), and its im-
provementdependson variousstepsofitemanalysis(whichI laterdefine
as covariancescaling;Upshaw, 1968). Unfortunately thesemethodshave
severaldrawbacksforthe sociologistand perhapsforothersocial scien-
tistsas well: themathematicalassumptionsforalpha reliabilityare often
not met; the usual steps of itemanalysis-throwingout "bad" itemsto
enhancealpha reliability-maynot in factproduceoptimumalpha reli-
ability;and item analysisdoes not include clear and systematicproce-
duresfordetectingand takingintoaccountmultidimensionality-that is,
the presenceof mutuallyindependentsubelustersof items withinthe
total composite.
This chapteraims to solve some of theseproblemsby describing
an approachto reliabilitybased on principal-component factoranalysis.
This approachrestson a different and, forsociologists,a morerealistic
definitionof compositereliability.Moreoverit leads to a little-known
measureof reliabilityI call theta (to distinguishit fromalpha reliabil-
ity) that assesses optimal reliability,and it providesfor a method of
factorscaling that can take account of multidimensionality in a set of
items,therebyenhancingreliabilityand validity.
Beforepresentingthesemethodsit is necessaryto reviewcurrent
alpha reliabilitytechniquesand the associated method of covariance
scaling.In thisway it willbe possibleto illustratemoreclearlythe spe-
cificadvantagesof theta reliabilityand factorscaling.

ALPHA RELIABILITY

Given an interestin assessingcompositereliability,decidingon


an appropriatecoefficienthas been a troublesomestep formanyinvesti-
gators.Althoughthe usual definitionof reliabilityis the simple prod-
uct-momentcorrelationbetween two parallel variables that measure
"identical" things,thereis no singlecoefficientthat has been adopted
universallyforcompositereliability.Guilford(1954) liststencoefficients,
and thisis not an exhaustivelist. Some ofthestandardones he givesare
the Spearman-Brownprophecyformula;the split-halfand odd-even

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 19

methods(both of whichdepend on the Spearman-Brownformula);the


Kuder-Richardsonformula20; and Cronbach's alpha. The reason for
thismultiplicity has to do withthe different waysofestimatingtheerror
componentsof a set of items.
Due in partto theconceptualand-computationalsimplicityofthe
method,it was probablythemostcommonlyused coefficient
split-half in
sociologicalresearchbeforethe middle1950s. Since that time,however,
Cronbach'salpha has become the popularmeasureof reliability(Cron-
bach, 1951): it is a generalformulathat subsumesmost of the split-half
and Kuder-Richardson ;2 it has also provedto be a lowerbound
coefficients
to the truereliability(Novick and Lewis, 1967). This lattercharacteris-
tic means that alpha is a conservativeestimateof the reliabilityof a
composite.Basically the alpha coefficient treatseach itemin a composite
as a parallelvariable. Since moderatedeparturesfromthe model can be
expected,the reliabilityforeach itemis estimatedby the averageinter-
itemcorrelation.If we let f be theaveragecorrelationbetweenitemsand
ifwe assumethattheitemsare in standardform(or thattheyhave equal
variances),then Cronbach'salpha is simply
aO=-pr/[l + r(p - 1)] (1)

wheref = mean inter-itemcorrelation;p = numberof items. In other


wordsalpha is identicalto the Spearman-Brownprophecyformulawith
the average correlationf used to estimatethe reliabilityof each item.
Model
The usual model forthe alpha reliabilityformulagivenin Equa-
tion (1) comesfrompsychologicaltest theory(Lord and Novick, 1968).
This modelassumesthat the observedscoreforany itemin a composite
can be partitionedaccordingto "true" and "error" components.For
item i and forany subjectj this model is

xij = Ti + eij (2)


wherexij = the observedscoreforitem i, subjectj; Tj = the truescore
forsubjectj; eij = the errorscoreforitemi, subjectj. That is, all items
measurethe subject on an underlyingpropertyTj to an equal extent.It
is normallyfurtherassumedthat the errorcomponentsare uncorrelated
withtruecomponents(forthesame itemas wellas acrossdifferent items)
and witheach other;in otherwords,all itemsare parallelforms.It would
2 Cronbach(1951) provedthat alpha is the mean ofall possiblesplit-half
reliabilities.

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
20 DAVID J. ARMOR

thenbe expectedthat all population(as opposed to sample) inter-item


correlationswouldbe equal, as would be all populationitemvariances.
Given moderatedeparturesfromthese assumptionsand giventhat the
inter-itemcorrelationsand item variances may not be all equal in a
sample,Equation (1) providesan estimatedreliabilityby standardizing
itemsand by averagingoverobservedcorrelationsto obtainan estimate
of the truepopulationinter-item correlation.
However adequate this model may be forpsychologicaltests,for
manycompositemeasuresin sociologytheseassumptionsare not realis-
tic. Since the contentof each item in a compositeusually differsby a
substantialdegree,particularlyin attitudinalor behavioralcomposites
(such as politicalefficacy,anomie,social class), we mightexpectthat the
true populationcorrelationsare not all equal. For example,in a social
class indexcomposedofmeasuresofincome,education,and occupational
status,one usuallyfindsthat thecorrelations are not all equal; the corre-
lations betweenincomeand the other two variables are usually lower
than the correlationbetweeneducation and occupation-even for the
fullU.S. Census (Siegel and Hodge, 1968). A similarcase mightbe made
forany numberof attitudescales wheninter-item correlationsare con-
sistentlyunequal acrossmany different samples.
A more parsimoniousand more realisticbasis for alpha can be
givenby utilizingthe familiarformulafordecomposingthe variance of
a sumofitems.Lettingxijstandfora scoreon itemi forsubject., letting
p be the numberof items, and lettingscale scoresXj = Xlj + X2j +
* + xj, we have
=x2 = 20IZ i + 21 cov(xi,xj) (3)
i<j
whereax2 = varianceof the sum; cK2 = varianceof itemi; cov(x, xj)=
covariancebetweenitemi and itemj. Or, morecompactly,
S = I+C (4)
That is, the varianceofa scale or compositesum S is equal to the sumof
the individualitemvariancesI plus the sum of all possiblecovariances
betweendifferent items C.
For a set of itemsin a compositethat are measuringthe same or
similarproperties,one would expect the covariancesto be large com-
pared to the itemvariances;therefore we mightmeasurethe composite
reliability(or scalability)by consideringthe ratio C/S: this is the pro-
portionof scale variance due to item covariation.Unfortunately this
ratiodoes not have an upperlimitof 1, sinceI is neverzeroexceptin the
trivialcase when all item variancesare zero. The maximumvalue for

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 21

C/S is (p - l)/p, and this occurs if everyitem is perfectlycorrelated


with everyotheritem, whichmeans that all oa, are equal and cov(xi,
X,) = o2. This suggestsan indexof compositereliabilitywiththe desir-
able rangeof 0 to 1 as follows:3

Composite reliability = -C/ (5)

That is, alpha can be derivedas the proportionof scale variancedue to


itemcovariationadjusted to providean upperlimitof 1.
Althoughthis derivationprovidesthe same coefficient, the ad-
vantages of the rationale (aside fromits simplicity)are that no postu-
lates have to be made about an underlying modelof trueand errorscore
componentsand that thereis no necessityto make the parallelvariable
assumptions(such as equalityoftruescoresand equal precisionofmeas-
urement).It is this latterset of assumptionsthat may be particularly
unrealisticin many sociologicalapplications.On the otherhand, if the
parallel formassumptionsare reasonableforall items in a composite,
thenalpha as definedin Equations (1) or (5) is an appropriatemeasure
of reliabilityin the traditionalsense. In eithercase alpha offersa single
criterionforestimatingthe internalconsistencyand coherencyof a set
of itemsmakingup a scale. The greaterthe covariation,given a fixed
numberof items,the greaterthe value foralpha. A vahle of zero means
that the average covariationis zero; a value of 1 means that all items
have intercorrelations equal to 1.
Computational Forms
The formulasforalpha given in Equations (1) and (5) are not
particularlyusefulforcomputation.It mightbe helpfulto summarize
the major computationalformsand pointout some practicaldifferences
betweenthem.It mightcome as a surpriseto some analyststhat not all
formsproduce the same answers,and, given various conditions,that
some formsare moreappropriatethan others.
In the followingformulasp is the numberof items; rij is the
product-moment correlationbetweenitemsx. and xj; a-2 is the variance
of item i; X is the sum of the itemscoreswithan associatedscale vari-
ance ax; and X is the arithmeticmean of the itemscoreswithan associ-
2
ated variance ax. Depending on how the compositescale scores are

3 This formulaassumesthat all itemsare scoredin the same directionor,


if not, that appropriatereversalsare made to eliminatenegativecovariances.
Withreal data it is always possibleto make reversalsso thatC remainspositive
(and so that alpha is always greaterthan or equal to zero).

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
22 DAVID J. ARMOR

formed,one of the followingformulaswillbe appropriate.If the scale is


formedby summingstandardizeditemscores,

~(6) p + 2'5~~~"
rij
i<j )
If the scale is formedby summingraw itemscores,

a = ( (7)
_ -
U2)
If the scale is formedby averagingraw itemscores,

a = ( P T ) (8)

The choicebetweentheseformulasis not just a matterof compu-


tationalconvenience;at least threeconsiderations apply. First,if one is
makingthetraditionalparallel-form assumptions,thenall itemvariances
shouldbe equal; if theyare not (as is usual withmost real data), then
differences wouldordinarily be seen as nonsubstantiveand shouldnot be
allowedto affectthe reliabilityestimate.In this case standardizationof
items removesvariance differences and Equation (6) can be used. Of
courseEquation (7) can also be used ifthe scale variancehappensto be
known.But Equation (6) is moreusefulbecause the scale variancedoes
not have to be known;it dependsonlyon theinter-item correlations and
the numberof items.
Equations (7) or (8) should be used wheneverraw item scores
are used to producethe scale scores.Since theyemploythe actual item
variances(whichare normallynot all equal), theseformulasdo not pro-
duce the same estimateas Equation (6). In effectthe alpha coefficient is
weightedaccordingto itemvariances.Whetherthe reliabilityaccording
to Equations (7) and (8) is higheror lowerthan that givenby Equation
(6) dependson complexrelationships betweentheitemvariancesand the
inter-item Basicallyscales formedfromrawitemscoreshave
correlations.
higherreliabilityif the itemswithhighervarianceshave highercorrela-
tions; if the oppositeholds,then Equation (6) producesa morereliable
scale. This relationshipshould become cleareraftertheta reliabilityis
derivedin the nextsection.
A thirdconsiderationarises when some items have missingob-
servationsdue to nonresponses.If scales are formedby a straightsum-
mation methodwith no special treatmentof nonresponses(that is, if
nonresponsesare in effecttreatedas zeros), subjectswithmany nonre-
sponseswill have spuriouslylow scale scores;the scale variancea2 will
be spuriouslylarge; and the resultingalpha will be spuriouslyhigh.In

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 23

thiscase it is betterto formscale scoresby averagingitemscores(based


on responsespresent)and to apply Equation (8). Equation (6) is also
appropriatein thiscase, assumingthat the rijare computedwithoutthe
nonresponsesforeach pair of items.
Covariance Scaling
Given that real data may departfromthe parallel-itemassump-
tions,techniquesare necessaryto decide on the adequacy of individual
items. It is possiblethat one or moreitemsmay not be measuringthe
propertyin commonto the rest of the items; if so, theirelimination
mightenhancereliabilityas well as the conceptualclarityof the com-
posite. Under the parallel-itemassumptions,the greaterthe numberof
itemsthe higherthe reliability(this relationshipis given by the Spear-
man-Brownprophecyformula).But in the absence oftheseassumptions
such is not necessarilythe case. Specificallyitemsthat do not correlate
sufficientlywithotheritemsin a composite(or correlatein a negative
directionafterappropriatereversals)may actuallyreducethe reliability
of a compositeand shouldtherefore be excluded.4
The techniqueof maximizingthe alpha reliabilityof a composite
undervarious constraintswill be called covariancescaling.Otherterms
used to describethis processincludeitemanalysis (Upshaw, 1968), test
construction (Cronbach,1960), summatedratings(Edwards, 1957), and
Likertscaling after the psychologistwho inventedthe agree-disagree
itemscoringscheme.Sociologistsalso oftenuse the termindexconstruc-
tionto denote a similarprocedure.The termcovariancescalingcan be
applied to all thesemethodssince theyare all based on the same basic
and self-evident assumption:if a set of itemsis measuringthe same or
similarpropertiesand the propertycomprisesa singlecontinuumor di-
mension,the itemsshouldall covaryto some extent.For a fixednumber
of items,the greaterand moreconsistentthe inter-item correlationsthe
morereliablethe composite.
There is no single,uniformprocedurethat all researchersfollow
when constructing a covariancescale. Nonethelessseveralsteps can be
identifiedin the literature;an investigatorwould normallyuse at least
one of them:

4 In factit can be shownthat,givenan ap basedon p items, theap-l


basedontheremaining itemsafteritemh hasbeenexcludedwillbe greater
than
ap if
_p
2rih < pap/[p - a, (p - 1)] [p - 1 - ap (p - 2)]
i3 h

whererihis thecorrelation
betweenitemi and itemh.

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
24 DAVID J. ARMOR

1. Inspectionof itemface contentto ensureclear and consistent


meanings
2. Calculation and inspectionof item-to-scalecorrelationsto
pinpointitems that are not contributingto the scale as a
whole5
3. Calculationof a T-test foreach itembetweenthe highest25
percentand lowest 25 percentof the compositescale scores
(usuallya substituteforstep 2)6
4. Examination of all inter-itemcorrelationsfor patterns of
loweror negativecorrelations
5. Recalculationof alpha reliabilityaftereliminatingitems ac-
cordingto information obtainedin steps 1 to 4
It is unlikelythat any but the most carefulresearchersgo throughall
thesesteps.Step 4 is probablythe one mostlikelyto be leftout unlessa
computeris beingused fortheanalysis.In factthisstepis notmentioned
in several standarddiscussionsof the methodof summedratings(Ed-
wards,1957; Upshaw, 1968). As willbecomeclear,however,step 4 con-
tains all the informationneeded to decide on scalability,and it may be
the mostimportantstep of all.
The pointin describingthesefamiliartechniquesis not so much
to instructthe reader in covariancescaling; ratherI want to be clear
about the meaningof my claim that the traditionalcovariancestepsre-
viewed here do not necessarilyresultin optimalscales fromthe stand-
point of compositereliability.Althoughthe applicationof covariance
scalingmay enhancereliability,the success of the applicationmay de-
pend moreon subjectivejudgmentsthan on analyticcriteria.Although
subjectivejudgmentscan never be eliminatedfromscaling techniques
(norshouldtheybe), the methodsof thetareliabilityand factorscaling
offera substantiallymoreanalyticapproachthat can help to reducethe
idiosyncraticvariationsof individualinvestigators.
Limitations of Alpha Reliability and Covariance Scaling
It has been pointedout that the methodof alpha reliabilityde-
pendson the assumptionthat all itemsin a compositeare parallelitems,
whichfurtherimpliesthat all itemsmeasure a singleunderlyingscale
propertyequally. Thereforethereare two major conditionsunderwhich
a set ofrealdata clearlyviolatestheassumptions:theitemsmay measure
a singlepropertybut do so unequally; the itemsmay measuretwo or

'7The item-to-scale correlationsshouldbe correctedfor


(or item-to-total)
itemcontributionbut frequentlytheyare not.
6 The proceduresforthisstep are describedin Edwards (1957).

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 25

moreindependentpropertieseitherequally or unequally.Althoughthe
techniquesof covariancescalingmay helpsome aspectsof the firstprob-
lem theymay not do so optimally.As faras the second problemis con-
cerned,covariancescalingoffers littleor no solutionat all. The littlehelp
it may offerdependsverymuchon thenumberofitemsand thepatience
of the investigator.
These two problemscan be illustratedby simple hypothetical
examples.Considera four-item compositewhoseintercorrelations are as
shownin Table 1. The firstthreeitemshave intercorrelations that are
quite highand consistent,rangingfrom0.4 to 0.6. But item4 has much
lowercorrelationswith the otherthreeitems (all being 0.1). Assuming
that the correlationsare based on a sufficiently large sample,it might
turnout thatthe0.1 correlationsare significantly different fromzeroand
an investigatormay thereforedecide to keep item 4 in the composite.
But ifitem4 is leftin the compositewithoutsomekindofweighting, the
compositereliabilitywillbe lowerthanifthe itemsare weightedaccord-
ing to theirdifferential The four-item
contributions. (unweighted)alpha
reliabilityis 0.63; the compositereliabilitywithweightingis 0.68 (apply-
ing the theta techniquedefinedin the nextsection).
The second problemis generallyharderto detectwithreal data,
but the simplehypotheticalexamplein Table 2 offersa cleardemonstra-
tion. Basically the existenceof two or moreindependentpropertiesin a
compositeis usually revealed wheneverthereare two or more sets of
itemswithrelativelyhighwithin-setcorrelationsand relativelylow be-

TABLE 1
Inter-Item Correlations for Hypothetical Four-Item Composite
Item 1 2 3 4
1 1.0
2 0.6 1.0
3 0.5 0.4 1.0
4 0.1 0.1 0.1 1.0

TABLE 2
Inter-Item Correlations Revealing Two Dimensions
Item 1 2 3 4 5 6
1 1.0
2 0.5 1.0
3 0.5 0.5 1.0
4 0.0 0.0 0.0 1.0
5 0.0 0.0 0.0 0.5 1.0
6 0.0 0.0 0.0 0.5 0.5 1.0

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
26 DAVID J. ARMOR

tween-setcorrelations. The degreeofindependenceamongtheproperties


increasesas the between-setcorrelationsapproachzero.
Table 2 illustratesa six-itemcompositewhose intercorrelations
revealtwo independentdimensions,one represented by items1 to 3 and
anotherby items4 to 6. Each subset has within-setcorrelationsof 0.5;
all the between-setcorrelationsare zero. If all six items are combined
into a single scale, the alpha reliabilitywill be 0.60 (r = 0.2); but if
items1 to 3 and items4 to 6 are combinedinto two separatethree-item
scales, theywould each have the substantiallyhigheralpha reliabilities
of 0.75. In factthe six-itemscale reliabilitywouldnot surpassthe relia-
bilitiesof the two three-item scales untiltheaverageofall thebetween-
set correlationssurpasses0.2 (holdingthe within-setcorrelationscon-
stantat 0.5). Quite aside fromthe questionofreliability, ofcourse,is the
factthatdecomposingthea priorisix-itemcompositeintotwoindepend-
ent subscales providesnew informationthat may have an important
bearingon conceptualand theoreticalissues.
Althoughmost investigatorscould discoverthe independentdi-
mensionsin simpleexampleswhosecorrelationalpatternsare as clear as
those in Table 2, the situationwith real data is generallymuch more
complex.As thenumberofitemsincreases(say, overten),as thenumber
of dimensionsincreases,and as items contributedifferentially to each
dimension,nonebut themostpatientand diligentanalystwouldbe able
to produceoptimumscalingwiththe usual alpha reliabilityand covari-
ance scalingtechniques.

THETA RELIABILITY

Withthe adventof computersolutionsformultivariatemethods,


manyresearchershave turnedto factoranalysisforassistancein identi-
fyingthe numberof propertiesor dimensionsin a set of data and in re-
latingspecificitemsto each dimension.With a fewnotable exceptions,
however,littleattempthas been made to relatefactoranalysisformally
to reliabilityand scalingmethods,and almostnoneofthisworkis widely
knownand applied in sociology(Lord, 1958; Bentler,1968; Heise and
Bohrnstedt,1971). Even factoranalysisis not as well understoodand
applied as it could be. When factoranalysisis applied to scalingtasks,
the applicationdependsmoreon the intuitionand experienceof the in-
vestigatorthan on analytictechniques;it remainsmoreof an art than a
technology.
Althoughthe interpretiveaspects of factoranalysis can never be
eliminated,thereare a numberof proceduresthat can aid the establish-
mentofoptimallyreliablescales. The keyto theseproceduresis a formal

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 27

connectionbetweenreliabilityand scaling providedby principal-com-


ponentfactoranalysis.7Otherfactor-analytic methodshave been devel-
oped, but principal-component analysisoffersthe most straightforward
and preciseconnectionbetweenreliabilityand scaling.The resultsof a
componentanalysisenable one to computean optimalreliabilitycoeffi-
cientI call thetaand to deriveoptimalscales througha seriesof steps I
call factorscaling.
The basic hypothesisof componentanalysisis that,givena set of
p iteins,the scoreof a subjecton each itemcan be decomposedinto any
numberof componentsor factors(up to the total numberof items).
Generallyonlya small numberof the largestfactorsin termsof the pro-
portionof the total itemvariationtheyrepresentare givensubstantive
meaning;the remainderare groupedtogetherand consideredas an error
componentor factor.Each nonerrorfactor,say m ofthem(withm < p),
is a hypotheticalvariablegivensubstantivemeaningby individualitems
accordingto factorloadings-a set of weightsdeterminedby the contri-
butionofeach itemto a givenfactor.This meansthateach itemcan con-
tributedifferentially to a givenfactor(or not at all if the factorloading
is zero). Moreover each factorrepresentsa statisticallyindependent
sourceofvariationamongtheset ofitems,and mostcomponent-analysis
solutionsextractthese factorsin an ordercorresponding to the size of
theircontribution; thatis, thefirstfactoraccountsforthemostvariance,
the second factorforthe second largestamountof variance,and so on.
From the standpointof scaling,principal-component analysis offersa
means of detectingthe most importantindependentdimensionsamong
a set of items (if thereis morethan one) and, foreach of these dimen-
sions,providesfordifferential contributions by individualitems.
A principal-component analysiscan be used to constructa set of
factorscores,one set foreach factor.A factorscoreis simplythe scoreof
a subjecton a givenfactor:it is in effecta compositescale scorebased on
a weightedsum of the individualitemsusingthe factorloadingsas the
weights.The task of the thetacoefficient is to providea compositerelia-
bilityestimateforfactorscoresbased on principal-component analysis.
In componentanalysisthe amountof varianceaccountedforby
a factoris called a rootand is denotedby the symbolXk (forthe kthfac-
tor).8 Althoughthese quantitiesare providedby most computersolu-
tions forcomponentanalysis,they actually have a simple relationship
7 By principal-component analysis is meanta methodof factoranalysis
wherebyunities(is) are leftin the diagonalof a correlationmatrixwhenfactor-
ing. See Harman (1967).
8 The fulltechnicaltermis latent
root.Othercommontermsare eigenvalue
and characteristicroot.

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
28 DAVID J. ARMOR

to the factorloadings:the kthrootis simplythe sum ofthe squared fac-


torloadingsforthe kthfactor.These are the quantitiesthat providethe
basis fortheta reliability.
Beforegivingthe formulaand the derivationforthetareliability,
it must be pointedout that thereare two generalcases in component
analysisthat need to be distinguished.First,in a single-factor solution
the firstfactorand its root Xi sufficefora completespecification of the
scale and its reliability.Second, if a multiple-factor
solutionis adopted
meaningfulinterpretation generallyrequiresa rotationof the m factors
that are retained.Rotationsare dealt within moredetail later; suffice it
to say herethatafterrotationthevarianceofa factor-whichstillequals
the sum of the squared rotatedloadings-no longerequals the mathe-
matical rootsderivedin the originalprincipal-component solution.Ac-
cordinglythesymbolXkis used to denotethevarianceofthe kthrotated
factor.
Given a set of p itemsand a single-factor solutionwithroot Xi,
the reliabilityof the compositescoresbased on thisfactoris givenby
0 = [p/(p - )1- (1/XI)] (9)

whereXi is the firstroot of a principal-component solution.Although


this formulais not new, it is littleknownin sociology(Bentler,1968).
It is mathematicallyequivalentto alpha fora compositescale formed
by weightingitemsaccordingto theirprincipal-component factorload-
ings; this has been shownby Lord (1958) to be the maximumpossible
alpha. By takingintoaccountdifferential itemcontributions to the cen-
in
tral property commonto a set of items,one obtains the maximum
compositereliabilityaccordingto the definition in
given Equation (5).
The situationfora multiple-factor solutionwithrotatedfactors
is somewhatmorecomplicated.LettingOhk stand forthe squared corre-
lation betweenthe originalunrotatedscores forfactorh and the new
factor k, and given a rotated m-factorsolution with original roots
Xl, X2, , Xm,the reliabilityof the kth set of rotatedfactorscoresis
givenby

k= [p/(p- 1)](1 - E hk/Xh) (10)

This formulaholds only fororthogonalrotations(such as the varimax


solution).The quantity khk is actually the elementin the hth row and
kth columnof the transformation matrixthat maps the originalfactor
loadingsinto the rotatedloadings.
Althoughtheta-staris mathematicallysimilarto theta,it will be
clearlaterthatit does not have quite the same utility.Since the number

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 29

of factorsm is by no means a determinatequantity,we cannotsay that


theta-staroffersmaximumreliabilityforeach subset of itemsin a com-
posite.AMoreover it turnsout that in spiteof the weighting, ifmorethan
one factoris presentoptimal reliabilityis seldom approached if all p
itemsare used to derivethe weightedscale scores.Nonethelesstheta-star
is the properformulaforreliabilitywhenone is usingthe completeset of
rotatedfactorscoresfroma principal-component analysis.
I now give more precisemeaningto the precedingideas, and I
providea derivationfortheta. Given an observedstandardizedscorexij
forsubjectj on item i, the basic model forcomponentanalysisis9
xij= ailflj + ai2f2j + * + aimfmj + eij ()
where aik is a factor loading of item i on factor k; fkj is a factor score for
subjectj on factork; eij is an errorcomponentdue to randomand other
specificinfluencesofitemi to the observedscore.If m = p or ifas many
factorsare extractedas thereare items,then eij = 0 forall i, j and the
originalscoresare decomposedperfectlywithouterror.This case is gen-
erallyuninteresting, however,sinceone ends up withas manyfactorsas
there are items. In most cases an investigatorinterpretsonly a small
numberoffactorsm; in thiseventtheremainingp-mfactorsare assumed
to representan errorcomponent.The problemof choosingthe cutofffor
m is taken up in a later section.
It is not necessaryto go into the mathematicsof exactly how
these quantitiesare obtained; mostresearchersrelyon widelyavailable
computerprogramsforderivingthem.10For the presentit suffices to say
that a standardprincipal-component analysisresultsin a set of factor
loadingsaik that measurethe magnitudeof the contributionof itemi to
factork on a scale rangingfrom-1.0 to + 1.0; a set ofrootsXk such that
Xk = 5 aik; and a set of factorscoresfkj that providethe score of sub-

ject j on factork. The factorloadingsand rootsare usuallydisplayedin


9 The assumptionofstandardizedscoresdoes iiotdestroy
thegenerality
of thetechnique.Thereis usuallyno a priorireasonwhyitemsthatmeasure
similarpropertiesshouldbe allowedto have differenitial
weightsby virtueof
different
arbitrarily variances;indeedthe wholepointof factoranalysisis to
base weightson contributionsto underlyingfactorswithoutregardto initial
variances.
10It can be notedin passingthat thefactorloadingsare simlply theset of
latent vectors (eigenvectors,characteristicvectors) which satisfythe charac-
teristicequation (R - XI) v = 0, whereR is the correlationmatrixwithunities
in the diagonal,X is a constant,I is the identitymatrix,and v is a latentvector.
The factorloadingsin Equation(11) are normalizedso that akak = Xk, whereak
is the vectorof factorloadingsforfactork; the lateintvectorsv are normallyas-
sumed to be normalizedto 1 (v'v = 1).

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
30 DAVID J. ARMOR

TABLE 3
Factor Loadings and Roots
Item Factor 1 Factor 2 ... Factor in
1 all al2 *-- a,m
2 a2l a22 * a2m

p apl ap2 ap--_

a k =1 X2 X,n

matrixformas shownin Table 3. The factorloadingsare necessarynot


onlyforconstructing thefactorscoresbut also forinterpreting thefactor
in termsof the contentof the originalitem. The higherthe absolute
value ofa loading,thegreaterthe contribution ofthatitemto thefactor
in question.11
A root Xk can be interpretedas the amount of variance in the
originalset of itemsaccountedforby factork. Since the total variance
in a set ofp standardizeditemsis just p, and sincethe sum of all p roots
mustalso equal p, theratioXk/PiS theproportion oftotalvariancedue to
factork. These proportions are used to givean idea about theimportance
of a given factor;moreoverthe sum of these proportionsform factors
tells us how much variationhas been explainedby the total set of m
factors.
Given the factorloadingsand roots,it is possible to derivethe
factorscoresfromthe followingsimplerelationship:

fkj = E (aik/Xk)Xij (12)

In words,the factorscores are just weightedaverages of the original


standardizedscoresusing the ratio aik/Xk as the weightforitem i and
factork.
In orderto derive theta, considera single-factor solution with
root Xi,loadings ail, and scoresfij. We shall use the basic of
definition
compositereliabilityprovidedin Equation (5) as thebasis of our deriva-
tion;thatis, C(p)/S(p - 1). Accordingly we needto knowthequantities

11A factor between


as thecorrelation
loadingcanbe interpreted an item
and thefactorscores.A negativeloadingmeansthattheitemhas a negative
withthefactor
correlation theitemscore,thelowerthefactor
scores(thehigher
scoreorviceversa).

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 31

C and S forthe factorscoresflj. We startwiththe knowNn


relationship
forS based on a weightedsum of item scores:
S = 2ZW ,Ti 2 + Zwiwh Cov(Xi, Xh) = I + C (13)

wherewi are the weights.Since in the case of factorscores the xi are


standardized,we have C. 2 = 1 and cov(xi, xh) = rih. Therefore,since
Wi = ail/Al,
S = f 22 (ail/X\)2+ (ail/Xl)(ahl/\l)rih
i h

= (at2i\/X\2 + (ailah1rih\/\2 (14)


2
And since EZai21 =

S= 1/X1+ (72 ailahlrih)/\2 (15)


i,--h J

From the basic equations of component analysis we know that


i,h ailahlrih = SO

E ailah1rih =
iHh
Z ailahlrih - Zi a2
i,h (16)
= X12-1
Thus
S = /X1+ (X2-Xl)/X2
1 1 1
~~~~~~~(17)
= 1/X1+ (1 -1/X1)
This means that forthe factorscoresbased on the firstprincipalcompo-
nent,S = 1, I = 1/X1,and C = 1 - 1/X1.For the compositereliability
fromEquation (5) we have

(p - 1)/ = [p/(p - 1)C/S = [p/(p - 1)(1 - 1/X1)= 0 (18)


whichis the formulafortheta givenin Equation (9).
That this formulafor theta provides the maximumcomposite
reliability(given our definition)followsfroma fundamentalproperty
of principalcomponents:that the quantityX1is the maximumvalue for
the quadratic form i,hailahlrih = X. That is, no set of weightsail
otherthan those providedby the firstprincipal-component factorload-
ings produce a value largerthan the firstroot.12Thus theta gives the
maximumvalue forthe ratio C(p)/S(p - 1).
12 In principal-component theorythisrelationshipis generallydefinedas
v1Rv1= Xi, wherethe firstprincipalcomponentis that vector v, which pro-
duces thelargestrootXi (see footnote10). Withtheusual normalization forfactor
loadingsthis is equivalentto a'Ral = X2.

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
32 DAVID J. ARMOR

The derivationfortheta-starfollowsa similar(ifmoreinvolved)


line. Startingwiththe usual relationshipS = I + C, forthe kthrotated
factorscoresf we have
2 p 22 p
S= =
S fk - Z zik -T + z Wikwjkrij Ik + Ck (19)

Since S = 1 forrotatedscoresas well,all we need to findforour ratio of


Equation (5) is the quantityCk = Z p WikWjkrij. In the case ofrotated
factorscores,the weightWik is morecomplexthan forunrotatedfactor
scores:
m
Wik = 'E (aihOhk)/\h (20)
h=l

where4hk iS the correlationbetweenoriginalfactorh scoresand the ro-


tated factork scores,or the (h, k) elementin the orthonormaltrans-
formationmatrix.Using this relationshipforCk we have
P P m m
Ck = E WikWjkrij =
iHj=l
E
iHj=l
[~h=lI (aihhk)\h [E (ajhOhk)/h
h=1
rij

P m m'(1
=~~E~ ,2rij
E(aihOhkajhOhk)/Xhr
+
+ , (aihchkaj0gk)/Xh\rij (21)
ii--dj-1 h=1 hp,hg=l

m 2p m P
= --I a'haJhrij + E (chk0gk)/(XhX\g) E2 aihaigrij
h=1 Ah iHj=1 h,g=1 idj1-

From the knownrelationshipsin principal-component analysis,we have


Z ipj aihajhrij = Xh2 hand Z ipj aihaigrij = 0. Therefore
Ck E2y(Xh- Ah) =Z phk(I- 1 )= hk- _E hklh/A
hsl Ah h=l A h=1 h=1

Since the quantities khkare froman orthonormal transformation


matrix,
we knowthat E am 1 = 1. We have, finally,

Ck =1-Z hk/Xh (23)


h=1

and thus our formulafortheta-star

k= - 1)][1 - hkh (24)

To sum up thereis a fairlysimpleconceptualbasis forthe rela-


tionshipa < 0. As we have said, alpha tendsto get largeras the amount
of covariationin a set of itemsincreases.But alpha assumes that each
itemis weightedequally and that each inter-item correlationis close to

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 33

r, the average correlation.When this conditionis present,theta and


alpha are approximatelyequal. When this is not the case, each item
contributesa different amount to the covariance.A factorloading ex-
pressessomethinganalogous to the average amountof covariationdue
to an item: those items with highercorrelationswith otheritems have
higherfactorloadings.Thereforea compositescale that weightsitems
accordingto thefactorloadingsexpressesa moreaccurateratioofcovari-
ance to scale variance.Theta and alpha are equal onlyin thelimitingcase
whenall inter-item correlationsrijare equal.
The relationshipbetweena and O*is moredifficult to state since
a is appropriateonly fora single-factor model. Given an m-factorsolu-
tion, one can constructm unweightedscales using the highestloading
itemson each factor.Lettinga' be the alpha reliabilityforthe subscale
correspondingto factor k, empiricalresultsshown later demonstrate
that usuallyO*< a'. Moreover,ifeach subsetofitemsis refactoredas a
single-dimension scale and a new theta reliabilityis computed,say Ok,
then generallya' and O' are approximatelyequal. Given a multiple-
factorsolution,therefore, optimallyreliablescales are derivedby apply-
ing the usual unweightedcovariancetechniquesto each subset of items
ratherthan by usingrotatedfactorscores.More is said on thisissue in
the nextsection.

IMPROVING RELIABILITY BY FACTOR SCALING

There are severalways in whichwe may use thetareliabilityand


principal-component analysis to enhancethe reliabilityof a composite.
First, as we have already seen, if one uses factorscores forcomposite
scale scoresinsteadofthe traditionalunweightedsum ofitemscores,the
methodof thetareliabilityshowsus that we are guaranteeda reliability
equal to or greaterthan that given by alpha reliability.Second, in the
case ofa single-factorsolutionwe muststillraisethe questionofwhether
weightingalone takes care ofitemsthat do not contributeto a factor(or
contributeto onlya smallextent).That is, willreliabilitybe enhanced
beyond that providedby factorscoring-by factorscales that exclude
low-loadingan item altogether?By factorscale I mean a composite
formedto measurea factorby usingonlythe highest-loading itemson a
factor.Finally,both reliabilityand conceptualclaritymay be improved
by adoptinga multiple-factor solution(whetheror not it was hypothe-
sized). If eitherof these last two conditionsis present,we must again
considertherelativebenefitsofrotatedfactorscoresversusfactorscales.
In thissectionI take up somepracticalconsiderations thatbear on these
issues.

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
34 DAVID J. ARMOR

Single-Factor Case
Althoughwe know that theta is algebraicallylargerthan alpha,
it mightbe usefulto presentsomesimplehypotheticalexamplesthat re-
veal how seriousthe differences mightbe and showhow we can improve
scale reliability.We shall examinean applicationoffactorscalingto real
data in a latersection.
Table 4 shows fourcases, the firstthreeof whichinvolve four
items.I have helditems1 to 3 constantso that thedifference in reliabili-
ties can be seen as we add differentfourthitems.
Case 1 reveals the largestdiscrepancybetweentheta and alpha
(0.05). The reasonis that the fourthitemhas low correlationswiththe
otheritemsand consequentlya low factorloading (0.24). A scale score
that weightsitemsaccordingto the loadingstherefore has a higherreli-
abilitythan a simplesummedscore.
In case 2 the fourthitem still has somewhathighercorrelations
and a higherloading (0.44), so that the difference betweentheta and
alpha is diminishedto 0.02. In case 3 thereis practicallyno difference
betweentheta and alpha even thoughthe correlationsand the loadings
do varyconsiderably.Clearlythetaand alpha differ in a substantialway
only whensome itemshave consistently lowercorrelationswithall the

TABLE 4
Comparing Theta and Alpha
Factor
Case Item Correlations Loading (ail)
Case I 1 2 3 4
Xi = 2.033 1 1.0 0.86
o = 0.68 2 0.6 1.0 0.82
a = 0.63 3 0.5 0.4 1.0 0.76
4 0.1 0.1 0.1 1.0 0.24

Case 2 1 2 3 4
Xi = 2.117 1 1.0 0.87
a = 0.70 2 0.6 1.0 0.80
a = 0.68 3 0.5 0.4 1.0 0.72
4 0.3 0.2 0.1 1.0 0.44

Case 3 1 2 3 4
X = 2.310 1 1.0 0.81
o = 0.756 2 0.6 1.0 0.80
a = 0.754 3 0.5 0.4 1.0 0.76
4 0.3 0.4 0.4 1.0 0.66

Case 4 1 2 .3
X= 2.004 1 1.0 0.87
0 =0.75 2 0.6 1.0 0.82
a = 0.75 3 0.5 0.4 1.0 0.76

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 35

remainingitems in a set. Nonethelesswe can see that theta is always


largerthan alpha.
Case 4 shows the statisticsfora scale formedby the firstthree
itemsalone. It is clearthat the three-item scale in case 4 is morereliable
than the weightedfour-item scale in cases 1 and 2 (0 = 0.75 compared
to 0.68 and 0.70), and it is almost as reliableas the four-item scale in
case 3. In otherwords,even thoughfactorscoresgive less weightto the
weak itemsand producea morereliablescale than a simple unweighted
summation,an even higherreliabilitymay be obtainedby throwingout
weak itemsaltogether.
This fourthcase motivates the procedure that I call factor
scaling.In some cases the highestreliabilityis obtainedby formingthe
scale using only items that load higheston a given factor.A scale so
formedis called a factorscale to distinguishit fromfactorscores. Al-
thoughthereis no preciseruleto definethe highestloadings,experience
has shownthat itemswithloadingsbelow0.3 shouldbe excluded.Items
in the 0.3 to 0.4 range are generallyborderline;that is, sometimesthey
increasereliabilityand sometimestheydecreaseit, but usuallytheydo
not affectit in any appreciableway.
One practicalconsequenceofconstructing scalesby usingonlythe
highest-loading itemsis that theta and alpha do not oftendiffer by any
importantdegree.This means that one can use the simple,unweighted
sum oftheitemscoresforthe compositescale withoutsacrificing reliabil-
ity. This procedureis onlyvalid, of course,whenthe itemsused do not
have widelydiffering factorloadings.Whetherone uses the unweighted
sums or refactorsto get factorscoresforthe subset of itemsto be used
in the scale is mainlya matterof convenience.
I must issue one importantcaution at this point. The most reli-
able scale forone sample of subjects may not correspondto the most
reliable scale for anothersample. When I suggestthat throwingout
itemsmay producea morereliablescale, this holds only in the context
of a singlesample. If one is buildinga scale of some type (ratherthan
analyzingdata froma particularsample), manysamplesand manyfac-
tor analyses must be taken beforea finalscale can be derived.In this
case a "bad" item is one that shows consistentlypoor factorloadings
acrossdifferent samples.Nonethelesswhenone is analyzingdata froma
particularsample and scales are beingused, thereis no reasonnot to ex-
clude weak itemsforthat analysiseven thoughtheymay be maintained
whencollectingdata fromothersamples.
Multiple-Factor Case
In the event that a set of itemsmeasuresmorethan one dimen-
sion,the methodof factorscalingrequiressome extension.Althoughthe

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
36 DAVID J. ARMOR

scaling operationsthemselvesare similarto those forthe single-factor


case, the mathematicaleleganceis somewhatdiminishedifthereis more
than one factor.The reasonshave to do withthe nonuniquenessof the
rotationalmethodsthat are used to interpreta multiple-factor analysis.
Given two or more clustersof intercorrelating items, the principal-
componentsolutionis notsuitableforfactorscalingbecause it maximizes
thevarianceofthefirstfactor,thenthesecondfactorafterthefirstfactor
varianceis removed,and so on. Therefore, giventwo or moreindepend-
entclustersofitems,unlesstheclustersare perfectly uncorrelated(which
is seldom the case) the firstfactorwill pick up more variance than it
should.
Figure 1 gives a geometricinterpretation of this problem.With-
out rotationthe firstfactorconsistsof strongand positiveloadingsby
all items whereasthe second factoris bipolar with both positive and
negativeloadings.The rotatedsolutionassociates each factorwithone
subsetof itemsor the other,facilitating both interpretation and scaling.
I want to stressthat rotationis not advocatedonlyas a convenience.As
we shall see in the real-dataexamplein a later section,rotationcan be
crucial foridentificationof thoseclustersofitemsthat producethe most
reliablescales.
Given a set ofm principal-component factorloadingsail, ai2, ,

aim and theirassociatedroots Xi, X2, * *y Xmfroma set of p items (1 <


,

mn< p), a new set ofrotatedloadingsa,*,a*2, *, a and scoresf;, *,


fmj can be foundby applyingan arbitrary orthogonaltransformation to
theoriginalloadingsand scores.Withone exceptionthe rotatedloadings
and scoreshave all the propertiesof the unrotatedloadingsand scores:
theyare statisticallyindependent;takentogethertheyexplainthe same
amountofthetotalvariation;and theyreproducetheoriginalcorrelation
matrixto thesame extent.The exceptionis thatthesumofsquaresofthe
loadingsX*, ,* (whereXk =
*, i a*k) do notequal theoriginal roots.
That is, we can say thatE km=l X = 57,=m X*but we also have that
Xk # X*.This means that althoughthe amount of variance accounted
forby in factorsis invariantunderarbitraryorthogonalrotations,the
portionof varianceattributableto each factoris not. Obviously,then,
dependingon the ro-
the scales derivedas well as theirreliabilitiesdiffer
tationalsolution.The onlyway to achievesome standardizationat this
stage is to specifythe rotationmethodwhenscales and reliabilitiesare
presented.
Althoughmany different rotationalschemeshave been devised
(Harmon,1967), the varimaxmethodseemsto be the mostwidelyused
orthogonalsolutionat the presenttime,and it is the one I shall assume
throughout.The varimaxmethodrotatesall m factorsuntil the factor

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 37

I 1*I 11*
0
1.0
0~~~~~~
ClusterA ail ClusterB a*
CILISLserAC /(luster B
-1.0 0 1.0M
a
a i2

-1.0

Uinrotate(I Rotated
ofunrotatedand rotatedfactors.
1. Geometricrepresentation
Figutre

loadingson all factorsattain maximumvariance.This means that each


factorhas itemswitheitherveryhighor verylow loadings(in absolute
value) withinthe constraintsimposedby the total varianceof the load-
ings.Since thissolutiontendsto give the mostsatisfactoryalignmentof
factorswithclustersof items,giventhe requirementof factorindepend-
ence, it seems the most suitablesolutionforfactorscaling.
The mostdifficult issueby farformostanalystshas to do withthe
numberof factorsm to extractand rotate. Clearly this numberdrasti-
cally affectsthe natureand reliabilityof the scales produced.Unfortu-
nately,however,Ino singleanalyticmethodcan solve this problempre-
cisely.About all I can offerhereare threerulesthatmay aid thedecision.
First,no factorshouldbe used ifits rootis near to or less than 1.
Since factoranalysis works on standardizeditems, the variance of a
singleitem is 1. Thereforea factorwhose root is near to 1 explainsno
morevariancethan a singleitem.
Second, afterthe mthroot, if the m + Ith and successiveroots
are fairlyequal exceptfora gradual tapering-off, thenm factorsshould
probablybe used (or at least rotatedforinterpretation).A large drop
fromone root to anotherfollowedby slightlydecreasingroots usually
rnarksthe end of meaningfulfactorsand the beginningof erroror spe-
cificvariancecomponents.Thereare statisticaltestsavailable fortesting
equality of the remainingroots afterm have been extracted,but they
depend heavilyon sample size and may be of limiteduse in large socio-
logicalstudies (Lawley and i-\iaxwell,1963).
Third, in many sociologicalstudies with inter-itemcorrelations
rangingfrom0.3 to 0.5, an m that accountsfor40 to 60 percentof the
total variance is a good solution.In this contextit can be pointedout
that, for ten items,single-factor solutionswhich account for no more

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
38 DAVID J. ARMOR

than 35 percentof the total variance can producescale reliabilitieson


the order ofO.8.13
Aside fromthese rules of thumb,most carefulanalysts rotate
suecessivenumbersof factorsuntiltheyfinda solutionthat is substan-
tivelymeaniingful and interpretable.Obviouslythis last criterionis as
importantas the others.
Given a satisfactorysolutionof in factorsas in the single-factor
case, thereare twomajor waysin whichto constructscales and compute
reliabilities.One methodis based on the rotatedfactorscores;the other
uses factorscales based on the highestloadingitems.
For principal-componientfactoranalysis,factorscoresforthe ro-
tated factorloadingscannotbe computedwitha simpleformulasuch as
Equation (12). Neverthelessfactorscoresare usuallyprovidedby most
computerprogramsforprincipal-component analysis.The procedurein-
volves rotatingthe originalfactorscores using the same orthogonal
transformation matrixthat is derivedforrotatingthe factorloadings.
Given rotated factorscoresfkj, a reliabilitycoefficient can be
computedforeach factork by applyingthe generalformulafortheta-
star given in Equation (10). It must be stressedthat these reliabilities
are not unique,however,sincetheydependon a givennumberoffactors
and on a givenrotationalmethod.I also stressthat thesereliabilitiesare
generallysignificantly lowerthan thoseobtainedby factorscaling.
As illustratedin the single-factorcase, even betterreliabilities
can be obtainedby formingscales usingonlythe itemsthat load highly
on a givenfactor.This methodis even moreimportantin the multiple-
factorcase sincethe existenceof two or morerotatedfactorsguarantees
that each factor*willhave many itemswithloadingsclose to zero. The
set of itemsthat loads highlyon each factor(say, above 0.40 or so with
no higherloadingson otherfactors)is takento forma scale. Since this
approachguaranteesrelativelyequal factorloadings(that is, equal item
w-eights), simpleunw-eighted summedscales can be constructed;alpha
reliabilitycan then be computed as a close approximationto theta.
Alternativelyeach subset of itemscan be refactoredfora single-factor
solution;both factorscoresand theta can thenbe obtained.Items that
load highon two or more factorsshould be excludedif one desiresto
maintainscale inidependence.
Althoughthe method of factorscaling generallyproduces the
most reliablescales, thereis a disadvantagein that the scales will not
of varianceex-
betweenthetaand the proportioin
13 The relationship)
)lained by thefirstfactor(say, Pi) is givenby Pi = 1/[p - 0 (p - 1)]. This can
be usedinconjunctionwithdiffering
numbers ofitemsto seewhatproportion of
varianceshouldbe explained
bythefirst toattaina giventhetareliability.
factor

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 39

necessarilybe orthogonal(uncorrelated).If the clusterof itemsthat de-


finesa factoris somewhatrelatedto anothercluster,however,thiscorre-
lation is probablymeaningful.If the correlationbecomes too large, of
course,it may mean that a smallernumberof factorswill be a better
solutionfromthe perspectiveof scalabilitythat we have developed.
Rather than givinghypotheticalexamplesto illustratethe mul-
tiple-factorcase, I shall turnimmediatelyto an exampleusingreal data.
The issues we have been discussingshouldbe made clearby way of this
completeillustration.

EXAMPLE OF FACTOR SCALING

To thispointmy comparisonof factorand covariancescalinghas


been primarilytheoreticalratherthan empirical.In orderto show how
to apply theseideas and howto see differencesbetweenthetwomethods,
thissectionpresentsa fullexampleusingreal data.
The data are drawnfroma surveyof collegeprofessors' viewsand
behaviorregardingthe VietnamWar (Armorand others,1967). A set of
13 attitudeitemswas designedto measurepoliticalideologyalonga gen-
eral conservative-liberal dimension.The intentionwas to constructan
attitudescale to measure politicalliberalismso that politicalideology
could be related to feelingsabout the Vietnam War. Althoughit was
knownthat politicalideologyis not a unidimensionalconstructin na-
tional cross-sections,
the special natureof the sample led to an expecta-
tion that therewould be a singleliberal-conservative dimensionin this
case. Since the originalintentionwas to createa singlescale, it is a par-
ticularlyappropriateexampleforcomparingunidimensionalcovariance
scalingwithmultidimensional factorscaling.
Covariance Scaling
Table 5 showstheitemwordingsalongwitha numberofstatistics
used in theconstruction ofa covariancescale. All itemswerescoredorigi-
nally on a 1 to 5 agree-disagreecontinuumwith5 indicating"strongly
agree"; itemsmarkedwitha minussignhave been reverse-scored so that
all itemsare scoredin the same direction(that is, agreeingmeans more
liberal). In additiononlyrespondentswho had no missingobservations
on any of the 13 itemswereincludedin the analysis.
If an analystconsidereditemface contentalone, withoutconsid-
erationof the quantitativeitem-analysisinformation shownin Table 5,
and developeda 13-itemscale by a straightsummationofitemscore,the
alpha reliabilitywouldbe 0.72 usingEquation (7). The alpha reliability
based on r (standardizeditems)is slightlyhigherin thiscase (0.73).

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
TABLE 5
Covariance Scaling forPolitical Ideology Scale
(a) Item-Scale Correlations
Itemb Uncorrectedc Correctedd
1 War is not justifiableunder any circumstances 0.608 0.483

2 Labor unions in large corporations should be


given a major part in deciding company policy 0.557 0.428

(-)3 A democratic form of governmentis a superior


systemto any otheryet devised 0.073 -0.062 a = 0.

4 Socialized medicinewould be a betterway to pro-


vide health servicesthan our presentsystem 0.503 0.384
5 It is up to the governmentto make sure that
everyonehas a secure job and a good standard of
living 0.434 0.289
(-)6 It is never wise to introduce changes rapidly, in
governmentor in the economic system 0.367 0.213
(-)7 In taking part in any formof world organization,
this countryshould make certain that none of its
independenceand power is lost 0.397 0.255

(-)8 Unfortunateas it may be, war is sometimesnec-


essary 0.535 0.411
9 An occupation by a foreignpower is betterthan a
nuclear war 0.604 0.481

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
(-)10 In general, complete economic security is bad;
most men wouldn't work if they didn't need
moneyforeating and living 0.483 0.339
11 War could always be avoided 0.440 0.285
(-) 12 Nationalization of the basic industries would
lead to an intolerable degree of governmental
control 0.566 0.435
13 Nationalization of the basic industrieswould lead
to a more equitable distributionof wealth 0.623 0.514

(b) Correlationsd,Means, and Variance


1 2 3 4 5 6 7 8 9 10
< 1 1.000
2 0.223 1.000
3 0.014 -0.048 1.000
4 0.103 0.334 -0.028 1.000
5 0.018 0.315 -0.261 0.324 1.000
6 -0.027 0.197 0.088 0.115 0.118 1.000
7 0.261 0.207 -0.121 0.005 0.010 0.193 1.000
8 0.771 0.197 -0.011 0.059 -0.028 -0.149 0.226 1.000
9 0.439 0.115 -0.039 0.325 0.119 0.187 0.260 0.322 1.000
10 0.036 0.305 0.005 0.156 0.388 0.244 0.106 0.024 0.065 1.00
11 0.622 0.062 -0.043 0.112 0.023 -0.159 0.045 0.579 0.354 -0.020
12 0.588 0.228 0.033 0.329 0.392 0.266 0.177 0.049 0.197 0.38
13 0.200 0.314 0.023 0.356 0.227 0.202 0.077 0.249 0.370 0.26
a N = 104 forall statistics.
bAll items scored on a 1 to 5 agree-disagreescale; reverseditems indicated by a minus sign
c Item contributionincluded.
d Significancelevels: r > 0.19 significantat 0.05
level; r > 0.25 significantat 0.01 level.

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
42 DAVID J. ARMOR

A considerationof item-to-scalecorrelationsreveals a substan-


tiallylower(and nonsignificant) correlationforitem3. Even ifthisitem
is eliminated,however,the alpha reliabilityof the summedscale based
on the remaining12 itemswouldbe unchanged.The remainingitemsall
have significantand fairlyconsistentitem-to-scalecorrelations;it is un-
likely,based on thesestatistics,that an investigatorwould change the
scale beyonddroppingitem3.
The inter-item correlationsgive moredetail about the itemrela-
tionships and show the problemwith item 3: it has a large (and sig-
nificant)negativecorrelationwithitem 4 and generallynegligiblecor-
relationswiththe remainingvariables. This clearlylowersthe average
correlation,althoughin this case not enoughto affectalpha reliability
significantly.In spite of the item-to-scalecorrelations,it is apparent
fromthe correlationmatrixthat a numberof the itemshave low corre-
lationsand that someeven have slightlynegativecorrelations.A careful
inspectionof the pairwisecorrelationsmightreveal other items that
shouldbe eliminatedor combinedseparatelyto formsubscales. Rather
thanundertaking thistediousjob, however,we turnto factorscalingfor
a morepreciseand compactmethodofexaminingthepatternofrelations
amongthe items.
Factor Scaling
The relevantstatisticsforfactorscalingare presentedin Table 6.
I shall discussonlythefirsttwo factors,whichaccountforabout 43 per-
cent of the total variance.Because the thirdroot was close to 1 (X3 =
1.236), it was decidedto considerat mostthe firsttwo factors.
solution,we shouldfocuson the prin-
If we adopt a single-factor
cipal-component loadingsforthe firstunrotatedfactor.The firstrootis
3.29, accountingforabout 25 percentof the total variance. As we can
see fromthe firstcolumnof Table 6, thereis considerablevariationin
the item contributionsto the firstfactor. The loadings range from
-0.08 foritem3 to 0.65 foritem 1. These differences resultin a higher
reliabilityfora 13-itemscale based on the factorI scores,whichtake
item weightinginto account. Theta is 0.75 forthe factorI scores,com-
pared to an alpha of0.72 forthe 13-itemsummedscale. Since in thisex-
ample only one item is trulyweak, the difference between alpha and
thetais not great.Obviouslythe difference betweenalpha and thetade-
pendson how many"bad" itemsoccurin a set ofitemsbeingused fora
scale.
Ratherthan exploringothercombinationsof first-factor itemsto
discoverthe best single scale, we must considerrejecting the single-
factorhypothesis.The secondrootis fairlylarge (X2 = 2.32, 18 percent

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 43

TABLE 6
Factor Scaling for Political Ideology Scalea
Unrotated Principal-
Component Loadings Rotated Loadings

Itemb I II I* II*
1 0.650 -0.615 0.032 -0.895
2 0.553 0.247 0.567 -0.212
3 -0.082 -0.030 -0.080 0.036
4 0.523 0.283 0.571 -0.165
5 0.438 0.477 0.647 0.033
6 0.262 0.450 0.502 0.137
7 0.377 -0.058 0.228 -0.306
8 0.593 -0.636 -0.023 -0.869
9 0.637 -0.184 0.325 -0.578
10 0.425 0.472 0.634 0.038
11 0.483 -0.637 -0.103 -0.793
12 0.535 0.504 0.735 -0.016
13 0.648 0.217 0.613 -0.300
Roots Xi = 3.289 X2 = 2.322 Xi = 2.813 X2 = 2.798
Percent of
total variance 25.3% 17.9% 21.7% 21.5%

Reliabilities
Factor Scores
O = 0.75 forfactor scores based on unrotated factor I
01 = 0.70 forfactor scores based on rotated factor I*
02 = 0.70 forfactor scores based on rotated factor II*
Factor Scales
01 -0.74 for socialism scale based on items 2, 4-6, 10, 12, and 13 [ai = 0.73
using Equation (6)]
02 = 0.85 forpacifismscale based on items 1, 8, and 11 [a2 = 0.85 using Equa-
tion (6)]
a N = 104 for all statistics.
I See Table 5 for item wording.

of the total variance), and a numberof items have higherloadings on


factorII than on factorI (namely,items5, 6, 8, 10, and 11). Therefore
we mustexaminethe two-factor solution.For the two-factorhypothesis
therotatedloadingsmustbe inspectedin orderto obtaina moreaccurate
interpretation.These are shownin columnsI * and II * in Table 6.
The rotated loadings clearly reveal two relativelyorthogonal
clustersof items (see Figure 2 for a geometricrepresentation).The
clusterof items (2, 4-6, 10, 12, 13) that largelydeterminesfactorI * has
contentconcernedmainlywitheconomicissues (the sole exceptionbeing
item6); the itemsthatdeterminefactorII * (1, 8, 11) deal withtheissue

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
44 DAVID J. ARMOR

Iioe ( I* (rotatedl)
I (unrlotaletdl) (12) *(?> II
0.7

0.4-
0 ~~~~0.3
Ci) b0~~~.2-

-0.3
-0).4
-0.5

tha facto I * rerset a soils dieso an tha fato II 1*


rpe
-0.7-
not made in ada -,cof seein -0.8
the fatranlss

Figure 2. Factor plot for the political ideology items.

of war. Item 3 (attitudetowarddemocracy)and item7 (maintenanceof


U.S. power) do not load substantiallyon eitherfactor.Item 9 ("better
red thandead") fallsbetweenthe two clusters.Accordingly we shall say
thatfactorI * representsa socialismdimensionand that factor * repre- I,
sents a pacifismdimension.This distinctioncertainlymakes sense con-
ceptually,althoughin the originalstudy this two-factor predictionwas
not made in advance of seeingthe factoranalysis.
Aside fromthe conceptualadvantagesof the two-factor solution,
there are some importantquantitativeadvantages as well1.First, the
two-factorsolutionaccounts for substantiallymore variance than the
single-factorsolution,witheach factoraccountingfora littlemorethan
21 percentof the total variance. Second, scale reliabilitiescan be in-
creasedsubstantially.The 0* reliabilitiesforthe scales based on the full
13-itemfactorscoresare somewhatlowerthan the unrotatedfactorn,
with tlieta-star0.70 forboth. If we are guided by the highestrotated
loadingsin Table 6 and clustersshownin Figure2, however,and if we
refactoritems2, 4-6, 10, 12, and 13 forthe socialismscale and items1,
8, and 11 forthe pacifismscale, we thenhave 0' reliabilitiesof0.74 and
0.85. The reliabilityof the socialismscale is slightlyhigherthan the
original13-itemalpha reliability;the pacifismscale reliabilityis con-

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 45

siderablylarger.Moreoverthe alpha reliabilitiesforthesetwo subscales


are 0.73 and 0.85, so that weightedscoringis unnecessaryand either
alpha or theta can be used to estimatereliability.
In effectthe factoranalysis enables us to find two relatively
homogeneousclustersof itemsthat can be used to constructtwo inde-
pendent subscales which closely approximatethe basic assumptions
behindthe alpha coefficient.
A slightlymorereliablescale (0 = 0.75) can be obtainedby add-
ingitems7 and 9 to thefactorI *scale, but thisdoes not seemjustifiable
in view of the largeloadingthat item9 has on factorII *. Moreoverthis
raises the correlationbetweenthe two subscales from0.09 (indicating
practicallyindependentscales) to 0.20. Addingitem 9 to the factor11*
scale substantiallyreduces the latter's reliability.Basically Figure 2
shows that items7 and 9 fall betweenthe two major clustersand that
theirinclusionin one scale or the other eitherdecreases reliabilityor
does not sufficiently increasereliabilityto justifythe resultantloss of
scale independence.
This examplemakes it clear that factorscalingcan substantially
improve scale reliability,particularlywhen multiple dimensionsare
present.Reliabilitycan be improvedin a single-factor case by identifica-
tionofitemsthatdo not contributeto the centraldimension.But when-
ever multipledimensionsare present,factorscalingis even moreimpor-
tant. In the multiple-factorcase the reliabilityof factorscoresis often
considerablylowerthanthereliabilityoffactorscales (otherthingsbeing
equal). In the presentexamplethe rotatedfactorscorereliabilitieswere
0.70 each; they were improvedto 0.74 and to 0.85 when factorscales
wereconstructed.These factorscales had one further advantage:it made
littledifference whetheralpha (unweighted)or theta (weighted)coeffi-
cientswereused to estimatereliability.
Validity
I have stated that the ultimatetest of the importanceof a scale
is not simplyits internalconsistency.The acid test of a sociological
measureis its validity; that is, whetherit is related to other,external
sociologicalvariableswithintheframework ofa model.If scale reliability
is importantand if we believe that we have enhanced the conceptual
clarityand reliabilityof our measureof politicalideology,we shouldbe
able to show improvedrelationshipswithothervariables.
I mentionedthat the purposeofbuildinga politicalideologyscale
was to relateit to supportforor oppositionto the VietnamWar. Using
a scale to measureattitudeson Vietnam,we can testtherelativevalidity
of the covarianceand factorapproachesby comparingthe predictive

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
46 DAVID J. ARMOR

power of the original 13-itemsummed scale with the socialism and


pacifismscales derivedby the methodof factorscaling.
The VietnamWar scale is scoredso that a highscore indicates
strongsupportforthe war and a low score denotesstrongopposition.
The regressionresultscan be summarizedas follows:
R2 withVietnam
War Scale
1. Covariancescaling: 13-itemsummedscale 0.26
2. Factor scaling:pacifismscale alone 0.25
pacifismplus socialismscale 0.30
In otherwordsthemodelthatuses thetwo morereliablescales raisesthe
explainedvariancefrom26 to 30 percentforour externalvariable.This
is directevidencethat the use of factorscalingto developmorereliable
scales can have a substantialimpacton the testofa model,even whena
covariancescale has an acceptablelevel of reliability.

C01PARISON OF T'HETA AND OMEGA

Beforeconcludingit is necessaryto make a fewcommentsabout


the relationshipbetweentheta and omega,a new reliabilitycoefficient
proposedby Heise and Bohrnstedt(1971). Their workrepresentsan im-
portantcontributionto understandingthe relationshipbetweeinfactor
analysisand reliability.They take a unique approachby usingthe tech-
nique of path analysistogetherwithtraditionalfactoranalysisto derive
measuresof reliabilityand validity.At the momentwe are most inter-
ested in theiromega,whichlike thetais based on factorainalysis.
Assumingstandardizeditems for convenience,the formulafor
omega reliabilityis
Q ( Erti?+Pr)
+L ) ( (25)

whereh2 is the communalityof item i. The major difficulty in relating


omega and theta stems fromthe relianceof omega on the conceptof
communalityhi. In the classic theoryof factoranalysis, communality
is definedas that componentof an item's variance shared in common
withthe remainingset of itemsbeingfactor-analyzed. It is to be distin-
guishedfromunique variance-that part of an item's variationdue to
the item alone (includingerrorcomponents).Unfortunately, since the
communalitydepends oIn the numberof factorsextractedthereis no
analyticmethodforcomputinga unique,invariantcommunality foreach
item.That is, since the communalityforitemi is definedas
2 2 2 2
h. =ail+ ai2 + +
*. faim (26)
solutionand not a
(where the aik are derived froma common-factor

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 47

principal-component solution) or the sum of squares of the factor load-


ingsof an itemon the m factorsextracted,we do not have an invariant
h2 unlessthenumberoffactorsm is also invariant.But as I have pointed
out,thenumberofmeaningful factorscannotbe determinedanalyticallv;
it dependson the typeof factoranalysisused and, mostof all, on inter-
pretivejudgmentsmade by the analyst.Thus forthe same set of items
omega differs fromone analysisto anotherdependingon themethodand
investigator.In generalthe greaterthe numberof factorsextracted,the
greaterthe omega coefficient.
Since experiencedsocial scientistshave acquired a hightolerance
forambiguity,the variant propertiesof omega may not be viewed as
seriousones. Thereis, however,a secondand moreseriousdifference be-
tweenomega and theta.We have seen that thereis no restriction on the
hypothesizednumberof factorsforomega. But if an analyst decides
thereare reallytwo or moreindependentfactorsin a set of items,each
factorshouldresultin a separatescale. That is the wholepointof apply-
ing factoranalysisin the firstplace. From the standpointof reliability
theoryand covariancescaling,thereis no morejustification in combining
uncorrelatedclustersof items into a singlescale than thereis in com-
bininga set of uncorrelatedindividualitems into a single scale. Thus
omega does not assess the reliabilityof separate scales in the event of
multipledimensions.
The comparisonof omega and theta can be done meaningfully
only if we considerthe single-factorcase. In additionit is importantto
2
stress that the hi for omega depends on a common-factorsolution, such
as the maximum-likelihood
method (Lawley and Mlaxwell,1963). The
a2 obtained from a principal-component solution should not be used in
2
Equation (26) to estimatethe hi. If thisis done the omega formulare-
sultsin a reliabilityestimatethat is spuriouslyhigh.'4
Assumingthat these conditionsare met, what can we say about
the differencebetweenomega and theta?First,practicalexperiencewith
common-factor and principal-component solutions reveals few differ-
ences betweenthe relativemagnitudesof the factorloadings.That is,
althoughthe loadingsdifferin absolute value both methodsgenerally
rank the itemsin the same orderwith respectto theircontributionto
the firstfactor.Thus if a scale is formedby takingthe items withthe
highestloadings,both methodsshould resultin similarscales. Second,
experiencealso indicatesthat the values of omega and theta are gener-
ally comparable,withvariationsusuallyunder0.01. The main pointto

14 If all intercorrelations
equalr, thena, 0, and Q shouldbe equal.But
using principal-componentloadings gives Q = a + (1 - r)/pX,, making
Q> or 0 unlessr = 1.

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
48 DAVID J. ARMOR

rememberis that the theta formulashould be used with principal-


componentanalysisand thatomegashouldbe used withcommon-factor
hypothesis.
solutionsassuminga single-factor

SUMMARY

We have reviewedthe classic methodof covariancescaling,out-


liningthe steps that investigatorsshould take to develop a composite
measure of some sociological construct.Although the basic idea of
covariancescalingis sound,sinceit is moreappropriateformanysocio-
logical conceptsthan otherscaling methods,it has several weaknesses
in the traditionalways it is applied to sociologicalvariables.Such steps
as item-to-scale correlations and alpha reliabilitydo not take properac-
countof varyingitemcontributions to a construct,and theycannotun-
cover the existenceof multiple,independentconstructsthat mightbe
presentin a set of items.Theta reliabilityand factorscaling,based on
traditionalprincipal-component factoranalysis,is offeredas a method
ofimprovingscale reliabilityand conceptualclarity.It presentsa means
of discoveringmultidimensionality and allows items to relate differen-
tially to these dimensions.These methodshave the advantage of con-
nectingmultidimensional scaling closelyto the traditionalconceptsof
measurementreliabilityand scalability.
Perhapsthebest way to summarizethetechniqueoffactorscaling
is to listthestepsan investigator can followwhenapplyingthetechnique
forimprovingscale reliability.
First,select a set of variables that representsa domain within
whichone or morescales or indicesis predicted.
Second,omitany subjects(or otherunitsofanalysis)withmissing
observationsforthepurposeoffactorscalingand reliabilitycomputation
(factoranalysiswith mean substitutionfor missingdata gives a good
approximation).
Third, apply a principal-component factoranalysis, extracting
factorsuntileithertherootsapproach1 or therootsbeginto taperoffby
approximately equal decrements.Onlyrarelywillit be necessaryto take
morethan fouror fivefactors;even feweris the norm.
Fourth,if a single-factor hypothesisis supportedby the factors,
the unrotatedfirst-factor loadingscan be used forinterpreting the scale.
If regularfactorscoresare used to create the scale, the theta formula of
Equation (9) shouldbe used forreliability.
Fifth,if the factorrevealsitemswithsmall loadings (say, under
0.3 or 0.4) higherreliabilitiescan be obtainedby eliminatingthe weak
itemsaltogether.In thiscase a simplesummedscale can be created(as-
suming approximatelyequal item variances; if not, items should be

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
THETA RELIABILITY AND FACTOR SCALING 49

standardized)and alpha reliabilitycan be computedfora reliabilityesti-


mate. Alternatively thesubsetcan be refactored in orderto obtainfactor
scoresand the corresponding theta coefficients.
Sixth, if a multiple-factorsolution is suggestedon eithercon-
ceptual or empiricalgrounds,rotationsof two or morefactorsshouldbe
done (usingthe varimaxor some similarmethod)untilan interpretable
solutionis obtained.Althoughthe rotatedfactorscorescan be used for
scales withreliabilitiescomputedaccordingto Equation (10), reliabilities
may be substantiallyhigherif only the highest-loadeditems on each
factorare used forscales. In this case, as with single-factor solutions,
scales can be constructedby simplesummationand reliabilityestimates
can be made usingalpha; alternativelyeach subsetcan be refactoredfor
factorscoresand theta.
I close with several cautions. First, reliabilityas we have dis-
cussed it pertainsonly to the internalconsistencyof a scale. High relia-
bilityis no guarantee,in and of itself,of a usefuland meaningfulcon-
struct.Second, thereare several otherrationalesformultidimensional
scalingand the covariancemodel may not be the best one forall socio-
logicalapplications.In particular,althoughnonmetricmultidimensional
scalingis not tied closelyto reliabilitytheorytheremay be applications
wherereliability(as defined)is not of primaryconcernand wherethe
nonmetricmethodsare more appropriatelytailoredto the problemat
hand.
Finally,even ifan analystfindsthe assumptionsof factorscaling
suitedto hisproblemhe cannotbe certainthatfactorscalingwillimprove
his scalingproblem.On the one hand the experiencedsociologistwho is
especiallysensitiveto the substantivemeasurementissuesin his fieldof
interestmay be able to constructhighlyreliablescales withlittleor no
quantitativeassistance (althoughhe may want to have a check on his
intuition!).On the other hand factorscaling cannot produce reliable
constructsout of a hodge-podgeof items assembledwithoutregardto
conceptualdomain and content.The view presentedhereis that factor
scaling is a quantitativeadjunct to scientificintuitionand judgment
thatmakesit possibleforall analysts-not just theparticularlysensitive
ones-to producecomparableresults.

REFERENCES

ARMOR, D. J. AND OTHERS


1967 "Professors'attitudes toward the Vietnam war." Public Opinion
Quarterly(Summer):159-175.
BUTLER, P. M.
1968 "Alpha-maximizedfactor analysis and its relation to alpha and
cannonicalfactoranalysis." Psychometrika
33 (March) :335-346.

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions
50 DAVID J. ARMOR

BLALOCK, H. M., JR.


tomeasurement
andthecausalapproach
1969 "Multipleindicators error."
Journal
American 75 (September):264-272.
ofSociology
BOHRNSTEDT, G. W.
1969 "A quick methodfordetermining and validityof
the reliability
scales."American
multiple-item Review
Sociological :542-
34 (August)
548.
CRONBACH, L. J.
alphaandtheinternal
1951 "Coefficient oftests."Psychometrika
structure
16 (September):297-334.
Testing.(2nded.) New York:Harper&
1960 EssentialsofPsychological
Row.
EDWARDS, A. L.
of Attitude
1957 Techniques New York: Appleton
Scale Construction.
Crofts.
Century
GUILFORD, J. P.
NewYork:McGraw-Hill.
Methods.
1954 Psychometric
HARMAN, H. H.
1967 ModernFactorAnalysis.(2nded.) Chicago:University
of Chicago
Press.
HEISE, D. R.
and stabilityin test-retest
1969 "Separatingreliability correlations."
American Review34 (February):93-101.
Sociological
HEISE, D. R. AND BOHRNSTEDT, G. W.
1971 "Validity,invalidity, Pp. 104-129inE. F. Borgatta
andreliability."
San Francisco:
Methodology.
(Ed.), Sociological Jossey-Bass.
LAWLEY, D. N. AND MAXWELL, A. E.
1963 FactorAnalysisas a Statistical London:Butterworth.
Method.
LORD, F. M.
1958 "SomerelationsbetweenGuttman'sprincipal of scale
components
analysisand otherpsychometric (Decem-
theory."Psychometrika
ber):291-296.
LORD, F. M. AND NOVICK, M. R.
1968 Statistical ofMentalTestScores.Reading,Mass.:Addison-
Theories
Wesley.
NOVICK, M. R. AND LEWIS, C.
alphaand thereliability
1967 "Coefficient measurements."
ofcomposite
32 (March):1-13.
Psychometrika
SIEGEL, P. M. AND HODGE, R. W.
1968 "A causalapproachto thestudyofmeasurement error."In H. M.
in Social Re-
Blalock,Jr.and A. B. Blalock(Eds.), Methodology
search.NewYork:McGraw-Hill.
UPSHAW, H. S.
1968 "Attitude measurement."In H. M. Blalock,Jr.and A. B. Blalock
NewYork:McGraw-Hill.
in SocialResearch.
(Eds.), Methodology

This content downloaded from 128.235.251.160 on Tue, 10 Feb 2015 01:34:33 AM


All use subject to JSTOR Terms and Conditions

You might also like