Professional Documents
Culture Documents
511–522, 2002
Ó 2002 Published by Elsevier Science Ltd.
Printed in Great Britain
www.elsevier.com/locate/worlddev 0305-750X/02/$ - see front matter
PII: S0305-750X(01)00114-0
be, a particularly fruitful interaction between US profession has, in fact, seen something of a
economics and other disciplines, namely the convergence similar to that already witnessed in
analysis of child survival. Gender is a crucial economics (Harriss, 2002). A review of the two
aspect of such analyses, so that the example is a most recent issues of the American Journal of
specific case of the more general point argued Political Science shows that just over one-third
by Jackson (2002) that feminist economics of the papers (10 out of 29) use mathematical
demonstrates the application of economic modeling, most of a sort similar to that found
techniques to an area previously analyzed from in the Economic Journal or the American Eco-
a sociological perspective. Section 5 concludes. nomic Review (Table 1). Eighty percent of the
papers present regression results. While some of
these are ordinary least squares, logistic re-
2. THE USE AND ABUSE OF gression is the most commonly applied tech-
TECHNIQUES nique and one paper uses vector auto
regressions (VARs). One of the papers is pri-
There is a perception among economists that marily concerned with appropriate statistical
quantitative techniques provide more ‘‘rigor’’ techniques (Zorn, 2001). Sociologists are apt to
than qualitative techniques. Hence it is often draw on data analysis as and when it is ap-
felt that economics, with its more rigorous propriate for their purpose, as in Durkheim’s
footing, is a sounder basis from which to for- classic study of suicide first published in 1897.
mulate policy advice. 2 Three observations may Again, in the United States a tradition has
be made with respect to such an attitude. First, grown up which routinely utilizes relatively
the use of quantitative techniques is common in sophisticated quantitative techniques. A review
several other disciplines, with the US social of two leading US sociology journals in the
science tradition utilizing these to a greater 1960s reported that over 90% of the papers
extent than that in Europe. Second, the real reported results from quantitative surveys
basis for ‘‘rigor’’ is the proper application of (Brenner, 1981, cited in Silverman, 1985). In-
techniques. Badly or misleadingly applied, both spection of the two most recent issues of the
quantitative and qualitative techniques give American Journal of Sociology found that two-
bad or misleading conclusions. Finally, differ- thirds of the papers use regression-based sta-
ent techniques are appropriate to different set- tistical analysis or similar ones (e.g., factor
tings. Later sections of this paper elaborate this analysis). Recent issues of the annual publica-
last point, showing how combining quantitative tion of the American Sociological Association,
and qualitative work can strengthen both. The Sociological Methodology, largely contain pa-
discussion turns now to the first two points. pers on advanced statistical techniques (e.g.,
Economics is not the only social science to Agresti, Booth, & Hubert, 2000).
rely on math, modeling and statistical data Turning to the second point, both quantita-
analysis. Demography has a strong mathemat- tive and qualitative approaches are about tak-
ical tradition, and is by its very nature rooted in ing observations of the world (data) and
analysis of quantitative data. Yet demogra- presenting them within a framework (a model).
phers routinely combine both quantitative and The important point is the relationship between
qualitative methods, and the discipline has not the data and the model, which comes first, and
seen the convergence toward a stylistic hege- which has the upper hand in this relationship.
mony which is evident from mainstream eco- In econometrics there is a distinction between
nomic journals. Political scientists are also no data analysis and data mining. Although both
strangers to either math or statistics, and the data analyst and data miner do similar things—
Table 1. Use of mathematical and statistical analysis in political science and sociology
American Journal of Political Science American Journal of Sociology
Mathematical modeling 5 0
Statistical analysis: regression or similar 18 4
Statistical analysis: other 0 2
Both modeling and statistical analysis 5 4
Neither 1 2
Total 29 12
Source: author’s examination of named journals.
COMBINING QUANTITATIVE AND QUALITATIVE 513
running many regressions and looking at di- their results and those of the World Bank was
agnostic tests—their objectives are very differ- ‘‘a mystery’’ (p. 1466). This claim is simply
ent. 3 The data analyst is looking for the untrue. The results are replicable, as five min-
interpretation most consistent with the data, utes with a spreadsheet would show.
i.e., letting the data tell the story. The data In such a climate, suspicions of data mining
miner knows what she is looking for and keeps can always arise. The World Bank’s work on
digging until she finds it. Then she stops and aid effectiveness, notably the paper of Burnside
that is the story she tells. Data miners are and Dollar (2000), has been criticized by many
equally at home using either quantitative or academic researchers on these grounds. For
qualitative data. some time the Burnside and Dollar data set was
The large qualitative analysis undertaken by not made publicly available and researchers
the World Bank, the Voices of the Poor— were unable to replicate the results using what
published in three volumes: Crying Out for ought to have been at least very similar data.
Change (Narayan, Chambers, Shah, & Petesch, Once the data became available the results were
2000a), Can Anyone Hear Us? (Narayan, Pa- indeed replicated but they remained unrobust.
tel, Schafft, Rademacher, & Koch-Schullem, Quite minor changes in model specification
2000b) and From Many Lands (Narayan & could change the results in such a way as to
Petesch, 2000)—has come under criticism for make them inconsistent with the arguments
its lack of rigor with respect to sampling and advanced by Burnside and Dollar and the
presentation of the results. Regarding the latter World Bank’s Assessing Aid report. 6
it is argued that in a mountain of transcripts it Where does this discussion leave us with re-
is, of course, possible to find a quote from a spect to qualitative data? Of course, qualitative
‘‘poor person’’ to fit the argument being made. data can be mined for ‘‘the right result,’’ but so
This is an accusation of data mining. But this is can quantitative data. There are well-estab-
hardly something of which economics is inno- lished techniques for the scientific analysis of
cent. Indeed, economics is predisposed toward qualitative data, for example, content analysis
a theory-driven approach in which the model is of different media—and textbooks telling the
first derived and then data fitted to that model. budding social scientist how to go about it (e.g.,
There is a saying that ‘‘if you torture the data Gilbert, 1993; Silverman, 1985, 1993). Com-
long enough [they] will confess.’’ 4 Certainly puter programs, such as QSR-Nudist, mean
any moderately competent econometrician, that ever larger data sets can be analyzed in
given enough variables, should be able to such ways. The real question here is one of
produce the desired sign and significance on professional respect, trust and integrity. Just as
most variables of interest. economists quite readily take as ‘‘right’’ the
In the absence of the data set and replication regression results they find in published papers,
of the results, it is impossible to know how they should accord the same professional re-
badly the data have been treated. It is not part spect to presentations of qualitative material.
of the refereeing process for economics journals Members of all disciplines should be on the
to replicate the econometrics. Indeed, authors look out for shady practices which distort
are not generally required to submit the data patterns in the data. 7 These problems are by
set to make this possible. Nor does economics no means unique to users of qualitative data.
appear to attach the same importance to rep-
lication as do the natural sciences. 5 In one
case, data were obtained which had been used 3. CREATING SYNERGY BETWEEN
for a paper in the Journal of Development QUANTITATIVE AND QUALITATIVE
Economics and three researchers were, inde- APPROACHES
pendently, unable to replicate the results. Even
where data are available most readers of course Two examples are discussed here in which the
take statements on trust. For example, the combination of quantitative and qualitative
World Bank report Adjustment in Africa pub- techniques can enrich one another. I go so far
lished the data it had used to show the benefi- as to say there is a synergy between them. That
cial impact of various policy changes on is, using the approaches together yields more
growth. Mosley, Subasat, and Weeks (1995) than the sum of the two approaches used in-
published a critique in World Development dependently. The first example discusses the
which included the claim that they could not lack of economic analysis of labor exchange in
replicate the results and the difference between rural Africa and points to the rich anthropo-
514 WORLD DEVELOPMENT
logical literature as a basis for developing payment, over which bargaining can occur, but
household models. The second point follows on there are many other forms of exchange rooted
from the first. This is how the problem of to varying degrees in social institutions. One
household economies of scale in consumption example, which can be found over much of the
can be resolved by realizing that the household continent, is of work parties or groups—com-
is the unit of production and the life cycle monly known as ‘‘beer parties’’ (or tea in
which follows from this. Muslim areas). The owner of the land provides
food and drink in return for the day’s work. It
8
(a) Labor exchange in rural Africa is prestigious to be able to arrange labor in this
way, and Kevane documents that older men
There is a view among many economists that will make their sons attend as part of the means
labor markets in rural Africa are very thin. A of ensuring attendance when they organize a
commonly cited statement of this position is similar event.
that of Binswanger, McIntire, and Udry (1989); The point made by Kevane can be general-
writing of the Sahel they say that ‘‘there is al- ized with evidence from throughout rural Af-
most no hiring or exchange of labour among rica. Labor exchange takes place when any
resident farmers during the peak labour sea- work is performed outside of the household.
son’’ (p. 125). 9 This view has been generalized Conversely, there is an exchange when any
for sub-Saharan Africa by Sahn and Sarris household production is performed by non-
(1994), who cite the view of Binswanger et al. household members. There are a wide range of
(1989) that rural Africa is typified by underde- market and nonmarket transactions through
veloped land and labor markets. They argue which labor exchange operates, which are
that labor and credit markets have only summarized in Table 2.
emerged in response to population growth and Economists model rural households as hav-
external trade opportunities and that there was ing a given labor supply (household size) and
little exchange of these factors before that. then deciding whether to hire labor in or out.
Hence, although their paper is titled ‘‘The But, as the last row of Table 2 shows, house-
evolution of states, markets, and civil institu- hold size is endogenous. Family size can be
tions in rural Africa,’’ and although they adopt deliberately enlarged to increase labor supply;
a broad definition of institutions to encompass Cheater (1984) (cited in den Ouden, 1995) calls
social relations, they do not touch upon any of this ‘‘the traditional idiom of accumulation.’’
the diverse forms of labor exchange which There are four main channels for adjusting
characterize rural Africa. family size. First, as economists have recog-
Kevane (1994) begins his paper analyzing nized, reproduction is an economic decision. If
labor exchange in Sudan with the quote from complementary assets are needed in production
Binswanger et al. given above. He goes on to then we would expect better-off households
show that, to the contrary, labor exchange is to have more children (though this is just
widespread. Some of this is in return for cash one determinant of the number of children).
hold life cycle matters for household well-being. most closely associated with the name of Mo-
Large families at certain stages of this cycle digliani, e.g., Ando & Modigliani, 1963). At
may be expected to be well off, but others may their simplest the models show expenditure to
not. Finally, there may be no paradox. Con- be based on expected lifetime earnings so that
sumption is not the thing that is valued; rather households are net savers in their working years
it is the prestige that follows from having large (except perhaps when starting out) and net
agricultural holdings, many cattle, and the borrowers in retirement. The story can be made
large family that goes with that. The first and more realistic by taking into account expendi-
third of these possibilities require further ex- tures related to property acquisition and child
planation. rearing. 14 The matter is more complicated
Quantitative analysis of the link between when the household is also involved in pro-
household size and poverty is complicated by duction. The most prosperous household will
the need to make two adjustments. The first be that with many healthy children of working
adjustment is to convert household size to adult age. That with a few young children, or an
equivalents to allow for the lower consumption older couple with no children, will be worse off,
requirements of children. 13 This adjustment is though the latter may well be absorbed into
commonly, though not always, made. The another household. There is clearly scope to
second adjustment, far less commonly made, is adapt modern theories of consumption to
to allow for economies of scale in household tropical settings, but only by taking into ac-
consumption, that is, the fact that two can live count the prevailing institutional setting. An-
more cheaply together than apart by virtue of thropological analysis of this social context
‘‘public goods’’ within the household. For ex- sheds light on how household size and com-
ample, charges for utilities may be the same for position affect household welfare. 15 Although
a household of two as for that of one, and present in some economic thinking (for exam-
certainly do not rise proportionately as house- ple, the work of Chayanov) recent household
hold size increases. Two problems arise if models for developing countries do not capture
economies of scale are not properly accounted these aspects.
for: systematic underestimation or overestima-
tion of poverty, misleading poverty correlates
with respect to household size. Both of these (c) Data collection
have important policy implications, so it would
be useful to know what light can be shed on The above is a broad outline of research is-
them by anthropological analysis. sues which may be tackled by economics, but
At least for rural households the key to this informed by information on institutional ar-
puzzle is likely to lie in the fact that the rangements provided by sociology. Moving
household is the unit of production as well as beyond modeling to data analysis requires that
consumption. Anthropologists have been the appropriate data be available. The current
critical of economists’ ready application of form of income and expenditure surveys is ill
the concept of ‘‘the household’’ to a wide adapted to the realities of rural life in much of
range of social settings (Hill, 1986). Of par- Africa. Termine (2001) reviews the Zambia
ticular importance for our purposes here is Living Conditions Monitoring Survey as a basis
that household size is chosen as part of the for analyzing rural labor markets. The small
production strategy. As discussed above, the labor market module asks questions such as
rich household head with control over ample whether the person has paid annual leave or is
resources will find ways to increase family in a superannuation scheme. Such questions
size to acquire labor. This fact is in contrast are peculiarly inappropriate to a rural laborer,
to the household models of economists which and a waste of an opportunity to collect labor
assume a household with a given endowment market data. The questionnaire does not allow
of labor (and sometimes land) to analyze if identification of payment in kind for particular
households are net suppliers or demanders of activities, and would miss activities such as
labor. beer parties (which are an important feature
Recognizing the household as the unit of of Zambian agricultural production, see
production also points to the importance of the Vaughan, 1998). Clearly there is a need for
household life cycle. The concept of the life data collection, as well as economic theory, to
cycle has been applied in Western economics to be informed by insights from anthropological
theories of consumption (life cycle theory is analysis.
COMBINING QUANTITATIVE AND QUALITATIVE 517
The paper of Venkatramani (1992) on the ternal one; Howard, 1994; Howard & Millard,
Kallars in Madurai district of Tamil Nadu put 1997). Howard (1994) describes how the rela-
this in a human perspective: tionship between birth order and traditional
land tenure systems can have consequences for
I killed my child to save it from the lifelong ignominy child health and survival. As the shortage of
of being the daughter of a poor family that cannot af- land is becoming increasingly acute, middle
ford to pay a decent dowry . . . But all the same, it was sons and daughters lose out on any inheritance.
extremely difficult to steel myself for the act. A mother
who has borne a child cannot bear to see it suffer even
So when times are hard parents must make a
for a little while, let alone bring herself to kill it. But I difficult choice between who to favor and who
had to do it, because my husband and I concluded to abandon. 20
that it was better to let our child suffer an hour or
two than suffer throughout life (p. 127).
(c) Mortality from medical and dietary beliefs
NOTES
1. For an elaboration of the ways in which quantitative stated that he had crossed out any sections of African
and qualitative techniques can be combined see Carv- poverty assessments referring to rural labor markets
alho and White (1996). since these did not exist.
2. An ethnology of disciplines could usefully explore 10. There are few systematic data to substantiate this
some oppositions—hard/soft, rigorous/arbitrary, sys- view (but there are some showing, for example, lower
tematic/impressionistic—including the macho language enrollments of orphans), but Howard and Millard’s
economists may use when talking about their own description of the death of the malnourished son of a
discipline. woman living in the compound of her well-off uncle is a
forceful illustration of the point (cf. Howard & Millard,
3. See Mukherjee, Wuyts, and White (1998) for a fuller 1997).
discussion of the philosophy underpinning data analysis.
11. The persistence of slavery is ignored in most
4. Quoted by Leamer (1983), attributed to Coase. poverty analysis. The tragic case of a slave ship off the
Leamer adds two more: ‘‘econometricians, like artists, West African coast at the time of writing the first draft
tend to fall in love with their models’’ and ‘‘there are two of this paper did little to help re-establish the issue on the
things you are better off not watching in the making: policy agenda. See www.antislavery.org.
sausages and econometric estimates’’ (1983).
6. This discussion ignores a number of other econo- 14. Model outcomes depend on whether a household
metric criticisms of the World Bank’s work, for which wishes to run down its assets in old age so that they
see Lensink and White (2000). are all consumed by the time of death, or if they wish
to make provision for later generations. These consid-
7. Or, in the extreme, simply falsify the data. One of erations are captured by overlapping generations
the best known examples of this is work by Burt to show models.
that IQ is hereditary rather than a function of upbring-
ing based on a study of twins separated at a young age
(see Joynson, 1989). Among the techniques he used to 15. See Whitehead (1990) for explicit discussion of the
establish this theory as accepted wisdom was to work implications of household structure for economic mod-
with younger researchers using data he provided, with eling.
the results published only under the name of the other
researcher. He then cited these papers as independent 16. Infants are those aged 12 months or less, and
verification of his views. The data, it transpired, were children those aged one to four years inclusive.
made up.
8. This section is partly based on White and Leavy 17. See Hanmer and White (1999) for a survey of such
(2001). papers.
9. A more extreme statement of this position was made 18. But, the possibility of determining sex before
by a former World Bank chief economist for Africa who birth means that discrimination can take place through
COMBINING QUANTITATIVE AND QUALITATIVE 521
sex-selective abortion. This has been the case for some 19. Official data in fact show an infant mortality rate
time in China, where abortion is readily available and of 1,200 per 1,000 live births!
shows up as the male–female ratio at birth being less
than the norm of 1.06. Croft shows that for mothers 20. This process is graphically described by Turnbull
with two or more children, all of which are female, the (1973) in his description of the response of the Ik to the
probability of a female fetus being aborted is very destruction of their hunting-based livelihood by the
high. creation of a National Park.
REFERENCES
Agnihotri, S. (2000). Sex ratio patterns in the Indian Hanmer, L., & White, H. (1999). Human develop-
population. New Dehli: Sage. ment in sub-Saharan Africa: the determinants of
Agresti, A., Booth, J., & Hubert, J. (2000). Random under-five mortality: with case studies of Zam-
effects modelling of categorical response data. So- bia and Zimbabwe. Report for Sida. The Hague:
ciological Methodology, 30, 27–80. ISS.
Ando, A., & Modigliani, E. (1963). The life cycle Harriss, J. (2002). The case for cross-disciplinary
hypothesis of saving. American Economic Review, approaches in international development. World
53, 55–84. Development, 30(3) (this issue).
Aunger, R. (1992). The nutritional consequences of Harriss-White, B. (2001). Development and productive
rejecting food in the Ituri forest of Zaire. Human deprivation: male patriarchal relations in business
Ecology, 20(3). families and their implications for women in South
Azevedo, M., Gwendolyn, P., & Lantum, D. (1991). India. Mimeo (abridged draft from Chapter 5 of
Culture, biomedicine and child mortality in Camer- How the Indian economy works, forthcoming from
oon. Social Science and Medicine, 32(12). Cambridge University Press).
Binswanger, H., McIntire, J., & Udry, C. (1989). Hill, P. (1975). The West African household. In J.
Production relations in semi-arid African agri- Goody (Ed.), Changing social structure in Ghana:
culture. In P. K. Bardhan (Ed.), The economic Essays in the comparative sociology of a new state and
theory of agrarian institutions. Oxford: Clarendon an old tradition. London: International African
Press. Institute.
Brenner, M. (1981). Social method and social life. Hill, P. (1986). Development economics on trial: The
London: Academic Press. anthropological case for a prosecution. Cambridge:
Burnside, C., & Dollar, D. (2000). Aid, policies, and Cambridge University Press.
growth. American Economic Review, 90(4), 847–868. Hopkins, A. G. (1973). An economic history of West
Carvalho, S. & White, H. (1996). Combining quantita- Africa. London: Longmans.
tive and qualitative approaches to poverty measure- Howard, M. (1994). Socio-economic causes and cultural
ment and analysis. World Bank Technical Paper No. expectations of childhood malnutrition among the
366. Washington, DC: World Bank. Chagga of Tanzania. Social Science and Medicine,
Cheater, A. P. (1984). Idioms of accumulation: Rural 38(2), 239–251.
development and class formation among freeholders in Howard, M., & Millard, A. (1997). Hunger and shame:
Zimbabwe. Gweru: Mambo Press. Child malnutrition and poverty on Mount Kilimanjaro.
Collier, P. & Dollar. D. (2001). Development effective- London: Routledge.
ness. What have we learned? Mimeo. Paper pre- Iliffe, J. (1987). The African poor. Cambridge: Cam-
sented to Experts’ seminar on aid effectiveness, bridge University Press.
selectivity and poor performers. Paris: OECD, Jan- Jackson, C. (2002). Disciplining gender. World Develop-
uary 17. ment, 30(3) (this issue).
Croll, E. (2000). Endangered daughters: Discrimination Joynson, R. B. (1989). The Burt affair. New York:
and development in Asia. London: Routledge. Routledge.
Falola, T., & Lovejoy, P. (1994). Pawnship in historical Kevane, M. (1994). Village labour markets in Shei-
perspective. In F. Toyin, & P. Lovejoy (Eds.), kan district, Sudan. World Development, 22(6),
Pawnship in Africa: Debt bondage in historical 839–857.
perspective. Boulder, CO: Westview. Leamer, E. (1983). Let’s take the con out of economet-
Gilbert, N. (Ed.). (1993). Researching social life. Lon- rics. American Economic Review, 23(1), 31–43.
don: Sage. Lensink, R., & White, H. (2000). Assessing aid: a
Grandin, B. (1989). Labour sufficiency, livestock man- manifesto for aid in the 21st century? Oxford
agement and time allocation on Maasai group Development Studies, 28(1).
ranches. Research in Economic Anthropology, 11, Mosley, P., Subasat, T., & Weeks, J. (1995). Assessing
143–178. adjustment in Africa. World Development, 23(9),
Hanmer, L., Pyatt, G., & White, H. (1999). What do the 1459–1473.
World Bank’s poverty assessments teach us about Mukherjee, C., Wuyts, M., & White, H. (1998). Econo-
poverty in Africa? Development and Change, 30, 795– metrics and data analysis for developing countries.
823. London: Routledge.
522 WORLD DEVELOPMENT
Narayan, D., Chambers, R., Shah, M., & Petesch, P. Termine, P. (2001). Using the Zambia living conditions
(2000a). Voices of the poor: Crying out for change. measurement survey to analyse rural labour markets.
New York: Oxford University Press for the World Mimeo. Paper presented at ESRC development
Bank. economics conference, Nottingham, April.
Narayan, D., Patel, R., Schafft, K., Rademacher, A., & Turnbull, C. (1973). The mountain people. London:
Koch-Schullem, S. (2000b). Voices of the poor: Can Cape.
anyone hear us? New York: Oxford University Press Vaughan, M. (1998). Exploitation and neglect: rural
for the World Bank. producers and the state in Malawi and Zambia. In D.
Narayan, D., & Petesch, P. (2000). Voices of the poor: Birmingham, & P. M. Martin (Eds.), History of
From many lands. New York: Oxford University Central Africa. Harlow: Longman.
Press for the World Bank. Venkatramani, S. H. (1992). Female infanticide: born to
den Ouden, J. H. B. (1995). Who’s for work? The die. In J. Bradford, & D. Russell (Eds.), Femicide:
management of labour in the process of accumula- The politics of killing women (pp. 125–132). Buck-
tion in three Adja villages, Benin. Africa, 65(1), 1–35. ingham: Open University Press.
Sahn, D., & Sarris, A. (1994). The evolution of states, White, H., & Leavy, J. (2001). Labour exchange in rural
markets, and civil institutions in rural Africa. Journal Africa: what do household models need to explain?
of Modern African Studies, 32(2), 279–303. Mimeo. Paper presented at ESRC development
Scheper-Hughes, N. (1992). Death without weeping: The economics conference, Nottingham, April.
violence of everyday life in Brazil. Berkeley, CA: White, H., Srinivasan, S. & Desmond, N. (2001).
University of California Press. Qualitative and quantitative approaches to poverty
Sen, A. (1990). More than a million women are missing. analysis: conflict, complementarity or synergy? Mi-
New York Review of Books, 20(December), 61–65. meo. The Hague: ISS.
Silverman, D. (1985). Qualitative method and sociology. Whitehead, A. (1990). Rural women and food produc-
Aldershot: Gower. tion in sub-Saharan Africa. In J. Dreze, & A. Sen
Silverman, D. (1993). Interpreting qualitative data. Lon- (Eds.), The political economy of hunger. Oxford:
don: Sage. Clarendon (three volumes).
Spielmann, K. (1989). A review: dietary restrictions on Zorn, C. (2001). Generalised estimating equation models
hunter-gatherer women and the implications for for correlated data: a review with applications.
fertility and infant mortality. Human Ecology, 17(3). American Journal of Political Science, 45(2), 470–490.