You are on page 1of 15

Methodology in Business Ethics Research: A Review D. M.

Randall
and Critical Assessment A. M. Gibson

ABSTRACT. Using 94 published empirical articles in depicted in Figure 1 are summarized. When weak-
academicjournals as a data base, this paper providesa critical nesses are noted, suggestions for a more defensible
review of the methodology employed in the study of ethical methodology are offered.
beliefs and behavior of organizational members. The review
revealed that full methodological detail was provided in less
than one half of the articles. Further, the majority of Methodology
empirical research articles expressed no concern for the
reliability or validity of measures, were characterized by low
An attempt was made to locate all published studies
response rates, used convenience samples, and did not offer a
theoretic framework, hypotheses, or a definition of ethics. that empirically investigated ethical beliefs and
Several recommendations, including a reviewer rating form behavior in organizations. Three computerized data
addressing methodological decisions and inclusion of meth- bases were searched: Management Contents, the Social
odologists on the review panel, are offered to improve &iences Citation Index (SSCI), and the BusinessPeriodi-
methodological rigor in published ethics research. cals Index (BPI). Management Contents indexes 500
U.S. and international journals on a variety of
business-related topics from 1974 to present. Using
At present, little is known about the quality of tides and descriptors, the following key words were
empirical research on the ethical beliefs and conduct searched: ethical conduct, ethical decision(s), and
of organization members. The purpose of this paper ethical situation(s). The SSCI contains over one
is to provide a critical review of the methodology in million citations from articles in key social science
business ethics articles and to suggest methodological journals and selected journals in related fields from
refinements. Using published empirical articles in
academic journals as a data base, methodological • Theory
decisions made by business ethics researchers in each HypOthesis
of the eight major stages of the research process Formation

Conceptualization ~ 1niceof Popula~onand


! ResearchMethod t Sampling
Donna M. Randall is an Associate Professor in Management and

I
Systems at Washington State University. Her research interests
include organizational commitment, media coverage of white- Operatio!nalizario/
collar and corporate crime, and ethical issues in management. Her
publications have appeared in such journals as Decision Observation
Sciences, Academy of Management Review, and Social
Science Quarterly.
Annetta M. Gibson is a doctoral student in accounting in the Data Pr!cessing
Department of Accounting and Business Law at Washington and Analysis
State University. She has a CPA and has worked as an auditor
for a number of years. Her research interests include impression * Adapted from Babble (1986).
management and ethical issues in auditing, accounting, and
management. Fig. 1. An overview of stages in the research process*

Journal of Business Ethics 9:457--471, 1990.


© 1990 KluwerAcademic Publishers. Printed in the Netherlands.
458 D. M. Randall and A. M. Gibson

1972 to present. Using tides and descriptors, the seek to generalize to a larger population? What
following key words were searched: ethical conduct, observation techniques were used? What statistical
ethical decision(s), ethical situation(s), business ethics, techniques were used to analyze the data?
managerial ethics, ethical beliefs, ethical conflict, Further, using Cabell's Director), of Publishing
ethical judgment, ethical attitudes, ethical percep- Opportunities in Business and Economics (1988 edition),
tions, ethical standards, unethical behavior, unethical data were gathered on the acceptance rate of each
decision(s), immoral behavior, managerial morality, journal publishing business ethics research (reflected
organizational ethics, and business philosophies. The in Table I). When a range of acceptance rates was
BPI was accessed on CD/Rom using a Wilsondisc provided, the mean of that range was used to calcu-
PC-Rom information retrieval system through the late the acceptance rate. Due to a skewed distribu-
key word, "business ethics." The data base indexes tion, the median acceptance rate was used to classify
304 periodicals on virtually every subject area related acceptance rates of the journals as high (26% or
to business from 1982 to the present. above) or low (25% or lower). As analysis revealed
In addition to the three computer searches, The that including studies from journals with nonavail-
Readers Guide to Periodical Literature from 1960 to able acceptance rates (18 studies) into the high
present and the Business Periodicals Index from 1958 acceptance rate category did not significantly change
to the present were searched manually using "busi- the findings, all subsequent analyses were run
ness ethics" as a key word. The Readers Guide to excluding the unrated studies.
Periodical Literature is a cumulative subject index to The code sheet was pretested for comprehensive-
approximately 175 periodicals published in the U.S. ness and clarity. Ten articles were coded independ-
Finally, bibliographies in all relevant articles were endy by the first and second authors. After agree-
reviewed to identify additional citations not obtained ment was reached on coding discrepancies and
through the computer or manual searches. refinements, all 94 articles were coded onto a revised
Each article was screened using clearly defined, code sheet by the second author. Using a random
prespecified criteria. Each article was reviewed to numbers table, the first author selected 15% (14
determine if: (1) the article contained an empirical articles) and independently coded the articles. The
research study, and (2) the study dealt with ethical interrater reliability between the two authors on the
beliefs (defined by Bowman, 1976, as judgments subsample of articles was 97.1%.
about what is right and wrong and whether or not
these judgments are good or bad) or ethical behavior
(defined by Runes, 1964, as '~ust" or "right" stand- Overview o f articles
ards of behavior between parties in a situation) of
organizational employees (employees, peers, or man- Empirical research on ethical beliefs and behavior in
agers). As the focus of the study was on ethical beliefs business organizations spans three decades beginning
and behavior, related articles on social responsibility, with a well-known study of attitudes of executives
corporate codes of conduct, and whistle-blowing toward business ethics (Baumhart, 1961). As Figure 2
were excluded from the study. illustrates, academic interest in business ethics re-
Of the over seven-hundred citations obtained, 94 mained fairly constant until the mid 1970s. The
articles met these two criteria and were included in 1980s reflected great interest in business ethics with
the study)' 2 Each of the 94 articles was analyzed in over 60% (66) of the articles published during this
depth. A code sheet was developed which consisted period. Interest has been particularly strong in recent
of a series of questions regarding key components of years, as reflected by the peak of published articles in
the research process depicted in Figure 1. Such 1987 and 1988.
questions included: Was a theory presented? Were Over the past three decades, a surprising variety
hypotheses specified? Were key constructs clearly of journals have published research on business
defined? Were the instruments used to measure the ethics. As reflected in Table I, the most popular
key constructs reliable and valid? Were the instru- publication outlet is the Journal of Business Ethics,
ments pretested? What research design was selected? publishing over one third (32) of the empirical
How was the sample selected? Did the researchers articles on ethical beliefs and behavior of organiza-
Methodology 459

TABLE I
Journals publishing articles on ethical beliefs and behavior in business

g¢ of acceptance
Name of journal articles (%) rate*

1. Journal of Business Ethics 32 34.0% 21--30%


2. Journal of Marketing 5 5.3% 10% or less
3. Akron Business and Economic Review 3 3.2% 11--20%
4. Business Horizons 3 3.2% 20--25%
5. Harvard Business Review 3 3.2% 5% or less
6. Journal of Applied Psychology 3 3.2% 6--10%
7. Journal of Business Research 3 3.2% 6--10%
8. Psychological Reports 3 3.2% na
9. Journal of Retailing 2 2.1% 21--30%
10. Management Review 2 2. I% 6--10%
11. Business and Society Review 2 2.1% 10%
12. Journal of SmallBusiness Management 2 2.1% 21--30%
13. Human Relations 2 2.1% na

14. Journal of Academy of Marketing Science 2 2.1% 11--20%


15. Journal of Purchasing and Materials Management 2 2.1% 21--30%
16. MSU Business Topics 2 2.1% na
17. Personnel Journal 2 2.1% 6--10%
18. California Management Review l 1.1% 6--10%
19. Public Personnel Management Journal 1 1.1% 11--20%
20. Academy of Management Journal 1 1.1% 6--10%
21. Atlantic Economic Review 1 1.1% na
22. Conference Board Record 1 1.1% 15--18%
23. Industrial Marketing Management 1 1.1% 11--20%
24. Journal of Advertising 1 1.1% 6--10%
25. Organizational Dynamics 1 1.1% 6--10%
26. Personnel Administration 1 1.1% na
27. Purchasing 1 1.1% na

28. Review of Business and Economic Research 1 1.1% 11--20%


29. Supervisor/Management 1 1.1% na

30. Journal of Purchasing 1 1.1% na


31. Supervision 1 1.1% 50% or more
32. Business and Society Review 1 1.1% 10%
33. Direct Marketing 1 1.1% na

34-. Canadian Business 1 1.1% na

35. European Journal of Marketing 1 1.1% na

36. Southern Business Review 1 1.1% ng

37. Management Decision 1 1.1% na

38. Marketing Education 1 1.1% na

Total 94 100.0%

* As reported in Cabell's DirectoryofPublishing Opportunitiesin Businessand Economics, 1988 ed.; na - not available.
460 D. M. Randall and A. M. Gibson

16

14

12

!-,,4
m ~

o 8 B m

Q.)
,.O
6

21
0
1961 1963 1965 1967 1969 1971 1973 1975 1977 1979 1981 1983 1985 1987 198~
Year of Publication
* Incompletedata available
Fig. 2. Businessethics articlesby year

tional members. With the exception of the Journal of ical study is essential for an accurate appraisal of the
Marketing, each of the remaining journals referenced research findings. Without full disclosure of research
in Table 1 published a maximum of three relevant methods, it will be difficult to evaluate the signifi-
articles. cance of the research findings and dangerous to draw
any practical implications from the study.
From the methodological description which was
Methodological critique provided, the following conclusions about empirical
research on ethical beliefs and conduct can be
While one might anticipate that empirical articles drawn:
published in academic journals would provide fairly
detailed methodological description, the review of
articles revealed that this was not the case. There was Theoreticframework
a surprizing amount of missing detail. Full method-
ological description appeared in less than one half Theory is a necessary component of the research
(45) of the 94 articles. While several studies neglected process. Positivists advocate a deductive approach to
to report only one or two methodological details, theory development in which a theoretic framework
other studies (e.g., Allen, 1980; Bezilla et al., 1976) is the appropriate starting place for empirical re-
reported only two methodological details - who was search. Once a theoretic base has been established,
sampled and the number of participants in the study business ethics researchers can make predictions
- without any other methodological description. about what beliefs and behavior would likely occur
A complete disclosure of methods in any empir- under given conditions, logically develop a research
Methodology 461

program to examine those predictions, interpret TABLE II


findings in light of the specified theory, and revise Summary of methodological decisions
the theory, if necessary. The business ethics articles
reviewed were characterized by a distinct absence of Tkeoreticframework ff~ %
theory. As reflected in Table II, 64% (60) of the no theory development 60 64
studies did not cite any previously established unclear if theory present 2 2
theoretic framework nor seek to develop one. weak theory development 8 9
In an effort to move beyond descriptive research, strong theory development 24 26
it is desirable that researchers specify a theoretic Hypothesis offered
framework. Researchers need not develop their own no 71 75
theoretic framework; a large number of theoretic yes 23 25
models currently await empirical testing (e.g., Bom- Conceptualization
mer et al., 1987; Trevino, 1986). Empirical studies by Definition offered to readers of article 21 22
Hegarty and Sims (1978) and Zey-Ferrell and Ferrdl Definition offered to study participants 7 7
(1982) provide excellent examples of how previously
existing theoretic frameworks can be used to guide
Operationalization
concern for reliability of instrument mentioned 18 19
empirical research on business ethics. concern for validity of instrument mentioned 18 19
instrument pretested 19 20

Hypotheses Population and sampling decisions


Study population
practicing managers and professionals 63 67
Once a theoretic framework has been established, students (graduate and undergraduate) 24 26
researchers typically develop and specify hypotheses combination of managers and students 7 7
or propositions predicting a particular relationship
between two or more variables, just as few empirical Sample design
articles on ethical beliefs and behavior presented a design not specified 13 14
other 11 12
theoretic framework, few presented testable hy-
convenience sampling 39 42
potheses (Table II). random sampling 31 33
As business ethic researchers move beyond ex-
ploratory research, it will be necessary to develop Response rate
specific research hypotheses. Hegarty and Sims not reported 10 11
(1978, 1979) aptly illustrated how testable hypothe- below 10% 0 0
10-19% 2 2
ses can be derived from a theoretic framework.
20-29% 10 11
30-39% 8 9
40-49% 10 11
Conceptualization 50-59% 4 4
60-69% 4 4
The conceptualization phase of the research process above 70% 4 4
involves defining the meaning of theoretic con- not applicable 42 45
structs. This stage is particularly troublesome in
Research design
ethics research as there is little consensus regarding survey research 76 81
what constitutes "ethical" beliefs or "ethical" behav- lab experiment/simulation 6 6
ior in an organizational setting (Baumhart, 1961; in-person interviews 4 4
Ferrell and Gresham, 1985; Vitell and Festervand, in-person interviews and survey 3 3
1987). Not only are ethical standards constandy not reported 5 5
changing over time, but they vary from one situation
Analytic teclmiques
or organization to another (Barnett and Karson, univariate techniques 33 35
1987; Ferrell and Gresham, 1985). As an illustration bivariate techniques 43 46
of the conceptual confusion characterizing business multivariate techniques 18 19
ethics research, Lewis (1985) identified 308 different
462 D. M. Randall and A. M. Gibson

definitions of the term "business ethics" in 254 by using interrater reliability checks, while Dubinsky
articles, books, and textbooks. and Ingram (1984) reported the Cronbach's Coeffi-
Rather than offering a wide variety of definitions cient alpha for all scales used in their research.
of ethical beliefs and behavior, the majority of Similarly, in seeking to enhance the content validity
business ethics articles reviewed offered no defini- of their instrument, Fritzsche and Becker (1984)
tion of ethical beliefs or conduct to study partici- developed a number of vignettes in each of five
pants or readers of the article (Table II). For instance, representative categories of ethical problems.
Allen (1980) asked survey respondents to indicate Pretesting can help increase the reliability and
which organizations and institutions were likely to validity of measures (as well as encourage scrutiny of
encourage poor ethics on the part of their members, question wording, question order, redundant ques-
without defining "poor ethics" or asking respondents tions, missing/inappropriate questions, confusing
what "poor ethics" meant to them. Such an approach response categories, and poor scale items (Bailey,
allows readers and study participants to create and 1982)). Yet, no pretest was conducted in 78% (73) of
respond to their own definitions of ethical beliefs the business ethics articles. While some of these
and conduct. articles (23) had adopted measures from other
To build a cumulative data base of business ethics studies, 50 of the articles utilized newly developed
research, key constructs in business ethics research measures without any reported pretest.
need to be clearly defined. Without a commonly When pretests were conducted they were appro-
shared understanding of what is meant by ethical priately conducted in the target population. For
beliefs and conduct, these terms will be ultimately example, Fritzsche and Becker (1984) pretested
defined by study participants and their definition vignettes on a sample of marketing managers (the
will remain unknown to researchers and readers of relevant population) by drawing a random sample
business ethic articles. Examples of clearly stated from a local chapter of the American Marketing
definitions of ethical beliefs and conduct can be Association. Similarly, Dubinsky and Gwin (1981)
found in Bowman (1976), Brown and King (1982), pretested descriptions of ethical situations with the
Brenner and Molander (1977), and Browning and assistance of purchasing and field sales personnel
Zabriskie (1983). across several industries (the relevant population).
In sum, business ethics researchers need to develop
reliable and valid instruments, to conduct pretests of
Operationalization those instruments, and to continue to conduct
pretests in the target population. The reliability of
Operationalization decisions involve timing reliable the instruments can be fairly easily assessed using
and valid measures for theoretical constructs. The measures of interrater reliability, internal consis-
articles reviewed reflected surprisingly little concern tency, or test-retest reliability. Whereas the validity
for either the validity or reliability of the research of instruments is typically more difficult to ascertain,
instruments (Table II). It appears that most business possible options might include the use of a valida-
ethics researchers assumed that their instruments tion procedure set forth by Churchill (1979), a
were reliable and had face validity. Unfortunately, as multitrait-multimethod matrix (Campbell and Fiske,
little consensus exists about the definition of ethical 1959), or a nomological net (Cronbach and Meehl,
beliefs and conduct, one researcher's judgment of 1955).
validity of an instrument may not be the same as
another's. In addition, if ethical beliefs and conduct
are multidimensional constructs, face validity will Population and sampling decisions
not assure that an instrument taps all relevant
dimensions. Population and sampling decisions concern identifi-
Some researchers did demonstrate a concern for cation of the group about which conclusions are to
the reliability and validity of the research instru- be drawn and the group that will be observed for
ments. For instance, Chonko and Hunt (1985) that purpose. Population aM sampling decisions
sought to enhance the reliability of their instruments generally involve four issues: who is sampled, what
Methodology 463

type of sample is drawn, what an acceptable response highly appropriate. In those instances where re-
rate is, and what size of sample is needed. searchers are interested in exploring the ethical
decision-making process, student samples can be
Study population. The bulk of articles on ethical substituted without a major threat to generalizability.
beliefs and conduct used practicing managers and
professionals as a sample (see Table II). It was Sampling design. Once a study population was selected,
particularly interesting to note the heavy reliance business ethics researchers relied primarily upon
upon samples of marketing managers. Over one- random or convenience sampling to select individ-
quarter (25) of the 94 articles explored ethical uals for inclusion in the study (Table II). Historically,
practices in marketing-related professions (sales, random samples have been preferred over conven-
purchasing, and advertising). This was undoubtedly ience samples as they offer the best assurance against
due to the belief that marketing managers encounter sampling bias (e.g., Lazerwitz, 1968). Convenience or
more situations that result in decisions with ethical "captive audience" samples offer no assurance of
impact than do executives in other functional areas representativeness and do not permit generalization
of the firm (Chonko and Hunt, 1985; Dubinsky and to a larger population. (However, when expert
Gwin, 1981; Dubinsky and Levy, 1985; Levy and opinion surveys are conducted (e.g., Schutte, 1965;
Dubinsky, 1983; Trawick and Darden, 1980). Stoffman, 1987), a convenience sampling design is
It was also surprising to note that relatively few highly appropriate.)
business ethics articles used student samples. Whereas In business ethics research, generalizations from
the use of student samples has been heavily criticized convenience samples to a larger population were
in organizational research due to problems of frequently made. For instance, Rizzo and Parka
generalizability (see Rosenthal and Rosnow, 1975), (1981) generalized from a convenience sample of
student samples are appropriate if they comprise the middle managers in public agencies in south Florida
population of interest or if the population of interest to all public managers; Trawick and Darden (1980)
is similar to the student sample on theoretically generalized from a convenience sample of marketing
relevant variables and the study is part of a research practitioners and educators (with an unspecified
program investigating generalizability to the popula- selection strategy) to all practitioners and educators;
tion of interest (Gordon et al., 1987). Furthermore, and, Dubinsky and Ingam (1984) similarly gener-
while studies examining attitudes have found size- alized from a convenience sample of salespersons
able discrepancies between students and other sub- (with an unspecified selection strategy) to all sales
jects, studies focusing on decision-making have personnel.
found considerable similarities in the decisions and Ideally, business ethics researchers should seek to
assumed behavior of student and nonstudent sam- decrease their reliance on convenience samples and
ples (Ashton and Kramer, 1980; Remus, 1986). increase the use of random samples. Random sam-
On those occasions when students were used as pies can be drawn using such techniques as member-
subjects in business ethics research, the purpose of ship lists of professional associations or a directory
the study was not to generalize to managers, but to of manufacturers. As random sampling may not
determine attitudes held specifically by business always be feasible due to time and resource con-
students toward ethical dilemmas (e.g., Beltrami et straints, at minimum, generalizations from conven-
al., 1984;Jones and Gautschi, 1988), the influence of ience samples to the larger population need to be
individual difference variables on ethical decision- curtailed.
making (e.g., McNichols and Zimmer, 1985; Tsalikis It was interesting to note that managerial and
and Nwachukwsu, 1988), the impact and effective- student samples are associated with distinct sampling
ness of business ethics courses (e.g., Hawkins and designs. A majority of the 63 studies involving
Cocanougher, 1984; Martin, 1982), and the willing- managers or professionals as the study population
ness of students to engage in unethical practces (e.g., were conducted using a random sampling technique,
Rosenberg, 1987; Worrell et al., 1985). through mail surveys, and had a mean sample size
Hence, the use of managerial samples for under- of 411. On the other hand, a majority of 24 studies
standing the ethical beliefs of managers appears to be involving students as the study population selected
464 D. M. Randall and A. M. Gibson

respondents through convenience sampling, utilized research is troubling as the data may not be repre-
questionnaires administered in a research setting, sentative of the population surveyed. For most
and had a mean sample size of 560. studies, Babble (1986) advised that a response rate of
at least 50% is adequate for analysis and reporting, a
Sample size. A large sample size can help minimize response rate of 60% is good, and 70% or more is very
sampling error. Many researchers view 100 subjects good. However, in research on sensitive topics, a
as a minimum sample size (Bailey, 1982). If one response rate of even 70% may be a severe problem,
compares sample sizes in business ethics research as nonrespondents are more likely to be similar in
against this standard, ethics research clearly surpasses important characteristics, thus introducing substan-
the minimum. The mean number of'subjects in the tial nonrandom error. Moreover, factors such as cost
studies reviewed was 434, with a range from 4 to of reducing the nonresponse bias need to be taken
2856 subjects. As the mean was skewed by four into consideration.
particularly large samples, the median of 196 pre- The typical response rate to surveys in the
sents a better estimate of typical sample size. business ethics articles was low (Table II). The mean
However, adequacy of sample size is not simply a response rate was 43%, ranging from 10% in a
function of the number of subjects, but rather Business and Society Review survey (1975) to 96% in a
depends on such factors as how the respondents study by McNichols and Zimmer (1985), with a
were selected (random or convenience), the distribu- median of 40%. A large number of factors may
tion of the population parameter (the variable of contribute to the low response rate. Bailey (1982)
interest), the purpose of the research project (ex- contended that response rates to social science
ploratory or applied), and the intended data analytic research are declining due to saturation. However,
procedures (to ensure adequate cell sizes for statisti- the articles reviewed revealed an insignificant rela-
cal analysis). Despite the importance of such factors, tionship between time and response rate to ethics
they were infrequently taken into consideration in research (r = -0.08,p > 0.31, n = 42).
the determination of appropriate sample size. It The cause of the low response rate to surveys in
appears, as is true of most social science research business ethics research probably lies more in the
(Simon and Burstein, 1985), that the sample size was nature of the research topic. Business ethics is a
fixed by the amount of money available or the particularly delicate topic to research. Managers may
sample size that similar pieces of research used in the not want to have their "ethics" observed or measured
past (e.g., Krugman and Ferrell, 1981; Posner and directly (Trevino, 1986), and few employees may be
Schmidt, 1987). For example, although Hogarth willing to divulge information to researchers that
(1978) determined that under certain conditions might be incriminating to them or to their friends.
groups with 8 to 12 experts have predictive ability Thus, people willing to participate in an experiment
close to the optimum, in the two surveys of expert or survey about ethics might have very different
opinion (Schutte, 1965; Stoffman, 1987), a limited attitudes than people more reticent to respond.
number of experts - seven and four, respectively - Due to the importance of acquiring an adequate
were approached. response rate, business ethics researchers may want
It would be desirable for business ethics research- to consider three suggestions. First, they may want to
ers to consider more carefully the issue of sample use "stronger" techniques to encourage a higher
size. Attention to the purpose of the research, response rate. These techniques might include
sampling design, and likely data analytic techniques personal interviewing (which typically has a higher
can aid in this effort. A pretest can help determine response rate than survey research), targeting and
how the population parameter is distributed in the intensively surveying a subpopulation (for a more
population. Such techniques as a Monte Carlo personalized appeal), persuading respondents of the
experiment or step-wise sample-size determination importance of the research topic and their candid
(Simon and Burstein, 1985) may also aid in deter- answers (in the cover letter as well as on the ques-
mining appropriate sample size. tionaire), or flee gifts provided with the survey (such
as financial incentives or donations to charities) (see
Response Rate. A low response rate in any type of Dillman, 1978, for a more complete discussion of
Methodology 465

these and other techniques to enhance the response would behave in that situation. Most often respond-
rate). ents were presented with 12 scenarios (with as few as
Second, researchers could seek to determine if one scenario and as many as 37), with each scenario
nonrespondents differ in any significant way from averaging five sentences in length.
respondents. For example, Armstrong and Overton The key problem with both the direct question
(1977) discussed the feasibility of using subjective and scenario formats is vagueness and generality.
estimates and extrapolation methods to estimate The problem situation is described so briefly to the
uonresponse bias. Of the business ethics studies respondent that it is difficult for him or her to
reporting a response rate less than 60%, only one evaluate and for the researcher to attain any reason-
study (Chonko and Hung, 1985) compared demog- able degree of within-subject reliability. Ethical
raphics of respondents and nonrespondents to assess choice is commonly believed to be situationally
the possibility of a response bias. A number of specific; the "best" action is not based on pre-existing
other studies (e.g., Browning and Zabriskie, 1983; values, but upon specifics of the action choice
Carroll, 1975; Fritzsche and Becker, 1984; Vitell and (Barnett and Karson, 1987). Contextual variables,
Festervand, 1987) cited extensive demographic detail such as the influence and actions of peers and
on the respondents, but failed to state how those data managers, available rewards and punishments, and
compared with characteristics ofnonrespondents. the probability of detection, may be essential pieces
Finally, when a low response rate is obtained and of information for realistic decision-making.
comparable data on respondents are not available, Two examples from the literature illustrate the
researchers may want to warn readers not to gener- problem of ambiguous and vague questions and
alize beyond the sample. For instance due to an scenarios. Using a previously developed instrument,
unexpectedly low (40%) response rate, Lincoln et al. Norris and Gifford (1988) asked respondents to
(1982) cautioned readers not to generalize to firms express their agreement or disagreement with the
beyond those in the sample. However, even if a action taken in the following situation: "A customer
caveat is incorporated into the article, one might calls the retailer to report that her refrigerator
question the utility of the study and its contribution purchased two weeks ago is not cooling properly and
to the literature. that all the food has spoiled. Action: The customer
should be reimbursed for the value of the spoiled
food." Hawkins and Cocanougher (1972) asked
students how ethical or unethical the following
Selection of researchdesign situation was: "A large manufacturer of technical
equipment requires that all its salesmen dress 'con-
As reflected in Table II, survey research was clearly
servatively' in dark suits with white shirts." When
the most common research design used by business
provided with such one-sentence descriptions and
ethics researchers (81% of the studies). Other designs,
no other contextual information, the only defensible
utilized much less frequently, included lab experi-
answer to the question, "What would you do?" is
ments and simulations (6%), in-person interviews
probably, "It depends." (Frederiksen, Jensen, and
(4%), and a combination of in-person interviews and
Beaton, 1972).
surveys (3%).
Scenarios and questions clearly need to be devel-
oped with a greater concern for realism. Fredrickson
Survey research. Survey research used either a direct (1986) suggested a detailed methodology for devel-
question format or scenarios. When a direct question oping scenarios in strategy research which can also
format was used, researchers typically asked re- be very appropriate in business ethics research.
spondents how they felt about a particular issue or if Structured interviews are conducted to identify
they would engage in a particular practice. Respond- issues facing the industry and possible actions on the
ents were presented with an average of 19 questions part of respondents. A scenario is developed using
one sentence in length. When scenarios were used, industry-specific terminology, usually with enough
respondents were typically presented with a hypo- detail for five, single-spaced pages. A pilot test is
thetical situation and asked to indicate how they then conducted to ensure that the scenario and
466 D. M. Randall and A. M. Gibson

accompanying questions are understandable and an ships and the control of exogenous variables. Within
accurate portrayal of the situation. The procedure the context of a lab experiment, in-basket exercises
provides all respondents with a standardized stimu- (e.g., Laczniak and Inderrieden, 1987) and computer
lus, sets forth a reasonably realistic, detailed situa- simulations (e.g., Hegarty and Sims, 1978) can be
tion, and the situation is written in such a way that it used.
generates the respondent's interest and therefore Despite the advantages of lab research, at present
"involvement" (Fredrickson, 1986). little is done from an experimental design point of
In addition to the problem of vague scenarios and view (Laczniak and Inderrieden, 1987) (also see
questions, a secondary problem accompanying sur- Table II). While only six of the 94 articles utilized a
vey research methodology relates to the heavy use of laboratory design, these articles demonstrate the
close-ended quer4ions. Respondents are frequently potential value of laboratory research. For instance,
asked to describe what they would do in a given using a computer simulation, Hegarty and Sims
situation by responding to close-ended questions. (1978) measured the effects of potential rewards and
The respondent does not have to devise a solution - punishments on students paying kickbacks in simu-
he or she merely has to evaluate those presented to lated business situations. The experimental design
him or her (Frederiksen et al., 1972). Problems do not consisted of a 3 × 3 factorial analysis of variance
typically present themselves in multiple-choice with selected personality and demographic variables
form. Sometimes it is necessary to invent a solution as co-variates. Subjects entered decisions into an
rather than to choose one of several presented on-line time-shared computer program, allowing
(Frederiksen et al., 1972). Furthermore, close-ended experimental control over key variables and an
questions are most appropriate when researchers assessment of causal relationships.
have a well-defined issue and know precisely what
dimension of thought they want the respondent to In-person interviews. In-person interviews can be very
use in providing an answer (Dillman, 1978). As useful in exploratory stages of a research project for
business ethics research is in an embryonic stage developing an understanding of a phenomenon.
(Laczniak and Inderrieden, 1987), feasible behavioral However, only seven of the 94 articles used in-
choices remain loosely defined. person interviews as a research design. Four of these
Despite the recognized difficulty of evaluating articles relied solely upon in-person interviews for
essay answers and the possible biasing effect of verbal data collection (Bird and Waters, 1987; Mayer, 1988;
facility (Frederiksen et al., 1972), a free-response Waters and Bird, 1987; Waters et al., 1986), while the
format to questions and scenarios may be superior to three other studies used a combination of in-person
the close-ended format typically used in business interviews and survey research (Avlonitis, 1983;
ethics research. However, the review of empirical Rudelius and Buchholz, 1979; Schutte, 1965).
research revealed that a free-response format was For in-person interviews to be useful, a complete
used in only five articles (Fritzsche and Becker, 1984; description of findings is needed. All four studies
Laczniak and Inderrieden, 1987; Mayer, 1988; Posner relying exclusively on in-person interviews con-
and Schmidt, 1987; Stoffman, 1987). tained an extensive discussion of the findings from
In brief, the questions and scenarios used in open-ended interviews. Of the studies using a com-
survey research are typically too vague and lack bination of in-person interviews and surveys, Schutte
realism, while the close-ended questions force re- (1965) and Avlonitis (1983) also provided a fair
sponses into pre-defined categories. Fredrickson's amount of methodological detail and discussion of
article (1986) provides a detailed description of how findings from the personal interviews. However, the
more valid questions and scenarios can be developed, remaining study (Rudelius and Buchholz, 1979)
and the use of more open-ended questions can help offered little detail about who and how many
overcome difficulties posed by the use of close- individuals were interviewed and the results of those
ended questions. personal interviews. In brief, in-person interviews
can serve a valuable role in exploratory research on
Laboratory designs. Unlike survey research, laboratory ethical beliefs and conduct if findings are sufficiently
research permits the examination of causal relation- detailed.
Methodology 467

Observation and stressing that there are no right or wrong


answers) will typically not eliminate the influence of
In business ethics research, self-report data are respondents' desire to place themselves in a favorable
almost invariably used as an observation technique as light. However, steps can and should be taken to
few alternative techniques exist. In fact, the review of reduce this bias. For example, the Crowne-Marlowe
empirical research revealed that self-report data Social Desirability (SD) Scale (Crowne and Marlowe,
were relied upon in almost 90% of the research 1964), used by Stevens (1984), can help detect a social
articles (all research designs involving surveys or desirability bias. In addition, Nederhof (1985) has
interviews). provided a comprehensive review of techniques to
Unfortunately, major differences often occur reduce or prevent social desirability bias.
between what people say they would do and what
they actually do. The use of self-report data raises
problems in that respondents may organize their
responses in light of what they feel "others" will
Analytic techniques
expect is appropriate for someone like them in a
particular kind of situation. Somewhat apprehensive Data processing and analysis refer to the transforma-
about the confidentiality of their responses and the tion of data into a form suitable for analysis and
relationship of researchers to managers, respondents drawing conclusions regarding the specified theory
may be sensitive when answering questions about or hypotheses. One data analytic technique is not
their ethical beliefs and conduct. necessarily superior to another; the appropriateness
The tendency of respondents to deny socially of the technique depends primarily upon the theory,
undesirable traits, to claim socially desirable ones, research hypotheses, and available data. In descrip-
and to say things which place the speaker in a tive studies and especially exploratory ones, uni-
favorable light has been termed a "social desirability" variate statistics (means, modes, percentages, or
bias (Nederhof, 1985). A social desirability bias may standard deviations) may be in order. If the interrela-
cause individuals to overreport actions that are tion of two or more variables is the object of interest,
perceived to be socially desirable ("ethical" conduct) a bivariate or a multivariate presentation of data may
and to underreport behavior ("unethical" conduct) be more appropriate.
perceived to be socially undesirable. It is also possible The business ethics articles reflected a heavy
that respondents may overstate the difference be- reliance on univariate and bivariate statistics (Table
tween their ethical beliefs and actions and the beliefs II). Thirty-five percent (33) of the articles reported
and actions of others. only univariate statistics (with 27 articles reporting
A number of empirical research articles have only percentages, three articles reporting only means,
employed questions that are highly susceptible to a and two articles reporting only rank orders). Forty-
social desirability bias. For instance, Brown and King six percent (43) of the articles used some form of
(1982) asked respondents to indicate agreement or bivariate statistics (primarily t-tests), and 19% (18) of
disagreement with the following statement, "Ethical the articles used some form of multivariate analysis
practices are good business in the long run." When (e.g., analysis of covariance, multiple regression, or
asking respondents what they would do in a par- loglinear logit analysis.)
ticular situation, Krugman and Ferrell (1981) gave Without complete methodological detail and an
respondents such options as, "Manipulating a situa- understanding of the purpose of the research project,
tion to make a subordinate look bad," or "Divulging it is difficult to specify which form of statistical
confidential information." The wording of such analysis should have been used. However, one might
statements would clearly seem to encourage a social anticipate that as research on business ethics pro-
desirability bias. gresses, the reliance on multivariate statistics will
It is admittedly difficult to counter social desira- increase. Indeed, a strong empirical relationship
bility biases. Phillips and Clancy (1970) contended between the passage of time and use of multivariate
that the usual precautions for eliminating the statistics (r = 0.39, p < 0.001, n = 94) can be
possible effects of the bias (e.g., assuring anonymity detected.
468 D. M. Randall and A. M. Gibson

Recommendations and conclusions business ethics researchers may simply need to apply
more conscientiously and to report more fully
After conducting structured personal interviews methodologies currently in use.
with more than 50 scholars in the area of ethics, As summarized in Table III, the critical review of
Fleming (1987) concluded that researchers are some- methodological decisions has identified key weak-
what dissatisfied with the present methodologies and nesses, strengths, and areas in need of improvement
are searching for new ones. From the review of in business ethics research. Unfortunately, Table III
empirical research, it appears that the dissatisfaction reveals that decisions made early in the research
is jusffied; the methodology in business ethics process (specification of theory, development of
research is clearly in need of improvement. However, hypotheses, conceptualization decisions, and opera-
in addition to searching for new methodologies, tionalization decisions) are most susceptible to criti-

TABLE III
Relative strengths and weaknessesof research decisions

Research stage relative strength possible improvements

Theory development weak Specificationof theory


Hypotheses weak Specification of hypotheses
Conceptualization weak Offer definition of ethical
conduct to reader and study
participants
Operationalization
reliability weak Use more reliable instruments
validity weak Use more valid instruments
pretesting weak Use more pretests
Population and
sampling decisions
study population strong Managerial samples for
beliefs; student samples
acceptable for decision-making
sample design moderate Less reliance on convenience
samples
sample size moderate Consider population
parameters, design, purpose
and analysis
response rate weak Use techniques to enhance
response rate
Selection of the research
design
survey research weak Need more specificand
realistic questions and
scenarios
laboratory research strong Continue to use lab and
simulations
in-person interviews moderate Need fuller findings and
description
Observation weak Reduce socialdesirability
bias
Data analysisand processing moderate Less reliance on univariate
statistics
Methodoloaoy 469

cism, establishing a shaky foundation for decisions approach vs. 30% of the articles published in journals
made in subsequent steps of the research process. with high acceptance rates).
Thus, methodological improvement in these initial In sum, while it would be desirable to make all of
steps in the research process should have the highest the refinements suggested above, it is important to
priority at this time. recognize that research in any area involves trade-
To encourage more methodological rigor in offs between the desirable and the practical (Bailey,
empirical research, editors of journals publishing 1982). Due to difficulties presented by such a
empirical business ethics research may consider sensitive research topic as business ethics, researchers
adopting a structured referee rating form asking may never be able to approach the study of business
reviewers to assess the methodological rigor of the ethics with an "ideal" scientific research process.
study. Such a form would not supplant traditional However, with complete reporting of methodologi-
substantive reviews; it would be provided as supple- cal decisions and the consideration of as many
mental to the open-ended responses typically solic- refinements in the review process as possible, a solid
ited from reviewers. methodological foundation can be established and
Editors of journals might also invite a meth- that foundation can make a substantial contribution
odologist in addition to substantive reviewers to to understanding and improving the ethical climate
comment on papers submitted for review, ask that of organizations.
one of the substantive reviewers specifically review
the paper for methodological adequacy, or solicit
commentaries to be published along with the paper. Acknowledgement
Such modifications in the review process should
encourage greater attention to and consideration of The authors wish to thankJ. Scott Armstrong for his
methodological issues on the part of potential comments on an earlier version of the manuscript.
authors, reviewers, and editors.
While all empirical articles need to demonstrate a
concern for more methodologically rigorous articles, Notes
TheJournal of Business Ethics may want to take a lead
in this effort as it is regarded as the "flagship'journal 1 Because the search was not exhaustive, no argument is
in ethics research. Yet, a comparison of the 32 made that all empirical research articles on ethical beliefs
articles published in Journal of Business Ethics against and behavior have been included. For instance, research on
the remaining 62 articles across all the methodologi- business ethics appearing in books, book chapters, and
conference papers have not been analyzed.The data gather-
cal dimensions discussed above revealed no substan-
ing procedure also suffers from whatever limitations may be
tive methodological differences, with one exception. said to accompany the publication decisions of the journals
The Journal of Business Ethics tended to publish fewer incorporated within the data bases.
studies using mail survey research than non-Journal 2 Limited empirical investigation of ethical beliefs and
ofBusiness Ethics articles (i.e., 28% of Journal of Business behavior has taken place (Trevino, 1986). Most articles
Ethics articles used a mail survey approach vs. 56% of approach ethics from a nonempirical perspective, e.g.,
non-Journal of BusinessEthics articles). developing theoretic models of ethical decision-making (e.g.,
It was also interesting to note that the more Trevino, 1986; Bommer et al., 1987), specifying taxonomies
"rigorous" journals, in terms of lower acceptance of moral rules (e.g., Forsyth, 1980), or providing practical
rates, did not generally publish more methodologi- guidance to managers (e.g.,Laczniak, 1983).
cally rigorous articles. No substantive methodologi-
cal differences were detected between studies pub-
References
lished in high and low acceptance journals, with
again the one exception of research design. Journals Allen, R. F.: 1980, 'IK in the Office', OrganizationalDynamics
with low acceptance rates (i.e., more rigorous jour- 8, 26--41.
nals) tended to publish more mail surveys than Armstrong, J. s. and T. S. Overton: 1977, 'Estimating
journals with high acceptance rates (i.e., less rigorous Nonresponse Bias in Mail Surveys',Journal of Marketing
journals). Specifically, 68°; of the articles in journals Researck14, 396-402.
with low acceptance rates used a mail survey Ashton, R. H. and S. S. Kramer, 'Students as Surrogates in
470 D. M. Randall and A. M. Gibson

Behavioral Accounting Research: Some Evidence',Journal Salespeople's Ethical Conflict: An Exploratory Investiga-
ofAccountingResearch 18, 1-15. tion',Journal ofBusinessEthics 3, 343-53.
Babble, E.: 1986, The Practice of Social Research, 4th ed. Dubinsky, A. J. and M. Levy: 1985, 'Ethics in Retailing:
(Wadsworth Publishing Co., Belmont, CA). Perceptions of Retail Salespeople',Journal of the Academyof
Bailey, K. D.: 1982, Methods of Social Research, 2nd ed. (Free MarketingScience 13(1), 1-16.
Press, New York). Ferrell, O. C. and L. G. Gresham: 1985, 'A Contingency
Barnett, J. H. and M. J. Karson: 1987, 'Personal Values and Framework for Understanding Ethical Decision Making
Business Decisions -- An Exploratory Investigation', in Marketing', Journal ofMarketing 49 (Summer), 87-96.
Journal ofBusinessEthics 6(5), 371-382. Fleming,J. E.: 1987, 'Authorities in Business Ethics', Journal of
Banmhart, R. C.: 1961, 'How Ethical are Businessmen?', BusinessEthics 6(3), 213-17.
HarvardBusinessReview 39(4), 6--8. Forsyth, D. R.: 1980, 'A Taxonomy of Ethical Ideologies',
Becker, H. and D. j. Fritzsche: 1987, 'Business Ethics: A Journal ofPersonalityand SocialPsychology39(1), 175-84.
Cross-Cultural Comparison of Managers' Attitudes', Frederiksen, N., O. Jensen and A. Beaton: 1972, Predictionof
Journal ofBusinessEthics 6, 289-95. OrganizationalBehavior (Pergamon, New York).
Bezilla, tL, J. B. Haynes and C. Elliott: 1976, 'Ethics in Fredrickson, j. w.: 1986, 'An Exploratory Approach to
Marketing Research', BusinessHorizons 19, 83--6. Measuring Perceptions of Strategic Decision Process
Bird, F. and j. A. Waters: 1987, 'The Nature of Managerial Constructs', StrategicManagementJournal 7, 473-83.
Moral Standards'Journal ofBusinessEthics 6, 1-13. Fritzsche, D.j. and H. Becker: 1984, 'Linking Management
Bommer, M., C. Gratta, J. Gravendu and M. Turtle: 1987, 'A Behavior in Ethical Philosophy - An Investigation',
Behavioral Model of Ethical and Unethical Decision- AcademyofManagementJournal 27, 166-75.
Maldng', Journal ofBusinessEthics 6, 265-80. Fulmer, R. M.: 1971, 'Business Ethics: Present and Future',
Bowman, J. S.: 1976, 'Managerial Ethics in Business and PersonnelAdministration 34, 48--55.
Government', BusinessHorizons 19, 48-54. Gordon, M. E., L. A. Slade and N. Schmitt: 1987, 'Student
Brenner, S. N. and A. Molander: 1977, 'Is the Ethics of Guinea Pigs: Porcine Predictors and Particularistic Phe-
Business Changing?' HarvardBusinessReview 55, 57-71. nomena', AcademyofManagementReview 12, 160--63.
Brown, D. J. and j. B. King: 1982, 'Small Business Ethics: Greenberg, J.: 1987, 'The College Sophomore as Guinea Pig:
Influences and Perceptions [Survey]', Journal of Small Setting the Record Straight', Academy of Management
BusinessManagement 20, 11--8. Review 12, 157--9.
Browning, J. and N. B. Zabriskie: 1983, 'How Ethical are Hawkins, D. I. and A. B. Cocanougher: 1972, 'Student
Industry Buyers?' Industrial Marketing Management 12, Evaluations of the Ethics of Marketing Practices: The
219-24. Role of Marketing Education',.Journal of Marketing 36(2),
Campbell, D. T. and D. W. Fiske: 1959, 'Convergent and 61-4.
Discriminant Validation by the Multi-trait Multi-method Hegarty, W. H. and H. P. SimsJr.: 1978, 'Some Determinants
Matrix', PsychologicalBulletin 56(2), 81- 105. of Unethical Decision Behavior', Journal of Applied
Carroll, A. B.: 1975, 'Managerial Ethics: A Post-Watergate Psychology63(4), 451-7.
View', BusinessHorizons 18, 75-80. Hegarty, W. H. and H. P. Sims: 1979, 'Organizational
Chonko, L. B. and S. D. Hunt: 1985, 'Ethics and Marketing Philosophy, Policies and Objective Related to Unethical
Management: An Empirical Examination', Journal of Decision Behavior: A Laboratory Experiment', Journal of
BusinessResearcfl 13, 339-59. Applied Psychology64, 331-8.
Churchill, G. A.: 1979, 'A Paradigm for Developing Better Hogarth, R. M.: 1978, 'A Note on Aggregating Opinions',
Measures of Marketing Constructs', Journal of Marketing OrganizationalBehaviorand Human PeoCormance21, 40--6.
Research 16, 64-73. Krugman, D. M. and O. C. Ferrell: 1981, 'The Organiza-
Cronbach, L.J. and P. E. Meehl: 1955, 'Construct Validity in tional Ethics of Advertising: Corporate and Agency
Psychological Tests', PsychologicalBulletin 52(4), 281-302. Views',JournaI ofAdvertising 10, 21-30.
Crowne, D. P. and D. Marlowe: 1964, The Approval Motive Laczniak, G.: 1983, 'Business Ethics: A Manager's Primer',
(Wiley, New York). Business 3(1), 23--9.
Dillman, D.: 1978, Mail and Telephone Surveys: The Total Laczniak, G. and E. J. Inderrieden: 1987, 'The Influence of
DesignMethod (John Wiley, New York). Stated Organizational Concerns Upon Ethical Decision-
Dubinsky, A. J. and J. M. Gwin: 1981, 'Business Ethics: Matdng', Journal ofBusinessEthics 6(4), 297-307.
Buyers and Sellers', Journal of Purchasing and Materials Lazernitz, B.: 1968, 'Sampling Theory and Procedures'. In
Management 17, 9--16. Methodology in Social Research, H. A. Blalock and A. B.
Dubinsky, A. J. and T. N. Ingram: 1984, 'Correlates of Blalock, eds., pp. 278-328, NY: McGraw-Hill.
Methodology 471

Levy, M. and A.J. Dubinsky: 1983, 'Identifying and Address- Schutte, T. F.: 1965, 'Executives' Perceptions of Business
ing Retail Salespeople's Ethical Problems: A Method of Etbics',Journal ofPurchasing,38-52.
Application',Journal ofRetailing 59, 46-66. Simon, j. L. and P. Burstein: 1985, BasicResearch Methods in
Lewis, P. V.: 1985, 'Defining 'Business Ethics': Like Nailing SocialScience,3rd Ed., (New York: Random House).
Jello to a Wall',Journal of BusinessEthics 4, 377-83. Stevens, G. E.: 1984, 'Business Ethics and Social Responsi-
Lincoln, D. J., M. M. Pressley and T. Little: 1982, 'Ethical bility: The Responses of Present and Future Managers',
Beliefs and Personal Values of Top Level Executives', Akron Businessand EconomicReview 15(3), 6-- 11.
Journal ofBusinessResearch 10, 475-87. Trawick, F. and W. R. Darden: 1980, 'Marketer's Perception
McNichols, L. W. and T. W. Zimmer: 1985, 'Situational of Ethical Standards in the Marketing Profession: Educa-
Ethics: An Empirical Study of Differentiators of Student tors and Practitioners', Review of Business and Economic
Attitudes'Journal ofBusinessEthics 4, 175--80. Research 16, 1-17.
Nederhof, A. J.: 1985, 'Methods of Coping with Social Trevino, L K.: 1986, 'Ethical Decision Making in Organiza-
Desirability Bias: A Review', EuropeanJournal of Social tions: A Person-Situation Interactionist Model', Academy
Psychology15, 263-80. ofManagementReview 11(3), 601-- 17.
Phillips, D. L. and K.J. Clancy: 1970, 'Some Effects of"Social Vitell, S. J. and T. A. Festervaud: 1987, 'Business Ethics:
Desirability" in Survey Studies', American Journal of Conflicts, Practices and Beliefs of Industrial Executives',
Sociology77(5), 921--40. Journal ofBusinessEthics 6, 111-22.
Posner, B. Z. and W. H. Schmidv. 1987, 'Ethics in American Waters, J. A. and F. Bird: 1987, 'The Moral Dimension of
Companies: A Managerial Perspective', Journal of Business Organizational Culture', Journal of Business Ethics 6, 15-
Ethics 6, 383-91. 22.
Remus, W.: 1986, 'Graduate Students as Surrogates for Waters, J. A., F. Bird and P. D. Chant: 1986, 'Everyday Moral
Managers in Experiments on Business Decision Making', Issues Experienced by Managers', Journal of BusinessEthics
Journal ofBusinessResearch 14, 19-25. 5, 373-84.
Rizzo, A. and T.J. Parka: 1981, 'Organizational Imperative Zey-Ferrell, M. and O. C. Ferrell: 1982, 'Role-set Configura-
and Supervisory Control: Their Effects on Managerial tion and Opportunity as Predictors of Unethical Behavior
Ethics', PublicPersonnelManagement10, 103-9. in Organizations', Human Relations 35(7), 587--604.
Rosenthal, R. and R. L. Rosnov< 1975, The VolunteerSubject,
(New York: Wiley). Department of Management and Systems,
Rudelius, W. and R. Buchholz: 1979, 'What Industrial Washington State University,
Purchasers See as Key Ethical Dilemmas', Journal of Pullman, WA 99164-4726,
Purchasingand MaterialsManagement 15, 2-10. U.S.A.
Runes, D. D. (Ed.): 1964, Dictionary of Philosophy, Peterson:
Littlefield, Adams & Company.

You might also like