You are on page 1of 33

The World Has ChangedHave Analytical Procedure

Practices?*

GREG TROMPETER, University of Central Florida

ARNOLD WRIGHT, Northeastern University

1. Introduction
Analytical procedures (APs) are required at the planning and review phases
of an audit (SAS No. 56, AICPA 1988). APs are valuable, since they allow
the auditor to consider the reasonableness of nancial results based on
expectations and thus obtain a broader view (see the forest for the trees)
than that provided by detailed substantive tests. Further, prior research has
shown that APs detect misstatements that other procedures would not iden-
tify and do so in a cost-efcient manner (Wright and Ashton 1989; Hylas
and Ashton 1982). Despite their mandatory use and potential value to the
audit, little recent academic research has focused on the actual practice of
APs. Nelson and Tan (2005) call for additional work in this area: We
believe it is particularly important for researchers to consider how analytical
procedures are changing in response to recent changes in the audit setting
(45). An inuential prior study conducted by Hirst and Koonce (1996) over
a decade ago serves as a baseline to identify changes in the use of APs.
We address this issue by examining the practices of the Big 4 CPA rms
in conducting APs. We focus on how practices have changed in light of
recent changes in the audit environment. Our analysis is structured around
the concepts of drivers and enablers of change. As will be discussed
more fully in the following section, a driver is a factor that creates the need
for change, while an enabler facilitates change. A review of developments in
the audit environment suggests that two factors, nancial reporting scandals
and the ensuing Sarbanes-Oxley legislation (2002), are likely to have driven
change in the use of APs, and two additional factors, technological
advances and changes in rm audit approaches, have likely facilitated or
enabled change.

* Accepted by Steven Salterio. We would like to acknowledge the research assistance of


Emily Lilly, Jeffrey Nonnenkamp, Jason Perillo, and Scott Wright, and the valuable
comments received from participants of the accounting workshops at Virginia Tech,
Boston College, and the Philadelphia Accounting Research Colloquium and from
Jean Bedard, Jim Bierstaker, Jeff Cohen, Greg Jenkins, Lisa Koonce, Linda McDaniel,
and Roger Simnett. In addition, we gratefully appreciate the participation of the 36
professionals who agreed to be interviewed as a part of this study.

Contemporary Accounting Research Vol. 27 No. 2 (Summer 2010) pp. 669700  CAAA
doi:10.1111/j.1911-3846.2010.01021.x
670 Contemporary Accounting Research

To examine whether and how APs have changed in recent years, we


interview 36 auditors from the Big 4 rms. While we observe some similari-
ties in the ndings when compared to the Hirst and Koonce 1996 study
(e.g., auditors continue to use fairly simple analytical procedures), there are
many signicant differences. Among the changes in APs identied in our
interviews is the signicant impact of technology in providing industry and
analyst databases enabling auditors to set comparison benchmarks, develop
more precise quantitative expectations, and use more nonnancial informa-
tion. Auditors also report relying more on lower-level audit staff to conduct
APs, conducting greater inquiry of non-accounting personnel, and a greater
willingness to reduce substantive testing as a result of APs. Finally, the
Sarbanes-Oxley Act has had an impact in greater knowledge of internal
controls, which is seen as the most important factor driving the use and
reliance on APs.
The remainder of this paper is organized into four sections. The next
section contains a discussion of the background for the current study. A
description of the method and a presentation of the ndings then follow.
The nal section summarizes the key results and identies implications for
practice and future research.

2. Background
During the past decade a number of signicant developments have affected
the audit environment. We examine whether and how APs have changed
in response. As previously mentioned, we distinguish between drivers and
enablers of change.

Drivers of change
A review of the literature reveals two prominent drivers of change in the
conduct of APs: corporate nancial reporting failures, and the enactment of
the Sarbanes-Oxley Act (2002). First, several major corporate frauds and a
record number of earnings restatements (Palmrose and Scholz 2004; Agra-
wal and Chadha 2005) led to signicant concerns regarding the efcacy of
nancial statements audits. There were allegations that had auditors been
aware of the clients industry conditions and conducted appropriate analyti-
cal procedures many of these frauds would have been detected by the audi-
tor, e.g., Lincoln Savings & Loan (Erickson, Mayhew, and Felix 2000). As
a result, one would expect that auditing rms have altered their practices in
conducting APs to protect against the reoccurrence of a signicant nancial
reporting failure and the attendant exposure to litigation and loss of reputa-
tion.
Second, as a result of public outcry emanating from the Enron and
WorldCom frauds, Congress enacted the Sarbanes-Oxley Act (2002). That
Act called for a number of signicant measures affecting the auditing pro-
fession such as requiring assurance on internal controls and the formation
of the Public Companies Accounting Oversight Board (PCAOB) to oversee

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 671

the auditing profession. The Sarbanes-Oxley Act (herein referred to as


SOX) is likely to have led audit rms to re-evaluate and alter their audit
approaches, including the performance of APs (Kinney 2005: Nelson and
Tan 2005).

Enablers of change
During the past decade there have also been two signicant enablers of
change in APs practices: technological advances, and changes in rm audit
approaches. First, the proliferation in recent years of nancial databases,
internet search capabilities, and analysts reports such as First Call make
nancial and nonnancial information about a client readily and easily
accessible to auditors. Such information is conceptually very valuable in
conducting APs to gain an understanding of the clients business, its indus-
try, and its strategies, and to form expectations (Bell, Peecher, and Solomon
2005). Thus, one might expect auditors to use such resources to arrive at
more informed expectations than the predominant use of simple this year
last year comparisons as reported in earlier research (Hirst and Koonce
1996).
Also, advances in technology have led auditing rms to increasingly use
computerization to improve audit efciency and effectiveness. For example,
rms routinely use electronic work papers and automate the audit process.
Computerization would make it easier to provide decision aids and check-
lists to conduct APs than in the past, such as identifying likely causes for
unexpected uctuations (Blocher and Willingham 1993).
Second, over the past decade auditing rms have increasingly adopted
a more holistic, business risk audit approach (e.g., PwC Winograd,
Gerson, and Berlin 2000; KPMG Bell, Marrs, Solomon, and Thomas
1997; Bell et al. 2005). This shift in eld (Hinings, Greenwood, Reay, and
Suddaby 2004) was facilitated by the AICPAs publication of the expecta-
tions gap standards in the late 1980s. In particular SAS No. 56 expanded
the use of APs, which had previously been used in the planning process to
direct the auditors attention towards areas that merited special attention
and in the nal review phase. However SAS No. 56 indicated that APs
could serve as substantive tests as well.
Subsequently, throughout the 1990s, some of the Big 6 rms developed
new audit methods that recognized the value of APs beyond simple plan-
ning tools. Development of such approaches as KPMGs Business Measure-
ment Process or Andersens Business Audit placed a new emphasis on the
use of APs. These approaches, while not fully adopted by all rms, result in
auditors gaining greater knowledge about a clients strategies, business pro-
cesses, and industry trends than previously, potentially resulting in greater
focus on APs and more sophisticated expectations. Further, auditors at all
levels are involved in gathering and assessing information regarding busi-
ness risks. Thus, staff at lower levels than in the past may be engaged in
conducting APs.

CAR Vol. 27 No. 2 (Summer 2010)


672 Contemporary Accounting Research

Prior research
Hirst and Koonce (1996) was the rst study to provide descriptive ndings
of audit practices related to the use of APs at the then Big 6 audit rms. As
such, it has served as an important benchmark for researchers, practitioners,
standard setters, and educators. Despite growing use of APs in practice and
the issuance of auditing standards (e.g., SAS No. 56, AICPA 1988), prior to
Hirst and Koonce there was only anecdotal evidence of how auditors con-
ducted APs at various stages of the audit.
In their study, Hirst and Koonce nd that auditors frequently rely on the
prior years account balances to form expectations rather than more sophisti-
cated approaches or decision aids. Reliance on prior balances is of concern,
since this approach considers only a narrow range of information and does
not take into account changes in the clients business risks and industry (Bell
et al. 2005; Bell et al. 1997; POB 2000, 43). For instance, clients engaged in
fraudulent nancial reporting may intentionally show a smooth earnings
trend from the prior year. Research ndings also suggest that auditors have
difculties considering patterns in nancial and nonnancial information and
in generating the correct cause, and also tend to focus on a (target) non-
error cause (e.g., Bedard and Biggs 1991; Asare et al. 2000).
Hirst and Koonce (1996) report that management is the most common
source consulted in searching for explanations for unexpected uctuations
in the planning and testing phases. Of course, reliance on management is
of concern due to the potential for bias in responses. As a result, one of
the recommendations of the Panel on Audit Effectiveness (POB 2000) was
for standard setters to Develop more guidance on when it is appropriate
(and when it is inappropriate) for the auditor to rely on managements
explanations . . . and on obtaining additional evidence to corroborate
those explanations (43). Finally, Anderson and Koonce (1998) nd that
quantifying expectations enhances the ability to discern the correct
cause(s) of a uctuation than when only considering the plausibility of a
client explanation.
Our review indicates that prior research on AP practices is limited and
conducted in the mid-1990s prior to the signicant changes, drivers, and
enablers affecting the auditing profession as identied previously. The next
section describes the method employed in the current study to examine how
these developments have impacted the use of APs.

3. Method
Instrument development and content
We developed a series of questions addressing the use of, and reliance on,
APs as well as changes in recent years in the performance of APs. Given
the interview nature of the study, the focus was on each participants expe-
riences in conducting analytical procedures. Since Hirst and Koonce (1996)
provide baseline data, our starting point in developing questions was the

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 673

survey instrument used in their study.1 We included questions from the ori-
ginal instrument that, in our judgment, reected ndings that provide signif-
icant insights and are noted frequently by subsequent researchers. (For
instance, Hirst and Koonce reported that, when confronted with a material
unexpected uctuation, auditors went to the client to identify the cause(s)
for the uctuation. We wanted to assess the extent to which that practice
still continues today.) Relevant interview questions are presented in Appen-
dix 1 and referenced to the discussion of the ndings by number (e.g., Q1).
We also included questions to address changes in the environment since
the early- to mid-1990s. For example we provided a list of six auditing tech-
niques that are commonly used in the conduct of APs.2 Participants were
asked to compare the frequency of the use of these techniques today to the
frequency of their use in the past. In designing the structure of the instru-
ment, we began each topic with Likert scale responses and then followed up
with more qualitative, open-ended questions.

Sample
We conducted interviews with 36 practicing auditors. Our sample was cho-
sen on an availability basis with the assistance of key rm personnel. To
obtain perspectives of auditors at different staff levels, we sought and
obtained the participation of approximately equal numbers of partners,
managers and seniors from each of the Big 4 rms (see Table 1). The sam-
ple size is the same as that used by Hirst and Koonce 1996; however, the
experience levels of the current sample are evenly balanced (with the excep-
tion of one more manager than seniors), whereas Hirst and Koonce
reported interviews with 17 seniors, 14 managers and only ve partners.
While participants in our study had experiences in several ofces across the
country, all were, at the time of interview, working in large ofces in the
northeastern United States. Initial interviews with three participants resulted
in minor modications to the interview instrument.
Descriptive statistics on the sample composition and interview length
are presented in Table 1. Interviews were conducted between late 2003 and
mid-2005. The duration of interviews ranged from 42 to 90 minutes with a
mean time for partners of 76 minutes (range 62 to 90 minutes) compared to
53 minutes for seniors (range 42 to 60 minutes). The longer duration of
partners interviews is indicative of their greater depth of knowledge.

Procedure
The interviews consisted primarily of structured questions which were
supplemented with open-ended (semi-structured) questions to further explore

1. We would like to thank Lisa Koonce for providing a copy of the interview template used
in the Hirst and Koonce 1996 paper.
2. These techniques include: account balance comparisons, ratio analysis, regression analy-
sis, analysis of nonnancial information, analysis of client budgets or forecasts, and
analysis of interim data.

CAR Vol. 27 No. 2 (Summer 2010)


674 Contemporary Accounting Research

TABLE 1
Sample composition and interview statistics

Panel A
Average length of Number of years
Rank N interview (in minutes) in public accounting

Partner 12 76 20.1
Manager 13* 62 6.9
Senior associate 11* 53 3.8

Notes:
* Each rm was requested to provide three partners, three managers and three
senior associates for interviews. However, one of the individuals who had
been a senior associate at the time the interview was arranged had been
promoted to manager by the date of the interview.

Panel B

Industry Frequency

Government 1
Financial services 9
Manufacturing 13
Non-prot 2
Technology 7
Retail 3
Healthcare 1

Total 36

participants experiences. Each of the 36 interviews was audiotaped and then


transcribed. One of the researchers read each transcript several times. Initial tran-
script analysis focused on gaining an understanding of how the use of APs had
been affected by hypothesized drivers and enablers: corporate nancial reporting
scandals, the Sarbanes-Oxley Act, technology and databases, and holistic audit
approaches. Subsequent analysis of each transcript focused on gaining a more
complete understanding of interviewees beliefs regarding the effectiveness
vs. the efciency of APs, the development of audit expectations, and fraud.
To gain additional insight with respect to the level of consensus across
participants, we conducted additionalmore quantitative analysis. For
questions with discrete responses (e.g., more, same, less or high, medium, low)
categorization was based on the related response scale. For qualitative res-
ponses, a preliminary coding scheme was developed by a graduate accounting

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 675

student, based on an initial analysis of several transcripts. This initial scheme


was rened based on a review by, and discussion with, the authors.3
In a further effort to identify statements that highlight the rms
changes in the performance of APs in response to drivers and enables in the
uid audit environment, we also conducted a key word search using the
term change to focus on statements that described changes in practice.
Finally, we also closely reviewed transcripts from the interviews with part-
ners and senior managers (i.e., personnel who would have signicant per-
spective to evaluate changes over time) to identify discussions of changes in
AP practices and the nature and basis for such changes.

4. Findings
As discussed, this study examines how the use of APs has changed in recent
years in response to signicant drivers and enablers in the audit environ-
ment. Factors that have impacted the use of APs in practice are considered
rst. Then the ndings are presented in the sequential stages of conducting
APs (Koonce 1993): developing expectations and identifying material uctu-
ations, and nding (hypothesis generation) and testing explanations. A sum-
mary of the implications of the ndings for future research is provided in
Table 2. These implications are explicitly linked to the discussion of the
results by reference to the numbered research issue (RI).

The use of APs


When asked about the use of APs over time (Q1), several participants noted
the impact of corporate scandals and Sarbanes-Oxley, both of the drivers
examined, on greater use of APs.

I would say the frauds and the corporate scandals have . . .made people
renew their focus on doing some good old-fashioned audit work, again
including good detailed analytics as part of a good, robust audit. (partner)

While this quote suggests that the scandals refocused the profession on
APs, the partner continues and provides a more in-depth description of how
the scandals affected the entire audit process including more rened APs:

I personally think that the profession as a whole got a little bit off track
maybe in the mid to late nineties in doing a little bit more top level audit-
ing. Not doing quite as much detailed auditing. And now I think theres a
renewed focus and I think rightly so of doing more detailed audit work
and that would include more detailed substantive analytics. (partner)

3. To test the reliability of coding, twelve of the transcripts (one-third) were recoded by a
second graduate accounting student and the results were compared to the initial coding.
Inter-rater agreement was 89.9 percent (Cohens Kappa Coefcient = .70 ; p = .001).
Differences were reconciled through discussion between the coders. Any remaining
disagreements were examined and coded by one of the authors.

CAR Vol. 27 No. 2 (Summer 2010)


676 Contemporary Accounting Research

TABLE 2
Future research issues (RI)

RI 1: Less experienced auditors give signicantly less attention to disaggregated


data in conducting APs than managers and partners. What benets can be derived
from training or decision aids for more junior level staff about the power of using
disaggregated data that provide more accurate and precise results than aggregate
models?
RI 2: Auditors report that the Sarbanes-Oxley Act (SOX) has had a substantial
effect in leading to broader consideration and knowledge of internal controls. How
does SOX impact the design and use of APs?
RI 3: Technology is playing a major role in allowing auditors to access a broad
array of information. How do auditors use such information in developing expecta-
tions and evaluating potential explanations for unexpected uctuations? How is the
reliability of this information considered? What is the nature of the information
search conducted and does this vary by client conditions (e.g., inherent risks)?
RI 4: Do auditors leverage the greater knowledge of business risks, strategies, and
processes obtained through the use of more holistic audit approaches in conducting
APs? How can this be enhanced?
RI 5: To what extent are auditors aware of metrics used by the marketplace and is
that knowledge incorporated into the use of APs?
RI 6: To what extent should the length of auditor-client tenure affect the use of, or
reliance on, APs?
RI 7: To what extent will external bodies such as the IAASB and the PCAOB drive
rms audit methods with respect to the future use of, and reliance on, APs?
RI 8: Auditors report relying on APs to decrease testing more often than in prior
studies. What factors lead to greater reliance on APs and how does this impact
audit efciency and effectiveness?
RI 9: There has been an apparent recent shift to assigning the conduct of APs to
less experienced associates and senior associates. Has this shift impaired audit effec-
tiveness? Are training (e.g., case examples and techniques of nancial statement
analysis) or decision aids needed?
RI 10: While interviewees state that they are likely to develop quantitative expec-
tations, there is little evidence that they formally develop a priori account
balance expectations in the absence of knowledge of unaudited account balances.
How do a priori expectations versus no expectations impact audit efciency and
effectiveness?
RI 11: Given the results showing a divergence in practice regarding whether expec-
tations are developed by the audit team or by each individual member, which of the
two approaches results in greater AP effectiveness and or efciency?
(The table is continued on the next page.)

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 677

TABLE 2 (Continued)

RI 12: How does the development of AP expectations impact fraud brainstorming


sessions required by SAS No. 99?
RI 13: While auditors tend to use simple APs, the current study also suggests that
they gather and consider a broader array of industry and company information,
particularly nonnancial information. How do auditors integrate such information
(e.g., in an informal, unstructured manner) and can structured tools provide
valuable assistance?
RI 14: Under what conditions is it most benecial to approach non-accounting
personnel? How should information gathered from such personnel be incorporated
into audit evidence?
RI 15: The ndings suggest that most auditors do not attempt to gather evidence to
refute a clients explanation for an unexpected uctuation. How does this affect
audit risk and the auditors business risk? How can this risk be mitigated?
RI 16: How does the psychology of risk affect the auditors willingness to rely on
APs? If auditors risk preferences cause them to over (under) rely on the results of
APs, what steps can rms take to encourage optimal use of APs?
RI 17: How does change, such as change in the use of APs, move through the
profession and become accepted by rms and by individuals within the rms? To
what extent are such changes driven by changes in regulation vs. driven by other
factors? How can technology lead to more timely adoption of improvements in
audit methods?

Quantitative results also conrm the greater use of APs (Q1). Sixty-one
percent (22) of the interviewees indicate APs are currently used more than
they have been in the past. When asked to identify factors accounting for
this increase, nearly 40 percent (14) participants cite the recent accounting
scandals.
Regarding the time allocated to APs (Q2), an analysis of data across
time periods reveals small increases in the number of budget hours allocated
to APs in each stage (planning, substantive testing, and overall review) of
the audit engagement. Interviewees estimated that, in the current period,
over 25 percent of the total audit budget was allocated to APs. By compari-
son, it was estimated that less than 21 percent of the budget was allocated
to APs during the period between two and seven years prior to our data
collection.4 This upward general trend is consistent with the ndings of
Ameen and Strawser (1994) who reported that, in the early 1990s, approxi-
mately 15 percent of the budget was allocated to APs.

4. Only partners and senior managers (less than half of the sample, n = 15) had sufcient
experience to respond to this part of the interview.

CAR Vol. 27 No. 2 (Summer 2010)


678 Contemporary Accounting Research

Participants were asked (Q2 and Q3) to assess the extent to which they
focus on APs at various stages of the audit (i.e., planning, substantive test-
ing, and overall review) and at various levels of aggregation (i.e., corporate,
division, and product level line of business, and for the review of the 10-Q).
In general, interviewees indicated that they focused moderate levels of atten-
tion on APs at various stages of the audit and at various levels of aggrega-
tion. However, responses also suggest that individuals with more experience
(partners and senior level managers) consistently focus more attention on
APs at the lowest level of aggregation (e.g., product line LOB) for all stages
of the audit. Given the nding that a greater focus on disaggregated data
allow APs to provide more accurate and precise results than would be
obtained by using aggregate models (Allen, Beasley, and Branson 1999), this
suggests that it may be benecial to place greater emphasis on the power of
analysis of disaggregated data in university studies or training classes for
lower level staff (see Table 2, RI 1).
Along with corporate scandals, the ensuing Sarbanes-Oxley Act is also
viewed as a driver of change in the use of APs. When questioned speci-
cally about it (Q3), 33 percent of the participants (12) indicate that
changes in the use of APs has been directly related to this law. However,
participants were equivocal in their assessments of whether SOX led to an
increase or decrease in the use of APs. The following two quotes from
partners suggest that APs will decrease, at least in the short run, as a
result of SOX:

I think one of the many unfortunate things [related to scandals, SOX


and PCAOB scrutiny] was I believe that people are going back to the
traditional way of doing auditing because they felt thats the safe haven.
When people were doing audits that way everybody was happy[because
of the perceived assurance from detailed testing]. (partner)

When you say Sarbanes-Oxley I assume you mean section 404 . . . . But
testing the internal controls, at least in this rst year of implementation,
is at such a detailed level around processes that were testing on such a
minute level of detail, you dont really need the analytics. An analytic is
to give you some sense of something from an overall perspective. If you
look at every single line item whats the point of doing the overall
perspective? (partner)

Yet the general belief is that, in the long run, enhanced internal controls
resulting from Section 404 audits will lead to enhanced ability to rely on
the results of analytical procedures. Ultimately, there is hope that more
in-depth reviews of internal controls will pave the way for a truly inte-
grated audit that can enhance the auditors efciency without sacricing the
effectiveness of the assurance function. The relation between controls and
the value of APs is addressed in the following quotes:

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 679

Now that its year two of 404 for an integrated audit, were going to try
to step away from substantive procedures and really get an idea of what
a truly integrated audit is. So relying on controls and maybe using
more analytical procedures to get more comfortable with account
balances and whatnot . . . . The ideal 404 integrated audit is going to be
relying on the controls within an effective environment and then saying,
okay, what can we do to relieve the substantive audit work because we
can rely on the controls they have on hand? And I can rely on APs.
(senior)

Concurring with that sentiment, other interviewees suggest that SOX


Section 404 has focused more attention on controls which, in turn, allows
the auditor to rely more heavily on APs:

We do a lot more of internal control work so therefore we get more


comfortable with the process around each account rather than substan-
tively auditing the account. . .we do a walkthrough of the controls and
then test the controls. Then at year end we only have to do analytics
over those accounts. So whereas we used to ignore kind of the controls
and assume that there were no effective controls over any counter pro-
cess [we now test controls and rely on analytics to] audit the accounting
year end. (manager)

The connection between strong internal controls and enhanced reliance


on APs was quite clear. Many participants indicated that SOX and or the
PCAOB had increased the focus on internal controls and, in turn the reli-
ability of APs. However, closer reading of the transcripts suggests a more
nuanced relationship. Many of the participants suggested that, at least in
the immediate post-SOX environment, characterized by heightened risk
sensitivity, reliance on APs would be greatest in low-risk situations. This
sentiment is highlighted in the following quotes:

I nd in my experience that the highest risk areas were always going to


fall to some level of detail testing. It might be a combination of analyti-
cal procedures and detail testing but areas where were not really overly
condent that we know whats going on we probably in some way
default to detail testing. . . . I think if it comes down to we say: Okay
we think theres a risk here that maybe theres some lack of documenta-
tion or improper accounting, then were always going to go back to
Okay lets get into the details. (senior)

The above quote provides insight into the use of APs and suggests that in
higher risk cases they may be most effective when combined with additional
tests of details. The following quote also addresses risk and the sometimes-
limited level of assurance that APs provide, by suggesting that it would be

CAR Vol. 27 No. 2 (Summer 2010)


680 Contemporary Accounting Research

unlikely that adjusting entries would be made based solely on the results
of APs.

The risk is too high. So in those instances we dont do that much analyti-
cal review. Its just: Substantively audit and thats that. I mean at the
end you would do an overall analytical review compared to last year but
thats it. You wouldnt analytically change account balances. (manager)

This is representative of statements made by several participants who


suggest that in the immediate wake of the scandals and the passage of
SOX, auditors are less willing to place reliance on APs in lieu of tests of
details except in low-risk cases. Whether auditors will increasingly rely on
APs as Section 404 audits strengthen internal controls, or continue to rely
on tests of details for high risk settings, is currently unclear (RI 2). This
issue will be discussed further in the nal section.
While SOX and risk management in response to nancial reporting
scandals were the most commonly cited explanations for the changing use
of APs (Q1, Q3), another theme is that technology, an enabler, allows for
greater use and reliance on analytics. This theme is highlighted by the fol-
lowing two quotes:

Whats different is the access to benchmarking and peer competitive


data that makes the analytical tools so much more powerful than they
were ve years ago, or [even] three years ago. (partner)

I would say were using them [APs] more because the information avail-
able is more robust, more timely, so therefore you can use more com-
parisons with competitors or just other companies in the marketplace so
you can get more out of analytics and theyre more available, so Id say
were using them more. (partner)

While all participants discussed how APs had been affected by the
expanded use of more powerful technology, mention of the effects of our
second hypothesized enabler more holistic audit approaches was not
universal. While two of the Big 4 rms appear to have fully adopted such
approaches, the other rms have not. This contrast is highlighted in the fol-
lowing quotes. The rst two quotes reect the views of professionals at
rms that appear to have more fully endorsed a holistic audit approach,
while the last quote reects the views of a professional at a rm that
appears to have less fully endorsed such an approach.

Well Im not sure if youre familiar with [my rms] audit methodology,
but the rst thing we do is we go in and test controls. That may be in
conjunction with the year end audit, we might go in a little earlier just
from a timing perspective, but well document the process. First we

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 681

identify the key process, we document the process, identify the key
controls and then test controls. You select a subset of those control
activities for testing, and if your control testing, if it goes well, if you
dont encounter any problems with control testing, your reliance
on analytical procedures is much higher than test of details. Whereas if
I go into a company that doesnt really have a good control environ-
ment, Im going to rely more heavily on test of details and kind of
ignore analytical procedures from a substantive testing standpoint.
(manager)

Another interviewee highlighted that information gathered using the


holistic audit approach strengthened the auditors knowledge of the client
and the power of APs:

As part of planning we have to put together [a proprietary schedule]. . . .


The rst page you talk about the industry and the changes that have
happened in the industry. Then who the major stakeholders are, just to
make us aware that these are the people who have a stake to be aware of
fraud and all that, who would have incentives in a way. Just to see if
theres pressure from upper management if they have a lot of stake in
the company. Then you look at factors that typically affect the company,
if theres any identied risks that we know about, if they have foreign
exchange rates that affect the nancials and things like that. Then we
identify what accounts it affects. Then we also come up with the
company goals. (senior)

However, a contrary view is offered by a senior partner at another rm


indicating that not all rms have fully adopted an AP intensive, holistic
audit approach:

In our rm we never placed heavy reliance on pure analyticals for audit.


We did look at relationships for what I call pure audit because we always
did combination testing. And analyticals help us understand the business
and understand some trends, but we always focused . . . much more on
substantive testing and control testing and less on analyticals . . . . We
have done a fair amount of analytical testing when it relates to revenue
and cost of sales relationships but beyond that it gives us a sense for the
business but it was never anything that Id rely on heavily. (partner)

Thus, the interviewees suggest a difference across rms in the extent to


which APs are relied upon for substantive testing. Yet, it is noteworthy
that these quotes emphasize the importance of internal controls. Thus,
differences between rms may change as section 404 internal control work
results in stronger systems of controls and, presumably, more trustworthy
input on which to base APs.

CAR Vol. 27 No. 2 (Summer 2010)


682 Contemporary Accounting Research

While various rms may be more or less willing to rely on APs as suf-
cient, competent audit evidence, all rms appear to rely on more powerful,
technology-driven APs to identify risk. This was noted by interviewees from
all rms and was often attributed to advances in technology as noted in the
following quote:

Our whole audit methodology is predicated on the risk model, where . . .


you make an evaluation on the controls: Strong, Not strong. Then that
translates into Whats the impact you get out of your analytical proce-
dures? And those are all risk assessments. You make your risk assess-
ments those are going to drive your substantive work. So its actually
built right into the model. The software, to some extent will tell you:
That doesnt make sense. Youve evaluated this risk factor as high,
youve got low value for your analytics, but youve got high risk. You
dont seem to have enough detailed substantive testing. (partner)

Thus, regarding our two hypothesized enablers, it is obvious that tech-


nology has facilitated profound change in the use of APs across the profes-
sion. More reliable, more easily accessed databases and more powerful
software have facilitated greater use and reliance on APs. Further, all rms
appear to take greater advantage of these more powerful, technology-driven
tests as attention-directing procedures to identify risks and to guide them in
the design and conduct of the audit. Yet, the impact of the holistic, risk-
based audit approach appears to have been more uneven across rms (RI 3
and 4).
In addition to our hypothesized enablers and drivers, participants raised
two additional factors that affected the use of, or reliance on, APs. First,
one interviewee, an audit manager, noted that she placed a greater emphasis
on APs when conducting an examination of publicly traded companies.
This focus stems from her observation that the market follows specic
ratios and that those are closely related to many of the APs that are per-
formed in the conduct of an audit. This links directly to the SECs concerns
stated in SAB No. 99 on materiality (SEC 1999) when the Commission
encouraged auditors to take market expectations into greater consideration
in the conduct of an examination. This leads to research opportunities on
market expectations and potential differences in the use of APs for public
vs private entities (RI 5).
A second factor that was addressed by another audit manager was
experience with a client as noted in the quote below:

. . . your experience with the client. How many years that youve been
there maybe. Whereas now Ive got some new clients . . . . Im a little
more hesitant to rely on analytical procedures than I would on the test
of details just because I dont really know enough about the client. I
guess I just dont have that comfort level yet being that its a rst-year

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 683

client. So even though analytical procedures might be completely appro-


priate given the circumstance, I might push towards more test of details
just for my own comfort level.

This also leads to potential research. Investigation into the extent to


which reliance on APs is appropriate for newer clients and long-standing
clients where an auditor may become complacent may prove bene-
cial (RI 6).
Finally, although not highlighted in our discussions, to the extent
that globalization affects U.S. auditing practices, there may be an
increasing reliance on APs as the International Standards on Auditing
(ISAs) migrate towards more risk-based methodologies that place greater
emphasis on the use of APs. In particular, beginning in 2004 with ISA
No. 315, and following with ISA No. 330, the International Auditing and
Assurance Standards Board (IAASB) has placed a greater focus on the
use of APs. While such pressures should lead to an increase in the use
of APs, it is not clear how the interaction between the IAASB, interna-
tionally, and the PCAOB, in the United States, will affect rms reliance
on APs (RI 7).

Efciency effectiveness of APs


Prior studies have found APs to be effective in detecting unexpected uctua-
tions (Wright and Ashton 1989; Hylas and Ashton 1982). Further, while
prior research has found auditors use the results of APs to highlight prob-
lem areas and expand testing in those areas, it has also reported that audi-
tors are reluctant to rely on the results of APs to reduce audit hours in
areas of less concern (Biggs, Mock, and Watkins 1989; Cohen and Kida
1989). This calls into question whether auditors take full advantage of the
efciencies that can be gained from the use of APs. However, the above
research was conducted more than a decade ago and it is possible that
changes in the audit environment have resulted in changes in the use of,
and reliance on, APs. For example, the emphasis on internal controls in
Section 404 of SOX has the potential to make APs more effective. Further,
changes in technology may make the use of APs more effective and ef-
cient. To examine this issue, we asked a series of questions (Q4Q7) on
auditor experiences regarding the effectiveness efciency of APs. In address-
ing the efciency of APs, one interviewee states:

Analytical procedure is supposed to save you some time versus test of


details. If youre going to spend all this time doing an analytical proce-
dure and you could have done a test of details in less time, then obviously
theres a disconnect there. So efciency is part of it as well. (manager)

This sentiment is reiterated by an interviewee from another rm who


continues to address effectiveness in particular the possibility that, in

CAR Vol. 27 No. 2 (Summer 2010)


684 Contemporary Accounting Research

certain cases, APs may not only be more efcient, but more effective as
well.

Well I would say analytics are pretty much always more efcient. I
mean [sometimes] you can just do an analytic thats going to save you a
ton of time than if youre testing the actual details . . . . I think you
look at relationships between different accounts more which I think
gives you a better insight into the numbers than if you just recalculate
something. I mean thats not very good audit evidence. (manager)

While there are situations where analytics may be both more efcient
and more effective, this may not be the typical case. More often there is a
tradeoff as highlighted in the following quote:

The tradeoff is precision versus scope. You have a nite amount of time
to get something done, not in theory but in practice. And if youre try-
ing to understand revenue you may be only able to test a certain num-
ber of transactions then youd know whether those transactions were
correct or not but that may only give you 20 percent coverage. Whereas
you do analytics, you do analytics on the total balance by stratifying it
and doing some analytics on different geographies or different product
lines, different business units, so you can get product coverage but not
the precision. So you have to consider inherent risk and stability and
then you can make your tradeoffs. (partner)

He continued noting that, in certain cases, APs by themselves may be


neither efcient nor effective:

Its only efcient to the extent that its completed the job. So you can
certainly analyze the balance but if you dont have enough condence
that the numbers right, then you need to go back and do more work.
So I think its really a combination of what are your various procedures
and wheres the risk inherently viewed. (partner)

In general, participants reinforced the importance of auditor judgment


in determining the sufciency of APs as audit evidence. The notion that
APs could generally serve as sufcient evidence to substantiate account bal-
ances was dismissed as it was tempered by the belief that the effectiveness
of APs was situation-specic and was based on a number of factors includ-
ing inherent risk, the importance and stability of the account, the nature of
the AP, and the level of precision desired for the test.
Further questioning on the the effectiveness of APs revealed that 69
percent of the interviewees indicated that they have uncovered issues with
APs that otherwise would not have been detected (Q4). As summarized by
a senior:

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 685

Thats why analytics are so good; its probably . . . the easiest way to
nd out something that you didnt already know. Its more of a step
back and just, Wow this really did change, why? What are the assump-
tions that might have changed underlying that? (senior)

To examine the impact of APs in the planning stage on subsequent


audit effectiveness and efciency of substantive testing, interviewees are
asked to assess whether such procedures ever reduce or increase audit hours
(Q5). Reminiscent of prior studies (e.g., Biggs et al. 1989; Cohen and Kida
1989), participants indicate, as a result of APs, audit hours are more likely
to increase (89 percent) than decrease (69 percent). However, the relative
frequency in which hours are decreased is substantially higher than reported
in earlier studies, suggesting auditors are more likely to take advantage of
APs to enhance audit efciency than previously (RI 8).5 One participant
emphasized the importance of developing expectations as a way of increas-
ing the power of APs:

If you adhere to the [rms audit] approach and the plan, and the
parameters, and you come up with what your expectations are, . . . and
your results are meeting your expectations, that should result in less
audit hours. (partner)

In effect, the claim is that the AP can be an effective substantive test


that reduces the need for additional testing if it is being used in conjunction
with fairly precise expectations that were developed a priori.
A second partner concurred with the possibility of reducing audit hours
based on the results of APs but he focused on the role that strong internal
controls play in enhancing the power of APs:

As I think about it, if we have very consistent trends in some of our


store operations, were done. [Because] The company has good controls.
If we have these inconsistent trends, then weve got to do much more
work. So yes it does reduce scope if theres good controls. (partner)

Thus, it appears that auditors may be more willing to reduce audit


hours based on the results of analytical testing, but it appears to be the
case only in situations where the audit environment (e.g., conditions of
precise expectations and or strong internal controls) allows for very power-
ful APs.

5. Under the condition of strong internal control, Cohen and Kida (1989) found that 25
percent of their sample of audit seniors and managers would reduce the hours allotted in
the initial audit budget, when APs indicated no likely errors. However, the remaining 75
percent would either increase hours or make no adjustments to the budget. In total,
under these conditions (i.e., strong internal control and no likely errors), the mean adjust-
ment from the initial budget was not signicantly different from zero.

CAR Vol. 27 No. 2 (Summer 2010)


686 Contemporary Accounting Research

Finally, some respondents suggested that they would not rely on the
results of APs to reduce overall audit hours, but would rely on them to
enhance the effectiveness of the audit by reducing hours in a specic area
and reallocate them to an area of higher risk:

Decrease the audit hours themselves? I never have . . . . What weve


usually done is: OK, this area is pretty clean, so maybe we only test
this analytically, but we have a problem here so were going to allocate
those hours [to the problem area]. If anything it would result in a
reallocation of hours versus an outright decrease per se. (senior
manager)

While some participants indicated a willingness to reduce audit hours as


a result of APs, as noted above, it was much more likely that interviewees
would increase hours. However, it is unclear to what extent those increases
are driven by a heightened concern over audit quality in the post-SOX envi-
ronment. In this regard, one participant stated:

I think [an] increase would be viewed as a good thing to the outside


world and I think were very much concerned with the outside world
today, more than we ever have been. Things that we are thinking about
now as were doing audits [are] OK, if this gets picked for review by
the PCAOB, is this going to hold up? What questions are they going to
ask? (partner)

Also indicative of the difference in the audit environment in the post-


SOX era is the following comment about the risks faced by the auditor and
how that is perceived by the audit committee:

I think in this day and age I think we are as auditors more responsive
to risks that we see at our clients . . . . And I think a big change in cor-
porate America is that not many CFOs and not many audit committees
want to be in a position to tell us to do less work. (partner)

Of course the extent to which this attitude will continue into the
future is uncertain as boards and business groups call for easing (or in
some cases the outright repeal) of certain SOX regulations and the SEC
and PCAOB attempts to balance audit quality and investor protection
against the costs of monitoring and disclosure that the registrant must
bear to satisfy the requirements of SOX (Burns 2007; Reilly 2006; Scannell
2007).

Design and performance of APs


We posed a series of questions regarding the design and performance of
APs (Q8Q10). When asked what factors are considered in the design of

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 687

APs (Q8), participants provided several answers. Fifty percent (18) state
that they considered prior years procedures, and 50 percent state that they
incorporate knowledge of changes and trends in the economy and at the cli-
ent. In addition, 44 percent (16) indicate that they consider the nature of
the business, and 31 percent (11) note that availability of data is an impor-
tant factor.
Estimates suggest that seniors and managers design (Q8) approxi-
mately 80 percent of the AP tests (means = 36 percent and 44 percent
respectively), with partners and staff doing considerably less (13 percent
and 6 percent respectively). In contrast, APs are performed (Q9) primarily
by the staff and seniors (48 percent and 35 percent respectively) with
less of this work being done by managers (12 percent) and partners (5
percent). With respect to the nature of the work being done (Q10), one
interviewee states:

Okay planning and substantive would typically be done by the staff and
the senior. Thats kind of getting down into the details. The overall
review I would say its more senior manager because I would say the
managers like to understand [so] that they can communicate effectively
all the changes in accounts to the partner. So theyre going to spend
more time reviewing it and in certain circumstances, actually doing it to
make sure they fully understand the changes between accounts from the
overall completion review stage of the audit. (manager)

In comparison, Hirst and Koonce (1996) report that 25 of 25 partici-


pants indicated that, in the planning stage of the audit, APs were usually
performed by the senior or manager with the manager being more involved
if the client is deemed to be of high risk. Further, they report that 9 of 10
auditors indicated that substantive testing APs are normally performed by
seniors. However, it was noted that newer associates (staff) might have been
assigned to conduct APs on straightforward or low-risk accounts. The
results in the current study suggest that, relative to practices reported previ-
ously (i.e., by Hirst and Koonce), there has been an increase in the tendency
to assign APs to less experienced staff. It appears that rms are more com-
fortable in assigning this work to associates, perhaps because they have
increased their investments in training junior associates as indicated in the
following response:

Yeah, I think [the rm] over the last two years . . . theyve really put a
focus on train the staff to do more effective and efcient analytical
procedures to get more bang for the buck. (manager)

In addition, it may be instructive to involve lower levels of staff in the


development of APs as a way to train them to better understand the clients
business as noted in the following partners quote:

CAR Vol. 27 No. 2 (Summer 2010)


688 Contemporary Accounting Research

But I think thats usually pushed down to our staff level and I think
that our seniors review it and they say, Thats a lousy expectation,
because they [the staff] are just kind of picking the path of least resis-
tance. And so our seniors push back on that. And then I think where
the managers and the partners come in is really doing more push back
and saying, You guys are still youre missing the boat. You didnt
factor in the fact that they opened a new building, in whatever loca-
tion. And you didnt factor that into your occupancy cost. Why is
that? So that push down is whats [important]. (partner)

Additionally, less experienced staff may be conducting more analytical


procedures because of the availability of more powerful, easy-to-use data-
bases that increase the productivity of lower level staff. Alternatively, it
may be a cause for concern if protability concerns are driving rms to
assign less qualied staff to conduct more complex audit procedures (RI 9).

Developing expectations and identifying material uctuations


Seventy-two percent (26) of the respondents indicated they develop quanti-
tative expectations (point estimate or range) in conducting APs (Q11). Fur-
ther, the precision of the expectation likely depends on the nature of the
account as noted in the following quote:

[My rms] approach is: You develop a very specic expectation on


account balance. And, again, it really should be very unique to each indi-
vidual account, how you develop that expectation. And, I think the other
example of citing salaries sometimes is pretty easy. You say, What was
last years salary? What was last years headcount? What were this years
raises? Whats this years headcount? You do the math. Theres your
expectation. Right? Plus or minus bonuses or some changes in manage-
ment, or directors, or something like that. With known factors you
should be able to get to a pretty close number on something like that.

Where an account balance is an estimate, for example, in allowance


accounts, we will often develop a range, kind of, you know, whats the
best case, whats the worst case, so theres your range. Does the client
balance fall within that range? Because . . . only time will tell what that
actual balance is . . . . The only way youre really ever going to get to
the known answer on that is probably wait a year. (partner)

When participants were asked how they developed expectations (Q12),


13 indicate they develop a threshold or range and 11 use professional
judgment (prior year numbers or budgets). This nding is in contrast with
that reported by Hirst and Koonce (1996), who nd auditors using APs as
a substantive test normally develop imprecise expectations of the direction
of the anticipated change in an account balance based on the prior year.

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 689

The apparent development of more precise expectations by auditors than


found previously appears to be due to the wide accessibility of industry and
other data through advances in technology in recent years such as the Inter-
net, and or to the adoption of more recent holistic auditing approaches.
When asked what is your primary point of comparison when using APs
(Q13), 89 percent indicate the prior year, 44 percent industry norms, and 25
percent expectations.
These ndings suggest that the profession appears to have progressed
from the early 1990s when APs frequently consisted of a simple comparison
of this years balances to last years balances accompanied by a uctuation
analysis to explain material year-to-year differences. (Of course one problem
with this type of analysis is that it may mask problems when an account
did not uctuate and, given appropriate knowledge of the clients business,
the auditor should have expected such a uctuation.) However, in most
cases, interviewees in the current study did not indicate that they developed
formal a priori account balance expectations. Rather, they most often relied
on prior year balances as their comparison benchmark. The authoritative
literature (SAS No. 56, AICPA 1988, 73) and prior research (McDaniel and
Kinney 1995) suggest that developing a priori expectations is most effective.
Thus, while our ndings indicate that the profession has progressed with
respect to developing expectations, they also suggest that further progress is
needed in the development of expectations (RI 10).
Fifty-eight percent (21) note that directional expectations are discussed
by the entire audit team, while the remaining 42 percent indicate each team
member develops their own expectations (Q14). Thus, there appears to be a
divergence in practices. An interesting issue for future research is which of
the two approaches results in greater audit effectiveness and or efciency
(RI 11). Also, how does the development of expectations impact the fraud
brainstorming sessions required by SAS No. 99 (AICPA 2002) (RI 12)?
Finally, with respect to developing expectations and observing uc-
tuations, we inquired about various AP techniques (Q15). Specically,
respondents were asked how often they use various audit techniques in con-
ducting analytical procedures on a scale of 1 (rarely) to 7 (often) over the last
2 years, 27 years, and over 7 years ago. Looking at the most recent data
auditors indicate they use a variety of techniques: account balance compari-
sons (mean 6.5), ratio analysis (5.3), nonnancial information (4.5), client
budgets or forecasts (4.6), and interim data (5.2). Regression analysis, how-
ever, is rarely employed (1.8). The only apparent trend suggesting a change in
the use of techniques is an increase in the use of nonnancial information:
last 2 years 4.5, 27 years 3.6, and over 7 years 3.4 (RI 13). A few salient
statements indicate views on changes in the use of various techniques:

We tend to use [ratio analysis] more than we used to because theres


more comparisons that you can use against more data thats available.
Because everythings on the Internet now. (partner)

CAR Vol. 27 No. 2 (Summer 2010)


690 Contemporary Accounting Research

The only thing I would say changed is the use of client budgets or fore-
casts. There wasnt really much of a focus on looking at client budgets
and forecasts, in my experience anyway. (manager)

Finding and testing explanations


As noted earlier, a concern is an over-reliance on management explanations
(without gathering additional corroborating evidence). This was also a
concern expressed by Hirst and Koonce 1996. While our results, as will be
discussed, indicate that client explanation is still the primary source of
evidence to explain unexpected results from APs, it is unclear the extent to
which auditors fully accept the clients explanation. As noted in the follow-
ing discussion, client inquiry is frequently only one source of information
that the auditor evaluates to determine the sufciency of evidence.
When asked: Once you observe a material unexpected difference, what
do you do next? (Q16), the pervasive response (86 percent or 31 partici-
pants) is go to the client for an explanation (53 percent rst response, 33
percent second response). Other responses include talk to a superior or
other audit team member (33 percent), do more work (25 percent), and
check expectations (22 percent).
Participants were also asked how often they look for an explanation
from various sources (Q17). Consistent with the above response, partici-
pants indicate client inquiry is the most frequently used source, followed by
other audit team members, self-generation, and prior year workpapers.
Results also indicate that decision aids and checklists are rarely used. When
asked how frequently various client personnel are approached for an
explanation (Q18), interviewees indicated that accounting staff are consulted
more frequently than non-accounting staff. However, over the past sev-
eral years, there seems to be an increased willingness to approach non-
accounting staff, which is consistent with the recommendations of the POB
(RI 14).
In all, these ndings further corroborate the widespread reliance on cli-
ent inquiry in searching for an explanation of an unexpected difference.
Further, when asked: When would you not ask a client for an explanation?
(Q19), 61 percent indicate never or rarely, 31 percent note they prefer
to generate their own explanation rst, and 22 percent state when fraud is
suspected.
However, one should not necessarily assume that auditors approach the
client and blindly accept the given response. The following quotes provide
insight into why client explanations may be sought, and indicate that, at
least in some cases, client inquiry is strategically driven:

Even if I didnt think they knew the answer or I was concerned they
werent going to give me a straight answer I would ask them just to
check it. Put them on record. (partner)

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 691

And often times you can nd out . . . by sitting down, looking them in
the eye, and asking them a question. If you get a blank stare that prob-
ably tells you youve got a control problem because theyre not even
aware if it . . . . [In such cases,] Ive really got to do a lot more work
because you dont have the right level of review and supervision in clos-
ing the books. So I always start with the client inquiry, no matter what,
because you can just corroborate your own expectations. (partner)

Although there is a tendency to seek client explanations, of concern is


when auditors accept the clients explanation without seeking corrobora-
tion. When asked about this (Q20), only 69 percent of the interviewees
indicated that they seek independent supporting information. Such infor-
mation included industry data, prior experience with client, talk to other
client personnel, obtain third party corroborating evidence (36 percent),
and perform further testing (22 percent). Without further, case-specic
information, this is of concern, since the evidence suggests that, nearly
one-third of the time, the interviewee does not follow up by seeking
corroborating evidence.
On further examination, in many cases there was an acknowledgement
of the importance of corroborating evidence, yet interviewees spoke of eval-
uating the clients explanation for reasonableness in determining the next
step. In these situations, the interviewees noted that professional judgment
is of the essence, because the auditor must determine whether to gather
additional evidence and how much is needed. This judgment is evidenced in
the following quotes:

[I] go talk to the client . . . show them what Ive got, and say Why
do you think this has happened? to try to get some explanation that
makes sense to me. And that may lead me to do deeper analytics or
it may lead me to do more detail testing. But I think my rst reac-
tion is . . . go talk to the client and decide what we do from there.
(manager)

Start right in with the discussion with client? Actually no. We would
check to make sure that our analytic made sense and we didnt miss
anything. Then we would have the conversation with the client about
the difference. Then you start to move down, is there more we have to
do here? Do we need to start to understand these balances that may
have shifted the ratio? An example in real estate is a bad storm that
wasnt anticipated. Should we be taking a look at that and vouch that?
To look at that invoice for repairs to see if there is something funky.
Make sure what we are doing is right, then go to the client and start a
discussion, and then validate what they were saying. If it doesnt make
sense, then we go and test some balances. (partner)

CAR Vol. 27 No. 2 (Summer 2010)


692 Contemporary Accounting Research

With respect to the nature of the corroborating evidence that auditors


seek, our ndings suggest a potential bias. When asked: How often do you
try to nd evidence that refutes a client explanation (0100 percent)? (Q21),
the mean was only 28 percent of the time, and few individuals could think
of an example of such a situation. This nding suggests that there is signi-
cant risk that auditors may follow a largely conrmatory strategy in evalu-
ating client explanations (RI 15).
While the evidence related to reliance on management explanations pre-
sents concerns about whether auditors subject such explanations to in depth
scrutiny, there may be a change on the horizon as a result of SOX, as noted
by one respondent:

I think that a lot of people would argue that were moving more in that
direction [i.e., gathering corroborating evidence, and in particular, evi-
dence that attempts to refute managements explanation] now under the
PCAOB because theres a feeling that the PCAOB wants auditors to
have a higher level of professional skepticism and one way that you do
that is instead of trying to support what somebodys telling you, you try
to disprove what somebodys telling you. And I think clearly you see
traces of that in SAS 99 in the fraud standards . .. it doesnt say this
[directly], but therefore I think people would come to the conclusion
that you dont accept whats being told to you at face value. You try to
go out not to support it but to try and disprove it . . ., being very
cautious about not getting on a witch-hunt. (partner)

5. Conclusions and implications for practice and future research


Analytical procedures (APs) are potentially powerful tests to detect mis-
statements and fraud (Wright and Ashton 1989). This study examines audi-
tor experiences in conducting APs through interviews. The eld study
nature of the investigation allows us to assess whether analytical procedure
practices have changed substantially in recent years as a result of signicant
developments in the audit environment, identied as drivers (forces for
change) and enablers (facilitators for change).
Evidence from interviewees supports the expectation that recent scan-
dals and the Sarbanes-Oxley Act of 2002 have been important drivers of
change in the use of APs. Further, we nd that changes in technology and,
to a lesser extent, rm audit approaches have facilitated or enabled such
changes.
The ndings show that on some dimensions there has been little
change in practice since the benchmark Hirst & Koonce 1996 study. For
instance, auditors continue to rely a great deal on clients for information
(e.g., inquiry, budgets) to set expectations and evaluate explanations.
Further, relatively simple analytical techniques are often employed, such
as a comparison of the current year to the prior year, and decision aids

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 693

are rarely used. The rst result raises concerns about whether auditors
over-rely on client representations with an attendant conrmatory bias.
Thus, auditing rms need to consider ways to mitigate this risk, such as
requiring self-generation of expectations and explanations prior to client
consultation.
We nd, however, that there have been a number of changes in AP
practice since the Hirst and Koonce study. For instance, auditors appear
to consult nonnancial client personnel more than in the earlier study.
This development seems promising, since such personnel are more likely
to be objective and have a broader business focus than accounting staff,
potentially providing valuable information in setting expectations and eval-
uating explanations. This is likely due to the signicant shift in the audit
approach of major rms towards a broad business, strategic focus (e.g.,
Bell et al. 1997; Winograd et al. 2000). As such, auditors are likely to be
more knowledgeable about business processes and, thus more comfortable
talking with nonnancial staff than previously. Further study is needed
regarding when such personnel are approached and how the information
gathered is used.
Auditors appear to develop more precise, quantitative expectations
than found in Hirst and Koonce 1996. Participant responses suggest that
this is because of the wide availability of industry and client information
from databases. Thus, technology is playing a major enabling role in pro-
viding auditors with access to a broad array of information in developing
expectations and also in evaluating potential explanations for unexpected
uctuations. An important issue for further study is how auditors use such
information.
While interviewees suggest that they are more likely to develop quan-
titative expectations, there is little evidence that they formally develop a
priori account balance expectations in the absence of knowledge of unau-
dited account balances. While it is likely more efcient to audit an
account with prior knowledge of the clients pre-audit balance, it may not
be as effective. Future research might examine this efciency-effectiveness
trade-off.
While auditors tend to use simple APs, the current study also suggests,
as noted, that they gather and consider a broader array of industry and
company information than in the past, particularly nonnancial information
that is widely available through the internet and databases. An important
issue for future research is to examine how auditors integrate such informa-
tion (e.g., in an informal, unstructured manner) and whether structured
tools can provide assistance.
Auditors also report relying on APs to decrease testing more often
than in prior studies. They note this is because of a greater focus on
audit planning and initial risk analysis, allowing the rm to capitalize
more on APs in enhancing audit efciency. Additionally, less experienced
staff appear to be conducting (but not designing) a greater portion of the

CAR Vol. 27 No. 2 (Summer 2010)


694 Contemporary Accounting Research

APs. Future research is needed to determine the reason(s) for this change,
for example, more rapid development of staff under recent audit
approaches, or a move for greater efciency. This nding also raises con-
cerns about whether staff auditors have sufcient knowledge and training
to prociently perform APs, potentially negatively impacting audit effec-
tiveness. This also suggests that rms establish training on the perfor-
mance of APs to include case examples and techniques of nancial
statement analysis.
Finally, auditors report that SOX has been an inuential driver in lead-
ing to broader consideration and knowledge of internal controls, which is
seen as the most important factor in driving the use of and reliance on APs.
A signicant practice issue is whether auditors are taking advantage of this
enhanced knowledge in the design and use of APs.
Going forward another important issue is the long term effects of SOX
on auditors reliance on APs. Our ndings suggest four possibilities:
The enhanced knowledge of the controls will enable auditors to more
truly integrate the SOX 404 audit and the nancial statement audit, and
place even greater reliance on APs.
Auditors will rely more on detail tests and less on APs in response to
greater perceived legal liability posed by the Act and the belief that detail
tests are easier to justify in a court of law or for PCAOB inspections to
substantiate conclusions than APs.
Auditors may increasingly rely on APs for substantive tests in low-risk
areas, but use them as attention-directing procedures and rely on tests of
details in higher-risk areas.
The Act will result in greater focus on both APs and substantive tests due
to the combined effects of enhanced knowledge and legal risks.
Thus, research is needed to explore this issue and consider promising
ways of capturing the potential of the knowledge gained by auditors as a
result of extended control evaluation and testing to fully capitalize on APs
in enhancing audit efciency and effectiveness. Finally, an important
research issue is how auditors might best leverage the enhanced knowledge
of business risks, strategies, and processes obtained through the use of
more holistic audit approaches in conducting APs (Trotman and Wright
2006).
This research is a descriptive study of AP practices and how they have
changed in response to the audit environment. The interviews that were
conducted were primarily structured in nature. Future research using other
methods (e.g., unstructured interviews, eld studies, experiments) is needed
to corroborate and extend the ndings. Further, a study grounded in a
larger theoretical framework would be valuable and extend our under-
standing of the use of APs. For example, we nd that while auditors are
unwilling to rely on APs for areas of high risk, they are more likely to
rely on APs to reduce audit hours than has been reported in prior

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 695

research. However, auditors are still more hesitant to reduce hours than
to increase hours based on the results of APs. Perhaps future research
could rely on psychological theories of risk, such as expected utility theory
(e.g., Ellsberg 1961); prospect theory (e.g., Kahneman and Tversky 1979);
and or security potential aspiration theory (e.g., Lopes and Oden 1999),
to examine why this is so. Could it be the case that more precise, and
defensible, expectations could result in more efcient and effective APs
that, in turn, would result in greater reliance on the part of auditors (RI
16)?
Further, with respect to social theory of change, Cullinan and Sutton
(2002) suggest that the AICPA approved a series of Statements on Auditing
Standards in 1988 to better meet expectations of users of nancial state-
ments. Included in these standards was the acceptance of APs as substantive
tests. Although this was a source of concern for the Chief Accountant of
the SEC (Turner 1998, 10), it set the stage for the development of the top-
down, risk-based audit approaches that we argue may be enablers of
change. The claim that the AICPA adopted standards to better meet users
expectations seems at odds with the Chief Accountants concern that the
profession had sacriced effectiveness for efciency. This inconsistency
raises questions about how such changes came to be viewed as acceptable.
Perhaps future research could take advantage of models such as the neo-
institutional model of change offered by Hinings et al. 2004 to examine
how such changes become possible and then accepted by an entire eld such
as the accounting profession (RI 17).

CAR Vol. 27 No. 2 (Summer 2010)


696 Contemporary Accounting Research

Appendix
Relevant interview questions
Q1. Are you using analytical procedures (APs) more, less, or the same as in
the past?
If there is a change, what has driven it?
Q2. About what percentage of audit time is typically spent performing and
reviewing APs (by the entire audit team)? Please explain.

> 7 years ago 27 years ago Last 2 years


Planning
Performance?
Review?
Substantive Testing
Performance?
Review?
Overall Review
Performance?
Review?
Q3. On a seven point scale (1 = little focus, 7 = much focus) how much attention
do you give to APs as indicated? Please explain.

Corporate Division Product LOB 10-K 10-Q

Planning
Substantive testing
Overall review

Has this changed over the course of your career? Was this true ten years ago? Since
Sarbanes-Oxley?
Q4. Have you ever discovered anything with APs that otherwise would not have
been detected? Examples?
h Yes
h No
Q5. Do you ever reduce audit hours based on your AP ndings during planning?
h Yes
h No
Increase them?
Has this changed in recent years?
Q6. How do you decide whether APs provide more assurance than tests of details?
Q7. How do you decide whether APs would be more efcient than a test of details?
Q8. Who designs the APs?
h Staff _________
h Senior _________
h Manager _________
h Partner _________
100%

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 697

What is considered during the design?


Q9. Who typically performs APs? What percentage of APs are done by each?
h Staff _________
h Senior _________
h Manager _________
h Partner _________
100%
Q10. Does the stage of the audit (planning, overall review) have an impact on who
performs the procedures? Please explain.
h Yes
h No
Q11. Do you formally develop quantitative expectations (point estimate or range)
for account balances, ratios or other relationships? If so how?
h Yes
h No
Q12. What do you take into consideration in developing directional (e.g., sales
increase) expectations?
Q13. In using APs, what is your primary point of comparison?
Q14. Are these directional expectations discussed by the entire team or does each
team member develop their own expectations?
h Entire team
h Individual
Q15. How often do you use the following audit techniques (1 = rarely, 7 = often)
in conducting APs?

> 7 years ago 27 years ago Last 2 years

Account balance comparisons?


Ratio analysis?
Regression analysis?
Non-nancial information?
Client budgets or forecasts?
Interim data?
Q16. Once you observe a material unexpected difference, what do you do next?
Q17. How often do you look for an explanation from the following sources (using a
seven-point scale where 1 = Very infrequently and 7 = Very frequently)?
During planning
___Client inquiry?
___Decision aids?
___Checklists?
___Self-generation?
___Other audit team members?
___Prior year workpapers?
During substantive testing
___Client inquiry?
___Decision aids?
___Checklists?
___Self-generation?

CAR Vol. 27 No. 2 (Summer 2010)


698 Contemporary Accounting Research

___Other audit team members?


___Prior year workpapers?
___Current year workpapers?
During overall review
___Client inquiry?
___Decision aids?
___Checklists?
___Self-generation?
___Other audit team members?
___Prior year workpapers?
___Current year workpapers?
Q18. Who at the client would you typically talk to in order to get an explanation?
(Please respond to the items below using a 7-point scale where 1 = Very infre-
quently and 7 = Very frequently)
___Staff accountant
___Accounting manager
___Controller
___CFO
___Non-accounting staff
___Non-accounting manager
___Non-accounting senior executive
Q19. When would you not ask a client for an explanation?
Q20. In seeking corroboration of client explanations, what types of information do
you seek?
Q21. How often would you try to nd evidence that refutes a client explanation
(0100 percent)? Can you give an example of such a situation?

References
Agrawal, A., and S. Chadha. 2005. Corporate governance and accounting scandals.
Journal of Law and Economics 48 (2): 371406.
Allen, R., M. Beasley, and B. Branson. 1999. Improving analytical procedures: A
case of using disaggregated multilocation data. Auditing: A Journal of Practice
and Theory 18 (2): 12842.
Ameen, E., and J. Strawser. 1994. Investigating the use of analytical procedures:
An update and extension. Auditing: A Journal of Practice and Theory 13 (2):
6976.
American Institute of Certied Public Accountants (AICPA). 1988. Analytical
procedures. Statement on Auditing Standards No. 56 New York: AICPA.
American Institute of Certied Public Accountants (AICPA). 2002. Consideration of
fraud in a nancial statement audit. Statement on Auditing Standards No. 99
New York: AICPA.
Anderson, U., and L. Koonce. 1998. Evaluating the sufciency of causes in audit
analytical procedures. Auditing: A Journal of Theory & Practice 17 (1): 112.
Asare, S., G. Trompeter, and A. Wright. 2000. The effect of accountability and time
budgets on auditors testing strategies judgments. Contemporary Accounting
Research 17 (4): 53960.

CAR Vol. 27 No. 2 (Summer 2010)


Analytical Procedure Practices 699

Bedard, J. C., and S. Biggs. 1991. Pattern recognition, hypotheses generation, and
auditor performance in an analytical task. Accounting Review 66 (3): 62242.
Bell, T., M. Peecher, and I. Solomon. 2005. The 2lst century public company audit.
New York: KPMG.
Bell, T., F. Marrs, I. Solomon, and H. Thomas. 1997. Auditing organizations through
a strategic-systems lens. New York: KPMG Peat Marwick LLP.
Biggs, S., T. Mock, and P. Watkins. 1989. Analytical procedures and processes in
auditing. Monograph No. 14. Vancouver: CCGA.
Blocher, E., and J. Willingham. 1993. Analytical review: A guide to performing
analytical procedures. New York: McGraw-Hill.
Burns, J. 2007. SEC sets sights on ne-tuning audit standards. Wall Street Journal,
April 4, C2.
Cohen, J., and T. Kida. 1989. The impact of analytical review results, internal
control reliability and experience on auditors use of analytical review. Journal
of Accounting Research 27 (2): 26376.
Cullinan, C., and S. Sutton. 2002. Defrauding the public interest: A critical
examination of reengineered audit processes and the likelihood of detecting
fraud. Critical Perspectives in Accounting. 13 (3): 297310.
Ellsberg, D. 1961. Risk, ambiguity and the Savage axioms. Quarterly Journal of
Economics 75 (4): 64369.
Erickson, M., B. Mayhew, and W. Felix. 2000. Why do audits fail? Evidence from
Lincoln Savings and Loan. Journal of Accounting Research 38 (1): 16594.
Hinings, C., R. Greenwood, T. Reay, and R. Suddaby. 2004. Dynamics of change
in organizational eld. In Handbook of Organizational Change and Innovation,
eds. M.S. Poole and A. H. Van de Ven, 30423. New York: Oxford University
Press.
Hirst, E., and L. Koonce. 1996. Audit analytical procedures: A eld investigation.
Contemporary Accounting Research 13 (2): 45786.
Hylas, R., and R. Ashton. 1982. Audit detection of nancial statement errors.
Accounting Review 57 (4): 75165.
International Auditing and Assurance Standards Board (IAASB). 2008a.
Understanding the entity and its environment and assessing the risks of
material misstatement. International Standard on Auditing 315. New York:
IAASB.
International Auditing and Assurance Standards Board (IAASB). 2008b. The
auditors response to Aassessed risk. International Standard on Auditing 330.
New York: IAASB.
Kahneman, D., and A. Tversky. 1979. Prospect theory: An analysis of decision
under risk. Econometrica 47 (2): 26391.
Kinney, W. 2005. Twenty-ve years of audit deregulation and re-regulation: What
does it mean in 2005 and beyond? Auditing: A Journal of Theory & Practice 24
(Supplement): 89109.
Koonce, L. 1993. A cognitive characterization of audit analytical review. Auditing:
A Journal of Practice and Theory 12 (Supplement): 5776.

CAR Vol. 27 No. 2 (Summer 2010)


700 Contemporary Accounting Research

Lopes, L., and G. Oden. 1999. The role of aspiration level in risky choice: A
comparison of cumulative prospect theory and SPA. Journal of Mathematical
Psychology. 43 (2): 286313.
McDaniel, L., and W. Kinney. 1995. Expectation-formation guidance in the
auditors review of interim nancial information. Journal of Accounting
Research 33 (1): 5976.
Nelson, M., and H. Tan. 2005. Judgment and decision making research: A task,
person, and interpersonal interaction perspective. Auditing: A Journal of
Practice and Theory 24 (Supplement): 4171.
Palmrose, Z., and S. Scholz. 2004. The accounting causes and legal consequences of
non-GAAP reporting: Evidence from restatements. Contemporary Accounting
Research 21 (1): 13980.
Public Oversight Board (POB). 2000. The Panel on Audit Effectiveness report and
recommendations.
Reilly, D. 2006. Moving the market: Audit watchdog proposes easing rules on
controls. Wall Street Journal, December 20, C3.
Sarbanes-Oxley Act. 2002. Public Law 107204, H.R. 3763.
Scannell, K. 2007. Moving the market: Softening a Sarbanes-Oxley thorn SEC
makes progress on tweaking the rules for unpopular provision. Wall Street
Journal, April 5, C2.
Securities and Exchange Commission (SEC). 1999. Materiality. Staff Accounting
Bulletin 99. Washington, DC: SEC.
Trotman, K., and W. Wright. 2006. Use of MBR, MII, and EBS evidence for risk
assessments: Scenarios for use of audit evidence. Working paper, University of
Illinois.
Turner, L. 1998. Letter from the Ofce of the Chief Accountant regarding the
19981989 audit risk alerts. Washington DC: SEC.
Winograd, B. N., J. S. Gerson, and B. L. Berlin. 2000. Audit practices of
Pricewaterhouse Coopers. Auditing: A Journal of Practice and Theory 19 (2):
17582.
Wright, A., and R. Ashton. 1989. Identifying audit adjustments with
attention-directing procedures. Accounting Review 64 (4): 71028.

CAR Vol. 27 No. 2 (Summer 2010)


Copyright of Contemporary Accounting Research is the property of Canadian Academic Accounting
Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email articles for
individual use.

You might also like