Comment Agile Testing

Definition

Testing approach on development projects using agile methods and paradigm

Organisations looking to get the most value from testing are often also interested in adopting software
development processes which reduce risks. Agile is a term which embraces a number of lifecycle models,
characterised by iterative development and cross-disciplinary working. These can reduce risk by providing
faster and clearer information on which to base project decisions. One facet of agile development is the
reduction in compartmentalisation of testing within the overall software development lifecycle. More
information can be found from the the Agile Alliance.
Tool vendors are moving to support agile development and testing within a unified environment. For
example, Microsoft's Visual Studio 2005 includes an integrated load testing tool. On the Open Source side,
Eclipse includes a web performance testing plug-in and Eclipse supports performance testing tools such as
JMeter (as well as proprietary tools including IBM's Rational Performance Tester).

Agile testing is a software testing practice that follows the statutes of the agile manifesto, treating software
development as the customer of testingAgile testing involves testing from the customer perspective as early
as possible, testing early and often as code becomes available and stable enough from module/unit level
testingSince working increments of the software is released very often in agile software development there
is also a need to test often. This is often done by using automated acceptance testing to minimize the
amount of manual labor. Doing only manual testing in agile development would likely result in either
buggy software or slipping schedules because it would most often not be possible to test the whole software
manually before every release

Agile software development is a conceptual framework for software engineering that promotes
development iterations throughout the life-cycle of the project.

There are many agile development methods; most minimize risk by developing software in short amounts
of time. Software developed during one unit of time is referred to as an iteration, which may last from one
to four weeks. Each iteration is an entire software project: including planning, requirements analysis,
design, coding, testing, and documentation. An iteration may not add enough functionality to warrant
releasing the product to market but the goal is to have an available release (without bugs) at the end of each
iteration. At the end of each iteration, the team re-evaluates project priorities.

Agile methods emphasize face-to-face communication over written documents. Most agile teams are
located in a single open office sometimes referred to as a bullpen. At a minimum, this includes
programmers and their "customers" (customers define the product; they may be product managers, business
analysts, or the clients). The office may include testers, interaction designers, technical writers, and
managers.Agile methods also emphasize working software as the primary measure of progress. Combined
with the preference for face-to-face communication, agile methods produce very little written
documentation relative to other methods. This has resulted in criticism of agile methods as being
undisciplined.

Contents:

 History,Principles behind agile methods — The Agile ManifestComparison with other

method,Suitability of agile methods,Agile data,Agile methods and method tailoring,Agile
methods,Measuring agility,Criticisim,Agile Principles

) In an agile environment, there is a strong focus on unit testing through automated test frameworks, of
which JUnit and its relatives are examples. Many agile environments use test-driven design, in which the
developers write tests that are designed to fail until code has been written to pass those tests successfully.
These tests, written by the developers, serve at least two highly important purposes. First, they help to
for the way in which the product should behave (that is, they are example-focused). drive
the design of the product by providing a set of conditions that the unit-level code must fulfill to be
considered complete. Second, they serve as powerful change detectors, so that the product can be
refactored and retested with confidence and ease; after each modification to the code, programmers run all
of the unit tests, and the unit tests have to pass before further work is done. In my observation and
experience, this is an extremely powerful and robust method for developing software, and it doesn't require
an agile project to use it.

(In fact, expert developers have been following practices like these for a long time. In the early days of
computing, there was a heavy focus on unit testing, at least in part because computer time was expensive;
computers and development environments weren't anything like as rapid and interactive as they are now;
and also because the hardware wasn't sufficiently reliable to trust without unit tests. Somewhere along the
line these practices got lost, and the agile development community has done a lot to reinvent and restore
them. Bravo.)

b) In an agile environment, there is a strong focus on developing user- or business-oriented tests up front;
these are often called acceptance tests, and they're typically written by customers or their representatives--
testers or business analysts. In the same way that unit tests are created for the functional units within it,
acceptance tests are oriented towards integrated or system-level views of the product. I observe that there
are two slightly different nuances. Sometimes user tests are designed as examples both example- and
milestone-focused. In an agile environment, typically these tests are automated. Agile teams often use
frameworks such as FIT or FitNesse, in which Sometimes, user tests used as benchmarks by which we
might assess that a given feature or set of features is complete (that is, they are milestone-focused). Often--
but not always--these tests are descriptions of the intended behaviour of the product are intermingled with
tables that give examples of intended functionality; developers code "fixtures" to allow the tables to interact
with code from the product; and the framework then allows the user to exercise the code in question and to
observe actual results. These tests tend to be very similar in nature to use cases, but are often more detailed
and elaborate thanks to the tables and the fact that the code is runnable. Agile teams might also use
automation tools such as WATIR, which drives Web applications using (as of this writing, Internet Explorer
and) the Ruby programming language. As with unit tests, when new functionality is implemented, these
tests are run to demonstrate functionality; and when code is modified, these tests are re-run in order ensure
that that which was working previously is still working.

I observe several potential problems with implementing this kind of testing. Outside of an agile context,
these tests are difficult to implement when automation is part of the picture and developers are not inclined
or mandated to colloborate with the testing process; FIT and FitNesse in particular require lots of help from
the developers. Second, even in an agile context, these tests require rather more involvement from
customers or customer representatives, and may depend on these people learning to use the tool when they
don't have the time or the inclination to do so. Third, these tests tend to be more confirmatory and less
investigative; they tend to be focussed on demonstrating that the product works, rather than checking to see
whether the product fails.

These processes can be implemented in a non-agile projects, and the risks above can be addressed, but it's
important to be aware of them.

• c) There is another way of thinking about testing in a agile way (note the reframe and the absence
of the capital "A"). I believe that can be done with agility in any project where the skills of the
tester and the context of the project support it. Such testing would, in my view, include

• lots of quick tests that have very low cost but uncertain value
• extremely rapid feedback to the developers, but also to the rest of the project community minimal
planning up front, in deference to learning more and focusing the practice as the project goes on and
we learn more about it
• generally preferring test ideas over test scripts
• generally focusing on investigation vs. confirmation
• lightweight approaches to test documentation
• with respect to automation, tests that can be rapidly and easily developed and evaluated by a machine
are automated; tests that have a high dependence on human cognition and evaluation may be executed
and observed manually--and automation is used to assist that effort; and testers continuously develop
and use automation skill to the extent that it helps themselves and the project.

• Testers develop and use skills of critical thinking, general systems thinking, and context-driven
thinking to identify risk, to create powerful tests, and to provide valuable information to
management so that it can make informed decisions about the product and the project.