World Health Report 2012:

No Health Without Research

Thematic collections :: Volume 2

World Health Report 2012:
No Health Without Research

Thematic collections :: Volume 2

 World Health Report 2012: No Health Without Research, 107p
iMedPub :: Thematic collections :: Volume 2

ISBN-13: 978-1461146223
ISBN-10: 1461146224

First Edition, 2011

Cover illustration: Jorge Quinteros

Design and layout: Marta Araya Marroni

bkmarta@gmail.com

Publisher: Internet Medical Publishing

http://www.imedpub.com/

Disclaimer: This book contains articles published under Creative Commons Attribution License. Creative
Commons Attribution License allows commercial re-use of all content.
 Foreword

It seems astonishing that in the 21st century decisions on

health care can still be made without a solid grounding

in research evidence. This is true even in clinical research,

whether for simple or complex interventions, where

systematic reviews time and time again conclude that the

evidence base is inadequate. It is even more true in the areas

of health policy and health systems, where quality research

is hampered further by a lack of shared definitions, a lack of

consensus on guiding principles, poor capacity (especially in

low-resource regions), and methodological challenges.

The WHR 2012 aims to provide impetus for a change to the

problematic state of affairs of health research. Given the

stated goals of the report, of particular importance is the

documentation and sharing of real experiences from the

countries where the research has been done.

Text taken from “Pang T, Terry RF, The PLoS Medicine Editors 2011 WHO/PLoS
Collection “No Health Without Research”: A Call for Papers. PLoS Med 8(1)”
World Health Report 2012: No Health Without Research

Contents
Debate
Are Patents Impeding Medical Care and Innovation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Viewpoints
Clinical Research in Resource-Limited Settings: Enhancing Research Capacity and Working Together to
Make Trials Less Complicated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Essays
Evidence-Based Priority Setting for Health Care and Research: Tools to Support Policy in Maternal, Neonatal,
and Child Health in Africa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Improving Implementation: Building Research Capacity in Maternal, Neonatal, and Child Health in Africa . . . . 15

Closing the Gaps: From Science to Action in Maternal, Newborn, and Child Health in Africa . . . . . . . . . . . . . . . . 18

Health in Action
AIDS Vaccine for Asia Network (AVAN): Expanding the Regional Role in Developing HIV Vaccines . . . . . . . . . . . 21

A Field Training Guide for Human Subjects Research Ethics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Developing ANDI: A Novel Approach to Health Product R&D in Africa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Laboratory Capacity Building in Asia for Infectious Disease Research: Experiences from the South East Asia
Infectious Disease Clinical Research Network (SEAICRN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Policy Forums
Registering New Drugs for Low-Income Countries: The African Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Defining Research to Improve Health Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

The Global Health System: Lessons for a Stronger Institutional Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

The Global Health System: Linking Knowledge with Action—Learning from Malaria . . . . . . . . . . . . . . . . . . . . . . . . 60

Guidelines and Guidance

Setting Research Priorities to Reduce Almost One Million Deaths from Birth Asphyxia by 2015 . . . . . . . . . . . . . . 66

Setting Implementation Research Priorities to Reduce Preterm Births and Stillbirths at the Community Level. . 77

Review
A Research Agenda to Underpin Malaria Eradication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Research Articles
NIH Disease Funding Levels and Burden of Disease . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Geographical Representativeness of Published and Ongoing Randomized Controlled Trials. The Example of:
Tobacco Consumption and HIV Infection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

Strategies for Increasing Recruitment to Randomised Controlled Trials: Systematic Review . . . . . . . . . . . . . . . . . 113

Development of and Access to Products for Neglected Diseases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Relation between the Global Burden of Disease and Randomized Clinical Trials Conducted in Latin America
Published in the Five Leading Medical Journals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Research Ethics Training in Peru: A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

iv iMedPub :: Thematic Collections :: Volume 2


The PLoS Medicine Debate
1
Are Patents Impeding Medical Care and Innovation?
E. Richard Gold1*, Warren Kaplan2*, James Orbinski3*, Sarah Harland-Logan4, Sevil N-Marandi5
1 Faculty of Law, McGill University, Montreal, Quebec, Canada, 2 Department of International Health, The Boston University School of Public Health, Boston,
Massachusetts, United States of America, 3 St. Michael’s Hospital, University of Toronto, Toronto, Canada, 4 Harvard University, Cambridge, Massachusetts, United States
of America, 5 University of Toronto, Toronto, Canada
Background to the debate: Pharmaceutical and medical
device manufacturers argue that the current patent
system is crucial for stimulating research and develop-
ment (R&D), leading to new products that improve
medical care. The financial return on their investments
that is afforded by patent protection, they claim, is an
incentive toward innovation and reinvestment into
further R&D. But this view has been challenged in recent
years. Many commentators argue that patents are stifling
biomedical research, for example by preventing research-
ers from accessing patented materials or methods they
need for their studies. Patents have also been blamed for
impeding medical care by raising prices of essential
medicines, such as antiretroviral drugs, in poor countries.
This debate examines whether and how patents are
impeding health care and innovation.
E. Richard Gold’s Viewpoint: We Could Increase
the Productivity of Biomedical Innovation
Systems by Rethinking How We Use Patents
The question posed in the title of this PLoS Medicine debate
seems to be a simple one, but there is a complex spectrum of
answers depending on how one interprets the question. In this
article, I lay out four different interpretations and their
corresponding answers.
How Are Existing Patent Rights Impeding Medical Care
and Innovation?
The narrowest version of the question focuses on the effect of
existing patents held by actors (industry, university, government
laboratories, etc.) on medical care and innovation.
In high-income countries, the evidence suggests that existing
patents increase the cost of medicines [1]. Whether patents
increase the cost of other services, such as diagnostics, is unclear
[2]. For example, in their recent analysis of patents on genetic
testing, Robert Cook-Deegan and colleagues concluded that
‘‘prices of patented and exclusively licensed tests are not
dramatically or consistently higher than those of tests without a
monopoly’’ [2]. What impact do existing patents have on the total
cost of medical care in rich countries? Again, the evidence is
unclear. Patents could conceivably reduce the total cost of care if
new patented medicines turn out to be cheaper than existing
medical interventions.
In those low- and middle-income countries in which current
medications are subject to patent rights, existing patents seem to
make medicines more expensive and increase the difficulty of
creating novel mechanisms through which to deliver medicines
[3,4]. In all countries, existing patents make research and
development more expensive for the simple reason that research-
ers and companies must clear patent rights to do their work.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
Whether this cost is offset by other benefits is a subject I turn to
next.
How Is The Prospect of Obtaining Patent Rights
Impeding Medical Care and Innovation?
The theory underlying patent rights is that patents encourage
people to invest in bringing a compound through clinical trials and
into practice [5,6]. The prospect of future patents may, therefore,
increase innovation today and may increase medical care by
encouraging manufacturers to introduce new medicines [7]. While
pharmaceutical companies spend almost twice as much on
marketing than on research [8], they nevertheless invest heavily
in developing new medicines.
Two questions remain, however. First, while patents provide an
incentive to bring a new product to market, are these incentives
better than those provided by alternative mechanisms? We know
that existing business strategies of both pharmaceutical and
biotechnology companies rely heavily on patents [6,9], but this
does not prove that they could not have developed strategies that
did not rely on patents. It appears that the biomedical industry’s
reliance on patents is historically arbitrary [10], rather than being
necessary to spur innovation. So, for example, would a prize
awarded to those who discover new medicines be a better
mechanism than using patents [11]? Neither theory nor evidence
provides a clear answer. Second, are the benefits of patents in
encouraging the development of new medicines offset by the
increased prices we pay for existing medicines and by the higher
fees that researchers must pay? Again, empirical research is
inconclusive but is strongest in the biomedical sector [10]. In the
end, we have no better answer today than in the 1950s when
economists Edith Penrose and Fritz Machlup concluded that the
evidence supporting or undermining the patent system is lacking
[12,13].
Citation: Gold ER, Kaplan W, Orbinski J, Harland-Logan S, N-Marandi S (2009) Are
Patents Impeding Medical Care and Innovation? PLoS Med 7(1): e1000208.
doi:10.1371/journal.pmed.1000208
Published January 5, 2010
Copyright: � 2009 Gold et al. This is an open-access article distributed under
the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the
original author and source are credited.
Funding: The authors received no specific funding to write this piece.
Competing Interests: ERG conducts research on intellectual property policy
with the Structural Genomics Consortium.
Abbreviations: HCT, health care technology; IP, intellectual property; R&D,
research and development; TRIPS, Trade in Intellectual Property Rights
Agreement.
* E-mail: richard.gold2@mcgill.ca (ERG); wak@bu.edu (WK); james.orbinski@
utoronto.ca (JO)
Provenance: PLoS Medicine initially commissioned two Viewpoints (those by
Warren Kaplan and by James Orbinski and colleagues) which were externally peer
reviewed by Richard Gold. The editors then commissioned Richard Gold to write a
third Viewpoint, which was not externally peer reviewed.
1 January 2009 | Volume 7 | Issue 1 | e1000208

World Health Report 2012: No Health Without Research
How Is The Patent System Impeding Medical Care and
Innovation?
If we look at the outcomes of biomedical innovation, a different
answer emerges. The patent system—not just patent rights but
how they are obtained and used—has resulted in an innovation
system characterized by a dramatic increase in health care costs
and decreasing (quantitatively and qualitatively) levels of innova-
tion, especially by dollar spent [9]. While one cannot say that these
problems are inherent in patent law they are, nevertheless, an
outcome of the manner in which actors deploy patent rights.
The evidence points to a crisis in biomedical innovation even if
not to a solution. While health care costs are increasing rapidly,
the fastest growing component of those costs are pharmaceutical
products [14]. The costs of developing a new medicine from
discovery through clinical trials appear to double every decade
[15]. Yet, despite increasing investments in research and
development, industry is producing fewer new drugs every year
of which a declining percentage is truly innovative [16]. Beyond
this, investments in the health needs of developing countries
remains very low by any standard, and patents continue to get in
the way of modifying existing medicines for the needs of those
countries [3].
All of this shows an industry in serious difficulty and a health
care system facing unsustainable cost increases and fewer new
products. There are many reasons for this crisis that stretch well
beyond the patent system. To the extent, however, that the
industry’s current business models are build around patents, the
patent system itself must shoulder its share of responsibility.
Would a Different Deployment of Patents Impede Health
Care and Innovation?
While some nongovernmental organizations call for the creation
of alternatives to the patent system, such as prize systems [11],
these are even less proven than the patent system in sustaining
innovation and increasing health care. A more practical solution to
the problems described above is not to do away with patent rights
but to use them more wisely. This means everything from deciding
not to apply for patents over certain inventions, as does the
Structural Genomics Consortium [17], to licensing out patent
rights widely for research purposes, as does the Health Commons
(http://www.healthcommons.net/), through to exploring the
possibility of creating ‘‘open source’’ licenses in biomedicine
[18]. None of these strategies require changing patent laws, but all
present a serious challenge to the way that universities and
industry obtain and deploy patent rights.
Conclusion
There are many questions buried within the question ‘‘How are
patents impeding medical care and innovation?’’ At bottom, we do know
that our biomedical innovation system is too expensive and too
unproductive and that patents play an important role within that
system. If, in the end, it is results that count, then it is certainly
time to question the business models that sit on top of our existing
patent system [9,18].
Warren Kaplan’s Viewpoint: The Evidence on
Whether Patents Impede Medical Innovation Is
Ambiguous
The complicated debate about whether or not patents impede
‘‘downstream’’ medical care and ‘‘upstream’’ medical innovation
is ultimately about access to such care and innovation, which are at
opposite ends of a ‘‘chain’’ of biomedicine.
PLoS Medicine | www.plosmedicine.org
 iMedPub :: Thematic Collections :: Volume 2
Access to Medical Care
Clinical research is costly, lengthy, and high-risk. Pharmaceu-
tical companies apply for patents on new drugs to gain market
exclusivity for a limited period. The aim of this exclusivity is to
generate revenue from sales and recoup the substantial cost of
drug development by collecting fees (‘‘royalties’’) from users of the
patented technology. Pharmaceutical companies also fund their
research from these fees. Patented pharmaceuticals cost more than
identical medicines that are off-patent. Access to medicines is
inhibited by high prices. Patents are a factor in inhibiting access to
pharmaceutical treatment, particularly in low- and middle-income
countries [8,20–22].
For medical devices, large manufacturers continue to benefit
from price increases on patent-protected devices. As with
pharmaceuticals, iterative improvements in performance and
safety of existing devices are typically patented. These patents
can, in principle, create barriers to market entry for new
competitors. Nonetheless, the literature on the impact of patents
on access to medical devices is slim compared to that for
pharmaceuticals.
The impact of gene patents on genetic testing has garnered
much recent press and academic interest [23–26], including a
lawsuit by the American Civil Liberties Union against Myriad
Genetics charging that patents on two human genes associated
with breast and ovarian cancer stifle research that could lead to
cures [27]. The lawsuit argues that the patents on these genes are
unconstitutional and invalid. But in fact, there is little in the way of
consistent evidence to suggest that gene patents inhibit patient
access to diagnostic tests, at least in the United States [24]. Even
so, when Mildred Cho and colleagues interviewed 132 directors of
clinical genetic testing laboratories, 53% of respondents reported
that patents or licenses had impeded their ability to develop and
provide genetic tests [28].
Patents are a critical factor affecting access to medical care, but
they are not the only factor. Other factors influencing medical care
include demand for a product and market size (e.g., a large market
and high demand for a product might lead to considerable
revenue for the company even at a lower price),
Access to Innovation
The proliferation of patents may block biomedical R&D
because researchers are unable to obtain the many different
permissions required (e.g., permission may be required to use
patented reagents, to try a patented method, and/or use a
patented device) [28]. This situation has been called the ‘‘antic-
ommons’’ problem [29]—R&D is inhibited by the presence of
many intellectual property owners’ exclusive and possibly
conflicting rights over devices and methods needed to perform
R&D on biomedical products.
However, there is little empirical evidence that an anticommons
problem is impeding innovation. For example, the French
Community Innovation Survey found that 14% of R&D
collaborating firms had to abandon or delay their innovation
projects because of difficulties in their partnerships (‘‘cooperation
failures’’), and the survey explored reasons for such failures.
Intellectual property rights were not a cause of cooperation
failure—in fact, the authors found that ‘‘industries where firms are
able to better appropriate their research results (through patents,
models and secrecy) present lower rates of ‘cooperation failures’’’
[30].
In their analysis of how patents affect medical innovation in
Australia, Dianne Nicol and Jane Nielsen concluded that ‘‘in
general the Australian industry seems to be avoiding an antic-
ommons situation, but the potential still exists for its emergence’’
2 January 2009 | Volume 7 | Issue 1 | e1000208

[31]. In the US and Japan, there is also very little evidence of an
anticommons problem preventing innovation [32]. John Walsh
and colleagues surveyed 507 academic biomedical researchers,
asking them about the impact of patents on access to the
knowledge and material inputs that are used in subsequent
research [33]. The authors concluded that ‘‘access to knowledge
inputs is largely unaffected by patents.’’ A survey of 70 attorneys,
scientists, and managers in the biomedical research industry did
not find evidence of the anticommons problem [34].
In contrast, Stephen Meurer has shown that the anticommons
problem prevented a group of about 100 academic biologists from
building a worldwide human mutations database [35]. The
biologists tried to trade their data for corporate support of the
database. Although they received an offer of US$2.3 million, a
deadlock occurred because most members of the group could not
afford the information costs needed to reach a decision—a
prediction of the anticommons.
Access to materials and/or data—such as cell lines, reagents,
genetically modified animals, and unpublished information—can
be restricted if these are owned by other researchers. In a survey of
agricultural biologists [36], and based on my own experience in
biotechnology, delayed or blocked access to such materials results
from having to negotiate material transfer agreements (the
University of California, Berkeley defines a material transfer
agreement as ‘‘a contract that governs the transfer of tangible
research materials between two organizations, when the recipient
intends to use it for his or her own research purposes’’ [37]).
Restrictions on access do not appear to depend on whether the
material is itself patented [38]. Typically, no issued patents exist on
such materials covered by these material transfer agreements. But
it is the possibility of future patent protection and the desire on the
part of the supplier to manage this uncertainty that slows down or
even eliminates such transfers of technology.
Conclusion
What are we to make of all this? The actual evidence on
whether patents impede innovation or inventiveness in biomed-
icine is, in a word, ambiguous. Yet firms clearly tend to avoid
research projects for which there are many existing patents [39].
Both the process of determining which potentially relevant patents
are important to a research project and the negotiations for access
to them can delay, but less often kills, innovation. In industry and
universities, researchers adopt strategies of ‘‘licensing, inventing
around patents, going offshore, the development and use of public
databases and research tools, court challenges and … using the
technology without a license (i.e. infringement) to achieve their
particular goals’’ [39].
This raises the question, What are these various ‘‘design
around’’ actions manifestations of, if not actual patent blockages or
threats of the same? We act as if the anticommons block to
innovation is real. Perception is reality. Patents, or perhaps only
the fear of their enforcement, inhibit biomedical innovation. If we
knew how strong the inhibition really was, we would be having a
different debate.
James Orbinski’s, Sarah Harland Logan’s, and
Sevil N-Marandi’s Viewpoint: Patents Skew
Biomedical Research Toward Problems of the Rich
World
If patents represent a bargain between the claimant to
intellectual property (IP) and the state, and on balance should
benefit society, a key question in this age of globalization is ‘‘which
society?’’ The United Kingdom’s Royal Society, an independent
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
academy of science, rightly argues that ‘‘uses of intellectual
property that benefit people in one part of the world but
conspicuously fail to benefit others, or even act to their detriment,
are not what the [patent] system is supposed to be about’’ [40].
For developing countries, patents can impede medical care by
pricing medicines and other health care technologies (HCTs) out
of the reach of patients or their health care systems. Pharmaceu-
tical companies have little interest in pricing drugs for developing
country markets because they are seeking to maximize global not
national profits, and do not want to set a low price precedent that
would increase demand in wealthy countries for similar low prices
[41]. For those with a purchasing power less than what is needed
to meet minimal needs—i.e., most of the 3.8 billion people who
live on less than US$2 per day [42]—access to HCTs is little more
than a discomforting dream. Further, if a treatment is too
expensive, other factors that can affect medicines availability, such
as drug distribution systems and rational drug use policies, become
moot. Indeed, it was only when generic competition lowered the
price of antiretroviral therapy for HIV—from more than
US$15,000 per patient per year in 2001 to less than US$99 in
2007—that the policy debate shifted from whether such therapy
was possible in resource-poor settings to how to strengthen health
infrastructure to provide comprehensive HIV health care for
people in such settings [43,44].
To increase access to existing HCTs, governments can make use
of fully legal safety provisions of the World Trade Organization’s
Trade in Intellectual Property Rights Agreement (TRIPS). These
provisions include compulsory licensing, which allows a govern-
ment to force a drug company to license its patent to a local
generic producer who must pay a royalty to the patent holder. But
a government is allowed to issue a compulsory license only after
price negotiations with the patent holder have failed. Nevertheless,
compulsory licensing remains a valuable tool, as memorably
shown in 2001 when South Africa issued compulsory licenses to
produce selected anttiretroviral drugs. Although 39 pharmaceuti-
cal companies attempted to sue South Africa’s government for
allegedly infringing on their patent rights, they ultimately chose to
withdraw this lawsuit in the face of immense public pressure [45].
The confrontation led the World Trade Organization to issue its
November 2001 Doha Declaration, which affirmed that ‘‘the
TRIPS Agreement does not and should not prevent members
from taking measures to protect public health’’ [46].
Current patent laws also skew biomedical research to products
that yield high profits rather than to global priority health needs in
both developed and developing countries. Currently, malaria,
pneumonia, diarrhea, and tuberculosis, which together account for
21% of the global disease burden, receive 0.31% of all public and
private funds devoted to heath research [47,48]. More than 1
billion people—the overwhelming majority of whom are in the
developing world—suffer from neglected tropical diseases, those
for which there are inadequate or nonexistent treatments and a
paucity of research and development [49]. Of the 1,556 new
pharmaceutical compounds that appeared on the market between
1975 and 2004, just twenty of these drugs—1.3%—were for
tropical diseases and tuberculosis [50].
The international debate around patents has been largely
framed in terms of ‘‘protection for’’ versus ‘‘access to’’ IP. If the
framing of the debate shifts to a focus on research and
development, this is likely to strengthen the leverage of developing
countries to change the dynamics of IP negotiations in trade
agreements [51]. Entirely shifting the debate from IP rights to the
R&D gap may help tackle the fundamental problem of a
monopoly-based innovation and access system. One example is
nonexclusive licensing practices, such as those used by the not-for-
3 January 2009 | Volume 7 | Issue 1 | e1000208

World Health Report 2012: No Health Without Research
profit Drugs for Neglected Diseases Initiative (http://www.dndi.
org/). The initiative finances R&D up front and offers the
outcome of its research on a nonexclusive basis to generic
producers, allowing for technology transfer and competition
among multiple producers [51]. Furthermore, universities cur-
rently hold important patents on many life-saving drugs, including
the antiretroviral drugs stavudine (Yale University), abacavir
(University of Minnesota), lamivudine (Emory University), and
enfuvirtide (Duke University) [52]. In recognition of these
university patents, Universities Allied for Essential Medicines
(http://www.essentialmedicine.org) proposes that ‘‘when a uni-
versity licenses a promising new drug candidate to a pharmaceu-
tical company, it should require that the company allow the drug
to be made available in poor countries at the lowest possible cost’’
[53]. Another alternative to overcoming current patent barriers is
the use of patent pools, as proposed by the WHO, Médecins Sans
Frontières, and UNITAID [54,55]. Here, a number of patents
held by different entities, such as companies, universities, or
research institutes, are pooled and made available to others for
production or further development—of, for example, pediatric
formulations or fixed-dose formulations. The patent holders
receive royalties that are paid by those who use the patents. The
pool manages the licenses, the negotiations with patent holders,
and the receipt and payment of royalties.
Other innovative policy proposals, such as the Heath Impact
Fund (a strategy to create a publicly funded ‘‘pot of gold’’ that
would attract the private sector to create R&D innovations that
References
1. Saha A, Grabowski H, Birnbaum H, Greenberg P, Bizan O (2006) Generic
Competition in the US Pharmaceutical Industry. Int J Econ Bus 13: 15–38.
2. Cook-Deegan R, Chandrasekharan S, Angrist M (2009) The dangers of
diagnostic monopolies. Nature 458: 405–406.
3. Gold ER, Piper T, Morin J-F, Durell LK, Carbone J, et al. (2007) A Preliminary
Legal Review of Proposed Medicines Patent Pool. Montreal: The Innovation
Partnership, Available: http://www.theinnovationpartnership.org/data/
documents/00000003-1.pdf. Accessed 17 September 2009. 161 p.
4. Attaran A (2004) How Do Patents And Economic Policies Affect Access To
Essential Medicines In Developing Countries? Health Aff 23: 155–166.
5. Grabowski H (2002) Patents, Innovation and Access to New Pharmaceuticals.
J Int Econ Law 5: 849–860.
6. Kieff FS (2008) On the Economics of Patent Law and Policy. In: Takenaka T, ed.
Patent Law and Theory: A Handbook of Contemporary Research. Northampton:
Edward Elgar Publishing. pp 3–65.
7. Gagnon MA, Lexchin J (2008) The cost of pushing pills: A new estimate of
pharmaceutical promotion expenditures in the United States. PLoS Med 5: e1.
doi:10.1371/journal.pmed.0050001.
8. Lanjouw JO (2005) Patents, price controls and access to new drugs: How policy
affects global market entry. NBER Working Paper 11321. Available: http://
www.nber.org/papers/w11321.pdf.
9. Munos BH, Chin WC (2009) A Call for Sharing: Adapting Pharmaceutical
Research to New Realities. Sci Transl Med 1: 9cm8. doi:10.1126/
scitranslmed.3000155.
10. Hall B (2007) Patents and patent policy. Oxford Rev Econ Policy 23: 568–587.
11. Love J, Hubbard T (2007) The Big Idea: Prizes to stimulate R&D for new
medicines. Chicago-Kent Law Rev 82: 1519–1554.
12. Penrose E (1951) The Economics of the International Patent System. Baltimore:
Johns Hopkins Press. 247 p.
13. Machlup F (1958) An Economic Review of the Patent System, Study No.15 of
Committee on Judiciary, Subcommittee on Patents, Trademarks, and
Copyrights, 85th Cong., 2d Sess. Washington, D.C.: U.S. Government Printing
Office.
14. Canadian Institute for Health Information (2009) Drug Expenditure in Canada
1985–2008. Ottawa: Canadian Institute for Health Information. 147 p.
15. DiMasi JA, Hansen RW, Grabowski H (2003) The price of innovation: new
estimates of drug development costs. J Health Econ 22: 151–185.
16. Gagnon M-A (2009) The Nature of Capital in the Knowledge-Based Economy:
The Case of the Global Pharmaceutical Industry. PhD Dissertation in Political
Science, York University.
17. Edwards AM, Bountra C, Kerr TJ, Willson TM (2009) Open access chemical
and clinical probes to support drug discovery. Na Chem Biol 5: 436–440.
18. Hope J (2008) BioBazaar: The Open Source Revoluation and Biotechnology.
Cambridge: Harvard University Press. 448 p.
PLoS Medicine | www.plosmedicine.org
 iMedPub :: Thematic Collections :: Volume 2
effectively address priority global heath needs) [56], should be
implemented. However, using patents as the financial incentive to
encourage the pharmaceutical industry to develop drugs for the
world’s poor is of limited use where the market is nonexistent
because neither governments nor patients can afford the end
product [57]. Instead, framing the issue around global R&D, as
opposed to international IP rights, will aid in developing public–
private partnerships and a set of novel policy alternatives that
support approaches to addressing the public health needs of
developing nations [58].
The patent system as it affects access to and innovation for
HCTs is broken. The system must be reformed so that public
goods—such as genuine innovation and access to HCTs—are not
sacrificed on the altar of private gain. This reform must prioritize
the public good, use innovative policy tools to harness the private
sector where it is possible to do so, and create public R&D
capacity where market forces and actors are likely to continue to
fail.
Author Contributions
ICMJE criteria for authorship read and met: ERG WK JO SHL SNM.
Wrote the first draft of the paper: ERG WK JO. Contributed to the writing
of the paper: ERG WK JO SHL SNM. Carried out some of the research
for this manuscript, and wrote a substantial portion of the first draft: SHL.
Contributed to the discussion on the current state of global R&D: SNM.
Researched and analyzed the framing and future framing of the global
patent policy debate: SNM.
19. Gold ER, Adams WA, Bernier L, Bubela T, Cassivi L, et al. (2008) Toward a
New Era of Intellectual Property: From Confrontation to Negotiation. Montreal:
The Innovation Partnership: Available: http://www.theinnovationpartnership.
org/data/ieg/documents/report/TIP_Report_E.pdf. Accessed 17 September
2009. 44 p.
20. t’Hoen EFM (2009) The Global Politics of Pharmaceutical Monopoly Power:
Drug patents, access innovation and the application of the WTO Doha
Declaration on TRIPS and Public Health. , Diemen-The Netherlands: AMB
Publishers, Available: http://www.msfaccess.org. Accessed 21 September 2009.
21. World Health Organization (2004) Report of the Commission on Intellectual
Property Rights, Innovation and Health (CIPIH), Geneva, Switzerland.
Available: http://www.who.int/intellectualproperty/en/. Accessed 18 Septem-
ber 2009.
22. ip-health-admin@lists.essential.org (11 June 2009) ‘‘Concerns voiced at TRIPS
Council over seizure of drugs’’.
23. Sevilla C, Julian-Reynier C, Eisinger F, Stoppa-Lyonnet D, Bressac-de Paillerets B
(2003) Impact of gene patents on the cost-effective delivery of care: The case of
BRCA1 Genetic Testing. Int J Tech Assess Health Care 19: 287–300.
24. Department of Health and Human Services (2009) Secretary’s Advisory
Committee on Genetics, Health, and Society Public Consultation Draft Report
on Gene Patents and Licensing Practices and Their Impact on Patient Access to
Genetic Tests (SACCHS). Available: http://oba.od.nih.gov/oba/SACGHS/
SACGHS%20Patents%20Consultation%20Draft%203%209%202009.pdf. Ac-
cessed 21 September 2009.
25. Stott M, Valentine J (2003) Impact of gene patenting on R&D and commerce.
Nature Biotechno 21: 729–731.
26. Gold ER, Carbone J (2008) Myriad Genetics: In the Eye of the Policy Storm,
International Expert Group on Biotechnology, Innovation and Intellectual
Property. Available: http://www.theinnovationpartnership.org/data/ieg/
documents/cases/TIP_Myriad_Report.pdf. Accessed 21 September 2009.
27. Patent Docs [Blog] (2009) Association for Molecular Pathology et al. v. United States
Patent and Trademark Office. 1:09-cv-04515; filed May 12, 2009 in the Southern
District of New York (exclusive rights to human BRCA genes violate
Constitutionally-protected speech by restricting research). Available: http://
www.patentdocs.org/2009/05/court-1.html. Accessed 21 September 2009.
28. Cho MK, Illangasakare S, Weaver MA, Leonard DGB, Merz JF (2003) Effects
of Patents and Licenses on the Provision of Clinical Genetic Testing Services.
J Mol Diagnostics 5: 3–8.
29. Heller MA, Eisenberg RS (1998) Can Patents Deter Innovation? The
Anticommons in Biomedical Research. Science 280: 698–701. Available:
http://www.sciencemag.org/cgi/content/full/280/5364/698. Accessed 21
September 2009.
30. Lhuillery S, Pfister E (2009) French CIS R&D cooperation and failures in
innovation projects: Empirical evidence from French CIS data. Res Policy 38:
45–57.
4 January 2009 | Volume 7 | Issue 1 | e1000208

31. Nicol D, Nielsen J (2003) Patents and Medical Biotechnology: An Empirical
Analysis of Issues Facing the Australian Industry. Center for Law and Genetics.
Occasional Paper 6, page 255, University of Tasmania. Available: http://www.
lawgenecentre.org/pub.php. Accessed 21 September 2009.
32. American Association for the Advancement of Science (2007) International
Intellectual Property Experiences - A report of four countries. Washington, D.
C.: Project on Science and Intellectual Property in the Public Interest, Available:
http://sippi.aaas.org/Pubs/SIPPI_Four_Country_Report.pdf. Accessed 21
September 2009.
33. Walsh JP, Cohen WM, Cho C (2007) Where excludability matters: Material
versus intellectual property in academic biomedical research. Res Policy 36:
1184–1203.
34. Walsh JP, Arora A, Cohen WM (2003) Science and the Law: Working Through
the Patent Problem. Science 299: 1021. Summary available: http://www.
sciencemag.org/cgi/content/summary/299/5609/1021. Accessed 21 Septem-
ber 2009.
35. Meurer SM (2006) Inside the Anticommons: Academic scientists’ struggle to
build a commercially self-supporting human mutations database, 1999–2001.
Res Policy 35: 839–853.
36. Lei Z, Juneja R, Wright BD (2009) Patents versus patenting: implications of
intellectual property protection for biological research. Nature Biotechnology 27:
36–40.
37. University of California, Berkeley, Sponsored Projects Office (2009) A Quick
Guide to Material Transfer Agreements at UC Berkeley Available: http://www.
spo.berkeley.edu/guide/mtaquick.html. Accessed 21 September 2009.
38. Rodriguez V (2008) Governance of material transfer agreements. Technol Soc
30: 122–128.
39. Organization for Economic Cooperation and Development (2002) Genetic
Inventions, Intellectual Property Rights and Licensing Practices: Evidence and
Policies. Available: http://www.oecd.org/dataoecd/42/21/2491084.pdf. Ac-
cessed 21 September 2009.
40. The Royal Society (2003) Keeping science open: the effects of intellectual
property policy on the conduct of Science. Available: http://royalsociety.org/
document.asp?id = 1374.
41. United Nations Development Program (1999) Human Development Report
1999. Available: http://hdr.undp.org/en/media/hdr_1999_en.pdf.
42. World Bank (2007) Poverty Analysis – Overview. Available: http://go.
worldbank.org/K7LWQUT9L0.
43. Médecins Sans Frontières (2007) Untangling the Web of Price Reductions: A
Pricing Guide for Developing Countries. 10th ed. Geneva: Médecins Sans
Frontières Campaign for Access to Essential Medicines.
44. Kim JY, Farmer P (2006) AIDS in 2006 – moving toward one world, one hope?
N Engl J Med 355: 645–647.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
45. Agovino T (20 April 2001) Companies fear precedent as they cut AIDS drug
prices for Africa. The Associated Press State & Local Wire. Available: http://
www.cid.harvard.edu/cidinthenews/articles/ap_042001.html.
46. DOHA WTO Ministerial (2001) TRIPS. Declaration on the TRIPS agreement
and public health. Adopted on 14 November 2001. WT/MIN(01)/Dec/2/
Available: http://www.wto.org/english/thewto_e/minist_e/min01_e/mindecl_
trips_e.htm.
47. Pogge T (2007) Could Globalisation be Good For World Health? Global Justice:
Theory Practice Rhetoric. Available: www.theglobaljusticenetwork.org/
wp-content/uploads/1_pogge.pdf.
48. Orbinski J (2008) Creating a World of Possibility: The Fight for Essential
Medicines. In: An Imperfect Offering: Humanitarian Action for the 21st
Century. New York: Walker & Company. 366 p.
49. Savioli L, Engels E, Daumerie D, Jannin J, Alvar J, et al. (2006) Response from
World Health Organization [reader response]. PLoS Med, Available: http://www.
plosmedicine.org/annotation/listThread.action?inReplyTo = info%3Adoi%2F10.
1371%2Fannotation%2F8e288736-efdf-4ded-8854-7785bee8401b&root = info%
3Adoi%2F10.1371%2Fannotation%2F8e288736-efdf-4ded-8854-7785bee8401b.
50. Chirac P, Torreele E (2006) Global framework on essential health R&D. Lancet
367: 1560–1561.
51. ’T Hoen E (2009) Rationale for the Pharmaceutical Patent System. In: The
Global Politics of Pharmaceutical Monopoly Power ’T Hoen E, ed. Diemen:
AMB Publishers.
52. Kapczynski A, Crone TE, Merson M (2003) Global Health and University
Patents. Science 301: 1629.
53. Universities Allies for Essential Medicines (2009) Our Proposals. Available:
http://www.essentialmedicine.org/our-proposals.
54. Médecins Sans Frontières (2008) MSF Welcomes UNITAID patent pool
endorsement. Available: http://www.msfaccess.org/media-room/press-releases/
msf-welcomes-unitaid-patent-pool-endorsement/. Accessed 12 November 2009.
55. World Health Assembly (24 May 2008) Global strategy and plan of action on
public health, innovation and intellectual property. Agenda item 11.6;
whA61.62. http://www.who.int/gb/ebwha/pdf_files/A61/A61_R21-en.pdf.
Accessed 12 November 2009.
56. Hollis A (2008) The Health Impact Fund: A Useful Supplement to the Patent
System? Public Health Ethics 1: 124–133.
57. Moran M, Ropars A-L, Guzman J, Diaz J, Garrison C (2005) The New
Landscape of Neglected Diseases Drug Development. London, UK: Pharma-
ceutical R&D Policy Project, London School of Economics, Available: http://
www.bvgh.org/documents/MMoranTheNewLandscape.pdf.
58. N-Marandi S (2009) Framing and Reframing of Global Patent Policy:
Implications on Access to Medicine in Developing Countries. Public Policy
and Governance Review 1(1), Autumn 2009.
5 January 2009 | Volume 7 | Issue 1 | e1000208

World Health Report 2012: No Health Without Research

Viewpoints

Clinical Research in Resource-Limited Settings:

Enhancing Research Capacity and Working Together to
Make Trials Less Complicated
Trudie A. Lang1,2*, Nicholas J. White1,3, Tran Tinh Hien4, Jeremy J. Farrar1,5, Nicholas P. J. Day1,3,
Raymond Fitzpatrick6, Brian J. Angus1, Emmanuelle Denis1, Laura Merson5, Phaik Yeong Cheah3, Roma
Chilengi2, Robert Kimutai2, Kevin Marsh1,2
1 Centre for Vaccinology and Tropical Medicine, University of Oxford, Churchill Hospital, Oxford, United Kingdom, 2 KEMRI-Wellcome Trust Research Programme, Kilifi
District Hospital, Kilifi, Kenya, 3 Mahidol Oxford Research Unit, Bangkok, Thailand, 4 Hospital for Tropical Diseases, District 5, Ho Chi Minh City, Vietnam, 5 Oxford University
Clinical Research Unit, District 5, Ho Chi Minh City, Vietnam, 6 Department of Public Health, University of Oxford, Old Road Campus, Oxford, United Kingdom

Clinical trials conducted in developing
countries differ in many respects to those
carried out in the West; for example, they
are usually conducted in vulnerable pop-
ulations, focus mainly on infectious diseas-
es, and often have severe endpoints. In
these regions, trial capacity lags behind
that of wealthier nations, particularly in
terms of the ability of research sites to lead
broad and independent clinical research
programmes. Product development trials
are important for the registration of new
treatments and vaccines, yet do not leave
sites with the skills to run their own trials,
as protocol design, operational planning,
and data management are typically con-
ducted remotely by the sponsor. There is
also a need for more disease management
trials to examine and then improve health
outcomes, but the capacity to design and
execute such studies is often absent. The
process of increasing clinical trial capacity
should be led by the research sites and
tailored to their needs, as trial methods
and guidelines need to be appropriately
designed and crafted to be fit for purpose
in the developing country context. We
discuss the need to address the deficit in
capacity and training and propose a
collaborative solution for identifying the
gaps and then designing methods, guid-
ance, and sharing approaches to make
clinical trials less daunting and cumber-
some, particularly when being planned for
resource-limited settings.
mous potential gain from effective new
interventions and because these popula-
tions have been under-represented in
clinical research to date. The human and
material resource capacity available to
ensure a high standard of design, man-
agement, and operation of clinical trials in
developing countries lags far behind that
available in wealthier nations.
Although many of the issues con-
fronting clinical trialists working in re-
source-limited settings are the same as
those affecting academic researchers in the
wealthier regions of the world, there are
significant differences which both highlight
the issues involved and require specific
attention. In contrast to clinical trials in
wealthy countries, those in developing
countries frequently have endpoints that
are severe disease outcomes or mortality.
They more often involve children, focus
predominantly on infectious diseases, and
are more often sponsored by not-for-
profit organisations. We illustrate this
difference by taking a random sample of
100 trials registered at ClinicalTrials.gov
for each of the top five countries in Europe
and Africa and then classifying them
(Figure 1). Public health problems that
are specific to developing countries also
urgently require more and better clinical
trials to inform policy, such as the
management of disease outbreaks (includ-
ing those with pandemic potential) in
displaced populations, in refugee camps,
and following natural disasters. These very
specialised situations and environments
need new, highly practical, and appropri-
ate interpretation of regulations and
guidelines to enable rapid and flexible
trial implementation.
All Those Guidelines and
Regulations
Over recent years there has been
massive proliferation of regulations affect-
ing the conduct of clinical trials. This
process began in 1964 with the Declara-
tion of Helsinki made by the World
Medical Association (WMA) in response
to a tightening of legislation following the
thalidomide disaster in the 1960s [1,2].
The implementation of the declaration
resulted in the US Food and Drug
Administration (FDA) having to reject trial
data from countries with ethical and safety
standards that differed from the US. The
perceived differences between standards
drove the harmonisation process led by
regulators from Japan, Europe, and the
US, and experts from the pharmaceutical
industry, who produced, in 1996, the
International Conference on Harmonisa-
tion of Technical Requirements for Reg-
istration of Pharmaceuticals for Human
Use - Good Clinical Practice (ICH-GCP)
guidelines [3].

Trials in Resource-Limited
Settings
Clinical trials establish the evidence
base for the prevention and treatment of
disease. They are critically important in
developing countries, not simply because
this is where the potential is greatest for
improving health in numerical terms (as
these regions have the highest diseases
burden), but also because there is enor-
Citation: Lang TA, White NJ, Hien TT, Farrar JJ, Day NPJ, et al. (2010) Clinical Research in Resource-Limited
Settings: Enhancing Research Capacity and Working Together to Make Trials Less Complicated. PLoS Negl Trop
Dis 4(6): e619. doi:10.1371/journal.pntd.0000619
Editor: Charles H. King, Case Western Reserve University School of Medicine, United States of America
Published June 29, 2010
Copyright: � 2010 Lang et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: The authors received no specific funding for this work.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: trudie.lang@ndm.ox.ac.uk

 www.plosntds.org 1 iMedPub :: Thematic

June 2010 | VolumeCollections
4 | Issue 6::| e619
Volume 2

Figure 1. Trials differ in Europe and Africa. Classification of a random sample of 100 trials for each of five countries in Europe (France, Germany, Italy,
Spain, and the United Kingdom) and Sub-Saharan Africa (Kenya, Mali, Tanzania, Uganda, and Zambia). Trials in Africa focus predominantly on paediatric
populations (A) and infectious disease (B) and are non-industry sponsored (C). Data were abstracted from the ClinicalTrials.gov website in August 2009.
doi:10.1371/journal.pntd.0000619.g001
Pre-dating ICH-GCP, the Council for
International Organizations of Medical
Sciences (CIOMS) produced its Internation-
al Ethical Guidelines for Biomedical Research
Involving Human Subjects in 1982 [4].
Revised in 2002, these guidelines are
intended to guide lower-income countries
in applying the ethical principles that were
laid out in the Declaration of Helsinki.
Another set of international clinical trial
guidelines was produced by the World
Health Organization in 1995 [5]. The
WHO Guidelines for good clinical practice
(GCP) for trials on pharmaceutical products
were developed to provide a global
standard for clinical trials. They were
intended to complement existing regula-
tions in those WHO member states that
had already enforced clinical trials legisla-
tion or to provide a basis for new
regulations in countries that had not.
However, neither the CIOMS nor the
WHO guidelines hold the force of law,
and ICH-GCP is now the de facto global
standard by which trials are run and has
become a legal requirement for clinical
trial conduct in many countries. Consis-
tent criticisms of ICH-GCP are that it is
outdated, that not enough countries were
involved in its development, and that it is
focussed on the needs of industry and drug
registration with minimal representation
from academia and noncommercial orga-
nisations [6,7,8,9]. It is also focussed on
drugs as the intervention and so is difficult
to apply to other trials and broader types
of clinical research that would benefit from
sensible and appropriate quality and
ethical guidance.
www.plosntds.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
The Declaration of Helsinki can also be
difficult to apply. For example, the posi-
tion of the WMA to insist that a medical
doctor be responsible for taking informed
consent is not practical or always appro-
priate. In our research sites, community-
based trials are very important to assess
proposed improved or new interventions.
Here, where there are rarely doctors
present, it would be inappropriate to
introduce a doctor where normally there
are nurses or clinical officers just for the
purposes of the trial. In these situations it
could be argued that the profession
responsible for administration of the
intervention is much better placed to
request fully informed consent from po-
tential participants.
In the US and Europe, as the regula-
tions and guidelines have become more
strongly enforced and embedded in legis-
lation, it has been increasingly recognised
that a high level of support is necessary to
help researchers run their trials. Recently
a network has been set up in the UK
specifically to support clinical trials in
children. The Medicines for Children
Research Network [10] is funded by the
UK’s Department of Health and recog-
nises that conducting trials in children has
very specific challenges and needs dedi-
cated experts to provide tools and guid-
ance. After the European Union made
ICH-GCP a legal requirement in 2004,
the UK’s Medical Research Council
(MRC) launched a Web site to help
noncommercial trialists find their way
through the guidelines and direct them
in what they need to ensure that their trials
2 June 2010 | Volume 4 | Issue 6 | e619
are legal and compliant [11]. Since 2004,
most UK universities now have clinical
trials offices that support their academics
in conducting clinical trials. Universities
have made these provisions because as
sponsors they bear the legal burden of
ensuring that trials do not breach
ICH-GCP.
In the US, the FDA has partnered with
Duke University to establish the Clinical
Trial Transformation Initiative [12].
Their aim is to generate evidence on the
conduct of clinical trials that will improve
their quality and efficiency. This is a US-
focussed exercise that will examine current
practice under FDA requirements and
make recommendations to improve trial
conduct in the US so it is more straight-
forward and attractive to researchers.
These various initiatives set out to
unravel the guidelines and facilitate trial
conduct for non-commercial researchers
in their distinct environments. The same is
needed for the unique setting of develop-
ing country-based trials. However, any
such initiative should be led from the
perspective of these regions and by the
researchers working there. It should be
highly collaborative and must reflect the
real issues and gaps, which will need to be
continuously captured and monitored. In
addition, it must be broad enough to
provide support for all areas of trial
conduct, from governance and insurance
issues through to trial design and opera-
tions, and also have a strong focus on
training and career development. Any
guidelines or recommendations put for-
ward must be derived through a partici-

World Health Report 2012: No Health Without Research
patory action research process to ensure
that they address the gaps and are
appropriate and practicable for this chal-
lenging environment.
Trials in Resource Limited
Settings, the Current Situation
The majority of clinical trials conducted
in developing countries have sponsors who
are based in Europe or the US. Trial
sponsors frequently demand that research
sites implement the sponsor’s own inter-
pretation of ICH-GCP, which is often over
and above what is actually required. This
is understandable as ICH-GCP is gener-
ally seen as the ‘gold standard’ for all
clinical research and the investigational
product research is typically aimed at US
FDA or European Medicines Agency
(EMA) licensure [9,13]. However, these
exacting standards and the associated
burden of process and paperwork can be
daunting for academic researchers and are
frequently inappropriate where they work
[8]. A more locally appropriate interpre-
tation of GCP guidelines is often possible
and provides just as high a standard in
terms of ethics and quality, but investiga-
tors either lack the confidence to develop
and propose pragmatic alternatives or are
not aware that they can and should.
As well as product development studies
there is also a need for more disease
management trials. These are often large
yet straightforward trials that assess wheth-
er new approaches could be made in
current treatment or care practices to
improve outcomes. Typically these trials
assess known drugs that have been widely
used in other settings. These studies can
make a significant impact on public health
practice. Examples of potential trial topics
include managing malnutrition or pre-
scribing antibiotics during childbirth. New
or adapted interventions can potentially
bring about dramatic reductions in mor-
tality, but they must be supported by
sound medical evidence, and researchers
need access to tools and training in order
to undertake trials to obtain this evidence.
Disease management studies such as these
that assess new uses for existing or licensed
practices or treatments are normally
associated with lower risk than trials that
evaluate new treatments. While the basic
principles of GCP are straightforward and
their application is important to ensure
high standards in ethics and data quality,
little guidance exists on how they should
be interpreted and applied in disease
management or other non-investigational
new product trials. It is important to
remember that these are guidelines, intend-
www.plosntds.org
 iMedPub :: Thematic Collections :: Volume 2
ed to be subject to varying interpretation
and application. Within ICH-GCP itself it
is repeatedly stated that the guidelines
should be interpreted and applied in a
manner appropriate to the risk of the
research. Pragmatic interpretation is need-
ed, because many aspects of ICH-GCP are
not applicable in disease management
trials and can be problematic when
applied to investigational new product
trials, especially in the specific situations
found in resource-limited settings. Addi-
tionally, there are fundamental areas, such
as randomisation, that ICH-GCP does not
cover. Robust randomisation of subjects
into groups is critical to rigorous trial
design, and its reliable implementation is
fundamental to producing a valid data-
set—i.e., the trial giving the right answer!
Straightforward guidance is required to
ensure that this can been done securely
and accurately.
The Need for a Research-Led,
Developing Country–Specific
Clinical Trial Programme
A major criticism of ICH-GCP is that it
was developed through a process of
informal consensus rather than through
research or evidence of best practices [6].
While programmes like the Duke-FDA
Clinical Trials Transformation Initiative
are seeking to gather evidence to improve
trial conduct in the US, no such initiatives
exist to support a broad range of trials in
the developing world.
A collaborative programme that is
designed specifically to support developing
country-based trials and is not disease
specific could benefit researchers in devel-
oping countries. It would work best if
participation were free and available on-
line. Such a collaboration could encourage
disease management research and product
development trials, and could give re-
searchers access to all they need from
training right through to template docu-
ments, suggested operating procedures,
and guidelines, all accessed from one site.
An important element of such a resource is
that it could offer advice, tools, and
guidance appropriately adjusted for all
types of trials with varying levels of risk. As
it would be specifically for researchers
working in these settings, it would be able
to focus on relevant issues such as
community participation in trials, which
is of particular importance when trials are
being conducted in vulnerable popula-
tions. Ultimately it could provide sensible
and pragmatic interpretations of good
clinical practice guidelines derived using
an evidence-based approach and make the
3 June 2010 | Volume 4 | Issue 6 | e619
conduct of high-quality trial conduct
easier, less cumbersome, and much less
daunting. Such a site would benefit from
being highly interactive and allow re-
searchers to share their tools, experiences
and interpretations of the guidelines.
It is important to emphasise that the
need to facilitate trial conduct in resource-
limited settings does not mean developing
an approach that is in any sense substan-
dard or inferior. Developing country trials
require at least the same degree of
attention to the quality of processes and
procedures and of data management as
that required in resource-rich settings—
and the attention to international ethical
standards may need to be even greater
where vulnerable populations are in-
volved. A focussed effort is needed to
establish a straightforward system by
which researchers can navigate the regu-
lations and guidelines and determine what
is needed for their planned research and
appropriate for the context in which it will
be carried out.
Training and Professional
Development
In Europe and the US being a Clinical
Trials Scientist or ’’trialist’’ is a well
recognised profession. There are well
established and recognised professional
bodies [14,15], and numerous vocational
and academic qualifications are available
from diplomas through to doctorates, all
specific to the science and profession of
designing and conducting clinical trials.
Clinical trialists of all disciplines can be
found in universities, health organisations,
medical research charities, and industry.
In the regions where we work there is
limited recognition of the clinical trialist as
a profession or viable career path. We
believe this is a key factor impeding
capacity development. Many still see the
running of trials as an administrative
function. Taking a clinical question and
then developing a protocol and conduct-
ing a clinical trial to answer that question
is a research discipline that requires
training, experience, and critical thinking.
Only when there is a critical mass of
skilled trial coordinators, laboratory tech-
nicians, data managers, statisticians, mon-
itors, research nurses, and investigators
will research sites be in a position to design
and lead their own programmes.
Clinical trial staff in these regions would
benefit from a free online continuing
professional development scheme that
allows them to register their role, experi-
ence, training, and core competencies, and
then build points to track their competen-

cies and new training. This scheme should
be linked to e-learning opportunities and
to a local network of shared real-life
training opportunities and mentoring.
Summary
Our aim is to raise awareness of the
issues faced by researchers in developing
countries and to introduce an initiative we
are developing.
We propose that the gaps and issues we
have outlined could be largely addressed
by building a community of researchers
from all the various roles who will be able
to access the information, guidance and
resources they need, whilst also be able to
share methods and pragmatic operational
practices that have been locally derived
and known to work. Some examples
include template consent forms, data
References
1. Hollister LE (1968) The Kefauver-Harris amend-
ments of 1962: a critical appraisal of the first five
years. J Clin Pharmacol 8: 69.
2. World Medical Association (1964) Declaration of
Helsinki (Human Experimentation: Code of
Ethics). BMJ 2: 177.
3. International Conference on Harmonisation of
Technical Requirements for Registration of
Pharmaceuticals for Human Use (1996) Guideline
for Good Clinical Practice E6(R1).
4. Council for International Organizations of Med-
ical Sciences (CIOMS) (2002) International
Ethical Guidelines for Biomedical Research
Involving Human Subjects.
5. World Health Organization (1995) Guidelines for
good clinical practice (GCP) for trials on phar-
maceutical products World Health Organization.
pp 97–137.
www.plosntds.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
management systems, and example proto-
cols and laboratory sample collection and
handling methods.
We emphasize that this initiative is
entirely based on an ethos of collabora-
tion, open access, and sharing practice;
indeed it will only be successful if research
groups both use the resource and contrib-
ute to its development. The development
of a prototype of web site for this initiative
is underway and can be found at http://
pilot.globalhealthtrials.org/. We are mak-
ing this public at this early juncture as we
are seeking involvement from our col-
leagues right from the outset in line with
the open and collaborative ethos that is
envisaged. Therefore, we encourage col-
leagues to become part of this initiative by
providing content, commenting on the
Web site, and sharing their operational
6. Grimes DA, Hubacher D, Nanda K, Schulz KF,
Moher D, et al. (2005) The Good Clinical
Practice guideline: a bronze standard for clinical
research. Lancet 366: 172–174.
7. White NJ (2006) Editorial: clinical trials in
tropical diseases: a politically incorrect view.
Trop Med Int Health 11: 1483–1484.
8. World Health Organization (2002) The Impact of
Implementation of ICH Guidelines in Non-ICH
Countries. Geneva.
9. Gajic A, Herrmann R, Salzberg M (2004) The
international quality requirements for the conduct
of clinical studies and the challenges for study
centers to implement them. Ann Oncol 15:
1305–1309.
10. Medicines for Children Research Network. Avail-
able: http://www.mcrn.org.uk/. Accessed June
7th 2010.
4 June 2010 | Volume 4 | Issue 6 | e619
tools. We also welcome all those engaged
in trials to register and build their own
personal professional development record
to track their career and training record,
and to provide a review structure.
Conclusion
To improve clinical trial conduct in
resource-limited settings we need easier
operational tools and guidance as well as
skilled staff. This needs to be more than
conducting externally sponsored trials that
are designed and led elsewhere. We
suggest true capacity-building might be
best achieved by establishing a community
of developing country based researchers to
share locally derived solutions and build a
set of validated methods and operational
tools that will enable pragmatic and locally
led development.
11. Medical Research Council Clinical Trials Tool
Kit. Available: http://www.ct-toolkit.ac.uk/. Ac-
cessed June 7th 2010.
12. Clinical Trials Transformation Initiative: Avail-
able: https://www.trialstransformation.org/. Ac-
cessed June 7th 2010.
13. Medical Research Council (1998) Guidelines for
Good Clinical Practice in Clinical Trials. Available:
http://www.mrc.ac.uk/Utilities/Documentrecord/
index.htm?d=MRC002416. Accessed June 7th 2010.
14. Association of Clinical Research Professionals
(ACRP). Available: http://www.acrpnet.org/.
Accessed June 7th 2010.
15. Institute of Clinical Research (ICR). Available:
http://www.icr-global.org/home/. Accessed 22
October, 2009.

World Health Report 2012: No Health Without Research
Essay
Evidence-Based Priority Setting for Health Care and
Research: Tools to Support Policy in Maternal, Neonatal,
and Child Health in Africa
Igor Rudan1,2*, Lydia Kapiriri3, Mark Tomlinson4, Manuela Balliet1, Barney Cohen5, Mickey Chopra6
1 Global Health Academy and Centre for Population Health Sciences, The University of Edinburgh Medical School, Edinburgh, Scotland, United Kingdom, 2 Croatian
Centre for Global Health, Faculty of Medicine, University of Split, Soltanska, Split, Croatia, 3 Department of Health, Aging and Society, McMaster University, Hamilton,
Ontario, Canada, 4 Department of Psychology, Stellenbosch University, Stellenbosch, South Africa, 5 Committee on Population, United States National Academy of
Sciences, Washington, D.C., United States of America, 6 UNICEF, New York, New York, United States of America
This paper is part of a PLoS Medicine
series on maternal, neonatal, and
child health in Africa
Priority Setting—Implicit or
Explicit?
Priority setting is required in every
health care system. It guides investments
in health care and health research, and
respects resource constraints. It happens
continuously, with or without appropriate
tools or processes. Although priority-set-
ting decisions have been described as
difficult, value laden, and political, only a
few research groups are focused on
advancing the theory of priority setting
and the development and validation of
priority setting tools [1–4]. These groups
advocate the use of their tools, but their
work is often not widely recognized,
especially among the policy makers in
developing countries, where these tools
would be most helpful [2].
Our primary objective in this essay is to
present the available tools for priority
setting that could be used by policy makers
in low-resource settings. We also provide
an assessment of the applicability and
strengths of different tools in the context
of maternal and child health in sub-
Saharan Africa.
The analyses of investments in neglect-
ed diseases showed that they lack trans-
parent priority-setting processes [2]. This
persisting situation results in remarkable
levels of inequity between investments in
different health priorities [1–6]. Therefore,
our secondary objective is to advocate for
the use of the tools that could lead to more
rational priority setting in sub-Saharan
The Essay section contains opinion pieces on topics
of broad interest to a general medical audience.
10 PLoS Medicine | www.plosmedicine.org
Africa. An optimal tool should be able to because it is not easy to validate the tools
draw on the best local evidence and guide or to link their output with concrete
policy makers and governments to identi- follow-up actions and policy development
fy, prioritize, and implement evidence- [9]. Indeed, it is difficult to prove beyond
based health interventions for scale-up and all doubt that investments in health care or
delivery. health research are valuable to society
when compared to alternative investments
Priority Setting in Low- such as infrastructure or the economy.
Resource Settings—Mixed However, there are many examples of
countries that have reduced their maternal
Evidence
and child disease burden substantially
Although there is currently insufficient from very high starting levels, and of
evidence that the use of priority-setting others that keep failing to achieve progress
tools improves health outcomes and re- [10]. We also have strong evidence on the
verses existing inequities, we have ample key determinants of those successes, which
evidence that the lack of a rational and has been incorporated into various prior-
transparent process generates inequity and ity-setting tools [1,4–9]. The few studies
stagnation in mortality levels [5,6]. Re- that have evaluated processes in low-
cently, Youngkong et al. conducted a resource settings not using priority-setting
systematic review of empirical studies on tools found that most of them fell short on
health care priority setting in low-income all four conditions of the ‘‘accountability
countries (Table 1) [7]. The review found for reasonableness’’ framework that assess-
that policy makers in developing countries ed their basic legitimacy and fairness
rarely consider using the available priority- [11,12].
setting tools, but also that the available Moreover, there is evidence on the
tools lack credibility for priority setting in interventions and health research needed
low-resource settings [7,8]. This is mainly to improve maternal and child survival in
Citation: Rudan I, Kapiriri L, Tomlinson M, Balliet M, Cohen B, et al. (2010) Evidence-Based Priority Setting for
Health Care and Research: Tools to Support Policy in Maternal, Neonatal, and Child Health in Africa. PLoS
Med 7(7): e1000308. doi:10.1371/journal.pmed.1000308
Published July 13, 2010
Copyright: � 2010 Rudan et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: The authors received no specific funding for this article.
Competing Interests: IR, LK, MT, and MC have all been involved in the development and implementation of
the CHNRI methodology. IR and MC have been consultants of Child Health and Nutrition Research Initiative of
the Global Forum of Health Research while developing the CHNRI methodology. The other coauthors have no
competing interests to declare.
Abbreviations: CAM, Combined Approach Matrix; CHNRI, Child Health and Nutrition Research Initiative;
CHOICE, Choosing Interventions that are Cost-Effective; COHRED, Council on Health Research for Development;
DALY, disability-adjusted life year; DCPP, Disease Control Priorities Project; EHCP, Essential Health Care Package;
ENHR, Essential National Health Research; LiST, Lives Saved Tool; MBB, Marginal Budgeting for Bottlenecks;
WHO, World Health organization.
* E-mail: igor.rudan@ed.ac.uk
Provenance: Commissioned; externally peer reviewed.
1 July 2010 | ::
iMedPub Volume 7 | Issue
Thematic 7 | e1000308
Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Table 1. Priority setting exercises for health care or health research in low resource settings.

Setting Participants Topic Criteria Process Outcome

Health care/health interventions – all low-resource countries (Refs. [13,21,22])

Low-resource globally TE All major diseases DCPP project consensus Systematic reviews Cost-effectiveness analysis
Low-resource globally TE, PM, OS Primary health care Yes, modified CHNRI CHNRI Specific list with scores and ranks
Low-resource globally TE, PM, OS Stillbirth prevention Yes, modified CHNRI CHNRI Specific list with scores and ranks
Health care/health interventions - national or sub-national level (Refs. [7,34,35,37])
Thailand PM, HM, HW, TE Several diseases Yes, through literature Semi-structured Interview Table with choice frequency
review
Chile None Health system Yes, through literature Secondary data analysis List with ranks for 56 choices
review
South Africa PM, NGO, TE HIV/AIDS Yes, through literature Group discussion and interview List with ranks by THREE chosen
review criteria
Tanzania PM Health system Yes, through group Group discussion and question. Ranking of criteria by importance
discussions
Tanzania PM, HP, GP, PA Health system Not transparent Group discussion Description of different views
Tanzania PM, HP, GP, PA Several diseases Yes, through literature Deliberative process List with ranks for NINE
review interventions
Argentina PM (at all levels) Health system Yes, focus group and Focus group and interviews List of criteria
interviews
Nepal PM, HP Several diseases Yes, literature review and Individual rating List with ranks for 33
group discussions interventions
Pakistan PM HIV/AIDS Yes, in-depth interview Interview Description of policy maker’s
views
Burkina Faso, Ghana, PM, HM, HW Safe motherhood programYes, self-administered Deliberative process and quest. Identifying three most important
Indonesia questionnaire priorities
Uganda PM, HW Health system Yes, interviews Semistructured interview Description of criteria used
Uganda PM, HW Health system Yes, one-on-one interviews Interview and document analysisDescription of criteria used
Ghana PM, HW Several diseases Yes, group discussion Individual rating List with ranks for interventions
Uganda PM, HW, GP Health system Yes, literature and self- Questionnaire with rating scale List of criteria and their weights
administered questionnaire
Uganda PM, HW, GP Health system Yes, group discussions and ‘‘Brainstorming’’ and List with ranks for criteria and
interviews questionnaire choices
Ghana PM, HM, TE, NGO Reproductive health Yes, literature review Interview and secondary data Demonstration of impact on
and interview priorities
Bosnia and Herzegovina None Health system Not transparent Secondary data analysis Description of criteria used
South Africa None Health system Yes, literature review Secondary data analysis List with ranks for interventions
India PM, TE Neonatal mortality Yes, literature review Lives Saved Tool (LiST) Effectiveness and impact on
and model mortality
Ghana and Mali PM, TE Child mortality Yes, literature review Lives Saved Tool (LiST) Effectiveness and impact on
and model mortality
Burkina Faso, Ghana, PM, TE Child mortality Yes, literature review Lives Saved Tool (LiST) Effectiveness and impact on
Malawi and model mortality
Health research – all low-resource countries (Refs. [23–29])
Low-resource globally TE, PM, HP, OS Mental health Yes, standard CHNRI CHNRI Specific list with scores and ranks
Low-resource globally TE, HP Maternal and child survivalNone; collective opinion Delphi Specific list of priorities with
ranks
Low-resource globally TE, PM, HP, OS Neonatal infections Yes, standard CHNRI CHNRI Specific list with scores and ranks
Low-resource globally TE, PM, HP, OS Childhood diarrhea Yes, standard CHNRI CHNRI Specific list with scores and ranks
Low-resource globally TE, PM, HP, OS Birth asphyxia Yes, standard CHNRI CHNRI Specific list with scores and ranks
Low-resource globally TE, PM, HP, OS Childhood pneumonia Yes, standard CHNRI CHNRI Specific list with scores and ranks
Low-resource globally TE, PM, HP, OS Zinc supplementation Yes, standard CHNRI CHNRI Specific list with scores and ranks
Low-resource globally TE, PM, HP, OS Research into disabilities Yes, modified CHNRI CHNRI Specific list with scores and ranks
Health research – national or sub-national level (Refs. [30,31])
Malaysia TE, PM, OS Health research Yes, transparent list CAM General recommendations
Cameroon Government Health research Not transparent ENHR and COHRED General objectives
officials

PLoS Medicine | www.plosmedicine.org 2 July 2010 | Volume 7 | Issue 7 | e1000308

iMedPub :: Thematic collections :: Volume 2 11
World Health Report 2012: No Health Without Research

Table 1. Cont.

Setting Participants Topic Criteria Process Outcome

Peru TE Health research Not transparent COHRED General recommendations

South Africa Government Health research Yes, transparent ENHR General recommendations
officials
South Africa TE, OS Child health research Yes, standard CHNRI CHNRI Specific list with scores and ranks
Brazil PM, TE, HP, multipleHealth research Yes, transparent COHRED General recommendations
OS
Philippines PM, TE, HP, OS Health research Not fully transparent COHRED General recommendations
Pakistan PM, TE, HP, NGO, PSHealth research Yes, transparent CAM General recommendations
Argentina PM, TE, HP Health research Yes, transparent CAM General recommendations

Abbreviations: GP, general population; HM, health managers; HP, health professionals; HW, health workers; NGO, non-governmental organization; OS, other
stakeholders; PA, patients; PM, policy-makers; PS, private sector; TE, technical experts.
doi:10.1371/journal.pmed.1000308.t001

low-resource settings. The key challenge is
how to motivate and educate policy
makers in sub-Saharan Africa to use the
available priority-setting tools to direct the
limited available resources into the most
effective interventions and health research.
We believe that addressing this challenge
is critical, because it has been repeatedly
shown that the scarcity of resources for
health in sub-Saharan Africa is only part
of the larger problem; the other part is that
the scarce available resources are not
being used efficiently by any standard,
leading to tragic consequences for the
population [2,4,6].
Emerging Tools for Evidence-
Based Priority Setting to Guide
Health Care Policy
Several tools and processes are begin-
ning to emerge as useful for priority setting
in low-resource settings. In Table 1 we
classify different methodologies by the
context (national/global level) and scope
(health care/health research prioritiza-
tion). We also provide some essential
information on the use of each method:
(i) the setting; (ii) participants included in
the process; (iii) the specific topic ad-
dressed; (iv) the criteria that were used for
prioritization; (v) the process that was
used; and (vi) the nature of the outcome.
An in-depth comparative analysis of all
these tools is beyond the scope of this
essay, but in Table 1 we provide references
to the key papers from which further
information about those methods can be
obtained ([13–37]; Lawn et al., manuscript
in preparation).
Table 1 shows that the ‘‘burden of
disease/cost effectiveness analysis,’’ pro-
moted by the Disease Control Priorities
Project (DCPP) [13], is an essential
component of several tools that have
been used for health care (interventions)
prioritization: for example, the Marginal
Budgeting for Bottlenecks (MBB) tool
developed by UNICEF and The World
Bank [14]; WHO-CHOICE (Choosing
Interventions that are Cost-Effective) de-
veloped by the World Health Organiza-
tion [14,15]; and Lives Saved Tool (LiST)
developed by Johns Hopkins University
scientists and the Futures Institute [16].
The DCPP approach for developing
countries uses information on the burden
of major diseases to assist decisions about
the potential of affordable and effective
interventions. The DCPP analysis identi-
fies the ‘‘best buys,’’ i.e., the most cost-
effective interventions in terms of DALYs
saved per unit cost, that should compose a
country’s essential health care package
(EHCP) [17]. The EHCP should then
influence program design and resource
reallocation to help governments achieve
the goal of reducing morbidity and
mortality.
However, the DCPP authors note that
factors other than cost-effectiveness influ-
ence priority setting in the real world, so
the available evidence has to be consid-
ered in the context of local realities
[13,17]. Both MBB and WHO-CHOICE
provide appropriate contextualization
tools. However, the LiST software goes
much further than any other tool in
several dimensions. LiST contains an
expansive evidence base of context-specific
intervention effectiveness, generated by
researchers from the WHO/UNICEF’s
Child Health Epidemiology Reference
Group (CHERG) [33]. It is a user-friendly
decision-making computer software avail-
able in the public domain. It enables
estimation of intervention impact on child
mortality at national, regional, and global
levels [16]. Further important advantages
of LiST include its validation in both
African and South Asian contexts [34,35],
an ability to perform very specific com-
parisons between alternative investment
strategies over a specified time frame in
terms of child survival outcomes [33–35],
its application of an equity lens [36], and
easy translation of outcomes into program
planning with convincing country-level
examples [37].
Prioritizing Gaps in Health
Research
Policy makers in low-resource settings
also need to set priorities for health
research. Table 1 shows that the CHNRI
methodology has recently been used by
several different groups to set health
research priorities at the highest interna-
tional level ([23–29], Lawn et al., manu-
script in preparation). However, there are
several other tools for setting research
priorities at the national level, which were
reviewed and evaluated by Tomlinson et
al. [30]. Whereas CHNRI method had its
first national-level implementation in
South Africa only recently [31], other
tools and processes have been dominant at
the national level. The Council on Health
Research for Development’s approach
(COHRED) has been implemented in
Brazil, Cameroon, Peru, and Philippines;
the Essential National Health Research
(ENHR) approach in Cameroon and
South Africa; and the Combined Ap-
proach Matrix (CAM) in Malaysia, Paki-
stan, and Argentina [30].
COHRED, ENHR, and CAM all were
developed by committees set up by
international agencies. All these methods
are very specific about context, and they
are excellent for organizing all the avail-
able information. However, they do little
to provide an algorithm, based on a
transparent set of criteria, that can distin-

PLoS Medicine | www.plosmedicine.org 3 July 2010 | Volume 7 | Issue 7 | e1000308

12 iMedPub :: Thematic Collections :: Volume 2

guish among many competing research
investment options [4,29]. This does not,
however, diminish their utility in most
situations where the development of an
evidence base is required. That phase can
then be followed by Delphi-type consulta-
tion processes among a designated set of
experts. For example, CAM does excep-
tionally well in addressing the two dimen-
sions of the context that it finds the most
important: the ‘‘public health’’ dimension
and the ‘‘institutional’’ dimension. Having
only two dimensions limits CAM’s flexi-
bility, though, and it is difficult to see how
additional dimensions—e.g., uncertainty
over the outcome (inherent to all health
research); accounting for investment styles;
accounting for the risk exposure and
benefit potential of each research option;
or the likelihood of obtaining funding
support from donors—could be added
[33]. The same limitation is also true for
COHRED and ENHR.
An emerging tool that is rapidly gaining
popularity in the area of health research
prioritization is the CHNRI methodology.
It was developed over four years (2005–
2008) with support from The World Bank
for a transdisciplinary exercise of 15
experts. The experts assessed principles
and practice of priority setting [4], reviewed
universal challenges [18], developed a
novel and robust conceptual framework
[18], and provided guidance for stakehold-
er involvement [19] and for implementa-
tion of the method [20]. Currently, they are
in the process of developing user-friendly
software that would enable simple, cheap,
and effective conducting of CHNRI exer-
cise via the internet.
The CHNRI methodology insists on
transparency about the context in which
priority setting takes place and the criteria
used. It was initially developed for health
research, but it has recently also been
successfully used for health care and health
interventions (Table 1) [21,22]. Like the
DCPP approach, it uses both cost-effec-
tiveness and potential impact on disease
References
1. World Health Organization (2010) Report of the
World Health Organization Expert Working
Group on Research and Development Financing
2010. Available: http://www.who.int/phi/docu-
ments/RDFinancingwithISBN.pdf. Accessed 10
March 2010.
2. Kapiriri L, Norheim OF, Martin DK (2009)
Fairness and accountability for reasonableness.
Do the views of priority setting decision makers
differ across health systems and levels of decision
making? Soc Sci Med 68: 766–773.
3. Klein R (1998) Puzzling out priorities. Why we
must acknowledge that rationing is a political
process. Br Med J 317: 959–960.
4. Rudan I, Gibson J, Kapiriri L, Lansang MA,
Hyder AA, et al. (2007) Setting priorities in global
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
burden as criteria. However, within a set
of ‘‘standard’’ criteria, CHNRI also uses
criteria relevant to the context—answer-
ability, deliverability, affordability, sustain-
ability, local capacity, likelihood of sup-
port, feasibility, equity, and others. The
process is usually designed by policy
makers or donors, conducted by technical
experts in a transparent way (e.g., each
vote counts equally), with a mechanism of
stakeholder involvement. Stakeholders can
assign different weights to the criteria used
in the CHNRI exercise. The outcome is a
comprehensive list with competing prior-
ities ranked according to the combined
scores they received in the process [18–
20]. Such a list is helpful to policy makers
because it provides an overview of
strengths and weaknesses of competing
investment options against many criteria,
based on the collective input of technical
experts. The list can also be adjusted by
taking the values of many stakeholders into
account.
Conclusions
The key challenges that need to be
overcome in sub-Saharan Africa to im-
prove the processes of prioritization in
health care and health research include
the following: increased acceptability and
popularity with local policy makers, ap-
preciation of the local context, clarity
about the criteria used, transparency in
the input from the stakeholders, and more
specific guidance on translation into pol-
icy. Many papers that analyze the strate-
gies for improving maternal and child
survival conclude with highlighting the
challenges such as integration, require-
ments for selection of community health
workers, operational research into systems,
among others. These are all admirable
and important future areas of research.
However, they are not exactly new,
ground-breaking, or very specific, and
the qualitative nature of the process
frequently does not provide sufficient
child health research investments: Assessment of
principles and practice. Croat Med J 48: 595–604.
5. Moran M, Guzman J, Ropars A-L, McDonald A,
Jameson N, Omune B, Ryan S, Wu L (2009)
Neglected disease research and development: how
much are we really spending? PLoS Med 6: e30.
doi:10.1371/journal.pmed.1000030.
6. Enserink M (2009) Some neglected diseases are
more neglected than others. Science 323: 700.
7. Youngkong S, Kapiriri L, Baltussen R (2009)
Setting priorities for health interventions in
developing countries: a review of empirical
studies. Trop Med Int Health 14: 930–
939.
8. Kapiriri L, Arnesen T, Norheim OF (2004) Is
cost-effectiveness analysis preferred to severity of
4 July 2010 | Volume 7 | Issue 7 | e1000308
guidance to policy makers on the specific
next steps. Tools such as LiST (for health
care/interventions) and CHNRI (for
health research) involve local experts and
incorporate issues of local context into
priority determination in a transparent,
user-friendly, replicable, quantifiable and
specific, algorithm-like manner. Both of
these tools were primarily developed to
address child health problems and should
be considered by policy makers in the area
of maternal and child health in sub-
Saharan Africa.
The use of scientific evidence and
principles in setting health priorities has
an enormous potential to lead to more
rational decision making, especially in low-
resource settings where decision making
has long lacked formal tools, processes, or
an evidence base. We believe one cannot
overstate the value of building and sup-
porting the capacity of local experts and
policy makers in sub-Saharan Africa to
initiate and assist their own national
government’s policy formation process in
maternal and child health, and of govern-
ment’s being able to generate rigorous
credible ‘‘home grown’’ advice [4,27,32].
Regardless of the limitations of the avail-
able tools, we strongly recommend their
use in development of sound maternal and
child health policies in sub-Saharan Africa
over the alternative of not using any
method. The use of such tools would
promote attention to objective evidence in
public policy debates, often leading to
decisions that are made are more clearly
and in the public interest [27,32].
Author Contributions
ICMJE criteria for authorship read and met: IR
LK MT MB BC MC. Agree with the
manuscript’s results and conclusions: IR LK
MT MB BC MC. Designed the experiments/
the study: MT. Wrote the first draft of the
paper: IR LK. Contributed to the writing of the
paper: IR LK MT MB BC MC.
disease as the main guiding principle in priority
setting in resource poor settings? The case of
Uganda. Cost Effect Resource Allocat 2: 1–11.
9. Allen L (2010) The art of evaluating the impact of
medical science. Bull WHO 88: 4.
10. Rudan I, Chan KY, Zhang JSF, Theodoratou E,
Feng XL, Salomon J, Lawn JE, Cousens S,
Black RE, Guo Y, Campbell H (2010) Causes of
deaths in children younger than 5 years in China
in 2008. Lancet 375: 1083–1089.
11. Kapiriri L, Martin DK (2006) Priority setting in
developing countries health care institutions: the
case of a Ugandan hospital. BMC Health Serv
Res 6: 127.
12. Mshana S, Shemilu H, Ndawi B (2007) What do
district health planners in Tanzania think about
13
World Health Report 2012: No Health Without Research
improving priority setting using ‘Accountability for
Reasonableness’? BMC Health Serv Res 7: 180.
13. Laxminarayan R, Mills AJ, Breman JG,
Measham AR, Alleyne G, et al. (2006) Advance-
ment of global health: key messages from the
Disease Control Priorities Project. Lancet 367:
1193.
14. UNICEF/The World Bank (2010) Marginal Budget-
ing for Bottlenecks. Available: http://www.who.int/
pmnch/topics/economics/costingtools_resources/
en/index.html. Accessed: 10 March 2010.
15. Adam T, Lim SS, Mehta S, Bhutta ZA,
Fogstad H, et al. (2005) Cost effectiveness analysis
of strategies for maternal and neonatal health in
developing countries. BMJ 331: 1107.
16. Victora CG (2010) LiST: using epidemiology to
guide child survival policymaking and program-
ming. Int J Epidemiol 39(Suppl 1): i1–2.
17. Bobadilla JL, Cowley P, Musgrove P, Saxenian H
(1992) Design, content and financing of an
essential national package of health services. Bull
World Health Organ 72: 653–662.
18. Rudan I, Chopra M, Kapiriri L, Gibson J,
Lansang MA, et al. (2008) Setting priorities in
global child health research investments: universal
challenges and conceptual framework. Croat
Med J 49: 307–317.
19. Kapiriri L, Tomlinson M, Gibson J, Chopra M,
El Arifeen S, et al. (2007) Setting priorities in
global child health research investments: Address-
ing the values of the stakeholders. Croat Med J
48: 618–627.
20. Rudan I, Gibson JL, Ameratunga S, El Arifeen S,
Bhutta ZA, et al. (2008) Setting priorities in global
child health research investments: Guidelines for
implementation of the CHNRI method. Croat
Med J 49: 720–733.
21. Walley J, Lawn JE, Tinker A, De Francisco A,
Chopra M, et al. (2008) Primary Health Care:
PLoS Medicine | www.plosmedicine.org
14
making Alma Ata a reality. Lancet 372:
1001–1007.
22. Pattinson R, Lawn JE, Darmstadt G, Rubens C,
Flenady V, et al. (2010) Priority interventions to
reduce the global burden of stillbirths. Lancet, In
press.
23. Tomlinson M, Rudan I, Saxena S, Swartz L,
Tsai AC, Patel V (2009) Setting investment
priorities for research in global mental health.
Bull World Health Organ 87: 438–446.
24. Costello A, Filippi V, Kubba T, Horton R (2007)
Research challenges to improve maternal and
child survival. Lancet 369: 1240–1243.
25. Bahl R, Martines J, Ali N, Bhan MK, Carlo W, et
al. (2009) Research priorities to reduce global
mortality from newborn infections by 2015.
Pediatr Inf Dis J 28(Suppl 1): S43–S48.
26. Fontaine O, Kosek M, Bhatnagar S, Boschi-
Pinto C, Chan KY, et al. (2009) Setting research
priorities to reduce global mortality from child-
hood diarrhoea by 2015. PLoS Med 6: e41.
doi:10.1371/journal.pmed.1000041.
27. Rudan I, El Arifeen S, Black RE, Campbell H
(2007) Childhood pneumonia and diarrhoea:
Setting our priorities right. Lancet Inf Dis 7:
56–61.
28. Brown KH, Hess SY, Boy E, Gibson RS,
Horton S, et al. (2009) Setting priorities for
zinc-related health research to reduce children’s
disease burden worldwide: An application of the
Child Health and Nutrition Research Initiative’s
research priority-setting method. Public Health
Nutr 12: 389–396.
29. Tomlinson M, Swartz L, Officer A, Chan KY,
Rudan I, Saxena S (2009) Research priorities for
health of people with disabilities: an expert
opinion exercise. Lancet 374: 1857–1862.
30. Tomlinson M, Chopra M, Hoosein N, Rudan I
(2010) Research priority setting processes at
5 July 2010 | Volume 7 | Issue 7 | e1000308
national level: Good practices for health research.
Health Syst Res Policy, In press.
31. Tomlinson M, Chopra M, Sanders D,
Bradshaw D, Hendricks M, et al. (2007) Setting
priorities in child health research investments for
South Africa. PLoS Med 4: e259. doi:10.1371/
journal.pmed.0040259.
32. Rudan I (2009) The complex challenge of setting
priorities in health research investments.
Indian J Med Res 129: 351–353.
33. Walker N, Fischer-Walker C, Bryce J, Bahl R,
Cousens S, CHERG Review Groups on Inter-
vention Effects (2010) Standards for CHERG
reviews of intervention effects on child survival.
Int J Epidemiol 39(Suppl 1): i21–31.
34. Friberg IK, Bhutta ZA, Darmstadt GL, Bang A,
Cousens S, et al. (2010) Comparing modelled
predictions of neonatal mortality impacts using
LiST with observed results of community-based
intervention trials in South Asia. Int J Epidemiol
39(Suppl 1): i11–20.
35. Hazel E, Gilroy K, Friberg I, Black RE, Bryce J,
et al. (2010) Comparing modelled to measured
mortality reductions: applying the Lives Saved
Tool to evaluation data from the Accelerated
Child Survival Programme in West Africa.
Int J Epidemiol 39(Suppl 1): i32–9.
36. Amouzou A, Richard SA, Friberg IK, Bryce J,
Baqui AH, et al. (2010) How well does LiST
capture mortality by wealth quintile? A compar-
ison of measured versus modelled mortality rates
among children under-five in Bangladesh.
Int J Epidemiol 39(Suppl 1): i186–92.
37. Bryce J, Friberg IK, Kraushaar D, Nsona H,
Afenyadu GY, et al. (2010) LiST as a catalyst in
program planning: experiences from Burkina
Faso, Ghana and Malawi. Int J Epidemiol
39(Suppl 1): i40–7.
iMedPub :: Thematic Collections :: Volume 2

Essay
Improving Implementation: Building Research Capacity
in Maternal, Neonatal, and Child Health in Africa
James Whitworth1, Nelson K. Sewankambo2, Valerie A. Snewin1*
1 Wellcome Trust, London, United Kingdom, 2 Makerere University, College of Health Sciences, Kampala, Uganda
This paper is part of a PLoS Medicine
series on maternal, neonatal, and
child health in Africa.
This PLoS Medicine series began by
outlining how much we fail mothers,
newborns, and children in Africa by not
implementing effectively what we know
saves lives and improves health [1,2]. It is
clear that countries in Africa are falling
behind not only on improving maternal,
newborn, and child health but on the
Millennium Development Goals 4, 5, and
6 more generally. Why is there such a
wide gap between what we know and what
we do? While technical knowledge about
what could be done is available, actual
implementation is neither straightforward
nor easy in the often difficult circumstanc-
es on the ground. The many competing
priorities—along with limited logistic ca-
pacity, a lack of political will, and
inadequate infrastructure—also constrain
the extent to which effective health
packages are delivered to those who need
them most.
Implementation Science
It is estimated that between 66% and
85% of Africa’s maternal, newborn, and
child (under 5 years) deaths could be
avoided by implementing current inter-
ventions [3,4]. Therefore, the priority for
maternal and child survival is not so much
the development of new technologies but
solving implementation issues, such as how
to scale up and evaluate interventions
within complex health systems. Such
implementation research should not only
focus the attention of policy makers and
implementers, but also improve decision
making, enhance efficiency, and build
understanding of why some programmes
work and others do not. But generating
the necessary robust evidence is not easy.
The Essay section contains opinion pieces on topics
of broad interest to a general medical audience.
iMedPub PLoS Medicinecollections
:: Thematic | www.plosmedicine.org
:: Volume 2
World Health Report 2012: No Health Without Research
First, we do not know how best to scale strengthen their often weak health systems
up interventions effectively. The recent while at the same time increase their own
evaluation of UNICEF’s Accelerated capacity to do research to improve the
Child Survival and Development pro- health of not only mothers, newborns, and
gramme in West Africa showed that, while children, but of their entire population? A
vertical preventive implementation did first step would be to listen to the voices of
improve coverage, there was no accelera- those grappling with the issues on the
tion in child survival [5]. Similar rigorous ground. Too often well-meaning initiatives
evaluations of other existing large-scale are developed in Washington, Geneva, or
implementation programmes such as PEP- London without incorporating the views of
FAR, the Global Fund to Fight HIV/ African scientists, policy makers, and civil
AIDS, Tuberculosis and Malaria, and society. Of course, the global community,
GAVI would help accelerate progress including the H8 group of health organi-
towards better implementation [6]. sations and the G20 group of major
However, the evaluation of complex advanced and emerging economies, has a
interventions is itself problematic and major role to play in realising the aims of
more work needs to be done on the building capacity in Africa, but this needs
development of robust and generally to be done while taking into account the
accepted methods for such evaluations voices of those on the ground. Until
[7,8]. While randomised controlled trials recently it has been difficult to obtain an
are considered the gold standard for authoritative voice that represents a wide
evaluating interventions, there is little spectrum of African scientists. But things
are changing, and the recently established
consensus on when these should be
Initiative to Strengthen Health Research
applied for evaluating complex interven-
Capacity in Africa (ISHReCA; http://
tions, or on what other methods are
ishreca.tropika.net/) aims to serve as a
appropriate and in what circumstances.
forum for African scientists to collate ideas
Engagement of Southern Voices on capacity building and to speak with a
collective voice. ISHReCA has identified a
and Institutions
series of key requirements for strengthen-
It is clear that there is a need to broaden ing health research capacity in Africa,
the base for health research in low- and focused around the need to improve the
middle-income countries, especially for research environment, and for supporting
implementation research [9]. But how both individuals and institutions [10,11].
can sub-Saharan African countries This effort is relevant across the whole
Citation: Whitworth J, Sewankambo NK, Snewin VA (2010) Improving Implementation: Building Research
Capacity in Maternal, Neonatal, and Child Health in Africa. PLoS Med 7(7): e1000299. doi:10.1371/
journal.pmed.1000299
Published July 6, 2010
Copyright: � 2010 Whitworth et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: The authors received no specific funding for this article.
Competing Interests: VAS and JW are employed by the Wellcome Trust. NKS is Chair of the steering
committee for the non-profit Initiative for Strengthening Research Capacity In Africa (ISHReCA) and Director of
Wellcome Trust funded THRiVE consortium, which consists of seven African institutions and two UK
universities.
Abbreviations: GAVI, Global Alliance for Vaccines and Immunisation; PEPFAR, US President’s Emergency Plan
for AIDS Relief; UNICEF, United Nations International Children’s Fund.
* E-mail: v.snewin@wellcome.ac.uk
Provenance: Commissioned; externally peer reviewed.
1 July 2010 | Volume 7 | Issue 7 | e1000299 15
World Health Report 2012: No Health Without Research
spectrum of scientific research, because,
while there is an imperative to implement
what we already know, there is still a need
to develop better interventions and deliv-
ery strategies.
Improving the Research
Environment
In many African countries legislation
needs to be modernised to support the
conduct of research, to exchange materials
and data, and to protect intellectual
property rights. African governments also
need to make greater efforts to support
research through strategic planning,
strengthened research governance, and
increased funding. Equally, African gov-
ernments need to develop strategic plans
for increasing and supporting human
resources for research for health in parallel
with the requirements of the programme
implementation work force. All this is
unlikely to happen without increased
engagement by scientists and advocates
to promote science within African societies
and to demonstrate the benefits that
investment in research can contribute to
health development and wealth creation.
Strong, sustained advocacy is required to
encourage policy makers to ensure that
research is supported by increased finan-
cial and political support [12]. National
Academies of Science (strengthened
through the African Science Academy
Development Initiative and Royal Socie-
ty–Pfizer African Academies Programme),
the African Academy of Sciences, and the
African Union could all be credible
advocates to promote the cause of science.
National governments should also increase
their research funding to match commit-
ments, for example, by allocating at least
2% of health ministry budgets to research
[13]. Competitive national grant schemes
with merit-based peer-reviewed assess-
ments are required. An example of such
a scheme is in Uganda, where the
government together with the World Bank
has funded the Millennium Science Initia-
tive. Calls for proposals are issued regu-
larly through the press. Applicants submit
proposals, which are reviewed by a team
of both Ugandan and international scien-
tists, and awards are made on a compet-
itive basis.
Supporting Individuals
There is an urgent need to build the
next generation of African scientists.
Schoolchildren need to be instilled with
excitement about science through their
teachers and curricula, otherwise they
PLoS Medicine | www.plosmedicine.org
16
are unlikely to choose to study science models. Such research group leaders
subjects at university. Universities need need to be supported with secure fund-
to promote and support research as well ing, for example through endowed posi-
as training and service, so that under- tions, to enable them to help and mentor
graduates are exposed to research and young researchers throughout their ca-
taught by researchers throughout their reers. Opportunities for funding need to
courses, hopefully leading them to view be diversified beyond the usual interna-
research as an attractive career option. tional foundations and agencies, to
However, to facilitate this credible career include national governments, private
paths must be created which offer donations, local charities, and corpora-
opportunities at every level. Attractive tions.
packages with competitive salaries, ca-
reer posts, and opportunities for training
Supporting Institutions
and travel are important, as is special
attention to the recruitment of women. The infrastructural base for research at
More programmes are needed that most institutions in Africa needs much
promote good mentoring and empower improvement. African governments need
junior scientists. For example, at Maker- to contribute more to providing basic
ere University in Kampala, clinical facilities, providing a foundation upon
scholarship positions have been created which external agencies can build. Fund-
to attract, mentor, and retain junior ing agencies and donors need to work
researchers, and there is a fast-track together to ensure that the true costs of
pathway for promotion based on re- research are provided for, to include such
search productivity. Senior scientists overhead as upgrading facilities and
themselves need to be identified who support services such as information
will act as research leaders and role technology, library services, ethical over-
Box 1. Initiatives and Networks for Research Capacity
Strengthening in Africa
N Healthy Newborn Network: http://www.healthynewbornnetwork.org
N African Health Research Forum and University Science, Humanities and
Engineering Partnerships in Africa (USHEPiA): http://web.uct.ac.za/misc/iapo/
ushepia/bg.htm
N European and Developing Countries Clinical Trials Partnerships (EDCTP)
Networks of Excellence: http://www.edctp.org/
N European Union funded Network for the Co-ordination and Advancement of
sub-Saharan Africa-EU Science and Technology Cooperation (CAAST-Net):
http://www.caast-net.org
N Health Research Capacity Strengthening initiative (HRCS) a partnership
between UK Department for International Development (DFID), International
Development Research Centre (IDRC), Canada and the Wellcome Trust: http://
www.wellcome.ac.uk/hrcs
N INDEPTH Network (International Network of field sites with continuous
Demographic Evaluation of Populations and Their Health in developing
countries): http://www.indepth-network.org/
N Initiative to Strengthen Health Research Capacity in Africa: http://ishreca.
tropika.net/
N Malaria IPTi network (funded by the Bill & Melinda Gates Foundation, WHO, and
UNICEF): http://www.ipti-malaria.org
N Medical Research Council (UK) and DFID African Research Leader scheme:
http://www.mrc.ac.uk/Fundingopportunities/Calls/AfricanResearchLeader/
MRC006652
N Neglected Tropical Diseases Fellowship Scheme (supported by a consortium of
European foundations): http://www.ntd-africa.net
N Netherlands African Partnership for Capacity Development and Clinical
Interventions against Poverty-related Diseases; Netherlands Organisation for
Scientific Research: http://www.nwo.nl/naccap
N Leverhulme Royal Society Africa Awards: http://royalsociety.org/Leverhulme-
Royal-Society-Africa-Awards/
N Wellcome Trust African Institutions Initiative: http://www.wellcome.ac.uk/aii
2 July 2010 | Volume 7 | Issue 7 | e1000299
iMedPub :: Thematic Collections :: Volume 2

sight, laboratory support, and financial
and research management. Clearly not
all universities or higher education facil-
ities can be supported in this way, and
priority should be given to those research
institutes and universities that have the
potential to flourish. Health research
funders and development agencies need
to ensure that there is greater harmonisa-
tion between themselves and increased
alignment with national health priorities,
to avoid unnecessary duplication of effort
and divergence of aims [14]. South
Africa, while not a typical sub-Saharan
country, has shown a promising way
forward. The University of Cape Town
Health Sciences Faculty has eight well-
funded research chairs, each provided
with two research assistants and research
funds. For every PhD and Masters degree
successfully completed, and every publi-
cation, universities in South Africa re-
ceive funding from government. This
helps to incentivise universities to train
research students and for researchers to
publish.
References
1. Kinney MV, Kerber KJ, Black RE, Cohen B,
Nkrumah F, et al. (2010) Sub-Saharan Africa’s
Mothers, Newborns, and Children: Where and
Why Do They Die? PLoS Med 7(6): e1000294.
doi:10.1371/journal.pmed.1000294.
2. Friberg IK, Kinney MV, Lawn JE, Kerber KJ,
Odubanjo MO, et al. (2010) Sub-Saharan
Africa’s mothers, newborns, and children: How
many lives could be saved with targeted health
interventions? PLoS Med 7: e295. doi:10.1371/
journal.pmed.1000295.
3. Jones G, Steketee R, Black R, Bhutta Z, Morris S,
et al. (2003) How many child deaths can we
prevent this year? Lancet 362: 65–71.
4. Kinney MV, Lawn JE, Kerber KJ, eds. (2009) Sci-
ence in Action: Saving the lives of Africa’s mothers,
newborns, and children. Cape Town, South Africa:
African Science Academy Development Initiative.
5. Bryce J, Gilroy K, Jones G, Hazel E, Black R, et
al. (2010) The Accelerated Child Survival and
Development programme in west Africa: a
retrospective evaluation. Lancet 375: 572–582.
6. Pisani E, Whitworth J, Zabac B, Abou-Zahret C
(2010) Time for fair trade in research data.
Lancet 375: 703–704.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
Developing Networks
Just as with intervention research, there is
an urgent need to evaluate initiatives that
aim to strengthen research capacity by using
robust and generalisable methods and to
share learning from them. Relatively few
examples of this process exist, and the
literature is sparse ([10] and Box 1). The
Wellcome Trust is supporting a thorough
evaluation of the recently launched African
Institutions Initiative [15].
Partnerships and networks should be
encouraged to promote North–South and
South–South interaction. Too often part-
nerships are developed between a north-
ern university research powerhouse and a
much smaller, less research-active, African
university. This imbalance is unlikely to
lead to serious sustainable capacity devel-
opment in the South. Equitable partner-
ships built upon mutual trust must be
encouraged [16]. Increased support for
South–South networks is also desirable so
that established universities can assist the
development of emerging neighbouring
7. Petersen S (2010) Assessing the scale-up of child
survival interventions. The Lancet 375: 530–531.
8. Mackenzie M, O’Donnell C, Halliday E,
Sridharan S, Platt S (2010) Do health improve-
ment programmes fit with MRC guidance on
evaluating complex interventions? BMJ 340:
401–403.
9. WHO Alliance for health policy and Systems Research.
Available: http://www.who.int/alliance-hpsr/en/.
Accessed 12 March 2010.
10. Whitworth JAG, Kokwaro G, Kinyanjui S,
Snewin VA, Tanner M, et al. (2008) Strengthen-
ing capacity for health research in Africa. Lancet
372: 1590–1593.
11. ISHReCA, http://ishreca.tropika.net/ (accessed
12 March 2010).
12. Accra Agenda for Action (2008) Available: http://
siteresources.worldbank.org/ACCRAEXT/Resources/
4700790-1217425866038/AAA-4-SEPTEMBER-
FINAL-16h00.pdf. Accessed 12 March 2008.
13. Bamako Call for Action at the Global Ministerial
Forum on Research for Health, Bamako, Mali,
November 2008. Available: http://www.tropika.
net/svc/specials/bamako2008/call-for-action/
call. Accessed 12 March 2010.
3 July 2010 | Volume 7 | Issue 7 | e1000299
institutions. Funding agencies, including
national governments, can promote col-
laborative networks to build lasting
change.
Conclusion
The high levels of maternal, newborn,
and childhood mortality and morbidity in
Africa are cause for an urgent response to
implementing interventions. Strong health
research systems and research pro-
grammes that address bottlenecks to
upscaling effective interventions should
be developed without delay. This effort
requires substantial and rapid investment
in the support of African scientists, insti-
tutions, and systems that will focus on
solutions to African problems.
Author Contributions
ICMJE criteria for authorship read and met:
JW NKS VAS. Agree with the manuscript’s
results and conclusions: JW NKS VAS. Wrote
the first draft of the paper: JW. Contributed to
the writing of the paper: NKS VAS.
14. ESSENCE for Health Research. Available:
http://essence.tropika.net/essence-stakeholders-
commons. Accessed 12 March 2010.
15. African Institutions Initiative on the Wellcome
Trust. Available: http://www.wellcome.ac.uk/
aii. Accessed 12 March 2010.
16. Swiss Commission for Research Partnerships with
Developing Countries (KFPE), Improving Im-
pacts of Research Partnerships (2006) Available:
http://www.kfpe.ch/key_activities/impact_study/
index.php. Accessed 12 March 2010.
17. Barros FC, Bhutta ZA, Batra M, Hansen TN,
Victora CG, et al. (2010) Global report on
preterm birth and stillbirth: evidence for effec-
tiveness of interventions. Available: http://www.
biomedcentral.com/1471-2393/10/S1/S3.
18. Victora CG, Rubens CE, GAPPS Review Group
(2010) Global report on preterm birth and
stillbirth: delivery of interventions. Available:
http://www.biomedcentral.com/1471-2393/10/
S1/S4.
17
World Health Report 2012: No Health Without Research
Essay
Closing the Gaps: From Science to Action in Maternal,
Newborn, and Child Health in Africa
Sara Bennett1*, Freddie Ssengooba2
1 Health Systems Program, Johns Hopkins School of Public Health, Baltimore, Maryland, United States of America, 2 School of Public Health, Makerere University, Kampala,
Uganda
This paper is part of a PLoS Medicine
series on maternal, neonatal, and
child health in Africa.
The previous papers in the PLoS Medicine
series [1,2] demonstrate that the technical
basis for improving maternal, newborn, and
child health (MNCH) in sub-Saharan Africa
is largely known, but too often policy and
practice are not well informed by science.
There are two distinct aspects to this ‘‘gap.’’
First there is a ‘‘science to policy and
practice’’ gap. Accumulated scientific re-
search on the severity of MNCH problems
and strategies to promote MNCH has, at
least in part, failed to ensure that MNCH
reaches the domestic policy agendas of
African countries, and stays there. Further-
more, local, context-specific evidence fre-
quently is not applied in planning and
programming interventions to address
MNCH. Second there is a ‘‘policy to
practice’’ gap: even where clear policy
commitments to MNCH are made, there
may be substantial challenges to getting such
policies implemented. These include chal-
lenges related to stakeholder management
through the implementation process and
challenges associated with the negotiation of
health system constraints. Many African
countries face weakened health systems
characterized by human resource shortages,
dysfunctional drug supply systems, decaying
health infrastructure, and weak supervisory
and governance mechanisms. Consequent-
ly, the global community is currently
strongly focused on strengthening health
systems [3] so that they can provide
adequate platforms for the delivery of a
range of services, including MNCH.
Our discussion focuses on the ‘‘science
to policy and practice’’ gap, in the belief
that action to address the second gap is
already mobilized, although clearly not yet
fully effective. In contrast, the first gap
remains neglected. This article first
The Essay section contains opinion pieces on topics
of broad interest to a general medical audience.
18 PLoS Medicine | www.plosmedicine.org
addresses what is already known about political priority given to maternal mor-
how scientific evidence has influenced tality. Through national-level interviews
MNCH policy and practice, then it and document review he assessed the
considers some of the key challenges in extent to which: (1) national political
closing the science to policy and practice leaders expressed sustained concern for
gap, and concludes by identifying promis- the issue; (2) the government enacted
ing paths for future action. policies that embraced strategies to ad-
Global and country-specific evidence on dress the problem; and (3) the government
maternal and child mortality, service allocated and released public budgets
coverage, and effective interventions to commensurate with the problem’s gravity.
improve MNCH has been key to stimu- He rated the political priority accorded to
lating greater global attention to these maternal mortality as low in the one
issues through monitoring efforts such as African country included in the study.
those of the Countdown group, which Despite intensive global advocacy efforts,
tracks progress towards the Millennium MNCH may not be a high policy priority
Development Goals (MDGs) that address for many African governments.
MNCH. However, much more needs to Where there is commitment to MNCH
be done to ensure that MNCH issues and an intention to support action to
reach national policy agendas and that address MNCH issues, African countries
they remain a high priority given the need to tailor strategies to match health
importance of policy consistency in pro- system capacity. Local data need to be
moting the MNCH agenda [1]. The compiled and analyzed to guide how
Countdown project assessed national pol- MNCH service packages can be integrated
icy for MNCH through select policy and delivered within the given resource
indicators (for example, the adoption and constraints. Evidence as to the use of
enactment of the International code on MNCH data in health planning is limited,
Marketing of Breast Milk Substitutes and but we know from multiple sources that, in
the presence of a costed implementation general, data quality is poor and the use of
plan for MNCH) [4]. It concluded that, data for planning and decision making is
while policies had improved in the 68 weak [6].
priority countries, policy environments
were not yet fully supportive and a major Challenges to Closing the
gap remains between policy and action.
MNCH Science to Policy and
Such indicators, however, are probably Practice Gaps
relatively insensitive measures of the true
political priority given by African leaders While the MDGs, including MDGs
to MNCH. Shiffman [5] analyzed the 4 and 5, remain the cornerstone of
Citation: Bennett S, Ssengooba F (2010) Closing the Gaps: From Science to Action in Maternal, Newborn, and
Child Health in Africa. PLoS Med 7(6): e1000298. doi:10.1371/journal.pmed.1000298
Published June 29, 2010
Copyright: � 2010 Bennett, Ssengooba. This is an open-access article distributed under the terms of the
Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any
medium, provided the original author and source are credited.
Funding: No specific funding was received for this piece.
Competing Interests: The authors have declared that no competing interests exist.
Abbreviations: MDG, Millennium Development Goal; MNCH, maternal, newborn, and child health
* E-mail: sbennett@jhsph.edu
Provenance: Commissioned; externally peer reviewed.
1 June 2010 ::| Volume
iMedPub 7 Collections
Thematic | Issue 6 | e1000298
:: Volume 2

government and development partner
policies, the harsh reality is that African
Ministries of Health face multiple com-
peting priorities. Obviously HIV/AIDS
and other infectious diseases continue to
demand time and attention, but so do
health worker concerns about pay, issues
of drug shortages, and emerging concerns
about the health effects of climate change
and chronic disease. In such contexts it is
naı̈ve to think that Ministries or Ministers
can remain consistently focused on critical
MNCH issues. Broader policy and advo-
cacy coalitions need to be developed and
employed to help promote government
accountability and maintain focus.
Unlike HIV/AIDS, for which a discrete
and close-knit group of affected persons
may be organized for effective advocacy,
MNCH beneficiaries are often diffuse and
not organized into a strong lobby. While
the White Ribbon Alliance, an interna-
tional coalition that advocates and raises
awareness concerning safe motherhood,
has had substantial achievements in rais-
ing the profile of maternal health issues,
advocacy coalitions for MNCH in Sub-
Saharan African countries are few. For
example, this Alliance has branches in
only nine sub-Saharan African countries.
Furthermore, MNCH does not lend itself
to a simple, ‘‘silver-bullet’’ fix. Instead it
raises a set of more complex (yet still
tractable) and context-specific policy is-
sues: are formal health services financially
and geographically accessible to women?
Do policies support community-based
workers who can identify and refer high-
risk women? Are there policies and
programs in place that promote appropri-
ate nutrition for girls and women through-
out their lives? Such complex policy
questions need to be complemented by
implementation research that supports
policy adaptation to local contexts.
Certain projects have demonstrated that
considerable health impacts can be
achieved through the local interpretation
and application of data to MNCH plan-
ning and programming. For example, in
Nepal an intervention that enabled wo-
men’s groups to review local evidence, and
to jointly plan, implement, and assess
interventions aimed at addressing local
perinatal problems led to increased cover-
age of antenatal care and attended deliv-
eries, and ultimately to a substantial drop
in neonatal mortality [7]. However such
projects have typically operated on a
relatively small scale and with quite
intensive technical support. In many
contexts the quality of routine data is
poor, and health staff and community
capacity to analyze data are limited. Even
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
when these two primary obstacles have
been tackled, the lack of an organizational
culture that supports evidence-informed
decision making has remained problemat-
ic [8]. Given the decentralized nature of
many African health systems, analytical
skills and a culture of evidence-informed
decision-making need to be developed in
district management teams and front-line
health workers, as well as in Ministries of
Health. Staff at all of these levels need to
be empowered to generate and use data
and operations research findings in their
planning and decision making processes.
What Must Be Done
The challenges in bridging the science
to policy and practice gap are consider-
able, but they are not insurmountable. As
for MNCH service packages, interventions
to strengthen the use of science in policy
and decision making will have positive
ramifications for the whole health system,
enabling the impact of ongoing health
systems strengthening investments to be
multiplied. Not all of the challenges
identified above can be addressed: we
propose three strategies that we believe
would have the greatest impact on
MNCH.
1. Develop MNCH Policy Networks
One of the key developments in policy
during the past decade has been an
increasing understanding of the impor-
tance of the ‘‘webs of influence’’ that guide
the exercise of power. Securing and
sustaining national political priority for
MNCH must go beyond politicians and
ministers and engage civil society, front-
line health workers, researchers, and the
media. This cannot be achieved in a ‘‘top-
down fashion,’’ instead it should be
stimulated by small pots of funds to foster
the emergence of grassroots groups and
coalitions. Powerful and persuasive evi-
dence that can be generated through the
Lives Saved Tool (LiST) [9], a computer-
based tool that allows users to predict the
impact of alternative packages of MNCH
services, needs to be packaged and com-
municated in ways that are easily accessi-
ble to all of these different groups. Such
policy networks can reinforce chains of
mutual accountability, particularly when
evidence regarding progress against
MNCH goals is available.
2. Mainstream the Use of MNCH
Science
Given the multiple competing demands
on decision makers it is critical to ensure
that the analysis and application of
2 June 2010 | Volume 7 | Issue 6 | e1000298
evidence to support MNCH fit into
planning and monitoring processes, again
strengthening the overall process of health
planning rather than adding an extra
burden. For example, rigorous efforts to
understand trends in MNCH and to plan
services to address MNCH should be
integrated into poverty reduction strate-
gies, sector-wide approaches, and other
ongoing processes that require the use of
evidence in policy and resource allocation
decisions.
3. Invest in Innovative Approaches
to Develop and Apply MNCH
Evidence
While there have been substantial
international efforts aimed at synthesizing
and packaging evidence so as to influence
MNCH policy, by and large this has not
been replicated at national levels. The
ability to generate, synthesize, and apply
evidence of different sorts—health data,
global health research, and experiences of
health practitioners and communities—is
critical to the development of adaptive
health systems able continually to
strengthen MNCH services. While some
investment has been made in developing-
country research capacities [10], much
more is needed to build local institutions
that can conduct relevant research for
MNCH policy and implementation.
Equally critical is investment in enhancing
skills and capacities to apply such research
findings in practice and in programs. This
is needed not only among policy makers
but across the whole range of actors
involved in MNCH policy networks.
As the main papers in the PLoS
Medicine series suggest [1,2], there are a
growing number of answers about what is
clinically and programmatically effective
in promoting MNCH. While weak health
systems continue to be a barrier to the
more rapid scale-up of MNCH and other
priority services, there has recently been
increased attention to health system
strengthening. We have argued that the
key challenge now is stimulating an
approach to MNCH policy development,
planning, and management that relies on
evidence in its multiple forms, to steer
strategy, and to facilitate the tailoring of
global solutions to local conditions. Fortu-
nately, there is also a growing body of
knowledge about how to promote the
appropriate use of science in policy and
practice: it is time to begin to apply what
we know about effective science to policy
and practice strategies to the MNCH field.
As we apply what we already know, we
should continue to build institutional and
individual capacity for the local adaptation
19
World Health Report 2012: No Health Without Research
and indigenization of global MNCH
evidence to national and subnational
contexts, resources, and constraints.
References
1. Kinney MV, Kerber KJ, Black RE, Cohen B,
Nkrumah F, et al. (2010) Sub-Saharan Africa’s
Mothers, Newborns, and Children: Where and
Why Do They Die? PLoS Med 7: e294.
doi:10.1371/journal.pmed.1000294.
2. Friberg IK, Kinney MV, Lawn JE, Kerber KJ,
Odubanjo MO, et al. (2010) Sub-Saharan
Africa’s Mothers, Newborns, and Children:
How Many Lives Could Be Saved with Targeted
Health Interventions? PLoS Med 7: e295.
doi:10.1371/journal.pmed.1000295.
3. Task force on Innovative International Financing for
Health Systems (2009) More money for health and
more health for the money Available: http://www.
internationalhealthpartnership.net/en/taskforce/
taskforce_reports Accessed: 11 March 2010.
PLoS Medicine | www.plosmedicine.org
20
Acknowledgments
Thanks to Mary Kinney and Joy Lawn for
comments on earlier versions of this essay.
4. The Countdown Working Group on Health
Policy and Systems (2008) Assessment of the
health system and policy environment as a critical
complement to tracking intervention coverage for
maternal, newborn, and child health. Lancet 371:
1284–1293.
5. Shiffman J (2007) Generating Political Priority for
Maternal Mortality Reduction in 5 Developing
Countries. Am J Public Health 97: 796–803.
6. Aqil A, Lippeveld T, Hozumi D (2009) PRISM: A
paradigm shift for designing, strengthening and
evaluating routine health information systems.
Health Policy Plan 24: 218–228.
7. Manandhar DS, Osrin D, Shrestha BP, Mesko N,
Morrison J, et al. (2004) Effect of a participatory
intervention with women’s groups on birth
3 June 2010 | Volume 7 | Issue 6 | e1000298
Author Contributions
ICMJE criteria for authorship read and met: SB
FS. Agree with the manuscript’s results and
conclusions: SB FS. Wrote the first draft of the
paper: SB. Contributed to the writing of the
paper: FS.
outcomes in Nepal: cluster-randomised controlled
trial. Lancet 364: 970–998.
8. Chalaugai CN, Moyo C, Koot J, Moyo HB,
Sambakunsi TC, et al. (2005) Design and
implementation of a health management infor-
mation system in Malawi: issues, innovations and
results. Health Policy Plan 20: 375–284.
9. LiST. Available: http://www.jhsph.edu/dept/
ih/IIP/list/index.html. Accessed: 2 June 2010.
10. Snewin V, Whitworth J, Sewankambo N (2010)
Improving Implementation: Building Research
Capacity in Maternal, Neonatal, and Child
Health in Africa. PLoS Med, In press.
iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Health in Action

AIDS Vaccine for Asia Network (AVAN): Expanding the

Regional Role in Developing HIV Vaccines
Stephen J. Kent1, David A. Cooper2, Mean Chhi Vun3, Yiming Shao4*, Linqi Zhang5, Nirmal Ganguly6,
Budiman Bela7, Hiko Tamashiro8, Rossana Ditangco9, Supachai Rerks-Ngarm10, Punnee Pitisuttithum11,
Nguyen Van Kinh12, Alan Bernstein13, Saladin Osmanov14, for the AIDS Vaccine for Asia Network
investigators and supporters
1 University of Melbourne, Parkville, Victoria, Australia, 2 University of New South Wales, Darlinghurst, New South Wales, Australia, 3 National Center for HIV/AIDS,
Dermatology and STIs (NCHADS), Phnom Penh, Cambodia, 4 State Key Laboratory for Infectious Disease Control and Prevention, National Center for AIDS/STD Control and
Prevention, Beijing, China, 5 Tsinghua University, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China, 6 Indian Council of Medical
Research, New Delhi, India, 7 University of Indonesia, Jakarta, Indonesia, 8 Hokkaido University, Hokkaido, Japan, 9 Research Institute for Tropical Medicine, Manila,
Philippines, 10 Department of Disease Control, Ministry of Public Health, Bangkok, Thailand, 11 Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand,
12 National Institute of Infectious and Tropical Diseases (NIITD), Hanoi, Viet Nam, 13 Global HIV Vaccine Enterprise, New York, New York, United States of America,
14 World Health Organization/Joint United Nations Programme on HIV/AIDS, Geneva, Switzerland

The Challenge
The HIV/AIDS pandemic continues to
spread and an AIDS vaccine is urgently
needed. While facing unprecedented chal-
lenges, AIDS vaccine development activi-
ties are continuing around the globe [1,2].
Recent results of the Thai Phase III vaccine
trial are renewing such efforts [3]. In
accordance with the goals of the Global
HIV Vaccine Enterprise (the Enterprise)
[4], there is now clear recognition of the
role that regional alliances can play in
fostering and facilitating AIDS vaccine
development [5], and there is broad
agreement that international collaborations
are the most effective way forward to
develop and evaluate the next generation
of AIDS vaccine candidates [6].
In response to these challenges, the
Asian region has recently formed the
AIDS Vaccine for Asia Network (AVAN),
with a clear vision and mission (Box 1).
The Need for AVAN: The HIV
Epidemic in the Asian Region
The total Asia population is approxi-
mately 4 billion people—just over 60% of
the world population. More than 500
million people are considered to be at-risk
for HIV exposure and infection, including
youth, injecting drug users (IDUs), sex
workers, men who have sex with men
(MSM), and mobile populations. Close to
5 million people have already been
infected with HIV across the region,
The Health in Action section is a forum for individuals
or organizations to highlight their innovative ap-
proaches to a particular health problem.
although the epidemic is highly variable the initial driving force of many HIV
across different countries, communities epidemics in Asian countries, sexual trans-
and populations [7]. mission has gradually taken over. Signifi-
Asian HIV sub-epidemics are driven cant increases in the number of new HIV
primarily by unsafe sex and drug injection infections are also expected to occur across
and are therefore currently concentrated the region among MSM. The geographic
in groups with higher risk for HIV; distribution of HIV subtypes and CRFs is
however, the epidemic is slowly spreading relatively homogeneous but varies by
into the general population. Although subcontinent: CRF_07B9/C, B9, and
impressive results have been achieved CRF01_AE in China [9–11]; subtype C
following the scaling up of available HIV in India [12]; and CRF01_AE in Thailand
prevention strategies in some countries [13] (Figure 1). Unfortunately, HIV inci-
[8], the level of prevention, care, and dence data remain incomplete and need to
be bolstered for optimal planning of
treatment coverage among risk-associated
vaccine studies. These interrelated factors
groups remains unacceptably low.
The main factors influencing HIV may affect AIDS vaccine research and
vaccine development in the region include development in the region and therefore
circulation of multiple HIV subtypes and should be given due consideration in
circulating recombinant forms (CRFs) planning future studies. The complexity
transmitted through the various modes of of the epidemic in the region poses
transmission, diverse host genetics, and substantial challenges but also affords
disparate social, cultural, and political tremendous opportunities to accelerate
contexts in the region. While IDUs were AIDS vaccine development considering
Citation: Kent SJ, Cooper DA, Chhi Vun M, Shao Y, Zhang L, et al. (2010) AIDS Vaccine for Asia Network (AVAN):
Expanding the Regional Role in Developing HIV Vaccines. PLoS Med 7(9): e1000331. doi:10.1371/
journal.pmed.1000331
Published September 21, 2010
Copyright: � 2010 Kent et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: No specific funding was received for this article.
Competing Interests: The authors have declared that no competing interests exist.
Abbreviations: ADCC, antibody-dependent cellular cytotoxicity; AVAN, AIDS Vaccine for Asia Network; BCG,
Mycobacterium bovis/Bacillus Calmette-Guérin; CAVD, Collaboration for AIDS Vaccine Discovery; CAVI, Chinese
AIDS Vaccine Initiative; CDC, Center for Disease Control and Prevention; CRF, circulating recombinant form;
DoD, Department of Defense; the Enterprise, Global HIV Vaccine Enterprise; EU, European Union; GMP, good
manufacturing practice; IAVI, International AIDS Vaccine Initiative; IDU, injecting drug user; MHRP, US Military
HIV Research Program; MSM, men who have sex with men; MVA, modified vaccinia Ankara; NIH, National
Institutes of Health; UNAIDS, World Health Organization (WHO)/Joint United Nations Programme on HIV/AIDS
* E-mail: yshao08@gmail.com
Provenance: Not commissioned; externally peer reviewed.

PLoS
iMedPub :: Medicinecollections
Thematic | www.plosmedicine.org
:: Volume 2 1 September 2010 | Volume 7 | Issue 9 | e1000331 21
World Health Report 2012: No Health Without Research

Summary Points Most notably, Phase III trials of the

ALVAC (Canarypox, vCP1521) and
N The HIV/AIDS pandemic continues to spread and an AIDS vaccine is urgently
AIDSVAX gp120 B/E prime-boost regi-
mens have also been completed. This
needed.
landmark trial, the largest AIDS vaccine
N Regional alliances and international collaborations can foster the development efficacy trial conducted to date involving
and evaluation of the next generation of AIDS vaccine candidates.
over 16,000 volunteers, showed a modest
N The importance of coordinating and harmonizing efforts across regional (31%) but significant reduction in new
alliances has become abundantly clear. HIV infections and is the first time that an
N We recently formed the AIDS Vaccine for Asia Network (AVAN) to help facilitate AIDS vaccine has exhibited protection
the development of a regional AIDS vaccine strategy that accelerates research against HIV acquisition [3].
and development of an AIDS vaccine through government advocacy, improved The Thai experience in successfully
coordination, and harmonization of research; develops clinical trial and conducting two Phase III efficacy trials
manufacturing capacity; supports ethical and regulatory frameworks; and has provided world-class expertise in
ensures community participation. collaborative teamwork, community en-
gagement activities, good clinical practice,
ethical compliance, sample repositories,
the relatively good infrastructure of doing of efforts will result in more productive
immunology studies, good laboratory
epidemiology study in the region. associations and acceleration of AIDS
practices, volunteer retention strategies,
vaccine development.
and large-scale data management ap-
Current Efforts in HIV Vaccine proaches. These efforts and investments
Research and Development in Thailand resulted in an extraordinary volunteer
Asia AIDS vaccine activities have a long- retention rate of 96% during the latest
standing history in Thailand, beginning in Phase III trial. The Thai experience offers
Significant basic and clinical HIV the mid-1990s [14], reflecting a strong a good example of what can be achieved
vaccine research and development efforts political will to invest in AIDS vaccine through international collaboration. In
are already underway across the region, research and development to stem the addition, several preclinical studies with
with notable achievements (see Table 1). epidemic in Thailand. Thailand has con- promising HIV vaccine candidates are
Several Asian countries now have Nation- ducted Phase I, II, and III trials with the now ongoing in Thailand. Researchers in
al AIDS Vaccine Plans and Strategies. AIDSVax gp120 B9/E vaccine, as well as Chulalongkorn University are working on
Further harmonization and consolidation studies of other candidate vaccines (Table 1). a HIV-1 AE/B mosaic DNA vaccine as
well as techniques to improve vaccine
delivery system. The Thailand Regional
Box 1. AVAN’s Vision and Mission Laboratories in the Comprehensive Anti-
body-Vaccine Immune Monitoring Con-
Vision
sortium are working with researchers from
N To develop a safe and effective AIDS vaccine and ensure its access as a part of a the Los Alamos National Library and the
National Cancer Institute on a cocktail of
comprehensive public health strategy for the control of new HIV infections
across the Asian region. the mosaic DNA vaccine and DNA/
vaccinia primed-boosted strategy. These
Goals candidates were tested in mice and have
shown a promising immunogenicity result.
N To accelerate the development of an AIDS vaccine through expanding capacity
for all aspects of AIDS vaccine research and development.
China
N To build up regional resource centres and collaborative platforms to promote
The Chinese government has significantly
innovative AIDS vaccine research strategies.
invested in AIDS vaccine research. China’s
N To enlarge the pipeline of candidate AIDS vaccines suitable for use among vaccine scientists have recently formed the
Asian populations. Chinese AIDS Vaccine Initiative (CAVI),
N To strengthen capacity and harmonize regulatory and ethical frameworks for supported by the government of China. The
the conduct of clinical trials that comply with internationally recognized initiative includes projects aimed at cohort
standards. development, vaccine vector design, estab-
N To actively involve community partners at all stages of AIDS vaccine lishment of clinical trial units, a manufac-
development, clinical trials, and future use. turing facility using principles of good
N To promote and share manufacturing and production capacity in the region, in manufacturing practice (GMP), a primate
compliance with GMP standards, both for clinical trials and for the potential of centre, a humanized mouse centre, and
high demand for an AIDS vaccine in Asia. technical platforms encompassing both neu-
N To engage with and advocate to governments and the private sector to commit tralizing antibody and T cell expertise.
to and provide political and financial support for all phases of AIDS vaccine Clinical studies conducted so far include
research and development, including licensing and future access. three Phase I trials: V3 peptide vaccine in
N To facilitate collaboration between Asian scientists and industry with the rest of 1993, DNA/MVA (modified vaccinia
Ankara) in 2005, and a DNA/Tiantan
the world.
N To align with the values and strategic goals of the Global HIV Vaccine vaccinia strain (replication-competent) in
2007 (Table 1). Pre-clinical studies with
Enterprise.
DNA and modified Tiantan vaccinia, as

PLoS Medicine | www.plosmedicine.org 2 September 2010 | Volume 7 | Issue 9 | e1000331

22 iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Figure 1. Regional distribution of HIV-1 subtypes and recombinants. The relative proportion of each subtype/recombinant is shown in the
pie chart for each country. The data on Figure 1 are from the former studies in the region and estimates made by AVAN Task Force members of the
manuscript based on the unpublished data of their own research projects.
doi:10.1371/journal.pmed.1000331.g001

well as new versions of DNA, vaccinia,
and adenovirus vector approaches for
inducing mucosal immunity show consid-
erable promise [15–17]. China’s AIDS
vaccine programme now has substantial
capacity in primate centres, vaccine pro-
duction, and clinical trial sites develop-
ment that will all help to facilitate further
research and development in the region.
India
AIDS vaccine trials have accelerated
significantly in India in recent years.
Activities include an adeno-associated
virus vector Phase I trial, modified MVA
vector vaccine trials [18,19], and a recent
DNA/MVA Phase I prime-boost trial
initiated in 2009 [20]. The International
AIDS Vaccine Initiative (IAVI) is support-
ing Indian AIDS vaccine trials and applied
research on neutralizing antibody immu-
nogens [21]. Exploration of several novel
concepts, including prime-boost regimens
with CD40L adjuvants, as well as research
towards development of improved Env-
based immunogens, is also progressing.
Improved awareness of AIDS vaccine
issues, training of staff, and development

PLoS Medicine | www.plosmedicine.org 3 September 2010 | Volume 7 | Issue 9 | e1000331

iMedPub :: Thematic collections :: Volume 2 23
World Health Report 2012: No Health Without Research

Table 1. AIDS Vaccine Trials across Asia to Date.

Country Vaccine Sponsor Subtype Phase/Time Reference

Thailand V3 peptides UBI Multiclade I/1994 [32]

gp120 Vaxgen, Chiron E, B, B9/E I/II,III/1995 [33]
gp160 DoD E I/II
Canarypox+gp 160 vs. gp 120 US NIH/DoD B/E I/II/2003 [34]
Adenovirus type 5 Merck B I/II/2003
MVA US NIH/DoD A/E I [35]
DNA+fowlpox Australia A/E I/2004 [36]
Canarypox+gp 120 US NIH/DoD B/E III/2006 [3]
China V3 peptides UBI Co. Multiclade I/1993 [31]
DNA+MVA Baike Co. CRF08_B/C I/2005
Tiantan vaccinia replicative China CDC/EU CRF 07_B/C I/2006 [15]
DNA+Tiantan replicative China CDC CRF 07_B/C I/2008 [37]
DNA+MVA Baike Co. CRF08_B/C II/2009
India Adeno-associated virus IAVI C I/2003 [18]
MVA IAVI C I/2005 [20]
AAV vs. DNA+MVA IAVI C I/2009 [38]
Australia DNA+fowlpox US NIH B I/2004 [21]
DNA+fowlpox Australia A/E I [36]

CDC, Center for Disease Control and Prevention; DoD, Department of Defense; EU, European Union.
doi:10.1371/journal.pmed.1000331.t001

of trial sites have substantially improved
the capacity of conducting AIDS vaccine
research and clinical trials within India.
Australia
Australia has long-standing capacities in
both fundamental HIV research and
clinical trial activities. A consortium,
termed the Australia-Thai HIV Vaccine
Consortium, conducted two recent trials,
first in Sydney and then in Bangkok, of a
DNA-prime and Fowlpox virus boost
vaccine using both subtype B and
CRF01_AE strains [22]. Australia was a
clinical trial site of the adenovirus-based
efficacy (STEP) trial that was not effica-
cious. Through collaborations, several new
vaccine candidates are emerging. These
include peptide-based vaccines, gp140
immunogens, recombinant influenza vec-
tors, and particle-based vaccine strategies
[23,24]. In addition, improved and sim-
pler assays to measure T cell immunity
and antibody-dependent cellular cytotox-
icity (ADCC) are emerging [25,26]. These
scientific approaches could be accelerated
into expanded clinical trials in future
collaborations across the Asian region.
Japan
Japan has developed a pipeline of
promising AIDS vaccine candidates.
These include recombinant Mycobacterium
bovis/Bacillus Calmette-Guérin (BCG)
vector-based regimens that are moving
towards clinical trial development in
collaboration with Thai groups [27]. In
addition, novel Sendai virus vectors are
being developed with the opportunity to
deliver vaccines mucosally to induce
mucosal immunity [28]. Sendai virus
vectors combined with DNA vaccine
candidates are moving towards clinical
trials in collaboration with IAVI.
The Evolution, Mission, and
Strategic Plan Development of
AVAN
The first Asian meetings devoted to
AIDS vaccines took place in Japan, China,
and Thailand in the late 1990s. The initial
efforts were followed by a World Health
Organization (WHO)/Joint United Na-
tions Programme on HIV/AIDS (UN-
AIDS) regional consultation on AIDS
vaccines in Japan in 2006 and in Beijing
in 2009 [29–31]. The challenges for
engaging countries and communities in
expanding pre-clinical and clinical trials,
and in enhancing regulatory and manu-
facturing capacity to accelerate the devel-
opment of AIDS vaccines in the region,
led to the establishment of AVAN, an-
nounced at the 2009 AIDS Vaccine
Conference in Paris. AVAN has had
regular consultations towards developing
an Asia-specific Strategic AIDS Vaccine
Plan in alignment with the Enterprise
Scientific Strategic Plan [6].
An AVAN Task Force was recently
established to foster further dialogue and
set the stage for the development of
principles and priorities for a focussed,
forward-looking and effective strategic
plan for the network. Eventually, the Task
Force will be replaced by a steering
committee to guide implementation of a
strategic plan. An initial AVAN Task
Force secretariat has been established,
currently based in China, with strong
support from the Chinese AIDS Vaccine
Initiative, WHO, the Enterprise, and the
Chinese government. The Task Force and
secretariat foster forums for consultation
and networking around many of the most
critical future issues. Regional workshops
are planned to further implement the
emerging strategic plan in June 2010 in
Bangkok, Thailand, and in August/Sep-
tember 2010 in Sapporo, Japan. AVAN
plans to support and promote the devel-
opment of regional resource centres to
provide information, a reagents repository,
protocols, and training on the many
aspects of AIDS vaccine research and
development. Coordination and harmoni-
zation of various aspects of AIDS vaccine
development, particularly regional vaccine
immunology evaluation laboratories and
regulatory reviews, is a high priority in
moving AIDS vaccine research forward

PLoS Medicine | www.plosmedicine.org 4 September 2010 | Volume 7 | Issue 9 | e1000331

24 iMedPub :: Thematic Collections :: Volume 2

across Asia. AVAN shall ultimately be-
come the key advocate for AIDS vaccine
research in the region, building upon the
trust created through working together in
collaborative partnerships in the region
and with support from the Enterprise.
Past Collaborations and Future
Opportunities
AVAN aims to make a significant impact
as the region undertakes a stronger role in
AIDS vaccine research and development.
Asian investigators have substantial capac-
ity to conduct basic research, which could
be significantly enhanced through better
collaboration. The current pipeline of new
vaccine candidates needs further expansion
with more innovative concepts and trans-
national efforts to accelerate research.
Clinical trial capacity could be quickly
exhausted with multiple, concurrent effica-
cy trials in the future. As partially effica-
cious AIDS vaccines are identified either in
Asia or around the world, determining their
effectiveness in regional epidemics involv-
ing differing HIV virus subtypes will be
critical. Considering the large number of
IDUs in many countries of the region still
with a high incidence of HIV infection,
Asia may serve as the ideal site suitable for
testing a vaccine to prevent parental HIV
transmission.
Building upon and maintaining the
substantial efficacy trials expertise in Thai-
land is both an opportunity and a
challenge. Substantial opportunities exist
to harmonize the regulatory and ethical
review of AIDS vaccine trials across the
region—too often, delays experienced in
initiating trials with vectors of known
safety profile are very lengthy. Data
management and data sharing across the
region can also be enhanced. Opportuni-
ties for training young immunologists and
virologists in the region to foster the next
generation of scientists need to be encour-
aged and supported. Technology transfer
from Western countries to Asian countries
also needs to be strengthened. Finally,
References
1. Buchbinder SP, Mehrotra DV, Duerr A,
Fitzgerald DW, Mogg R, et al. (2008) Efficacy
assessment of a cell-mediated immunity HIV-1
vaccine (the Step Study): a double-blind, rando-
mised, placebo-controlled, test-of-concept trial.
Lancet 372: 1881–1893.
2. Fauci AS, Johnston MI, Dieffenbach CW,
Burton DR, Hammer SM, et al. (2008) HIV
vaccine research: the way forward. Science 321:
530–532.
3. Rerks-Ngarm S, Pitisuttithum P, Nitayaphan S,
Kaewkungwal J, Chiu J, et al. (2009) Vaccination
with ALVAC and AIDSVAX to Prevent HIV-1
Infection in Thailand. N Engl J Med 361:
2209–2220.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
tapping into the large cost-effective vac-
cine manufacturing capacity in several
Asian countries is an opportunity and
priority.
International groups have previously
provided high level support for AIDS
vaccine activities in the Asian region,
including research and technical transfer,
funding and policy. In the past two
decades, the US National Institutes of
Health (NIH) has provided extensive
support for basic and clinical research
across many Asian countries and funded
the majority of the RV 144 trial. The
Walter Reed Army Institute of Research
and the US Military HIV Research
Program (MHRP), in collaboration with
the Royal Thai Army, sponsored several
clinical trials in Thailand, including the
RV144 trial. IAVI is supporting clinical
trial capacity building along with policy
and preparedness activities. The Euro-
Vacc AIDS Vaccine network has included
the Chinese HIV strains in their develop-
ment work in view of future plans of
conducting trials in the region. The
Collaboration for AIDS Vaccine Discov-
ery (CAVD), supported by the Bill &
Melinda Gates Foundation, is also looking
towards enhancing research on AIDS
vaccines in Asia in coming years. WHO-
UNAIDS and the Global HIV Vaccine
Enterprise are actively engaging in policy
development, assisting the development of
national AIDS vaccine plans, and provid-
ing a global umbrella for AIDS vaccine
development activities in Asia. Further
investment in AIDS vaccine development
by the most developed nations in the Asian
region would be welcome. The consolida-
tion of this collaborative support will be
critical in progressing future vaccine
development efforts.
Conclusions
The development of an effective AIDS
vaccine has never been more urgent,
particularly for the hundreds of millions
of people across Asia at substantial risk of
4. Klausner RD, Fauci AS, Corey L, Nabel GJ,
Gayle H, et al. (2003) Medicine. The need for a
global HIV vaccine enterprise. Science 300:
2036–2039.
5. Kaleebu P, Abimiku A, El-Halabi S, Koulla-
Shiro S, Mamotte N, et al. (2008) African AIDS
vaccine programme for a coordinated and
collaborative vaccine development effort on the
continent. PLoS Med 5: e236. doi:10.1371/
journal.pmed.0050236.
6. Coordinating Committee of the Global HIV/
AIDS Vaccine Enterprise (2005) The Global
HIV/AIDS Vaccine Enterprise: scientific strate-
gic plan. PLoS Med 2: e25. doi:10.1371/journal.
pmed.0020025.
5 September 2010 | Volume 7 | Issue 9 | e1000331
acquiring HIV. Similarly, the necessity of
coordinating and harmonizing efforts
across regional alliances has become
abundantly clear. AVAN has been initiat-
ed to meet these needs and actively
facilitate the development of a regional
AIDS vaccine strategy that accelerates
research and development of an AIDS
vaccine through government advocacy,
improved coordination and harmoniza-
tion of research; develops clinical trial and
manufacturing capacity; supports ethical
and regulatory frameworks; and ensures
community participation.
Acknowledgments
The authors gratefully acknowledge the enthu-
siasm of AVAN investigators and supporters,
including: Zhiwei Chen, Hong Kong, China;
Jose Esparza, Bill & Melinda Gates Foundation,
Seattle, USA; Jean-Louis Excler, WHO consul-
tant; Jorge Flores, Division of AIDS Research,
NIH, USA; Cate Hankins, UNAIDS, Switzer-
land; Fera Ibrahim, Indonesia; Edward Kar-
amov, Russian Federation; Joan Kaufman,
IAVI, USA; Sonali Kochhar, IAVI India;
Bonnie Mathieson, Division of AIDS Research,
NIH, USA; Sanjay Mehandale, India; Van
Kinh Nguyen, Viet Nam; Kiat Ruxrungtham,
Thailand; Eric Sandström, Sweden; Xuan Lien
Truong, Viet Nam; and Naoki Yamamoto,
Japan. The authors also acknowledge the help
of Tim France (http://www.iniscommunica-
tion.com/) for technical editing of the paper
and the help of the AVAN Task Force
Secretariat: Tao Teng, Hua Liang, and Xin
Xin.
Author Contributions
ICMJE criteria for authorship read and met:
SJK DAC MCV YS LZ NKG BB HT RD
SRN PP NVK AB SO. Agree with the
manuscript’s results and conclusions: SJK
DAC MCV YS LZ NKG BB HT RD SRN
PP NVK AB SO. Wrote the first draft of the
paper: SJK LZ HT SRN SO. Contributed to
the writing of the paper: DAC MCV YS LZ
NKG BB HT RD SRN PP NVK AB SO.
Participated in discussion through teleconfer-
ence of the AVAN task force and provided
information concerning the efforts in HIV
vaccine research in Indonesia: BB.
7. UNAIDS (2010) Joint United Nations Pro-
gramme on HIV/AIDS homepage. Available:
http://www.unaids.org/en/CountryResponses/
Regions/Asia.asp. Accessed 17 August 2010.
8. Commission on AIDS in Asia (2008) Redefining
AIDS in Asia: crafting an effective response.
Report of the Commission on AIDS in Asia.
HMarais, ed. New Delhi: Oxford University
Press.
9. Lu L, Jia M, Ma Y, Yang L, Chen Z, et al. (2008)
The changing face of HIV in China. Nature 455:
609–611.
10. Wang W, Jiang S, Li S, Yang K, Ma L, et al.
(2008) Identification of subtype B, multiple
circulating recombinant forms and unique re-
25
World Health Report 2012: No Health Without Research
combinants of HIV type 1 in an MSM cohort in
China. AIDS Res Hum Retroviruses 24:
1245–1254.
11. Hemelaar J, Gouws E, Ghys PD, Osmanov S
(2006) Global and regional distribution of HIV-1
genetic subtypes and recombinants in 2004. AIDS
20: W13–W23.
12. Deshpande A, Jauvin V, Pinson P, Jeannot AC,
Fleury HJ (2009) Phylogenetic analysis of HIV-1
reverse transcriptase sequences from 382 patients
recruited in JJ Hospital of Mumbai, India,
between 2002 and 2008. AIDS Res Hum
Retroviruses 25: 633–635.
13. Wirachsilp P, Kantakamalakul W, Foongladda S,
Chuenchitra T, Kohriangudom S, et al. (2007)
Surveillance of subtype and genetic variation of
the circulating strains of HIV-1 in Thailand.
Southeast Asian J Trop Med Public Health 38:
814–827.
14. Pitisuttithum P (2008) HIV vaccine research in
Thailand: lessons learned. Expert Rev Vaccines
7: 311–317.
15. Shao Y, Li T, Shen X, Zhang J, Zhu H, et al.
(2008) The safety and immunogenicity of the
replicative Tiantan vaccinia HIV vaccine in
phase I clinical trial. Poster No P13-18, AIDS
Vaccine 2008, Cape Town, South Africa, Oct
13–16.
16. Huang X, Lu B, Yu W, Fang Q, Liu L, et al.
(2009) A novel replication-competent vaccinia
vector MVTT is superior to MVA for inducing
high levels of neutralizing antibody via mucosal
vaccination. PLoS ONE 4: e4180. doi:10.1371/
journal.pone.0004180.
17. Dai K, Liu Y, Liu M, Xu J, Huang W, et al.
(2008) Pathogenicity and immunogenicity of
recombinant Tiantan Vaccinia Virus with deleted
C12L and A53R genes. Vaccine 26: 5062–5071.
18. Mehendale S, van Lunzen J, Clumeck N,
Rockstroh J, Vets E, et al. (2008) A phase 1
study to evaluate the safety and immunogenicity
of a recombinant HIV type 1 subtype C adeno-
associated virus vaccine. AIDS Res Hum Retro-
viruses 24: 873–880.
19. Ramanathan VD, Kumar M, Mahalingam J,
Sathyamoorthy P, Narayanan PR, Solomon S,
et al. (2009) A Phase 1 study to evaluate the safety
and immunogenicity of a recombinant HIV-1
subtype C modified vaccinia Ankara virus vaccine
PLoS Medicine | www.plosmedicine.org
26
candidate in Indian volunteers. AIDS Res
Human Retroviruses 25: 1107–1116.
20. Berkley S (2008) HIV vaccine trials in India. Nat
Biotechnol 26: 495; author reply 496.
21. International AIDS Vaccine Initiative (2010)
Working with communities: India. Available:
http://www.iavi.org/working-with-communities/
country-programs/Pages/india.aspx. Accessed 17
August 2010.
22. Kelleher AD, Puls RL, Bebbington M, Boyle D,
Ffrench R, et al. (2006) A randomized, placebo-
controlled phase I trial of DNA prime, recombi-
nant fowlpox virus boost prophylactic vaccine for
HIV-1. AIDS 20: 294–297.
23. De Rose R, Fernandez CS, Smith MZ, Batten CJ,
Alcantara S, et al. (2008) Control of viremia
following immunotherapy of SIV-infected ma-
caques with peptide pulsed blood. PLoS Pathog 4:
e1000055. doi:10.1371/journal.ppat.1000055.
24. De Rose R, Zelikin A, Johnston APR, Sexton A,
Chong SF, et al. (2008) Binding, internalisation
and antigen presentation of vaccine-loaded na-
noengineered capsules in blood. Advanced Ma-
terials 20: 4698–4703.
25. Stratov I, Chung A, Kent SJ (2008) Robust NK
cell-mediated human immunodeficiency virus
(HIV)-specific antibody-dependent responses in
HIV-infected subjects. J Virol 82: 5450–5459.
26. Zaunders JJ, Munier ML, Seddiki N, Pett S, Ip S,
et al. (2009) High levels of human antigen-specific
CD4+ T cells in peripheral blood revealed by
stimulated coexpression of CD25 and CD134
(OX40). J Immunol 183: 2827–2836.
27. Hiroi T, Goto H, Someya K, Yanagita M,
Honda M, et al. (2001) HIV mucosal vaccine:
nasal immunization with rBCG-V3J1 induces a
long term V3J1 peptide-specific neutralizing
immunity in Th1- and Th2-deficient conditions.
J Immunol 167: 5862–5867.
28. Kawada M, Tsukamoto T, Yamamoto H,
Iwamoto N, Kurihara K, et al. (2008) Gag-
specific cytotoxic T-lymphocyte-based control of
primary simian immunodeficiency virus replica-
tion in a vaccine trial. J Virol 82: 10199–10206.
29. Esparza J (1999) AIDS vaccine research in Asia:
needs and opportunities. Report from a UN-
AIDS/WHO/NIID meeting Tokyo, 28–30 Oc-
tober 1998. AIDS 13: UNAIDS 1–UNAIDS 13.
6 September 2010 | Volume 7 | Issue 9 | e1000331
30. Excler JL, Pitisuttithum P, Rerks-Ngarm S,
Shao Y, Zhang L, et al. (2008) Expanding
research capacity and accelerating AIDS vaccine
development in Asia. Southeast Asian J Trop
Med Public Health 39: 766–784.
31. Rerks-Ngarm S, Pitisuttithum P, Ganguly N,
ZhangLi, Tamashiro H, et al. (2010) Defining the
objectives of the AIDS Vaccine for Asia Network:
WHO-UNAIDS/Global HIV Vaccine Enter-
prise regional consultation on expanding AIDS
vaccine research and development capacity in
Asia. Curr Opin HIV AIDS 5: 435–452.
32. Lenihan F (1993) China and Thailand to start
trials of AIDS vaccines. BMJ 306: 1564–1565.
33. Pitisuttithum P, Gilbert P, Gurwith M,
Heyward W, Martin M, et al. (2006) Random-
ized, double-blind, placebo-controlled efficacy
trial of a bivalent recombinant glycoprotein 120
HIV-1 vaccine among injection drug users in
Bangkok, Thailand. J Infect Dis 194: 1661–1671.
34. Thongcharoen P, Suriyanon V, Paris RM,
Khamboonruang C, de Souza MS, et al. (2007)
A phase 1/2 comparative vaccine trial of the
safety and immunogenicity of a CRF01_AE
(subtype E) candidate vaccine: ALVAC-HIV
(vCP1521) prime with oligomeric gp160
(92TH023/LAI-DID) or bivalent gp120
(CM235/SF2) boost. J Acquir Immune Defic
Syndr 46: 48–55.
35. Earl PL, Cotter C, Moss B, VanCott T, Currier J,
et al. (2009) Design and evaluation of multi-gene,
multi-clade HIV-1 MVA vaccines. Vaccine 27:
5885–5895.
36. De Rose R, Chea S, Dale CJ, Reece J,
Fernandez CS, et al. (2005) Subtype AE HIV-1
DNA and recombinant Fowlpoxvirus vaccines
encoding five shared HIV-1 genes: safety and T
cell immunogenicity in macaques. Vaccine 23:
1949–1956.
37. Shao Y, Li T, Wolf H, Liu Y, Wang H, et al.
(2009) The safety and immunogenicity of HIV-1
vaccines based on DNA and replication compe-
tent vaccinia vector in phase I clinical trial. Poster
NoP14-15LB, AIDS Vaccine 2009, Paris, Oct
19–22.
38. Kumar S, Aggarwal P, Vajpayee M, Pandey RM,
Seth P (2006) Development of a candidate DNA/
MVA HIV-1 subtype C vaccine for India.
Vaccine 24: 2585–2593.
iMedPub :: Thematic Collections :: Volume 2

Health in Action
A Field Training Guide for Human Subjects Research
Ethics
Maria W. Merritt1*, Alain B. Labrique2, Joanne Katz3, Mahbubur Rashid4, Keith P. West Jr.3, Joan Pettit5
1 Johns Hopkins Berman Institute of Bioethics and Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United
States of America, 2 Department of International Health and Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United
States of America, 3 Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America, 4 JiVitA Project
(USAID-Bangladesh Health and Population Programmes - Johns Hopkins University), Dhaka, Bangladesh, 5 Institutional Review Board Office, Johns Hopkins Bloomberg
School of Public Health, Baltimore, Maryland, United States of America
The Training Gap
Investigators who conduct research with
human subjects are responsible for the
protection of participants’ rights, safety,
and welfare, and for scientific integrity [1–
6]. Each investigator and research site
must look to the local laws and ethical
standards that apply to their role in a
research project [7]. As a matter of routine
practice, investigators often delegate to
other study team members research activ-
ities involving direct contact with human
participants. We propose the term data
collector to designate a distinct role on the
study team for field workers who seek
informed consent or collect data through
direct contact with individuals. When
investigators delegate such activities to
data collectors, the investigators’ ultimate
responsibility for human subject protection
and scientific integrity is in no way
diminished and requires them to train
these personnel in the principles and
practice of research ethics. Due to a lack
of standard training guidelines specific to
field workers in low-resource settings,
training may vary between sites and
principal investigators. Consequently, the
effective implementation of research ethics
principles may also vary widely and
arbitrarily across settings.
Several of us (ABL, JK, MR, KPW)
(Figure 1) regularly conduct large commu-
nity trials to evaluate the efficacy and
effectiveness of new interventions intended
to alleviate major global causes of illness
and death, such as diarrheal diseases,
respiratory infections, and childhood and
maternal undernutrition. We locate our
studies at sites where the targeted problem
confers a significant burden and research
findings will be most directly applicable—
The Health in Action section is a forum for
individuals or organizations to highlight their
innovative approaches to a particular health prob-
lem.
PLoS
iMedPub :: Medicinecollections
Thematic | www.plosmedicine.org
:: Volume 2
World Health Report 2012: No Health Without Research
most often in low-resource settings, among grams are not written at an appropriate
populations exposed to social or economic educational level for field workers, many
stress, and within cultural frameworks that of whom have limited formal education.
range from sub-Saharan to Southeast The freely available Family Health Inter-
Asian [8–16]. As our studies can range in national (FHI) Research Ethics Training
size from hundreds to tens of thousands of Curriculum, for instance, is a comprehen-
participants, we find that tailoring re- sive 267-page document designed to train
search ethics training to the many cadres international scientists [17]. The research
of workers potentially engaged in human ethics capacity-building guide offered by
subject contact is both time-consuming the World Health Organization (WHO) is
and difficult to standardize. We are targeted to a similar educational level [18].
challenged by the multiplicity of spoken By contrast, field workers in low-resource
languages, cultures, levels of literacy, and settings need easily accessible, pared-down
educational backgrounds of the workers instruction on the essentials of protecting
hired from the various communities asso- human subjects, including how to obtain
ciated with our research sites. consent in a respectful and diligent
Although training tools exist, some— manner, how to protect the confidentiality
such as the Collaborative Institutional of the data they are collecting, and a few
Training Initiative (CITI) or National other key ethical concepts. In addition,
Institutes of Health (NIH) online ethics they need practical support to perform
training materials—are too complex for their jobs well, such as guidance on
broad application in field settings. Such specific behaviors to direct their interac-
programs are cumbersome to use in tion with human subjects.
settings with limited access to computers
and the internet. Some require institution-
The Field Training Guide
al affiliations or annual institutional fees.
Translations of online training programs As a first step toward filling the training
into non-English languages are limited, gap, we used input from a broadly repre-
and the attempt to translate them into sentative group of experienced investigators
local languages is challenging and unduly at our institution to produce a Field
burdensome, potentially resulting in less Training Guide. In August 2009, we posted
effective training. Moreover, existing pro- the current version of the Guide on our
Citation: Merritt MW, Labrique AB, Katz J, Rashid M, West KP Jr., et al. (2010) A Field Training Guide for Human
Subjects Research Ethics. PLoS Med 7(10): e1000349. doi:10.1371/journal.pmed.1000349
Published October 5, 2010
Copyright: � 2010 Merritt et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: MWM’s work on this manuscript was supported by a career development award from the Greenwall
Faculty Scholars Program (http://www.greenwallfsp.org/). JK’s work on this manuscript was partially supported
by US National Institutes of Health grants ES015558 and HD061712. The funders had no role in the decision to
submit the manuscript or in its preparation.
Competing Interests: The authors have declared that no competing interests exist.
Abbreviations: CITI, Collaborative Institutional Training Initiative; FHI, Family Health International; JHSPH,
Johns Hopkins School of Public Health; NIH, National Institutes of Health; WHO, World Health Organization.
* E-mail: mmerritt@jhsph.edu
Provenance: Not commissioned; externally peer reviewed.
1 October 2010 | Volume 7 | Issue 10 | e1000349 27
World Health Report 2012: No Health Without Research

Summary Points staff will be engaged in, and thus tailor the
level of training needed for that cadre
N Community trials of interventions to address major global causes of illness and
above a threshold of basic respect for
persons and confidentiality. For example,
death are often located in low-resource settings, where research findings will be
most directly applicable. interviewers collecting in-depth data on
N Although investigators delegate research activities involving human subject
socioeconomic characteristics or sexual
behaviors have a different level of respon-
contact to local field workers, they retain ultimate responsibility for human
subject protection and scientific integrity. sibility than workers doing simple head

N To train every cadre of field worker in research ethics requires simplified training
counts, and need to understand their
ethical obligations in greater depth.
guidelines that can be easily translated and adapted for use in a wide variety of
settings and cultural frameworks, especially where field workers have limited
formal education. Orientation to Basic Principles of
N Field workers need appropriate training materials, tailored to varying levels of Community Research
Technologically and culturally appropri-
human subject responsibility, that focus on basic principles of community research.
N We have produced a Field Training Guide for Human Subjects Research Ethics, ate training materials are needed to orient
higher-level cadres of field workers to basic
which is freely available to the public. In this article we address how to identify
field training needs and meet high standards of research ethics at every level of principles of community research, such as
human subject interaction. equipoise (to explain that research would
not be ethical if an answer were known to
exist), randomization, and placebo controls
publicly accessible IRB web site (http:// cultural challenge of encouraging data (how it is that some people may get an
www.jhsph.edu/bin/u/p/Field%20Guide_ collectors to admit to mistakes or inadver- active substance and others not). Training
25Feb10.pdf) [19]. Investigators working in tent procedural lapses, or to ask questions should prepare field workers to respond
various countries and situations were en- about things they do not understand. appropriately when participants perceive
couraged to adapt the delivery of content to Our IRB Office now routinely refers them as medical workers and ask them for
their specific projects. investigators to the Guide as an approved advice exceeding their level of competence,
The Guide is innovative in two primary curriculum in human subjects research or when they feel compelled to intervene in
ways. First, it identifies the distinct role of training for study staff. Users have trans- a situation of acute medical need but are
data collector and situates the responsibilities lated the Guide into French, Mandarin, neither trained nor mandated to do so.
of the role within the institutional context and Thai (Text S2–S4). Bangla and Nepali Investigators need to define lines of re-
governing the activities of the entire translations are in progress. The Nepali course for field workers who find them-
research team [20]. The Guide defines data translation will be used to test and obtain selves facing such dilemmas [21,22].
collectors as study team members ‘‘who will feedback on the Guide’s content while
(1) obtain informed consent from research training workers for new trials in Nepal. Social Complexities of Community
participants, or (2) collect data from human Research
subjects through individual or focus group Additional Training Needs Approaching subjects to seek informed
interviews, testing, physical measurements, consent or collect data in community
or other procedures involving direct con- Our ongoing experiences in the field research usually involves contact not only
tact’’ [19]. As an agent representing study and in the IRB review process indicate with the prospective subject but also with
investigators and the institutions charged needs for additional materials to support senior decision makers in the family. The
with ethical oversight of the research investigators’ delivery of ethical training to very event of a home visit by a field worker
protocol, the data collector bears two weighty field workers. has the potential to breach a subject’s
responsibilities: to act respectfully and confidentiality by exposing private circum-
otherwise appropriately toward each study Levels of Authority in Interaction stances to the neighborhood, as when a field
participant with whom he or she has with Human Subjects worker tests a subject for pregnancy and
contact for research purposes; and to The consenting process should always be brings antenatal supplements on subsequent
safeguard the confidentiality and integrity carried out by a responsible cadre of visits. Field workers in community settings
of the data that he or she collects [19]. workers articulate in the local language are often pressed by local government
Second, the Guide presents the two and culture. In some research settings, officials, opinion leaders, family heads, or
main responsibilities of the data collector’s workers authorized to administer informed NGO workers to divulge information about
role in concrete terms that trainers can consent are the same individuals who study subjects, as in studies of sexually
readily convey to workers in the field. This collect all the data, whereas in other settings transmitted conditions or other sensitive
includes detailed instruction on specific consent is only within the purview of matters. Field workers need to be trained
behaviors that promote basic ethical supervisor-level staff. The current version to manage these complex social situations so
principles: for example, paying attention of our Guide is oriented toward this higher- as to protect subjects’ privacy and the
to body language when seeking informed level cadre of workers. In settings where confidentiality of personal information.
consent or recording interview responses, data are collected by a distinct cadre of
and safeguarding data collection sheets to workers, usually less educated, who are not Opening a Conversation on
protect the confidentiality of personal authorized to administer consent, a need Field Training Tools for
information [19]. The Guide conveys the exists for field training materials targeted Research Ethics
importance of correct, accurate data specifically to them. Investigators could use
recording and careful, systematic trans- a checklist to assess the level of human Our Field Training Guide is a first step
mittal of data records. It also addresses the subject interaction a particular cadre of toward developing locally adaptable research

PLoS Medicine | www.plosmedicine.org 2 October 2010 | Volume 7 | Issue 10 | e1000349

28 iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

adaptation by investigators beyond our

institution, assuming that they already have
in place an organizational infrastructure
through which they normally train and
supervise field personnel in the activities
necessary to conduct research. We encour-
age readers to examine the Guide, try it out,
translate it, and identify potential improve-
ments. A welcome step forward would be
systematic evaluation of the Guide, or of
users’ adaptations thereof, as compared with
other training tools. We invite other investi-
gators and institutions to join us in conver-
sation about how to address field training
needs so as to meet high standards of
research ethics at every level of human
subject interaction.

Supporting Information
Text S1 JHSPH Field Training Guide:
English.
Found at: doi:10.1371/journal.pmed.
1000349.s001 (0.04 MB PDF)
Text S2 French translation of JHSPH
Field Training Guide courtesy of Trans-
Perfect Translations.
Found at: doi:10.1371/journal.pmed.
1000349.s002 (0.05 MB PDF)
Text S3 Mandarin translation of JHSPH
Field Training Guide courtesy of Youfa
Wang.
Found at: doi:10.1371/journal.pmed.
1000349.s003 (0.31 MB PDF)
Text S4 Thai translation of JHSPH
Field Training Guide courtesy of David
Celentano and Louise Walshe.
Found at: doi:10.1371/journal.pmed.
1000349.s004 (0.17 MB PDF)

Figure 1. Field interviewer Lipi Begum, of the JiVitA Project in rural Gaibandha, Author Contributions
Bangladesh, conducts an enrollment interview with a consented study participant. In ICMJE criteria for authorship read and met:
addition to building rapport, speaking in the local dialect, using linguistic terms of respect, and MWM ABL JK MR KPW JP. Agree with the
clearly explaining the objective of the interview, specific behaviors that convey respect for the manuscript’s results and conclusions: MWM
participant in the Gaibandha setting are eye contact, the equal position of the interviewer to the ABL JK MR KPW JP. Wrote the first draft of
participant (seated on the bed), and the prominent display of an identification badge. Specific the paper: MWM. Contributed to the writing of
behaviors should always be tailored to the local setting. For example, while eye contact conveys the paper: MWM ABL JK MR KPW JP.
respect in the setting depicted here, it does not do so universally and would be inappropriate in Coordinated co-authors’ contributions to the
some other settings. Image Credit: Alain B. Labrique; available under Creative Commons paper through successive drafts, revisions, and
Attribution License.
final version: MWM. Conducted field studies
doi:10.1371/journal.pmed.1000349.g001
and ethics training practices that informed the
Field Guide and paper; helped write the Field
ethics training tools for study teams working an extensive search, no other comparably Guide: ABL JK MR KPW. Wrote the first draft
in a variety of settings around the world. simple field training guide is publicly avail- of the Field Guide and coordinated subsequent
To the best of our knowledge, following able. We intend our Guide to be suitable for revisions: JP.

References
1. World Medical Association (2008) The Declaration
of Helsinki: Ethical Principles for Medical Research
Involving Human Subjects. Available: http://www.
wma.net/en/30publications/10policies/b3/index.
html. Accessed 15 March 2010.
2. Council for International Organizations of Medical
Sciences (CIOMS) in collaboration with the World
Health Organization (WHO) (2002) International
Ethical Guidelines for Biomedical Research Involv-
ing Human Subjects. Available: http://www.cioms.
ch/publications/layout_guide2002.pdf. Accessed 6
September 2010.
3. National Commission for the Protection of
Human Subjects of Biomedical and Behavioral
Research (1979) The Belmont Report: Ethical
Principles And Guidelines For The Protection Of
Human Subjects Of Research. Available: http://
ohsr.od.nih.gov/guidelines/belmont.html. Ac-
cessed 15 March 2010.
4. U.S. Department of Health and Human Services,
National Institutes of Health, and Office for
Human Research Protections (1991, revised
2005) The Common Rule, Title 45 (Public
Welfare), Code of Federal Regulations, Part 46

PLoS Medicine | www.plosmedicine.org 3 October 2010 | Volume 7 | Issue 10 | e1000349

iMedPub :: Thematic collections :: Volume 2 29
World Health Report 2012: No Health Without Research
(Protection of Human Subjects). Available:
http://ohsr.od.nih.gov/guidelines/45cfr46.html.
Accessed 15 March 2010.
5. U.S. Department of Health and Human Services,
Food and Drug Administration (1980) Title 21
(Food and Drugs), Code of Federal Regulations,
Part 50 (Protection of Human Subjects). Avail-
able: http://www.accessdata.fda.gov/scripts/cdrh/
cfdocs/cfCFR/CFRSearch.cfm?CFRPart = 50.
Accessed 15 March 2010.
6. Nuffield Council (2002) The Ethics of Research
Related to Healthcare in Developing Countries.
Available: http://www.nuffieldbioethics.org/go/
ourwork/developingcountries/introduction. Ac-
cessed 3 June 2010.
7. International Compilation of Human Research
Protections. Available: http://www.hhs.gov/
ohrp/international/HSPCompilation.pdf. Ac-
cessed 3 June 2010.
8. Klemm RDW, Labrique AB, Christian P,
Rashid M, Shamim AA, et al. (2008) Newborn
vitamin A supplementation reduced infant mor-
tality in rural Bangladesh. Pediatrics 122:
180–181.
9. Tielsch JM, Khatry SK, Stoltzfus RJ, Katz J,
LeClerq SC, et al. (2007) Effect of daily zinc
supplementation on child mortality in southern
Nepal: a community-based, cluster randomised,
placebo-controlled trial. Lancet 370: 1230–1239.
10. Tielsch JM, Darmstadt GL, Mullany LC,
Khatry SK, Katz J, et al. (2007) Impact of
PLoS Medicine | www.plosmedicine.org
30
newborn skin-cleansing with chlorhexidine on
neonatal mortality in southern Nepal: A commu-
nity-based, cluster-randomized trial. Pediatrics
119: e330–e340.
11. Semba RD, Ndugwa C, Perry RT, Clark TD,
Jackson JB, et al. (2005) Effect of periodic vitamin
A supplementation on mortality and morbidity of
human immunodeficiency virus-infected children
in Uganda: A controlled clinical trial. Nutrition
21: 25–31.
12. Mebrahtu T, Stoltzfus RJ, Chwaya HM, Jape JK,
Savioli L, et al. (2004) Low-dose daily iron
supplementation for 12 months does not increase
the prevalence of malarial infection or density of
parasites in young Zanzibari children. J Nutr
134 (11): 3037–3041.
13. Christian P, West Jr. KP, Khatry SK,
LeClerq SC, Pradhan EK, et al. (2003) Effects
of maternal micronutrient supplementation on
fetal loss and infant mortality: a cluster-random-
ized trial in Nepal. Am J Clin Nutr 78:
1194–1202.
14. West Jr. KP, Katz J, Khatry SK, LeClerq SC,
Pradhan EK, et al. (1999) Double blind, cluster
randomised trial of low dose supplementation
with vitamin A or b carotene on mortality related
to pregnancy in Nepal. BMJ 318: 570–575.
15. West KP, Jr., Pokhrel RP, Katz J, LeClerq SC,
Khatry SK, et al. (1991) Efficacy of vitamin A in
reducing preschool child mortality in Nepal.
Lancet 338: 67–71.
4 October 2010 | Volume 7 | Issue 10 | e1000349
16. Sommer A, Tarwotjo I, Djunaedi E, West KP, Jr.,
Loedin AA, et al. (1986) Impact of vitamin A
supplementation on childhood mortality: A ran-
domized controlled community trial. Lancet 327:
1169–1173.
17. Family Health International Research Ethics Train-
ing Curriculum. Available: http://www.fhi.org/
NR/rdonlyres/eciqqcmgblapkofxnolmj2e7hkjenvln
rgjo2f4iassesnalxiw2z66chszjaty3ridg7rqwdef6gl/
ResearchEthics.pdf. Accessed 3 June 2010.
18. World Health Organization (2009) Research
Ethics Committees: Basic Concepts for Capaci-
ty-Building. Available: http://www.who.int/eth/
Ethics_basic_concepts_ENG.pdf. Accessed 3
June 2010.
19. Johns Hopkins School of Public Health (JHSPH)
(2009) Human Subjects Research Ethics Field
Training Guide. Available: http://www.jhsph.
edu/bin/u/p/Field%20Guide_25Feb10.pdf. Ac-
cessed 15 March 2010.
20. Hardimon MO (1994) Role Obligations. J Philos
XCI. pp 333–363.
21. Merritt MW, Taylor HA, Mullany LC (2010)
Ancillary Care in Community-Based Public
Health Intervention Research. Am J Public
Health 100: 211–216.
22. Hyder AA, Merritt MW (2009) Ancillary Care for
Public Health Research in Developing Countries.
JAMA 302: 429–431.
iMedPub :: Thematic Collections :: Volume 2

Health in Action
Developing ANDI: A Novel Approach to Health Product
R&D in Africa
Solomon Nwaka1*, Tshinko B. Ilunga2, Jorge Santos Da Silva3, Emiliano Rial Verde3, Doan Hackley3,
Raymond De Vré3, Tom Mboya-Okeyo4, Robert G. Ridley1
1 UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (TDR), World Health Organization, Geneva, Switzerland, 2 African
Development Bank, Tunis-Belvedère, Tunisia, 3 McKinsey & Company, Geneva, Switzerland, 4 Permanent Mission of the Republic of Kenya to the United Nations and Other
International Organizations, Geneva, Switzerland
A Need for African-Led Health
R&D Innovation
The health status of the African popu-
lation remains behind that of populations
in Europe and North America, as well as
many other developing regions with sim-
ilar affluence (Figure S1). For example,
Africa is especially affected by a series of
infectious diseases that are responsible for
more than half of its disability-adjusted life
years (DALYs) and over 6 million deaths
per year (Figure S2). For the 18 diseases
listed in Figure S2 Africa has over 30%,
and in some cases over 90%, of the
worldwide disease burden, even though it
represents only 15% of the global popula-
tion. Currently, there are limited or no
affordable therapies or vaccines for many
of these conditions, and diagnostic meth-
ods, where they exist, are often inadequate
to deploy in the field for large populations
[1–5]. Accurate quantification of the
economic impact of disease burden is
difficult. However, the negative impact of
these diseases to the African gross domes-
tic product (GDP) may run into tens of
billions of dollars (US) each year [5–7].
Despite some welcome increases in
global R&D investment in recent years
(e.g., through public private partnerships),
it is generally agreed that research and
product pipelines for the diseases that
disproportionately affect developing coun-
tries are grossly inadequate. This has
recently been reinforced by the 192
countries composing the World Health
Assembly through resolutions supporting
GSPOA (the Global Strategy and Plan of
Action on Public Health, Innovation and
Intellectual Property) [8,9]. There is
urgent need for enhanced research in
developing countries. Our analysis suggest
The Health in Action section is a forum for
individuals or organizations to highlight their
innovative approaches to a particular health
problem.
iMedPub PLoS Medicinecollections
:: Thematic | www.plosmedicine.org
:: Volume 2
World Health Report 2012: No Health Without Research
that, in addition to the scant research for Drug and Diagnostics Innovation (ANDI)
several diseases, only 21% of total bio- in contributing to the implementation of
medical research publications and 31% of these goals.
clinical trials in Africa are related to
diseases that account for approximately Health R&D in Africa: Variable
50% of the African disease burden Capacity and Lack of
(Figure 1).
Intra-African Collaboration
One of the underlying emphases of the
GSPOA was that a sustainable global We mapped the African health research
solution requires greater efforts to build landscape by building a database of all
and utilize innovation capabilities in peer-reviewed research articles in biomed-
developing countries, enhance their access ical fields that had at least one African-
to information and technology, and forge based author, during the 5-year period
collaborative networks. Implementing 2004 to 2008. The methodology for data
these goals and a sustainable pan-African collection and analysis is presented in
health product R&D endeavor requires Figure 2 (additional information is also
provided in Text S1 and Table S3). The
capacity, infrastructure, leadership, financ-
ing, and an understanding of the status of affiliation of authors in a total of 31,279
health R&D in the African continent. In a articles identified were processed to deter-
related development, the African Union mine the lead and collaborating institu-
have committed to raising spending on tions in each article. For every institution,
scientific research and innovation to 1% of the number of individual collaborations
GDP in recognition that such funding is a was quantified and mapped.
necessary prerequisite for sustainable de- We identified about 2,700 institutions in
velopment [10]. We therefore assessed the Africa as lead institutions based on the fact
African biomedical research landscape that they were the corresponding institu-
including the level of intra-African exper- tions for articles cited in the peer-reviewed
tise and collaboration to support the article database. These are present in 47 of
development of the African Network for the 53 African countries (excluding Cape
Citation: Nwaka S, Ilunga TB, Da Silva JS, Rial Verde E, Hackley D, et al. (2010) Developing ANDI: A Novel
Approach to Health Product R&D in Africa. PLoS Med 7(6): e1000293. doi:10.1371/journal.pmed.1000293
Published June 29, 2010
Copyright: � 2010 Nwaka et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: The work is supported by the Special Programme for Research and Training in Tropical Diseases,
World Health Organization. The funder played no role in the decision to submit the article or in its preparation.
Competing Interests: JDS, ERV, DH, and RDV are paid consultants by WHO/TDR in the development of the
strategic business plan for ANDI. TBI is a member of Pfizer’s Emerging Markets Customer Advisory Board.
Abbreviations: ANDI, African Network for Drugs and Diagnostics Innovation; AMANET, African Malaria
Network Trust; AAVP, African AIDS Vaccine Programme; DALY, disability-adjusted life year; EDCTP, European
and Developing Countries Clinical Trials Partnership; GDP, gross domestic product; GSPOA, Global Strategy and
Plan of Action on Public Health, Innovation and Intellectual Property; R&D, research and development; TDR,
UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases; UNESCO,
United Nations Educational, Scientific and Cultural Organization; WHO, World Health Organization; AFRO, WHO
African Regional Office; EMRO, WHO Eastern Mediterranean Regional Office
* E-mail: nwakas@who.int
Provenance: Not commissioned; externally peer reviewed.
1 June 2010 | Volume 7 | Issue 6 | e1000293 31
World Health Report 2012: No Health Without Research

Summary Points R&D capacity is uneven in Africa. These

findings are consistent with, but go
N A novel approach to supporting health product research and development beyond, an earlier report on product
R&D landscape in Africa [11,12].
(R&D) and access in Africa is urgently needed. Successful implementation of
such an endeavor requires sustainable capacity, infrastructure, funding, Among the top 20 most productive
leadership, and an understanding of the status of health R&D in the African institutions, i.e., those with the highest
continent. number of articles published, we found
N As part of the development of the African Network for Drugs and Diagnostics that only three African countries are
represented (South Africa, Egypt, and
Innovation (ANDI), we analyzed biomedical research output and collaborative
research undertaken across the continent by evaluating peer-reviewed articles Nigeria). This analysis show that portions
published between 2004 and 2008, as well as other innovation indicators, such of Western and Central Africa are signif-
as R&D investments and manufacturing capacity. icantly lagging behind (Figure 3). This
N Significant health R&D capacity exists in different parts of Africa, but this trend is further confirmed by patent
productivity, which is concentrated in a
capacity is fragmented, uncoordinated, and not properly utilized to address
African health problems. Most biomedical collaborations of African institutions few countries in Africa (data not shown).
are with institutions in Europe and the United States rather than with other These patterns underscore the value of
African institutions. This lack of intracontinental collaboration, combined with increasing collaboration across African
low levels of investment, contributes to gaps in the continental research countries to both increase and leverage
agenda, a lack of local ownership of research undertaken on the continent, and the available expertise to enhance R&D
suboptimal utilization of available research capability. capacity.
N We discuss the establishment of ANDI as a new approach to address these We next assessed the nature of collab-
challenges, through the creation, coordination, and funding of African health orative activity by African institutions.
R&D networks focused on the discovery, development, and delivery of tools to While 77% of articles in our database
address Africa’s unique health needs. are authored in collaboration, only 5.4%
involve institutions in more than one
African country, and fewer than 1%
Verde, Equatorial Guinea, Liberia, São clinical trial activity in Africa also high- involve more than two African countries.
Tomé and Principe, Somalia, and Bur- lights the existence of significant capacity, The vast majority of collaborations are
undi). This significant number indicates as well as pharmaceutical manufacturing with external partners in Europe and the
that quality R&D capacity exists in the capacity with over 120 companies identi- United States. Even the most collaborative
continent. Figure 3 illustrates centres of fied and some patent activity (data not African institutions have little collabora-
expertise with more than 30 biomedical shown). However, the data also highlight tion with African countries other than
publications. Concurrently, mapping of the challenging reality that distribution of their own (Table S1). This further con-

Figure 1. Diseases disproportionately affecting Africa are under-prioritized. Only 21% of all articles with at least one African author focus
on the conditions causing highest disease burden in Africa. This reduced research focus mirrors the low number of products being tested in clinical
trials (only ,1/3 of trials). African DALYs were obtained from the Global Burden of Disease project of WHO. Peer-reviewed articles with at least one
African author published in the biomedical fields during the 5-year period between 2004 and 2008 were identified by querying the Thomson Web of
Science database for African countries in the affiliation field (Figure 2 and Text S1). Absolute numbers of clinical trials were identified using the
ClinicalTrials.gov database (as of September 2009). All trials for drugs and biological products with at least one center in Africa were considered and
all 53 focus countries were scanned. Only trials ‘‘currently open’’ or ‘‘recently completed’’ (i.e., concluded within 48 months of scan date) were
counted. A total of 1,627 trials were identified, out of which 511 trials focus on drugs or biologicals for diseases identified as causing the highest
disease burden in Africa (see left column of charts).
doi:10.1371/journal.pmed.1000293.g001

PLoS Medicine | www.plosmedicine.org 2 June 2010 | Volume 7 | Issue 6 | e1000293

32 iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Figure 2. Flow chart: Creation of Bibliographic databases.

doi:10.1371/journal.pmed.1000293.g002

Figure 3. Distribution of R&D capacity in Africa. Mapping of the top African cities by research output shows hotspots of R&D activity, but also
highlights inequities in R&D productivity across the continent. To create this mapping we used articles with Africa-based authors in the
corresponding address. A total of 20,119 articles were identified (Figure 2 and Text S1), but for simplicity only African cities with over 30 articles from
2004 to 2008 are marked. This represents 16,647 articles (circle diameter indicates number of published articles; a total of 91 cities in 28 countries are
identified but only the top 40 are labeled).
doi:10.1371/journal.pmed.1000293.g003

PLoS Medicine | www.plosmedicine.org 3 June 2010 | Volume 7 | Issue 6 | e1000293

iMedPub :: Thematic collections :: Volume 2 33
World Health Report 2012: No Health Without Research

firms the low degree of intra-African R&D
collaboration. The focus of collaboration
with the United States and European
countries is illustrated by the R&D
network mapping for two critical diseases
disproportionately affecting Africa: HIV/
AIDS and malaria (Figures 4 and S3).
These figures identify the top 20 city nodes
of collaboration using measures of ‘‘de-
gree’’ and ‘‘betweenness’’ for each disease
and mapping their main collaborative
partners (see Text S1). These measures
are commonly used as measures of net-
work centrality [13–17]. Other diseases
disproportionately affecting Africa show a
similar pattern (Table S2).
A closer look at the HIV/AIDS and
malaria network maps reveals some subtle
differences in relation to these two diseas-
es: (a) there are significantly more research
articles for HIV/AIDS than for malaria,
but the percentage of African collabora-
tion is approximately twice as high for
malaria (13%) as for HIV/AIDS (7%)
(Table S2); (b) malaria collaborations are
more widely spread across Africa than
they are for HIV/AIDS; (c) HIV/AIDS
research more strongly partners with the
US, while malaria research more strongly
partners with Europe. A clear explanation
for these differences will require more
work. However, we believe that the higher
collaboration seen for malaria is a reflec-
tion of a longer history of local malaria
research in Africa.
Implications of Data for the
Development of African-Led
Health R&D Innovation
While the extra-African collaboration
should be encouraged, the lack of intra-
African collaboration suggests that African
institutions do not have adequate leader-
ship and ownership of the research being
done in the continent. The sustainability
of research undertaken in Africa may also
be an issue, especially when it is under-
taken with short-term funds coming from,
and directed from, external sources. The
poor intra-African collaboration was con-
firmed by interviews in Africa. Over 170
stakeholders were interviewed across Afri-
can countries/regions including ministries
of health, science and technology, and
higher education; national academies of
science; pharmaceutical companies and
research centers; and networks and uni-
versities in South Africa, Nigeria, Egypt,
Kenya, and Burkina Faso [6]. These
interviews identified key factors believed
to hamper collaboration, ownership, and
leadership of research in Africa. These
include the lack of knowledge about R&D
done in other African countries, the
deficient networking infrastructure, and
the absence of financial incentives to spur
cooperative research within the continent.
Increasing R&D activities for under-
researched diseases and promoting collab-
orative networks within Africa will require
robust African-based funding mechanisms
to complement current funding that is
coming mostly from outside Africa. Based
on data from the UNESCO Science
Report 2005, Africa spent 0.3% of GDP
on R&D in 2002, in contrast to a global
value of 1.7%. Increased local funding and
intra-African coordination to complement
external support and coordination are
essential to spur much-needed health
R&D and empower Africa in driving its
own R&D agenda. Overdependence on
external sources alone will continue to
leave a substantial portion of Africa health
needs unaddressed, and will not resolve
the leadership and ownership gap. The
implementation of ANDI is hoped to help
in addressing these challenges.
The African Network for Drugs
and Diagnostics Innovation
The global momentum to increase
participation and leadership of low-in-
come countries in their own R&D pro-

Figure 4. Collaboration bias towards the US and Europe for the HIV/AIDS R&D network. Network mapping of most collaborative
institutions leading articles identified as directly related to HIV/AIDS and published from 2004 to 2008. There are eight top centers in Africa, mostly in
the Southern region. Few articles are published in collaboration between these African centers but there is active collaboration with institutions in
the Northern Hemisphere, mainly in the US. Yellow circles indicate institutions in Africa and red circles mark institutions outside of Africa. Circle
diameter indicates the count of HIV/AIDS articles identified in the analyzed period. Only the top most collaborative institutions in the network and
the links to and from Africa among them are shown. Institutions outside Africa that do not show connections (e.g., Institut Pasteur) are linked to other
institutions outside Africa or to institutions not identified as the most collaborative.
doi:10.1371/journal.pmed.1000293.g004

PLoS Medicine | www.plosmedicine.org 4 June 2010 | Volume 7 | Issue 6 | e1000293

34 iMedPub :: Thematic Collections :: Volume 2

grams received a major push through
World Health Assembly resolutions on
the Global Strategy and Plan of Action on
Public Health, Innovation and Intellectual
Property [8,9]. This guiding framework
calls for improvement in health R&D
innovation through greater engagement
of developing countries, investment in
local capacity and capability building
including support for regional R&D col-
laborative networks, and devising mecha-
nisms to ensure financial sustainability of
local R&D efforts. This is consistent with
other high-level pan-African declarations,
for example the Algiers declaration of
2008 [18]. This is a propitious moment to
address the need for medicines in Africa,
with a pragmatic and sustainable model
that: (a) promotes the assembly of African
R&D networks that can better use the
technology and human capital already
present on the Continent, (b) sustainably
funds R&D projects aligned with African
health priorities and led by African R&D
centers, (c) ensures African ownership of
the R&D agenda, and (d) supports broader
south–south and south–north collabora-
tion and technology transfer.
A strategic business plan to guide the
implementation and financing of drugs
and diagnostics research—ANDI—has
been developed by a series of partners
such as WHO through TDR, AFRO
(WHO African Regional Office), and
EMRO (WHO Eastern Mediterranean
Regional Office), the African Develop-
ment Bank, the European Union, and
several national African institutions [6].
The plan calls for the establishment of an
African innovation fund to support ANDI
activities. Discussions are ongoing to
formally establish an African-led gover-
nance structure for ANDI under the
auspices of an African institution in
2010, and to operationally launch the
initiative with a set of well-defined projects
in 2011.
The business plan’s development in-
volved multiple consultative discussions,
analyses, and over 170 stakeholder inter-
views. The plan calls for a US$600 million
endowment fund in Africa that can
complement other, more classical, dona-
tions to generate a sustainable income of
up to US$30 million annually to support
African health product innovation. ANDI
aims to partner, fund, and coordinate
research by creating a portfolio of collab-
orative project networks and partnerships
as well as building capacity and support
for infrastructural development. It will also
advocate for more investment for research
to be done throughout Africa and support
local intellectual property management
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
capability for enhanced access to medi-
cines. Recent discussions with various
stakeholders have also emphasized the
need for ANDI to remain open to include
research on a broad range of products,
including drugs, diagnostics, vaccines, and
medical devices. They have also stressed
the need for downstream research to
strengthen and support health systems.
ANDI will complement external R&D
efforts, such as those promoted by product
development partnerships that focus on
one or a few diseases. These include,
Medicines for Malaria Venture (www.
mmv.org), Drugs for Neglected Diseases
Initiative (www.dndi.org), Foundation
for Innovative New Diagnostics (www.
finddiagnostics.org) and Malaria Vaccine
Initiative (www.malariavaccine.org/) [19].
It will also complement and partner other
continental initiatives such as EDCTP
(European and Developing Countries
Clinical Trials Partnership, http://www.
edctp.org/), AMANET (African Malaria
Network Trust, http://www.amanet-trust.
org/), and AAVP (African AIDS Vaccine
Programme, http://www.who.int/vaccine_
research/diseases/hiv/aavp/en/) [20],
which focus primarily on clinical research.
It will also complement capacity-building
efforts such as those of the Wellcome
Trust focusing on academic biomedical
research capacity in specific countries
such as Kenya and Malawi. ANDI covers
the entire innovation value chain from
discovery to manufacture and links both
health and innovation sectors to economic
development.
Funds have been secured, including
from the European Union, to support the
initial establishment of ANDI. However,
further resources are being sought to
operationalise it in Africa. Next steps for
the establishment of ANDI include: (a) the
formal establishment of a governance
structure; (b) the selection of host sites for
the regional and sub-regional offices; (c)
the selection and establishment of an
initial set of projects and technological
support platforms; and (d) recruitment of
staff. It is anticipated that these milestones
will be achieved by 2011. Progress towards
these activities will be reported at the 3rd
ANDI stakeholders’ meetings scheduled
for Nairobi in October 2010.
The establishment of ANDI as a func-
tional and successful organization located
in Africa—managed and governed by
African institutions, implementing product
R&D, and ensuring sustainable access to
new drugs and diagnostics innovations—
will help fill the gaps identified in this work,
including deficient investment in product
R&D within Africa; a lack of collaboration
5 June 2010 | Volume 7 | Issue 6 | e1000293
among African scientists and between the
African public and private sector; and poor
awareness of the link between research and
economic development.
Supporting Information
Figure S1 Life expectancy in Africa.
Life expectancy in Africa is in general
lower than in other areas of the developing
world (e.g., Asia). This is independent of
economic development, as measured by
GDP per capita. GDP per capita is shown
in a logarithmic scale, and correspond to
purchasing power parity (PPP) in 2007. All
measurements obtained from the United
Nations [5].
Found at: doi:10.1371/journal.pmed.
1000293.s001 (0.16 MB TIF)
Figure S2 Disease burden caused by
diseases disproportionally affecting Africa.
Diseases that show high relative impact in
Africa, i.e., cause over 30% of total global
disease burden as measured by DALYs.
DALY is a time-based measure combining
years of life lost to premature mortality
and years of life lost to time lived in states
of less than full health. These are respon-
sible for 54% of all African DALYs and
approximately half of all deaths.
Found at: doi:10.1371/journal.pmed.
1000293.s002 (0.41 MB TIF)
Figure S3 Collaboration bias towards
the US and Europe for the malaria R&D
network. Network mapping of most col-
laborative institutions leading articles iden-
tified as directly related to malaria and
published from 2004 to 2008. There are
nine top centers in Africa spread across the
continent, except in the northern region.
Few articles are published in collaboration
between African centers, but these are
active in collaborating with institutions in
the Northern Hemisphere, mainly in
Europe. Yellow circles indicate institutions
in Africa; red circles mark institutions
outside of Africa. Circle diameter indicates
the count of malaria articles identified in
the analyzed period. Only the top most-
collaborative institutions in the network
and the links to and from Africa among
them are shown.
Found at: doi:10.1371/journal.pmed.
1000293.s003 (0.55 MB TIF)
Table S1 Most-collaborative African
institutions.
Found at: doi:10.1371/journal.pmed.
1000293.s004 (0.41 MB TIF)
Table S2 African R&D output in eight
key disease areas in Africa.
Found at: doi:10.1371/journal.pmed.
1000293.s005 (0.18 MB TIF)
35
World Health Report 2012: No Health Without Research
Table S3 Subject areas excluded to
restrict the dataset to biomedical publica-
tions.
Found at: doi:10.1371/journal.pmed.
1000293.s006 (0.72 MB TIF)
Text S1 Methodology.
Found at: doi:10.1371/journal.pmed.
1000293.s007 (0.03 MB DOC)
References
1. Commission for Macroeconomics and Health
(2001) Investing in Health for Economic
Development.
2. World Health Organization (2004) World Health
Report 2004. Geneva: WHO.
3. International Aids Vaccine Initiative, Spurring
Innovation in AIDS Vaccine R&D: What will it
take? (2009)
4. Chirac P, Torreele E (2006) Global framework on
essential health R&D. Lancet 367: 1560–1561.
5. United Nations (2010) UNdata. Available:
http://data.un.org. Accessed 3 January 2010.
6. The Strategic and business plan for the African
Network for Drugs and Diagnostics Innovation
(ANDI) October 2009. Available: http://meeting.
tropika.net/andi2009/files/2009/10/sbp-final_
web.pdf. Accessed 3 January 2010.
7. Resolution of the 2nd ANDI stakeholders meeting,
Cape Town October 2009. Available: http://
meeting.tropika.net/andi2009/files/2009/10/
andi-endorsement.pdf. Accessed 3 January 2010.
8. World Health Assembly (2008) Global strategy
and plan of action on public health, innovation
and intellectual property. WHA 61.21. Available:
PLoS Medicine | www.plosmedicine.org
36
Acknowledgments
We would like to thank the ANDI Task Force—
Alex Ochem, Sanaa Botros, Uford Inyang,
Peter Atadja, Kevin McCarthy, and Albrecht
Jahn—for their support and input, as well as all
the stakeholders interviewed. We would also like
to thank Anna Tarasova, Lili Duan, Denis
Konouck, Bernadette Ramirez, Dominique
Besson, and Foluke Fakorede for technical
support.
http://apps.who.int/gb/ebwha/pdf_files/A61/
A61_R21-en.pdf. Accessed 3 January 2010.
9. World Health Assembly (2009) Global strategy
and plan of action on public health, innovation
and intellectual property. WHA 62.16. Available:
http://apps.who.int/gb/ebwha/pdf_files/A62/
A62_R16-en.pdf. Accessed 3 January 2010.
10. Khan MJ (2008) Africa’s plan of action for science
and technology and indicators: South African
experience. African Stat J 6: 163–176.
11. Mboya Okeyo T, Ridley R, Nwaka S (2009) The
African Network for Drugs and Diagnostics
Innovation. Lancet 371: 1507–1508.
12. African Network for Drugs and Diagnostics Inno-
vation (ANDI) (2008) Part I: Health product R&D
landscape in Africa, and Part 2: Collection of
meeting, abstract. Founding meeting in Abuja,
Nigeria. Available: http://www.who.int/tdr/news-
events/news/pdf/ANDI-rd-landscape-abstracts.
pdf. Accessed 3 January 2010.
13. Shaw ME (1954) Group structure and the
behavior of individuals in small groups. J Psychol
38: 139–149.
6 June 2010 | Volume 7 | Issue 6 | e1000293
Author Contributions
ICMJE criteria for authorship read and met: SN
TBI JSdS ERRV DH RDV TMO RGR. Agree
with the manuscript’s results and conclusions:
SN TBI JSdS ERRV DH RDV TMO RGR.
Designed the experiments/the study: SN TBI.
Analyzed the data: SN TBI JSdS ERRV DH
RDV. Collected data/did experiments for the
study: SN JSdS ERRV DH RDV. Wrote the
first draft of the paper: SN. Contributed to the
writing of the paper: TBI JSdS ERRV DH
RDV TMO RGR.
14. Nieminen J (1974) On centrality in a graph.
Scand J Psychol 15: 322–336.
15. Anthonisse JM (1971) The rush in a directed
graph. Technical Report BN 9/71. Amsterdam:
Stichting Mathematisch Centrum.
16. Freeman LC (1977) A set of measures of centrality
based upon betweenness. Sociometry 40: 35–41.
17. Analytic Technologies (1999) Ucinet 5.0 version
1.00 [computer program].
18. The Algiers Declaration (2008) Ministerial Con-
ference on Research for Health in the African
Region.
19. Tucker T, Makgoba M (2008) Public-private
partnerships and scientific imperialism. Science
320: 1016–1017.
20. Kaleebu P, Abimiku A, El-Halabi S, Koulla-
Shiro S, Mamotte N, et al. (2008) African AIDS
Vaccine Programme for a coordinated and
collaborative vaccine development effort on the
continent. PLoS Med 5: e236. doi:10.1371/
journal.pmed.0050236.
iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Health in Action

Laboratory Capacity Building in Asia for Infectious

Disease Research: Experiences from the South East Asia
Infectious Disease Clinical Research Network (SEAICRN)
Heiman F. L. Wertheim1,2,3*, Pilaipan Puthavathana4, Ngoc My Nghiem1,5, H. Rogier van Doorn1,3,5,
Trung Vu Nguyen6, Hung Viet Pham7, Decy Subekti1,8, Syahrial Harun9, Suhud Malik9, Janet Robinson10,
Motiur Rahman10, Walter Taylor1,3,11, Niklas Lindegardh1,3,11, Steve Wignall1,8, Jeremy J. Farrar1,3,5,
Menno D. de Jong1,5,12
1 South East Asia Infectious Diseases Clinical Research Network, 2 Oxford University Clinical Research Unit, National Hospital of Tropical Diseases, Hanoi, Vietnam, 3 Centre
for Tropical Medicine, Nuffield Department of Clinical Medicine, Oxford University, Oxford, United Kingdom, 4 Siriraj Hospital, Mahidol University, Department of
Microbiology, Bangkok, Thailand, 5 Oxford University Clinical Research Unit, Hospital of Tropical Diseases, Ho Chi Minh City, Vietnam, 6 National Hospital of Tropical
Diseases, Hanoi, Vietnam, 7 National Hospital of Pediatrics, Hanoi, Vietnam, 8 South East Asia Infectious Disease Clinical Research Network (SEAICRN) Coordinating Center,
Eijkman Institute, Jakarta, Indonesia, 9 National Institute of Health Research and Development, Jakarta, Indonesia, 10 Family Health International, Bangkok, Thailand,
11 Mahidol-Oxford Research Unit, Mahidol University, Bangkok, Thailand, 12 Academic Medical Center, Amsterdam, The Netherlands

Introduction
The South East Asia Infectious Disease
Clinical Research Network (SEAICRN) is a
collaborative partnership of hospitals and
institutions in Thailand, Vietnam, Indone-
sia, and Singapore. The network strives to
advance scientific knowledge and clinical
and laboratory management of infectious
disease in general, and influenza in partic-
ular, through integrated, collaborative clin-
ical research in the South East Asia region
(see http://www.seaicrn.org/) [1]. The
establishment of this network helps fulfill
the declaration of the World Health
Assembly in 2005 that urged its member
states to ‘‘strengthen national laboratory
capacity for human and zoonotic influen-
za,’’ and ‘‘to support an international
research agenda to reduce the spread and
impact of pandemic influenza viruses’’
[2,3]. In addition it also supports the
recently revised International Health Reg-
ulations (IHR 2005), which came into effect
in 2007, by increasing surveillance and
response capacity in strategic hospitals,
national institutes, and referral laboratories
and by promoting international collabora-
tion and sharing of data [3].
SEAICRN’s first intervention study was
a multicenter randomized clinical trial in
four Asian countries that compared the
efficacy of high dose (150 mg bid) versus
standard dose (75 mg bid) oseltamivir for
severe influenza (Table 1). Subsequently
The Health in Action section is a forum for
individuals or organizations to highlight their
innovative approaches to a particular health
problem.
the SEAICRN initiative developed a series tory capacity building program and dis-
of research protocols related to influenza cuss how this benefitted the institutions,
and other infectious diseases being imple- the research studies, and the region. The
mented in participating institutions. To US National Institutes of Health’s (NIH)
make these studies possible, and to comply National Institute for Allergy and Infec-
with international laboratory and clinical tious Disease (NIAID) and Wellcome
trial standards, the SEAICRN recently Trust funding support for this program
chose to enhance the capacity and quality has been described previously [1].
of both the research and clinical laborato-
ries at all participating hospitals and
Creating Laboratory Capacity
institutions in the region.
The key objective of the laboratory Given that SEAICRN laboratories had
strengthening program was to enhance different levels of organization and exper-
laboratory facilities; ensure availability of tise, it was considered important to create
necessary equipment; build human re- reference laboratories for different aspects
source capacity by teaching, training, of the research activities in each country
and mentoring; and ensure quality labo- (Figure 1). Due to the initial focus on
ratory management and testing to comply influenza therapeutics, a SEAICRN clinical
with good clinical laboratory practice pharmacology laboratory was established
(GCLP) and other international standards at the Department of Tropical Medicine,
such as the ISO 15189 [4,5]. Here we Mahidol University, Bangkok, Thailand.
present the experiences of the SEAICRN To enroll patients into the first clinical
in setting up and conducting the labora- trial, participating sites needed to be able
Citation: Wertheim HFL, Puthavathana P, Nghiem NM, van Doorn HR, Nguyen TV, et al. (2010) Laboratory
Capacity Building in Asia for Infectious Disease Research: Experiences from the South East Asia Infectious
Disease Clinical Research Network (SEAICRN). PLoS Med 7(4): e1000231. doi:10.1371/journal.pmed.1000231
Published April 6, 2010
Copyright: � 2010 Wertheim et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: This work was undertaken as part of the the South East Asia Infectious Disease Clinical Research
Network (www.seaicrn.org). The SEAICRN is supported by the National Institute Allergy and Infectious Diseases
(USA), the Wellcome Trust (UK), and the national authorities of Indonesia, Thailand, and Viet Nam. The funders
had no role in the decision to submit the article or in its preparation.
Competing Interests: The authors have declared that no competing interests exist.
Abbreviations: GCLP, good clinical laboratory practice; MDL, molecular diagnostic laboratory; pH1N1,
influenza A (H1N1); RT-PCR, reverse transcriptase polymerase chain reaction; SEAICRN, South East Asia
Infectious Disease Clinical Research Network; SOP, standard operating procedure
* E-mail: heiman.wertheim@gmail.com
Provenance: Not commissioned; externally peer-reviewed.

iMedPub PLoS Medicinecollections

:: Thematic | www.plosmedicine.org
:: Volume 2 1 April 2010 | Volume 7 | Issue 4 | e1000231 37
World Health Report 2012: No Health Without Research

Summary Points diagnostic laboratory (MDL), with all the

required infrastructure, equipment, re-
N Enhancing laboratory capacity is essential for generating reliable and accurate agents, and consumables. Physical separa-
tion of spaces for reagent preparation,
data from clinical research, especially in resource-constrained settings.
N Local well-trained laboratory experts and scientists are important to research,
nucleic acid extraction, and amplification
was ensured in all laboratories to prevent
and must participate actively in scientific activities and continuing education
programs. PCR contamination. Also, a new biosafety
level 3 laboratory facility was established
N Improving laboratory capacity is more than supplying new equipment and
at the Hospital for Tropical Diseases, Ho
reagents; it also includes a long-term commitment to staff training, quality
Chi Minh City, Viet Nam for isolation of
control, and biosafety.
H5N1 viruses and emerging (pandemic)
N Improved laboratory capacity optimizes responses to an epidemic or an influenza viruses. Additionally, in Viet
outbreak of a novel virulent pathogens, and can support international agendas
Nam, (pyro)sequencing capacity was es-
to reduce the impact of pandemic influenza viruses.
tablished to characterize viruses and detect
mutations in the influenza virus genome
to screen patients by influenza reverse ly. To be able to screen patients by RT- that confer antiviral drug resistance.
transcriptase polymerase chain reaction PCR, several laboratories were renovated Site staff from Bangkok, Jakarta, Ban-
(RT-PCR) reliably, rapidly, and accurate- to house, for the first time, a molecular dung, Hanoi, and Ho Chi Minh City were

Table 1. List of approved protocols that will be or are being executed by SEAICRN.

Study Clinical Trial

Number Registry Number Title Status

SEA001 NCT00298233 High-dose versus standard-dose oseltamivir for the treatment of severe influenza Study finished.
and avian influenza. A phase ii double-blind, randomized clinical trial Awaiting analysis
SEA002 NCT00439530 Pharmacology Study of oseltamivir and probenecid in healthy Asian subjects Published
SEA003 NCT00921726 Phase 1, open-label study to evaluate potential pharmacokinetic interactions between Study finished.
orally-administered oseltamivir and intravenous zanamivir in healthy Thai adult subjects Awaiting analysis
SEA004 NCT00980109 Long term influenza prophylaxis with inhaled zanamivir or oral oseltamivir Ongoing
SEA005 Pending An open label study to evaluate the safety, tolerability and efficacy of intravenous Will start 2010
zanamivir administered twice daily for 5 days in hospitalized subjects with
confirmed H5N1 influenza infection.
SEA006 — Clinical database of patients with H5N1 infection Ongoing
SEA007 — Host genetic susceptibility to human influenza A/H5N1 Ongoing
SEA008 — Cardiorespiratory function and radiological changes in lung structure in Indonesian Ongoing
patients who survive highly pathogenic influenza A/H5N1—A 3 year prospective study
SEA013 Pending Open-label study to evaluate potential pharmacokinetic interactions between orally Tentatively scheduled
administered oseltamivir and intravenous peramivir in healthy Thai adult subjects after January 2010
SEA019 — Non-Influenza etiologies of acute respiratory illness in Southeast Asia Ongoing
SEA022 Pending Oseltamivir treatment in severe, virologically confirmed influenza A or B in hospitalized Expected to start
children less than one year of age in Hanoi, Ho Chi Minh City and Bangkok in 2010
SEA023 — The viral etiology of hospitalized children less than one year of age with an acute, Expected to start
community acquired, lower respiratory tract infection in Hanoi and Bangkok in 2010
SEA024 Pending Efficacy of primaquine against relapse by Plasmodium vivax in Indonesia Expected to start
in 2010
SEA025 — A survey of acute febrile illnesses in adults and children at the Cipto Mangunkusumo Expected to start
hospital, Jakarta, Indonesia in 2010
SEA026 — Characterization of the human carboxylesterase 1 (CES 1) mutation(s) which may be Ongoing
responsible for markedly reduced conversation of oseltamivir phosphate to oseltamivir
carboxylate, and determination of frequency in SE Asian population
SEA027 — Acute post-infectious measles encephalitis Finished. Awaiting
analysis
SEA028 — Hand, foot, and mouth disease Protocol in development
SEA029 Pending A randomized, open-label, comparative trial of the efficacy of itraconazole versus Will start beginning
Amphotericin B in the treatment of penicilliosis in HIV-infected persons of 2010
SEA030 Pending An open label study to evaluate the safety, tolerability and efficacy of intravenous Pending
zanamivir administered twice daily for 5 days in hospitalized subjects with confirmed
seasonal influenza infection
SEA032 NCT00985582 Oseltamivir treatment in adults and children with mild or moderate influenza Ongoing
H1N12009 infection—a clinical, virological and pharmacokinetic study

Study SEA002 is finished and already published [13].

doi:10.1371/journal.pmed.1000231.t001

PLoS Medicine | www.plosmedicine.org 2 April 2010 | Volume 7 | Issue 4 | e1000231

38 iMedPub :: Thematic Collections :: Volume 2

Figure 1. South East Asia Infectious Disease Clinical Research Network sites and
laboratories.
doi:10.1371/journal.pmed.1000231.g001
trained in influenza real-time RT-PCR
diagnostics and molecular diagnostics in
general, including principles of unidirec-
tional workflow and prevention of carry-
over contamination. These trainings were
performed on an ongoing basis at the
reference laboratory in Ho Chi Minh City,
followed by on-site training. To ensure
quality, each MDL laboratory was en-
rolled in at least two different external
quality assurance programs for molecular
influenza diagnostics. Furthermore, profi-
ciency testing for diagnosing influenza by
RT-PCR was performed before sites were
allowed to start screening patients for the
studies.
Each hospital is encouraged to use the
MDL for purposes other than SEAICRN-
related research activities, such as HIV,
hepatitis, dengue, meningitis, and enceph-
alitis testing by molecular techniques. The
philosophy is that it is important and
highly beneficial to increase expertise in
detecting infectious agents by molecular
diagnostics and so stimulate the imple-
mentation of other relevant molecular
diagnostic tests. For example, after imple-
mentation of a Streptococcus suis–specific
molecular test in the MDL of the National
Hospital of Tropical Diseases, Vietnam, it
was found that S. suis is a common cause of
bacterial meningitis in adults in Hanoi [6].
In preparation for the start of the
clinical trials, dedicated laboratory trial
staff members were appointed to become
staff of the national institutions. They
received training on processing, labeling,
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
testing, storing, shipping, and document-
ing data from trial specimens according to
protocol requirements. A centralized spec-
imen labeling and database system was set
up to track and trace all stored specimens
for the SEAICRN studies. Currently the
SEAICRN uses Freezerworks software
(Dataworks Development; http://www.
freezerworks.com/) to track study speci-
mens and aliquots. Sites received deep
freezers for adequate specimen storage
and power back-up systems were installed
where needed to ensure an uninterrupted
power supply. Table 2 summarizes the
required laboratory capacity and what was
done to achieve this.
The laboratory capacity program of the
SEAICRN enabled us to rapidly respond to
changes in influenza epidemiology, such as
the spread of naturally occurring oseltamivir
resistance in seasonal H1N1 viruses and the
recent emergence of the novel influenza A
(H1N1) strain (pH1N1) in April 2009. In
response to the increasing rates of oseltami-
vir resistance in seasonal H1N1 viruses,
before the emergence of pH1N1, the
laboratory committee decided to test Asian
isolates and respiratory specimens from
influenza patients in our studies [7,8]. This
step generated valuable resistance data from
the region, which were instantly shared with
the national and international medical and
scientific community. In April 2009, when
confronted with a pandemic threat of this
novel influenza virus, we prepared collabo-
rating laboratories by providing them with
sequence information of this strain and
3 April 2010 | Volume 7 | Issue 4 | e1000231
primers and probes to enable them to detect
it by RT-PCR. Furthermore, we were able
to describe the first worldwide transmission
of oseltamivir resistant pH1N1 in a com-
munity cluster [9].
Laboratory Quality
Enhancement Program
In partnership with Family Health
International (FHI), the SEAICRN initi-
ated a clinical laboratory quality improve-
ment program in the region. This includ-
ed: (i) baseline assessment of each hospital
clinical laboratory against international
standards, ( ii) inventory, maintenance,
and calibration of instruments, (iii) enroll-
ment in external quality assurance pro-
gram, (iv) assessment of training needs, (v)
review of normal reference values used in
the laboratory, and (vi) accreditation status
of the laboratories by local and interna-
tional accreditation bodies. The baseline
assessments evaluated the overall quality
of the clinical laboratory and helped to
develop short- and long-term recommen-
dations for improvement. Based on the
assessment and recommendations, subse-
quent follow-up visits and technical sup-
port were provided to each hospital to
implement those recommendations.
Laboratory management staff in each
hospital were trained and encouraged to
implement a quality management system.
This included development and imple-
mentation of quality and technical manu-
als, standard operating procedures (SOPs),
and a document control system. To help
to improve laboratory quality it was
necessary to appoint a senior staff member
as a Quality Officer to oversee all aspects
of laboratory quality and the quality
enhancement program. Furthermore,
each laboratory was supported with nec-
essary instruments, and staff were encour-
aged to establish and monitor an equip-
ment maintenance and calibration
program. All staff from each laboratory
were trained on standards developed by
the technical committee of the Interna-
tional Organisation for Standardisation
(ISO15189:2003 and ISO15190:2003,
http://www.iso.org/) [10].
During the clinical trials it was noticed
that occasionally reported biochemistry
results were implausible. In response, a
continuous improvement system was im-
plemented to ensure that laboratory test
results are reviewed by a qualified person
to identify performance issues. In addition,
advice was given on proper use of internal
controls, how to monitor and investigate
the possible cause(s) of controls not
meeting acceptance criteria, and correc-
39
World Health Report 2012: No Health Without Research

Table 2. Activities by SEAICRN to achieve required laboratory capacity for their research projects.

Context Capacity/Expertise Required Activity

Infrastructure Uninterrupted power supply UPS systems installed

Sufficient freezer capacity for study specimen storage
MDL
Quality and safety External quality assurance (EQA) of laboratory tests
Equipment maintenance
Temperature logs
Laboratory safety
Laboratory test normal ranges
Laboratory result documentation
Laboratory accreditation
Specimen/aliquot tracking and tracing
Shipment of infectious substances
Specific laboratory Influenza molecular diagnostic test
testing
Neuraminidase inhibitor resistance testing
Oseltamivir plasma concentrations
Laboratory tests for adverse events
Blood culture
Other Listen for local needs and challenges and provide
technical assistance when able to do so.
Provided freezers
Create MDL where needed and train staff.
Provided EQA (Thai program)
Ensured maintenance
Set up temperature logging system
Provide safety cabinets and biosafety training
Provided guidance in establishing normal ranges
Improved where needed
Laboratory quality enhancement program implemented
Implemented a central labeling system and database
Training in shipping biological substances
Set up CDC assay for influenza real-time RT-PCR
Set up neuraminidase inhibition assay and molecular
based test (sequencing)
Set up pharmacokinetic laboratory
In case unavailable, set up referral system.
Provided blood culture system
Provided technical assistance and support as part of
the program

doi:10.1371/journal.pmed.1000231.t002

tive and preventive action measures when
performance issues are identified. Labora-
tories were encouraged to complement the
capacity of each other by establishing a
specimen testing referral linkage among
them.
The overall activities of the SEAICRN
created opportunities for laboratory staff
to exchange experiences with senior labo-
ratory personnel. In addition, a laboratory
networking system and a platform for
sharing experiences among laboratories
was established through the SEAICRN
annual meetings and regular teleconfer-
ences. Also, as part of the development of
new clinical protocols there has been a
strong interaction between laboratory
experts, clinicians, and trial support staff.
Having such systems in place have proved
crucial to the ability of these clinical
laboratories to respond quickly to the
spread of oseltamivir-resistant influenza
A/H1N1 and pH1N1. With the infra-
structure in place it proved possible to
respond immediately to the pH1N1 pan-
demic. It was also possible to implement a
clinical research protocol in Viet Nam and
start to make the data available in less than
a month from the appearance of the first
cases.
The ultimate goal of the program was to
improve the quality of laboratory services,
achieve compliance with GCLP standards,
gain ISO 15189 accreditation, and facili-
tate better clinical care and collaborative
research. Setting such goals makes the
process more real and motivates laborato-
ry staff to participate. Several laboratories
in Thailand had already started working
toward accreditation independently of the
SEAICRN, and these efforts helped facil-
itate the SEAICRN program at those sites.
Two laboratories in Thailand received
‘‘The Association of Medical Technolo-
gists of Thailand’’ accreditation, and one
laboratory in Thailand and two in Viet-
nam have gained ISO15189 accreditation.
In addition, another laboratory in Thai-
land and one in Indonesia are nearly ready
for accreditation inspection.
Discussion
Although GCLP is the standard in
developed countries, it can pose a dispro-
portionate burden and significant chal-
lenge in resource-constrained settings
(see Box 1). However, it was clear that
laboratory staff were motivated to im-
prove the quality of their testing and
services when working toward a concrete
endpoint, such as ISO15189 accredita-
tion. A GCLP standard for resource-
constrained settings that is simpler and
less demanding on resources is urgently
needed [5,11].
It is important that laboratory enhance-
ment programs focus not only on the
particular disease of research interest, so that
the investments can also benefit other diseases
and routine patient care [12]. Setting up the
SEAICRN has served as a catalyst for
laboratories to implement molecular diag-
nostic techniques as part of routine care, not
only for influenza but for a range of other
pathogens beyond direct research needs. The
SEAICRN has provided tools, training
(including short courses, Masters, and PhD
programs), and protocols to facilitate these
processes. Currently, six scientists from Asia
are enlisted in a PhD program, nine in a
Masters program, three in a Bachelor’s
program, and 295 in a short-term fellowship.
A total of 2,146 individuals have participated
in operational training activities organized or
made possible by the SEAICRN.
The sharing of data has been a
contentious issue over the last few years.
The SEAICRN has a policy that all the
protocols, case record forms, SOPs, and
manuals of operations for all studies and
for all processes are made available via the
internet at the start of all studies (http://
www.seaicrn.org/). During the pH1N1
pandemic, a number of non-Asian coun-
tries used these protocols, adapted them to
their own local circumstances, including
translation, and implemented them.
SEAICRN is also committed to sharing

PLoS Medicine | www.plosmedicine.org 4 April 2010 | Volume 7 | Issue 4 | e1000231

40 iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Box 1. Key Challenges was able to respond immediately in

assessing oseltamivir resistance in seasonal
1. Improving laboratory capacity requires investment in both time and money. It is influenza A/H1N1 in 2008, providing
essential that long term goals are set, as opposed to short term fixes, and that vital information to the scientific and
the finances are made available to enable and sustain continuous quality medical community. We believe this
improvements. program has improved the clinical and
research laboratory capacity and quality in
2. Laboratories will be at different stages of implementing an international-level
the region. Besides enhancing research in
quality system. A capacity building program should be tailored to each
Asia, the investments have also helped
laboratory taking cultural, technical, and financial differences into account.
improve health care for all patients served
Working towards accreditation is a concrete and achievable goal and helps to
at the participating hospitals. Our pro-
motivate staff.
gram addresses the issues raised by the
3. International accreditation (e.g. ISO15189:2003) may not always be desirable World Health Assembly in 2005 and in
and sustainable in resource-constrained settings as the ongoing costs may be the revised International Health Regula-
too high. It is worthwhile to explore alternate accreditation models in such tions, as it supports the region to be more
settings, which can be used as a stepwise approach to full international
prepared to deal with potential pandemic
accreditation.
influenza viruses and other endemic and
4. Improving laboratory quality systems requires financial commitment and emerging infectious diseases, as we have
support from the hospital director and a time commitment from laboratory recently experienced, and promotes inter-
leadership and staff. All need to be convinced that improvement is essential for national collaboration and sharing of data.
delivering good quality health care. Research activities should not be solely
responsible for funding the running costs of quality systems as this would not
Acknowledgments
be sustainable with the systems potentially ending when the research funding
stops. We would like to thank all the members of the
5. Not all countries have accreditation authorities that can accredit laboratories as Reference Laboratory committee and the
Clinical Laboratory Committee for their com-
qualified people are not available. The capacity for national or regional
mitment to the SEAICRN. We thank the
accreditation should be improved in the near future. clinicians and nurses who send us numerous
specimens daily and for their positive interac-
tions. We would like to thank Le Viet Thanh for
all data with all institutions and national across the region are committed to making the SEAICRN map and Caroline
and international authorities, in line with continuing the network, and funding has Fukuda for her continuous support. We also
the IHR 2005. However, it is also crucial been secured to ensure its sustainability thank Dean Sherwood for helping us to set up
for scientists and clinicians within coun- beyond that date. There is a clear and maintain the SEAICRN database system
tries and within regions to know each commitment to extend the network be- for tracking study specimens in the region.
other, to have built sustainable relation- yond the current centers in the four
ships and networks in a manner that countries in Asia. Other centers have Author Contributions
encourages and facilitates the exchange expressed an interest in joining or sup- ICMJE criteria for authorship read and met:
of experience and information within the porting such an expansion. The long-term HFLW PP HRvD NMN TVN HPV DS SH
regional clinical and scientific community. aim is to build a sustainable clinical SM JR MR WT NL SW JJF MDdJ. Wrote the
Clearly, the success and sustainability of research network with its center of gravity first draft of the paper: HFLW PP SH.
a research network depends on funding. firmly based in Asia. Contributed to the writing of the paper: HFLW
The current support for the SEAICRN Having a wider perspective has proven HRvD NMN TVN HPV DS SM JR MR WT
ends in September 2010, but investigators invaluable. Furthermore, the SEAICRN NL SW JJF MDdJ.

References
1. Higgs ES, Hayden FG, Chotpitayasunondh T,
Whitworth J, Farrar J (2008) The Southeast Asian
Influenza Clinical Research Network: develop-
ment and challenges for a new multilateral
research endeavor. Antiviral Res 78: 64–68.
2. (2005) World Health Assembly. Strengthening
pandemic-influenza preparedness and response:
WHA58.5. May 2005. Available: http://
wwwwhoint/gb/ebwha/pdf_files/WHA58/
WHA58_5-enpdf Accessed 21 April 2009.
3. PLoS Medicine Editors (2007) How is WHO
responding to global public health threats? PLoS
Med 4: e197. doi:10.1371/journal.pmed.0040197.
4. Ezzelle J, Rodriguez-Chavez IR, Darden JM,
Stirewalt M, Kunwar N, et al. (2008) Guidelines
on good clinical laboratory practice: bridging
operations between research and clinical research
laboratories. J Pharm Biomed Anal 46: 18–29.
5. Stevens W (2003) Good clinical laboratory
practice (GCLP): the need for a hybrid of good
laboratory practice and good clinical practice
guidelines/standards for medical testing labora-
tories conducting clinical trials in developing
countries. Qual Assur 10: 83–89.
6. Wertheim HF, Nguyen HN, Taylor W, Lien TT,
Ngo HT, et al. (2009) Streptococcus suis, an
important cause of adult bacterial meningitis in
northern Vietnam. PLoS One 4: e5973.
doi:10.1371/journal.pone.0005973.
7. Sheu TG, Deyde VM, Okomo-Adhiambo M,
Garten RJ, Xu X, et al. (2008) Surveillance for
neuraminidase inhibitor resistance among human
influenza A and B viruses circulating worldwide
from 2004 to 2008. Antimicrob Agents Che-
mother 52: 3284–3292.
8. Besselaar TG, Naidoo D, Buys A, Gregory V,
McAnerney J, et al. (2008) Widespread oseltami-
vir resistance in influenza A viruses (H1N1),
South Africa. Emerg Infect Dis 14: 1809–1810.
9. Le QM, Wertheim HF, Tran ND, van
Doorn HR, Nguyen TH, Horby P (2010) A
community cluster of oseltamivir-resistant cases
of 2009 H1N1 influenza. N Engl J Med 362:
86–87.
10. Burnett D, Blair C (2001) Standards for the
medical laboratory–harmonization and subsidiar-
ity. Clin Chim Acta 309: 137–145.
11. Kuepfer I, Burri C (2009) Reflections on clinical
research in sub-Saharan Africa. Int J Parasitol 39:
947–954.
12. Kruk ME (2008) Emergency preparedness and
public health systems lessons for developing
countries. Am J Prev Med 34: 529–534.
13. Wattanagoon Y, Stepniewska K, Lindegardh N,
Pukrittayakamee S, Silachamroon U, et al. (2009)
Pharmacokinetics of high-dose oseltamivir in
healthy volunteers. Antimicrob Agents Che-
mother 53: 945–952.

PLoS Medicine | www.plosmedicine.org 5 April 2010 | Volume 7 | Issue 4 | e1000231

iMedPub :: Thematic collections :: Volume 2 41
World Health Report 2012: No Health Without Research
Policy Forum
Registering New Drugs for Low-Income Countries: The
African Challenge
Mary Moran1*, Nathalie Strub-Wourgaft2, Javier Guzman1, Pascale Boulet2, Lindsey Wu1, Bernard
Pecoul2
1 Policy Cures, Sydney, Australia, 2 Drugs for Neglected Diseases initiative (DNDi), Geneva, Switzerland
Introduction
What is the best strategy to approve
novel drugs for disease such as sleeping
sickness that predominantly affect patients
in Africa? How can African regulators best
be supported to evaluate these drugs for
their own populations? For many years,
African medicines regulatory authorities
(MRAs) have relied on stringent regulators
in developed countries to assess novel
pharmaceutical products such as drugs
and vaccines for use in African popula-
tions. However, a recent shift in the drug
product environment for Africa has put
this approach under strain. A score of new
products are now being, or have been,
developed specifically for diseases of the
developing world (Table 1), creating new
challenges for regulators in Africa and
elsewhere.
However, it is not at all certain that
African regulatory authorities currently
have the capacity to meet these new
demands. A study conducted by the World
Health Organization (WHO) in 2010
concluded that 90% of MRAs in sub-
Saharan Africa ‘‘were in a situation which
did not allow them to adequately carry out
regulatory functions,’’ and thus could not
guarantee the safety and efficacy of
medicines to be used in their country [1–
3]. While undoubtedly improving, growth
in African regulatory capacity is not
keeping up with these new challenges.
The growing demand to assess novel
neglected disease (ND) products for African
use has generated a range of responses from
policymakers and product developers, as
outlined below. While each approach offers
unique benefits, none is ideally suited as a
primary vehicle for drug registration for
Africa. There is also no guidance to product
developers in choosing between approach-
The Policy Forum allows health policy makers
around the world to discuss challenges and
opportunities for improving health care in their
societies.
42 PLoS Medicine | www.plosmedicine.org
es, and little or no integration between opment Partnerships (PDPs) have typically
approval mechanisms (see Figure 1). It is used this approach because it offers clear
now critical to review how novel ND drugs protocols and rules, liability management
are assessed and approved for African use. and, in the case of ODL, tax breaks, free
This article is based on research conducted scientific advice, and market exclusivities.
for a report titled ‘‘Registering New Drugs: Firms also welcome the access Western
The African Context’’ [4], commissioned regulatory approval provides to early
by the Drugs for Neglected Diseases commercial returns on products with
initiative (DNDi), and builds upon this overlapping rich and poor markets.
work with additional research and analysis. While bringing decades of regulatory
experience to the table, use of Western
authorities to review ND drugs also has
Western Regulatory Approval
drawbacks. It delays access for African
Routes
patients since African MRAs often wait for
Historically, the majority of new ND the Western MRA decision before com-
drugs have been first submitted to well- mencing action, and it puts ND product
established Western regulatory authorities decisions in the hands of regulators who
(e.g., United States Food and Drug have less experience in tropical disease
Administration [FDA], European Medi- products, presentations, and epidemiolo-
cines Agency [EMA], SwissMedic), either gy, and who are not accountable for the
for routine regulatory review or under needs and safety of target African patients.
specific pathways such as Orphan Drug For instance, Western regulations may
legislation (ODL) or expedited approval omit data requirements vital for safe large-
mechanisms. Multinational pharmaceuti- scale use in Africa (e.g., trials assessing the
cal companies and some Product Devel- safe interaction of HIV and malaria
Citation: Moran M, Strub-Wourgaft N, Guzman J, Boulet P, Wu L, et al. (2011) Registering New Drugs for Low-
Income Countries: The African Challenge. PLoS Med 8(2): e1000411. doi:10.1371/journal.pmed.1000411
Published February 1, 2011
Copyright: � 2011 Moran et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: MM, JG, and LW have support from the Drugs for Neglected Disease initiative (DNDi) for this work.
MM, JG, and LW have received funding from the World Bank (via Global Forum for Health Research), the Bill &
Melinda Gates Foundation, the Council on Health Research for Development (COHRED), the Wellcome Trust,
the UK Department of Health, the World Health Organization, and the International AIDS Vaccine Initiative. The
funders helped in the organization of the report on which this article is based, in the preparation and writing of
the manuscript and decision to publish, and had input into the study design; however, they played no role in
data collection or analysis.
Competing Interests: MM, JG, and LW have relationships with the World Bank (via Global Forum for Health
Research), the Bill & Melinda Gates Foundation, the Council on Health Research for Development (COHRED),
the Wellcome Trust, the UK Department of Health, the World Health Organization, and the International AIDS
Vaccine Initiative, all of which may have an interest in the submitted work. No authors’ spouse, partner, or
children has financial relationships that may be relevant to the submitted work. DNDi develops new drugs or
new formulations of existing drugs for patients suffering from HAT (sleeping sickness), visceral leishmaniasis
(VL), and Chagas disease.
Abbreviations: ASAQ, artesunate-amodiaquine; ASMQ, artesunate-mefloquine; AZCQ, azithromycin-chloro-
quine; DNDi, Drugs for Neglected Diseases initiative; EMA, European Medicines Agency; FDA, United States
Food and Drug Administration; MRAs, medicines regulatory authorities; ND, neglected disease; ODL, Orphan
Drug legislation; PDP, product development partnership; PEPFAR, President’s Emergency Plan for AIDS Relief;
TB, tuberculosis; WHO, World Health Organization.
* E-mail: mmoran@policycures.org
Provenance: Not commissioned; externally peer reviewed.
1 February 2011 ::| Volume
iMedPub 8 Collections
Thematic | Issue 2 | e1000411
:: Volume 2
 World Health Report 2012: No Health Without Research

Summary Points affect millions of patients in the developing

world.
N A recent shift in the drug product environment for Africa has seen a score of
Neglected Disease–Specific
new products being developed specifically for diseases of the developing
world, creating new challenges for regulators in Africa and elsewhere. However, Regulatory Pathways
it is not at all certain that African regulatory authorities currently have the
capacity to meet these new demands. Policymakers have responded to these
N The growing demand to assess novel neglected disease (ND) products for
shortcomings by developing regulatory
pathways tailored for ND products, in-
African use has generated a range of responses from policymakers and product
developers, but there is limited guidance for product developers in choosing cluding the EMA’s Article 58, WHO drug
between approaches, and little or no integration between approval mecha- prequalification, and FDA ‘‘tentative ap-
nisms. proval’’.
N We discuss the various mechanisms in which novel ND drugs are assessed and
Article 58
approved for developing country use, and put forth six recommendations to
create an efficient integrated system of national, regional, and international Article 58, established by the European
approvals to achieve an optimal drug registration approach for Africa that can Commission (EU) in 2004, aims to
reliably evaluate safety, efficacy, and quality of drugs for African use. facilitate and assist developing country
registration of medicines by providing the
same scientific assessment (‘‘opinion’’) on
drugs). Rifapentine, a novel tuberculosis US market in October 1999 due to a one products used outside the EU as for the
(TB) drug registered under US Orphan in 10,000 risk of intussusception in chil- EU, but incorporates WHO in the review
Drug provisions, ultimately could not be dren. This precluded its subsequent intro- process. Article 58’s strength lies in its
used in African TB patients despite being duction in the developing world. While combination of stringent review stan-
approved by the FDA because the trial this risk-benefit analysis may have been dards, efficiency (average review time is
design excluded HIV-positive patients. valid for the US, where rotavirus causes 2.5 months), and structured input from
While HIV is less commonly associated less than 60 deaths per year, the vaccine WHO disease experts from disease-en-
with TB in the US, it represents up to 70% was likely to have a much more favorable demic countries. However, it has fallen
of TB patients in some sub-Saharan Africa risk-benefit ratio in Africa, where rotavirus victim to underutilization (only four
countries, making the efficacy data sub- is responsible for approximately 5% of product applications have been submitted
mitted to the FDA inadequate for use in deaths in children under the age of five (a since 2004), largely due to a lack of
African populations [5]. Furthermore, the mortality rate of 183/100,000). Many of incentives for product developers to use
relative risk-benefit of ND drugs can be these problems are heightened in the case this route. In particular, Article 58 does
dramatically different in Africa and the of regulatory pathways such as Orphan not offer tax breaks or market exclusivi-
West, where analysis against the same Drug approval and FDA Accelerated ties; does not result in European market-
criteria can lead to completely different Review, which allow clinical trials to be ing approval; is not linked to Orphan
conclusions. For example, the first rotavi- abridged or downsized in order to expe- Drug approval; and does not formally
rus vaccine, RotaShield, developed by dite registration of treatments for diseases expedite approval through WHO drug
Wyeth-Ayerst and licensed by the FDA that are rare and life-threatening in the prequalification, although this may be
in August 1998, was withdrawn from the Western context (such as malaria), but changing.

Table 1. Sample of novel neglected disease products presented to regulators since 2005 [8,11,12,20–23].

Novel Neglected Disease Products Regulatory Stage

Artesunate-amodiaquine ASAQ (malaria) Approved by 24 African countries

WHO prequalified (October 2008)
Artesunate-mefloquine ASMQ (malaria) Approved by Brazilian ANVISA (April 2008)
Coartem Dispersible (malaria) Approved by 14 African countries
Approved by Swissmedic (December 2008)
WHO prequalified (February 2009)
Intramuscular paromomycin (visceral leishmaniasis) Received FDA and EMA orphan drug designation (March 2005)
Approved by Drugs Controller General of India (August 2006)
Eurartesim (malaria) Submitted to EMA for approval (July 2009)
Moxifloxacin (TB) Clinical development plan submitted to developing country and/or Western regulators
PA-824 (TB) Clinical development plan submitted to developing country and/or Western regulators
Arterolane/PQP (malaria) Clinical development plan submitted to developing country and/or Western regulators
Azithromycin-chloroquine AZCQ (malaria) Clinical development plan submitted to developing country and/or Western regulators
Fexinidazole (sleeping sickness) Clinical development plan submitted to developing country and/or Western regulators

Additional source: correspondence with Novartis.

doi:10.1371/journal.pmed.1000411.t001

PLoS Medicine | www.plosmedicine.org 2 February 2011 | Volume 8 | Issue 2 | e1000411

iMedPub :: Thematic collections :: Volume 2 43
World Health Report 2012: No Health Without Research
Figure 1. Neglected disease drug registration timeline [7,9,11–19]. Additional source: correspondence with Novartis.
doi:10.1371/journal.pmed.1000411.g001
FDA PEPFAR-Linked Approvals
Following the launch of the US Presi-
dent’s Emergency Plan for AIDS Relief
(PEPFAR), the FDA introduced expedited
approval in 2004 for HIV drugs purchased
with PEPFAR funds for use outside the
US. Seventy-one of the 100 products fully
or tentatively approved (products still
under patent in the US are given ‘‘tenta-
tive approval’’ until the patent expires) in
association with PEPFAR as of June 2009
were generic formulations of existing
drugs; 22 were new combinations or
regimens of existing drugs not previously
authorised in the US; and seven were
pediatric re-formulations. The approval
process is integrated with WHO prequal-
ification through the exchange of reviews
and the automatic inclusion of FDA-
reviewed drugs in the WHO prequalifica-
tion list: as of February 2010, 41% (113
drugs) of WHO prequalification drugs
were PEPFAR approvals [6,7]. While
helpful and efficient in assessing non-novel
HIV drugs associated with PEPFAR, this
program’s usefulness is limited by its
disease and product restrictions.
WHO Drug Prequalification
In 2001, the WHO began the drug
prequalification program as a ‘‘surrogate’’
regulatory approval mechanism on which
PLoS Medicine | www.plosmedicine.org
44
international procurement groups such as
the Global Fund to Fight AIDS, Tubercu-
losis and Malaria could rely while develop-
ing country capacity for drug regulation
was being strengthened. Evaluations are
conducted by mixed teams of developed
and developing country experts, with
around one-third of reviewers from Africa.
WHO prequalification has been relied
upon by African MRAs as a proxy for their
own drug assessments and approvals.
WHO prequalification focuses on only a
few diseases (in particular, HIV, malaria,
and TB), with the majority of approved
products being generic HIV drugs. As of
June 2009, the program had pre-qualified
280 drugs—86% for HIV (241), 7% for
TB (20), and 6% for malaria (16) (Figure 2).
Just over half (56%) of these were generics,
and 21% were new fixed-dose combina-
tions or formulations of existing drugs. A
further 23% were innovative drugs that
had been approved by a stringent MRA
prior to the WHO prequalification pro-
cess.
WHO prequalification (in tandem with
FDA tentative approval) has vastly accel-
erated African access to HIV, and to a
lesser degree, malaria products; neverthe-
less, it could be further optimized. It
covers only a few of the major diseases of
Africa, and does not include a review of
3 February 2011 | Volume 8 | Issue 2 | e1000411
novel ND products. Due to its voluntary,
no-fee, capacity-building approach, WHO
prequalification can be slow (averaging 2
years) and it would benefit from more
seamless integration with product reviews
by stringent MRAs.
Alternative Approval Strategies
In response to the drawbacks of both
standard and ND-specific regulatory re-
view, product developers have begun
exploring alternatives, some of which offer
insights for drug registration in Africa.
Parallel approvals have been a common
strategy for many PDPs, with dossiers
submitted simultaneously to Western and
developing country MRAs. The aim is to
achieve high regulatory standards while
expediting African registration. In prac-
tice, however, time gains are often illusory,
as most African MRAs wait on WHO or
Western approval before commencing
their own process. Parallel approval also
fails to assist or build the regulatory
capacity of African MRAs.
Another potential strategy is twinned
review, under which developing country
regulators assess a pharmaceutical dossier
in consultation with, or alongside, revie-
wers from stringent regulatory agencies.
Twinned reviews can offer a potentially
iMedPub :: Thematic Collections :: Volume 2

Figure 2. WHO prequalified drugs by disease [7].
doi:10.1371/journal.pmed.1000411.g002
superior outcome by combining Western
experience in product assessment with
developing country expertise on endemic
diseases, while expediting African regula-
tory approval and leaving risk-benefit
analysis and decisions to MRAs responsible
for areas where products will be used. More
importantly, twinned review can build
African MRA capacity through first-hand
training for developing country regulators
by Western regulatory experts. Neverthe-
less, there has not yet been a formal
twinned regulatory review of any new ND
product, although in 2008 the WHO
organized a joint ‘‘practice’’ review of the
artesunate-amodiaquine (ASAQ) dossier
developed by the DNDi and involving
regulators from African MRAs and the
EMA. The implementation of twinned
reviews will require resources and commit-
ment by both Western and developing
country regulators to move forward, but
early stage joint reviews, such as those
facilitated by the WHO with The Gambia,
Mali, Ghana, and Senegal for the clinical
trial application of the PATH Meningitis
Vaccine Program’s conjugate vaccine, are
certainly a step in the right direction.
Product developers can also seek first
approval from developing country MRAs
without seeking prior, parallel, or twinned
approval by WHO prequalification or a
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
stringent regulatory agency. Used primar-
ily by PDPs or developing country man-
ufacturers, this option offers rapid access
for domestic populations. For example,
artesunate-mefloquine (ASMQ), devel-
oped by DNDi and Brazil’s Farminguin-
hos/Fiocruz, was first registered in Brazil
in April 2008 [8], and is currently under
assessment by the WHO prequalification
program. ASAQ, jointly developed by
DNDi and Sanofi-Aventis, was first regis-
tered by the Moroccan regulatory author-
ity in February 2007 and then received
WHO prequalification in October 2008
[9], and the Institute for One World
Health first registered intramuscular par-
omomycin for the treatment of visceral
leishmaniasis in India in August 2006 [5].
Discussion
An optimal drug registration approach
for Africa should reliably evaluate safety,
efficacy, and quality of drugs for African use.
It should include African expertise, contrib-
ute to building African regulatory capacity,
and, ultimately, expedite African access by
reducing duplicative and sequential reviews
by different regulators. However, as the
above overview shows, the current system of
ND drug approval is still far from achieving
these goals. It is often inefficient, uses
4 February 2011 | Volume 8 | Issue 2 | e1000411
regulatory resources wastefully, and creates
lengthy delays for patient access. Capacity-
building opportunities for African regulators
are routinely lost and, in the worst case,
regulatory processes and decisions may not
meet Africa’s needs for the best, safest, and
most appropriate drugs.
The following proposals are aimed at
rapidly moving the current regulatory
paradigm to the optimal scenario:
1. Institute formal twinned regulatory
review; that is, any review of a novel
ND product by a stringent MRA (or
WHO prequalification) should formal-
ly include regulators from relevant
endemic countries.
2. Automatic WHO prequalification of all
novel ND products approved by strin-
gent MRAs using standard regulatory
pathways, and which meet WHO
treatment recommendations. (With
the exception of approvals under the
Accelerated approval (FDA)/Condi-
tional approval (EMA) mechanisms.
Approvals under ODL should be
reviewed on a case-by-case basis.)
3. Itegrate Article 58 with other approval
mechanisms by allowing automatic
WHO drug prequalification for prod-
ucts given a positive opinion under
Article 58; AND allow positive Article
45
World Health Report 2012: No Health Without Research
58 opinions to provide European
market access either by conversion to
EMA approval with a single European
bridging study; OR link to automatic
EU Orphan approval, which would
additionally provide eligibility for tax
breaks and market exclusivities.
4. Select experienced Western MRAs to
conduct prequalifications on behalf of,
and in addition to, the WHO.
5. Conduct a strategic review of WHO
drug prequalification disease and prod-
uct priorities, along the lines of WHO
Strategic Advisory Group of Experts
(SAGE) reviews for vaccines (estab-
lished by the Director-General of the
World Health Organization in 1999 to
provide guidance on the work of the
WHO Immunization, Vaccines and
Biologicals Department), to identify
additional priority diseases or products
to be addressed by WHO prequalifica-
tion (and/or outsourced to reference
MRAs for prequalification).
6. Fund Centres of Regulatory Excellence
in each of Africa’s main regions that
would conduct:
# Joint review of product dossiers for
the region (with external support as
necessary).
References
1. Belgharbi L (2007) Vaccine regulatory issues in
African countries: building & sustaining national
capacity. EDCTP consultative meeting; 11 June
2007; Geneva, Switzerland. Available: http://
www.edctp.org/fileadmin/documents/Regulatory
_meeting_Lahouari_Belgharbi.pdf. Accessed 4
January 2011.
2. World Health Organization (2006) Comite Region-
al de l’Afrique. Cinquante-sixieme session, Addis-
Abeba, Ethiopie, 2 aout-1er septembre 2006.
Autorites de reglementation pharmaceutique: situ-
ation actuelle et perspectives: Report du Directeur
regional. Available: http://afrolib.afro.who.int/
RC/RC%2056/Doc_Fr/AFR%20RC56%2011%
20AUTORITES%20%20REGLEMENT%20
PHARMACEUTIQUE.pdf. Accessed 23 Sep-
tember 2010.
3. World Health Organization (2010) Regulatory
harmonization: updating medicines regulatory
systems in sub-Saharan African countries. WHO
Drug Information 24(1): 6–20. Available: http://
whqlibdoc.who.int/druginfo/24_1_2010.pdf. Ac-
cessed 4 January 2011.
4. Moran M, Guzman J, McDonald A, Wu L,
Omune B (2010) Registering new drugs: the
African context. London: Health Policy Division,
The George Institute for International Health.
Available: http://www.dndi.org/images/stories/
advocacy/regulatory-report_george-institute-
dndi_jan2010.pdf. Accessed 4 January 2011.
5. US Food and Drug Administration (2010) Orange
book: approved drug products with therapeutic
equivalence evaluations. Available: http://www.
accessdata.fda.gov/scripts/cder/ob/docs/obdetail.
cfm?Appl_No = 021752&TABLE1 = OB_Rx. Ac-
cessed 4 January 2011.
6. US Food and Drug Administration (2010) Presi-
dent’s Emergency Plan for AIDS Relief: Approved
PLoS Medicine | www.plosmedicine.org
46
# Joint good manufacturing practices
plant inspections for the region.
# Clinical trial regulation, including
joint regional review/approval.
# ‘‘Twinned’’ reviews i.e., formal partic-
ipation in external regulatory reviews
such as FDA reviews, Article 58
assessments, or WHO prequalification.
# Training and regulatory fellowships,
including attachments to stringent
external regulators and time with
their national regulatory authority.
Collectively, these measures would
improve the quality of ND drug reviews
for the targeted populations; create an
efficient integrated system of national,
regional, and international approvals;
expand the scope of regulatory support
for Africa to include many more diseases
and products; provide an institutional
pathway to train and retain African
regulators; and build African capacity to
manage its own regulatory tasks. To move
these ideas forward, it will be up to key
policymakers in Africa and donor coun-
tries, funders of ND research and devel-
opment, innovators, and, more impor-
tantly, regulatory agencies to reach a
consensus on how these can be best
implemented to ultimately benefit pa-
and tentatively approves antiretrovirals in associa-
tion with the President’s Emergency Plan.
Available: http://www.fda.gov/internationalpro-
grams/fdabeyondourbordersforeignoffices/asiaan-
dafrica/ucm119231.htm. Accessed 4 January 2011.
7. Cipla (2007) Once-daily dosage lamivudine anoth-
er ARV first for Cipla Medpro. Available: http://
www.ciplamedpro.co.za/news.php?nid = 26. Ac-
cessed 4 January 2011.
8. Act with ASMQ (2008) A worldwide public
partnership makes available a new, once-a-day
fixed-dose combination against malaria. Available:
http://www.actwithasmq.org/index2.php?inter= 0&
&hight = 0. Accessed 4 January 2011.
9. Act with ASAQ (2010) ASAQ: hope for malaria.
Available: http://www.actwithasaq.org/en/
asaq1.htm. Accessed 4 January 2011.
10. Nwaka S, Ilunga TB, Da Silva JS, Rial Verde E,
Hackley D, et al. (2010) Developing ANDI: a novel
approach to health product R&D in Africa. PLoS
Med 7(6): e1000293. doi:10.1371/journal.pmed.
1000293. Available: http://www.plosmedicine.
org/article/info%3Adoi%2F10.1371%2Fjournal.
pmed.1000293. Accessed 4 January 2011.
11. Institute for One World Health (2010) Next steps.
Available: http://www.oneworldhealth.org/
next_steps. Accessed 4 January 2011.
12. Novartis (2008) Novartis receives FDA priority
review for CoartemH, potentially the first artemi-
sinin-based combination treatment (ACT) for
malaria in the US [press release]. Basel: Novartis.
Available: http://hugin.info/134323/R/1251164/
271951.pdf. Accessed 6 January 2011.
13. World Health Organization (2010) HA210 World
Health Organization public assessment report. Avail-
able: http://apps.who.int/prequal/WHOPAR/
WHOPARPRODUCTS/TriomunePart7v10.pdf.
Accessed 4 January 2011.
5 February 2011 | Volume 8 | Issue 2 | e1000411
tients. The WHO, as a credible and
trusted multilateral agency, can potential-
ly play a large role in leading these efforts,
as seen in recent pan-African initiatives
such as the African Network for Drugs &
Diagnostics Innovation [10].
In the face of scarce regulatory re-
sources and large gaps in capacity, these
proposals could address the immediate
need for efficient, appropriate regulatory
approval of new ND products, while
building a sustained and independent
African regulatory infrastructure in a
way that truly addresses African needs
and realities.
Author Contributions
ICMJE criteria for authorship read and met:
MM NSW JG PB LW BP. Agree with the
manuscript’s results and conclusions: MM
NSW JG PB LW BP. Designed the experi-
ments/the study: MM JG BP. Analyzed the
data: MM JG LW. Collected data/did exper-
iments for the study: MM JG LW. Wrote the
first draft of the paper: MM JG LW. Contrib-
uted to the writing of the paper: MM NSW JG
PB LW BP.
14. US Food and Drug Administration (2009) FDA
Approves Coartem tablets to treat malaria. Available:
http://www.fda.gov/NewsEvents/Newsroom/
PressAnnouncements/2009/ucm149559.htm. Ac-
cessed 4 January 2011.
15. World Health Organization (2010) WHO list of
prequalified medicinal products. Available:
http://apps.who.int/prequal/. Accessed 4 Janu-
ary 2011.
16. Institute for One World Health (2009) Visceral
leishmaniasis. Available: http://www.oneworld
health.org/leishmaniasis. Accessed 4 January
2011.
17. World Health Organization (2008) HA343 World
Health Organization public assessment report. Avail-
able: http://apps.who.int/prequal/WHOPAR/
WHOPARPRODUCTS/HA343Part7v2.pdf. Ac-
cessed 4 January 2011.
18. European Medicines Agency (2005) Background
information on the procedure. Available: http://
www.ema.europa.eu/docs/en_GB/document_
library/EPAR_-_Procedural_steps_taken_before
_authorisation/human/000594/WC500043717.
pdf. Accessed 16 March 2010.
19. Gilead (2010) Gilead international access opera-
tions. Available: http://www.gilead.com/pdf/
GAP_Registration_Status.pdf. Accessed 4 Janu-
ary 2011.
20. Medicines for Malaria Venture (2010) MMV
Interactive Science Portfolio, Q4 2010. Geneva:
Medicines for Malaria Venture. Available:
http://www.mmv.org/research-development/
science-portfolio. Accessed 6 January 2011.
21. Global Alliance for TB Drug Development (2011)
TB Alliance portfolio March 2010. New York:
Global Alliance for TB Drug Development.
http://www.tballiance.org/new/portfolio/html-
portfolio.php. Accessed 6 January 2011.
iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

22. Wells T (2009) Building a robust portfolio of new 258406/building_a_robust_portfolio_of_new_ org/portfolio/fexinidazole.html. Accessed 6
medicines. Drug Discovery World. Available: medicines.html. Accessed 6 January 2011. January 2011.
http://www.ddw-online.com/therapeutics/ 23. Drugs for Neglected Disease Initiative (2010)
Fexinidazole (HAT). Available: http://www.dndi.

PLoS Medicine | www.plosmedicine.org 6 February 2011 | Volume 8 | Issue 2 | e1000411

iMedPub :: Thematic collections :: Volume 2 47
World Health Report 2012: No Health Without Research

Policy Forum

Defining Research to Improve Health Systems

Jan H. F. Remme1, Taghreed Adam2, Francisco Becerra-Posada3, Catherine D’Arcangues4, Michael
Devlin5, Charles Gardner6, Abdul Ghaffar2, Joachim Hombach7, Jane F. K. Kengeya1, Anthony Mbewu6,
Michael T. Mbizvo4, Zafar Mirza8, Tikki Pang9, Robert G. Ridley1, Fabio Zicker1, Robert F. Terry9*
1 UNICEF/UNDP/World Bank/World Health Organization Special Programme for Research and Training in Tropical Diseases, Geneva, Switzerland, 2 The Alliance for Health
Policy and Systems Research, Geneva, Switzerland, 3 Council on Health Research for Development, Mexico City, Mexico, 4 World Health Organization, Department of
Reproductive Health and Research, Geneva, Switzerland, 5 Council on Health Research for Development, Geneva, Switzerland, 6 Global Forum for Health Research,
Geneva, Switzerland, 7 World Health Organization, Initiative for Vaccine Research, Geneva, Switzerland, 8 World Health Organization, Department of Public Health,
Innovation and Intellectual Property, Geneva, Switzerland, 9 World Health Organization, Department of Research Policy and Cooperation, Geneva, Switzerland

Introduction
A major obstacle to achieving the health-
related Millennium Development Goals
(MDGs) is the weakness of the health
systems in many low- and middle-income
countries, and their struggle to effectively
provide health care to populations in need
[1,2]. Several global health initiatives have
been created over the last decade to support
the delivery of available interventions for
priority health problems, and in recent years
there have been some major new initiatives
to support health system strengthening
[1,3,4]. These developments have been
accompanied by a growing recognition of
the role of research in improving health
systems and health care delivery. The
ministerial summit on health research that
was held in Mexico in 2004 concluded that
research has a crucial part to play in
strengthening health systems and in improv-
ing the equitable distribution of quality
health services for populations in need, and
the summit called for greater support for
such research [5]. Since then, the number of
research initiatives on health systems in low-
and middle-income countries has increased
substantially [6,7]. This is a positive devel-
opment that we would like to see expanded
and accelerated to build up evidence-based
knowledge to improve the effectiveness of
health systems. Unfortunately, these initia-
tives have also led to growing confusion
about what type of research is involved and
at whom that research is targeted. The fact
that the various research initiatives originate
from different research backgrounds (bio-
medicine, social sciences, organization of
services, health economics, etc.) has led to
an inconsistent use of terminology to
describe the research. Multiple definitions
of operational research, implementation
The Policy Forum allows health policy makers
around the world to discuss challenges and
opportunities for improving health care in their
societies.
research, and health systems research have and consistency for non-specialists, scien-
been proposed in recent years [2,6–18], and tists, policymakers, and donors.
many of these define the scope of their As a starting point, the three research
research very broadly, resulting in consid- areas described here refer to research
erable overlap between definitions. Opera- domains that differ at their core in the
tional research and implementation re- type of research questions they address, in
search are sometimes used interchangeably how they are organized, and in how they
in the literature, or are classified as health interface with the health system. Below,
systems research [10–12,17]. we briefly describe the proposed three
So does this matter? domains, indicate where they differ and
We believe it does, because the resulting overlap, how they complement each other,
confusion may create duplications and and how they could more effectively
inefficiencies both in the funding for interact for greater impact of the overall
different research efforts and among those research effort. We hope that, in this year
seeking to understand and use the evi- with an unprecedented focus on research
dence. It indicates a lack of shared to strengthen health systems with the First
conceptual clarity among scientists and Global Symposium on Health Systems
decision-makers about the scope, nature, Research this paper will contribute to
methodologies, and issues to be addressed greater clarity and more efficient ap-
by the research involved [6]. This makes proaches to fulfil the overall objective of
efforts to retrieve relevant evidence on strengthening health systems to improve
particular topics even more complex than population health [19].
it already is, negatively affecting the
credibility of the research itself. Defining the Research Domains
The aim of this paper is to present
working definitions of operational re- Building on the numerous definitions
search, implementation research, and that exist in the literature under the overall
health systems research in the context of umbrella of research to improve health
research to strengthen health systems, with systems, three domains of research can be
the intention of providing greater clarity defined using their primary characteristics:
Citation: Remme JHF, Adam T, Becerra-Posada F, D’Arcangues C, Devlin M, et al. (2010) Defining Research to
Improve Health Systems. PLoS Med 7(11): e1001000. doi:10.1371/journal.pmed.1001000
Published November 16, 2010
Copyright: � 2010 World Health Organization; licensee Public Library of Science (PLoS). This is an Open
Access article in the spirit of the Public Library of Science (PLoS) principles for Open Access http://www.plos.
org/oa/, without any waiver of WHO’s privileges and immunities under international law, convention, or
agreement. This article should not be reproduced for use in association with the promotion of commercial
products, services or any legal entity. There should be no suggestion that WHO endorses any specific
organisation or products. The use of the WHO logo is not permitted. This notice should be preserved along
with the article’s original URL.
Funding: No direct funding was received for this study. JHFR was employed under contract from TDR (Special
Programme for Research and Training in Tropical Diseases) to write the initial draft. The salary for RFT was
provided under a grant from the Bill & Melinda Gates Foundation (number 49275.01). Contributions from all
other authors were salaried by their institutions during the period of writing.
Competing Interests: The authors have declared that no competing interests exist. The views expressed here
are those of the authors and not of the organization they represent.
Abbreviations: MDG, Millennium Development Goal
* E-mail: terryr@who.int
Provenance: Not commissioned; externally peer reviewed.

48 PLoS Medicine | www.plosmedicine.org 1 November 2010 | ::Volume

iMedPub 7 | Issue
Thematic 11 | e1001000
Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Summary Points The users of the research outputs

(published results, findings, methodologies,
N Research has an important role to play in strengthening health systems to etc.) fall broadly into three groups with
operational research being predominant-
improve system performance and public health impact.
N The multiple definitions of operational research, implementation research, and
ly, but not exclusively, of use to health
care providers; implementation research
health systems research creates confusion and negatively affects the credibility
and progress of the research. predominantly of use to managers of
programmes scaling up an intervention;
N The aim of this paper is to present working definitions of operational research,
and research on the health system as a
implementation research, and health systems research to provide greater clarity
for non-specialists, scientists, policymakers, and donors working to strengthen whole (or one of its building blocks) of
health systems. most use to those who manage or need to
make policy for the health system.
In another characteristic, the impor-
the focus of the research, the users of the The focus of the research within a tance of how amenable the research is to
research outputs, and the utility of the health system [20] is explained more fully adaptation and use in other contexts or
research outputs (see Figure 1 and Table 1). in the following sections. locations—also varies across the three

Figure 1. Research to improve health systems.

doi:10.1371/journal.pmed.1001000.g001

PLoS Medicine | www.plosmedicine.org 2 November 2010 | Volume 7 | Issue 11 | e1001000

iMedPub :: Thematic collections :: Volume 2 49
World Health Report 2012: No Health Without Research

Table 1. Defining research to improve health systems.

Research Domain Primary Characteristic

Focus of the Research Users of the Research Outputs Utility of the Research Outputs*

Operational Operational issues of specific health Health care providers programme Local
programmes managers
Implementation Implementation strategies for specific Programme managers, R&D managers Local/broad
products or services
Health System Issues affecting some or all of the Health system managers, policy makers Broad
building blocks of a health system

*How amenable the research outputs are to adaptation, scaling up or use or in other contexts or locations.
doi:10.1371/journal.pmed.1001000.t001

domains. We define this characteristic as
the utility of the research, where utility
describes the fact, character, or quality of
something being useful or serviceable [21].
While it is well established that all research
to support health systems is context-
specific, careful consideration of study
design and reporting of context-specific
factors generally improve the application
of this research to other settings. This can
be achieved in varying degrees of success
across the three domains. For example,
operational research tends to address a
local problem, taking into account the
particular context in which it occurs.
Therefore, the research utility of the
outputs would not be readily applicable
to other settings without careful and
considerable adaptation.
As it moves through implementation
research and onto health systems research,
the utility of the research tends to be
broader and have increasingly common
points of comparison with other contexts.
Therefore, it is not uncommon to be able to
generate lessons from this type of research
that are applicable to other settings within
the country or even for other countries. So,
using research utility as defined here,
operational research generates research
outputs that are generally local in their
utility, and health systems research has a
primary characteristic of generally being
more amenable to adaptation and applica-
tion in other contexts, i.e., a broad utility.
Again the degree of utility is wholly
dependent on the use of an appropriate
study design or protocol that takes into
consideration of contextual factors.
In subsequent sections we will elaborate
on these definitions using case studies and
examples to illustrate them.
1. Research Domain: Operational
Operational research aims to develop
solutions to current operational problems
of specific health programmes or specific
service delivery components of the health
system, e.g., a health district or a hospital.
Table 2 gives some selected examples of
research questions that illustrate the local
nature of the type of problems that this
research addresses. These are problems
that confront a local disease control
programme, health district, or health
clinic during the execution of its routine
operations, and for which practically
useful answers or solutions are urgently
needed to allow operations to proceed
more effectively. This research is charac-
terized by a strong problem-solving focus
and an urgency to find solutions. Its
demand-driven nature and close associa-
tion with health care delivery and routine
health care operations ensure operational
relevance of the research activities and
rapid uptake and local utilization of
research findings. The operational prob-
lems are often identified through routine

Table 2. Examples of research questions for the three research domains.

Research Domain Research Question* Reference

Operational Can the ‘‘communication for behavioural impact’’ (COMBI) strategy improve the poor compliance with [46,47]
mass drug administration for LF elimination in Tamil Nadu, India?
Which locations should be targeted for delivering HIV prevention services in Kawempe district, Uganda? [9]
Which of the current ART payment strategies in use in Nairobi should be retained for the new integrated [48]
programme?
Should the sleeping sickness programme in Equator Nord province, DRC, change its first-line drug? [49]
Implementation How to deliver ivermectin for onchocerciasis control and ensure sustained high treatment coverage in [50]
isolated rural communities?
How to improve access to vaccination among children who are currently not reached by immunisation services? [25]
How to implement antenatal syphilis screening—one-stop versus conventional service? [51]
How to effectively implement a new intervention package for kala azar elimination in the Indian subcontinent? [52]
Health system To what extent do health services reach the poor? How can this be improved? [32]
Should fees be charged to clients who use health centres for curative services? [17]
How effective are different policies for attracting nurses to rural areas? [53]
What has been the impact of the rapid scale-up of HIV programmes on fragile health systems? [54]

*As noted above, depending on how the question is phrased and the research is designed; some of these questions can be addressed in several research domains.
doi:10.1371/journal.pmed.1001000.t002

PLoS Medicine | www.plosmedicine.org 3 November 2010 | Volume 7 | Issue 11 | e1001000

50 iMedPub :: Thematic Collections :: Volume 2

monitoring and evaluation activities for
which this research, where it exists, can be
an important complement that allows a
health programme to explore new ideas
and experiment with potentially more
effective approaches to its operations.
A wide range of study designs and
research methods are used, ranging from
descriptive and analytical studies to
operational experiments and the use of
mathematical modelling. The research
often starts with exploratory studies to
better define the problem and its deter-
minants, and to identify potential solu-
tions that can subsequently be tested
under operational conditions. The re-
search requires the involvement of, and is
usually executed by, health staff who have
the necessary research experience and/or
by scientists from collaborating local
research institutions. In their definition
of operational research, Zachariah et al.
[8] stress the local, programme-based
focus of research that ‘‘can enhance the
quality, effectiveness or coverage of pro-
grams in which the research is being
done.’’ Similarly, WHO refers to research
‘‘for programme decision making to
achieve a specific outcome’’ [22]; and
the Global Fund refers to research that
‘‘provides decision-makers with informa-
tion to enable them to improve the
performance of their programs’’ [9].
Hence, the results of this research tend
to have a local utility and, because of its
design, are not generally amenable for
adaptation and use in other settings. This
in no way devalues the research, and
useful lessons—such as methodological
approaches—could be applied elsewhere
if reported. However, operational re-
search is still not commonly undertaken,
and many of the lessons that could be
learnt remain unpublished.
Many health programmes and health
system managers do not see operational
research as a priority, and it is sometimes
perceived as a waste of time and resourc-
es, distracting from the need for opera-
tional action on the basis of ‘‘common
sense’’ [23]. Such attitudes tend to soften
with exposure to properly executed oper-
ational research that delivers practical
results, but quality operational research
does not come easily, given the general
lack of research capacity and research
funding at the operational level. Several
global health initiatives offer additional
funding for operational research but most
of these funds are not taken up at country
and programme level because of the lack
of appreciation for this type of research
and insufficient local research capacity
[10].
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
2. Research Domain:
Implementation
Implementation research aims to devel-
op strategies for available or new health
interventions in order to improve access
to, and the use of, these interventions by
the populations in need. Table 2 provides
some examples of this type of research for
which the starting point is the availability
of an intervention or intervention pack-
age that has been proven efficacious in
previous research, but for which major
questions remain as to how to scale up the
intervention and ensure effective integra-
tion within the health system. This re-
search is characterized by a focus on the
need for innovative approaches and/or
ensuring the effectiveness of implemented
interventions. Examples may include mass
treatment with ivermectin for onchocerci-
asis or the introduction of new evidence-
based birth practices for isolated commu-
nities where there are no formal health
services or maternity clinics. This research
often addresses implementation of newly
developed products, such as a pharmaceu-
tical, medical device, or vaccine, where
this research represents the last phase of
the product development pipeline. How-
ever, in this definition implementation
research also covers such interventions as
service delivery approaches, behavioural
interventions, or understanding the impact
of a payment mechanism.
Many promising health interventions
have had only limited impact on the
burden of disease in low- and middle-
income countries because of implementa-
tion problems that were not identified,
researched, and addressed. For example,
research on the impact of insecticide-
treated nets to reduce malaria was stopped
too soon: phase IV effectiveness trials were
not followed up by implementation re-
search, and 15 years later the utilization of
nets is still low in Africa. Hence, it is
critical to include research on implemen-
tation as an extension of the development
phase or R&D pipeline when testing a new
intervention.
The examples of implementation re-
search above tend to be developed as
focused studies with clear research ques-
tions. Multicentre and multicountry stud-
ies are often used, as these help clarify
which findings are location-specific and
which are more generalisable. Large-scale
implementation studies often have two
phases. The first phase consists of descrip-
tive, formative research to better under-
stand the major implementation challeng-
es and to design potential implementation
strategies. This is often followed by a
second phase, in which the most promising
4 November 2010 | Volume 7 | Issue 11 | e1001000
implementation strategies are tested and
compared in large-scale experimental
studies in settings at the appropriate level
of the health system [24,25]. Social science
research methods are extensively used
(including qualitative research methods
for stakeholder analysis and process eval-
uation) as well as methods for determining
the cost of implementation strategies at
different levels of the formal and informal
health system.
Implementation research is usually un-
dertaken by multidisciplinary research
groups drawn from many countries, in-
cluding those where the study is located.
There is often a network of such groups,
supported by international research insti-
tutions and expertise as required. The
necessary local research capacity in be-
havioural sciences, health economics, and
epidemiology is still inadequate in many
low-income countries, and building such
research capacity remains a top priority
[7,14]. Where adequate research capacity
does exist, it is often isolated, and special
efforts may be needed to identify and
involve such groups in research initiatives.
With regard to existing definitions,
Sanders et al. refer to ‘‘research to
promote the uptake and successful imple-
mentation of evidence-based interventions
and policies’’ [12], and Allottey et al. to
‘‘evidence that informs effective, sustained
and embedded adoption of interventions
by health systems and communities’’ [13].
The Special Programme for Research and
Training in Tropical Diseases (TDR)
defines implementation research as ‘‘re-
search to significantly improve access to
efficacious interventions by developing
practical solutions to common implemen-
tation problems’’ [14]. As these definitions
indicate, implementation research is inter-
vention-specific, but in contrast to opera-
tional research, it is often designed with
the intention of creating outputs that can
be applicable beyond the local environ-
ment in which the research is done.
The relevance of implementation re-
search is increasingly being recognized,
and several convincing examples in recent
years have demonstrated the effectiveness
of this type of research in enabling
implementation and scale-up of priority
health interventions [26,27]. However,
compared to the corresponding invest-
ment in R&D, implementation research
is receiving only limited financial sup-
port; it will be important to correct this
imbalance.
3. Research Domain: Health System
Health systems research addresses
health system and policy questions that
51
World Health Report 2012: No Health Without Research
are not disease-specific but concern sys-
tems problems that have repercussions on
the performance of the health system as a
whole. It addresses a wide range of
questions, from health financing, gover-
nance, and policy to problems with
structuring, planning, management, hu-
man resources, service delivery, referral,
and quality of care in the public and
private sector. Table 2 gives a few
examples of research questions that illus-
trate the nature of the research involved
(e.g., studies on the effectiveness of differ-
ent policies for attracting nurses to rural
areas). Health systems issues are often
highly context-specific, and many case
studies try to elucidate a certain health
system challenge within its specific envi-
ronment.
However, with appropriate study design
and planning, health systems research can
not only answer policy questions relevant
to the specific health system in which the
research is undertaken, but can generate
valuable lessons that are more amenable
to adaptation and adoption in other
settings. This is particularly true when a
systems perspective is used, i.e., by con-
sidering all the positive and negative
effects of a particular system-level inter-
vention, this research can provide a robust
and accurate understanding of health
systems challenges and their potential
solutions, thereby improving the utility of
the findings in other settings [28]. This
systems approach, in combination with
stakeholder engagement, also informs the
definition of priority research questions to
address health systems challenges.
Health systems research by necessity is
highly multidisciplinary, with a strong em-
phasis on social sciences, economics, and
anthropological investigations, for example
on community perceptions of health care.
Much ongoing research consists of descrip-
tive, comparative, and evaluation studies and
secondary analytical research. Although
experimental studies are less common, partly
because of operational and ethical challenges
in experimenting at the health system level,
they can be very informative and provide
convincing evidence on the benefit of
innovations in health system efficiency and
health impact [29]. Most research is under-
taken through collaboration between aca-
demic institutions, with a major role being
played by a few institutions with special
expertise in health systems research or in one
of its research disciplines (e.g., health eco-
nomics and policy analysis). Health planners
and decision-makers may contribute to
defining the research questions, but are
otherwise not much involved in undertaking
the research itself [6].
PLoS Medicine | www.plosmedicine.org
52
The research in this domain falls under
the general definition by the Alliance for
Health Policy and Systems Research
(HPSR) as: ‘‘The production of new
knowledge to improve how societies orga-
nize themselves to achieve health goals.’’
The Alliance for HPSR further clarifies
that ‘‘the prime focus of health policy and
systems research is not a specific disease
or service, but rather the health system
as a whole. However, health systems
research sometimes adopts a disease or
service specific focus’’ (http://www.who.
int/alliance-hpsr/en/ [17]). More specifi-
cally, it can address any or all of the six
building blocks of health systems identified
by the WHO [20]: service delivery, infor-
mation and evidence, medical products and
technologies, health workforce, health fi-
nancing, and leadership and governance. In
doing so, it should explicitly acknowledge
the importance of the continuous interac-
tions between the different building blocks
of the health systems and the different
sectors (including non-health sectors) in-
volved, as well as all the other characteristics
of complex health systems [28]. Another
definition offered by Varkevisser et al. [30]
refers to health systems research as ‘‘re-
search that enhances the efficiency and
effectiveness of the health system.’’
Research on health systems addresses a
huge research area that has only been
marginally covered to date [6]. Because of
the multitude of system challenges and
their complex multidimensional environ-
ment, research prioritization is essential
and some recent priority-setting initiatives
are seen as being timely [20,31–33]. Due
to the relative scarcity of research capacity
to undertake this type of research, efforts
to improve the design, robustness, and
applicability of the evidence generated in
one setting to another would be highly
desirable. Systems thinking methods and
approaches can offer tremendous help and
guidance on this [28]. By using a system-
atic, comprehensive way of examining the
design and evaluation of potential health
systems interventions, and ensuring in-
volvement and ownership of all stakehold-
ers involved, the utility and pay back from
the evidence generated from this research
greatly increases.
Related Research Areas and
Research Terms
There are a number of related research
areas that may overlap with the above
research domains but are out of the scope
of this paper. Some of them are briefly
described here. First, monitoring and
evaluation aims to track the progress of a
5 November 2010 | Volume 7 | Issue 11 | e1001000
health intervention and to determine
whether it is having its intended impact.
As a routine operational activity, it is
usually not regarded as research by itself,
although its findings are instrumental for
identifying priority problems for research.
However, the term evaluation can some-
times refer to the more formal evalua-
tion designs such as process, economic,
or impact evaluations, or can be used
separately to answer questions related to
the three research domains described in
this paper [34].
Intervention science—the development
of new and improved health interven-
tions—is another important area of re-
search that may considerably overlap with
implementation research [35,36]. This
science is mainly concerned with biomed-
ical research, where the early stages of
intervention development are often far
removed from the field. However, as the
development phase is nearing its comple-
tion, evaluation of the intervention effec-
tiveness is usually done under real-life
conditions [37]. Hence, towards the end of
the development process, intervention
development and implementation re-
search tend to closely overlap.
Some terms are very similar to those
used to describe the three research do-
mains but have been used in different ways
in the literature. Operations research
usually refers to the use of mathematical
optimization methods for operational de-
cision making, but this is sometimes also
called operational research [34,35,38].
Implementation science has been de-
fined as the study of methods to promote
the systematic uptake of research findings
into routine clinical practice [39], and as
such is complementary to knowledge
translation (another term with its own
diversity of definitions!) [40–42].
Implementation science has also been
said to be similar to translational research
or defined as ‘research that identifies
barriers to proven interventions and that
facilitates the creation of strategies to
overcome them’ and in this sense imple-
mentation science is equal to implemen-
tation research as described above
[15,43,44].
The term health services research
suggests research that focuses on the
service component of the health system,
but it is often defined more broadly and
used interchangeably with health systems
research [17,45].
Conclusion
Definitions are meant to clarify. But if
too many different definitions for the same
iMedPub :: Thematic Collections :: Volume 2

terms abound, so much confusion results
that they become an obstacle to progress.
Our aim here is not to establish which of
these definitions are correct or to launch
an intense debate about definitions that
would distract from the need to support
the research itself and the use of research
results to improve health. Instead, we seek
to provide a simple framework that is
easily understood by both experts in the
field and the managers, policy makers, and
donors working to improve health systems
and deliver better health care. We have
tried to map the three main research
domains, the research targets, and the
users, and to highlight the importance of
context and study design in the subsequent
utility of the research findings.
Research on operational problems, on
implementation strategies, and on health
system challenges all involve multidisci-
plinary research that tends to use the same
type of quantitative and qualitative re-
search methods. But these three research
domains differ in the type of research
questions they address, in the way they are
designed, and in their expected outcomes.
Operational research and implementation
research are action-oriented, respond to
operational problems or implementation
challenges, and work towards developing
targeted solutions. Research on health
system challenges addresses more com-
plex, systems problems and is geared
towards improving the understanding of
References
1. Fryatt R, Mills A, Nordstrom A (2010) Financing
of health systems to achieve the health Millenni-
um Development Goals in low-income countries.
Lancet 375: 419–426.
2. Madon T, Hofman KJ, Kupfer L, Glass RI (2007)
Public health implementation science. Science
318: 1728–1729.
3. Samb B, Evans T, Dybul M, Atun R, Moatti JP,
et al. (2009) An assessment of interactions
between global health initiatives and country
health systems. Lancet 373: 2137–2169.
4. Travis P, Bennett S, Haines A, Pang T, Bhutta Z,
et al. (2004) Overcoming health-systems con-
straints to achieve the Millennium Development
Goals. Lancet 364: 900–906.
5. World Health Organization (2005) Report from
the Ministerial Summit on Health Research 2004,
Mexico City, Mexico. Geneva: World Health
Organization.
6. Bennett S, Adam T, Zarowsky C, Tangcharoen-
sathien V, Ranson K, et al. (2008) From Mexico
to Mali: progress in health policy and systems
research. Lancet 372: 1571–1578.
7. World Health Organization (2009) Scaling up
research and learning for health systems: now is
the time. Report of a High Level Task Force.
Geneva: World Health Organization.
8. Zachariah R, Harries AD, Ishikawa N, Rieder HL,
Bissell K, et al. (2009) Operational research in low-
income countries: what, why, and how? Lancet
Infect Dis 11: 711–717.
9. The Global Fund and World Health Organiza-
tion (2007) Guide to operational research in
programmes supported by the Global Fund.
Geneva: The Global Fund.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
what works for whom and under what
circumstances. It provides guidance on
what might work better within the system
as a whole.
The three research domains are not
mutually exclusive, and there are large
overlapping areas. Research on operation-
al problems is about local problem solving,
but not all the problems it addresses are
truly local. Many occur in a similar
manner in multiple locations and may
represent implementation problems for
specific interventions that might be effi-
ciently tackled by implementation re-
search, or are representative of a systems
problem that could be effectively ad-
dressed through health systems research.
New implementation strategies are often
designed to overcome specific health
system failures, e.g., how to improve access
to vaccination among children who are
currently not reached by immunization
services or home treatment for malaria in
communities where formal health systems
are not able to effectively provide such
treatment. In such situations, implemen-
tation research develops innovative solu-
tions that are in effect improvements of the
health system and that could be regarded
as health systems research, especially when
these innovations affect more than just a
single intervention. Such overlap between
the three research domains provides
opportunities for cross-fertilization that
should lead to greater consideration by
10. World Health Organization and the Global
Fund (2008) Framework for Operations and
Implementation Research in Health and Disease
Control Programmes. Geneva: World Health
Organization.
11. Horstick O, Sommerfeld J, Kroeger A, Ridley R
(2010) Operational research in low-income coun-
tries. Lancet Infect Dis 10: 369–370.
12. Sanders D, Haines A (2006) Implementation
research is needed to achieve international health
goals. PLoS Med 3: e186.
13. Allotey P, Reidpath DD, Ghalib H, Pagnoni F,
Skelly WC (2008) Efficacious, effective, and
embedded interventions: implementation re-
search in infectious disease control. BMC Public
Health 8: 343.
14. UNICEF/UNDP/World Bank/WHO Special
Programme for Research and Training in
Tropical Diseases (2003) Implementation Re-
search in TDR: conceptual and operational
framework. Geneva: World Health Organization.
Report No.: TDR/IDE/SP/03.1.
15. Eccles MP, Armstrong D, Baker R, Cleary K,
Davies H, et al. (2009) An implementation
research agenda. Implement Sci 4: 18.
16. Bhattacharyya O, Reeves S, Zwarenstein M
(2009) What is implementation research: ratio-
nale, concepts and practice. Research on Social
Work Practice 19: 491–502.
17. Alliance for Health Policy and Systems Research
(2007) What is health policy and systems research
and why does it matter? Geneva: World Health
Organization.
18. Mills A, Gilson L, Hanson K, Palmer N,
Lagarde M (2008) What do we mean by rigorous
6 November 2010 | Volume 7 | Issue 11 | e1001000
operational and implementation research-
ers of the wider system implications of
their research. Accordingly, this should
encourage the expansion of study designs
and an appreciation of the feasibility of
experimentation with different health sys-
tem solutions.
We hope the above helps clarify what
research to improve health systems is
seeking to achieve. In essence it is quite
simple: it involves operational research on
operational problems, implementation re-
search on implementation strategies for
available interventions, and research on
health system challenges as the main focus
of health systems research. To improve
health care delivery to poor populations,
all of these research domains are very
much needed.
Author Contributions
ICMJE criteria for authorship read and met:
RFT JHFR TA FBP CD MD CG AG JH
JFKK AM MTM ZM TP RGR FZ. Agree with
the manuscript’s results and conclusions: RFT
JHFR TA FBP CD MD CG AG JH JFKK AM
MTM ZM TP RGR FZ. Conceived and
designed the experiments: RFT JHFR CD
MD CG JH MTM TP RGR. Wrote the first
draft: RFT JHFR FBP CD MD CG JH MTM
TP RGR. Wrote the paper: RFT JHFR TA
FBP CD MD CG AG JH JFKK MTM ZM TP
RGR FZ. Developed original concept: JHFR
CD MD CG JH MTM TP RGR RFT.
Commented on drafts: TA FBP AG JFKK
ZM FZ.
health-systems research? Lancet 372: 1527–
1529.
19. World Health Organization and partners (2010)
The First Global Symposium on Health Systems
Research (HSR) - Science to Accelerate Universal
Health Coverage, Montreux, November 2010.
Available: http://www.hsr-symposium.org/. Ac-
cessed 10 September 2010.
20. World Health Organization (2007) Everybody’s
business: strengthening health systems to improve
health outcomes. WHO’s framework for action.
Geneva: World Health Organization.
21. The Oxford English Dictionary Second Edition.
22. World Health Organization, Department of
Reproductive Health and Research (2003) Ex-
panding capacity for operations research in
reproductive health. Geneva: World Health
Organization Report No.: WHO/RHR/02.18.
23. Zachariah R, Ford N, Draguez B, Yun O, Reid T
(2010) Conducting operational research within a
non governmental organization: the example of
Medecins Sans Frontieres. Internat Health 2:
1–8.
24. CDI Study Group (2010) Community-directed
interventions for priority health problems in
Africa: results of a multicountry study. Bull World
Health Organ 88: 509–518.
25. World Health Organization (2009) Meeting of the
Strategic Advisory Group of Experts on Immu-
nization, October 2009 – conclusions and rec-
ommendations. Wkly Epidemiol Rec 84: 517–
532.
26. Pagnoni F (2009) Malaria treatment: no place like
home. Trends Parasitol 25(3): 115–119.
53
World Health Report 2012: No Health Without Research
27. Remme JHF (2004) Research for control: the
example of onchocerciasis. Trop Med Int Health
9: 243–254.
28. de Savigny D, Adam T (2009) Systems thinking
for health systems strengthening. Geneva: World
Health Organization, Alliance for Health Policy
and Systems Research.
29. De Savigny D, Kasale H, Mbuya C, Reid G
(2008) Fixing health systems. 2nd ed. Ottawa:
International Development Research Centre.
30. Varkevisser CM, Pathmanathan I, A.T. B (2003)
Designing and conducting health systems re-
search projects: Proposal development and field-
work. Ottawa: International Development Re-
search Centre.
31. Task Force on Heath Systems Research (2004)
Informed choices for attaining the Millennium
Development Goals: towards an international
cooperative agenda for health-systems research.
Lancet 364: 997–1003.
32. Ranson K, Law TJ, Bennett S (2010) Establishing
health systems financing research priorities in
developing countries using a participatory meth-
odology. Soc Sci Med 70: 1933–1942.
33. Ranson MK, Chopra M, Atkins S, Dal Poz MR,
Bennett S (2010) Priorities for research into
human resources for health in low- and middle-
income countries. Bull World Health Organ 88:
435–443.
34. Last JM (2001) A dictionary of epidemiology;
fourth edition. Oxford: Oxford University Press.
35. Craig P, Dieppe P, Macintyre S, Michie S,
Nazareth I (2008) Developing and evaluating
complex interventions: the new Medical Research
Council guidance. BMJ 337: a1655.
36. Remme JH, Blas E, Chitsulo L, Desjeux PM,
Engers HD, Kanyok TP, et al. (2002) Strategic
emphases for tropical diseases research: a TDR
perspective. Trends Parasitol 18: 421–426.
37. Edejer TT, Baltussen R, Adam T, Hutubessy R,
Acharya A, et al. (2003) WHO guide to cost-
PLoS Medicine | www.plosmedicine.org
54
effectiveness analysis. Geneva: World Health
Organization.
38. Guiasu S (2009) Probabilistic models in operations
research. New York: Nova Science Publishers.
39. Eccles MP, Mittman BS (2006) Welcome to
Implementation Science. Implement Sci 1: 1–3.
40. McKibbon KA, Lokker C, Wilczynski NL,
Ciliska D, Dobbins M, et al. (2010) A cross-
sectional study of the number and frequency of
terms used to refer to knowledge translation in a
buddy of health literature in 2006: a Tower of
Babel? Implement Sci 5: 1–11.
41. Wallin L (2009) Knowledge translation and
implementation research in nursing. Int J Nurs
Stud 46: 576–587.
42. Walker AE, Grimshaw J, Johnston M, Pitts N,
Steen N, et al. (2003) PRIME–PRocess modelling
in ImpleMEntation research: selecting a theoret-
ical basis for interventions to change clinical
practice. BMC Health Serv Res 3: 22.
43. Woolf SH (2008) The meaning of translational
research and why it matters. JAMA 299:
211–213.
44. Gray J (2010) Global health experts seek to
transform programs through implementation
science. Global Health Matters 9: 10–12.
45. Lohr KN, Steinwachs DM (2002) Health services
research: an evolving definition of the field.
Health Serv Res 37: 7–9.
46. Ramaiah KD, Das PK, Appavoo NC, Ramu K,
Augustin DJ, et al. (2000) A programme to
eliminate lymphatic filariasis in Tamil Nadu state,
India: compliance with annual single-dose DEC
mass treatment and some related operational
aspects. Trop Med Int Health 5: 842–847.
47. Ramaiah KD, Vijay Kumar KN, Hosein E,
Krishnamoorthy P, Augustin DJ, et al. (2006) A
campaign of ‘‘communication for behavioural
impact’’ to improve mass drug administrations
against lymphatic filariasis: structure, implemen-
tation and impact on people’s knowledge and
7 November 2010 | Volume 7 | Issue 11 | e1001000
treatment coverage. Ann Trop Med Parasitol
100: 345–361.
48. Zachariah R, Van Engelgem I, Massaquoi M,
Kocholla L, Manzi M, et al. (2008) Payment for
antiretroviral drugs is associated with a higher
rate of patients lost to follow-up than those offered
free-of-charge therapy in Nairobi, Kenya.
Trans R Soc Trop Med Hyg 102: 288–293.
49. Robays J, Nyamowala G, Sese C, Betu Ku Mesu
Kande V, Lutumba P, et al. (2008) High failure
rates of melarsoprol for sleeping sickness, Dem-
ocratic Republic of Congo. Emerg Infect Dis 14:
966–967.
50. UNICEF/UNDP/World Bank/WHO Special
Programme for Research and Training in
Tropical Diseases (1996) Community-directed
treatment with ivermectin: report of a multi-
country study. Geneva: World Health Organiza-
tion Report No.: TDR/AFT/RP/96.1.
51. Munkhuu B, Liabsuetrakul T, Chongsuvivatwong V,
McNeil E, Janchiv R (2009) One-stop service for
antenatal syphilis screening and prevention of
congenital syphilis in Ulaanbaatar, Mongolia: a
cluster randomized trial. Sex Transm Dis 36:
714–720.
52. Mondal D, Singh SP, Kumar N, Joshi A,
Sundar S, et al. (2009) Visceral leishmaniasis
elimination programme in India, Bangladesh,
and Nepal: reshaping the case finding/case
management strategy. PLoS Negl Trop Dis 3:
e355. doi:10.1371/journal.pntd.0000355.
53. Blaauw D, Erasmus E, Pagaiya N, Tangcharoen-
sathein V, Mullei K, Mudhune S, et al. (2010)
Policy interventions that attract nurses to rural
areas: a multicountry discrete choice experiment.
Bull World Health Organ 88: 350–356.
54. Rabkin M, El-Sadr WM, De Cock KM (2009)
The impact of HIV scale-up on health systems: A
priority research agenda. J Acquir Immune Defic
Syndr 52(Suppl 1): S6–11.
iMedPub :: Thematic Collections :: Volume 2

Policy Forum
The Global Health System: Lessons for a Stronger
Institutional Framework
Suerie Moon1*, Nicole A. Szlezák1, Catherine M. Michaud2, Dean T. Jamison3, Gerald T. Keusch4,
William C. Clark1, Barry R. Bloom5
1 Sustainability Science Program, John F. Kennedy School of Government, Harvard University, Cambridge, Massachusetts, United States of America, 2 Harvard Initiative for
Global Health, Harvard University, Cambridge, Massachusetts, United States of America, 3 Department of Global Health, University of Washington, Seattle, Washington,
United States of America, 4 Global Health Initiative, Boston University, Boston, Massachusetts, United States of America, 5 Harvard School of Public Health, Boston,
Massachusetts, United States of America
This is the fourth in a series of four articles
that highlight the changing nature of global
health institutions.
Introduction
The global health system is in a period of
rapid transition, with an upsurge of funds
and greater political recognition, a broader
range of health challenges, many new
actors, and the rules, norms and expecta-
tions that govern them in flux. The
introductory article of this series (Szlezák
et al. [1]) laid out some of the many
challenges facing the global health system.
This system is defined as the constellation of
actors (individuals and/or organizations)
‘‘whose primary purpose is to promote,
restore or maintain health [2]’’ and ‘‘the
persistent and connected sets of rules
(formal or informal), that prescribe behav-
ioral roles, constrain activity, and shape
expectation [3]’’ among these actors. The
second article (Frenk [4]) defined the key
attributes of national health systems as a
core component of the global system. The
third article (Keusch et al. [5]) analyzed the
institutional evolution of one of the system’s
most important functions—the integration
of research, development, and delivery.
This concluding article draws on the
others in the series. It also draws from a
year-long effort that included case studies,
two international workshops of scholars
and practitioners (further details at http://
www.hks.harvard.edu/centers/cid/programs/
sustsci/events/workshops/2008/institutions),
and ongoing discussions by the authors, to
summarize lessons learned and propose
future actions to strengthen the system as a
whole. The project used as a case study the
global health system’s evolving response to
The Policy Forum allows health policy makers
around the world to discuss challenges and
opportunities for improving health care in their
societies.
iMedPub ::PLoS Medicine
Thematic | www.plosmedicine.org
collections :: Volume 2
World Health Report 2012: No Health Without Research
malaria. Nevertheless, the workshops and WHO and the UN Childrens Fund (UNI-
discussions that informed this analysis CEF)—with input from national govern-
drew from a broader range of cases, and ments and a few foundations. It was
we believe lessons learned may usefully exemplified by iconic programs such as
apply beyond malaria alone. Furthermore, the eradication initiatives for malaria and
while recognizing the many determinants smallpox in the 1950s–70s. Agenda-setting
of health and interlinkages between health is well captured by a ‘‘punctuated equilib-
and other issue areas such as trade and rium’’ model, in which long periods of
environment [6,7], we limit our scrutiny relative stability in agendas are sporadically
here to the global health system. broken by sudden bursts of high-level
The project concluded that an effective attention in public and policy circles [8,9].
global health system must accomplish at Agendas may vary because of crises, such as
least five core functions: agenda-setting; natural disasters or epidemics, or from
financing and resource allocation; research recognition of the human and economic
and development (R&D); implementation costs of inaction, as with noncommunicable
and delivery; and monitoring, evaluation, diseases [10]. History indicates that these
and learning. We discuss here ways to episodes of high attention are fleeting;
improve each of the five functional areas, seizing these brief opportunities to produce
consider the implications for the role of lasting change usually requires adapting
the World Health Organization (WHO), governance structures to accommodate
and make recommendations for future new actors and interests [11].
action. Our case study of malaria found that,
after undergoing a half-century of fluctu-
Key Functions of the Global ating global attention, malaria re-emerged
Health System on the global agenda in the late 1990s.
Agenda-Setting Central to its reemergence was the crea-
In the past, global agenda-setting in tion of a novel global governance struc-
health took place within the framework of ture, the Roll Back Malaria (RBM)
the United Nations (UN)—primarily at Partnership, launched by WHO. RBM
Citation: Moon S, Szlezák NA, Michaud CM, Jamison DT, Keusch GT, et al. (2010) The Global Health System:
Lessons for a Stronger Institutional Framework. PLoS Med 7(1): e1000193. doi:10.1371/journal.pmed.1000193
Academic Editor: Gill Walt, London School of Hygiene and Tropical Medicine, United Kingdom
Published January 26, 2010
Copyright: � 2010 Moon et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: This work was supported by a grant to the Institutional Innovations in Global Health project by the
John F. Kennedy School of Government at Harvard University, Cambridge, Massachusetts (US) under its ‘‘Acting
in Time’’ program. Additional support was received from the Burroughs Wellcome Fund, Research Triangle
Park, North Carolina (US). The funders had no role in the decision to publish or preparation of the manuscript.
Competing Interests: Suerie Moon has been a consultant for UNITAID and WHO.
Abbreviations: ACT, artemisinin combination therapy; GFATM, Global Fund to Fight AIDS, Tuberculosis and
Malaria; INCLEN, International Clinical Epidemiology Network; M&E, monitoring and evaluation; PDP, public–
private product development partnership; R&D, research and development, RBM, Roll Back Malaria; TDR,
Special Programme for Research and Training in Tropical Diseases; UN, United Nations; UNICEF, United Nations
Children’s Fund; WHA, World Health Assembly; WHO, World Health Organization.
* E-mail: suerie_moon@hksphd.harvard.edu
Provenance: Commissioned, externally peer reviewed.
1 January 2010 | Volume 7 | Issue 1 | e1000193 55
World Health Report 2012: No Health Without Research
now includes over 100 organizations
including endemic country governments,
donors, civil society organizations, the
private sector, and academia.
Once an issue garners attention and
attracts many new actors and activities,
effective governance requires a process for
setting an agenda for action within the issue
area. Coordination is ultimately essential;
however, as several experienced partici-
pants in our workshops pointed out, few
organizations wish to be coordinated,
because of the costs and loss of autonomy
entailed. Thus coordination and some
degree of harmonization of multiple inde-
pendent activities are likely to emerge only
after the construction of consensus on a
widely shared set of rules, roles, and
expectations. To get to this consensus,
participants must share a clear set of goals
and perceive the process as inclusive,
transparent, technically credible, and fair.
Effective agenda-setting for action,
when achieved, can provide a framework
(albeit no guarantee) for coordination at
global and national levels. The 2008
Global Malaria Action Plan, which was
negotiated within the RBM framework,
exemplifies how global agenda-setting for
action within an issue area can be
achieved [12]. A similar institution, the
Stop TB Partnership, has also created a
coordinating framework for tuberculosis
control.
Underlying such institutional frame-
works must be scientifically valid metrics,
evidence of the problem’s importance, and
recognition that tools exist, however im-
perfect, that could improve health out-
comes. Finally, the framework requires
that the affected countries and the public,
who are ultimately co-producers of health,
be represented as key participants [4].
These partnerships, anchored by the
legitimacy of the WHO, represent creative
approaches to eliciting the broad partici-
pation necessary to construct widely ac-
cepted agendas and forge consensus at the
global level.
Financing and Resource Allocation
International financing and resource
allocation for health in developing coun-
tries have long been subject to three
fundamental questions [13]: (1) How
should the priorities of donors be balanced
with those of recipients? (2) How should
resources be allocated to different diseases
or issue areas? (3) How can sufficient
investment into health, which has tradi-
tionally been underfunded relative to
need, be ensured?
In the past, international resources for
health flowed primarily through bi-/mul-
PLoS Medicine | www.plosmedicine.org
56
tilateral donors and WHO with only a few
exceptions, e.g., the Rockefeller Founda-
tion. Over the past decade a variety of
actors, including philanthropists, advocacy
groups, civil society, and public and
private sector organizations, have cata-
lyzed an unprecedented increase in the
flow of international financing for health
[14]. In the case of malaria, funding has
increased tenfold [12]. The Global Fund
to Fight AIDS, Tuberculosis and Malaria
(GFATM) represents a model for enabling
some coherence in the resource allocation
process for its mandated diseases. The
GFATM balances two often-competing
objectives: providing reassurance to do-
nors by only funding projects adhering to
international ‘‘best practices,’’ and de-
manding that country applications dem-
onstrate meaningful and widespread na-
tional ownership [15]. By unifying
multiple funding streams and oversight
processes, the GFATM also tries to lighten
the burden put on national health systems
by reporting requirements and lack of
coordination among multiple donors—an
issue exacerbated by the recent increase in
players in the global health system.
While the upsurge of financing for
malaria is welcome, it also points to a
current governance gap in the overall
system: there are no clear norms for how
resources should be allocated across dif-
ferent health needs. The Global Burden of
Disease and Disease Control Priorities
Projects have provided country estimates
of years of healthy life lost to illness and
injury, identified major risks, and estimat-
ed the cost-effectiveness of interventions
[16,17]. However, widely accepted princi-
ples on how to translate these figures into
resource allocation decisions are lacking.
Major discrepancies exist between re-
sources provided for specific diseases and
their relative burden, e.g., HIV/AIDS
versus chronic diseases, and between the
burdens of disease within specific countries
and their ability to attract resources to
address them [18].
A related question is how to increase
funding levels further to meet the full
spectrum of global health needs, and how
to sustain those levels in the long run.
Especially in difficult economic times, it is
critical to ensure continuity by insulating
finance arrangements to the greatest
extent possible [19]. In all World Bank
regions, external development assistance
represents less than 3% of total health
spending, with the exception of sub-
Saharan Africa where it accounts for
21% [20]. However, international financ-
ing is critical for providing global public
goods and for the lowest-income countries
2 January 2010 | Volume 7 | Issue 1 | e1000193
that rely on aid to meet basic health needs.
Financial fluctuations can be disruptive in
all countries, but for the poorest, a sudden
drop in aid can be devastating. The long-
term sustainability of financing will rest on
three elements: (1) demonstrating results;
(2) making financing arrangements more
politically acceptable by mobilizing more
resources from middle-income countries;
and (3) developing innovative financing
mechanisms that are less vulnerable to
politicized budgeting processes. One such
innovative model is UNITAID, which
purchases health products for use primar-
ily in low-income countries and is funded
through national airline taxes. Of the 29
committed UNITAID donors, three-quar-
ters are low- or middle-income countries,
emphasizing the idea that all nations—
even the poorest—can contribute to sus-
tainable global health finance [21].
Research and Development
In the past, health technologies such as
drugs, vaccines, and diagnostics were
developed primarily by and for popula-
tions in the industrialized world. Today
there is increasing evidence of contribu-
tions from the South to global health
research. Investments in human capacity
that began in the 1970s are now bearing
fruit as scientists from Africa, Asia, and
Latin America take a key role in advanc-
ing research, as in the case of malaria [5].
After a period of neglect, there is now a
resurgence of R&D aimed specifically at
developing new tools for the health needs
of developing countries. Since traditional
market incentives such as the patent
system are unlikely to generate the neces-
sary innovation, much of malaria R&D is
now taking place through public–private
product development partnerships (PDPs),
which receive significant philanthropic,
public, and private investments [22]. In
contrast to classic private-sector product
development, however, the PDPs have an
explicit objective of jointly achieving
affordability and innovation suited to
developing country contexts. PDPs are
redefining the roles of public and private
sectors and promoting new expectations
for the development of health technologies
as global public goods.
Experience with malaria offers several
lessons for R&D in other health areas.
First, investments in human capital are
essential but take many years to bear
fruit. Here the long-term commitment of
the UNICEF/UNDP/World Bank/WHO
Special Programme for Research and
Training in Tropical Diseases (TDR) and
the recent 30 million commitment from
the Wellcome Trust for research capacity
iMedPub :: Thematic Collections :: Volume 2

building in Africa should be noted [23].
Nevertheless, greater training in laboratory
sciences, health economics, management,
program evaluation, and implementation
research are clearly needed. Capacity-build-
ing of developing country researchers and
research organizations (e.g., universities,
public research institutes) should receive
greater emphasis and be scaled up today.
Second, considerations of access to products
should be built into R&D processes from
their inception. The WHO Global Strategy
and Plan of Action on Public Health,
Innovation and Intellectual Property, ap-
proved at the 2008 World Health Assembly
(WHA), is an important contribution to
rethinking the governance of the R&D
system, and merits the constructive engage-
ment of all concerned parties [24].
Implementation and Delivery
As the essential link between global
actors and local populations, national
health systems are a critical part of the
global health system. Health systems
must accomplish multiple challenging
tasks. These include: providing preven-
tive and primary care services; develop-
ing a health workforce; devising equita-
ble financing arrangements; regulating
the private sector; and leveraging vertical
programs (such as malaria control) to
strengthen, rather than distort, the over-
all health system (‘‘diagonalization’’ [25])
([4]).
Health system performance varies wide-
ly but the reasons for this variation remain
poorly understood. For example, Eritrea,
Ethiopia and Rwanda have reduced
malaria-related morbidity and mortality
dramatically [26,27]. Eritrea, for one,
credits the RBM strategy and community
health workers as key components of their
approach [28]. However, it is unclear why,
largely using the same strategies recom-
mended to all endemic countries, others
were less successful. Recent analyses of
health systems performance point to
leadership, community involvement, dis-
trict-level focus, use of data to set priorities
and track progress, and prioritizing equi-
table access as key factors that have
enabled significant improvements in
health outcomes in some countries
[4,29]. Even when public sector delivery
capacity is weak, some countries have still
managed to expand primary health care
coverage and improve childhood survival
by engaging the private for-profit as well
as nonprofit sectors [29]. These non-state
actors can energize national health systems
by sharing knowledge of how better to
achieve efficiency, outreach, and user
satisfaction. A comprehensive operational
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
and policy research agenda is needed to
understand fully which policies and prac-
tices best strengthen national health sys-
tems [30,31].
Monitoring, Evaluation, and
Learning
Reliable information on the impact of
health programs is critical to setting
priorities, measuring efficacy, and main-
taining global support for any interven-
tion. Yet the global health system current-
ly poorly manages monitoring and
evaluation (M&E). There is no consensus
on key questions regarding who should be
responsible for M&E, how it should be
carried out, how available the information
should be, and how it should be used.
National and subnational organizations
for conducting M&E and promoting
critical learning are relatively weak, and
incentives for strengthening them are
almost nonexistent.
For example, there remain enormous
gaps in knowledge about malaria. Precise
annual and seasonal malaria incidence
and mortality data, or the percentage of
children with fevers that actually have
malaria, are unavailable in most endemic
countries and districts [28,32–34]. How to
mobilize communities to make full use of
bed nets, artemisinin combination thera-
pies (ACTs), and indoor spraying remain
critical research issues. With increasing
funds being expended on programs, it is
shortsighted that so little is spent on
operational research, on learning what
works in specific contexts and how best to
engage communities to use the tools
available [35]. An essential step toward
sufficient investment in M&E is to ac-
knowledge and plan for its costs, both in
dollars and, more importantly, in the
limited time available from experienced
managers and researchers. This research is
vital, yet ironically it is too rarely funded
by major donors nor requested by imple-
menting countries.
M&E should be an integral part of all
program planning, yet it is too often an
afterthought. Furthermore, effective M&E
of programs and interventions, as well as
learning from experience, requires that
M&E efforts achieve technical credibility,
maintain legitimacy (i.e., general acceptance
of their authority), and produce knowledge
that is salient for end-users. These three
objectives often compete directly with one
another [36]. For example, there is often
tension between the goal of producing
data for internal learning and that of
monitoring by outside parties (e.g., higher-
level officials or donors). When evaluation
data are linked to funding, as in currently
3 January 2010 | Volume 7 | Issue 1 | e1000193
favored ‘‘performance-based funding,’’ the
accuracy of the data provided may
diminish. It may be necessary to create
institutional ‘‘safe spaces’’ where failures
can be divulged to encourage genuine
learning rather than used to assign blame
for underperformance [37,38].
Similarly, at the global level, all coun-
tries provide health statistics to WHO, but
the quality and validity of these data vary
greatly. Yet WHO is a producer, reposi-
tory, and evaluator of evidence, and
simultaneously a political organization
representing 193 countries, and so it will
be particularly difficult for the organiza-
tion to meet the demands for saliency,
credibility, and legitimacy. Academic re-
searchers, who also play a critical role in
M&E, face a contrasting set of difficulties.
While they may achieve technical credi-
bility, it is more challenging to achieve
political legitimacy and ensure that the
knowledge produced is salient to end users
and policy-makers.
Several new organizations are address-
ing some of these issues. For example, the
Institute for Health Metrics and Evalua-
tion [39] and the WHO-hosted Health
Metrics Network [40], both funded in part
by the Bill & Melinda Gates Foundation,
were recently created to develop methods
to acquire, analyze and disseminate rele-
vant and reliable information on burden of
disease and global health. There are
promising models, some country-based
such as the India Program Evaluation
Network coming out of the International
Clinical Epidemiology Network (IN-
CLEN) [41] and the India network,
IndiaCLEN [42], others international,
such as the Southern-led INDEPTH
epidemiological network [43] and the
Cochrane Collaboration [44]. The chal-
lenge is not only to secure the evidence,
but also to have the political and proce-
dural legitimacy that, to date, few organi-
zations other than the UN agencies have
been chartered to provide.
The Role of WHO
Cutting across these five core functions
is the question of how changes in the
global health system redefine the role of
WHO. WHO is facing ‘‘an urgent need to
define and assert a clear and effective role
for itself, as never before’’ [45]. There are
at least three key roles that we believe only
WHO can fulfill and therefore must do
well. The first is global stewardship, i.e.
identifying needs to be met and taking a
leadership role in setting global norms.
Second is as a provider of operational
support to countries: WHO has a unique
57
World Health Report 2012: No Health Without Research
capacity to engage the best experts
worldwide, which should enable it to
provide technical assistance to govern-
ments through normative guidelines and
recommendations reflecting best evidence
and practice. To retain the legitimacy to
do so, it must maintain the highest
technical and ethical standards [46]. The
third is its special role in governance: as
the major global intergovernmental health
organization, WHO has a unique conven-
ing power and mandate for decision-
making on major health-related issues. Its
governing body, the World Health Assem-
bly, with its 193 Ministers of Health,
provides WHO its singular legitimacy to
carry out these mandated roles of stew-
ardship, country support, and governance,
albeit with a high degree of bureaucrati-
zation and politicization. Yet WHO’s
regular budget resources are remarkably
limited. For the 2006–7 biennium, the
formal budget assessed on countries was
less than $1 billion (with voluntary contri-
butions the total budget was just over $3
billion); the following period, three-fourths
of the budget was allocated to the regions
[47]. This excessive budgetary decentral-
ization undermines WHO’s capacity to
deliver the global public goods demanded
of it.
Inadequate levels of core funding have
resulted in predictable consequences for
performance. For example, WHO’s ma-
laria program has experienced a number
of difficulties in recent years, and while
inadequate funding is not the only cause it
is an important one. When WHO fails to
lead, new global partnerships such as
RBM (originally created by WHO’s Di-
rector-General) have stepped in, received
external funding, and, to an extent, WHO
has been marginalized. For WHO to fulfill
the key roles for which it is uniquely
charged, it will need strong leadership,
strengthened technical expertise, and
clearer focus. The current economic crisis
provides an opportunity for WHO to
redefine and strengthen its core functions,
recognize what it cannot do well, and
delegate to or partner with other organi-
zations. If it can define its core functions
and strategic role credibly, the organiza-
tion will justify—and perhaps be more
likely to receive—the greater resources it
will need to fulfill its central global mission.
Lessons and Future Needs
Several general lessons have emerged
from our study of institutions in the global
health system.
N In the present complex global envi-
ronment no single actor can or should
PLoS Medicine | www.plosmedicine.org
58
set the agenda for action. Global
partnerships similar to those that have
transformed malaria and the infectious
disease agenda will be needed to
mobilize resources for other health
problems, such as chronic diseases.
An example is the new Global Alliance
on Chronic Diseases [48]. Broad-
based, participatory processes for
agenda setting, anchored by WHO’s
global political legitimacy, will be
required to define priorities, avoid
unnecessary duplication, and share
knowledge. There is clearly a tension
between WHO as an intergovernmental
organization and WHO as a partner in
multiple partnerships where it must
share power with a broader set of
nongovernmental actors including civil
society organizations, foundations,
and the private sector. Widely accept-
ed procedural principles including
transparency, broad participation, in-
tegrity of data, and equity should be
adopted to construct the consensus
necessary for effective coordination.
N Sustainability depends on strengthen-
ing national health systems, which are
the essential link between global
knowledge and best practices, and
local health needs and impact. Dis-
ease-specific international funding also
has much to contribute. But it can
distort national priorities, pull resourc-
es from less-popular programs, and
ultimately undermine the overall per-
formance of the health system [49].
Country experts are often in a better
position to set priorities than outside
consultants. Donors should allow
greater flexibility for recipient coun-
tries to direct a portion of received
funds beyond narrow programmatic
interventions to strengthening national
health systems. This will require the
development of clearly defined goals
and performance indicators for key
functions of health systems such as
service provision, research, health
worker development, and equity of
access.
N Ironically, the proliferation of global
actors threatens to weaken health
systems by placing additional reporting
burdens on already thinly stretched
health ministries [49]. By channeling
multiple funding streams into a single
source for HIV/AIDS, tuberculosis,
and malaria, the GFATM offers an
instructive example of how to distrib-
ute the resources of various donors in a
way that is sensitive to national health
systems’ priorities and constraints. As
new global health initiatives arise to
4 January 2010 | Volume 7 | Issue 1 | e1000193
address the wave of emerging health
challenges, the global health system
should identify and adopt analogous
ways to streamline reporting and,
more generally, to minimize the
additional transaction costs put on
countries.
N Systematic investment in creating new
and improving existing M&E pro-
grams should become second nature
for all global health activities. The
global health system has two important
functions to fulfill. First, it needs to set
the tone and actively foster the idea
that M&E is crucial to global health.
Second, it needs to support the sys-
tematic exchange, coordination, and
streamlining of M&E efforts. Over
time, this investment will contribute
to building robust M&E systems and to
generating reliable, comparable data
to inform action.
N There is compelling evidence that
long-term investments in education
and training at many levels (e.g.,
national, provincial, district) can result
in large payoffs for improved health
[5,50]. The global health system
should prioritize additional invest-
ments in longer-term, multidisciplinary
education and training for leadership
in the complex public health, medical,
management, economic, education,
communications, and policy aspects
of health systems, and in the function-
ing of health systems overall.
N Finally, it will be critical to support
research that provides the evidence
and knowledge bases for prioritiza-
tion, resource allocation, and the
development and evaluation of new
tools and interventions. In particular,
operational research will be crucial
to learning how to use the tools that
are available, take them to scale, and
engage populations to become co-
producers of health rather than pas-
sive recipients of services. More
broadly, research should be promoted
to understand variation in the perfor-
mance of different national health
systems, and thus to identify system
designs that can be adapted to local
circumstances to help translate global
aspirations into meaningful impact on
people’s lives.
Author Contributions
ICMJE criteria for authorship read and met:
SM NAS CMM DTJ GTK WCC BRB. Wrote
the first draft of the paper: SM NAS. Contrib-
uted to the writing of the paper: SM NAS
CMM DTJ GTK WCC BRB. One of the two
iMedPub :: Thematic Collections :: Volume 2

original conceptualizers of the study, and PI on
the grant that supported it: WCC. Chaired the
References
1. Szlezá k NA, Bloom BR, Jamison DT,
Keusch GT, Michaud CM, et al. (2010) The
Global Health System: Actors, Norms, and
Expectations in Transition. PLoS Med 7:
e1000183. doi:10.1371/journal.pmed.1000183.
2. World Health Organization (2000) World health
report 2000 – health systems: Improving perfor-
mance. Geneva: WHO, Available: http://www.
who.int/whr/2000/en/whr00_en.pdf. Accessed:
3 July 2009.
3. Keohane RO (1988) International institutions:
Two approaches. Int Stud Q 32: 379–396.
4. Frenk J (2010) The Global Health System:
Strengthening National Health Systems as the
Next Step for Global Progress. PLoS Med 7(1):
e1000089. doi:10.1371/journal.pmed.1000089.
5. Keusch GT, Kilama WL, Moon S, Szlezak NA,
Michaud CM (2010) The Global Health System:
Linking Knowledge with Action—Learning from
Malaria. PLoS Med 7(1): e1000193. doi:10.1371/
journal.pmed.1000193.
6. Lee K, Sridhar D, Patel M (2009) Bridging the
divide: Global governance of trade and health.
Lancet 373: 416–422.
7. Costello A, Abbas M, Allen A, Ball S, Bell S, et al.
(2009) Managing the health effects of climate
change: Lancet and University College London
Institute for Global Health Commission. Lancet
373: 1693–1733.
8. Downs A (1972) Up and down with ecology: The
issue attention cycle. Public Interest 28: 38–50.
9. Shiffman J, Beer T, Wu Y (2002) The emergence
of global disease control priorities. Health Policy
Plan 17: 225–234.
10. Kingdon JW (1995) Agendas, alternatives and
public policies. New York: Harper Collins.
11. Baumgartner FR, Jones BD (1993) Agendas and
instability in American politics. Chicago: The
University of Chicago Press.
12. Roll Back Malaria (RBM) Partnership (2008) The
global malaria action plan: For a malaria free
world. Available: http://www.rollbackmalaria.
org/gmap/. Accessed: 3 July 2009.
13. Hecht R, Shah R (2006) Recent trends and
innovations in development assistance for health.
In: Jamison D, Breman JG, Measham AR,
Alleyne G, Claeson M, et al, editors. Disease
Control Priorities in Developing Countries, 2nd
edition. New York and Washington D.C.: Oxford
University Press and The World Bank. 243 p.
Available: http://files.dcp2.org/pdf/DCP/
DCP13.pdf. Accessed: 3 July 2009.
14. Ravishankar N, Gubbins P, Cooley RJ, Leach-
Kemon K, Michaud CM, et al. (2009) Financing
of global health: Tracking development assistance
for health from 1990 to 2007. Lancet 373:
2113–2124.
15. Szlezák N (2008) Global health in the making:
China, HIV/AIDS and the Global Fund to Fight
AIDS, Tuberculosis and Malaria [PhD disserta-
tion]. Cambridge (Massachusetts): Harvard Uni-
versity.
16. Lopez AD, Mathers CD, Ezzati E, Jamison DT,
Murray CJL, eds (2006) Global Burden of Disease
and Risk Factors. Oxford and New York: Oxford
University Press, Available: http://www.dcp2.
org/pubs/GBD. Accessed: 3 July 2009.
17. Jamison D, Breman JG, Measham AR, Alleyne G,
Claeson M, et al. (2006) Disease Control Priorities
in Developing Countries, 2nd edition. New York
and Washington D.C.: Oxford University Press
and The World Bank. pp 87–102. Available:
http://www.dcp2.org/pubs/DCP. Accessed: 3
July 2009.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
steering group for the project and supervised the
research and workshop planning: WCC. In-
18. Sridhar D, Batniji R (2008) Misfinancing global
health: A case for transparency in disbursements
and decision-making. Lancet 372: 1185–1191.
19. LaFranchi H (2009) Funds tighten for fighting
AIDS and malaria worldwide. The Christian
Science Monitor. (2 February) Available: http://
www.csmonitor.com/2009/0202/p02s02-usfp.
html. Accessed: 3 July 2009.
20. Schweitzer J (2008) Unpublished World Bank
data, presented at ‘‘Forum on Research Agendas
in Health Systems.’’ Harvard School of Public
Health. 4–5 December 2008.
21. UNITAID (2007) How it is financed: The air
ticket levy. 2009(February 26). Available: http://
www.unitaid.eu/index.php/en/The-air-ticket-
levy.html. Accessed: 5 December 2008.
22. Moran M (2005) A breakthrough in R&D for
neglected diseases: New ways to get the drugs we
need. PLoS Med 2: e302. doi/10.1371/jour-
nal.pmed.0020302.
23. Looi M (2009) Platform for research - African
Institutions Initiative. News and Features. The
Wellcome Trust. Available: http://www.well-
come.ac.uk/News/2009/Features/WTX055738.
htm. Accessed: 10 September 2009.
24. World Health Assembly (2008) Global strategy
and plan of action on public health, innovation
and intellectual property. WHA61.21 (Sixty-First
World Health Assembly). Available: http://apps.
who.int/gb/ebwha/pdf_files/A61/A61_R21-en.
pdf. Accessed: 3 July 2009.
25. Sepúlveda J, Bustreo F, Tapia R, Rivera J,
Lozano R, et al. (2006) Improvement of child
survival in Mexico: The diagonal approach.
Lancet 368: 2017–2027.
26. World Health Organization. (2008) World ma-
laria report 2008. WHO/HTM/GMP/2008.1.
Available: http://www.who.int/malaria/
wmr2008/malaria2008.pdf. Accessed: 3 July
2009.
27. Barat LM (2006) Four malaria success stories:
How malaria burden was successfully reduced in
Brazil, Eritrea, India and Vietnam. Am J Trop
Med Hyg 74: 12–16.
28. Ministry of Health Eritrea (2007) Malaria control
in Eritrea: A success story. Available: http://
www.hks.harvard.edu/var/ezp_site/storage/
fckeditor/file/pdfs/centers-programs/centers/
cid/ssp/docs/events/workshops/2008/institu-
tions/Meky_2008_Malaria_Control_in_Eritrea.
pdf. Accessed: 5 December 2008.
29. Rohde J, Cousens S, Chopra M,
Tangcharoensathien V, Black R, et al. (2008)
30 years after Alma-Ata: Has primary health
care worked in countries? Lancet 372: 950–961.
30. Putzel J (2004) The global fight against AIDS:
How adequate are the national commissions? J Int
Dev 16: 1129–1140.
31. Mills A, Rasheed F, Tollman S (2006) Strength-
ening health systems. In: Jamison D, Breman JG,
Measham AR, Alleyne G, Claeson M, et al,
editors. Disease Control Priorities in Developing
Countries, 2nd edition. New York and Washing-
ton D.C.: Oxford University Press and The
World Bank. pp 87–102. Available: http://files.
dcp2.org/pdf/DCP/DCP03.pdf. Accessed: 3 July
2009.
32. Guerra CA, Gikandi PW, Tatem AJ, Noor AM,
Smith DL, et al. (2008) The limits and intensity of
Plasmodium falciparum transmission: Implica-
tions for malaria control and elimination world-
wide. PLoS Med 5: e38. doi/10.1371/jour-
nal.pmed.0050038.
33. Snow RW, Guerra CA, Noor AM, Mying HY,
Hay SI (2005) The global distribution of clinical
5 January 2010 | Volume 7 | Issue 1 | e1000193
volved in the conception of the project and
organization of the meetings: BRB.
episodes of Plasmodium falciparum malaria.
Nature 434: 214–217.
34. Bell DR, Jorgensen P, Christophel EM,
Palmer KL (2005) Malaria risk: Estimation of
the malaria burden. Nature 437: E3–4.
35. van Kerkhoff L, Szlezák NA (2006) Linking local
knowledge with global action: Examining the
Global Fund to Fight AIDS, Tuberculosis and
Malaria through a knowledge systems lens. Bull
World Health Organ 84: 629–635.
36. Mitchell R, Clark W, Cash D (2006) Information
and influence. In: Mitchell R, Clark W, Cash D,
Dickson N, eds. Global Environmental Assess-
ments: Information and Influence. Cambridge
(Massachusetts): MIT Press. pp 307–337.
37. Clark WC, Holliday L (2006) Linking knowledge
with action for sustainable development: The role
of program management – summary of a
workshop. Washington D.C.: The National
Academies Press, Available: http://www.nap.
edu/openbook.php?record_id = 11652. Accessed:
3 July 2009.
38. Kristjanson P, Reid R, Dickson N, Clark W,
Romney D, et al. (2009) Linking international
agricultural research knowledge with action for
sustainable development. Proc Natl Acad Sci U S A
106: 5047–5052.
39. Institute for Health Metrics and Evaluation
(IHME) (2009) Institute for health metrics and
evaluation. 2009(6/29). Available: http://www.
healthmetricsandevaluation.org/. Accessed: 3 Ju-
ly 2009.
40. World Health Organization (WHO). About the
health metrics network. 2009(10/3). Available:
http://www.who.int/healthmetrics/about/en/.
Accessed: 10 October 2009.
41. International Clinical Epidemiology Network
(INCLEN) (2000) History: The INCLEN trust.
2009(29 June). Available: http://www.inclentrust.
org/index.php?option = content&task = view&i-
d = 284&Itemid = 338. Accessed: 3 July 2009.
42. Indian Clinical Epidemiology Network. About
IndiaCLEN. 2009(6/29). Available: http://www.
indiaclen.org/indiaclen.htm. Accessed: 3 July
2009.
43. INDEPTH Network (2009) INDEPTH network.
2009(6/29). Available: http://www.indepth-net-
work.org/. Accessed: 3 July 2009.
44. The Cochrane Collaboration (2009) The Co-
chrane collaboration. 2009(6/29). Available:
http://www.cochrane.org/. Accessed: 3 July
2009.
45. Lee K (2008) The World Health Organization
(WHO). Oxford: Routledge.
46. Yamey G (2002) WHO in 2002: Why does the
world still need WHO? BMJ 325: 1294–1298.
47. World Health Organization (WHO) (2009) Draft
proposed programme budget. Available: http://
www.who.int/gb/ebwha/pdf_files/MTSP-08-
13-PPB-10-11/PPB-1en.pdf. Accessed: 10 Sep-
tember 2009.
48. The Lancet (2009) The Global Alliance for
Chronic Diseases. The Lancet 373(9681):
2084–2084.
49. World Health Organization Maximizing Positive
Synergies Collaborative Group (2009) An assess-
ment of interactions between global health
initiatives and country health systems. The Lancet
373(9681): 2137–2169.
50. Jamison EA, Jamison DT, Hanushek EA (2007)
The effects of education quality on income
growth and mortality decline. Econ Educ Rev
26: 771–788.
59
World Health Report 2012: No Health Without Research
Policy Forum
The Global Health System: Linking Knowledge with
Action—Learning from Malaria
Gerald T. Keusch1*, Wen L. Kilama2, Suerie Moon3, Nicole A. Szlezák3, Catherine M. Michaud4
1 Department of International Health, Boston University School of Public Health, Boston, Massachusetts, United States of America, 2 African Malaria Network Trust, Dar es
Salaam, Tanzania, 3 Sustainability Science Program, John F. Kennedy School of Government, Harvard University, Cambridge, Massachusetts, United States of America,
4 Harvard Initiative for Global Health, Harvard University, Cambridge, Massachusetts, United States of America
This is the third in a series of four articles
that highlight the changing nature of global
health institutions.
Introduction
Conducting basic research, translating it
into the development of new health tools,
and delivering products to patients in need
of them are core functions of an effective
global health system [1]. Yet performing
these functions is a particular challenge for
diseases that primarily affect the poor in
low-income countries, partly because ef-
forts to understand diseases and develop
tools to combat them are often detached
from efforts to deliver interventions. For
malaria, the global health system has
evolved over the past century to integrate
better the research, development, and
delivery (R&D&D) of new products to
treat and control the disease. This article
traces that evolution and extracts lessons
applicable to the many new challenges
currently facing the global health system.
Historically, global investment in ma-
laria research has been disproportionately
small relative to its disease burden.
Research funding in endemic countries
was seriously limited by resource and
capacity constraints, while funding agen-
cies in industrialized countries were pri-
marily concerned with domestic health
issues, with the important exception of
military needs to control malaria. Recent-
ly, however, global malaria R&D invest-
ments have increased dramatically, from
an estimated $84 million in 1993 [2] to
$323 million in 2004 [3], with a new focus
on malaria’s impacts on people in endemic
countries.
In malaria control, there has been a
concomitant shift from time-limited, cen-
The Policy Forum allows health policy makers
around the world to discuss challenges and
opportunities for improving health care in their
societies.
60 PLoS Medicine | www.plosmedicine.org
tralized efforts—often relying on single This article explores the changing
interventions—toward a more decentral- global health system for malaria research
ized, continuous effort using multiple and the delivery of research products to
approaches. Malaria is no longer seen those at risk, including the organizations
primarily as a biomedical problem, but and actors involved, and the arrangements
rather as a complex ecological system in that govern their interactions (for more
which humans, mosquitoes, and parasites about these actors and arrangements, see
are interconnected. Malaria has also in- the first article in this four-part series [1]).
creasingly been characterized as a ‘‘global’’ Following Alilio and colleagues [4], we
and regional rather than a national or local have divided the evolution of malaria
problem. This has led to changed concepts R&D&D into three periods (Table 1);
of governance. Such governance has although these divisions are somewhat
changed in two ways: (1) from an essentially arbitrary, they highlight major shifts in
‘‘top-down’’ process from international to the system’s development. Finally we
national or local players to an active address the lessons learned and speculate
interplay between local and global players, about the future.
and (2) from a system that centered on the
World Health Organization (WHO), with
Phase I. Late Nineteenth
little attention to national governments in
endemic countries, to one in which state Century through the 1950s:
and non-state actors cooperate across National Public Goods
multiple dimensions, emphasizing inclusion R&D
and engagement of local communities. The early driver of malaria research was
Today, for the first time, the principal the desire of the European colonial powers
constraints to malaria control may be more to protect their own nationals and the
political and managerial than financial or economic interests in their colonies. This
technical. investment led to many discoveries, in-
Citation: Keusch GT, Kilama WL, Moon S, Szlezák NA, Michaud CM (2010) The Global Health System: Linking
Knowledge with Action—Learning from Malaria. PLoS Med 7(1): e1000179. doi:10.1371/journal.pmed.1000179
Academic Editor: Gill Walt, London School of Hygiene and Tropical Medicine, United Kingdom
Published January 19, 2010
Copyright: � 2010 Keusch et al. This is an open-access article distributed under the terms of the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,
provided the original author and source are credited.
Funding: This work was supported by a grant to the Institutional Innovations in Global Health project by the
John F. Kennedy School of Government at Harvard University, Cambridge, Massachusetts (US) under its ‘‘Acting
in Time’’ program. Additional support was received from the Burroughs Wellcome Fund, Research Triangle
Park, North Carolina (US). The funders played no role in the decision to submit the article or in its preparation.
Competing Interests: The authors have declared that no competing interests exist.
Abbreviations: CDC, US Centers for Disease Control; DDT, dichloro-diphenyl-trichloroethane; DFID,
Department for International Development (UK); DNDi, Drugs for Neglected Diseases Initiative; GFATM, Global
Fund for AIDS, Tuberculosis and Malaria; GND, Great Neglected Diseases Biomedical Research Network; IoWH,
Institute for One World Health, ITN, insecticide-treated bed net; MIM, Multilateral Initiative on Malaria; MMV,
Medicines for Malaria Venture; MVI, Malaria Vaccine Initiative; NGO, non-governmental organization; NIH, US
National Institutes of Health; PDP, Product Development Partnership; PEPFAR, President’s Emergency Program
for AIDS Relief; PMI, President’s Malaria Initiative; R&D, research and development; R&D&D, research and
development and delivery; RBM, Roll Back Malaria; UNICEF, United Nations Children’s Fund; USAID, United
States Agency for International Development; WHO, World Health Organization
* E-mail: keusch@bu.edu
Provenance: Commissioned, externally peer reviewed.
1 January 2010 ::| Volume
iMedPub 7 Collections
Thematic | Issue 1 | e1000179
:: Volume 2
 World Health Report 2012: No Health Without Research

Table 1. Evolution of institutional arrangements for malaria R&D.

Phase Purpose of R&D Institutions Targeted End-Users Funding Targeted Diseases

I: Late Nineteenth Century National public goods Industrialized countries Public, private Malaria, yellow fever
through the 1950s
II: 1960s–80s International health programs (e.g., TDR, Developing countries Public, philanthropic Malaria and other tropical
Fogarty International Center, Rockefeller infectious diseases
Foundation)
III: 1990s–2000s Global health partnerships neglected Developing countries Public, philanthropic, Malaria, tuberculosis, HIV/
disease R&D (e.g., PDPs) private sector AIDS and neglected tropical
infectious diseases
IV: The Future Global public goods for global health Global Public, philanthropic, All types
private sector

doi:10.1371/journal.pmed.1000179.t001

cluding identification of the cause, vector,
and transmission cycle of malaria. Later,
when malaria debilitated allied soldiers in
World War II (WWII), military needs
drove malaria R&D. None of the principal
malaria medicines of the twentieth century
would have been discovered without
military R&D [5–8]. Even insecticide-
treated bed nets (ITNs) [9] and household
spraying with DDT were used effectively
by the allied militaries in WWII [6,10].
During this long period, innovation
followed a distinct trickle-down pattern.
Researchers in the North produced knowl-
edge to serve their own national needs,
and only later was it applied for the benefit
of low-income countries. While these
R&D efforts ultimately created global
benefits, the institutions that guided and
benefited from the research were in rich
countries. The drawback for low-income
countries was that tools developed for
militaries of the North were not necessarily
well-suited for civilians in the South. Cost
was not a major issue for the North, and
because antimalarial drugs were targeted
at adults, testing in children was a low
priority, although children account for
most malaria deaths. As the US Military
Infectious Diseases Research Program
recently pointed out, ‘‘Preventing death
in children and keeping soldiers healthy
and effective are distinct goals requiring
different research strategies’’ [11].
Delivery
The association between swamps, mos-
quitoes, and malaria has long been appre-
ciated [12]. By the time of the Roman
Empire, bed nets, decoy animals to attract
mosquitoes, swamp drainage, and housing
prohibitions in mosquito breeding areas
were used to control malaria. The cluster-
ing of ‘‘marsh fever’’ among those living
near smelly swamps led to the miasma
theory of disease, that foul ‘‘mala aria’’ (bad
air in Italian, from which the name malaria
derives) from decomposed matter (mias-
mas) was the cause. Efforts at control were
local and often misguided, yet sometimes
effective, for example drainage of swamps
and closing of mill ponds in the US in the
nineteenth century.
Evidence-based systematic attempts to
control malaria at a population scale date
from the beginning of the twentieth
century, and were based on the under-
standing of the transmission cycle and
recognizing quinine’s therapeutic value.
Control programs were used in large,
expensive works projects threatened by
malaria (and yellow fever) and targeted at
workers and managers from industrialized
countries, such as the Suez and Panama
Canal projects. Multiple strategies were
adopted, including manual clearance of
mosquito larvae, removal of breeding sites,
leveling and oiling of roads to eliminate
water pools, use of clothing to prevent
mosquito bites at dusk, burning of pyre-
thrum indoors, larviciding with chemicals,
treatment with quinine, use of window
screens, and collection of indoor resting
mosquitoes post-feeding. These strategies
were effective in the limited scope of the
effort. In the late 1930s, Fred Soper of the
Rockefeller Foundation and 40,000 work-
ers in Brazil successfully eradicated Anoph-
eles gambiae, which had recently been
imported, using pyrethrum spraying, lar-
viciding with Paris Green (copper acet-
oarsenite), and elimination of breeding
sites [13].
In 1946, the US Centers for Disease
Control (CDC) was established in Atlanta,
Georgia as the successor to the WWII
Malaria Control in War Areas Agency,
primarily to eradicate malaria in the
southern states. According to a history of
the CDC, ‘‘Pursuit of malaria was by far
the most absorbing interest of CDC during
its early years, with over 50 percent of its
personnel engaged in it’’ [14]. Malaria
transmission in the US was eliminated.
Phase II. 1960s–1980s: The
International Health
Perspective
Phase I had involved nationally focused
programs concerned with domestic social
well-being (malaria in the southern US),
economic gain (the canal projects), or
military needs (wars in malaria zones).
But subsequent years witnessed a phase of
internationalization in public health, with
rapid decolonization, the launch of na-
tional foreign aid initiatives amidst height-
ened Cold War tensions, and new faith in
the potential of science and technology.
R&D
In phase II, the relevant actors were
increasingly viewing the world as interde-
pendent [15], with greater emphasis on
international health needs. The Special
Programme for Research and Training in
Tropical Diseases (TDR) was established
within WHO in 1975, and played a key
role in building malaria research capacity
in developing countries, particularly in
Africa where few malaria researchers
existed at the time of political indepen-
dence for the former colonies. TDR also
established international networks of aca-
demic centers for tropical disease research,
a model that the public–private product
development partnerships would later
emulate [16]. These national–internation-
al partnerships proved to be essential for
the development of critical new tools,
including artemisinin combination therapy
and ITNs still in use today [9,17,18].
During this era the private foundations
reemerged as a force, for example the
Great Neglected Diseases (GND) of Man-
kind Biomedical Research Network
launched by Kenneth Warren of the
Rockefeller Foundation [19]. Catalyzed
by GND funds, a stream of young
scientists from developing and developed
nations were attracted to work with

PLoS Medicine | www.plosmedicine.org 2 January 2010 | Volume 7 | Issue 1 | e1000179

iMedPub :: Thematic collections :: Volume 2 61
World Health Report 2012: No Health Without Research
established researchers in this global
network on problems such as malaria in
the laboratory and the field.
Compared to the previous period,
institutions for R&D were broader in
scope, more international, and targeted
low-income country needs. However, as
the Rockefeller Foundation’s Tim Evans
later noted, GND produced ‘‘improved
basic knowledge about poorly understood
tropical diseases….[but] no explicit strat-
egy to translate new knowledge into drug
or vaccine development’’ [20]. By the late
1980s the GND was winding down, TDR
was seriously underfunded for its broad
mandate, and the pharmaceutical industry
had largely withdrawn from tropical
infectious disease research. The existing
R&D system could not meet the vast
health needs of low-income countries. The
research enterprise had simultaneously
succeeded and failed.
Delivery
In 1955, based on the wartime success
of DDT, WHO initiated an ambitious
attempt to eradicate malaria by eliminat-
ing the vector. However, by 1969 the
Global Malaria Eradication Programme
(in fact it was never global, excluding
much of sub-Saharan Africa from the
outset) was considered to have failed.
The program had nevertheless achieved
considerable success in 25 countries in
Africa, Asia, Europe, North America, and
the Caribbean, primarily relatively rich
and island countries and a few poor
countries with good health infrastructure
and seasonal malaria [21]. There were
many reasons to give up the effort,
including donor funding fatigue, local
resistance to the imposition of control
measures, insecticide resistance, and the
difficulty of mosquito eradication in many
ecosystems. Efforts reverted to control
[22], and with the momentum for primary
health care and the 1978 Alma Ata
declaration calling for ‘‘Health for All’’
by the year 2000, malaria control was
incorporated into primary care programs.
With the loss of visibility, combined
with waning global interest and dwindling
funding for research and control, malaria
was soon overshadowed by the emerging
HIV/AIDS epidemic in the 1980s. Be-
tween 1975 and 1994, malaria control was
financed mostly as bilateral assistance to
endemic countries, with yearly contribu-
tions of less than $20 million, for an
estimated $364 million over 20 years. The
impact of the ‘‘Silent Spring’’ movement
and the near total cessation of DDT
production [23], accompanied by rapid
spread of resistance to the nearly ideal
PLoS Medicine | www.plosmedicine.org
62
antimalarial chloroquine, contributed to
the resurgence of malaria, including in
places where it had formerly been con-
trolled. Malaria research and control had
itself become a neglected initiative.
Phase III. 1990s to the Present:
Global Health and Malaria
Research and Control
Many factors have led to the consider-
ation of health as a global imperative over
the past two decades, particularly the
disparate burden of HIV in poor nations
and the AIDS activist movement, which
revived a human rights approach to health
care. With these changes in the value
system and increasing attention to the
concept of global public goods, malaria
R&D and delivery have become priorities
again.
R&D
In 1990, the independent Commission
on Health Research for Development
argued in its seminal report, Health Re-
search-Essential Link to Equity in Development,
that research had long been ‘‘under-
recognized and neglected’’ as a tool to
mitigate growing global inequities in
public health [24]. With increasing glob-
alization of trade, travel, information, and
disease, health in general and R&D in
particular were increasingly framed as
‘‘global’’ rather than ‘‘international,’’ con-
cerned with ‘‘the health needs of the
people of the whole planet above the
concerns of particular nations’’ [25]. This
change also underscored ‘‘the growing
importance of actors beyond governmen-
tal or intergovernmental organizations’’
[24]. The report of the WHO Ad Hoc
Committee on Health Research Relating
to Future Intervention Options set prior-
ities for global health research and recom-
mended an approach to allocate research
funding [26]. Because confidence in the
leadership at WHO among key global
players was at an all-time low, an inde-
pendent organization, the Global Forum
for Health Research, was established in
Geneva to catalyze and monitor invest-
ments in research relevant to the world’s
poorest people [27].
Malaria was a good example of a
neglected disease in 1990, as both public
and private actors had largely retreated
from malaria research, even though drug-
resistant malaria was spreading across the
globe. In 1996, Harold Varmus, then
Director of the US National Institutes of
Health (NIH), concluded that malaria
R&D merited increased funding because
of its global impact and the potential for
3 January 2010 | Volume 7 | Issue 1 | e1000179
scientific progress with increased funding.
The same year, the UK-based Wellcome
Trust reported on the domination of
malaria research by scientists from the
North [2]. In part because WHO was not
deemed to have sufficient scientific depth
or resources to address these disparities,
the Multilateral Initiative on Malaria
(MIM) was established in 1998 as a joint
effort of northern country health research
and bilateral aid agencies [28]. MIM
rapidly improved channels for information
flow between researchers in the North and
South through data sharing and internet-
based library access; established a reposi-
tory for patient-, parasite-, and vector-
derived chemical entities and genomes for
research; provided research funds for
African scientists through TDR; and
initiated a regular Pan African Malaria
conference to bridge the malaria research
and control communities. The MIM
Secretariat, based successively at the Well-
come Trust, the Fogarty International
Center at NIH, and the Karolinska
Institute, moved to its first African home
in Tanzania in 2006, although securing
long-term financial support remains a
vexing problem. Enlightened leadership
and commitment from the elite science
funding agencies was the essential catalyst
behind these changes.
By the late 1990s, the increasing self-
confidence of senior African scientific
leaders and maturation of young African
malaria researchers into senior leaders,
and recognition of their contributions to
knowledge generation, placed them at the
center of research planning and progress
in malaria. Trainees in basic sciences,
entomology, epidemiology, biostatistics
and bioinformatics, sociology, behavioral
sciences, and public health now could play
key roles and become deeply involved in
institutional leadership and management.
The creation of MIM also pushed forward
the visibility of malaria as a global
problem, engaged leading research fund-
ing institutions in the North, and support-
ed a global research network to link
research to control. These actions laid
the immediate groundwork for the launch
of public-private–product development
partnerships (PDPs) for malaria.
With the support of major foundations,
PDPs emerged in the 1990s to address a
glaring failure of existing institutional
arrangements for R&D—that market-in-
centives had proven insufficient to drive
investment in new tools for neglected
diseases [29]. PDPs have redefined roles
and expectations, with the public sector
playing a stewardship and funding role,
the private sector contributing materials
iMedPub :: Thematic Collections :: Volume 2

and know-how, and private philanthropy
investing a significant share of the funds.
New PDPs doing malaria research have
been created, including Medicines for
Malaria Venture (MMV), Malaria Vac-
cine Initiative (MVI), Drugs for Neglected
Diseases Initiative (DNDi), and the Insti-
tute for OneWorld Health (iOWH). Thus
far, two new fixed-dose combination
malaria treatments based on artemisinin
derivatives (DNDi) [30], a lower-cost
synthetic method to produce artemisinin
(iOWH) [31], and a licensed pediatric
formulation of an artemisinin combination
drug and a pipeline of new compounds in
development to address emerging drug
resistance (MMV) [32] have resulted.
Though relatively new, PDPs have rein-
vigorated product development for malar-
ia and other neglected diseases [33].
Furthermore, by placing affordability and
accessibility at the center of their missions,
they promote the concept of health R&D
as a global public good [34]. Importantly,
PDPs are explicitly expected to develop
products well-adapted for use in low-
income countries [35]. Nevertheless, the
PDPs are relatively young, and it remains
to be seen if they can efficiently deliver on
their early promise over the long haul.
Local initiatives are now apparent as
well. The African Malaria Network Trust
[36] and the Malaria Clinical Trials
Alliance [37] are African-led initiatives to
strengthen malaria-related R&D capaci-
ties in Africa. They collaborate with
northern partners and malaria PDPs to
support African research institutions to
develop products up through Phase III
clinical trials. This reflects the recognition
of African malaria scientists with the skills
to conduct basic and clinical research and
compete for funding, and reinforces the
new norm that neglected disease research
should involve endemic-country scientists
and be targeted to meet low-income
country needs.
Delivery
In 1992, the WHO Ministerial Confer-
ence on Malaria in Amsterdam [38]
outlined a broad set of measures to reduce
the burden of malaria, including early
diagnosis and treatment, selective and
sustainable preventive measures, early
identification of epidemics and rapid
responses to contain them, and strength-
ening of local capacities in basic and
applied research. Much of this agenda
was supported by northern research agen-
cies, not WHO. To reestablish a central
role for WHO, the newly elected Director
General, Gro Bruntland, in 1998 estab-
lished the Roll Back Malaria (RBM)
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
Partnership as a ‘‘Cabinet Project’’ report-
ing directly to her [39]. It signaled a new
order of business at WHO—a global
program partnership—responding to the
widely held belief that malaria could not
be controlled by governments and WHO
alone but needed multiple public and
private partners to succeed. The World
Bank, UNICEF, DFID, USAID, founda-
tions, NGOs, and the private sector
quickly joined RBM, together with na-
tional governments and their malaria
control agencies.
RBM’s mandate was to seek greater
funding, raise awareness of malaria as a
global problem, harmonize activities of the
partners and support development of
effective national programs. However,
heavy-handed management by the Secre-
tariat at WHO led to dissatisfaction with
progress among the partners and with the
manner in which they were being en-
gaged. An external evaluation, required by
the partners, damned with faint praise the
accomplishments of the first four years,
noting that advocacy was not supported by
data, decision-making was inefficient, ac-
countability within the Partnership was
lacking, reductions in the malaria burden
had been ‘‘slower than anticipated,’’
countries ‘‘receive inadequate and some-
times inconsistent technical advice,’’ and
insufficient attention was given to ‘‘multi-
sectoral approaches, particularly as re-
gards private sector activity’’ [40]. Since
then, RBM’s performance has improved.
RBM’s recently issued Global Malaria
Action Plan outlining strategies, costs,
goals, and timelines is a major accom-
plishment, with multiple partner inputs.
RBM is commissioning an independent
evaluation to appraise the ‘‘governance;
management; ability to convene, coordi-
nate and harmonize RBM partners and
stakeholders; and its impact on malaria
efforts at country level’’ [41].
The Global Fund to Fight AIDS,
Tuberculosis and Malaria (GFATM) was
founded in 2002 as a new international
financing mechanism for these three
diseases and to harness the capacities of
public, private, and civil society actors at
both global and national levels [42].
GFATM was based on the premise that
success depended on the involvement of
multiple state and non-state actors. It was
explicitly created as a public–private entity
outside of and independent of the UN
system. Furthermore, the concentration of
funds from multiple public and private
sources in GFATM (which totaled $1.6
billion for malaria control between 2002
and 2007) was intended to decrease prior
fragmentation of funding schemes. Despite
4 January 2010 | Volume 7 | Issue 1 | e1000179
increasing multilateralism from many
global health actors, the US has been a
reluctant partner in GFATM since
its founding, preferring to invest most of
its significantly increased commitment
through bilateral programs, such as the
President’s Emergency Fund for AIDS
Relief (PEPFAR) and the President’s
Malaria Initiative (PMI). Total US pledges
to GFATM from its inception through
January 2009 amount to $4 billion. The
Obama administration’s budget request
for 2010 includes a 36% increase in
malaria support, a 2.5% increase for
PEPFAR, but no change in funding for
GFATM, thus continuing the major
emphasis on bilateral program support.
Phase IV. The Future: Lessons
Learned and Global Public
Goods for Global Health
The past 30 years has witnessed signif-
icant shifts in the types of actors and the
roles they play in malaria research and
control, with gradually increasing integra-
tion of the R&D&D communities. With
these changes, a number of new modes of
operation have been established that seem
certain to continue, such as: (1) a more
central role for endemic-country research-
ers in an increasingly globalized research
system; (2) direct funding to local research-
ers and institutions; (3) the involvement of
affected communities not only as targets of
interventions but as co-producers of re-
sults; (4) new actors taking on tasks
formerly vested in WHO; and (5) new
PDPs to drive research to unmet needs
and new product development. These
developments bode well for achieving the
prospects for new, effective, adapted, and
affordable tools for malaria
A new challenge to the global health
system is the recent decision by the Bill
and Melinda Gates Foundation, in addi-
tion to its support of malaria PDPs, to
place malaria eradication back on center
stage [43]. Not all experts agree that
malaria eradication is feasible or desirable
[44]. Regardless of the validity of the
criticism, it is necessary to continue to
develop and apply new tools to eliminate
malaria as a significant public health
problem, as disease reduction is the
necessary antecedent to any attempt at
eradication of the parasite.
New mechanisms for partnerships
among global and national organizations
have pioneered new approaches to financ-
ing and governance of programs, such as
GFATM and RBM, along with major
bilateral investments, such as PEPFAR
and PMI. In addition, the interests of the
63
World Health Report 2012: No Health Without Research
science community now connect to product
development to tackle growing drug and
insecticide resistance. These innovations
have focused on neglected infectious dis-
eases that by definition affect only develop-
ing countries. Such innovations leave the
question unanswered of who contributes to
and who benefits from R&D for diseases
that affect all countries, such as noncom-
municable diseases, including cancer, car-
diovascular disease and stroke, diabetes,
and obesity [45–50] The challenge of
building effective new R&D&D arrange-
ments in the twenty-first century for all
health needs of all people should be
informed by past developments in malaria.
The most relevant developments to draw
upon are the challenges of filling the
institutional gaps within the global health
system to link R&D with delivery; of
effectively connecting local and interna-
tional researchers; and finally of ensuring
support for the generation of global public
goods for global health. A step in that
direction has recently been taken by the
leading national research agencies of a
number of developed and developing
nations. These have come together to form
the Global Alliance for Chronic Diseases
[50], with a pledge to invest in research in a
coordinated manner, and to scale up
promising interventions to achieve targeted
goals of disease reduction.
Conclusions
In this paper, we have reviewed the
century-long effort to research malaria, to
References
1. Szlezá k NA, Bloom BR, Jamison DT,
Keusch GT, Michaud CM, et al. (2010) The
Global Health System: Actors, Norms, and
Expectations in Transition. PLoS Med 7:
e1000183. doi:10.1371/journal.pmed.1000183.
2. Anderson J, Maclean M, Davies C (1996) Malaria
research: An audit of international activity. The
Wellcome Trust Unit for Policy Research in
Science and Medicine. Available: http://www.
wellcome.ac.uk/stellent/groups/corporatesite/@
sitestudioobjects/documents/web_document/
wtd003220.pdf. Accessed 15 September 2008.
3. Malaria Research and Development Alliance
(2005) Malaria research and development:
An assessment of global investment. Avail-
able: http://www.malariaalliance.org/PDFs/
RD_Report_complete.pdf. Accessed 15 Septem-
ber 2008.
4. Alilio MS, Bygbjerg IC, Breman JG (2004) Are
multilateral malaria research and control pro-
grams the most successful? Lessons from the past
100 years in Africa. Am J Trop Med Hyg
71(Suppl 2): 268–278.
5. Burckhalter JH (1950) Modern antimalarial
drugs. Trans Kansas Acad Sci 53: 433–440.
6. Ockenhouse C, Magill A, Smith D, Milhous W
(2005) History of U.S. military contributions to
the study of malaria. Military Med 170: 12–16.
7. Russell PF (1963) Introduction. In Preventive
Medicine in World War II, Volume VI Commu-
PLoS Medicine | www.plosmedicine.org
64
develop tools for control, and to imple-
ment them. It is clear from our review that
support for and inclusion of local research
institutions in global health research is
essential to develop well-adapted health
tools and to strengthen collaborations
between global and local actors in imple-
mentation. Such support and inclusion is a
necessary precursor to the emergence of
stronger and more integrated global re-
search, development, and delivery, which
we have termed the R&D&D system. The
role of WHO in this global system must
evolve as a partnership with other actors.
Building an effective global health system
takes times. It required decades to build up
research capacity in malaria-endemic
countries to the present level, when local
researchers can play an integral role in
malaria R&D&D. Investments in capacity
building in other relatively neglected areas,
such as noncommunicable diseases, must
begin today if we expect similar dividends
in the future.
R&D must connect closely to the
challenges of implementation. The histor-
ical divide between academic research,
industry development, and those who
implement in the real world cannot
continue if ‘‘acting in time,’’ translating
knowledge into action, is a critical goal.
Those in the R&D world must understand
what the control community has to deal
with, and the latter need to know what is
in the R&D pipeline in order to identify
the delivery constraints that must be
solved.
nicable Diseases: Malaria. United States Army,
Medical Department. Washington, D.C.: Office of
the Surgeon General, Department of the Army.
8. Honigsbaum M (2001) The fever trail: In search
of the cure for malaria. New York: Farrar, Straus
and Giroux.
9. Binka F, Akweongo P (2006) Prevention of
malaria using ITNs: Potential for achieving the
millennium development goals. Curr Mol Med 6:
261–267.
10. Bruce-Chwatt LJ, ed (1985) Essential malariology.
2nd edition. London: William Heinemann.
11. United States Army Medical and Materiel
Command (2005) What makes the Military
Infectious Diseases Research Program (MIDRP)
Unique? Bethesda (Maryland): Military Infectious
Diseases Research Program, Available: https://
www.midrp.org/pdf_files/INFO_PDF/midrpunique.
pdf. Accessed 12 May 2008.
12. CDC (2004) The history of malaria, an ancient
disease. Available: http://www.cdc.gov/malaria/
history/index.htm. Accessed 8 February 2009.
13. Soper FL, Wilson DB (1943) Anopheles gambiae in
Brazil, 1930 to 1940. New York: The Rockefeller
Foundation. 262 p.
14. CDC (2009) Our History - Our Story. Available:
http://www.cdc.gov/about/history/ourstory.
htm. Accessed 8 February 2009.
15. Keohane RO, Nye JS (1973) Power and interde-
pendence. Survival 15: 158–165.
5 January 2010 | Volume 7 | Issue 1 | e1000179
Enlightened leadership within organiza-
tions comes with a commitment to scaling
up the level of R&D and capacity-building
investments, harnessing the potential
gained from connecting researchers in
the North and the South, and articulating
the messages to decision makers and the
general public to gain support. The new
Global Alliance for Chronic Disease
appears to have heard the message, as
these issues are highlighted in its mandate.
Finally, the case study of malaria
suggests that a multiplicity of partnership
models is useful, particularly for diseases
that require multiple interventions and
continuing R&D. The global health sys-
tem of the future must identify ways to
include those who suffer from diseases,
those who contribute to R&D, and those
who deliver interventions, sharing the
responsibility to link better knowledge
with action for those in need.
Author Contributions
ICMJE criteria for authorship read and met:
GTK WLK SM NAS CMM. Wrote the first
draft of the paper: GTK. Contributed to the
writing of the paper: GTK WLK SM NAS
CMM. Co-planned and organized the work on
which this and the other papers in the global
health system series are based: GTK. An
original conceptualizer of the study, and co-PI
on the grant supporting it: NAS. Involved in
conceptualization and initial drafting of this
paper, and provided input throughout the
writing process: NAS.
16. World Health Organization, Special Programme
for Research and Training in Tropical Diseases
(WHO/TDR) (2007) Making a Difference: 30
Years of Research and CapacityBuilding in
Tropical Disease. Geneva: WHO, Available at:
http://www.who.int/tdr/about/history_book/
pdf/anniversary_book.pdf. Accessed 1 May
2008.
17. Alonso P, Lindsay SW (1991) The effect of
insecticide-treated bed nets on mortality of
Gambian children. Lancet 337: 1499.
18. Lengeler C, Snow RW (1996) From efficacy to
effectiveness: Insecticide-treated bednets in
Africa. Bull WHO 74: 325–332.
19. The Rockefeller Foundation (1979) The Presidents
Review and Annual Report, 1979. Available: http://
www.rockfound.org/library/annual_reports/1970-
1979/1979.pdf Accessed 8 February 2009.
20. Evans TG (2002) Health-related global public
goods: Initiatives of the Rockefeller Foundation.
New York: United Nations Development Pro-
gramme, Office of Development Studies, Avail-
able: http://www.undp.org/ods/pdfs/evans.pdf.
Accessed 1 June 2008.
21. Packard RM (2007) The making of a tropical
disease: A short history of malaria. Baltimore
(Maryland): Johns Hopkins University Press. 320 p.
22. Scholtens RG, Kaiser FL, Langmuir AD (1972)
An epidemiologic examination of the strategy of
malaria eradication. Int J Epid 1: 15–24.
iMedPub :: Thematic Collections :: Volume 2

23. (2009) Silent Spring. Wikipedia. Available:
http://en.wikipedia.org/wiki/Silent_Spring. Ac-
cessed 24 September 2009.
24. Commission on Health Research for Develop-
ment (1990) Health research: Essential link to
equity in development. New York: Oxford
University Press.
25. Brown TM, Cueto M, Fee E (2006) The World
Health Organization and the transition from
international to global public health. Am J Public
Health 96: 62–72.
26. (1996) Investing in health research and develop-
ment: Report of the Ad Hoc Committee on
Health Research Relating to Future Intervention
Options. Document TDR/Gen/96.1. Geneva:
WHO.
27. Ramsay S (2001) No closure in sight for the 10/
90 health-research gap. Lancet 358: 1348.
28. Heddini A, Keusch GT, Davies CS (2004) The
multilateral initiative on malaria: past, present
and future. Am J Trop Med Hyg 71(Suppl 2):
279–282.
29. Trouiller P, Olliaro P, Torreele E, Orbinski J,
Laing R, et al. (2002) Drug development for
neglected diseases: A deficient market and a
public-health policy failure. Lancet 359:
2188–2194.
30. World Health Organization. New type of R&D
cooperation spawns malaria drugs. Available at:
http://www.who.int/bulletin/volumes/84/1/
news20106/en/ Accessed 15 December 2009.
31. The Artemisinin Enterprise. Semisynthetic artemisi-
nin through fermentation. Available at: http://www.
rollbackmalaria.org/extra/artemisininenterprise/
project1.html Accessed 15 December, 2209.
32. MMV Portfolio, Q3, 2009. Available at: http://
www.mmv.org/IMG/pdf/MMV_Portfolio__Q3_
2009-_FINAL.pdf Accessed 15 December 2009.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
33. Moran M, Ropars A, Guzman J, Garrison C (2005)
The new landscape of neglected disease drug
development. London: The Wellcome Trust and
The London School of Economics and Political
Science, Available: http://www.wellcome.ac.uk/
stellent/groups/corporatesite/@msh_publishing_
group/documents/web_document/wtx026592.
pdf. Accessed 9 July 2008.
34. Shotwell SL (2007) Product development and IP
strategies for global health product development
partnerships. Chapter 12.9 in: Intellectual Prop-
erty Management in Health and Agriculture
Innovation. A Handbook of Best Practices
Kratiger A, Mahoney RT, Nelsen L, et al., eds.
DavisCalifornia, United States: MIHR, Oxford
UK and PIPRA.
35. Frenk J (2010) The Global Health System:
Strengthening National Health Systems as the
Next Step for Global Progress. PLoS Med 7:
e1000089. doi:10.1371/journal.pmed.1000089.
36. (2009) African Malaria Network homepage.
Available: http://www.amanet-trust.org/. Ac-
cessed 17 February 2009.
37. (2009) The Malaria Clinical Trials Alliance
homepage. Available at: http://www.indepth-
network.net/mcta/mctaindex.htm. Accessed 17
February 2009.
38. Trigg PI, Kondrachine V (1998) Commentary:
malaria control in the 1990’s. Bull World Health
Organ 76: 11–16.
39. (1999) Roll Back Malaria. Report of the Director
General. Available at: http://apps.who.int/gb/
archive/pdf_files/EB103/ee6.pdf Accessed 16
December 2009.
40. (2002) Final Report of the External Evaluation of
Roll Back Malaria. Achieving Impact: Roll Back
Malaria in the Next Phase. Available at: http://
6 January 2010 | Volume 7 | Issue 1 | e1000179
rbm.who.int/cmc_upload/0/000/015/905/ee_toc.
htm Accessed 16 December 2009.
41. (2009) Independent evaluation of the Roll Back
Malaria Partnership. Request for Proposals. Avail-
able at: http://www.rollbackmalaria.org/partnership/
secretariat/docs/RBMSEC2009INF112.pdf Accessed
16 December 2009.
42. (2009) The Global Fund homepage. Available:
http://www.theglobalfund.org/en/about/. Ac-
cessed 8 February 2009.
43. Bill & Melinda Gates Foundation (2007) Bill and
Melinda Gates call for new global commitment to
chart a course for malaria eradication. Available:
http://www.gatesfoundation.org/press-releases/
Pages/course-for-malaria-eradication-071017-2.
aspx. Accessed 8 February 2009.
44. Greenwood B (2009) Can malaria be eliminated?
Trans R Soc Trop Med Hyg 103(Suppl): S2–S5.
45. World Health Organization (2005) Preventing
chronic diseases: A vital investment. Geneva:
World Health Organization, Available: http://
www.who.int/chp/chronic_disease_report/en/
index.html Accessed 21 November 2008.
46. Horton R (2005) The neglected epidemic of
chronic disease. Lancet 366: 1514.
47. Fuster V, Voute J (2005) MDGs: Chronic diseases
are not on the agenda. Lancet 366: 1512–1514.
48. Strong K, Mathers C, Leeder S, Beaglehole R
(2005) Preventing chronic diseases: How many
lives can we save? Lancet 366: 1578–1582.
49. Horton R (2007) Chronic diseases: the case for
urgent global action. Lancet 370: 1881–1882.
50. Beaglehole R, Ebrahim S, Reddy S, Voute J,
Leeder S, et al. (2007) Prevention of chronic
diseases: A call to action. Lancet 370: 2152–2157.
51. Global Alliance for Chronic Diseases. Available
at: http://www.ga-cd.org/ Accessed 16 Decem-
ber 2009.
65
World Health Report 2012: No Health Without Research

Guidelines and Guidance

1

Setting Research Priorities to Reduce Almost One Million

Deaths from Birth Asphyxia by 2015
Joy E. Lawn1*, Rajiv Bahl2, Staffan Bergstrom3, Zulfiqar A. Bhutta4, Gary L. Darmstadt5, Matthew Ellis6,
Mike English7, Jennifer J. Kurinczuk8, Anne C. C. Lee9, Mario Merialdi10, Mohamed Mohamed11, David
Osrin12, Robert Pattinson13, Vinod Paul14, Siddarth Ramji15, Ola D. Saugstad16, Lyn Sibley17, Nalini
Singhal18, Steven N. Wall19, Dave Woods20, John Wyatt21, Kit Yee Chan22", Igor Rudan23"
1 Saving Newborn Lives/Save the Children, Cape Town, South Africa, 2 Department for Child and Adolescent Health and Development, World Health Organization,
Geneva, Switzerland, 3 Division of Global Health, Karolinska Institutet, Stockholm, Sweden, and Averting Maternal Death and Disability Program, Columbia University, New
York, New York, United States of America, 4 Division of Women & Child Health, the Aga Khan University, Karachi, Pakistan, 5 Family Health Division, Global Health Program,
Bill & Melinda Gates Foundation, Seattle, Washington, United States of America, 6 Community Child Health Partnership, Southmead Hospital, Bristol, United Kingdom,
7 KEMRI–Wellcome Trust Programme, Centre for Geographic Medicine Research–Coast, Nairobi, Kenya, and Department of Paediatrics, University of Oxford, Oxford,
United Kingdom, 8 The National Perinatal Epidemiology Unit, University of Oxford, Oxford, United Kingdom, 9 Department of International Health, Johns Hopkins
Bloomberg School of Public Health, Baltimore, Maryland, United States of America, 10 Department of Reproductive Health and Research, World Health Organization,
Geneva, Switzerland, 11 George Washington University, Washington, D.C., United States of America, 12 Centre for International Health and Development, UCL Institute of
Child Health, London, United Kingdom, 13 MRC Maternal and Infant Heath Care Strategies Research Unit at the University of Pretoria, Pretoria, South Africa,
14 Department of Pediatrics, All India Institute of Medical Sciences, New Delhi, India, 15 Department of Pediatrics, Maulana Azad Medical College, New Delhi, India,
16 Department of Pediatric Research, Oslo University Hospital Rikshospitalet, University of Oslo, Norway, 17 Nell Hodgson Woodruff School of Nursing, Atlanta, Georgia,
United States of America, 18 Department of Pediatrics, University of Calgary, Calgary, Alberta, Canada, 19 Saving Newborn Lives/Save the Children, Washington, D.C.,
United States of America, 20 University of Cape Town and the Perinatal Education Programme, Observatory, South Africa, 21 Centre for Philosophy, Justice and Health,
University College of London, London, United Kingdom, and Center for Women’s Health, University College of London, London, United Kingdom, 22 Nossal Institute of
Global Health, University of Melbourne, Melbourne, Australia, 23 Croatian Centre for Global Health, University of Split Medical School, Split, Croatia, and the Centre for
Population Health Sciences, The University of Edinburgh Medical School, Edinburgh, United Kingdom

Introduction The terms and definitions used to describe a baby affected by

The Millennium Development Goals (MDGs), ratified by
almost every country in the world, have catalyzed policy attention
and investment for child survival (MDG 4) and maternal health
(MDG 5) [1]. MDG 4 aims for a two-thirds reduction in deaths of
children under 5 years of age between 1990 and 2015. Despite
almost no progress for MDG 4 on a global level during the 1990s,
there has been increasingly rapid progress with several recent
landmark achievements since about 2005. The number of child
deaths has been reduced to about 8 million per year, despite the
continuing increase in the global child population [2,3], and a
number of low-income countries are now on track for the goal [3].
On the African continent, which has had the slowest progress,
several countries have moved from the ‘‘no progress’’ to the ‘‘rapid
progress’’ group, and two low-income African countries (Eritrea
and Malawi) are on track to achieve their MDG 4 goal [4,5].
Global Burden of ‘‘Birth Asphyxia’’
Most of the child mortality reduction in recent decades, however,
is attributed to progress in tackling infectious causes of deaths (such
as measles, malaria, pneumonia, and diarrhea) in post-neonatal
infants and children aged 1–4 years. Reductions in deaths that occur
in the neonatal period (the first 28 days after birth) have been
relatively limited. When the MDGs were signed in the year 2000,
approximately 37% of under-five child deaths occurred in the
neonatal period [6]; this has since risen to over 41% [7], a total of
3.6 million deaths. Mortality in the first week after birth, the early
neonatal period, has shown the least progress, with no measurable
change at global level in the last decade. If progress towards MDG 4
is to be accelerated, then urgent attention is required to reduce
neonatal deaths. It also links closely with advancing MDG 5 since
women’s health and health care, especially at the time of birth, are
major determinants of early neonatal deaths, especially those due to
preterm birth and complications at birth.
birth complications have evolved over time, driven not only by a
greater understanding of pathophysiology and clinical manifesta-
tions but also by increasing litigation in high-income countries [8].
‘‘Birth asphyxia’’ is an imprecise term; it was broadly defined by
the World Health Organization (WHO) in 1997 as the clinical
description of a newborn who ‘‘fails to initiate or maintain regular
breathing at birth’’ [9]. This term applies to an important clinical
condition—the need for resuscitation—but is not predictive of
outcome. Nor does it imply a particular causation (e.g.,
Citation: Lawn JE, Bahl R, Bergstrom S, Bhutta ZA, Darmstadt GL,
et al. (2011) Setting Research Priorities to Reduce Almost One Million Deaths
from Birth Asphyxia by 2015. PLoS Med 8(1): e1000389. doi:10.1371/journal.pmed.
1000389
Published January 11, 2011
Copyright: � 2011 Lawn et al. This is an open-access article distributed under
the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the
original author and source are credited.
Funding: JEL is funded by Saving Newborn Lives/Save the Children through a
grant from the Bill & Melinda Gates Foundation. IR received support as a
consultant of the Child Health and Nutrition Research Initiative during the
conduction of this study. There were no other sources of funding, and all co-
authors (except IR) volunteered their time to conduct this study. The funders had
no role in the study design, data collection, analysis, decision to publish or
preparation of the manuscript.
Competing Interests: Zulfiqar Bhutta and David Osrin are members of the
Editorial Board of PLoS Medicine. None of the other authors declare any
competing interests.
* E-mail: joylawn@yahoo.co.uk
" Joint senior authors.
Provenance: Commissioned; externally peer reviewed.
Abbreviations: AEA, average expert agreement; CAH, World Health Organiza-
tion Child and Adolescent Health and Development Department; CHNRI, Child
Health Nutrition Research Initiative; MDG, Millennium Development Goal; NE,
neonatal encephalopathy; NIH, US National Institutes of Health; RPS, research
priority score; World Health Organization.

66 PLoS Medicine | www.plosmedicine.org 1 January 2011 | ::

iMedPub Volume 8 | Issue
Thematic 1 | e1000389
Collections :: Volume 2

Summary Points
N Intrapartum-related neonatal deaths (previously called
‘‘birth asphyxia’’) are the fifth most common cause of
deaths among children under 5 years of age, accounting
for an estimated 814,000 deaths each year, and also
associated with significant morbidity, resulting in a
burden of 42 million disability adjusted life years (DALYs).
N This paper uses a systematic process developed by the
Child Health Nutrition Research Initiative (CHNRI) to
define and rank research options to reduce mortality
from intrapartum-related neonatal deaths by the year
2015, in order to advance Millennium Development Goal
(MDG) 4 for child survival.
N A list of 61 research questions was developed and scored
by 21 technical experts. The top one-third of the ranked
research investment options was dominated by delivery
(implementation) research, whilst discovery (basic sci-
ence) questions were not ranked highly, especially for
expected reduction of mortality and inequity in the short
time to 2015.
N Among the top four research questions, two relate to
generation of demand for facility care at birth with
specific mechanisms (such as transport and communi-
cation schemes, or financial incentives and conditional
cash transfers). The other two top ranked priorities relate
to use of community cadres and the roles they might
effectively play—for example, screening for complica-
tions or supportive transfer to facilities and companion-
ship at birth. The highest ranked discovery question
concerned the interaction of hypoxia and infection, and
the highest ranked epidemiologic question addressed
prediction of intrapartum hypoxic injury.
N This exercise highlights the need for current research
investments to focus on studies most likely to result in
accelerated progress towards MDG 4 and in the countries
where the most deaths occur.
intrapartum hypoxia) since the baby may not be breathing for
other reasons, such as prematurity. Three consensus statements
have recommended that terms such as ‘‘birth asphyxia’’,
‘‘perinatal asphyxia’’, ‘‘fetal distress’’, ‘‘hypoxic-ischemic enceph-
alopathy’’, or ‘‘post-asphyxial encephalopathy’’ should not be used
unless evidence of acute intrapartum causation is available [10–
12]. These consensus statements suggested the term ‘‘neonatal
deaths associated with acute intrapartum events,’’ which is
cumbersome. Since the late 1990s, the Scottish and UK
Confidential Enquiries have included the term ‘‘intrapartum-
related neonatal death,’’ which has also been used in a recent
supplement on the topic [8]. The terminology used in interna-
tional health estimates and policy has been slower to change, but
in this paper we use the new term, intrapartum-related neonatal
deaths.
Each year an estimated 814,000 children die of intrapartum-
related causes [13]. Intrapartum-related neonatal deaths are the
fifth most common cause of under-five child deaths after
pneumonia, diarrhea, preterm birth complications, and neonatal
infections [13]. They rarely feature on lists of child survival
priorities, compared to other conditions such as malaria that
account for fewer child deaths [14]. The burden of intrapartum
complications is underestimated if only liveborn babies are
considered since an additional 1.02 million stillbirths occur in
the intrapartum period [15], which accounts for approximately
one-third of the world’s total 3.2 million stillbirths [16]. However,
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
stillbirths are not included in MDG tracking or Global Burden
estimation (Figure 1). The Global Burden of Disease 2004 report
allocated 42 million disability adjusted life years (DALYs) to ‘‘birth
asphyxia’’, which is twice the number of DALYs allocated to
diabetes and around 75% of the DALYs for HIV/AIDS [17].
Mismatch of Burden and Research Investment
In evidence-based decision making, research investment would
be matched with burden. There is, however, a well-described
mismatch between burden and research investment, particularly
for conditions common in low-income settings [18,19]. This
mismatch is referred to as the 10/90 gap, whereby 10% of
research expenditure is directed at 90% of the world’s burden of ill
health. The roots of this disparity are complex (see Figure 2, left
side). Even in high income countries, the research investment for
neonatal deaths is a small fraction of regular investments in
research on other conditions [20]. Although the United States
National Institutes of Health (NIH) invests approximately US$700
million on research relevant to perinatal conditions, this is less
than 1% of total NIH funding (http://report.nih.gov/rcdc/
categories/) and is primarily focused on preterm birth at around
US$1,200 per case compared to US$18,000 per case for breast
cancer and ovarian cancer. Yet the NIH allocates over US$1.9
billion to biodefense research. For low- and middle- income
countries, which experience 98% of total neonatal deaths and a
similar burden of stillbirths, the investment in research funding for
neonatal survival is extremely low, perhaps around US$20 million
per year, and the funds allocated to address intrapartum-related
conditions are even lower. Defining specific funding allocations for
research on intrapartum-related neonatal deaths is not possible in
current research resource reporting, for either high- or low-income
countries.
Given the large burden, the mismatch with investments and the
short time frame before the MDG targets in 2015, evidence-based
priority setting is imperative to accelerate progress in mortality
reduction [20]. While there are strategies to reduce intrapartum-
related neonatal deaths, the focus has been on having a functional
health system to provide care at birth [21], with little consensus on
how to strengthen weak systems over time [22], or how to address
the 60 million annual home births [23,24]. A recent series of
papers involved a systematic review of evidence for interventions
to reduce intrapartum-related deaths and screened almost 30,000
abstracts [8]. Several reviews summarized the evidence for
obstetric care and for neonatal resuscitation [25,26] and
summarized health system actions that are needed [22,27]. These
reviews focused on the need for effective implementation strategies
for intrapartum care in varying health system contexts and
consistent measurement of pregnancy outcomes including mater-
nal, neonatal and stillbirths. A previous survey of 173 policymakers
and program managers reported on implementation gaps in
programs to address intrapartum-related deaths [28]. The level of
evidence for many intrapartum interventions is low and, while
randomized trials for many accepted intrapartum care interven-
tions may not be considered ethical, all recent reviews have
highlighted the need for more investment in research [22–28]. As
yet, no publication has set out a systematic research agenda on this
topic.
Priority Setting for Research Investments
The Child Health and Nutrition Research Initiative (CHNRI),
linked with the Global Forum for Health Research, has
summarized methodologies [29] developed over the last 20 years
to set priorities for global health research investments (http://
2 January 2011 | Volume 8 | Issue 1 | e1000389
67
World Health Report 2012: No Health Without Research
Figure 1. The burden of intrapartum-related neonatal deaths, intrapartum stillbirths, maternal deaths, and the unknown
associated burden of neonatal morbidity and disability. Data sources: neonatal deaths [13], stillbirths [15,16], maternal deaths [48], place of
birth [8]. No systematic estimates are currently available.
doi:10.1371/journal.pmed.1000389.g001
www.chnri.org/publications.php). Previous methods have includ-
ed the Combined Approach Matrix [30] and the Delphi process
[31]. These were the starting points for the development of the
novel CHNRI approach to research priority setting, based on a
well-defined context, transparent criteria, and independent input
from investors, technical experts, and other stakeholders [31] (see
Figure 2, right side). The CHNRI methodology has been proposed
as a tool that could be used by those who develop research policy
or invest in health research [32,33]. The process examines (i) the
full spectrum of research investment options, (ii) the potential risks
and benefits that could result from investments in different
research options, and (iii) the likelihood of achieving reductions of
persisting burden of disease and disability through investments.
CHNRI methodology has now been applied to a wide range of
topics that include childhood pneumonia [34], diarrhea [35],
neonatal infections [36], zinc supplementation [37], mental health
[38], disability [39], primary health care [40], and also country-
level priority setting in South Africa [41].
Several analyses have shown that around two-thirds of neonatal
[42] and child deaths [43] could be prevented with existing
interventions, but that there is a gap in current coverage, especially
for the poorest families. The WHO’s Child and Adolescent Health
and Development Department (CAH) identified a need for a
PLoS Medicine | www.plosmedicine.org
68
systematic approach to setting priorities for health research that
could reduce this gap through health systems research while
maximizing reductions in the five main causes of child death
within the short time frame to the MDG 4 target. CAH recognized
the potential usefulness of the CHNRI methodology, and in 2008
initiated a process to identify health research priorities to reduce
mortality from the top five causes of child death, including
intrapartum-related neonatal deaths. Several hundred technical
experts from diverse backgrounds and all regions of the world took
part in the exercise. In this paper we present the results and
highest ranked research priorities to reduce mortality from
intrapartum-related neonatal deaths by 2015.
Methods
The CHNRI methodology for setting priorities in health
research investments has four stages: defining the context and
criteria for priority setting with input from investors and policy-
makers; listing and scoring research investment options by
technical experts using predetermined criteria (Box 1); weighting
the criteria according to wider societal values with input from
other stakeholders; and computation and discussion of the scores
and agreement between experts [32,33,44]. The CHNRI
3 January 2011 | Volume 8 | Issue 1 | e1000389
iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Figure 2. Conceptual framework for Child Health and Nutrition Initiative (CHNRI) showing steps from health research investment
to a decrease in burden of death, disease, or disability. Investment decisions in health research are based on a range of factors and processes
(left side). The CHNRI framework identifies criteria to discriminate between competing research options (right side): (1) answerability; (2) effectiveness;
(3) deliverability; (4) maximum potential for disease burden reduction; and (5) predicted equity effect in the population. These five criteria are used to
score the list of research options in the CHNRI research priority setting process [32–34].
doi:10.1371/journal.pmed.1000389.g002

methodology, validity and potential limitations are detailed in setting a relatively short time frame (2015), research requiring a
Table S1. longer lead time was less likely to be highly ranked.

Stage 1: Define the Context and Criteria for Priority
Setting
The aim of this particular exercise was to inform key global
donors, investors in health research (especially of public funds),
and international agencies about research investment policies that
are expected to address MDG 4 in the most effective way. In
choosing to focus on mortality, we nonetheless acknowledge the
importance of non-fatal outcomes, such as the considerable
burden of morbidity and sequelae related to intrapartum insults.
In addition, while focusing on one condition (intrapartum-related
neonatal deaths), there would be expected beneficial effects of
investments from such research on related outcomes such as
maternal deaths and stillbirths, and perhaps on the function of
health systems and primary health care [9]. Furthermore, by
Stage 2: List and Score Research Options Using
Predetermined Criteria
Individuals with a wide range of technical expertise and regional
representation were identified by a core team and by WHO staff
and sent a formal invitation to participate. A list of research
questions was drafted by the core team expert group based on
recent systematic reviews [22–27] and a previous survey of experts
[28]. The research questions were organized using the framework
shown in Table S2. The expert group then reviewed the questions,
adding to and refining the list. The final questions were sent to
each technical group member in an Excel (Microsoft Word 2007)
format for scoring.
Based on CHNRI methodology (Figure 1), five scoring criteria
were applied: (i) answerability in an ethical way; (ii) likelihood of

PLoS Medicine | www.plosmedicine.org 4 January 2011 | Volume 8 | Issue 1 | e1000389

iMedPub :: Thematic collections :: Volume 2 69
World Health Report 2012: No Health Without Research
Box 1. Questions Answered by Technical Experts to Assign Intermediate Scores to Competing
Research Options.
Possible answers: Yes = 1; No = 0; Informed but undecided
answer: 0.5; Not sufficiently informed: blank.
CRITERION 1: Likelihood that research would lead to
new knowledge (enabling a development/planning
of an intervention) in ethical way.
1. Would you say the research question is well framed and
endpoints are well defined?
2. Based on: (i) the level of existing research capacity in
proposed research; and (ii) the size of the gap from current
level of knowledge to the proposed endpoints; would you
say that a study can be designed to answer the research
question and to reach the proposed endpoints of the
research?
3. Do you think that a study needed to answer the proposed
research question would obtain ethical approval without
major concerns?
CRITERION 2: Assessment of likelihood that the inter-
vention resulting from proposed research would be
effective.
1. Based on the best existing evidence and knowledge, would
the intervention which would be developed/improved
through proposed research be efficacious?
2. Based on the best existing evidence and knowledge, would
the intervention which would be developed/improved
through proposed research be effective?
3. If the answer to either of the previous two questions is
positive, would you say that the evidence upon which
these opinions are based is of high quality?
CRITERION 3: Assessment of deliverability, afforda-
bility, and sustainability of the intervention resulting
from proposed research.
1. Taking into account the level of difficulty with intervention
delivery from the perspective of the intervention itself (e.g.,
design, standardization, safety), the infrastructure required
(e.g., human resources, health facilities, communication
and transport infrastructure) and users of the intervention
(e.g. need for change of attitudes or beliefs, supervision,
existing demand), would you say that the endpoints of the
research would be deliverable within the context of
interest?
2. Taking into account the resources available to implement
the intervention, would you say that the endpoints of the
research would be affordable within the context of
interest?
3. Taking into account government capacity and partnership
requirements (e.g., adequacy of government regulation,
monitoring and enforcement; governmental intersectoral
coordination, partnership with civil society and external
donor agencies; favorable political climate to achieve high
coverage), would you say that the endpoints of the
PLoS Medicine | www.plosmedicine.org
70
research would be sustainable within the context of
interest?
CRITERION 4: Assessment of maximum potential of
disease burden reduction. As this dimension is con-
sidered "independent" of the others, in order to score
competing options fairly, their maximum potential to reduce
disease burden should be assessed as potential impact
fraction under an ideal scenario, i.e., when the exposure
to targeted disease risk is decreased to 0% or coverage of
proposed intervention is increased to 100% (regardless of
how realistic that scenario is at the moment—that
aspect will be captured by other dimensions of priority
setting process, such as deliverability, affordability and
sustainability).
Non-existing interventions*
Maximum potential to reduce disease burden should be
computed as "potential impact fraction’’ for each proposed
research avenue, using the equation: PIF = [S(i = 1 to n) Pi (RRi-
1)]/[S(i = 1 to n) Pi (RRi-1)+1];
where PIF is ‘‘potential impact fraction’’ to reduce disease
burden through reducing risk exposure in the population
from the present level to 0% or increasing coverage by an
existing or new intervention from the present level to 100%;
RR is the relative risk given exposure level (less than 1.0 for
interventions, greater than 1.0 for risks), P is the population
level of distribution of exposure, and n is the maximum
exposure level.
Existing interventions**
Maximum potential to reduce disease burden should be
assessed from the results of conducted intervention trials; if
no such trials were undertaken, then it should be assessed as
for non-existing interventions.
Then, the following questions should be answered:
1. Taking into account the results of conducted intervention
trials**, or for the new interventions the proportion of
avertable burden under an ideal scenario*, would you say
that the successful reaching of research endpoints would
have a capacity to remove 5% of disease burden or more?
2. To remove 10% of disease burden or more?
3. To remove 15% of disease burden or more?
CRITERION 5: Assessment of the impact of proposed
health research on equity.
1. Does the present distribution of the disease burden affect
mainly the underprivileged in the population?
2. Would you say that either (i) mainly the underprivileged, or
(ii) all segments of the society equally, would be the most
likely to benefit from the results of the proposed research
after its implementation?
3. Would you say that the proposed research has the overall
potential to improve equity in disease burden distribution
in the long term (e.g., 10 years)?
5 January 2011 | Volume 8 | Issue 1 | e1000389
iMedPub :: Thematic Collections :: Volume 2

effectiveness; (iii) likelihood of deliverability, affordability, and
sustainability; (iv) maximum potential impact on burden reduc-
tion; and (v) predicted impact on equity. The experts made a
judgment on each proposed research question by answering the
questions presented in Box 1.
Stage 3: Solicit Input From Societal Stakeholders to
Weight the Criteria
The five criteria for scoring (answerability, efficacy and
effectiveness, deliverability, disease burden reduction, and effect
on equity) may be perceived to be of varying importance and the
value given to each criterion may vary with the perspective of
stakeholders. For example, parents who have experienced a
neonatal death may rate mortality reduction higher than a
research funder who may value answerability, or a health system
planner who may be most concerned with deliverability. Hence,
CHNRI undertook an exercise to poll a wide range of stakeholders
and to weight the criteria based on values assigned by them, as
described elsewhere [44]. The weights applied in this exercise are
explained in detail in Table S1.
Stage 4: Compute "Research Priority Scores" and Average
Expert Agreement
Completed worksheets were returned to the group coordinator.
The overall research priority score (RPS) was computed as the
mean of the scores for the five criteria, weighted according to the
input from the stakeholders (Table S1), according to the formula:
RPS ~ððCriterion 1 score � 0:96ÞzðCriterion 2 score � 0:86Þ
zðCriterion 3 score � 0:86ÞzðCriterion 4 score � 1:75Þ
zðCriterion 5 score � 0:91ÞÞ=5
Average expert agreement (AEA) scores were also computed for
each research question as the average proportion of scorers who
agreed on the 15 questions asked. This was computed for each
scored research investment option as:
1 encephalopathy, with extremely low scoring for mortality
AEA ~ | reduction. Only two delivery research questions were in the
15
X15
Nðscorers who provided most frequent responseÞ
q~1
Nðscorers who provided any responseÞ
(where q is a question for which experts are being asked to
evaluate competing research investment options, ranging from 1 to
15). For further details regarding the choice of methods,
agreement statistics, and interpretation see Table S1.
Results
Of the 26 experts who were approached and agreed to
participate, 21 returned their scoring sheets within the allocated
time, resulting in a completion rate of 81%. The scorers were
evenly distributed across four regions (Africa [29%], Americas
[29%], Asia/Middle East [19%] and Europe [24%]), and the
regional distribution for non-responders was similar. Only 19% of
responders (four) and non-responders (one) were female. Expertise
covered clinical provision (midwifery, neonatology, obstetrics,
pediatrics, and disability care), perinatal epidemiology, public
health, and basic science, as well as both researchers and research
funders. The full list of technical experts who were invited to
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
participate, their expertise, and reasons for non-participation (for
those who declined or failed to respond in time) are presented in
Table S3.
The full list of 61 research options and the scores from each
individual scorer are presented in Table S4. Questions are
organized by delivery (health system research questions), develop-
ment, discovery science, and epidemiology research themes. More
questions were listed for delivery (28) than for development (11) or
epidemiology (17), and far fewer for discovery (5). The scores
ranged from 37 to 92 (potential 0 to 100), although almost all
scores were over 50, suggesting that few of the research options
were considered of little merit, and that the scoring system was
able to help distinguish between a long list of mostly valuable
options.
Table 1 shows the ten highest ranked questions after weighting.
Of these, seven (70%) are related to delivery research, two to
development research, one to epidemiological and none to
discovery science questions. In the top ten ranked questions, the
scores varied from 84 to 92. AEA varied from 0.42 to 0.79. Not
surprisingly, the highest ranked research options tended to have a
higher AEA. The lowest AEA, and also the lowest RPS, was for
the question regarding amnioinfusion, suggesting that the question
or the intervention may not have been well understood.
Two of the top four questions relate to how to most effectively
generate demand for facility care at birth with specific mechanisms
such as transport and communication schemes, or financial
incentives and conditional cash transfers. The other two relate
to use of community cadres and the roles that they might
effectively play; for example, in screening for complications or for
supportive transfer to facilities and companionship at birth.
Table 2 shows the ten lowest ranked questions after weighting,
with a range of RPS scores (37–58) and AEA scores (0.42–0.64).
Most of the five discovery research questions are among the lower
ranks, being placed at 59, 55, 54, 39, and 25, respectively. Only
two of the 11 development questions were in the lowest ten.
However, the lowest-ranked question related to development/
adaptation of amnioinfusion, which was ranked very low for
burden effect and also for effectiveness. Of the 17 epidemiology
questions, three fell in the lowest ten ranks and two related to early
identification of infants with developmental delay after neonatal
lowest ten ranks, the lowest of which was about operationalizing
care for diabetes in pregnancy in weak health systems (rank 58);
this question scored low for burden reduction as well as for
deliverability.
Table 3 shows the highest ranked questions for each of the four
different research categories (description, discovery, development,
and delivery). The epidemiology questions with the highest ranks
(8, 11, and 19), were all questions with obvious clinical
implications—for example, early prediction of intrapartum
complications, risk of neonatal encephalopathy, or the need for
resuscitation. The highest ranked discovery question (25th) related
to the interaction of intrapartum infection/pyrexia and hypoxic
injury. Several development research options were ranked highly
(6, 10, and 16), and related to innovative technology for neonatal
resuscitation, for detection of fetal distress and to approaches to
maintaining provider competence for skills.
Discussion
As far as we know, this is the first systematically ranked research
priority list for addressing the burden of almost 1 million
intrapartum-related neonatal deaths, mostly occurring in the
6 January 2011 | Volume 8 | Issue 1 | e1000389
71
World Health Report 2012: No Health Without Research

Table 1. The 15 research questions that achieved the highest overall research priority score (RPS), with average expert agreement
(AEA) related to each question (total of 61 questions).

Research Answer- Effec- Deliver- Burden

Rank Proposed Research Question Type able? tive? able? reduction? Equitable? AEA RPS

1 Can community cadres of workers identify a limited number Delivery 93 88 85 77 94 0.78 91.9
of high-risk conditions/danger signs (e.g., multiple pregnancy,
breech, short maternal stature, etc.) and successfully refer
women for facility birth? What is the predictive value and
cost effectiveness?
2 What strategies are effective in increasing demand for, and Delivery 90 88 77 82 93 0.79 91.2
use of, skilled attendance (e.g., conditional cash transfers)?
3 Behavioral/community participation package to improve Delivery 92 78 94 75 95 0.79 90.6
recognition and acting for simplified danger signs for mother
in labor, including transport and phone/radio communication
("emergency preparedness")?
4 Effectiveness of community cadre roles, e.g., social support, Delivery 83 78 96 73 95 0.74 88.9
bringing to facility when woman is in labor, danger
recognition/referral?
5 Does regular use of perinatal audit reduce the incidence of Delivery 83 97 82 68 98 0.74 88.4
adverse outcomes related to acute intrapartum events?
6 Can simpler/cheaper/more robust technology be developed Development 95 93 87 59 100 0.78 88.1
for neonatal resuscitation (e.g., bag-and-mask, suction
devices), and for resuscitation training (resuscitation
dummies) and more feasible models of maintaining
clinical competency for resuscitation?
7 Does regular use of perinatal audit improve adherence to Delivery 78 92 82 72 93 0.69 86.6
clinical standards for intrapartum care (e.g., use of
partograph, monitoring of fetal heart rate, resuscitation etc.)?
8 Can specific maternal complications (e.g., obstructed Epidemiology 85 81 82 72 91 0.74 86.2
labor, hypertension, retained twin) with higher risk of
intrapartum stillbirth, early neonatal death, or other
unfavorable intrapartum-related outcomes be more
simply predicted at an earlier stage?
9 Can simpler clinical algorithms (recognition and Delivery 93 81 93 53 100 0.79 84.4
management) be developed and validated for babies
who require resuscitation at birth, and does this
increase met need for resuscitation at birth?
10 Can low-cost, robust, simple fetal heart monitors be Development 94 86 69 64 93 0.75 83.7
developed and tested that are more user friendly than
the Pinard—e.g., adaptations of Doppler FHM? Does use of
such a device improve fetal heart rate monitoring and
reduce intrapartum stillbirths and asphyxia-related outcomes?

doi:10.1371/journal.pmed.1000389.t001

world’s poorest families and in settings with too few frontline
health workers. Three-quarters of the top ten priorities, and most
of the top one-third of 61 research investment options, were
dominated by delivery research (implementation). This is not
surprising given the large number of preventable deaths with
known solutions and the short time frame to give results in order to
contribute to achievement of MDG 4 in 2015. The greatest
immediate mortality gains could be achieved through better
implementation of existing interventions, and greater investment
in implementation research is an urgent need. The high-priority
research questions identified in this exercise also have high scores
for improving equity given the marked inequity in current
coverage data regarding care at birth [1,8,22].
Given 60 million home births each year, it is appropriate that
the top four priorities relate to closing the gap in skilled attendance
at the time of birth for women and their babies, mainly by trying
to bring them into facilities for birth through ‘‘pull’’ approaches
(conditional cash transfers) or better linkages such as transport and
communications, and to revisiting evidence-based, selective
approaches to identifying pregnancies at greatest risk. Other
themes in the ‘‘top ten’’ include improving facility based care with
strategies such as audit (ranked 5, 7), and innovations for low-cost,
simpler technology (ranks 6, 10), in addition to more questions
regarding roles for community cadres (ranks 8 and 9).The scores
for the top ten ranked options were close and it is possible that
with a larger group of experts the rank orders would differ.
However, whilst delivery research investment is most likely to
result in burden reduction in the shorter term, development and
discovery research remain essential to develop new interventions
to feed the delivery research pipeline [18]. The highest ranked
question from the discovery research options was only at 25 out of
61. The ten lowest ranked options included the other four of the
five initial discovery research options. This may reflect a systematic
bias introduced by the specified context of a very short time frame
(5 years). Discovery research often takes longer to be translated
into measurable benefits in terms of mortality burden reduction,
and by definition the link to reduction in mortality and inequity is
less direct. The highest ranked discovery question related to the
interaction of hypoxia and infection, which is particularly relevant
in high burden settings where the prevalence of both conditions is

PLoS Medicine | www.plosmedicine.org 7 January 2011 | Volume 8 | Issue 1 | e1000389

72 iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Table 2. The 15 research questions that achieved the lowest overall research priority score (RPS), with average expert agreement
(AEA) related to each question (total of 61 questions).

Research Burden
Rank Proposed Research Question Type Answerable? Effective? Deliverable? reduction? Equitable? AEA RPS

52 What is the magnitude of misclassification between Epidemiology 77 72 58 18 67 0.57 55.8

fresh stillbirths and early neonatal deaths, and
which factors affect this misclassification? What
decision rules (applicable in the community and
hospital settings) can be used to differentiate?
53 What is the positive and negative predictive value Epidemiology 85 57 47 12 77 0.61 52.4
of a very low (,3) and a moderately low (4–6)
Apgar score for neonatal encephalopathy (NE),
death, etc.
54 Can new, simple to use, robust technology be Development 75 62 23 26 79 0.66 52.4
developed to better detect neonatal
fetal distress or NE in low-income settings? e.g.,
amplitude-integrated EEG (cerebral function
monitor, CFM) to identify NE for postnatal
therapeutic interventions.
55 What are the longer term outcomes of NE (6 Epidemiology 79 81 32 11 74 0.64 51.8
months, 1 y, 5 y, and school function at 10 y),
and is there an increased risk of death as well
as disability and reduced school performance?
56 Would novel micronutrient approaches reduce Discovery 72 60 61 24 48 0.49 51.6
cerebral damage after insult (magnesium,
nitrates, combinations etc.)?
57 Does early identification of babies with Delivery 83 43 47 4 78 0.60 46.9
development problems following NE improve
utilization of services (feeding, physiotherapy,
speech and language, hearing) and/or outcomes
(hearing, vision, school performance)?
58 Can care of diabetes in pregnancy be Delivery 71 44 35 25 59 0.53 46.9
operationalized in context of weak health
systems to reduce the risk of large for gest
age babies?
59 Would other novel drug treatments reduce Discovery 70 60 42 10 37 0.51 41.7
cerebral damage after insult (allopurinol, epo,
opioids, etc.)?
60 Are there genes or other biomarkers that predict Discovery 50 62 10 18 48 0.60 37.0
susceptibility to intrapartum hypoxic injury?
61 Can the procedure of amnioinfusion be adapted Development 50 50 27 10 52 0.42 36.0
to lower resource settings and would this impact
asphyxia-related outcomes? Are there clinically
important risks from the procedure?

doi:10.1371/journal.pmed.1000389.t002

high. Initial, small studies of head cooling for neonatal
encephalopathy in high burden settings raise the question of
whether infection may be a factor in the possible increased risk
observed with cooling [45].
The development and epidemiological research questions
mainly fell in the middle band. The highest ranked development
option refers to simpler, cheaper, more robust technology for
neonatal resuscitation, which is clearly critical given the major
unmet need [26]. The highest ranked epidemiology question also
echoed the need to revisit the radical move away from risk
screening, asking if specific maternal complications (e.g., obstruct-
ed labor, hypertension, retained twin) with higher risk of
intrapartum stillbirth, early neonatal death, or other unfavorable
asphyxia-related outcome could be more simply detected at an
earlier stage.
Although the CHNRI methodology represents a systematic
attempt to deal with many of the challenges inherent in the complex
process of research investment priority setting, there are still possible
biases [29]. The initial list of questions is critical—if a given research
investment option is not included, it cannot be scored and drops
from view. Another important possible source of bias arises from the
selection and response of the expert technical group. A larger
scoring group and deliberate attempts to widen regional and
professional variation appear to help reduce the risk of bias; in
addition, due to independent scoring of lists, the ranking is less likely
to be dominated by eloquent individuals, as may happen in
traditional group discussion approaches to research priority setting.
Limitations of CHNRI methodology and validation exercises are
described and discussed in greater detail in Table S1.
Conclusions
A strong political commitment has been made to MDG 4 and 5,
but this commitment requires systematic changes in health
research investment. Current investments mainly target the
diseases prevalent in high-income countries and tend to favor
basic research. This exercise highlights the research investments
most likely to result in rapid progress towards MDG 4 in the

PLoS Medicine | www.plosmedicine.org 8 January 2011 | Volume 8 | Issue 1 | e1000389

iMedPub :: Thematic collections :: Volume 2 73
World Health Report 2012: No Health Without Research

Table 3. Top three research questions within each instrument of health research: description (epidemiology), discovery (basic
research), development (translational research), and delivery (operations research).

Description (Epidemiology) Rank

1. Can specific maternal complications (e.g., obstructed labor, hypertension, retained twin) with a higher risk of intrapartum stillbirth, early neonatal 8
death, or other unfavorable asphyxia-related outcome be more simply detected at an earlier stage?
2. What are the maternal and antenatal/intrapartum care risk factors for NE graded for mild, moderate, and severe in various settings? 11
3. What is the prevalence of babies requiring resuscitation in various settings? What is the prevalence for preterm and term babies? 19

Discovery (Basic Research)

1. Can the synergy of infections/maternal pyrexia with neonatal encephalopathy (NE) be addressed through interventions e.g., antibiotic therapy for 21
pyrexial women in labor?
2. Can new, simple to use, robust technology be developed to better detect fetal distress and NE in low-income settings? e.g., amplitude-integrated EEG 54
(CFM) to identify NE for postnatal therapeutic interventions.
3. Would novel micronutrient approaches reduce cerebral damage after insult (magnesium, nitrates, combinations, etc.)? 56

Development

1. Can simpler/cheaper/more robust technology be developed for neonatal resuscitation, (e.g., bag-and-mask, suction devices) and for resuscitation 6
training (resuscitation dummies) and more feasible models of maintaining clinical competency for resuscitation?
2. Can low-cost, robust, simple fetal heart monitors be developed and tested that are more user friendly than the Pinard—e.g., adaptations of 10
doppler FHM? Does use of such a device improve fetal heart rate monitoring and reduce intrapartum stillbirths and asphyxia-related outcomes?
3. Does distance learning for intrapartum care improve competencies? How do skills and cost compare with conventional training? How often does 16
provider re-training need to be performed to maintain competency in neonatal resuscitation, for different cadres?

Delivery

1. Can community cadres of workers identify a limited number of high-risk conditions/danger signs (e.g., multiple pregnancy, breech, short 1
maternal stature, etc.) and successfully refer women for facility birth? What is the predictive value and cost effectiveness?
2. What strategies are effective in increasing demand for (and use of) skilled attendance? e.g., conditional cash transfers, etc. 2
3. Behavioral/community participation package to improve recognition and acting on simplified danger signs for mother in labor, including 3
transport and phone/radio communication ("emergency preparedness").

doi:10.1371/journal.pmed.1000389.t003

countries with the most deaths. These primarily address delivery
research and development research, particularly to increase the
reach of some high impact interventions for the poorest and most
heavily affected families. Competing research questions may all
contribute to MDG 4 and certainly for the longer term agenda
more investment is also required in discovery science. A more
systematic approach with strategic investment in different
instruments of health research would be expected to accelerate
progress towards mortality reduction. While newborn survival has
gained rapid attention in recent years, attention has yet to connect
to adequate action [46]. Further progress in reducing deaths will
depend on systematically addressing implementation and knowl-
edge gaps, and targeted innovation where most of the deaths
occur.
We challenge the research community, research funding
organizations, and national research organizations to systemati-
cally address at least the top ten ranked research questions before
2015. These research options have the potential to prevent almost
1 million unnecessary neonatal deaths that occur every year, and
also reduce an additional one million intrapartum stillbirths and
the closely associated 350,000 maternal deaths [47].
Supporting Information
Table S1 The CHNRI methodology for setting priorities in
health research investments.
Found at: doi:10.1371/journal.pmed.1000389.s001 (0.03 MB
PDF)
Table S2 CHNRI’s starting framework from which a listing of
many research options (level of 3–5-year research program) and
research questions (level of individual research papers) were being
proposed by technical experts to develop a consolidated list of
research questions.
Found at: doi:10.1371/journal.pmed.1000389.s002 (0.02 MB
PDF)
Table S3 Composition of the group of technical experts.
Found at: doi:10.1371/journal.pmed.1000389.s003 (0.03 MB
PDF)
Table S4 Research options scored (61) and example of CHNRI
scoring sheet.
Found at: doi:10.1371/journal.pmed.1000389.s004 (0.11 MB
XLS)
Acknowledgments
The authors thank the CHNRI team who helped to develop these
methods, particularly Prof. Bob Black, Dr. Shams el Arifeen, and Prof.
Harry Campbell. We acknowledge an additional scorer, Prof. G. Justus
Hofmeyr, who is not named as an author. We also thank the team at WHO
coordinating the priority setting process for MDG 4, especially Olivier
Fontaine. The views expressed are the responsibility of the authors and do
not represent the view of the World Health Organization.
Author Contributions
ICMJE criteria for authorship read and met: J. Lawn R. Bahl S. Bergstrom
Z. Bhutta G. Darmstadt M. Ellis M. English J. Kurinczuk A. Lee M.
Merialdi M. Mohamed D. Osrin R. Pattinson V. Paul S. Ramji O.
Saugstad L. Sibley N. Singhal S. Wall D. Woods J. Wyatt K. Chan I.
Rudan. Agree with the manuscript’s results and conclusions: J. Lawn R.
Bahl S. Bergstrom Z. Bhutta G. Darmstadt M. Ellis M. English J.

PLoS Medicine | www.plosmedicine.org 9 January 2011 | Volume 8 | Issue 1 | e1000389

74 iMedPub :: Thematic Collections :: Volume 2

Kurinczuk A. Lee M. Merialdi M. Mohamed D. Osrin R. Pattinson V.
Paul S. Ramji O. Saugstad L. Sibley N. Singhal S. Wall D. Woods J. Wyatt
K. Chan I. Rudan. Designed the experiments/the study: J. Lawn R. Bahl
S. Bergstrom G. Darmstadt I. Rudan. Analyzed the data: J. Lawn K. Chan
I. Rudan. Collected data/did experiments for the study: J. Lawn M. Ellis J.
Kurinczuk L. Sibley S. Wall. Wrote the first draft of the paper: J. Lawn I.
Rudan. Contributed to the writing of the paper: R. Bahl S. Bergstrom Z.
Bhutta G. Darmstadt M. Ellis M. English J. Kurinczuk A. Lee M. Merialdi
D. Osrin R. Pattinson V. Paul O. Saugstad N. Singhal S. Wall D. Woods J.
Wyatt K. Chan I. Rudan. Contributed to the scoring for the research
priorities: Z. Bhutta. Was an expert collaborator who participated in the
expert survey: M. Ellis. Part of expert panel providing data used: M.
English. Read and agree with submission of the final version of the paper:
J. Kurinczuk. The paper is based on surveying physicians & researchers
who are public health experts and working in the field of decreasing
mortalities related to birth process for mothers and their babies. The survey
References
1. Bhutta ZA, Chopra M, Axelson H, Berman P, Boerma T, et al. (2010)
Countdown to 2015 decade report (2000-10): taking stock of maternal, newborn,
and child survival. Lancet 375: 2032–2044.
2. Rajaratnam JK, Marcus JR, Flaxman AD, Wang H, Levin-Rector A, et al.
(2010) Neonatal, postneonatal, childhood, and under-5 mortality for 187
countries, 1970-2010: a systematic analysis of progress towards Millennium
Development Goal 4. Lancet 375: 1988–2008.
3. You D, Jones G, Hill K, Wardlaw T, Chopra M (2010) Levels and trends in
child mortality, 1990-2009. Lancet 376: 931–933.
4. Lawn JE, Costello A, Mwansambo C, Osrin D (2007) Countdown to 2015: will
the Millennium Development Goal for child survival be met? Arch Dis Child 92:
551–556.
5. Kinney MV, Kerber KJ, Black RE, Cohen B, Nkrumah F, et al. (2010) Sub-
Saharan Africa’s mothers, newborns, and children: where and why do they die?
PLoS Med 7: e1000294. doi:10.1371/journal.pmed.1000294.
6. Lawn JE, Cousens S, Zupan J (2005) 4 million neonatal deaths: When? Where?
Why? Lancet 365: 891–900.
7. Lawn JE, Kerber K, Enweronu-Laryea C, Massee BO (2009) Newborn survival
in low resource settings–are we delivering? BJOG 116(Suppl 1): 49–59.
8. Lawn JE, Lee AC, Kinney M, Sibley L, Carlo WA, et al. (2009) Two million
intrapartum stillbirths and neonatal deaths: where, why, and what can we do?
Int J Gynaecol Obstet 107: S5–S19.
9. WHO (1997) Basic newborn resuscitation: a practical guide. Geneva: World
Health Organization, Available: http://www.who.int/reproductivehealth/
publications/maternal_perinatal_health/MSM_98_1/en/index.html. Accessed
2 December 2010.
10. [No authors listed] (1996) Use and abuse of the Apgar score. Committee on
Fetus and Newborn, American Academy of Pediatrics, and Committee on
Obstetric Practice, American College of Obstetricians and Gynecologists.
Pediatrics 98: 141–142.
11. MacLennan A (1999) A template for defining a causal relation between acute
intrapartum events and cerebral palsy: international consensus statement. BMJ
319: 1054–1059.
12. [No authors listed] (1998) ACOG committee opinion. Inappropriate use of
the terms fetal distress and birth asphyxia. Number 197, February 1998
(replaces no. 137, April 1994). Committee on Obstetric Practice. American
College of Obstetricians and Gynecologists. Int J Gynaecol Obstet 61: 309–
310.
13. Black RE, Cousens S, Johnson HL, Lawn JE, Rudan I, et al. (2010) Global,
regional, and national causes of child mortality in 2008: a systematic analysis.
Lancet 375: 1969–1987.
14. Martines J, Paul VK, Bhutta ZA, Koblinsky M, Soucat A, et al. (2005) Neonatal
survival: a call for action. Lancet 365: 1189–1197.
15. Lawn J, Shibuya K, Stein C (2005) No cry at birth: global estimates of
intrapartum stillbirths and intrapartum-related neonatal deaths. Bull World
Health Organ 83: 409–417.
16. Stanton C, Lawn JE, Rahman H, Wilczynska-Ketende K, Hill K (2006)
Stillbirth rates: delivering estimates in 190 countries. Lancet 367(9521):
1487–1494.
17. WHO (2008) The global burden of disease: 2004 update. Geneva: World Health
Organization.
18. Moran M, Guzman J, Ropars A-L, McDonald A, Jameson N, et al. (2009)
Neglected disease research and development: how much are we really spending?
PLoS Med 6: e30. doi:10.1371/journal.pmed.1000030.
19. Enserink M (2009) Some neglected diseases are more neglected than others.
Science 323: 700.
20. Lawn JE, Rudan I, Rubens C (2008) Four million newborn deaths: Is the global
research agenda evidence-based? Early Hum Dev 84: 809–814.
21. Filippi V, Ronsmans C, Campbell O, Graham WJ, Mills A, et al. (2006)
Maternal health in poor countries: the broader context and a call for action.
Lancet 368: 2123–2124.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
examined current and future research priorities concerning birth related
mortalities. The survey was thorough and comprehensive. It required a
significant amount of time to read, analyze, and answer each questions
considering multiple factors. All participating experts are considered
authors on this paper. I was a responder to the study: M. Mohamed.
Contributed expert opinion to the process described in the paper: D. Osrin.
Scored questions: R. Pattinson. Participated in the systematic assessment
and ranking of the research priorities. Contributed to the interpretation of
findings. Helped in refining the manuscript: V. Paul. Contributed to the
technical inputs for the paper and commented on final draft of the paper:
S. Ramji. Contributed with data: O. Saugstad. Helped with the coding of
the different research priorities: N. Singhal. Took part as a technical expert
contributing and commenting on the research questions: D. Woods. Was
involved in contributing data for the research process and reviewed,
checked, and approved various drafts of the manuscript: J. Wyatt.
22. Lawn JE, Lee AC, Kinney M, Bateman M, Paul V, et al. (2009) Reducing
intrapartum-related deaths and disability: can the health system deliver?
Int J Gynaecol Obstet 107: S123–S142.
23. Darmstadt GL, Lee AC, Cousens S, Sibley L, Zulfiqar A, et al. (2009) 60 million
non-facility births: who can deliver in community settings to reduce intrapartum-
related deaths? Int J Gynaecol Obstet 107: S89–S112.
24. Lee AC, Lawn JE, Darmstadt GL, Osrin D, Kumar V, et al. (2009) Linking
families and facilities for care at birth: what works to avert intrapartum-related
deaths? Int J Gynaecol Obstet 107: S65–S88.
25. Hofmeyr GJ, Haws RA, Bergstrom S, Okong P, Lee AC, et al. (2009) Obstetric
care in low resource settings: what, who, how, and overcoming challenges to
scale up. Int J Gynaecol Obstet 107: S21–S45.
26. Wall S, Lee AC, Niermeyer S, English M, Keenan WJ, et al. (2009) Neonatal
resuscitation in low resource settings: what, who, how and overcoming
challenges to scale up. Int J Gynaecol Obstet 107: S47–S64.
27. Pattinson R, Kerber K, Waiswa P, Day L, Mussell F, et al. (2009) Perinatal
mortality audit: counting, accountability and overcoming challenges in scaling
up. Int J Gynaecol Obstet 107: S113–S122.
28. Lawn JE, Manandhar A, Haws RA, Darmstadt GL (2007) Reducing one million
child deaths from birth asphyxia – a survey of health systems gaps and priorities.
Health Res Policy Syst 16: 5–9.
29. Rudan I, Gibson J, Kapiriri L, Lansang MA, Hyder AA, et al. (2007) Setting
priorities in global child health research investments: assessment of principles
and practice. Croat Med J 48: 595–604.
30. Ghaffar A (2009) Setting research priorities by applying the combined approach
matrix. Indian J Med Res 129: 368–375.
31. Costello A, Filippi V, Kubba T, Horton R (2007) Research challenges to
improve maternal and child survival. Lancet 369(9569): 1240–3.
32. Rudan I, Gibson JL, Ameratunga S, el Arifeen S, Bhutta ZA, et al. (2008)
Setting priorities in global child health research investments: guidelines for
implementation of CHNRI method. Croat Med J 49(6): 720–733.
33. Rudan I, Chopra M, Kapiriri L, Gibson J, Ann LM, et al. (2008) Setting
priorities in global child health research investments: universal challenges and
conceptual framework. Croat Med J 49(3): 307–17.
34. Rudan I, El AS, Black RE, Campbell H (2007) Childhood pneumonia and
diarrhoea: setting our priorities right. Lancet Infect Dis 7(1): 56–61.
35. Fontaine O, Kosek M, Bhatnagar S, Boschi-Pinto C, Chan KY, et al. (2009)
Setting research priorities to reduce global mortality from childhood diarrhoea
by 2015. PLoS Med 6(3): e1000041.
36. Bahl R, Martines J, Ali N, Bhan MK, Carlo W, et al. (2009) : Research priorities
to reduce global mortality from newborn infections by 2015. Pediatr Infect Dis J
28(1 Suppl): S43–48.
37. Brown KH, Hess SY, Boy E, Gibson RS, Horton S, et al. (2009) Setting
priorities for zinc-related health research to reduce children’s disease burden
worldwide: an application of the Child Health and Nutrition Research
Initiative’s research priority-setting method. Public Health Nutr 12(3): 389–96.
38. Tomlinson M, Rudan I, Saxena S, Swartz L, Tsai AC, et al. (2009) Setting
priorities for global mental health research. Bull World Health Organ 87(6):
438–446.
39. Tomlinson M, Swartz L, Officer A, Chan KY, Rudan I, et al. (2009) Research
priorities for health of people with disabilities: an expert opinion exercise. Lancet
374(9704): 1857–62.
40. Walley J, Lawn JE, Tinker A, de Francisco A, Chopra M, et al. (2008) Primary
health care: making Alma-Ata a reality. Lancet 372(9642): 1001–7.
41. Tomlinson M, Chopra M, Sanders D, Bradshaw D, Hendricks M, et al. (2007)
Setting Priorities in Child Health Research Investments for South Africa. PLoS
Med 4(8): e259.
42. Darmstadt GL, Bhutta ZA, Cousens S, Adam T, Walker N, et al. (2005)
Evidence-based, cost-effective interventions: how many newborn babies can we
save? Lancet 365(9463): 977–988.
43. Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS (2003) How many child
deaths can we prevent this year? Lancet 362(9377): 65–71.
10 January 2011 | Volume 8 | Issue 1 | e1000389
75
World Health Report 2012: No Health Without Research
44. Kapiriri L, Tomlinson M, Chopra M, El AS, Black RE, Rudan I (2007) Setting
priorities in global child health research investments: addressing values of
stakeholders. Croat Med J 48(5): 618–27.
45. Robertson NJ, Nakakeeto M, Hagmann C, Cowan FM, Acolet D, et al. (2008)
Therapeutic hypothermia for birth asphyxia in low-resource settings: a pilot
randomised controlled trial. Lancet 372: 801–803.
46. Shiffman J (2010) Issue attention in global health: the case of newborn survival.
Lancet 375: 2045–2049.
PLoS Medicine | www.plosmedicine.org
76
47. Bhutta ZA, Darmstadt GL, Haws RA, Yakoob MY, Lawn JE (2009) Delivering
interventions to reduce the global burden of stillbirths: improving service supply
and community demand. BMC Pregnancy Childbirth 9(Suppl 1): S7.
48. WHO (2010) Trends in maternal mortality: 1990 to 2008. Estimates developed
by WHO, UNICEF, UNFPA, World Bank. Geneva: WHO, Available: http://
whqlibdoc.who.int/publications/2010/9789241500265_eng.pdf. Accessed 2
December 2010.
11 January 2011 | Volume 8 | Issue 1 | e1000389
iMedPub :: Thematic Collections :: Volume 2

Guidelines and Guidance
1
Setting Implementation Research Priorities to Reduce
Preterm Births and Stillbirths at the Community Level
Asha George1*, Mark Young1, Abhay Bang2, Kit Yee Chan3, Igor Rudan4,5, Cesar G. Victora6, Mickey
Chopra1, Craig Rubens7, and the GAPPS expert group on community based strategies and constraints"
1 Health Section, UNICEF, New York, New York, United States of America, 2 Society for Education, Action and Research in Community Health, Gadchiroli, India, 3 Nossal
Institute of Global Health, University of Melbourne, Melbourne, Australia, 4 Croatian Centre for Global Health, University of Split Medical School, Split, Croatia, 5 Centre of
Population Health Sciences, The University of Edinburgh Medical School, Edinburgh, Scotland, United Kingdom, 6 University of Pelotas, Pelotas, Brazil, 7 Global Alliance to
Prevent Prematurity and Stillbirth, Seattle Children’s Hospital, Seattle, Washington, United States of America
Introduction
It is estimated that 3.2 million stillbirths occur each year
globally, 1 million of which happen during birth [1]. In addition,
complications from preterm birth (before 37 completed weeks of
gestation) are the leading cause of death for newborns,
contributing an additional 1 million or 12% of child deaths
[2,3]. In 2009, more than 200 stakeholders attended the
International Conference on Prematurity and Stillbirth convened
by the Global Alliance to Prevent Prematurity and Stillbirth
(GAPPS, http://www.gapps.org/). The community expert group
at the conference included 15 members drawn from technical and
funding organizations in addition to program implementers and
researchers from around the world (see Acknowledgments section
for specific names). In their discussions, the group framed efforts to
address preterm and stillbirths within the broader context of
maternal–newborn interventions. As most of the evidence
supporting these interventions emanates from research projects
in controlled settings in specific contexts, the group identified the
main challenge being implementing interventions at scale in
different contexts. Based on these discussions, the group began a
research prioritization exercise for implementation research on
community-based maternal-newborn interventions that address
prematurity and stillbirths at scale in different contexts. In this
paper, we present the results of this exercise.
Methods
A number of research prioritization efforts have recently been
applied to various health topics and health system themes [4–7].
The GAPPS community expert group chose the methodology
proposed by the Child Health and Nutrition Initiative (CHNRI) to
systematically list and score research questions. The CHNRI
methodology was selected because its conceptual framework [8–
10] has been used in numerous areas by different national and
international organizations [11–16] (further information on
CHNRI methodology, validity, and potential limitations are
discussed in Table S1). The group followed three main stages to
derive research priorities (detailed in Box 1). Briefly, guided by the
CHNRI methodology the group evaluated 55 research questions
against five main criteria:
1. Is the research question answerable in an ethical way?
2. Does the research question have the potential to reduce the
disease burden (due to prematurity and stillbirths)?
3. Is it likely that the proposed research would address obstacles to
scaling up?
4. Would the proposed research attract funding support and
national policy attention?
iMedPub ::PLoS Medicine
Thematic | www.plosmedicine.org
collections :: Volume 2
World Health Report 2012: No Health Without Research
5. Would the research results be owned by local actors, including
political authorities and elected representatives, health workers,
district managers, and communities?
Respondents were 39% women and diverse in terms of regional
representation (26% sub-Saharan Africa, 16% Asia, 16% Latin
America, 10% Europe, 32% North America). While a substantial
number of respondents were based in North America, they all
work full-time in developing country contexts. Half of the
respondents were based in research institutions, whereas the other
half were in charge of implementing programs whether through
nongovernmental organizations, UNICEF country offices, or
USAID headquarters. Nonrespondents were not significantly
different from respondents (Table S2).
Results
The research question that was highlighted as the most
important out of all 55 reviewed was ‘‘Evaluate ways to reduce
the financial barriers to facility births at the community level—
e.g., user fee exemptions, emergency loans, conditional cash
Citation: George A, Young M, Bang A, Chan KY, Rudan I, et al. (2011) Setting
Implementation Research Priorities to Reduce Preterm Births and Stillbirths at the
Community Level. PLoS Med 8(1): e1000380. doi:10.1371/journal.pmed.1000380
Published January 4, 2011
Copyright: � 2011 George et al. This is an open-access article distributed under
the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the
original author and source are credited.
Funding: IR and CGV received support as consultants from the Child Health
and Nutrition Research Initiative (CHNRI) and the Global Alliance to Prevent
Prematurity and Stillbirths (GAPPS). GAPPS, with support from the Bill & Melinda
Gates Foundation, March of Dimes, PATH, Save the Children, UNICEF, and WHO,
also funded the meeting in which the research prioritization process was initiated.
There were no other sources of funding and all authors (except IR and CGV)
volunteered their time to conduct this study. The funders had no role in study
design, data collection and analysis, decision to publish, or preparation of the
manuscript.
Competing Interests: While this paper was being developed IR was a
consultant for the Child Health and Nutrition Initiative (CHNRI) of the Global
Forum for Health Research. Because the authors all work in this research area, it is
conceivable that they could benefit from any increased attention and financing
regarding implementation research in preventing prematurity and stillbirth.
Abbreviations: AEA, average expert agreement; CHNRI, Child Health Nutrition
Research Initiative; GAPPS, Global Alliance to Prevent Prematurity and Stillbirth;
MDG, Millennium Development Goal; RPS, research priority score; WHO, World
Health Organization
* E-mail: ageorge@unicef.org
" Membership of the GAPPS expert group on community based strategies and
constraints is provided in the Acknowledgments.
Provenance: Not commissioned; externally peer reviewed.
1 January 2011 | Volume 8 | Issue 1 | e1000380 77
World Health Report 2012: No Health Without Research
Summary Points
N Preterm birth complications are the leading cause of
neonatal mortality, contributing 1 million deaths annu-
ally. Stillbirths account for another 3.2 million deaths.
Both causes of perinatal mortality are inextricably linked
to maternal health and to conditions at birth.
N While some community-based interventions have
proved effective in controlled settings and specific
contexts, the implementation research challenge is to
understand how to sustain these interventions at scale in
different contexts.
N A systematic process based on the Child Health and
Nutrition Research Initiative (CHNRI) methodology was
used to score and rank implementation research
questions regarding community-based maternal–new-
born interventions that address prematurity and still-
births in different contexts at scale.
N Of the 55 research questions that were reviewed in this
way, the top five addressed equity (e.g., reaching the
poor and marginalized, reducing financial barriers),
behavioral practices and skills (e.g., engaging with social
norms, identifying prematurity), and quality of care
provided by community health workers. The top 15
questions encompassed issues pertaining to behavioral
interventions, community health workers, referral, and
managing health systems.
transfers, transportation vouchers, etc.’’ Other research questions
among the top five prioritized also addressed equity issues
(reaching the poor and marginalized), but also behavioral practices
and skills (engaging with social norms, identifying prematurity) and
service delivery (measuring and maintaining quality of care
provided by community health workers). The remaining top ten
research questions (Table 1) include other behavioral skills and
practices (thermal care and feeding for preterm babies, birth
planning), concerns about how to best motivate and compensate
community health workers and their supervisors, and different
dimensions of making referrals more effective. Congruent with the
priority need to measure and maintain quality of care by
community health workers, rational drug use by community
health workers and community engagement with regard to audits
was also listed among the top 25 research questions that received
an overall research priority score (RPS) of 0.75 or greater (Table 2).
Table 3 shows the ten research questions that were assigned the
lowest RPSs. Several broad policy questions (human resource
planning, gender profiles, budget flows, accountability, and
monitoring systems) are listed here, along with some questions
related to the sequencing of community interventions and one
specific question regarding private provider practice (delayed cord
clamping). Questions from almost all research avenues were found
among the bottom ten research questions, suggesting that no one
area was completely discriminated against by the scoring.
Furthermore, even these lower-ranked research questions received
relatively high RPSs compared to those arising from other
CHNRI exercises. The RPS for all 55 questions ranged from
0.86 to 0.56, in contrast to other CHNRI exercises, which have
generated RPS ranges from 0.90 to 0.25 [12–16]. This suggests
that respondents collectively considered all implementation
research questions as fairly important.
Research questions did vary in specificity. For example, broad
questions such as ‘‘evaluate community-based strategies to reach
the poor and marginalized’’ were scored alongside very specific
questions like ‘‘evaluate ways to provide thermal care and feeding
PLoS Medicine | www.plosmedicine.org
78
Box 1. CHNRI Process
Stage 1: Defining the research context, questions,
and criteria for priority setting
When: May–September 2009
How: Group discussions and subsequent e-mails
Results:
N Consensus on research context defined by space
(developing countries), time (the next 5–10 y), the
population of interest (children under five years of age),
and disease burden of interest (preterm and stillbirths).
Respondents were also asked to keep in mind that all
research questions started with the following introduc-
tion: ‘‘When implementing a community based maternal
newborn intervention package that addresses prematu-
rity and stillbirths in different contexts at scale…’’
N Consensus around 55 implementation research ques-
tions grouped according to the following research
domains: community engagement, behavioral skills and
practices, community health workers, rational drug use,
management health systems, and referral.
N Consensus on the five criteria used to rank the research
questions: ethical answerability, disease burden reduc-
tion, ability to support scale-up, likelihood to attract
financial and policy support, ownership by local actors.
Stage 2: Enlisting experts to systematically score
the research questions
When: October 2009 – March 2010
How: Preliminary e-mails sent to 85 leading experts on
community based approaches and maternal-newborn
health in developing countries identified through a
literature search and through snowballing of program
managers. The spreadsheet was also translated into French
and Spanish in order to ensure the participation of
colleagues from Francophone Africa and Latin America.
Results:
N 42 experts agreed to participate
N 31 experts were able to complete the spreadsheets,
independently scoring the 55 research questions by each
of the five criteria by answering ‘‘Yes’’ (1 point), ‘‘No’’ (0
points), undecided (0.5 points), or insufficiently informed
to answer the question (missing input).
Stage 3: Computing and writing up results
When: March–August 2010
How: An intermediate score was calculated for each of the
five criteria and the RPS computed as the mean of all five
intermediate priority scores [8–10] (Table S3). AEA scores
were computed for each research question as the average
proportion of scorers that agreed on the 55 questions
asked (Table S1).
Results:
N 29 correctly completed spreadsheets analyzed with all 55
research questions systematically scored and ranked in
order of priority and agreement.
N Draft circulated to all participants for feedback before
being finalized.
2 January 2011 | Volume 8 | Issue 1 | e1000380
iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Table 1. The ten research questions that received the highest overall RPS.

Burden Scale National

Rank Proposed Research Question Answerable? Reduction? Up? Policy? Ownership? RPS AEA

1 Evaluate ways to reduce the financial barriers to 0.930 0.663 0.845 0.877 0.895 0.858 0.821
facility births at the community level (user fee
exemptions, emergency loans, conditional cash
transfers, transportation vouchers, etc)
2 Develop and validate strategies to identify preterm 0.942 0.640 0.750 0.795 0.821 0.832 0.801
babies at community level by CHWs and family members
3 Evaluate different methods of behavior change 0.904 0.696 0.909 0.886 0.772 0.829 0.794
that overcome harmful practices and promote
positive cultural and social norms
4 Evaluate effective community-based strategies to 0.895 0.670 0.843 0.911 0.868 0.825 0.772
reach the poor and marginalized
5 Evaluate ways to measure and maintain quality 0.967 0.698 0.851 0.737 0.776 0.825 0.794
of care provided by CHWs
6 Evaluate ways to provide thermal care and feeding 0.958 0.686 0.802 0.737 0.798 0.822 0.777
for the preterm baby
7 Evaluate financing measures at the community 0.915 0.500 0.848 0.729 0.877 0.817 0.779
level that improve referral
8 Evaluate ways to motivate and compensate CHWs 0.983 0.596 0.929 0.700 0.817 0.814 0.785
and their supervisors
9 Evaluate how to maximize referral compliance 0.959 0.587 0.796 0.772 0.833 0.813 0.757
especially for the poor and marginalized
10 Evaluate ways to engage communities in birth 0.908 0.630 0.740 0.741 0.888 0.812 0.759
planning for normal and at risk pregnancies

doi:10.1371/journal.pmed.1000380.t001

the preterm baby.’’ Both broad and specific questions were ranked
in the top and bottom ten implementation research questions,
suggesting that no bias existed against the kind of question asked.
The CHNRI methodology evaluates certain dimensions of each
research question according to defined criteria. For example,
‘‘Evaluate methods and levels of accountability that can be
ensured’’ was not considered to affect disease burden and
‘‘Evaluate ways to ensure delayed cord clamping in deliveries
assisted by private providers’’ was not scored as likely to attract
funding support or national policy attention. Among the five
criteria, the most discriminative was the one related to disease
burden reduction, while the least discriminative was the one
regarding ethical answerability.
As mentioned, the relatively high mean scores assigned to
questions across all criteria (apart from disease burden reduction)
indicate that most of the respondents were fairly optimistic about
the value of implementation research questions. Average expert
agreement (AEA) ranged from 0.82 to 0.49. Similar to other
CHNRI exercises, AEA showed a direct positive association with
RPSs, indicating that there was more agreement among experts
about what were the priority research questions. This is a property
that is inherent to the way AEA is measured: very high or very low
RPS scores require high levels of expert agreement, while
substantial disagreement among experts will lead to RPS moving
closer to a mean value [12-16].
To determine whether any systematic bias existed against
certain questions due to the profile of the respondent, we
analyzed scores for researchers and implementers separately. We
found at least a 10% difference in the scoring assigned for 20% of
the research questions (Table 4). The 11 questions for which
there was a significant difference between researchers and
implementers are spread across each research avenue, suggesting
no one particular research area was affected by this difference of
opinion. In ten out of these 11 questions, implementers ranked
the implementation research question as being of higher value
than researchers.
Discussion
The top 25 research questions that have been prioritized span a
broad range of issues (Table 2). These implementation research
priorities address fostering and sustaining specific behavioral skills
and practices at the community level, engaging communities in
monitoring service delivery through audits, and improving
referral. With regard to service delivery, a host of implementation
research questions about the management of community health
workers, along with the health system supports they require to
function, were stressed. Finally, issues of equity, financing, and
referral were highlighted, reflecting how community-based
approaches cannot be dealt with in isolation from broader health
system concerns.
While many of the implementation research priorities identified
can be generalized across community-based maternal, newborn,
and child health areas, a few distinctions may be particular to this
specific exercise. Issues related to referral were present three times
within the top 25 research questions. There is little implementa-
tion research on linking families from homes to facilities, or
referral more broadly, in low-income countries [17–19]. While
important gains have been made with task-shifting, effective and
equitable referral remains vital, because the most serious cases of
prematurity and other birth complications cannot be handled at
the community level.
Implementation research questions related to community
engagement and some other broader policy concerns central to
managing health systems, such as human resource planning and
monitoring systems, were overall not given high priority by

PLoS Medicine | www.plosmedicine.org 3 January 2011 | Volume 8 | Issue 1 | e1000380

iMedPub :: Thematic collections :: Volume 2 79
World Health Report 2012: No Health Without Research

Table 2. Top 25 research questions by research area with a research priority score of 0.7 or above.

Rank Research Area Research Questions

12 Community engagement Evaluate how community audits could improve access and quality of services
14 Evaluate how community engagement improves referral and counter-referral
2 Behavioral skills and practices Develop and validate strategies to identify preterm babies at community level by CHWs and family members
3 Evaluate different methods of behavior change that overcome harmful practices and promote positive cultural and
social norms
6 Evaluate ways to provide thermal care and feeding for the preterm baby
10 Evaluate ways to engage communities in birth planning for normal and at risk pregnancies
13 Assess the impact of initiation and continuation of Kangaroo Mother Care at home on survival of preterm/LBW
babies in setting with high home births
15 Evaluate ways to ensure the sustained use of insecticide-treated bed nets by pregnant women and newborns
19 Evaluate ways to garner community support to ensure early and sustained breastfeeding
23 Evaluate ways to maintain CHW neonatal resuscitation skills
22 Rational drug use Assess methods to ensure rational drug use among CHWs
5 Community health worker Evaluate ways to measure and maintain quality of care provided by CHWs
8 Evaluate ways to motivate and compensate CHWs and their supervisors
16 Evaluate how CHWs can improve referral and counter-referral
17 Evaluate ways to assure continuous supply of essential medicines and inputs for CHWs
20 Evaluate ways to improve retention of CHWs
21 Evaluate how to measure good supervision for CHWs and different ways of providing it
24 Assess the optimal number of activities and population coverage required to maintain case load and skills of CHWs
25 Evaluate the equity impacts and effectiveness of CHW services when delivered with user fees or drug cost-recovery
fees
1 Management and health systems Evaluate ways to reduce the financial barriers to facility births at the community level (user fee exemptions,
emergency loans, conditional cash transfers, transportation vouchers, etc)
4 Evaluate effective community-based strategies to reach the poor and marginalized
11 Evaluate demand-side financing mechanisms (e.g. insurance, demand side subsidies, vouchers)
7 Referral Evaluate financing measures at the community level that improve referral
9 Evaluate how to maximize referral compliance especially for the poor and marginalized
18 Evaluate the barriers at the community and provider level that cause poor referral

doi:10.1371/journal.pmed.1000380.t002

respondents. Nonetheless, even the bottom ten research questions
received high RPSs relative to other CHNRI exercises. This could
be because the other exercises had more discriminatory criteria or
because previous exercises compared different kinds of research
(basic science versus implementation research). It may be easier for
experts to discern between very different research areas (basic
science versus implementation research) than to discern between
areas of implementation research, which they may consider to be
of relatively similar importance.
In addition, many of the implementation research questions do not
by themselves contribute to improved maternal newborn outcomes.
Their value comes forth when combined with other implementation
issues that together make a more comprehensive and coherent
community-based response with linkages to primary health care
service delivery. It might therefore be difficult for respondents to think
about specific implementation research questions in isolation from
their broader social and health systems contexts.
The partiality toward some areas of implementation research
could reflect the profile of respondents. A comparison of scoring
by implementers and researchers did show some differences—not
across any particular kind of research question, but in the direction
of the bias, with implementers ranking implementation research
questions higher than did researchers. The reasons for this
difference among 20% of the questions are not known, but seem to
indicate that implementers perceive the results of implementation
research to be more powerful if effectively implemented.
While the CHNRI methodology provides a systematic and
transparent way to rank research questions that purposefully
avoids biases introduced by group dynamics dominated by
powerful individuals, it still is a very lengthy process to
undertake. Respondents must score 55 research questions
according to five criteria that have three subcomponents each,
resulting in 825 dimensions to respond to in the spreadsheet.
This makes it a complex spreadsheet and likely does not help
response rates. Eliciting participation via e-mail alone was not
successful—only 42 out of 85 experts responded to the
preliminary e-mail. The 42 experts that did express interest
reflected a group that was more familiar with the GAPPS
conference and had current working relationships with the lead
authors who managed the exercise.
Despite these drawbacks, this exercise represents an important
collaboration between researchers and program implementers to
jointly identify the key implementation research questions vital to
improving community-based maternal and newborn interventions
that address preterm and stillbirths. The exercise also developed
new criteria deemed more appropriate to implementation
research, which require further testing and refinement to improve
their discriminatory power.

PLoS Medicine | www.plosmedicine.org 4 January 2011 | Volume 8 | Issue 1 | e1000380

80 iMedPub :: Thematic Collections :: Volume 2
 Table 3. The ten research questions that received the lowest overall RPS.

iMedPub :: Thematic collections :: Volume 2

PLoS Medicine | www.plosmedicine.org
Rank Proposed Research Question Answerable? Burden Reduction? Scale Up? National Policy? Ownership? RPS AEA

46 Assess the gender distribution of CHWs and its implications in terms 0.925 0.343 0.602 0.619 0.633 0.639 0.574
of their acceptability and effectiveness
47 Assess how CHWs and other kinds of frontline health workers are 0.925 0.271 0.556 0.583 0.692 0.638 0.593
represented in human resource policies, strategies and legislation
48 Evaluate methods of integrating community-based data collection 0.930 0.298 0.636 0.526 0.579 0.628 0.565
into district HMIS

5
49 Evaluate methods and levels of accountability that can be ensured 0.650 0.345 0.565 0.510 0.608 0.618 0.540
50 Assess the methods of tracking budget allocations and flow 0.889 0.256 0.482 0.636 0.651 0.611 0.548
51 Determine the minimum set of indicators required and the most 0.825 0.298 0.609 0.535 0.544 0.608 0.547
effective monitoring system
52 Evaluate the sequencing and linking of different community level 0.696 0.385 0.610 0.479 0.590 0.591 0.536
interventions
53 Evaluate different stages of community engagement (consultation, 0.816 0.267 0.663 0.453 0.548 0.587 0.518
cooperation, co-learning, collective action), including their phasing,
cost and effectiveness
54 Evaluate ways to ensure delayed cord clamping in deliveries assisted 0.933 0.278 0.478 0.343 0.616 0.573 0.532
by private providers
55 Assess the optimal number of community groups that a community 0.923 0.208 0.471 0.365 0.611 0.562 0.497
engagement facilitator can support

doi:10.1371/journal.pmed.1000380.t003

January 2011 | Volume 8 | Issue 1 | e1000380

81
World Health Report 2012: No Health Without Research
82
Table 4. Eleven research questions with a 10% or greater difference in RPS between implementers and researchers.

Difference in Difference in Difference in Difference in Difference in

Answerability Burden Reduction Scale-Up National Policy Ownership Difference Difference

PLoS Medicine | www.plosmedicine.org

Rank Proposed Research Question Criterion Criterion Criterion Criterion Criterion RPS AEA

37 Assess what communities consider as maternal-newborn health 0.131 0.291 0.184 -0.030 0.099 0.135 0.650
priorities and how communities compare maternal-newborn
health with other development priorities
22 Evaluate ways to improve retention of CHWs 20.038 0.140 0.206 0.218 0.032 0.111 0.720
World Health Report 2012: No Health Without Research

35 Evaluate different training approaches (including refresher 20.027 0.293 0.074 0.211 0.001 0.111 0.657
training) for CHWs and their supervisors
19 Evaluate ways to assure continuous supply of essential 0.083 0.265 0.162 0.135 0.028 0.135 0.751

6
medicines and inputs for CHWs
36 Evaluate methods to prevent misuse of oxytocics 20.101 20.136 20.092 20.112 20.112 20.111 0.653
31 Determine culturally appropriate means to deliver skin to skin care 0.095 0.217 0.228 0.200 0.005 0.149 0.677
(formative research of the cultural barriers, design of local solutions)
15 Assess the impact of initiation and continuation of Kangaroo 0.022 0.105 0.237 0.154 20.020 0.100 0.739
Mother Care at home on survival of preterm/LBW babies in
setting with high home births
13 Evaluate demand-side financing mechanisms (e.g. insurance, 0.106 0.167 0.165 0.152 20.021 0.114 0.764
demand side subsidies, vouchers)
53 Determine the minimum set of indicators required and the 0.197 0.182 0.186 0.070 0.021 0.131 0.547
most effective monitoring system
44 Measure the extent of household expenditures and their 0.053 0.175 0.370 20.010 0.137 0.145 0.609
equity impacts
51 Evaluate methods and levels of accountability that can be ensured 0.162 0.245 0.149 0.145 0.034 0.147 0.540

doi:10.1371/journal.pmed.1000380.t004

January 2011 | Volume 8 | Issue 1 | e1000380

iMedPub :: Thematic Collections :: Volume 2

Success in reducing stillbirth and prematurity rates, and in
increasing the survival of preterm infants in low-income countries,
is strongly dependent on achieving high and equitable coverage
with existing cost-effective interventions [20,21]. Yet coverage of
such interventions remains unacceptably low in most countries.
For example, across 68 countries with the highest mortality, only
54% of women deliver with a skilled birth attendant and 38%
receive a postnatal visit [22]. Furthermore, coverage levels are
particularly low among poor and rural families in these countries.
Community-based interventions are therefore essential to reach
population subgroups whose current access to health facilities is
severely limited. The effect of expanding coverage of family and
community care to 90% can by itself lead to a 15%–32%
reduction in neonatal mortality [22]. Nonetheless, the knowledge
gaps around how to sustain these programs at scale in different
contexts remain significant.
Conclusion
While important reviews [23–28] have helped to spur attention
to community-based maternal newborn issues, with intriguing
results for specific interventions [29,30], the implementation
research priorities identified in this article will, we hope, help to
secure further research attention and financing for this important
area. Priority research areas identified include equity concerns
(such as removal of financial barriers and responsiveness to the
poor and marginalized), specific behavioral skills and practices,
and the management of community health workers including
referral care. The challenge is now raised; will communities,
governments, donors, research institutions, and international
organizations respond?
Supporting Information
Table S1 The CHNRI methodology for setting priorities in
health research investments.
Found at: doi:10.1371/journal.pmed.1000380.s001 (0.04 MB
DOC)
Table S2 Profile of respondents and non-respondents.
Found at: doi:10.1371/journal.pmed.1000380.s002 (0.04 MB
DOC)
Table S3 All 55 implementation research questions scored and
ranked.
Found at: doi:10.1371/journal.pmed.1000380.s003 (0.15 MB
DOC)
Acknowledgments
The authors thank the CHNRI team who helped to develop these
methods, particularly Robert Black, Shams El Arifeen, and Harry
References
1. Lawn JE, Lee ACC, Kinney M, Sibley L, Carlo WA, et al. (2009) Two million
intrapartum-stillbirths and neonatal deaths: Where, why and what can be done?
Int J Gynaecol Obstet 107: S5–S19.
2. Rubens CE, Victora CG, Gravett MG, Nunes TM, eds (2010) Global report on
preterm birth & stillbirth: The foundation for innovative solutions and improved
outcomes. BMC Pregnancy Childbirth 10(Suppl 1).
3. Black RE, Cousens S, Johnson HL, Lawn JE, Rudan I, et al. (2010) Child Health
Epidemiology Reference Group of WHO and UNICEF. Global, regional, and
national causes of child mortality in 2008: a systematic analysis. Lancet 375:
1969–1987.
4. Sharan P, Gallo C, Gureje O, Lamberte E, Mari JJ, et al. (2009) World Health
Organization-Global Forum for Health Research - Mental Health Research
Mapping Project Group. Mental health research priorities in low- and middle-
income countries of Africa, Asia, Latin America and the Caribbean.
Br J Psychiatry 195: 354–363.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
Campbell. We thank Paola Canahuati, the UN translation service,
Mariame Sylla, and Jose Rolando Figueroa for translating the CHNRI
scoring spreadsheet from English into French and Spanish. We are grateful
to Zulfiqar Bhutta, Neal Brandes, and Luc de Bernis for their expert review
of an earlier draft of this manuscript.
The GAPPS expert group on community-based strategies and
constraints includes those who attended the Seattle GAPPS meeting and
those who participated in the research prioritization exercise: Kaosar
Afsana (Bangladesh Rural Advancement Committee, Dhaka, Bangladesh),
Fernando Althabe (Institute for Clinical Effectiveness and Health Policy,
Buenos Aires, Argentina), Shams Arifeen (International Centre for
Diarrheal Disease Research, Bangladesh, Dhaka, Bangladesh), Abhay
Bang (Society for Education, Action and Research in Community Health,
Gadchiroli, India), Maria Bardolet (UNICEF, Niamey, Niger), Fernando
Barros, Cesar G. Victora (University of Pelotas, Pelotas, Brazil), Neal
Brandes, Troy Jacobs (USAID Washington, DC, USA), Guy Clarysse
(UNICEF, Dakar, Senegal), Joseph de Graft-Johnson (Saving Newborn
Lives, Washington, DC, USA), Alfredo Fort (Macro International,
Calverton, MD, USA), Hernando Gaitan (Institute of Clinical Research,
National University of Columbia, Bogota, Columbia), Asha George, Mark
Young (UNICEF New York, USA), Kate Gilroy, Vishwajeet Kumar,
Henry Perry (John Hopkins Bloomberg School of Public Health,
Baltimore, USA), Zelee Hill (University College London, London, UK),
Hakeem Jokhio (Aga Khan University, Karachi, Pakistan), Nilda Lambo
(UNICEF, Lusaka, Zambia), Tippawan Liabsuetrakul (Faculty of Medi-
cine, Prince of Songkla University, Hat Yai, Songkla, Thailand), Stephanie
Martin (Program for Appropriate Technology in Health, Seattle, WA,
USA), Grace Mlava (UNICEF, Lilongwe, Malawi), David Marsh, Salim
Saddrudin (Save the Children, Washington, DC, USA), Indira Narayan
(BASICS/USAID Washington, DC, USA), Pius Okong (St. Francis
Nsambya Hospital, Kampala, Uganda ), Adama Ouedrago (UNICEF,
Niamey, Niger), Luwei Pearson (UNICEF, Addis Ababa, Ethiopia), Stefan
Peterson (Karolinska Institute, Stockholm, Sweden), Shamim Qazi (WHO,
Geneva, Switzerland), M. G. Sall (University of Dakar, Dakar, Senegal),
Sharad Sharma (Department of Health Services, Kathmandu, Nepal),
Judith Standley (Independent Program Consultant, Israel), Aboubacry
Thiam (BASICS/USAID, Dakar, Senegal), Celestin Traore (UNICEF
Bujumbura, Burundi).
Author Contributions
ICMJE criteria for authorship read and met: AG MY AB KYC IR CGV
MC CR. Agree with the manuscript’s results and conclusions: AG MY AB
KYC IR CGV MC CR. Designed the experiments/the study: AG MY IR
CGV MC. Analyzed the data: AG MY KYC IR MC. Collected data/did
experiments for the study: AG CGV. Enrolled patients: AG. Wrote the first
draft of the paper: AG MY. Contributed to the writing of the paper: AG
MY KYC IR CGV MC CR. Participated in designing the study and
contributed to writing the manuscript by way of comments: AB. Led the
development of the CHNRI methodology with assistance of the large
multidisciplinary team of experts: IR. Suggested the original idea: CGV.
Reviewed the data analysis: CR.
5. Ranson K, Law TJ, Bennett S (2010) Establishing health systems financing
research priorities in developing countries using a participatory methodology.
Soc Sci Med 70: 1933–1942.
6. Ranson K, Chopra M, Atkins S, Dal Poz MR, Bennett S (2010) Priorities for
research into human resources for health in low- and middle-income countries.
Bull World Health Organ 88: 435–443.
7. Robotin MC, Jones SC, Biankin AV, Waters L, Iverson D, et al. (2010) Defining
research priorities for pancreatic cancer in Australia: Results of a consensus
development process. Cancer Causes Control 21: 729–736.
8. Rudan I, Gibson J, Kapiriri L, Lansang MA, Hyder AA, et al. (2007) Setting
priorities in global child health research investments: Assessment of principles
and practice. Croat Med J 48: 595–604.
9. Rudan I, Chopra M, Kapiriri L, Gibson J, Ann Lansang M, et al. (2008) Setting
priorities in global child health research investments: Universal challenges and
conceptual framework. Croat Med J 49: 307–317.
7 January 2011 | Volume 8 | Issue 1 | e1000380
83
World Health Report 2012: No Health Without Research
10. Rudan I, Gibson JL, Ameratunga S, El Arifeen S, Bhutta ZA, et al. (2008)
Setting priorities in global child health research investments: Guidelines for
implementation of the CHNRI method. Croat Med J 49: 720–733.
11. Rudan I, El Arifeen S, Black RE, Campbell H (2007) Childhood pneumonia and
diarrhoea: setting our priorities right. Lancet Infect Dis 7: 56–61.
12. Tomlinson M, Chopra M, Sanders D, Bradshaw D, et al. (2007) Setting
priorities in child health research investments for South Africa. PLoS Med 4:
e259. doi:10.1371/journal.pmed.0040259.
13. Tomlinson M, Swartz L, Officer A, Chan KY, Rudan I, et al. (2009) Research
priorities for health of people with disabilities: An expert opinion exercise.
Lancet 374: 1857–1862.
14. Tomlinson M, Rudan I, Saxena S, Swartz L, Tsai AC, et al. (2009) Setting
investment priorities for research in global mental health. Bull World Health
Organ 87: 438–446.
15. Bahl R, Martines J, Ali N, Bhan MK, Carlo W, et al. (2009) Research priorities
to reduce global mortality from newborn infections by 2015. Pediatr Inf Dis J
28(Suppl 1): S43–S48.
16. Fontaine O, Kosek M, Bhatnagar S, Boschi-Pinto C, Chan KY, et al. (2009)
Setting research priorities to reduce global mortality from childhood diarrhoea
by 2015. PLoS Med 6: e41. doi:10.1371/journal.pmed.1000041.
17. Macintyre K, Hotchkiss DR (1999) Referral revisted: Community financing
schemes and emergency transport in rural Africa. Soc Sci Med 49: 1473–1487.
18. Murray SF, Pearson SC (2006) Maternity referral systems in developing
countries: current knowledge and future research needs. Soc Sci Med 62:
2205–2215.
19. Lee AC, Lawn JE, Cousens S, Kumar V, Osrin D, et al. (2009) Linking families
and facilities for care at birth: What works to avert intrapartum-related deaths?
Int J Gynaecol Obstet 107(Suppl 1): S65–S88.
20. Darmstadt GL, Walker N, Lawn JE, Bhutta ZA, Haws RA, et al. (2008) Saving
newborn lives in Asia and Africa: cost and impact of phased scale-up of
interventions within the continuum of care. Health Policy Plan 23: 101–117.
21. Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, et al. (2005) BMJ 331: 1107.
PLoS Medicine | www.plosmedicine.org
84
22. Darmstadt GL, Bhutta ZA, Cousens S, Adam T, Walker N, et al. (2005)
Evidence-based, cost-effective interventions: How many newborn babies can we
save? Lancet 365: 977–988.
23. Bhutta ZA, Darmstadt GL, Hasan BS, Haws RA (2005) Community-based
interventions for improving perinatal and neonatal health outcomes in
developing countries: A review of the evidence. Pediatrics 115(Suppl 2):
519–617.
24. Bhutta ZA, Ali S, Cousens S, Ali TM, Haider BA, et al. (2008) Alma-Ata:
Rebirth and Revision 6 Interventions to address maternal, newborn, and child
survival: What difference can integrated primary health care strategies make?
Lancet 372: 972–989.
25. Darmstadt GL, Lee AC, Cousens S, Sibley L, Bhutta ZA, et al. (2009) 60 Million
non-facility births: Who can deliver in community settings to reduce
intrapartum-related deaths? Int J Gynaecol Obstet 107(Suppl 1): S89–S112.
26. Lewin S, Munabi-Babigumira S, Glenton C, Daniels K, Bosch-Capblanch X,
et al. (2010) Lay health workers in primary and community health care for
maternal and child health and the management of infectious diseases. Cochrane
Database Syst Rev 3: CD004015. doi:10.1002/14651858.CD004015.pub3.
27. Nair N, Tripathy P, Prost A, Costello A, Osrin D (2010) Improving newborn
survival in low-income countries: Community-based approaches and lessons
from South Asia. PLoS Med 7: e1000246. doi:10.1371/journal.pmed.1000246.
28. Bang AT, Bang RA, Reddy HM (2005) Home-based neonatal care: Summary
and applications of the field trial in rural Gadchiroli, India (1993–2003).
J Perinatol 25: S108–S122.
29. Azad K, Barnett S, Banerjee B, Shaha S, Khan K, et al. (2010) Effect of scaling
up women’s groups on birth outcomes in three rural districts in Bangladesh: A
cluster-randomised controlled trial. Lancet 375: 1193–1202.
30. Tripathy P, Nair N, Barnett S, Mahapatra R, Borghi J, et al. (2010) Effect of a
participatory intervention with women’s groups on birth outcomes and maternal
depression in Jharkhand and Orissa, India: A cluster-randomised controlled
trial. Lancet 375: 1182–1192.
8 January 2011 | Volume 8 | Issue 1 | e1000380
iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

Review

A Research Agenda to Underpin Malaria Eradication

Pedro L. Alonso1,2*, Graham Brown3, Myriam Arevalo-Herrera4,5, Fred Binka6, Chetan Chitnis7, Frank
Collins8, Ogobara K. Doumbo9, Brian Greenwood10, B. Fenton Hall11, Myron M. Levine12, Kamini
Mendis13, Robert D. Newman13, Christopher V. Plowe14, Mario Henry Rodrı́guez15, Robert Sinden16,
Laurence Slutsker17, Marcel Tanner18
1 Barcelona Centre for International Health Research (Hospital Clı́nic, Universitat de Barcelona), Barcelona, Spain, 2 Centro de Investigaçao em Saude da Manhiça, Maputo,
Mozambique, 3 Nossal Institute for Global Health, University of Melbourne, Melbourne, Australia, 4 Immunology Institute-Universidad del Valle, Cali, Colombia, 5 Centro
de Investigación Caucaseco, Cali, Colombia, 6 School of Public Health, University of Ghana, Accra, Ghana, 7 International Center for Genetic Engineering and
Biotechnology, Delhi, India, 8 University of Notre Dame, Notre Dame, Indiana, United States of America, 9 University of Bamako, Bamako, Mali, 10 London School of
Hygiene & Tropical Medicine, London, United Kingdom, 11 National Institute of Allergy and Infectious Diseases (NIAID), Baltimore, Maryland, United States of America,
12 Center for Vaccine Development, University of Maryland, Baltimore, Maryland, United States of America, 13 Global Malaria Program, World Health Organization,
Geneva, Switzerland, 14 Howard Hughes Medical Institute/University of Maryland School of Medicine, Baltimore, Maryland, United States of America, 15 Instituto Nacional
de Salud Pública, Cuernavaca, Mexico, 16 Imperial College, London, United Kingdom, 17 US Centers for Disease Control and Prevention, Atlanta, Georgia, United States of
America, 18 Swiss Tropical and Public Health Institute and University of Basel, Basel, Switzerland

elimination, and prevention of reintroduction phases; Morocco,

Abstract: The interruption of malaria transmission
worldwide is one of the greatest challenges for interna-
tional health and development communities. The current
expert view suggests that, by aggressively scaling up
control with currently available tools and strategies, much
greater gains could be achieved against malaria, including
elimination from a number of countries and regions;
however, even with maximal effort we will fall short of
global eradication. The Malaria Eradication Research
Agenda (malERA) complements the current research
agenda—primarily directed towards reducing morbidity
and mortality—with one that aims to identify key
knowledge gaps and define the strategies and tools that
will result in reducing the basic reproduction rate to less
than 1, with the ultimate aim of eradication of the parasite
from the human population. Sustained commitment from
local communities, civil society, policy leaders, and the
scientific community, together with a massive effort to
build a strong base of researchers from the endemic areas
will be critical factors in the success of this new agenda.
Introduction
The unacceptable health burden of malaria, and its economic
and social impacts on development, have made it a focal point of
the international development agenda, and the world has
embraced an ambitious plan for scaling up malaria control that
progresses towards country-by-country and regional elimination
and the ultimate goal of global eradication [1]. Over the past
decade, resources and control efforts have intensified to a level not
seen since the early days of the World Health Organization’s
Global Malaria Eradication Program (GMEP) in the late 1950s.
Nonetheless, in 2009, with 3.28 billion people living in areas that
have some risk of malaria transmission and about 1.2 billion
people (one-fifth of the world’s population) living in areas with a
high risk of transmission (more than one reported case per 1,000
population per year), there were about 225 million cases of clinical
malaria and 781,000 malaria-related deaths. Today, there is
ongoing malaria transmission in 106 countries. Eighty-one of these
countries are focusing on control, while 25 are in pre-elimination,
the United Arab Emirates, and Turkmenistan have recently been
certified as malaria free [2–4].
These statistics emphasize the direness of the current malaria
burden but also benchmark the accomplishments and progress
that have been achieved in malaria control. Following declarations
at the Malaria Forum in October 2007 convened by the Bill &
Melinda Gates Foundation, and subsequent support voiced by the
World Health Organization (WHO), the Roll Back Malaria
(RBM) Partnership, and many other organizations and institu-
tions, the paradigm of malaria control and elimination has been
extended to encompass an ultimate goal of malaria eradication
[1,2,5]. The question is no longer whether international agencies
and national health authorities should be mobilized to pursue the
goal of malaria eradication, but rather when and how.
A key question, however, is whether elimination from all regions
of the world (eradication) is feasible with the current tools and state
of knowledge. For a number of reasons, we believe that the answer
is ‘‘no.’’ First, malaria is not a single disease. The five Plasmodium
species (falciparum, vivax, ovale, malariae, knowlesi) that cause human
malaria are transmitted by more than 30 Anopheline mosquito
species with diverse breeding and feeding habits, and result in
different disease spectra in different population target groups and
epidemiological settings. Second, current malaria control and
elimination programs face remarkable heterogeneity of transmis-
Citation: Alonso PL, Brown G, Arevalo-Herrera M, Binka F, Chitnis C, et al. (2011) A
Research Agenda to Underpin Malaria Eradication. PLoS Med 8(1): e1000406.
doi:10.1371/journal.pmed.1000406
Published January 25, 2011
Copyright: � 2011 Alonso et al. This is an open-access article distributed under
the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the
original author and source are credited.
Funding: malERA received a grant from the Bill & Melinda Gates Foundation. The
funders had no role in study design, data collection and analysis, decision to
publish, or preparation of the manuscript.
Competing Interests: GB declares that he is Chair of MALVAC, Chair USAID
Malaria Vaccine Development Program Scientific Consultants Group, a
member of the Board of Roll Back Malaria, and a member of the APMEN
Advisory Board.
Abbreviations: GMEP, Global Malaria Eradication Program; TPP, target product
profile.

Review articles synthesize in narrative form the best available evidence on a topic. * E-mail: palonso@clinic.ub.es
Provenance: Submitted as part of a Supplement; externally peer reviewed.

iMedPub ::PLoS Medicine

Thematic | www.plosmedicine.org
collections :: Volume 2 1 January 2011 | Volume 8 | Issue 1 | e1000406 85
World Health Report 2012: No Health Without Research
Summary Points
N Malaria remains a major global public health problem,
but a recent paradigm shift has moved the emphasis
from control of malaria to the interruption of malaria
transmission and ultimately malaria eradication
N The Malaria Eradication Research Agenda (malERA)
initiative was convened in 2008 to define the knowledge
base, strategies, and tools required to eradicate malaria
from the human population
N A two-year consultative process has resulted in the
preparation of a detailed research and development
agenda for malaria eradication, which is reported in this
Supplement
N Implementation of this research agenda might enable
the elimination of malaria, even in the most difficult
areas
N However, to achieve the aim of malaria eradication in a
timely manner, commitment to implementing this
agenda must begin immediately
sion dynamics of malaria in endemic areas, including differences in
parasite, vector, human, social, and environmental factors. Third,
operational limitations include underperforming health services,
lack of political will, insufficient financial, social and human
resources, and for some areas, inadequate tools to interrupt
transmission given an exceedingly high force of transmission. Each
country presents different combinations of these problems and
their determinants. Thus, a widely held view suggests that with
currently available tools, much greater gains could be achieved,
including elimination from a number of countries and regions, but
that even with maximal effort we will fall short of elimination in
many areas and of global eradication [6]. For definitions of terms
used regarding malaria eradication see Box 1.
Mixed Success and Failure of Past Malaria Control
and Elimination Efforts
A detailed discussion of all the factors involved in the partial
success of the past eradication campaign is beyond the scope of
this introduction, but three critical elements can be highlighted.
First, there was insufficient recognition of the heterogeneity of
malaria transmission and disease. Much of the optimism that
inspired WHO GMEP in 1955 was based on the successful
outcomes of earlier control programs that benefited from a
combination of biological, parasitological, social, and environ-
mental factors that favoured success (e.g., the rarity of DDT-
resistant Anophelines and of chloroquine-resistant parasites).
Second, the first WHO GMEP (1955-1969) was predicated on
an assumption that the available knowledge and tools were
sufficient to achieve worldwide eradication. A single strategy that
would work everywhere—‘‘one size fits all’’—proved to be ill-
founded because it underestimated the challenges of dealing with
the extremely efficient vectors in Africa (An. gambiae) and with
transmission by outdoor-feeding mosquitoes that were not
susceptible to attack by indoor residual insecticide. It also did
not allow for the lack of safe drugs for mass administration to
remove all infectious parasites from symptomatic and asymptom-
atic carriers, particularly from people carrying P. vivax or P. ovale,
species that establish latent liver infections that are responsible for
relapses months or years following initial infection. Third,
insufficient research in biomedical and social sciences and
inadequate local application of research findings across a wide
PLoS Medicine | www.plosmedicine.org
86
Box 1. Clarifying the Goals and Definitions
N Control: Reduction of disease incidence, prevalence,
morbidity, or mortality to a locally acceptable level as a
result of deliberate efforts; continued intervention
measures are required to maintain the reduction.
N Elimination: Reduction to zero of the incidence of
locally transmitted malaria infection in a defined
geographical area as a result of deliberate efforts;
continued intervention measures are required to prevent
reestablishment of transmission.
N Eradication: Permanent reduction to zero of the global
incidence of malaria as a result of deliberate efforts;
intervention measures are no longer needed [1].
N What species? Although the eradication of P. falci-
parum, the most serious form of malaria, would
constitute an historic public health achievement, the
coexistence of transmission of P. falciparum and P. vivax
in many areas of the world together with the fact that
they are the species responsible for the major burden of
disease, make it necessary to aim for the eradication of
both.
variety of settings are widely viewed to have contributed to
demoralization and waning effort when tools proved ineffective or
could not be adequately implemented. The neglect of malaria
research during and after the campaign did long-term damage.
These elements resulted in a lack of progress that in turn
compromised continued financial support [7].
Current Malaria Control Efforts: The Goal of
Eradication and Its Research and Development
Implications
The past decade has witnessed renewed investment in malaria
control and substantial increases in funding for malaria research.
The Roll Back Malaria Global Malaria Action Plan (GMAP) and
WHO have recently revised and updated the strategy and the steps
for scaling up and sustaining malaria control (Figure 1). In
addition, the Malaria Elimination Group (MEG), a group of
scientists, public health decision makers, control program
managers, and funders, has compiled a guide to policy makers
for areas that embark or have embarked on elimination strategies
[8].
Reductions in disease incidence are being documented, even in
some areas of sub-Saharan Africa that constitute the heartland of
malaria transmission [2]. There are, however, significant threats to
current progress that cannot be ignored, and unmet needs that will
continue to be central to the global research agenda for improving
malaria control and eventually achieving eradication. Notable
examples are the emergence of artemisinin resistance and the
consequent need for improved strategies to contain dissemination
of resistant parasite strains coupled with accelerated research into
potential new drugs for first-line treatment [9,10]. Similarly, new
insecticides are urgently needed to replace those threatened by
increased mosquito resistance [11], and accelerated development
of vaccines that can impact on malaria incidence, disease, and
death remains a high priority [12].
Complementing the current research agenda—primarily directed
towards improving malaria control and reducing morbidity and
mortality—with research on developing tools, interventions, and
strategies to interrupt transmission and ultimate eradication of the
parasite from the human population constitutes a true paradigm shift.
2 January 2011 | Volume 8 | Issue 1 | e1000406
iMedPub :: Thematic Collections :: Volume 2

Figure 1. Epidemiological milestones [1,23]. Image credit: Fusión Creativa.
doi:10.1371/journal.pmed.1000406.g001
The malERA Initiative
To catalyze this paradigm shift towards malaria elimination and
eradication, it was necessary to design a process to bring together
the best scientific minds in the malaria community. That process is
the Malaria Eradication Research Agenda (malERA) initiative,
which was established to complement GMAP and which aims to
define the critical knowledge base, strategies, and tools required to
reduce the basic reproduction rate (R0 or the number of secondary
cases arising from a single case) to less than one.
Scientists involved in malaria research were challenged to develop
a multidisciplinary, global research and development agenda that
would be actionable by research and public health agencies and
funders/sponsors and available for discussion and debate through
publication in a readily accessible format. The process engaged more
than 250 scientists in a series of 20 consultations around the world
(Figure 2) and was managed by a three-tier governance structure
(Figure 3). The rest of this article briefly introduces the work
undertaken by the various malERA Consultative Groups and
presented in the other articles in this Supplement.
Tools to Interrupt Malaria Transmission
To reduce the basic reproduction rate to less than 1, and hence
to interrupt transmission, interventions are needed to reduce the
reservoir of infection, the time that a person or a mosquito is
infectious, and the rate at which infections are spread. This goal
can be achieved by drugs or vaccines directed against the parasite
or by new tools that attack the vector, with the support of
improved diagnostics and surveillance.
Drugs: Single Encounter Radical Cure and Prophylaxis
In the recent past, drug development efforts were guided by the
need for first-line drugs to treat P. falciparum infections with an
increasing emphasis on drugs with a short half-life that potentially
minimize the risk of development of resistance rather than on
drugs with a long half-life that have benefits for dosing and post-
treatment prophylaxis [13]. Treatment of infected individuals with
a variety of drug regiments has been used successfully in
combination with intensive vector control to eliminate malaria
from areas with relatively strong health systems and stable
populations. However, interruption of malaria transmission is
likely to require a new set of drugs and formulations.
As described in more detail in the article by the malERA
Consultative Group on Drugs [14], such drugs will need to be used
both in stable transmission areas and in complex urban or remote
rural areas, with poorly functioning health systems where concerted
campaigns may be the only way of achieving high coverage or
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
preventing reintroduction by migrants or travelers from endemic
regions. For such campaigns to impact effectively on inaccessible
populations, a single encounter between health providers and
target populations is critical. Single Encounter Radical Cure and
Prophylaxis (SERCaP) has a target product profile (TPP) that
includes radical cure, defined as elimination of all parasites (including
the long-lived hypnozoites of P. vivax or P. ovale in the liver), suitability
for mass administration (including administration to healthy sub-
jects and the consequent need of a very good safety profile), and
prophylaxis for at least 1 month after treatment, to outlast the typical
development period of Plasmodia parasites in Anopheline mosquitoes.
A drug with this profile would perform in a similar way to a highly
efficacious pre-erythrocytic (infection-preventing) vaccine.
A drug with this TPP may take a long time to develop, but the
development of new drugs that meet some of these essential
requirements could dramatically improve chances of eradication.
For example, development of new safe and effective drugs that block
the infectivity of the mature sexual forms of P. falciparum gametocytes
and/or the dormant hepatic forms (hypnozoites) of P. vivax could have
a profound impact on transmission rates and would be valuable tools
in the efforts to contain and eliminate parasite strains resistant to first-
line treatment drugs. Presently, only the 8-aminoquinolines are
known to be effective against both P. vivax hypnozoites and P.
falciparum stage-five gametocytes. Unfortunately this class of drugs has
significant side-effects in some individuals, particularly hemolysis in
those with G6PD deficiency, that compromise their widespread use in
mass administration for elimination [14].
Vaccines that Interrupt Malaria Transmission
Vaccines currently in clinical development have the primary aim
of reducing morbidity and mortality from P. falciparum in young
children living in highly endemic countries. However, with the new
goal of elimination and eradication, vaccines that will reduce and
contribute to interruption of transmission also need to be developed.
The broader concept of ‘‘vaccines that interrupt malaria transmis-
sion (VIMT)’’ is introduced by the malERA Consultative Group on
Vaccines to replace the term ‘‘transmission blocking vaccines’’
(TBVs), which has been used widely to refer to vaccines that target
only the sexual and mosquito stages of the parasite [15]. VIMT
could include antivector vaccines that target mosquito molecules
essential for parasite development, highly effective pre-erythrocytic
or erythrocytic stage vaccines, and vaccines targeting parasite
antigens of sexual and mosquito stages of the infection. The desired
TPP identified by the Consultative Group for VIMT indicates that
they should be effective against both P. falciparum and P. vivax, suitable
for administration to all age groups, and should impact transmission.
Other issues discussed by the group in their article include the need
3 January 2011 | Volume 8 | Issue 1 | e1000406
87
World Health Report 2012: No Health Without Research
Figure 2. Consultative process towards a consolidated research and development agenda for malaria eradication. Image credit:
Fusión Creativa.
doi:10.1371/journal.pmed.1000406.g002
for validated functional assays that measure the reduction in
infectivity at the individual level after vaccination that could be
used as surrogate measures to predict reductions in transmission
rates at the community level. Such surrogate measures will be critical
components of a regulatory pathway leading to licensure. Standard-
ized, specific and sensitive methods for assessment of transmis-
sion rates, particularly when intensity is low, will be critical in the
assessment of vaccine efficacy in interrupting transmission following
large-scale deployment of vaccination as an elimination tool [15,16].
Vector Control
The overarching goal of vector control is to reduce the vectorial
capacity of local vector populations below the critical threshold to
prevent ongoing or epidemic transmission. Because it takes a
relatively long time (days) after ingestion for Plasmodia to become
infective to humans in its Anopheles vectors, the most effective
vector control strategies currently in use rely on interventions like
indoor residual insecticide spraying and insecticide treated bednets
(ITNs) that reduce vector daily survival rates [17].
The malERA Consultative Group on Vector Control identifies
three critical challenges in its article [18]. The most pressing
challenge is the development of a coherent research agenda for
discovering and developing a broader range of insecticides, with
novel modes of action that can circumvent emerging resistance to
existing insecticides, in particular, pyrethroid-based insecticides
[11]. The second challenge is the development of interventions that
affect vectors that do not rest or feed indoors and are therefore not
susceptible to current tools. The final critical challenge is the
development of novel approaches that permanently reduce the high
PLoS Medicine | www.plosmedicine.org
88
vectorial capacities of the dominant malaria vectors in sub-Saharan
Africa. Genetic control programs based on permanent reduction of
the vectorial capacities of natural vector populations have received
the most attention to date [19,20], but the Consultative Group also
considers the development of other novel approaches [18].
Diagnostics
Methods for measuring transmission are central to an
elimination agenda. Current methods for measuring transmission
that may be applied in endemic areas are time-consuming,
expensive, and too insensitive for use in conditions of low and
nonuniform infection [21,22]. Some years after regional elimina-
tion, as immunity declines, infection is likely to be symptomatic
and may become the best marker of resumed transmission.
However, during the early elimination phase in regions previously
experiencing high transmission, populations will retain clinical
immunity and will not experience symptomatic disease with every
infection [23]. Thus, the main challenge identified by the malERA
Consultative Group on Diagnoses and Diagnostics and discussed
in detail in their article and in the article on Cross-cutting Issues
for Eradication [24,25] is to find a robust, sensitive, and specific
standardized method for assessment of transmission intensity in
the intervening period when transmission continues at low and
nonrandom levels. Improved serological tests have been suggested
[26], but other minimally invasive biomarkers could be consid-
ered. This information will be essential for modeling potential
effects of various interventions alone, or in combination, and for
assessing efficacy of transmission–reducing vaccines and drugs.
Other challenges for diagnostics discussed by the Consultative
4 January 2011 | Volume 8 | Issue 1 | e1000406
iMedPub :: Thematic Collections :: Volume 2

Figure 3. The malERA governance bodies. Image credit: Fusión Creativa.
doi:10.1371/journal.pmed.1000406.g003
Group include the need for tools that can rapidly detect and
monitor unexpectedly high transmission that leads to outbreaks
and that can identify reintroduction of infections that may be
asymptomatic [16,24].
Beyond the Tools: Supporting Strategies and the
Knowledge Base
Modeling and Harmonized Data Systems
Substantial advances have been made recently in computational
approaches for modeling malaria epidemiology and in model-
based approaches to economic evaluation [27–29]. As discussed by
the malERA Consultative Group on Modeling [16], a significant
research challenge for malaria eradication will be to integrate
these new approaches into the planning of elimination, surveil-
lance, monitoring, and evaluation, and to create appropriate
interfaces for different user communities, including researchers,
global and national policy makers, and local-level planners.
Modeling can inform the definition of TPPs for new tools and
intervention strategies and will be needed throughout a global
eradication campaign to analyze the likely effects on malaria and
of various elimination strategies and the costs of these strategies
[30].
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
Importantly, a single unifying model will be insufficient to meet
all these needs, so multiple modeling efforts need to be coordinated
and made accessible to everyone. This harmonization and
validation process will require close, iterative collaboration between
software engineers, researchers, and malariologists to develop the
necessary computer systems and connectivity (cyberinfrastructure).
It will also necessitate the creation and maintenance of properly
annotated and accessible malariometric databases that include all
the research results needed to insert parameters into the models and
the model outputs. How this can be achieved is considered in detail
by the Consultative Group in their article [16].
Enabling Technologies and Platforms
The development of new tools for elimination is critically
dependent on a vibrant and coherent agenda for basic sciences.
We believe there are at least two potentially transformative
developments that need to be pursued. First, continuous
laboratory culture of P. vivax, P. ovale, and P. malariae needs to be
developed to provide an essential platform for studying the biology
of the liver stages and sexual forms of these parasites. These forms
could be important targets of intervention strategies with drugs,
vaccines, or other biological or chemical agents aimed at
interrupting transmission. Second, systems analyses of transcrip-
5 January 2011 | Volume 8 | Issue 1 | e1000406
89
World Health Report 2012: No Health Without Research
Box 2. Key Examples of Critical Research
Needed to Support Elimination and
Eradication of Plasmodium falciparum and
Plasmodium vivax.
N In vitro culture and study of hypnozoites (persistent liver
stages) of P. vivax
N Drugs to be used for mass drug administration to clear
infections and provide prophylaxis to prevent new
infections
N Vaccines that target different stages of the parasite life
cycle, or the mosquito, with the key goal of interrupting
transmission
N New vector control approaches for (i) outdoor biting/
resting mosquitoes and (ii) achieving permanent reduc-
tions of vectorial capacity in areas where transmission is
predominantly due to the highly efficient vector A.
gambiae
N New approaches for fast and accurate assessment of
transmission at community level
N When to press the elimination button? Tool kits to
scientifically determine ‘‘health system readiness’’ for a
switch to elimination efforts
N New collaborative approaches to use of mathematical
modeling to inform TPPs, and expected outcomes of
mixes of intervention
N Strengthened monitoring and evaluation tools and
strategies for interrupting transmission that are linked
and embedded in the health and social systems
tion, proteome, and metabolome libraries, rapid screening of drug
libraries, high-throughput approaches to antigen identification,
and the functional definition of gene products are all feasible but
not yet fully exploited, but would bring important new tools to the
bench scientist and to field operations. These and other aspects of
enabling technologies and platforms are considered in detail in the
articles prepared by the malERA Consultative Groups on Basic
Science and Enabling Technologies and on Cross-cutting Issues
for Eradication [25,31].
Figure 4. Key research and development issues and their position in relation to the different epidemiological phases towards
eradication. Image credit: Fusión Creativa.
doi:10.1371/journal.pmed.1000406.g004
PLoS Medicine | www.plosmedicine.org
90
Health Systems Integration, Operational
Research, and Effectiveness-Decay Analysis
The previous formal attempt at global eradication of malaria
(1955–1969) depended largely on vertical operations that often
bypassed health systems and their health services because it was
assumed that eradication operations could be run most efficiently
in this way. Many of the elimination efforts failed, because the
health systems failed, leading to a pessimistic view that malaria can
only be eliminated where economic progress, governance, and
efficient health systems are in place to support maintenance of
conditions necessary to block transmission [32,33].
It is now clear that the long-term solution to malaria elimination
and eradication will require a systems approach in which malaria-
specific interventions and actions are integrated into existing
health systems [34]. To achieve this, research is needed into health
systems, their readiness to optimize novel programs, systems, tests,
or other interventions, and their continuing performance [35–37].
During their deliberations, the malERA Consultative Group on
Health Systems and Operational Research identified the need for
a substantial research approach to establish and validate a tool kit
that allows effectiveness-decay analysis of health system impedi-
ments to effective and equitable coverage of malaria interventions
and that allows decisions to be made on the degree of possible
integration of interventions into an existing health system [16,38].
A further critical component of the research agenda identified by
this Consultative Group is the development and validation of a
decision-making framework to guide the move from control to
elimination.
Finally, but equally importantly, the article by the malERA
Consultative Group on Monitoring, Evaluation, and Surveillance
considers the need to investigate the performance of surveillance,
monitoring, and evaluation by new and old technologies [39,40] and
to evaluate optimal strategies for implementation of surveillance as an
active responsive intervention to further reduce transmission [41].
Training
The last time the world community tried to eliminate malaria,
so the joke goes, the only thing that was eliminated was
malariologists. For a renewed malaria eradication campaign to
6 January 2011 | Volume 8 | Issue 1 | e1000406
iMedPub :: Thematic Collections :: Volume 2

have a chance to succeed, it will be essential to train the
malariologists and scientists in the multiple disciplines needed for
an eradication campaign that might last 50 years, especially in
endemic countries. This need cannot be overemphasized. The
malaria research community remains small and often dominated
by the views and strategies of scientists who sit far away from the
problems. A massive effort to train, empower, and sustain research
capacity in the endemic countries will be a critical factor for the
success of improved control efforts and for the ultimate elimination
and eradication of malaria.
Concluding Remarks
The past 2 years have reinvigorated an old malaria paradigm in
which reduction of transmission is the driving strategy for malaria
interventions. The malaria community has now used the malERA
process to propose a research and development agenda that will be
essential for regional elimination and eventual global eradication
of malaria. Not every tool or strategy considered by the malERA
Consultative Groups (see Box 2) will be essential in every situation
(see Figure 4), but the complexity and heterogeneity, and in some
places, the sheer intensity of transmission, demand that we start
without delay to prepare for the most difficult challenges. This
focus on the end goal of eradication must not displace our
determination and efforts to continue to scale up ongoing efforts
for control and to include a research agenda for reducing
morbidity in areas of continuing moderate or high transmission.
Rather, it must encourage us to supplement our efforts with a
structured agenda that can realize the ultimate goal of eradication
envisaged by the Global Malaria Action Plan and the Roll Back
Malaria Partnership. An important lesson we can learn from other
disease elimination efforts is that complacency is dangerous. The
parasite and the vector are always evolving, and the human
environment is always changing. Thus, new research questions will
continually arise during the course of elimination [42], and active
malaria research, particularly on the development of new tools,
must continue up to the point when eradication is finally achieved.
We anticipate that the results of research efforts proposed by our
Consultative Groups for each stage of progression, from scaling up
for improved control to the elimination phases, will have great
synergy in design and application.
Past efforts at disease eradication, successful or otherwise, have
highlighted the importance of sustained commitment from local
communities, civil society, policy leaders, and the scientific
community to implement research in the context of the desired
integration of services, sector wide approaches, harmonisation of
activities, and long-term funding commitment. Thus, research
areas such as social science or research into direct and indirect
economic benefits of malaria eradication also need to be
strengthened. With these drivers in place, and the development
References
1. Roll Back Malaria Partnership (2008) The global malaria action plan for a
malaria free world. Geneva: RBM, Available: http://www.rollbackmalaria.org/
gmap/gmap.pdf.
2. World Health Organization (2010) World malaria report. Geneva: World
Health Organization, Available: http://whqlibdoc.who.int/publications/2010/
9789241564106_eng.pdf.
3. World Health Organization (2010) Weekly epidemiological record 85, 229-236.
Available: http://www.who.int/wer/2010/wer8524.pdf.
4. World Health Organization (2010) Weekly epidemiological record 85, 461-472.
Available: http://www.who.int/wer/2010/wer8547/en/index.html.
5. Bill & Melinda Gates Foundation Malaria Forum (2007) Day-2 transcript. 17-
10-2007. Available: http://www.gatesfoundation.org/speeches-commentary/
Pages/melinda-french-gates-2007-malaria-forum.aspx.
PLoS Medicine | www.plosmedicine.org
iMedPub :: Thematic collections :: Volume 2
World Health Report 2012: No Health Without Research
of the new tools we describe briefly here and in the other articles in
this Supplement, it may be possible to fulfil the dream that malaria
eradication can be achieved within the lifetime of young scientists
just embarking on their careers, even in the most difficult areas
where current tools/strategies have proven to be insufficient. The
time course may be long, but to have a chance of realizing that
dream, the commitment to starting those research and develop-
ment efforts must begin now.
Supporting Information
Text S1 malERA governance bodies
Found at: doi:10.1371/journal.pmed.1000406.s001 (0.05 MB
DOC)
Text S2 malERA launch meeting participants
Found at: doi:10.1371/journal.pmed.1000406.s002 (0.06 MB
DOC)
Acknowledgments
The Malaria Eradication Research Agenda (malERA) initiative was
constituted as a scientific consultative process to identify knowledge gaps
and new tools that will be needed to eradicate malaria globally. It was
managed by three governance bodies comprising a Steering Committee, an
International Advisory Committee, and a Leadership Council. Continuity
and cross-sector communication within the different program elements was
facilitated by a Secretariat based at the Barcelona Centre for International
Health Research (Hospital Clı́nic, Universitat de Barcelona), led by
Almudena Legarda in close collaboration with Jessica Milman at the Bill &
Melinda Gates Foundation. Very special thanks to Matiana González-
Silva, scientific writer, and Carolyn Daher, Patricia Garcı́a, and Desiree
van der Mei. For a listing of the malERA governance bodies see Text S1.
Participants in the malERA initiative launch meeting are detailed in Text
S2.
Author Contributions
ICMJE criteria for authorship read and met: PLA GVB MA FB CEC FHC
OKD BG BFH MML KM RDN CVP MHR RES LS MT. Agree with the
manuscript’s results and conclusions: PLA GVB MA FB CEC FHC OKD
BG BFH MML KM RDN CVP MHR RES LS MT. Wrote the first draft
of the paper: PLA GVB MT. Contributed to the writing of the paper: PLA
GVB MA FB CEC FHC OKD BG BFH MML KM RN CVP MHR RES
LS MT. Contributed to the development of the research agenda: PLA
GVB MA FB CEC FHC OKD BG BFH MML KM RDN CVP MHR
RES LS MT. Members of the malERA steering committee which
coordinated the malERA review process and guided the direction of the
other papers in the series: PLA MA FB CEC FHC OKD BG BFH KM
CVP MHR RES LS MT. Members of the malERA International Advisory
Committee: GB MML. Contributed conceptually to the whole process
leading to this paper: PLA GVB MA FB CEC FHC OKD BG BFH MML
KM RDN CVP MHR RES LS MT.
6. Mendis K, Rietveld A, Warsame M, Bosman A, Greenwood B, et al. (2009)
From malaria control to eradication: The WHO perspective. Trop Med Int
Health 14: 802–809.
7. Nájera J, González-Silva M, Alonso, PL (2010) Some lessons for the future from
the Global Malaria Eradication Programme (1955-1969). PLoS Med 8:
e1000412. doi:10.1371/journal.pmed.1000412.
8. Feachem RGA, Phillips AA, Targett GA, eds (2009) Shrinking the malaria map:
A prospectus on malaria elimination. San Francisco: The Global Health Group,
Global Health Sciences University of California, San Francisco.
9. Plowe CV (2009) The evolution of drug-resistant malaria. Trans R Soc Trop
Med Hyg 103: S11–S14.
10. Dondorp AM, Nosten F, Yi P, Das D, Phyo AP, et al. (2009) Artemisinin
resistance in Plasmodium falciparum malaria. N Engl J Med 361: 455–467.
7 January 2011 | Volume 8 | Issue 1 | e1000406
91
World Health Report 2012: No Health Without Research
11. Ranson H, Abdallah H, Badolo A, Guelbeogo WM, Kerah-Hinzoumbé C, et al.
(2009) Insecticide resistance in Anopheles gambiae: data from the first year of a
multi-country study highlight the extent of the problem. Malar J 8: 299.
12. Greenwood B, Fidock DA, Kyle DE, Kappe SH, Alonso PL, et al. (2008)
Malaria: Progress, perils and prospects for eradication. J Clin Invest 118:
1266.
13. Nyunt MM, Plowe CV (2007) Pharmacologic advances in the global control and
treatment of malaria: combination therapy and resistance. Clin Pharmacol Ther
82: 601–605.
14. The malERA Consultative Group on Drugs (2011) A research agenda for malaria
eradication: Drugs. PLoS Med 8: e1000402. doi:10.1371/journal.pmed.1000402.
15. The malERA Consultative Group on Vaccines (2011) A research agenda for
malaria eradication: Vaccines. PLoS Med 8: e1000398. doi:10.1371/jour-
nal.pmed.1000398.
16. The malERA Consultative Group on Modeling (2011) A research agenda for
malaria eradication: Modeling. PLoS Med 8: e1000403. doi:10.1371/
journal.pmed.1000403.
17. Enayati A, Hemingway J (2010) Malaria management: Past, present, and future.
Ann Rev Entomol 55: 569–591.
18. The malERA Consultative Group on Vector Control (2011) A research agenda
for malaria eradication: Vector control. PLoS Med 8: e1000401. doi:10.1371/
journal.pmed.1000401.
19. Terenius O, Marinotti O, Sieglaff D, James AA (2008) Molecular genetic
manipulation of vector mosquitoes. Cell Host Microbe 13: 417–423.
20. Sinkins SP, Gould F (2006) Gene drive systems for insect disease vectors. Nat
Rev Genet 7: 427–35.
21. Bell D, Wongsrichanalai C, Barnwell JW (2006) Ensuring quality and access for
malaria diagnosis: how can it be achieved? Nat Rev Microbiol. pp S7–S20.
22. Paris DH, Imwong M, Faiz AM, Hasan M, Yunus EB, et al. (2007) Loop-
mediated isothermal PCR (LAMP) for the diagnosis of falciparum malaria.
Am J Trop Med Hyg 77: 972–976.
23. World Health Organization (2007) Malaria elimination: a field manual for low
and moderate endemic countries. Geneva: World Health Organization.
24. The malERA Consultative Group on Diagnoses and Diagnostics (2011) A
research agenda for malaria eradication: Diagnosis and diagnostics. PLoS Med
8: e1000396. doi:10.1371/journal.pmed.1000396.
25. The malERA Consultative Group on Integration Strategies (2011) A research
agenda for malaria eradication: Cross-cutting issues for eradication. PLoS Med
8: e1000404. doi:10.1371/journal.pmed.1000404.
26. Bousema T, Youssef RM, Cook J, Cox J, Alegana VA, et al. (2010) Serologic
markers for detecting malaria in areas of low endemicity, Somalia 2008.
Emerging Infect Dis 16: 392–399.
27. Ghani AC, Sutherland CJ, Riley EM, Drakeley CJ, Griffinet JT, et al. (2009)
Loss of population levels of immunity to malaria as a result of exposure-reducing
PLoS Medicine | www.plosmedicine.org
92
interventions: consequences for interpretation of disease trends. PLoS One 4:
e4383. doi:10.1371/journal.pone.0004383.
28. McKenzie FE, Bossert WH (2005) An integrated model of Plasmodium falciparum
dynamics. J Theor Biol 232: 411–426.
29. Smith T, Killeen GF, Maire N, Ross A, Molineaux L, et al. (2006) Mathematical
modeling of the impact of malaria vaccines on the clinical epidemiology and
natural history of Plasmodium falciparum malaria: Overview. Am J Trop Med Hyg
75: 1–10.
30. Smith DL, Smith T, Hay SI (2009) ‘‘Measuring malaria for elimination.
Feachem RGA, Phillips AA, Targett GA, eds. Shrinking the malaria map: A
prospectus on malaria elimination. San Francisco: The Global Health Group,
Global Health Sciences University of California, San Francisco. pp 108–126.
31. The malERA Consultative Group on Basic Science and Enabling Technologies
(2011) A research agenda for malaria eradication: Basic science and enabling
technologies. PLoS Med 8: e1000399. doi:10.1371/journal.pmed.1000399.
32. Alilio MS, Bygbjerg IC, Breman JG (2004) Are multilateral malaria research and
control programs the most successful? Lessons from the past 100 years in Africa.
Am J Trop Med Hyg 71: 268–278.
33. Nájera JA (2001) Malaria control: Achievements, problems and strategies.
Parassitologia 43: 1–89.
34. Tanner M, de Savigny D (2008) Malaria eradication back on the table. Bull
World Health Organ 86: 82–83.
35. de Savigny D, Adam T (2009) Health systems thinking for health systems
strengthening. Geneva: World Health Organization.
36. World Health Organization (2007) Everybody’s business: Strengthening health
systems to improve health outcomes: WHO’s framework for action. Geneva:
World Health Organization.
37. World Health Organization (2008) World health report 2008 – Primary health
care: More than ever. Geneva: World Health Organization.
38. The malERA Consultative Group on Health Systems and Operational Research
(2011) A research agenda for malaria eradication: Health systems and
operational research. PLoS Med 8: e1000397. doi:10.1371/jour-
nal.pmed.1000397.
39. Hay SI, Smith DL, Snow RW (2008) Measuring malaria endemicity from
intense to interrupted transmission. Lancet Infect Dis 8: 369–378.
40. Drakeley C, Cook J (2009) Potential contribution of sero-epidemiological
analysis for monitoring malaria control and elimination: Historical and current
perspectives. Adv Parasitol 69: 299–352.
41. The malERA Consultative Group on Monitoring, Evaluation, and Surveillance
(2011) A research agenda for malaria eradication: Monitoring, evaluation, and
surveillance. PLoS Med 8: e1000400. doi:10.1371/journal.pmed.1000400.
42. Breman JG, Quadros CA, Dowdle WR, Foege WH, Henderson DA, et al.
(2011) The role of research in viral disease eradication and elimination
programs: Lessons for malaria eradication. PLoS Med 8: e1000405.
doi:10.1371/journal.pmed.1000405.
8 January 2011 | Volume 8 | Issue 1 | e1000406
iMedPub :: Thematic Collections :: Volume 2
 World Health Report 2012: No Health Without Research

NIH Disease Funding Levels and Burden of Disease

Leslie A. Gillum1, Christopher Gouveia1, E. Ray Dorsey3, Mark Pletcher2, Colin D. Mathers4, Charles E.
McCulloch2, S. Claiborne Johnston1,2*
1 Department of Neurology, University of California San Francisco, San Francisco, California, United States of America, 2 Department of Epidemiology and Biostatistics,
University of California San Francisco, San Francisco, California, United States of America, 3 Department of Neurology, Johns Hopkins University Medical Center, Baltimore,
Maryland, United States of America, 4 Department of Health Statistics and Informatics, World Health Organization, Geneva, Switzerland

Abstract
Background: An analysis of NIH funding in 1996 found that the strongest predictor of funding, disability-adjusted life-years
(DALYs), explained only 39% of the variance in funding. In 1998, Congress requested that the Institute of Medicine (IOM)
evaluate priority-setting criteria for NIH funding; the IOM recommended greater consideration of disease burden. We
examined whether the association between current burden and funding has changed since that time.

Methods: We analyzed public data on 2006 NIH funding for 29 common conditions. Measures of US disease burden in 2004
were obtained from the World Health Organization’s Global Burden of Disease study and national databases. We assessed
the relationship between disease burden and NIH funding dollars in univariate and multivariable log-linear models that
evaluated all measures of disease burden. Sensitivity analyses examined associations with future US burden, current and
future measures of world disease burden, and a newly standardized NIH accounting method.

Results: In univariate and multivariable analyses, disease-specific NIH funding levels increased with burden of disease
measured in DALYs (p = 0.001), which accounted for 33% of funding level variation. No other factor predicted funding in
multivariable models. Conditions receiving the most funding greater than expected based on disease burden were AIDS
($2474 M), diabetes mellitus ($390 M), and perinatal conditions ($297 M). Depression ($719 M), injuries ($691 M), and
chronic obstructive pulmonary disease ($613 M) were the most underfunded. Results were similar using estimates of future
US burden, current and future world disease burden, and alternate NIH accounting methods.

Conclusions: Current levels of NIH disease-specific research funding correlate modestly with US disease burden, and
correlation has not improved in the last decade.

Citation: Gillum LA, Gouveia C, Dorsey ER, Pletcher M, Mathers CD, et al. (2011) NIH Disease Funding Levels and Burden of Disease. PLoS ONE 6(2): e16837.
doi:10.1371/journal.pone.0016837
Editor: Joseph Ross, Yale University School of Medicine, United States of America
Received September 17, 2010; Accepted January 3, 2011; Published February 24, 2011
Copyright: � 2011 Gillum et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by NIH/NCRR/OD UCSF-CTSI grant KL2 RR024130. The funders had no role in study design, data collection and analysis,
decision to publish, or preparation of the manuscript.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: Clay.johnston@ucsfmedctr.org

Introduction correlated with funding. Disability-adjusted life years (DALYs)—a

The National Institutes of Health (NIH) is the largest public
funder of biomedical research worldwide [1,2], with a budget that
has grown from $11.9 billion in 1996 to $28.5 billion in 2006 [3].
In the mid-1990s, Congress and the public raised concerns that
disease-specific funding allocations by the NIH failed to adequate-
ly reflect burden of disease and incorporate public input [4]. In
response, Congress requested that the Institute of Medicine (IOM)
assess the NIH funding apportionment processes. In its 1998
report, Scientific Opportunities and Public Needs: Improving Priority Setting
and Public Input at the National Institutes of Health, the IOM
recommended improved tracking of disease-specific funding and
development of a new priority-setting process [4].
A landmark study comparing disease burden to NIH funding
levels was published in 1999 [5]. For 29 common conditions,
the study examined a variety of measures of societal burden,
recognizing that none by itself completely captures relative impacts
of diseases. Disease incidence and prevalence were unrelated
to funding, while mortality and years of life lost (YLLs) weakly
measure that estimates the equivalent number of healthy years lost
due to disability or early death [6,7]—were more strongly
predictive. Using DALYs as the best single predictor, only 39%
of the variance in NIH funding could be explained. The prior
analysis was limited to evaluation of univariate predictors, and did
not attempt to evaluate whether funding aligned with other
measures of disease burden. The NIH Reform Act of 2006 re-
emphasized the NIH’s role in identifying research to meet public
health challenges, and mandated submission of a biennial report to
Congress regarding disease-specific funding amounts [8]. There
has been no recent comprehensive study of US disease burden and
NIH funding, and an analysis of only one of its institutes has been
performed [9].
To determine whether the NIH has developed processes that
better align funding with burden, we assessed the correlation
between NIH funding and burden of disease, and compared
results with those reported 10 years ago [5]. We also considered
other potential predictors of funding and assessed the association
of NIH funding with estimates of future and global disease burden.

iMedPub :: Thematic collections :: Volume 2 93

PLoS ONE | www.plosone.org 1 February 2011 | Volume 6 | Issue 2 | e16837
World Health Report 2012: No Health Without Research
NIH Funding Levels and Burden of Disease

Methods Institutes and Centers categorized spending in a variety of

In a cross-sectional study, we compared measures of US and
world disease burden and sociopolitical factors from 2004 to NIH
funding levels in 2006. The study design was modeled on methods
previously established [5], which used 1994 burden data and 1996
NIH funding to reflect an expected lag in availability of data on
disease burden. Each disease was defined using pre-specified sets of
International Classification of Diseases, 9th revision, Clinical Modification
(ICD-9) codes, which were applied to public sources of information
on disease burden [10].
Data Sources
Amounts of NIH funding for disease categories were obtained
directly from the NIH for the year 2006 (Table 1). These estimates
were annually consolidated from figures supplied by indivi-
dual NIH Institutes and Centers (http://www.nih.gov/news/
fundingresearchareas.htm) [11]. In 2006 and prior years, NIH
manners to satisfy diverse reporting requirements, and calculated
condition-specific total funds in a non-mutually exclusive manner.
Thus, funding for a particular trial or biomarker could have been
attributed to multiple conditions.
Data denoting disease burden were collected from multiple sources
(Table 1). First, world and North American disease-specific data were
obtained from the Global Burden of Disease (GBD) Project the
World Health Organization (WHO) [12,13,14]. We used GBD’s
North American data for US estimates. The GBD systematically
collects timely disease-specific epidemiologic information and models
missing data to estimate a variety of measures of burden [15,16]. We
used 2004 GBD estimates because they were likely to be the most
recent publicly available data at the time decisions were made for the
2006 funding cycle. No other centralized, systematic source of broad
national disease-specific burden estimates was identified.
We evaluated five disease burden categories from the GBD:
incidence, prevalence, mortality, YLLs, and DALYs [14]. YLLs

Table 1. NIH Research Funds and Measures of Disease Burden for 29 Conditions.

Measure of Disease Burden in North America*

Condition or Disease NIH Research Funds thousands (rank)

millions of dollars Years of Disability-Adjusted

(% of total) Incidence Prevalence Mortality Life Lost Life-Years

AIDS 2902 (24.3) 142 (19) 1275 (15) 14 (16) 279 (13) 583 (14)
Diabetes mellitus 1038 (8.7) 1200 (9) 21663 (2) 84 (7) 563 (7) 1473 (6)
Perinatal conditions 789 (6.6) 45 (21) 3516 (7) 17 (14) 578 (6) 793 (10)
Breast cancer 718 (6.0) 222 (13) 1875 (11) 53 (10) 488 (8) 684 (12)
Dementia 643 (5.4) 714 (10) 3108 (8) 132 (5) 306 (11) 1359 (8)
Alcohol abuse 511 (4.3) 2641 (6) 9553 (4) 8 (17) 121 (16) 1837 (4)
Dental and oral disorders 413 (3.5) 109774 (1) 41152 (1) 0 2 (27) 267 (18)
Cirrhosis 408 (3.4) 43 (22) 303 (21) 30 (12) 360 (10) 455 (16)
Ischemic heart disease 398 (3.3) 1336 (7) 2347 (10) 531 (1) 2695 (2) 3048 (3)
Schizophrenia 364 (3.1) 42 (23) 1561 (13) 1 (24) 5 (25) 522 (15)
Injuries 355 (3.0) 3747 (5) 241 (23) 182 (2) 3448 (1) 4484 (2)
Pneumonia 351 (3.0) 4178 (4) 75 (27) 68 (9) 294 (12) 315 (17)
Prostate cancer 348 (2.9) 149 (18) 1032 (16) 38 (11) 152 (14) 253 (20)
Stroke 342 (2.9) 373 (12) 2733 (9) 176 (4) 791 (4) 1336 (9)
Depression 335 (2.8) 16417 (3) 8207 (5) 1 (24) 3 (26) 4564 (1)
Asthma 283 (2.4) 1278 (8) 19100 (3) 4 (20) 53 (19) 755 (11)
Colorectal cancer 269 (2.3) 150 (17) 713 (18) 70 (8) 466 (9) 609 (13)
Lung cancer 266 (2.2) 196 (15) 706 (20) 181 (3) 1331 (3) 1384 (7)
Sexually transmitted diseases 264 (2.2) Not available Not available 0 2 (27) 65 (26)
Parkinson’s disease 208 (1.7) 90 (20) 1025 (17) 21 (13) 66 (18) 263 (19)
Tuberculosis 150 (1.3) 11 (27) 11 (28) 1 (24) 7 (24) 10 (29)
Multiple sclerosis 110 (0.9) 9 (28) 176 (25) 4 (20) 45 (20) 118 (24)
Epilepsy 103 (0.9) 207 (14) 1677 (12) 2 (23) 31 (22) 160 (22)
Ovarian cancer 102 (0.9) 24 (26) 143 (26) 17 (14) 140 (15) 161 (21)
Cervical cancer 97 (0.8) 29 (24) 225 (24) 7 (18) 86 (17) 125 (23)
Chronic obstructive pulmonary disorder 67 (0.6) 429 (11) 6923 (6) 132 (5) 644 (5) 1647 (5)
Uterine cancer 28 (0.2) 25 (25) 301 (22) 7 (18) 45 (20) 84 (25)
Peptic ulcer disease 17 (0.1) 192 (16) 712 (19) 4 (20) 24 (23) 40 (27)
Otitis media 17 (0.1) 17679 (2) 1360 (14) 0 1 (29) 35 (28)

Funding rates by disease were obtained from the NIH for 2006. Estimates of incidence, prevalence, mortality, disability-adjusted life-years lost, and years-of-life-lost are
total annual counts, in 1000s, for North America, obtained from the 2004 update of the World Health Organization’s Global Burden of Disease project.
doi:10.1371/journal.pone.0016837.t001

94 iMedPub :: Thematic Collections :: Volume 2

PLoS ONE | www.plosone.org 2 February 2011 | Volume 6 | Issue 2 | e16837

for each disease were calculated by totaling the differences
between life expectancy and age at death. DALYs were estimated
based on YLL and on standardized weighting schemes for
disability applied to those surviving with disease [6,7]. GBD
estimates for the US were based on analyses of comprehensive
death certificate data, national incidence estimates, and a
systematic review of published epidemiological studies.
To evaluate other estimates of disease burden, we used 2004 US
inpatient and outpatient healthcare databases categorized by
diagnosis, each of which includes large samples with weights to
generate nationally representative estimates (Supporting Table
S1). The number of hospital discharges, total length of stay, and
mean hospital charges for disease-specific principal diagnoses were
derived from the National Inpatient Sample (NIS) [17]. The
number of visits to emergency departments and outpatient hospital
clinics were derived from the National Hospital Ambulatory Care
Survey (NHAMCS) and the National Ambulatory Care Survey
(NAMCS) [18]. In both NIS and NHAMCS, community hospitals
were defined as non-federal, short-stay hospitals [19,20].
Public interest could also influence funding levels through
lobbying efforts, additional funding support from private founda-
tions, or by directly stimulating the interest of investigators, and
this could influence funding levels. To begin to assess the influence
of public interest, we determined the number of disease-specific
newspaper articles published in the top 10 US newspapers with
highest distribution, as well as broadcast television news reports
from national networks, using disease-specific keyword searches of
LexisNexis News [21] and Vanderbilt Television News Archive
[22]. To estimate the influence of specific interest group advocacy,
the total US disease-specific charity revenue was similarly
calculated for public charities receiving more than $500,000 in
public support [23].
Scientific productivity in a given area could stimulate further
interest from researchers and the NIH, and could result in targeted
funding. As surrogate measures of scientific productivity, we tallied
the number of disease-specific patents submitted [24] and articles
published and listed in PubMed [25] in the 10 medical journals
with the highest impact factor scores using key word searches [26].
Analysis
The relationship between 2006 NIH funding levels and 2004
US disease burden metrics was designated the a priori primary
analysis. All predictor and outcome variables were log-transformed
to reduce positive skew.
Univariate linear regression was first performed to replicate the
prior study design [5]. This approach was preferred to correlation
to allow for adjustment and to acknowledge that funding level was
the dependent variable to be predicted. Expected funding levels in
2006 were calculated with correction for the log-transformation by
applying measures of disease burden to fitted models predicting
funding [27,28]. To account for inflation, the actual and expected
funding levels for 1996 were inflated to 2006 dollars [29].
Standard, forward stepwise multivariable linear regression analysis
included all variables of disease burden, including those with
p,0.05 in the final model. We did not evaluate interactions. F-
statistics were used to estimate differences in explanatory power
(significant if p,0.05). A separate forward stepwise multivariable
model was constrained so that diseases with no burden would
receive zero funding, mimicking a theoretical funding process that
determines allocations proportional to the disease burden. For this
model, we defined the dependent variable as the ratio of dollars to
DALYs for each of the included conditions. Separate multivariable
models included measures of public interest and scientific
productivity. In additional sensitivity analyses, we evaluated the
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
NIH Funding Levels and Burden of Disease
association of funding levels with worldwide disease burden in
2004, and with projections for 2015 and 2030 for both the US and
worldwide [30]. A final analysis evaluated whether a new NIH
accounting method for disease-specific funding introduced in 2007
produced different results. Explanatory power of the models was
estimated with adjusted R2 values; an adjusted R2 value was also
determined for the analysis of 1996 funding levels using data
derived from this prior publication. The Stata statistical package
(Version 10, College Station, Texas) was used for all analyses.
Results
In 2006, the total NIH budget was $28.5 billion, with $11.9
billion devoted to the 29 included conditions. Disease funding
ranged from $17 million (M) for peptic ulcer disease and otitis
media, to $2902 M for AIDS, with a median of $335 M
(6standard deviation $537 M; Table 1). Other metrics from the
GBD (Table 1); US inpatient, emergency room, and outpatient
(Supporting Table S1); and public interest and scientific
opportunity (Supporting Table S2) varied by disease.
In the univariate analysis, NIH funding was most strongly
associated with burden of disease measured in DALYs (p = 0.001;
Table 2). YLLs (p = 0.03), inpatient hospital discharges (p = 0.05),
and total hospital days (p = 0.02) were also associated with funding
levels.
In standard multivariable analysis, DALYs was the only
significant predictor of NIH funding level retained in the final
model, so the analysis became univariate. In 2006, the degree of
correlation between NIH funding and disease burden as measured
by DALYs alone was less than in 1996: Only 33% of NIH funding
variance was explained in 2006 compared to 39% in 1996.
Differences between actual and expected funding based on burden
of disease in DALYs were estimated for 2006 and compared to 1996
funding levels (Table 3; Figure 1). Depression received the least
funding compared to expected, and AIDS the most, consistent with
findings from 1996. Relative to expected funding, AIDS, diabetes,
and perinatal conditions were the three diseases with the largest
amounts of funding, while depression, injuries, and COPD received
the least funding (Table 3). The largest positive 10-year gains in
actual NIH funding relative to expected were AIDS (+$809 M),
perinatal conditions (+$420 M), and diabetes (+$193 M); by
contrast, injuries (2$578 M), depression (2$541 M), chronic
obstructive pulmonary disease (2$512 M), and ischemic heart
disease (2$459 M) decreased most sharply (Figure 1).
In standard multivariable regression models including measures
of public interest and scientific productivity, the total charity
revenue for a given disease in 2006 (p = 0.04) was also predictive of
funding in addition to DALYs (p = 0.006). A model including both
variables explained 41% of the variation in NIH funding levels.
In multivariable models constrained to require that diseases
resulting in no burden of illness receive no NIH funding
(equivalent to requiring an intercept of zero-zero in the regression
line, expressed in the dashed line of Figure 2), expected funding
amounts were generally similar to those found with the
unconstrained multivariable model (Table 3; Figure 3).
To determine if NIH funding might better correlate with world
or future disease burden, we performed sensitivity analyses with
global measures and future projections (2015 and 2030), all
derived from the GBD project (Supporting Table S3). When
restricted to global measures, mortality (p = 0.05) and DALYs
(p = 0.001) were predictive of funding in univariate analyses
(adjusted R2 values 0.11 and 0.30), but only DALYs were retained
in all the multivariable models of both global measures and future
predictions. Correlation of funding with disease burden was not
95
3 February 2011 | Volume 6 | Issue 2 | e16837
World Health Report 2012: No Health Without Research
NIH Funding Levels and Burden of Disease

Table 2. Univariate Predictors of NIH Disease-Specific Funding in Fiscal Year 2006.

Predictor Predicted change in funding associated with a 2-fold increase in the predictor*

Relative Increase 95% CI P-Value Adjusted R-squared Value

Disease Burden Measure

Incidence 1.05 (0.91–1.20) 0.51 20.02
Prevalence 1.15 (0.97–1.36) 0.10 0.06
Mortality 1.12 (0.98–1.27) 0.08 0.08
Disability-adjusted life-years 1.37 (1.16–1.63) 0.001 0.33
Years-of-life-lost 1.15 (1.01–1.31) 0.03 0.12
Number of hospital discharges 1.21 (1.00–1.46) 0.05 0.11
Total hospital days 1.24 (1.03–1.48) 0.02 0.15
Average hospital charges 1.49 (0.76–2.93) 0.23 0.02
Outpatient physician visits 1.11 (0.90–1.37) 0.30 0.004
Outpatient hospital and emergency room visits 1.10 (0.92–1.31) 0.27 0.01
Public Interest Measure
Newspaper articles 1.13 (1.02–1.25) 0.02 0.14
Television news broadcasts 1.19 (1.04–1.36) 0.02 0.17
Charity revenue 1.07 (1.02–1.11) 0.004 0.24

*Predictors and outcome are log-transformed to reduce positive skew.

doi:10.1371/journal.pone.0016837.t002

improved when data utilizing new NIH accounting methods was
used (adjusted R2 = 0.27) compared to prior methods (adjusted
R2 = 0.34) applied to 2007 data, the first year for which the new
methods were available.
Discussion
In the 10 years since an initial assessment of the correlation of
NIH funding with disease burden [5] and an IOM report
recommending new NIH funding priority-setting criteria [4], NIH
funding is no better aligned with US disease burden. Furthermore,
diseases that were previously funded more than expected–such as
AIDS, breast cancer, and diabetes–continue to receive funding
greater than predicted by burden of disease, while most conditions
that were previously underfunded remain underfunded. Adding
measures of disease burden to the model and constraining it to
assure that diseases with no burden would receive no funding
minimally affected the overall relationship between burden
and funding. Neither global nor future disease burden were
more closely related to NIH funding, and newly implemented
NIH disease-specific accounting practices did not improve the
correlation.
Although the IOM report on NIH funding recommended
ongoing assessment of alignment of NIH funding with disease
burden, it also recognized other important criteria for setting
funding priorities [4]. These criteria included research quality,
scientific innovation and opportunity, portfolio diversification, and
infrastructure building. Other experts have proposed a similar
framework to guide funding decisions [31]. Additional factors to
consider may include transmissibility or population risk, collateral
benefits to disease control, and public interest. Given these other
potential contributors to decision-making about disease funding,
perfect alignment with DALYs or any other measure of disease
burden would not be expected.
It is unclear why particular conditions remain under- or
overfunded relative to disease burden. The difficulty of attributing
basic science research—a large portion of the NIH portfolio—to
96
PLoS ONE | www.plosone.org
individual diseases complicates implementation of any disease-
based allocation process [32]; however, some discrepancies are
particularly dramatic. Spending for AIDS research, the disease
with greatest funding compared to expected, may be justified due
to the potential threat associated with its spread, to past successes
in treating and even eliminating other infectious diseases, and to a
greater burden in lower income countries. However, AIDS
funding remained greater than predicted even when worldwide
and projected burden were considered, and strong political
influences may be important in maintaining high levels of funding
in the US. Also, congressionally-mandated research support for
rare illnesses may explain greater funding for some diseases with
little burden [33,34,35]. Conditions typically associated with
substance use or mental health diagnoses tended to be under-
funded (e.g., lung cancer, chronic obstructive pulmonary disease,
alcoholism, and depression). Charity revenue, used as a proxy for
disease-specific interest-group advocacy, was associated with
funding levels and may have contributed through lobbying efforts
or by providing collateral support for research and training to
encourage NIH submissions in specific disease areas. The
availability of proven cost-effective interventions (e.g., tobacco-
related prevention strategies) [36] may also influence funding since
development of new interventions may be unnecessary when
effective strategies have already been identified [37].
Over the 10-year interval, funding for several conditions notably
increased compared to expected. For example, the National
Institute of Allergy and Infectious Diseases initiated a concerted
response to bioterrorism following the terrorist attacks on Sept. 11,
2001, anthrax incidents, and severe acute respiratory syndrome
(SARS) outbreak in 2002, and this may have augmented support
for pneumonia-related research [38]. Conversely, the relative
funding increase for perinatal conditions appears primarily due to
a 55% reduction in associated DALYs over the last decade since
funding increases over the same period paralleled overall growth
in the NIH budget [39]. Finally, an increase in relative funding for
diabetes research may have been precipitated by Congressional
actions in 1997 requiring development of a comprehensive
iMedPub :: Thematic Collections :: Volume 2
4 February 2011 | Volume 6 | Issue 2 | e16837
 World Health Report 2012: No Health Without Research
NIH Funding Levels and Burden of Disease

Table 3. Ranked Differences between Expected and Actual NIH Funding According to Year of Funding and United States Disease
Burden Measure(s) Used.

Millions of Dollars
(ascending rank*)

1996 2006

Standard Standard Multivariable with Constrained

Univariate{ Univariate{ Public Interest Variables1 Multivariable"

Condition or Disease
Depression 2178 (1) 2719 (1) 2689 (2) 2951 (2)
Perinatal Conditions 2124 (2) 297 (27) 114 (23) 2194 (11)
Stroke 2121 (3) 2278 (6) 2288 (5) 2170 (13)
Injuries 2113 (4) 2691 (2) 2123 (12) 2721 (3)
Chronic obstructive pulmonary disorder 2101(5) 2613 (3) 2357(3) 2554(5)
Pneumonia 252(6) 27 (21) 154 (27) 270 (27)
Peptic ulcer disease 246(7) 2105 (14) 275 (14) 218 (19)
Lung cancer 246(8) 2364 (5) 2328 (4) 21358 (1)
Schizophrenia 237(9) 244 (18) 129 (24) 94 (24)
Ischemic heart disease 230(10) 2490 (4) 2731 (1) 2721 (4)
Uterine cancer 229(11) 2146 (10) 294 (13) 2111 (15)
Asthma 225(12) 2198 (8) 2235 (6) 40 (21)
Otitis media 219(13) 297 (15) 270 (15) 227 (18)
Colorectal cancer 216(14) 2168 (9) 21 (19) 2519 (7)
Ovarian cancer 215(15) 2135 (11) 2196 (9) 2337 (9)
Epilepsy 210(16) 2133 (12) 2226 (7) 2101 (16)
Parkinson’s disease 23(17) 290 (17) 2183 (10) 2363 (8)
Cervical cancer 28(18) 2113 (13) 244 (17) 2223 (10)
Prostate cancer 32 (19) 56 (22) 23 (18) 68 (22)
Tuberculosis 44 (20) 89 (23) 96 (22) 97 (25)
Multiple sclerosis 52 (21) 295 (16) 2222 (8) 296 (17)
Sexually transmitted disease 58 (22) 110 (24) 153 (26) 198 (26)
Alcohol abuse 61 (23) 2202 (7) 152 (25) 2152 (14)
Cirrhosis 67 (24) 25 (20) 255 (16) 2178 (12)
Dental and oral disorders 130 (25) 113 (25) 74 (20) 90 (23)
Dementia 183 (26) 18 (19) 2159 (11) 2524 (6)
Diabetes mellitus 197(27) 390 (28) 160 (28) 382 (28)
Breast cancer 346 (28) 258 (26) 92 (21) 39 (20)
AIDS 1664 (29) 2474 (29) 2306 (29) 1835 (29)

*Ascending rank, from most underfunded disease condition (indicated by negative numbers) to most overfunded (positive numbers) as predicted by each model.
{ Univariate linear regression of the association between 2004 disease-specific Disability-Adjusted Life-Years (DALYs) and the outcome NIH dollars. Differences between
expected and actual funding levels in 1996 are adjusted for inflation to 2006 dollar equivalents, but are otherwise unchanged from those reported by Gross et al.
{ Standard univariate linear regression of the outcome NIH dollars as predicted by disease-specific DALYs. A stepwise forward multivariable model retained only DALYs
as a predictor.
1 Standard multivariable linear regression of the outcome NIH dollars as predicted by disease-specific DALYs and charity revenue.
" The constrained model is the multivariable linear regression model where the predicted NIH dollars are obligated to be proportional to disease-specific DALYs after
adjustment for total number of hospital discharges and average hospital charges.
doi:10.1371/journal.pone.0016837.t003

diabetes research plan and allocating $150 million to a new
funding program for Type I Diabetes Research [40].
Lack of improvement in alignment between funding and disease
burden may not indicate neglect of the 1998 IOM recommenda-
tions by the NIH; there are several other possible explanations.
First, basic science research has consistently accounted for 55% of
NIH spending and it is difficult to credit specific disease for much
of this research, contributing uncertainty to the analysis and
reducing correlations between funding levels and disease burden.
Second, NIH funds committed at the time of the 1998 IOM
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
publication could not be redistributed until their associated
projects were completed, sometimes five or more years later; as
a result, reallocations would be delayed for several years. Third,
the distribution of funding among NIH Institutes is determined by
Congress and incorporates input from NIH itself, scientists, health
care providers, and special interest groups. Thus, fiscal and
political constraints likely additionally tempered the NIH’s ability
to implement the IOM recommendations. Finally, without regard
to overall disease funding alignment, substantial financing ($1.8
billion, 6.3% 2006 NIH budget) was invested in the creation of
97
5 February 2011 | Volume 6 | Issue 2 | e16837
World Health Report 2012: No Health Without Research
NIH Funding Levels and Burden of Disease

Figure 1. Ten-year Comparison of Differences Between Actual and Expected Disease-Specific NIH Funding Relative to US Burden of
Disease in DALYs. A comparison of differences between actual and expected funding values as predicted by DALYs burden alone in 1996 (light blue)
and 2006 (navy). Negative values reflect actual funding dollars less than expected and positive values represent actual funding dollars more than expected.
doi:10.1371/journal.pone.0016837.g001

Figure 2. NIH Funding in 2006 and US Disease Burden in DALYs in 2004 for 29 Common Medical Conditions. The solid line represents
the results of a traditional multivariable analysis, showing the relationship between US disease-specific DALYs burden and actual 2006 NIH funding
dollars. The dashed line projects NIH funding levels in a similar multivariable model that requires that a disease with no burden receives no funding
(constrained model). Though the models produce similar results, several diseases that would be considered overfunded in one model are considered
underfunded in the other. For example, cervical cancer appears to be overfunded relative to the dashed line, while it is underfunded relative to the
solid one.
doi:10.1371/journal.pone.0016837.g002

98 iMedPub :: Thematic Collections :: Volume 2

PLoS ONE | www.plosone.org 6 February 2011 | Volume 6 | Issue 2 | e16837
 World Health Report 2012: No Health Without Research
NIH Funding Levels and Burden of Disease

Figure 3. Differences Between Actual and Expected Disease-Specific Funding in 2006. Determinations of actual funding relative to
expected funding were generally similar among separate analytic models predicting funding levels from disease burden measures. Univariate results
are based on DALYs alone (navy), the only variable retained in a stepwise forward multivariable model. A traditional multivariable model including
public interest variables (grey-blue) retained only DALYs and total charity revenue in the model. A constrained multivariable (light blue)
model required an intercept of zero-zero to impose a requirement that conditions with no burden received no funding and retained DALYs, total
number of US hospital discharges, and mean charge per hospitalization in 2004.
doi:10.1371/journal.pone.0016837.g003

three new centers (National Institution for Biomedical Imaging
and Bioengineering, Center for Complementary and Alternative
Medicine, and National Center for Minority Health and Health
Disparities) [41], establishment of the cross-cutting Roadmap
Initiatives [42], and expanded emphasis on career training awards
[43].
The NIH has recently taken steps to integrate more effectively
the IOM priority-setting criteria. In 2007, the Division of Program
Coordination, Planning, and Strategic Initiatives (DPCPSI) was
established for the purposes of identifying scientific opportunities,
public health challenges, and scientific knowledge gaps, and to
improve portfolio analysis and priority-setting [44]. In 2009, a
more consistent and transparent system for the tracking of disease-
based funding was launched. Also, as part of the NIH’s $10.4
billion allocation in the American Recovery & Reinvestment Act,
$400 million will be dedicated to comparative effectiveness
research that specifically evaluates the effects of clinical manage-
ment on comprehensive public health outcomes such as mortality,
morbidity, and quality of life [45].
Our study has several limitations. First, the accounting of
disease funding in 2006 by NIH is not standardized nor is the
reliability known [32]. However, no alternative source of
information is publicly available and historical and new account-
ing methods yielded similarly poor correlations with burden.
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
Second, we did not evaluate other sources of federal, nonprofit,
and industry funding. Conditions well-funded by organizations
other than the NIH may justify a corresponding decrease in NIH
funding [46]. However, other sources of federal funding–The
Centers for Disease Control and Prevention, Agency for
Healthcare Research and Quality, and the Food and Drug
Administration–distributed only 3.3% of all Department of Health
and Human Services dollars dedicated to life sciences research in
2006. Similarly, the total life science research dollars spent by the
Department of Defense and Department of Veterans Affairs was
equivalent to less than 7% of the total amount disbursed by the
NIH [47,48]. Thus, the total dollars spent on complementary
projects by other federal institutions does not fully explain the poor
correlation between disease burden and NIH funding. Although
biomedical and pharmaceutical industry research funding in the
United States was 1.6 times the amount allocated by the NIH in
2006, fully 49% was dedicated to supporting clinical trials less
likely to have a widespread public health benefit [49]. Unfortu-
nately, funding by disease for these sectors is not available, except
for limited therapeutic areas [50], and no such figures exist for
private foundations. Third, global estimates of disease burden
from the WHO GBD are uncertain due to incompleteness and
bias, particularly in low-income countries [51]. However, these
estimates are the best available and are particularly accurate for
99
7 February 2011 | Volume 6 | Issue 2 | e16837
World Health Report 2012: No Health Without Research
the US. Fourth, to permit comparisons to the prior study and to
make the study feasible, many diseases and conditions funded by
the NIH were omitted from our analysis. Still, an assessment of the
responsive of NIH to prior recommendation was central to our
study and power would not be expected to dramatically impact
our findings. Fifth, the prior analysis utilized estimates for world
market economies [5], while we used North American numbers.
The data we used better approximates US burdens, strengthening
our conclusions, but could affect the assessment of interval change.
Since we found a poorer correlation with funding than previous
estimates, this did not impact our conclusions. Finally, none of the
measures of disease burden individually or collectively fully
captures the health and economic cost of these conditions. A
better metric might capture the true societal cost of disease
through a comprehensive assessment of total healthcare costs and
a valuation of both deaths and DALYs [52].
Overall, funding levels today are less well aligned with DALYs
compared to 10 years ago, suggesting that the IOM’s 1998
priority-setting recommendations have not been implemented
effectively. Accounting for other measures of disease burden did
not substantially improve alignment. As a recipient of substantial
governmental support, clear articulation of the rationale for NIH
spending may be expected by Congress and the public, and a lack
of clear alignment with measures of public burden could
encourage special interests to further erode the scientific
independence of NIH or to raise questions about its management.
References
1. Moses H, 3rd, Dorsey ER, Matheson DH, Thier SO (2005) Financial anatomy
of biomedical research. JAMA 294: 1333–1342.
2. European Science Foundation (2007) Present status and future strategy for
medical research in Europe. European Science Foundation, www.esf.org,
accessed October 28, 2008.
3. NIH Office of Budget NIH appropriations history by institute and center. Office
of Budget, Office of the Director, National Institutes of Health, Department of
Health and Human Services, http://officeofbudget.od.nih.gov/UI/Appropria-
tionsHistoryByIC.htm, accessed May 5, 2008.
4. Institute of Medicine (1998) Scientific opportunities and public needs: Improving
priority setting and public input at the NIH. National Institutes of Health
Priority Setting Committee, Institute of Medicine. National Academy Press,
http://www.nap.edu, accessed March 1, 2008.
5. Gross CP, Anderson GF, Powe NR (1999) The relation between funding by the
National Institutes of Health and the burden of disease. N Engl J Med 340:
1881–1887.
6. Morrow RH, Bryant JH (1995) Health policy approaches to measuring and
valuing human life: conceptual and ethical issues. Am J Public Health 85:
1356–1360.
7. Murray CJL (1996) Rethinking DALYs. In: Murray CJL, Lopez AD, eds. The
global burden of disease. Cambridge: Harvard School of Public Health on behalf
of the World Health Organization and the World Bank. pp 1–98.
8. National Institutes of Health Reform Act of 2006.
9. Nahin RL (2005) Identifying and pursuing research priorities at the National
Center for Complementary and Alternative Medicine. FASEB J 19: 1209–1215.
10. World Health Organization (2006) Global burden of disease: Cause Category
and ICD-9 Codes.: World Health Organization. The World Bank Group,
http://www.dcp2.org/pubs/GBD/3/Table/3.A2, accessed February 8, 2008.
11. National Institutes of Health (2008) Estimates of funding for various diseases,
conditions, research areas. National Institutes of Health, Department of Health
and Human Services, http://www.nih.gov/news/fundingresearchareas.htm,
accessed May 16, 2007.
12. Lopez AD (2005) The evolution of the Global Burden of Disease framework for
disease, injury and risk factor quantification: developing the evidence base for
national, regional and global public health action. Global Health 1: 5.
13. Lopez AD, Mathers CD, Ezzati M, Murrary CJL, Jamison DT (2006) Global
burden of disease and risk factors. New York: Oxford University Press.
14. World Health Organization (2008) The global burden of disease: 2004 update.
Geneva, Switzerland: WHO Press.
15. Murray CJ, Lopez AD (1997) Global mortality, disability, and the contribution
of risk factors: Global Burden of Disease Study. Lancet 349: 1436–1442.
16. Murray CJL, Lopez AD, Harvard School of Public Health, World Health
Organization, World Bank (1996) The global burden of disease: a comprehen-
sive assessment of mortality and disability from diseases, injuries, and risk factors
in 1990 and projected to 2020. CambridgeMA: Harvard University Press. xxxii,
990 p.
100
PLoS ONE | www.plosone.org
NIH Funding Levels and Burden of Disease
The use of more consistent accounting methods for disease
funding, more comprehensive measures of burden and future risk
that include impact on health and expenditures, and the timely
dissemination of benchmarks on the alignment of disease burden
to funding could help to make NIH funding priorities more
rationale and transparent.
Supporting Information
Table S1 Additional Measures of Disease Burden for 29
Conditions.
(DOC)
Table S2 Public Interest and Other Measures for 29 Conditions.
(DOC)
Table S3 World and Future Disability-Adjusted Life-Years as
Predictors of NIH Disease-Specific Funding in Fiscal Year 2006.
(DOC)
Author Contributions
Conceived and designed the experiments: LAG ERD MP CDM SCJ.
Analyzed the data: LAG SCJ. Contributed reagents/materials/analysis
tools: LAG CEM SCJ. Wrote the paper: LAG SCJ. Acquisition of data:
LAG CG CDM SCJ. Critical revision for intellectual content: LAG ERD
MP CDM CEM SCJ. Study Supervision: SCJ LAG. Obtained funding:
LAG.
17. Healthcare Cost and Utilization Project (2008) HCUPnet: National statistics on
hospital stays. Agency for Healthcare Research and Quality, http://hcupnet.
ahrq.gov/, accessed December 15, 2007.
18. National Center for Health Statistics (2008) Ambulatory health care data:
National Ambulatory Medical Care Survey (NAMCS). Centers for Disease
Control and Prevention, US Department of Health and Human Services,
http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm, accessed November
29, 2007.
19. Agency for Healthcare Research and Quality (2005) Introduction to the HCUP
Nationwide Inpatient Sample (NIS), http://www.hcup-us.ahrq.gov/db/nation/
nis/NIS Introduction 2005.jsp, accessed May 1, 2008.
20. National Center for Health Statistics (2008) Ambulatory health care data:
National Hospital Ambulatory Medical Care Survey (NHAMCS). Centers for
Disease Control and Prevention, US Department of Health and Human
Services, http://www.cdc.gov/nchs/about/major/ahcd/ahcd1.htm, accessed
November 29, 2007.
21. LexisNexis News (2008) ed: Reed Elsevier Inc, http://www.lexisnexis.com,
accessed June 5, 2008.
22. Vanderbilt Univeristy (2008) Vanderbilt Television News Archive. http://
tvnews.vanderbilt.edu/, accessed July 15, 2008.
23. Charity Navigator (2006) Your guide to intelligent giving, www.charitynavigator.
com, accessed August 14, 2008.
24. United States Patent and Trademark Office (2008) USPTO patent full-text and
image database: patent applications. United States Patent and Trademark
Office, http://patft.uspto.gov/, accessed July 11, 2008.
25. National Center for Biotechnology Information (2008) PubMed Central. US
National Library of Medicine, http://www.ncbi.nlm.nih.gov/sites/entrez/,
accessed August 7, 2008.
26. Saha S, Saint S, Christakis DA (2003) Impact factor: a valid measure of journal
quality? J Med Libr Assoc 91: 42–46.
27. Bradu D, Mundlak Y (1970) Estimation of lognormal linear models. Journal of
the American Statistical Association 65: 198–211.
28. Goldberger AS (1968) The interpretation and estimation of Cobb-Douglas
functions. Econometrica 36: 464–472.
29. Friedman MS (2008) The Inflation Calculator. S. Morgan Friedman, http://
www.westegg.com/inflation/, accessed November 11, 2008.
30. Mathers CD, Loncar D (2006) Projections of global mortality and burden of
disease from 2002 to 2030. PLoS Med 3: e442.
31. Lichtenberg FR, American Enterprise Institute for Public Policy Research (1999)
The allocation of public funds for biomedical R&D. Washington, D.C.: AEI
Press. v, 41 p.
32. National Institutes of Health (1993) Cost-savings resulting from NIH research
support. 2nd ed. NIH Publication 93-3109. Rockville, MD.
33. Food and Drug Administration (2008) Clinical studies of safety and effectiveness
of orphan products. Research project grant (RO1). RFA-FD08-001. U.S. Food
and Drug Administration, U.S. Department of Health and Human Services,
iMedPub :: Thematic Collections :: Volume 2
8 February 2011 | Volume 6 | Issue 2 | e16837

http://grants.nih.gov/grants/guide/rfa-files/RFA-FD-08-001.html, accessed
November 1, 2008.
34. Food and Drug Administration (2008) Office of Orphan Products Development.
U.S. Food and Drug Administration, http://www.fda.gov/orphan/, accessed
November 1, 2008.
35. NIH Office of Rare Diseases (2008) National Institutes of Health, Department of
Health and Human Services, http://rarediseases.info.nih.gov/, accessed
November 1, 2008.
36. Ad Hoc Committee on Health Research Relating to Future Intervention
Options (1996) Investing in health research and development. Geneva: World
Health Organization.
37. Katz DA, Muehlenbruch DR, Brown RL, Fiore MC, Baker TB (2004)
Effectiveness of implementing the agency for healthcare research and quality
smoking cessation clinical practice guideline: a randomized, controlled trial.
J Natl Cancer Inst 96: 594–603.
38. National Institute of Allergy and Infectious Diseases (2006) Fiscal Year 2007
Budget Justification. http://www3.niaid.nih.gov/about/overview/budget/
PDF/fy07 justification.pdf, accessed Nov. 1, 2008.
39. Gitterman DP, Hay WW, Jr. (2008) That sinking feeling, again? The state of
National Institutes of Health pediatric research funding, fiscal year 1992–2010.
Pediatr Res 64: 462–469.
40. NIH Almanac (2008) National Institute of Diabetes and Digestive and Kidney
Illnesses — Mission and history. http://www.nih.gov/about/almanac/organization/
NIDDK.htm, accessed Jan. 12, 2009.
41. National Institutes of Health (2009) NIH Institutes, Centers and Offices.
National Institutes of Health, Department of Health and Human Services,
http://www.nih.gov/icd/index.html, accessed April 27, 2009.
42. NIH Office of Budget (2009) NIH Common Fund/Roadmap. Office of Budget,
Office of the Director, National Institutes of Health, Department of Health and
Human Services, http://officeofbudget.od.nih.gov/ui/2008/Final%20Roadmap.
pdf, accessed April 27, 2009.
43. National Institutes of Health (2008) NIH career award funding 1998–2007.
Office of Research Information Systems, National Institutes of Health,
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
NIH Funding Levels and Burden of Disease
Department of Health and Human Services, http://report.nih.gov/NIH
Investment/PDF sectionwise/NIH Extramural DataBook PDF/NEDB%
20CAREER%20DEVELOPMENT.pdf, accessed April 27, 2009.
44. National Institutes of Health (2007) NIH director selects Dr. Alan M. Krensky as
NIH deputy director for the Office of Portfolio Analysis and Strategic Initiatives
(OPASI) National Institutes of Health, Department of Health and Human
Services, http://www.nih.gov/news/pr/jan2007/opasi-25.htm, accessed May
5, 2009.
45. National Institute of Neurological Disorders and Stroke (2009) Grant
opportunities in comparative effectiveness research. National Institutes of
Health, Department of Health and Human Services, http://www.ninds.nih.
gov/recovery/arra-funding/go-cer.htm, accessed May 5, 2009.
46. Varmus H (1999) Evaluating the burden of disease and spending the research
dollars of the National Institutes of Health. N Engl J Med 340: 1914–1915.
47. National Science Foundation (2007) Federal funds for research and develop-
ment: Fiscal years 2004–06. NSF 07-323. John E. Jankowski, project officer.:
National Science Foundation, Division of Science Resources Statistics, http://
www.nsf.gov/statistics/nsf07323/pdf/tab3.pdf, accessed May 1, 2008.
48. Department of Veterans Affairs (2006) Fiscal Year 2006 performance and
accountability report: consolidated financial statements. Washington D.C.:
Department of Veterans Affairs, Office of Management. pp 285–290.
49. ArchstoneConsulting, Burns L (2009) The biopharmaceutical sector’s impact on
the U.S. economy: analysis at the national, state, and local levels. Stamford, CT:
Archstone Consulting. 18 p.
50. Dorsey ER, Vitticore P, De Roulet J, Thompson JP, Carrasco M, et al. (2006)
Financial anatomy of neuroscience research. Ann Neurol 60: 652–659.
51. Mathers CD, Salomon JA, Ezzati M, Begg S, Lopez AD (2006) Sensitivity and
uncertainty analyses for burden of disease and risk factor estimates. Global
burden of disease and risk factors. New York: Oxford University Press. pp
399–426.
52. Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS (2006) Effect of a
US National Institutes of Health programme of clinical trials on public health
and costs. Lancet 367: 1319–1327.
101
9 February 2011 | Volume 6 | Issue 2 | e16837
World Health Report 2012: No Health Without Research

Geographical Representativeness of Published and

Ongoing Randomized Controlled Trials. The Example of:
Tobacco Consumption and HIV Infection
Nizar Ahmad1,2,3*, Isabelle Boutron1,2,3, Agnes Dechartres1,2,3, Pierre Durieux4,5, Philippe Ravaud1,2,3,4
1 INSERM, U738, Paris, France, 2 AP-HP (Assistance Publique des Hôpitaux de Paris), Hôpital Hôtel Dieu, Centre d’Epidémiologie Clinique, Paris, France, 3 Université Paris
Descartes, Faculté de Médecine, Paris, France, 4 Centre de Médecine Fondée sur les Preuves (EHESP, HAS, INSERM, APHP), Paris, France, 5 Santé Publique et Informatique
Médicale, Faculté de Médecine, Université Paris Descartes, Paris, France

Abstract
Background: The challenge for evidence-based healthcare is to reduce mortality and the burden of diseases. This study
aimed to compare where research is conducted to where research is needed for 2 public health priorities: tobacco
consumption and HIV infection.

Methods: We identified randomized controlled trials (RCTs) included in Cochrane systematic reviews published between
1997 and 2007 and registered ongoing RCTs identified in January 2009 through the World Health Organization’s
International Clinical Trials Registry Platform (WHO-ICTRP) evaluating interventions aimed at reducing or stopping tobacco
use and treating or preventing HIV infection. We used the WHO and World Bank reports to classify the countries by income
level, as well as map the global burden of disease and mortality attributable to tobacco use and HIV infection to the
countries where the trials performed.

Results: We evaluated 740 RCTs included in systematic reviews and 346 ongoing RCTs. For tobacco use, 4% of RCTs
included in systematic reviews and 2% of ongoing trials were performed in low- and middle-income countries, even though
these countries represented 70% of the mortality related to tobacco use. For HIV infection, 31% of RCTs included in
systematic reviews and 33% of ongoing trials were performed in low- and middle-income countries, even though these
countries represented 99% of the mortality related to HIV infection.

Conclusions: Our results highlight an important underrepresentation of low- and middle-income countries in currently
available evidence (RCTs included in systematic reviews) and awaiting evidence (registered ongoing RCTs) for reducing or
stopping tobacco use and treating or preventing HIV infection.

Citation: Ahmad N, Boutron I, Dechartres A, Durieux P, Ravaud P (2011) Geographical Representativeness of Published and Ongoing Randomized Controlled
Trials. The Example of: Tobacco Consumption and HIV Infection. PLoS ONE 6(2): e16878. doi:10.1371/journal.pone.0016878
Editor: Geneviève Chêne, Institut National de la Santé et de la Recherche Médicale, France
Received September 17, 2010; Accepted January 6, 2011; Published February 9, 2011
Copyright: � 2011 Ahmad et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by a grant "Recherche sur la Recherche" from the "Délégation Interrégionale à la Recherche Clinique (DIRC), Ile de France" and
a grant from the Institut National du Cancer (INCA), France. The funders had no role in study design, data collection and analysis, decision to publish, or
preparation of the manuscript.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: nizar.ahmad@htd.aphp.fr

Introduction services, etc.). Addressing context-specific questions is fundamental to

The main challenge for public health policies is to reduce mortality
and the burden of disease. This aim implies having adequate evidence
to guide health care providers and policymakers for implementing the
most efficient and cost-effective interventions. More than two-thirds
of the world’s population live in low- and middle-income countries
[1,2], and 93% of the burden of preventable mortality occurs in these
countries [3]. The shortage of resources in low- and middle-income
countries paradoxically increases the need for reliable healthcare
evidence to prioritize the use of these scarce resources [4,5]. Such
evidence is essential to determine the prevention and therapeutic
strategies that work best but also under which circumstances they
work and how best to deliver them [6].
In this context, it could be useful to tailor research to the needs of
the particular population and to the context (socio-cultural, access to
designing interventions that improve health [7,8]. Results of research
performed in high-income countries cannot be easily transposed to
low- and middle-income countries. The extent to which questions
addressed by researchers in high-income countries are relevant to
patients and physicians in low-income countries is largely unknown.
This study aimed to compare where research is conducted to where
research is needed according to the attributable burden of disease,
mortality and prevalence. We compared high-income countries to low-
or middle-income countries defined by the 2008 World Bank
classification. We considered both current available evidence (i.e.,
randomized controlled trials [RCTs] included in systematic reviews)
and awaiting evidence (i.e., RCTs registered on the World Health
Organization’s International Clinical Trials Registry Platform [WHO-
ICTRP]) that provide data for current clinical decision making and
future public health policies. We focused on 2 public-health priorities,

102 iMedPub :: Thematic Collections :: Volume 2

PLoS ONE | www.plosone.org 1 February 2011 | Volume 6 | Issue 2 | e16878

Figure 1. Flow diagram of Cochrane systematic reviews selected in the study and retrieval of randomized controlled trials (RCTs)
from these reviews.
doi:10.1371/journal.pone.0016878.g001
tobacco use and HIV infection, because they are ranked among the top
5 leading causes of mortality in the world [9].
Methods
I. Classification of countries
We classified the countries according to the 2008 World Bank
classification that divides the world into income categories (low,
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs
middle, and high) according to economies in terms of gross region
national income per capita and in geographic regions for low- and
middle-income economies only. We considered trials performed in
high-income countries (country with a per capita income of $$11
906 in 2008 [e.g., USA, European Union [EU], Canada, Australia
and other]) as one group and trials performed in low- and middle-
income countries as another group [9,10]. In addition, we grouped
low- and middle-income countries into 6 geographic regions
103
2 February 2011 | Volume 6 | Issue 2 | e16878
World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs

Figure 2. Flow diagram of reports of ongoing RCTs selected from International Clinical Trial Registries* via the World Health
Organization (WHO) portal. * International clinical trial registries: Clinicaltrials.gov, Australian New Zealand Clinical Trials Registry, Chinese Clinical
Trial Register, Clinical Trials Registry - India, German Clinical Trials Register, Iranian Registry of Clinical Trials, Japan Primary Registries Network,
International Standard Randomised Controlled Trials Number, The Netherlands National Trial Register, Sri Lanka Clinical Trials Registry.
doi:10.1371/journal.pone.0016878.g002

Table 1. Characteristics of RCTs of tobacco use and HIV infection included in selected Cochrane systematic reviews and ongoing
RCTs selected from international clinical trial registries.

Total Tobacco use HIV infection

Ongoing Ongoing Ongoing

All RCTs RCTs All RCTs RCTs All RCTs RCTs

n = 1 086 (%) n = 740 (%) n = 346 (%) n = 668 (%) n = 556 (%) n = 112 (%) n = 418 (%) n = 184 (%) n = 234 (%)

Intervention
N PT 489 (45.0) 283 (38.2) 206 (59.5) 252 (37.7) 201 (36.2) 51 (45.5) 237 (56.7) 82 (44.6) 155 (66.2)
N NPT 597 (55.0) 457 (61.8) 140 (40.5) 416 (62.3) 355 (63.8) 61 (54.5) 181 (43.3) 102 (55.4) 79 (33.8)

Number of participating countries

N1 1 004 (92.5) 695 (94.0) 309 (89.3) 639 (95.7) 529 (95.1) 110 (98.2) 365 (87.3) 166 (90.2) 199 (85.0)
N 1–5 38 (3.5) 15 (2.0) 23 (6.7) 8 (1.2) 6 (1.1) 2 (1.8) 30 (7.2) 9 (4.9) 21 (9.0)
N 6–10 10 (0.9) 4 (0.5) 6 (1.7) 4 (0.6) 4 (0.7) 0 6 (1.4) 0 6 (2.6)
N .10 12 (1.1) 4 (0.5) 8 (2.3) 2 (0.3) 2 (0.4) 0 10 (2.4) 2 (1.1) 8 (3.4)
N Not reported 22 (2.0) 22 (3.0) -- 15 (2.2) 15 (2.7) -- 7 (1.7) 7 (3.8) --

RCTs: randomized controlled trials. PT: pharmacologic intervention. NPT: nonpharmacologic intervention.
doi:10.1371/journal.pone.0016878.t001

104 iMedPub :: Thematic Collections :: Volume 2

PLoS ONE | www.plosone.org 3 February 2011 | Volume 6 | Issue 2 | e16878
 World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs

Table 2. Tobacco use: smoking prevalence, burden of disease (in disability-adjusted life-years [DALYs]), attributable mortality,
RCTs included in Cochrane systematic reviews and ongoing RCTs selected from international clinical trial registries for high-income
and low- and middle-income countries.

Attributable Smoking Total PT NPT

DALYs Attributablemortality prevalence Total NPT ongoing ongoing ongoing
Countries (thousands) (thousands) (millions) RCTs PT RCTs RCTs RCTs RCTs RCTs

n = 72 919 n = 4 802 n = 11 220 n = 541 n = 200 n = 341 n = 112 n = 51 n = 61

(%) (%) (%) (%) (%) (%) (%) (%) (%)

High-income countries 18 900 (25.9) 1 462 (30.4) 202 (18.0) 517 (95.6) 187 (93.5) 330 (96.8) 110 (98.2) 51 (100) 59 (96.7)
Low- or middle-income 54 019 (74.1) 3 340 (69.6) 920 (82.0) 24 (4.4) 13 (6.5) 11 (3.2) 2 (1.8) 0 2 (3.3)
countries
N East Asia and Pacific 16 518 (22.7) 1 059 (22.1) 429 (38.0) 9 (1.6) 4 (2.0) 5 (1.5) 2 (1.8) 0 2 (3.3)
N Eastern Europe and 14 769 (20.3) 897 (18.7) 122 (11.0) 7 (1.3) 4 (2.0) 3 (0.9) 0 0 0
central Asia
N South Asia 14 452 (19.8) 879 (18.3) 178 (15.0) 1 (0.2) 0 1 (0.3) 0 0 0
N Latin America and 3 957 (5.4) 250 (5.2) 98 (9.0) 5 (0.9) 4 (2.0) 1 (0.3) 0 0 0
Caribbean
N Middle east and North 2 153 (2.9) 121 (2.5) 37 (3.0) 1 (0.2) 1 (0.5) 0 0 0 0
Africa
N Sub-Saharan Africa 2 171 (3.0) 1 (0.2) 0 1 (0.3) 0 0 0

RCTs: randomized controlled trials. PT: pharmacologic intervention. NPT: nonpharmacologic intervention. GBD: global burden of disease.
doi:10.1371/journal.pone.0016878.t002

according to the 2008 World Bank classification (i.e., East Asia
and Pacific, Eastern Europe and Central Asia, Latin America and
Caribbean, Middle East and North Africa, South Asia, Sub-
Saharan Africa).
II. Where research is conducted
To describe the context (i.e., geographic region and socioeco-
nomic situation of the countries where the trial took place) of
research performed in these fields, we focused on the context of
currently available evidence and awaiting evidence that could be
used for clinical decision making. Because RCTs are the gold
standard to assess the effectiveness of interventions, we evaluated the
context of all RCTs included in Cochrane systematic reviews (i.e.,
context of current evidence) and all ongoing RCTs in the registries
of the WHO-ICTRP (i.e., context of awaiting evidence). We
focused on systematic reviews performed by the Cochrane
collaboration because this international, not-for-profit organisation
is a source of high-quality, reliable health information providing up-
to-date knowledge about the effects of health care. Further, an
important goal of the Cochrane collaboration is the dissemination of
information to low- and middle-income countries. The Cochrane
Library is freely available to all residents of low-income countries.
a. Identification of RCTs included in Cochrane
systematic reviews. We identified all systematic reviews of
interventions aimed at reducing or stopping tobacco use and those
aimed at treating or preventing HIV infection published in the
Cochrane database of systematic reviews between January 1997
and December 2007. We searched for the following terms in the
title, abstract or the MeSH terms: ‘‘smoking cessation’’ OR
‘‘tobacco use cessation’’ OR ‘‘smoking reduction’’ OR ‘‘tobacco
reduction’’ OR ‘‘smoking abstinence’’ OR ‘‘tobacco abstinence’’
for tobacco use and ‘‘HIV’’ OR ‘‘human immunodeficiency virus’’
OR ‘‘AIDS’’ OR ‘‘acquired immunodeficiency syndrome’’ OR
‘‘sexually transmitted diseases’’ for HIV infection. Titles and
abstracts were then screened by one of us (NA) to identify the
relevant systematic reviews. The full texts for selected abstracts
were retrieved and reviewed by one of us (NA) to determine the
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
eligibility of systematic reviews for inclusion. Another reviewer (IB)
checked the adequate selection of the abstracts and confirmed the
adequate exclusion of full-text articles.
Reports were included if the study was identified as a systematic
review or meta-analysis of interventions aimed at reducing tobacco
consumption or preventing or treating HIV infection. We
excluded systematic reviews focusing on a specific setting, except
when this setting concerned developing countries (e.g., interven-
tion for tobacco cessation in a dental setting) or a specific
population (e.g., tobacco cessation for hospitalised patients).
Excluding these systematic reviews allowed for a relatively
homogeneous sample of systematic reviews aimed at providing
conclusions on treatment effect whatever the context. We also
excluded systematic reviews of prevention or treatment of
complications of HIV infection (e.g., opportunistic infections,
Kaposi’s sarcoma) and those evaluating an intervention for
another disease among subjects infected with HIV (e.g., treatment
of anaemia or tuberculosis infection in people with HIV infection).
b. Identification of ongoing RCTs. In January 2009, we
searched the WHO-ICTRP (http://www.who.int/trialsearch/)
for all ongoing RCTs registered in the platform’s 10 clinical
trials registries: Australian New Zealand Clinical Trials Registry
(ANZCTR), Chinese Clinical Trial Register (ChiCTR),
ClinicalTrials.gov, Clinical Trials Registry - India (CTRI),
German Clinical Trials Register (DRKS), Iranian Registry of
Clinical Trials (IRCT), International Standard Randomised
Controlled Trials Number (ISRCTN.org), Japan Primary
Registries Network, The Netherlands National Trial Register
(NTR), and Sri Lanka Clinical Trials Registry (SLCTR). We used
this platform because it allows access to all primary registries
meeting the WHO criteria and the requirements of the
International Committee of Medical Journal Editors [11].
We searched for RCTs in the ‘‘conditions’’ menu of the
platform using the topics ‘‘smoking’’ for tobacco use and ‘‘HIV
infection’’ for HIV infection and in the ‘‘recruitment’’ menu of the
‘‘advanced search’’ feature, using ‘‘recruiting’’ to select only
ongoing RCTs.
105
4 February 2011 | Volume 6 | Issue 2 | e16878
World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs

106 iMedPub :: Thematic Collections :: Volume 2

PLoS ONE | www.plosone.org 5 February 2011 | Volume 6 | Issue 2 | e16878

Figure 3. Prevalence, burden of disease (in disability-adjusted life-years [DALYs]), attributable mortality and percentage of RCTs
included in Cochrane systematic reviews and ongoing RCTs selected from international clinical trial registries for different regions
of the world for trials of tobacco use (A) and HIV infection (B).
doi:10.1371/journal.pone.0016878.g003
c. Data collection. One of us (NA) screened all records
obtained by this search to select the relevant RCTs (i.e., all RCTs
of interventions aimed at reducing or stopping tobacco
consumption or preventing or treating HIV infection).
For RCTs included in systematic reviews, we collected data on
the individual RCTs included in the review. To avoid collecting
data several times from an RCT included in several systematic
reviews, we recorded all RCTs in an EndNote data file (EndNote
for Windows Version X2, Bld (3210)). We systematically searched
this file for reports with the same authors and excluded duplicates.
If the country where the trial was performed was not mentioned
in the systematic review, we retrieved the published report for the
trial to obtain more information.
As a quality assurance exercise, another author (AD) collected
data on a random sample of 102 RCTs included in systematic
reviews.
For ongoing trials, we collected data from the trial records
available in the WHO-ICTRP and in the primary registry’s
records.
For each RCT included in a systematic review or registered, we
checked whether the trial was performed in a high-income or a
low- or middle-income country. We considered trials performed in
both high-income and low- and middle-income countries as trials
of low- and middle-income countries.
We also collected data on the intervention (pharmacological or
nonpharmacologic treatment) and the number of participating
countries.
III. Where research is needed
The following indicators were elaborated:
1. Prevalence of tobacco use and HIV infection. To
obtain the prevalence of tobacco use and HIV infection in the
world, in high-income countries and in different regions of low-
and middle-income countries, we used the World Bank reports for
number of smokers [12] and WHO reports for number of people
living with HIV infection [13,14].
2. Mortality and global burden of disease (GBD). The
global burden of disease (GBD) reflects the burden of disease by
disability-adjusted life years (DALYs). This time-based measure
combines years of life lost due to premature mortality and that lost
due to time lived in states of less than full health. The DALY
measure was developed in the original GBD 1990 study to assess
the burden of disease consistently across diseases, risk factors and
regions [15].
We used the data (WHO assessment of the GBD for 2000–2002
compiled and updated by the World Bank) related to the GBD
(DALYs) and mortality attributable to tobacco use and HIV
infection in the world, in high-income countries and in low- and
middle-income countries [16].
IV. Geographic representation
For an adequate representation of the contrast between the area
where trials are needed and the area where trials are conducted,
we constructed an area cartogram according to the method
developed by Gastner and colleagues [17]. Area cartograms are
maps in which the sizes of geographic regions such as countries are
not proportional to the area on the ground but, rather, appear in
proportion to their population or some other demographic feature
such as disease prevalence. We used such maps to compare the
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs
area for number of people smoking and with HIV infection to
where RCTs have been and are being conducted.
Statistical analysis
We used descriptive statistics; categorical variables are described
with frequencies and percentages. All data analyses involved use of
SAS for Windows, Release 9.1 (SAS Inst., Cary, NC).
Cartographic representation was developed with use of Migratio
8.0 (Stéphane Le Rouzic, Migratio.fr, Rennes, France) and
ScapeToad cartography software (Chôros Laboratory, EPFL-
ENAC-INTER, Lausanne, Switzerland).
Results
Where research is conducted
The flow of RCTs included in the selected systematic reviews
and ongoing RCTs through the study is in Figures 1 and 2,
respectively. The electronic search yielded 118 systematic reviews,
of which 57 (28 of tobacco use, 29 of HIV infection) were selected.
From these systematic reviews, data for 1 012 trials were retrieved;
143 duplicate reports and reports of 129 nonrandomised trials
were excluded. We included in the final analysis data for 740
RCTs (556 of tobacco use, 184 of HIV infection). Of the 940
ongoing RCTs (266 of tobacco use, 674 of HIV infection)
identified in the international clinical trial registries, we included in
the final analysis data for 346 (112 of tobacco use, 234 of HIV
infection).
The characteristics of the RCTs and ongoing RCTs are in
Table 1.
Geographic representation
Tobacco use trials. The geographic comparison of location
of RCTs of tobacco use and mortality and GBD (in DALYs)
attributable to tobacco use is in Table 2 and Figure 3A. In total,
96% of RCTs included in systematic reviews and 98% of ongoing
RCTs were performed in high-income countries. Both kinds of
trials were performed mainly in the USA (314 [57%] and 68
[61%], respectively) and the EU (144 [26%] and 27 [24%],
respectively). Only 24 (4%) RCTs included in systematic reviews
and 2 (2%) ongoing RCTs were carried out in low- or middle-
income countries, although these countries represented 74% of the
GBD (in DALYs) and 70% of the mortality attributable to tobacco
use and 82% of the smokers. Only 11 of 341 (3%) trials and 2 of 61
(3%) ongoing RCTs assessing nonpharmacologic treatment were
performed in low- and middle-income countries. The highest
prevalence of tobacco use concerns East Asia and Pacific countries
(38%). This area represents 22% of the mortality attributable to
tobacco use. In this area, only 9 RCTs were performed, and only 2
registered RCTs are ongoing. Similarly, no ongoing trial is
registered in Eastern Europe and Central Asia, South Asia, Latin
America and Caribbean, the Middle East and North Africa, and
Sub-Saharan Africa, which overall represent nearly half of the
mortality attributable to tobacco use. Figure 4 highlights the
differences in geographic representation between regions with a
high prevalence of smokers and where trials are conducted.
HIV infection trials. The geographical comparison of
location of RCTs and mortality and GBD (in DALYs)
attributable to HIV infection is in Table 3 and Figure 3B. In
total, 70% of RCTs included in systematic reviews and 67% of
107
6 February 2011 | Volume 6 | Issue 2 | e16878
World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs

108 iMedPub :: Thematic Collections :: Volume 2

PLoS ONE | www.plosone.org 7 February 2011 | Volume 6 | Issue 2 | e16878
 World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs

Figure 4. Area cartograms showing the sizes of countries in proportion to A) the number of smokers* and B) the number of RCTs
(RCTs included in Cochrane systematic reviews and ongoing RCTs) aimed at reducing or stopping tobacco use{. * Source: World
Bank’s 2005 World Development Indicators, 2005. Table 2.18 Health: risk factors and future challenges. The World Bank sourced this in turn from the
World Health Organization’s 2004 World Health Report, Tobacco Control Country Profiles 2003. { The number of RCTs (primary and ongoing) was
classified by main regions according to the 2008 World Bank classification: high-income countries, East Asia and Pacific, Eastern Europe and Central
Asia, Latin America and Caribbean, Middle East and North Africa, South Asia, Sub-Saharan Africa. For the construction of the map we divided the
number of trials performed in each region by the number of countries in each region.
doi:10.1371/journal.pone.0016878.g004

ongoing RCTs were performed in high-income countries. Only 54
(31%) RCTs and 77 (33%) ongoing RCTs were carried out in low-
or middle-income countries, although these countries represented
99% of the GBD, 99% of the mortality and 94% of the people
with HIV infection. About 16% (16/99) of trials and 26% (20/78)
of ongoing RCTs assessing nonpharmacologic treatments were
performed in low- and middle-income countries.
Only 37 (21%) RCTs and 40 (17%) ongoing RCTs were
performed in Sub-Saharan Africa, which represents 84% of the
GBD and 84% of the mortality attributable to HIV infection and
68% of the people with HIV infection. Similarly, only 2 ongoing
RCTs are being performed in South Asia, although this area
represents 9% of the GBD, 9% of the mortality and 12% of the
people with HIV infection. Figure 5 highlights the differences in
geographic representation between the regions with a high
prevalence of people with HIV infection and where trials are
conducted.
Discussion
This study evaluated the context of clinical research (i.e., where
research is performed) for 2 public health priorities: tobacco use
and HIV infection. We described the location of 740 RCTs
included in Cochrane systematic reviews and 346 ongoing RCTs
identified through international clinical trial registries. Our results
highlight a gap between where trials are conducted and where
research is needed in terms of GBD, mortality and prevalence of
tobacco use and HIV infection. Most ongoing and RCTs were
performed in high-income countries, even though most of the
mortality and GBD concerns low- and middle-income countries.
Only 4% and 32% of the total and ongoing RCTs performed in
the fields of tobacco use and HIV infection, respectively, were
conducted in low- and middle-income countries. However, low-
and middle-income countries represent 70% and 99% of the
mortality attributable to tobacco use and HIV infection,
respectively [13,18].
The number of trials of tobacco use performed in low- and
middle-income countries was particularly low (4%) as compared
with trials of HIV infection performed in these countries (about
one-third).
Previous studies have highlighted the underrepresentation of
research addressing priority issues for low- and middle-income
countries [19]. Some authors highlighted the poor association of
GBD and reports of RCTs published in high-impact-factor
journals [19,20]. Swingler et al. [21] analysed nearly 3 000
systematic reviews and assessed the correlation between the
number of systematic reviews undertaken and the burden of
disease: only a few systematic reviews focused on some diseases
affecting a large number of the world’s population. Sheriff showed
that only 3% of mental disease research is performed in low- and
middle-income countries [22], but the religious and cultural
individuality of these countries could greatly hamper implemen-
tation of any mental health interventions recommended by
research performed in high-income countries. These low research
percentages are probably related to the limited financial
investment in research for low- and middle-income countries

Table 3. HIV infection: AIDS prevalence, burden of disease (in disability-adjusted life-years [DALYs]), attributable mortality, RCTs
included in Cochrane systematic reviews and ongoing RCTs selected from International Clinical Trials Registries for high-income
and low- and middle-income countries.

Attributable Attributable Total PT NPT

DALYs mortality HIV infection Total NPT ongoing ongoing ongoing
Countries (thousands) (thousands) prevalence(thousands) RCTs PT RCTs RCTs RCTs RCTs RCTs

n = 67 568 (%) n = 2 440 (%) n = 33 200(%) n = 177(%) n = 78(%) n = 99(%) n = 234(%) n = 156(%) n = 78(%)

High-income countries 445 (0.6) 15 (0.6) 2 100 (6.3) 123 (69.5) 40 (51.3) 83 (83.8) 157 (67.1) 99 (63.5) 58 (74.3)
Low- or middle-income 67 141 (99.4) 2 425 (99.4) 31 100 (93.7) 54 (30.5) 38 (48.7) 16 (16.2) 77 (32.9) 57 (36.5) 20 (25.7)
countries
N Sub-Saharan Africa 56 795 (84.1) 2 057 (84.3) 22 500 (67.9) 37 (20.9) 29 (37.1) 10 (10.1) 40 (17.1) 28 (17.9) 12 (15.3)
N South Asia 5 846 (8.7) 214 (8.8) 4 000 (12.0) 1 (0.6) 0 0 2 (0.9) 0 2 (2.6)
N Latin America and 2 099 (3.1) 74 (3.1) 1 800 (5.4) 5 (2.8) 2 (2.6) 2 (2.0) 9 (3.8) 7 (4.5) 2 (2.6)
Caribbean
N East Asia and pacific 2 026 (2.9) 68 (2.8) 800 (2.4) 9 (5.1) 5 (6.4) 4 (4.1) 12 (5.1) 10 (6.4) 2 (2.6)
N Eastern Europe and 277 (0.4) 8 (0.3) 1 600 (4.8) 0 0 0 2 (0.9) 0 2 (2.6)
central Asia
N Middle east and North 99 (0.2) -- -- 0 0 0 0 0 0
Africa
N Multiple regions -- 2 (1.1) 2 (2.6) -- 12 (5.1) 12 (7.7) --

RCTs: randomized controlled trials. PT: pharmacologic intervention. NPT: nonpharmacologic intervention. GBD: global burden of disease.
doi:10.1371/journal.pone.0016878.t003

iMedPub :: Thematic collections :: Volume 2 109

PLoS ONE | www.plosone.org 8 February 2011 | Volume 6 | Issue 2 | e16878
World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs

110 iMedPub :: Thematic Collections :: Volume 2

PLoS ONE | www.plosone.org 9 February 2011 | Volume 6 | Issue 2 | e16878

Figure 5. Area cartograms showing the sizes of countries in proportion to A) the number of people with HIV infection* and B) the
number of RCTs (primary and ongoing RCTs) aimed at treating or preventing HIV infection{. *Source: report on the global AIDS
epidemic, UNAIDS/WHO, July 2008. {The number of RCTs (primary and ongoing) was classified by main regions according to the 2008 World Bank
classification: high-income countries, East Asia and Pacific, Eastern Europe and Central Asia, Latin America and Caribbean, Middle East and North
Africa, South Asia, Sub-Saharan Africa. For the construction of the map we divided the number of trials performed in each region by the number of
countries in each region.
doi:10.1371/journal.pone.0016878.g005
because of the phenomenon of the ‘‘10/90 gap’’. The Global
Forum has highlighted that of the US$73 billion invested annually
in global health research, less than 10% is spent on research into
the health problems that account for 90% of the GBD [4,23,24].
This gap is also reflected in the low proportion of publications
from research performed in low- and middle-income countries
[25].
Contrary to previous research, we focused on 2 public health
priorities (tobacco use, responsible for non-communicable chronic
diseases, and HIV infection, a communicable chronic disease)
relevant for high-, low- and middle-income countries. Our study is
the first to focus on existing evidence (trials included in systematic
reviews) and on awaiting evidence that will guide future evidence-
based practices.
With the evidence of the effectiveness of an intervention, some
might assume that the challenge is to make the intervention
available in low- and middle-income countries [26]. However, the
results of trials performed in high-income countries may not be
easily applied to low- and middle-income countries. Many
interventions shown to be efficacious in high-income countries are
not similarly effective when carried out in other contexts [26,27].
The populations can differ, with people consulting late, frequently
using self-medication, unable or unwilling to adhere to treatment,
and having several co-morbidities (malnourishment, anaemia,
malaria, etc.), in addition to the behavioural and cultural differences
[28,29]. Further, the contextual factors differ greatly, particularly
the social and cultural context, as well as the available facilities and
infrastructures. The lack of studies performed in a relevant location
is particularly problematic for nonpharmacologic interventions
targeting behavioural changes (e.g., education, counselling), because
the results of such trials could be strongly influenced by cultural
conditions [20,30]. The influence of the cultural context is
important, even between different high-income countries [31].
Recently, the Global Alliance for Chronic Disease (GACD) was
created to address this imbalance issue [32]. The GACD brings
together 6 major national health research councils representing
80% of all public research funding in the world to coordinate
research activities that address the prevention and treatment of
chronic disease on a global scale. One essential goal of the GACD
is to focus on chronic disease in low- and middle-income countries
and to coordinate research into low-cost interventions. This
initiative should be important in limiting the worldwide imbalance
in health research resources [7].
Similarly, the High Level Taskforce on Innovative International
Financing for Health Systems was created in September 2008 to
References
1. ONU (2005) Population Division: World population prospects, the 2004
revision. New York: United Nations, Department of Economic and Social
Affairs. 91 p.
2. McDevitt TM World population profile. Report WP/98. US Census Bureau website
(accessed 2009) http://www.census.gov/ipc/www/wp98.html.
3. Harris E, Tanner M (2000) Health technology transfer. Bmj 321: 817–820.
4. McMichael C, Waters E, Volmink J (2005) Evidence-based public health: what
does it offer developing countries? J Public Health (Oxf) 27: 215–221.
5. Oxman AD, Bjorndal A, Becerra-Posada F, Gibson M, Block MA, et al. (2010)
A framework for mandatory impact evaluation to ensure well informed public
policy decisions. Lancet 375: 427–431.
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
Geographical Representativeness of RCTs
help fill national financing gaps to reach the health Millennium
Development Goals in low-income countries. One key challenge is
to obtain evidence for what works according to different settings
[33].
Our study has some limitations. First, we focused on only 2
medical areas, and these results should be confirmed in other
medical areas. However, we chose tobacco use and HIV infection
because they are among the first 5 causes of mortality in the world
[9]. Second, we selected only RCTs included in Cochrane
systematic reviews because these systematic reviews are known
to be of high quality, and we did not consider non-Cochrane
systematic reviews [34,35,36,37,38]. As well, the Cochrane
collaboration seeks to actively encourage the participation of
reviewers from developing countries and to prioritize reviews
focusing on determinants of health that are particularly pertinent
to low- and middle-income countries [1,4]. Finally, our results may
underestimate the number of ongoing RCTs performed in low-
and middle-income countries because we focused on only available
data (i.e., trials that were registered in a trial registry available on
the WHO portal). We cannot exclude that some trials performed
in low-income countries were not registered.
In conclusion, clinical research into tobacco use and HIV
infection is not performed in the world locations most affected in
terms of GBD and mortality because we found important
underrepresentation of such research in low- and middle-income
countries. The GBD measured by mortality or DALYs could be a
useful aid in deciding where to conduct health research. The
measure should help in conducting RCTs in an appropriate
context to improve clinical research practice and decrease the
GBD [39].
Acknowledgments
We are grateful to Isabelle Pane for help in cartography representation and
to Laura Smales for copyediting.
Author Contributions
Conceived and designed the experiments: NA IB PR. Performed the
experiments: NA IB AD PD PR. Analyzed the data: NA. Contributed
reagents/materials/analysis tools: NA IB AD PR. Wrote the paper: NA IB
PR. Data collection: NA AD. Critical revision of manuscript for important
intellectual content: IB AD PD PR. Data interpretation: NA IB AD PD
PR.
6. Piot P, Bartos M, Larson H, Zewdie D, Mane P (2008) Coming to terms with
complexity: a call to action for HIV prevention. Lancet 372: 845–859.
7. Barreto ML (2009) Health research in developing countries. Bmj 339: b4846.
8. Say L, Raine R (2007) A systematic review of inequalities in the use of maternal
health care in developing countries: examining the scale of the problem and the
importance of context. Bull World Health Organ 85: 812–819.
9. Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ (2006) Global and
regional burden of disease and risk factors, 2001: systematic analysis of
population health data. Lancet 367: 1747–1757.
10. World Bank Country Classification. World Bank website (accessed 2009) http://
www.iqla.org/joining/World-Bank_Classification-List_2009.pdf.
111
10 February 2011 | Volume 6 | Issue 2 | e16878
World Health Report 2012: No Health Without Research
11. International Clinical Trials Registry Platform (ICTRP) World Health
Organization website (accessed 2009) http://www.who.int/ictrp/network/
primary/en/index.html.
12. Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, et al. (2006)
Disease Control Priorities in Developing Countries; Second, editor: A copublication
of The World Bank and Oxford University Press. 1. 400 p.
13. AIDS epidemic update: December 2007. United Nations Programme on HIV/
AIDS (UNAIDS) and World Health Organization. UNAIDS website (accessed
2009) http://data.unaids.org/pub/epislides/2007/2007_epiupdate_en.pdf.
14. Report on the Global HIV/AIDS Epidemic. UNAIDS website (accessed
2009) http://data.unaids.org/pub/GlobalReport/2008/JC1511_GR08_
ExecutiveSummary_en.pdf.
15. WHO (2008) The global burden of disease: 2004 update. Geneva. 146 p.
16. Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJL (2006) Global
Burden of Disease and Risk Factors: A copublication of Oxford University Press
and The World Bank.
17. Michael T Gastner, MEJN (2004) Diffusion-based method for producing
density-equalizing maps. Proc Nat Acad Sci USA. pp 7499–7504.
18. World Health Statistics Report. World Health Organization website (accessed
2009) http://www.who.int/whosis/whostat/EN_WHS08_Full.pdf.
19. Perel P, Miranda JJ, Ortiz Z, Casas JP (2008) Relation between the global
burden of disease and randomized clinical trials conducted in Latin America
published in the five leading medical journals. PLoS ONE 3: e1696.
20. Rochon PA, Mashari A, Cohen A, Misra A, Laxer D, et al. (2004) Relation
between randomized controlled trials published in leading general medical
journals and the global burden of disease. Cmaj 170: 1673–1677.
21. Swingler GH, Volmink J, Ioannidis JP (2003) Number of published systematic
reviews and global burden of disease: database analysis. Bmj 327: 1083–1084.
22. Sheriff RJ, Adams CE, Tharyan P, Jayaram M, Duley L (2008) Randomised
trials relevant to mental health conducted in low and middle-income countries: a
survey. BMC Psychiatry 8: 69.
23. Yusuf S (2002) Clinical research and trials in developing countries. Stat Med 21:
2859–2867.
24. Louis J. Currat, Andrés de Francisco, Nchinda T The 10/90 report on health
research 2001–2002: The Global Forum for Health Research website (accessed
2009) http://www.globalforumhealth.org/Media-Publications/Publications/10-
90-Report-on-Health-Research-2001-2002.
112
PLoS ONE | www.plosone.org
Geographical Representativeness of RCTs
25. Horton R (2003) Medical journals: evidence of bias against the diseases of
poverty. Lancet 361: 712–713.
26. Buekens P, Keusch G, Belizan J, Bhutta ZA (2004) Evidence-based global
health. JAMA 291: 2639–2641.
27. Volmink J, Siegfried NL, van der Merwe L, Brocklehurst P (2007) Antiretrovirals
for reducing the risk of mother-to-child transmission of HIV infection. Cochrane
Database Syst Rev 24: CD003510.
28. Siegfried N, Clarke M, Volmink J (2005) Randomised controlled trials in Africa
of HIV and AIDS: descriptive study and spatial distribution. Bmj 331: 742.
29. Chinnock P, Siegfried N, Clarke M (2005) Is evidence-based medicine relevant
to the developing world? PLoS Med 2: e107.
30. Caldwell JC (1993) Health transition: the cultural, social and behavioural
determinants of health in the Third World. Soc Sci Med 36: 125–135.
31. Eisinger F, Geller G, Burke W, Holtzman NA (1999) Cultural basis for
differences between US and French clinical recommendations for women at
increased risk of breast and ovarian cancer. Lancet 353: 919–920.
32. Daar AS, Nabel EG, Pramming SK, Anderson W, Beaudet A, et al. (2009) The
global alliance for chronic diseases. Science 324: 1642.
33. Fryatt R, Mills A, Nordstrom A (2010) Financing of health systems to achieve the
health Millennium Development Goals in low-income countries. Lancet 375:
419–426.
34. Moseley AM, Elkins MR, Herbert RD, Maher CG, Sherrington C (2009)
Cochrane reviews used more rigorous methods than non-Cochrane reviews:
survey of systematic reviews in physiotherapy. J Clin Epidemiol 62: 1021–1030.
35. Jorgensen AW, Hilden J, Gotzsche PC (2006) Cochrane reviews compared with
industry supported meta-analyses and other meta-analyses of the same drugs:
systematic review. Bmj 333: 782.
36. Olsen O, Middleton P, Ezzo J, Gotzsche PC, Hadhazy V, et al. (2001) Quality of
Cochrane reviews: assessment of sample from 1998. Bmj 323: 829–832.
37. Shea B, Moher D, Graham I, Pham B, Tugwell P (2002) A comparison of the
quality of Cochrane reviews and systematic reviews published in paper-based
journals. Eval Health Prof 25: 116–129.
38. Jadad AR, Moher M, Browman GP, Booker L, Sigouin C, et al. (2000)
Systematic reviews and meta-analyses on treatment of asthma: critical
evaluation. Bmj 320: 537–540.
39. Michaud CM, Murray CJ, Bloom BR (2001) Burden of disease—implications
for future research. JAMA 285: 535–539.
iMedPub :: Thematic Collections :: Volume 2
11 February 2011 | Volume 6 | Issue 2 | e16878
 World Health Report 2012: No Health Without Research

Strategies for Increasing Recruitment to Randomised

Controlled Trials: Systematic Review
Patrina H. Y. Caldwell1,2*, Sana Hamilton1, Alvin Tan3, Jonathan C. Craig1,2,4
1 Centre for Kidney Research, The Children’s Hospital at Westmead, New South Wales, Australia, 2 Discipline of Paediatrics and Child Health, University of Sydney, New
South Wales, Australia, 3 The Children’s Hospital at Westmead, New South Wales, Australia, 4 School of Public Health, University of Sydney, New South Wales, Australia

Abstract
Background: Recruitment of participants into randomised controlled trials (RCTs) is critical for successful trial conduct.
Although there have been two previous systematic reviews on related topics, the results (which identified specific
interventions) were inconclusive and not generalizable. The aim of our study was to evaluate the relative effectiveness of
recruitment strategies for participation in RCTs.

Methods and Findings: A systematic review, using the PRISMA guideline for reporting of systematic reviews, that compared
methods of recruiting individual study participants into an actual or mock RCT were included. We searched MEDLINE,
Embase, The Cochrane Library, and reference lists of relevant studies. From over 16,000 titles or abstracts reviewed, 396
papers were retrieved and 37 studies were included, in which 18,812 of at least 59,354 people approached agreed to
participate in a clinical RCT. Recruitment strategies were broadly divided into four groups: novel trial designs (eight studies),
recruiter differences (eight studies), incentives (two studies), and provision of trial information (19 studies). Strategies that
increased people’s awareness of the health problem being studied (e.g., an interactive computer program [relative risk (RR)
1.48, 95% confidence interval (CI) 1.00–2.18], attendance at an education session [RR 1.14, 95% CI 1.01–1.28], addition of a
health questionnaire [RR 1.37, 95% CI 1.14–1.66]), or a video about the health condition (RR 1.75, 95% CI 1.11–2.74), and also
monetary incentives (RR1.39, 95% CI 1.13–1.64 to RR 1.53, 95% CI 1.28–1.84) improved recruitment. Increasing patients’
understanding of the trial process, recruiter differences, and various methods of randomisation and consent design did not
show a difference in recruitment. Consent rates were also higher for nonblinded trial design, but differential loss to follow
up between groups may jeopardise the study findings. The study’s main limitation was the necessity of modifying the
search strategy with subsequent search updates because of changes in MEDLINE definitions. The abstracts of previous
versions of this systematic review were published in 2002 and 2007.

Conclusion: Recruitment strategies that focus on increasing potential participants’ awareness of the health problem being
studied, its potential impact on their health, and their engagement in the learning process appeared to increase recruitment
to clinical studies. Further trials of recruitment strategies that target engaging participants to increase their awareness of the
health problems being studied and the potential impact on their health may confirm this hypothesis.
Please see later in the article for the Editors’ Summary.

Citation: Caldwell PHY, Hamilton S, Tan A, Craig JC (2010) Strategies for Increasing Recruitment to Randomised Controlled Trials: Systematic Review. PLoS
Med 7(11): e1000368. doi:10.1371/journal.pmed.1000368
Academic Editor: Isabelle Boutron, University Paris Descartes, France
Received May 26, 2010; Accepted September 30, 2010; Published November 9, 2010
Copyright: � 2010 Caldwell et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: PHYC was funded by the University of Sydney Postgraduate Award Scholarship and The Centre for Kidney Research at the Children’s Hospital at
Westmead. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing Interests: The authors have declared that no competing interests exist.
Abbreviations: CI, confidence interval; RCT, randomised controlled trial; RR, relative risk.
* E-mail: Patrinac@chw.edu.au

iMedPub :: Thematic collections :: Volume 2 113

PLoS Medicine | www.plosmedicine.org 1 November 2010 | Volume 7 | Issue 11 | e1000368
World Health Report 2012: No Health Without Research
Introduction
The randomised controlled trial (RCT) provides the most
reliable evidence for evaluating the effects of health care
interventions [1,2], but the successful conduct of clinical RCTs
is often hindered by recruitment difficulties [3]. Inadequate
recruitment reduces the power of studies to detect significant
intervention effects [4], causes delays (which may affect the
generalizability of the study if standard care changes over time),
increases costs, and can lead to failure to complete trials [5,6].
With increasing reliance on clinical RCT findings for clinical and
regulatory decision making, the success of future RCTs depends
on employing effective and efficient methods for recruiting study
participants [7].
Historically recruitment of participants for RCTs has been by
‘‘trial and error’’ [8], by using a number of different strategies and
modifying strategies according to the observed effects on
recruitment. More recently, novel strategies have been developed
to facilitate adequate and timely recruitment [3,4]. Although there
have been two previous systematic reviews on strategies to
enhance recruitment to research [9,10], they identified specific
individual interventions. However, these interventions could not
be combined to offer useful general advice for recruitment for
clinical RCTs.
The aim of this study was to identify effective recruitment
strategies for clinical RCTs by systematically reviewing rando-
mised studies that compare consent rates, or other methods of
measuring consent for two or more recruitment methods used, to
approach potential RCT participants for trial participation (these
studies are termed recruitment trials).
Methods
A protocol for this systematic review had not been registered
before the review commenced, although the abstracts of previous
versions of this systematic review were published in 2002
(International Clinical Trials Symposium: improving health care
in the new millennium) [11] and 2007 (3rd International Clinical
Trials Symposium) [12] (Text S1).
Selection Criteria
All randomised and quasi-randomised studies that compared
two or more methods of recruiting study participants to a real
phase III RCT or mock RCT (where no actual trial occurred)
were included. Studies that assessed recruitment to observational
studies, questionnaires, health promotional activities, and other
health care interventions and nonrandomised studies of recruit-
ment strategies were excluded. Where more than one publication
of the same study existed, the publication with the most complete
data was included.
Literature Search
Studies were identified from MEDLINE (1950 to April, week 4,
2009), Embase (1980 to week 17, 2009), and The Cochrane Library
(Cochrane Library, issue 3, 2009) (Figure 1). The MEDLINE and
Embase databases were searched using text words and subject
headings (with unlimited truncations) for ‘‘recruitment,’’ ‘‘enrol-
ment,’’ and ‘‘accrual’’ combined with ‘‘random’’ and ‘‘trials’’ and
‘‘participate’’ or ‘‘consent’’ or ‘‘recruit’’ with unlimited truncations.
The Cochrane Library was searched using ‘‘recruitment’’ com-
bined with ‘‘random and trial,’’ and ‘‘consent or accrual.’’ The
search strategy changed slightly with time as a result of changes in
MEDLINE Mesh heading definitions. Reference lists of relevant
studies were also searched and non-English language papers were
114
PLoS Medicine | www.plosmedicine.org
Recruitment Strategies for Trials
translated. Two of three reviewers (PHYC, AT, or SH) indepen-
dently screened each study title and abstract for eligibility, retrieved
full text articles of all potentially relevant studies, and extracted data
from the retrieved papers using a form that was designed by the
authors. Disagreements were resolved by discussion with a third
reviewer (JCC).
Data Extraction
Data were extracted without blinding to authorship, on the
recruitment methods evaluated, the population setting, and the
trial design, as well as risk of bias items such as randomisation,
allocation concealment, blinding of outcome assessors, loss to
follow up, and intention-to-treat analysis. These elements were
each assessed separately using the method developed by the
Cochrane Collaboration [13].
Outcomes Assessed
The primary outcome of interest was consent rates for the
different recruitment strategies. Because studies differed in
definitions of consent rates, where possible we recalculated the
consent rate of each recruitment method by dividing the number
of participants exposed to the recruitment method who actually
consented for clinical study participation by the total number of
potential participants exposed to that method (see Figure 2). For
studies where information was insufficient to calculate consent
rates, other measures of consent success described in the study
were reported. For mock trials, willingness to consent to
participate (i.e., potential participants acknowledging that they
would be willing to participate in the trial or willingness to be
contacted for participation in future trials) was the outcome
measure. Consent rates and other outcome measures were
compared using intention-to-treat analysis.
Statistical Methods
Where possible we used relative risk (RR) and their 95%
confidence intervals (CIs) to describe the effects of different
strategies in individual recruitment trials. Where more than two
strategies were used in a single recruitment trial, the numerator
and denominator from the standard (control) recruitment strategy
was divided by the number of intervention strategies for each
comparison so that the control numbers would not be overrep-
resented [13].
Results
Literature Search
From 16,703 unique titles and abstracts, 396 articles were
retrieved and 37 eligible publications identified (Figure 1).
Collectively this total assessed recruitment outcomes in at least
59,354 people who were approached for clinical study participa-
tion, of whom 18,812 consented to participate (Table 1). (Not all
studies identified the number of potential participants who were
approached).
Quality of Included Studies
There were 23 parallel group RCTs, six quasi-RCTs (including
one using paired data), and eight cluster RCTs. Of the 37
included recruitment trials, only 12 studies (32%) had clear
allocation concealment, two (4%) specified blinding of outcome
assessors (no study had blinding of participants as this would have
been difficult to achieve), 15 (40%) recorded loss to follow-up
information, and 14 (38%) used intention-to-treat analysis (see
Table 2).
iMedPub :: Thematic Collections :: Volume 2
2 November 2010 | Volume 7 | Issue 11 | e1000368

Figure 1. Literature search.
doi:10.1371/journal.pmed.1000368.g001
Characteristics of Included Studies
Of the 37 included studies, 17 assessed treatment comparisons,
11 were prevention studies, and nine mock studies (where
participants declared their willingness to participate in a trial but
no actual trial occurred).
Figure 2. Consent rate for RCTs.
doi:10.1371/journal.pmed.1000368.g002
iMedPub :: Thematic collections :: Volume 2
PLoS Medicine | www.plosmedicine.org
World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials
There were 66 different types of recruitment strategies that were
broadly categorised into four groups: novel trial designs (nine
studies), recruiter differences (eight studies), incentives (two
studies), and provision of trial information (19 studies), with one
study looking at both novel trial design and incentives [14].
Standard recruitment is defined as when the investigator invites
the potential participant to enrol in the study and treatment
allocation is randomly assigned after consent has been given, with
routine treatment being provided where consent is not given.
Types of Recruitment Strategies Studied
Novel trial designs. Avenell and Hemminki [15,16]
compared a standard placebo-controlled design with a non-
blinded trial design (both for prevention studies) (see Figure 3 and
Table 3). In the nonblinded trial design arm, randomisation
occurred before participants were approached, and participants
were informed of the treatment they were randomised to receive
prior to giving consent. Consent rates were higher for the
nonblinded trial design compared with standard trial design
where randomisation occurred after consent for trial participation
(RR 1.14, 95% CI 1.02–1.28 and RR 1.28, 95% CI 1.19–1.37,
respectively) [15,16]. Welton [17] compared a noninferiority
clinical study (where both arms of the trial had an active
treatment) with a placebo-controlled study of hormone replace-
ment for postmenopausal women. Willingness to enrol in the
clinical study appeared to be higher for the noninferiority study
compared with the placebo-controlled study, although results were
only just statistically significant (39% versus 30%, RR 1.31, 95%
CI 1.01–1.70).
Gallo and Myles (both for mock studies) compared standard
randomisation (random assignment for all participants and
standard care for nonparticipants) with different types of
115
3 November 2010 | Volume 7 | Issue 11 | e1000368
World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Table 1. Included studies.

Health Recruitment n Invited to

Trial Year of Country Problem Intervention Strategy n Recruited Participate
Type Author Publication of Trial Studied Arms of RCT Studied for Trial in Trial

Treatment Du [39] 2008 USA Lung Mixed treatments Information 26 126

cancer (multiple trials) provision
Hutchison 2007 UK Multiple Mixed treatments Information 128 173
[38] cancers (multiple trials) provision
Monaghan 2006 Multinational BP control Antihypertensive Recruiter 7,847 167 sites
[51] in diabetics versus placebo differences
Litchfield 2005 UK Diabetes Two insulin Recruiter 73 80
[42]a delivery systems differences
Kimmick 2005 USA Multiple Mixed treatments Recruiter 1,097 unknown
[40] cancers (multiple trials) differences
Nystuen 2004 Norway Absentee Follow up versus Information 97 703
[31] employees standard care provision
Donovan 2003 UK Prostate Surgery versus Recruiter 103 150
[23] cancer radiotherapy differences
versus monitoring
Coyne 2003 USA Multiple Chemotherapy Information 147 226
[48] cancers (multiple trials) provision
Quinaux [41] 2003 France Breast cancer Chemotherapies Recruiter 362 unknown
differences
Tworoger 2002 USA Breast cancer Aerobic exercises Information 376 4,999
[37] versus stretching provision
Fleissig 2001 UK Multiple Mixed treatments Recruiter 205 265 (15
[49] cancers (multiple trials) differences recruiters)
Miller 1999 USA Depression Psychotherapy Recruiter 50 347
[43] versus antidepressants differences
versus both
Cooper 1997 UK Menorrhagia Medical management Trial design 187 273
[22] versus surgery
Berner 1997 USA Gynaecological Mixed treatments Information 9 120
[45] cancers (multiple trials) provision
Aaronson 1996 The Multiple Chemotherapy Information 146 346
[20] Netherlands cancers (multiple trials) provision
Wadland 1990 USA Smoking Nicotine gum versus Information 52 104
[35] standard care provision
Simes 1986 Australia Multiple Mixed treatments Information 50 57
[33] cancers (multiple trials) provision
Prevention Leira 2009 USA Aspiration Ranitidine versus Information 52 100
[29] pneumonia placebo provision
Mandelblatt 2005 USA Breast Tamoxifen versus Information 325 450
[3]a cancer Raloxifene provision
Avenell 2004 UK Fractures Vitamins versus Trial design 367 538
[21]a placebo/no treatment
Ford 2004 USA Multiple Screening tests versus Information 376 12,400
[25] cancers standard care provision
Hemminki 2004 Estonia Postmenopausal Hormone replacement Trial design 1,823 4,295
[27]a health risks versus placebo/ no
treatment
Larkey 2002 USA cardiovascular Hormone replacement Recruiter 13 34+
[50] disease, cancer therapy and dietary differences
and osteoporosis modification and
calcium and vitamin
D supplements
Kendrick 2001 UK Home safety Safety equipment Information 374 2,397
[4] versus usual care provision
Kiernan 2000 USA Healthy diet Additional goal Information 9 561
[28] setting techniques provision
versus standard care

116 iMedPub :: Thematic Collections :: Volume 2

PLoS Medicine | www.plosmedicine.org 4 November 2010 | Volume 7 | Issue 11 | e1000368
 World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Table 1. Cont.

Health Recruitment n Invited to

Trial Year of Country Problem Intervention Strategy n Recruited Participate
Type Author Publication of Trial Studied Arms of RCT Studied for Trial in Trial

Welton 1999 UK menopausal Hormone replacement Trial design 150 492 (438)
[46]a symptoms and therapies versus placebo
osteoporosis
Rogers 1998 USA Risk for life Follow up versus Trial design 44 57
[32] threatening illness standard care
Valanis 1998 USA Lung cancer Vitamins versus Information 451 22,546
[34] placebo provision
Mock trial Halpern 2004 USA Hypertension Different Incentives+trial 66–94 142
[47] hypertensives design
Ellis 2002 Australia Breast cancer Chemotherapy Information 26 180
[24] versus Tamoxifen provision
Martinson 2000 USA Smoking cessation Peer, mail, and phone Incentives 1,560 4,046
[6]a and prevention contacts versus
standard care
Wragg 2000 UK Postmenopausal Hormone replacement Information 22 50
[44] health risks versus placebo provision
Myles 1999 Australia Anaesthesia for Experimental drug Trial design 429 770
[30] surgery versus standard care
Weston 1997 Canada Premature labour Induced labour versus Information 43 90
[5]a expectant management provision
Gallo 1995 Italy Hypothetical Experimental drug Trial design 1,620 2,035
[26]a disease versus standard drug
Llewellyn- 1995 Canada Bowel cancer Chemotherapy Information 52 102
Thomas [2]a versus monitoring provision
Simel [36]a 1991 USA Variable presenting Standard versus Trial design 55 100
health problems new medication
Total 18,812 59,354+

a
Studies showed a statistically significant difference in consent rates between recruitment strategies.
BP, blood pressure.
doi:10.1371/journal.pmed.1000368.t001

randomisation designs [18,19]. Strategies included increasing or
decreasing the chance of receiving the experimental treatment;
experimental treatment for all participants and standard treatment
for nonparticipants (where potential participants are informed that
they have been randomised to receive the experimental treatment,
but if they do not consent, they would receive the standard
treatment); standard care for all participants and experimental
treatment for nonparticipants (where potential participants are
informed that they have been randomised to receive the standard
treatment, but if they do not consent, they would receive the
experimental treatment); and random assignment of treatment for
participants and choice of treatment for nonparticipants. The only
randomisation strategy that influenced consent was the ‘‘pre-
randomisation to standard drug’’ (standard care for all participants
and experimental treatment for nonparticipants) in Gallo’s study
[18], which significantly reduced the consent rate compared with
standard randomisation (RR 0.60, 95% CI 0.53–0.69) [18].
However, this was not demonstrated in Myles’ study [19].
Cooper compared standard consent with partially randomised
patient preference where patients could choose to be randomised
or choose their own (medical or surgical) treatment [20]. Patients
who chose their own treatment were excluded in our analysis, as
choice of treatment conflicts with the purposes of random
allocation of treatment, and only patients who chose to be
randomised were compared with those receiving standard RCT
consent (where they were offered the opportunity to participate in
iMedPub :: Thematic collections :: Volume 2
PLoS Medicine | www.plosmedicine.org
a clinical study where treatment was randomly allocated for
participants). This study tested whether allowing a patient choice
of treatments increased consent for choosing to have their
treatment randomised, compared with simply inviting them to
participate in a clinical RCT (without mentioning choice of
treatment). There was no difference in consent rates between the
standard consent and choosing to be randomised (RR 0.95, 95%
CI 0.81–1.11).
Rogers compared ‘‘opting in’’ with ‘‘opting out’’ [21] where
consent was sought for participation or for nonparticipation,
respectively. In the ‘‘opting out’’ arm, consent rate for clinical
study participation was calculated as the proportion who did not
sign the consent form (for refusing participation). There was no
difference in consent rates between the two groups (RR 1.07, 95%
CI 0.81–1.41).
Simel compared consenting to a clinical study assessing
standard medication versus a new medication that worked twice
as fast with a clinical study comparing standard medication with a
new medication that worked half as fast as the standard
medication [22]. Participants were not informed that this was a
mock trial. This study was designed to assess patients’ competence
and judgement regarding clinical study participation. Not
surprisingly, more patients consented to a clinical study comparing
the faster new medication than to a clinical study comparing a
slower new medication (67% versus 41%, RR 0.62, 95% CI 0.42–
0.91), with a more marked difference among those who voluntarily
117
5 November 2010 | Volume 7 | Issue 11 | e1000368
World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Table 2. Quality of included studies.

Blinding of
Type Allocation Outcome Loss to Follow Intention-to- Quality
Trial Type Author Of RCT Concealment Assessors Up Mentioned Treat Analysis Items

Prevention Avenell [21] Parallel Yes No Yes Yes 3

Prevention Rogers [32] Parallel Yes Yes No Yes 3
Treatment Monaghan [51] Cluster RCT Yes Unclear Unclear Yes 2
Treatment Hutchison [38] Parallel Yes Unclear Unclear Yes 2
Treatment Cooper [22] Parallel Yes No No Yes 2
Treatment Tworoger [37] Parallel Unclear Unclear Yes Yes 2
Treatment Coyne [48] Cluster RCT Unclear No Yes Yes 2
Treatment Du [39] Parallel Unclear Yes Yes No 2
Prevention Kendrick [4] Parallel Yes No Yes Unclear 2
Prevention Hemminki [27] Parallel Yes No Unclear Yes 2
Prevention Ford [25] Parallel Unclear No Yes Yes 2
Prevention Leira [29] Parallel No Unclear Yes Yes 2
Mock trial Weston [5] Parallel Yes No Yes Unclear 2
Mock trial Ellis [24] Parallel Yes No Yes Unclear 2
Mock trial Llewellyn-Thomas [2] Parallel Yes No Yes No 2
Mock trial Martinson [6] Cluster RCT Yes No Unclear Yes 2
Treatment Donovan [23] Parallel Yes No No No 1
Treatment Wadland [35] Parallel Unclear No Yes Unclear 1
Treatment Aaronson [20] Parallel Unclear No Yes Unclear 1
Treatment Berner [45] Quasi-RCT No Unclear Yes Unclear 1
Treatment Nystuen [31] Parallel No Unclear Unclear Yes 1
Prevention Larkey [50] Cluster RCT Unclear No Yes No 1
Prevention Valanis [34] Parallel Unclear No No Yes 1
Prevention Welton [46] Quasi-RCT No No Yes Unclear 1
Mock trial Simel [36] Parallel Unclear No No Yes 1
Treatment Quinaux [41] Cluster RCT Unclear Unclear Unclear Unclear 0
Treatment Kimmick [40] Cluster RCT Unclear Unclear Unclear Unclear 0
Treatment Litchfield [42] Cluster RCT Unclear Unclear Unclear Unclear 0
Treatment Fleissig [49] Cluster RCT Unclear No No Unclear 0
Treatment Simes [33] Parallel No No No Unclear 0
Treatment Miller [43] Quasi-RCT No No No Unclear 0
Prevention Kiernan [28] Parallel Unclear No No Unclear 0
Prevention Mandelblatt [3] Quasi-RCT No No Unclear Unclear 0
Mock trial Gallo [26] Parallel Unclear No No Unclear 0
Mock trial Myles [30] Parallel Unclear No No Unclear 0
Mock trial Wragg [44] Quasi-RCT Unclear No No Unclear 0
Mock trial Halpern [47] Paired data No No Unclear Unclear 0

doi:10.1371/journal.pmed.1000368.t002

mentioned the medication’s speed of action as a factor in their
decision regarding clinical study participation, which may reflect
better understanding of the trial information.
Halpern [14] used a factorial design to assess willingness to
participate in a number of mock trials using paired data from the
same individuals with variations in clinical study designs (as well as
variation in monetary incentives, which will be discussed later
under ‘‘incentives’’). There were no differences in consent rates
statistically.
Recruiter differences. Eight recruitment trials compared
recruiter differences (see Figure 4 and Table 4). Three cluster
118
PLoS Medicine | www.plosmedicine.org
RCTs compared different strategies for engaging recruiters (e.g.,
standard contact versus additional monitoring and contact with
recruiters [23–25]). Outcome measures were different for each of
the studies and therefore results could not be combined. In
Quinaux’s study, 186 patients from 34 control centres enrolled
compared with 176 total patients from 34 monitored centres [23].
In Kimmick’s study, 1,161 elderly patients (36% of total patients in
first year and 31% in second year) from the control centres
enrolled compared with 1,075 (32% in first year and 31% in
second year) from the centres who received additional training and
contact with investigators [24]. Monaghan’s study assessed median
iMedPub :: Thematic Collections :: Volume 2
6 November 2010 | Volume 7 | Issue 11 | e1000368
 World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Figure 3. Consent rates for novel trial designs. RR, intervention recruitment strategy/standard recruitment strategy. Used total number/number
of intervention strategies to calculate RR, so that the number of patients on standard strategies were not overrepresented; S, random assignment for
participants, standard care for nonparticipants; 2, patients are told physician believes the experimental drug may be superior. Increased chance of
receiving the experimental drug after consenting; 3, patients are told that they are allowed to increase or decrease their chance of receiving the new
experimental drug after consenting; 4, experimental drug for participants, standard care for nonparticipants; 5, standard drug for participants,
experimental drug for nonparticipants; 6, random assignment for participants, choice of either treatments for nonparticipants.
doi:10.1371/journal.pmed.1000368.g003

number of patients recruited per site with 37.0 patients from the 82
control sites compared with 37.5 patients from the 85 sites with
increased contacts with investigators [25]. In all three studies,
increased contact with investigators did not statistically increase
consent rates, and appeared to actually lower enrolment. One
recruitment trial that compared untrained recruiters with training of
recruiters [26] found statistically more patients enrolled when the
recruiter was trained (28 trained recruiters enrolled 13 patients versus
28 untrained recruiters who enrolled no patients). Fleissig compared
standard recruitment with providing recruiters with information
about patient preferences [27], with no differences in consent rates
between the two methods (RR 1.09, 95% CI 0.96–1.25).
Donovan and Miller compared recruiter roles (doctor versus
nurse RR 0.94, 95% CI 0.76–1.17 [28], and senior investigator
versus research assistant RR 0.69, 95% CI 0.41–1.15 [29]).
Although there was no difference in consent rates between the
recruiters, costs were higher for the more senior person (mean cost
of £43.29 versus £36.40 and US$78.48 versus US$50.28 per
patient randomised, respectively).
Litchfield compared internet-derived database handling with
paper-based database handling [30]. Although proportionately
more patients enrolled with the paper-based database, the internet
database was more efficient (with shorter time required for data
collection and more patients being exposed to the trial). 100% of
paper-based database versus 87% internet database groups
enrolled (RR 0.87, 95% CI 0.78–0.96), with the internet database
being preferable for recruiters.
Incentives. Martinson and Halpern assessed incentives for
increasing recruitment (see Figure 5 and Table 5) [14,31]. In the
Martinson study, compared to no incentives, any monetary
incentive increased survey response rates and willingness to be
contacted regarding a smoking cessation trial. The study did not
measure actual recruitment to the clinical study. Consent rate for
no incentives was 29% compared with 41% for prepaid US$2 cash
incentive (RR 1.43, 95% CI 1.19–1.72); 44% for US$15 cash
incentive contingent on completion of survey (RR 1.53, 95% CI
1.28–1.84); and 39% for US$200 prize draw (RR 1.36, 95% CI
1.13–1.64).
The Halpern study assessed the effect of variations in monetary
incentives on the willingness to participate in a number of mock
clinical studies (of varying trial designs that was mentioned earlier).
iMedPub :: Thematic collections :: Volume 2
PLoS Medicine | www.plosmedicine.org
Patients’ willingness to participate increased as the payment level
increased from US$100 to US$2,000 irrespective of the risk of
adverse effect and risk of being assigned to placebo, although the
difference was not statistically significant.
Methods of providing information. Nineteen recruitment
trials compared different methods of providing information to
participants, including how the information was presented and
what information was provided (see Figure 6 and Table 6).
There were six recruitment trials that related to mailing of
recruitment material for the clinical study. The methods used to
enhance recruitment were the addition of: a questionnaire that
focused on the health problem studied (Kendrick [32]); a personal
letter inviting participation (Kiernan and Tworoger [33,34]); use
of bulk mailing or first class stamps (Tworoger [34]); an advanced
postcard alerting recipients to look for the recruitment packet
(Valanis [35]); a reminder phone call for nonresponders of mailed
recruitment material (Nystuen [36]); and increasingly intensive
interventions (for African Americans), which included a follow-up
eligibility-screening phone call, an enhanced recruitment letter
featuring a prominent African American man, recruitment by an
African American member of the research team, and involvement
of church-based project sessions (Ford [37]). Kendrick’s addition
of the questionnaire that focused on the health problem studied
(RR 1.37, 95% CI 1.14–1.66) [32] was the only mailing strategy
that increased the consent rate compared with standard mailing of
recruitment material. The personal letter [33,34] using bulk mail
or first class mail [34], advanced postcard warning [35], and
reminder phone calls [36] did not significantly increase consent
rates (see Table 6).
Leira compared standard consent (being invited to participate in
the clinical study when the investigators met the patient during
helicopter retrievals) with advanced notification of the clinical
study with telephone and faxing of informed consent documents
prior to arrival of investigators in the helicopter [38]. The
intention-to-treat analysis showed no statistical difference between
the two recruitment strategies (RR 1.08, 95% CI 0.74–1.57),
although 42% of the intervention group did not actually receive
the intervention (fax and telephone call) because of technical and
logistic reasons. Coyne compared an easy-to-read consent
statement with standard consent [39] but showed no significant
difference in consent rates (RR 1.11, 95% CI 0.94–1.31).
119
7 November 2010 | Volume 7 | Issue 11 | e1000368
World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Table 3. Studies of novel trial designs.

Standard Recruitment Consent Rate Experimental Recruitment Consent Rate RR

Study Strategy n/N (95% CI) Strategies n/N (95% CI) (95% CI)

Myles [30] One-sided informed 84/151 56% (48–64) One-sided physician 91/150 61% (52–69) 1.10
consenta modifiedb (0.80–1.50)
One-sided patient 85/150 57% (48–65) 1.03
modifiedc (0.75–1.41)
Prerandomised to 90/169 53% (45–61) 0.96
experimental drugd (0.70–1.33)
Prerandomised to 79/149 53% (45–61) 0.96
standard druge (0.69–1.33)
Gallo [26] One-sided informed 521/622 84% (81–87) Prerandomised to 642/730 88% (86–90) 1.05
consenta experimental drugd (0.98–1.12)
Prerandomised to 156/307 51% (45–56) 0.60
standard druge (0.53–0.69)f
Two-sided informed 301/376 80% (76–84) 0.95
consentg (0.88–1.03)
Avenell [21] Standard placebo- 233/358 65% (60–70) Nonblinded trial 134/180 74% (67–81) 1.14
controlled design design (1.02–1.28)f
Hemminki [27] Standard placebo- 796/2,136 37% (35–39) Nonblinded trial 1,027/2159 48% (46–50) 1.28
controlled design design (1.19–1.37)f
Rogers [32] Opting-in consent for 24/32 75% (57–89) Opting-out consent 20/25 80% (59–93) 1.07
participation for nonparticipation (0.81–1.41)
Cooper [22] Standard informed 97/138 70% (62–78) Partially randomised 90/135 67% (58–75) 0.95
consent patient preferenceh (0.81–1.11)
Simel [36] Consent for trial of usual 35/52 67% (53–80) Consent for trial of usual 20/48 41% (28–57) 0.62
treatment versus new treatment versus new (0.42–0.91)f
treatment that may work treatment that may
twice as fast work half as fast
Halpern [47] A- 10% risk of adverse 26/64 41% (29–54) 20% risk of adverse 23/64 36% (24–49) 1.08
US$100 incentive effects effects (0.59–2.00)
10% risk of adverse 26/64 41% (29–54) 30% risk of adverse 18/64 28% (18–41) 1.44
effects effects (0.72–2.89)
20% risk of adverse 23/64 36% (24–49) 30% risk of adverse 18/64 28% (18–41) 1.33
effects effects (0.65–2.72)
Halpern [47] A- 10% risk of adverse 33/64 52% (39–64) 20% risk of adverse 26/64 41% (29–54) 1.31
US$1,000 incentive effects effects (0.77–2.22)
10% risk of adverse 33/64 52% (39–64) 30% risk of adverse 23/64 36% (24–49) 1.42
effects effects (0.81–2.46)
20% risk of adverse 26/64 41% (29–54) 30% risk of adverse 23/64 36% (24–49) 1.08
effects effects (0.59–2.00)
Halpern [47] A- 10% risk of adverse 35/64 55% (42–67) 20% risk of adverse 29/64 45% (33–58) 1.20
US$2,000 incentive effects effects (0.74–1.94)
10% risk of adverse 35/64 55% (42–67) 30% risk of adverse 25/64 39% (27–52) 1.38
effects effects (0.82–2.33)
20% risk of adverse 29/64 45% (33–58) 30% risk of adverse 25/64 39% (27–52) 1.15
effects effects (0.66–2.02)
Halpern [47] B- 10% assigned to 21/62 34% (22–47) 30% assigned to 20/62 32% (21–45) 1.10
US$100 incentive placebo placebo (0.55–2.21)
10% assigned to 21/62 34% (22–47) 50% assigned to 19/62 31% (20–44) 1.10
placebo placebo (0.55–2.21)
30% assigned to 20/62 32% (21–45) 50% assigned to 19/62 31% (20–44) 1.00
placebo placebo (0.49–2.06)
Halpern [47] B- 10% assigned to 27/62 44% (31–57) 30% assigned to 25/62 40% (28–54) 1.08
US$1,000 incentive placebo placebo (0.61–1.90)
10% assigned to 27/62 44% (31–57) 50% assigned to 23/62 37% (25–50) 1.17
placebo placebo (0.65–2.10)
30% assigned to 25/62 40% (28–54) 50% assigned to 23/62 37% (25–50) 1.08
placebo placebo (0.59–1.99)
Halpern [47] 10% assigned to 28/62 45% (33–58) 30% assigned to 26/62 42% (30–55) 1.08
B- US$2,000 placebo placebo (0.61–1.90)
incentive

120 iMedPub :: Thematic Collections :: Volume 2

PLoS Medicine | www.plosmedicine.org 8 November 2010 | Volume 7 | Issue 11 | e1000368
 World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Table 3. Cont.

Standard Recruitment Consent Rate Experimental Recruitment Consent Rate RR

Study Strategy n/N (95% CI) Strategies n/N (95% CI) (95% CI)

10% assigned to placebo 28/62 45% (33–58) 50% assigned to placebo 27/62 44% (31–57) 1.00
(0.58–1.73)
30% assigned to placebo 26/62 42% (30–55) 50% assigned to placebo 27/62 44% (31–57) 0.93
(0.53–1.64)
Welton [46] Standard placebo- 65/218 30% (24–36) Noninferiority trial design 85/218 39% (33–46) 1.31
controlled design (1.01–1.70)f

RR, experimental recruitment strategy/standard recruitment strategy. Used total number/number of experimental strategies to calculate RR, so that standard was not
overrepresented. Halpern’s study used each participant more than once.
a
Random assignment for participants, standard care for nonparticipants.
b
Patients told physician believes the experimental drug may be superior. Increased chance of receiving the experimental drug after consenting.
c
Patients are told that they are allowed to increase or decrease their chance of receiving the new experimental drug after consenting.
d
Experimental drug for participants, standard care for nonparticipants.
e
Standard drug for participants, experimental drug for nonparticipants.
f
Studies showed a statistically significant difference in consent rates between recruitment strategies.
g
Random assignment for participants, choice of either treatments for nonparticipants.
h
Patients could choose to be randomised or choose their own treatment, but only those who chose to be randomised were compared with standard treatment.
doi:10.1371/journal.pmed.1000368.t003

Three recruitment trials looked at increasing participants’
understanding of the clinical trial process, which did not appear
to affect recruitment [40–42]. Ellis compared standard informed
consent with the addition of an educational booklet on clinical
trials [40]. There was no difference in consent rates (unadjusted)
between the two groups (RR 0.88, 95% CI 0.46–1.66). However,
after adjusting for potential confounders (demographic variables,
disease variables, preference for involvement in clinical decision
making, anxiety, depression, and attitudes to clinical trials),
participants receiving the educational booklets were significantly
less likely to consent to clinical study participation (OR 0.22, 95%
CI 0.04–1.0). Du compared standard care with the addition of a
brief video about cancer clinical studies among patients with lung
cancer [41]. Consent rates were not statistically different between
the two groups. Hutchison compared standard care (where
patients discuss clinical care and clinical study participation with
the administration of a trial-specific information sheet and consent
form) with the addition of an audiovisual patient information tool
(with choice of video, CD-Rom, or DVD format), which addressed
clinical trial information [42], with no difference in consent rates
between the two groups (76% versus 72%, RR 0.95, 95% CI
0.80–1.13).
Three recruitment trials assessed strategies that aim to increase
participants’ understanding of their underlying condition. Llewel-
lyn-Thomas compared tape recorded reading of clinical study
information with an interactive computer program where
participants (who were oncology patients receiving radiation
therapy) were actively involved in the information search process
[43]. The consent rate was higher for participants in the
interactive group (RR 1.48, 95% CI 1.00–2.18). Weston compared
standard informed consent with the addition of a video explaining
trial information and the health problem studied [44]. The
consent rate was higher in the video group when initially assessed
(RR 1.75, 95% CI 1.11–2.74), but this did not reach statistical
significance at 2 wk follow-up (not shown on Table 6). Berner’s
recruitment trial compared standard care (verbal communication)
with the addition of patient information files containing clinical
information on cancer specific to the patient [45]. There was no
difference in the rate of recruitment to cancer trials in both groups
(7% versus 7%, RR 0.89, 95% CI 0.24–3.38), although not all
patients were eligible for clinical study enrolment.
Three recruitment trials compared standard consent with
additional personal contact with research staff (a study coordinator
reading and explaining the clinical study, Wadland [46];
additional phone-based contact with an oncology nurse, Aaronson
[47]; and an additional educational session about the disease and
risks and benefits of clinical study participation for an oncology
prevention study, Mandelblatt [48]). There was no difference in
consent rates between standard consent and the study coordinator
reading and explaining the clinical study (RR 1.12, 95% CI 0.76–
1.65) [46] or additional phone-based contact with the oncology
nurse (RR 0.87, 95% CI 0.76–1.01) [47]. However there was
higher consent for participants who attended the education session
(RR 1.14, 95% CI 1.01–1.28) [48].
There were two recruitment trials assessing framing of
recruitment information. In Simes’ 1986 trial of recruitment for
a cancer treatment study [49], total disclosure of information
about the clinical study was compared with an individual

Figure 4. Consent rates for recruiter differences. RR, intervention recruitment strategy/standard recruitment strategy.
doi:10.1371/journal.pmed.1000368.g004

iMedPub :: Thematic collections :: Volume 2 121

PLoS Medicine | www.plosmedicine.org 9 November 2010 | Volume 7 | Issue 11 | e1000368
World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Table 4. Studies of recruiter differences.

Standard Recruitment Consent Rate Experimental Consent Rate RR

Study Strategy n/N (95% CI) Recruitment Strategies n/N (95% CI) (95% CI)

Donovan [23] Recruitment by 53/75 71% (59–81) Recruitment by nurse 50/75 67% (55–77) 0.94
urologist (0.76–1.17)
Miller [43] Recruitment by senior 28/162 17% (12–24) Recruitment by research 22/185 12% (8–17) 0.69
investigator assistant (0.41–1.15)
Fleissig [49] Standard consent, 96/130 74% (65–81) Doctors shown patient’s 109/135 81% (73–87) 1.09
doctors not aware of responses to questionnaire (0.96–1.25)
patients’ personal regarding personal
preferences preferences and trial
participation before
recruiting patients for trial
Litchfield [42] Paper-based data 28/28 screened 100% (88–100) Internet data capture 45/52 screened 87% (74–94) 0.87
recording (0.78–0.96)a
Quinaux [41] Centres not monitored 186/34 centres Monitored centres 176/34 centres
Larkey [50] Recruiters not trained 0/28 recruiters Recruiters trained 13/28 recruiters
Kimmick [40] Standard recruitment, 777 (year 1)+384 Additional seminar, 691 year 1)+384
website access and (year 2) = 1,161 educational materials, (year 2) = 1,075
periodic notification list of available protocols,
email and mail reminders,
and case discussion
seminars for recruiters
Monaghan [51] Usual communication 37 (median) per Frequent email contact 37.5 (median) per
site at 82 sites and individual feedback site at 85 sites
about recruitment to
the recruiter

RR, experimental recruitment strategy/standard recruitment strategy.

a
Studies showed a statistically significant difference in consent rates between recruitment strategies.
doi:10.1371/journal.pmed.1000368.t004

approach where doctors informed patients about the clinical study
in a manner they thought best. This study assessed both
willingness to enrol in the clinical study and actual study
participation. There were no differences in actual consent rates
between the total disclosure and individual approach groups (RR
1.13, 95% CI 0.93–1.38). However, actual consent rates were
higher than the stated willingness to participate in the clinical
study (actual consent rates were 82% and 93% in the total
disclosure and individual approach groups, respectively, compared
with rates of 65% and 88%, respectively, for willingness to
participate in the clinical study). Wragg compared framing of
recruitment information explicitly (to provide the best current
estimates of effect for the experimental treatment) with framing
information ambiguously (to emphasise the uncertainty and
relative costs and benefits of the experimental treatment) [50].
There was no difference in consent rates between the ‘‘ambigu-
ously framed’’ group and the ‘‘explicitly framed’’ group (RR 1.90,
95% CI 0.97–3.70).
Discussion
Trials of recruitment strategies have evaluated all steps in the
recruitment process, including different methods of trial design,
randomisation, provision of information, and recruiter differences.
In this systematic review, we found that strategies that increased
potential participants’ awareness of the health problem being
studied by engaging them in the learning process significantly
increased consent rates (both for ‘‘real’’ and mock trials). These
strategies included the addition of a questionnaire that focused on
the health problem studied and additional educational sessions,
videos, and interactive programs about the diseases studied
[32,43,44,48]. Strategies that increased understanding of the
clinical trial process (e.g., provision of an educational booklet [40],
video [41], or audiovisual patient information tool [42] on clinical
trials or provision of an easy-to-read consent statement [39])
showed no evidence of improved recruitment. This finding
suggests that it is increased education about the health problem

Figure 5. Consent rates for incentives. RR, intervention recruitment strategy/standard recruitment strategy. Used total number/number of
intervention strategies to calculate RR, so that the number of patients on standard strategies were not overrepresented; S, random assignment for
participants, standard care for nonparticipants; 1, small incentives (US$2 prepaid cash incentive); 2, larger incentive (US$15) contingent on response;
3, US$200 prize draw.
doi:10.1371/journal.pmed.1000368.g005
122 iMedPub :: Thematic Collections :: Volume 2
PLoS Medicine | www.plosmedicine.org 10 November 2010 | Volume 7 | Issue 11 | e1000368
 World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Table 5. Studies of incentives.

Standard Consent Rate Experimental Consent Rate RR

Study Recruitment Strategy n/N (95% CI) Recruitment Strategies n/N (95% CI) (95% CI)

Martinson [6] No incentives 288/996 29% (26–32) US$2 small prepaid cash 423/1,021 41% (38–45) 1.43
(1.19–1.72)a
Large cash incentives 452/1,021 44% (41–47) 1.53
contingent on response (1.28–1.84)a
(US$15)
US$200 prize draw 397/1008 39% (36–42) 1.36
(1.13–1.64)a
Halpern [47] A-10% US$100 26/64 41% (29–54) US$1,000 33/64 52% (39–64) 0.76
risk of adverse effect (0.45–1.30)
US$100 26/64 41% (29–54) US$2,000 35/64 55% (42–67) 0.72
(0.43–1.21)
US$1,000 33/64 52% (39–64) US$2,000 35/64 55% (42–67) 0.94
(0.60–1.48)
Halpern [47] A-20% US$100 23/64 36% (24–49) US$1,000 26/64 41% (29–54) 0.92
risk of adverse effect (0.30–1.70)
US$100 23/64 36% (24–49) US$2,000 29/64 45% (33–58) 0.80
(0.45–1.43)
US$1,000 26/64 41% (29–54) US$2,000 29/64 45% (33–58) 0.87
(0.50–1.51)
Halpern [47] A-30% US$100 18/64 28% (18–41) US$1,000 23/64 36% (24–49) 0.75
risk of adverse effect (0.37–1.53)
US$100 18/64 28% (18–41) US$2,000 25/64 39% (27–52) 0.69
(0.35–1.39)
US$1,000 23/64 36% (24–49) US$2,000 25/64 39% (27–52) 0.92
(0.50–1.70)
Halpern [47] B- 10% US$100 21/62 34% (22–47) US$1,000 27/62 44% (31–57) 0.79
assigned to placebo (0.43–1.45)
US$100 21/62 34% (22–47) US$2,000 28/62 45% (33–58) 0.70
(0.43–1.45)
US$1,000 27/62 44% (31–57) US$2,000 28/62 45% (33–58) 1.00
(0.58–1.73)
Halpern [47] B- 30% US$100 20/62 32% (21–45) US$1,000 25/62 40% (28–54) 0.77
assigned to placebo (0.40–1.48)
US$100 20/62 32% (21–45) US$2,000 26/62 42% (30–55) 0.77
(0.40–1.48)
US$1,000 25/62 40% (28–54) US$2,000 26/62 42% (30–55) 1.00
(0.56–1.80)
Halpern [47] B- 50% US$100 19/62 31% (20–44) US$1,000 23/62 37% (25–50) 0.83
assigned to placebo (0.42–1.64)
US$100 19/62 31% (20–44) US$2,000 27/62 44% (31–57) 0.71
(0.38–1.36)
US$1,000 23/62 37% (25–50) US$2,000 27/62 44% (31–57) 0.86
(0.48–1.54)

RR, experimental recruitment strategy/standard recruitment strategy. Used total number/number of experimental strategies to calculate RR, so that standard was not
overrepresented. Halpern’s study used each participant more than once.
a
Studies showed a statistically significant difference in consent rates between recruitment strategies.
doi:10.1371/journal.pmed.1000368.t005

being studied rather than education about the clinical trial process
that increased trial participation. There were insufficient data to
evaluate whether the effects of the different recruitment strategies
were constant across all health conditions, but no there was no
clear trend for these strategies to be context specific (see Table 1).
The recruitment trials on how recruitment information was
provided (the technique of information presentation, how
information was framed, who presented the information, and
when the information was presented) did not show a difference
between strategies, demonstrating that how or when the
information was presented or who presented the information did
iMedPub :: Thematic collections :: Volume 2
PLoS Medicine | www.plosmedicine.org
not influence recruitment, but rather the information provided. A
recent study (which was published after completion of our last
search update) also showed that publicity about the trial did not
increase recruitment [51].
Although a previous observational study showed that framing of
recruitment information to emphasise uncertainty enhanced
recruitment [52], when this was tested by the rigor of RCT
methodology [49,50], we found that framing did not appear to
influence recruitment. Unexpectedly we found that the role of the
recruiter also did not show evidence of influencing recruitment
(although costs were higher for senior recruiters [28,29]).
123
11 November 2010 | Volume 7 | Issue 11 | e1000368
World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Figure 6. Consent rates for methods of providing information. RR, intervention recruitment strategy/standard recruitment strategy. Used
total number/number of intervention strategies to calculate RR, so that the number of patients on standard strategies were not overrepresented; S,
standard informed consent; B, bulk mailing; 1, enhanced recruitment letter and screening by African American interviewer; 2, enhanced recruitment
letter, screening by African American interviewer and baseline information collected via telephone interview; 3, enhanced recruitment letter,
screening by African American interviewer and church-based project sessions; 4, bulk mailing with letter; 5, first-class mailing; 6, first-class mailing
with letter.
doi:10.1371/journal.pmed.1000368.g006

In our review, one recruitment trial identified that a
noninferiority clinical study (with active treatment arms) had
higher consent rates compared with a placebo-controlled clinical
study. This finding is consistent with previous findings that patients
preferred ‘‘trials with all active arms to placebo-controlled trials’’
[53]. Also, recruitment trials that compared standard placebo-
controlled design with a nonblinded trial design demonstrated that
patients were more willing to participate in a clinical study if they
knew which treatment they were receiving when consenting, even
if the treatment was randomly predetermined. These studies
illustrate people’s anxieties regarding the unknowns of clinical trial
participation. Despite the higher consent rates for the nonblinded
trial design, the differential loss to follow up in the two treatments
arms of the nonblinded trial is likely to jeopardise validity of the
results, as comparison of outcomes between the two treatment
groups would be subject to selection bias. For example, patients
may be more likely to drop out if they were unhappy with the
treatment they were assigned. In the two included studies of
nonblinded trial designs, there were higher drop outs in the active
treatment arms compared with the placebo arms.
The inclusion of recruitment trials of recruitment to mock
clinical studies enabled assessment of recruitment strategies, which
for equity reasons would be difficult to compare (such as different
randomisation designs, different monetary incentives). Some
strategies may be acceptable when used in isolation, but
inappropriate when more than one are used within the same
clinical study: for example mock trials that tested the hypothesis
that potential participants are more willing to participate in a study
if they had an increased chance of receiving the experimental
treatment is a strategy that has been adopted by many vaccine and
other clinical studies in the belief that potential participants are
more likely to participate if they believed they had a higher chance
of receiving the (desirable) experimental treatment. However, we
124
PLoS Medicine | www.plosmedicine.org
found that increasing the likelihood of receiving the experimental
treatment [19] (or reducing the risk of receiving placebo) [14] did
not appear to affect the consent rate, demonstrating that people’s
decisions for clinical study participation are not influenced by
whether they are more or less likely to receive a particular
treatment. Other strategies are more controversial: for example,
the only consent strategy that appeared to affect the consent rate
for a mock trial was ‘‘prerandomisation to standard drug’’ [18],
where participants were given the standard drug and nonpartic-
ipants were given the experimental drug. Fewer people were
willing to consent to this type of clinical study than to a clinical
study of standard randomisation for all participants. It is unlikely
that such a method could ethically be employed in a real situation.
Monetary incentives appeared to increase consent compared to no
monetary incentives [31], but the amount of money appeared to
be less important [14].
As results of mock clinical studies are based on whether
participants are willing to enrol in a clinical study (rather than
whether they actually consented), extrapolation to real clinical
studies may not be realistic. Stated ‘‘willingness to participate’’
and actual participation may also differ. In the recruitment trial
comparing standard consent to the addition of a video
explaining clinical trial information and the health problem
studied for a mock clinical study, although statistically more
participants from the video group were willing to enrol in the
clinical study, this number became not statistically significant
2 wk later [44]. Conversely, in Sime’s 1986 study [49], more
participants actually consented to clinical study participation
than had indicated willingness to participate, perhaps reflecting
patients’ deference to doctors’ advice in the 1980s (when there
was less emphasis on patient autonomy compared with today). It
also showed the influence of the doctor on patient behaviour
[53].
iMedPub :: Thematic Collections :: Volume 2
12 November 2010 | Volume 7 | Issue 11 | e1000368
 World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials

Table 6. Studies of methods of providing information.

Standard Consent Experimental Consent

Recruitment Rate Recruitment Rate
Study Strategy n/N (95% CI) Strategies n/N (95% CI) RR (95% CI)

Kendrick [4] Standard informed 157/1,194 13% (11–15) Additional home safety 217/1,203 18% (16–20) 1.37 (1.14–1.66)a
consent (mailing) questionnaire
Kiernan [28] Standard informed 0/191 0% (0–2) Additional personal letter 9/370 2% (1–5) 9.83 (0.58–168.04)
consent (combination of general
(mailing of flyer) letter+Hispanic specific letter)
Valanis [34] Standard informed 225/11,273 2% (2–6) Advanced postcard 1 wk prior 226/11,273 2% (2–2) 1.0 (0.84–1.21)
consent (mailing) to mailing of recruitment packet
Nystuen [31] Standard informed 42/347 12% (9–16) Additional reminder phone call 55/356 15% (12–19) 1.28 (0.88–1.85)
consent (mailing) for nonresponders
Ford [25] Standard informed 95/3,297 3% (2–4) Enhanced recruitment letter+ 78/3,079 3% (2–3) 0.87 (0.58–1.31)
consent (mailing)+ screening by African American
screeningb interviewer
Enhanced recruitment letter+ 87/3,075 3% (2–3) 0.97 (0.65–1.45)
screening by African American
interviewer+baseline information
collected via telephone interview
Enhanced recruitment letter+ 116/2,949 4% (3–5) 1.35 (0.92–1.99)
screening by African American
interviewer+church-based project
sessions
Tworoger Bulk mailing no 86/1,250 7% (6–8) Bulk mailing with letter 87/1,251 7% (6–9) 1.00 (0.67–1.50)
[37] letters
First class mailing no letters 102/1,249 8% (7–10) 1.17 (0.79–1.75)
First class mailing with letters 101/1,249 8% (7–10) 1.16 (0.78–1.73)
Leira [29] Standard informed 25/50 50% (36–65) Advanced notification with 27/50 54% (39–68) 1.08 (0.74–1.57)
consent phone and fax
Llewellyn- Tape recording of 21/50 42% (28–57) Interactive computer program 31/50 62% (47–75) 1.48 (1.00–2.18)a
Thomas [2] trial information for participants
Weston [5] Standard informed 17/48 35% (22–51) Additional video about the 26/42 62% (46–76) 1.75 (1.11–2.74)a
consent health condition
Berner [45] Standard informed 4/50 7% (2–19) Additional written 4/56 7% (2–17) 0.89 (0.24–3.38)
consent (verbal) cancer-specific information
Ellis [24] Standard informed 14/42 33% (20–50) Additional education booklet 12/41 29% (16–46) 0.88 (0.46–1.66)
consent on trials
Du [39] Standard informed 10/63 16% (8–27) Additional video about 16/63 25% (15–38) 1.60 (0.79–3.25)
consent clinical trials
Hutchison [38] Standard informed 66/87 76% (66–84) AVPI tool to explain about 62/86 72% (61–81) 0.95 (0.80–1.13)
consent trials, video+DVD/CD
Coyne [48] Standard informed 93/137 68% (59–76) Easy-to-read consent 67/89 75% (65–84) 1.11(0.94–1.31)
consent statement
Wadland [35] Patients reading 25/53 47% (33–61) Study coordinator reading and 27/51 53% (39–67) 1.12 (0.76–1.65)
trial information explaining the study to patients
Aaronson [20] Standard informed 78/90 87% (78–93) Additional phone-based contact 68/90 76% (65–84) 0.87 (0.76–1.01)
consent with oncology nurse
Mandelblatt [3] Standard informed 147/218 67% (61–74) Additional brief educational session 178/232 77% (71–82) 1.14 (1.01–1.28)a
consent (brochure) and discussion about the trial
Simes [33] Total disclosure 23/28 82% (63–94) Individual approach 27/29 93% (77–99) 1.13 (0.93–1.38)
Wragg [44] Explicit informationc 8/26 31% (14–52) Ambiguous informationd 14/24 58% (37–78) 1.90 (0.97–3.70)

RR, experimental recruitment strategy/standard recruitment strategy. Used total number/number of experimental strategies to calculate RR, so that standard was not
overrepresented.
a
Studies showed a statistically significant difference in consent rates between recruitment strategies.
b
Standard informed consent and screening (used total number/number of experimental strategies to calculate RR, so that standard was not overrepresented).
c
Provides the current best estimates of effect of the experimental treatment.
d
Emphasises the current state of uncertainty.
doi:10.1371/journal.pmed.1000368.t006

Although there have been two previous systematic reviews on
strategies to enhance recruitment to research [9,10], our study is
the latest and has a more targeted and rigorous search method.
iMedPub :: Thematic collections :: Volume 2
PLoS Medicine | www.plosmedicine.org
We conducted a more comprehensive search (with inclusion of
more databases than Watson’s study [10]) and included earlier as
well as later studies, and also studies of recruitment for mock trials
125
13 November 2010 | Volume 7 | Issue 11 | e1000368
World Health Report 2012: No Health Without Research
to test recruitment strategies that would otherwise be difficult to
compare for equity reasons. Our methods were also more rigorous
(with two reviewers examining all titles, abstracts, and relevant
papers) with an inclusion criteria targeting recruitment of
participants for RCTs only (excluding studies about recruitment
to observational studies, questionnaires, health promotional
activities. and other health care interventions). We targeted
recruitment to RCTs in which recruitment is more difficult
because potential participants must consent to participation in
research in which their treatment is unknown. The Mapstone
study conducted in 2002 and published in 2007 [9] included
recruitment for any type of research studies, and the Watson study
[10], although targeting recruitment strategies used for RCTs,
searched only from 1996 to 2004 with a limited number of
electronic databases (without hand searching), using only the
keywords ‘‘recruitment strategy’’ or ‘‘recruitment strategies.’’ Our
study has identified more studies than the previous reviews (37
compared with 14 and 15 studies), and provides a better
understanding of the factors that influence clinical RCT
participation for potential participants. Although both previous
studies highlighted effective and ineffective strategies, there was no
attempt to examine the differences between successful and
unsuccessful recruitment strategies.
Our findings are consistent with the health belief model that
people are more likely to adopt a health behaviour (such as
participation in a clinical study) if they perceive they are at risk of a
significant health problem [54]. The importance of informing
potential participants about the health problem being studied and
engaging them in the learning process is not only educational and
constructive, but is also likely to enhance clinical trial participa-
tion.
Limitations
Because of major differences in recruitment methods, popula-
tions, and types of clinical studies that were recruiting as well as
outcomes measured, we did not combine the results statistically in
a meta-analysis. In many of the smaller recruitment trials, the
failure to find a significant difference in consent rates could be
related to the sample size (type II error). There may also be
publication bias. However, as more than 70% (27/37) of the
included studies had a nonsignificant result, we are hopeful that
publication bias may be minimal. Given that the interventions we
are considering are of noncommercial value we would suggest that
publication bias may be less likely than for other interventions.
The majority of the included trials were conducted in developed
countries, with a substantial proportion in the US. We
acknowledge that developed countries’ health systems may be
very different from those of less-developed countries and hence the
results of this systematic review may not be generalizable to other
countries.
The main limitation of the study, due to the prolonged conduct
of the study (from 2000 to 2009), was that the search strategy had
References
1. Barton S (2000) Which clinical studies provide the best evidence? The best RCT
still trumps the best observational study. BMJ 321: 255–256.
2. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS (1996)
Evidence based medicine: what it is and what it isn’t. BMJ 312: 71–72.
3. Lovato LC, Hill K, Hertert S, Hunninghake DB, Probstfield JL (1997)
Recruitment for controlled clinical trials: literature summary and annotated
bibliography. Control Clin Trials 18: 328–352.
4. Swanson GM, Ward AJ (1995) Recruiting minorities into clinical trials: toward a
participant-friendly system. J Natl Cancer Inst 87: 1747–1759.
5. Walson PD (1999) Patient recruitment: US perspective. Pediatrics 104: 619–622.
6. Easterbrook PJ, Matthews DR (1992) Fate of research studies. J R Soc Med 85:
71–76.
126
PLoS Medicine | www.plosmedicine.org
Recruitment Strategies for Trials
to be modified with subsequent search updates owing to changes
in MEDLINE Mesh heading definitions. Because of these changes
(and the large number of titles and abstracts searched), the reason
for exclusion of each study cannot be provided. The abstract of the
first version of this systematic review (which included nonrando-
mised studies owing to the lack of randomised recruitment trials on
the subject at the time) was published in conference proceedings in
2002 [11], and a later version that was limited to randomised
studies was published in conference proceedings in 2007 [12].
Conclusion
Our systematic review of recruitment strategies for enhancing
participation in clinical RCTs has identified a number of effective
and ineffective recruitment strategies. Grouped together, the
statistically significant strategies either engaged participants in
learning about the health problem being studied and its impact on
their health or else informed participants of the treatment they
have been randomised to receive (nonblinded trial design).
However, as there was differential loss to follow up in the different
treatment arms with nonblinded trial design, this trial design is
likely to jeopardise the validity of the results. The use of monetary
incentives may also increase recruitment, but as this was tested in a
mock trial, and as another mock trial did not show any difference
in consent rates between different amounts of monetary incentives,
this finding needs to be interpreted with caution.
Future RCTs of recruitment strategies that engaged participants
in the learning process using various methods of delivering the
recruitment material compared with standard recruitment may
confirm the effectiveness of this concept. This research may be
particularly useful for testing strategies that expose large number
of potential participants to recruitment information such as
interactive internet strategies.
Supporting Information
Text S1 PRISMA checklist.
Found at: doi:10.1371/journal.pmed.1000368.s001 (0.07 MB
DOC)
Acknowledgments
Thanks to Ravinder Summan, Rebecca George, Phyllis Butow, Mike
Jones, Gabrielle Williams, Premala Sureshkumar, Tamara Borysko, and
others at Centre for Kidney Research for their help in preparation of this
manuscript.
Author Contributions
ICMJE criteria for authorship read and met: PHYC SH AT JC. Agree
with the manuscript’s results and conclusions: PHYC SH AT JC. Designed
the experiments/the study: PHYC JC. Analyzed the data: PHYC SH JC.
Collected data/did experiments for the study: PHYC SH AT. Wrote the
first draft of the paper: PHYC SH. Contributed to the writing of the paper:
PHYC SH AT JC.
7. Baines CJ (1984) Impediments to recruitment in the Canadian National Breast
Screening Study: response and resolution. Control Clin Trials 5: 129–140.
8. Zifferblatt SM (1975) Recruitment in large-scale clinical trials. Weiss SM, ed.
Proceedings of the National Heart and Lung Institute Working Conference on
health behavior. Bethesda: NIH. 187–195.
9. Mapstone J, Elbourne D, Roberts I (2007) Strategies to improve recruitment to
research studies. Cochrane Database Syst Rev: MR000013.
10. Watson JM, Torgerson DJ (2006) Increasing recruitment to randomised trials: a
review of randomised controlled trials. BMC Med Res Methodol 6: 34.
11. Caldwell P, Craig J, Hamilton S, Butow PN (2002) Strategies for recruitment to
RCTs: a systematic review of controlled trials and observational studies. 2002
Oct 21; Sydney, Australia; International Clinical Trials Symposium: improving
iMedPub :: Thematic Collections :: Volume 2
14 November 2010 | Volume 7 | Issue 11 | e1000368

health care in the new millennium. http://www.ctc.usyd.edu.au/education/
4news/Symposium2002_report/caldwell.htm.
12. Caldwell P, Craig J, Hamilton S, Butow PN (2007) Systematic review of
strategies for enhancing recruitment of participants for randomised controlled
trials. 2007 Sep 24; Sydney, Australia; International Clinical Trials Symposium.
13. Higgins J, Green S, Cochrane C (2008) Cochrane handbook for systematic
reviews of interventions. Chichester, England: Wiley-Blackwell.
14. Halpern SD, Karlawish JH, Casarett D, Berlin JA, Asch DA (2004) Empirical
assessment of whether moderate payments are undue or unjust inducements for
participation in clinical trials. Arch Intern Med 164: 801–803.
15. Avenell A, Grant AM, McGee M, McPherson G, Campbell MK, et al. (2004)
The effects of an open design on trial participant recruitment, compliance and
retention–a randomized controlled trial comparison with a blinded, placebo-
controlled design. Clin Trials 1: 490–498.
16. Hemminki E, Hovi S-L, Veerus P, Sevon T, Tuimala R, et al. (2004) Blinding
decreased recruitment in a prevention trial of postmenopausal hormone therapy.
J Clin Epidemiol 57: 1237–1243.
17. Welton AJ, Vickers MR, Cooper JA, Meade TW, Marteau TM (1999) Is
recruitment more difficult with a placebo arm in randomised controlled trials? A
quasirandomised, interview based study. BMJ 318: 1114–1117.
18. Gallo C, Perrone F, De Placido S, Giusti C (1995) Informed versus randomised
consent to clinical trials. Lancet 346: 1060–1064.
19. Myles PS, Fletcher HE, Cairo S, Madder H, McRae R, et al. (1999)
Randomized trial of informed consent and recruitment for clinical trials in the
immediate preoperative period. Anesthesiology 91: 969–978.
20. Cooper KG, Grant AM, Garratt AM (1997) The impact of using a partially
randomised patient preference design when evaluating alternative managements
for heavy menstrual bleeding. BJOG 104: 1367–1373.
21. Rogers CG, Tyson JE, Kennedy KA, Broyles RS, Hickman JF (1998)
Conventional consent with opting in versus simplified consent with opting out:
an exploratory trial for studies that do not increase patient risk. J Pediatr 132:
606–611.
22. Simel DL, Feussner JR (1991) A randomized controlled trial comparing
quantitative informed consent formats. J Clin Epidemiol 44: 771–777.
23. Quinaux E, Lienard JL, Slimani Z, Jouhard A, Piedbois P, Buyse M (2003)
Impact of monitoring visits on patient recruitment and data quality: case study of
a phase IV trial in oncology. Control Clin Trials 24: 99S. [conference abstract].
24. Kimmick GG, Peterson BL, Kornblith AB, Mandelblatt J, Johnson JL, et al.
(2005) Improving accrual of older persons to cancer treatment trials: a
randomized trial comparing an educational intervention with standard
information: CALGB 360001. J Clin Oncol 23: 2201–2207.
25. Monaghan H, Richens A, Colman S, Currie R, Girgis S, et al. (2007) A
randomised trial of the effects of an additional communication strategy on
recruitment into a large-scale, multi-centre trial. Contemp Clin Trials 28: 1–5.
26. Larkey LK, Staten LK, Ritenbaugh C, Hall RA, Buller DB, et al. (2002)
Recruitment of Hispanic women to the Women’s Health Initiative: the case of
Embajadoras in Arizona. Control Clin Trials 23: 289–298.
27. Fleissig A, Jenkins V, Fallowfield L (2001) Results of an intervention study to
improve communication about randomised clinical trials of cancer therapy.
Eur J Cancer 37: 322–331.
28. Donovan JL, Peters TJ, Noble S, Powell P, Gillatt D, et al. (2003) Who can best
recruit to randomized trials? Randomized trial comparing surgeons and nurses
recruiting patients to a trial of treatments for localized prostate cancer (the
ProtecT study). J Clin Epidemiol 56: 605–609.
29. Miller NL, Markowitz JC, Kocsis JH, Leon AC, Brisco ST, et al. (1999) Cost
effectiveness of screening for clinical trials by research assistants versus senior
investigators. J Psychiatr Res 33: 81–85.
30. Litchfield J, Freeman J, Schou H, Elsley M, Fuller R, et al. (2005) Is the future
for clinical trials internet-based? A cluster randomized clinical trial. Clin Trials 2:
72–79.
31. Martinson BC, Lazovich D, Lando HA, Perry CL, McGovern PG, et al. (2000)
Effectiveness of monetary incentives for recruiting adolescents to an intervention
trial to reduce smoking. Prev Med 31: 706–713.
32. Kendrick D, Watson M, Dewey M, Woods AJ (2001) Does sending a home
safety questionnaire increase recruitment to an injury prevention trial? A
randomised controlled trial. J Epidemiol Community Health 55: 845–846.
iMedPub :: Thematic collections :: Volume 2
PLoS Medicine | www.plosmedicine.org
World Health Report 2012: No Health Without Research
Recruitment Strategies for Trials
33. Kiernan M, Phillips K, Fair JM, King AC (2000) Using direct mail to recruit
Hispanic adults into a dietary intervention: an experimental study. Ann Behav
Med 22: 89–93.
34. Tworoger SS, Yasui Y, Ulrich CM, Nakamura H, LaCroix K, et al. (2002)
Mailing strategies and recruitment into an intervention trial of the exercise effect
on breast cancer biomarkers. Cancer Epidemiol Biomarkers Prev 11: 73–77.
35. Valanis B, Blank J, Glass A (1998) Mailing strategies and costs of recruiting
heavy smokers in CARET, a large chemoprevention trial. Control Clin Trials
19: 25–38.
36. Nystuen P, Hagen KB (2004) Telephone reminders are effective in recruiting
nonresponding patients to randomized controlled trials. J Clin Epidemiol 57:
773–776.
37. Ford ME, Havstad SL, Davis SD (2004) A randomized trial of recruitment
methods for older African American men in the Prostate, Lung, Colorectal and
Ovarian (PLCO) Cancer Screening Trial. Clin Trials 1: 343–351.
38. Leira EC, Ahmed A, Lamb DL, Olalde HM, Callison RC, et al. (2009)
Extending acute trials to remote populations a pilot study during interhospital
helicopter transfer. Stroke 40: 895–901.
39. Coyne CA, Xu R, Raich P, Plomer K, Dignan M, et al. (2003) Randomized,
controlled trial of an easy-to-read informed consent statement for clinical trial
participation: a study of the Eastern Cooperative Oncology Group. J Clin Oncol
21: 836–842.
40. Ellis PM, Butow PN, Tattersall MH (2002) Informing breast cancer patients
about clinical trials: a randomized clinical trial of an educational booklet. Ann
Oncol 13: 1414–1423.
41. Du W, Mood D, Gadgeel S, Simon MS (2008) An educational video to increase
clinical trials enrollment among lung cancer patients. J Thorac Oncol 3: 23–29.
42. Hutchison C, Cowan C, McMahon T, Paul J (2007) A randomised controlled
study of an audiovisual patient information intervention on informed consent
and recruitment to cancer clinical trials. Br J Cancer 97: 705–711.
43. Llewellyn-Thomas HA, Thiel EC, Sem FW, Woermke DE (1995) Presenting
clinical trial information: a comparison of methods. Patient Educ Couns 25:
97–107.
44. Weston J, Hannah M, Downes J (1997) Evaluating the benefits of a patient
information video during the informed consent process. Patient Educ Couns 30:
239–245.
45. Berner ES, Partridge EE, Baum SK (1997) The effects of the pdq patient
information file (pif) on patients’ knowledge, enrollment in clinical trials, and
satisfaction. J Cancer Educ 12: 121–125.
46. Wadland WC, Hughes JR, Secker-Walker RH, Bronson DL, Fenwick J (1990)
Recruitment in a primary care trial on smoking cessation. Fam Med 22:
201–204.
47. Aaronson NK, Visser-Pol E, Leenhouts GH, Muller MJ, van der Schot AC,
et al. (1996) Telephone-based nursing intervention improves the effectiveness of
the informed consent process in cancer clinical trials. J Clin Oncol 14: 984–996.
48. Mandelblatt J, Kaufman E, Sheppard VB, Pomeroy J, Kavanaugh J, et al. (2005)
Breast cancer prevention in community clinics: Will low-income Latina patients
participate in clinical trials? Prev Med 40: 611–612.
49. Simes RJ, Tattersall MH, Coates AS, Raghavan D, Solomon HJ, et al. (1986)
Randomised comparison of procedures for obtaining informed consent in
clinical trials of treatment for cancer. Br Med J (Clin Res Ed) 293: 1065–1068.
50. Wragg JA, Robinson EJ, Lilford RJ (2000) Information presentation and
decisions to enter clinical trials: a hypothetical trial of hormone replacement
therapy. Soc Sci Med 51: 453–462.
51. Pighills A, Torgenson DJ, Sheldon T (2009) Publicity does not increase
recruitment to falls prevention trials: the results of two quasi-randomized trials.
J Clin Epidemiol 62: 1332–1335.
52. Leader MA, Neuwirth E (1978) Clinical research and the noninstitutional
elderly: a model for subject recruitment. Journal J Am Geriatr Soc 26: 27–31.
53. Caldwell PH, Butow PN, Craig JC (2003) Parents’ attitudes to children’s
participation in randomized controlled trials. J Pediatr 142: 554–559.
54. Becker MH (1974) The health belief model and personal health behavior.
Health Educ Monogr 2.
127
15 November 2010 | Volume 7 | Issue 11 | e1000368
World Health Report 2012: No Health Without Research
Editors’ Summary
Background. Before any health care intervention—a
treatment for a disease or a measure such as vaccination
that is designed to prevent an illness—is adopted by the
medical community, it undergoes exhaustive laboratory-
based and clinical research. In the laboratory, scientists
investigate the causes of diseases, identify potential new
treatments or preventive methods, and test these
interventions in animals. New interventions that look
hopeful are then investigated in clinical trials—studies that
test these interventions in people by following a strict trial
protocol or action plan. Phase I trials test interventions in a
few healthy volunteers or patients to evaluate their safety
and to identify possible side effects. In phase II trials, a larger
group of patients receives an intervention to evaluate its
safety further and to get an initial idea of its effectiveness. In
phase III trials, very large groups of patients (sometimes in
excess of a thousand people) are randomly assigned to
receive the new intervention or an established intervention
or placebo (dummy intervention). These ‘‘randomized
controlled trials’’ or ‘‘RCTs’’ provide the most reliable
information about the effectiveness and safety of health
care interventions.
Why Was This Study Done? Patients who participate in
clinical trials must fulfill the inclusion criteria laid down in the
trial protocol and must be given information about the trial,
its risks, and potential benefits before agreeing to participate
(informed consent). Unfortunately, many RCTs struggle to
enroll the number of patients specified in their trial protocol,
which can reduce a trial’s ability to measure the effect of a
new intervention. Inadequate recruitment can also increase
costs and, in the worst cases, prevent trial completion.
Several strategies have been developed to improve
recruitment but it is not clear which strategy works best. In
this study, the researchers undertake a systematic review (a
study that uses predefined criteria to identify all the research
on a given topic) of ‘‘recruitment trials’’—studies that have
randomly divided potential RCT participants into groups,
applied different strategies for recruitment to each group,
and compared recruitment rates in the groups.
What Did the Researchers Do and Find? The researchers
identified 37 randomized trials of recruitment strategies into
real and mock RCTs (where no actual trial occurred). In all,
18,812 people agreed to participate in an RCT in these
recruitment trials out of at least 59,354 people approached.
Some of these trials investigated novel strategies for
recruitment, such as changes in how patients are
randomized. Others looked at the effect of recruiter
differences (for example, increased contact between the
health care professionals doing the recruiting and the trial
investigators), the effect of offering monetary incentives to
participants, and the effect of giving more information about
the trial to potential participants. Recruitment strategies that
improved people’s awareness of the health problem being
studied—provision of an interactive computer program or a
video about the health condition, attendance at an
educational session, or inclusion of a health questionnaire
in the recruitment process—improved recruitment rates, as
128
PLoS Medicine | www.plosmedicine.org
Recruitment Strategies for Trials
did monetary incentives. Increasing patients’ understanding
about the trial process itself, recruiter differences, and
alterations in consent design and randomization generally
had no effect on recruitment rates although consent rates
were higher when patients knew the treatment to which
they had been randomly allocated before consenting.
However, differential losses among the patients in different
treatment groups in such nonblinded trials may jeopardize
study findings.
What Do These Findings Mean? These findings suggest
that trial recruitment strategies that focus on increasing the
awareness of potential participants of the health problem
being studied and its possible effects on their health, and
that engage potential participants in the trial process are
likely to increase recruitment to RCTs. The accuracy of these
findings depends on whether the researchers identified all
the published research on recruitment strategies and on
whether other research on recruitment strategies has been
undertaken and not published that could alter these
findings. Furthermore, because about half of the recruit-
ment trials identified by the researchers were undertaken in
the US, the successful strategies identified here might not
be generalizable to other countries. Nevertheless, these
recruitment strategies should now be investigated further
to ensure that the future evaluation of new health care
interventions is not hampered by poor recruitment into
RCTs.
Additional Information. Please access these Web sites via
the online version of this summary at http://dx.doi.org/
10.1371/journal.pmed.1000368.
N The ClinicalTrials.gov Web site is a searchable register of
federally and privately supported clinical trials in the US
and around the world, providing information about all
aspects of clinical trials
N The US National Institutes of Health provides information
about clinical trials
N The UK National Health Service Choices Web site has
information for patients about clinical trials and medical
research
N The UK Medical Research Council Clinical Trials Units also
provides information for patients about clinical trials and
links to information on clinical trials provided by other
organizations
N MedlinePlus has links to further resources on clinical trials
(in English and Spanish)
N The Australian Government’s National Health and Medical
Research Council has information about clinical trials
N WHO International Clinical Trials Registry Platform aims to
ensure that all trials are publicly accessible to those making
health care decisions
N The Star Child Health – International Forum of Standards
for Research is a resource center for pediatric clinical trial
design, conduct, and reporting
iMedPub :: Thematic Collections :: Volume 2
16 November 2010 | Volume 7 | Issue 11 | e1000368
 World Health Report 2012: No Health Without Research

Development of and Access to Products for Neglected

Diseases
Joshua Cohen1*, Maria Staroselsky Dibner2, Andrew Wilson1
1 Tufts University Center for the Study of Drug Development, Boston, Massachusetts, United States of America, 2 Lahey Clinic, Department of Medicine, Burlington,
Massachusetts, United States of America

Abstract
Introduction: Prior research on neglected disease drug development suggested inadequate funding was responsible for
relatively few new approvals. In response, significantly more resources have been allocated towards development of drugs
targeting neglected diseases. Our objective was to reassess drug development between1975 and 1999, evaluate progress in
neglected disease drug development since 2000, and explain how increased numbers of approvals are a necessary but
insufficient condition to improving access.

Methods: To assess numbers of approvals targeting neglected diseases, we employed two distinct methodologies: First, to
revisit numbers published in Trouiller et al. (2002) we used their method to count marketed new chemical entities (NCEs)
between 1975 and 1999. Second, using the G-Finder report as a benchmark, we identified which diseases are currently
considered ‘‘neglected’’ to tally approvals in the 1975–1999 and 2000–2009 periods. Searching PharmaProjects and IMS R&D
Focus databases as well as websites from numerous drug regulatory agencies, we identified new drug approvals and
indications. Also, we examined the World Health Organization’s (WHO) Essential Drug List (EDL) to see which drugs and
indications were on the list.

Findings: Upon recount, using Trouiller et al. methodology, we found that between 1975 and 1999 more NCEs (n = 32)
targeting tropical diseases and tuberculosis were approved than reported in Trouiller et al. (n = 16). Using the G-Finder
method of defining neglected diseases, we found 46 new drug approvals between 1975 and 1999. WHO included 85% of
these drugs on the EDL. In the period 2000 to May 2009, despite much greater funding, only 26 new drugs and vaccines for
neglected diseases were marketed. Of these, WHO placed 50% on the EDL.

Conclusions: Product approvals for neglected diseases have increased, though progress has been uneven, with malaria
appearing to benefit most in the short run from increased funding, while less success has been booked in other disease
categories. Uneven progress suggests funding could be better targeted, particularly with regard to neglected diseases that
have hitherto received scant attention. In addition, policymakers should focus on other aspects related to access. Besides
drug development, there are the issues of EDL listing, architecture, availability, affordability, and adoption.

Citation: Cohen J, Dibner MS, Wilson A (2010) Development of and Access to Products for Neglected Diseases. PLoS ONE 5(5): e10610. doi:10.1371/
journal.pone.0010610
Editor: Joseph S. Ross, Mount Sinai School of Medicine, United States of America
Received January 15, 2010; Accepted April 8, 2010; Published May 12, 2010
Copyright: � 2010 Cohen et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors have no support or funding to report.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: joshua.cohen@tufts.edu

Introduction industry to dedicate research and development (R&D) resources to

Neglected diseases are infectious diseases that primarily,
though not exclusively, affect vulnerable populations in devel-
oping countries where poor sanitation and lack of access to
health care foster disease transmission and vector proliferation.
These diseases, which include malaria, tuberculosis, diarrheal
diseases, and kinetoplastids such as leishmaniasis, cause 35,000
deaths per day in the developing world along with significant
morbidity.[1]
There is great interest in the public health community in
developing new products to treat or prevent these diseases.
However, given limited health care budgets in most developing
nations, the general public’s weak purchasing power, and the
correspondingly low likelihood of a satisfactory return on
investment, there is comparatively little incentive for private
developing medicines for these markets.[2]
In a widely cited 2002 study, Trouiller et al. reported that of
1393 new chemical entities (NCEs) marketed between 1975 and
1999, only 16 targeted ‘‘tropical diseases’’ and tuberculosis.[3]
Furthermore, Trouiller et al. found that in 1999 less than $70
million was invested in drug research and development for
malaria, tuberculosis, leishmaniasis, and African trypanosomias
combined.[4]Their study galvanized thought leaders to proclaim
the necessity of greater investment in neglected disease drug
development. As such it served as a clarion call to action for
governments, non-profit foundations, private-public partnerships,
and the private industry to earmark more resources to battle this
public health problem.
Since 1999, funding has greatly increased.[5] Additionally,
definitions of neglected disease have expanded beyond the tropical

iMedPub :: Thematic collections :: Volume 2 129

PLoS ONE | www.plosone.org 1 May 2010 | Volume 5 | Issue 5 | e10610
World Health Report 2012: No Health Without Research
diseases and tuberculosis included in Trouiller et al. Large new
cooperative ventures have begun to take shape, including the
Global Network for Neglected Tropical Disease Control, Medi-
cines for Malaria Venture (MMV), the Drugs for Neglected
Diseases Initiative (DNDi), and numerous partnerships with
pharmaceutical companies, including Merck, GlaxoSmithKline,
Pfizer, Novartis and Sanofi-Aventis. According to the G-Finder
report, in 2007, $2.5 billion was invested in neglected disease drug
development, with $980 million targeting malaria, tuberculosis,
and the kinetoplastids combined. Nearly 80% of funds poured into
the ‘‘big three:’’ HIV/AIDS, tuberculosis, and malaria. Of this
amount, nearly 90% came from public or philanthropic donors,
with the National Institutes of Health and the Bill and Melinda
Gates Foundation leading the way.[6]
In comparison with a decade ago, more resources are being
spent to address the problem of neglected diseases. A way of
addressing the question of whether increased funding has been
effective is to mark progress in new approvals. A progress report
was published in 2006, which showed several new approvals over
the 2000–2004 period.[7] However, our study provides an
updated, in-depth examination of new approvals through May
of 2009. We also analyze whether new approvals are being
included in the WHO’s EDL. Together with WHO treatment
guidelines, the EDL forms the basis for public health policy in
many developing countries. Finally, we examine the larger
question of pharmaceutical access.
First, our paper revisits numbers of approved drugs targeting
‘‘tropical diseases’’ and tuberculosis previously published by
Trouiller et al. Second, we mark progress in neglected disease
drug approvals since 1999. Finally, we explain how increased
numbers of approvals are a necessary but insufficient condition to
improving access.
Methods
To assess new approvals targeting neglected diseases, we
employed two distinct methodologies: First, we used the method
in Trouiller et al. to count marketed new chemical entities (NCEs)
between 1975 and 1999, as well as fixed dose combination
products. Second, referencing the authoritative G-Finder report as
a benchmark, we identified currently defined neglected diseases to
tally approved products and indications in the 1975–1999 and
2000–2009 periods. The G-Finder report investigated funding
allocated to the following diseases: malaria, tuberculosis, bacterial
pneumonia and meningitis, pneumonia, rotavirus, enterotoxigenic
and enteroaggregative E. Coli, cholera, shigella, cryptosporidium,
giardia, Chagas disease, leishmaniasis, African trypanosomiasis,
roundworm (ascariasis), hookworm (ancylostomiasis & necatoria-
sis), whipworm (trichuriasis), strongyloidiasis and other intestinal
roundworms, lymphatic filariasis (elephantiasis), onchocerciasis
(river blindness), schistosomiasis (bilharziasis), tapeworm (cyster-
cercosis/taeniasis), leprosy, trachoma, Buruli ulcer, Dengue fever,
rheumatic fever, typhoid and paratyphoid fever, and HIV/AIDS
(products with applications specific to the developing world, such
as vaccines, microbicides and pediatric label extensions). This
enumeration of diseases builds upon Trouiller et al. Besides the
inclusion of Buruli ulcer, bacterial pneumonia and meningitis,
rheumatic fever, typhoid and paratyphoid fever, and the exclusion
of Japanese encephalitis, the biggest difference is that HIV/AIDS
drugs and anti-diarrhoeals are included in the G-Finder count, so
long as they are indicated for conditions pertinent to the
developing world. In the case of HIV/AIDS products this implies
pediatric use, vaccines and microbicides. Anti-diarrhoeal drugs
were only counted if indicated for cholera, shigella, or cryptospo-
130
PLoS ONE | www.plosone.org
Drug Development and Access
ridium. Finally, anti-diarrhoeal vaccines were included if they
targeted one or more diseases across the entire diarrhoeal
spectrum.
In contrast to Trouiller et al., we categorized indications
separately in our G-Finder based count. If a drug received
approval for more than one disease, we counted each instance as a
newly approved indication though only one instance as a newly
approved drug. For example, mebendazole can be used for four
different helminths, so it was counted for four indications, but just
once as a newly approved drug. Note that new doses or
formulations of (combination) drugs did not count towards new
approvals.
We identified new product approvals and indications using the
PharmaProjects and IMS R&D Focus databases, websites from the
Food and Drug Administration, European Medicines Agency, and
drug regulatory agencies of France, Germany, the Netherlands,
the United Kingdom, China, Kenya, and India. We also
examined the most recent version of the EDL to see which drugs,
indications, and vaccines are recommended for use.[8]
Results
Using the same NCE count methodology employed in Trouiller
et al. we found that their figure of 16 appears to have
undercounted the total number of drugs approved for ‘‘tropical
diseases’’ and tuberculosis between 1975 and 1999; namely, 32.
Moreover, the Trouiller et al. list is inaccurate as five of the 16
drugs were not approved between 1975 and 1999.
According to our tally based on a G-Finder definition of
neglected diseases, 46 new products were approved between 1975
and 1999 targeting neglected diseases, with a total of 56
indications. Of these, 6 were for pediatric HIV (note, no
microbicides or vaccines were approved), 7 for malaria, 12 for
tuberculosis, three for bacterial pneumonia and meningitis, two
new drugs and four new indications for diarrheal diseases, two for
kinetoplastids, 9 new drugs and 16 new indications for helminths,
two for leprosy and one each for trachoma, rheumatic fever, and
typhoid fever. No new products were approved for Buruli ulcer
and Dengue fever. Of the 46 new drug approvals, 39 (85%) were
placed on the EDL. And, of the 56 new indications approved for
marketing, 46 (82%) were added to the EDL.
In table 1 we list the original Trouiller et al. numbers, our
recount of Trouiller et al., an analysis of numbers based on a
broader G-Finder definition of neglected diseases, and a tally of
percentages of drugs and indications on the EDL. The appendix
(Appendix S1) includes tables which provide a detailed count of all
approvals, their year of approval, and the regulatory authority that
made the initial approval.
Between 2000 and May 2009, 26 products for neglected
diseases were marketed with a total of 26 indications. Of these, the
WHO had placed 50% on the 2009 EDL. The greatest number of
approvals occurred in malaria with 11 new drugs being marketed.
An additional 10 new HIV/AIDS drugs were granted pediatric
labeling; one new drug and two vaccines for diarrheal diseases; one
vaccine was developed against bacterial meningitis, and one new
drug was approved for kinetoplastids. No other disease category
had any new drugs approved in the last 9 years. Table 2 lists all
products approved for neglected diseases between 2000 and May
2009 along with their EDL status.
For a complete listing of all products and indications approved
in the 1975–1999 and 2000–2009 periods, please see appendix
(Appendix S1).
The percentage of approved neglected disease products
sponsored by the private pharmaceutical industry dropped from
iMedPub :: Thematic Collections :: Volume 2
2 May 2010 | Volume 5 | Issue 5 | e10610
 World Health Report 2012: No Health Without Research
Drug Development and Access

Table 1. Numbers of 1975-1999 Approvals for Neglected Diseases.

Trouiller et al. new chemical

entities (NCEs) targeting ‘‘tropical
Disease Category diseases’’ and tuberculosis Our Analysis
Our tally of
G-Finder defined
Our tally of G-Finder indications
Our recount of defined drugs targeting targeting
Trouiller et al. NCEs neglected diseases neglected diseases

HIV/AIDS* n/a** n/a** 6*** 6

Malaria 4 7 7 7
Tuberculosis 3 12 12 12
Bacterial Pneumonia and Meningitis n/a** n/a** 3 3
Diarrheal Diseases n/a** n/a** 2 4
Kinetoplastids 5 2 2 2
Buruli Ulcer n/a** 0 0 1
Dengue Fever 0 0 0 0
Helminths 4 9 9 16
Leprosy 0 2 2 2
Trachoma n/a** n/a** 1 1
Rheumatic Fever n/a** n/a** 1 1
Typhoid and Paratyphoid Fever n/a** n/a** 1 1
Total Approvals 16 32 46 56
Percentage on Essential Drug List 94% 85% 85% 82%

*In their analysis, Trouiller et al. included 26 HIV drugs-20 anti-virals and 6 drugs for ‘‘opportunistic diseases’’-but as a separate (non-neglected) disease category.
**Disease categories Trouiller et al. did not include in their analysis.
***HIV/AIDS drugs with applications specific to the developing world, such as vaccines, microbicides and pediatric label extensions. Sources: PharmaProjects, IMS R&D
Focus, http://www.accessdata.fda.gov/Scripts/cder/DrugsatFDA, http://eudrapharm.eu/eudrapharm/searchbykeyword.do, http://www.cbg-meb.nl/CBG/en/human-
medicines/actueel, http://www.pharmacyboardkenya.org/index.php?id = 13&dpgndg1 = 42&an = , http://www.who.int/medicines/publications/
08_ENGLISH_indexFINAL_EML15.pdf, http://www.cdsco.nic.in/, http://www.afssaps.fr/, http://www.mhra.gov.uk/index.htm, http://www.bmg.bund.de.
doi:10.1371/journal.pone.0010610.t001

83% to 46% between the two time periods, while the percentage
sponsored by public-private partnerships increased from 15% to
46% (see table 3). Here, sponsorship implies the sponsor paid for a
product’s development through the clinical phases.
It may be problematic to mark progress using numbers of
approvals. Clearly, many of the product development efforts that
began in 2000–2009 have not (yet) resulted in new product approvals,
given the (variable) length of time between initial funding of R&D and
registration. Indeed, collecting data on new products in clinical
development, we see promising signs as 97 are currently in the
pipeline despite the fact that certain therapeutic areas continue to be
neglected (see table 4). This said, we do not think it is far-fetched to
already be looking for indications of progress in terms of numbers of
new approvals, particularly given the fact that the conventional 10–15
year period to register a drug may not invariably apply as numerous
products currently being developed by PPPs have been granted
accelerated approval times.[9]
Discussion
Regardless of how one does the counting, the 1975–1999 period
was not good for neglected disease product development.
However, since Trouiller’s call to action, there has been progress
in neglected disease product development, albeit in uneven strides.
For example, malaria has seen a 250% increase in numbers of new
products compared to the 1975–1999 period. Nevertheless,
malaria appears to be the exception rather than the rule. While
tuberculosis has received similar funding to malaria, not a single
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
new tuberculosis drug has been approved in the last nine years.
Although this may be in part due to longer development times for
tuberculosis products, we also observe fewer products in the
clinical development pipeline than with malaria. Likewise, despite
HIV/AIDS R&D towards applications specific to the developing
world totaling $1.8 billion in 2007, of which 64% went to vaccine
development and 18% to microbicides, no vaccines or microbi-
cides have been approved. Finally, not a single new product has
been approved in the last 9 years in disease categories that include
Buruli ulcer, Dengue fever, trachoma, rheumatic fever, or typhoid
and paratyphoid fevers.
Here, we do not wish to leave the impression that the now
predominant PPP model is not promising. Besides having
numerous products in the clinical development pipeline that
may prove invaluable, PPPs have demonstrated an ability to
develop drugs with high health impact. PPP-based products, such
as the arthemeter + lumefantrine combination, are making a
difference in the developing world.
Separate from R&D funding levels and new approvals is the
issue of inclusion on the EDL, which does not guarantee access but
may act as a lever to increase access. The EDL recommends what
it considers to be the most essential, yet cost-effective drugs
available. As such, it forms the basis for public health policy in
many developing nations.[10] There are clear linkages between
EDL, clinical practice guidelines, and drug procurement practices.
These relationships underscore the ‘‘operational, educational, and
symbolic functions of an essential drugs list:’’[11] Operational in
that the EDL identifies medicines for priority attention for
131
3 May 2010 | Volume 5 | Issue 5 | e10610
World Health Report 2012: No Health Without Research
Drug Development and Access

Table 2. Numbers of 2000–2009 Approvals for G-Finder Table 4. Neglected Disease Products in Clinical Development
Defined Neglected Diseases. as of July 2009.

Disease Category Our Analysis Disease Category Drugs Vaccines Microbicides

Drugs/Vaccines Indications HIV/AIDS 0 24 4

HIV/AIDS* 10 10 Malaria 9 19 —

Malaria 11 11 Tuberculosis 5 7 —

Tuberculosis 0 0 Bacterial Pneumonia and Meningitis 0 4 —

Bacterial Pneumonia and 1 1 Diarrheal Diseases 0 11 —

Meningitis Kineptoplastids 4 2 —
Diarrheal Diseases 3 3 Buruli Ulcer 0 0 —
Kinetoplastids 1 1 Dengue Fever 0 3 —
Buruli Ulcer 0 0 Helminths 1 1 —
Dengue Fever 0 0 Leprosy 0 0 —
Helminths 0 0 Trachoma 0 0 —
Leprosy 0 0 Rheumatic Fever 0 0 —
Trachoma 0 0 Typhoid and Paratyphoid Fever 0 3 —
Rheumatic Fever 0 0 Total 19 74 4
Typhoid and Paratyphoid 0 0
Fever Sources: BIO Ventures for Global-R&D Landscape, ,http://www.bvgh.org/
resources/landscape/default.asp.; Cowen and Co. Pharmaceuticals Industry
Total Approvals 26 26 Overview, May 2009; PhRMA Medicines in Development Factsheets for HIV/
Percentage on Essential 50% 50% AIDS, infectious disease, and biotechnology; International Federation of
Drug List Pharmaceutical Manufacturers and Associations, ‘‘Pharmaceutical Industry R&D
for Diseases of the Developing World-2009’’, ,http://www.ifpma.org/
*HIV/AIDS drugs with applications specific to the developing world, such as documents/NR12400/Status_RnD_for_DDW_07Jul09.pdf.; ClinicalTrials.gov;
vaccines, microbicides and pediatric label extensions. Sources: PharmaProjects, Moran, et al. (2007) ‘‘The Malaria Product Pipeline: Planning for the Future.’’ The
IMS R&D Focus, http://www.accessdata.fda.gov/Scripts/cder/DrugsatFDA, George Institute for International Health, ,http://www.thegeorgeinstitute.org/
http://eudrapharm.eu/eudrapharm/searchbykeyword.do, http://www.cbg-meb. research/health-policy/current-projects/the-malaria-product-pipeline-planning-
nl/CBG/en/human-medicines/actueel, http://www.pharmacyboardkenya.org/ for-the-future.cfm.; websites, press releases, and reports from various PDPs
index.php?id = 13&dpgndg1 = 42&an = , http://www.who.int/medicines/ and NGOs including the Meningitis Vaccine Project, PATH (including Malaria
publications/08_ENGLISH_indexFINAL_EML15.pdf, http://www.cdsco.nic.in/, Vaccine Initative), Medicines for Malaria Venture, Alliance for Microbicide
http://www.afssaps.fr/, http://www.mhra.gov.uk/index.htm, http://www.bmg. Development, Drugs for Neglected Diseases Initiative, International AIDS
bund.de. Vaccine Initiative, AIDS Vaccine Advocacy Coalition, Global Alliance for TB Drug
doi:10.1371/journal.pone.0010610.t002 Development Note to table 4: Table 4 presents a snapshot of active drugs,
vaccines, and microbicides in clinical development for neglected diseases as of
July 2009. Using the inclusion criteria outlined in Moran et al., the majority of
acquisition and distribution; educational in that the EDL products were located using commercial and public reports from trade
influences drug utilization patterns; and symbolic in that the associations (PhRMA, IFPMA), commercial entities (Cowen and Co.), various
product development partnerships (PDPs) and NGOs (BioVentures for Global
EDL confers global recognition and a preferred position in
Health), and government agencies such as the NIH and US Department of
pharmaceutical management. Defense. Products were only included when their status could be verified
Of the drugs targeting neglected diseases that were developed through secondary sources, which included clinical trial records from
between 1975 and 1999, 85% are on the EDL. By comparison, only ClinicalTrials.gov, information posted on company websites, and website
information from the sources listed in the table. The table should not be
50% of products approved in the 2000–2009 period are on the EDL. considered an exhaustive list of every product in development. First, only new
This could be due to lag time as the WHO deliberates following each products were listed, which excludes previously approved products that are
now being developed for wider use. For example, the dihydroartemisinin +
piperaquine combination currently in pre-registration was left out because it
Table 3. Comparison of Approvals by Source of Sponsorship. was approved in Kenya in 2005 (see table 2), despite being considered by some
analysts as part of the pipeline. Second, our reliance on public and commercial
sources potentially overlooks other sources.
doi:10.1371/journal.pone.0010610.t004
1975–1999 2000–2009

Number of Individual Products Approved 46 26

new approval, but may also be the result of the higher cost of certain
newer drugs.[12] In a severely budget-constrained environment the
Number of Products Sponsored by PPP 7 (15%) 12 (46%)
(% of Total)
low cost of older drugs may tip the scales in their favor. Needless to
say, non-admittance to the EDL can serve as an access barrier, with
Number Sponsored by Private Industry (%) 38 (83%) 12 (46%)
possibly deleterious health outcomes.
Number Sponsored by Other/Unknown (%) 1 (2%) 2 (8%) It is often assumed that once a product is added to the EDL, the
Sources: PharmaProjects, IMS R&D Focus, http://www.accessdata.fda.gov/
access problem is resolved. However, besides EDL listing,
Scripts/cder/DrugsatFDA, http://eudrapharm.eu/eudrapharm/ numerous barriers to access persist, the foremost being resource
searchbykeyword.do, http://www.cbg-meb.nl/CBG/en/human-medicines/ constraints. Others include limited capacity of public health
actueel, http://www.pharmacyboardkenya.org/index. systems, lack of political commitment in terms of distribution and
php?id = 13&dpgndg1 = 42&an = , http://www.who.int/medicines/publications/
08_ENGLISH_indexFINAL_EML15.pdf, http://www.cdsco.nic.in/, http://www.
delivery of products and services, international trade and patent
afssaps.fr/, http://www.mhra.gov.uk/index.htm, http://www.bmg.bund.de. disputes, and cultural attitudes towards disease and products to
doi:10.1371/journal.pone.0010610.t003 remedy or prevent disease.[13]
132 iMedPub :: Thematic Collections :: Volume 2
PLoS ONE | www.plosone.org 4 May 2010 | Volume 5 | Issue 5 | e10610

Access also comprises availability, affordability, adoption, and
architecture. Availability refers to ‘‘the logistics of making, ordering,
shipping, storing, distributing and delivering a new health technology
to ensure it reaches the hands (or mouth) of the user.’’[14] This
implies that even if a product travels the drug development route
successfully, reaches market, and is recommended by WHO for use,
there is still the question of how it is to be distributed to the people
who need it. Affordability encompasses both the individual patient’s
ability to pay and that of governments and other payers. This is less of
a concern for medications such as ivermectin, which Merck donates.
But, it may be an acute issue for drugs such as artemisisin-based
combination therapies.[15] Adoption runs the gamut from a
product’s recommendation by international agencies like WHO to
its acceptance by local policymakers in developing countries, to
patients as well as health care providers, some of whom may have
misgivings about taking certain products. Lastly, architecture refers to
the organizational dimension of access; decisions about organiza-
tional structure that are required for coordinating availability,
affordability, and adoption. In this context, consider, for example,
the anti-helminths. Given that there are already 9 drugs that are
effective at treating helminths, it appears that the need for
development of new anti-helminths is less critical than the need for
improved access. Hence, the helminths appear to be neglected
diseases not because of a drug deficit but due to limited effective
means of getting these drugs to the people who need them.
In sum, funding of neglected disease R&D is highly concen-
trated, with significant funding flowing into HIV/AIDS, malaria
and tuberculosis product development. Progress is lopsided, with
marked strides in the area of malaria research, yet few end
products in others. This suggests the infusion of more money itself
is insufficient, while better targeting of funds may be warranted.
Moreover, a balanced comprehensive approach to address the
neglected disease problem will involve not only drug development
but also attention paid to public health infrastructure and
capacity-building to improve access.
References
1. Fehr A, Thurmann P, Razum O (2006) Editorial: drug development for
neglected diseases: a public health challenge. Trop Med Int Health 11(9):
1335–8.
2. Trouiller P, Torreele E, Olliaro P, White N, Foster S, et al. (2001) Drugs for
neglected diseases: a failure of the market and a public health failure. Trop Med
Int Health 6(11): 945–51.
3. Trouiller P, Olliaro P, Torreele E, Orbinski J, Laing R, et al. (2002) Drug
development for neglected diseases: a deficient market and a public-health policy
failure. Lancet 22;359(9324): 2188–94.
4. Ensuring success and targeting eradication: MMV business plan 2008-
2012. http://www.mmv.org/sites/default/files/uploads/docs/press_releases/
FINALMMV_Business_Plan_2008-2012.pdf, accessed 3/01/2010.
5. Moran M (2005) A breakthrough in R&D for neglected diseases: new ways to get
the drugs we need. PLoS Med 2(9): e302.
6. Moran M, Guzman J, Ropars A, McDonald A, Jameson N, et al. (2009)
Neglected disease research and development: how much are we really spending?
PLoS Med 3;6(2): e30.
7. Chirac P, Torreele E (2006) Global framework on essential health R&D. Lancet
367(9522): 1560–1.
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
Drug Development and Access
Supporting Information
Appendix S1 *Trouiller et al. did not include year of approval
for each drug, nor the regulatory agency that made the initial
approval. Using multiple sources, including IMS R&D Focus and
PharmaProjects, we identified both year of approval and the
regulatory agency that made the initial approval. **Trouiller et al.
included fivedrugs on their 1975–1999 approvals list that were not
approved during that period. First, they included pyrazinamide as
a separate NCE. Our research found that this drug was first
approved in 1954. However, we did find that pyrazinamide was
approved as part of a combination product in 1998 (isoniazid +
rifampicin + pyrazinamide + ethambutol). Second, Trouiller et al.
included the atovaquone + proguanil combination as a 1975–1999
approval. However, this combination product was approved in
2000. Therefore, in our list we included it as a 2000–2008
approval. Third, Trouiller et al. included benznidazole as a drug
approved between 1975 and 1999. Our research found that
benznidazole was first approved in 1972. Therefore, we did not
include it on our list of 1975–1999 approvals. Fourth, Trouiller et
al. included nifurtimox as a drug approved between 1975 and
1999. Our research found that nifurtimox was first launched in
1967 as a drug targeting Chagas’ disease. Therefore, we did not
include it on our list of 1975–1999 approvals. Fifth, Trouiller et al.
included pentamidine in the 1975–1999 approvals list. Our
research found that pentamidine was first approved in France in
1956 as a kinetoplastid.
Found at: doi:10.1371/journal.pone.0010610.s001 (0.14 MB
DOC)
Author Contributions
Conceived and designed the experiments: JPC MSD. Analyzed the data:
JPC MSD AW. Wrote the paper: JPC MSD AW.
8. WHO Model List of Essential Medicines. 15th List March 2007. http://www.
who.int/medicines/publications/08_ENGLISH_indexFINAL_EML15.pdf ac-
cessed 20/01/2010.
9. Nwaka S, Ridley R (2003) Virtual drug discovery and development for neglected
diseases through public-private partnerships. Nat Rev Drug Discov 2(11):
919–28.
10. Robertson J, Hill S (2007) The essential medicines list for a global patient
population. Clin Pharm Ther 82(5): 498–500.
11. Revised procedures for updating the WHO Model List of Essential Drugs: a
summary of proposals and process. http://www.searo.who.int/en/Section1257/
Section2263/info-kit/who-model-drug_list.pdf accessed 03/01/2010.
12. Reich M, Frost L (2008) Access: How Do Good Technologies Get To Poor
People in Poor Countries. Harvard Center for Population and Development
Studies, Cambridge, Massachusetts. See http://www.accessbook.org accessed
11/11/2009.
13. Resisting arrest: fighting malarial drug resistance. Economist, 15 May 2009
http://www.economist.com accessed 3/01/2010.
14. Frost L, Reich M (2009) Creating access to health technologies in poor countries:
Health Aff 28(4): 962–73.
15. Dentzer S (2009) Eliminating neglected diseases in poor countries: a
conversation with Andrew Witty. Health Aff 28(3): w411–w46.
133
5 May 2010 | Volume 5 | Issue 5 | e10610
World Health Report 2012: No Health Without Research

Relation between the Global Burden of Disease and

Randomized Clinical Trials Conducted in Latin America
Published in the Five Leading Medical Journals
Pablo Perel1, J. Jaime Miranda1,2*, Zulma Ortiz3, Juan Pablo Casas1
1 Department of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, United Kingdom, 2 School of Public Health and
Administration, Universidad Peruana Cayetano Heredia, Lima, Peru, 3 Epidemiological Research Institute, National Academy of Medicine, Buenos Aires, Argentina

Abstract
Background: Since 1990 non communicable diseases and injuries account for the majority of death and disability-adjusted
life years in Latin America. We analyzed the relationship between the global burden of disease and Randomized Clinical
Trials (RCTs) conducted in Latin America that were published in the five leading medical journals.

Methodology/Principal Findings: We included all RCTs in humans, exclusively conducted in Latin American countries, and
published in any of the following journals: Annals of Internal Medicine, British Medical Journal, Journal of the American
Medical Association, Lancet, and New England Journal of Medicine. We described the trials and reported the number of RCTs
according to the main categories of the global burden of disease. Sixty-six RCTs were identified. Communicable diseases
accounted for 38 (57%) reports. Maternal, perinatal, and nutritional conditions accounted for 19 (29%) trials. Non-
communicable diseases represent 48% of the global burden of disease but only 14% of reported trials. No trial addressed
injuries despite its 18% contribution to the burden of disease in 2000.

Conclusions/Significance: A poor correlation between the burden of disease and RCTs publications was found. Non
communicable diseases and injuries account for up to two thirds of the burden of disease in Latin America but these topics
are seldom addressed in published RCTs in the selected sample of journals. Funding bodies of health research and editors
should be aware of the increasing burden of non communicable diseases and injuries occurring in Latin America to ensure
that this growing epidemic is not neglected in the research agenda and not affected by publication bias.

Citation: Perel P, Miranda JJ, Ortiz Z, Casas JP (2008) Relation between the Global Burden of Disease and Randomized Clinical Trials Conducted in Latin America
Published in the Five Leading Medical Journals. PLoS ONE 3(2): e1696. doi:10.1371/journal.pone.0001696
Editor: Jocalyn Clark, Public Library of Science, Canada
Received September 25, 2007; Accepted January 25, 2008; Published February 27, 2008
Copyright: � 2008 Perel et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: JJM is supported by a Wellcome Trust Research Training Fellowship (GR074833MA). The authors have no additional support or funding to report.
Competing Interests: The authors have declared that no competing interests exist.
*E-mail: jaime.miranda@lshtm.ac.uk

Introduction medicine and public health worldwide. In relation to this group of

It has been estimated that less than 10% of health research
spending is directed toward diseases or conditions that account for
90% of the global burden of disease, a phenomenon referred to as
the ‘‘10/90 gap’’.[1] The gap is also reflected in the low
proportion of publications arising from research in low and
middle income countries, where most of the global burden of
disease occurs.[2]
Addressing context specific research questions are fundamental
to designing interventions that improve health. Randomized
controlled trials (RCTs) are the gold standard to assess the
effectiveness of interventions. An unbiased design and adequate
reporting of RCTs that specifically address diseases which affect
low and middle income countries can have an impact in changing
medical practice and influencing public health policy.
Five of the world’s leading medical journals—Annals of Internal
Medicine, British Medical Journal (BMJ), Journal of the American
Medical Association (JAMA), Lancet, and New England Journal of
Medicine (NEJM)—have together the highest impact in medical
research, and therefore, they can have a large influence on clinical
134
PLoS ONE | www.plosone.org
medical journals, one of its editors stated that ‘‘Their international
success brings global responsibilities to the communities they serve
and profit from...’’ and that they should raise ‘‘the priority of
papers from less-developed countries in line with global burdens of
disease’’.[3] Unfortunately, there is good evidence of the under
representation of research addressing common conditions from
developing countries in these leading journals.[2,4,5,6]
Bibliometric analyses—a set of methods used to study or
measure texts and information—have been done to illustrate the
gaps of the published medical literature, either in terms of subjects
covered, representation of journal’s editorial boards, and specif-
ically, the under-representation of low and middle income
countries in high-impact general medical journals.[6,7,8,9]
Latin-American and Caribbean countries have experienced a
rapid epidemiological transition and since 1990 non communica-
ble diseases and injuries already account for the majority of death
and disability-adjusted life years in the region.[10,11] No previous
studies have investigated the relationship between RCTs and the
burden of disease in Latin America. Description of this
relationship can provide information about gaps between the
iMedPub :: Thematic Collections :: Volume 2
1 February 2008 | Volume 3 | Issue 2 | e1696
 World Health Report 2012: No Health Without Research
GBD and Trials in LatinAmerica

health needs and the research conducted in the region. This Table 1. General Characteristics of the reports included
information can contribute to the establishment of a research
agenda and prioritize neglected conditions.
We analyzed the relationship between the global burden of Publications per journal n (%)
disease and RCTs conducted in Latin America published in five
Lancet 30 (46)
leading medical journals.
BMJ 12 (18)

Methods NEJM 12(18)

Eligibility Criteria JAMA 6 (9)

We included all RCT s in humans, exclusively conducted in Annals 6 (9)
Latin American countries (by this we mean that the population Publications per country* n
was located in Latin America irrespective of the origin of the Brazil 14
researcher) and published in any of the following journals: Annals of
Mexico 12
Internal Medicine, BMJ, JAMA, Lancet and NEJM. The definition
used for RCTs was an experiment in which investigators randomly Argentina 9

allocate eligible participants into an intervention group (arm) each Chile 6

of which receives one or more of the interventions (or none) that Venezuela 5
are being compared.[12] Haiti 5
Peru 5
Identification of the trials Colombia 4
We searched the PubMed database registers from 1990 up to
Unit of randomization n (%)
December 2006 using the Cochrane Sensitive Search Strategy, which is
highly sensitivity to identify RCTs, and combined it with the terms Individuals 54 (81)
‘‘Latin America’’ or ‘‘Latin America’’ and all the country names Clusters 12 (19)
from the region.[13] We limited the search strategy to the five Date of publication n (%)
selected medical journals (Annals of Internal Medicine, BMJ, JAMA, 1990–1992 10 (15)
Lancet and NEJM). (See Appendix S1 for the full search strategy).
1993–1995 14 (22)
1996–1998 10 (15)
Data extraction
1999–2001 8 (12)
From selected trials, the following data were extracted: name of
the first author, year of publication, country(ies) where the study 2002–2004 12 (18)
was conducted, number of participants, description of disease 2005–2006 12 (18)
evaluated according to groups of global burden of disease and type Type of funding n(%)
of funding. Data extraction and data entry was made indepen- Non commercial 42 (64)
dently by two reviewers. Differences in data extraction were
Commercial 6 (9)
resolved by a third party.
Both 12 (18)

Data analysis Not reported 6(9)

We conducted a descriptive analysis listing the number of Sample Size mean (range)
RCTs, participants and countries in which the studies were Patients included per report 275 (26 to 348,139)
conducted. We also reported the number of RCTs according to Conditions according to the Global n (%)
the main categories of the global burden of disease. Burden of Disease Study
Communicable diseases 38 (57)
Results Maternal perinatal and nutritional 19 (29)

Of 181 reports initially obtained, 66 (36%) fulfilled the inclusion
criteria and were retrieved for the analysis.
Table 1 displays the general characteristics of the reports
included. The majority of the trials were published in Lancet and
the countries with the highest numbers of publications were Brazil
and Mexico. Cluster trials accounted for one-fifth of all the
publications. We found no defined trend in the frequency of
publications during this time. Most of the trials were non-
commercial.
Communicable diseases accounted for 38 (57%) reports.
Maternal, perinatal, and nutritional conditions accounted for 19
(29%) trials, and 9 (14%) addressed non-communicable diseases.
Within the latter group, 4 trials evaluated cardiovascular disease, 1
cancer and 4 other non-communicable diseases. We found no
RCTs assessing interventions for injuries. The most common
individual condition evaluated was traveler’s diarrhea, which was
studied in 4 (6%) of the trials. (See table 2 for the full details of
reports included).
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
Non-Communicable diseases 9 (14)
doi:10.1371/journal.pone.0001696.t001
We found an overall poor correlation between the global
burden of disease and the topic of the published RCTs conducted
in Latin America. The mismatch is evident in the following figures:
non-communicable diseases represent 48% of the burden of
disease and 14% of the RCTs; no RCTs addressed injuries despite
these representing 18% of the burden of disease in 2000 (see
Figure 1).
Discussion
There is an evident mismatch between the burden of disease in
and publications from Latin America. Non communicable diseases
and injuries account for up to two thirds of the burden of disease in
Latin America but these topics are seldom addressed in published
135
2 February 2008 | Volume 3 | Issue 2 | e1696
World Health Report 2012: No Health Without Research
GBD and Trials in LatinAmerica

Table 2. Full details of the included reports

Global Burden of
Year Journal Participants Cluster Funding Specific Condition Disease Classification Countries

1990 Lancet 81621 no Non-commercial Salmonela Tiphy Communicable Chile

1990 Jama 227 no Non-commercial traveller’s diahrrea Communicable Mexico
1991 NEJM 101 no Commercial Infant meningitis Communicable Costa Rica
1991 NEJM 1194 no Non-commercial Hipertensive disorders Maternal, perinatal Argentina
in pregnancy and nutrition
1991 NEJM 86 no Commercial Infant diarrea Maternal, perinatal Costa Rica
and nutrition
1992 NEJM 2235 no Non-commercial Delivery and low birth weight Maternal, perinatal Argentina, Brasil, Cuba,
and nutrition and Mexico
1992 Jama 191 no Both traveller’s diahrrea Communicable Mexico
1992 NEJM 110 no Not reported Leshmaniasis Communicable Colombia
1992 Lancet 29113 no Non-commercial Lepra Communicable Venezuela
1992 Jama 278 no Commercial DTP vaccine response Communicable Chile
1993 Lancet 2606 no Non-commercial Delivery, episiotomy Maternal, perinatal Argentina
and nutrition
1993 Annals 126 no Commercial Artritis reumatoidea NCD Mexico
1993 Lancet 11124 si Both Diarrhea and respiratory Communicable Haiti
infections in infants
1993 Lancet 118 no Non-commercial HiV and Tuberculosis Communicable Haiti
1993 NEJM 275 no Both Diarrea Communicable Peru
1993 Lancet 1548 no Non-commercial Malaria Communicable Colombia
1993 Lancet 159 no Non-commercial Infant diarrea Communicable Guatemala and Brazil
1993 Lancet 4534 no Commercial Acute Miocardial Infarction NCD Argentina, Brasil, Chile,
Paraguay, Uruguay and
Venezuela
1994 Lancet 88 no Non-commercial traveller’s diahrrea Communicable Belize
1994 Lancet 1563 no Not reported Colera Communicable Peru
1994 Lancet 275 no Non-commercial Kangaroo low birth Maternal, perinatal Ecuador
weight infants and nutrition
1994 Lancet 516 no Not reported Heart Failure NCD Argentina
1994 Lancet 141 no Non-commercial Infant nutrition Maternal, perinatal Honduras
and nutrition
1994 Lancet 1240 no Non-commercial Diarrea e infecciones Maternal, perinatal Brasil
respitatorias pediatricas and nutrition
1996 Lancet 130 no Non-commercial Chagas Communicable Brasil
1997 NEJM 2207 no Both Rotavirus Diarrrea Communicable Venezuela
1997 BMJ 472 no Non-commercial Childhood Pneumonia Communicable Brasil
1997 Lancet 113 no Non-commercial Filariasis in childrem Communicable Haiti
1997 Annals 187 no Not reported Leshmaniasis Communicable Colombia
1997 Lancet 202 no Both Coronary Heart Disease NCD Argentina
1997 BMJ 591 no Non-commercial Blood pressure of children Maternal, perinatal Argentina
and nutrition
1998 Annals 176 no Non-commercial Malaria Communicable Colombia
1998 Lancet 627 no Non-commercial Haemofilus in infants Communicable Chile
1998 NEJM 53 no Not reported Acute Respiratory Syndrome Communicable Brasil
1999 BMJ 101 no Both Snake bite Communicable Brasil
1999 Jama 543 no Non-commercial Meningitis Communicable Chile
1999 Lancet 130 no Non-commercial Breastfeeding Maternal, perinatal Mexico
and nutrition
2000 Lancet 233 no Non-commercial Tuberculosis Communicable Haiti
2000 Lancet 12926 no Non-commercial HIV Communicable Nicaragua
2000 Annals 42 no Non-commercial HIV Communicable Haiti
2000 NEJM 135 no Commercial Infant diarrea Communicable Peru

136 iMedPub :: Thematic Collections :: Volume 2

PLoS ONE | www.plosone.org 3 February 2008 | Volume 3 | Issue 2 | e1696
 World Health Report 2012: No Health Without Research
GBD and Trials in LatinAmerica

Table 2. cont.
Global Burden of
Year Journal Participants Cluster Funding Specific Condition Disease Classification Countries

2001 Jama 1193 no Both Respiratory tract infection Communicable Dominicana

in infants
2002 BMJ 2913 si Both Leshmaniasis Communicable Venezuela
2002 Lancet 26 no Non-commercial Altitude polycythaemia NCD Bolivia
2003 BMJ 301 no Non-commercial Agitation in mental health Maternal, perinatal and Brasil
nutrition
2003 Lancet 240 no Non-commercial Depression Maternal, perinatal and Chile
nutrition
2004 Lancet 70 clusters si Non-commercial Infant and maternal health Maternal, perinatal and Honduras
nutrition
2004 BMJ 210 si Non-commercial Snake bite Communicable Ecuador
2004 BMJ 139 si Non-commercial Infant malnutrition Maternal, perinatal and Jamaica
nutrition
2004 Lancet 149276 si Non-commercial Delivery Maternal, perinatal and Argentina, Brasil, Cuba,
nutrition Guatemala and Mexico
2004 Jama 795 si Non-commercial Infant anaemia and nutrition Maternal, perinatal and Mexico
nutrition
2004 BMJ 120 no Non-commercial Tetanus Communicable Brasil
2004 NEJM 68 no Not reported Cardia arrest NCD Brasil
2004 NEJM 120 no Both Neurocisticercosys Communicable Peru
2005 Lancet 348139 si Non-commercial Tuberculosis Communicable Brasil
2005 Lancet 350 no Non-commercial Breastfeeding Maternal, perinatal and Brasil
nutrition
2005 BMJ 1518 no Non-commercial Heart Failure NCD Argentina
2005 Lancet 277 si Non-commercial Infan nutrition Maternal, perinatal and Peru
nutrition
2005 Annals 210 no Both traveller’s diahrrea Communicable Mexico
2005 NEJM 162 no Both Lupus NCD Mexico
2006 BMJ 232 no Non-commercial Depression Maternal, perinatal and Jamaica
nutrition
2006 Lancet 476 no Non-commercial Infant anaemia Maternal, perinatal and Mexico
nutrition
2006 BMJ 10049 si Both Dengue Communicable Venezuela
2006 Lancet 2373 si Non-commercial Parasites Communicable Ecuador
2006 BMJ 10954 si Non-commercial HIV Communicable Mexico
2006 Annals 30 no Non-commercial Glioblastoma NCD Mexico

doi:10.1371/journal.pone.0001696.t002

RCTs in the selected sample of journals. Most of the reports that
were retrieved evaluated infectious diseases, and within this group,
the predominant condition studied was traveler’s diarrhea, a
condition that may be more relevant to travelers from developed
countries visiting less developed areas. Previous studies have
reported the paucity of RCTs addressing the diseases of low and
middle income countries.[5,14,15] However, to the best of our
knowledge previous studies have not explored the relation between
global burden of disease and RCTs in Latin America.
The gap identified in our study may have different explanations
including lack of original research that mirror the burden of
disease or a higher rate of RCTs publications addressing
communicable, maternal, perinatal and nutritional conditions.
The latter could be due to better quality of the design and
reporting of the original studies. Alternatively editors and
reviewers from the leading journals could be biased to include
publications from Latin America when they focus on these
conditions.
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
Why is important to conduct RCTs in Latin America?
As previously stated, context is crucial to decide which
interventions are effective in specific populations because the
effect—and therefore the impact—of some interventions could
differ according to the setting. This is of particular relevance for
those interventions that address behavioral modifications or
involve health services.[16] Preventive interventions in injuries
targeting behaviors that could be strongly influenced by cultural
conditions or the adherence to medication in chronic conditions
like cardiovascular diseases are some of the situations where RCTs
are needed. A good example of this type of research is the DIAL
Trial which addressed the case management of patients with
chronic heart failure.[17] Furthermore, for chronic diseases the
relevant outcome of interest is quality of life, so local evidence is
necessary since this type of ‘‘soft’’ outcome is more culturally
dependent. Another important reason to conduct trials in the
region is that certain diseases predominantly occur in Latin
America, such as Chagas disease[18] or Hemorrhagic Argentine
137
4 February 2008 | Volume 3 | Issue 2 | e1696
World Health Report 2012: No Health Without Research
Figure 1. Relation between Global Burden of Disease and RCTs in LAC published in the five leading medical journals. The figure
shows an evident mismatch: non-communicable diseases represented 48% of the burden of disease in LAC and 14% of the RCTs. No RCTs addressed
injuries despite these representing 18% of the DALYs in 2000.
doi:10.1371/journal.pone.0001696.g001
Fever. These conditions may be of more interest to the region’s
population and scientific community.
Why is important to publish the results of the RCTs?
A fundamental step after study completion is to publish the
findings in scientific journals. The publication in the five leading
medical journals, in particular, can influence clinical practice and
policy.[19] These journals are particularly relevant in low and
middle income where limited resources require that medical
libraries subscribe to only a few international journals, and
therefore, prioritize those with the higher impact. The publication
of RCTs conducted on the region will not only disseminate the
results appropriately but will also raise the awareness of the topic
and potentially could help to increase the funding for research in
these conditions.
Limitations
A limitation of our study is that we only considered five leading
medical journals. We acknowledge that many of the trials
conducted in Latin America may have been published in different
journals, however we considered that those published on the
leading journals will have the larger visibility and impact amongst
the medical and health care community.
Another important limitation is that we included RCTs
exclusively conducted in Latin America and, therefore, we
excluded international trials which may have enrolled participants
from this region. Most of the international trials are sponsored by
the pharmaceutical industry, which may answer important
questions but are less likely to influence the public health of low
and middle income countries in the short term.[20] Nonetheless,
the Latin America region has also seen good examples of
international trials that study generic drugs for prevalent diseases
or conditions with direct relevance to the region. For example,
Latin America has had an active participation in the MRC
CRASH Trial,[21] which evaluated the effect of corticosteroids on
head injury, or the CREATE-ECLA trial,[22] which evaluated the
138
PLoS ONE | www.plosone.org
GBD and Trials in LatinAmerica
effect of the glucose-insulin-potassium (GIK) infusion in patients
with myocardial infarction. Another good example is the Magpie
trial, which assessed the effects of magnesium sulphate in
approximately 10,000 women with pre-eclampsia, including
women from Latin America, and resulted in a change of clinical
practice.[23] Conducting regional RCTs will also promote the
development of skills and infrastructure which will allow Latin
America’s researchers to participate in international collaborations
addressing these generic but locally relevant questions.
Finally, a further limitation of this report is that we only
included RCTs, and it is possible that observational studies carried
out in topics such as non communicable diseases and injuries are
less likely to be published in the five leading medical journals
which may favor RCTs.
What are the future challenges?
In this paper we only focused on published reports, yet it is
widely recognized that only a small proportion of studies reach the
publication stage. Unfortunately, at the moment, there is not any
source with complete information about RCTs conducted
available in Latin America. The World Health Organization is
actively promoting an international initiative to develop registers
of controlled trials.[24] In Latin America, the Colombian Branch
of The Iberoamerican Cochrane Network developed the Latin
American Ongoing Clinical Trials Register (LATINREC) which,
when fully active, will represent a unique opportunity to obtain a
detailed profile of the research conducted in the region.[25] The
analysis of this register will confirm or not the gap found in this
report and it will enable researchers and policy makers to draw an
appropriate research profile of the region. It will be possible to
identify duplication of work, inequitable funding of research,
neglected diseases and other aspect such as source of funding and
quality of design and reporting of clinical trials.[26]
In addition, editorial boards should not only avoid ‘‘bias against
the diseases of poverty’’ but, in the context of the epidemiological
transition occurring in developing regions, such as Latin America,
iMedPub :: Thematic Collections :: Volume 2
5 February 2008 | Volume 3 | Issue 2 | e1696

they should also avoid the bias of what they consider diseases of
poverty. This predisposition could partially explain why while
Latin America experienced an increase of non-communicable
diseases and injuries in the last 20 years, this epidemic was not
mirrored by the RCTs published in the five leading medical
journals.
Supporting Information
Appendix S1
Found at: doi:10.1371/journal.pone.0001696.s001 (0.03 MB
DOC)
References
1. Global Forum for Health Research (2002) The 10/90 report on health research
2001–2002 Geneva: Global Forum for Health Research.
2. Horton R (2003) Medical journals: evidence of bias against the diseases of
poverty. Lancet 361: 712–713.
3. Horton R (2000) North and South: bridging the information gap. Lancet 355:
2231–2236.
4. Obuaya C-C (2002) Reporting of research and health issues relevant to resource-
poor countries in high-impact medical journals. Euro Science Edit 28: 72–77.
5. Rochon PA, Mashari A, Cohen A, Misra A, Laxer D, et al. (2004) Relation
between randomized controlled trials published in leading general medical
journals and the global burden of disease. CMAJ 170: 1673–1677.
6. Sumathipala A, Siribaddana S, Patel V (2004) Under-representation of
developing countries in the research literature: ethical issues arising from a
survey of five leading medical journals. BMC Medical Ethics 5: 5.
7. Bakke P, Rigter H (1985) Editors of medical journals: Who and from where.
Scientometrics 1: 11–22.
8. Langer A, Diaz-Olavarrieta C, Berdichevsky K, Villar J (2004) Why is research
from developing countries underrepresented in international health literature,
and what can be done about it? Bull World Health Organ 82: 802–803.
9. Keiser J, Utzinger J, Tanner M, Singer BH (2004) Representation of authors
and editors from countries with different human development indexes in the
leading literature on tropical medicine: survey of current evidence. BMJ 328:
1229–1232.
10. Murray CJL, Lopez AD (1996) The global burden of disease: a comprehensive
assessment of mortality and disability from diseases, injuries, and risk factors in
1990 and projected to 2020. Cambridge, Mass.: Published by the Harvard
School of Public Health on behalf of the World Health Organization and the
World Bank. 43 p.
11. Perel P, Casas JP, Ortiz Z, Miranda JJ (2006) Noncommunicable Diseases and
Injuries in Latin America and the Caribbean: Time for Action. PLoS Medicine
3: e344.
12. Higgins JPT, Green S, eds (2005) Cochrane Handbook for Systematic Reviews
of Interventions 4.2.5 [updated May 2005]. In: The Cochrane Library I, 2005,
editor. Chichester, UK: John Wiley & Sons, Ltd.
13. Glanville JM, Lefebvre C, Miles JN, Camosso-Stefinovic J (2006) How to
identify randomized controlled trials in MEDLINE: ten years on. J Med Libr
Assoc 94: 130–136.
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
GBD and Trials in LatinAmerica
Acknowledgments
Our special gratitude to Dr. Luis Gabriel Cuervo for his comments to an
earlier version of this report and to Dr. Jocalyn Clark for her valuable
editorial comments.
Author Contributions
Conceived and designed the experiments: PP. Analyzed the data: JM PP
ZO. Wrote the paper: JM PP JC. Other: Interpreted data: ZO JC JM.
14. Sumathipala A, Siribaddana S, Patel V (2004) Under-representation of
developing countries in the research literature: ethical issues arising from a
survey of five leading medical journals. BMC Med Ethics 5: E5.
15. Isaakidis P, Swingler GH, Pienaar E, Volmink J, Ioannidis JP (2002) Relation
between burden of disease and randomised evidence in sub-Saharan Africa:
survey of research. BMJ 324: 702.
16. Rothwell PM (2005) External validity of randomised controlled trials: ‘‘to whom
do the results of this trial apply?’’. Lancet 365: 82–93.
17. GESICA Investigators (2005) Randomised trial of telephone intervention in
chronic heart failure: DIAL trial. BMJ 331: 425–.
18. (2006) Chagas’ disease–an epidemic that can no longer be ignored. Lancet 368:
619.
19. Kenyon S, Taylor DJ (2002) The effect of the publication of a major clinical trial
in a high impact journal on clinical practise: the ORACLE Trial experience.
BJOG 109: 1341–1343.
20. Bruzzi P (2008) Non-drug industry funded research. BMJ 336: 1–2.
21. Roberts I, Yates D, Sandercock P, Farrell B, Wasserberg J, et al. (2004) Effect of
intravenous corticosteroids on death within 14 days in 10008 adults with
clinically significant head injury (MRC CRASH trial): randomised placebo-
controlled trial. Lancet 364: 1321–1328.
22. Mehta SR, Yusuf S, Diaz R, Zhu J, Pais P, et al. (2005) Effect of glucose-insulin-
potassium infusion on mortality in patients with acute ST-segment elevation
myocardial infarction: the CREATE-ECLA randomized controlled trial. JAMA
293: 437–446.
23. Altman D, Carroli G, Duley L, Farrell B, Moodley J, et al. (2002) Do women
with pre-eclampsia, and their babies, benefit from magnesium sulphate? The
Magpie Trial: a randomised placebo-controlled trial. Lancet 359: 1877–1890.
24. Gulmezoglu AM, Pang T, Horton R, Dickersin K (2005) WHO facilitates
international collaboration in setting standards for clinical trial registration.
Lancet 365: 1829–1831.
25. Reveiz L, Delgado MB, Urrutia G, Ortiz Z, Garcia Dieguez M, et al. (2006) The
Latin American Ongoing Clinical Trial Register (LATINREC). Rev Panam
Salud Publica 19: 417–422.
26. Siegfried N, Clarke M, Volmink J (2005) Randomised controlled trials in Africa
of HIV and AIDS: descriptive study and spatial distribution. BMJ 331: 742–.
139
6 February 2008 | Volume 3 | Issue 2 | e1696
World Health Report 2012: No Health Without Research

Research Ethics Training in Peru: A Case Study

A. Roxana Lescano1*, David L. Blazes1, Silvia M. Montano1, Zoe Moran1, Cesar Naquira2, Edwin Ramirez3,
Reidar Lie4, Gregory J. Martin1, Andres G. Lescano1, Joseph R. Zunt5
1 United States Naval Medical Research Center Detachment, Lima, Peru, 2 Peruvian Ministry of Health, Instituto Nacional de Salud, Lima, Peru, 3 Hospital Nacional Dos de
Mayo, Lima, Peru, 4 Department of Clinical Bioethics, National Institutes of Health, University of Washington, Seattle, Washington, United States of America, 5 Departments
of Neurology, Global Health and Medicine, University of Washington, Seattle, Washington, United States of America

Abstract
With the rapidly increasing number of health care professionals seeking international research experience, comes an urgent
need for enhanced capacity of host country institutional review boards (IRB) to review research proposals and ensure
research activities are both ethical and relevant to the host country customs and needs. A successful combination of
distance learning, interactive courses and expert course instructors has been applied in Peru since 2004 through
collaborations between the U.S. Naval Medical Research Center Detachment, the University of Washington and the
Department of Clinical Bioethics of the National Institutes of Health to provide training in ethical conduct of research to IRB
members and researchers from Peru and other Latin American countries. All training activities were conducted under the
auspices of the Peruvian National Institute of Health (INS), Ministry of Health. To date, 927 people from 12 different Latin
American countries have participated in several of these training activities. In this article we describe our training model.

Citation: Lescano AR, Blazes DL, Montano SM, Moran Z, Naquira C, et al. (2008) Research Ethics Training in Peru: A Case Study. PLoS ONE 3(9): e3274. doi:10.1371/
journal.pone.0003274
Editor: Cesar Augusto Ugarte-Gil, Instituto de Medicina Tropical Alexander Von Humboldt, Peru
Received April 25, 2008; Accepted August 15, 2008; Published September 26, 2008
This is an open-access article distributed under the terms of the Creative Commons Public Domain declaration which stipulates that, once placed in the public
domain, this work may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose.
Funding: This work was partially supported by work unit number No. 847705 82000 25GB B0016 and by Grants number R21NS048838, AI27757 and
1D43TW007393-01, Title: Fogarty International Center Grant, AIDS Research and (CFAR) Grant and Fogarty Global Infectious Diseases Training.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: Roxana.Lescano@med.navy.mil

Introduction
Interest in global health research and training is rapidly
increasing, especially among medical students. [1] As more U.S.
medical centers strive to provide international research and
clinical experiences, the requirement for certified international
institutional review boards (IRB) in the research arena has also
increased. In addition, large pharmaceutical companies, conduct
29% to 70% of their clinical trials in developing countries,
although clear figures are unavailable. [2] Transjurisdictional
research requires the approval of both originating country and the
country of operation. IRBs or Ethics Committees in developed
countries often have little grasp of the conditions for ethical review
in other and particularly developing countries. Additionally, there
is concern in developed countries that research, particularly
industry sponsored, is migrating ‘off shore’ due to lower costs, but
more particularly, less burdensome regulatory environments.
To ensure international research protects the rights and welfare
of human subjects, the Office of Human Research Protection
(OHRP) of the U.S. Health and Human Services requires all
federally-sponsored research conducted on human subjects at
international sites have approval by an IRB holding an OHRP
Federal wide Assurance (FWA).
Each institution with a FWA is responsible for ensuring
investigators conducting HHS-supported human subjects research
understand and act in accordance with the requirements of the
HHS regulations for the protection of human subjects. Therefore,
as stated in the Terms of the FWA, OHRP strongly recommends
that institutions and their designated IRBs establish training and
oversight mechanisms to ensure investigators maintain continuing
140
PLoS ONE | www.plosone.org
knowledge of, and comply with relevant ethical principles and
federal regulations, written IRB procedures, OHRP guidance,
state, local laws and international laws, and institutional policies
for the protection of human subjects. In addition, OHRP
recommends investigators complete institutional educational
training before conducting human subjects research; in some
instances, such as for the National Institute of Health, training is
mandatory for all key personnel conducting NIH-sponsored
human subjects research. In addition to the ethical aspects of
clinical research, other areas of equal importance include
requirements for authorship and dissemination of research results.
One of the conclusions of the Ethics of Research Related to
Healthcare in the Developing Countries, specifies that ‘there is an
urgent need to further education and training to ensure that those
[researchers] in developing countries are able to discuss ethical
issues effectively with external sponsors and others and to have
mechanisms in place to deal with issues that arise. [3]
Strengthening bioethics training for both young and seasoned
researchers in Latin America is a vital need, particularly as
foreign-funded research conducted in this part of the world
continues to increase. Training resources in human research
protection are available over the internet and several of them are
free of charge. The Collaborative Institutional Training Initiative
(CITI) modules offer one of the most complete programs, ranging
from Basic Aspects of Human Subject Protection, Good Clinical
Practices, Responsible Conduct of Research among many others.
Many of their modules have been translated to Spanish and
adapted to local practices. The Collaborative Institutional
Training Initiative (CITI) modules were first introduced in Peru
at the 2007 Conference in Lima and were very well received by
iMedPub :: Thematic Collections :: Volume 2
1 September 2008 | Volume 3 | Issue 9 | e3274

the audience. Between May and August 2007, 804 individuals had
requested access to the Spanish version of the Basic course
module, 90% of whom were Peruvian, The Office of Human
Research Protection (OHRP), the National Cancer Institute
(NCI), and Family Health International (FHI) also have human
subject protection training modules geared to investigators and/or
IRB members.. These resources are especially useful to existing
research programs in the United States, but may not be as relevant
for scientific communities in the developing world in the absence
of structured institutional Human Research Protection Programs
and lack the one-on-one approach. The Helsinki Declaration
issued by the World Medical Association and the International
Ethical Guidelines for Biomedical Research published by the
International Organizations of Medical Sciences (CIOMS) are
essential reference documents for the IRB community, as well, and
are discussed fully in several of these training events.
Although there is general agreement among investigators that
training in ethical aspects of research is essential to conducting
good and ethical studies, traditionally, clinical research grants have
not provided funds for human research protection training. In
addition, government and academic institutions at international
sites always have limited discretionary funds for this type of
training. Live courses and workshops led by experts in the field of
research ethics are often prohibitively costly or available in limited
geographic areas, but are especially valuable for encouraging one-
on-one interaction with other investigators and opportunities for
trainees to learn from case-based discussions or clarify areas of
uncertainty–which are common in international biomedical ethics.
Recently international and national agencies such as the World
Health Organization, the European Commission, the US National
Institutes of Health, and the Wellcome Trust have shown an
interest in addressing this concern and have funded initiatives for
training programs and capacity building regional and national
workshops. Here we will report on our experience in developing a
series of live courses and workshops that could provide useful
information for these newly developed programs.
Materials and Methods
The Training Model
The US Naval Medical Research Center Detachment Peru
(NMRCD-Peru) is one of five overseas US military infectious
diseases laboratories and the only one located in the Americas. The
central geographic location in Peru has made it an easy-to-access
country for South American colleagues who desire to participate in
training activities. Through collaboration with the University of
Washington, the NIH Department of Clinical Bioethics, the
Peruvian National Institute of Health, the Peruvian Institutional
Review Board Network and local Peruvian universities, NMRCD
became the center for bioethics research training for participants
from across Latin America from 2004 through 2007. Although
universities provide the ideal location and resources for training
activities, good networking, the support of domestic and interna-
tional government and private institutions and a strong commit-
ment from the Peruvian IRB Network made it possible for a series of
training efforts to be provided by NMRCD in Peru.
The training model followed by NMRCD combined distance
learning, interactive teaching and high level expert teaching in
workshops, courses, conferences, webminars and videoconferences as
the key element for success. Courses and conferences involved
didactic sessions and mock IRB discussions conducted by experts
from the U.S, and from Peru to provide a more relevant approach.
Topics included conducting ethical research, informed consent,
placebo versus standard of care, research with children, authorship,
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
Research Ethics Training
feedback to research subjects, repository and tissue sample banks and
international collaborative research. Each participant completed a
test at the end of the event and received a certificate of attendance.
Participants included IRB Chairs and members, researchers, persons
directly involved in clinical research and professionals directing offices
in academic or research organizations. All courses and conferences
were co-sponsored by the Peruvian Medical Board Association and
local Universities and continuing medical education credits were
awarded to participants who completed the post-test.
The most popular training courses have been the Conferences
on Ethical Aspects of International Collaborative Research held in
Lima, and Iquitos, a city in the Peruvian Amazon region; and the
Conference in Ethics in Collaborative International Research:
Practical Issues and Constructive Tools for Latin American
Research Teams held in Lima and Arequipa, a city in the
Southern Andes. The satellite conferences gathered approximately
80 students each, while the Lima-based conferences had nearly
200 participants each. The lectures presented various topics of
crucial importance to ethics in research, such as coercion, undue
inducement, exploitation, informed consent, research with vulner-
able populations, placebo versus standard of care, research with
children, criteria for authorship, feedback to the research subjects,
repository and tissue sample banks and international collaborative
research. To better illustrate topics and make them more relevant
to South America, faculty incorporated results and case-studies
from research conducted in Peru and other parts of the world.
Faculty members were well-published speakers from the U.S.
OHRP and NIH Department of Clinical Bioethics, the Univer-
sities of Washington and Texas and Peruvian academic and
regulatory entities and IRBs. The events were organized in
collaboration with Peruvian organizations, such as the Universi-
dad Peruana Cayetano Heredia, Universidad Nacional Mayor de
San Marcos, the Peruvian Medical Association, the Peruvian
Ministry of Health, the Peruvian National Institute of Health. The
University of Washington provided funding and coordination for
the Ethics in Research Courses conducted in Lima and Iquitos in
2005 and the Lima Conference on Ethical Aspects of International
Collaborative Research and course in Arequipa in the 2007. The
U.S. NIH Department of Clinical Bioethics played a crucial role in
providing training through the 2004–2007 video conference
course entitled NIH Ethics and Regulatory Practices in Clinical
Research, the three Latin American Conferences on Ethical
Aspects of International Collaborative Research held in Lima in
2005, 2006 and 2007, and the Ethics in Research course held in
Iquitos in 2006. The videoconference course includes 7 sessions
and is simulcast to multiple sites throughout the U.S., Peru,
Mexico and Puerto Rico.
Since 2004, other training activities have included: webcast
broadcasting on ethics in international clinical trials and the
Anniversary of the Belmont Report; and two workshops for IRB
Administrators (Figure 1).
Results
A total of 927 (258 of whom were IRB members) from 12
different countries in the Americas participated in training courses
between 2004 and 2007 (Figure 2). Of the 927 participants, 836
were Peruvian and 510 of these were Peruvian MoH staff. Forty-
nine percent (49%) of the participants were women. Suggestions
received from the students encouraged the organization of more
courses and post-test results demonstrated recognition of basic
concepts of ethics in research.
A total of 137 of the 804 Peruvians who registered to take the
CITI online modules completed the CITI course entitled Basic
141
2 September 2008 | Volume 3 | Issue 9 | e3274
World Health Report 2012: No Health Without Research
Figure 1. Types of training events held in Peru, 2004–2007.
doi:10.1371/journal.pone.0003274.g001
Aspects in the Protection of Human Research Subjects within the
first 10 months.
The NIH Ethics and Regulatory Practices in Clinical Research
video conference course has been so successful in Peru that since
2005, two additional sites in Lima were registered directly with the
NIH to broadcast the course sessions from their own facilities.
Participants in the videoconference series maintained a 70–80%
attendance rate and anonymous surveys showed high satisfaction
with the overall course (Figure 3). A total of 95 Peruvians received
NIH certificates of participation from 2004 through 2007.
Discussion
The developing world needs both, bioethics training and IRB
capacity reinforcement to ensure research conducted in each
country is compliant with international standards, while at the
same time, sensitive to the needs of the local populations. Our
Figure 2. Courses and participants, 2004–2007.
doi:10.1371/journal.pone.0003274.g002
142
PLoS ONE | www.plosone.org
Research Ethics Training
collaborative bioethics conferences co-hosted by Peruvian aca-
demic and governmental organizations, U.S. NMRCD, and U.S.
based government and academic institutions provided a unique
opportunity for fostering adherence to ethical standards of
research in this region. In Peru, this partnership has been
extremely fruitful, with the participation of over 927 professionals
from 12 different Latin American countries.
We believe inclusion of expert speakers, a diverse curriculum
and the investment and commitment of local partners made our
conferences successful. Involvement of Peruvian members in both
the presentations and mock IRBs promoted the inclusion of topics
relevant to the developing world and fostered greater understand-
ing between investigators and IRB members from developing and
developed countries. Although conference training lacked an
applied, practical component, many of the participants had
extensive experience already and benefited from reinforcement
of theoretical concepts and examples from research conducted in
other parts of the world. This training model can be easily
reproducible by other developing world countries.
Training courses on bioethics are essential for encouraging
acknowledgement and understanding of the importance of ethical
conduct among persons conducting clinical research in the
developing world. In addition, these courses strengthen the
capabilities of IRB members and encourage better functioning of
existing IRBs and the creation of new ones. Our courses and
conferences were perceived as very useful by the Latin American
scientific community, with a-growing number of attendees
registering for these events and requesting additional training
opportunities. We believe the next step is to target more advanced
individuals, such as IRB chairs and members and develop
intensive site evaluations to assist with setting up systems for
record-keeping, IRB activity monitoring, tracking modifications
iMedPub :: Thematic Collections :: Volume 2
3 September 2008 | Volume 3 | Issue 9 | e3274

Figure 3. Training activities conducted.
doi:10.1371/journal.pone.0003274.g003
and continual renewals, as well as encouraging local hosting of
additional training courses.
Although our courses and conferences have received positive
evaluations from the participants, we recognize that such
evaluations provide limited evidence of the usefulness of the
training program. Ideally, we would like to know if the training
program contributes to better research ethics review and
ultimately to better protection of research participants. We are
not aware of any formal evaluations of training programs using
such criteria, and it would be almost impossible to do such an
evaluation in a rigorous manner. One could, however, measure
the level of knowledge and understanding of ethical principles and
human subjects regulations before and after a series of training
courses. Again, there are no standardized instruments for such
evaluations available right now. Given the increasing interest in
funding training workshops by international agencies, we believe
that the development of such an instrument should be of high
priority.
References
1. Drain PK, Primack A, Hunt DD, Fawzi WW, Holmes KK et al (2007) Global
health in medical education: a call for more training and opportunities. Acad
Med 82: 226–230.
iMedPub :: Thematic collections :: Volume 2
PLoS ONE | www.plosone.org
World Health Report 2012: No Health Without Research
Research Ethics Training
Acknowledgments
Disclaimer:
The views expressed in this article are those of the author and do not
necessarily reflect the official policy or position of the Department of the
Navy, Department of Defense, nor the U.S. Government.
Copyright statement
Some of the authors are military service members and employees of the
U.S. Government. This work was prepared as part of their official duties.
Title 17 U.S.C.
1 105 provides that ‘Copyright protection under this title is not available
for any work of the United States Government’. Title 17 U.S.C. 1 101
defines a U.S. Government work as a work prepared by a military service
member or employee of the U.S. Government as part of that person’s
official duties.
Author Contributions
Conceived and designed the experiments: RL AGL. Performed the
experiments: RL ZM RL JRZ. Analyzed the data: RL DLB ZM RL GM
AGL. Contributed reagents/materials/analysis tools: DLB SMMT CN ER
RL GM JRZ. Wrote the paper: RL DLB GM AGL JRZ.
2. WEMOS 2007. (2007) A bitter pill. The risk of carrying out clinical trials in the
developing world. Available: http://www.wemos.nl. Accessed 19 Mar 08.
3. Nuffield Council on Bioethics (2005).
143
4 September 2008 | Volume 3 | Issue 9 | e3274
 ABOUT THE PUBLISHER

iMedPub is a new approach to scientific publishing.As an open service to scientists, it

is driven by researchers for researchers, while serving the interests of the general public.
Therefore, we are a collaborative publishing house. iMedPub disseminates research in a
tiered system, beginning with our specialty journals and working upwards.

Vision

The grand vision of iMedPub is a world where all people have an equal opportunity
to seek, share and create knowledge. To help actualize our grand vision, iMedPub
provides open and free access to all of its publications. iMedPub is bilingual English-
Spanish and we publish journals, books and websites in both languages.

Mission

iMedPub aims to uphold the rights of authors, address their needs, and foster a
rapid, convenient, unbiased, and comprehensive publishing environment, which not only
guarantees the highest quality constructive peer-review process, but also provides an
evaluation system that involves the entire research community.
To fulfill this mission, iMedPub applies the most advanced Internet technologies to
bring scholarly publishing into a new generation.

Dedication to Quality

Each article published by iMedPub, is a landmark of the highest quality, due to true
collaborative interactions between authors and the highest quality reviewers. iMedPub
supports only the highest quality reviews, and operates according to an unbiased
review system.
Other titles in these series:

• Social Medicine in the 21st Century

 The World Health Report 2012, the biannual flagship report of the

World Health Organization, focuses for the first time in its history

on the theme of research for better health. Decisions on healthcare

are still made without a solid grounding in research evidence, and

an impetus is required for this state of affairs to change. Aimed

at ministers of health, the report provides new ideas, innovative

thinking, and pragmatic advice on how to strengthen health

research systems.

WHO and PLoS have launched an initiative to encourage researchers

to complement and substantiate the key messages in World Health

Report 2012 by creating a special WHO/PLoS Collection. PLoS invited

the submission of papers, especially from low- and middle-income

countries, on topics related to strengthening of key functions and

components of national health research systems.

The World Health Report 2012 focuses on eight specific areas,

discussed in the editorial, within the theme of ‘No Health Without

Research’.

Now iMedPub brings this collection to you within a book.