Evidenced Based Dentistry

Variation, Certainty, Evidence, and Change in Dental Education: Employing Evidence-Based Dentistry in Dental Education
Valria Coelho Cato Marinho,______; Derek Richards,_____; Richard Niederman,____
Abstract: Variation in health care, and more particularly in dental care, was recently chronicled in a Readers Digest investigative report. The conclusions of this report are consistent with sound scientific studies conducted in various areas of health care, including dental care, which demonstrate substantial variation in the care provided to patients. This variation in care parallels the certainty with which clinicians and faculty members often articulate strongly held, but very different opinions. Using a casebased dental scenario, we present systematic evidence-based methods for accessing dental health care information, evaluating this information for validity and importance, and using this information to make informed curricular and clinical decisions. We also discuss barriers inhibiting these systematic approaches to evidence-based clinical decision making and methods for effectively promoting behavior change in health care professionals. Dr. Marinho is ____________, Department of Epidemiology and Public Health, University College London; Dr. Richards is _____________, Centre for Evidence-based Dentistry, Oxford University, Oxford; Dr. Niederman is _________, Office of Evidence-Based Dentistry, Harvard School of Dental Medicine and ___________, Office of Evidence-Based Dentistry, Forsyth Institute, Boston. Direct correspondence to Dr. Richard Niederman, 188 Longwood Ave, Boston, MA 02115; Rniederman@hms.harvard.edu e-mail. Key words: evidenced based dentistry, curriculur decision making Submitted for publication 2/7/01, accepted for publication 4/17/01

ariation in health care is chronicled yearly by the Dartmouth Atlas of Healthcare (www.dartmouthatlas.org/default.php). Its evidence indicates that extraordinary variations in care and outcomes are associated with three factors: 1) poor quality of science underlying clinical care; 2) poor quality of clinical decision making; and 3) variations in clinical skill. These factors may also account, in part, for the recent report in Readers Digest1 highlighting the variability of treatment plans and the highly regarded series of scientific studies demonstrating significant variability in dental diagnosis and treatment.2 In an Internet-linked, twenty-four-hour-per-day, seven-day-per-week, 365-day-per-year information society, in which clinical information and technology are continuously increasing, life-long learning is a challenge for health care students, faculty members, and clinicians. Learning involves identifying and evaluating new methods that might improve care and prognosis, determining when to implement those that appear to improve care, and discarding old diagnostics and therapeutics that prove to be unsound. The combination of these elements with ones own

inclination to change is another component of clinical variability. With this information river constantly flowing, it is understandable why variation in care can be anticipated to increase. One quantitative example is the recent estimate from the restorative dental literature.3 This work indicates that more than 400 high-quality human clinical trials per year appear in more than fifty journals. Thus to stay current, one would need to identify, peruse, read, and evaluate more than one article per day 365 days per year. Interestingly, care variability is most clearly observable on the clinical teaching floor. There, students routinely note the certainty with which instructors provide apparently antithetical clinical opinions. Conversely, clinical teaching leaders are continually looking for better ways to calibrate their faculty. Or, lacking faculty calibration, the clinical departments celebrate the breadth of experiences their students are exposed to from faculty with divergent clinical philosophies (one wonders whether an educated patient would celebrate this diversity of opinion equally). Meanwhile, patients want clinicians who

May 2001

Journal of Dental Education

449

know, rather than believe, that a prescribed treatment works best. Variability has another context: health care disparities. Health care disparities and access to health care are prominent issues, as clearly indicated in the Surgeon General s Report on Oral Health (www.nidcr.nih.gov/sgr/oralhealth.htm). Governments, health authorities, and insurers are seeking sound data for making informed decisions about resource allocation during an era in which health care costs are escalating faster than the consumer price index. One method for identifying, evaluating, and implementing good clinical data in a timely fashion is evidence-based health carethe integration of the best clinical evidence with the clinicians experience and the patients needs. The Centre for EvidenceBased Medicine at Oxford University provides methods for this approach (cebm.jr2.ox.ac.uk/docs/ glossary.html). In this article we present a clinical scenario, occurring on the teaching clinic floor, that generates a controversy in therapeutic approach. Then, using evidence-based methods, we demonstrate how one might consider incorporating these methods in solving this clinical dilemma. We also discuss methods for effectively promoting behavior change among health care professionals.

Scenario
Following an initial examination at Dental School Universitys oral diagnosis clinic, a twelveyear-old boy, Fredrick Lopez, is referred to the thirdyear comprehensive dental care clinic. After introductions, the third-year dental student, Melanie Tsai, asks what brings Fredrick to the clinic. Fredricks mother, Mary, indicates that Fredrick has some brown spots on his back teeth. On further questioning, Melanie finds that this is Fredricks first dental appointment, primarily because Mary is a single working parent, has a limited income, and cannot afford expensive dental care. Melanie completes a history and identifies no other significant findings and no contraindications to treatment. The bite wing radiographs do not indicate any caries. However, Melanie identifies the brown spots on the occlusal surfaces of both the maxillary and mandibular first and second molars. The third molars have not erupted. Using an explorer, Melanie finds what may be sticking in the brown

areas of the fissures. She isnt sure, however, whether they are carious lesions. Melanie calls over Dr. Janson, one of the operative instructors, to review her findings. Dr. Janson indicates that the brown spots are carious lesions and that they need restorations. However, given Fredricks age, Dr. Janson recommends that Dr. Lynn, a pediatric dentist be consulted too. When Dr. Lynn examines Fredrick, she disagrees with Dr. Jansons assessment. Dr. Lynn says the spots are not carious lesions, but rather deep fissures with staining. Thus what Fredrick needs are either sealants or fluoride varnish, not restorations. Dr. Janson and Dr. Lynn proceed to discuss the relative risks and benefits of each treatment, while Melanie and Fredrick, and his mother listen. Before departing, the instructors ask Melanie to complete her exam, prepare a treatment plan, and talk to them after clinic. After they leave, Mary Lopez asks Melanie: Whats best for Fredrick? Melanie replies that there are several options to consider and that she will prepare a treatment plan and call Mrs. Lopez after she consults with her instructors. This scenario highlights two common teaching issues. First, among practicing clinicians, there are significant variations in clinical diagnosis and treatment recommendations.2 Second is the question of treatment effectiveness: What s best for Fredrick? It is the second question that evidence-based dentistry can address. Out of a resolution of the evidence-based question can evolve an answer to the first question on variation in care. In the following paragraphs, the four-step method of evidence-based health care is applied to answer the clinical question.

Evidence-Based Health Care

1. Articulate an answerable question.
Carefully articulating an answerable question provides two benefits: it ensures that everyone understands the issue, and it foreshadows step two searching for the best evidence. For clarity, clinical questions are normally framed in terms of the problem (P), intervention or exposure (I/E), comparison (C), and outcome (O) (Centre for Evidence-Based Medicine cebm.jr2.ox.ac.uk/docs/focusquest.html).
Journal of Dental Education I Volume 65, No. 5

450

The same PICO/PECO format can be used for clinical questions of diagnosis (caries diagnosis), prognosis, therapy (e.g., sealants vs. fillings), harm, or quality of care. In the scenario there are several questions. The first is the diagnosis of caries. The second is the restoration of caries versus the sealing of caries. The third is sealing versus remineralization. For this scenario weve elected to examine the third question (Table 1). However, the same method can be used to examine the first two questions.

Table 2. Search strategy

Step 1 2 3 4 5 6 7 Search Concept Exp Dental Caries Pit and Fissure Sealants Exp Fluorides, Topical 1 and 2 and 3 Limit 4 to (human and English language) Limit 5 to (meta analysis or randomized controlled trial) From 6 keep 1, 3, 4 Result 22435 1835 6727 141 98 12 3

2. Search for the best evidence.

a. MEDLINE searching. Conducting an efficient search of the literature in electronic databases to find the best evidence is the second step. There are several complications to this and several solutions. The first complication is the selection of a database. Three widely used medical bibliographic databases are available: MEDLINE (largely U.S. English language), EMBASE (largely European), and LILACS (largely Spanish language). MEDLINE remains the standard English-language bibliographic database for biomedical information with free Internet access from the National Library of Medicine (www.ncbi.nlm.nih.gov/PubMed/). EMBASE is a proprietary database owned by Elsevier with access on a subscription or fee basis. LILACS (Literatura Latino-Americana e do Caribe em Cincias da Sade) is a database of Latin American and Caribbean Health professionals (www.epm.br/ cochrane/lilacs.htm). We elected to use MEDLINE as the database and, using the PICO formatted question (Table 1), constructed a search strategy using each of the key concepts (caries, pit and fissure sealants, and topical fluorides) (Table 2). The second issue is the type of research and the level of evidence that needs to be considered. A large proportion of the biomedical literature, for example, focuses on laboratory or animal research. This literature, while applicable by extrapolation, lacks immediate clinical applicability to humans. To overcome this problem, limits can be placed on the identified
Table 1. Asking evidence-based questions
Concept Format Clinical Question Problem (P) In patients with P, In patients with possible caries, Intervention (I) would I, would sealants,

articles so that the search screens for articles that address only humans (Table 2, step 5). Further, a number of articles appear in foreign language journals. To facilitate reading by English-speaking clinicians, limits can be placed so that only those articles in English are identified (Table 2, step 5). Finally, the highest levels of evidence are preferred over lower levels of evidence (Table 3, Centre for EvidenceBased Medicine cebm.jr2.ox.ac.uk/docs/levels.html). High levels of evidence (e.g., randomized controlled trials or systematic reviews with meta-analyses) can be efficiently located in electronic databases, such as MEDLINE (Table 2, step 6). Implementing the PICO question on sealants compared to varnishes in a MEDLINE search with the indicated limits generated three relevant citations (among a total of twelve possible), all reporting on one study that is a randomized controlled trial of sealants versus varnish on molar teeth of 314 children.4,5,6 b. Other resources. Among many, six other, primary and so-called secondary, evidence-based information sources are of key interest. These sources include: 1) The Cochrane Library (www.cochrane.org; www.update-software.com/cochrane/cochraneframe.html). The Cochrane Library contains both the Database of Systematic Reviews (CDSR) and the Cochrane Controlled Trials Register (CCTR). The database offers full text, highly structured, systematic, health care reviews, with explicit inclusion and exclusion criteria, quality assessments, and meta-analysis. Included in the Cochrane Library is the Cochrane Oral Health Group (www.cochrane-oral.man.ac.uk/ default.htm), with three reviews completed and

Comparison (C) when compared to C, when compared to fluoride varnish,

Outcome (O) result in O? result in fewer caries?

May 2001

Journal of Dental Education

451

Table 3. Levels of evidence

Level of Evidence 1 2 3 4 5 Study Type Systematic review of randomized controlled trials, and randomized controlled trials Systematic review of cohort studies, and cohort studies Systematic review of case-control studies, and case-control studies Case series Expert opinion

5) The University of Sheffield School of Health and Related Research (ScHARR, www.shef.ac.uk/~scharr/ir/netting/). ScHARR offers Internet resources for finding information (netting the evidence). 6) The Centre for Evidence-Based Dentistry (www.ihs.ox.ac.uk/cebd/). The Centre for Evidence-Based Dentistry is a central reference source for all topics related to evidence-based dentistry. It provides training in evidence-based dentistry and is the site for the editorial office for the evidence-based dentistry journal.

more than thirty in various stages of progress w w w. u p d a t e - s o f t wa r e . c o m / a b s t r a c t s / g080index.htm). The Trials Register is the result of hand searching the worlds health care journals and contains a database of more than 8,000 randomized controlled human dental trials. 2) The York University Centre for Reviews and Dissemination (agatha.york.ac.uk/welcome.htm). York University Centre for Reviews and Dissemination contains three databases and one publication of dental interest. The Database of Abstracts of Reviews of Effectiveness (DARE) is a database of high-quality systematic research reviews on the effectiveness of health care interventions. It contains approximately sixty dental-related systematic reviews. National Health Service (NHS) Economic Evaluation Database (NHS EED) is a database of structured abstracts on economic evaluations of health care interventions. It contains approximately fifty dental-related evaluations. The Health Technology Assessment (HTA) database contains abstracts of technology assessments, including approximately fifteen dental-related abstracts. Effective Healthcare Bulletins published bimonthly by the University of York NHS Centre for Reviews and Dissemination currently contains a review of dental restorations (www.york.ac.uk/inst/crd/ ehc52.htm). 3) Bandolier (www.jr2.ox.ac.uk/Bandolier/). Bandolier is a print and Internet journal about health care that uses evidence-based methods to distill bullet points from primary health care journals for use by clinicians and consumers. It currently has approximately six dental-related topics, including a significant review of pain abatement. 4) The Centre for Evidence-Based Medicine at Oxford (cebm.jr2.ox.ac.uk/). The Centre for Evidence-Based Medicine provides tools and training for learning, practicing, and teaching evidence-based health care.
452

3. Critically appraise the evidence.

Once the best evidence has been identified, three questions are asked of all papers: Is the study valid? Are the results clinically important? and Are the results relevant to my patients? Standardized worksheets are the easiest way to calibrate this process (see Centre for Evidence-Based Medicine at cebm.jr2.ox.ac.uk/docs/teachingresources.html). The first question on validity concerns the articles closeness to the truth or unbiased results. Review of the studies identified in our search4,5,6 confirmed that fifteen classes of children were assigned at random to three independent groups (sealant group, varnish group, and no treatment group). The dentist examining the children was unaware of group assignment (blinded), no exclusions were made during the follow-up (and natural losses were reported), and the groups were similar at the start of the trial. Considering the question of clinical importance next, the study results indicated a 48 percent risk reduction (RR) in occlusal caries developing in first molars after two years with sealants compared to fluoride varnishes. RR = average DMFS change in sealant group average DMFS change in varnish group x 100 average DMFS change in varnish group Similarly, there was 68 percent risk reduction in occlusal caries in sealants compared to controls. A 38 percent risk reduction in occlusal caries was also noted with fluoride varnishes when compared to controls. Note, however, that this was a selected group of patients in that sealants or varnishes were being applied at six-month intervals in children with low exposure to fluoride and with low to moderate caries levels.

Journal of Dental Education I Volume 65, No. 5

This randomized controlled trial demonstrates the benefits of varnish over control and of sealants over varnish in first molar fissures over twenty-four months. Even more interesting, the results were better for smooth surface caries, with a reduction of 87 percent for sealants compared to controls and 66 percent for varnish compared with controls. Similar results were reported after forty-eight months. The question is: is 38 percent or 68 percent clinically significant? Normally, the question to be asked is 68 percent of what? Thus, if an individual has a low caries incidence (e.g., one caries in two years), a 68 percent reduction yields about one caries in six years. Conversely, for an individual with a high caries risk (e.g., two caries in one year), a 68 percent reduction yields two caries in three years. The question becomes: is this benefit worth the cost of sealing all the posterior teeth over three years? Finally, answering the question Is this information relevant to my patient? depends heavily on clinical judgment. Put most simply, how similar (or different) is my patient or my patients problem from those in the study? In this case scenario, is the cause of caries in the seven- to eight-year-olds similar to that found in twelve-year-olds, and do these causes affect first molars differently from second molars? Is the cause of caries different in seven- to eightyear-olds and twelve-year-olds? Is the effect of prevention and treatment the same or different?

Discussion
In reality, most clinical care and policy decisions are complex, time-dependent, and based on clinical experience, patient/population needs, and evidence with varying degrees of certainty. The evidence-based approach clarifies the level of certainty underlying a portion of these complex decisions that occurring in the clinical literature. Further, because it starts with the health problems faced by patients, rather than with the existing evidence that researchers have chosen to generate, this inevitably means that sometimes the evidence available may be far from adequate. It will, however, be the best available. Implementing evidence-based health care is also a daunting task for busy students, teaching faculty, and clinicians to undertake for all decisions. These methods can, however, facilitate discussions about the certainty of clinical decision-making, inform decisions around curriculum development, identify important evidence gaps that can drive research funding, and ultimately support the implementation of the best evidence in practice. There are, however, a number of steps and barriers to implementing evidence-based clinical practice (Figure 1).7 This diagram, in the form of a funnel, represents a clear contrast of interests. At the top of the funnel, investigators and funding agencies (governments, product companies, and pharmaceutical developers) determine areas of research interest and investment for clinical trials. At the bottom of the funnel, the circles represent the patient and clinician who present with and solve actual clinical problems, respectively. Similar dichotomies of interest occur at each level, and the stakeholders at each level of the funnel may have entirely different needs and interests. As a result, the application of these different stakeholders needs may harmonize or conflict with one another. Further, they may harmonize or conflict with the patients perception of the problem, the evidence, and/or the clinicians preferred solution. Thus while the evidence may be sound, the certainty of an outcome may be in question. Further complicating the flow of information through the funnel, in addition to the personal preferences of individuals, are access to information and willingness to change behaviors. The major portion of this manuscript addressed access to information. Here we will briefly address the willingness of clinicians to change behaviors. Classic studies indicate

4. Apply the evidence

Having identified evidence that is valid, clinically important, and relevant, one now has a basis on which decisions can be made, after considering other issues related to each particular situation. In our example, the evidence came from well-conducted randomized controlled trials comparing the effectiveness of sealants with varnishes.4,5,6 All of the studies came from the same investigators. This is the best currently available evidence addressing our particular question. Other investigators using other patients in other countries have not addressed this question. So, how certain does one need to be? Some dental practitioners and decision-makers may choose to be cautious about adopting the findings from just one series of studies. They may prefer to wait for further evidence. Others may accept the reliability of the beneficial evidence and decide to proceed and use sealants and/or varnishes instead of fillings. Finally, other issues arise, not the least of which is patient preferences (e.g., chief complaint, time, money).

May 2001

Journal of Dental Education

453

Generate Clinical Evidence e.g., clinical trials Synthesize Evidence e.g., systematic reviews

Develop Policy e.g., guidelines Apply Policy

Clinical Problem

Patient Preference

Clinician Preference

Measure Outcomes
Figure 1. Pathways and barriers to evidence-based health care implementation
In this figure each trapezoid represents a portion of a funnel that takes a larger volume of information and distills it for the next step. The gaps between the trapezoids represent the barriers to implementation. The circles represent the multiple stakeholders, and thus the complexity of implementing clinical policy or guidelines and measuring outcomes in clinical practice. Source: Haynes B, Haines A. Getting research findings into practice: barriers and bridges to evidence based clinical practice. Brit Med J 1998;317:273-6.

that clinicians, even when provided with sound information, can take fifteen years to substantially change the way they practice. 9 The Cochrane Collaborations (www.cochrane.org/) Effective Practice and Organization of Care Groups focus is the systematic review of trials examining clinician behavior. A recent review, highlighted in the BMJ,8 identifies mechanisms that tend to be effective and ineffective in altering clinicians behavior (Table 4). Interestingly, even the interventions identified as

consistently effective produce change in clinician behavior less than 50 percent of the time. The most effective methods are those used by the pharmaceutical industry, called detailing, in which representatives visit doctors offices to inform them of new drugs and offer samples. Also effective are the combinations of two or more methods of variable effectiveness. For example, an audit and feedback of a clinicians practice habits or a local consensus may have little effect on altering behavior. However, when

454

Journal of Dental Education I Volume 65, No. 5

Table 4. Promoting behavioral changes of health care professionals

Effectiveness Consistently effective Intervention Educational outreach for prescribing Reminders (manual or computerized) Two or more of the following: audit and feedback, reminders, local consensus, marketing Interactive educational meetings Audit and feedback (e.g., summary of clinical performance) Local opinion leaders Local consensus processes Patient mediated interventions Educational materials Didactic educational meetings

Variable effectiveness

Little effect Source: Bero et al. (1998)

combined, they can have a significant effect. Most interesting, perhaps, is the finding that, in spite of the massive amount of lecture-based continuing education programs and printed continuing education materials distributed to practitioners, these two techniques have little effect in altering behavior. All of this suggests a significant need for additional studies that measure behavioral changes in health care providers.9,10 In conclusion, the needed transitional elements required to evolve from the individualized educational and decision methods we currently use to a more evidence-based approach include: 1) evidencebased health care training programs for pre- and postdoctoral students and faculty; 2) faculty and students who are willing to embrace change; 3) chair-side access to systematic reviews of the evidence; and 4) trained faculty who are willing to implement new effective health care methods and discard established treatments with poor effectiveness. Most important are administrators and alumni who understand, appreciate, and are willing to support and invest in these opportunities. In sum, evidence-based health care has the potential to improve health care by providing mechanisms for transforming the teaching and practice of oral health care professionals as they continue to face an exploding volume of literature, rapid introduction of new technologies, deepening concern about health care disparities, and increasing attention to the quality and outcomes of oral health care.

REFERENCES
1. Echenbarger W. How honest are dentists? Readers Digest 1997; Feb: 50-56. 2. Bader JD, Shugars DA. What do we know about how dentists make caries-related treatment decisions? Comm Dent Oral Epidem1997;25:97-103. 3. Nishimura K, Rasool F, Ferguson MB, Sobel M, Niederman R. Benchmarking the clinical prosthetic dental literature on MEDLINE. J Prosth (in press) 2001. 4. Bravo M. Baca P. Llodra JC. Osorio E. A 24-month study comparing sealant and fluoride varnish in caries reduction on different permanent first molar surfaces. J Public Health Dent 1997;57:184-6. 5. Bravo M, Garcia-Anllo I, Baca P, Llodra JC. A 48-month survival analysis comparing sealant (Delton) with fluoride varnish (Duraphat) in 6- to 8-year-old children. Comm Dent Oral Epidem 1997;25:247-50. 6. Bravo M, Llodra JC, Baca P, Osorio E. Effectiveness of visible light fissure sealant (Delton) versus fluoride varnish (Duraphat): 24-month clinical trial. Comm Dent Oral Epidem 1996;24:42-6. 7. Haynes B, Haines A. Getting research findings into practice: barriers and bridges to evidence based clinical practice. BMJ 1998;317:273-6. 8. Bero L, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson AM. Getting research findings into practice: closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. Brit Med J 1998;317:465-8. 9. Davis O, OBrien MA, Freemantle N, et al. Impact of formal continuing medical education: do conferences, workshops, rounds and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282:867-74. 10. Davis O. Does CME work? an analysis of the effects of educational activities on physician performance on health care outcomes. Int J Psychiatry Med 1998;28:21-39.

Acknowledgments
We thank Amid Ismail, Jim Bader, and Bill Hendricson for their very helpful comments on the various versions of this manuscript.

May 2001

Journal of Dental Education

455