Laying It Table

A look at the state of Cupping

by Andi C. Trindle MersCh

on the

s I said in the first half of this two-part state of the coffee industry series, when youve been doing something for a while, you become compelled to step back and take a

look to make sure youre doing something worthwhile and that youre doing it the right way. This sentiment is especially true if youre a perfectionist. Although I have grudgingly abandoned the idea of achieving self-perfection, I seem to still expect it of systems: iPhones, computers, cable TVand cupping. Coffee cuppingas our industrys formal and, thus far, only universally agreed-upon sensory qualityassessment tool that has very real financial implicationsis a system that should be perfect. Right? By all accounts the industry has come a long way over the years, but perfection in cupping is probably as likely as perfection in me. As an industry, we may need to pursue the next evolution of cupping and, at the very least, we need some reflection and realignment.
continued on page 26

photo courtesy of Font Coffee

24

roast

S e p t e m b e r | Oc t o b e r 2 0 12

25

Laying It on the Table | Cupping

(continued)

Why We Cup
Before we dig into some of the reasons for re-evaluation of the cupping process, lets look at some of the reasons why we cup in the first place. According to the Specialty Coffee Association of Americas cupping curriculum, we cup coffees for the following reasons: Purchasing Quality and price discovery Subject to Approval of Sample (SAS) applications Quality assurance (incorporating roast profiling, as well as consistency monitoring) Blending Palate enrichment Education/training These are some valuable motivationsparticularly those that influence which coffees are bought and sold and at what prices. And, as Brian Aliffi, green coffee sourcing manager at Minnesota-based Caribou Coffee, states, Cupping provides a simple tool for evaluating multiple samples in a convenient time frame and with a minimum required sample size. Without spending any more time acknowledging the value and need for cupping, we can probably all agree that we need to have a formal, efficient system for evaluating coffees. But

Is the Cupping Process Working?

Short answer: yes. More accurate answer: not wholly. As Martin Diedrich, founder of California-based Kean Coffee, says, Its so amazing to go to origin and be able to speak the same language. This simple statement reflects whats generally working with cupping. We have developed an industry language that allows us to understand each other across borders. This language of ours includes vocabulary and numbersi.e., descriptors and scoreswhich theoretically help us agree on an objective coffee quality. From here, we can make informed buying and selling decisions that allow our businesses to succeed while, in turn, securing the growth of specialty within the greater coffee industry. Getting back to the longer and more realistic answer of not wholly, however, imperfections of the cupping process are manyfrom protocols to calibration to bias.

photo by Chad Trewick

lighting, and many other factors exist to ensure that a cupping session is as scientific as possible, thereby yielding (in theory) objective results. Because coffee tasting is inherently rife with uncontrollable variables, such as the differences between tasters (taste buds, taste experience, psychology and physiology to name a few), its particularly important to identify and systematically manage these controllable variables. The SCAA, Cup of Excellence and CQI training programs, among other programs and companies in the coffee industry, have done an excellent job instilling cupping protocols around the globe in recent years. Although these organizations impart slight variation in protocols, they generally espouse similar guidelines, and all agree that consistent practices from session to session are critical. However, in general, scientific practices are not consistently followed in most cupping labs, says K.C. OKeefe, founder of Caf Verde Peru, a coffeehouse in Lima, and the chair of the SCAAs professional cupper development committee. This introduces unacceptable margin for errors in extraction and flavor development in the cup, OKeefe notes.
continued on page 28

Protocols
Protocols is the term generally adopted within our industry to identify the best practice/industry guidelines for preparing a formal cupping. Protocols for consistency in roast degree, steeping time, grind degree, water quality and temperature,

26

roast

S e p t e m b e r | Oc t o b e r 2 0 12

27

Laying It on the Table | A Look at the State of Cupping

Indeed, my own impetus for writing this article came from my frustrations owing to variant protocols in a lab setting. Earlier this year, I found myself participating in origin cuppings where some important, basic protocols werent followed. The intention to host a formal cupping was very clear, and these producers (unlike most around the globe) had appropriate equipment and the general setup for professional cupping. (Credit must go to dedicated NGO, industry partners, and government organizations around the world for devotedly dispensing cupping training and supplies in recent decades.) Nonetheless, training and/ or resources simply didnt support meticulous analysis. In these cases, water temperature wasnt monitored, roast degrees between samples were wide, and cuppers did not rinse their spoons from cup to cup. There were more problems than these, but these particular inconsistencies stuck out as highly problematic. Could I really make an informed buying decision or even provide helpful commentary about coffees where so many influential variables were uncontrolled? (Note: A buying decision on the ground at origin is separate from a final pre-shipment sample approval process.)

(continued)

To be clear, protocol lapses are in no way limited to origin-side operations. I posit that many professional coffee buyers and sellers dont maintain proper protocols 100 percent of the time and some dont even attempt them. In addition, some question whether existing protocols are sufficient and wholly accurate. As one example, Eton Tsuno, director of coffee at Sacramentos Temple Coffee Roasters, suggests that water protocols should be published in exact gram weight of water in addition to traditional volume recommendations, since this echoes the current habits of baristas in measuring and evaluating espresso and filter drip extractions. The quality of water is even more important as water comprises approximately 98.75 percent of the brewed beverage in a traditional cupping. Despite the importance of water quality, however, labs around the world (and even within the United States) dont use a standardized water base. For Christopher Schooley, a roaster who works with Coffee Shrub, a huge weakness of current cupping practice exists in the roast degree
continued on page 30

I posit that many professional coffee buyers and sellers dont maintain proper protocols 100 percent of the time and some dont even attempt them.

28

roast

S e p t e m b e r | Oc t o b e r 2 0 12

29

Laying It on the Table | A Look at the State of Cupping

recommendation. You have to look at more than one roast, he says, because there are problems with taking a qualitative measurement of both acidity and body, two things greatly influenced by roast development, unless you are looking at a couple of different roasts. OKeefe also feels that the sample-roasting specifications are too lax and instead requires that his samples are roasted within 10:3011:15 [minutes] rather than the eight- to 12-minute window of the SCAA. Certainly, there are many other conditions and variables in cupping that might be better controlled or differently managed. Without postulating conclusions here, I suggest to the industry that we review existing protocols and best practices to see where we can improve upon them in the interest of achieving greater objectivity and balance of results. Of course, until everyone is rigorously employing protocols, creating more of them or adapting them doesnt fix anything.

(continued)

Calibration
photo by Chad Trewick

Even if we had a perfect tasting process available to us and we employed its protocols religiously, a more difficult challenge manifests in the concept of calibration. Calibration is the notion that, assuming equality of sample and process, cuppers around the world are consistently infusing vocabulary and scoring with like meaning. Are the words citrus, fruity, savory and floral applied consistently among cuppers of the same coffee, for example, or does fruity mean cherry to some and over-fermented to others? Is an 85-point coffee in one lab at least in the range of 8486 among cuppers of the same coffee elsewhere? For cupping to serve its critical purpose as a buying and selling tool and price determiner, we have to have semantic consensus among our professional tasters. As Tsuno states, [Without calibration] pricing will break down, since in our small sector of coffee, price should be in direct relation to quality/ cupping score. (Tsuno also suggests that availability, sustainability and traceability are part of the pricing equation.) If we arent speaking the same language, how do we determine fairly if a coffee is worth a differential of +50 versus +250? And, are we doing ourselves a disservice by selling coffees to consumers at prices they arent really worth? Over the years, I have frequently questioned whether we are universally calibrated as tasters. For example, many times, I have been witness to or part of debates over whether a fruity sample (in the same cupping session) is over-fermented and defective or whether it is a 90-plus-rated coffee. Well-respected and experienced tasters can land on opposite sides of this debate.
photo by Chad Trewick

continued on page 32

30

roast

S e p t e m b e r | Oc t o b e r 2 0 12

31

Laying It on the Table | A Look at the State of Cupping

Tsuno, having also experienced this particular argument, states, I believe many people need more training in identifying uncontrolled ferment as a defect since what we like about honeys and naturals is a result in controlled ferment. Tsuno is exactly right: as an industry, we need more training to calibrate. In this particular case, we need more training of processing manifestations

(continued)
later in the article), but some of it is because we dont have universal language agreement. Despite the frequency of these debates over the years, to my pleasant surprise when I questioned some fellow tasters about calibration, they were largely positive about their success in matching their results with their outside partners. For Aliffi, Caribou finds that they trend in the same direction as our industry contemporaries more often than not. OKeefe asserted that his industry partners consistently score/calibrate within our results, but he also acknowledges that they have some trading partners whose results are very inconsistent. For OKeefe, these inconsistencies are attributable to counterparts neither scientifically running the lab, nor consistently cupping, though, and not problems of semantic variation. Aliffi also says that for calibration the greatest challenge is practices in the lab. Without doubt, poor protocols yield result variation, but part of the problem is semantic. We are not speaking the same languageboth in words and numbers. Scoring certainly shows itself to be widely varied among different tasters and different groups of tasters. Obviously, exact score matching isnt possible or even ideal, but it seems important that we are generally in agreement with a fairly narrow range of, say, two to three points as a two- to three-point variation can put a coffee into a different quality and corresponding price categorization. This group score just manifested itself in our office as we hosted a tasting of the Nicaragua Cup of Excellence (COE) coffees. Generally speaking, the entire group of more than 12 outside tasters, plus our four cuppers, ranked the various lots with scores ranging from 82 to 90 and generally a solid few points (or more) lower than the COE jury panel across the board (all jury scores were above 85). Protocol variation like roast degree might very well have been different and the influence of time between the jurors evaluation and ours was likely impactful (as an agricultural product, green coffees properties will change over time regardless of sample preparation and evaluation methodology), but who is right in these scenarios, and can there even be a right score? After all, cultural consumer preferences are distinctly different and one communitys good is another communitys outstanding, which warrant very different scores. Still, with purchasing decisions and price points at stake, calibration has merit. The good news is that some level of calibration among knowledgeable and practiced tasters can happen. Tsuno advises that one should learn protocol from an industry professional, and constantly review and communicate scores with other cuppers. In particular to scoring calibration, he suggests cupping true commercial, 60- to 70-point coffees and 90-plus-point coffees in order to put 80-point coffees into relation. (Although we have many discrepancies when looking at smaller score ranges, we are calibrated enough as an industry that this 10-pointrange categorization is generally agreed upon and, therefore, a tasting can be organized effectively.) For Caribou, Aliffi suggests that continued exposure to events through the SCAA and around the industrythe likes of barista competitions, Coffees of the Year judging, COE events and Rainforest Alliance cuppingshas rounded our abilities and helped us develop and maintain our consistency. OKeefe echoes COE participation as an important calibration tool, in addition to honest, open sharing of blind cupping scoring/categorization with your trading partners. a generally poor reputation for quality. Similarly, if a well-known industry taster has scored a coffee highly, then others may trend their own scoring on the high side. And, of course, the reverse is true as well. A sample might be very good, but, because a cupper is disinclined to believe it based on past experience, they score more conservatively, awarding it an 85 score versus a deserved 88. Or, sometimes, because we feel that everyone elsea COE jury panel or a group of vocal roasters, for exampleare overrating a particular coffee, we are more critical of it in an effort to assert our nonbias. Allowing these influences to impact our evaluation is, of course, unprofessional, and wed all like to believe we do not allow them.
continued on page 34

both controlled and uncontrolledto find some objective end to these debates. However, even if we can come to agreement and can taste accurately whether fruit is caused by happenstance versus deliberate effort in processing, I suspect we will still have variation on what is positive fruit versus fruit that is defective. Some of these issues get into bias (which we will discuss

Bias
One of the most prevalent problems with the cupping process is the influence of bias. Fortunately, many biases are readily apparent and relatively easily mitigated. For example, if a cupper knows that a particular sample is from a producer who pays scrupulous attention to quality or from a farm with an excellent reputation, they may be inclined, without conscious awareness, to score it higher than they would score a coffee coming from an unknown farm or from an origin with

32

roast

S e p t e m b e r | Oc t o b e r 2 0 12

33

Laying It on the Table | A Look at the State of Cupping

However, even among experienced tasters, these biases can occur despite best intentions and firm belief that we are in control of our partialities. Human psychology is very powerful. So, cup blind. This simply means, cup without knowing what youre cupping. Have someone else set up your tasting sessions ideally using codes only and dont even try to figure out or guess what you may be tasting. Just taste and evaluate honestly and with focus. Blind cupping isnt always possible or practical, and sometimes there are distinct advantages to cupping with knowledge of what you are tastingcalibration training, for example. Nonetheless, as participants in a process with results that impact pricing paid to farmers and feedback that can influence their future care of their beans, we should control the controllable variables.

(continued)

The Next Phase

Ultimately, I feel that the cupping process is a worthwhile and important one, but its a process fraught with imperfections; the current cupping process will never achieve true objectivity. Of course, we dont necessarily need complete objectivity; there are cultural preferences and biases that create a need for subjective variety in the marketplace. The key as professionals, however, is in having a system with as much impartiality as possible. As specialty coffee tasters, it can be easy to think of our job as just choosing the very best coffees from around the world to showcase, but we should remember that our decisions at the cupping table have real financial implications for hard-working coffee producers around the globe. So, knowing the limitations of the

cupping process, as Schooley recommends, it would be considerably helpful for [buyers] to look at coffees a couple of different times and in different iterations before making a final judgment. Maybe we can all agree to do at least that while we re-evaluate and improve cupping evaluation in the long run.

coffee trader with Atlantic Specialty Coffee, Inc. in

ANDI C. TRINDLE MERSCH is a green

California, where she also runs the quality-control lab. She has been working in specialty coffee since 1989. Andi has volunteered with the SCAA Training Committee since 1995 and joined the board of directors in March 2010. She can be reached at atrindle@ecomtrading.com.

photo by Chad Trewick

34

roast

S e p t e m b e r | Oc t o b e r 2 0 12

35