Evaluating Roast

Written By Stephanie Ratanas - September 17 2012


geoff watts
September 28 2012

This post made my heart smile. Bravo, Stephanie, for being so clear and honest and thoughtful in your writing. And for taking the time to share on a topic that is of great interest to everyone who loves coffee but that is all too often charged (npi) with a lot of hyperbole, stubbornness, dogma, and polarized opinion. You are right to ask so many open ended questions. We’ve all got so much to learn about roasting science, sensory evaluation, and how we can (or should) measure quality. Sometimes the definitions and metrics that emerge from our collective attempt to make sense of coffee quality end up feeling sort of obtuse or detached. It is nice to step back and think about why we score things a certain way, and how relevant those metrics are if the ultimate goal is maximum pleasure. Anyway, it is great to see that Roasters are finally getting their own competition that should serve to celebrate the artisanship, skill, and science that make roasting so freaking fascinating and so vitally important in the quality chain. Big props to everyone involved in making that happen, and good luck. And by the way I love the suggestion that the competition could evolve into a something where roasters are required to demonstrate different skills (using different kinds of coffees) and where success could be measured in a way that does not rely too heavily on a handful of judge’s individual definitions of beauty.

September 28 2012

Hey, could you repost a version of this on the RG forum so the rest of our peeps can check it out? Excellent post by the way. I think it’s really in line with most of our concerns as Roasters about the whole “competition” thing.

Hanna Neuschwander
September 26 2012


Excellent food for thought in here. I recently wrote a book on roasters (a guidebook of sorts) on the west coast. For the book I interviewed over 60 roasters. It was amazing to me how ingrained the cultural preferences you describe are (and I think you’re right to call them “cultural”). Almost every smaller specialty roaster I interviewed, though not all of them, talked about “letting the coffee speak for itself.” Even though the roasting styles varied from hardcore Scandanavian style light roasts to full city, roasters used the same language to talk about how they weren’t intervening. (This in itself is problematic: We don’t have a good language for defining roast styles. “Light” and “dark” are too blunt. “Well developed” is too subjective. No one seems to like the old school “cinnamon” and “full city”, and besides that they seem too arbitrary and culturally specific—they wouldn’t translate well across national lines. But that’s a separate blog post.)

This idea of “nakedness”, I think, has been a profoundly useful way of advancing the conversation about coffee and origin to this point, but I see its potential to undermine the good work of roasters going forward. It ignores reality: Roasting coffee fundamentally alters it. That’s the value the roaster adds. I don’t understand why so many folks are reluctant to own that fact. If all a roaster is doing is “not f*cking up” green coffee, why can’t I just buy some of the same high quality green from Sweet Marias and do the same thing myself? What good is a coffee roaster to me?

In a number of cases, the roasters I talked with were quite young/new to the business. I think to a degree this line of argument had become a protective cover for the fact that they were still getting their roasting chops in order. It’s easier to lay your success or failure at the feet of your beans than it is the skill of your roasting. I drank a lot of coffees that were flat out poorly roasted. But I’ve watched every one of those roasters get better.

I think roasting competitions could be a major way of advancing the overall project of roasting great coffee, of allowing roasters to differentiate themselves, and of encouraging innovation. And, as you suggest, of putting a finer point on some of the tricky questions we haven’t poked too many sticks at yet—the most essential of which is: how do you define the skills of roasting, and usefully measure them? Given the incredible curiosity at the heart of specialty coffee, defining some of these things will certainly drive younger and established roasters to become better, and will give them some useful tools for doing so.

Hanna Neuschwander

September 20 2012

Thanks for your comments, guys. There is so much to talk about on this topic, hopefully we can keep it going!

I definitely think there is a way to have great, bright acidity and balanced sweetness— but these things are also a matter of culture, taste and preference. What may be interpreted as “harsh” acidity to some, might be very balanced and wonderful to others. This competition really sparked my curiosity about various approaches to roasting across cultures and markets. Obviously, we have a lot of variation in what we do here in the states, which is interesting in itself.


This Nordic Barista Cup video (you may want to skip ahead, the beginning is sort of boring) is interesting. The same coffees were roasted on six different machines, and the results were cupped. Obviously you can’t taste through the video, so you’re not really getting the whole experience, but the most interesting part to me was how very different the roasts were executed by different people. Very different placements of first crack, different finish times and temps and percentage loss—but all roasted to the same “level”. Some of this of course has to do with the machine you’re working with, and what makes sense with that equipment, but some of it is also style. How did everyone get to the approach they have to roasting? Was it taught? Based on experimentation? Based on equipment limits or advantages? There are so many approaches it seems that we’re a ways off from coming up with any “standard”, and even when and if that happens it will be broken and needs to be. On a related note, Jon’s blog on here “Coffee Under Pressure” addresses the same issue of standards in regards to espresso. Check it out.

Keep it coming.

Marshall Hance
September 19 2012

I roast light. Ten to fifteen degrees (Fahrenheit) post first crack. There’s such thing as sweetness in harmony with, even dominating acidity at this level that I’m not hearing echoed above. Either way, you great raise a question that’s on my mind: with so many ways to roast a great coffee, how is one approach judged better? I personally like the Nordic Roaster event that puts the attendees in charge of judging.

Joe Marrocco
September 19 2012

Excellent post. I like it that you have obviously mulled I’ve these thoughts and questions, developing them quite a bit over the past few months.

Evaluating roast in light of a particular coffee’s attributes, whether positive or negative, seems to be a logical mode of judging. Having the judges familiar with the coffee prior to judging is crucial. It is almost like the opposite of a normal cupping procedure. Normally the cupper needs to be very familiar with the cupping score sheet and use it to reveal the nature of a coffee. In this format, however, it is necessary to be very familiar with the coffee and see how a roast alters the original scoring of that coffee.

Rethinking cupping roasts is not only important for this competition, but vital for wholesale/retail roasters. When making purchasing decisions, traditional cupping as score sheets are really about as good as we’ve got, right now (another discussion). But, I find so many roasters use their cupping rooms primarily for this. Cupping and analysizing production and how the roasts are affecting the coffee is really, in my opinion, far more valuable to a roasting company. Vetting coffees is fun and important, but developing and sustaining a profile or expression of that coffee is the weight of a roaster’s job. We need to develop better tools for this. Competition has done this in the world of espresso evaluation. My hope is that this competition does the same for roast evaluation. Not to flatter, but truthfully, it is honest, open, thoughtful posts like this that get the ball rolling. Thanks for sharing.

…typed on my phone… Sorry for mistakes.

September 19 2012

Awesome post, Steph! I think that you hit the nail on the head with a number of your observations on the problems of evaluating a roast. I don’t believe that giving a finite score to something that has such a range is really realistic. I also think the evaluation tool of cupping can be a major hang up and lends itself to the confusion with evaluating the coffee or the roast. I also think that some of the issues in terms of hyper light roasting style have to do with such a heavy emphasis placed on acidity, that that’s where all the exotic fruit flavors are, as if it’s the only element in a coffee or at least the most important one, while sweetness is just assumed. when cupping you check a box for sweetness, but you qualitatively measure acidity. Isn’t it easier to get people excited about a sweet cup of coffee rather than an acidic one?

Leave a comment

Comments have to be approved before showing up