First off, I’m super bad at blogging on time/when things happen. This topic is relevant now, but based off of an experience I had a few months ago. In June of this year I was asked to participate in the first ever World Coffee Roasting challenge in Vienna, Austria. This event was a trial for the World Coffee Roasting Championship, which will start officially next year at the SCAE World of Coffee event in Nice, France. The challenge is a “demonstration of a competitive event designed to feature the talent and skill of the dedicated men and women involved in the artisanal craft of roasting specialty coffee.” I participated in the friendly challenge with six other folks (all men…) from Guatemala, Australia, The Netherlands, Norway, South Africa and Taiwan. Here’s a video that pretty much explains the competition, which is more entertaining than reading whatever I would write:
I hope that this trial run gave the organizers some good feedback about how to make it a success next year. The logistics of running a roasting competition are a little ridiculous, taking into account the equipment necessary, and the amount of time each competitor needs to familiarize themselves with the equipment, coffees, and the actual roasting and resting. It’s also not exactly a spectator coffee sport, like the Barista competition or the Brewer’s Cup, which include presentations, talking and numerous small tasks that make it worth watching for some (while I do think these competitions are good for the industry, they bore me pretty easily as well). I know roasting is an interesting topic for people, particularly customers of ours who aren’t yet very familiar with the process, but I never quite get it when people ask if they can come in and watch roasting; it’s boring to watch and most likely the person roasting isn’t going to be able to give enough additional attention to explain what’s going on, particularly when there’s a full day of production happening.
Aside from meeting some other awesome roasters and having a great time doing the challenge, the event put some more gas on the fire regarding thoughts I’d already had about roasting and posed questions I’d love to spark further dialogue.
Here are some thoughts:
- How do we evaluate roast across different roasting styles/cultures/ideas of what is a “good” roast?
- How much are people able to really evaluate roast separately from the evaluation of the coffee itself?
- What kinds of prerequisites are needed or “tests” do we give judges to make sure they can accurately do that?
We were given three different coffees to choose from, all generously provided by Café Imports. A Colombia, Mexico and a Brazil. All participants graded and evaluated each coffee to decide which coffee they would use to compete. The kind of obvious choice was the Colombia: high density, few defects and a pretty sparkling acidity. There’s the most room for error in this coffee because you’re not depending so much on capturing subtle nuanced flavors with a roaster you’re mostly unfamiliar with, and there’s a decent amount of acidity to fall back on and boost the overall cup should you take the roast a bit too far after first crack or allow too much drying time or development before first crack. In the spirit of challenge, I chose the Mexico, which had a really nice subtle floral quality and some distinct strawberry notes as well—easily a delicious coffee but less easy to nail in a few tries on equipment you’re not too familiar with (a challenge for all of us, the roasters provided were 6 kilo Giesen production roasters, which none of us used on a regular basis!).
<-- Here I am, lone user of Fahrenheit.
In the final cupping, two of us roasted the Mexico, four the Colombia and one person roasted Brazil. These were all cupped blindly together. The cupping was pretty interesting, as it seemed like there were a lot of different approaches, some very light and in my opinion, on the verge of underdevelopment, some with pretty obvious roast characteristics. While I do believe cupping blindly is the best way to evaluate coffees, it seemed that in this case it might have been good to reveal the coffees at least. The three judges cupped sample roasts of all the coffees, before cupping our production roasts, and I hope that they would be able to figure out what the coffees were but who knows.
The differences in the roasts were really significant to me. The one that stuck out the most was the “Scandinavian” style approach to roasting, which is very, very, very light. Now, Dogwood definitely roasts light, but we’re quite a few notches deeper than most of these style roasts I’ve had, which are at times even lighter than our “sample” roasts. As a parting gift, Simo of Oslo based Solberg & Hansen, well-respected roaster that’s been around forever, gave me two coffees which I drank on my vacation in Berlin over the next few days. While I could tell these were great coffees, the roast was so light it really didn’t make a good Aeropress or drinking coffee, in my opinion. I’ll just say it: the über-light “Scandinavian” style roast is totally not my thing. Sure, it shows the coffee naked and may be good for evaluating the quality of the coffee, but there is a lot of depth and sweetness that I think gets left out with this light of a roast, and the wheaty, stark acidity overpowers a lot of the delicious potential of the coffee. In Vienna, away from the competition, I had a similar experience at a few of the specialty shops I checked out.
In our discussions about roast styles, Simo commented on the general “American” style of roasting which is for the most part really dark and oily and gross. For the most part this is true I suppose, if you take the mass of coffee roasting and consumption into account. In the specialty sector, I think most roasters concentrate on a well-developed light roast, but stop before any carbon roast characteristics enter into the equation. This is how I’d categorize Dogwood’s roasting style, “well-developed light roasts.”
So, for a competition like this that includes different production roasting styles from around the world, what kinds of attributes and levels are the “standards” for judging this type of competition? Below is the score sheet used for the competition, which I believe is based off the Brewer’s Cup Compulsory Service score sheet.
So, this is pretty similar to the green coffee score sheet, which does make sense in some ways. When we’re evaluating the roast of a coffee, we are looking at the acidity, the body, flavor etc etc., but we’re looking at it in relation to that specific coffee that we hopefully already know a lot about. So the question about acidity really is, how did you take advantage of the acidity that was present in the coffee, how did you manipulate it to make it taste great? If the acidity wasn’t the particular strong point of this coffee but it has great flavor attributes or body, how did you make those prominent in the cup and make this coffee taste as great as it can?
On this sheet there’s a place for “defect” as well. I’ll go ahead and assume that this means roast defect. How should this be scored, and what is a considered a defect? Some are visible defects, like scorching or tipping, but what about the ones that aren’t necessarily visible? I’d say that over development is a defect, but what means over development to me, might not mean the same thing to another competitor from another roasting “school of thought.” Also, roast defects are likely something that would affect the whole roast, so multiplying by cups isn’t necessarily the best option.
The judges were overall not very calibrated, and had also been drinking espresso all day, for whatever that’s worth. I got scored an 87ish something from one judge and a 74 from another. Hm. They apparently had all agreed on scores for the sample roasts, so they felt the same way about the coffees in their naked state, but that’s one interpretation of the coffee. Unless we were all doing sample roasts, which is totally not the point of the competition, the sample roast provided in isolation was maybe not the best way to base all the judging.
[Side note: These are all just observations and thoughts I’ve developed based on this experience, I hope that what may be interpreted as criticisms are understood to be general thoughts and prompt discussion on roast evaluation in general, and hopefully help with further development of the competition.]
At what point do you “know” a coffee well enough in its current state (age, storage etc.) to be able to judge roast? It may benefit the overall competition if the coffee could still be a surprise to the roasters, but that the judges were given more time to learn the coffees well; maybe roast it themselves, cup it several times and play with profiles before the competition.
At Dogwood we cup our production roasts every day, and as the coffee ages we revisit those profiles, playing around with the coffee on the sample roaster and in our production roaster to see what kinds of flavors we can pull out, what’s available in the coffee at a certain point in time. Below I’ve pasted a recently revised version of our profile sheet, the nifty bit of paper that Eddie uses to record roasts. Originally we had the green coffee cupping form at the bottom, but soon realized that scoring with something designed to evaluate coffee outside of roast (as much as possible) every day wasn’t really helpful to our evaluation of the roast. We were mostly just taking notes and not scoring.
With this sheet we’re not looking at a numerical value, we’re looking at how we did with these different aspects of the coffee based on what we know they should be. So let’s say we do a roast of the Nereo Ramirez Costa Rica, and we cup it and find that the acidity was much more prominent, complex and delicious than the last roast or what we usually see from it, we’d make note of that and take a look at the roasts and see what might have improved that: time of first crack? Rate of temperature increase after first crack? The weather outside? Eddie’s outfit choice that day? Etc. The same goes if something was less than it should be. This type of evaluation gives us a more comparative value rather than an overall numeric value from a form that isn’t necessarily even relevant. The “defect disguise” category is generally for coffees that are aging. It sounds a little harsher than it is, because we generally try not to source defective coffees! Everyone has a coffee once in a while a little longer than they should, and if it’s not at the point where we need to just kill it, how are we doing with minimizing that “loss of organic matter” taste in the cup? Or, for instance if we’re roasting a Rwanda or Burundi, we may want to keep track of how many potato cups we get throughout the time we have the coffee, which may be helpful information for future purchasing.
Back to the cross-cultural roast evaluation topic. There are similarities here between roast evaluation and green coffee evaluation and the conflict of taste and style. Great coffees can be widely understood to be great coffees, but there’s still the element of taste and style preference. The difference here is that people who evaluate coffee hopefully cup enough that they can understand what a good wet-hulled Sumatran coffee tastes like, or a Brazil, or a Kenya or a natural… even if they hate it. This is done internally at roasting companies that have various roast styles. People can probably understand what a “good” French Roast is, even if they think it’s awful. if it’s part of the quality control they need to do at their particular roasting company; how does that work in this type of competition, though? Do the judges approach it as “this is a very good Scandinavian style roast” or “this is a very good dark roast”? A dark roast doesn’t necessarily show the skills of the roaster person to maximize the flavors and strengths of a particular coffee, but if the challenge was to cover up a defect or something, then it would.
I’d like to see the World Roasting Competition evolve into some very specific skills roasters deal with every day. It would be great if all participants had to roast three very different coffees (different origins, densities etc.) and maybe something with some sort of problem such as super high moisture content or something baggy. The scoring would be based on how well you could apply your roasting skills to each specific coffee. You wouldn’t roast a high grown, dense Kenya the same way you’d roast a softer Brazil coffee, for example, so how DO you approach those coffees? How do you approach them differently and make each one cup out amazingly as a production roast in their own way?
This blog has posed a lot of questions and hopefully material for discussion. Roasting is really fun, and I’m so glad that Trish asked me to participate in this trial run. I’m excited to see where this competition goes, and I hope that the United States will eventually have an in country competition and send someone to the Championship. I also hope there are more women next time!
Send me any thoughts you might have on this topic! Hate mail also accepted.
Director of Coffee, Dogwood Coffee Co.
Thank you also to Trish Rothgeb and Filip Akerblom for asking me to participate and putting the competition together, Giesen for donating actual production size roasters (though still small for me!), Cropster, Sinar Technology, Probat, Loring, Curtis, Boot Coffee and Café Imports and I’m sorry if I forgot anyone.
Here's the group of rad roasters/guinea pigs: