Standards on Tap

Cold beer. Hot coffee. Spicy food. Fresh-smelling deodorant. Learn how sensory scientists evaluate aroma, mouthfeel, flavors, and more in the things you use and enjoy every day.

Join host Dave Walsh and his guest Ali Schultz, sensory manager at New Belgium Brewing, as they discuss standards for sensory evaluation from ASTM committee E18.

Follow Us

• Twitter @ASTMIntl
• Facebook @ASTMInternational
• Instagram @astmintl
• YouTube @ASTMIntl
• LinkedIn @ASTM International

Presented by ASTM International

www.astm.org

Transcript

Dave Walsh (00:17):

Welcome to Standards Impact, ASTM's official podcast. Today's event on Standards for Sensory Evaluation covers a topic that is sure to be one of our most popular ever, and that is beer. Of course, sensory evaluation is a critical tool in product development and quality control, and it's used to evaluate a wide range of products, from cheese to fish, to even underarm deodorant. But we're going to start off today's event talking beer, with a sensory evaluation expert who happens to have made this beverage a central part of her career. I'm your host, Dave Walsh, editor-in-chief of Standardization News, and today I talk with Ali Schultz, Sensory Manager at New Belgium Brewing and Bell's Brewery. She's also a member of the Committee on Sensory Evaluation, E18. All right, so thanks for being with us today, Allie.

Ali Schultz (01:02):

Yeah, it's awesome to be here.

Dave Walsh (01:04):

So I'm sure you can guess in advance what my first question is going to be about, and it relates to your job at New Belgium Brewing. It's what most people would consider the best part of your job, and that is beer and specifically drinking and tasting it. So what can you tell us about the process of evaluating and assessing beer? Is it as glamorous as it sounds? Are there woozy people all over your office all day? <laugh> What is it like?

Ali Schultz (01:26):

Uh, a little bit of column A, a little bit of column B, I would say. <laugh> Um, usually how I, I start off describing my job is something that you nailed, saying that we don't drink beer, we evaluate it. And there's a very long process to become trained to do this. And also just in the evaluation of beer itself, we train them on an 11-step process, which who knew there was that much detail in drinking a beer or sorry, evaluating a beer. Uh, even I'm guilty of that sometimes. And so it's a really fun job, um, because like you said, it's, it's really fun working with beer. It's fun working with a product that people enjoy versus like, you know, everybody likes cheese, but it's not on the same level as beer, let's be honest. So I mentioned we train our panelists and they go through quite a rigorous training process.

(02:20):

We do spike the beer with off flavors and then have panelists, unknowing what it spiked with, have to identify the off flavor. And they go through this for, you know, about two years before we let them on panel and start using their data. And that's the not fun part is, you know, we're spiking things that smell like garbage, that smell like baby puke, uh, because these are real potential off flavors in beer, and we drink these types of things so that the customer doesn't have to. But got to bless the panelists hearts for just, you know, putting these types of beers in their mouths, knowing they're spiked with something bad and really trying their hardest to be like, "Ah, yes, this is puke." <laugh> <laugh> Something else that's really interesting about, uh, how we train our panelists is we allow anybody to be a panelist. You know, some people have more natural acuity than others in terms of being able to perceive different aromas or tastes, but we really can train that into anybody.

(03:26):

So it's really a matter of focusing and being interested in this process and that's what really makes somebody a good panelist isn't their natural talents, but their interest and enthusiasm. And I do always tell them, "The one thing I don't care about is how much they like it. I just really don't care. If you wanna tell somebody, go call your mom." <laugh> Like we, we're gonna be drinking a lot of different styles of beer because whatever comes through the brewery, they're being asked to evaluate and make sure it is, uh, what we call quote, true to brand. And so, you know, maybe you hate imperial IPAs. I don't really care. You're gonna drink it. <laugh> And so I, I find myself in the same boat. There's, you know, I don't love all of our brands. I definitely have my favorites, but I have to drink my least favorites as well and be able to identify off flavors or if the balance of malt and hops is off in them.

(04:26):

So that is maybe a downside, I guess. And another one I tell panelists is they will become beer snobs. And that sounds kind of cool, but it's actually the worst because you can't turn it off. You go to a bar or a brewery that you really used to like and you're like, "Oh, no." <laugh> This, this beer has off flavors in it and it just really can kind of ruin the experience. So I don't know anything about like wine or spirits. I just enjoy drinking those and I try not to think too much about it because with beer, I know too much and it's, it's impossible to go back at this point. <laugh>

Dave Walsh (05:07):

Well, for the layperson, many would wonder why is sensory evaluation so important to not just the beer industry, but the, the food and beverage industry in general, whether it's cheese or any other product, I think everyone can instinctively understand why it's important when you're creating, say, fat tire from new Belgium, but once it's up and running and off to the races, we would think, "Oh, you know, just make a million gallons now, you're fine." Why is it so important to your industry and to the food and beverage industry in general?

Ali Schultz (05:36):

Yeah, a lot of reasons, and obviously I'm very biased towards thinking sensory is the best. <laugh> <laugh> But with beer and with a lot of food products, well, most food products, like you, you know, they're natural products. There is automatically variation in the process. We use agricultural products in beer like malt and hops, and yeast is a living organism. And so all of these things factor into making each batch slightly different. There is no such thing as two identical batches, that the beginning of a, a run to the end of the run might be slightly different, even as much as we try and have it be consistent, it's just unrealistic. And that is true of any food product, and I bet a lot of non-food products as well. So testing is important, not just sensory testing, but all testing is important for that reason, but sensory specifically is important because I, usually we're cheaper.

(06:34):

At New Belgium, our panelists are all coworkers, so they're just paid, you know, their normal salary and they take time out of their day to participate in panels, so we're not paying them extra for that. Compare that to the cost of a gas chromatograph up in our analytical lab, and they have multiples of those, and that's just one of their instruments. And so we pay our panelists in snacks, so I give them a bag of pretzels and they're good to go. <laugh> <laugh> And that's a bit different than a gas chromatograph. The other thing that I think really is important to remember in sensory is we are a lot more sensitive, especially with certain compounds like sulfur compounds than instrumentation. One example of an anomaly we just had with our water is with geosmin, which if you're not familiar with that compound, that is the main aroma compound in beets.

(07:27):

So that kind of earthy, soilly beet aroma. Humans are extremely sensitive to that. We can pick it up at one part per trillion, which is one drop in five Olympic swimming pools, and instruments can get nowhere close to that. And it is, it is a very recognizable off flavor that, that consumers would certainly pick up. So we're a lot more sensitive than instrumentation in a lot of ways. And then I think another big reason, arguably, perhaps the biggest reason is sensory allows us to look at products holistically. It sounds like I'm bashing analytical. I'm not. I love my analytical chemistry coworkers. They do absolutely amazing work and the work I do isn't possible without them, but they can measure a beer's hop compounds. They can understand how much mercene and linolu and caryophylline is in an IPA, but that doesn't tell you what it tastes like.

(08:26):

And it's really sensory that bridges that gap between what are the components of a beer to how does it taste to the consumer? You know, we can describe harsh bitterness, whereas analytical can just measure IBUs. We can say, "Ah, this has this level of esters and even this isomal acetate is turning into this bubble gum instead of just banana." Whereas micro is like, "Ah, yes, we used this specific Belgian yeast strain." So we help bridge that gap between science and somebody who's just enjoying a beer doesn't necessarily wanna think about the science, but wants a really well-rounded product. And then kind of a smaller piece in that, aside from holistic is, is there a perceptible difference? So analytical, again, could measure different things about the beer, but does that matter? Like they could say the pH is slightly off, it's, it's a point higher, maybe we should freak out about that, um, because it's not in spec, which is, you know, all well and good, but then does that have an impact on the consumer?

(09:26):

Can we tell? And so sensory is, it's like we're a robot consumer almost where we have the training to be able to use our words to describe how the beer is different or how it tastes, but yet we're also like a human being where we're using all of our senses to evaluate this beer. Again, I said sensory is pretty awesome, but I think those combinations of we're usually cheaper, we're more sensitive and we're holistic because we're doing it like the consumer is, is really what makes sensory so awesome.

Dave Walsh (10:00):

It's funny, I think you've just identified a job that is in no danger of being taken away by AI. It's so inherently subjective. You know, you even mentioned, "I just like it. " So that's something that a gas chromatograph hooked up to ChatGPT probably couldn't determine what people would just like for no other reason than it appealed to them.

Ali Schultz (10:18):

Oh, yeah. And I mean, we do run consumer studies at New Belgium as well in our tasting room and people love participating in them. We give them little stickers afterwards that say, "I drank beer for science," and they just go nuts over those stickers. <laugh> Um, but as much as people are trying to predict what consumers like, it's very challenging. And so you really have to give consumers the product and ask them what they think about it because yeah, we tend to know, even though consumers don't say it, they do like sweeter things. In, in terms of not necessarily beer, but sometimes with beer, yes. But, you know, a lot of the canned cocktails, which we've dabbled in at New Belgium, it, people are like, "I don't like really sweet things." And then you try the ones that are on the market and you're like, "Holy crap." <laugh> I am one of those people too.

(11:07):

I'm not like, I'm not shaming anybody. I am that person too. And so it's, it's tough to use AI to be like, "Oh, what are people saying they like? " Because what they say isn't necessarily how they truly feel. And so that makes it even harder for AI to participate and to understand what they're doing. And so we see AI as being helpful, less so in evaluating products, but more so in helping us with data analysis. You know, if you're doing a focus group with a bunch of consumers to understand how would they prefer to drink a new type of product, do they want it with a straw, do they want it with ice, things like that, AI can help us take the transcript and pick out themes from it, something like that. So you're right. There's even been some things of like, I can't remember if it's called like an artificial tongue or an electric tongue or something like that, but I think that might only just do, you know, things like the basic tastes, sweet sour bitter and maybe some compounds.

(12:08):

But again, we're, we're kind of lacking that holistic sense like, sure, it can tell us the compounds in there, it can tell us what's sweet and bitter and maybe we have an understanding of the balance of sweet and bitter that consumers like in an IPA versus a wheat beer, but it really doesn't get us all the way there. And then we're talking about like a lot of costs and if you're not even sure the answer you're getting is the right one, then why risk it, I guess?

Dave Walsh (12:35):

Well, of course your committee, the committee on sensory evaluation has a lot more going on than just beer, even though I would enjoy talking about that for another hour or so, but- <laugh> You have standards for topics that range from evaluating contaminated fish to testing the odor and flavor of drinking water, the standards in E18 run the gamut. So what can you tell us about some of the newest developments coming out of your committee? And new standards can be included in that, but also like what are some of the most important standards that E18 oversees?

Ali Schultz (13:05):

Definitely, yeah. It's an exciting time to be an E18. I am the, um, subcommittee chair for E1806, which is food and beverage. So those are the ones that I'm, I'm most familiar with, but I do have another example from another one that I'll get to in a second. Two of the ones that are up and coming standards in food and beverage subcommittee are one on coffee and tea, which the more we've dived into it, the more we've realized, holy cow, this is a can of worms because you have hot products, you have cold products, you have bags and loose leaf tea, you have Keurig pods, you have potential additives, um, that we all need to factor into the document. And when I say additives, I don't mean like preservatives. I mean, I take my coffee with so much cream and sugar, it tastes like a candy bar, whereas my husband drinks it black.

(13:58):

And so how do you factor that in when you're testing it? Do you let panelists do what they want or do you set a standard where it's like everybody has to drink it black to get the true flavor? And maybe that depends on why you're testing it. You know, is it for consumers or is it for quality? So that's just one layer of the complexity of coffee and tea. And then the other one that is so fascinating that I knew nothing about before I really started working on it is, uh, chemistetic heat sensations. So chemistetic heat is that spicy note we get from capsaicin or ginger, chili peppers, even black pepper. So that one has been so fascinating to learn about because I've worked with some incredibly, incredibly intelligent women on understanding how our bodies respond to this and how the heck you test something because, uh, you know, you have some hot sauce on your burrito and then I can't taste anything else for like five minutes afterwards.

(14:55):

And how am I supposed to taste three different samples of this and rate the heat intensity? So that has been a really awesome standard to help work on and shepherd through the process because I think there's a lot of, a lot of questions on how do we test these products that seem kind of untestable for sensory because of this long linger or the complexity of them. So that's the food and beverage. And then one that I think continues to be a perennially popular standard is the ad claims one. And ad claims is a very popular topic because ad claims are the type of thing where it's like, nine out of 10 dentists say this or our potato chip tastes as good as the leading brand. All of those have to be extremely vetted because you can be sued over that thing. And, and there's a lot of lawyers and judges and, and companies, frankly, that their job is to deal with lawsuits regarding these claims.

(15:54):

Even something like, uh, you know, we had a product that was like made with real fruit juice, which it was, but that's not enough, apparently. You have to prove that making it with real fruit juice makes a difference in the product. And so that, that blew my mind. And so ad claims, because of the level of scrutiny it receives with big companies potentially duking things out saying, "No, my potato chip is better than your potato chip." Add claims with its extreme complexity continues to be one of our most popular standards in E18.

Dave Walsh (16:28):

So those are all really interesting standards and interesting applications of sensory evaluation, but I did want to ask you an additional question, which is what are some of the most unusual applications of sensory evaluation? And by unusual, I kind of mean gross. I mean, I know there's some, there are some pretty good ones. Like I mentioned rotten fish already, that's pretty nasty. So what, what are some of the strangest applications of this technology, but also if there's a standard for it, that, that could dovetail nicely, but it could just be the application.

Ali Schultz (16:55):

Yeah, totally. Well, you stole my thunder with the contaminated fish. And interestingly, I, I didn't know this until I started working on this standard since the contaminated fish is un- also under my subcommittee. I get all the good ones. <laugh> But, um, it's actually for fish exposed to oil spills. You know, when you have the oil spills and the gulfs, that's the method that is used to test whether they can be consumed by people. So it's fun to make fun of it, but obviously it serves a very real need. <laugh>

Dave Walsh (17:24):

Yeah, for real.

Ali Schultz (17:25):

Geez.

Dave Walsh (17:25):

Um,

Ali Schultz (17:26):

And it used to be called exposed fish, which was really funny. <laugh> And we would, we would make a joke of like, "Ah, this fish is like flashing you or something." <laugh> But we changed the name. The other one that astounds me that people do this, it's for non-food product and it is called the Standard Guide for the Sensory Evaluation of Axillary Deodorancy. So you have people that you train to smell other people's armpits.

Dave Walsh (17:57):

Ugh.

Ali Schultz (17:57):

And I am not joking. <laugh> And, and I was reading the document a little bit ago and it's actually quite fascinating because you have to recruit the people that smell the armpits and you have the people that are the armpits who wants to sign up to be like, "Yes, I do want people to smell my literally sweaty armpit." <laugh> And they have to test these, I think they call them subjects, the people who are the armpit, they have to scream these people to be like, "Well, we need you to not smell too bad or not smell too not bad." So it's like this like sweet range of smelliness and then they have to compare one arm to an, like one armpit to the other armpit and make sure there's not too big of a difference between how you sweat between the armpits. And I was like, "Oh my gosh, my mind is being blown right now." <laugh>

Dave Walsh (18:48):

I don't know if they could pay me enough for that one. <laugh>

Ali Schultz (18:51):

Right? Yeah, it is crazy. But I mean, think of how practical this is. Like when I purchase deodorant, I don't want to smell. Yeah. That is why I buy it. <laugh> And so you need to have somebody doing this testing and the, the best way to do it is to have somebody like run on a treadmill and then see how it works and have somebody who is trained to rate, rate your body odor. I just, it's just crazy to me. Um, but it, that's one of the, the cooler grocer ones. And I think a lot of personal care products <laugh> do end up in that realm. And there is a lot of sensory testing that goes on, but that deodorancy one is, uh, one of my favorites. <laugh>

Dave Walsh (19:32):

Well, it kind of brings me to an interesting follow-up question. The technology used in sensory evaluation is largely just human senses. And I'm wondering, is there any high-tech equipment to be used? Is there any aid that you can give to a sensory evaluator to either taste beer or smell someone's underarm or, or whatever? Is there technology that could assist this field in the future?

Ali Schultz (19:54):

Kind of. I mean, I think there's just the same type of thing, but at different levels of fanciness, dare I say. So for example, in, in beer, I talked about we spike samples of beer with mecaptin, which is the, the garbage flavor. And we spike it by simply adding in the chemical compound that we purchase from, you know, one of the chemical companies like Millipore Sigma or something like that. But it's, it's very interesting. Like beer, it's easier, you're putting it in your mouth, but when you're talking about room odors and how do you make a room smell like one thing and then clear it out and have it smell again? And so I think sometimes it's the delivery that is where the technology comes into play. So you'll have things like airflow where it, you know, it goes past your nose or you might have a certain liquid stream going past your tongue.

(20:49):

I've never used any of these, but I've read some papers on it where it's the delivery method where it's like this very unique kind of complex thing to just really try and isolate how humans are experiencing the product, if you will, that we're trying to evaluate. And so that I think is where we see some of this technology come in. But really, I would say that the fundamentals of sensory are like, you know, in order to train somebody on this, you need to give them examples of the thing you're trying to train them on. This is what mercaptan smells like. This is what isoamyl acetate that banana aroma smells like. This is what geosmin smells like. And I think it's just how the panelist is experiencing those might be different. And then also, how are you delivering the product to be evaluated? In beer, we just drink it.

(21:39):

Whereas with maal odors or bad odors in rooms, you might be wafting a stream of air under their nose instead of literally putting them in a room, although that's done too. So I don't really see it getting super complicated, at least on the evaluation end of things.

Dave Walsh (21:58):

I also wanted to ask about the standards development process and how your committee works from the inside as well when these standards are created. I know for E18 in particular, collaboration is important because these are sensory evaluation standards, but therefore the agriculture industry, the beer industry, the dairy industry, it's all these different fields that need to be brought together to have input on these standards. So how important do you think collaboration is for your committee and how will it help all of those industries? Why is it necessary?

Ali Schultz (22:29):

I can't overstate how important all of that collaboration and that consensus is. I don't remember the exact adage, but it's like all of us are smarter than one of us or something like that, where that is, I think, the strength of ASTM is we take a bunch of people's opinions and their personal experiences and their knowledge, and we combine them into this holistic document that has been debated and dragged through the dirt and then brought up from the ashes of that into this Phoenix, if you will, of a standard. And then it's not even done then. You know, within seven years, you're reviewing it and making it even better each time. And I think personally, I have really enjoyed watching that process and I have learned so much from the other people on ASTM, to be honest, giants in my field. I'm talking about people that write sensory textbooks that I studied in school.

(23:25):

I got to sit next to them and learn from them. And then I've taken that knowledge and practiced it at New Belgium, and then I can come back and contribute to ASTM to further the sensory field and to make sure that people are doing sensory science well because I think people hear sensory science and they're like, "What? That's not real." <laugh> And they're like, "Oh, you drink beer? I can drink beer." And I'm like, "No, we evaluate beer." And I think that's a, a very common misconception is anybody can put a food product in their mouth, but that's not science. And so how do we elevate this field and make sure we're all doing the same science in the same way? Especially like, I, I totally get it. Like plant managers, they don't have time to like read textbooks. Are you kidding me? And so how can we give them the tools to do the science in the best way possible, knowing that they don't have a whole fully trained panel.

(24:21):

They have outline workers that can maybe take 10 minutes out of their day to evaluate product. How do we make sensory science accessible to that situation in a way that makes them produce the best product possible? It's all about statistics. And a lot of the statisticians, they're so smart, holy cow. <laugh> But they will get into like drop down, drag out fights about the best way to do something and all the rest of us in the room are like, "Whoa, this is crazy." And they're saying words, I'm like, "I vaguely know what you're talking about. " <laugh> But at the end, you end up with something where I can feel really confident in using it because I have seen it be debated and the best outcome has been decided upon.

Dave Walsh (25:00):

That's a great take on the general process, but I wanted to get a little more granular and ask about how the specific standards development process that ASTM uses, you know, a consensus process, I wanted to ask, how important has that been in making these standards more effective and more accurate? Bringing all those stakeholders together as ASTM does, how important is that to the work of E18?

Ali Schultz (25:22):

It's really huge. I mean, you'll read sensory textbooks, different ones, and for the most part they're the same, but there are different ways of doing things. And it's tough to know, especially if you're a young professional, you get kind of shoved into the sensory role, you might be a chemist and they're like, "Well, we need somebody to do sensory and you're kind of a scientist. Here, do this. " So like if you have no experience with sensory, you need to start somewhere. And I think that's where ASTM's strength comes in is we provide these documents that have been vetted through this consensus process to come up with the best way to do things. And I think the other benefit of this consensus process is we have a lot of people with a lot of different experiences. We have professors, we have people that are in companies where all they do is sensory and they take on projects from businesses to do sensory for them.

(26:15):

It's kind of a third party sensory testing, if you will. And then we have people in industry like myself where I do pretty much all of our sensory in- house. And so we all are bringing these different perspectives to the table and making something that is useful. You know, we're not pie in the sky. Well, this is the way you ... In certain cases, it is like, yes, you have to do it this way because this is otherwise crap sensory science. But in certain circumstances, we're like, look, this is the best way to do it. This is something that you need to be thinking about if you can't do it that way. So at least we're acknowledging some ways you could do things differently if you can't do it the absolutely perfect way because we do live in the real world.

Dave Walsh (26:56):

Well, you mentioned the word young professional in your answer, and that is one of the key priorities for ASTM always is bringing in the next generation of experts, the next generation of standards professionals. And so what would you say to a young professional starting out in your field in terms of engaging ASTM and the world of standards? I'm sure many young professionals get involved in the field of agriculture or dairy, as I mentioned, and they don't really have familiarity with standards, and suddenly they're kind of thrust into that world. So what would you say to them and has it helped your career and been worthwhile?

Ali Schultz (27:28):

Oh my gosh, yes. <laugh> So for somebody that isn't in sensory but needs to use our documents, I think it would be so invaluable. Like I mentioned, a lot of people don't even know what sensory science is. And then if they do know, they're like, "Oh, well, I could just put it in my mouth." And we're like, "No, n- I mean, you can, but you need to understand that this is a science and there's a right way and a wrong way to do things in order to get actionable results, to be able to tell that your product is consistent from batch to batch, or if it's not, how to identify what's wrong with it. " And so young professionals, you know, this is kind of like a leg up. I think being a young professional is so hard, especially if you're coming right from the classroom, you had somebody you could ask.

(28:15):

You always had a teacher or a professor where you'd be like, "I don't get this. Can you explain this to me? " That does not exist in the real world, to my constant sadness. And so these standards are a part of that, is like, "Look, we're telling you how to do this. You don't have to reinvent the wheel. It's right here. Here's the wheel. Put it on your car and you're good to go. " And then I think for people interested in participating in ASTM, especially sensory professionals, I cannot overstate how big of an impact working with the other ASTM members has been for my personal growth and my networking and my career. Sensory science is very well respected in the brewing industry, however, it's not always done the best it could be. And that's not necessarily the brewery's fault. A lot of breweries are pretty small and they have slim numbers of people on their staff.

(29:08):

So you might have one person doing all of the analytical chemistry, all of the micro, and then, you know, if time, they'll do some sensory too, but that's typically like, "Well, I'm gonna put it in my mouth and make sure it tastes like it did last time." And different breweries do differently. But I think that's a good example of, like, you work with what you have, but then there's not a lot of, like, growth opportunities or, or knowledge in sensory. And so by being a member of ASTM, I have learned so much. I like to think about it like I am the little fish in the big pond of sensory gods <laugh>, if you will, and I do honestly mean that. And so the way that young professionals can participate in ASTM, both from the standards themselves and from participating in committees and hearing these debates is just unbelievable.

(29:59):

I, I I love it. And I think the other really cool thing, the only sensory job I've ever had has been in beer. I know very little about how to do sensory on a burrito. Like that is a very complex product compared to beer. Beer at least is all homogenous. A burrito has like beans and cheese and rice and a tortilla. Like what the crap? I don't know how to evaluate that. But I think being in ASTM has exposed me to other aspects of this field that I love. And it's really allowed me to, again, personally grow and I feel like I am a much more well-rounded sensory scientist because of my participation in ASTM and viewing all of these standards and hearing about work that sensory scientists do around the world.

Dave Walsh (30:45):

Well, I think we're getting to the end of our time at this point, but I, I feel like we could pick any application of sensory evaluation and do a whole event on it, like especially the deodorant standard and just really gross everyone out. <laugh> But unfortunately, that is really the end of our event. So we thank you for your time today, Ali. We really appreciate it.

Ali Schultz (31:02):

I had a great time talking with you. This was so much fun.

Dave Walsh (31:07):

If you wanna learn more about any of the standards discussed in this episode, visit astm.org for all the latest. And if you enjoyed the show, remember to like and subscribe so you never miss an episode. I'm Dave Walsh, and this has been Standards Impact, presented by ASTM International.