For example, Monday’s (3/8/10) show will be on the commercialization of outer space…and beer. On Thursday (3/11/10) we’ll take your questions on taxes…and beer. On Thursday, March 18th, we’ll talk with audio ecologist Gordon Hempton, about his book One Square Inch of Silence, and his search for areas of North America that offer completely uninterrupted natural silence (and beer). You get the idea.

While we’ve had a good chuckle about this proposed “beer factor,” there is something a bit more serious to consider – does it matter how many comments we receive during or after each Radio Smart Talk program? Should we consider a show with 52 comments a success, and one with only 6 a failure? Just how do we measure success for our program each day?

This was one of the quandaries I shared with students of Russ Eshleman’s commentary writing class at Penn State this past Monday. I was invited by Eshleman to give a guest lecture about how we prepare, produce and evaluate Radio Smart Talk, and what sets it apart from other talk shows. I talked with the class about how and why the show began, how we develop topics, conduct research, and decide on guests for each show. And then, we talked about evaluation. How do we know if what we did was useful? Informative? Valuable? How do we measure success? And, perhaps most important, (and something Eshleman asked me to discuss with the class) how do we actively ensure we don’t let the bias so pervasive in talk radio and television today creep into what we do?

Well, there are short-term and long-term answers to those questions.

In the short term, we have the daily feedback from you, the number and quality of phone calls, e-mails, texts, tweets, letters, and so forth. But if we only consider the short-term, using the above “beer factor” measurement as a guide, we’d conclude that while Tuesday’s show on beer sales (52 web comments) was a success, our conversation from 2/17/10 about whether we’re living in The Era of the Cheater (which yielded but four comments online) was somehow a failure. Not every on air discussion lends itself to an online conversation. And not every online conversation necessarily proves the success of the previous on air dialogue. This also doesn’t address the bias question – we could shirk our responsibility to objectivity and fairness and probably yield a lot of comments! But, of course, we won’t do that.

So, short term measurements are just not enough. We also look over the long-term – we still watch that listener feedback, but ask ourselves, in a larger sense, can we see a trend developing – are there more quality conversations taking place online associated with Radio Smart Talk now than there were, say, six months ago? We also gather ratings data from Arbitron (the company that measures ratings for the radio industry) that tells us whether more or fewer people are listening to the show day to day and week to week, from one year to the next. (I’m pleased to say the answer is more of you are listening – thanks for that!)

We also do consider things like sponsorship and fundraising, as potential ways, in the long-term, to measure whether people find what we’re doing valuable. And we’re pleased when we win awards, or receive publicity from other media about our show. And we do surveys, from time to time, to gather even more feedback. (Through the end of this month, you can answer some questions about Radio Smart Talk as part of a survey about 93.3 in Chambersburg).

The feedback we generate from all of these resources gives us a broad range of metrics to use to evaluate our success. And, if we ever somehow cross that line from objective reporting and interviewing to some sort of bias or agenda, it’s a guarantee those same resources (read: you) will tell us!

But there’s one final factor we consider when evaluating success – a gut check. We have a pretty good sense when we’ve presented a program that was useful, informative, relevant, valuable and objective. We also have a pretty good sense right away of how we could make the program better.

I hope the budding journalists I spoke with on Monday will constantly evaluate the work they do as they enter the news industry. I hope they’ll seek feedback from their listeners and readers. I hope they’ll also trust their gut to tell them when they’re providing a good public service, through objective reporting of news, and fair-minded interviewing, rather than succumbing to the trappings of in-your-face, biased punditry…and beer.

- Craig Cohen