Rebuttal to “Audiophoolery”

This is a rebuttal to Ethan Winer’s article “Audiophoolery” posted on the eSkeptic web site on January 6, 2010.

–

Dear eSkeptic staff and readers:

As a skeptic and an audio professional, I was disheartened to read Ethan Winer’s article “Audiophoolery” in the January 6th issue of eSkeptic.

Mr. Winer does make many valid points, though it should be noted that there is good science to dispute some of his assertions. But my primary objection is that he fails to be sufficiently skeptical.

There are two distinct schools of thought regarding the evaluation of audio equipment. One view, expressed frequently by Mr. Winer, is that everything we need to know about audio reproduction and psychoacoustics is already understood to a sufficient degree that we can measure any perceivable differences between different pieces of equipment. The other view is that there are perceivable elements of the listening experience that are, as of yet, not measurable.

Throughout my nearly 30 years working in various parts of the industry, I’ve had considerable exposure to both sides of this argument, and watched for decades as the two camps have fought it out repeatedly in publications, scientific studies and online discussion boards. The specific points of contention are too numerous to list here, but suffice it to say the debate is ongoing and lively.

As skeptics, our challenge is to not fall into the trap of arguing: “Everything there is to know about the topic at hand is already known, and since that set of knowledge does not include what you claim to experience, your claim cannot be true.” Such a position is just as dogmatic as the ones held by those making claims on the other side. Instead, we should be able to say, “What you claim to experience may or may not be true. Please design experiments using acceptable methodology to prove or disprove those claims and we will adjust our beliefs accordingly.”

There is a lot of ongoing research in the field of psychoacoustics, much of it contradictory. That fact alone clearly demonstrates that we don’t yet understand everything there is to know about how the ear/brain system works. Mr. Winer acknowledges this by stating, “it’s futile to claim I know what someone else can or cannot hear,” but then goes on to contradict himself with the contention that “there is no need to reproduce ultrasonic content because nobody can hear it,” and, “with all modern equipment, jitter is so much softer than the music that you’ll never hear it.” So which is it? Does Mr. Winer know what others hear or not?

There are unquestionably some wacky claims out there about audio equipment, especially high-end accessories. I have personally conducted hundreds of blind listening tests, many of which have demonstrated the manufacturers’ claims to be bogus. But I’ve also conducted tests where 100 percent of the subjects heard a consistent difference between two pieces of equipment that should have sounded identical. I cannot explain that result, nor can I determine what aspect of the signal was being transformed, whether one of the two pieces was in any way better, if the reason for the discrepancy had anything to do with one manufacturer’s claims for the product, or even if one of the test components was inadvertently affecting something else in the signal chain. All I can say is that there was a perceivable, repeatable and unexplained effect.

Our goal should be to explain that effect, not dismiss it as illusory. If we doggedly refuse to even acknowledge that there is something more to understand about how we experience music, we should expect that many such mysteries will remain unexplained.

The most glaring example of our lack of understanding is the poor state of audio reproduction today. There are almost no systems in the world that can fool the human ear, even the ear of an untrained listener. If the goal of audio reproduction is to faithfully reproduce the experience, we have failed miserably. With today’s understanding of human physiology and the technology that has resulted from that limited understanding, achieving the goal of a faithful reproduction, indistinguishable from the live performance, is nearly impossible.

When I engineer orchestral recordings, I often step out of the booth and listen to what the orchestra sounds like in the room. No matter how subjectively “good” my recording is, what comes out of the speakers sounds nothing like the live performance. Any attempt to fool a blindfolded listener into believing he was in the room with the orchestra would fail miserably. We’re just not there yet technologically, and with attitudes like Mr. Winer’s, we’ll never get there. There is still much work to be done in this field.

A suitable analogy for the high-end audio consumer is the wine connoisseur. Only a small percentage of the wine-drinking public would consider themselves connoisseurs, yet for that small percentage, there are magazines, reviews, ratings and endless discussions of quality and preference.

Wine connoisseurs have highly developed palates that can discern differences between vintages that most of us cannot. Logically, they are the people most likely to pay for high-priced wine.

When a novice is talking with such a person about wine, he’s far more likely to say something like, “I don’t know much about wine,” than to say, “You’re nuts… all wine is basically the same because I can’t taste the difference.” Yet with audio, that’s what many people do. First, they assume, often times erroneously, that they would not perceive a difference, and then go on to deny the sensory skills and adaptations that the connoisseur has spent years cultivating.

Another plight the audiophile community shares with wine aficionados is a problem of language. English lacks auditory adjectives. We have plenty of words to describe our visual and tactile experiences, relatively few to describe taste and smell, and almost none to describe what we hear. We’re forced to use visual adjectives to describe our auditory experiences, which is why descriptions of sound come out sounding so ridiculous.

When I’m speaking to another audio engineer, I can say “there’s a 2 dB bump at 5k” and he knows exactly what that sounds like. But to the layperson, or even an aficionado who has not spent years playing with parametric equalizers, that means nothing, so I would have to describe it as “somewhat bright” or “a little harsh.” Mr. Winer even succumbs to the same limitation when forced to describe harmonic distortion as adding a “slightly thick” quality.

Our descriptions aren’t as flowery as the reviews in high-end audio magazines, but it’s not difficult to see how a person whose job it is to describe something unseen can get carried away. Here’s a description cobbled together from a few recent wine reviews:

Boasts a beautifully pure bouquet of white flowers. Full-bodied, structured, and rich with ripe tannin, abundant fruit, and plenty of extract. Aromas and flavors of black plums and blackberries with hints of pepper, oak and dark cherries.

To someone who’s not a wine expert, that sounds ridiculous and pompous. But we don’t tell the millions of wine connoisseurs that they’re just imagining the differences. We don’t tell them that everything there is to know about the taste of wine can be measured in a lab, and because we cannot measure a difference between the various vintages, everything the connoisseurs are describing is just some kind of collective delusion. That would be pretentious on our part. Yet with audiophiles, that’s exactly what we do.

Skepticism is not about replacing blind faith in mysticism with a new blind faith in science, a field that has the constant evolution of knowledge as its central premise. We must be skeptical about the science too. The assertion that science has already explained everything there is to understand about any topic is itself anti-scientific, and not in any way skeptical.

For an example of misplaced faith in science, one need look no further than the collection of drugs approved and then withdrawn by the FDA after they were found to be dangerous. Pharmaceutical trials count among some of the most important experiments performed, yet under real-world conditions, the underlying science has repeatedly turned out to be fatally wrong.

That’s not to say science as a whole is bad or that we shouldn’t subject drugs to rigorous scientific testing. But it shows that the scientific method is fallible. To believe otherwise is to replace our skepticism with a new kind of faith in a different “higher power.”

Engineers say that one test is worth a thousand expert opinions. What continues to surprise me is the number of people who describe themselves as skeptics, yet still refuse to sit down and do their own listening tests, or at the very least, accept that there is something to learn from those who have. Rejection of evidence out of hand is not skepticism, it’s dogmatism.

Many years ago, before I got into engineering and production, I worked at an audio store. Some of our customers, usually the electrical engineers working for nearby defense contractors, would actually refuse to listen to the equipment before buying it, not believing that an audition would tell them anything they needed to know about how it sounded. To them, the spec sheet was enough to make a determination. Well, that’s fine if you’re going to take the gear home and hook it up to test equipment every day, but if you’re actually going to listen to music through it, you really should experience what it sounds like.

On the occasions that I could convince such people to listen to the equipment, it was amazing to me how easily some of them were willing to dismiss their personal experiences because someone had told them that those experiences could not possibly be. I have met so many highly educated engineers who outright refuse to believe what their own ears tell them. Instead, they’ll make up excuse after excuse of why it can’t be so. That tells me that these people are being grossly miseducated, to the degree that they cannot even acknowledge reality.

The whole point of audio equipment is to reproduce music for humans. Test equipment doesn’t go out and buy audio gear. Test equipment doesn’t enjoy listening to music. The idea that music reproduction equipment should be designed for listeners instead of test gear seems obvious to me, but it baffles many designers.

Nonetheless, there is a small cadre of designers who buck this trend, running every piece of equipment they design through a rigorous series of listening tests. For that daring approach, they are labeled snake-oil salesmen and their customers are discounted as fools. While it’s certain that there is a level of quackery in this field, that is not the field’s entire makeup. In fact, I’d say that the quacks are taking advantage of the groundwork laid by those who really understand what’s going on.

Interestingly, this kind of human-based research and development is not so cavalierly dismissed in other fields. Food scientists measure and test each new formulation in the lab, then pass it to a series of human tasters to determine what’s most marketable. The human is the final consumer, so the people chosen for the testing are representative of the market the product is destined for. High-end audio manufacturers use the audiophile community to evaluate their gear because that’s their market.

With respect to audio equipment, I’m skeptical about manufacturers’ claims, skeptical about magazine reviews, and skeptical about the engineering assumptions; but I’m also skeptical about claims regarding my ability to hear the difference between two pieces of gear, and most importantly, skeptical about any claimed ability to reproduce audio that’s indistinguishable from the source.

The only convincing information comes from a personal audition of the equipment in controlled listening situations, hopefully blindly. A choice based upon preconceptions, no matter what the source of those preconceptions, is a worthless venture. Moreover, preconception and closed-mindedness are not the skeptical path.

I submit that we need to be equal-opportunity skeptics. The kind of one-sided skepticism that seems increasingly popular these days is disturbing. It’s as if skepticism itself is being corrupted by the polemics of the non-skeptics. We should be able to rise above that. A true skeptic looks for evidence supporting all claims, not just those he is arguing against.