----- Original Message -----
From: "Richard L. Hess" <[log in to unmask]>
> Hello, Steve,
> I will tell you a little story about a bad test design and a good test design.
> The bad test started at a lecture I was giving. Part of my standard
> lecture shows "what is possible" at a given era by highlighting
> high-quality sound from a given era (going back to a 1935 steel tape
> copy). Someone asked about MP3 and I had one of the 1980s selections
> both in my demo as a WAV file and on my Palm T3 as an MP3. The Palm
> sounded way worse than the Sony CD walkman I was using for the rest
> of the demo.
> When I got home, I took the original file, made a high-quality MP3
> within Samplitude, converted it back to WAV and then cut between the
> two recordings. I now demo the cut recording and the MP3 is almost
> identical to the WAV file.
> So, just as with A-D and D-A converters and even CDs themselves which
> over time have, for some people, received the reputation of "not
> sounding good" for perhaps the wrong reasons. Clearly, here we were
> hearing the deficiencies in the Palm T3 audio system as opposed to
> the deficiencies in the MP3 format.
> I believe that my test is one of the few ways that one can do a test
> and remove most of the external variables. I'm passing this story on
> as an object lesson and as a caveat to anyone doing a listening test:
> make sure that you're really listening to what you think you're
> listening to and do NOT make assumptions. I believe that it is almost
> impossible to do the test that you describe using A/B hardware
> without the hardware differences influencing the rating of the format.
> Oh, and I emailed the organizer of the lecture this explanation and
> requested she mail it to all attendees. I think she did.
The harsh reality is this: most folks who visit such comparisons
"arrive at the front door" with their own firm (and prejudiced)
convictions...and are thus looking for anything that suppoprts their
The other harsh reality (which I know all too well) is that most of
the parties debating these details are at least into middle age, if
not past...and as such have lost a significant part of the hearing
they once had!
We might make it a requirement that anyone intending to pass judgment
on audio formats be required to submit a current audiogram of their
Steven C. Barr