Print

Print


Jon,

Well said.

The following AES pre-print from the 1991 (91st) convention is worth 
reading; I include three paragraphs below. --Marcos

TEN YEARS OF A/B/X TESTING
David Clark
DLC Design
Farmington Hills, MI 48335, USA
ABSTRACT
Experience from many years of double-blind listening tests
of audio equipment is summarized. The results are generally
consistent with threshold estimates from psychoacoustic
literature, that is, listeners often fail to prove they can hear
a difference after non-controlled listening suggested that there
was one. However, the "fantasy" of audible differences continues
despite the "fact" of audibility thresholds.
0 INTRODUCTION
Ten years ago the present author presented a paper to the
AES on double-blind testing using the A/B/X technique [1]. For
the next five years, a device to conveniently implement this test
was commercially available. It was thought by the author and his
associates that general use of this system would resolve "The
Great Debate" [2] of whether or not small differences in audio
components were audible. The debate rages on, however. It is
generally agreed by both sides that listeners fail to hear small
differences in double-blind tests, but some "believers" dismiss
the test as being inadequate to reveal what they hear clearly
when not being tested [3].
Questioning the test methodology is consistent with the
scientific method, particularly if it leads to a better test.
The problem here is that the challenge rarely leads to a better
test or vindication of the old test and acceptance of the
results. It could be argued that believing without scientific
confirmation is a harmless part of human nature. This is
acceptable for many of our personal choices from lucky charms to
comfortable furniture. Audio engineering, however, is the
business of delivering sound efficiently. It becomes an ethical
and perhaps legal question when it is claimed that improved sound
quality is delivered despite failure of tests to prove it.

Jon Noring wrote:
> Jerry wrote:
>
>   
>>> I think with experience, we are learning to spot the faults in digital
>>> recordings, which are not audible at first.
>>>       
>
>   
>> I do not dispute the fact that some can hear things that others do not. But
>> how much of this relates to the audio mikeing, mixing, amplification,
>> equalization, etc. process and how much to the digital part? Have those who
>> discern faults in digital recordings conducted blind comparisons of audio
>> and digital playbacks that originated from the same master?
>>     
>
> I was definitely a little concerned whether to bring this up because
> the views concerning digitization of audio, whether for or against,
> oftentimes become almost religious.
>
> And there are so many aspects to the digitization issue, it becomes
> difficult to sort out even the known facts based on objective data
> (which includes properly done listening tests.) And whether to go
> digital or analog depend upon the particular purpose. In this group,
> we are talking about the archiving of transfers from analog sources,
> such as LPs, 78s, magnetic tape, etc.
>
> As an example, in private email I was pointed to the 1984 Boston Audio
> Society A/B test, which I remember studying a number of years ago but
> forgot about until now. After reading it, it strongly indicated, but not
> without some controversy, that even with 1984 digitization equipment
> using 44.1/16-bit sampling, the "golden ears" could not discern any
> better than guessing between "A" and "B", similar to the thought-
> experiment I proposed in my prior message:
>
> A: LP/78 playback --> pre-amp --> amplifier --> speakers
>
> B: LP/78 playback --> pre-amp --> AD --> DA --> amplifier --> speakers
>
> (There are a couple things that could obviously be done with the
> Boston test to alleviate most of its controversies, such as using a
> computer, or a coin toss, to reassign which is A and which is B for
> each A/B comparison test, so as to remove the A or B preconceived
> bias -- that is, the person may, after a few listens, develop a
> preference for A or B for some subjective reason -- by mixing this up,
> and letting the listener know A and B *may* switch at random at every
> listen, the listener is less likely to develop a bias towards either A
> or B due to subjective reasons having nothing to do with the test.)
>
> I recall reading about a few other similar comparison tests done since
> then, and I've concluded to myself for the above A and B chains,
> that with today's much better digitization equipment, and the higher
> 96k/24bit sampling, that there will be no question NO ONE will be
> able to tell the difference between A and B by careful listening
> tests of actual audio material. (There might be faint discernment,
> though, of the difference in noise floors when there is no input
> signal. Inserting the A-D and D-A units in the stream adds a very
> small amount of noise in the analog sections that we don't have in
> the "A" system, which may actually favor "A". For archival purposes,
> though, for the analog option we'd insert an analog recorder and
> playback between the pre-amp and amp, and we know they will likewise
> add a small amount of noise. This suggests refinements to the A/B
> test.)
>
> Of course, this is an assertion of belief based on my interpretation
> of the data I've studied, so I propose this test be rerun, with
> changes to alleviate the criticisms of the Boston Audio tests. Then
> invite the "golden ears", especially those who believe digital audio of
> any kind creates artifacts that the trained "golden ear" can discern,
> to the listening, and see if any of the listeners can do better than
> pure guessing (i.e., to meet statistical significance) as to which is
> the full analog and which has the added A-D and D-A insertion.
>
> Of course, from the archivist's perspective, there are two choices for
> archiving:
>
> 1) place an analog recording system after the pre-amp
>
> 2) place a digital recording system after the pre-amp
>
> One thing I do know, the analog recording system will distort the
> signal (and add its own noise) much more upon recording than the
> digital recording system will, and in my view this distortion is
> unacceptable. It's bad enough that we have to deal with the vagaries
> of the analog transfer (cartridge and pre-amp) which itself distorts
> the signal from what was originally applied to the master.
>
> (As an aside, it's amazing that the significance of the obvious
> difference in "sound" between different cartridges among the
> pro-analog crowd is totally lost on them. If each cartridge sounds
> different, that's because each cartridge is distorting the audio in a
> different way. This is "distortion", not accurate reproduction. Sure,
> the "distortion" may be euphonic -- which I think contributes to the
> belief that analog "sounds better" than digital, but nevertheless it
> is an alteration of the analog signal which should be of concern to
> the archival community.)
>
> Thus, if the archivist has only one option for recording, they should
> choose professional-grade 96k/24 bit (or better) digitization with
> lossless compression. Doing both digital and analog recording, of
> course, makes sense, and should be done, but not for audio quality
> purposes, but rather for archival diversification. No one should do
> only analog archival recording since that will be of lower accuracy to
> the source due to inevitable distortion added by the analog recorder
> (and during playback as well) and as stated above (yes as a statement
> of belief) will not do any better, audio-quality-wise, than high-
> quality digitization.
>
> All in my rabid opinion, of course. <smile/>
>
> Jon Noring
>
>
> (p.s., just to note, I *love* the sound of good tube amplifiers, but I
> know the reason I love the sound is because of euphonic distortion. I
> have no difficulties with all the euphonic distortions added by analog
> equipment for *listening enjoyment*, but for archival purposes, digital
> is the only way to go as explained above. Now just tell those who love
> "vinyl" over CD that the reason they like the vinyl sound better is
> because of greater euphonic distortion from the whole analog chain of
> mastering the LP and the cartridge playback. They believe it is
> because the vinyl is more precise and accurate to the original... I
> remember a statement by a professional violinist who is also an
> audiophile, and he loved digital recording and playback because it
> is more accurate to the original -- he should know. He stated that in
> reality violins are pretty "harsh" instruments, and that the real
> character of violins can only be brought out by digital recording and
> playback. But most people prefer the sound of violins after having
> passed through a number of analog stages of recording and playback
> because of the reshaping of the sound by euphonic distortion. This to
> me is further evidence that as archivists we must be concerned with
> the most accurate reproduction of the original sound, and not be
> swayed by the crowd who believe analog systems are more accurate
> because "they sound better" -- they are not. One can always take the
> digital recording and upon playback run it through a euphonic
> "smoother" to recreate the sound of "vinyl" for those who just gotta
> have that sound.)
>