Skip to main content

Topic: TAS at it again: bit-identical sources sound different. They just do. (Read 11602 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • krabapple
  • [*][*][*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Seen today on the Hoffman forum:

"In the latest Absolute Sound magazine there is an article on computer audio which claims (get this):

2 bit identical files on a hard drive, played back with JPlay (which reads hard drive to memory, then plays from memory), sound different. One file was ripped at half the speed of the other, both with JRiver, both confirmed with AccurateRip"

I haven't read the TAS article, but I followed up online and found one of its authors responding to queries about it, on another forum.  Here's what he says:

Quote
As one of the authors of this article, let me make a few comments. First we have taken some pains to describe our methods in detail in order to encourage readers to repeat our tests for themselves. Only then can they be in a position to dispute or confirm our findings. Opinions don't really count for much. Experimental results are more compelling.

In the quote cited by bramankp, we agree that one can make copies that are truly "identicle" but one cannot conclude that they therefore must sound the same. There are many factors that can change the sound between the information storage source and the DAC. For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different. Bits are bits all right, but time- and amplitude-based jitter mechanisms can account for many of the sonic differences we report.

To rossop: See Part 3 of our article in TAS #220 on FLAC. There are many sources and reasons why FLAC and WAV sound different, sometimes to a disturbingly large degree.


 

His other replies further down that thread are no more convincing,e.g

Quote
Lets make sure we are talking about the same experiment first. Using JRMC, under ripping options, there is a sub-menu for read speed: 1x, 2x, 4x, 8x, 16x (version 15 of the software). This is what we were testing. As an aside, it is an Interesting question to ask why the manufacturer includes this option in their software.  Second, our hearing sensitivity and acuity was highly conditioned after listening to the same test tracks over and over again for 8 months. Third, I have over 30 years of professional industry listening training. Fourth, our auditioning system was optimized with respect to power cabling, digital cabling, AC  grounding, vibration control, and speaker resolution. If your situation is not the same, then you have not replicated our experimental conditions exactly.  I contend that if the differences are hearable on any system, they are not meaningless, but rather indicate a real effect regardless of whether they can be heard in all systems or just select systems.  And the history of science and engineering includes many examples of disputed concepts being eventually proven correct. If you think the results in Part 2 are outrageously wrong, wait until you read what we have to say about FLAC files in Part 3.

On the other hand, there are criticisms that can be made of our procedures. For example, testing was usually conducted under single, not double blind conditions. Our results lack proper statistical rigor; generally speaking, we looked to replicate our results on two different systems and duplicate our tests at least once. It is also the reason why these results were submitted for publication to TAS and not the JAES. For the record, we did employ listening panels and numerous replications (at least 4) for what we felt were going to be the most controversial findings.  In the case of the particular experiment we are speaking about, similar results were obtained some time ago with an independent computer and software and at an independent time. Had we tried to repeat everything we did in triplicate with at least 2 weeks intervals between listening tests, we probably would not have lived long enough to complete this study, or at least would no longer be married.




As we enter 2012, I want to say Thank you, HA, for being an antidote to audiophile diseases like TAS.
  • Last Edit: 31 December, 2011, 03:15:33 PM by krabapple

  • benski
  • [*][*][*][*][*]
  • Developer
TAS at it again: bit-identical sources sound different. They just do.
Reply #1
Not that there's an ounce of sense in these people, but maybe someone should explain to them what disk cache is or a sound-card buffer?  It sounds as if they believe that files are read off the hard drive "just in time" and go straight from the SATA controller to the DAC 
  • Last Edit: 31 December, 2011, 04:09:43 PM by benski

TAS at it again: bit-identical sources sound different. They just do.
Reply #2
It may be useful to remember that these magazines and writers, for the most part, have a vested interest in the sales of "audiophile" equipment and recordings. In light of that, their views and beliefs can be seen as self-serving rather than simply delusional. That's not to say that they're not delusional; just that many of them profit from the propagation of these "beliefs."

  • Speedskater
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #3
Another audio forum has a thread on the magazine article:

http://www.whatsbestforum.com/showthread.p...d-music-reprodu
Kevin Graf :: aka Speedskater

  • Wombat
  • [*][*][*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #4
They just are starting a new market indeed and breed a new generation of Audiophiles!
Now there isn´t new hardware and gear coming into their labs, it is Software!!
People that buy these things and believe these esotheric feelings will need such tests in the future for telling them how it sounds.
Everyone that sees evidence in these reports will always want to know how the new versions of his beloved software sounds in contrast to others.

Happy newe year btw. and a big Cheers to all of you. May the new year bring you health and clean ears
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!

  • Goratrix
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #5
And another one of those threads over at the SH forum:

Where is the magic in a SHM disk?

  • db1989
  • [*][*][*][*][*]
  • Global Moderator
TAS at it again: bit-identical sources sound different. They just do.
Reply #6
Wow, this is absolutely terrible.

Quote
As one of the authors of this article, let me make a few comments. First we have taken some pains to describe our methods in detail in order to encourage readers to repeat our tests for themselves. Only then can they be in a position to dispute or confirm our findings. Opinions don't really count for much. Experimental results are more compelling.
As though they can appeal to the authority of science to make themselves sound trustworthy whilst flouting all of its proper procedures.

Quote
In the quote cited by bramankp, we agree that one can make copies that are truly "identicle" but one cannot conclude that they therefore must sound the same. There are many factors that can change the sound between the information storage source and the DAC. For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different. Bits are bits all right, but time- and amplitude-based jitter mechanisms can account for many of the sonic differences we report.
Notwithstanding that this assertion is mince, it is completely irrelevant. This study compared two files that were merely ripped at different speeds, then verified to be bit-identical before being played from the same disc through the same player. So, what possible relevance do any of these imagined phenomena have? Whence the difference? How can a difference emerge after bit-comparison, in response to a different ripping speed earlier in time? Do bits have memory like homoeopathic water? Just wow. How this writer thinks this is an adequate explanation is beyond me. (Then again, perhaps one can [pretend to?] believe anything when profit is at stake.)

Quote
Using JRMC, under ripping options, there is a sub-menu for read speed: 1x, 2x, 4x, 8x, 16x (version 15 of the software). This is what we were testing. As an aside, it is an Interesting question to ask why the manufacturer includes this option in their software.
Tinfoil hat nonsense. Something exists, so there must be a divine purpose for it, but one that is being suppressed by agenda-laden scientists! Nasty people!

Quote
Second, our hearing sensitivity and acuity was highly conditioned after listening to the same test tracks over and over again for 8 months. Third, I have over 30 years of professional industry listening training.
Appeal to authority, particularly distasteful in the second and more personal case, which just reads as Look at me! Please! and the obviously untrue implication that idiots and hacks cannot fashion comfortable and safe careers for themselves.

Quote
If your situation is not the same, then you have not replicated our experimental conditions exactly.
Malignant narcissism. We cannot be incorrect, naturally; so if you fail to agree with us, you are flawed. (Never mind that our experimental conditions are laughable.)

Quote
I contend that if the differences are hearable on any system, they are not meaningless, but rather indicate a real effect regardless of whether they can be heard in all systems or just select systems.
Whilst I doubt anything was heard, this is an astonishing abuse of reason.

Quote
And the history of science and engineering includes many examples of disputed concepts being eventually proven correct. If you think the results in Part 2 are outrageously wrong, wait until you read what we have to say about FLAC files in Part 3.
Turns out the Earth is actually flat after all!

Quote
On the other hand, there are criticisms that can be made of our procedures.
Where to begin?!

Again, just . . . Wow. I shall have to refrain from reading any more things like this (never mind the original article), for so many reasons.
  • Last Edit: 01 January, 2012, 10:57:09 AM by db1989

  • kraut
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #7
Quote
for so many reasons.


One of them to keep your sanity. Nice take-down
Quote
For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different.
.

Where does he get the data from for such a contention - his behind maybe

  • krabapple
  • [*][*][*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #8
Another audio forum has a thread on the magazine article:

http://www.whatsbestforum.com/showthread.p...d-music-reprodu



I see there that Clawson & Zeilig's article was rejected first by Stereophile for technical issues

  • JimH
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #9
While you guys get a lot of things right, maybe even most things, I think you occasionally underestimate the importance of opinion, equipment, and hearing.

I don't agree with all that the authors said in the article, but I think they honestly tried to be objective (if you can apply that word to something as subjective as music).  They spent about a year working on the article.  I talked with Chuck Zielig 10 or 15 times and he was generally very precise and demanding about details.

I don't think that Stereophile rejected their aticle for technical reasons (or for any other reasons).  I believe that the authors just got tired of waiting to hear from Stereophile and took their work to TAS.

I wish you a very Happy New Year!

  • kraut
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #10
While you guys get a lot of things right, maybe even most things, I think you occasionally underestimate the importance of opinion, equipment, and hearing.

That is exactly where the article seems to break down. The flawed methodology that actually ignores simple procedures that will ensure reproducible results that are worth further investigations, contentions that are just that without any evidence.
Opinions are dime a dozen and worthless without evidence.
I am of the opinion that stereophile is a worthless rag, and I have as evidence 12 years of collected bullshit in my attic. There is at least some evidence for my opinion.

Quote
For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different.

Without proper testing another bullshit statement that in this forum would not be permitted to be posted. Not even a cautious: might, no an outright will, as if they really had done any testing.
From what I understand - all they have done compare different sample rates, and nothing that gives them the right to the stated assertions.

  • db1989
  • [*][*][*][*][*]
  • Global Moderator
TAS at it again: bit-identical sources sound different. They just do.
Reply #11
While you guys get a lot of things right, maybe even most things, I think you occasionally underestimate the importance of opinion, equipment, and hearing.

I don't agree with all that the authors said in the article, but I think they honestly tried to be objective (if you can apply that word to something as subjective as music).

Could you explain, generally or with particular reference to this article, “the importance of opinion, equipment, and hearing”? Which parts of it are worth agreeing with? I’m genuinely curious.

In objective terms—because your asserting the subjectivity of music as art is irrelevant to discussion of sound as a physical phenomenon—the test is fundamentally useless because it clearly was not performed blind.

However, its participants managed to go beyond the otiosity of sighted tests to the absurdity of concluding that bit-identical files, played in precisely the same way, could nonetheless sound different because of the speed at which they were ripped—as though this can somehow introduce a departure from their binary identity after this being verified, by what supposed means I cannot even imagine. The madness of this line of reasoning is profound.

At this point, I almost feel as though I must cut sighted tests some slack! Despite being devoid of objectivity, unreliable at best, and typically useless, they do not necessarily lead to this kind of nonsense: that requires a truly wild imagination, wholly unfettered by objectivity and reason.

Honestly, I’m not going out of my way to be disparaging, but I cannot conceive of any way to rehabilitate this ‘study’. It was based on unscientific methodology, and the biased results thus generated were presented as evidence of a totally ridiculous conjecture.

I really can’t believe that someone can hold ideas such as these, never mind sell them to consumers un(der)educated in the most basic means of evaluating claims presented to them. I’m genuinely saddened by it. (A shame, then, that this is one of the less harmful abuses of reason in our world.)

  • krabapple
  • [*][*][*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #12
While you guys get a lot of things right, maybe even most things, I think you occasionally underestimate the importance of opinion, equipment, and hearing.

I don't agree with all that the authors said in the article, but I think they honestly tried to be objective (if you can apply that word to something as subjective as music).  They spent about a year working on the article.  I talked with Chuck Zielig 10 or 15 times and he was generally very precise and demanding about details.


But about statistics, not so much, apparently. 

Excusing that by writing  "this was aimed at Stereophile or TAS,  not intended for a journal like JAES" , as the 'HiFi Doctor' Zeilig did, is pretty weak.

Quote
I don't think that Stereophile rejected their aticle for technical reasons (or for any other reasons).


Partly true -- it wasn't literally rejected .  But it was offered to Stereophile, where it was not accepted outright, and that was due to 'technical issues' with the work....unless John Atkinson, Stereophile's editor-in-chief, was lying when he wrote this on WBF:

Quote
[The authors] offered this computer audio article to Stereophile but I felt there were some technical issues that needed to be addressed before the article could be published. Rather than address those issues, the authors withdrew it and offered it to TAS, where it appears to have been published as is.
  • Last Edit: 01 January, 2012, 09:56:47 PM by krabapple

  • Nessuno
  • [*][*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #13
Quote
For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different.

Without proper testing another bullshit statement that in this forum would not be permitted to be posted.


Oh, let's go, this statement is the easiest to test and indeed something everyone experiences many times a day using computers.
It's just like when you open the same text document reading it from your old and evilishly fragmented hard disk or from a new and heavenly formatted usb memory stick: the very words and sentences you read are different, try and see it with your eyes!         

Now the next audiophool quest will be: which is the best sounding filesysytem? And how much data journaling affects jitter? 
... I live by long distance.

  • Destroid
  • [*][*][*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #14
May I interrupt the logical arguments and ask if they bothered trying to burn it to vinyl first? Or better yet, burn it to water =D
"Something bothering you, Mister Spock?"

  • smok3
  • [*][*][*][*][*]
  • Moderator
TAS at it again: bit-identical sources sound different. They just do.
Reply #15
Quote
Do bits have memory like homoeopathic water?

  happy new year!

seriously: If bashing this kind of "tests" is the only thing that keeps hydrogenaudio still kicking, it is well worth it.
p.s. gotta find those golden cables now (left them on the roof freezing over night, so they will forget the music i was listening yesterday, hello!)
  • Last Edit: 02 January, 2012, 05:21:21 AM by smok3
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung

  • john33
  • [*][*][*][*][*]
  • Developer
TAS at it again: bit-identical sources sound different. They just do.
Reply #16
It just beggars belief that anyone with even a basic understanding of digital audio could even consider asserting that bit-identical audio files, played on the same equipment could possibly sound different no matter how the audio was originally ripped. I can't even imagine the mind set that would permit you to consider that there was anything to test, nevermind persuade yourself, and others that there was any difference to be considered. That he even states that he has "over 30 years of professional industry listening training" only serves to indicate how flawed that training must have been.

Anyway, just my 2 cents. I've been trying to resist making any comment here but have finally given in!

Happy New Year, everyone.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

TAS at it again: bit-identical sources sound different. They just do.
Reply #17
I don't think that Stereophile rejected their aticle for technical reasons (or for any other reasons).  I believe that the authors just got tired of waiting to hear from Stereophile and took their work to TAS.


While it it is true that it took me longer than the authors liked to go through their article - it was 16,000 words - I felt there were methodological problems and technical misstatements that would need to be addressed. I discussed these issues with Chuck Zeilig last May, but his reaction was to withdraw the offer to have the article published in Stereophile. It has subsequently been published in The Absolute Sound in its original form, as far as I can tell from the first two installments.

If you check with Chuck, he can confirm that the discussion with him, in which I let him know of some of my reservations, took place on Thursday May 26 and his withdrawal was on Friday July 1.

John Atkinson
Editor, Stereophile

  • JimH
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #18
Hi John,
I can understand why it took a long time to review the article.  It is big.  I also understand that it might have needed your editing attention.  In the end, it did get published and it will probably continue to be scrutinized and criticized for some time to come.

I know that Stereophile will continue to give computer audio close attention and I look forward to reading more there.

For those who haven't read the article, it's not just about how some zeroes and ones are better than others.  I think it will help convince mainstream audiophiles that computer audio has come of age.  Of course, if you're a forum member here, you've known that for a while.

No matter whether you agree with what audiophiles believe, I hope you will find it in your hearts to give them credit for their passion.

Jim Hillegass
CEO, JRiver

  • db1989
  • [*][*][*][*][*]
  • Global Moderator
TAS at it again: bit-identical sources sound different. They just do.
Reply #19
In the end, it did get published and it will probably continue to be scrutinized and criticized for some time to come.
But probably not to the degree that it deserves, especially not in its original location of TAS. If anything, it may just help to further entrench established (and baseless) prejudices against particular aspects of digital audio, and to add to the ranks of the misled (and the coffers of the snake-oilers).

Quote
For those who haven't read the article, it's not just about how some zeroes and ones are better than others.  I think it will help convince mainstream audiophiles that computer audio has come of age.  Of course, if you're a forum member here, you've known that for a while.

No matter whether you agree with what audiophiles believe, I hope you will find it in your hearts to give them credit for their passion.
Computer audio has come of age because nonsense like the above (which is merely one facet of the article, by the way) is able to be published and believed? Audiophiles deserve credit because they are able to say things like this and find an audience? Forgive me if I cannot make the logical connection in either of these examples.

Also, as per my above post #12, which you seem to have overlooked, I am still curious about how you would defend this article (beyond what seems to be your idea that any talk about audio is good talk about audio).

  • Deep_Elem
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #20
So when I copy audio files from one part of my system to another, e.g. a different hard drive, should I copy them at half speed or full speed in order to get the best sound?

Yes I'm kidding.

  • Juha
  • [*][*][*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #21
So when I copy audio files from one part of my system to another, e.g. a different hard drive, should I copy them at half speed or full speed in order to get the best sound?

Yes I'm kidding.



Loopback cable (DA-AD conversion) is worth to try (if those are CD rips). 


Juha

  • Speedskater
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #22
The same two authors did publish a 7 page article in the June 1986 issue of Stereophile.  The Stereophile editors did have a dis-claimer about the authors conclusions. The article's premise was using commercial recording of the same music in LP record, CD and cassette tape, determine which play-back system sounded best.
Kevin Graf :: aka Speedskater

  • Wombat
  • [*][*][*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #23
Me doesn´t know much but there are many that know nothing and use excactly such publications to argue.
I just was in contact with some audiophiles at diyaudio.com that use terms and examples of such reviewers and marketing to argue in the forum all the time. They just repeat what they read without understanding anything of it, leave alone being able to conclude anything themself.
These individuals must be leaded and most likely highly weclome such pamphlets as another prove of their "knowledge"
I learned the last days at diyaudio that 1-bit DSD sounds better because it creates REAL samples of music and linear filters are the most destructive thing that can be done to audio. I was prompted to practice, learn and finaly to use my ears!
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!

  • JimH
  • [*][*][*]
TAS at it again: bit-identical sources sound different. They just do.
Reply #24
Audiophiles deserve credit because they are able to say things like this and find an audience? Forgive me if I cannot make the logical connection in either of these examples.

They're saying what they believe to be true.  It's up to you to accept or reject it.  As I said, I don't believe it all.

Quote
Also, as per my above post #12, which you seem to have overlooked, I am still curious about how you would defend this article

I'm not defending the entirety of the article.  I believe they have a right to a fair and objective analysis of their work.  I think some of the people who are the most negative about it may not have read it.

Quote
(beyond what seems to be your idea that any talk about audio is good talk about audio).

I don't think I said that.  As I've said, I'm a scientist and an engineer.  I like proof.

On our forum, we occasionally encounter some questionable opinions about audio.  Here's an example:  "... ape sounds better than flac ..."

On some forums, this might be out of bounds, but we let this kind of discussion continue for a while.  It generally blows itself out.
  • Last Edit: 02 January, 2012, 02:16:50 PM by JimH