Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Lossless vs. Redbook tests? (Read 116553 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Lossless vs. Redbook tests?

Reply #25 –
Quote
Is it necessary to be so insulting?

Maybe, I'm not sure whether or not he's really got the message already. What I'd like the OP to do is to really take my claim seriously for a moment and try to understand that it's exactly equivalent to the claim that he is making about lossless music. If it helps him to understand then I think it's a good thing.

Lossless vs. Redbook tests?

Reply #26 –
Quote from: uart on
Quote
Is it necessary to be so insulting?

Maybe, I'm not sure whether or not he's really got the message already. What I'd like the OP to do is to really take my claim seriously for a moment and try to understand that it's exactly equivalent to the claim that he is making about lossless music. If it helps him to understand then I think it's a good thing.


uart, I think it's pretty safe to say that you're the one who's not seeing the big picture, as I have already mentioned earlier.

There are serious epistemological issues to the entire concept of testing different lossless codecs. You can't make the case based on theoretical impossibility in a situation like that. You have to argue outside the box.

Frankly, I think at least 80% of the posters in this thread simply are not comprehending the question. That ain't TOS2, that's the bare truth.

Lossless vs. Redbook tests?

Reply #27 –
Quote from: 2Bdecided on
Well, on really really bad old on-board sound cards, you can sometimes hear what the CPU is doing. Literally. You can hear when it's idle, when it's not doing much, and when it's working flat-out. It creates different sounds, distortion, or interference on the audio output.

You can actually hear it on most on-board sound cards if you have a powerful graphics card. It's happened on all my desktop builds, and I had to buy an external USB card to fix the problem.

Lossless vs. Redbook tests?

Reply #28 –
Quote from: AV-OCD on
My motivation is this comment made in that other forum:

"I know of some very knowledgeable engineers who have measured differences between Apple Lossless files and AIFF files. They measured the files, once a sonic difference was heard in an A/B comparison, in the first attempt to quantify the difference. The measurements were carried out on several computers with some of the best test equipment available."


I kinda wanted to know the quote location, and Google is my friend. Glad to see another Computer Audiophile exile! Bolds are mine.

Quote
Hi WoodsDweller - I don't agree with your above statement. I'm not trying to start a flame throwing thread here, just relaying the information I have.  I know of some very knowledgeable engineers who have measured differences between Apple Lossless files and AIFF files. They measured the files, once a sonic difference was heard in an A/B comparison, in the first attempt to quantify the difference. The measurements were carried out on several computers with some of the best test equipment available. I'll try to get one of these people to post in this thread so it doesn't look like I am making things up :~) Please keep in mind that the goal of the engineers I know was to explain the sonic differences they heard between lossless compressed files and uncompressed files. No agenda is being pursued. They are just asking why and trying to show a measurable & repeatable reason.

I have yet to see anyone claim that a computer can't uncompress a music file fast enough or anything that relates to speed of decompression. I believe the common theories are more related to the decompression of music in real time and decompression errors, not a lack of computing power.

__________________ Chris Connaker

Founder
Computer Audiophile





Quote
For example when opening a compressed / zipped word document once in a while there is a hiccup. I wouldn't call the hiccup a big rather it's just something that happens because we're using PCs with many things going on. I'm guessing your still not going to buy this approach and I'm totally cool with that. I just like discussing the topic. As long as everyone is cool with each other we can all gain something.


Quote
"Do what makes you happy!" Ah, now we're talking. Back to enjoying this wonderful hobby whether it's 8 bit lossy compressed or 24/192 DDP files from the mastering house! As Cheryl Crow says, "If it make you happy, it can't be that bad.'


Whatever.






Lossless vs. Redbook tests?

Reply #29 –
Quote from: AV-OCD on
That's where the speculation is headed in the thread on the other site, that the data is all there bit for bit, but there may be something happening in the decompression of the file.

Causality! Seems like many audiophiles have a problem with it.

Lossless vs. Redbook tests?

Reply #30 –
Quote from: Axon on
Quote from: uart on
Quote
Is it necessary to be so insulting?

Maybe, I'm not sure whether or not he's really got the message already. What I'd like the OP to do is to really take my claim seriously for a moment and try to understand that it's exactly equivalent to the claim that he is making about lossless music. If it helps him to understand then I think it's a good thing.


uart, I think it's pretty safe to say that you're the one who's not seeing the big picture, as I have already mentioned earlier.

There are serious epistemological issues to the entire concept of testing different lossless codecs. You can't make the case based on theoretical impossibility in a situation like that. You have to argue outside the box.

Frankly, I think at least 80% of the posters in this thread simply are not comprehending the question. That ain't TOS2, that's the bare truth.



'serious epistemological issues'
Aren't you two days late with this? 

Lossless vs. Redbook tests?

Reply #31 –
Quote
My Experience

Larry,

Basically it has been found that it does not matter what type of computer or operating systems. Flat PCM files like WAV and AIFF will always sound better than Lossless files (FLAC, ALAC=Apple Lossless, etc...).

Why? I don't know it really does not make any sense. We have tested the file integrity and they all match up, but there is a difference when you hear it.

The good news is that you don't have to rip all the files again. The Lossless as I said are bit copies and can be converted to AIFF without loss of integrity.

I am working on a test bed to determine bit accuracy at several levels as well as sonics using some testing files I create and will post some results when they are finished. Problem is I have never been busier and testing like this can take weeks worth of time.

I did some THD measurements and most of them were only a few percentages different than their Lossless companions. But the levels were so low that I don't think that is an aspect of what we are hearing.

Thanks
Gordon
__________________

J. Gordon Rankin
~~~~~~~~~~
Wavelength Audio


 

and immediately,  posters there are taking this clown's utterly unsupported or verified claims as 'truth'  - because hey, he runs a tube amp company!

THIS is how audiophile mythology gets started and propagated...from 'on high'.  Watch for the 'EVERYONE KNOWS that lossless compressed files sound worse than .wavs'  meme to appear in Stereophile and TAS soon.

Lossless vs. Redbook tests?

Reply #32 –
I seem to recall JA being pretty skeptical about the bits != bits thing (if only in the context of spreading lies about MP3).

Anyways, we can laugh at Computer Audiophile and its posters all day, but that's not going to do much positive in the context of this thread. If we're that up tight about it one of us should try posting there (and risk the ban). I think it's important to focus most on the important issues here: how much a test of this could accomplish, how much should people trust their own ears, etc. Of course, that last point has been driven to death on this forum.

Lossless vs. Redbook tests?

Reply #33 –
whoo dogies, I and a bunch of other people got snagged on a private email thread with that gordin rankin guy and his 'thd studies'.  the nonsense was unbearable.  there's a quick 2 step process to address this:

1. quick, simple explanation of how lossless works, and possibly double-blind methodology for demonstrating audible differences

2. move on.  the people who get #1 will get it and the audiophools will claim the rest.  don't worry, most people actually do get #1

resist the urge to explain it further to people who don't get #1, the nonsense just explodes and the thread gets huge.

Lossless vs. Redbook tests?

Reply #34 –
I don't think there is anything more to be said. Well-put, Josh. Thread closed.

Lossless vs. Redbook tests?

Reply #35 –
I couldn't let this one get away, so after some extremely long missives with AV-OCD and Canar, I'd like to put them together and explain why Josh's (correct!) message is not the last word on this.

AV-OCD is asking an astonishingly good question for a new user: If this is a forum where blind testing is the gold standard, why is one of the most widely assumed beliefs (bits is bits) not backed up by blind testing?

Virtually all of the responses (besides my own, *cough*) responded by demonstrating the impossibility of the lossless!=WAV argument on theoretical grounds. Which everybody, including the OP, agrees with. But strictly speaking, those answers are wrong. He's not asking whether or not it's true. He's asking, why are there no tests? In an extremely literal manner of speaking, theoretical demonstrations of impossibility are not acceptable means of argument according to TOS8: "Acceptable means of support are double blind listening tests (ABX or ABC/HR) demonstrating that the member can discern a difference perceptually, together with a test sample to allow others to reproduce their findings." I don't see anything there about theoretical derivations, and even less than that about derivations based on computer engineering. The OP was, quite simply, taking TOS8 at face value, and everybody else misinterpreted it.

People are only calling this question stupid because, frankly, they do not comprehend it. It's a common human response. "This topic makes no sense to me, therefore it must be stupid/crazy/fraudulent." It's something I have come to expect from other forums, but certainly not this one.

jcoalson's response was that those people are never going to be convinced, and that one should just focus on logically communicating to those who think logically. I do agree with that, but not necessarily that they're crazy or stupid or even fraudulent (although people like Kait and Bybee definitely seem to fall into the latter camp, and I can rattle off a lot of people who fall in the first two). What all of this boils down to is: how fallible is your personal experience? If you dogmatically hold that it is infallible, basically, nothing's going to help you, and you will have to invent an extremely elaborate philosophy to explain away what you perceive. Hence beliefs like lossless != WAV when you somehow got some magic fuzzies in your stomach when you listened to uncompressed music. And beliefs like power cables, etc. If your perception is fallible, a certain reliance on numeric and blind testing is completely natural and logical, in order to come to firm conclusions and discuss issues rationally with other people. Hence, HydrogenAudio.

---

But none of that actually answers AV-OCD's question. Here's my answer: nobody here sees the point to testing this. Everybody here fully expects that sort of test to have only one answer (no difference). This expectation, as has been plentifully demonstrated in this thread, is backed up by a preponderance of theoretical and logical evidence. If even the theoretical possibility of this happening is eliminated, that puts quite a damper on anybody actually wanting to spend time investigating it. There is simply too much cost for too little reward. And again, that's not a bad thing. I mean, what would you think would happen if you asked a scientist to test beyond the shadow of a doubt whether or not the earth was flat? Or if creationism was true?

At some point, you're going to wind up wrestling a pig. Whatever result you come up with - and you're damn well going to predict the result in advance - "the opposition" will come up with an ad hoc hypothesis to perfectly explain it. That's exactly how so many objections to ABX testing even exist in the first place - they gave results that audiophiles did not agree with, so, they trumped up statistical errors, or invented errors out of whole cloth, to let them "logically" dismiss the test as meaningless. So, not only are we grappling with the fact that we're testing something we almost certainly know the answer to already (so we don't expect much reward from the test), the test must be large to be particularly statistically valid (so the cost of the test is very high) - it's quite likely that such a test result will not actually convince anybody. That is, in the event of an ABX failure, the reward for conducting the test is zero. That's a pretty convincing reason for me not to try and run a test like that!

It's important to note that reliance of blind testing only extends as far as people expect it to be necessary. This exposes a certain human imperfection to the whole science, but it's wholly natural. "Real" science doesn't have up-to-date, valid results in every field of study, and especially not for the principles considered elementary or patently obvious or axiomatic or what have you. That doesn't mean that science is wrong in any way, but it does mean that philosophies of science can get more complicated than people give them credit for. We don't go around requiring ABX results to verify that volume controls work. That doesn't mean we're not being objective about it; we just simply don't see the point of testing that. But there certainly can be a point for, say, seeing if a 0.01db volume change is audible. The same logic applies with lossless vs WAV. Any logical analysis of computer playback from a hardware and software standpoint (assuming the usual caveats about properly functioning computers, sound cards, etc) will state, unequivocally, that there is absolutely no stage in the file decoding and DAC process where such a difference could exist. The CA guys will of course claim there is some kind of difference in there - but they sure as hell can't offer a logical explanation for it!

When you get down to it, it really is a lot like the debates over the Nyquist-Shannon Theorem (of which many have been mentioned here in the past). That theorem proves, unequivocally, that analog audio can be encoded digitally with virtually no distortion (the distortion mechanisms that do exist are shown to be so trivial in modern systems as to be nonexistent). But that doesn't stop a lot of audiophiles from claiming that Nyquist was wrong, because their ears obviously tell them otherwise. And people do blind tests revolving around aspects of the analog<->digital process... and they fail. Almost every time (and the times they do pass always relate to serious faults in the test). At some point, you've gotta just back off and make sure everybody else knows how wrong these people are. Which was exactly the conclusion reached by jcoalson (and Canar).

Lossless vs. Redbook tests?

Reply #36 –
Axon, this is getting silly.

If someone comes along wondering whether or not there are people who can breathe under water, do we need to rally everyone round to conduct a test to prove in a statistically valid way that all our members cannot breathe under water?

The problem I had with the OP's original question is that a step was missing.
a) OP reads something and is inquisitive (no problem)
b) OP immediately looks for evidence / research to confirm or deny this.

What's missing is the OP's experience. If the second step had been "well, can I hear a difference?" then the OP's question would have been what's the correct method for testing FLAC vs. WAV and everyone at HA would have said ABX with helpful links to foobar's ABX comparitor and a warm welcome to HA.

I think the reason for the strong opinions on this is simply that people at HA are used to doing ABX tests and thus are fully aware of the futility of testing what is essentially WAV vs. WAV. I guarantee that if the OP was used to conducting blind ABX tests the OP wouldn't have asked the question. Likewise, the OP can't breathe under water and is presumably not active on the Atlantis Forum asking whether this ability in humans has been properly tested.

So, I agree with you that all this theory is not the right way to go, but where I disagree is that the OP is the one that should be encouraged to engage in the practical: Do your own ABXing and start with WAV vs. LAME -V 0, and ask 2 questions:

1) Can I hear a difference?
2) Are the bits identical?

The OP's likely answer is NO, NO.

Then move to WAV vs. LossyWAV --standard, likely answer, NO, NO. Then the OP can decide whether or not he/she can be bothered to get to the one where the answer is NO, YES. In so doing they will have answered their own question as to why no one else thinks it's worth doing.

Now I'm off to set up a poll to see how many HA members can breathe under water, and will probably come to the statistically valid conclusion that listening to LAME at -V 2 makes you less likely to breathe under water.

C.

[EDIT: Added some stuff about Atlantis]
PC = TAK + LossyWAV  ::  Portable = Opus (130)

Lossless vs. Redbook tests?

Reply #37 –
I see Axon's point here though. This, to me, involves truth. It is true that the difference is not discernible. I am completely convinced of it. I can believe in and hold close to that truth, because its basis is in hard mathematical logic. If math fails me here, if logic fails me here, it will mark the first time that math has actually failed me. I have failed to comprehend mathematics for sure in the past, and by fixing my comprehensions I fix my understanding of reality.

This brings us to the question about the power of mathematics. Even mathematics boils down to a set of truths that are simply taken to be self-evident. What's the point? Carefully considering what you hold to be true, I guess.

Lossless vs. Redbook tests?

Reply #38 –
The reason for the strong opinions is because people don't want moronic topics hanging around, polluting the forum.

But if every thread has to devolve (i.e. TOS 5) into a conversation about whether certain people are nice or not, than being blunt is counter intuitive. 


BTW, I'm not sure why anyone hasn't mentioned this, but it is very possible that a CD and a wav or flac file ripped from that CD are ABX'able, when the CD has scratches that produce errors during playback, and those errors are recover-able through secure ripping. In this case the archive file would certainly sound better than the CD, since it wouldn't be skipping during playback.

Also, with regards to those "very knowledgeable engineers", if they encoded their ALAC files using ffmpeg, it is possible that they experienced the known length mismatch bug. But it's probably more likely they were blowing smoke out their asses.

However, both above examples are exceptions, and not the rule.
The rule is (as has already been tirelessly repeated): Lossless compression is lossless.
elevatorladylevitateme

Lossless vs. Redbook tests?

Reply #39 –
Quote from: Canar on
Even mathematics boils down to a set of truths that are simply taken to be self-evident. What's the point? Carefully considering what you hold to be true, I guess.

I believe that 0 = 0. I also believe that 1 = 1. Surely these are universal truths. Unless something is broken somewhere in the conversion or playback processes, a 0 will still be a 0 and a 1 will still be a 1.

Unless someone can come up with a convincing argument to prove to me beyond a shadow of a doubt that 0 <> 0 and/or 1 <> 1, I'm going to carry on believing that lossless = lossless. How can it be wrong to do so... or am I still missing the point? 

Cheers, Slipstreem. 

Lossless vs. Redbook tests?

Reply #40 –
Quote from: AV-OCD on
"I know of some very knowledgeable engineers who have measured differences between Apple Lossless files and AIFF files. They measured the files, once a sonic difference was heard in an A/B comparison, in the first attempt to quantify the difference. The measurements were carried out on several computers with some of the best test equipment available."


Hello AV-OCD.

Well, so far everyone told you what Lossless audio compression is and what it means - it's like winzip or winrar, but specifically made for audio data. You could as well compress audio with winzip or winrar - you couldn't play these files if you don't unpack them first, but contents of the file is not changed.
Accept that - it is proven theory and hifists are just scared of admiting that they (mostly) overpaid their equipment, so they will invent whatevers just to justify them. I know, I am battling with them at a regular basis 

But, what is of the most interest - those engineers you mentioned - how did they measure the files, and with what test equipment? Can you get the complete procedure so we can verify and repeat the test?
I guess I know the answer, but I will give it a shot
Error 404; signature server not available.

Lossless vs. Redbook tests?

Reply #41 –
@Axon: That last post of yours was a bit tough to read.. Anyway, let's think about it again:

If the hypotesis is that Lossless is unABXable, you cannot use a double blind test to proove it (a non existence of a difference in a single test cannot proove the non existence of a difference in the whole group)

if we took the opposite route, and said that lossless is ABXable, and searched for methods to find that this can never happen, we would be on mathematics again, because tests can only extend to their size (and thus being representative of just that reduced group). Doing a public listening test with ABX files with (suspectedly) all fails could be as much taken as a "probably transparent". Not as strong as the mathematically explanation of such.


So, since we have a mathematical explanation of why they are equal, we don't spend time on that, but instead on the other parts that can cause a difference (player/decoder, faulty hardware, etc..)

Lossless vs. Redbook tests?

Reply #42 –
If most (all?) people cannot even distinguish mp3 encoding from sources, why would anyone be able to distinguish lossless from the source?

Regarding the missing explanations on the setup where these "engineers" claim they hear a sonic difference, would it by any chance be possible that they knew the source playing and was fooled by placebo?
Can't wait for a HD-AAC encoder :P

Lossless vs. Redbook tests?

Reply #43 –
Quote from: odyssey on
If most (all?) people cannot even distinguish mp3 encoding from sources, why would anyone be able to distinguish lossless from the source?

Regarding the missing explanations on the setup where these "engineers" claim they hear a sonic difference, would it by any chance be possible that they knew the source playing and was fooled by placebo?

I think that’s true for the vast majority of “tests” done by these subjectivist audiophile communities.

Lossless vs. Redbook tests?

Reply #44 –
OK, let's ten or so of us post some ABX results here of flac vs. wav, as we do for mp3s.  Then no one can say 'there are no tests here', and all pedantic purpose will have been served.

Lossless vs. Redbook tests?

Reply #45 –
I apologize if this particular point has been done to death. I read a little here and there in the thread without seeing it, but the topic doesn’t interest me enough to plow through the whole thing. FLAC playback can indeed sound different than wav playback.

I FLAC encoded some CDs with the idea of listening, from the computer, while doing other things on the computer. It didn’t work out and I had to use the wav versions. No doubt the data was the same in both, but the process of decoding the FLAC in real time apparently was too much for this computer. It was [almost ok, but every once in awhile there was a “millisecond” glitch, or several in a fairly short stretch, that eventually got to be quite annoying.

Considering what I’ve been subjected to in enduring some other people’s mindless music playing, this problem was pretty mild. I suspect, considering what they put up with regularly, that some of those other people would never have noticed this problem. However, such a playback difficulty as I experienced could become much more severe with the computer’s attention even more heavily weighted towards other tasks so that noone could possibly miss it.

Decoding/playback quality is probably on a continuous gradient, depending on the decoding system’s capacity. Can we definitely say that it might not also be possible that the problem be so slight that one (or at least many people) might not be able to consciously identify it as such, yet be able to ABX the difference?

Lossless vs. Redbook tests?

Reply #46 –
Quote from: AndyH-ha on
Decoding/playback quality is probably on a continuous gradient, depending on the decoding system’s capacity. Can we definitely say that it might not also be possible that the problem be so slight that one (or at least many people) might not be able to consciously identify it as such, yet be able to ABX the difference?

FLAC playback is not very CPU intensive. What system did you have this problem on? I suspect things like bad drivers, bad decoder or maybe even background processes eating up all resources (malware etc.) to be more likely the cause of this problem. Also as I just mentioned, it might very well just be placebo. If you for some reason has let your mind to believe that flac playback has this problem, you may actually believe you hear this.

As said earlier in this thread, you cannot ABX FLAC against WAV, because the FLAC is decoded prior to ABX'ing anyway.
Can't wait for a HD-AAC encoder :P

Lossless vs. Redbook tests?

Reply #47 –
Quote from: Axon on
why are there no tests?


You can't prove the non-exitistence of a phenomenon. You can only prove its exististence. For the question at hand this means that an ABX test proving that someone heard a difference between FLAC/ALAC and WAV/AIFF would be needed. Since no one provides a test result where he or she heard a difference, I'd guess there is none.

Ask yourself two questions:

As a proband, could you fake an ABX test result if you don't hear a difference? (Answer: No, you cant')
As a proband, could you fake an ABX test result if you hear a difference? (Answer [may Obama forgive me  ]: Yes, you can.)

So I guess we'll have to wait for a successful FLAC/ALAC vs WAV/AIFF ABX test ...


Cheers
Thomas




Lossless vs. Redbook tests?

Reply #48 –
Quote from: krabapple on
OK, let's ten or so of us post some ABX results here of flac vs. wav, as we do for mp3s.  Then no one can say 'there are no tests here', and all pedantic purpose will have been served.

Great! All we need is an ABX tool that doesn't first decode to WAV

Otherwise, we need to think of something else. How about we pump out FLAC via S/PDIF to a S/PDIF input (or AES to AES), record it, then repeat the process for the WAV? Sample-align the recorded files, trim to identical lengths and run a bit comparator. If all other factors remain constant, the result should be sufficient. Anyone see any potential flaws with such a methodology?

Lossless vs. Redbook tests?

Reply #49 –
Quote from: Ron Jones on
Quote from: krabapple on
OK, let's ten or so of us post some ABX results here of flac vs. wav, as we do for mp3s.  Then no one can say 'there are no tests here', and all pedantic purpose will have been served.

Great! All we need is an ABX tool that doesn't first decode to WAV


Look, *I get it* -- the capillary-thin reed this is all hanging on is that on-the-fly decoding is degrading the sound, whereas predecoding before playback gives us identical wav vs wav comparison in our ABX, hence no difference.  2bdecided went over all that earlier...noting that if the former is happening *something's probably wrong with your playback setup*. 

D'you think these subtleties are really what animates those who are claiming audible difference on CA?  I don't.  You get the usual audiophile melodrama -- multiple people saying, in effect, "I switched back to wav and suddenly everything sounded *right* again!".  Gordon Rankin is already claiming that the wav *always* sounds better than the FLAC.   

Let's look at it another way.: 
Are mp3s converted to .wav audio by ABX software? 
-If so then virtually all mp3 ABX tests reported at HA are invalid, by the criterion being applied here.  Oh well, back to the drawing board!
-If not, then we already know that on-the-fly decoding of a LOSSY compressed file can be indistinguishable from source, in ABX tests, to most listeners, with most material.  So, is OTF decoding of FLAC not only worse than OTF decoding of MP3, but worse *enough*  to plausibly produce an audible difference -- and not just a rare one, but one that a half dozen people in a small sample on CA already claim to hear either all or a signficant part of the time?


Quote
Otherwise, we need to think of something else. How about we pump out FLAC via S/PDIF to a S/PDIF input (or AES to AES), record it, then repeat the process for the WAV? Sample-align the recorded files, trim to identical lengths and run a bit comparator. If all other factors remain constant, the result should be sufficient. Anyone see any potential flaws with such a methodology?


Expect some to say there's no analog stage, so you aren't really modeling what they're hearing.  You could do D->A->D on FLAC and WAV, and compare those, meanwhile also comparing them to the all-digital pipe you describe. Obviously there will be measurable differences between various pairs of such a comparison matrix, but their magnitude should correlate well to 'positive' or 'negative' ABX results.