Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Lossless vs. Redbook tests? (Read 118230 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Lossless vs. Redbook tests?

Reply #50 –
Quote from: krabapple on
Are mp3s converted to .wav audio by ABX software? 
-If so then virtually all mp3 ABX tests reported at HA are invalid, by the criterion being applied here.  Oh well, back to the drawing board!

Yes mp3 are decoded to wav as well but they obvoiusly forget that ABX tests reveals ANY audible differences by the simple test. You can't find differences that may (not) be there.

So I suppose many users of at CA are probably fooled by placebo. They should really lookup this term before claiming one thing is better than another without conducting an objective and throughful test.

Quote from: krabapple on
Expect some to say there's no analog stage, so you aren't really modeling what they're hearing.  You could do D->A->D on FLAC and WAV, and compare those, meanwhile also comparing them to the all-digital pipe you describe. Obviously there will be measurable differences between various pairs of such a comparison matrix, but their magnitude should correlate well to 'positive' or 'negative' ABX results.

Now, don't give them ideas...

Just comparing SPDIF output is actually the best solution (imho) presented in this thread, but don't be surprised, when the bits still come out exactly as they originated

Edit: I take that back what I previously wrote... I didn't see the irony  Sorry krabapple, I was a little quick reading your post.
Edit 2: Cleaned up me mess...
Can't wait for a HD-AAC encoder :P

Lossless vs. Redbook tests?

Reply #51 –
Um, guys. Again, you are missing the point here.

My point was not that we should be testing this. It was that we don't test these things, and we shouldn't, for extremely good reasons. krabapple has reiterated all the obvious reasons (any test we could construct could be dismissed as artificial and/or improperly constructed). But this fact exposes a significant limitation in TOS8 which the OP picked up on.

Fundamentally, audio discussion on HydrogenAudio relies upon a solid basis of audio, electrical, and computer engineering. You can't convince somebody that something is impossible when they already know it is theoretically impossible! I think most of this topic could be resolved if we simply extended TOS8 to include, among allowable objective evidence, firm proofs of impossibility based on grounds of mathematics or engineering - where the statement of audibility explicitly excludes distortion effects that are assumed not to exist in the implemented system - power supply noise, broken cards/drivers causing skips, etc. need not apply.

There's obviously some leeway here as to what constitutes "proof", but it can't be that hard to show that foobar2000 will always get the bits out to the card in the same fashion every time, if it doesn't break catastrophically, right?

Lossless vs. Redbook tests?

Reply #52 –
The "proof" would be just like this:
WAV file is read into a buffer of PCM samples (step 1), then sent to a D/A converter with right timing (step 2).
Losslessly encoded file is read into an input buffer (step 0), decoded to a buffer of PCM samples (step 1) and sent to a DAC with right timing (step 2).
Because what we have at step 1 is strictly identical (by definition of lossless compression) and step 2 is always the same regardless of input format, the output is identical too.
Full-quoting makes you scroll past the same junk over and over.

Lossless vs. Redbook tests?

Reply #53 –
Quote from: AndyH-ha on
I apologize if this particular point has been done to death. I read a little here and there in the thread without seeing it, but the topic doesn’t interest me enough to plow through the whole thing. FLAC playback can indeed sound different than wav playback.


Well, if it truly does sound different, you should be able to ABX it quite easily.  Otherwise this is nothing more than a case involving the placebo affect (or what was previously mentioned).

Quote from: AndyH-ha on
I FLAC encoded some CDs with the idea of listening, from the computer, while doing other things on the computer. It didn’t work out and I had to use the wav versions. No doubt the data was the same in both, but the process of decoding the FLAC in real time apparently was too much for this computer.


I am not sure what kind of hardware you are running, what CD ripper you were using, or what settings you were using but I really do find this hard to believe.  I have an old Pentium II 266MHz computer with 128MB of RAM that can playback FLAC files (at the default -q value) without stuttering.  Granted, foobar2000 consumes all of the PC's resources but it can still be done.  RockBox can even be installed on older iPods running specs that are far worse and still playback FLAC files just fine.  So there is something that isn't right on your end, this does not sound like a problem with FLAC.  Someone already mentioned previous reasons why you could be having issues.

Lossless vs. Redbook tests?

Reply #54 –
Quote from: krabapple on
You could do D->A->D on FLAC and WAV, and compare those, meanwhile also comparing them to the all-digital pipe you describe. Obviously there will be measurable differences between various pairs of such a comparison matrix, but their magnitude should correlate well to 'positive' or 'negative' ABX results.

I don't like the idea of giving "them" measurable differences, as the common audiophile will extrapolate meaningless differences in measured output to "night and day" differences if it suits the agenda. If it can be demonstrated that what's being output by a sound card is identical regardless of whether an application is decoding FLAC into a buffer or just shuffling raw PCM into a buffer, that's as far as any test needs to go to satisfy the interest of the OP. The idea of recording the S/PDIF output is a little crude, yeah, but it's at least a "real world" scenario, and people tend to like it when anything moves out of the realm of mathematical "theory" into the realm of the real world.

Of course we all know what the end result would be of such a test, but if the test parameters are scientifically sound, I don't see any harm in performing it.

Lossless vs. Redbook tests?

Reply #55 –
Taking up Axon's line, at a purely theoretical level:

we know that lossless=lossless, but how do we know this? I mean, can they be proven to be lossless (in a computer science/mathematical sense of proven), or are they designed to be lossless and shown by test to be working as designed? I seem to remember some talk of testing in the early days of lossless, so it might be, finally, an empirical question.

Lossless vs. Redbook tests?

Reply #56 –
Yes, it can be proven that a lossless file is lossless using a mathematical procedure.  Simply do a bit-for-bit comparison to a PCM WAV version of the file (ie rip a track to lossless and then to PCM WAV).  I believe that is a mathematical method for comparing a lossless track and a raw PCM WAV track.

Lossless vs. Redbook tests?

Reply #57 –
I know that when you compare a WAV file with a file that has gone WAV -> Lossless -> WAV, they will be identical, but that isn't quite what I meant by proof. I'd call that an empirical test of the lossless nature of the process. What I meant by proof was being able to demonstrate, in advance, that the algorithms as implemented necessarily will do it right. Like the way the sampling theorem or Fourier transforms are mathematically guaranteed.

Partly idle curiosity, but linked to axon's interest in what should be tested, and what need not be tested. Sometimes the audiofringe says the scientific-minded people have closed minds: well, you have to close your mind to some things, but it would be nice to know what need never ever be investigated.

Lossless vs. Redbook tests?

Reply #58 –
Quote from: MichaelW on
I know that when you compare a WAV file with a file that has gone WAV -> Lossless -> WAV, they will be identical, but that isn't quite what I meant by proof. I'd call that an empirical test of the lossless nature of the process. What I meant by proof was being able to demonstrate, in advance, that the algorithms as implemented necessarily will do it right. Like the way the sampling theorem or Fourier transforms are mathematically guaranteed.

Partly idle curiosity, but linked to axon's interest in what should be tested, and what need not be tested. Sometimes the audiofringe says the scientific-minded people have closed minds: well, you have to close your mind to some things, but it would be nice to know what need never ever be investigated.


All the encoder operations need to be mathematical reversible if it's to be theoretically lossless. A lossless encoder, like any other piece of software, could have a bug in the program code. And yes one possible result of such a bug could be to give non-bit identical ouptut. Can any complex piece of software be shown to be entirely bug free at absolutely 100% certainly? Maybe not,  but since bit identical is pretty easy to test for, I think it would be very safe to assume that any well tested lossless codec would have a probabilty of very close to zero of harbouring this type of bug.

BTW. When I recently transcoded my 20GB of monkey's audio lossless to TAK lossless I did (out of paranioa) take a small random sample of files and test for bit identical decodes (APE->WAV compared with TAK->WAV). I'm happy to report that every file compare came back "no difference found".

Lossless vs. Redbook tests?

Reply #59 –
Quote from: Axon on
Fundamentally, audio discussion on HydrogenAudio relies upon a solid basis of audio, electrical, and computer engineering. You can't convince somebody that something is impossible when they already know it is theoretically impossible! I think most of this topic could be resolved if we simply extended TOS8 to include, among allowable objective evidence, firm proofs of impossibility based on grounds of mathematics or engineering - where the statement of audibility explicitly excludes distortion effects that are assumed not to exist in the implemented system - power supply noise, broken cards/drivers causing skips, etc. need not apply.


Seconded. 

Lossless vs. Redbook tests?

Reply #60 –
Don't know if any of you are following that CA thread, but someone there, in response to challenge from me to show some ABX results,  claims to have gotten positive AIFF vs FLAC ABX results 'about a year ago'. The result reporting is obviously not up to HA standard, but here's the kind of 'authority' we're dealing with:

http://www.computeraudiophile.com/content/...s#comment-16205


Quote
My test was done about a year ago for my own purposes, and not for anyone else's. I cannot remember what the exact source material was, but it was 24 bit FLAC downloaded from one site or another on the web. It might have been a Linn Sampler, but I just don't remember - I had no intentions of publishing the results because it was just a 'proofing' for my satisfaction. It is quite possible that you feel an obligation to accumulate and share this sort of thing, but in all probability I won't be reading that stuff.

From what I recall, and I know you won't be impressed, there were two files in question which we made three FLAC and three AIFF copies of (total of four X 2= 8)- differently named so that we'd know what was what. The guy that helped me put these into the library of VLC (I think it was) and scrambled them three times - once for each iteration of the test. A FLAC vs. PCM comparison is not easy on the Mac as we have no foobar-like playback software with its built-in A/B function. I used AKG 240 series headphones. RME software was used to check output levels. I believe that I correctly ID'd 20 of 24. I'm not really sorry that I cannot be much more specific than that as you have grown quite tiresome in your short time here so far - I feel like I'm baby sitting the nephews. I'm sure that we are the same for you. How about you changing your 'channel'? That way you can 'watch' what you want.

Lossless vs. Redbook tests?

Reply #61 –
I call BS on that person's claims (that is unless VLC was incorrectly decoding the FLAC files but I highly doubt that).  Additionally, I know that Mac OS X ABX software is out there so they could have done an ABX test instead of just having a friend switch between tracks.  Might as well ask the person to conduct their "blind tests" between PCM WAV and AIFF versions of the same file.  Throw in Apple lossless, TAK, and a few other lossless (but apparently aren't lossless for whatever "reasons") formats just to show that they aren't really lossless.

So I would brush off their post and not worry about it.  The whole "is lossless really lossless" topic has been discussed to death all over the internet.  For the most part, hydrogenaudio users show that lossless is really lossless while all these other audiophool websites think otherwise.  This is something that I just don't understand.  I heard these types of discussions back when Blu-ray and HD-DVD were new.  People were saying that the PCM soundtracks to the movies produced better sound than either the lossless Dolby TruHD or lossless DTS-HD MA masters.  The funny thing is that the PCM soundtracks are often mastered at a slightly higher volume producing the false perception of an increase in quality.  I showed people this by having a 30 second clip.  Both clips were PCM WAV files.  One of them was made from the lossless file (but maybe it isn't lossless  ) and another was made from a 96kbps iTunes mp3.  The source lossless file was mastered at around -95dB.  I then took the 96kbps mp3 file and increased it's volume to -100dB.  I could easily ABX the results when volume matched.  Anyway, I posted the two files and everyone said that the WAV made from the louder mp3 was superior to the other file.  I then matched the volumes and no one was able to readily tell the difference.  Some people said the lossless version was better while others said the mp3 version was better (of course these people refused blind ABX tests but I was still able to get my point across).

My main point is that this lossless vs uncompressed "battle" has been going on for awhile and I don't it is going to stop.  The use of louder multi-channel PCM audio with Blu-ray movies hasn't helped either.

Lossless vs. Redbook tests?

Reply #62 –
@uart
What I think you are telling me is that the algorithms in lossless encoding can be mathematically demonstrated to be reversible, and so lossless, and so do not need testing, but any actual implementation is subject to bugs, like all real-world objects. So lossless testing is about the implementation, which is what needs verifying.

I should have been able to work that out for myself, but I didn't, and thank you.

Lossless vs. Redbook tests?

Reply #63 –
Quote from: krabapple on
You could do D->A->D on FLAC and WAV, and compare those, meanwhile also comparing them to the all-digital pipe you describe.
That's a good idea in theory. In practice it's surprisingly hard to do. Because of the difficulty of synchronisation, it's difficult to be sure there's no difference, while it's also difficult to conclusively catch genuine differences.

I say "difficult" - it's not impossible. If you can make it work, it's a reasonable way of verifying that the PC hardware, drivers, OS and software are performing as expected.

Cheers,
David.

Lossless vs. Redbook tests?

Reply #64 –
Quote from: Axon on
Fundamentally, audio discussion on HydrogenAudio relies upon a solid basis of audio, electrical, and computer engineering. You can't convince somebody that something is impossible when they already know it is theoretically impossible! I think most of this topic could be resolved if we simply extended TOS8 to include, among allowable objective evidence, firm proofs of impossibility based on grounds of mathematics or engineering - where the statement of audibility explicitly excludes distortion effects that are assumed not to exist in the implemented system - power supply noise, broken cards/drivers causing skips, etc. need not apply.
I think this is implicit in the way we work here, but it would require great care to add it to TOS8.

We already do something similar - e.g. TOS 8 requires that we ABX - but for "obvious" cases (e.g. 32kbps mp3 vs CD!) we don't bother. What exactly counts as obvious? Well, that can be (and is) worked out within each thread - if anyone doubts it's obvious, ABX can be requested in that thread. There is no point trying to codify this in the forum rules. I think it would make the forum worse to define exactly where ABX is not required in the rules - it's better left on a case-by-case basis. It also makes for better, more enlightening discussions.


I think the same applies here - TOS 8 requires that we ABX - but for cases where there is no mechanism for a difference to occur, we don't bother. If anyone is in doubt, ABX can be requested in a given thread, and the reasons discussed.

I'd be strongly against having a list of cases where we don't need to ABX in the forum rules.

Obviously if a given case has already been discussed, a link can be provided to the appropriate thread, and if it re-occurs often it can even be placed in the FAQ / knowledge base.

But at the end of the day, ABX is the rule - when people don't need to do them, it's because they are convinced they will fail (or pass). Yet if we want to be really sure there's not something unexpected happening, the appropriate mechanism is still ABX.


I'm sure some people would have put "correct" dithering to 16-bits (of normal music) on the "no need to ABX" list - yet there are positive ABX results to be found here on HA. So, let's leave the discussion open. We'll learnt more, both about differences, and the absence of differences, that way.

Cheers,
David.

Lossless vs. Redbook tests?

Reply #65 –
That's really the strangest thread in a long time. How can even the less simple minded participants here stay serious for so long? What's next? Do we want to discuss wether the world is at max 7000 years old?

The whole thread is a joke. Look at the complexity involved in just an average computer (or even cd player), the hundreds of algorithms and protocols involved on the different physical layers. Look at how laughable measly 44100 16-bit samples per second look in comparison to the total data moved within the system on all layers - and that error free within 0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001 ranges. A sound card receives an identical bitstream from both flac and wave files and this bitstream is send over the same error corrected bus as banks, the military, and scientists rely on for billions of transactions each day. But our golden ear visitors can hear a difference!

Some understand that FLAC implies an algorithm and an that that's not equal to its implementation. Applause! Now just put this into relation to the other 99999 implementations involved just for writing this text (or implementing a redbook cd player).

There have been very precise and complete answers very early in this thread. That should be it. Don't argue with idiots.

Reminds me of the bikeshed.

Lossless vs. Redbook tests?

Reply #66 –
Quote from: rpp3po on
Don't argue with idiots.
Who is arguing? The OP has been absent for at least two pages. The subsequent discussion has been more interesting than the original (silly) premise.

You provided a link to this...
Quote
Code: [Select]
      +------------------------------------------------------------+
      |    Warning:  You have not read all emails in this thread yet. |
      |    Somebody else may already have said what you are about to  |
      |    say in your reply.  Please read the entire thread before   |
      |    replying to any email in it.                               |
      |                                   |
      |                 [CANCEL]                              |
      +------------------------------------------------------------+


Cheers,
David.

Lossless vs. Redbook tests?

Reply #67 –


Maybe you are right. I just ignored the thread from the beginning expecting the usual religious noise and then it popped back and back to the front page even extending to |1||2|. Then I just skimmed through it, found exactly what I had expected and some (insightful) lengthy comments from the usual suspects.

It was probably anger about myself that I read through all of this against knowing it better from the start and this feeling just motivated a short rant.

My association to the link provided was related to parkinson's law. For example, the concept of PCM can easily be understood by many people*. Now certain claims are made and criticized based on this simple knowledge, which all seem to be simple truisms in relation to this knowledge, but are plain BS in the bigger picture. Understanding fundamental differences between FLAC and PCM is a bike shed. But to get any audio out of this bike shed you need to build the attached aircraft carrier below first. And the design and implementation of the aircraft carrier will make a lot more difference than theoretically possible flaws within the FLAC implementation or its realtime decoding overhead.

That's a very loose transformation from parkinson's law, which rather talks about how it is easier to get an atomic power plant approved by a board of directors than a bike shed.

* Well, at least the general concept, not necessarily its implications (quantization noise, filtering, etc.), but that's a different matter.

Lossless vs. Redbook tests?

Reply #68 –
Well, I attempted a simple test yesterday -- one even more crude than my previous idea -- using the "What U Hear" recording feature on my Auzentech X-Fi, since I don't have a 75 ohm coaxial cable for a S/PDIF-to-S/PDIF connection on hand to route from my X-Fi's outputs into my Pro Tools rig. Assumably the "What U Hear" input is what the drivers intercept just before a stream hits the converters, so I figured that, if I configured it correctly, it would be an acceptable means of testing lossless vs. uncoded playback. I was somewhat wrong.

I set up the X-Fi to output ASIO via foo_out_asio in foobar and recorded the "What U Hear" twice -- once for a FLAC of Modern Guilt off of Beck's Modern Guilt album (which is apparently pretty complex at an average of 1043kbps for FLAC -8) and once for the WAV of the same track. The sound card itself was configured for bit-matched playback (not that it matters when outputting through ASIO), set to 44.1 kHz clocking (again...) and the recording options were set to 16-bit, 44.1 kHz stereo direct to WAV. I sample-aligned the resultant files and cut them to identical lengths, created a stereo session with the left channels of both files to left and right, inverted the phase on the right channel, set my master output to mono and bounced the output without dither into 16-bit, 44.1 kHz.

What I was left with was, either fortunately or unfortunately depending on how you look at it, dither. Even when the full path is 16-bit, the X-Fi's recording function apparently still dithers with slight noise shaping, which is something I actually wasn't really expecting (Creative doing something correct, I mean). Since dither's naturally just random noise not correlated between recordings in any way, phase-inverting doesn't really successfully eliminate it, so there it was. There was naturally no indication of any musically-correlated noise whatsoever. The noise maintained the exact same average level regardless of what was happening with the tracks from which it was derived, even during silent passages, which is perfectly typical and expected behavior of any non-auto-blacking dither.

When you compare the waveforms themselves at the sample level, you can of course note some slight differences -- some samples being one or two steps higher or lower in amplitude, but nothing potentially audible. The grand majority of samples between the FLAC and WAV recordings share the exact same amplitude.

This test certainly won't satisfy any voodoo-believing audiophiles, but it certainly satisfies me...not that I needed to be.

Lossless vs. Redbook tests?

Reply #69 –
You should set in bold letters that these differences are necessary artifacts of the DA AD conversion stage (at least in this routing configuration) and no slight differences between FLAC and WAV. If you repeat the same for WAV vs. WAV you will get the same differences. I'm sure you know that, I repeated this just for clarity.

Lossless vs. Redbook tests?

Reply #70 –
If the full path is 16-bit, then dithering is the wrong thing to do. There would be no reason.

If the input and output are 16-bits, but the path is a higher bitdepth which doesn't "know" the input is only 16-bits, then dither is fine.

I'm guessing if you ask for 24-bits, you'll get something more useful for your comparison.

(It's the action of "what you hear", rather than the comparison itself, that interests me! It means "what you hear" isn't bit-perfect - unlike, say, Total Recorder).

Cheers,
David.

Lossless vs. Redbook tests?

Reply #71 –
Quote from: rpp3po on
You should set in bold letters that these differences are necessary artifacts of the DA AD conversion stage
Why would "What you Hear" do D>A>D?

Cheers,
David,

Lossless vs. Redbook tests?

Reply #72 –
Quote from: rpp3po on
You should set in bold letters that these differences are necessary artifacts of the DA AD conversion stage (at least in this routing configuration) and no slight differences between FLAC and WAV.

My understanding is that there's no conversion stage with "What U Hear". As I've come to understand, it's simply intercepting the stream prior to being fed to the DAC(s). Unfortunately I couldn't find any concrete technical data as to exactly where "What U Hear" is pulled. I admittedly could be totally, flat-out wrong here.

Quote from: 2Bdecided on
If the full path is 16-bit, then dithering is the wrong thing to do. There would be no reason.

Granted, but it seems like the right thing to do if the source bit depth is unknown. Although the card is configured to 16-bit, that information may not necessarily be passed to the recording function. Rather than grabbing that information, Creative probably just designed the recording function to add dither regardless of how the rest of the variables are configured. So, it's just dithering full time just as a precautionary, lazy measure. It's both right and wrong, but at least fairly harmless given that it's not any kind of elaborately-shaped dither.

Quote from: 2Bdecided on
(It's the action of "what you hear", rather than the comparison itself, that interests me! It means "what you hear" isn't bit-perfect - unlike, say, Total Recorder).

Maybe I'll repeat the test with Total Recorder. Would that work with ASIO, do you know?

Lossless vs. Redbook tests?

Reply #73 –
I got it wrong. I didn't even think about why in heaven a card would dither as default behavior when there is no intermitting AD stage. I don't consider this correct behavior. "What you hear" should be what you hear. If I was comparing different dither algorithms I don't want them to be mangled up by Creative's.

Still you will surely not have got different results for WAV/WAV vs. WAV/FLAC.

Lossless vs. Redbook tests?

Reply #74 –
Just out of curiosity, did you make sure that all of the various analog inputs were muted in the mixer? The "What You Hear" input on Creative cards will include line-in, CD, etc if they aren't muted in the playback Mixer settings. (Not sure if an analog input would look like dither or not)