Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: MP3 decoders test : MAD isn't so good! (for me...) (Read 93091 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MP3 decoders test : MAD isn't so good! (for me...)

Reply #75 –
Garf: Nope Mr Lavry doesn't suggest listening to dither at levels where you just start to hear it. Quite the opposite. Since, as Mr Lavry says, the threshold detectibility curves are completely different for quiet and loud levels --> "Noise shaping listening tests must be done at "reasonable" volume levels."

I'd hate to appeal to authority, but do you know who Dan Lavry is? If you don't I recommend spending some time at the forum I linked to above. Regular posters include Mr Lavry himsel and also Bob Katz and Nika Aldrich (it's a scientific and objectivist forum).

MP3 decoders test : MAD isn't so good! (for me...)

Reply #76 –
Quote
Garf: Nope Mr Lavry doesn't suggest listening to dither at levels where you just start to hear it.


Where on earth did I say this?

MP3 decoders test : MAD isn't so good! (for me...)

Reply #77 –
Garf, answer to your question:

Quote
If I get your proposal, you want to evaluate the SNR gain in the frequency domain, and then turn up the dither as loud as possible while still being inaudible.

This sounds great in theory, but you might run into practical issues. You'll end up blasting VERY LOUD HF noise through the reproduction system, and not all of them will like that. Think blown tweeters, so be carefull


Furthermore, you also state that it isn't right to criticize the test as flawed unless you come up with a better test. I suggest you read what Mr Lavry wrote again since that is exactly what he has done.

MP3 decoders test : MAD isn't so good! (for me...)

Reply #78 –
Quote
Garf, answer to your question:

Quote
If I get your proposal, you want to evaluate the SNR gain in the frequency domain, and then turn up the dither as loud as possible while still being inaudible.

This sounds great in theory, but you might run into practical issues. You'll end up blasting VERY LOUD HF noise through the reproduction system, and not all of them will like that. Think blown tweeters, so be carefull

[a href="index.php?act=findpost&pid=356086"][{POST_SNAPBACK}][/a]


Gah! I'm not talking about the volume level in the test there, I'm talking about TUNING THE NOISE THE NOISESHAPER ADDS. That's exactly why it says "while still being inaudible" and not what you say.

Quote
Furthermore, you also state that it isn't right to criticize the test as flawed unless you come up with a better test. I suggest you read what Mr Lavry wrote again since that is exactly what he has done.


He's doing exactly the same thing! He warns against the effect changing the volume level has on the shape of the audibility curves, and that too large volume levels hence invalidate the test. Spoon already warned against this and I agree with the principle. My previous posts elaborates on that; I explain that there is a smooth tradeoff between making the testing easier and getting a large error on the curve you will end up with. Furthermore, practical limitations of playback material might cause limits contrary to what a theorethically optimal test or dither design will produce. The dither you design should be resistant against those, or it's not going to be very usable in practise.

Clearer now?

Edit: PS. I say it seems they also turn up the volume, just not to an unreasonable level, because of this quote: "The listener must resist the temptation to turn the volume up to unreasonable levels." They could have said "to turn the volume up. PERIOD", but they didn't.

MP3 decoders test : MAD isn't so good! (for me...)

Reply #79 –
Well I don't know what you are arguing then. I stated that Guru's test is flawed, which you apparently agree with, so what exactly are you in disagreeance with?

MP3 decoders test : MAD isn't so good! (for me...)

Reply #80 –
Quote
I'm getting a 403 Forbidden response from http://www.foobar2000.net/.

Is this just me, or does Guruboolez need to kick his server administrator?
[a href="index.php?act=findpost&pid=356024"][{POST_SNAPBACK}][/a]

I'm getting it too.

MP3 decoders test : MAD isn't so good! (for me...)

Reply #81 –
Thanks for the confirm.  I thought it was just me for a while.

Unfortunately I think guruboolez has other things on his mind. 
I'm on a horse.

MP3 decoders test : MAD isn't so good! (for me...)

Reply #82 –
Quote
Well I don't know what you are arguing then. I stated that Guru's test is flawed, which you apparently agree with, so what exactly are you in disagreeance with?
[a href="index.php?act=findpost&pid=356100"][{POST_SNAPBACK}][/a]


I compared it to saying Newton was wrong because we know what Einstein said. Yes, but...

We can work with a suboptimal, less accurate result if it allows us to do practical things easier. Of course, the theorethically correct method would be even better.

But that doesn't mean the original was "flawed" in as so much the the results are worthless. Just like the test you qoute isn't, although they apparently did adjust the volume. I would be impressed if anyone actually manages to ABX dither vs truncation at completely normal playback volume. It would be a stricter version of the "MAD Challenge" and even that is insanely hard to pass. Let alone repeat it to determine an optimal dither.

The tradeoff is "less accurate" vs. "more accurate". Not "fatally flawed" vs. "100% correct".

Given that auditory capabilities change even between humas, you won't get a 100% method, anyway. Might as well call any test "flawed" then. But I will ignore you and make something that demonstrably works, despite being based on slightly inaccurate results

MP3 decoders test : MAD isn't so good! (for me...)

Reply #83 –
Quote
The tradeoff is "less accurate" vs. "more accurate". Not "fatally flawed" vs. "100% correct".


Nobody has said anything about fatally flawed. Just flawed. Huge difference.

When it's all said and done more professional people have conducted solid ABX test evaluating the efftects of dither, come to conclusions, while also recommending guidelines for how to evaluate dither. Guidelines which says a big no-no to tests such as performed by Guru.

The scientific minded audio world does not begin or end with HA.

MP3 decoders test : MAD isn't so good! (for me...)

Reply #84 –
Amplification of low-level dithered and requantized test signals up to clearly audible levels is useful for checking if the dither is properly eliminating quantization distortion and does not produce noise modulation. And is useful for this whether the dither is flat or noise shaped. In this context, MAD dither amplitude was (last time I checked) insufficient because it caused both artifacts, and this could be checked by the mentioned amplification procedure.

Said this, I find useful the procedure of comparing just dither noise audibility (I mean dither without any test signal, that is, dithered "silence") at barely audible levels, given that there is little background noise and dither noise levels are low enough. If the noise cause by one of the dithers at test is barely audible and the other is not audible, then the one not audible is better for sure, under those conditions.

For tests at real world conditions, when not applying extra amplification in order to make dither noise audible, if no one of the dithers is audible by itself... then it really doesn't matter.

MP3 decoders test : MAD isn't so good! (for me...)

Reply #85 –
Sorry for delay.

Quote
Btw, it appears that your testing procedure for dither evaluation is flawed.
http://recforums.prosoundweb.com/index.php.../48/0#msg_num_1
[a href="index.php?act=findpost&pid=355762"][{POST_SNAPBACK}][/a]


Btw, it appears that your testing procedure for dither evaluation is flawed.

A listening test performed on unusual (and impracticable in real life!) condition is inherently flawed  In my case, I could only conclude that dither x is better than dither y on the tested conditions: over-amplified playback of ultra-quiet recordings. I can’t extrapolate the result to normal playback condition: conclusion won’t be valid anymore I suppose. The practical purpose of my test is therefore close to zero. I did it by curiosity, and because it was for me the only way to evaluate the performance of different MP3 decoders. My poor hearing abilities don’t allow me to differentiate MAD from another MP3 decoder (excepted the reputed Nitrane?). And even with these extreme testing conditions, the test was difficult to perform (my audio hardware is rather inexpensive and my amp produce a strong amount of noise at high volume).

BTW, I’m amazed to see that some people are able to make a distinction between different dithered noise shaping technology on 16 bit or more material. I haven’t reached this state – far from it!