I tried to compare different mp3 decoders, in order to evaluate the benefits of dithering (MAD and foobar2000) and noise shaping (foobar2000 only), against simplier decoders, as LAME or Fraunhofer (Winamp 5.0). I didn't hope anything first. But I've finally succeed with one sample, and decided then to resume the test with others samples.
I've wrote a complete review in french, but I can't translate it properly. In few words:
• For this test, I didn't reproduced real listening conditions. Difference are so subtle that I have to increase the level of my amp in order to maximise the audible difference.
• In order to make it possible without being completely deaf in few seconds, I've selected very quiet samples. With classical music, it's not really hard to find such samples (even full tracks are sometimes totally quiet). I didn't evaluate any decoders on pre-instrumental silence, but only when music was playing
Four decoders were tested:
• MAD, basically in order to evaluate the real performance of this mythic decoder, and technically in order to rate the benefits of dithering WITHOUT noise shaping.
• LAME, to mesure the difference between MAD and a simplier decoder.
• Fraunhofer: in order to evaluate possible difference with LAME (both encoders are really similar, so difference may not be audible or existent)
• foobar2000: to see if noise shaping could make a difference with MAD dithering.
Eight samples were introduced:
I've mixed different musical genre: lyrical, orchestral, piano, chamber, electronic. I've tried to mixed dithered and undithered mastered CD too.
CONCLUSIONS:
MAD isn't impressive. On four samples, result was worse than simplest decoders: increase in noise, without audible benefit on other points of the signal (aliasing, higher definition, etc...). But on some other samples, real improvements were noticed (less aliasing, less naturel and less synthetic sound)
LAME and Fraunhofer (WA5) are close, but not totally identical. Both are noisy, but less than MAD (good point). With some samples, sound was irregular, unatural... MAD partailly correct this feeling, and foobar2000 was perfect in comparison. Something like aliasing or moiré was introduced by the decoding stage. Slight difference between LAME and Fraunhofer (hard to ABX : up to 30...50 trials), in favour of Fraunhofer, less synthetic and maybe slightly more noisy
foobar2000 : each time winner. The less noisy of all challengers on all samples. The only additionnal noise audible was a very high-pitched band, very typical, and audible on extreme conditions (near silence sound, and with my amp pump to the maximum). All aliasing, irregularities... were gone: instruments are smooth, naturel. Nothing synthetic at all. With one sample (the #7), the sound was clearly better than the reference file (CD!). The lack of dithering on mastering seem to be prejudicial on very low volume part, and foobar2000 decoding correct the aliasing present on the CD. Simply amazing conclusion....
All results are here :
www.foobar2000.net/mp3decoder (http://www.foobar2000.net/mp3decoder)
http://www.foobar2000.net/mp3decoder/conclusion.htm (http://www.foobar2000.net/mp3decoder/conclusion.htm)
(I've translated in english the summary of my comments).
Comments are welcome.
P.S. It's not a MAD CHALLENGE. I repeat again that the listening conditions of the test are not reproductible on daily music listening. I've just put the decoder behind a magnifying glass, by using low volume samples.
I suppose you tested with ABC/HR blind rating although you don't say it in the message?
Your pages here (http://www.foobar2000.net) would be fantastic but too bad you chose to use french..
The "better than CD" sentence is maybe a bit controversial though, but I understand what you are after...
ABC/HR, yes (the log files are available on the site, or here (http://www.foobar2000.net/mp3decoder/files).Of course, the notation or rank can't be the same with this test than for a 64 kbps listening test. Difference between different decoders was magnified, as well as samples were magnified by increasing the listening volume.
For french language, ask my parents or english teachers: I didn't choose my country
Impressive. Thanks for your effort. Several people before have noted that MAD's dithering is "sub-optimal" and inferior to fb2k's noiseshaped dithering.
See also this thread:
http://www.hydrogenaudio.org/forums/index....showtopic=14960 (http://www.hydrogenaudio.org/forums/index.php?showtopic=14960)
One "problem" with Guru's interesting results are that those are presented on www.foobar2000.net. Not very unbiased sounding place despite of all the blind ratings and ABX -results..
I agree
I'm amazed (but very glad) that you've managed to subjectively confirm my old objective test results: MAD was/is measurably inferior to the best.
http://mp3decoders.mp3-tech.org/24bit.html (http://mp3decoders.mp3-tech.org/24bit.html)
(compare the various MAD compiles with l3dec)
I felt quite harsh for pointing this out at the time. I assumed no one would ever hear the difference with real music. It's interesting that it can be audible (in extreme conditions). FWIW the differences in dither algorithms could be greater with a very clean 24-bit source. (you state you used CDs)
It also suggests that the assumption which runs through the foobar2k GUI (which seems to be "this program performs very accurate processing, but it's probably impossible to hear the difference") is too modest!
Anyway, I wish I had your ears guruboolez!
Cheers,
David.
Anyway, I wish I had your ears guruboolez!
Golden ears would probably perceive a difference between different decoders on real listening conditions. Common ears, like mine, need to push the volume to extreme position. Try, please... I was the first to be surprise to hear differences.
At the beginning, I mainly heard different noise level... Then, I was able to ditinguish different things (that may explain why, at the end of the test, notation between decoders was more distant).
I've uploaded the lossless samples (which are -funny things- clearly inferior to the 320 CBR encoding filesize) in optimfrog format, so everybody could make the same test.
Good thorough test
Thanks for taking the time to do this... and all the other things you've contributed.
Golden ears would probably perceive a difference between different decoders on real listening conditions.
You are so modest, but this is pretty amazing stuff.
Thank you.
hmm. maybe it should be added that foobar is not the decoder itself. it uses the mpg123 library for decoding which is available for Winamp, too afaik. So the separation Winamp vs foobar is not correct and looks even more strange, as this test is provided on the foobar website
better would be: MAD vs LAME vs Fraunhoffer vs Mpeg123
Yes, and no.
I didn't test mpg123 alone, because there was dithering/noise shaping... from foobar2000. The most interessing thing to note is that mp4, vorbis, mpc... playback will profit from the same technique.
I explained that in french : the test isn't an audio player comparison (mainly because audio players are not only decoders). The test is not really a mp3 decoder test, but more a decoding technique test : dithered vs non-dithered decodings and noise-shaped dithering against simple dithering. LAME and Fraunhofer, sharing the same principles, are really close, if not the same on three or four samples. Dithering introduced a real difference (not always good, because MAD files were systematically the easiest to ABX), and noise shaping other differences.
I wonder why other audio players didn't use these techniques. Is it really hard to program? Dithering/noise shaping is very common on studio mastering: most of my CDs have visible noise shaping. So if studios use them, it must be good.
But are you sure that these improvements in sound quallity are the benefit of the dithering/noise shaping or the benefit of the decoder library itself. Maybe the same increase in quality could be heard by using the in_mpg123 plugin in winamp, too.
I would rather test Winamp in_mpg123 vs Foobar mpg123, so you can be sure that the dithering and noise shaping was the determining factor in increasing the quality. Now it can be the decoder itself.
Don't other players just do the obvious, simple thing: decode to 16-bits and truncate?
You can't noise shape and dither unless you have more than 16-bits available to start with. It must be there inside the decoder (well, at least partly, or at least in some decoders?) - but foobar is the first player where all these issues have been properly thought through.
Don't forget just how bad Winamp has been at times during its history. The old Nitrane decoder was barely 14-bit accurate at times - no point dithering it!
It seems that some players (did or do) think it's enough just to get something resembling music out of an mp3 - the quality wasn't/isn't important.
Cheers,
David.
P.S. surely mpg123 and lame give basically the same decodes (with occasional LSB differences), and all FhG decoders are likewise similar (with occasional LSB differences). FhG vs lame (=mpg123?) gives continuous LSB differences; whats more, the lame decoder apparently includes -90dB harmonic distortion when decoding to 16-bit - or at least it did when I tested it years ago.
huh? I cannot really follow your post.
I don't think the Shibatch mpg123 library and lame are related in any kind. 16+ bit output should be possible with winamp mpg123 plugin as well.
Are you confusing something?
Oh damn! I've always thought dithering was no good.
huh? I cannot really follow your post.
I don't think the Shibatch mpg123 library and lame are related in any kind. 16+ bit output should be possible with winamp mpg123 plugin as well.
Are you confusing something?
http://lame.sourceforge.net/ (http://lame.sourceforge.net/)
claims that
"LAME 3.xx uses software from:
Decoding engine (for the frame analyzer): mpglib, from the mpg123 project."
I've never investigated the details of this statement, but the lame output and the Winamp mpg123 output are amazingly similar.
Cheers,
David.
they adopted and improved hipp's mpglib and integrate for decoding
...and also made a hip interface (libmp3hip) for a straight decoding library
later
oops, then the fault was on my side. Good to know, though.
If you are correct, these conclusons can be drawn from the test:
- Neither MAD. nor mpg123, nor FhG can outperform its competitors (one better here, the other better there)
- only a combination with dithering and noise shaping will offer signifivantly better quality.
Right?
Oh damn! I've always thought dithering was no good.
oh no, dithering is very important, especially when you reduce the resolution from 24 bits to 16 bits. it's consecutive re-dithering which is a questionable practice, especially when the source is only 16 bits. however, if we use mp3s as an example, if the bitrate is high enough, it may be possible to retrieve data above 16 bits, thus preserving more ambiance and detail which would surely be lost if we were to truncate at 16 bits. there is no guarantee, however, that you will hear a lot of difference or any difference at all. the side effect of re-dithering is more noise in the audio signal but this can be somewhat offset with proper noise shaping, although this is very subjective.
the side effect of re-dithering is more noise in the audio signal but this can be somewhat offset with proper noise shaping, although this is very subjective.
If the original dithering was noise shaped, it was mainly removed during mp3 encoding because of the applied lowpass filter.
oops, then the fault was on my side. Good to know, though.
If you are correct, these conclusons can be drawn from the test:
- Neither MAD. nor mpg123, nor FhG can outperform its competitors (one better here, the other better there)
- only a combination with dithering and noise shaping will offer signifivantly better quality.
Right?
Wrong.
Read this quote from an earlier post from guruboolez on this thread again:
Golden ears would probably perceive a difference between different decoders on real listening conditions. Common ears, like mine, need to push the volume to extreme position. Try, please... I was the first to be surprise to hear differences.
There are no significant differences of any kind.
I've done a similar test (~ 1 year ago I guess, but I can't find where I posted it ):
- Take some mp3s (3.90.2 --alt-preset standard IIRC)
- Use mp3directcut to apply fadeouts (0 -> -40 dB IIRC)
- Decode using foobar2000, winamp, xmplay
- Apply fadeins opposite to the mp3directcut fadeouts (e.g. 0 -> +40 dB)
- Listen + compare
IIRC the results were these:
- With Winamp (native decoder, not MAD) there was clearly audible truncation distortion
- With foobar2000 (I can't remember what dither type I used - I don't even know if there were options to choose) there was no distortion at all but a clearly audible noise floor (of course exagerated by amplification)
- XMplay's decoder was some kind of compromise: Less distortion than Winamp and lower audible noise level than fb2k
This way a similar test can be performed without need for very good (=low noise floor) equipment. It's still hard to tell how these results are related to listening under real life conditions though.
Edit: Seems like I never posted about the test unfortunately. Some time before (I think even before I knew foobar2000) I did the same test XMPlay (2.5) vs. MAD, the result is here (http://www.un4seen.com/forum/?board=2;action=display;threadid=1615)
Later, I repeated it with Winamp 2.8x, fb2k 0.dunno and XMPlay 2.6 as described above - well, if anyone's really interested, it should be very easy to repeat ....
I wonder why other audio players didn't use these techniques. Is it really hard to program? Dithering/noise shaping is very common on studio mastering: most of my CDs have visible noise shaping. So if studios use them, it must be good.
They slow down the player for something which doesn't have a benefit in casual listening. This is why it defaults to off in foobar too.
The foobar decoder may have been based on mpglib, it has been improved quite a bit since.
The dithering/noiseshaping code is from me, based on SSRC and suggestions from KikeG.
I wonder why other audio players didn't use these techniques. Is it really hard to program? Dithering/noise shaping is very common on studio mastering: most of my CDs have visible noise shaping. So if studios use them, it must be good.
They slow down the player for something which doesn't have a benefit in casual listening. This is why it defaults to off in foobar too.
I understand. But all players have a preference/option box. Allowing dithering isn't the same thing than forcing it. If people want a theoretical increase in quality, why not? It's not a bad thing... on a changelog
MAD isn't impressive. [..] on some other samples, real improvements were noticed (less aliasing, less naturel and less synthetic sound)
less natural and less synthetic seem to contradict Maybe you could clear up what you meant here?
Also I could not find which version of MAD you tested, (cause the PCM dithering algorithm was changed in version 14.0).
Edit: Ah it was in the French piece: est la 0.15.0 (beta, datant de juin 2003)
Thanks for testing and writing to this forum in English
Sorry, I did a mistake. MAD is sometimes MORE natural and LESS synthetic.
But this is feeling only. I don't really know if this 'natural' sound is a consequence of noise added by MAD dithering. Small amount of noise isn't always bad. For exemple, video decoders have 'film' or 'grain' effects: image is less clean, but looks sometimes better with than without.
oh no, dithering is very important, especially when you reduce the resolution from 24 bits to 16 bits. it's consecutive re-dithering which is a questionable practice, especially when the source is only 16 bits. however, if we use mp3s as an example, if the bitrate is high enough, it may be possible to retrieve data above 16 bits, thus preserving more ambiance and detail which would surely be lost if we were to truncate at 16 bits. there is no guarantee, however, that you will hear a lot of difference or any difference at all. the side effect of re-dithering is more noise in the audio signal but this can be somewhat offset with proper noise shaping, although this is very subjective.
Humm, interesting... Unfortunately I don't have any 24-bit audio to test it. Thanks a lot.
I've quickly tested MAD 24 bits (dithered I suppose) decoded files against FB2K 24 bits dithered/noise shaped through foobar2000 ABX component, and the two files were less noisy than the best 16 bits decoded files. Both files sounded similar.
It seems that ABC/HR don't support 24 bits input files.
So what is the point of application in plain terms for the non-technical person? Does this mean we should use Foobar to batch encode mp3 from waves OR that playback is best when using Foobar OR some box should/shouldn't be checked when playing them back or encoding or ABXing? Help the Average Joe here understand the real-world significance of the discussion, please. Thanks in advance.
Did you even try to read the discussion?
Did you even bother looking up what encoding and decoding are?
Yup but I don't get it - clearly you are much smarter than I. Or, you at least have years of training and/or experience I do not have (and can't get in the next 3 days before this thread dies off) that enables your participation in the discussion. Take it as a challenge. You genius try to explain, in non-tech terms, the real world application of the discussion for the average reader. Or if you are not interested in such a discourse, perhaps someone else is. Thanks in advance for your help.
Since you challenged me, I'll try to do so:
Decoding is the process of uncompressing compressed (encoded) data like MP3 back to uncompressed PCM Audio. Guruboolez' tried to objectivly measure the quality of different MP3 decoders with and without different types of dithering (http://www.mtsu.edu/~dsmitche/rim420/reading/rim420_Dither.html).
You are most likely not to hear any differences at all under normal listening conditions on normal music (especially loud pop/rock music will mask most differences), but Guruboolez tried to "magnify" the differences by choosing quiet passages of music and listening to them at a high volume.
Guruboolez' liked foobar2000's decoder and noise-shaped dithering best.
So what is the point of application in plain terms for the non-technical person? Does this mean we should use Foobar to batch encode mp3 from waves.
Only if you want to apply DSPs or volume changes (e.g. replaygain) before encoding. Volume changes can be applied after encoding as well (replaygain/mp3gain) BTW.
OR that playback is best when using Foobar
Theoretically, yes. That's what the test results suggest. Under real listening conditions there shouldn't be an audible difference though.
OR some box should/shouldn't be checked when playing them back or encoding or ABXing?
For theoretically best playback quality check "Dither" box in output settings if you use 16bit output. If you use 24bit output (and have a soundcard capable of 24bit ) dither will only (slightly) increase CPU load but have no advantages.
Help the Average Joe here understand the real-world significance of the discussion, please.
To understand the technical background better you might want to have a look at the FAQ or the Wiki ("Knowledge Base" on HA portal).
I think the 'real world' conclusion is:
1) Not all CD's are properly mastered
2) Differences between MP3 decoders are only audible in extreme circumstances
3) Taking 2 into account, foobar does well, and MAD isn't what it is hyped up to be
Would conclusions 2 and 3 be based only on Guru's test?
As in, I'm quite sure many of us would have already encountered many people on other forums who claim the direct opposite to 3. How are they going to be persuaded otherwise?
Thank you. I understand. Great article on dithering - very approachable by a lay person. Follow up questions then (if the answers aren't beyond the average Joe's discussion and I'm using the correct terminology): Why not do this on every recording? Is there a way to ensure that it is when recording digitally or is it dependant on choice of equipment? Does "upsampling(?)" to 24 bits during decoding/playback take care of the problem or is this artificial in some way?
I assume this explains some of the the background noise in some cases on soft recording passages (Jazz and classical is my main listening) or distorted/strange tone qualities/characteristic sounds of certain instruments played at soft volumes? Perhaps this problem isn't as uncommon as it seems to classical listeners, as turning up the volume to hear soft passages is frequently required to hear what is even being played...
Thanks everyone!
Would conclusions 2 and 3 be based only on Guru's test?
As in, I'm quite sure many of us would have already encountered many people on other forums who claim the direct opposite to 3. How are they going to be persuaded otherwise?
Not only, I knew of the differences before he did the test but it's nice to see it confirmed
Well, there's an objective blind test showing that foobar is better. I don't know any showing the opposite, do you?
Not only, I knew of the differences before he did the test but it's nice to see it confirmed
Well, there's an objective blind test showing that foobar is better. I don't know any showing the opposite, do you?
Don't worry, your question is indeed rhetorical.
3) Taking 2 into account, foobar does well, and MAD isn't what it is hyped up to be
With the asterisk that this test was done at 16bit, and that based on gurobolez's 'quick test' MAD *might* well be about as good as fb2k at 24bit.
(edited for clarity)
Anyway, I wish I had your ears guruboolez!
Golden ears would probably perceive a difference between different decoders on real listening conditions. Common ears, like mine, need to push the volume to extreme position.
Nonsense. You have made enough merits to be considered member of the "golden-ear" club. Some people being able to hear differences at real listening conditions? Maybe, for some really gifted people, and when listening in really good conditions (good headphones, low ambient noise), but even in this case audible differences would probably be very small. Anyway, without actual results it's hard to say.
Try, please... I was the first to be surprise to hear differences.
I'm not very surprised, if using low amplitude music, amplifying the result, and using discriminating ears. When I tested it, MAD dithering showed to be clearly sub-optimal, due to lack of dither amplitude, that caused visible distortion and noise pumping on synthetic test signals.
I'm not sure I understand all of this, but I have this question.
All of my MP3s or MPCs are 16 bit. Now there are instances where there is more than 16 bits of information to decode at one time? The option to dither (in fb2k) would allow playback without truncating these instances to 16 bits. My audigy2 supports 24 bit playback, so if I chose 24 bit playback (in fb2k) then these instances would still not be truncated without the need of dithering?
Sorry if I have this completely wrong but I'm drinking beer right now. :X
does foobar also use the noise shaping (when enabled) on wav files or only on encoded material ? as i want bit identical output on my spdif out i guess i better leave it off.
All of my MP3s or MPCs are 16 bit.
They aren't.
In general, you can't say that a lossy file has a certain bit resolution.
Theoretically, an mp3 file can be decoded to any desired bit depth. And I think that by modifying e.g. gain information, you can create mp3s that would decode to something other than digital silence only, say, at 24 bits.
Whether decoding to a bit depth higher that 16 makes any difference in the real-world listening conditions is a different question, though.
Interesting discussion about dithering. Just a small off-point question. In the vorbis plugin for winamp 5, there is a 'dither' checkbox. Since lossy files are not bit resolution specific, does that mean when the vorbis plugin truncates to the (default) 16 bits, it does some sort of dithering? Also, in the Directsound SSRC output plugin, there is a dithering option and noise shaping. If I am doing dithering in the output plugin, does that mean the vorbis plugin dithering is redundant or vice versa?
@Quantum: I don't know if I fully understand your question. But I think that if Vorbis decoder truncate to 16-bit, then it won't be a form of dither. So I think you should enable Dither on it's decoder (on Winamp). I've always used it disabled and I really love my Vorbis. hehe
I haven't enabled dithering on fb2k either. I've always had a sort of predujice against this word hehe. Dithering for pictures su**s.
I'm not sure I understand all of this, but I have this question.
All of my MP3s or MPCs are 16 bit. Now there are instances where there is more than 16 bits of information to decode at one time? The option to dither (in fb2k) would allow playback without truncating these instances to 16 bits. My audigy2 supports 24 bit playback, so if I chose 24 bit playback (in fb2k) then these instances would still not be truncated without the need of dithering?
Sorry if I have this completely wrong but I'm drinking beer right now. :X
LOL! Quite a sensible question then!
The FAQ on these pages (scroll down) may help, though you've kind of got the answer already...
http://mp3decoders.mp3-tech.org/24bit.html (http://mp3decoders.mp3-tech.org/24bit.html)
http://mp3decoders.mp3-tech.org/24bit2.html (http://mp3decoders.mp3-tech.org/24bit2.html)
If I could find the test files, I'd even think about testing foobar. But I have 1000001 things to do first.
Cheers,
David.
@Quantum: I don't know if I fully understand your question. But I think that if Vorbis decoder truncate to 16-bit, then it won't be a form of dither. So I think you should enable Dither on it's decoder (on Winamp). I've always used it disabled and I really love my Vorbis. hehe
I haven't enabled dithering on fb2k either. I've always had a sort of predujice against this word hehe. Dithering for pictures su**s. Should I enable it?
Ah ok, I'll rephrase it to see if its a bit understandable.
Essentially, there are two places in Winamp where there is a dithering option. One is in the Vorbis input plugin, which is a check box. I got that checked and using 16-bit decoding. The second place to dither is in Peter's DirectSound SSRC output plugin (the one which allows you to do resampling to 48 kHz). In this second one, you can choose noise distribution as well as dithering using a spectral shape (triangular).
Are these two dithering options different and independent of each other and should I have one turned on instead of the other?
I think that if Vorbis decoder decodes do 16-bit, it will already be 16-bit (constant) stream that will be fed to Peter's DSound SSRC plugin. So I believe it won't mess up with it anymore. Now, if you turn off Dithering on Vorbis decoder but tell it to decode at 16-bit it will truncate the file and the final stream will also be 16-bit, constant, truncaded. I think that if you want to use Pete's dither algorithm you should tell Vorbis decoder to decode to 24-bit non dithered.
I'm not really sure if it is correct. Perhaps someone could make things more clear.
What happens if I process my music in Winamp/mpg123 plugin in 24bit/32bit (should be possible, there is an option for it) and downsample it to 16 bit with PP's out_ds using dithering and noiseshaping. Shouldn't the result be the same or very similar (considering garf's and PPs improvements done to the mpg123 base) as with foobar?
...
fed to Peter's DSound SSRC plugin.
...
Hi... could someone please tell me something about this DirectSound plugin? I've only just installed Winamp5, and I'm still searching for the best possible audio quality (which is why I read this thread, since I'm using the MAD input...)
I searched the plugins on Winamp's site, but I couldn't find 'peter's dsound ssrc plugin'...
thanks
d. (newbie extraordinaire)
Try:
ftp://x-sound.com:x-music@www.x-sound.net...t_ds_v2.2.6.exe (http://ftp://x-sound.com:x-music@www.x-sound.net/software/other/out_ds_v2.2.6.exe)
This is another interesting site for winamp 2.x plugs:
http://support.xmplay.com/Plugins_index.html (http://support.xmplay.com/Plugins_index.html)
The installers here will only install to a folder where "XMPlay.exe" is present, but any binary file with that name will do.....
This is really a great site:
http://www.x-sound.com/index?update=005 (http://www.x-sound.com/index?update=005)
To bad I don't understand japanese... but they keep good track of updates for a lot of fiiine 'ole proggys....
helpful, thanks!
d.
I searched the plugins on Winamp's site, but I couldn't find 'peter's dsound ssrc plugin'...
You can get it here (http://www.public.asu.edu/~abarber2/Stuff.html) (direct link (http://www.public.asu.edu/~abarber2/mirror/out_ds.exe)).
Most of above was over my head, but this might be the right thread to ask a question about something I never understood. What is the effect and interaction of the dithering to get something higher than 16-bit audio quality on a 16-bit CD format and thence to MP3? I'm thinking about SBM and the JVC K-sumpn or other (which I thought best).
DMP Records did a re-release using such shaping/dithering technques of some of their earliest DDD recordings, intended to pass through info from the higher sampling rates they had used for Flim & the BBs -- I thought the update Cds super. As a result, I tried upsampling to 48khz (Shibatch?) many over-16bit Cds as I encoded them at the highest feasible LAME VBR settings; I thought that made the MP3s sound better, but I'll confess I was frittering to kill time in wee hours of late 2001-early 2002.
Perhaps related is why LAME results from horribly digitized analog-recorded music often improved on the CD; the most noticeable was '70s Tangerine Dream like Phaedra where, once processed by Dibrom, the music seemed clearer -- is that just that Dibrom's codecs knew what was spurious distortion and eliminated it?
• For this test, I didn't reproduced real listening conditions. Difference are so subtle that I have to increase the level of my amp in order to maximise the audible difference.
• In order to make it possible without being completely deaf in few seconds, I've selected very quiet samples. With classical music, it's not really hard to find such samples (even full tracks are sometimes totally quiet). I didn't evaluate any decoders on pre-instrumental silence, but only when music was playing
Sorry to bring up this over 1 year old thread (it was recently referenced).
On to business, I think this test is flawed...to listen to dither (around 1 bits worth) the volume must be increased by a large factor. The best dither on the planet would be one that kept it's noise just below audible threshold (ok there is some give an take with this, as peoples hearing differs and playback hardware is different, but only to an extent), and by just below I mean right upto the threshold. A dithers strength is proportional to the strength of this inaudible noise, multiplying up the dither noise (and making it audible) proves nothing, just that there is noise that can be heard when multiplied, with such a test only the weakest dither or no dither at all would come out top.
A dithers strength is proportional to the strength of this inaudible noise, multiplying up the dither noise (and making it audible) proves nothing, just that there is noise that can be heard when multiplied, with such a test only the weakest dither or no dither at all would come out top.
[a href="index.php?act=findpost&pid=351210"][{POST_SNAPBACK}][/a]
You are completely wrong, and apparently misunderstand why dither is used at all.
It's also quite easy to prove this assertion wrong by actually doing what you describe.
As for the test being flawed; I think it's stated several times in this thread the effects of dither are below the audible threshold for any practical purpose. So any comparison will push practicality, and I think guruboolez` stated approach is sensible. But do the results matter? I would say they usually don't, and thats why dither defaults to off in fb2k.
>apparently misunderstand why dither is used at all
I was under the impression it was to increase perceived bit depth, there are other reasons?
I am not questioning that part of the test (ok I will also...an mp3 created from a 16 bit CD? so suddenly there is 20 bits worth of dynamic range? a great test for dither).
Instead generate test sines (uncompressed) on the 16th+n bit and dither down to 16 bit then run listening tests, that would be a good test for dither.
>effects of dither are below the audible threshold for any practical purpose
So it is ok to multiply these up so they can be heard and pass comments on how one has more noise than the other? (I am aware there are many different types of dither, in this test only the ones which push the created noise into the higher frequencies would come out well), AFAIK most strong commercial dithers (as actually used on CDs) are not this type.
>apparently misunderstand why dither is used at all
I was under the impression it was to increase perceived bit depth, there are other reasons?
"Wrong reason"
There are a lot of threads about dithering and dithering + noiseshaping around here; look for posts by KikeG, 2BDecided etc...
http://www.hydrogenaudio.org/forums/index....showtopic=29912 (http://www.hydrogenaudio.org/forums/index.php?showtopic=29912)
You dither to decorrelate the quantization error from the input signal.
I am not questioning that part of the test (ok I will also...an mp3 created from a 16 bit CD? so suddenly there is 20 bits worth of dynamic range?
Sure!
Nobody says there are 20 bits of dynamic range in the mp3 itself. However, if you lower the volume of the MP3 decoded output (in floating point), dither when converting to 16 bits, and up your amplifier, you are certainly going to hear how the sub-16th-bit resolution affects the output.
Instead generate test sines (uncompressed) on the 16th+n bit and dither down to 16 bit then run listening tests, that would be a good test for dither.
Maybe. But why do this when it can be tested with actual music, too?
>effects of dither are below the audible threshold for any practical purpose
So it is ok to multiply these up so they can be heard and pass comments on how one has more noise than the other?
Know another method to evaluate differences in something that should not be inaudible?
You can instead go to 10 bit resolution instead of 16 bit or so. Then no amplification is needed - the dither quality will be observable in normal conditions.
I don't know which comparison is more valid. I believe results will be similar if not identical.
(I am aware there are many different types of dither, in this test only the ones which push the created noise into the higher frequencies would come out well), AFAIK most strong commercial dithers (as actually used on CDs) are not this type.
[a href="index.php?act=findpost&pid=351221"][{POST_SNAPBACK}][/a]
The high end ones most certainly are.
(I am aware there are many different types of dither, in this test only the ones which push the created noise into the higher frequencies would come out well), AFAIK most strong commercial dithers (as actually used on CDs) are not this type.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=351221")
The high end ones most certainly are.
[a href="index.php?act=findpost&pid=351273"][{POST_SNAPBACK}][/a]
See here:
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=9192]http://www.hydrogenaudio.org/forums/index.php?showtopic=9192[/url]
Thanks for the links, will research.
On to business, I think this test is flawed...to listen to dither (around 1 bits worth) the volume must be increased by a large factor. The best dither on the planet would be one that kept it's noise just below audible threshold (ok there is some give an take with this, as peoples hearing differs and playback hardware is different, but only to an extent), and by just below I mean right upto the threshold. A dithers strength is proportional to the strength of this inaudible noise, multiplying up the dither noise (and making it audible) proves nothing, just that there is noise that can be heard when multiplied, with such a test only the weakest dither or no dither at all would come out top.
[a href="index.php?act=findpost&pid=351210"][{POST_SNAPBACK}][/a]
I did this test, which included dithered¹ and
undithered² decoded files. According to your claim, the undithered decoding should end on top. But according to my results, they end last.
I didn't translate the test in english, so I guess you can't read my detailed results. But the marks are a good indication: undithered and inferior dithering (MAD) produce a more annoying form of distortion than a good dithering (perceptible by a tiny HF noise).
In one case (one Mahler sample), the reference CD (which is undithered) is lower quality than the MP3 encoding decoded with dithering!
¹ MAD and foobar2000
² LAME and Fraunhofer decoders.
...(less aliasing, less naturel and less synthetic sound)[a href="index.php?act=findpost&pid=174740"][{POST_SNAPBACK}][/a]
What do you mean by "aliasing" here? I'm curious as to how an MP3 decoder could affect the AA filter of a DAC (or SRC)..
...(less aliasing, less naturel and less synthetic sound)[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=174740")
What do you mean by "aliasing" here? I'm curious as to how an MP3 decoder could affect the AA filter of a DAC (or SRC)..
[a href="index.php?act=findpost&pid=351985"][{POST_SNAPBACK}][/a]
The sample is still online:
[a href="http://foobar2000.net/mp3decoder/files/lossless_mahler3.ofr]http://foobar2000.net/mp3decoder/files/lossless_mahler3.ofr[/url]
Result and comment are here:
http://foobar2000.net/mp3decoder/test7.htm (http://foobar2000.net/mp3decoder/test7.htm)
The sample is still online:
http://foobar2000.net/mp3decoder/files/lossless_mahler3.ofr (http://foobar2000.net/mp3decoder/files/lossless_mahler3.ofr)
Result and comment are here:
http://foobar2000.net/mp3decoder/test7.htm (http://foobar2000.net/mp3decoder/test7.htm)
I don't doubt that you heard what you heard. My problem is that you used the word "aliasing".
My understanding of aliasing is as follows:
Aliasing in digital audio is a phenomenon that occurs due to the fact that several waveforms can mathematically occupy the same sample points; the illegal waveforms are aliases. All the aliasing waveforms are frequencies > fs/2 --> an oversampling DAC contains a low pass filter to remove those frequencies. A SRC also needs to deal with this issue.
Of course, I could be wrong. In that case, please correct me.
By aliasing I meant a weird form of "geometric" distortion easily audible with resampling (16->8 bits for example).
I defined it (in french) in the beginning of sample-02 (http://foobar2000.net/mp3decoder/test2.htm).
I put online this extreme example to illustrate this phenomenon:
http://foobar2000.net/mp3decoder/files/aliasing.ape (http://foobar2000.net/mp3decoder/files/aliasing.ape)
(http://foobar2000.net/mp3decoder/files/aliasing.png)
You can see the geometric distortion (something like a wave form) on the middle.
By aliasing I meant a weird form of "geometric" distortion easily audible with resampling (16->8 bits for example).
Ok, that's not aliasing - in all likelihood it's just dithering noise.
Aliasing has a very specific look in the frequency plane - the signals are mirrored around fs/2 (22.05 kHz if the source is 44.1 KS/s).
Anyway, this was just a sidetrack. Thank you for your tests.
Btw, it appears that your testing procedure for dither evaluation is flawed. This is what Dan Lavry of Lavry Engineering (famous DAC/ADC maker) has to say:
Dithered noise shaping technology has been incorporated into a handful of hardware devices. While all are based on the same concepts, some perform better than others. After simulating and listening to all available public domain algorithms, Lavry Engineering came to some conclusions in forming a basis for Acoustic Bit Correction™. The principal conclusions are:
a. The practice of greatly amplifying low level signals to determine triangular flat PDF (probability density function) dither reveals the effectiveness of distortion and noise modulation elimination. This practice yields misleading results when testing unflattened dithers and/or noise shapers. It conflicts directly with L. Fielder’s findings showing completely different threshold delectability curves for quiet and loud levels. Noise shaping listening tests must be done at "reasonable" volume levels.
b. Given the above requirement, our listening tests concluded a strong preference for "triangle high pass" dither (this dither is produced by simultaneously adding a new random number and subtracting the previous value). Such dither is frequency-shaped to carry more high frequency energy (the energy content at low frequencies is minimal).
c. Listening tests revealed a preference for smoothly varying noise-shaping curves. Peaks and notches seem to irritate the listener (admittedly while turning the volume up). In addition, despite the temptation to optimize the noise shaping curve to the average listener’s hearing threshold, given a significant variation from listener to listener requires reasonable compromises in tailoring such a curve. In other words, smooth the curve.
The improvements offered by dither and noise shaping vary with source material and final word length. An A/B/X test at 16-bit level, requires a quiet environment and low level (loudness) audio. The listener must resist the temptation to turn the volume up to unreasonable levels. The practice of truncating to short word length (8-12 bits) should be avoided. The ideal noise-shaping curve may be irritating at loud levels.
Lavry Engineering’s listening tests were based on test tones and repeating loops of quiet passages of various material (mostly classical music) with flat amplifier response. Listening to test tones was straightforward: we used the Model AD122-96 MKIII test tone generator mode switching the Acoustic Bit Correction™ on and off. The frequency and amplitude programmability was very useful.
http://recforums.prosoundweb.com/index.php.../48/0#msg_num_1 (http://recforums.prosoundweb.com/index.php/m/0/8804/48/0#msg_num_1)
...so I am not alone in my thinking. It it like those who dare question that the world is not flat, discussion is good and shouldn't be counted with a 'do the results matter?', of course they matter. It is the basis of all scientific testing, as apposed to theological ideas.
Dither is one of those difficult areas, since my original post I have spent a further 20 hours reading up and still think (70% sure) you cannot mess with the signal as it invalidates the process. I am of the view the best way to test dithers is in the frequency domain, this deternimes if the dither is effective (such as > -100 db dynamic range at certain frequencies) and here is the clincher, as long as the noise added remains inaudible in normal circumstances then the dither is working as should.
Well, if I read the quote, they used low volume music and *did* turn the volume up, just not unreasonably loud.
Again, I don't see any other way to practically test this. You can analyse the dither in other ways, but the actual tuning of the noise shaper is a psychoacoustics process.
Hearing beyond the 15th bit is extremely hard, how are you going to test different dithers for audibility there? You'll have to go a similar way, set up a soundproof room and spend a lot on time on listening tests.
Don't misunderstand me. I fully agree that the most solid test is in the actual playback conditions, 16 bit, volume at a default setting. It's just very hard. Each time you turn the knob, testing gets easier, and you will deviate very slightly. It's not like the hearing curves suddenly turn upside down. I think you are right to criticise the test, but it's not sensible to say the results are completely flawed if you haven't been able to perform a better one - because while the hearing curves do change with loudness, they change smoothtly, it's not like they turn upside down.
It's like saying Newton is wrong because you observed a relativistic effect. Yes, you might be right, but until Einstein comes along Newtons results are certainly very usable
If I get your proposal, you want to evaluate the SNR gain in the frequency domain, and then turn up the dither as loud as possible while still being inaudible.
This sounds great in theory, but you might run into practical issues. You'll end up blasting VERY LOUD HF noise through the reproduction system, and not all of them will like that. Think blown tweeters, so be carefull
Good luck!
PS. The fact that audibility and masking levels are dependant on loudness and the listener isn't news...You will be able to apply a lot more dither if the person listening is 40 years old and deaf to 13kHz than against a 15 year old that hears 18kHz.
I'm getting a 403 Forbidden response from http://www.foobar2000.net/ (http://www.foobar2000.net/).
Is this just me, or does Guruboolez need to kick his server administrator?
Garf: Nope Mr Lavry doesn't suggest listening to dither at levels where you just start to hear it. Quite the opposite. Since, as Mr Lavry says, the threshold detectibility curves are completely different for quiet and loud levels --> "Noise shaping listening tests must be done at "reasonable" volume levels."
I'd hate to appeal to authority, but do you know who Dan Lavry is? If you don't I recommend spending some time at the forum I linked to above. Regular posters include Mr Lavry himsel and also Bob Katz and Nika Aldrich (it's a scientific and objectivist forum).
Garf: Nope Mr Lavry doesn't suggest listening to dither at levels where you just start to hear it.
Where on earth did I say this?
Garf, answer to your question:
If I get your proposal, you want to evaluate the SNR gain in the frequency domain, and then turn up the dither as loud as possible while still being inaudible.
This sounds great in theory, but you might run into practical issues. You'll end up blasting VERY LOUD HF noise through the reproduction system, and not all of them will like that. Think blown tweeters, so be carefull
Furthermore, you also state that it isn't right to criticize the test as flawed unless you come up with a better test. I suggest you read what Mr Lavry wrote again since that is
exactly what he has done.
Garf, answer to your question:
If I get your proposal, you want to evaluate the SNR gain in the frequency domain, and then turn up the dither as loud as possible while still being inaudible.
This sounds great in theory, but you might run into practical issues. You'll end up blasting VERY LOUD HF noise through the reproduction system, and not all of them will like that. Think blown tweeters, so be carefull
[a href="index.php?act=findpost&pid=356086"][{POST_SNAPBACK}][/a]
Gah! I'm not talking about the volume level in the test there, I'm talking about TUNING THE NOISE THE NOISESHAPER ADDS. That's exactly why it says
"while still being inaudible" and not what you say.
Furthermore, you also state that it isn't right to criticize the test as flawed unless you come up with a better test. I suggest you read what Mr Lavry wrote again since that is exactly what he has done.
He's doing exactly the same thing! He warns against the effect changing the volume level has on the shape of the audibility curves, and that too large volume levels hence invalidate the test. Spoon already warned against this and I agree with the principle. My previous posts elaborates on that; I explain that there is a smooth tradeoff between making the testing easier and getting a large error on the curve you will end up with. Furthermore, practical limitations of playback material might cause limits contrary to what a theorethically optimal test or dither design will produce. The dither you design should be resistant against those, or it's not going to be very usable in practise.
Clearer now?
Edit: PS. I say it seems they also turn up the volume, just not to an unreasonable level, because of this quote: "The listener must resist the temptation to turn the volume up to unreasonable levels." They could have said "to turn the volume up. PERIOD", but they didn't.
Well I don't know what you are arguing then. I stated that Guru's test is flawed, which you apparently agree with, so what exactly are you in disagreeance with?
I'm getting a 403 Forbidden response from http://www.foobar2000.net/ (http://www.foobar2000.net/).
Is this just me, or does Guruboolez need to kick his server administrator?
[a href="index.php?act=findpost&pid=356024"][{POST_SNAPBACK}][/a]
I'm getting it too.
Thanks for the confirm. I thought it was just me for a while.
Unfortunately I think guruboolez has other things on his mind.
Well I don't know what you are arguing then. I stated that Guru's test is flawed, which you apparently agree with, so what exactly are you in disagreeance with?
[a href="index.php?act=findpost&pid=356100"][{POST_SNAPBACK}][/a]
I compared it to saying Newton was wrong because we know what Einstein said. Yes, but...
We can work with a suboptimal, less accurate result if it allows us to do practical things easier. Of course, the theorethically correct method would be even better.
But that doesn't mean the original was "flawed" in as so much the the results are worthless. Just like the test you qoute isn't, although they apparently did adjust the volume. I would be impressed if anyone actually manages to ABX dither vs truncation at completely normal playback volume. It would be a stricter version of the "MAD Challenge" and even that is insanely hard to pass. Let alone repeat it to determine an optimal dither.
The tradeoff is "less accurate" vs. "more accurate". Not "fatally flawed" vs. "100% correct".
Given that auditory capabilities change even between humas, you won't get a 100% method, anyway. Might as well call any test "flawed" then. But I will ignore you and make something that demonstrably works, despite being based on slightly inaccurate results
The tradeoff is "less accurate" vs. "more accurate". Not "fatally flawed" vs. "100% correct".
Nobody has said anything about
fatally flawed. Just flawed. Huge difference.
When it's all said and done more professional people have conducted solid ABX test evaluating the efftects of dither, come to conclusions, while also recommending guidelines for how to evaluate dither. Guidelines which says a big no-no to tests such as performed by Guru.
The scientific minded audio world does not begin or end with HA.
Amplification of low-level dithered and requantized test signals up to clearly audible levels is useful for checking if the dither is properly eliminating quantization distortion and does not produce noise modulation. And is useful for this whether the dither is flat or noise shaped. In this context, MAD dither amplitude was (last time I checked) insufficient because it caused both artifacts, and this could be checked by the mentioned amplification procedure.
Said this, I find useful the procedure of comparing just dither noise audibility (I mean dither without any test signal, that is, dithered "silence") at barely audible levels, given that there is little background noise and dither noise levels are low enough. If the noise cause by one of the dithers at test is barely audible and the other is not audible, then the one not audible is better for sure, under those conditions.
For tests at real world conditions, when not applying extra amplification in order to make dither noise audible, if no one of the dithers is audible by itself... then it really doesn't matter.
Sorry for delay.
Btw, it appears that your testing procedure for dither evaluation is flawed.
http://recforums.prosoundweb.com/index.php.../48/0#msg_num_1 (http://recforums.prosoundweb.com/index.php/m/0/8804/48/0#msg_num_1)
[a href="index.php?act=findpost&pid=355762"][{POST_SNAPBACK}][/a]
Btw, it appears that your testing procedure for dither evaluation is flawed.
A listening test performed on unusual (and impracticable in real life!) condition is inherently flawed In my case, I could only conclude that dither
x is better than dither
y on the tested conditions: over-amplified playback of ultra-quiet recordings. I can’t extrapolate the result to normal playback condition: conclusion won’t be valid anymore I suppose. The practical purpose of my test is therefore close to zero. I did it by curiosity, and because it was for me the only way to evaluate the performance of different MP3 decoders. My poor hearing abilities don’t allow me to differentiate MAD from another MP3 decoder (excepted the reputed Nitrane?). And even with these extreme testing conditions, the test was difficult to perform (my audio hardware is rather inexpensive and my amp produce a strong amount of noise at high volume).
BTW, I’m amazed to see that some people are able to make a distinction between different dithered noise shaping technology on 16 bit or more material. I haven’t reached this state – far from it!