[span style='font-size:14pt;line-height:100%']INTRODUCTION[/span]
During a sleepless night, I decided to perform my own listening test, with samples based on my musical tastes. I chose a friendly bitrate encoding (~130 kbps) : in my opinion, it’s a good compromise for limited hardware space (notebook) and good quality (my preliminary experience with lower bitrate was catastrophic). Of course, testing encoding at this bitrate range is much easier. I also had in mind the idea to build a comfortable library for my notebook, and the idea to clarify some feelings I had about different encoders, on my music only. Among them :
- I suspected lame mp3 to be a good audio format on some (quiet) situation, and to not suffering as much as with pop, metal or electronic music.
- I suspected Nero AAC to be simply worse than Lame MP3 ; I did some encodings on common music the last times, and I was disappointed by the poor results I obtained for a VBR setting (-streaming). I couldn’t conclude on Nero worse quality based on three or four samples. This extend test will be more conclusive.
- I suspected QuickTime AAC and WMA 9 PRO to reach a near-transparency quality at this bitrate.
[span style='font-size:14pt;line-height:100%']SAMPLES[/span]
Unfortunately, I worked with a limited choice of samples. With the start of the new university year, I had to leave my summer house. My discs are not for the moment in my new apartment : only the few one I bought since September. I choose samples from them, and completed the gallery with samples I burned on CD-R some times ago. Most of them are music from baroque period. It’s my favorite one. Therefore, I couldn’t materially built a coherent sample library for this test with romantic or contemporary music. That’s why there’s no piano, no Beethoven, no electronic samples here. That’s why, too, so many harpsichord (hard-to-encode instrument) is present, during singing, sacred, chamber or orchestral music. I did my best to find some samples without this instrument : more of the half.
Therefore, this test only had a limited impact: results are useful for baroque music only ; they may be different for other kind of “classical” ; and they must be totally different for jazz, country, R&B or anything else. Of course, notation is mine, dependant on my own subjectivity, my on hardware setting, and on the mood I had during the test.
[span style='font-size:14pt;line-height:100%']ENCODERS[/span]
I encoded them with foobar2000 (except for WMA9 and PsyTEL AAC). All encodings were replaygained, original too. Decoding stage include the calculated gain (without dithering – maybe not the best choice). I didn’t make any offset correction (needed for some AAC encoding).
Challengers are :
• AAC Faac (don’t know the exact version : I used Case diskwriter component bundled with foobar2000 0.7.1 beta 4), quantizer 128, LC, m/s coding and with TNS.
• AAC Nero (aac.dll 2.5.6.3 & aacenc32.dll 2.5.5.8) : -streaming, LC, High quality
• AAC PsyTEL 2.15 –streaming [ADDED ONE WEEK LATER: I did a big mistake, and placed the good aacenc32.dll in the wrong location Therefore, I tested an old encoder : 2.5.1.6. Some odd artifacts (Dorilla, Passacaglia) are not present in 2.5.5.8. See here for complete report and additional test] (http://www.hydrogenaudio.org/forums/index.php?showtopic=14091&st=39)
• AAC QuickTime 6.3, CBR 128, High Quality
All AAC files were decoded with faad2 (foobar2000)
• MP3 Lame 3.90.3 –alt-preset 134
• MPC 1.14 –quality 4 –xlevel
• OGG vorbis GT3b1 –q 4.25
• WMA 9 PRO VBR-2pass 128 kbps
(I chose abr 134 in order to be close to 128 kbps, because ABR mode isn’t reliable with most classical stuff. I used –b 4.25 for Vorbis, according to the average size collected some times ago, before Roberto’s test: it was a small mistake).
[span style='font-size:14pt;line-height:100%']DECODING & PLAYBACK[/span]
All files (including reference) were replaygained with foobar2000, and then decoded with calculated gain and without dithering, No offset correction was applied (needed for correcting some AAC encoding offset) : I didn’t have the correct value.
Note than one sample (« Laudate pueri ») wasn’t properly encode/decode with PsyTEL : 15 seconds are missing, due to a bug (« negative scalfactor… »). I rated the quality for the first part only.
Comparisons were made with ff123 tool (ABC/HR). I didn’t take the time for ABXing (except for one sample).
I worked with my notebook, an its weak AC Audio hardware. My desktop computer is with all my CDs : away. Headphone is a Philips SBC-HP910.
[span style='font-size:14pt;line-height:100%']RESULTS[/span]
(http://membres.lycos.fr/guruboolez/AUDIO/test_MF_early/early128.png)
[span style='font-size:14pt;line-height:100%'](few) COMMENTS[/span]
• The most shocking results are Nero AAC poor performances. The quality is terribly poor. Faac isn’t for the moment a good encoder ; Nero, on my 18 samples, isn’t really better, but is slower and with a slightest higher average weight… I suspect (and hope) that a bug in current version is responsible. “Dorilla” sample should be an interesting one; “cello” too, but in a more common way. “Passacaglia” sample was never annoying, except with Nero encoder: maybe a problem with background noise, noticed last week with the “biniou” sample.
Interesting thing to note: PsyTel is better for my ears than Nero. Only one sample encoded with Nero ancestor was considered worse than with latest Nero encoder (“hornpipe” sample).
ADDED ONE WEEK LATER: The biggest and unexpected flaws I heard, and badly rated, with Nero AAC, are removed with the latest stable and official AAC encoder, 2.5.5.8. See
here for complete report and additional test] (http://www.hydrogenaudio.org/forums/index.php?showtopic=14091&st=39). Nevertheless, the more usual distortions are still present.
• Lame mp3 isn’t a bad choice for this music, and for my ears. Better than Faac, Nero and PsyTEL AAC, not too far from overall MPC and Vorbis performances.
• Vorbis surprised me : on Roberto’s multiformat 128 test, vorbis wasn’t too competitive, due to hiss/HF boost and dirty feeling it gives to me. Here, problem wasn’t always perceptible. Nevertheless, on some samples, this added noise gives the impressions of coarse manufactured instruments, rough tones, etc… Good ranking for vorbis ; excessive bitrate consumption will be partially corrected with a conventional setting (-b 4). Few pre-echo problems : I used GT3b1 encoder (but it was maybe not the best choice – I forgot that foobar2000 component was based on this particular –and particularly good– library)
• MPC was very good during Roberto’s test; with classical (baroque) only, performances are not so exciting. Nevertheless, mpc stays competitive in quality area. Unfortunately, and as expected, average bitrate isn’t competitive at all: strings, harpsichord, and ‘tonal music’ in general are bitrate greedy.
• AAC QuickTime is very pleasant. I must confess that I’m a bit disappointed: I hoped better results. Some distortions (flanging) are still noticeable with this very good encoder. Best encoder on 6 samples (33%).
With classical, QuickTime is in my opinion the very best AAC encoder at mid-bitrate.
• WMA9 PRO : I expected very good results. They are excellent. Very few flaws (more annoying was the unstable noise with “Requiem” ultra-quiet part). I couldn’t distinguish the encoded from the reference on 4 samples. Best format for 10 samples (55%).
This new encoder fits to my ears :-)
Note : I’ve got a very limited internet connection. I will try to upload the 18 samples, but I need time for it. I will begin this night (i.e. in 6 hours), with the most important one (useful, in my opinion, for Ivan). Then, I will try to finish the next week on a DSL connection. Samples will be hosted on HA server (Upload Forum). Thanks for comprehension.
Note.2 : bitrate table will follow. Total size of all files are at the bottom of the previous table.
Note.3 : I posted this result in AAC forum, just because 4 different AAC encoders were tested, though it wasn't the goal of the test.
http://membres.lycos.fr/guruboolez/AUDIO/t...early/early.htm (http://membres.lycos.fr/guruboolez/AUDIO/test_MF_early/early.htm)
EDIT : total size for PsyTEL and QuickTime are changed (html table only) ; I forgot to include one sample.
Nero's performance on tonal instruments is due to tonality estimation algorithm in Nero AAC - it has been on my replacement schedule for a long time, but SBR development was taking all of my research time.
Now I am working on completely new tonality estimation method, and it is under in-depth tests. I hope it will manage to be completed until the next web release (actually, it is a work of couple of months, it is finally been debugged and now the parameters are being calibrated for each bit rate) - some initial feedback resulted in very positive ranking of older problem clips.
guruboolez: plese PM me if you want to test new tonality algorithm.
Once again Guru, you grace us with your amazing ears! Thank you for your hard work.
Very interesting that overall, WMAPro9 does so well. Considering the source of the test, it speaks volumes. People complain about DRM with WMA, but it seems to me there is an option to turn it off in the settings of the player. Since down the road more portables will support the newer MS codec, this is important.
I also find it interesting you find the Nero encoder to be such a poor performer, considering how it has obtained "darling" status around here as of late. With so many people finding it to be great, it demonstrates how much can be ignored during casual listening. You listen to nero files while surfing the web or answering email, your brain is not on an artifact "hunt"
I also find it interesting you find the Nero encoder to be such a poor performer, considering how it has obtained "darling" status around here as of late. With so many people finding it to be great, it demonstrates how much can be ignored during casual listening. You listen to nero files while surfing the web or answering email, your brain is not on an artifact "hunt"
If you read carefully, the results are here based on one particular music genre - classic music, and Nero still has some issues with tonality estimation (which are being worked out now) that directly affect classic music.
I am quite sure that if you include other genres that you will get different result
Fascinating results. Especially interesting to me since you used the same Vorbis -q setting I use for normal listening (though classical is in the minority of my listening, and I use Post 1.0 CVS rather than GT3b1, so my nominal bitrate's a little lower). Psy-models are different between the two versions, so I can't extrapolate too much to how Post 1.0 CVS could compare in this test, but these results are still important to know.
I'm very surprised to see the WMA-Pro results. Not that you rated it the highest overall, but how dramatic of a difference you found between it and most of the other formats (only 0.28 between it and second place QT AAC, but 0.71 between it and tied-for-third Vorbis and MPC). If/when WMA-Pro gets decent hardware support, it'll be a serious contender with this kind of performance (if it's anything close in other genre's as well, that is).
Thanks for the work you did on this, guruboolez!
Edit: Clarification.
I also find it interesting you find the Nero encoder to be such a poor performer, considering how it has obtained "darling" status around here as of late.
I think the "darling" status comes from the fact that Nero is currently one of the very few encoder which are rapidly evolving, and the developer (Ivan) is actively participating here.
What comes to Guru's results, they are really no surprise to me (and shouldn't be to Guruboolez either imo, he has sent quite a few harpsichord samples here). One of the biggest weaknesses of Nero AAC has been samples like harpsichord, pipes, and in general music with only a single very defined instrument which need very precise tonality estimation especially with VBR coding. Many of these samples have been posted to HA also, so the problem has been known. In that regard there's nothing new here.
Too bad for Ivan, only one very specific genre of music, like baroc, is Guru's favorite and is tested by him.
But, it indeed doesn't show the whole picture at all.
The new development versions have already had very remarkable improvement in this area, so again too bad for Ivan that Guru decided to do his big test already.
Nero AAC is going through rapid development phases, and it is already better in this area of music than the test here indicates. I think Guru will notice this too when he gets the chance to test the new versions.
Another thing. With this kind of music CBR/ABR mode has the advantage, because VBR quality and bit allocation depends heavily on whether tonality estimation is working quite flawlessly or not (this is emphasized with the tested low-medium bitrate vbr). But CBR/ABR allocates pretty much always so much bits that possible deficiencies in tonality estimation don't appear so easily. It would have been interesting to see how wma9 pro does with VBR, would it still be on top? That's another reason why people shouldn't make too conclusive conclusions about this test. It's no surprise that Quicktime does good. It's excellent with CBR.
Obviously I'm not surprised that the top two codecs in this tests use CBR/ABR.
And I also wonder why Guru chose to use ABR with WMA, but VBR with Nero. I'm pretty sure that the tested Nero codec would have performed clearly better in CBR mode with several of those samples... Not to mention that Nero vbr streaming is rated 110-120kbps average (although because of not well working TE in the tested codec version, it surely is often much lower), while Vorbis for example gets over 128kbps nominal vbr mode, although this is not the deciding factor here.
Maybe by the time I am ready to upgrade to Nero 6.0, the improvements Ivan is working on will be fully implemented. At this juncture, I am in no hurry to upgrade.
Having the new version of foobar2000 burn audio cds from just about any codec source, in dao mode, without worrying about a passle of plugins, has not inspired me to seek out Nero 6.0 yet Maybe after Christmas or so.
It would have been interesting to see how wma9 pro does with vbr, would it still be on top? That's another reason why people shouldn't make too conclusive conclusions about this test.
18 samples encoded with WMA VBR-2 PASS 128 : 7,31 Mo (7 673 606 octets)
18 samples encoded with WMA VBR QUAL 90 : 7,32 Mo (7 677 178 octets)
Bitrate are fortunately very close with one of the few offered VBR setting of wma pro. A fair comparison is possible. I'll probably made another one the next week
And I also wonder why Guru chose to use ABR with WMA, but VBR with Nero. I'm pretty sure that the tested Nero codec would have performed clearly better in CBR mode with several of those samples...
I didn't choose any single VBR pass setting, for practical reason. I didn't have too much experience with WMA PRO. Therefore, it's hard for me to estimate average bitrate of the samples I progressively chose during the test. The only data I have are these :
http://membres.lycos.fr/guruboolez/AUDIO/a...assical_VBR.txt (http://membres.lycos.fr/guruboolez/AUDIO/aac128/classical_VBR.txt)
http://membres.lycos.fr/guruboolez/AUDIO/a...etailed_VBR.txt (http://membres.lycos.fr/guruboolez/AUDIO/aac128/classical_detailed_VBR.txt)
VBR Q90 is nice for something near 130 kbps. But for classical music only, if I remember correctly the discussions around the call for data of Roberto's for the test he launched some months ago. ABR 128 was the more universal setting. And the safest too.
I didn't chose ABR/CBR with Nero, because I had bad experiences with it, compared to VBR -streaming. Lowpassing, for exemple, is really noticeable with CBR 128. This opinion is based on the few tests I made to defend Roberto's choice of QuickTime against Ahead -streaming.
I've no scientific knowledge about audio encoding. I'm a good HA pupil : I learned than VBR is the best way to obtain a good quality. If I had opt for CBR with Nero, many people will certainly asked me why, by a polite or less polite way (ask Roberto : he have good experience now of this kinf of controversy )
If you want, I can perform a new test during this week, by comparing WMAPRO ABR/VBR and Nero AAC ABR/VBR. I probably have time for this.
I have my revenge...thanks to guruboolez.
Thank you very much guruboolez !
One made fun of me because I used the WMA with the DRM...and now one finally proves the superiority of the WMA.
God heard me: victory!
WMA Rules.
That's it
One made fun of me because I used the WMA with the DRM...and now one finally proves the superiority of the WMA
Well, on some other music genres WMA is not that superior But for classical music it surely is not bad
Nick Jr III> Don't forget that WMA PRO is not traditionnal WMA. It's a different audio format.
WMA PRO is very competitive - WMA standard isn't. And WMA PRO is far from perfection. Try with sharp electronic music for exemple (death2.wav sample on Roberto's test was a good alarm). On a test based on this kind of music, I would bet that WMA PRO will be last, behind gogo mp3, on some interesting samples.
Nevertheless, you're right : WMA PRO need consideration. We can't admit primitive M$ bashing anymore.
Well, on some other music genres WMA is not that superior But for classical music it surely is not bad
I would like to know for which type of music the WMA is worse?
isn't this rather a mistrust towards WMA ?
God heard me: victory!
WMA Rules.
Right, here we have 1 test by 1 person with specific settings and quite homogenous samples. And people start this.
I could set up a test which gives completely different results, except that I probably couldn't do much to save FAAC..
Try with sharp electronic music for exemple (death2.wav sample on Roberto's test was a good alarm). On a test based on this kind of music, I would bet that WMA PRO will be last, behind gogo mp3, on some interesting samples.
too bad...
I would like to know for which type of music the WMA is worse?
isn't this rather a mistrust towards WMA ?
WMA has problems with percussions, music with attacks and strong stereo separation, etc...
I would like to know for which type of music the WMA is worse?
isn't this rather a mistrust towards WMA ?
WMA has problems with percussions, music with attacks and strong stereo separation, etc...
thank you for this reply, Ivan.
Problems ? even with WMA Pro ?
@128 - http://audio.ciara.us/test/128extension/results.html (http://audio.ciara.us/test/128extension/results.html)
Samples: 41_30sec, Death2, waiting
@64 - http://audio.ciara.us/test/64test/results.html (http://audio.ciara.us/test/64test/results.html)
Samples: experiencia, gone, mybloodrusts, polonaise, riteofspring, waiting
During the 64 kbps listening test of Roberto, WMA9 standard was used, not WMA PRO. This last format is really better at low bitrate than common WMA (in VBR mode, only available for low bitrate). I'm not sure that we can extrapolate results from one format to another.
P.S. I forgot to upload some samples last night. Is someone interested by the 18 clips ? 35 MB.
@128 - http://audio.ciara.us/test/128extension/results.html (http://audio.ciara.us/test/128extension/results.html)
Samples: 41_30sec, Death2, waiting
Thank you Ivan !
perhaps WMA 10 could...
regards,
Nick
In the meantime, Microsoft is still teh evil.
I also find it interesting you find the Nero encoder to be such a poor performer, considering how it has obtained "darling" status around here as of late.
I think the "darling" status comes from the fact that Nero is currently one of the very few encoder which are rapidly evolving, and the developer (Ivan) is actively participating here.
I don't know about other persons, but I like it because it gives (IMHO) clearly the best results for rock, metal and electronic music. WMA does quite bad here. in comparison. So I guess the music genre matters a lot, and I'm not a 'classical' person
I finally had time enough for a specific WMA comparison: WMA PRO ABR against WMA PRO VBR. Size are near the same (see above).
Some difference heard on ABA were not significant on ABX direct comparison between two encodings.
Nevertheless, slight but significative differences appeared :
- additional noise with VBR
+ reduced distortions with VBR
+ less artifact on specific case (drum roll) with VBR
http://membres.lycos.fr/guruboolez/AUDIO/t...WMA_ABR_VBR.htm (http://membres.lycos.fr/guruboolez/AUDIO/test_MF_early/WMA_ABR_VBR.htm)
I finally had time enough for a specific WMA comparison: WMA PRO ABR against WMA PRO VBR. Size are near the same (see above).
Some difference heard on ABA were not significant on ABX direct comparison between two encodings.
Nevertheless, slight but significative differences appeared :
- additional noise with VBR
+ reduced distortions with VBR
+ less artifact on specific case (drum roll) with VBR
http://membres.lycos.fr/guruboolez/AUDIO/t...WMA_ABR_VBR.htm (http://membres.lycos.fr/guruboolez/AUDIO/test_MF_early/WMA_ABR_VBR.htm)
WMA 9pro q90 gives me quite high bitrate on average, with other than classical baroc music. I'm getting pretty high bitrate on average, like closer to 170kbps. Did you choose q90 based only on the bitrates from the baroc music? If so, you probably gave it relatively quite high quality level...
Yes. WMA9 PRO VBR Q90 is bitrate friendly with classical. I had, of course, to encode many discs in order to have a more precise idea of average bitrate with classical, in general (not only baroque instruments).
I know that with other musical genre, Q90 gives higher bitrate. That's mainly why I choose ABR 128 for the whole listening test: no bitrate deviation. Roberto made the same choice for his 128 public test: Q75 is closer to 128 kbps with most music, but too low with classical samples.
PS : I forgot to say that the WMA test was made with another soundcard and another headphone (Terratec & Beyer, and not AC & Philips).
I know that with other musical genre, Q90 gives higher bitrate. That's mainy why I choose ABR 128 for the whole listening test: no bitrate deviation. Roberto made the same choice for his 128 public test: Q75 is closer to 128 kbps with most music, but too low with classical samples.
Imo you should have used Q75... If it gives too low bitrate on classical music, that's the codec's problem. You didn't tweak the other codecs' vbr quality levels either, rather used what generally gives about 128kbps with large selection of material (well, Nero streaming LC is somewhat lower; 110-120 on average).
I know that with other musical genre, Q90 gives higher bitrate. That's mainy why I choose ABR 128 for the whole listening test: no bitrate deviation. Roberto made the same choice for his 128 public test: Q75 is closer to 128 kbps with most music, but too low with classical samples.
Imo you should have used Q75... If it gives too low bitrate on classical music, that's the codec's problem. You didn't tweak the other codecs' vbr quality levels either, rather used what generally gives about 128kbps with large selection of material (well, Nero streaming is somewhat lower; 110-120 on average).
Yes, I have : I choosed --preset 134 for the same reason.
Don't forget that I did this test for classical only, and some practice purpose. I'm not going to rate Q75, and compare it to ABR 128, if average bitrate is near 90 kbps with my music.
If I had test some metal, or jazz gallery, Q75 would probably be my choice. Here, Q90 seemed to be more accurate.
N.B. Nero STREAMING gives higher bitrate with classical samples than with metal one. It's really close to 130 kbps. With some pure strings discs, average bitrate is superior to 140 kbps (not 100% sure though).
Only exception is mpc : --radio gives too high bitrate here, compared to others file format settings.
Yes, I have : I choosed --preset 134 for the same reason.
Don't forget that I did this test for classical only, and some practice purpose. I'm not going to rate Q75, and compare it to ABR 128, if average bitrate is near 90 kbps with my music.
If I had test some metal, or jazz gallery, Q75 would probably be my choice. Here, Q90 seemed to be more accurate.
N.B. Nero STREAMING gives higher bitrate with classical samples than with metal one. It's really close to 130 kbps. With some pure strings discs, average bitrate is superior to 140 kbps (not 100% sure though).
Only exception is mpc : --radio gives too high bitrate here, compared to others file format settings.
Right. Problem here is that you use relatively higher vbr quality level for one codec, based on how it acts with your music. You don't need to be a prophet to guess that wma9 pro q90 does pretty good, if it's average with vast selection of material is close to 170kbps (I can't say for sure is it). Q90 is pretty high quality level. When you set a vbr quality level, you set (simplified) a certain level of how much (audible) quantization noise is allowed in codec's opinion (but the point here is that psychoacoustics is often wrong, and if you artifically pump up the bitrate, you aren't getting very useful data generally, only for very specific case).
But if you encode other than classical music, you need to use q75 to be close to 128kbps. Your test doesn't say anything about Q75 which is more closer to 128kbps with non-classical and general music (all mixed). I hope people understand that these results are very very restricted on very specific cases. I fear most don't understand.
Ps. also don't look at the Nero Streaming bitrates of any development version yet.
Based on your file sizes, the Nero release codec you used stays clearly under 128kbps.
Also because vbr codecs do act differently, imo the only usable result to say anything about some codec's true quality on a certain quality level, is to use vast selection of material and use the average bitrate from that as indicator.
It matters not how it performs quality wise on other music, this test is for classical only so q90 was a fair setting.
If you want to do yor own test on a different type of music then please do, don't rubbish the results of this test.
IMHO, I think it actually *is* a real world scenario for some people to select incremental -q settings to target certain nominal bitrates for certain music types. For me, Vorbis (1.0) -q 4.00 averaged ~120kbps on a cross-section of my music, while -q 4.25 more closely hit my desired average of 128kbps (it actually averages ~129kbps) in the interest of getting my music encoded with as many bits as possible while not causing buffering problems on my more limited playback hardware. Other people may have other, comparable reasons, but I wouldn't venture whether this would be a majority or not. However, this would be one reason for a normal user to "focus" a codec's nominal bitrate for their musical taste by using not-so-general quality settings.
Also, I personally see the value in having both types of listening tests...the codec tests using a wide variety of music types as in Roberto's tests, and also "style-specific" tests, such as the one guruboolez has performed here. I've noted people saying (in this very thread as a matter of fact) that different codecs perform differently with different types of music, so that's clear justification to have "style-specific tests" to compliment the "general tests". And accordingly, quality settings / nominal bitrates should be chosen for the style-specific tests to make sure that each codec is throwing an equal number of bits at each sample, as closely as possible anyway.
If q75 "generally" provides 128kbps for a codec on most types of music, then that would be fine if you're testing "most types" of music. This was a test on Baroque-era classical, so crippling one codec by using a -q setting because it gives the desired-target bitrate with rock is not fair nor realistic because rock's not being tested. And for people who only listen to classical and not any rock, what nominal bitrate rock music hits with the codecs being tested will mean nothing to them.
By "focusing" each codec's -q setting for the specific type of music (based on the music's characteristics), you are measuring (I would think) more precisely how that codec uses the (approximate) same number of bits to encode the music vs. each other codec. It levels the playing field, so to speak. If a codec's q75 gives a significantly lower nominal bitrate (whether it's "supposed to" or not) with the type of music being tested than the other codecs give, then consider that a person who listens to this type of music (and not much else) would like to know which codec provides the best performance when each codec is generating very similar bitrates and filesizes with their music.
My $0.02.
I also agree with guruboolez' methodology. He's not trying to extrapolate results to other musical styles, he's testiong only one style. For that reason, adapting quality settings is acceptable.
Also, keep in mind that if he used q75, he would be flooded with whiner posts shouting "APPLES AND ORANGES! APPLES AND ORANGES!"
@guruboolez: Thanks a lot for another very interesting listening test. You're a legend
Based on your file sizes, the Nero release codec you used stays clearly under 128kbps.
I'm not sure. I didn't have exact average bitrate, only filesize :
mp3 abr 134 : 7 615 221
QT AAC cbr 128 : 7 856 071
WMA 128 : 7 673 606
Nero : 7 537 148
Maybe under 128 kbps, but really close to others encoding.
The problem with Nero AAC ont his kind of music : bitrate is often higher than with other musical genre, and with probably a worse quality than with other genres...
I hope people understand that these results are very very restricted on very specific cases. I fear most don't understand.
I hope too
Also, keep in mind that if he used q75, he would be flooded with whiner posts shouting "APPLES AND ORANGES! APPLES AND ORANGES!"
I don't see anybody shouting, even though MPC -radio gives clearly higher bitrate. I see just tweaking of some codecs, but not others. A bit mixed bag in my opinion.
Also, if the bitrate is tweaked on some codecs according to "classical baroc music 128kbps", I'd like to see it based on clearly more samples than 18. Let's say at least 5 or 10 CDs.. Why? Because the volume of the recordings can affect on the codec's bit allocation, regardless of music.
mp3 abr 134 : 7 615 221
QT AAC cbr 128 : 7 856 071
WMA 128 : 7 673 606
Nero : 7 537 148
Maybe under 128 kbps, but really close to others encoding.
Ok, hopefully I'm not counting wrong. Assuming QT AAC cbr 128 is 128kbps:
mp3 abr 134: 124.0 kbps
WMA pro 128 vbr-2pass 128: 125.0 kbps
Nero: 122.8 kbps
Vorbis -q4.25: 131.6 kbps
MPC -radio: 139.3 kbps
FAAC: 116.1 kbps
WMA9 pro q90 size is not visible anywhere, so can't really say...
I don't see anybody shouting, even though MPC -radio gives clearly higher bitrate. I see just tweaking of some codecs, but not others. A bit mixed bag in my opinion.
Erm.. read my post again. IF he did so, people would be shouting.
MPC radio indeed uses a higher bitrate, but not by a margin of 40kbps like would happen with WMA q75 vs. WMA q90.
Again, I completely agree with what Guruboolez did. For the "tweaking of some codecs, and not others": This is an ABR/VBR test, so allowing some codecs to scale for a little more or a little less is perfectly acceptable. But allowing a codec to scale to 40 kbps less is unacceptable! That would render completely unusable results for that codec.
I would say, from the fair amount of experience I have, that allowing a codec to scale to 10% more or less than the base bitrate (128, in this case) is acceptable. That's why I tried to limit the bitrates in my 64kbps test to scale in the range of 60 to 70kbps (WMA behaved badly, I admit). If you get higher deviation than 10%, issues of fairness and usefulness of the test start becoming critical.
Regards;
Roberto.
MPC radio indeed uses a higher bitrate, but not by a margin of 40kbps like would happen with WMA q75 vs. WMA q90.
Again, I completely agree with what Guruboolez did. For the "tweaking of some codecs, and not others": This is an ABR/VBR test, so allowing some codecs to scale for a little more or a little less is perfectly acceptable. But allowing a codec to scale to 40 kbps less is unacceptable! That would render completely unusable results for that codec.
Allowing codecs to scale 10% is acceptable, but imo you shouldn't mix 2 kind of testing methods in 1 test. Either you go with "quality level principle", or "bitrate tweaked principle", but both used in the same test for different VBR codecs (or some vague mixed method) and then refer to "10% acceptable scalability", I really don't know...
Also, I don't know if wma q90 (or q75) should be included in this test, because they just don't seem to give bitrates even close to 128 with Guru's samples. I tested the samples Guru posted with q90 and the bitrates were close to 150kbps and 170kbps. This is more than 10%. Some sample which had a long quiet section was 122 though.
So, you have this q90 on the upper part and FAAC with its 116kbps on the lower part. Doesn't really matter if FAAC was any other encoder, it still wouldn't be hard to say that it loses here, and I don't know what new information this brings. Those codecs which are "allowed" manually to scale, win.
I do think that the original test which didn't include wma9 pro q90 is more acceptable though, although I still don't like the mixing of testing principles.
I think JohnV is at least 99% right. This same thing bothered me with rjamorim 128kbps extension test. Because most of the encoders arent quality based they should't adjust to one quality. And encoder witch have quality scale, they are not comparable with each others. I always though that we where trying to find most efficien encoder at close as possible to one bitrate.
So one comment to this thread and no more whining for me.
While we are in WMA topic, how can we encode wma pro in a easy method ? Can we use Windows media player, or the only options for wma pro encoding are dbpoweramp and the windows media encoder who give me really hard time to encode something with ?
While we are in WMA topic, how can we encode wma pro in a easy method ? Can we use Windows media player, or the only options for wma pro encoding are dbpoweramp and the windows media encoder who give me really hard time to encode something with ?
i'd also like to know this. i also tried to encode a .wav with WMEncoder, and choose Q90 (VBR).. it made a 1-pass .wma that was identical in bitrate and size (down to bits) as to the same .wav file encoded in WMP9 with WMA9 Q90 (also that a 1-pass, cant seem to find a 2-pass option in any of these apps).. so that leads me to think that what i encoded with WMEncoder indeed was not a WMA9 Pro file..
I see two different kind of controversy here.
• First, "apple and oranges": I tweaked some encoders, and not another - that's bad.
• Second one: the choice of a wma VBR preset that isn't accurate.
I will try to answer to both problems with my little and approximate English. Sorry for all mistakes and my childish expression, but I really had to defend myself. Then, in a second message, I will add some capital precision about the version of Nero AAC I used, follow by another test. People interested by classical music may directly go to this last one.
I. MY CHOICES
I had to explain the methodology I used.
My goal was to build a good idea about encoders performance with the music I mostly listen. I had to find the best encoder for me, at a bitrate close to 130 kbps. Why 130 kbps and not less, or more? Not less, because the preliminary tests -very limited- I did (HE/LC-AAC, Vorbis) were not conclusive: I need something near transparency, and these low-bitrate encodings were too far from this state. Not more, because I'm short in space on my notebook. Therefore, something between 120 and 140 kbps is a good compromise. Of course, I prefer 120 to 140; but if at 140 kbps the quality jump is consequent, I can accept this slight sacrifice in disk space.
Then, for each encoder, I had to find the ideal setting: something between 120 and 140 kbps that will give me the best perceptible quality. I refused CBR, for obvious reasons : I ALWAYS read on HA than VBR is the best encoding method, followed by ABR, then by CBR., except some rare case (lame and low-mid bitrate VBR for example). I'm honestly surprised to read from JohnV that "with this kind of music CBR/ABR mode has the advantage, because VBR quality and bit allocation depends heavily on whether tonality estimation is working quite flawlessly or not" [JohnV, Posted: Oct 11 2003, 07:49 PM]. There are some people here listening to classical, and they would be happy to hear that ABR/CBR are better choice than VBR. Or is this affirmation applicable to low-mid bitrate only? Does it only mean that VBR is just using too much bits for a given quality and is, in one word, inefficient?
So, I chose VBR everywhere, except for QuickTime (no VBR, no ABR=>CBR only), LAME (no VBR preset=>ABR) and WMA (abr, for reasons I explained above, on a previous message).
After the choice of encoding method, I had to fix an adequate setting. I must precise that I didn't select the 18 samples before the listening phase, but progressively found them. In other words, I began with one sample, encoded it, listened to them, rated them, then I took another sample, encoded it, etc... This implies that I had fixed an encoding value for each encoder BEFORE I began the test. This implies too that I didn't proceed by any bitrate measurement phase. I fixed at the very beginning some settings, based on my own experience, others experience, or my total ignorance. Here is the explanation of my choice:
• Lame was set at ABR 134, because I have encoded a lot of classical with ABR for my hardware device, and I always saw lower bitrate (5-7 kbps) that what I asked. I use this general correction for every encoding I made or will make with LAME. Lame ABR is not accurate with classical : it's the case since Dibrom removed something like a bitrate correction during the --dm-preset era (he explicitly did that two years ago - I suppose that he could confirm this minor change, and if not, consider that claim as a memory flaw, or as a product of my fertile imagination). Therefore, --preset abr (x + 5...7 kbps) is what I would suggest for each people encoding classical. Note that final result was really nice: exactly the same size as QuickTime CBR 128 !
• Vorbis was set at -b 4.25, just because this value seemed to be the best according to the public results Roberto asked for some months ago. I used it too, because I often saw Vorbis giving lower bitrate with classical than with other genres at the same bitrate. Results told me that -b4 may be a better choice, but it was too late.
• Faac was set a q 128, because I totally ignore the behaviour of this encoder. I chose 128 value, just because Lame, QuickTime and WMA were intended to be close to 128. BTW, I include Faac encoder just because I had a free place in ABC/HR: I didn't expect Faac to be competitive – I just found interesting to have some idea about the quality of this encoder (used on some audioripper, as dBpowerAmp [Edit: after verification, dBPoweAmp is using fastenc AAC] or EasyCDDA Extractor). Same thing for TNS and M/S stereo: I didn't know how they will perform; I used just them: by ignorance of this encoder.
• WMA9 was set at ABR 128, just because I know pure VBR to be too much fluctuant (between 60 and 240 kbps for two different stereo classical tracks). ABR was the safest choice, even if it may not be the best one (VBR generally > ABR). I wrongly thought it will be the less problematic setting...
• I chose VBR for Nero, because some test I did some months ago were clearly in favor of -streaming against CBR 128. I haven't idea about ABR performances. The fact was, that I forgot that Nero AAC has an ABR mode - and even if I had remembered it, I would chose again VBR, for the theoretical benefits of this encoding method. I took -streaming profile, for two reasons. First, I noticed that average bitrate with pop music is visibly inferior to 128 kbps; but I noticed too that with classical (especially baroque, strings), bitrate is really close to 128 kbps, and sometimes higher than this symbolic value. Second reason: -normal profile is clearly too high for the bitrate range I set. Again, the fact gave me reason: with 2.5.5.8 encoder, average bitrate for the 18 samples was exactly 128 kbps - the encoding of the entire 18 discs would probably not being too far from this value.
• Same thing for PsyTEL : VBR against CBR/ABR, even if bitrate is commonly a bit higher than 128 kbps.
As you can see, I didn't perform any kind of tweak. The last proof of my claim is the mpc choice I made. I chose --radio setting, even if I knew that this profile gives higher bitrate that average value calculated on a wide variety of musical genres. I did this in order for two reasons:
• TO NOT TWEAK the encoder.
• TO KEEP the entire COHERENCY of my test (stay with the most immediate and easy to get setting for mpc : -q4, and not something like -q 3.68).
Just imagine the reproach, if I had compare mpc --thumb profile area, against the others competitor...
Obviously, I didn’t tweak anything for this test[/i]. The changes I made were made before the test: I used the settings I will use for all encodings I will make for listening purpose. I can’t be blame for having some knowledge about bitrate behaviour with classical listening. Is it a fault to anticipate some variations, and correct them with my own experience I accumulate during two years ? In my opinion, definitely not.
It's really hard for me to understand what you said by a "mixing of testing method". It's not very pleasant too to learn that the effort I did for this test is ruined because I "mix 2 kind of testing method", and that my results are just something like tolerable, "acceptable" [JohnV, Oct 14 2003, 05:58 AM].
II. WMA Q75 or WMA Q90
As I said before, I had in mind to test early music only, with a great dominant of baroque samples. The performances in quality, or the behaviour in bitrate variations, that both occurs on Metallica-like music, is something I don't care AT ALL. I’m not going to use an audio format giving excellent results on electronic but weak ones with violin, if I’m Paganini’s fan. In the same way, a person interested by a ~180 kbps audio format is not going to reject mpc –standard because complete works of Domenico Scarlatti (33 CDs) reach 220 kbps on average with the same preset.
Personally, I'm not rejecting pure knowledge, and I also would be interested by both bitrate and quality performance with Ride the lightning. But I'm not going to take these performances in count, on music I don't listen to, just in order to make pleasure to all people listening metal music every day.
After all, ff I never heard about HA, I would probably never see full album reaching 170 kbps with WMA9 PRO Q90. For my own purpose, based on my own -preset and future- experience of classical encoding, Q90 is the VBR setting close to 130 kbps.
Then, WMA VBR encoding performances are designed by a very abstract way: Q10, Q25...Q98. I don't see anywhere Microsoft claiming that Q75 is intended to be close to 120 kbps, nor that Q90 setting should be close to 180 kbps. Therefore, I found difficult to consider as a "codec problem" [JohnV,Oct 13 2003, 01:53 PM] the average bitrate of Q90 setting with classical music. I don't see why we should say that with classical music bitrate is "too low" [JohnV,Oct 13 2003, 01:53 PM]. Isn't the contrary valid too? Why isn't the bitrate considered as "too high" with others genre?
So, I can't judge the measured bitrate with classical as a problem. Of course, it would be one, if overall quality was bad, betraying any flaw of the psy-model or something similar. But it isn't the case. WMA VBR gives me something like 140 kbps (+10%) with classical music; therefore, I expect some improvement in quality compared to ABR 128. Quality is slightly improved compared to ABR (except increase in noise), therefore, VBR is doing its job.
As a conclusion, I would say that I didn't consider at all as a "problem", or as a fault, WMA Q90 performances: quality and bitrate are coherent, at least, on the 18 samples I tested. I need valid argument to accept the idea of a "problem": nobody on HA is blaming mpc --standard for the very low bitrate measured on piano encoding, or saying that's a "problem". Au contraire, everybody applaud on the clever behaviour of well-tuned VBR encoder. As long as no audible or unexpected problems are detected, a VBR encoder is allowed to reduce the needed bitrate without encountering any blame. This is at least what I conclude after reading HA during two years. The same policy used for mp3, mpc and vorbis should be applied for WMA PRO without an hesitation or hair-splitting.
I finally opted for Q90, just because I knew that it's close to 128 kbps. In order to help Roberto for his listening test, I've encoded two anthologies of classical music in order to build a good idea about bitrate behaviour of each profile. Two discs, but 30 tracks, from 30 different discs, including recording of various genre, period and even recording techniques (two or three of them were mono - some others very noisy). Average bitrate for these 30 tracks at Q90 is 131 kbps. At Q75, bitrate is 88 kbps. With the 18 samples I choose, coming from 18 different discs (all stereo), bitrate were very similar: 139 & 93 kbps. Incidentally, results were published, here, on HA. I even gave the links on page 1. Therefore, I was surprised to read the ask for encoding "5 or 10 CDs. Why? Because the volume of the recordings can affect on the codec's bit allocation, regardless of the music" [JohnV, Oct 13 2003, 05:27 PM]. 48 tracks & samples, coming from 48 different classical discs, are not enough for giving a positive idea about average bitrate?
Few others of your arguments are annoying me a bit – probably bad luck: "I tested the samples Guru posted with q90 and the bitrate were close to 150 kbps and 170 kbps". Three only are superior to 170 kbps (Dorilla, Questo core and Viva Rey Ferrando). 13 are inferior to 150 kbps. I know that I didn’t have uploaded all 18 files, so I can’t put the shame on you. Nevertheless, it’s a VBR test: and it surely not very conclusive to anticipate the average bitrate of the whole gallery with only the quarter of them. Using this biased argument against the validity of my test is something I don’t really understand. Nor like.
Q90 is, with the limited experience I have, and the limit I fixed for my test, the fairest challenger for a comparison with ABR or CBR 128 kbps. Q75 is clearly inferior to 100 kbps. Now, do you really consider as serious, or fair, or scientific, a test comparing WMA at 90 kbps against others encoders, measured between 120 and 140 kbps? Personally, I don't. To be honest, I found it unreasonable.
Other thing I don’t (literally – my limited English is guilty) understand: "those codecs which are 'allowed' manually to scale, win.". Does it mean that scalable encoders are real winners? Winners of the test are the two less tweakable encoders: QuickTime AAC (112-128-160 kbps steps) and WMAPRO ABR (128-192-256 kbps). And the biggest values aren’t the best one: vorbis (135 kbps) and mpc (142 kbps) were clearly behind the two strict 128 kbps encodings.
You don't like the idea of mixing different principles, I don't like it too.
Problem is, if people had to choose what they expect to be the best encoding at a given bitrate, they have the right to compare CBR, ABR and VBR, at a given bitrate. You can't forbid someone using a VBR preset, useful with the kind of music he listen to, just because this preset is useless with other kind of music. You can't forbid this person to compare this VBR setting with some CBR settings, because few encoders don't offer VBR or ABR method. Encoders are not developed for being played in ABC/HR only. Encoders are not developed for people listening various genre of music only. You are maybe right by saying that we need to separate two different groups of encoders for testing purpose, in order to respect some testing principle uniformity. But a castrated test is absolutely useless for practical purpose. If one people is asking for building a 200 kbps mp3 library, and hesitate between CBR 192 and --preset standard, what would be your answer? VBR preset? Or will this answer look something like: "I can't answer this, because the two settings are not directly comparable, and I don't like the idea of mixing two different testing principles"?
I did a test with the settings I would use for a practical and daily use. Nothing was arranged by me in order to scale the average bitrate of 18 grain-of-sand samples on a desired value. I used unilateral principle, simulated a real usage by using the same setting for both daily listening and occasional testing. In one word, it was totally coherent.
There's one moment where you're obliged to test each encoders, with their best weapons, even if the test looks like a confuse bric-à-brac of encoding principles. That's what I made. I find it coherent, and useful in the same way.
Complete table: bitrate, size and notation (http://membres.lycos.fr/guruboolez/AUDIO/test_MF_early/Bitrate_Table_complete.htm)
Warning: I obtained all bitrate values by using a simple rule of three, and using the average value of QuickTime AAC CBR 128 and WMA ABR 128 as reference. I removed all tags before calculation (except replaygain tags)
EDIT: Crossed dBpowerAmp name.
The new .dll of Ivan Dimkovic I tested this week gave me the same bitrate than the encoder I used for the initial test. Audio stream is bit-to-bit identical. Explanation: I mixed up Nero5 and Nero6 encoder path. This is annoying, because it implies that I didn’t test the latest Nero encoder [2.5.5.8] during first listening test (8-challengers). The encoder used was an old version, from this summer, one of the first including he-aac profile. Exact number is 2.5.1.6…
I’m confused, especially for Ivan. I’ve just denied four month of his work. These months are probably more important for Nero AAC than for other encoders, as musepack or vorbis. Nero AAC is actually a young encoder, and progress is probably constant, and fast.
Therefore, before I start a new listening test, comparing old and experimental encoders, I decide to add a third encoder in the arena:
• 2.5.1.6 – old & uninteresting version of aacenc32.dll I tested first with 7 other challengers
• 2.5.5.8 – the latest released version of aacenc32.dll, bundled with Nero 6.0.19
• 2.5.6.2 – the experimental version of aacenc32.dll, named 2.5.6.2, that Ivan kindly sent me.
By testing 2.5.1.6 and 2.5.5.8, I can obtain a precise idea of the progress of the encoder, in order to evaluate 2.5.5.8 progress and to estimate the virtual position of this encoder in the 8-challengers test.
I initially planed to ask Ivan agreement before publishing results of an unofficial encoder. But results are so positive, and because I did a first and deplorable mistake last week, I have decided to bypass its authorization. Results are here:
http://membres.lycos.fr/guruboolez/AUDIO/t...Grille_Nero.htm (http://membres.lycos.fr/guruboolez/AUDIO/test_MF_early/Grille_Nero.htm)
Note that notation should not be directly compared with notes I gave few days ago, with others competitors.
Initial comment:
- bitrate of 1.6 & .5.8 encoders are really close. 126 kbps against 128 kbps. More problematic for my purpose, the value I obtained for 2.5.6.2 –streaming: 148-149 kbps. +16%. Of course, it’s a less interesting bitrate for my purpose, and a really annoying one for my test (excessive if compared to most challengers). It’s the biggest deviation of the whole test (I’m not considering Q75 suggestion as a serious one: -27%). I honestly don’t have any idea of the average bitrate I would obtain with other musical genres. Nevertheless, for my favorite music, and the targeted bitrate area I’m looking for, this new Nero AAC encoder need a downgrade to –internet profile to stay in bitrate competiton.
Quality jump is really impressive, and bitrate inflation couldn’t be invoked to explain the whole progress. Some of the worst distortions audible with 2.5.5.8 encoder at –streaming profile (average 128 kbps) are indeed gone with 2.5.6.2 –internet (124 kbps).
Some progress are clearly audible between 2.5.1.6 and 2.5.5.8 release. For exemple, Dorilla sample: destroyed with the old encoder, now noise-free with the newest (though bad distortions). Passacaglia (organ sample) encoded with 2.5.1.6 was clearly the worse one during the 8-challengers test. With 2.5.5.8, it’s near transparency, as other formats. But if I except these two samples, the overall progress is really slim: small improvements on some samples, small degradation on others. Finally, the progress between 2.5.1.6 and 2.5.5.8 is real, but it’s more like a bug correction, with audible and considerable effect on limited situations. 2.5.5.8 is probably closer to PsyTEL performances than to Faac ones. That’s a good step, but the major one, for my needs, will be the next aacenc32.dll release.
Bitrate and size are available on the same table as before:
membres.lycos.fr/guruboolez/AUDIO/test_MF_early/Bitrate_Table_complete.htm
Note that I carefully checked the integrity of each PCM and each encoded files. I noticed some corruption and discordance between some encoding sets, and I corrected them.
The new .dll of Ivan Dimkovic I tested this week gave me the same bitrate than the encoder I used for the initial test. Audio stream is bit-to-bit identical. Explanation: I mixed up Nero5 and Nero6 encoder path. This is annoying, because it implies that I didn’t test the latest Nero encoder [2.5.5.8] during first listening test (8-challengers). The encoder used was an old version, from this summer, one of the first including he-aac profile. Exact number is 2.5.1.6…
Ok, well it's very good you found this rather umm fundamental issue..
More problematic for my purpose, the value I obtained for 2.5.6.2 –streaming: 148-149 kbps. +16%. Of course, it’s a less interesting bitrate for my purpose, and a really annoying one for my test (excessive if compared to most challengers).
I don't think you should publicly post results about development builds. Send your results to Ivan privately. First of all, like you noticed, the profiles of 2.5.6.2 and further are not yet tweaked to correspond the changes in psychoacoustics.
I don't think you should publicly post results about development builds. Send your results to Ivan privately. First of all, like you noticed, the profiles of 2.5.6.2 and further are not yet tweaked to correspond the changes in psychoacoustics.
As I said it, I first had in mind to do it. But I did a big mistake last week, giving to a wrong and outdated version of Nero AAC encoder, a really bad notation. Therfore, I think that these new promising results would be welcome. There are some fan of Ivan works: I had to compensate for my mistake...
There are only results, comparing Nero to Nero. Exciting values, or promising words as yours in first page, are so different?
Nevertheless, if Ivan want it, I can remove the link and correct the html table (or you, JohnV, if Ivan is asking this before I read it).
I don't think you should publicly post results about development builds. Send your results to Ivan privately. First of all, like you noticed, the profiles of 2.5.6.2 and further are not yet tweaked to correspond the changes in psychoacoustics.
As I said it, I first had in mind to do it. But I did a big mistake last week, giving to a wrong and outdated version of Nero AAC encoder, a really bad notation. Therfore, I think that these new promising results would be welcome. There are some fan of Ivan works: I had to compensate for my mistake...
There are only results, comparing Nero to Nero. Exciting values, or promising words as yours in first page, are so different?
Nevertheless, if Ivan want it, I can remove the link and correct the html table (or you, JohnV, if Ivan is asking this before I read it).
Well, it's fine by me, as long as people remember that your 2.5.6.2 results (http://membres.lycos.fr/guruboolez/AUDIO/test_MF_early/Grille_Nero.htm) are results from a development build which is still evolving before the next stable..
Well, it's fine by me, as long as people remember that your 2.5.6.2 results (http://membres.lycos.fr/guruboolez/AUDIO/test_MF_early/Grille_Nero.htm) are results from a development build which is still evolving before the next stable..
Should be OK now
Thanks Guru.
What comes to your explanation about the test.. well, it was long.
The biggest questional thing still imo was the q90 (the winner), which has "offset" very high, and gives according to you about 140 with these samples. As I said earlier, imo neither q75 or q90 was very good for this test. Imo q90 is fundamentally in different class than others. It's like driving a Ferrari 140 km/h and claiming that it's in the same class with an old Volkswagen which can barely reach 120km/h, because they are going nearly as fast. Considering the q90 overall average bitrate is very high (Ferrari), and vbr bitrate with your samples near 140, and considering that between 120-140 the quality difference of vbr codecs can be relatively much higher than for example between 140-160...
Anyway, this is just my opinion and others can and probably will disagree.
I don't think it does anybody any good to continue chewing this. In any case, your test wasn't flawed in that sense, that there's reason to invalidate it. Imo it could have been a bit better in some ways, but I think that is also a matter of opinion. Well the usage of a quite old Nero codec was quite considerable technical mistake though, but fortunately you noticed it and reacted (You might want to correct your original message's Nero codec version number also)
And in any case HA needs blind testings and testers with good hearing like you, so I hope you weren't discouraged (from the length and tone of your explanation, it seems you weren't, which is good. )
Thank for your positive answer. I spent long hours to wrote my defence, and I expected your comments. I'm not to continue the debate, because I don't want to spend my week-end in an english dictionnary
The first message is now corrected, including precisions about the encoder version I unfortunately used.
I'm not discouraged anymore (but I was last week). Afterall, I'm doing tests for my own usage, not only for the others.
Finally, I noticed that WMA9PRO VBR90 is probably flawed: I've saw some low-intensity stereo piano tracks with a bitrate inferior to 60 kbps, and here, distortions are easily noticeable (but I didn't ABX them). If I could confirm this, this will end the debate: ABR128 would be the safest competitor (and the more universal) for this bitrate area.
Quicktime 6.4 (released with iTunes for Windows) claims enhanced AAC encoding. By enhanced I don't know if Apple means speed or quality or both (can I hope?). How 'bout re-running your test using just WMA Pro and AAC using QuickTime 6.4? Enquiring minds want to know...
Kind of off-topic, but still a bit on-topic (as i didnt want to start a single thread for a simple question) :
What would be the best way to use the WMA9 Pro codec in? as there is no 16-bit 44khz 2-channel preset to use, so my question is, is there a negative side on using 24-bit on samples that are 16 bit ?
I also encoded the From Dusk Till Dawn soundtrack with both WMA9 Pro (Q90, VBR, 44 kHz, 2 channel 24 bit) and WMA9 Std (Q90, VBR, 44 kHz, stereo 16 bit), and i noticed that the filesizes was very different, the filesize off the WMA9 Pro encode was actually alot smaller than the one encoded with WMA9 Std - examples:
Track 13, Tito & Tarantula - After Dark:
---------------------------------------------
WMA9 Pro filesize: 5,16 MB (5 415 279 bytes)
WMA9 Pro bitrate: 171 kbps
WMA9 Std filesize: 6,31 MB (6 625 061 bytes)
WMA9 Std bitrate: 224 kbps
Track 2, The Blasters - Dark Night:
----------------------------------------
WMA9 Pro filesize: 4,51 MB (4 735 743 bytes)
WMA9 Pro bitrate: 180 kbps
WMA9 Std filesize: 5,09 MB (5 345 409 bytes)
WMA9 Std bitrate: 205 kbps
To be honest, i think it's strange that a 24bit file is smaller than a 16bit one, and i would also asume that the Pro codec would generate bigger filesizes/bitrates at the same quality level ? Has anyone done a good test off the WMA9 Std. codec and compared it with the Pro codec, at Q90 (VBR, no 2-pass "ABR") ?
While we are in WMA topic, how can we encode wma pro in a easy method ? Can we use Windows media player, or the only options for wma pro encoding are dbpoweramp and the windows media encoder who give me really hard time to encode something with ?
There comes with wmencoder (or wmencoder sdk, don't remember) a windows script to encode a file (windows media encoding script).
Later, I found on usenet an modified version where you can specify the artist/genre/album/etc tags.
With that script I can now encode WMA9Pro or Std directly from cdex. I can mail a copy of the script if somebody is interessted.
I Just made my own little experiment with WMa9pro 128K vbr 2 pass with the first track of AC/DC album "High voltage". The intro of that track (just a raw guitar) seems to be a killer for nero AAC @ 128 kbits (sounds like a cellular phone), for wma9std 128 vbr 2 pass (sounds like a robot) and for wma9pro 128 vbr 2 pass (you hear cracks, just like a vinyl LP !).
My solution was to encode wma9 Q90. Both Wma9pro or std seems to sound good, but the wma9std file was bigger (I remember it was >5Megs) than the wma9pro one (it is 4.39 Megs). Fast generalisation conclusion : wm9pro q90 seems good quality/compression ratio and safer than wma vbr 2 pass.
With that script I can now encode WMA9Pro or Std directly from cdex. I can mail a copy of the script if somebody is interessted.
Please send it to me by email: borge_i@hotmail.com
My solution was to encode wma9 Q90. Both Wma9pro or std seems to sound good, but the wma9std file was bigger (I remember it was >5Megs) than the wma9pro one (it is 4.39 Megs). Fast generalisation conclusion : wm9pro q90 seems good quality/compression ratio and safer than wma vbr 2 pass.
WMA9 Pro/Std 2-pass isn't actually VBR, its more like ABR. 1-pass is true VBR (atleast it seems this way).
As in my examples a bit up, WMA9 Std filesize is much bigger than WMA9 Pro.. but still people say that WMA9 Pro has much better encoding quality..
dBPowerAmp Music Converter is very good for ripping/encoding to WMA9 Pro i think, but i'd still like to try your script, maybe i could use that with EAC then.
To be honest, i think it's strange that a 24bit file is smaller than a 16bit one, and i would also asume that the Pro codec would generate bigger filesizes/bitrates at the same quality level ? Has anyone done a good test off the WMA9 Std. codec and compared it with the Pro codec, at Q90 (VBR, no 2-pass "ABR") ?
When talking about WMAPro "24bit" just means that it is capable of encoding audio files with 24bit resolution. But since we are talking about lossy mode here: As you probably know, lossy codecs save space by 'throwing away' or adding indaudible information, e.g. low volume signals masked by loud signals. At normal volume information carried by bits # 17 ... 24 is masked anyway. So a lossy compressed file created from a 24bit source won't need more space than one created from a 16 bit source in most cases.
WMA9 Pro is not just an extension of WMA9 Std. It's a different codec. If WMA9 Pro is *better* than Std. it should produce more transparent quality at identical file sizes OR smaller file sizes at identical/comparable quality, shouldn't it? That's exactly what you observed.