Greetings!
I am planning to start a multiformat listening test at 128 kbps on
November, 30th which should end on December, 11th. The test start can be postponed if necessary.
Edit: The test has been postponed: Stard date is December 5th and end date is December 25th if everything goes well.
Edit: The discussion about codecs is now
over.
The following codecs are going to be included:
- iTunes / QuickTime AAC
- Nero Digital 4.2.1.0
- LAME 3.97
- Ogg Vorbis, AoTuV 4.5
- WMA Professional 9.1
- Shine as low anchor
I'm open to suggestions regarding the settings to use. The only thing that's for sure is iTunes / QT AAC which will be used at 128 kbps VBR.
I'd like to ask for Garf's or Ivan's opinion regarding Nero Digital (CBR or VBR, etc.).
As for the rest of the formats, suggestions from the community are welcome, too.
I have some samples in mind already and we'll start discussing them soon. While at it, would you guys prefer 18 or 12 samples?
Regards,
Sebastian
IMHO
1. MusePack - on the last test it performed very well at that bitrate.
2. WMA - lot of people use it - it would be good to prove WMA is not the best codec as lot of people in the world think.
So my vote for MusePack and WMA.
I'm with de Mon.
wma standard and pro
Hi. Thanks for your efforts in organizing the test.
I would suggest skipping musepack as well for the reasons already mentioned and also because it is rarely used at this bitrate.
WMA standard would be the low anchor here I think.If WMA Pro is finding hardware support these days it might be useful to have it blind tested as well.
It breaks my open source loving heart that FAAC is not more actively developed. If it has not changed significantly then testing is probably not warranted at this time.
$.02
There is room for two other competitors or one competitor and a low anchor. Maybe MusePack and some version of WMA (Professional or Standard)?
There should be one WMA at least, otherwise the retards populating Slashdot will claim it wasn't tested because we are afraid our beloved codecs would be owned badly by it.
I wouldn't feature MPC. But that's what you should expect from an MPC hater (http://www.hydrogenaudio.org/forums/index.php?showtopic=35030&view=findpost&p=308437) like me.
While at it, would you guys prefer 18 or 12 samples?[a href="index.php?act=findpost&pid=341355"][{POST_SNAPBACK}][/a]
18. Otherwise a well know person around here will use the few samples as an excuse for his format of choice not winning, in case it loses (and if it wins, he'll stay quiet)...
WMA standard would be the low anchor here I think.
[a href="index.php?act=findpost&pid=341373"][{POST_SNAPBACK}][/a]
For the low anchor, I suggest using something that sounds bad for sure, like l3enc.
WMA standard would be the low anchor here I think.
[a href="index.php?act=findpost&pid=341373"][{POST_SNAPBACK}][/a]
For the low anchor, I suggest using something that sounds bad for sure, like l3enc.
[a href="index.php?act=findpost&pid=341388"][{POST_SNAPBACK}][/a]
Atrac?
ATRAC3 has certainly improved since SonicStage 2.0 (especially in 2.2 I think) but would still be the low anchor. Probably more interesting than including another MPEG layer 3 codec like l3enc. So I vote for ATRAC3 @ 132kbps with SonicStage 3.2 or 3.3.
New Nero AAC LC/HE Encoder is 4.2, not 3.0...
So I vote for ATRAC3 @ 132kbps with SonicStage 3.2 or 3.
[a href="index.php?act=findpost&pid=341397"][{POST_SNAPBACK}][/a]
If most people are against MusePack... we could really include ATRAC. With ATRAC included - the test will break TWO BIG MYTHES - about WMA and ATRAC superiority which is quite widespread in other communities.
So I vote for ATRAC3 @ 132kbps with SonicStage 3.2 or 3.
[a href="index.php?act=findpost&pid=341397"][{POST_SNAPBACK}][/a]
If most people are against MusePack... we could really include ATRAC. With ATRAC included - the test will break TWO BIG MYTHES - about WMA and ATRACK superiority which is quite widespread in other communities.
[a href="index.php?act=findpost&pid=341403"][{POST_SNAPBACK}][/a]
Maybe. The last time the atrac community just attacked the testing methodology used.
Die Hards........
So I vote for ATRAC3 @ 132kbps with SonicStage 3.2 or 3.
[a href="index.php?act=findpost&pid=341397"][{POST_SNAPBACK}][/a]
If most people are against MusePack... we could really include ATRAC. With ATRAC included - the test will break TWO BIG MYTHES - about WMA and ATRACK superiority which is quite widespread in other communities.
[a href="index.php?act=findpost&pid=341403"][{POST_SNAPBACK}][/a]
This sounds like a good idea, and at the bitrate that the test will be done, the results should be very interesting.
If most people are against MusePack... we could really include ATRAC. With ATRAC included - the test will break TWO BIG MYTHES - about WMA and ATRAC superiority which is quite widespread in other communities.[a href="index.php?act=findpost&pid=341403"][{POST_SNAPBACK}][/a]
Didn't we do it already more than an year ago?
Testing ATRAC3 is beating a dead horse, IMO, even if it improved since SS 2.0. I doubt it got much better (Sony has other things to worry about, like DRMing their format to the limit and spreading rootkits throughout the world), it's forcedly DRMd - DRM is not an option like in WMA - and even though it was bad, it wasn't bad enough to act like an anchor. The anchor is there to avoid people ranking samples too low just because there isn't anything worse, even though said samples are prefectly acceptable otherwise.
IMO, good anchors would be old or weird encoders from RRW (like l3dec, MBaacenc, QDesign, AUPECg2), or a lowpass.
If you go with WMA, I'd suggest to go for WMA Standard since WMA Pro does not play on any portable device and thus is completely useless in a real-world-scenario.
... (Sony has other things to worry about, like DRMing their format to the limit and spreading rootkits throughout the world) ...
If you go with WMA, I'd suggest to go for WMA Standard since WMA Pro does not play on any portable device and thus is completely useless in a real-world-scenario.
[a href="index.php?act=findpost&pid=341413"][{POST_SNAPBACK}][/a]
I agree with that.
I don't - retards from Slashdot (which rjamorin mentioned) will claim that Pro isn't included because of its superiority (and that it will Soon be supported by portables anyway...)
since WMA Pro does not play on any portable device and thus is completely useless in a real-world-scenario.[a href="index.php?act=findpost&pid=341413"][{POST_SNAPBACK}][/a]
Erm... you could use pretty much the same argument for MPC. And that didn't keep it from being tested on my older tests.
I don't - retards from Slashdot (which rjamorin mentioned) will claim that Pro isn't included because of its superiority (and that it will Soon™ be supported by portables anyway...)
Indeed, I agree. F*** SlashDot. We're not testing for the clueless hordes populating that place.
I just thought of another reason not to use ATRAC3: Sebastian is a nice guy. He doesn't deserve to feel the infinite pain involved with installing and using SonicStage
By the way, has WMA Pro ever really been compared in a listening test like this? If so, could I be provided with a link? If not, could it be included in this test? I'm curious to see how good it really is.
If so, could I be provided with a link?[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=341435")
[a href="http://www.rjamorim.com/test/128extension/results.html]http://www.rjamorim.com/test/128extension/results.html[/url]
Quite outdated though.
L3enc would be an interesting low anchor, as it would allow to check evolution of "state of the art" encoders. However, it might be a little too good for a low anchor.
However, it might be a little too good for a low anchor.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=341438")
Even 1.0?
[a href="http://www.rjamorim.com/rrw/l3enc.html]http://www.rjamorim.com/rrw/l3enc.html[/url]
A low anchor would be nice, but personally I would probably prefer WMA Standard and WMA Pro.
I think ATRAC would be a waste of time because:
1. It's already pretty well known by most people who actually do any real research on the topic that it's inferior to just about all modern alternatives.
2. It's fading in popularity, and will continue to do so quite rapidly in the near future.
3. The people that feel that ATRAC are superior seem quite resistant to persuasion by evidence.
So, ATRAC isn't really a worthwhile choice.
I don't think MPC is really a good choice either, because:
1. It isn't really used at this bitrate (not explicitly that is, although VBR may sometimes cause it to average out at this bitrate), even if it can perform well.
2. I don't know if it's fading in popularity, but it's not really growing either. In terms of relevancy for the results, I think MPC would be less useful for a large audience.
3. It hasn't seen any serious development in a long time, whereas other alternatives might have, and would be more interesting from a results perspective -- MPC's performance is pretty well established either way, so it could serve possibly as some sort of anchor, but beyond that it's not going to provide much new data.
I understand that a low anchor is important, especially in order to provide a reference point illustrating just how much some of the codecs in this test have improved either upon recent iterations, or older/inferior codec designs in general.
Maybe the best choice would be to add WMA Standard and WMA Pro, AND a low anchor. That has the danger of making the test too tedious and complex, but maybe it is still doable...
I don't understand why everyone insist to inlude WMA Standard instead of the Pro version... You don't cripple other formats the same way, but wma everybody wants WMA to be bad and just to make sure it is, only test the ancient WMA Std. Ask Woodinville which version to use, and I'm pretty sure he suggest using Pro instead of Std.
I definitely would like to see both WMA Standard and Pro included. I think testing against a lower anchor can be done in an another test. You can say whatever you want about WMA, but it at least had gapless support way before some of the other codecs had it (and some still don't have it, yes, I'm looking at you Apple). And I'm very much interested in the differences between WMA Standard and Pro.
QUOTE(JeanLuc @ Nov 13 2005, 12:23 AM)
If you go with WMA, I'd suggest to go for WMA Standard since WMA Pro does not play on any portable device and thus is completely useless in a real-world-scenario.
*
I agree with that.
I think we can drop WMA or even FAAC safely now, there's no improvements (version is still the same) since the last test, it is fine to just refer to the previous results
Unlike iTunes, Nero, LAME...they are all improved
WMA Pro vbr 128 kbps 2-pass is not in the previous test, it is interesting and more meaningful to add WMA Pro this time
I think we can drop WMA or even FAAC safely now, there's no improvements (version is still the same) since the last test, it is fine to just refer to the previous results
Actually, if you're referring to the Multiformat 128kbps listening test done by rjamorim in May 2004,
WMA has since been upgraded to version 9.1, a fact that seems to escape many people. Microsoft has done a rather poor job of making this widely known, so it's understandable that it's flown well below the radar of most listeners.
Besides that, it's the codec supplied on the majority of new PCs sold today. Omitting it from the test would be like omitting the Honda Accord from a roundup of family sedans.
Actually, if you're referring to the Multiformat 128kbps listening test done by rjamorim in May 2004, WMA has since been upgraded to version 9.1, a fact that seems to escape many people. Microsoft has done a rather poor job of making this widely known, so it's understandable that it's flown well below the radar of most listeners.
I was under the impression the WMA Std did not get an upgrade with the WM9.1 release. Has anyone actually compared the output of 9 and 9.1 to see if there is in fact an update?
Ogg Vorbis, AoTuV 4.5 if it's OK with Aoyumi
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=341355")
I just received green light from Aoyumi. He also told me to use a quality preset, so I think I'll go with something around 4 (exact quality level depends on the samples and the resulting average bitrate).
I just thought of another reason not to use ATRAC3: Sebastian is a nice guy. He doesn't deserve to feel the infinite pain involved with installing and using SonicStage
[a href="index.php?act=findpost&pid=341430"][{POST_SNAPBACK}][/a]
Well, just in case...
[a href="http://www.transilvania2000.com/www.maresweb.de/SonicStage.png](http://www.transilvania2000.com/www.maresweb.de/SonicStageT.png)[/url]
Running on a virtual machine with Windows 98.
I was under the impression the WMA Std did not get an upgrade with the WM9.1 release. Has anyone actually compared the output of 9 and 9.1 to see if there is in fact an update?
I assure you that Microsoft didn't just change the version from 9 to 9.1 just for fun. Forums regarding the topic:
http://www.hydrogenaudio.org/forums/index....showtopic=31029 (http://www.hydrogenaudio.org/forums/index.php?showtopic=31029)
and
http://www.hydrogenaudio.org/forums/index....topic=27037&hl= (http://www.hydrogenaudio.org/forums/index.php?showtopic=27037&hl=)
According to the second link, there is an ABX-audible difference between the two versions. I use WMA for my personal ripping purposes, and the main improvement I've noticed is a bitrate increase for each VBR Quality setting. At the highest setting it's a nice compromise between the quality of the lossless and the compatibility of the lossy versions of the format.
I don't understand why everyone insist to inlude WMA Standard instead of the Pro version... You don't cripple other formats the same way, but wma everybody wants WMA to be bad and just to make sure it is, only test the ancient WMA Std. Ask Woodinville which version to use, and I'm pretty sure he suggest using Pro instead of Std.
One is playable on portable players, the other one is not.
Btw, they are NOT the same format, but two different formats.
I vote for Atrac3 and WMA Standard because I have a feeling that Atrac3 could beat WMA nowadays.
I don't understand why everyone insist to inlude WMA Standard instead of the Pro version... You don't cripple other formats the same way, but wma everybody wants WMA to be bad and just to make sure it is, only test the ancient WMA Std. Ask Woodinville which version to use, and I'm pretty sure he suggest using Pro instead of Std.
One is playable on portable players, the other one is not.
Btw, they are NOT the same format, but two different formats.
[a href="index.php?act=findpost&pid=341482"][{POST_SNAPBACK}][/a]
And not even Windows Media Player supports ripping to WMA Pro.
And not even Windows Media Player supports ripping to WMA Pro.
[a href="index.php?act=findpost&pid=341487"][{POST_SNAPBACK}][/a]
The freely available Windows Media Encoder 9 does. WMA Pro is aimed more at content producers than consumers such as most of us.
For what it's worth, here's my take.
My preffered linup would be Nero, iTunes, Vorbis, Lame, WMAStd, WMAPro, and a Low Anchor. Obviously, this introduces one more codec than on previous 128kbps tests conducted by Roberto. In the first 128kbps test their were six competitors and no low-anchor. Theoretically, the low anchor should be easy to ABX from the original, thus not really require any extra time or effort. As such I think having seven codecs is acceptable, as long as one is a low anchor.
As far as bitrates go, you'll probably want to independantly verify, but I've encoded 291 tracks from an assortment of genre's using the new Nero codec (4.2.1.0). The closest preset I found was the VBR/Stereo - Internet 90-100 kbps [LC AAC]. On my test tracks, this provided an average bitrate of 133kbps. Some may consider this too far from the target bitrate, but I do believe it is probably not possible to get closer to the target of 128kbps.
For anyone about to bring up the issue of fairness, I quote Roberto:
The quality settings chosen for the VBR codecs were chosen because they average out to about 128kbps over a number of encoded albums. It would be unfair to tie the hands of VBR codecs and punish them for being smart about where to spend what turns out to be the same number of bits over the long run.
For reference, at 133kbps, my tracks total 1024mb. If the average were 128kbps, they would total 985mb. This represents only a 4% increase of bitrate/filesize. I would imagine the margin of error for this test is going to negate a 4% bitrate increase.
WMA Std will probably make a suitable low anchor, since this is a low bitrate test after all. WMA Pro is an interesting contender, and has not been tested adequately.
ATRAC3 has been tested to death, and the ppl who still use it are not interested in listening test results. Keep the rootkit company's software safely away
seeing as how HA is seen as some-what reliable source for audio info (writers from Wired come here) i think WMAPro should be tested just to prove that Pro doesnt always meen better (see http://www.rjamorim.com/test/64test/results.html (http://www.rjamorim.com/test/64test/results.html) and http://www.rjamorim.com/test/32kbps/results.html (http://www.rjamorim.com/test/32kbps/results.html) for example).
Theoretically, the low anchor should be easy to ABX from the original, thus not really require any extra time or effort. As such I think having seven codecs is acceptable, as long as one is a low anchor.
The problem with several codecs is not only the fatigue, but the need to iterate codecs several times to come to a ranking among them. I realized that at my low bitrate tests. Even though there was very little fatigue involved, as reference was instantly recognizable for most cases, people spent too much time checking samples again and again to see who won over who.
Some may consider this too far from the target bitrate, but I do believe it is probably not possible to get closer to the target of 128kbps.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=341501")
In my tests, the limit I tried to pursue was 10% bitrate difference from the format consuming least bits to the format consuming most. Therefore, 133 would not be a problem as long as there is no format consuming less than 120.
seeing as how HA is seen as some-what reliable source for audio info (writers from Wired come here) i think WMAPro should be tested just to prove that Pro doesnt always meen better (see [a href="http://www.rjamorim.com/test/64test/results.html]http://www.rjamorim.com/test/64test/results.html[/url] and http://www.rjamorim.com/test/32kbps/results.html (http://www.rjamorim.com/test/32kbps/results.html) for example).[a href="index.php?act=findpost&pid=341521"][{POST_SNAPBACK}][/a]
Neither of these tests featured WMA Pro. Pro actually doesn't even go down to 32kbps, and only goes down to 64kbps using a VBR mode that's completely unreliable as far as bit allocation goes.
I'd say ATRAC doesn't really have to be tested since it's used by a very limited user group, unlike MP3, AAC, Vorbis or WMA.
Also, I don't think that the ATRAC lovers will accept the test results anyways.
Out of curiosity, I'm interested how WMA Pro, ATRAC3 or Musepack would compete. But nevertheless, WMA Standard and FhG-fastenc are more often used with commercial audio-software.
I suggest Coding Technologies aacPlus @ 64 kb/s as low anchor. Maybe it's quite close to 128 kb/s with some samples.
I suggest Coding Technologies aacPlus @ 64 kb/s as low anchor. Maybe it's quite close to 128 kb/s with some samples.
[a href="index.php?act=findpost&pid=341556"][{POST_SNAPBACK}][/a]
Sorry? If this encoder has 64 kbit/s CBR mode, how do you expect it to get close to 128?
As for me, I vote for WMA Std and Pro to be included in this test. And as low anchor... maybe FAAD or Blade (less likely).
Sorry? If this encoder has 64 kbit/s CBR mode, how do you expect it to get close to 128?
[a href="index.php?act=findpost&pid=341559"][{POST_SNAPBACK}][/a]
Sounds like he means quality, not bitrate. Somehow I believe that there really are some samples to which this would be a reasonable assumption. So including 64 kbps HE AAC would be interesting indeed and so how far low bitrate codecs have came since 1995.
But somehow I believe that including a lower bitrate codec without including FhG @ 128 would be unapproprite.
Out of curiosity, I'm interested how WMA Pro, ATRAC3 or Musepack would compete. But nevertheless, WMA Standard and FhG-fastenc are more often used with commercial audio-software.
[a href="index.php?act=findpost&pid=341556"][{POST_SNAPBACK}][/a]
I don't think MusePack should be tested at 128 kbps because it was optimized for higher bitrates. As for ATRAC3, see my comment above.
Including WMA is another story.
One more question regarding samples - what would you say about using the same sample set as in Roberto's last multiformat 128 kbps test?
One more question regarding samples - what would you say about using the same sample set as in Roberto's last multiformat 128 kbps test?
[a href="index.php?act=findpost&pid=341570"][{POST_SNAPBACK}][/a]
Shouldn't this be discussed in it's own thread?
Sounds like he means quality, not bitrate. Somehow I believe that there really are some samples to which this would be a reasonable assumption. So including 64 kbps HE AAC would be interesting indeed and so how far low bitrate codecs have came since 1995.
[a href="index.php?act=findpost&pid=341567"][{POST_SNAPBACK}][/a]
True, I didn't get that. Maybe because I was assumpting that even low anchor encoder should get close to 128 kbps bitrate...
Sorry? If this encoder has 64 kbit/s CBR mode, how do you expect it to get close to 128?
[a href="index.php?act=findpost&pid=341559"][{POST_SNAPBACK}][/a]
Sounds like he means quality, not bitrate. Somehow I believe that there really are some samples to which this would be a reasonable assumption. So including 64 kbps HE AAC would be interesting indeed and so how far low bitrate codecs have came since 1995.
But somehow I believe that including a lower bitrate codec without including FhG @ 128 would be unapproprite.
[a href="index.php?act=findpost&pid=341567"][{POST_SNAPBACK}][/a]
Well, the development of low-bitrate codecs should be tested separetely, I guess. I had a disussion about a 64 kbps listening test a few months ago, but the test was cancelled because Apple didn't release an HE-AAC encoder as I expected and at that time, the new Nero version was also about to come out "soon".
One more question regarding samples - what would you say about using the same sample set as in Roberto's last multiformat 128 kbps test?
[a href="index.php?act=findpost&pid=341570"][{POST_SNAPBACK}][/a]
Shouldn't this be discussed in it's own thread?
[a href="index.php?act=findpost&pid=341576"][{POST_SNAPBACK}][/a]
Well, yes, you are right, but I wanted to see what you think about the idea before opening a special thread just for this.
For my own listening test (test is finish - results tomorrow), I used:
• AAC: Nero Digital VBR
• AAC: Apple iTunes VBR
• MP3 LAME 3.98 VBR
• Vorbis: aoTuV VBR
as low anchor, a very old AAC encoder
as high anchor, LAME 3.97 -V2 --vbr-new
I discarded atrac3+, MPC or WMA (both Std and PRO). Two of them are not usable with portable players (I don't cound PDA in this category), and the two remaining (atrac3+ and WMAstd) are really poor and simply uninteresting. Testing them is fine for curiosity, but for practical usage it's a waste of time IMO.
The closest preset I found was the VBR/Stereo - Internet 90-100 kbps [LC AAC]. On my test tracks, this provided an average bitrate of 133kbps. Some may consider this too far from the target bitrate, but I do believe it is probably not possible to get closer to the target of 128kbps.
[a href="index.php?act=findpost&pid=341501"][{POST_SNAPBACK}][/a]
I got 125,7 kbps for Nero Digital -internet high with my 150 reference tracks.
For the samples "non-classical" samples, I got 133 kbps with the same setting.
Note if iTunes will be tested in VBR mode, there will be the same issue. iTunes AAC works as iTunes MP3 in VBR mode: the target bitrate also corresponds to the minimal bitrate. In other words, the encoder don't encode anything below 128 kbps (apart digital silence maybe). Consequently, the bitrate is necessary superior to 128 kbps. I got 133.33 kbps for classical and 137.x for non-classical.
Same issue will occur with LAME -V5.
Only exception is aoTuV, which benefits for a very flexible VBR mode.
These problems might be a good reason to discard all CBR encoder from the test and to use VBR encodings only outputting to the same approximate bitrate.
LAME -V5, iTunes AAC and Nero Digital are fortunately is this case. Same for Vorbis of course. I'm not sure that WMA VBR (Std and Pro) could be close to ~130 kbps. If they're not, it may end the debate about using or not this format in the test.
kl33per, or someone else, could you try different WMA settings to see if a 1-pass VBR mode could stay close to 130-135 kbps?
Sorry for posting this in three replies...
About WMA or ATRAC as low anchor:
- during the first multiformat test, Blade was used as low anchor. Final note = 1.99.
http://www.rjamorim.com/test/128extension/results.html (http://www.rjamorim.com/test/128extension/results.html)
- during the second listening test, WMA standard get a much higher note (3.65).
In other words, WMA seems to be too good to be considered as low anchor.
Sorry for posting this in three replies...
About WMA or ATRAC as low anchor:
- during the first multiformat test, Blade was used as low anchor. Final note = 1.99.
http://www.rjamorim.com/test/128extension/results.html (http://www.rjamorim.com/test/128extension/results.html)
- during the second listening test, WMA standard get a much higher note (3.65).
In other words, WMA seems to be too good to be considered as low anchor.
[a href="index.php?act=findpost&pid=341710"][{POST_SNAPBACK}][/a]
Yes, as mentioned already. If we discard ATRAC, we could test WMA and feature a low anchor. WMA is used by a lot more people than ATRAC.
Whether the standard encoder should be tested because it works on portables and it's also available in a lot of music stores or the professional due to its higher quality is uncertain. What do you guys think?
If we discard ATRAC, we could test WMA and feature a low anchor. WMA is used by a lot more people than ATRAC.
Whether the standard encoder should be tested because it works on portables and it's also available in a lot of music stores or the professional due to its higher quality is uncertain. What do you guys think?
[a href="index.php?act=findpost&pid=341721"][{POST_SNAPBACK}][/a]
I'd say include both. It would be interesting to see how their quality are related to each other and whether pro is able to compete with the best codecs out there, and it would make wma supporters silent, they couldn't complain about the lack of wma pro in the test.
Yes, as mentioned already. If we discard ATRAC, we could test WMA and feature a low anchor. WMA is used by a lot more people than ATRAC.
Whether the standard encoder should be tested because it works on portables and it's also available in a lot of music stores or the professional due to its higher quality is uncertain. What do you guys think?
[a href="index.php?act=findpost&pid=341721"][{POST_SNAPBACK}][/a]
I think including WMA standard is a must: It is widely used by end users, plays on practically all portables and is used by most music download stores.
The problem with WMA Pro, IMO, is that Microsoft never targeted it at the end user market, and probebly never will.
It's simple to notice that: All online music stores are selling WMA Std, and Windows Media Player won't rip your CDs to WMA Pro either. Microsoft isn't pushing it for portable player adoption either.
Therefore, I wonder what would be the real use of testing it. Microsoft itself doesn't want us using it.
If I knew Microsoft would never push it for the consumer market, I probably wouldn't have featured it even in my first multiformat test. I only did it because I believed, at the time, that Pro was set to replace Std.
If it is correct that MS will never target WMA Pro to the end user market, I agree. But with the competition from AAC, I think it is strange that WMA Pro won't replace WMA Std at some point. Personally, I think it will.
What about itunes mp3 as the low anchor? It's got to be the most popular of the old mp3 encoders and lots of people still use it. It scored pretty miserably at the last mp3 test
What about itunes mp3 as the low anchor? It's got to be the most popular of the old mp3 encoders and lots of people still use it. It scored pretty miserably at the last mp3 test
[a href="index.php?act=findpost&pid=341750"][{POST_SNAPBACK}][/a]
Well, it lost, but 3 is still pretty high as low anchor.
The problem with WMA Pro, IMO, is that Microsoft never targeted it at the end user market, and probebly never will.
It's simple to notice that: All online music stores are selling WMA Std, and Windows Media Player won't rip your CDs to WMA Pro either. Microsoft isn't pushing it for portable player adoption either.
Therefore, I wonder what would be the real use of testing it. Microsoft itself doesn't want us using it.
If I knew Microsoft would never push it for the consumer market, I probably wouldn't have featured it even in my first multiformat test. I only did it because I believed, at the time, that Pro was set to replace Std.
[a href="index.php?act=findpost&pid=341742"][{POST_SNAPBACK}][/a]
Actually, recent Windows Vista Betas now have WMA Pro as a ripping option in WMP 11. Perhaps MS is deciding to make it mainstream after all. If it's going to be a standard feature in the next gen mainstream OS, it would make sense to include it in the test for the purpose of comparison.
Actually, recent Windows Vista Betas now have WMA Pro as a ripping option in WMP 11. Perhaps MS is deciding to make it mainstream after all. If it's going to be a standard feature in the next gen mainstream OS, it would make sense to include it in the test for the purpose of comparison.
[a href="index.php?act=findpost&pid=341757"][{POST_SNAPBACK}][/a]
But by then, this test will be outdated. For at least the next two years or so, WMA Pro is targeted at professionals who use it at high resolutions and higher bitrates than 128kbps. Whenever WMA Pro becomes a consumer codec (for use with portables/music stores etc), I agree it should be tested.
Somebody mentioned also VBR...it would be nice to include CBR vs. VBR comparison for MP3. Too bad it would be too much work.
And I still think that WMA Pro should be included, people will have harder time discrediting this test.
Maybe it would be better to define the scope of the test, and then codecs could be included or excluded at the hand of that.
Now I see many arguments (popularity, completeness, support for portables) for and against inclusion of certain codecs, but any of those arguments are only good or bad if you know exactly what it is exactly you are trying to test.
If you just want to test what (on average) the best codec is, then you should definetly include WMA Pro. But if you want to test what the best codec for portable use is, it shouldn't be included.
Since I couldn't really find exactly what is being tested, the discussion about yes/no certain codecs seems a bit messy to me.
Maybe it would be better to define the scope of the test, and then codecs could be included or excluded at the hand of that.
Now I see many arguments (popularity, completeness, support for portables) for and against inclusion of certain codecs, but any of those arguments are only good or bad if you know exactly what it is exactly you are trying to test.
If you just want to test what (on average) the best codec is, then you should definetly include WMA Pro. But if you want to test what the best codec for portable use is, it shouldn't be included.
Since I couldn't really find exactly what is being tested, the discussion about yes/no certain codecs seems a bit messy to me.
[a href="index.php?act=findpost&pid=341764"][{POST_SNAPBACK}][/a]
I agree. It all depends on the scope of what formats the generous tester wants to listen to. If it is more of a mainstream listening test then I don't think WMA pro should be included but ATRAC3 should definately be included as well as WMA std, mpeg-4 AAC (iTunes), Nero AAC, and Lame 3.97b1. If it is more of a test to include a broad range of formats then WMA pro, WMA std, AAC (iTunes and Nero), Lame 3.97b1, and other formats should be included. This is a lot of work for the listener and I appreciate all his work.
Personally, I would like to see ATRAC3 just to show people on the PSP boards how bad the format is (or convince other people who dive into the Sony formats). I would also like to see WMA std as this format is everywhere. I would also like to top the rumors that a 64kbps WMA is the same as a 128kbps mp3. I would also like to see QuickTime 7 (iTune 6) and Nero 7's AAC encoders going up against Lame 3.97b1.
Again, thanks for the work.
side note, maybe 11 days is too little? I would rather have it open by 20 days.
What about using the encoder with the oldest sourcecode as a low anchor: blade. It should reliably deliver the worst result (and if it doesn't, that's a very interesting result too), and will show how much mp3 has actually progressed.
With sony dropping minidisk, i'd personally favour wma pro over atrac for inclusion in the test.
What about using the encoder with the oldest sourcecode as a low anchor: blade. It should reliably deliver the worst result (and if it doesn't, that's a very interesting result too), and will show how much mp3 has actually progressed.
With sony dropping minidisk, i'd personally favour wma pro over atrac for inclusion in the test.
[a href="index.php?act=findpost&pid=341807"][{POST_SNAPBACK}][/a]
I second that.
What about using the encoder with the oldest sourcecode as a low anchor: blade.[a href="index.php?act=findpost&pid=341807"][{POST_SNAPBACK}][/a]
That criteria is kinda wacko. You mean the oldest open source MP3 encoder? Then it would probably be 8hz-MP3. The oldest MP3 encoder is L3enc (and hey, it's based on source code!). But there are even older encoders like Indeo Audio, IMA ADPCM and Lucent's PAC.
What about using the encoder with the oldest sourcecode as a low anchor: blade.[a href="index.php?act=findpost&pid=341807"][{POST_SNAPBACK}][/a]
That criteria is kinda wacko. You mean the oldest open source MP3 encoder? Then it would probably be 8hz-MP3. The oldest MP3 encoder is L3enc (and hey, it's based on source code!). But there are even older encoders like Indeo Audio, IMA ADPCM and Lucent's PAC.
[a href="index.php?act=findpost&pid=341814"][{POST_SNAPBACK}][/a]
I think he meant using Blade because it's not much different than the original specification source code (and it was well spread) which I have read a couple of times on these boards.
...
[a href="index.php?act=findpost&pid=341764"][{POST_SNAPBACK}][/a]
...
[a href="index.php?act=findpost&pid=341780"][{POST_SNAPBACK}][/a]
Popular formats, that's why I think ATRAC should be left out.
Since WMA Standard is more popular than Professional, I'd also go with Std. and a low anchor like l3enc.
If you really want to include a low anchor, then I'd vote for HE-AAC. And the other contender should then be WMA Standard. While I'd like to see how it compares to WMA Pro, the lack of compatibility and encoding tools make WMA Pro pretty much useless.
my vote would be L3enc as the low anchor, then nero, itunes, vorbis, lame, and unfortunately, both wma std and pro. these tests are benchmarks, and they really need to be as conclusive as possible. testing 7 encoders will certainly be a tedious task for all of the participants, but we, the ha.org crowd are a tough bunch, we can handle it. maybe we might consider leaving the test open for a few extra days to get more participants...
i just really think we would be remiss to leave out wma std because it is so widely used, or wma pro, because many "audiophiles" will argue that it is a viable competitor to the other encoders tested (and it likely will be more viable than wma std, at least). therefore, its performance will need to be tested against the other new encoders...
just my 2 cents. and btw, many thanks to sebastian mares for volunteering to set this up for all of us.
If you really want to include a low anchor, then I'd vote for HE-AAC. And the other contender should then be WMA Standard. While I'd like to see how it compares to WMA Pro, the lack of compatibility and encoding tools make WMA Pro pretty much useless.
[a href="index.php?act=findpost&pid=341859"][{POST_SNAPBACK}][/a]
Use HE-AAC as a low anchor, force Nero to output a 128kbps HE-AAC stream
Then use the test results as a graphic reminder of why HE-AAC should never, ever be used above 64kbps.
It's a better low anchor than random_ancient_MP3codec IMHO.
I would like to see how Atrac3Plus performs. I am an owner of Sony portable device and I use it because the device is able to play Atrac files gaplessly. I use only 256kbps with that format but I accidentally once used 64kbps A3P and was surprised that it didn't sound THAT BAD. So I'm quite curious. I really would like if the word "Plus" actually means any progress and step to quality(I've never used the old Atrac so I don't know how it sounds like).I think HA is the only place where I can find the unbiased results without any hype.
Btw, SonicStage 3.3 (the last one since it's being replaced with the Connect Player) support ATRAC3+ at 128kpbs (vs older ATRAC3 132kpbs)
And since Sony's MP3 players, CD, Hi-MD and the PSP(?) support both MP3/ATRAC3, it could be interesting to have it in
Some peoples may be tempted to convert to ATRAC3+ since playback time is about 25% longer (lower power consumption I guess)
I'd like to see WMA Pro included, because I'm curious to see if Microsoft is actually capable of producing a competitive audio codec.
I have to ask a basic/important question: Who are the results of this test targeted at?
If it's the frequenters of HA then one set of codecs may be appropriate, but if it's general consumers of 128kbps digital audio (either purchased or home-brewed) then the codecs chosen should be relevant to the general consumer market (current, popular, accessible).
I vote for WMA STD, and a low anchor. We can come back to WMA PRO in 2007 once Vista is released (the codec may well have changed/updated by then anyway).
Popular formats, that's why I think ATRAC should be left out.
Since WMA Standard is more popular than Professional, I'd also go with Std. and a low anchor like l3enc.
In that case WMA Std, but not Pro, yes. Although, as low anchor maybe it is interesting to use HE-AAC(v2)...
Btw, SonicStage 3.3 (the last one since it's being replaced with the Connect Player) support ATRAC3+ at 128kpbs (vs older ATRAC3 132kpbs)
And since Sony's MP3 players, CD, Hi-MD and the PSP(?) support both MP3/ATRAC3, it could be interesting to have it in
Some peoples may be tempted to convert to ATRAC3+ since playback time is about 25% longer (lower power consumption I guess)
[a href="index.php?act=findpost&pid=341946"][{POST_SNAPBACK}][/a]
Yes, but how many people use ATRAC with that hardware? I'd say only the minority - the majority using MP3 for compatibility reasons.
I have to ask a basic/important question: Who are the results of this test targeted at?
If it's the frequenters of HA then one set of codecs may be appropriate, but if it's general consumers of 128kbps digital audio (either purchased or home-brewed) then the codecs chosen should be relevant to the general consumer market (current, popular, accessible).
I vote for WMA STD, and a low anchor. We can come back to WMA PRO in 2007 once Vista is released (the codec may well have changed/updated by then anyway).
[a href="index.php?act=findpost&pid=341950"][{POST_SNAPBACK}][/a]
As stated, the test should feature popular formats which you would probably meet in "everyday's life". 128 kbps is also an excellent bitrate for portable players, music store downloads, etc.
As you mentioned, WMA Pro should be tested when it's used by the large mass.
Btw, SonicStage 3.3 (the last one since it's being replaced with the Connect Player) support ATRAC3+ at 128kpbs (vs older ATRAC3 132kpbs)
And since Sony's MP3 players, CD, Hi-MD and the PSP(?) support both MP3/ATRAC3, it could be interesting to have it in
Some peoples may be tempted to convert to ATRAC3+ since playback time is about 25% longer (lower power consumption I guess)
[a href="index.php?act=findpost&pid=341946"][{POST_SNAPBACK}][/a]
Yes, but how many people use ATRAC with that hardware? I'd say only the minority - the majority using MP3 for compatibility reasons.
[a href="index.php?act=findpost&pid=342036"][{POST_SNAPBACK}][/a]
I think a lot of peoples download/own MP3s at 320kpbs CBR and transcode/re-encode when they transfer it to thier flash memory device
Some may be also ripping a CD directly to it
Yes, but still, compared to WMA, it's used by far less people and that's why I think it shouldn't be included. The question is whether it's really important to replace the low anchor with another format.
WMA (pro or std) has to be tested and I think most of you agree. It's too important to be left out. So this makes room for really another competitor or a low anchor. I'd go with the second.
Hey, you're running the test, so you decide what to include. You can't wait for everybody to come to consensus, that's just nog going to happen.
I would suggest you wait until January to choose the codecs. If you do not, you may obsolete your test.
I think we can drop WMA or even FAAC safely now, there's no improvements (version is still the same) since the last test, it is fine to just refer to the previous results[a href="index.php?act=findpost&pid=341456"][{POST_SNAPBACK}][/a]
I though you were required to have evidence for making such assertions.
I think we can drop WMA or even FAAC safely now, there's no improvements (version is still the same) since the last test, it is fine to just refer to the previous results[a href="index.php?act=findpost&pid=341456"][{POST_SNAPBACK}][/a]
I though you were required to have evidence for making such assertions.
[a href="index.php?act=findpost&pid=342064"][{POST_SNAPBACK}][/a]
Identical version numbers aren't good enough for you?
I would suggest you wait until January to choose the codecs. If you do not, you may obsolete your test.
Why?
I think we can drop WMA or even FAAC safely now, there's no improvements (version is still the same) since the last test, it is fine to just refer to the previous results[a href="index.php?act=findpost&pid=341456"][{POST_SNAPBACK}][/a]
I though you were required to have evidence for making such assertions.
[a href="index.php?act=findpost&pid=342064"][{POST_SNAPBACK}][/a]
Identical version numbers aren't good enough for you?
WMA's been upgraded to version 9.1 across the board (std, pro, lossless, voice) since the last test, in case you missed the memo (courtesy of the absence of an official announcement from MS, most people seem to have, so don't worry ). There have been improvements. FAAC I don't know much about at all.
In addition, the Pro arm of the WMA codec is set to be included among the ripping options available in WMP 11, so progress is being made.
Identical version numbers aren't good enough for you?[a href="index.php?act=findpost&pid=342066"][{POST_SNAPBACK}][/a]
I wonder, have you explored WMP 10's encoding options?
Identical version numbers aren't good enough for you?[a href="index.php?act=findpost&pid=342066"][{POST_SNAPBACK}][/a]
I wonder, have you explored WMP 10's encoding options?
[a href="index.php?act=findpost&pid=342102"][{POST_SNAPBACK}][/a]
Well, no. I misread kotrtim's statement.
Identical version numbers aren't good enough for you?[a href="index.php?act=findpost&pid=342066"][{POST_SNAPBACK}][/a]
I wonder, have you explored WMP 10's encoding options?
[a href="index.php?act=findpost&pid=342102"][{POST_SNAPBACK}][/a]
Few people do, yet they are usually the first to say something derogatory about WMP or WM formats at any chance they get without having even tried either of them. It's practically prejudicial.
Personally I don't comment on formats I have no direct experience with. As such, the formats I'd like to see compared are: WMA 9.1 Standard, WMA 9.1 Pro, LAME 3.97 MP3, OGG (not familiar with the specs) and MPC (same) at the very least. Is there any merit to including an FHG MP3 encoder also for comparative purposes? I'm curious to see how well it does against LAME.
Personally I don't comment on formats I have no direct experience with. As such, the formats I'd like to see compared are: WMA 9.1 Standard, WMA 9.1 Pro, LAME 3.97 MP3, OGG (not familiar with the specs) and MPC (same) at the very least. Is there any merit to including an FHG MP3 encoder also for comparative purposes? I'm curious to see how well it does against LAME.
[a href="index.php?act=findpost&pid=342109"][{POST_SNAPBACK}][/a]
When reading the results I would also want all of those included, but while doing the actual testing I would want as few as possible, and absolutely no more than 5 + low anchor... High anchor is useless in a 128 kbit test. My choice would be: Vorbis, Lame, iTunes, WMA Pro + low anchor. If people insist to have a 5:th competitior, then maybe include MPC or Nero or WMA Std or ATRAC or something else that was discussed before....
Really, though, what is the fascination with WMA? Pro may be a significant improvement but the closed nature of the format offers it no benefits over the competitors, other than hopeful inclusion in future hardware. I'd like to see Pro in the mix if only to show that it has nothing to offer over the competitors, quality-wise. Yes, I'm making a fairly baseless prediction.
Yes, but still, compared to WMA, it's used by far less people and that's why I think it shouldn't be included. The question is whether it's really important to replace the low anchor with another format.
WMA (pro or std) has to be tested and I think most of you agree. It's too important to be left out. So this makes room for really another competitor or a low anchor. I'd go with the second.
[a href="index.php?act=findpost&pid=342055"][{POST_SNAPBACK}][/a]
WMA Pro is a good contender choice, it's shipped with Vista beta already which makes it already more widespread than ATRAC3 will ever be.
Plus I just want to see how good a WMA product with the label "Pro" could be
Really, though, what is the fascination with WMA? Pro may be a significant improvement but the closed nature of the format offers it no benefits over the competitors, other than hopeful inclusion in future hardware. I'd like to see Pro in the mix if only to show that it has nothing to offer over the competitors, quality-wise. Yes, I'm making a fairly baseless prediction.
[a href="index.php?act=findpost&pid=342116"][{POST_SNAPBACK}][/a]
^emphasis mine
As far as I know, the fact
in itself that a project is either open or closed source does not affect the quality of its output. An open source project with poor developers will be outdone by a closed source project with good developers, and the reverse is true also.
As to the "fascination" with WMA - it's a common format. Get used to it. That's just a fact. WMA Pro will become more common also when WMP 11 comes out, as screenshots show it being included as an encoding option. As such, they should be compared. That doesn't mean it will necessarily triumph or lose, but there can't be a fair comparison if a major competitor is not included.
That's just a fact. WMA Pro will become more common also when WMP 11 comes out, as screenshots show it being included as an encoding option.[a href="index.php?act=findpost&pid=342119"][{POST_SNAPBACK}][/a]
WMP9 beta included PRO encoding. It included even lossless encoding as an option. (I remember burning audio CDs full of samples just to test PRO, because WMP wouldn't take WAV as input, only CDDA). Both options disappeared in the final version.
I see too much wishful thinking in this thread. People here are pushing PRO much more than Microsoft itself.
I see too much wishful thinking in this thread. People here are pushing PRO much more than Microsoft itself.
[a href="index.php?act=findpost&pid=342130"][{POST_SNAPBACK}][/a]
I think it's sort of hard to know what Microsoft is planning with Pro, so maybe everyone should just stop making assumptions one way or another.
The question should be: Is WMA Pro worth testing?
I think the answer is probably yes. Whether or not it is actually being marketed one way or another is pretty much irrelevant for two reasons:
1. It has the potential to be competitive in this test.
2. It has the potential to become a widespread format.
Most of the other alternatives proposed for testing in place of WMA Pro (most notably ATRAC) fail to meet one or both of those conditions.
Given that one of the presumed goals of this test is to highlight progress in various encoding technologies, and given the fact that WMA is very widespread, it really doesn't make a lot of sense to me to leave Pro out of the picture -- especially when just about everyone here knows that WMA Std is going to most likely be inferior to Pro across the board. Leaving it out amounts to an oversight that people will likely (and perhaps rightly) point to as a flaw in the test.
There is something to be said for keeping the number of codecs tested to a low number, and even perhaps delaying the testing of WMA Pro until some point in the future. However, I believe that will only lessen the relevance of this test perhaps in the somewhat near future.
So, again, my vote is for WMA Std and Pro either with or without an additional codec in the form of an anchor, and no ATRAC.
...
I see too much wishful thinking in this thread. People here are pushing PRO much more than Microsoft itself.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=342130")
Personally, I don't promote it at all. And I don't use it also...
I just think including WMA Pro is a way to gain a bit of "credibility" - this community will find a bit easier to promote its views if it'll be a bit more "marketable" to common folks, by including the tech in which large part of them believe (but that's still only a "bit" of course...many don't care about ABX, or even quality for that manner...)
BTW:
...
I wouldn't feature MPC. But that's what you should expect from an [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=35030&view=findpost&p=308437]MPC hater[/url] like me.
...
MPC has one tremendous advantage applicable to people who download their music (discussing this practise here only) and want very good quality, but not at the expense of lossless (and if they don't own portable player of course )
When you download something in MPC, it's almost guaranteed it'll be very good (especially since REX guidelines are very often used). With OGG...not always, you can find files encoded in ancient versions (and see this only afer download), besides greater popularity leads to less strict ripping (OTOH that's a plus when it comes to actually finding something). And MP3...like OGG, but much more...
MPC has one tremendous advantage applicable to people who download their music (discussing this practise here only) and want very good quality, but not at the expense of lossless (and if they don't own portable player of course )[a href="index.php?act=findpost&pid=342138"][{POST_SNAPBACK}][/a]
Good point. But I think it's not really a good idea to feature a format because it's good for file trading purposes
I see too much wishful thinking in this thread. People here are pushing PRO much more than Microsoft itself.
[a href="index.php?act=findpost&pid=342130"][{POST_SNAPBACK}][/a]
Some of us would prefer to see a more futureproof test, that's all, even to silence the critics on Slashdot, as a few have commented. If you exclude Pro, a lot of people might discount your test as being unfair. As a compromise, you could do a limited test of it.
why so? i personally don't touch anything WMA, and i manage to get along pretty well.
A lot of people don't buy Hondas or Fords and get along quite well too. That doesn't mean Car and Driver should exclude the brands from a roundup of family sedans. That would be an unfair test. Another example: a roundup of pickups that excluded the Toyota Tundra because it's not a traditional American brand. WMA's either 2nd or 3rd behind MP3 in popularity, and Pro is no longer in beta. If you kick either out, you run the risk of people seeing the test as being incomplete (see Dibrom's post).
As for being enthusiastic about the codec, that's because I like it, just like other users like OGG, MPC, etc. Apparently a lot of people seem to have problems with others liking MS products or anything closed source, mostly for prejudicial reasons as I posted earlier. That and it seems to be cool to hate MS. They have the right to do that if they wish. None of what I said suggested WMA would win or lose. I'd just like to see it fairly represented, instead of bashed by people who've never personally touched it, listened to it, or otherwise used it.
I had one post in another thread where I spoke about my personal experiences with different formats (I was new to HA and didn't know about the ABX rule, I apologized and recanted) but I don't bash other formats, especially ones I haven't used. People seem to enjoy doing that to WMA, however, and that's just wrong, fairly speaking. Nothing I've posted in this thread asserts the superiority or inferiority of WMA. I want to see a fair test and the results therof.
I have an idea here, lets try to be democratic!
Can the poll be tweaked so that it can let a person pick exactly 5 or 6 choices
depending on the maximum limited spaces for these formats....
I see some boards are able to do this, I'm not sure "Invision Power Board v2.0.4" can do this.
1. It has the potential to be competitive in this test.
2. It has the potential to become a widespread format.
Most of the other alternatives proposed for testing in place of WMA Pro (most notably ATRAC) fail to meet one or both of those conditions.
Given that one of the presumed goals of this test is to highlight progress in various encoding technologies, and given the fact that WMA is very widespread, it really doesn't make a lot of sense to me to leave Pro out of the picture -- especially when just about everyone here knows that WMA Std is going to most likely be inferior to Pro across the board. Leaving it out amounts to an oversight that people will likely (and perhaps rightly) point to as a flaw in the test.
i completelly agree.
A lot of people don't buy Hondas or Fords and get along quite well too. That doesn't mean Car and Driver should exclude the brands from a roundup of family sedans. That would be an unfair test. Another example: a roundup of pickups that excluded the Toyota Tundra because it's not a traditional American brand. WMA's either 2nd or 3rd behind MP3 in popularity, and Pro is no longer in beta. If you kick either out, you run the risk of people seeing the test as being incomplete (see Dibrom's post).
at the risk of being off topic, would you buy a Honda (WMA) if it had a lot of restrictions, you have to take it only to honda to reapair it, or a toyota, without restrictions, and open to anyone to repair (Vorbis)?
What about using the encoder with the oldest sourcecode as a low anchor: blade.[a href="index.php?act=findpost&pid=341807"][{POST_SNAPBACK}][/a]
That criteria is kinda wacko. You mean the oldest open source MP3 encoder? Then it would probably be 8hz-MP3. The oldest MP3 encoder is L3enc (and hey, it's based on source code!). But there are even older encoders like Indeo Audio, IMA ADPCM and Lucent's PAC.
[a href="index.php?act=findpost&pid=341814"][{POST_SNAPBACK}][/a]
Heh, you obviously know far more about this than i do.
As markanini said, i suggested blade because it used the original specification source code, but seeing as that code was meant as a demonstration, i suppose l3enc would be a good option also. The other encoders you mention aren't mp3 encoders are they? I meant mp3 of course.
Blade was spread though, i'm not sure which of these two encoders spit out more files to the web.
As for being enthusiastic about the codec, that's because I like it, just like other users like OGG, MPC, etc. Apparently a lot of people seem to have problems with others liking MS products or anything closed source, mostly for prejudicial reasons as I posted earlier. That and it seems to be cool to hate MS. They have the right to do that if they wish. None of what I said suggested WMA would win or lose. I'd just like to see it fairly represented, instead of bashed by people who've never personally touched it, listened to it, or otherwise used it.
I had one post in another thread where I spoke about my personal experiences with different formats (I was new to HA and didn't know about the ABX rule, I apologized and recanted) but I don't bash other formats, especially ones I haven't used. People seem to enjoy doing that to WMA, however, and that's just wrong, fairly speaking. Nothing I've posted in this thread asserts the superiority or inferiority of WMA. I want to see a fair test and the results therof.
[a href="index.php?act=findpost&pid=342147"][{POST_SNAPBACK}][/a]
Nownow, you are making assumptions and you surely know the old cliche' about those. I've used WMA in the not so distant past and found that horrid metallic high frequency clang noticeable at even 128 with the std codec. It is true that Pro improves on this issue (and very decently from my recollection) but I cannot very easily encode to this format with the software of
my choice, as with the more open formats in discussion here today (thanks for your clarification of my post above, kwanbis. )...and, again, the only place it will play is on my PC. My Iaudio I5 flash player and Iriver CD mp3 player (both of which support modern incarnations of the vorbis encoder) both only support the wretchedly outdated std but no pro, so it's off-limits to me because the metallic ringing is something my tin ears can't even stand. This is where I have trouble understanding blatant support for the wma format, as good as Pro may or may not be! Nero HE-AAC v.2 codec is also a fairly closed format with limited (nonexistent at this stage as far as I know) hardware support beyond Windows-based PCs but after hearing the amazing clarity of sound at even 40 kbps, I can't help but give the project respect. In reference to Dibrom's post, I also can only comment on the basis of the WMA presence/support in the past and up to this point, so my comments may very well become obsolete come 2006 or at some unknown date in the future...
I apologize if I've said too much, I don't mean to incite a format war.
Back to the thread topic and in reference to what I mentioned earlier, I would definitely vote for wma pro for inclusion in this test...in regards to std, only if we're wishing to win over a handful of the more inquisitive type from "Joe Public" would I be interested in its inclusion. ATRAC3...what kind of future does it have now that the MD has reached near obsolescence?
Nero HE-AAC v.2 codec is also a fairly closed format
[a href="index.php?act=findpost&pid=342187"][{POST_SNAPBACK}][/a]
There is nothing closed about HE-AAC v2.
You can get the full specification from the standardization organisations, and there's a complete open sourced GPL decoder. (There's another open source one for HE-AAC v1, and another open source one for HE-AAC v2)
at the risk of being off topic, would you buy a Honda (WMA) if it had a lot of restrictions, you have to take it only to honda to reapair it, or a toyota, without restrictions, and open to anyone to repair (Vorbis)?
[a href="index.php?act=findpost&pid=342152"][{POST_SNAPBACK}][/a]
Only good mechanics can fix the car - Honda sure has them, and I'm not going to do it myself.
Realistically, it's 100% true for audio codecs as well. A codec is not 'open to anyone to repair'. It takes skill and understanding, and very few people have that and the time to commit to a work on a format they're not payed for.
(Offtopic or not, well, the original statement was, IMHO, a silly argument not to test WMA (Pro))
Ivan / Garf, could you please tell me which settings you think I should use with Nero?
I have a question about who should participate in the test.
I have not done more that 20 abx test and the only thing I successfully(?) abxed was the Fraunhofer codec @ 128kbs (6/6). Every other codec I have tried in that range was 5/6 or 5/7. I would say I do not have great hearing.
Should people like me take the test?
the more participants we have, the more statistical accuracy we have, regardless of their hearing ability. if nothing else, your participation will add an element of the "average listener" to the results.
so if i may speak for sebastian mares, yes, by all means, please participate.
edit: typos
I have a question about who should participate in the test.
I have not done more that 20 abx test and the only thing I successfully(?) abxed was the Fraunhofer codec @ 128kbs (6/6). Every other codec I have tried in that range was 5/6 or 5/7. I would say I do not have great hearing.
Should people like me take the test?
[a href="index.php?act=findpost&pid=342431"][{POST_SNAPBACK}][/a]
Null results are not bad results. If only people that can hear a difference are participating, the average rating of codecs is lower, and differences between codecs are larger than those realisticly speaking are.
I would suggest you wait until January to choose the codecs. If you do not, you may obsolete your test.
[a href="index.php?act=findpost&pid=342063"][{POST_SNAPBACK}][/a]
Sorry, I can't delay the test if I don't even know what to wait for.
What I'd like to know is whether you guys would like to see 5 competitors + an anchor or 6 competitors + an anchor.
Four codecs are already set: LAME, Nero, iTunes AAC and Vorbis. We can now either have WMA and ATRAC or WMA Pro and WMA Standard as additional formats plus an anchor or decide between WMA Pro, WMA Standard and ATRAC and use one of the formats plus an anchor.
keep in mind that, as stated earlier, when using something like l3enc as an anchor, it will be very easily abxable; it really wont add to the difficulty of the test for the participants. therefore, i think the 6 competitors + an anchor is really our best bet.
again, i think wma pro and std should be the 2 additional formats; not atrac.
What I'd like to know is whether you guys would like to see 5 competitors + an anchor or 6 competitors + an anchor.
[a href="index.php?act=findpost&pid=342567"][{POST_SNAPBACK}][/a]
By experience, I wouldn't discard the anchor. By heart, I'd rather give the free place to another contender (such as WMA) and not to something uninteresting. But by reason, I now understand that anchors are really necessary to moderate some notations. They're maybe even more necessary for collective tests. Anchors (low and high) should homogenize the notation principle between different testers. Therefore, "temperementic" or "harsh" listeners won't give 1.0 to the worst contender; the low anchor is here to temper the reaction. I don't know if I'm clear. But as someone which was used to rate very strictly some weak encodings, I feel that anchors could give more homogeneity to the test. If I'm not wrong, it's one of the purpose of the anchor.
I'd use:
• low anchor (a really poor one, otherwise it's not a low anchor anymore)
• AAC Nero
• AAC iTunes
• MP3 LAME
• Vorbis
These five are enough I'd say. It's easier to people to request for WMA, WMAPro, Atrac3, Atrac3+, MPC, FhgMP3, HE-AAC... than performing a complete tests with 18 samples and 6...8 files to listen, evaluate, rank, reevaluate, ABX, re-rank, etc...
WMAPro is a very interesting (and not very well-known) format; the only problem is that Microsoft apparently don't plan to promote this new format and to replace the old WMA Std one.
WMA is well-supported, but quality is much less interesting. I don't think that people interested by HQ implementations of AAC and by Vorbis are also interested by WMA at this bitrate. I don't mean that testing WMA is really uninteresting, but if we consider the current choice of encoders, WMA looks a bit like an intruder in this arena. With four real and strong contender, I believe that several users will aready have severe troubles to perform the test; that's why it's maybe not necessary to increase the required effort.
keep in mind that, as stated earlier, when using something like l3enc as an anchor, it will be very easily abxable; it really wont add to the difficulty of the test for the participants. therefore, i think the 6 competitors + an anchor is really our best bet.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=342571")
I hope you realize that testing 6 real contenders is already something difficult.
And with more contenders, the "statistical noise" ([a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=23355&view=findpost&p=236453]ff123[/url]) will be higher. It means that with more contenders, the risk of ending the test with a conclusion like "everything is tied" is simply higher (correct me if I'm wrong).
Regarding the low anchor, I am offering to provide a compile of Shine.
This mp3 encoder would be wonderfull as a low anchor.
• low anchor (a really poor one, otherwise it's not a low anchor anymore)
• AAC Nero
• AAC iTunes
• MP3 LAME
• Vorbis
WMAPro is a very interesting (and not very well-known) format; the only problem is that Microsoft apparently don't plan to promote this new format and to replace the old WMA Std one.
i undesrtand what you say about low anchor, and i think it makes sence. what i do think is that WMA Pro is important enought to be included, as what said, MS plans to include it under WMP 11. WMA Std is also important but has already been tested. I would trade Nero for WMA Pro. Nero appears to be always trailing iTunes, so, why test it again? You did a 200 samples test yourself, this would be much less, but 1 or 2 more codecs, do you think it would be more dificult than what you tested?
Regarding the low anchor, I am offering to provide a compile of Shine.
This mp3 encoder would be wonderfull as a low anchor.
[a href="index.php?act=findpost&pid=342576"][{POST_SNAPBACK}][/a]
Will it be CBR 128 dual channel? That would make a pretty good low anchor.
BTW, I think l3enc may be too good as a low anchor if used as it is intended (128 Joint Stereo). I have some older encodings with it, and while they are by no means great, depending on the difficulty of the samples, I find it usually not too offensive WRT artifacting - waaay better than e.g. Blade.
as what said, MS plans to include it under WMP 11. WMA Std is also important but has already been tested.[a href="index.php?act=findpost&pid=342577"][{POST_SNAPBACK}][/a]
What does it mean? WMAPro is supported on playback (and also encoding) since WMP9 if I remember correctly. What really matters IMO is not the software support of WMAPro (maybe as large as WMA std one) but the hardware one.
I would trade Nero for WMA Pro. Nero appears to be always trailing iTunes, so, why test it again?
Ask Garf
You did a 200 samples test yourself, this would be much less, but 1 or 2 more codecs, do you think it would be more dificult than what you tested?
Keep in mind that HA.org count 25.000 members but only one silly enough to perform complete listening tests and to publish them What I can endure is apparently higher than most people on this board would do (by lack of time, motivation, envy, curiosity, practise...). Ask Roberto some precision about the number of download of the each samples and the number of results finally sent two weeks later. Many were discouradged.
i would like to participate in the test. it's ok if my results are regardless, i just want to be able to download the sample files. so - will the test be open for the public?
armin
Sebastian,
follow the good advice given to you earlier.
The most crucial question to ask yourself is:
What is my research question?
What question do I want the test to answer?
Some possible examples:
- Ranking of general audio accuracy (comparison against original) of most common available audio codecs (at 128bkps)?
- Ranking of current available audio codecs in terms of audio accuracy for enthusiasts to use (i.e. HA community and the ilk) at 128kbps (regardless of codec obscurity)?
- Quality ranking of current audio codecs with most likely to have a future support and wide future use?
- Quality ranking of widely hardware (DAP, not just PDA) supported audio codecs by their audio accuracy at 128kbps.
- More ranking information about general audio codec performance ranking that adds new information to the previous test results from Rjamorim's test.
Everything else will come out of that:
Codecs, anchors, number of samples, number of testers, who are best test listeners, statistical handling, etc.
If you just want to do a generic 128kbps test, then it really doesn't matter how the codecs are chosen. Just pick them like an enlightened dictator would. You have enough knowledge yourself to do this. Letting the mob vote will just give you mob results
My own research question would probably be something like the following:
Relative audio quality ranking of new & improved common audio codecs at 128kbps (disregarding those that have already been tested recently):
Personally I'd pick:
- Lame 3.97 (the basic standard, one of the most widely deployed, still actively developed, improving)
- iTunes AAC (the standard for buying music off the net, widely deployed, widely used, growing in popularity, apparently also improving)
- Nero Digital AAC (widely deployed software, one of big contenders to QT/iTunes, AAC is a big part of the future, in many ways)
- Ogg Vorbis AoTuv (only really open/free alternative, gaining ground, gaingin in quality, gaining slowly more hw support)
- WMA Pro (supported or not, widely deployed or not, people CAN use it and WILL use it, if it proves to be superior to WMA std, most importantly it has not been tested thoroughly and people will just say "what if..." if you don't test it)
I would not pick myself (for reasons given):
- Atrac (biggest in Japan and most of them will not read/participate in this test anyway, elsewhere losing popularity / dead-end format, die-hards won't care whatever the result is, earlier version already tested, unlikely to have made big advances )
- WMA std (tested, .1 improvement unlikely to be stellar advances)
But I digress, it's not my test. I'm not the one to decide.
You decide, but do think of the question: what do you want the test to find out?
regards,
halcyon
PS Those thinking that a number of listeners will automatically improve the accuracy of the results... it's not just about the number of listeners (or tracks for that matter). It's whether the sampling matches the research question or not. You have to decide your test question, before you can decide what the optimal sampling is for your test.
Will it be CBR 128 dual channel? That would make a pretty good low anchor.
BTW, I think l3enc may be too good as a low anchor if used as it is intended (128 Joint Stereo). I have some older encodings with it, and while they are by no means great, depending on the difficulty of the samples, I find it usually not too offensive WRT artifacting - waaay better than e.g. Blade.
Yes, Shine is CBR dual channel, with some added bonus: no bit reservoir, no short blocks.
Some would say that its tonal purity is way higher than BladeEnc.
i would like to participate in the test. it's ok if my results are regardless, i just want to be able to download the sample files. so - will the test be open for the public?
armin
[a href="index.php?act=findpost&pid=342584"][{POST_SNAPBACK}][/a]
The test will be public, yes.
Could the test be made available in user~helpful download packs, including the normalised lossless samples - a smaller subset for hurried or slow testers, as well as the full set of samples for those who can commit time to completing the whole?
Or choose which genre of samples to do at a time?
Ive wondered about adapting abchr java comparator to work as the front end to a user-data collecting server script. That could make it easy as pie for fickle users to participate in tests. -Include in the front end a little bit of gameplay - like a score league table, the more disorganised of us could still end up devoting hours to trying to improve our abx ranking scores'
Ok that system would require someone to donate the coding and server time - just having a beginner's download pack would help in the same direction.
All this sounds very interesting.
Since I wasn't around when the last test was conducted - how exactly do you go about distributing a test like this? Are we given a package to download, which includes software and a load of samples and instructions?
I read your suggestions, ChiGung, about automating data collection - so how was this done the last time around?
OMG, so much to reply to...
Regarding the low anchor, I am offering to provide a compile of Shine.
This mp3 encoder would be wonderfull as a low anchor.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=342576")
I completely agree
Please provide us with a compile anyway. So that we can already have an idea of its superior quality. If you would like me to host it at RareWares, send it to my e-mail.
MS plans to include it under WMP 11.[a href="index.php?act=findpost&pid=342577"][{POST_SNAPBACK}][/a]
I think that's unclear, at the moment. Sure, it's an optional format in WMP 11 beta's ripping function. But then again, WMP9 beta featured ripping to WMA Pro and WMA Lossless, and the final version ended up sacking these options.
I'm still confident Microsoft doesn't want end users playing with WMA Pro. Why?
- They aren't pushing for portable player support. Name me one single DAP supporting WMA Pro. Also, WMA Pro isn't a mandatory format in Microsoft's latest DRM initiative - PlaysForSure.
- It's not a ripping format option in WMP. This might change in WMP11, but might not as well.
- Digital music stores selling WMA aren't offering Pro even as an option.
- Microsoft is probably afraid of the endless compatibility complaints from users that own daps that can only play WMA Std.
- They are still confident that WMA Std. is twice better than MP3.
I have some older encodings with it, and while they are by no means great, depending on the difficulty of the samples, I find it usually not too offensive WRT artifacting - waaay better than e.g. Blade.[a href="index.php?act=findpost&pid=342578"][{POST_SNAPBACK}][/a]
But what version of l3enc was used to encode these samples? I suspect most samples available online were encoded with 2.x series l3enc - mostly 2.72. I'm proposing 1.0 here.
What does it mean? WMAPro is supported on playback (and also encoding) since WMP9 if I remember correctly.
No, WMP never supported WMA Pro ripping, except on some beta WMP9 builds.
Ask Roberto some precision about the number of download of the each samples and the number of results finally sent two weeks later. Many were discouradged.[a href="index.php?act=findpost&pid=342579"][{POST_SNAPBACK}][/a]
On my second Multiformat test, the complete samples package (170Mb) was downloaded more than 500 times through BitTorrent alone. I'm not counting here HTTP downloads and independent sample packages downloads.
All in all, I got results from avg. 30-35 different people. Most of these didn't send in a complete results set.
Could the test be made available in user~helpful download packs, including the normalised lossless samples - a smaller subset for hurried or slow testers, as well as the full set of samples for those who can commit time to completing the whole?
Or choose which genre of samples to do at a time?
It depends on Sebastian's decision, but I don't think he would have any reason to do differently than that. I'm offering the server resources to him so that he can host all sample packages. I'll also set up a torrent tracker.
Ok that system would require someone to donate the coding and server time - just having a beginner's download pack would help in the same direction.[a href="index.php?act=findpost&pid=342612"][{POST_SNAPBACK}][/a]
I can donate the server resources.
Since I wasn't around when the last test was conducted - how exactly do you go about distributing a test like this? Are we given a package to download, which includes software and a load of samples and instructions?[a href="index.php?act=findpost&pid=342652"][{POST_SNAPBACK}][/a]
You are given a package with all executable files: Java ABC/HR, command line decoders, and batch files to process the samples.
Then, you download each sample package at a time or the whole sample collection at once.
Download this, and read the readme:
[a href="http://pessoal.onda.com.br/rjamorim/abc-hr_bin.zip]http://pessoal.onda.com.br/rjamorim/abc-hr_bin.zip[/url]
You'll get an idea on how a listening test works.
Regards;
Roberto.
At least WMA is alot better than MP3, anyone agrees?? and since MP3 and WMA are the generally accepted formats on the Digital players... I guess that we can't run much from those two formats.
At least WMA is alot better than MP3...
[a href="index.php?act=findpost&pid=342663"][{POST_SNAPBACK}][/a]
Uhh?
At least WMA is alot better than MP3, anyone agrees??
...mpffff...pffff....HA HA HA !!!
Good joke.
At least WMA is alot better than MP3, anyone agrees?? and since MP3 and WMA are the generally accepted formats on the Digital players... I guess that we can't run much from those two formats.
[a href="index.php?act=findpost&pid=342663"][{POST_SNAPBACK}][/a]
Let's say they both have their strengths and their weaknesses...
At least WMA is alot better than MP3, anyone agrees??
oh yes, sure ... WMA is the best codec ever ...
At least WMA is alot better than MP3, anyone agrees?? and since MP3 and WMA are the generally accepted formats on the Digital players... I guess that we can't run much from those two formats.
[a href="index.php?act=findpost&pid=342663"][{POST_SNAPBACK}][/a]
Whether WMA is better than MP3 is debatable: it depends on what you mean by "better."
If you're talking about WMA Std. vs the best MP3 encoder, quality wise, then probably no. If you're talking about WMA Pro, then the situation might be quite different (again, this is only speaking quality-wise -- i.e., not considering compatibility, userbase, etc.).
One thing that is quite clear though, and that I think speaks in favor of WMA and negatively of MP3 is that WMA is quite likely to see continued development in the future. MP3 will not.
Sure, the LAME devs have made a good encoder quality wise and they have been able to improve it's quality over time, but at the end of the day there is a very real limitation posed by the current MP3 specification on what is possible for the format. Some problems, such as the well known scalefactor band issue, are not solvable in a way that would still be standard MP3.
Microsoft, however, is able to rev. their specification if they desire, and IMO, it probably wouldn't be nearly as hard for them to get their partners to support a subsequent revision as might be the case for other formats (witness the stillbirth of MP3Pro or the lack of support for some of the latest features for AAC in some popular players). This is because Microsoft already controls everything (which has it's own significant downsides as well) such as the libraries for decoding, encoding, etc.
Some people will probably want to crucify me for saying this, but I'd personally rather see WMA in this test than MP3, if it came down to having to be
that picky simply because of the fear of too many formats. MP3 performance is pretty well understood by now -- it has been discussed for years on the board here, and it has been featured in almost all of the major listening tests. Yet some potential competitors in this test (WMA Pro) still haven't seen much exposure.
I don't say any of this because I'm a fan of WMA either -- I don't use the format and probably won't use the format anytime soon. Even if I wanted to, it's not a practical solution for me on the Mac. However, it's too widespread of a format to ignore I think, in the case of WMA Std, and has too much potential to become widespread, in the case of WMA Pro.
I'd like to see a little bit more variety and a bit of change in this test. I'd also like to see this crowd potentially regain a bit of credibility that I think was lost in the last few rather ugly WMA oriented threads. For a supposedly format agnostic group, I think maybe we're not living up to that principle in some regards.
Haha! Gabriel sent me Shine. Thank-you very much, dude.
Quoting the author:
Here is the awaited Shine encoder.
It should feature wonderfull tonal purity due to a top secret psymodel,
so secret that you can not spot it, even in the source code. (it's
probably the only mp3 encoder to not use scalefactors)
I think that I can hardly do worst.
http://www.rarewares.org/files/mp3/Shine.zip (http://www.rarewares.org/files/mp3/Shine.zip)
At least WMA is alot better than MP3, anyone agrees??
...mpffff...pffff....HA HA HA !!!
Good joke.
[a href="index.php?act=findpost&pid=342665"][{POST_SNAPBACK}][/a]
At least WMA is alot better than MP3, anyone agrees??
oh yes, sure ... WMA is the best codec ever ...
[a href="index.php?act=findpost&pid=342690"][{POST_SNAPBACK}][/a]
Ah yes, another bashfest
After the last few WMA threads, do we really need this?
Haha! Gabriel sent me Shine. Thank-you very much, dude.
Quoting the author:
Here is the awaited Shine encoder.
It should feature wonderfull tonal purity due to a top secret psymodel,
so secret that you can not spot it, even in the source code. (it's
probably the only mp3 encoder to not use scalefactors)
I think that I can hardly do worst.
http://www.rarewares.org/files/mp3/Shine.zip (http://www.rarewares.org/files/mp3/Shine.zip)
[a href="index.php?act=findpost&pid=342696"][{POST_SNAPBACK}][/a]
Sounds kind of like HE-AAC. Even less harsh in highs
Oops did I just break TOS 8?
Haha! Gabriel sent me Shine. Thank-you very much, dude.
Quoting the author:
Here is the awaited Shine encoder.
It should feature wonderfull tonal purity due to a top secret psymodel,
so secret that you can not spot it, even in the source code. (it's
probably the only mp3 encoder to not use scalefactors)
I think that I can hardly do worst.
http://www.rarewares.org/files/mp3/Shine.zip (http://www.rarewares.org/files/mp3/Shine.zip)
[a href="index.php?act=findpost&pid=342696"][{POST_SNAPBACK}][/a]
Wow!, Shine will be a great low anchor IMHO!
Some people will probably want to crucify me for saying this, but I'd personally rather see WMA in this test than MP3, if it came down to having to be that picky simply because of the fear of too many formats. MP3 performance is pretty well understood by now -- it has been discussed for years on the board here, and it has been featured in almost all of the major listening tests. Yet some potential competitors in this test (WMA Pro) still haven't seen much exposure.
[a href="index.php?act=findpost&pid=342694"][{POST_SNAPBACK}][/a]
One could also argue that this is a reason for MP3 to be included: As people know the behaviour of the codec, it would be a kind of reference to compare other codecs with, and as Lame still has real improvement, this isn't a void point. Personally, I would go with this point of view.
Microsoft, however, is able to rev. their specification if they desire, and IMO, it probably wouldn't be nearly as hard for them to get their partners to support a subsequent revision as might be the case for other formats (witness the stillbirth of MP3Pro or the lack of support for some of the latest features for AAC in some popular players). This is because Microsoft already controls everything (which has it's own significant downsides as well) such as the libraries for decoding, encoding, etc.
I don't think it would be so easy to rev. their specification breaking backwards compatibility with WMA Standard. Otherwise, they would be forcing WMA Pro down everyone's throats.
The big concern, IMO, is backwards compatibility on hardware players. Software players can be quickly updated with a Windows Update. But several hardware players simply can't have their firmware updated. Others can have it updated, but are long out of market and the developers probably aren't interested in releasing update firmware with revised WMA decoder. And even current players will depend on users going out of their ways to flash a new firmware. Most users can't be arsed to care to do that (but can be arsed to scream loudly if his WMAs stop playing)
and has too much potential to become widespread, in the case of WMA Pro.
I agree it has lots of potential, but Microsoft chose to solemnly ignore this potential and keep pushing WMA Standard for the consumer market.
Ah yes, another bashfest
After the last few WMA threads, do we really need this?[a href="index.php?act=findpost&pid=342697"][{POST_SNAPBACK}][/a]
We need, since - as I understand it - the posters are bashing a bold, unsubstantiated claim from a newbie user that runs counter previous knowledge. They (well, Gabriel at least) aren't necessarily bashing WMA itself.
Some people will probably want to crucify me for saying this, but I'd personally rather see WMA in this test than MP3, if it came down to having to be that picky simply because of the fear of too many formats. MP3 performance is pretty well understood by now -- it has been discussed for years on the board here, and it has been featured in almost all of the major listening tests. Yet some potential competitors in this test (WMA Pro) still haven't seen much exposure.
[a href="index.php?act=findpost&pid=342694"][{POST_SNAPBACK}][/a]
One could also argue that this is a reason for MP3 to be included: As people know the behaviour of the codec, it would be a kind of reference to compare other codecs with, and as Lame still has real improvement, this isn't a void point. Personally, I would go with this point of view.
[a href="index.php?act=findpost&pid=342706"][{POST_SNAPBACK}][/a]
The reference for comparison should be served by the possible anchor, if one is to be employed.
Beyond that, I think the point you raised does not by itself outweigh the other issues I touched upon in my post.
There's already some redundancy in the test (2 AAC encoders), and 1 encoder (LAME) that has seen some improvement since the last version, but is still an MP3 encoder and I suspect will not be so fundamentally better than its last showing that it would be worth excluding other possibilities that are, at least to a significant portion of people outside of HA, quite relevant.
In addition to the issues I already raised in my previous point, the collection of codecs right now seems to me not to be the most efficient spread for providing the most significant results for the largest group of people.
Again, whether or not that is of real importance depends primarily on what the purpose of this test is supposed to be...
I agree tests should be made appealing to the broadest audience. But you gotta remember the vast majority of listeners come from within this place, therefore, it's only proper to give some priority to what they want in tests, and only later consider what outsider people wants to see.
Microsoft, however, is able to rev. their specification if they desire, and IMO, it probably wouldn't be nearly as hard for them to get their partners to support a subsequent revision as might be the case for other formats (witness the stillbirth of MP3Pro or the lack of support for some of the latest features for AAC in some popular players). This is because Microsoft already controls everything (which has it's own significant downsides as well) such as the libraries for decoding, encoding, etc.
I don't think it would be so easy to rev. their specification breaking backwards compatibility with WMA Standard. Otherwise, they would be forcing WMA Pro down everyone's throats.
I didn't say it would be
easy, just that it wouldn't be nearly as hard as getting people on board with an "enhanced" MP3. The latter is practically impossible, the former is merely difficult to very hard.
The big concern, IMO, is backwards compatibility on hardware players. Software players can be quickly updated with a Windows Update. But several hardware players simply can't have their firmware updated. Others can have it updated, but are long out of market and the developers probably aren't interested in releasing update firmware with revised WMA decoder. And even current players will depend on users going out of their ways to flash a new firmware. Most users can't be arsed to care to do that (but can be arsed to scream loudly if his WMAs stop playing)
Sure, but players become old and are eventually replaced. Eventually it could happen that a new format revision could take over, simply because it's feasible on one end (the software adoption side of thing). Again, with MP3, this won't happen. The people that
could possibly make it happen have moved on (rightly so) to AAC.
and has too much potential to become widespread, in the case of WMA Pro.
I agree it has lots of potential, but Microsoft chose to solemnly ignore this potential and keep pushing WMA Standard for the consumer market.
For now, yes. But this doesn't mean indefinitely. I wouldn't be so bold as to say "never"
Ah yes, another bashfest
After the last few WMA threads, do we really need this?[a href="index.php?act=findpost&pid=342697"][{POST_SNAPBACK}][/a]
We need, since - as I understand it - the posters are bashing a bold, unsubstantiated claim from a newbie user that runs counter previous knowledge. They (well, Gabriel at least) aren't necessarily bashing WMA itself.
[a href="index.php?act=findpost&pid=342710"][{POST_SNAPBACK}][/a]
I made this statement to discourage other people from jumping on the "me too!" bandwagon and derailing this thread.
I agree it has lots of potential, but Microsoft chose to solemnly ignore this potential and keep pushing WMA Standard for the consumer market.
For now, yes. But this doesn't mean indefinitely. I wouldn't be so bold as to say "never" [a href="index.php?act=findpost&pid=342716"][{POST_SNAPBACK}][/a]
True, but I still don't see it happening in the foreseeable future. IMO, the best hint that WMA Pro isn't approaching the consumer market any time soon is that Microsoft left it out of their PlaysForSure campaign. Here they had this great opportunity to force WMA Pro down everyone's throats - since PlaysForSure is only being implemented on new hardware, therefore there should be no issue adding a new codec while at it - and they decided to stick to WMA Standard neverthless.
May they change their mind in the future and start seriously pushing WMA Pro? Of course. But by then, Sebastian's test results will have lost their usefulness anyway (I don't think Microsoft could pull this stunt in less than an year starting from their current situation regarding WMA Pro), so I don't think it's even worth worrying too much about it.
Oh my, where is this going...
Please, please, don't use Shine as a low anchor. It's an encoder nobody uses, it's absolutely irrelevant today and we already know that it sucks. Why test it again? We have the great opportunity to test one of the modern codecs that claim superior quality at half the bitrate, so it would be a huge mistake to include Shine instead. I personally think that HE-AAC is perfectly suited for the low anchor position. But again, please, forget about Shine, it's a dead horse.
Now whether WMA should be included, I personally think that yes, it should be included. But the test will be very tough, lots of encoders and the quality will be high so it won't be easy. And while I first thought that it would be interesting to include WMA Pro, I changed my mind. It's by far the least popular format, and it's future is uncertain, so I think we can omit it in this test.
Irregardless of WMA Pro's future (as said before, it's all speculation at this stage, innit?), it has yet to be incorporated into any proper double blind test that I am aware, so I'm all for it. The burning curiosity as to how well it stacks up against the latest and greatest codecs from the other players is my main reasoning for wanting inclusion.
So Dibrom, do you feel mp3 development has reached a ceiling? Will version 4.0 break prior compatibility or just be another evolutionary step for mp3 development (with ever decreasing gains since the ceiling is so close)?
Oh my, where is this going...
Please, please, don't use Shine as a low anchor. It's an encoder nobody uses, it's absolutely irrelevant today and we already know that it sucks. Why test it again? We have the great opportunity to test one of the modern codecs that claim superior quality at half the bitrate, so it would be a huge mistake to include Shine instead. I personally think that HE-AAC is perfectly suited for the low anchor position. But again, please, forget about Shine, it's a dead horse.
Now whether WMA should be included, I personally think that yes, it should be included. But the test will be very tough, lots of encoders and the quality will be high so it won't be easy. And while I first thought that it would be interesting to include WMA Pro, I changed my mind. It's by far the least popular format, and it's future is uncertain, so I think we can omit it in this test.
[a href="index.php?act=findpost&pid=342741"][{POST_SNAPBACK}][/a]
Souldn't the low anchor be something that is just obviously easy to ABX, but still be in the same league as the other competitors?, does it matter how old/new, simple/advanced it is?, I mean, Shine produces mp3's, a format everybody knows, and the results it produces fill the requirements of the low anchor position, I've tried it, and for a 128kbps listening test the quality of Shine is nicely suited for this, I think. As for using HE-AAC, isn't this supposed for something other than trasparency?, the formats in this test are more inclined towards transparency than acceptable qaulity at super-low bitrates, I mean, HE-AAC and AAC/mp3/Vorbis/WMA are very different players with different purposes in mind.
Just my 2 Pesos.
EDIT: coherence.
we already know that it sucks
[a href="index.php?act=findpost&pid=342741"][{POST_SNAPBACK}][/a]
And this is exactly why I am using it as low anchor.
Irregardless of WMA Pro's future (as said before, it's all speculation at this stage, innit?), it has yet to be incorporated into any proper double blind test that I am aware, so I'm all for it. The burning curiosity as to how well it stacks up against the latest and greatest codecs from the other players is my main reasoning for wanting inclusion.
[a href="index.php?act=findpost&pid=342748"][{POST_SNAPBACK}][/a]
But you obviously don't realize that including another codec again increases the difficulty of the test. Is it really wotrh it because of a codec that is IRRELEVANT today? I don't think so.
So Dibrom, do you feel mp3 development has reached a ceiling? Will version 4.0 break prior compatibility or just be another evolutionary step for mp3 development (with ever decreasing gains since the ceiling is so close)?
[a href="index.php?act=findpost&pid=342748"][{POST_SNAPBACK}][/a]
Please, breaking the compatibility of mp3? Where did you get that from?
Souldn't the low anchor be something that is just obviously easy to ABX, but still be in the same league as the other competitors?, does it matter how old/new, simple/advanced it is?, I mean, Shine produces mp3's, a format everybody knows, and the results it produces fill the requirements of the low anchor position, I've tried it, and for a 128kbps listening test the quality of Shine is nicely suited for this, I think. As for using HE-AAC, isn't this supposed for something other than trasparency?, the formats in this test are more inclined towards transparency than acceptable qaulity at super-low bitrates, I mean, HE-AAC and AAC/mp3/Vorbis/WMA are very different players with different purposes in mind.
[a href="index.php?act=findpost&pid=342754"][{POST_SNAPBACK}][/a]
Again, this test will be very tough. Do you really want to spend all the effort to get results for a codec that never had any real world value? The results you will get are worthless. It's not even a codec that was ever used by a user base in the past, like for example Xing. Then you could at least use the results to see how the mp3 format has developed since.
I can't even properly express all my concerns right now because I'm so worked up about this issue. I'm afraid that this test is going in a wrong direction.
we already know that it sucks
[a href="index.php?act=findpost&pid=342741"][{POST_SNAPBACK}][/a]
And this is exactly why I am using it as low anchor.
[a href="index.php?act=findpost&pid=342759"][{POST_SNAPBACK}][/a]
Great. But as rjamorim said, there are tons of shitty encoders. What is it that makes Shine worth including in this test?
Great. But as rjamorim said, there are tons of shitty encoders. What is it that makes Shine worth including in this test?
It would not be included in the test, but used as a low anchor.
Listen to its results, you will immediately understand the difference.
Great. But as rjamorim said, there are tons of shitty encoders. What is it that makes Shine worth including in this test?
It would not be included in the test, but used as a low anchor.
Listen to its results, you will immediately understand the difference.
[a href="index.php?act=findpost&pid=342771"][{POST_SNAPBACK}][/a]
Ermm... I don't even know what to say... I expected that somebody will respond with some BS, but not you...
How is a low anchor "not included in a test"? You obviously INCLUDE it in a test. It's just that it is predestined to sound the worst. But it obviously HAS to be included.
And yes, I know that it sounds bad. But again, there are tons of encoders that produce shitty encodes. The point is to include one that is of some significance to the real world usage.
So Dibrom, do you feel mp3 development has reached a ceiling?
This is getting kind of off topic, so if you want to discuss it further you should really probably start a new thread.
My answer is yes, but my answer was yes a long time ago. MP3 is an obsolete format and everyone working on codecs these days (including the LAME developers) know this. LAME provides impressive quality given the MP3 limitations, but there is a real limit to what can be achieved with the format. At this point, LAME is really pretty much limited to fine tuning its psymodel. We are possibly going to see improvement on difficult samples and at times an improvement in quality stability. We might see very mild improvements in quality at lower bitrates than was possible with previous versions. But we are not going to see much more than that.
Will version 4.0 break prior compatibility or just be another evolutionary step for mp3 development (with ever decreasing gains since the ceiling is so close)?
[a href="index.php?act=findpost&pid=342748"][{POST_SNAPBACK}][/a]
Umm.. I don't see why the LAME devs would break compatibility with the MP3 spec with LAME 4.0 because nothing would play the file, and they'd have a hard time getting everyone to use their own decoder. I've never heard any plans of this. As far as I understand it, 4.0 is an encoder rewrite, focusing primarily on efficiency, and possibly quality also. The current LAME sources are very messy, so I suppose that's part of the motivation for the rewrite, but you'll have to ask a LAME developer if you want a real answer.
I'm afraid that this test is going in a wrong direction.
[a href="index.php?act=findpost&pid=342765"][{POST_SNAPBACK}][/a]
I'd have to say I feel the same way. Oh well...
How is a low anchor "not included in a test"? You obviously INCLUDE it in a test. It's just that it is predestined to sound the worst. But it obviously HAS to be included.
That's nit-picking. What Gabriel meant was that it is not the result of the low-anchor that people are interested in. It is merely a reference point to not get too much difference in people's ratings. Since the a low-anchor must be of obvious worse quality than the other contenders, there is no point in drawing any conclusion from the low-anchor score itself. We already knew it would do worse before the test started.
How is a low anchor "not included in a test"? You obviously INCLUDE it in a test. It's just that it is predestined to sound the worst. But it obviously HAS to be included.
And yes, I know that it sounds bad. But again, there are tons of encoders that produce shitty encodes. The point is to include one that is of some significance to the real world usage.
What I mean by "not included" is that it should be obvious that it is lower, thus not requiring substained care from the test participant. If the low anchor is a good low anchor, you usually can not precisely deduce anything about it (except that it is bad) regarding the final ranking/notation as it can be an order of manitude lower than the contenders.
I think that a low anchor should be significantly lower, and I am afraid that an encoder with significance to real use might be too high.
A good alternative would be a pcm 8bits with noticable lowpass.
edit: stephanV already posted a similar answer in the meantime
Please, breaking the compatibility of mp3? Where did you get that from?
Gambit, it was a simple question for
Dibrom. No need to flame, as I am not a developer and am unaware of the details surrounding Lame 4.
Thanks for you answers, Dibrom...I expected as much. I won't continue any further off topic here.
But you obviously don't realize that including another codec again increases the difficulty of the test. Is it really wotrh it because of a codec that is IRRELEVANT today? I don't think so.
I do realize this but after years of skipping over this format, I think the time has come.
But you obviously don't realize that including another codec again increases the difficulty of the test. Is it really wotrh it because of a codec that is IRRELEVANT today? I don't think so.
The anchor is not meant to be relevant. It is meant to sound guaranteedly bad.
According to Nero developers, their HE AAC down to 32kbps is good enough to challenge MP3 at 128. So, no, your proposal for anchor is not good enough (or, rather, "bad enough")
And yes, I know that it sounds bad. But again, there are tons of encoders that produce shitty encodes. The point is to include one that is of some significance to the real world usage.
[a href="index.php?act=findpost&pid=342773"][{POST_SNAPBACK}][/a]
Like what? Instead of just criticizing everything you read, maybe you could give some examples. The only other bad MP3 encoders I can think of are Xing, Blade and the iTunes encoder that lost in Roberto's last MP3 listening test. However, Xing is too good IMHO and I am not sure if the encoder in iTunes didn't change.
Were you thinking about another format? What should it be? WMA is too good for a low anchor. Maybe an old AAC encoder that was used in the past?
I'm afraid that this test is going in a wrong direction.
[a href="index.php?act=findpost&pid=342765"][{POST_SNAPBACK}][/a]
I'd have to say I feel the same way. Oh well...
[a href="index.php?act=findpost&pid=342777"][{POST_SNAPBACK}][/a]
And why is that?
You said
the collection of codecs right now seems to me not to be the most efficient spread for providing the most significant results for the largest group of people.
[a href="index.php?act=findpost&pid=342712"][{POST_SNAPBACK}][/a]
So, what would your collection of codecs look like? The only encoders that are set for this listening test are LAME (MP3), iTunes and Nero AAC and Vorbis. These are some of the most wide-spread formats at this bitrate. What is left is WMA which is also pretty popular, but we didn't decide anything. I think the discussion shouldn't be whether or not WMA should be included, but which WMA version to feature in the test.
I think Roberto is right. If Microsoft really wanted the people to use WMA Pro, they would've added it to PlaysForSure. On the other hand, if we are testing the creme de la creme of the encoders (we're testing LAME and not FhG which is included in MMJB and other rippers for the mass; and we're testing AoTuV and not the Xiph builds), maybe we should test WMA Pro and not Standard.
And yes, I know that it sounds bad. But again, there are tons of encoders that produce shitty encodes. The point is to include one that is of some significance to the real world usage.
Now why does the anchor have to have any significance to the real world?? Isn't the anchor just a reference in order to know how to rate the others??
According to Nero developers, their HE AAC down to 32kbps is good enough to challenge MP3 at 128. So, no, your proposal for anchor is not good enough (or, rather, "bad enough")
[a href="index.php?act=findpost&pid=342790"][{POST_SNAPBACK}][/a]
Depends on who takes the test (I stated *very clearly* what I claimed, so don't try to twist my words.). You cannot check the audience on such a test, so any reference to my claim is effectively bogus.
Based on my experience, the result you actually get back will be from the experienced ones more than from people that couldn't tell a difference.
I'm afraid that this test is going in a wrong direction.
[a href="index.php?act=findpost&pid=342765"][{POST_SNAPBACK}][/a]
I'd have to say I feel the same way. Oh well...
This is getting a bit extreme. This discussion is held to have some general consensus about how it should be performed. I certainly can't say that leaving out MP3/LAME would make the test more interesting or relevant (the contrary is true). Also adding a low anchor that isn't really one, doesn't make much sense either.
I think Roberto is right. If Microsoft really wanted the people to use WMA Pro, they would've added it to PlaysForSure. On the other hand, if we are testing the creme de la creme of the encoders (we're testing LAME and not FhG which is included in MMJB and other rippers for the mass; and we're testing AoTuV and not the Xiph builds), maybe we should test WMA Pro and not Standard.
There is a difference though. LAME and FhG both produce MP3, Xiph and AoTuV both produce Vorbis, but WMA Std is not WMA Pro. If you want to test the most popular formats/codecs (which you said you wanted to do) test WMA Std, not Pro.
How about this:
1) Instead of Nero @ 128kbps, use Nero HE-AACv2 at a lower bitrate
2) Include CT HE-AACv2 at the same bitrate
This gets you
1) Anchor(s)
2) Some result on new Nero vs CT
3) Some result on low bitrate HE-AAC vs near-transparent codecs
4) More likelihood of an 'interesting' result (including 2 AAC codecs will most likely give a statistical tie. IMHO the 'relative quality' of HE-AAC would be more interesting to find)
is FAAC completely out of question? - this could be a 'medium' anchor (i mean even if the wma-pro wins, do we really care?)
is FAAC completely out of question? - this could be a 'medium' anchor (i mean even if the wma-pro wins, do we really care?)
[a href="index.php?act=findpost&pid=342869"][{POST_SNAPBACK}][/a]
I don't think there is a point to including a codec that hasn't changed since the last time it was tested.
How about this:
1) Instead of Nero @ 128kbps, use Nero HE-AACv2 at a lower bitrate
2) Include CT HE-AACv2 at the same bitrate
This gets you
1) Anchor(s)
2) Some result on new Nero vs CT
3) Some result on low bitrate HE-AAC vs near-transparent codecs
4) More likelihood of an 'interesting' result (including 2 AAC codecs will most likely give a statistical tie. IMHO the 'relative quality' of HE-AAC would be more interesting to find)
[a href="index.php?act=findpost&pid=342868"][{POST_SNAPBACK}][/a]
It's an 128 kbps listening test. Also, if I include CT, it would be the third AAC encoder - that isn't really multi-format any longer.
is FAAC completely out of question? - this could be a 'medium' anchor (i mean even if the wma-pro wins, do we really care?)
[a href="index.php?act=findpost&pid=342869"][{POST_SNAPBACK}][/a]
As stated above, three AAC encoders are too much.
If you want to compare various AAC encoders, wait for the next AAC listening test.
It's an 128 kbps listening test. Also, if I include CT, it would be the third AAC encoder - that isn't really multi-format any longer.
[..]
As stated above, three AAC encoders are too much.
If you want to compare various AAC encoders, wait for the next AAC listening test.
[a href="index.php?act=findpost&pid=342875"][{POST_SNAPBACK}][/a]
If you are going to do an AAC listening test too, then perhaps it makes more sense to exclude Nero from this test.
I can live with that as long as we are eventually tested
It's an 128 kbps listening test. Also, if I include CT, it would be the third AAC encoder - that isn't really multi-format any longer.
[..]
As stated above, three AAC encoders are too much.
If you want to compare various AAC encoders, wait for the next AAC listening test.
[a href="index.php?act=findpost&pid=342875"][{POST_SNAPBACK}][/a]
If you are going to do an AAC listening test too, then perhaps it makes more sense to exclude Nero from this test.
I can live with that as long as we are eventually tested
[a href="index.php?act=findpost&pid=342876"][{POST_SNAPBACK}][/a]
If I am going to run an AAC listening test, it will most probably be at a lower bitrate and after Apple released an HE-AAC encoder, but that's another story.
But you obviously don't realize that including another codec again increases the difficulty of the test. Is it really wotrh it because of a codec that is IRRELEVANT today? I don't think so.
The anchor is not meant to be relevant. It is meant to sound guaranteedly bad.
According to Nero developers, their HE AAC down to 32kbps is good enough to challenge MP3 at 128. So, no, your proposal for anchor is not good enough (or, rather, "bad enough")
[a href="index.php?act=findpost&pid=342790"][{POST_SNAPBACK}][/a]
Well, it's not (and yes, I did test that) and this is a great opportunity to prove it. And the people that can't hear a difference between 32kbps HE-AAC and the top 128kbps encoders, certainly won't hear the much smaller differences between the various 128kbps encodes. So they aren't really well suited for this test.
if we are testing the creme de la creme of the encoders (we're testing LAME and not FhG which is included in MMJB and other rippers for the mass; and we're testing AoTuV and not the Xiph builds), maybe we should test WMA Pro and not Standard.
I believe so. So what if Pro isn't popular? Maybe, if it performs well, it will become so.
I don't agree with discounting it because it is not being pushed by Microsoft, or simply because it has been developed by Microsoft. If it is a potential contender, unlike Standard, then it should be seriously considered, IMH(and unknowledgeable)O.
When did users of this forum become so concerned whether Microsoft thought it had a future?
So, here is my idea for the codec collection before the weekend starts:
LAME 3.97
Nero AAC (whatever version will be up-to-date)
QuickTime/iTunes AAC (whatever version will be up-to-date)
AoTuV 4.5
WMA (Pro or Std - discussion is still open)
HE-AACv2 @ 32 kbps as low anchor
So, what do you think?
If it's OK, the last thing we need to decide about the codecs is whether to use WMA Pro or Std.
So, here is my idea for the codec collection before the weekend starts:
LAME 3.97
Nero AAC (whatever version will be up-to-date)
QuickTime/iTunes AAC (whatever version will be up-to-date)
AoTuV 4.5
WMA (Pro or Std - discussion is still open)
HE-AACv2 @ 32 kbps as low anchor
So, what do you think?
If it's OK, the last thing we need to decide about the codecs is whether to use WMA Pro or Std.
[a href="index.php?act=findpost&pid=342974"][{POST_SNAPBACK}][/a]
I think this is a great setup.
I would vote for WMA Pro, merely because it hasn't been tested properly before.
As said before, Pro all the way. Anybody with XP can encode with it at no additional cost.
So, here is my idea for the codec collection before the weekend starts:
LAME 3.97
Nero AAC (whatever version will be up-to-date)
QuickTime/iTunes AAC (whatever version will be up-to-date)
AoTuV 4.5
WMA (Pro or Std - discussion is still open)
HE-AACv2 @ 32 kbps as low anchor
So, what do you think?
If it's OK, the last thing we need to decide about the codecs is whether to use WMA Pro or Std.
[a href="index.php?act=findpost&pid=342974"][{POST_SNAPBACK}][/a]
WMA Std, please. WMA Pro is not currently a consumer product, and stands a good chance of changing when it does become a consumer product (with Vista).
Even if Pro is superior in terms of quality, it is more or less only of academic interest to the consumer population at this point in time.
I'm happy with it so far. And I vote for WMA Standard too.
I'm happy with it so far. And I vote for WMA Standard too.
[a href="index.php?act=findpost&pid=342989"][{POST_SNAPBACK}][/a]
+1
So, here is my idea for the codec collection before the weekend starts:
LAME 3.97
Nero AAC (whatever version will be up-to-date)
QuickTime/iTunes AAC (whatever version will be up-to-date)
AoTuV 4.5
WMA (Pro or Std - discussion is still open)
HE-AACv2 @ 32 kbps as low anchor
So, what do you think?
If it's OK, the last thing we need to decide about the codecs is whether to use WMA Pro or Std.
[a href="index.php?act=findpost&pid=342974"][{POST_SNAPBACK}][/a]
My vote is for WMA Pro.
And I also suggest excluding Nero from this group. It has been confirmed before that at this bitrate iTunes is at least equal to or better than Nero. Like Garf, I'm more interested in seeing how it performs at really low bitrates in another test.
If only one WMA codec is in, definitely test WMA Pro (as the other encoders in the test are the best representatives for their codec). It's a known fact that Standard performs slightly worse, but it would still be interesting to see exactly how big a difference there really is.. Perhaps both should be tested instead of Nero?
And I also suggest excluding Nero from this group. It has been confirmed before that at this bitrate iTunes is at least equal to or better than Nero. Like Garf, I'm more interested in seeing how it performs at really low bitrates in another test.
[a href="index.php?act=findpost&pid=342992"][{POST_SNAPBACK}][/a]
Sorry, but the four codecs mentioned are fixed.
So, here is my idea for the codec collection before the weekend starts:
LAME 3.97
Nero AAC (whatever version will be up-to-date)
QuickTime/iTunes AAC (whatever version will be up-to-date)
AoTuV 4.5
WMA (Pro or Std - discussion is still open)
HE-AACv2 @ 32 kbps as low anchor
So, what do you think?
If it's OK, the last thing we need to decide about the codecs is whether to use WMA Pro or Std.
[a href="index.php?act=findpost&pid=342974"][{POST_SNAPBACK}][/a]
I would probably drop one of the AAC codecs and include both WMA Std and Pro.
As Garf already pointed out, testing 2 AAC codecs is somewhat redundant and will probably provide similar enough results to end up in a tie or very close to one, and the end result of the whole test will be less interesting because of it.
On the other hand, WMA Std and Pro are two clearly different codecs which will probably provide more significantly different results.
If you did this, every codec included in the test would be different, making for a more comprehensive result. If it's a multiformat listening test, why not make it as multiformat as possible? I think a nice goal of the test would be to highlight major differences between codecs, not so much subtle differences among two competing implementations of the same codec -- that would be better left for another test IMO.
Other than that, I think it's looking good.
okay, I'm beginning to like the sound of both WMA Std and WMA pro now as well when put in the above light. It truly is AAC overkill...
This is so close to being decided after 7 pages, I can feel it (maybe by page 10? ).
I would probably drop one of the AAC codecs and include both WMA Std and Pro.
same here, drop NERO's, this way, we would be testing 6 diferent technologies and encoders =)
HE-AACv2 @ 32 kbps as low anchor
I really like to see how 128 kbps HE-AAC compared to other non-SBR 128 kbps
Its really interesting to see how good SBR can reproduce high frequency, can it really trick us.
I would probably drop one of the AAC codecs and include both WMA Std and Pro.
Agree, Guru has tested it thoroughly with hundreds of samples, iTunes has more transparent samples compared to Nero..... I completely agree that we should only have one representative for each format...well, of course Apple will be representing AAC in this "tournament"
This is a "multiformat test", not "single format- different encoders" test....just my viewpoint. We can test iTunes vs Nero vs etc etc next time
WMA and WMA PRO is 2 different formats, and I've heard that WMA PRO is written by the inventor of AAC?
mp3 - LAME
AAC - iTunes
Vorbis - aoTuV
WMA - WMA9.1
WMA PRO - WMA9.1 PRO
HE-AAC - Nero or CT??? (I suggest 128k)
EDIT: Nero doean't have 128kbps for HE-AAC.... only CT offers up to 128kbps
EDIT: Nero doean't have 128kbps for HE-AAC.... only CT offers up to 128kbps
[a href="index.php?act=findpost&pid=343004"][{POST_SNAPBACK}][/a]
That's because it doesn't make sense to use it at 128kbps. The format was not designed for that bitrate, and the AAC devs have said many times that standard AAC should be used instead.
So, here is my idea for the codec collection before the weekend starts:
QuickTime/iTunes AAC (whatever version will be up-to-date)
[a href="index.php?act=findpost&pid=342974"][{POST_SNAPBACK}][/a]
I hope the VBR switch is turned on for the iTunes sample.
Would 96K WMA standard be an acceptably bad low anchor?
I really like to see how 128 kbps HE-AAC compared to other non-SBR 128 kbps
Its really interesting to see how good SBR can reproduce high frequency, can it really trick us
It would not be an anchor any longer. The point of an anchor is to be clearly worse/better than all the others.
iTunes has more transparent samples compared to Nero.....
Still, they
are different. Not like lame 3.90 vs. 3.97. Not like aoTuV 4.51 vs. Xiph 1.1.1. iTunes' encoder is
not an improvement over Nero's, in any way. Still, both are fairly popular on the AAC scene, both keep improving and updating and stuff. More of that, they are constantly competing with each other.
Finally, what we test here is not just a fixed quality parameter at 128 kbps, but the relative differencies between them.
I'd say, some are satisfied with both LAME and aoTuV @ 128 kbps and give both of them 4/5 ("perceptible but not annoying") on a transparence scale. Does it make them equal in terms of quality? Hell no.
Guruboolez's tests are the best illustration for that: he gives 2.x and 3.x to a [semi-]transparent encodings
for a reason, and that reason is
relativity. Pure subjective measuring must be done when there is
something to measure against.
I hope I've made myself clear.
BTW, I vote for WMA Std for the reasons mentioned above by Roberto and the others. WMA Pro is not popular at all, isn't HW-supported, isn't constantly improving, and just doesn't worth it — it's 99% clear that it will not win when there is Vorbis and AAC.
HE-AACv2 @ 32kbps as a low anchor sounds curious at least, but will it be low enough for an anchor? How about doing a couple of encodings of the popular samples (6—10) and making them available for us to judge just in terms of desired quality?
BTW, I vote for WMA Std for the reasons mentioned above by Roberto and the others. WMA Pro is not popular at all, isn't HW-supported, isn't constantly improving, and just doesn't worth it — it's 99% clear that it will not win when there is Vorbis and AAC.
[a href="index.php?act=findpost&pid=343019"][{POST_SNAPBACK}][/a]
Emphasis mine.
And you know this because... you've tested it recently? Because others have tested it recently?
This is exactly the attitude I think needs to be avoided, and is part of the problem Woodenville pointed out. Nobody is testing it, and it sounds as if maybe there have been some version changes, yet people are making all kinds of assumptions (and TOS 8 violations I might add).
We should just test the damn thing and then we can actually say with some authority. I'm really surprised at all the excuses I keep seeing for why it shouldn't be tested. Really, I thought these tests were designed to illustrate codec quality and to
learn something, but it seems as if many people are not directly interested in these goals.
Why continue to test almost the exact same codec configuration (which is likely to produce similar and expected results as the last time) and continue to ignore some rather obvious choices which could provide interesting results? Why introduce redundancy in the test (by testing two AAC encoders) and ignore other formats which fit more in line with the multiformat purpose?
Also, it's rather curious that you suggest not testing WMA Pro because you say it's certain that it won't "win" with AAC and Vorbis, but yet you suggest testing WMA Std instead, which (at least by design) should not be even as good as Pro...
MP3 - LAME
AAC - iTunes
Vorbis - aoTuV
WMA - WMA9.1
WMA PRO - WMA9.1 PRO
anchor - whatever (or no anchor)
looks very logical ... maybe adding the famous anchor, whatever that may be...
I'd say, some are satisfied with both LAME and aoTuV @ 128 kbps and give both of them 4/5 ("perceptible but not annoying") on a transparence scale. Does it make them equal in terms of quality? Hell no.
No.... I'm not looking at the final scale of Guru's test, what I'm counting is how many 5 iTunes got...
I thought "5" means imperceptible difference?
whether it is mp3, aac, wma or anything , if they all got a "5", it means they are of the same quality, = original (perceptually)
I know what you mean, if 4.5 is given to both different codecs/encoders, it doesn't mean that everybody will be giving the same rating....but once its "5", it should be the same.
So, here is my idea for the codec collection before the weekend starts:
LAME 3.97
Nero AAC (whatever version will be up-to-date)
QuickTime/iTunes AAC (whatever version will be up-to-date)
AoTuV 4.5
WMA (Pro or Std - discussion is still open)
HE-AACv2 @ 32 kbps as low anchor
. . . .
[a href="index.php?act=findpost&pid=342974"][{POST_SNAPBACK}][/a]
I would probably drop one of the AAC codecs and include both WMA Std and Pro.
. . . .
[a href="index.php?act=findpost&pid=342996"][{POST_SNAPBACK}][/a]
Agreed:
LAME 3.97
iTunes AAC
AoTuV 4.5
WMA Pro
WMA Std
Low Anchor (Nero HE-AAC @ 32kb/s)
I have some samples in mind already and we'll start discussing them soon. While at it, would you guys prefer 18 or 12 samples?
[a href="index.php?act=findpost&pid=341355"][{POST_SNAPBACK}][/a]
I would prefer 18 samples, and a bit longer time to complete the test than the 12 days you proposed. People have plenty of time for theese things during the holidays when all the packets have been opened and all the firecrackers cracked...
If I may be so bold and start talking about samples already, I suggest that you mix both typical hard samples with easier samples so that flexible vbr codecs will not have a bitrate advantage to more cbr like codecs?
WMA Pro
WMA Std
Too much attention to Microsoft. Lets exlude one of them (you decide which one), and include either
1. MP3 from Fhg (WMP) or
2. Musepack
or both.
That would be more interesting and varied
Sorry, but as I mentioned already, the four encoders LAME, AoTuV, Nero Digital and QuickTime / iTunes AAC are fixed.
The argument that Guru performed a test already doesn't hold. Garf said that the results are not meaningful because they come from one person. Also, I am sure a lot of people would like to know how the encoder they waited for so long (Nero AAC) performs compared to its greatest rival (iTunes AAC).
2. Musepack
[a href="index.php?act=findpost&pid=343052"][{POST_SNAPBACK}][/a]
I think the majority agrees that MusePack shouldn't be tested at this bitrate.
What about testing an MP3 codec from Fraunhofer IIC ?
Yes that is right, Sebastian. But what do you think about what I've said ?
Don't you think we should give a chance to MPC to restore its reputation ?
[a href="index.php?act=findpost&pid=343056"][{POST_SNAPBACK}][/a]
No.
Really, at this bitrate, it would lose against Vorbis and AAC. Also, MPC is not a popular format at this bitrate. It's not used in music stores and it has no HW support.
I'm surprised to see people agree to discard Nero Digital AAC from the test. The new encoder and the enhancements in the LC core were anounced more than two years ago, and now that this encoder is available, people don't want to test it? And why not discarding iTunes AAC? Because my test conclude on iTunes superiority? So should we rely on a personal evaluation or shouldn't we give credit to this form of listening test? It's time to make a choice
All mentionned encoders have their own place in this test. And atrac3+ (I dislike it) no less than WMA or WMAPro. Therefore, there are 8 possible competitors:
- AAC Apple
- AAC Nero
- atrac3 or atrac3plus
- MP3 LAME
- MPC
- Vorbis
- WMA
- WMAPro
And anchors (at least a low one) must be included. Therefore, we have nine possible encoders, and we must choose between all of them. Each suppression is eaqually contestable.
So we need to precise the purpose of this test: what's the goal of this test? Which criterion for choosing the tested encoders?
I see few criterions:
- portable players: the we must go for AAC, ATRAC, MP3, VORBIS, WMA[std].
- constantly developed encoders: AACs [both Nero and Apple], MP3, Vorbis
- High quality encoders: AAC [both Nero and Apple], MP3, MPC, Vorbis, WMAPro
- encoders mostly used in HA.org at this bitrate : AAC [Nero, Apple], MP3, Vorbis [just a guess, but it seems likely to me]
- popular formats: MP3 [not necessary LAME], WMA, Vorbis, AAC [iPod -> iTunes].
Some associations are not really coherent in my opinion. WMA & WMAPro is one of them. Choosing WMA but not ATRAC is also contestable, if we keep in mind that Sony push a lot of energy to give visibilty to his formats (atrac->atrac3->atrac3+; Connect Music Store; DRM; support in flash players, HDD, CD; gapless: this format is maybe poor but is simply more complete than AAC, Vorbis and MP3!).
I strongly believe that we need to clarify what we're going to test (not the encoders themselves, but the goal).
____
About anchor:
People really need to understand what anchors are for. They're not another contenders -not even a poor one-, they're not here to give us additional information but to increase the accuracy of all others informations. It's right that some encoders are more interesting as anchor than others, but this point is secondary. The primary goal of anchors is to avoid floating notation; the rest is accessory.
If one true contender behaves poorly, how will the listener react? Some of them (especially trained one, more inclined to artefact intolerance) might rate it very harshly, but some other trained listeners might make the effort of relativization and grant a less aggressive mark. In other words, two listeners detecting the same amount of annoyance could rate very differently the same encoding. The low anchor is here to prevent this and to put some uniformity in the test. But to achive this, it's not enough to appoint a slightly or less slightly inferior encoder as low anchor: a very poor quality is really a requirement. That's why encoders like WMA, modern Fhg MP3 or HE-AAC at 64 kbps or even 32 with PS can't be used as low anchor. It's very likely that these encoders are poorer than the other contenders, but they're certainly not as poor to most hearings.
If people want to test how modern low bitrate encoders behave compared to... let's say MP3 at 128 kbps, it would be better to dedicate a test to this question. A low bitrate comparison with MP3@128 kbps as high anchor is feasible, and was already done. If I understood correctly, Sebastian Mares is interested to organize it once Apple will release it's own HE-AAC tool. But there's no need to risk to hamper this test by using a fake-anchor selected for our curiosity. Let's be clear: I'm also interested to see how new HE-AAC encoders perform at 64, 48 and 32 kbps in a collective test [I have my own opinion], but this question is clearly not a part of this test. There's also no need to arbitrary choose HE-AAC over Vorbis or atrac3plus at such low bitrate; no need also to choose one HE-AAC implementation over another one.
A low anchor is a low anchor: something bad that should remind to all listeners what "crap sound" mean. The choice for low anchors is therefore limited: it's either a very poor lossy encoder (a good one but deliberately untuned may also work) or something lowpassed/downsampled. I clearly prefer the first solution, simply because the quality problems of the low anchor would come from the same family as those ones audible with the true contenders.
So we need to precise the purpose of this test: what's the goal of this test? Which criterion for choosing the tested encoders?
[a href="index.php?act=findpost&pid=343061"][{POST_SNAPBACK}][/a]
Yes, you are right. So, considering this bitrate, I'd say popular and portable formats.
Although ATRAC is portable, it's not very wide-spread.
MusePack is not portable and it's not "popular" at this bitrate. People using MPC usually encode their tracks at bitrates like 192 kbps and more since that's what the format was designed for. While some people are still working on MPC, the changes are rather minor and I doubt that MPC has seen improvements at 128 kbps.
This makes 6 competitors.
Nero and iTunes AAC should be included because of the competition and because of the announced enhancements.
MP3 is a very popular format and I think we should see how LAME performs against the state-of-the-art codecs available now.
Vorbis is also an important codec. It's becoming more and more portable and quality-wise it might be superior to AAC.
As I already said couple of times, these four encoders are fixed. Please stop disussing whether or not LAME or Nero should be dropped.
What is left now is the question about WMA. Thinking about it, I came to the conclusion that WMA standard is the best choice if we are testing popular and portable formats like I said. Roberto is perfectly right - if MS wanted us to use WMA Pro, they would've included it in PlaysForSure. If WMA Pro will replace WMA Std in 2006 or whenever Vista is expected to be release, fine, we can test it in 2006. By that time, this test is going to be outdated anyways.
I think we're going in circles now. Sebastian has made up his mind on the criteria. My choice would be to test for:
High quality encoders
irregardless of format popularity.
But I am just a sole voice. How to cross this impasse.
But I am just a sole voice.
Now we're a duet.
I think we're going in circles now. Sebastian has made up his mind on the criteria.
[a href="index.php?act=findpost&pid=343075"][{POST_SNAPBACK}][/a]
Well, yes, since I see this is going nowhere. 10 votes for WMA pro, 10 votes for WMA std.
As stated, WMA pro can be tested in a year when it's more popular (if).
But I am just a sole voice.
Now we're a duet.
[a href="index.php?act=findpost&pid=343076"][{POST_SNAPBACK}][/a]
- The final decision rests with the test administrator, of course...I don't envy his position too much with all these dissenting views.
So we need to precise the purpose of this test: what's the goal of this test? Which criterion for choosing the tested encoders?
[a href="index.php?act=findpost&pid=343061"][{POST_SNAPBACK}][/a]
Yes, you are right. So, considering this bitrate, I'd say popular and portable formats.
[a href="index.php?act=findpost&pid=343069"][{POST_SNAPBACK}][/a]
It suits to me. I consider the
portable criterion as the most important one as long as 128 kbps is targeted. I don't think that people here are using 128 kbps for "archiving" or something close to that, but it seems that current encoders may reach an excellent quality on portable players.
I don't really like the "
popular" criterion. Excepted if by popular you mean "popular on HA.org and other internet forums". It's not by elitism, but it's just that I fear that aoTuV beta 4 or LAME VBR don't meet the "popular" criterion.
But if your test is for
portable and popular formats tested at their best possible settings, then it's OK (for me, and for you I suppose ).
What is left now is the question about WMA. Thinking about it, I came to the conclusion that WMA standard is the best choice if we are testing popular and portable formats like I said. Roberto is perfectly right - if MS wanted us to use WMA Pro, they would've included it in PlaysForSure. If WMA Pro will replace WMA Std in 2006 or whenever Vista is expected to be release, fine, we can test it in 2006. By that time, this test is going to be outdated anyways.
I fully agree with you. I'm very interested by WMAPro, but in its current state, it's just for fun and curiosity. I'd rather test the new WMAPro encoder which will be released... if ever a new WMAPro will be released (
we never knows)
I'd go for WMA Std, which makes sense.
So we need to precise the purpose of this test: what's the goal of this test? Which criterion for choosing the tested encoders?
[a href="index.php?act=findpost&pid=343061"][{POST_SNAPBACK}][/a]
Yes, you are right. So, considering this bitrate, I'd say popular and portable formats.
[...]
As I already said couple of times, these four encoders are fixed. Please stop disussing whether or not LAME or Nero should be dropped.
What is left now is the question about WMA. Thinking about it, I came to the conclusion that WMA standard is the best choice if we are testing popular and portable formats like I said. Roberto is perfectly right - if MS wanted us to use WMA Pro, they would've included it in PlaysForSure. If WMA Pro will replace WMA Std in 2006 or whenever Vista is expected to be release, fine, we can test it in 2006. By that time, this test is going to be outdated anyways.
[a href="index.php?act=findpost&pid=343069"][{POST_SNAPBACK}][/a]
So popular and portable is the target. It's the first time it's mentioned in this thread, but it's important to know. Then you should def pick WMA Std. But you should also include ATRAC which is very popular and portable (there are many md players around).
As you might have guessed, I'm more interested in quality... but that will come some other day when I have the time to conduct my own tests.
Btw, since it's an open discussion forum, I think it would be difficult to force people to stop discussing certain aspects unless you lock the thread...
I think you agree that you cannot make everyone happy. Of course this test is also about quality, but it's also about popular formats. How many people actually use WMA Pro at 128 kbps (or at all)? I already said that if MS wants us to use WMA Pro in 2006, I have no problem in testing that format in 2006.
Today however, the music stores that let you download WMA sell WMA standard, the audio players supporting WMA support WMA standard, Windows Media Player encodes to WMA standard...
Regarding ATRAC, I think it's not such an important format. The only software that can encode to ATRAC is Sonic Stage AFAIK. The only portables supporting ATRAC are from Sony. Also, MDs use another ATRAC version if I am not mistaken, so it has no sense. It might only be interesting for people having a CD player that can also play ATRAC files.
Having read this thread I'd go with these
1. Nero AAC
2. iTunes AAC
3. Lame MP3
4. WMA std
5. Vorbis Aotuv
6. Super ultra crappy anchor
The most important kind of annoyance characterizing atrac3/atrac3plus is that that this format is very restricted even for software (on playback). You absolutely need SonicStage installed to play atrac3 files. There exists winamp or DirectShow plug-ins, but they're based on SonicStage DRM routines. You can't even burn your atrac3/atrac3plus on CD with a basic burning applications and play your files on another computer on a re-installed OS. The DRM protection don't allow this (you must backup licenses with the encodings), and you can't disable the DRM protection.
This is the most important difference between atrac3 and WMA (and other formats).
That's why I would also discard atrac from this test: it's the only format usable on portable but not on PC
Now situation may have change. I haven't installed SonicStage for a long time, and the DRM restriction are maybe softened. Could someone confirm what I said?
I know this will upset Gambit, but I just encoded a few samples with Nero HE-AACv2 at 32 kbps and the results from my quick listening tests show that it's not suitable as low anchor.
I can post some 20 seconds clips of Cockney Rebel - Sebastian, Sash! - Stay, Jam & Spoon feat. Rea - Set Me Free and some others if you'd like to convince yourself. I think I'd have to use some really shitty encoder like Shine (sorry Gabriel ) and leave the comparison of HE-AACv2 for another test.
Having read this thread I'd go with these
1. Nero AAC
2. iTunes AAC
3. Lame MP3
4. WMA std
5. Vorbis Aotuv
6. Super ultra crappy anchor
[a href="index.php?act=findpost&pid=343087"][{POST_SNAPBACK}][/a]
Couldn't agree more...
Having read this thread I'd go with these
1. Nero AAC
2. iTunes AAC
3. Lame MP3
4. WMA std
5. Vorbis Aotuv
6. Super ultra crappy anchor
[a href="index.php?act=findpost&pid=343087"][{POST_SNAPBACK}][/a]
i also vote for this
Nero will do one update of the codec, in order to include some quality updates.
I hope it will make it for this test, as I believe it would be worth it
1. Nero AAC
2. iTunes AAC
3. Lame MP3
4. WMA std
5. Vorbis Aotuv
6. Super ultra crappy anchor
If the premise is portable and popular then the above makes sense - WMA Pro obviously doesn't fit that criteria yet. If it were quality only then it would be different...
Edit: I'd be interested to know the % of people who use Nero AAC on a portable though...
Nero will do one update of the codec, in order to include some quality updates.
I hope it will make it for this test, as I believe it would be worth it smile.gif
sebastion hasn't set the date yet, hasn't he?
If it's just a few weeks before the next quality update, I think it's worth waiting, since this test, besides comparing multiformat, the objective of this test is to compare the quality of Nero & iTunes.
Nero will do one update of the codec, in order to include some quality updates.
I hope it will make it for this test, as I believe it would be worth it
[a href="index.php?act=findpost&pid=343097"][{POST_SNAPBACK}][/a]
I think what really matters here is... how long will it take?
sebastion hasn't set the date yet, hasn't he?
I am planning to start a multiformat listening test at 128 kbps on November, 30th which should end on December, 11th. The test start can be postponed if necessary.
We'll try to meet the Nov 30th deadline - but there might be a delay of 2-3 days maybe.
I hope not.
Here (http://www.rarewares.org/sebastian/low_anchor_samples.zip) are some samples I encoded for the low anchor descision. The ZIP contains some LAME -V5 encodes and Nero HE-AACv2 @ 32 kbps samples. IMO, they sound too good.
Edit: I only want a quick playback test, no need for ABXing and other fancy stuff.
Having read this thread I'd go with these
1. Nero AAC
2. iTunes AAC
3. Lame MP3
4. WMA std
5. Vorbis Aotuv
6. Super ultra crappy anchor
[a href="index.php?act=findpost&pid=343087"][{POST_SNAPBACK}][/a]
Couldn't agree more...
[a href="index.php?act=findpost&pid=343093"][{POST_SNAPBACK}][/a]
This list looks good to me too.
I would like to suggest a "para contest" for the dropouts. After the main test a smaller scale additional test could be organized. It could be as simple as a thread for posting personal ABX results.
If this "para contest" brings up something significant it would very interesting and perhaps some conclusions could be made. The common thing with the main test should be at least the same test samples. If the lossless samples from the original test were available, people could make their own encodings for this less formal test. However, the encoders and the proper switches for each encoder should be defined beforehand and the correct use of them should be compulsory for getting the results included.
The participants could be:
- The test winner of the main test (or one of them if a sole winner was not found)
- WMA Pro at ~128 kbps VBR
(I found an interesting thing about WMA Pro options. It has only
24-bit 2-channel settings. I have no idea what Microsoft means by this. I thought lossy encoders don't have fixed bit depths.)
- Musepack at ~128 kbps
- The WMP10 version of the Fraunhofer MP3 encoder at 128 kbps CBR (no VBR options are available). I guess this encoder has the largest installed base of all 128 kbps capable MP3 encoders because the Windows Update installs it automatically. It would make a sort of anchor, though probably not a low anchor...
- WMA Pro at ~128 kbps VBR
(I found an interesting thing about WMA Pro options. It has only 24-bit 2-channel settings. I have no idea what Microsoft means by this. I thought lossy encoders don't have fixed bit depths.)
maybe it means it will decode at 24-bit by default
Lossy encoders will encode differently for different bitdepth i suppose,
If I'm right
16-bit can store up to 96 dB range of information
24-bit can store up to 144dB range of information
mp3 does not support 24-bit input.
Lossy encoders will encode differently for different bitdepth i suppose,
[...]
mp3 does not support 24-bit input.
[a href="index.php?act=findpost&pid=343132"][{POST_SNAPBACK}][/a]
No, no no no!
...
[a href="index.php?act=findpost&pid=343130"][{POST_SNAPBACK}][/a]
Dude, that makes no sense. The formats were dropped for a reason. MusePack is supposed to perform bad at 128 kbps, WMA pro will be tested when MS thinks it's time for it to be used en-mass.
- The WMP10 version of the Fraunhofer MP3 encoder at 128 kbps CBR (no VBR options are available). I guess this encoder has the largest installed base of all 128 kbps capable MP3 encoders because the Windows Update installs it automatically. It would make a sort of anchor, though probably not a low anchor...[a href="index.php?act=findpost&pid=343130"][{POST_SNAPBACK}][/a]
Well, I doubt it would make a high anchor.
MusePack is supposed to perform bad at 128 kbps,
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=343136")
Uhm, no:
[a href="http://www.rjamorim.com/test/multiformat128/plot18z.png]http://www.rjamorim.com/test/multiformat128/plot18z.png[/url]
Well, does it even make sense to test a dead format? I really doubt that we're going to see improvements. And anyways, people using MPC are using it at bitrates over 160 kbps.
Here (http://www.rarewares.org/sebastian/low_anchor_samples.zip) are some samples I encoded for the low anchor descision. The ZIP contains some LAME -V5 encodes and Nero HE-AACv2 @ 32 kbps samples. IMO, they sound too good.
Edit: I only want a quick playback test, no need for ABXing and other fancy stuff.
[a href="index.php?act=findpost&pid=343124"][{POST_SNAPBACK}][/a]
Wow! I'd never thought 32kbps could sound so good.. It's easily on par with older mp3 encoders at 112-128kbps imho.
mp3 does not support 24-bit input.
Not that sure
Here (http://www.rarewares.org/sebastian/low_anchor_samples.zip) are some samples I encoded for the low anchor descision. The ZIP contains some LAME -V5 encodes and Nero HE-AACv2 @ 32 kbps samples. IMO, they sound too good.
Edit: I only want a quick playback test, no need for ABXing and other fancy stuff.
[a href="index.php?act=findpost&pid=343124"][{POST_SNAPBACK}][/a]
Excepted
EnolaGay and the vocal part of
Waiting, the sound quality is indeed amazing for such bitrate, and could compete with poor and even decent encoders set at 128 kbps.
It's clearly too good for a low anchor.
...
[a href="index.php?act=findpost&pid=343130"][{POST_SNAPBACK}][/a]
Dude, that makes no sense. The formats were dropped for a reason. MusePack is supposed to perform bad at 128 kbps, WMA pro will be tested when MS thinks it's time for it to be used en-mass.[a href="index.php?act=findpost&pid=343136"][{POST_SNAPBACK}][/a]
Makes no sense? I know the formats were dropped for a reason. That's why I spoke about "dropouts" and "para contest"
No one has to officially arrange this alternative test. Anyone can post ABX findings to HA. Many people have shown interest towards WMA Pro and some have mentioned Musepack too (actually, I guess "
supposed to perform bad" kinda asks to be proved). After participating the main test it would be interesting and easy to ABX a couple of additional samples and post the results. I just meant that if some guidelines were defined it might make the findings more relevant.
Personally I am going to try WMA Pro with the test samples because I am interested.
Actually, I have already tried to ABX a few samples encoded at ~200 kbps with WMA Pro, Musepack and Vorbis. So far I have nothing to report because unfortunately I failed to ABX them.
I just wanted to get the idea out for public discussion. However, because this is a bit OT I guess I should have posted to a separate thread. Sorry for that.
[span style='font-size:7pt;line-height:100%']Edit: typo[/span]
I'll add my vote for Latexxx's list of codecs, although it would be interesting in the future to see WMA std compared to WMA pro.
what's the goal of this test? Which criterion for choosing the tested encoders?
Thank you for putting in plaing English what I was trying to say.
Test has reduced meaning, if it is not set up to test a specific question.
I personally don't think "codecs that are widely available for portable use" is a the most interesting selection criteria.
Why?
Many smaller artifacts are completely masked in portable listening settings (bad player, bad headphones, increased self-noise, lots of background noise).
For most people, even 48-64kbps could be indistinguishable from originals when walking in a city and listening to the signals from sub-optimal portable/headphones (and no, I'm not going to try to ABX this )
This is also the reason, why I'm more interested in WMA Pro than WMA Std (I don't think the portable support is _that_ relevant, but at 128kbps or above, sound quality is when listening in a quiet non-portable environment).
Std has been tested (even if it may have _slightly_ improved since last tested). Pro has been not at all, but it is possible to use it in a wide variety a of settings.
I hate to jump in the middle of discussion - but since some people raised concerns about inclusion of WMAPro and MPC...
Hmm - why don't we organise an extension test after the initial 128 kbps multiformat test? Choose three winners from the multiformat test, and conduct "extension" test with MPC anc WMAPro as the additional codecs?
This way we would basically be able to test all state-of-the-art codecs, performing at the range that should give optimum quality for most of the users.
Std has been tested (even if it may have _slightly_ improved since last tested). Pro has been not at all, but it is possible to use it in a wide variety a of settings.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=343193")
[a href="http://www.rjamorim.com/test/128extension/presentation.html]http://www.rjamorim.com/test/128extension/presentation.html[/url]
Hmm - why don't we organise an extension test after the initial 128 kbps multiformat test? Choose three winners from the multiformat test, and conduct "extension" test with MPC anc WMAPro as the additional codecs?
[a href="index.php?act=findpost&pid=343195"][{POST_SNAPBACK}][/a]
It's not a bad idea. But it would make two consecutive multiformat listening tests at 128 kbps: would people be motivated to start a second test?
If the goal of the test is to compare the best available encoders, I'd rather see pre-tests, pools or two semi-final (four encoders in each) in order to choose the very best encoders.
Two pre-tests with 6...8 samples and four encoders first, and a final one with 15...20 samples and the four best format.
But it's probably too complicated. I'm not sure that listeners are motivated enough. One single test, 5 encoders and one anchor is way more simplier.
True - then, again - extension test could be organised after the sufficient break - if some sources say that MS will release new encoder in January - then, great - let's wait one month and do an extension test?
I believe that 30-45 days of break between tests would generate enough "rest" and get back the motivation of the people to do a second one.
http://www.rjamorim.com/test/128extension/presentation.html (http://www.rjamorim.com/test/128extension/presentation.html)
Silly me, had completely missed that. Thanks.
I hate to jump in the middle of discussion - but since some people raised concerns about inclusion of WMAPro and MPC...
Hmm - why don't we organise an extension test after the initial 128 kbps multiformat test? Choose three winners from the multiformat test, and conduct "extension" test with MPC anc WMAPro as the additional codecs?
This way we would basically be able to test all state-of-the-art codecs, performing at the range that should give optimum quality for most of the users.
[a href="index.php?act=findpost&pid=343195"][{POST_SNAPBACK}][/a]
True - then, again - extension test could be organised after the sufficient break - if some sources say that MS will release new encoder in January - then, great - let's wait one month and do an extension test?
I believe that 30-45 days of break between tests would generate enough "rest" and get back the motivation of the people to do a second one.
[a href="index.php?act=findpost&pid=343201"][{POST_SNAPBACK}][/a]
That is pretty much what I said about testing WMA Pro when it becomes more popular. Maybe MS has plans to push the format in 2006.
Anyways, regarding MPC, what's the point in testing a dead format? No HW support (maybe with the Rockbox firmware, but that's used by geeks only ), uncertain patent status, painfully slow development (if any), no multi-channel, no high sampling rates, slow seeking, cannot be used in movies...
By the way, you can't even say that the format is the best since Vorbis beat it last time and I am sure AAC slowly reaches its level, too (if it hasn't already).
if some sources say that MS will release new encoder in January - then, great - let's wait one month and do an extension test?
[a href="index.php?act=findpost&pid=343201"][{POST_SNAPBACK}][/a]
It would be an encoder update. Does this event really justify an extension test? I'm not against the idea, but each test means effort (for all listeners and for the organizer) and I'm not sure that many people would be ready to periodically restart the same exercise.
___
About the test itself:
It's maybe time to investigate about the average output of each encoder and if necessary to find the corresponding setting. Sebastian said that iTunes AAC 128
VBR will be included. The current VBR mode of iTunes (apparently no frames are encoded at less than 128 kbps) implies an average bitrate higher than 128 kbps. It's probably something between 134 and 140 kbps. We can't lower it. And we need to precisely know this value.
The same goes for LAME -V5 (with or without --athaa-sensitivity 1) and also for Nero Digital VBR.
To know it, we need to encode several albums or tracks with each encoder. I could only do it for classical music and it's not enough. That's why different members should do it with various genres of music. I remember that ff123, JohnV, Spoon and probably other members did it in the past. It would be nice to see gather as many information and as soon as possible.
If people have time, they could try to get the corresponding bitrate of WMA VBR and WMA PRO VBR (even if this last one isn't forecast yet). Why WMA especially? Because the VBR scale of these encoders have few levels. There's a real risk that neither VBR 50 nor VBR 75 correspond to what we need. If it occurs, this may end the debate. I don't think it would be wise to use WMA at 128 kbps CBR against different contenders at VBR and with 135 kbps on average. Except if Sebastian is fond of criticism and sarcasms... And WMAPro 2-pass is not a good solution either (I don't know how 2 pass mode is working for audio, and the bitrate distribution may be different if we encode a full track: a short sample cut from a full track would therefore be different from a short sample directly encoded with two passes).
Are some people volonteers to build a bitrate table?
WMA will be bitrate managed. -a_mode 3 -a_setting 128_44_2. According to some quick tests, the average bitrate really is 128 kbps.
LAME's -V5 produces a bitrate over 128 kbps. It's actually something like 130 to 150 kbps with the samples I tested (from my private music collection).
I didn't check iTunes or Vorbis yet. I would like Gabrei and Aoyumi to tell me which settings they think I should use. Fine tunings if the avg. bitrate is too high can be made later. Anyways, I actually wanted to talk about settings tomorrow.
What I want to know is whether or not people agree that HE-AACv2 at 32 kbps is too good for a low anchor. If yes, I think I will fall back to Shine.
WMA will be bitrate managed. -a_mode 3 -a_setting 128_44_2. According to some quick tests, the average bitrate really is 128 kbps.
[a href="index.php?act=findpost&pid=343206"][{POST_SNAPBACK}][/a]
What kind of encoding method is it? ABR? A true and unrestricted VBR mode can't always stays close to the targeted value.
For Lame I'd suggest -V5 --vbr-new. Overall it should average in the 130kbps range, but this should be tested on a big corpus of samples.
If bitrate is not suitable, then I think that there is only the abr option.
For Lame I'd suggest -V5 --vbr-new.
[a href="index.php?act=findpost&pid=343212"][{POST_SNAPBACK}][/a]
May I eventually suggest --athaa-sensitivity 1 (http://forum-images.hardware.fr/icones/smilies/whistle.gif)
WMA will be bitrate managed. -a_mode 3 -a_setting 128_44_2. According to some quick tests, the average bitrate really is 128 kbps.
[a href="index.php?act=findpost&pid=343206"][{POST_SNAPBACK}][/a]
What kind of encoding method is it? ABR? A true and unrestricted VBR mode can't always stays close to the targeted value.
[a href="index.php?act=findpost&pid=343208"][{POST_SNAPBACK}][/a]
2-pass, bitrate-managed VBR
Edit: Available modes are:
0: 1-pass CBR (default).
1: 2-pass CBR.
2: Quality-based VBR.
3: Bit rate-based VBR (two-pass).
4: Bit rate-based peak VBR (two-pass)
WMA will be bitrate managed. -a_mode 3 -a_setting 128_44_2. According to some quick tests, the average bitrate really is 128 kbps.
[a href="index.php?act=findpost&pid=343206"][{POST_SNAPBACK}][/a]
What kind of encoding method is it? ABR? A true and unrestricted VBR mode can't always stays close to the targeted value.
[a href="index.php?act=findpost&pid=343208"][{POST_SNAPBACK}][/a]
2-pass, bitrate-managed VBR
[a href="index.php?act=findpost&pid=343215"][{POST_SNAPBACK}][/a]
If two pass encoding for audio is comparable to the video encoding process, shouldn't we encode the full track with this mode and then extract the corresponding range to get the sample? Are direct encoding of 15...30 sec samples representative of the quality we could get in real conditions (full track encodings)?
For Lame I'd suggest -V5 --vbr-new.
[a href="index.php?act=findpost&pid=343212"][{POST_SNAPBACK}][/a]
May I eventually suggest --athaa-sensitivity 1 (http://forum-images.hardware.fr/icones/smilies/whistle.gif)
[a href="index.php?act=findpost&pid=343214"][{POST_SNAPBACK}][/a]
iirc this is not available in Lame 3.97 beta
We can either use 3.98 alpha or ask John33 to compile 3.97 beta in debug mode
well...yes of course.
but uses a codec compiled in debug mode is pretty useless concerning a "normal" usage (i.e archiving). It's more interesting for developpers. but back to the topic now.
Oh, and I think the test should be open for at least a month. 12 days are not long enough IMO.
[a href="index.php?act=findpost&pid=343237"][{POST_SNAPBACK}][/a]
I think I could extend it to last until December 18th. Anything more than that probably doesn't make sense because of Christmas / New Year and the related "problems".
Oh, and I think the test should be open for at least a month. 12 days are not long enough IMO.
i think the same, a month is not that much, for such a test.
Oh, and I think the test should be open for at least a month. 12 days are not long enough IMO.
[a href="index.php?act=findpost&pid=343237"][{POST_SNAPBACK}][/a]
I think I could extend it to last until December 18th. Anything more than that probably doesn't make sense because of Christmas / New Year and the related "problems".
[a href="index.php?act=findpost&pid=343245"][{POST_SNAPBACK}][/a]
What problems? To be honest, I think people will have more time to test during the holidays. They will have some days free, and you know we are nerds, many of us use the free days to work on our hobby coding projects and stuff like that. So a listening test is perfect for that time. (Yes, I need to get a life. )
...(Yes, I need to get a life. )
[a href="index.php?act=findpost&pid=343270"][{POST_SNAPBACK}][/a]
Most of us need to...
I encoded the same 25 various genre music tracks I used in my LAME 3.97b1 VBR bitrate test (http://www.hydrogenaudio.org/forums/index.php?showtopic=37011&view=findpost&p=328558).
The first column shows the Monkey's Audio (Normal) bitrates. It might give some idea about the file complexity.
I used Vorbis aoTuV beta 4.51 (at -q4 and -q4.25) and the newest iTunes 128 kbps VBR AAC (QT 7.03). The LAME bitrates are from my previous test.
(http://kotisivu.mtv3.fi/alexb/ha/128test_bitrates.gif)
(http://kotisivu.mtv3.fi/alexb/ha/128test_bitrates_gr.gif)
Getting the iTunes bitrates was tedious. The encoder was quite slow and I couldn't find any application that could show the correct bitrates in one view. I ended up opening one file at a time in the QT player and checking the separate "Movie Info" window.
I tried also WMA standard at VBR quality 50 (44.1 kHz stereo), but it produced too low bitrates. The average bitrate was 105 kbps.
I couldn't find any application that could show the correct bitrates in one view.[a href="index.php?act=findpost&pid=343280"][{POST_SNAPBACK}][/a]
Foobar can do that quite easily (one of the very few uses I have for it )
Getting the iTunes bitrates was tedious. The encoder was quite slow and I couldn't find any application that could show the correct bitrates in one view. I ended up opening one file at a time in the QT player and checking the separate "Movie Info" window.
[a href="index.php?act=findpost&pid=343280"][{POST_SNAPBACK}][/a]
Tried Mr QuestionMan? It should show the correct bitrate.
Hmm - why don't we organise an extension test after the initial 128 kbps multiformat test? Choose three winners from the multiformat test, and conduct "extension" test with MPC anc WMAPro as the additional codecs?
[a href="index.php?act=findpost&pid=343195"][{POST_SNAPBACK}][/a]
It's not a bad idea. But it would make two consecutive multiformat listening tests at 128 kbps: would people be motivated to start a second test?
[...]
But it's probably too complicated. I'm not sure that listeners are motivated enough. One single test, 5 encoders and one anchor is way more simplier.
[a href="index.php?act=findpost&pid=343200"][{POST_SNAPBACK}][/a]
It sounds like a good idea to me. I'd sure participate in the second test.
I couldn't find any application that could show the correct bitrates in one view.[a href="index.php?act=findpost&pid=343280"][{POST_SNAPBACK}][/a]
Foobar can do that quite easily (one of the very few uses I have for it )
[a href="index.php?act=findpost&pid=343282"][{POST_SNAPBACK}][/a]
Foobar 0.8.3 doesn't understand new iTunes files at all. It shows 128 kbps for all files and cannot play them. Foobar 0.9 beta can read the changed format, but I don't have it installed right now.
I trusted the two decimal digit QT display and rounded the bitrates mathematically correctly.
I tried Mr QuestionMan now. It can show the bitrates, but some roundings are strange. For example, when QT shows 141.21 Mr Q displays 142.
EDIT
For some reason the foobar 0.8.3 playback problem disappeared and even the dynamic bitrate display works. However it can't show the average bitrate. It is still 128 kbps.
Regarding ATRAC, I think it's not such an important format. The only software that can encode to ATRAC is Sonic Stage AFAIK. The only portables supporting ATRAC are from Sony. Also, MDs use another ATRAC version if I am not mistaken, so it has no sense. It might only be interesting for people having a CD player that can also play ATRAC files.
[a href="index.php?act=findpost&pid=343084"][{POST_SNAPBACK}][/a]
These arguments don't make much sense if you're serious about testing portable and popular formats. Let's take a look at them one by one:
1. So what if the only software that can encode to ATRAC is SonicStage? Btw, there are several hardware encoders.
2. There are plenty of MD players from other manufacturers than Sony, so that statement is not only irrelevant but also false.
3. Of course the version of ATRAC that can be played on MD players should be used. In the same way that you choose WMA Standard rather than Pro because it's the one that can be played on portable devices. Simple.
4. Again, you forget all the MD players out there...
And as far as popularity is concerned, I'm confident the combination MD player + ATRAC is more popular than aoTuV Vorbis or Nero AAC on any other portable device.
MD is not popular AFAIk, i know 1 person that has it, and like 300 that has MP3 or WMA. i don't know in other regions.
edit: in fact Sony is halting production iirc.
If you want to estimate bitrates, I suggest you provide links and commandlines to be used for all the codecs.
I think I will be able to produce some results (for the small amount (11) of lossless images I have on my laptop).
MD is not popular AFAIk, i know 1 person that has it, and like 300 that has MP3 or WMA. i don't know in other regions.
[a href="index.php?act=findpost&pid=343331"][{POST_SNAPBACK}][/a]
It sure is different for different regions. For example it's much more popular in Japan than in Europe. Although MP3 players now sell better than MD players you can still find a considerable amount of MD devices in Japanese electronic stores.
I encoded the same test file set with the WMA Pro encoder at VBR quality 50 (24-bit, stereo)
Here's a table made by Mr QuestionMan:
25 test files / WMA Pro VBR q50 (http://kotisivu.mtv3.fi/alexb/ha/MrQAnalysis_WMAPro50.html)
With this format Mr Q. seems to show the correct bitrates. I compared the values with WMA Tag dumps made with J. River Media Center.
Here is an example of such a dump (The "CurrentBitrate" and "OptimalBitrate" lines show the exact average bitrate):
Track #24: Yello - Planet Dada:Audio Codec: Windows Media Audio 9.1 Professional
Format: VBR Quality 50, 44 kHz, 2 channel 24 bit 1-pass VBR
Protected: No
Seekable: Yes
Trusted: No
WMA Tag dump:
Signature_Name:
WMFSDKVersion: 10.00.00.3802
WMFSDKNeeded: 0.0.0.0000
Duration: 1901730000.0000000000000000
Bitrate: 151664.0000000000000000
Seekable: Yes
Stridable: No
Broadcast: No
Is_Protected: No
Is_Trusted: No
HasAudio: Yes
HasImage: No
HasScript: No
HasVideo: No
CurrentBitrate: 142517.0000000000000000
OptimalBitrate: 142517.0000000000000000
HasAttachedImages: No
Can_Skip_Backward: No
Can_Skip_Forward: No
FileSize: 3412243.0000000000000000
HasArbitraryDataStream: No
HasFileTransferStream: No
WM/ContainerFormat: 1.0000000000000000
IsVBR: Yes
ASFLeakyBucketPairs: <unknown data>
I calculated the bitrates also with Excel, the average was 133 kbps and the median was 132 kbps.
(...)
What I want to know is whether or not people agree that HE-AACv2 at 32 kbps is too good for a low anchor. If yes, I think I will fall back to Shine.
[a href="index.php?act=findpost&pid=343206"][{POST_SNAPBACK}][/a]
why not use HE-AAC at a ridiculously high bitrate instead? It'll make a great low anchor.
One other thing: Is foobar2k reliable when it comes to bitrates? If not, what program should one use?
These arguments don't make much sense if you're serious about testing portable and popular formats. Let's take a look at them one by one:
1. So what if the only software that can encode to ATRAC is SonicStage? Btw, there are several hardware encoders.
2. There are plenty of MD players from other manufacturers than Sony, so that statement is not only irrelevant but also false.
3. Of course the version of ATRAC that can be played on MD players should be used. In the same way that you choose WMA Standard rather than Pro because it's the one that can be played on portable devices. Simple.
4. Again, you forget all the MD players out there...
And as far as popularity is concerned, I'm confident the combination MD player + ATRAC is more popular than aoTuV Vorbis or Nero AAC on any other portable device.
[a href="index.php?act=findpost&pid=343312"][{POST_SNAPBACK}][/a]
1. Software: It's a big limitation which makes it unpopular if you ask me. Hardware: Yes, but if I am not mistaken, they use another ATRAC encoder (I remember reading that there is a difference between hardware and software encoders).
2. My fault, I was unclear. I was referring to non-MD units.
3. As I said already, as far as I know, the encoders used in MDs are different than the encoder of the current SonicStage software.
Edit: Isn't there a format difference between SP and LP modes in MDs (I mean, not only bitrate is different, but the format, too)?
Another problem is that we already have five competitors. Adding ATRAC to the list would mean adding an additional competitor or dumping the low anchor - both aren't such good ideas if you ask me.
If people really want an extension test, we could test ATRAC in the second one.
Regarding the test duration, I will think about it. The "problems" I was referring to are shopping, wives and children being disappointed that daddy spends more time with his headphones and PC than with them...
Ah, and don't forget one thing... I don't think it's a good idea to test WMA when you're drunk. Who knows, maybe it will win then.
One other thing: Is foobar2k reliable when it comes to bitrates? If not, what program should one use?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=343349")
Yes, but not for AAC (see [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38516]this thread[/url]). It was maybe corrected for 0.9 beta 11.
MrQuestionMan is more accurate for AAC/MP4.
(...)
What I want to know is whether or not people agree that HE-AACv2 at 32 kbps is too good for a low anchor. If yes, I think I will fall back to Shine.
[a href="index.php?act=findpost&pid=343206"][{POST_SNAPBACK}][/a]
why not use HE-AAC at a ridiculously high bitrate instead? It'll make a great low anchor.
[a href="index.php?act=findpost&pid=343345"][{POST_SNAPBACK}][/a]
The few Dire Straits and Pink Floyd samples I just encoded at HE-AAC 96 kbps sound way too good.
I encoded the same 25 various genre music tracks I used in my LAME 3.97b1 VBR bitrate test (http://www.hydrogenaudio.org/forums/index.php?showtopic=37011&view=findpost&p=328558).
[a href="index.php?act=findpost&pid=343280"][{POST_SNAPBACK}][/a]
Thank you VERY much for doing this!
It confirms without contestation that iTunes & LAME -V5 are highly comparable (same bitrate, same encoding mode). Same for Vorbis of course. That's luck! The table don't indicate the bitrate for Nero Digital, so I suppose that you don't have the encoder.
May I ask you to try
-V5 --vbr-new --athaa-sensitivity 1 (it requires 3.98 alpha), just to get a better idea of the impact on filesize with non-classical music?
I encoded the same 25 various genre music tracks I used in my LAME 3.97b1 VBR bitrate test (http://www.hydrogenaudio.org/forums/index.php?showtopic=37011&view=findpost&p=328558).
[a href="index.php?act=findpost&pid=343280"][{POST_SNAPBACK}][/a]
Thank you VERY much for doing this!
It confirms without contestation that iTunes & LAME -V5 are highly comparable (same bitrate, same encoding mode). Same for Vorbis of course. That's luck! The table don't indicate the bitrate for Nero Digital, so I suppose that you don't have the encoder.
May I ask you to try -V5 --vbr-new --athaa-sensitivity 1 (it requires 3.98 alpha), just to get a better idea of the impact on filesize with non-classical music?
[a href="index.php?act=findpost&pid=343381"][{POST_SNAPBACK}][/a]
I am working on Nero Digital. I downloaded Nero 7 and I am experimenting with the options.
I can try the athaa sensitivy switch too.
Having read this thread I'd go with these
1. Nero AAC
2. iTunes AAC
3. Lame MP3
4. WMA std
5. Vorbis Aotuv
6. Super ultra crappy anchor
[a href="index.php?act=findpost&pid=343087"][{POST_SNAPBACK}][/a]
After having read this very lengthy tread I have to agree with the above selection of encoders.
OK, I am glad you agree with the current selection of codecs.
Regarding MPC, ATRAC and WMA Pro, they can be tested in an extension test if you really want it and if there are enough testers, although it makes no sense IMO (because of the reasons already mentioned).
Does anyone have an idea what Nero developers recommend?
Nero has a nice set of useful pdf manuals available, but AAC options are not explained in detail. (... or at least I couldn't find the right manual/chapter.)
Garf? Ivan?
Does anyone have an idea what Nero developers recommend?
Nero has a nice set of useful pdf manuals available, but AAC options are not explained in detail. (... or at least I couldn't find the right manual/chapter.)
Garf? Ivan?
[a href="index.php?act=findpost&pid=343400"][{POST_SNAPBACK}][/a]
I already received a PM regarding Nero and I still think it's my business to take care of that.
The settings will be published after the updated encoder is going to be released.
BTW, thanks for the bitrate table.
And as far as popularity is concerned, I'm confident the combination MD player + ATRAC is more popular than aoTuV Vorbis or Nero AAC on any other portable device.
[a href="index.php?act=findpost&pid=343312"][{POST_SNAPBACK}][/a]
I also agree with you. That's why I considered as "ambiguous" the choice of
popularity as criterion for this test. It becomes easy to object the choice of
aoTuV as vorbis encoder; LAME is surely popular but
3.97b1 surely less (Winamp devs are waiting for a final version for example and are keeping in the meantime 3.96.1) and
-V5 --vbr-new probably not.
But I can see different reasons to discard atrac3/atrac3plus from this test. Dibrom already gives some of them, and I can add:
- no VBR mode (it may be a serious cause of troubles if all 4 or 5 other contenders are using VBR and atrac not).
- very restricted usage (DRM can't be disabled, making atrac3 encodings by far less friendly than even WMA/WMAPro)
[- secondary reason: inferior bitrate. Atrac3
plus now offers 128 kbps and not 132 kbps (it was the case for atrac3); I can hear from now and dispite my nasty cold the outcry]
- it ends last during the previous multiformat listening test, and even if progress were introduced in recent encoders, it's unlikely that atrac3plus is now competitive with modern implementation of AAC, Vorbis or MP3.
- atrac3plus has much less support than atrac3 (the latter is supported by all MD released in the five last years, not atrac3plus),
and if popularity is the criterion then atrac3 should logically by selected over the new atrac3plus, even if the latter is supposed to be better. And if we do select atrac3 over atrac3plus, I let you imagine the circus:
and why the hell on HA.org are always bashing Sony products and starting biased listening tests proving nothing because my player has 1-bit digit amp and sound much better than even CD with more bass than MP3 192 especially if the hardware encoder is used to encode CD because the hardware ship emebedd the last generation of audiophile encoder also present in Sony ES products and I fuck you all I'm an audiophile...
atrac3/atrac3plus are surely popular, but I don't think that users are using this format outside a direct association with a MD or another compatible device. All other formats are sometimes less popular but are at least usable for different purpose.
Also, and others might see this differently, this test is for me a display of the current state of the codecs, at this time.
So, after I look at the results, I want to know what encoder gives me the highest quality when I encode my CD to 128kbps. This is not a listening test oriented at the developers to help them tweak the encoder.
This test is for the users, to see which codec performs best. For example: Joe Average ripped his CDs with EAC and LAME, John Doe ripped his in iTunes. After they look at this test, they want to know whose encodings are better.
Using special tweakings to encode the samples could be considered cheating from that point of view.
BTW, Gabriel explained why changes can't be immediate for LAME. This encoder is widely used by end-users, commercial softwares and the output files are supposed to be readable on more than 10.000 portable, DVD, and video games players. They can't take the risk to partially break the compatibility by changes that marginally affect the quality.
What's ironic is that --vbr-new plays better with iPods that have the stuttering issues, so we all know at least V2 --vbr-new will be an improvement on 80% or so of all mp3 players out there.. The message has always been to stick with the presets to get the best possible quality, so if that's changing for 3.97 final, then it's really confusing for the end user (as if LAME wasn't abstruse enough before). As Dibrom pointed out, "new" & "old" doesn't say squat to a user in terms of how it affects their encodings as well.
AFAICS it weren't exactly HA recommendations for LAME that were used in Roberto's multiformat listening test. I don't know why this is such an issue now. Using the old HA recommendations LAME couldn't even be included in this test, because there doesn't seem to be a preset fitted for it. Using the developers recommendation is always the best thing to do in such a test.
[slightly OT]IMO the HA recommendations and presets in the form 3.90.3 were possibly the worst thing that ever happened to LAME. For the user the presets lacked granularity (which is now reasonably fixed with the V switch) and the analness with which these presets were defended made it very hard for developers to make any kind of progress.[/slightly OT]
Another thing about MDs and ATRAC. If I am not mistaken, only MD-LP2 comes close to 128 kbps. MD-LP4 uses something around 64 kbps and MD-SP uses ~300 kbps. Also, HW MD encoders usually produce files with inferior quality than SW encoders because they have no floating point unit, the CPU and memory are limited... So the only reason why we might want to test ATRAC3 is because of Sony CD players that also support ATRAC3.
vorbis is important because: 1) is the only patent free encoder and 2) is the other only codedc, besides MP3, AAC and WMA, that is being push by many comercial comapnies. MD is being fased out even by Sony.
[a href="index.php?act=findpost&pid=343468"][{POST_SNAPBACK}][/a]
I refered to aoTuV beta 4 as selected encoder, probably not very popular compared to the current CVS one. Not to Vorbis itself
I refered to aoTuV beta 4 as selected encoder, probably not very popular compared to the current CVS one. Not to Vorbis itself
oh, sorry, didn't catch that part. ... anyway, it would probably become vorbis' next version.
AFAICS it weren't exactly HA recommendations for LAME that were used in Roberto's multiformat listening test.
[a href="index.php?act=findpost&pid=343437"][{POST_SNAPBACK}][/a]
This is not exactly correct. HA.org "recommendations" (3.90.3 abr) were followed for the
first multiformat listenening test: MP3 failed badly, and if you read on comments posted after this test, it became "obvious" that MP3 was outdated.
But for the
second multiformat listening test, HA.org recommendations (which didn't changed) were not followed, and MP3 (3.96 vbr) suddenly appeared as a very competitive format, on par with iTunes AAC. It also proved to everyone that VBR is perfectly suitable for 130 kbps encoding with LAME, fact that was never admitted on this board before.
Moral of the story: don't invoke HA.org recommendations as reference document as long as LAME quality is concerned.
Ok... I have done my first bitrate test for Vorbis aoTuV beta 4.5 at -q 4
I will be using these samples in all my bitrate tests.
bol Silver Sun 1 120
bol Silver Sun 2 111
bol Silver Sun 3 122
bol Silver Sun 4 119
bol Silver Sun 5 134
bol Silver Sun 6 126
bol Silver Sun 7 116
bol Silver Sun 8 118
bol Silver Sun 9 108
Dream Theater Images and Words 1 129
Dream Theater Images and Words 2 132
Dream Theater Images and Words 3 130
Dream Theater Images and Words 4 127
Dream Theater Images and Words 5 133
Dream Theater Images and Words 6 131
Dream Theater Images and Words 7 117
Dream Theater Images and Words 8 132
Dream Theater Scenes from a Memory 1 130
Dream Theater Scenes from a Memory 2 133
Dream Theater Scenes from a Memory 3 130
Dream Theater Scenes from a Memory 4 119
Dream Theater Scenes from a Memory 5 129
Dream Theater Scenes from a Memory 6 133
Dream Theater Scenes from a Memory 7 125
Dream Theater Scenes from a Memory 8 133
Dream Theater Scenes from a Memory 9 134
Dream Theater Scenes from a Memory 10 130
Dream Theater Scenes from a Memory 11 132
Dream Theater Scenes from a Memory 12 126
Howard Shore Fellowship of the Ring 1 114
Howard Shore Fellowship of the Ring 2 122
Howard Shore Fellowship of the Ring 3 120
Howard Shore Fellowship of the Ring 4 117
Howard Shore Fellowship of the Ring 5 119
Howard Shore Fellowship of the Ring 6 116
Howard Shore Fellowship of the Ring 7 120
Howard Shore Fellowship of the Ring 8 117
Howard Shore Fellowship of the Ring 9 115
Howard Shore Fellowship of the Ring 10 118
Howard Shore Fellowship of the Ring 11 120
Howard Shore Fellowship of the Ring 12 113
Howard Shore Fellowship of the Ring 13 122
Howard Shore Fellowship of the Ring 14 119
Howard Shore Fellowship of the Ring 15 116
Howard Shore Fellowship of the Ring 16 118
Howard Shore Fellowship of the Ring 17 118
Howard Shore Fellowship of the Ring 18 117
Howard Shore The Two Towers 1 126
Howard Shore The Two Towers 2 115
Howard Shore The Two Towers 3 122
Howard Shore The Two Towers 4 113
Howard Shore The Two Towers 5 120
Howard Shore The Two Towers 6 115
Howard Shore The Two Towers 7 123
Howard Shore The Two Towers 8 112
Howard Shore The Two Towers 9 118
Howard Shore The Two Towers 10 108
Howard Shore The Two Towers 11 112
Howard Shore The Two Towers 12 126
Howard Shore The Two Towers 13 117
Howard Shore The Two Towers 14 116
Howard Shore The Two Towers 15 117
Howard Shore The Two Towers 16 127
Howard Shore The Two Towers 17 123
Howard Shore The Two Towers 18 124
Howard Shore The Two Towers 19 121
Howard Shore The Return of the King 1 118
Howard Shore The Return of the King 2 118
Howard Shore The Return of the King 3 126
Howard Shore The Return of the King 4 125
Howard Shore The Return of the King 5 115
Howard Shore The Return of the King 6 128
Howard Shore The Return of the King 7 121
Howard Shore The Return of the King 8 114
Howard Shore The Return of the King 9 116
Howard Shore The Return of the King 10 121
Howard Shore The Return of the King 11 130
Howard Shore The Return of the King 12 130
Howard Shore The Return of the King 13 122
Howard Shore The Return of the King 14 116
Howard Shore The Return of the King 15 120
Howard Shore The Return of the King 16 117
Howard Shore The Return of the King 17 116
Howard Shore The Return of the King 18 113
Howard Shore The Return of the King 19 117
Schtimm Featuring... 1 107
Schtimm Featuring... 2 117
Schtimm Featuring... 3 123
Schtimm Featuring... 4 113
Schtimm Featuring... 5 132
Schtimm Featuring... 6 114
Schtimm Featuring... 7 93
Schtimm Featuring... 8 120
Schtimm Featuring... 9 118
Schtimm Featuring... 10 126
Schtimm Featuring... 11 140
Schtimm Featuring... 12 120
Schtimm plays Mrakoslav Vragosh 1 120
Schtimm plays Mrakoslav Vragosh 2 119
Schtimm plays Mrakoslav Vragosh 3 109
Schtimm plays Mrakoslav Vragosh 4 136
Schtimm plays Mrakoslav Vragosh 5 125
Schtimm plays Mrakoslav Vragosh 6 120
Schtimm plays Mrakoslav Vragosh 7 145
Schtimm plays Mrakoslav Vragosh 8 126
Schtimm plays Mrakoslav Vragosh 9 125
Schtimm plays Mrakoslav Vragosh 10 122
Schtimm plays Mrakoslav Vragosh 11 123
Schtimm The Alcoholovefi Collection 1 120
Schtimm The Alcoholovefi Collection 2 125
Schtimm The Alcoholovefi Collection 3 128
Schtimm The Alcoholovefi Collection 4 134
Schtimm The Alcoholovefi Collection 5 138
Schtimm The Alcoholovefi Collection 6 128
Schtimm The Alcoholovefi Collection 7 138
Schtimm The Alcoholovefi Collection 8 116
Schtimm The Alcoholovefi Collection 9 128
Schtimm The Alcoholovefi Collection 10 124
VA That's Jazz CD 1 1 133
VA That's Jazz CD 1 2 145
VA That's Jazz CD 1 3 125
VA That's Jazz CD 1 4 136
VA That's Jazz CD 1 5 132
VA That's Jazz CD 1 6 106
VA That's Jazz CD 1 7 97
VA That's Jazz CD 1 8 124
VA That's Jazz CD 1 9 138
VA That's Jazz CD 1 10 129
VA That's Jazz CD 1 11 120
VA That's Jazz CD 1 12 145
VA That's Jazz CD 1 13 125
VA That's Jazz CD 1 14 117
VA That's Jazz CD 1 15 143
VA That's Jazz CD 1 16 121
VA That's Jazz CD 1 17 141
VA That's Jazz CD 2 1 137
VA That's Jazz CD 2 2 97
VA That's Jazz CD 2 3 119
VA That's Jazz CD 2 4 127
VA That's Jazz CD 2 5 111
VA That's Jazz CD 2 6 133
VA That's Jazz CD 2 7 125
VA That's Jazz CD 2 8 101
VA That's Jazz CD 2 9 119
VA That's Jazz CD 2 10 122
VA That's Jazz CD 2 11 111
VA That's Jazz CD 2 12 89
VA That's Jazz CD 2 13 90
VA That's Jazz CD 2 14 75
VA That's Jazz CD 2 15 79
VA That's Jazz CD 2 16 134
VA That's Jazz CD 2 17 128
VA That's Jazz CD 2 18 121
VA That's Jazz CD 2 19 86
VA That's Jazz CD 2 20 95
VA That's Jazz CD 2 21 119
VA That's Jazz CD 2 22 110
VA That's Jazz CD 2 23 109
bol - Silver Sun: Experimental Jazz
Dream Theater: Progressive rock/metal
Schtimm: Jazz with some rock elements ans some balads
LotR soundtrack: classical
That's Jazz: Jazz
Results for Vorbis q 4:
Mean: 120.9 kbps
Median 120.5 kbps
Standard Deviation: 11.5 kbps
Sample Range: 70 kbps
Mean Deviation: 8.2 kbps
Quartile Deviation: 6.0 kbps
Quartiles: 1:116.0, 2: 120.5, 3: 128.0
Student-t 0.95 confidence interval [119.1, 122.7]
Student-t 0.99 confidence interval [118.5, 123.3]
Student-t two-sided hypothesis test vs. 128 kbps:
P-value: 1.30*10^(-12)
Preliminary conclusion: q 4 is not appropriate.
I have also encoded some samples, all from the original wave files with:
1) Itunes 6 (QT 7.0.3), mode: 128 Kbits VBR
2) Nero 7, aacenc 4.2.1.0
3) lame -V5 --vbr-new
4) vorbis q4.7
5) wma @ Q50
I encoded 20 songs from varying styles of music. Conversion was done with foobar (for lame), dbpoweramp (for itunes, nero and wma) and oggdropxpd (for vorbis).
(http://www.leminator.org/pics/bitratetest.png)
(the row at the bottom is the average bitrate for a codec over all used songs)
My results for wma seem to differ quite a bit from Alex b's results. Here, while using the same preset (Q50), the average bitrate is only 98 Kbits/sec. (it seems unfair to compare wma with this preset against the other samples which average 134 Kbits/sec. With wma included the average for all samples drops to 127 Kbits/sec. Unfortunately, if you go to Q75 the bitrate is much too high...)
It seems to me that for WMA, a constant bitrate will be needed to give a view which is more fair when compared with the other codecs...
Thanks, but please, use [ CODEBOX ] [ /CODEBOX ] (without space) instead of [ CODE ] [ /CODE ].
- done -
my post could be deleted
Thanks for the tests guys!
As regards --athaa-sensitivity 1 (a1) with LAME, wouldn't it be the most pragmatical solution to include it if the average bitrate still fits within the range of the other contenders and not to include it if the average bitrate gets too high?
AFAIK, it is uncontested that a1 improves the quality, so it would be foolish not to include it if the bitrate fits just because it is not considered to be an "elegant" solution to the problems LAME has at this bitrate - it's the only solution we have at the moment, so it should be used. If LAME is to be tested at its full potential, it definitely has to be included IMHO.
As regards --athaa-sensitivity 1
Right now, this switch
IS NOT AVAILABLE with 3.97b.
As regards --athaa-sensitivity 1
Right now, this switch IS NOT AVAILABLE with 3.97b.
[a href="index.php?act=findpost&pid=343496"][{POST_SNAPBACK}][/a]
Is it available in 3.98a2?
Yes, but I would not recommend using an alpha version for a public listening test.
Yes, but I would not recommend using an alpha version for a public listening test.
[a href="index.php?act=findpost&pid=343499"][{POST_SNAPBACK}][/a]
Not even 3.97a12?
sTisTi> you just expressed my innermost thoughts
Even if --athaa-sensitivity 1 is not the right answer to current quality issues audible with -V5 preset, it's at least a temporary fix, which works pretty well, and which is available for the end users by using 3.98 alpha - available on rarewares.
I wouldn't consider it as cheating. There is also a precedent: Roberto's 32 kbps listenening test used an unofficial build of Nero Digital providing Parametric Stereo support 18 months before the official release, and nobody complained about cheating.
I also listed what command lines or encoders which were used in previous tests and which weren't recommended at this time:
64kbps public listening test: (http://www.rjamorim.com/test/64test/presentation.html)
• Lame MP3 encoder 3.90.3 --alt-preset 128 --scale 1 => --scale 1 was recommended nowhere
Multiformat at 128kbps public listening test (http://www.rjamorim.com/test/multiformat128/presentation.html)
• LAME encoder 3.96 -V5 --athaa-sensitivity 1 => 3.96, V5 and --athaa... were not recommended.
32kbps public listening test (http://www.rjamorim.com/test/32kbps/presentation.html)
• Ahead HE AAC+PS 32kbps CBR High Quality => unofficial build, technological demonstration (by the way amazing, if we consider the first place of this preliminary encoder) => not accessible to most users, temporary available as link on HA.org and released for the purpose of the test only.
• Ogg Vorbis post-1.0.1CVS --managed -b 32 resampled with SSRC => use of SSRC instead of default resampler
If I perfectly understand why Sebastian prefers follow developers recommendations, I can't honestly understand some comments about cheating or the full controversy about what recommended is and what should be tested.
As regards --athaa-sensitivity 1
Right now, this switch IS NOT AVAILABLE with 3.97b.
[a href="index.php?act=findpost&pid=343496"][{POST_SNAPBACK}][/a]
You still have 10 days to introduce it
(I'm joking)
What I don't understand is that Gabriel recommends to use --vbr-new, but says that --athaa-sensivita 1 should not be used by the user, but should be something that the encoder handles; shouldn't the encoder handle --vbr-new, too?
That's why I think if --athaa-sensivity 1 provides better results, it should be used.
Correct me if I am wrong since I am very tired ATM.
My results for wma seem to differ quite a bit from Alex b's results. Here, while using the same preset (Q50), the average bitrate is only 98 Kbits/sec.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=343483")
How did you computed the bitrate? Manually or by using a dedicated software? I noticed issues with foobar2000 and MrQuestionMan, [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38955&view=findpost&p=343531]here[/url].
What I don't understand is that Gabriel recommends to use --vbr-new, but says that --athaa-sensivita 1 should not be used by the user, but should be something that the encoder handles; shouldn't the encoder handle --vbr-new, too?
[a href="index.php?act=findpost&pid=343512"][{POST_SNAPBACK}][/a]
Both commands are different.
--athaa-sensitivity is one of the numerous switch that directly affect quality; the switch is not even documented, and is also removed from the latest beta release. I suppose that this command is considered as something that shouldn't be tweaked by end-users. The current presets of LAME are supposed to use in the most efficiently way commands like --athaa-sensitivity 1, --ns-bass, etc... If they don't, they must be changed.
The remaining question is: why --athaa-sensitivity 1 is not the defaulted one if it increases the quality (not only for classical music: it concerns DVDrip first and foremost as well)? Gabriel told me that --athaa-sensitivity tool was not intended to correct such problems. A better solution should be found. Using this command will (partially) solve and mask issues that should be corrected elsewhere. That's why he's opposed to make --athaa-sensitivity 1 as default. [Gabriel, or Robert, beat me if I'm wrong].
Personaly, I would regret to start a test with -V5 when one known issue is not corrected AND when a temporary fix is also available. We couldn't wait for the next LAME release, but we already could use LAME with an improved quality. I don't know how many samples will benefit from --athaa-sensitivity 1 in the upcoming test. Maybe none...
I wouldn't worry too much if such tests were periodically organized. But as we don't know how many time we have to wait for the next one, I feel that it's very important to use for each contender the best available setting.
Listening tests precede popularity and sometimes real life usage, not the opposite.
But you don't equally apply this to format decision (i.e. in the case of WMA Pro inclusion), do you?
Just playing the devil's advocate here.
Also, HW MD encoders usually produce files with inferior quality than SW encoders because they have no floating point unit, the CPU and memory are limited... [a href="index.php?act=findpost&pid=343469"][{POST_SNAPBACK}][/a]
Have you seen any listening tests regarding this that I haven't? Otherwise how can you claim that?
I believe that current popularity and compatibility shouldn't be considered in this test. Listenings tests should tell us which encoders/formats are the best a which are not good. So I would like to see in this test following formats:
AAC LC (not sure which encoders, iTunes, Nero or WinAMP?), MP3 (LAME with all tweaks), WMA Std, WMA Pro, ATRAC (I'm not sure which version is best for this bitrate), MPC (Can it perform well at such a low bitrate?) and of course OGG Vorbis.
Vlada
So I would like to see in this test following formats:
AAC LC (not sure which encoders, iTunes, Nero or WinAMP?), MP3 (LAME with all tweaks), WMA Std, WMA Pro, ATRAC (I'm not sure which version is best for this bitrate), MPC (Can it perform well at such a low bitrate?) and of course OGG Vorbis.
[a href="index.php?act=findpost&pid=343597"][{POST_SNAPBACK}][/a]
I'd like to see them all, but choices must be done. And to make them, criterion such as popularity or hardware support are needed.
I can see how gurubool could be frustrated, having put in 100s of hours almost single-handedly documenting the performance of Lame alphas,
not get his best recommendation of a good and safe tweak ,'defaulted' -because the hack is just unrecommened by the coder/s,
even though all gurus hard earned data suggests its beneficial.
What must go through the mind of the tester, spending hours chugging through sample after sample after sample? It would seem to me a tiresome meditation,
<shouts from sideline>Come on coders, default the mans tweak wontcha
<runs away...>
One other thing: Is foobar2k reliable when it comes to bitrates? If not, what program should one use?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=343349")
Yes, but not for AAC (see [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38516]this thread[/url]). It was maybe corrected for 0.9 beta 11.
MrQuestionMan is more accurate for AAC/MP4.
[a href="index.php?act=findpost&pid=343367"][{POST_SNAPBACK}][/a]
foobar2000 beta 12 is now computing correctly the corresponding bitrate of iTunes VBR encodings. Thanks to Peter
I as well agree with gambit and peter....... and would like to add something which i mentioned before in other threads:
Every option and feature is to some extend a *flaw*, because it adds complexity. Tools like applications are there to simplify tasks, to make them more efficient. Complexity runs contrary to this goal.
This doesn't mean, that an application with no options is automatically better than another app with more options. If the advantages and "gain" from implementing an option/feature outweights the added complexity, then its justified.
In regards to encoders, i think the following sums it up quite nicely: the job of the user is to tell the encoder *what* he wants, NOT *how* it should be done. The job of the encoder is to deliver what the user wants, and automatically choose the optimal method *how* to achieve it. Just as with the taxi-driver, the more guides the encoder needs on the "how" the less usable and more flawed it is.
One more thing: from a social-design POV, added complexity suggests to the user that it is NECESSARY to tweak those options to achieve optimal results - because else, they wouldn't be there, right? Thus, added complexity provokes malicious tweaking and user-errors.
As for those ultra-knowledgeable-users: they shouldn't be users - they should be testers and developers. This in turn means that the stock releases aren't meant for them anyways.
- Lyx
The debate around the LAME setting is legitimate (that's precisely the purpose of this thread) but some arguments are senseless in my opinion. Is it safe to use a special switch? Should we use something which is not present in latest version? [a href="index.php?act=findpost&pid=343618"][{POST_SNAPBACK}][/a]
While I see no problem in using any available switches, I feel that if it is known beforehand that they will not be available in betas or releases, they should not be used.
I also see a lot of arguments in this thread which would apply to release versions of an encoder. With alphas or betas developers have carte blanche, imo.
What do you guys want to test?
Do you want to test how good the encoders perform for average-joe in everyday-usage?
Do you want to test how good a PROTOTYPE (using experimental switches) performs for HA.ORG *developers*?
No matter in which utopia you folks live, those two are NOT the same.
In case you want to test the former scenario, then only the defaults should be used with no additional switches. Using additional switches IS developing... no matter if you change the source, or experimental switches... its essentially the same - think otherwise? then consider the following:
Imagine an encoder, which is coded in a programming language which is compiled at runtime. Thus, no compilation of the code is necessary.... similiar to a script. Now, what is the difference between changing a few numbers in the source, and making use of experimental switches? Where does usage end, and where does development begin? I'll jump right to the answer: at the moment where you begin to tell the encoder "how" it shall do it, you are developing the encoder! You're doing the kind of work which is the job of a developer, not a user.
- Lyx
In case you want to test the former scenario, then only the defaults should be used with no additional switches.
[a href="index.php?act=findpost&pid=343733"][{POST_SNAPBACK}][/a]
So, in your opinion, we should use LAME with -b 128.
Vorbis should be used at something producing ~128 kbps.
WMA 1-pass CBR at 128 kbps.
iTunes CBR 128 kbps.
And finally Nero at whatever comes close to 128 kbps.
Then, when all codecs are pretty much tied, get bombarded by Garf that the test is not meaningful since I used suboptimal settings, get bombarded by Gabriel (sorry, I know you won't do it ) that the test is useless since 1: I used suboptimal settings and 2: the test didn't help him at all because he was interested in something totally different like how the new VBR engine performs, etc.
Way to go! Show me one listening test that meets your criteria.
In case you want to test the former scenario, then only the defaults should be used with no additional switches.
[a href="index.php?act=findpost&pid=343733"][{POST_SNAPBACK}][/a]
So, in your opinion, we should use LAME with -b 128.
While i worded a single word the wrong way ("additional" instead of "experimental") in the above quote, if you would have read my post completely as well as my previous one, then you could understood the difference between telling the encoder the "how" or "what".
- Lyx
You take a cab and tell the driver the address. You expect him to take you there, without asking you for directions. A bad driver will not know the way and you will have to guide him. This driver should probably look for another job. A good driver will take you there, but maybe you will be able to point out a shortcut which will take you there faster. The perfect driver will take you there without any help and he will know all the shortcuts already.
In regards to encoders, i think the following sums it up quite nicely: the job of the user is to tell the encoder *what* he wants, NOT *how* it should be done. The job of the encoder is to deliver what the user wants, and automatically choose the optimal method *how* to achieve it. Just as with the taxi-driver, the more guides the encoder needs on the "how" the less usable and more flawed it is.
at the moment where you begin to tell the encoder "how" it shall do it, you are developing the encoder! You're doing the kind of work which is the job of a developer, not a user.
Do you want to test how good a PROTOTYPE (using experimental switches) performs for HA.ORG *developers*?
[a href="index.php?act=findpost&pid=343733"][{POST_SNAPBACK}][/a]
I doubt that any developer would get informative results with the upcoming test. To test the impact of additional switchs, we should at least compare it to the standard commandline (without switch). And this won't be the case.
Using additional switches IS developing...
No, it's simple usage. Developing implies development: in other words movement, progress, experimentation - something active and thoughtful in the same time. Usage is at the opposite: something static, without any questioning behind.
I am wondering (just out of curiosity)... How come nobody complained that Roberto used all kinds of "funky" switches during his tests?
Examples:
LAME --alt-preset 128 --scale 1 in 64 kbps listening test
Adobe Audtion FhG MP3 64kbps CBR, Current codec, allow M/S, no I/S, allow narrowing, no CRC in 64 kbps listening test
MPC --quality 4 --xlevel in 128 kbps extension listening test
LAME -V5 --athaa-sensitivity 1 in multiformat 128 kbps listening test
...
I think the discussion about LAME is leading to nowhere.
I will use what Gabriel recommended, and this is -V5 --vbr-new.
If you want to discuss about why --vbr-new is not default whatsoever, please use the splitted thread.
Do you want to test how good a PROTOTYPE (using experimental switches) performs for HA.ORG *developers*?
[a href="index.php?act=findpost&pid=343733"][{POST_SNAPBACK}][/a]
I doubt that any developer would get informative results with the upcoming test. To test the impact of additional switchs, we should at least compare it to the standard commandline (without switch). And this won't be the case.
Correct. I didn't say that one is good and the other one is bad. I just said that the two are not the same and that you need to be aware and decide what this test is meant for: Do you want to betatest to help development(in this case the results will NOT represent real-world usage), or do you want to determine real-world performance for users(in this case, the results will be less useful for developers, because WIP-tech will not be covered).
Using additional switches IS developing...
No, it's simple usage.
When you tell the taxi-driver how to get to the target, then are you doing the taxi-drivers job?
Developing implies development: in other words movement, progress, experimentation - something active and thoughtful in the same time.
What is the difference between a developer testing out the effect of experimental switches and coming up with something which works best(and which then should be defaulted) - and a user doing the very same thing? The only difference is that the former is official, while the latter is inofficial(and therefore doesn't get applied to the main distribution).
- Lyx
I am wondering (just out of curiosity)... How come nobody complained that Roberto used all kinds of "funky" switches during his tests?
Because back then either people were not aware of this inconsistency, or the few that were didn't get enough support from regulars.
When you tell the taxi-driver how to get to the target, then are you doing the taxi-drivers job?
[a href="index.php?act=findpost&pid=343757"][{POST_SNAPBACK}][/a]
Are you the one who's driving? Are you responsible for the traffic etc.?
As I already said... I am not looking for popular settings, but popular formats (MP3, AAC, Vorbis, WMA) at their BEST settings.
What is the problem?
Using additional switches IS developing...
No, it's simple usage.
When you tell the taxi-driver how to get to the target, then are you doing the taxi-drivers job?
[{POST_SNAPBACK}][/a]
(http://index.php?act=findpost&pid=343757")
I'm suggesting, he's driving.
Sebastian: what about the suggestion I made for the criterion of the test:
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38979&view=findpost&p=343624]http://www.hydrogenaudio.org/forums/index....ndpost&p=343624[/url]
- VBR
- compatible with hardware
- best available settings
Also, HW MD encoders usually produce files with inferior quality than SW encoders because they have no floating point unit, the CPU and memory are limited... [a href="index.php?act=findpost&pid=343469"][{POST_SNAPBACK}][/a]
Have you seen any listening tests regarding this that I haven't? Otherwise how can you claim that?
[a href="index.php?act=findpost&pid=343595"][{POST_SNAPBACK}][/a]
If I tie your hands and tell you to cook a 5 star menu, do you think you would be able to do as well as another guy who can use his hands?
Hardware encoders don't have a floating point unit. They have limited CPU and memory which can be seen as handicaps. Also, while software encoders can experience minor updates, hardware encoders can't if they are not firmware upgradeable.
And as I said already, only MD-LP2 comes close to the bitrate tested in this listening test.
Sebastian: what about the suggestion I made for the criterion of the test:
http://www.hydrogenaudio.org/forums/index....ndpost&p=343624 (http://www.hydrogenaudio.org/forums/index.php?showtopic=38979&view=findpost&p=343624)
[a href="index.php?act=findpost&pid=343761"][{POST_SNAPBACK}][/a]
Thanks God at least one person is understanding me!
As I already said... I am not looking for popular settings, but popular formats (MP3, AAC, Vorbis, WMA) at their BEST settings.
What is the problem?
[a href="index.php?act=findpost&pid=343760"][{POST_SNAPBACK}][/a]
Hehe, call it an identity-crisis of ha.org concerning large-scale listening-tests. In the past, we tested non-realworld usage... we essentially tested just for ourself - checking how good the various encoders are for ha.org members who know about optimal tweaks. Many however were not aware of this and confused this with the realworld-performance of those encoders. Now we're beginning to understand the differences and are asking ourself: "what do we test for?"
As I already said... I am not looking for popular settings, but popular formats (MP3, AAC, Vorbis, WMA) at their BEST settings.
What is the problem?
[a href="index.php?act=findpost&pid=343760"][{POST_SNAPBACK}][/a]
Hehe, call it an identity-crisis of ha.org concerning large-scale listening-tests. In the past, we tested non-realworld usage... we essentially tested just for ourself - checking how good the various encoders are for ha.org members who know about optimal tweaks. Many however were not aware of this and confused this with the realworld-performance of those encoders. Now we're beginning to understand the differences and are asking ourself: "what do we test for?"
[a href="index.php?act=findpost&pid=343766"][{POST_SNAPBACK}][/a]
And what stops John Doe from saying "Hmm... Seems that --vbr-new performs better. Maybe I should add it to my command line when encoding."?
And what stops John Doe from saying "Hmm... Seems that --vbr-new performs better. Maybe I should add it to my command line when encoding."?
[a href="index.php?act=findpost&pid=343767"][{POST_SNAPBACK}][/a]
Thats a hypothetical future-event. John Doe doesn't visit ha.org. You cannot test the future, only make asumptions.
It seems he does (http://www.hydrogenaudio.org/forums/index.php?showuser=11006).
It seems he does (http://www.hydrogenaudio.org/forums/index.php?showuser=11006).
[a href="index.php?act=findpost&pid=343771"][{POST_SNAPBACK}][/a]
I think that *finally*, all is setup.
What this pre-test thread has shown is that we are in need of several tests, not just one. So let's first do this one, and later, whatever we feel in need.
And what stops John Doe from saying "Hmm... Seems that --vbr-new performs better. Maybe I should add it to my command line when encoding."?
[a href="index.php?act=findpost&pid=343767"][{POST_SNAPBACK}][/a]
Thats a hypothetical future-event. John Doe doesn't visit ha.org. You cannot test the future, only make asumptions.
[a href="index.php?act=findpost&pid=343768"][{POST_SNAPBACK}][/a]
Well, I was pretty much a John Doe when I came to HA the first time. I was encoding my audio files from CD using a 486 at 33 MHz and was using the Windows audio recorder for that. After spending 15 minutes for a 5 minutes track, I used Blade to encode the file to MP3.
If everyone would think like you, there wouldn't be any websites at all. Darin would've said "no need to create HA, noone will ever visit it".
And what stops John Doe from saying "Hmm... Seems that --vbr-new performs better. Maybe I should add it to my command line when encoding."?
[a href="index.php?act=findpost&pid=343767"][{POST_SNAPBACK}][/a]
Thats a hypothetical future-event. John Doe doesn't visit ha.org. You cannot test the future, only make asumptions.
[a href="index.php?act=findpost&pid=343768"][{POST_SNAPBACK}][/a]
A few John Doe's have become HA regular visitors over time (me, for example), maybe looking for Easy CD-DA Extractor best practices and then switched to EAC after browsing a few threads.
EDIT: Grammar.
If everyone would think like you, there wouldn't be any websites at all. Darin would've said "no need to create HA, noone will ever visit it".
[a href="index.php?act=findpost&pid=343777"][{POST_SNAPBACK}][/a]
You are confusing determination of how things are, with having an ideological intention. You cannot test dreams, only reality. Reality is that ha.org does not have millions of members, so it represents just a tiny fraction of the userbase of said codecs.
Let's face it, ha.org was not created and does not exist to be the home of MILLIONS of users. It is an "audio technology enthusiast's resource" - does that sound like average-joe??? Sure, happenings on ha.org sometimes have a big impact on the digital audio world as a whole...... but that happens indirectly in most cases - like....... defaulting certain settings in the encoders after they were test-driven here ;-)
Hardware encoders don't have a floating point unit. They have limited CPU and memory which can be seen as handicaps. Also, while software encoders can experience minor updates, hardware encoders can't if they are not firmware upgradeable.
And as I said already, only MD-LP2 comes close to the bitrate tested in this listening test.
[a href="index.php?act=findpost&pid=343763"][{POST_SNAPBACK}][/a]
I hope you do realize that this argument doesn't hold much water. The performance of a lossy perceptual encoder does depend on the quality and degree of optimization of its psymodel and the assorted coding tools it has at its disposal. And
not on cpu speed, available memory or the availability of an fpu (even though those
may help achieve higher quality). For example, an encoder with a simple but well-tuned psymodel can very well be superior to one that makes use of complex but suboptimal psymodel.
Sorry about the OT, but I just had to address this. (And for the record, I refuse to stoop to flawed analogies.)
As for the debate at hand, as far as I see it you're free to test anything you want as long as you make sure that you give a proper consistent motivation for your codec choices and ensure that you test all codecs under the same premises so as to not compare the infamous apples and oranges.
Good luck with your test, I hope you won't allow the heated discussion to discourage you from going through with it.
Hehe, call it an identity-crisis of ha.org concerning large-scale listening-tests. In the past, we tested non-realworld usage... we essentially tested just for ourself - checking how good the various encoders are for ha.org members who know about optimal tweaks. Many however were not aware of this and confused this with the realworld-performance of those encoders. Now we're beginning to understand the differences and are asking ourself: "what do we test for?"
[a href="index.php?act=findpost&pid=343766"][{POST_SNAPBACK}][/a]
We can only test for ourselves, never for John Doe. John Doe won't visit HA, and John Doe won't read the listening test results. But if he does anyway, he can also read the settings used.
If you want to test CBR 128 kbps, go ahead, but such a test has no meaning. It would be disregarding any kind of development ever made on a codec. Any developer will ignore the test or shout that is performed in a bad way, and so will any user who has gotten out of the "real-usage" stage of audio encoding.
It makes no sense to conduct a test for people who have no interest in it whatsoever and probably aren't even aware the test exists. Just a waste of effort.
My results for wma seem to differ quite a bit from Alex b's results. Here, while using the same preset (Q50), the average bitrate is only 98 Kbits/sec.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=343483")
How did you computed the bitrate? Manually or by using a dedicated software? I noticed issues with foobar2000 and MrQuestionMan, [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38955&view=findpost&p=343531]here[/url].
[a href="index.php?act=findpost&pid=343534"][{POST_SNAPBACK}][/a]
Manually, with some minor aid from excel
Sorry, but what's the difference between mean and average? I always thought they would be the same thing.
Edit: Is mean highest value + lowest value / 2 and avarage sum of all values / number or items?
Sorry, but what's the difference between mean and average? I always thought they would be the same thing.
Edit: Is mean highest value + lowest value / 2 and avarage sum of all values / number or items?
[a href="index.php?act=findpost&pid=343914"][{POST_SNAPBACK}][/a]
Mean is the arithemetic average - sum / number of items. I don't believe highest+lowest / 2 is a commonly used statistic. Popular measures of central tendency include median (middle value in an ordered series) and mode (most common value).
Sorry, but what's the difference between mean and average? I always thought they would be the same thing.
Edit: Is mean highest value + lowest value / 2 and avarage sum of all values / number or items?
[a href="index.php?act=findpost&pid=343914"][{POST_SNAPBACK}][/a]
Mean is the arithemetic average - sum / number of items. I don't believe highest+lowest / 2 is a commonly used statistic. Popular measures of central tendency include median (middle value in an ordered series) and mode (most common value).
[a href="index.php?act=findpost&pid=343917"][{POST_SNAPBACK}][/a]
Ah, right. Max + Min / 2 is midrange.
In case you want to test the former scenario, then only the defaults should be used with no additional switches.
[a href="index.php?act=findpost&pid=343733"][{POST_SNAPBACK}][/a]
So, in your opinion, we should use LAME with -b 128.
Vorbis should be used at something producing ~128 kbps.
WMA 1-pass CBR at 128 kbps.
iTunes CBR 128 kbps.
And finally Nero at whatever comes close to 128 kbps.
[a href="index.php?act=findpost&pid=343742"][{POST_SNAPBACK}][/a]
When you tell the taxi-driver how to get to the target, then are you doing the taxi-drivers job?
[a href="index.php?act=findpost&pid=343757"][{POST_SNAPBACK}][/a]
Are you the one who's driving? Are you responsible for the traffic etc.?
[a href="index.php?act=findpost&pid=343758"][{POST_SNAPBACK}][/a]
You fail to understand the difference and the root of the problem.
When you tell the taxi-driver how to get to the target, then are you doing the taxi-drivers job?
[a href="index.php?act=findpost&pid=343757"][{POST_SNAPBACK}][/a]
Are you the one who's driving? Are you responsible for the traffic etc.?
[a href="index.php?act=findpost&pid=343758"][{POST_SNAPBACK}][/a]
You fail to understand the difference and the root of the problem.
[a href="index.php?act=findpost&pid=343936"][{POST_SNAPBACK}][/a]
The example was bad. A taxi driver's job is not only knowing the city - that's a guide's job for example. If you have to tell a taxi driver how to start the engine, how to push the brakes, how to change gears, tell him about road signs, then yes, he's a bad taxi driver.
Edit: By the way, if you ever used routing software, you would've noticed that it asks you if you want to use the fastest or the shortest road, it asks you if you want to use freeways, etc. IMO, --vbr-new is like telling a routing software that you want to use the fastest road and not the shortest.
Anyways, I don't want to start the shit again just because some of you are so narrow minded. As I stated multiple times, I am not looking at populat settings, but at popular formats at their best settings to see what the format itself is capabile of achieving. Discussion about LAME is over now since I don't see the point in continuing something that has no sense. If we are really talking about John Doe, there's no need for listening tests since John Doe is ripping his music using Windows Media Player at 128 kbps CBR with DRM (since that's the default in WMP if I am not mistaken). Probably, John Doe doesn't even know what VBR is and what it's good for.
If we are really talking about John Doe, there's no need for listening tests since John Doe is ripping his music using Windows Media Player at 128 kbps CBR with DRM (since that's the default in WMP if I am not mistaken). Probably, John Doe doesn't even know what VBR is and what it's good for.
[a href="index.php?act=findpost&pid=343950"][{POST_SNAPBACK}][/a]
I think you're underestimating poor John Doe
WMP has encoded to mp3 as default for over a year now.
If we are really talking about John Doe, there's no need for listening tests since John Doe is ripping his music using Windows Media Player at 128 kbps CBR with DRM (since that's the default in WMP if I am not mistaken). Probably, John Doe doesn't even know what VBR is and what it's good for.
[a href="index.php?act=findpost&pid=343950"][{POST_SNAPBACK}][/a]
I think you're underestimating poor John Doe
WMP has encoded to mp3 as default for over a year now.
[a href="index.php?act=findpost&pid=343951"][{POST_SNAPBACK}][/a]
Ah, sorry, didn't catch up with that. Well, he's using whatever WMP defaults to.
What about Jane Doe ? She's got to put something on her iPod Mini after all......
Now that the thread has been purged and to avoid unecessary debate, I think it would be nice to recap what has been decided from the beginning:
• How many encoders are going to be included in each package?
Six. It's already hard enough with six and adding another one would increase the difficulty, lower the significance, frighten away potential testers and increase the size of each package.
• Are anchors planned?
Yes: but a low one only. The anchor is absolutely not interested by itself. His only purpose is to increase the uniformity of the notation between all testers. All we need is a very poor quality: average or unoptimal [HE-AAC at 96 kbps as example] settings are not suitable to be low anchors.
As a consequence, there are 5 vacant places for competitors (5 + 1 anchor).
• What is the purpose of the test?
The purpose is to evaluate different format with the best available settings and with recommendable encoders (no outdated ones, no experimental ones). Quality is going to be maximized for the purpose of the test, which will not represent what average people are using.
• Which format should we use?
The choice of format shouldn't be arbitrary. That's why criterions are needed to ensure a coherent choice. The choice of criterion itself is debatable, and final decision has inevitably an unavoidable part of arbitrariness (there are no bad criterions, but choice must be made). Current criterions are:
- hardware compatibility: the tested formats should be independant to the computer environment and therefore currently compatible with several players (like Jukebox, DVD...)
- VBR: all formats must, when possible and if recommended, be set in VBR mode, assuming that VBR is the best coding method at this bitrate, and that most encoders now offer VBR options. Using one CBR encoder among four VBR ones may appear as highly unfair (at least if the CBR one don't end at the first place...) and will cause unecessary debate about Apples and Oranges.
- relevant choice: MP3Pro at 128 kbps is playable on some devices; MPEG1 layer 2 is also working... but these encoders are not relevant at ~130 kbps.
As first consequence formats like MPC, WMAPro or VQF are not going to be test [reason No.1]; same apply to atrac3 and atrac3plus [reason No.2].
As second consequence four formats remain: AAC, MP3, Vorbis, WMA.
We have four formats but five vacant place. It's a good occasion to include two different high quality implementations of AAC: the new generation Nero Digital encoder and the new iTunes VBR one. It may appear as favouritism, but such decision is mostly motivated by the fact that AAC is the only format benefiting from active competition between different companies (Apple, Coding Technologie, Nero Digital); for MP3, Vorbis and WMA, there is only one active branch of development.
[span style='font-size:14pt;line-height:100%']What has still be to discussed:[/span]
• the average bitrate and the tolerance. Assuming that all encoders are set with VBR, the average bitrate can't be identical for each competitor. A margin must be set, as well as the average bitrate of the test (which seems to be closer to be a 133...135 kbps listening test than a stricty 128 kbps one). For encoders benefiting for a precise VBR scale (only vorbis is in the current state), the appropriate -q setting has to be select
• a bitrate table must be built. Some members start to submit data. Additional information is still welcome and should be posted in the dedicated thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=38955).
• samples: we should start to discuss about samples. Are we going to collect new samples? Wouldn't it be better to use again the same collection that was used last time (as Sebastian suggested it)? By using the same samples than before, comparison between tests should be feasable (but still not perfect: testers have probably changed in the meantime). And it would be interesting to compare results to mesure the progress of each encoders.
[span style='font-size:14pt;line-height:100%']What shouldn't be discussed:[/span]
• The version of the encoder or the setting if developers could give their opinion. They are talking the responsability of bad results if their own choices were not optimal.
• off-topic consideration, such as the cooking abilities of developers, their musical tastes or the way they're developing encoders.
____
I hope that this summary reflects Sebastian decisions. I'm just trying to help, not making a putsch If there's a mistake, don't hesitate to correct them.
Personally I would choose some new samples and keep some of the old. As samples are supposed to be representative, comparison with older tests is still possible (imo).
I'm glad we are back on topic, and that you took this inniciative as this is something nobody is more qualified at than you.
Regarding the samples. I think we should definitely include some new samples. Maybe it would be a good idea to keep the samples that performed worst and add some new ones.
I'm glad we are back on topic, and that you took this inniciative as this is something nobody is more qualified at than you.
Regarding the samples. I think we should definitely include some new samples. Maybe it would be a good idea to keep the samples that performed worst and add some new ones.
[a href="index.php?act=findpost&pid=344037"][{POST_SNAPBACK}][/a]
I think this is a good idea, but I think that one or two of the samples that performed well need to be included for reference.
Additional information is still welcome and should be posted in the dedicated thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=38499).
Probably you meant that thread: Bitrate estimates for the upcoming 128 kbps test (http://www.hydrogenaudio.org/forums/index.php?showtopic=38955)
Regarding sample selection, I think it would be worth attempting a stab at approximating the bitrate distribution of a large number of samples. The first step would be to look at the bitrate histograms of the selected codecs to see what's going on over a large sample group.
I'd guess that there would need to be more samples on the low side of the average. Also, samples with extreme bitrate deviations, both high and low, might be better discarded, as you wouldn't expect to see a lot of them in a small group of random samples.
ff123
BTW, I like a lot of the choices made so far: only 5 codecs under test, a real low anchor, a defined test objective with rationale for the choices explained (although not universally agreed upon, which is to be expected), and a test administrator who doesn't wither under criticism.
To sort out one last thing about the encoder (this is related to the samples, too)... Garf asked me if he can send me an "unofficial" encoder if the bug-fixed release of the Nero AAC encoder doesn't make it into the Nero Update before the test starts. Since I cannot wait for him to discuss samples, I have no problem testing an unofficial build as long as I am allowed to distribute it (e.g. make it available for download on HA). The reason why I am "demanding" that is to make sure I don't receive an encoder optimized for the samples used in the test only.
Nice to see that the test objective is coming together.
1) I think a "yet-to-be-released" encoder version would be nice, if it is also a "will-be-released-soon" encoder.
2) If it won't be released at all (perhaps due to test results), but will again be delayed/fixed, then I think testing it will not be fair and worthwhile.
What do you think?
I understood that the situation is as described in 1). If it's like 2), then I think it's not a good idea to include it.
After all, we are testing for users primarly, not for developers. Right?
To sort out one last thing about the encoder (this is related to the samples, too)... Garf asked me if he can send me an "unofficial" encoder if the bug-fixed release of the Nero AAC encoder doesn't make it into the Nero Update before the test starts. Since I cannot wait for him to discuss samples, I have no problem testing an unofficial build as long as I am allowed to distribute it (e.g. make it available for download on HA). The reason why I am "demanding" that is to make sure I don't receive an encoder optimized for the samples used in the test only.
[a href="index.php?act=findpost&pid=344049"][{POST_SNAPBACK}][/a]
This is definitely possible - it is certainly not the intention of nobody inside Nero to generate an encoder which is "produced" for a specific critical sample set. We never did it, and I am pretty sure we won't do it in the future
Also, new Nero Digital AAC update will prolly bear "ABR" as the recommended codec setting for the 128 kbps range.
I think this is a good idea, but I think that one or two of the samples that performed well need to be included for reference.
[a href="index.php?act=findpost&pid=344038"][{POST_SNAPBACK}][/a]
IMO, only really hard to encode samples (not necessarily high-bitrate samples) should be used. If I think of guruboolez' test, where lots of encodings got 4-5 even for his critical ear, and if I think of my own pretty hopeless attempts at ABXing anything besides killer samples at 128kbps with modern encoders, I fear we might get not very discriminative results for the different encoders.
Egor> the link is corrected. Thank you for the verification
ff123> I'm also favourable to your idea. In my opinion, it's the only advantage of changing the samples; otherwise, the "comparison" argument looks strong enough to keep them all.
About extreme samples, I must say that I'm not agree. Especially for low bitrate. As example, I have reconstituted a useful library of LAME VBR encodings to feed my portable players, and IIRC (I can't check now), 10% of the encodings have a really low bitrate (60...100 kbps for 131 kbps on average and 137 as median). And from my experience, all VBR encoder are very far to perform eaqually for the quality point of vue. You maybe remember the
Debussy sample tested last year, revealing to some people that different VBR implementations are reacting differently on such musical content. These samples are not exceptional I'd say, and one at least should be included in the test.
The same maybe apply for extremely high bitrate. I'm not concerned, but I suppose that people listening to some kind of electronical music are often encountering such tracks. Again, different encoders don't react eaqually, and this difference would be interested to test (even if one sample is not enough to conclude anything strong for such material).
With 18 samples, including one very low and very high bitrate encoding
(assuming that all VBR implementations are excessively reacting to such content) shouldn't be a problem. 16 vacant places for less exceptional content (again, assuming that very low and very high bitrate are exceptional, and I not entirely agree with it) are remaining.
In my own bitrate table, I have such "extreme" samples. Here are the average value for full tracks (bitrate are even more "extreme" with shorter pieces of music)
MEAN | WMA LAME iTunes aoTuV
A03_emese 178 | 226,4 157,5 143,8 184,0
E01_MODERN_CHAMBER_A_brass 98 | 80,5 78,3 130,2 103,9
E20_MODERN_ORCHESTRAL_E_strings 99 | 63,5 91,5 130,5 112,1
S30_OTHERS_Accordion_A 98 | 74,5 78,9 128,8 108,6
S54_WIND_Trombone_A 98 | 78,0 73,9 129,9 108,9
Only iTunes looks apathetic. I explained why the bitrate don't go below 128 kbps (there's a bitrate floor!). It doesn't go very high either (track A03). But all remaining encoders have similar reactions: either an inflated bitrate or an weedy one.
Sebastian Mares> For the experimental encoder of Nero Digital, I'm a bit puzzled. The test would include on one hand four encoders released for general purpose and widely available to the public and on the other hand one encoder released for the only purpose of the test. I must precise before someone would be tempted to invoke something like a "anti-Nero crusade" that I'd be the first one to be pleased to see very fast improvements of Nero Digital and that I'm in fact currently very keen to see so fast reactions of Ivan and all the Nero Digital team.
I'm just concerned by the equity of the test. Is it fair to accept at the last minute something unofficial and apparently released with the sole object to improve the performance for the test? There were concerns about the inclusion or not of --athaa-sensitivity, which isn't available in latest beta, but what should we say to a whole encoder which isn't available at all at Nero.com and for a majority of Nero users? If we accept this, shouldn't we also ask to Microsoft, Apple, Aoyumi, Xiph, LAME's team if they also can't provide a preview of their upcoming work?
I repeat: I only asking questions, not giving answers. I did my test on my side, and didn't hesitate to use the most advanced encoders (like LAME 3.98 alpha). I
never hesitate in fact (I was the only one to test lzst year aoTuV beta 3).
In other word, I'm rather in favour of the experimental encoder. But the questions are legitimate in my opinion.
Sebastian Mares> For the experimental encoder of Nero Digital, I'm a bit puzzled. The test would include on one hand four encoders released for general purpose and widely available to the public and on the other hand one encoder released for the only purpose of the test.
This is a non-issue, Nero AAC encoder that will be used in the test
will be in the next regular web-release of Nero7. We had similar cases earleir and all changes ended up in the next public build of N7.
Will the next update of Nero Digital include the tested encoder and produce exactly the same output? If the answer is positive, I agree with you about the non-issue
It will be identical copy
It will be identical copy
[a href="index.php?act=findpost&pid=344141"][{POST_SNAPBACK}][/a]
For me everything is all right then.
To sort out one last thing about the encoder (this is related to the samples, too)... Garf asked me if he can send me an "unofficial" encoder if the bug-fixed release of the Nero AAC encoder doesn't make it into the Nero Update before the test starts. Since I cannot wait for him to discuss samples, I have no problem testing an unofficial build as long as I am allowed to distribute it (e.g. make it available for download on HA). The reason why I am "demanding" that is to make sure I don't receive an encoder optimized for the samples used in the test only.
[a href="index.php?act=findpost&pid=344049"][{POST_SNAPBACK}][/a]
This is definitely possible - it is certainly not the intention of nobody inside Nero to generate an encoder which is "produced" for a specific critical sample set. We never did it, and I am pretty sure we won't do it in the future
Also, new Nero Digital AAC update will prolly bear "ABR" as the recommended codec setting for the 128 kbps range.
[a href="index.php?act=findpost&pid=344077"][{POST_SNAPBACK}][/a]
It was a just-in-case scenario. Anyways, I am glad that it's OK with you.
Regarding the samples, I think we could use 12 old samples (from Roberto's listening test) and 6 new ones. I also re-downloaded the samples from my 64 kbps listening test thread and am listening to them now.
Could anyone post 0:57 - 1:32 2:54 - 3:24 of "Sash! - Stay" (FLAC, WV or MAC)? I hope it's OK even if it's 35 seconds instead of 30...
Edit: I am asking for the long version of the song (~6 minutes) not the radio edit which is only ~3 minutes long.
First 30 seconds of "Sash! - Mysterious Times" would be cool, too.
I have one comment to target bitrate: Is the goal to have a ~130 kbps bitrate for every file or to achieve this bitrate as average of all samples.
The first possibility would show better how codecs perform at the same bitrate, where the other way would show how can codecs react on more/less complicated sources.
Lyx explained it better.
I have one comment to target bitrate: Is the goal to have a ~130 kbps bitrate for every file or to achieve this bitrate as average of all samples.
None of the above. The target bitrate is the resulting average bitrate when encoding entire tracks of a wide range of music-genres. You could call it the "real-world average".
The first possibility would show better how codecs perform at the same bitrate...
[a href="index.php?act=findpost&pid=344343"][{POST_SNAPBACK}][/a]
On a single sample ONLY which is unrealistic and uninteresting - if a sample is difficult, then the encoder is EXPECTED to increase the bitrate *on this part of the track*.
- Lyx
I started reading the closed topic in mp3 general forum,
and found there, that MPC is not considered to be tested in this multiformat test due to missing portable hardware support.
This isn't valid, because there is hardware support,
PPC, pda.
Because MPC is an advanced codec, it should be tested also, I think, testing does not hurt.
More reasons to include MPC:
In last listening tests MPC has won or won tied together with Vorbis.
Because MPC has not changed (a lot) during times, MPC could act as a stable anchor, so improvements of other formats during the times in developmnt can be measured.
Then we should look to future, PPC/pda, programmable hardware devices will come more and more, and then you have the possibility to play mpc even more.
(It is already a pleasure to play MPC in my car via pda, also used for gps navigation.)
regarding mp3 lame settings selection for this test, i agree with gabriel,
that the simple 3.97b1 -V5 --vbr-new should be tested.
Other tests should be carried out in special lame (dev-) tests, like V5 vbr-new vs plain V5, and with/without the ath 1 switch.
I have one comment to target bitrate: Is the goal to have a ~130 kbps bitrate for every file or to achieve this bitrate as average of all samples.
[a href="index.php?act=findpost&pid=344343"][{POST_SNAPBACK}][/a]
The same debate happened for most (if not all) previous listening tests involving VBR encoders.
Testing all samples at exactly the same bitrate could be fun, but such test wouldn't correspond to a realistic scenario. I don't know anyone reencoding each track of his library to obtain for each of them a precise bitrate.
Unrealistic scenario, but also
impossible: such process is only conceivable with encoders offering a precise VBR scale. For iTunes, LAME, Nero Digital, WMA, if the bitrate reach 124 kbps 'only', the use of the next VBR level will lead to 150 or 170 kbps.
That's why such testing principle requires CBR encoders, not VBR ones.
The first possibility would show better how codecs perform at the same bitrate,
To some extend, the second too. All encoders should lead to the same bitrate. But not on short samples, not on complete track and not even for a complete album -- all encoder should achieve the same bitrate on a full library.
That's the way all people (I suppose) are thinking and acting. --preset standard correspond to something close to 192 kbps, even if the bitrate fluctuate within each track, even if most tracks don't correspond to the expected bitrate. On average, and dispite all dissimilarties, the user will get ~192 kbps with his VBR preset. That's why we can defend the paradoxical idea that all files are going to be tested on the same bitrate even when they aren't obviously encoded at the same bitrate.
I started reading the closed topic in mp3 general forum,
and found there, that MPC is not considered to be tested in this multiformat test due to missing portable hardware support.
This isn't valid, because there is hardware support,
PPC, pda.
Because MPC is an advanced codec, it should be tested also, I think, testing does not hurt.
More reasons to include MPC:
In last listening tests MPC has won or won tied together with Vorbis.
Because MPC has not changed (a lot) during times, MPC could act as a stable anchor, so improvements of other formats during the times in developmnt can be measured.
Then we should look to future, PPC/pda, programmable hardware devices will come more and more, and then you have the possibility to play mpc even more.
(It is already a pleasure to play MPC in my car via pda, also used for gps navigation.)
regarding mp3 lame settings selection for this test, i agree with gabriel,
that the simple 3.97b1 -V5 --vbr-new should be tested.
Other tests should be carried out in special lame (dev-) tests, like V5 vbr-new vs plain V5, and with/without the ath 1 switch.
[a href="index.php?act=findpost&pid=344347"][{POST_SNAPBACK}][/a]
Sorry, but the codecs are now set. I cannot increase the number of codecs tested and IMO, WMA is more popular than MPC so it has priority.
MPC, ATRAC and WMA Pro could be tested in an extension test if people really want it and there are enough testers.
I started reading the closed topic in mp3 general forum,
and found there, that MPC is not considered to be tested in this multiformat test due to missing portable hardware support.
This isn't valid, because there is hardware support,
PPC, pda.[a href="index.php?act=findpost&pid=344347"][{POST_SNAPBACK}][/a]
This is not portable players. There are small computers running with OS and allowing the installation of media players and different codec. My Compaq Presario is also portable and play OptimFrog --extrahigh without problem. It's not a valid reason to claim that OptimFrog is supported by portable players
Lets wait that any brand will officially support MPC to say that this encoder corresponds to the required criterion.
Because MPC is an advanced codec, it should be tested also, I think, testing does not hurt.
Did you read what was decided previously? Advanced is not a criterion of choice. MPC is a pertinent choice, but less than AAC, MP3, Vorbis and WMA.
BTW, an advanced encoder should at least offer a proper seeking
Then we should look to future, PPC/pda, programmable hardware devices will come more and more, and then you have the possibility to play mpc even more.
In few years there may be more compatible players with musepack than musepack users themselves
The debate for MPC, WMAPro, VQF, HE-AAC is over. None of them will be tested here - excepted maybe if we find a very good reason to discard one of the current competitor.
The target bitrate is the resulting average bitrate when encoding entire tracks of a wide range of music-genres. You could call it the "real-world average". ...[a href="index.php?act=findpost&pid=344346"][{POST_SNAPBACK}][/a]
I thought this was obvious and already settled.
In my opinion the only possible alternative could be the average bitrate of the complete tracks used for making the samples, but that would not be as representative (as already explained in this thread several times).
BTW, I am just about to post a new table in the "bitrate" thread.
P.S. My WMA findings were commented earlier:
My results for wma seem to differ quite a bit from Alex b's results. Here, while using the same preset (Q50), the average bitrate is only 98 Kbits/sec. (it seems unfair to compare wma with this preset against the other samples which average 134 Kbits/sec. With wma included the average for all samples drops to 127 Kbits/sec. Unfortunately, if you go to Q75 the bitrate is much too high...)[a href="index.php?act=findpost&pid=343483"][{POST_SNAPBACK}][/a]
I used WMA
Pro at q50. I already mentioned earlier that Standard produced about 105 kbps with my test tracks. On the other hand q75 makes way too big files. The only possible "fair" way to use WMA Standard is VBR 2-pass 128 kbps. Though, that is a bit unfortunate because the VBR 2-pass option is not available in the standard WMP10 GUI. It can be found e.g. in the Windows Media Encoder and dbPowerAMP GUIs.
while it is quite obvious that, in theory, (and mostly in practice too) VBR quality-based should give better results than CBR, I don't think it should be a necessity to include it. The only thing required is: how big is the file. (which translates directly in a maximum bitrate).
What I'm trying to say is: yes, a cbr mode _should_ perform worse than a vbr mode, but it's not a reason to exclude a codec from a test. If a cbr-codec loses out compared to vbr, tough luck. The inverse can also be true: have a look at the results from roberto's tests in which itunes AAC @ CBR performed as well or even better than nero digital @ VBR.
If a file encoded by a particular codec sounds great, I couldn't care less whether it's vbr or cbr...Which mode to use is really the problem of the guy who has to recommend the settings for that encoder.
while it is quite obvious that, in theory, (and mostly in practice too) VBR quality-based should give better results than CBR, I don't think it should be a necessity to include it. The only thing required is: how big is the file. (which translates directly in a maximum bitrate).
What I'm trying to say is: yes, a cbr mode _should_ perform worse than a vbr mode, but it's not a reason to exclude a codec from a test. If a cbr-codec loses out compared to vbr, tough luck. The inverse can also be true: have a look at the results from roberto's tests in which itunes AAC @ CBR performed as well or even better than nero digital @ VBR.
If a file encoded by a particular codec sounds great, I couldn't care less whether it's vbr or cbr...Which mode to use is really the problem of the guy who has to recommend the settings for that encoder.[a href="index.php?act=findpost&pid=344363"][{POST_SNAPBACK}][/a]
I don't think any encoders have been excluded because of a missing usable VBR option.
iTunes's default AAC mode is not CBR, the bitrate varies. The new VBR mode makes it vary more, mostly towards upper bitrates.
WMA VBR 2-pass is a good encoding mode at 128 kbps. Most likely it would be better for WMA results than the CBR mode.
Actually, now when I think this VBR matter again, I realize that the only fair system would be to encode the complete tracks, decode the tracks and only after that separate the sample passages.
This is especially true with WMA 2-pass, because as far as I know it is designed to first measure the whole file and after that make the adjustments for getting the defined average bitrate. I'll try to test the 2-pass behavior with some track that compresses otherwise well, but has a small complex and difficult to encode passage. I'll encode the whole file and only the complex passage separately.
I think this has been discussed earlier, but I didn't search for it.
What I'm trying to say is: yes, a cbr mode _should_ perform worse than a vbr mode, but it's not a reason to exclude a codec from a test.
[a href="index.php?act=findpost&pid=344363"][{POST_SNAPBACK}][/a]
Indeed. But this is not the invoked reason. Comparing VBR to CBR always caused problems (mostly about fairness). Moreover, it becomes clear that two encoders at least won't reach 128 kbps, but rather 134...135 kbps (iTunes, LAME). Then we'll get 128 CBR vs 135 VBR. And for same samples, 128 vs 150.
For the two first listening tests at this bitrate, it was very hard to avoid the CBR vs VBR confrontation. VBR encoders were necessary (MPC, Vorbis) and CBR had to be included because the best AAC challenger didn't offer any VBR mode.
Presently, he have a chance to avoid all debate on this subject, because most encoders (atrac3 is the only exception, with vqf ) offer VBR. And the most important point is that in all likehood each encoder has a VBR preset corresponding to 130...135 kbps (I'm still unsure for WMA though).
Testing VBR and CBR encodings has nothing wrong in my opinion, but for the peace of the debate, it would be nice if we could avoid this situation. In fact, it only penalize atrac3plus. Whether atrac3plus should be tested or not has been discussed. The format is maybe popular, but nobody was so far interested enough by the relative quality of current encoder at this bitrate to start and publish ABX tests. If only some users brought the proof that atrac3plus was a real improvement and an interesting codec, then the inclusion of atrac3plus may be a subject to debate.
As a consequence, most encoders corresponding the main criterion (hardware compatibility) could be tested with VBR mode.
I made the WMA 2-pass encoding test files, but I couldn't find a way to analyze them. I checked out about 10 different player programs, but none could show the dynamic bitrate. Also, I couldn't find a way to split a WMA file losslessly for analyzing the joined file in two separated parts.
The samples are in this package: WMA2passSamples.rar (659 KB) (http://www.archive.org/audio/etree.php), if someone knows how to analyze them.
The package has these WMA VBR 2-pass 128 kbps files:
Symphony of life, 10 s (simple and quiet)
Planet Dada, 10s (complex and loud)
Joined, 20 s (I joined the original wave samples before encoding)
[span style='font-size:8pt;line-height:100%']Edit: a small fix, Edit 2: link removed.[/span]
So we all agree taht the second possibility I offered (average bitrate of all samples) is better. But now my question is: Do some encoders allow you to choose how much can the bitrate vary from the average? If I understood it correctly, iTunes can.
This can lead to situation, where one encoder will have 80 kbps and 180 kbps samples, while the other one would use bitrate 110-150 kbps. Of course a good encoder should try to reach same quality for all samples, but are we sure the encoders will?
This might be another ineteresting aspect of the test. What do you think?
Vlada
Do some encoders allow you to choose how much can the bitrate vary from the average?
[a href="index.php?act=findpost&pid=344409"][{POST_SNAPBACK}][/a]
Yes, some do. LAME for example and Vorbis if I am not mistaken, although it never worked on my side with Vorbis.
This can lead to situation, where one encoder will have 80 kbps and 180 kbps samples, while the other one would use bitrate 110-150 kbps. Of course a good encoder should try to reach same quality for all samples, but are we sure the encoders will?
[a href="index.php?act=findpost&pid=344409"][{POST_SNAPBACK}][/a]
As long as the average bitrate is around 128 kbps or whatever was specified, it's irrelevant if an encoder has peaks of 320 kbps or only 160 kbps.
If encoder A decided to encode a complicated part of the sample at 320 kbps and encoder B only allocated 160 kbps, we can say that encoder A was smart (assuming it sounds better than B and the average bitrate is not too far away from the desired one).
From the oggenc manpage:
--advanced-encode-option bitrate_average_window=NN
Set the managed bitrate window to NN seconds. The bitrate will be forced to the specified average over a floating window of this length. May be fractional (e.g. 3.5)
I never tried it, but it seems this only applies with the bitrate management engine enabled, i'm not aware of any option that influences how reactive the bitrate is to the nature of the content in the normal VBR mode.
The only other thing you can do is limit min and max bitrate.
Personally i'm all for constant quality and variable bitrate. Even if bitrates deviate significantly on testing samples, as long as bitrates match nominal on a large, varied selection of real world music (which the samples were picked from), that's fine with me. In such a case the bitrate querness really says more about the selection of samples, than the about codec.
Well, in a perfect world with perfect, constant quality VBR codecs, anyway.
So we all agree taht the second possibility I offered (average bitrate of all samples) is better.[a href="index.php?act=findpost&pid=344409"][{POST_SNAPBACK}][/a]
Hmm... not the average bit rate of samples. The optimal settings that produce the desired average bitrate on a wide variety of complete tracks should be used as already explained:
The target bitrate is the resulting average bitrate when encoding entire tracks of a wide range of music-genres. You could call it the "real-world average".[a href="index.php?act=findpost&pid=344346"][{POST_SNAPBACK}][/a]
All encoders should lead to the same bitrate. But not on short samples, not on complete track and not even for a complete album -- all encoder should achieve the same bitrate on a full library.[a href="index.php?act=findpost&pid=344348"][{POST_SNAPBACK}][/a]
But now my question is: Do some encoders allow you to choose how much can the bitrate vary from the average?[a href="index.php?act=findpost&pid=344409"][{POST_SNAPBACK}][/a]
The recommend settings, which are going to be used in this test don't allow that.
If I understood it correctly, iTunes can[a href="index.php?act=findpost&pid=344409"][{POST_SNAPBACK}][/a]
iTunes has only one tick box for the new VBR option. The older AAC mode in iTunes is not a quality based VBR mode even the bitrate varies a bit. All encoded files have the same average bitrate.
This can lead to situation, where one encoder will have 80 kbps and 180 kbps samples, while the other one would use bitrate 110-150 kbps. Of course a good encoder should try to reach same quality for all samples, but are we sure the encoders will?[a href="index.php?act=findpost&pid=344409"][{POST_SNAPBACK}][/a]
After the test we may know better. Bitrate is not the only variable that affects the encoding quality. Different encoders use the available bits differently.
[span style='font-size:7pt;line-height:100%']Edit: typo[/span]
iTunes has only one tick box for the new VBR option. The older AAC mode in iTunes is not a quality based VBR mode even the bitrate varies a bit. All encoded files have the same average bitrate.
One remark regarding AAC (and MP3, for that matter) and "CBR" - in fact, CBR in AAC is also quality-controlled due to the presence of the bit-reservoir. Bit rate allocation for each frame is usually done by checking the fullness of the bit reservoir, and adding/donating bits to/from the average frame bit rate, depending exactly on the
psymodel demands and previous frame complexity statistics - so, even "CBR" (which it is not really, in fact) is quality-controlled.
You would be surprised with how small bit-reservoir you could prevent pre-echoes and artifact peaks - value of 1000 bits would already help a lot, and AAC typically uses ~10000 bits at this average bit rate. Also - it is a very blurry line between CBR/ABR and VBR in these codecs.
Real CBR in MP3 and AAC would mean that every single frame is exactly 128000/(sample_rate / frame_size) bits big... which would definitely kill the quality on low and mid bit rates.
That mode is only used in low-delay configurations (even then it is desirable to have small bit reservoir of, say 500 or so bits) - I don't think any of the AAC codecs on the market use that mode for offline encoding not constrained with the lowest-delay requirements.
I made this picture for explaining better what I meant with the 2-pass VBR problem in case the short samples are encoded instead of the complete tracks:
(http://kotisivu.mtv3.fi/alexb/ha/2pass.png)
I guess the same can be more or less possible with AAC because the encoders tend to make less varied average bitrates than LAME or Vorbis.
[span style='font-size:7pt;line-height:100%']Edit: changed a couple of words[/span]
I agree with the problem pointed out by guru and alex b regarding wma 2-pass vbr. If this mode is going to be used, the best method is to encode the entire track and then decode to wav and extract the appropriate section.
ff123
Would a tool like this work to cut samples out of wma tracks?
http://www.softaward.com/6451.html (http://www.softaward.com/6451.html)
It would also be nice if there were some sort of command-line wma decoder that could be distributed with the test zipfile, to cut the download size.
I guess the same can be more or less possible with AAC because the encoders tend to make less varied average bitrates than LAME or Vorbis.
It is certainly technically possible to do 2-pass VBR in AAC (in fact I know one encoder that does it ) - it is just question is it worth the effort compared to bitres-managed "CBR" - I don't really believe it is the case.
We are here talking about two paradigms:
#1 - "Encode so that average of many tracks is 128 kbps"
or
#2 - "Encode, so that EVERY track averages to 128 kbps"
Obviously, Vorbis and LAME VBR use #1 - and, some others, like WMA are closer to #2.
IMO, it is actually comparing apples and oranges - but, hey, we have to compare something, and comparing "most widespread coding solutions that average @130 kbps" is the clostest we can get.
Ivan, how does Nero behave in the example situation pictured? Will the separately encoded complex passage be different from the complex passage that is encoded as part of the complete track? (Assuming the used setting is the "Internet" VBR.)
Hard to tell as there is ongoing rework of VBR mode..
Basicaly - we could offer four modes:
1. AAC CBR - compatible with the ISO 14496 bit-reservoir constraints, basically what people call CBR
2. AAC ABR - larger deviation, still near the target bit rate but would need larger pre-buffering for streaming - pretty similar to iTunes "VBR mode"
3. AAC Managed VBR - with constraints such as minimum and maximum bit rate, etc... that is most similar to Vorbis I would say. It would reach average bit rate on the wide selection of audio samples - but could give higher bit rates for some hard samples.
4. AAC Psymodel-threshold-simulator - complete variable bit rate, driven only by psychoacoustic threshold - not so predictable.
We are still investigating which mode to use - I guess either 2 or 3 for this test but not yet sure.
Thanks Ivan,
I think only the encoders that use the mode 4. are suitable for encoding directly the short wave samples.
The full tracks should be encoded when an encoder uses the modes 2-3.
Would a tool like this work to cut samples out of wma tracks?
http://www.softaward.com/6451.html (http://www.softaward.com/6451.html)[a href="index.php?act=findpost&pid=344440"][{POST_SNAPBACK}][/a]
Unfortunately no. I already tried it earlier on today. The claim of "no quality loss" is false. The tool obviously decodes and encodes WMA files. The user has to select the output format and encoding settings. Besides, the cutting precision is only one full second.
I also tried a couple of other tools that I found, but they weren't any better.
Whether atrac3plus should be tested or not has been discussed. The format is maybe popular, but nobody was so far interested enough by the relative quality of current encoder at this bitrate to start and publish ABX tests. If only some users brought the proof that atrac3plus was a real improvement and an interesting codec, then the inclusion of atrac3plus may be a subject to debate.
[a href="index.php?act=findpost&pid=344378"][{POST_SNAPBACK}][/a]
I'm sorry but I have to respond to this. In my opinion, which I have voiced very clearly in this thread before, neither ATRAC nor WMA Standard are interesting encoders to include in a listening test. But Sebastian made it very clear that the criterion for inclusion was not going to be "interesting" but rather "portable and popular". By that criterion you should logically include both WMA Standard and ATRAC.
However, since the codecs are now decided, it's rather pointless to continue arguing about this. I'm just pointing out what I think is a flawed argument in your reasoning.
I'm just pointing out what I think is a flawed argument in your reasoning.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=344464")
I don't think so. See my [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=344017]recapitulation[/url]. I don't know if Sebastian is agree or not, but he hasn't objected to it so far.
- popularity is not a criterion anymore
- portable, VBR and relevancy are the choosen ones.
Atrac3plus is popular, relevant but CBR only: it don't respect one of the criterion => not suitable for this test.
AAC, MP3, Vorbis, WMA are compatible with dedicated players, offer VBR encoding, and are relevant for 128 kbps encodings.
I'm just pointing out what I think is a flawed argument in your reasoning.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=344464")
I don't think so. See my [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=344017]recapitulation[/url]. I don't know if Sebastian is agree or not, but he hasn't objected to it so far.
- popularity is not a criterion anymore
- portable, VBR and relevancy are the choosen ones.
Atrac3plus is popular, relevant but CBR only: it don't respect one of the criterion => not suitable for this test.
AAC, MP3, Vorbis, WMA are compatible with dedicated players, offer VBR encoding, and are relevant for 128 kbps encodings.
[a href="index.php?act=findpost&pid=344467"][{POST_SNAPBACK}][/a]
There's also the point that Atrac users are, more often than not, unwilling to acknowledge test results. While that is not a test criterion, it's a valid point to help decide between WMA Std. and Atrac.
What I'm trying to say is: yes, a cbr mode _should_ perform worse than a vbr mode, but it's not a reason to exclude a codec from a test.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=344363")
Indeed. But this is not the invoked reason.
I don't think so. See my [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=344017]recapitulation[/url]. I don't know if Sebastian is agree or not, but he hasn't objected to it so far.
- popularity is not a criterion anymore
- portable, VBR and relevancy are the choosen ones.
Atrac3plus is popular, relevant but CBR only: it don't respect one of the criterion => not suitable for this test.
AAC, MP3, Vorbis, WMA are compatible with dedicated players, offer VBR encoding, and are relevant for 128 kbps encodings.
[a href="index.php?act=findpost&pid=344467"][{POST_SNAPBACK}][/a]
In the post I quoted just before, you state that lack of VBR shouldn't be decisive, and now you do. Which one is it?
In the post I quoted just before, you state that lack of VBR shouldn't be decisive, and now you do. Which one is it?
[a href="index.php?act=findpost&pid=344481"][{POST_SNAPBACK}][/a]
I never implied that VBR must be a criterion. For proof: previous listening tests with iTunes CBR. If a format is already good without VBR, then it would be nice to see a comparison with other formats.
The problem with atrac3plus is that we don't even know if it's good. Some people are claiming that, but they didn't make any effort to give valid arguments to this idea (like ABX tests on multiple samples).
That's why I proposed VBR as criterion for this test and to avoid further debate on this subject. AAC, MP3, VORBIS and WMA are VBR. There's no place for another competitor. A choice is necessary. I know that people are interested to see atrac3plus included in this test, but we can't satisfy everyone: otherwise the test will include 15 codecs with bitrate going from 32 to 200 kbps and with 86 samples covering all desired musical genre.
I never implied that VBR must be a criterion
But you have clearly said that it is a criterion for this test.
The problem with atrac3plus is that we don't even know if it's good.
Yes, so better not test it.
A choice is necessary. I know that people are interested to see atrac3plus included in this test, but we can't satisfy everyone: otherwise the test will include 15 codecs with bitrate going from 32 to 200 kbps and with 86 samples covering all desired musical genre.
[a href="index.php?act=findpost&pid=344483"][{POST_SNAPBACK}][/a]
Yes, that's a valid argument that I accept. But I don't agree with the choices done. But as you said, you can't please everybody...
Sebastian - here are the first 30 seconds of Sash - Mysterious Times anyway. (The 3.39 "radio" version.):
http://davidnaylor.org/temp/Sash_-_Mysterious_Times_30sec.wv (http://davidnaylor.org/temp/Sash_-_Mysterious_Times_30sec.wv)
I have the "Maxi Version" and the "John B. Norman" versions too, if you prefer.
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
Edit: It turns out they don't have lossless at allofmp3.com after all. My memory is not the best, let's say.
Sebastian - here are the first 30 seconds of Sash - Mysterious Times anyway. (The 3.39 "radio" version.):
http://davidnaylor.org/temp/Sash_-_Mysterious_Times_30sec.wv (http://davidnaylor.org/temp/Sash_-_Mysterious_Times_30sec.wv)
I have the "Maxi Version" and the "John B. Norman" versions too, if you prefer.
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
Edit: It turns out they don't have lossless at allofmp3.com after all. My memory is not the best, let's say.
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]
Thank you!
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]
I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]
I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
[a href="index.php?act=findpost&pid=344597"][{POST_SNAPBACK}][/a]
Is it the long version, or the radio edit?
The problem with atrac3plus is that we don't even know if it's good.
Yes, so better not test it.
I believe Guru's point to be: if the MD community doesn't bother about ABX tests, why should we?
The problem with atrac3plus is that we don't even know if it's good.
Yes, so better not test it.
I believe Guru's point to be: if the MD community doesn't bother about ABX tests, why should we?
[a href="index.php?act=findpost&pid=344635"][{POST_SNAPBACK}][/a]
Because NO ONE else does listening tests, regardless of format..
the point is, MD is old, unpopular*, retired by sony, and the users don't care about listening tests, so why test it again?
* i can se here (at work) at least 200 MP3 players, and 1 MD.
the point is, MD is old, unpopular*, retired by sony, and the users don't care about listening tests, so why test it again?
* i can se here (at work) at least 200 MP3 players, and 1 MD.
[a href="index.php?act=findpost&pid=344673"][{POST_SNAPBACK}][/a]
I don't understand why these kind of issues keep surfacing. I thought I had made myself clear, but apparently not. So here we go again.
1. So what if it's old? MP3 is also old but it's tested.
2. I thought we discussed popular before... It may be unpopular in some parts of the world, but in others it's still popular. Japan being one of them.
3. Where did you get that idea from? Show me some official information from Sony that they have retired ATRAC, or I claim you just made that up on the spot.
4. First you said that you hardly know any MD users, and now you claim that they don't care about listening tests. How can you know that if you don't know any users? How many in the MP3 and WMA userbase care about this listeing test, you think? Besides, that was not a criterion when choosing codecs as stated before.
Anyway, why bother trying to justify the choice of codecs anymore? It's set, and it won't change, so there's no point in discussing it further. Or do you just want to create an aura of unanimity? The fact is, there will always be other opinions, no matter how you do.
seen responses from then on minidsic forum. thats why i know they don't care. and, besides japan, what other place its popular you say? and what numbers do you know in japan to backup your claim?
edit: tried to find the atrac sony news, but no luck so far.
According to a listening test (http://www.sony.net/Products/ATRAC3/tech/ITS_test_report.pdf) organised by SONY back in 2003, Atrac3plus @ 132 kbps is only on par with WMA std @ 128 kbps and MusicMatch MP3 @ 128 kbps.
@ErikS:
Why would you like to see Atrac3plus included in the test, when it most certainly will fall short when compared to the VBR codecs?
BTW: Sony had the same tracks tested in Germany with identical results (http://www.sony.net/Products/ATRAC3/tech/TESTfactory_Listening_test.pdf).
So, even when SONY gets to select the tracks for the listening test, atrac3plus don't outperform Fraunhofer MP3 @ 128 kbps CBR, not even WMA std @ 128 kbps CBR!
Furthermore it's a bit perculiar that only SONY was allowed to encode the tracks. They provided the WAV file of the decoded Atrac3plus tracks for the listening test to the test labs. The test labs never got to play with the atrac3plus encoder and for all I know the WAV files provided by SONY could have been tampered with.
Edited: Added comment about second listening test.
seen responses from then on minidsic forum. thats why i know they don't care. and, besides japan, what other place its popular you say? and what numbers do you know in japan to backup your claim?
edit: tried to find the atrac sony news, but no luck so far.
[a href="index.php?act=findpost&pid=344765"][{POST_SNAPBACK}][/a]
I know Japan because I'm here right now. In any given electronics shop which sells portable music players I go to (except the apple store) has a ratio of MD players which is about 1/4 compared to the total number. That's the source I use. Sure, it's a bit unscientific, but at least it should give an idea about popularity. I don't know how it looks in other countries except Sweden and Singapore (where MD is all but nonexistant). You tell me it's not popular in your country either, so you could have a point.. perhaps it's only in the backwards country Japan it is popular. Why don't you check on the forum you visited if they are all Japanese?
Another question: don't you think more people know about MD than know about Vorbis capable players? Yet, there is no problem to include Vorbis as a popular portable codec...
Why would you like to see Atrac3plus included in the test, when it most certainly will fall short when compared to the VBR codecs?
[a href="index.php?act=findpost&pid=344770"][{POST_SNAPBACK}][/a]
This question can be divided in two, and I will give two answers to it.
1. I don't want to see ATRAC included. Scroll back a couple of pages and you will see which codecs I would have wanted in the test.
2. What does expected results have to do with selection when you state that the criteria are "portable and popular"? I've seen many people add various arguments to why this codec should be included and another one not, but if you want to be consistent you should state the criteria clearly before the test and then stick to the logical consequeces of it.
Counter question: why would you like to see WMA Standard included in the test, when it most likely will fall short when compared to the other codecs?
Counter question: why would you like to see WMA Standard included in the test, when it most likely will fall short when compared to the other codecs?
[a href="index.php?act=findpost&pid=344776"][{POST_SNAPBACK}][/a]
Because it supports VBR, it's used on a lot of music stores, many portable audio players support it (and I don't mean mass, but a lot of different brands), there are several DVD players supporting it. That's why it has priority over ATRAC
The discussion about codecs is over now.
The discussion about codecs is over now.
[a href="index.php?act=findpost&pid=344801"][{POST_SNAPBACK}][/a]
I think you're sending a conflicting message here. The same as before, you made it appear that you still wanted to discuss things, yet you said your decisions were fixed.
And once again, you make it appear as if you are not decided:
There is room for two other competitors or one competitor and a low anchor. Maybe MusePack and some version of WMA (Professional or Standard)?
... and yet you say the "discussion is over."
If you don't want to talk about format choices anymore, you need to update your original post and make it
very clear that you've made up your mind and that your decision is
final. Otherwise people are going to keep discussing, and other people are going to keep getting annoyed.
Anyway, why bother trying to justify the choice of codecs anymore? It's set, and it won't change, so there's no point in discussing it further. Or do you just want to create an aura of unanimity? The fact is, there will always be other opinions, no matter how you do.
[a href="index.php?act=findpost&pid=344741"][{POST_SNAPBACK}][/a]
... if you want to be consistent you should state the criteria clearly before the test and then stick to the logical consequeces of it.
[a href="index.php?act=findpost&pid=344776"][{POST_SNAPBACK}][/a]
I can't agree with either of these sentiments more.
I hope at this point that argument over format choices is ended soon and that a final decision is made already. There's not going to be unanimous agreement no matter what.
And I hope that this whole debate can serve as an example for the future: New tests need to have a
very clearly stated purpose along with
well defined decision making criteria,
before debate gets under way for particular test decision details. I believe that a lot of the disagreement and strife that followed from this debate stems rather directly from this ambiguity and the fact that so much was open to different interpretation (and in fact even reinterpretation at various points, IMO).
And I hope that this whole debate can serve as an example for the future: New tests need to have a very clearly stated purpose along with well defined decision making criteria, before debate gets under way for particular test decision details. I believe that a lot of the disagreement and strife that followed from this debate stems rather directly from this ambiguity and the fact that so much was open to different interpretation (and in fact even reinterpretation at various points, IMO).
Precisely. For the sake of practicality and so that the test can actually occur instead of being held up by arguments, I'll drop my request for the inclusion of WMA Pro. But I'll also agitate for the inclusion of only 1 AAC encoder - iTunes - for the same reasons.
And I hope that this whole debate can serve as an example for the future: New tests need to have a very clearly stated purpose along with well defined decision making criteria, before debate gets under way for particular test decision details.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=344811")
I agree with you. But do you mean that the test conducer must chose himself the purpose of the test before any debate, or that some room for debate should be allowed by the conducer?
Take a look on this debate. The progress is not confused (excepted one off-topic debate):
1/ Sebastian Mares has started the debate with four contenders (AAC, AAC, MP3, Vorbis - which are the only encoders developed with regularity and the most popular at this bitrate here on HA.org). He was also very clear on the number of contender (6, with or without anchors). It was not a bad basis but rather a fertile one.
2/ During several pages, there were a debate about:
- MPC
- WMA and/or WMAPro
- atrac
2a/ Quickly, Sebastian conclude than (real) portability is an obligatory requirement: WMAPro and MPC are exclude.
2b/ He also considered popularity as a second one.
3/ To avoid endless debate about atrac3 being as popular as WMA, about Vorbis being less popular as atrac3 or different considerations on this subject, I proposed to replace the unclear "popularity" criterion by[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=344017] the "VBR" argument[/url]. I still consider that if a fifth encoder must be add to the four VBR ones, it mustn't be a CBR one. Consequently, with "portability and VBR" as criterions, WMA [std] is the only remaining choice.
The whole debate was maybe not conduced with very precise and clearly defined steps (1/ criterions 2/ formats 3/ bitrate table & encoders settings 4/ samples 5/ additionnal debate about minor points: like volume normalisation, etc...), but Sebastian is also beginning in this task, and he didn't performed poorly.
Now what we need is to focuse our energy on precise details. The first of it is: the bitrate table, which is necessary to fix the encoders settings. It seems clear that iTunes AAC, LAME MP3, Vorbis are perfectly comparable with VBR at similar bitrate. If Nero Digital AAC is going to be include in an ABR mode (following Ivan's decision), the bitrate shouldn't be a problem either.
We have still a problem with WMA VBR. The current datas are contradictory. We don't know if we have to make do with 2-pass mode. If 2-pass will appear to be necessary, then it will have a big impact on the next step: the choice of samples. Why? Because we can't simply encode last-year samples in two pass mode, but we need to collect the full tracks first, then encode them and finally cut the right part. We're not sure to contact each sample's submitter. Consequently, if WMA 2-pass will be used, it would probably better to build a completely new sample gallery.
Once the choice of settings made, we need to debate about samples. Should we keep last year' samples? Is it simply conceivable (cf. WMA 2-pass issues)? Should we include critical samples or keep the gallery free of them? Should we be careful about the bitrate distribution of the gallery and take care to use as many samples with high bitrate than samples with low bitrate? Should we discard samples revealing abnormal behaviour with one encoder (i.e 250 kbps when all others are close to 130)? Should we stay with 30 seconds samples?
Once choice of samples ready, we could still ask some question: should we suggest to testers to not play with the volume knob during the test (
controled volume condition)? Should we recommand
ABX tests in each case or not? Shouldn't we think about creating a something like a
quick-start guide for newbies, explaining how to set ABC/HR but also
how to react during the test (not trying to lower a slider for each par, be careful about ranking, when ABX are necessary and how does it work)? When I discussed on french forums about the previous collective tests, many interersted people were completely lost and confused with ABCHR and ABX, marks and ABX scores... I supposed that some of them gave up or sent results which were finally discarded (example: peoples only perfomed (successful) ABX tests and wasn't aware that grading each encoder was also necessary).
That's what I suggest as roadmap for the next days. What do other people think about it?
For those interested:
Reliable sources within Sony that have given me consistant and 100% accurate information in the past, recent documents that have been passed along my way, and the blantant obviousness of Sony's direction from not only the recent webcast along with their Annual Report for 2005 and other various public presentations have given a clear indication as to what's going to occur to the format. Right now there's a very slim chance that Hi-MD may even see a third generation, and if it were, it would probably be the definitive last. I'm not so sure anymore that the Connect Player will even support MD. There's something else on the horizon now, a unified approach. I can't say any more than this, but it will boil down to one of these scenarios, trust me: (I) Minidisc is no longer developed, focus on other products and introduce similar functionality. (II) Put a radical spin on MD (Video, DVR, etc) and see what happens.
Regardless, fifteen years is a damn short product lifecycle. If they had played the market a little more efficently and not fallen into the deep hole that Sonicstage was, it would've been different. But alas, such is life, such is Sony.
Link: minidisc.org (http://forums.minidisc.org/index.php?showtopic=12392)
EDIT: this is a post by kurisu, one of the minidisc.org admins, and this is the first reply:
With all the different people saying MD was on its way out, I never really gave it much credibility since somebody's ALWAYS saying MD is dead. When I hear news like this from you, I know it's not a good day.
I just hope they choose to revitalize MD/Hi-MD
As a sidenote I want to add, that a proposed test like this with popular portable formats is very good, eg. to get each year a "market" overview about the approached qualities which hopefully progress to the better side.
(Though, HA is the place, where progress is shown to public besides commercial developments. If we have advanced formats like lame, mpc, ogg, wavpack (lossy hybrid), then it is up to us, to compare them publicly with the more or less commercial popular formats).
Though, I want to emphasize, that the lot of work with these tests should have 2 results:
1. ranking of current encoder versions/formats
2. ranking or possibility to compare current encoder versions/formats with older/other/unchanged encoder versions/formats.
To get a possibility to compare the new encoders with older ones, to see the progress, it is needed, that a stable/old anchor encoder is added to new tests.
So, for this test, the mpc version tested in the other or previous 128k multiformat tests comes to my mind, because the encoder hasn't changed much recently, and has obviously certain value from past, present & future.
Or is there another encoder in this currently proposed test setup, which could be seen as anchor to get a connection to the older tests ?
Of course, if too much formats are tested, the work will increase too much. Decisions are up to the conductor.
Of course, if too much formats are tested, the work will increase too much. Decisions are up to the conductor.
[a href="index.php?act=findpost&pid=344899"][{POST_SNAPBACK}][/a]
The problem is here. Using MPC as reference for comparison would suppose either an increase of the whole difficulty of the exercise or to discard one of the current competitor.
The first solution is not a good one in my opinion. Especially at this bitrate and if we expect from many people to take part to this test.
The second solution is conceivable but it would suppose new debates about what necessary is and what could be discarded; moreover, if MPC is finally included in the test other people could complain about the exclusion of atrac3plus or WMAPro and could be tempted to try to argue in order to include them at the last minute. Again, debate may be endless.
We have still a problem with WMA VBR. The current datas are contradictory. We don't know if we have to make do with 2-pass mode. If 2-pass will appear to be necessary, then it will have a big impact on the next step: the choice of samples. Why? Because we can't simply encode last-year samples in two pass mode, but we need to collect the full tracks first, then encode them and finally cut the right part. We're not sure to contact each sample's submitter. Consequently, if WMA 2-pass will be used, it would probably better to build a completely new sample gallery.[a href="index.php?act=findpost&pid=344874"][{POST_SNAPBACK}][/a]
I suppose WMA 2-pass 128 kbps is the only possible WMA standard setting. On yesterday I posted a new table of my 25 various track set, which produces a nice 130-135 kbps overall average with the other codecs. WMA 50 resulted about 100 kbps and WMA 75 resulted over 155 kbps.
I think the following procedure would be a possible solution for making the conditions equal for all codecs without having to hunt the original full tracks.
1. A "standard" audio track should be made by combining small varied parts from several audio tracks. The standard track would produce identical average bitrates with all codecs. It could be e.g. exactly 149 seconds long.
2. The test sample should be combined like this with the standard audio track:
standard track (149 s) _ digital silence (1 s) _ test sample (5-30s) _ digital silence (1 s) _ standard track (149 s)3. After encoding the file is decompressed and exactly 150 s is removed from the beginning and the end.
4. The resulting sample is saved in a lossless format.
I am not sure if the one second digital silence is needed at the joint points. At least it would make easier to see the correct position in a wave editor. I suppose it would also give the encoders free space for using the bit reservoir systems at the beginning and the end of the samples.
This procedure would minimize the possible effect of the missing track parts and actually make the comparison more equal because the missing parts could contain anything from very simple to very complex data.
I think this should be done for all codecs unless the codec developers confirm that encoding the complete track would produce an identical sample passage with the separately encoded short sample file.
I think the following procedure would be a possible solution for making the conditions equal for all codecs without having to hunt the original full tracks.
[a href="index.php?act=findpost&pid=344905"][{POST_SNAPBACK}][/a]
The conditions may be eaqual but really far to be representative. It looks needlessly complicated IMO. I understand the reasons of this ("confirm that encoding the complete track would produce an identical sample passage with the separately encoded short sample file") but I don't believe that significant difference may occur between an encoding coming from one short sample and an encoding extracted from the full track. There are probably differences (frames boundaries are not identical), but they should be minor ones (as minor as the impact of additionnal samples introduced by the drive's offset during ripping).
The only possible problem concern 2-pass encoding. We don't exactly know how it works, and how the bitrate is distributed. If the distribution on the second part is working like video encoders are working, then we 1/ can't use short samples; 2/ need to use the complete tracks, not an artificial substitute. For the sake of equity, we can't take the risk to handicap one competitor
4. The resulting sample is saved in a lossless format.
Then we would get 7 lossless files in each archive (6 contenders + reference). Some archives will reach 20 MB. The full test package will probably exceed 300 MB. DSL internet access are very common, but not systematical.
The conditions may be eaqual but really far to be representative. It looks needlessly complicated IMO. I understand the reasons of this ("confirm that encoding the complete track would produce an identical sample passage with the separately encoded short sample file") but I don't believe that significant difference may occur between an encoding coming from one short sample and an encoding extracted from the full track. There are probably differences (frames boundaries are not identical), but they should be minor ones (as minor as the impact of additionnal samples introduced by the drive's offset during ripping).
The only possible problem concern 2-pass encoding. We don't exactly know how it works, and how the bitrate is distributed. If the distribution on the second part is working like video encoders are working, then we 1/ can't use short samples; 2/ need to use the complete tracks, not an artificial substitute. For the sake of equity, we can't take the risk to handicap one competitor
I agree mostly. I suppose it is quite safe to encode short samples with LAME and Vorbis. But: Ivan wasn't sure about Nero when I asked this (at least not yet). Do we know how QT AAC works? (Perhaps I could try to test it.) I really tried to help with WMA 2-pass, but I've failed to analyze it. - Anyone willing to contact Microsoft?
Then we would get 7 lossless files in each archive (6 contenders + reference). Some archives will reach 20 MB. The full test package will probably exceed 300 MB. DSL internet access are very common, but not systematical.
[a href="index.php?act=findpost&pid=344910"][{POST_SNAPBACK}][/a]
It could be a problem, I admit. How big, I don't know. Naturally, it could be a bandwidth problem for the test organizers, but for file distribution P2P networks can be used and recommended.
Would it really be a problem for testers? I downloaded the 101 MB Nero 7 installer just for checking the bitrates for this test. Windows XP SP2 update is 260 MB. Many people can access a faster connection at work or by asking a friend to help, even the home connection is a dial-up. Perhaps a poll could tell the answer.
Would it really be a problem for testers? I downloaded the 101 MB Nero 7 installer just for checking the bitrates for this test. Windows XP SP2 update is 260 MB. Many people can access a faster connection at work or by asking a friend to help, even the home connection is a dial-up. Perhaps a poll could tell the answer.
[a href="index.php?act=findpost&pid=344928"][{POST_SNAPBACK}][/a]
I agree. I think anyone interested enough to sit through the test is, if he/she doesn't have DSL at home, probably interested enough to get the package through work or school.
I think people on dial-up are used to doing that kind of workaround.
I think the following procedure would be a possible solution for making the conditions equal for all codecs without having to hunt the original full tracks.
1. A "standard" audio track should be made by combining small varied parts from several audio tracks. The standard track would produce identical average bitrates with all codecs. It could be e.g. exactly 149 seconds long.
2. The test sample should be combined like this with the standard audio track:
standard track (149 s) _ digital silence (1 s) _ test sample (5-30s) _ digital silence (1 s) _ standard track (149 s)
3. After encoding the file is decompressed and exactly 150 s is removed from the beginning and the end.
4. The resulting sample is saved in a lossless format.
[a href="index.php?act=findpost&pid=344905"][{POST_SNAPBACK}][/a]
Thats seems like a bloody good idea. The whole process could be automated keeping it relatively easy to prepare samples for WMA.
WMA should definitely be able to benefit from such a scheme, if WMA find the sample difficult.
Whether the resulting bitrate will match or even get closer to the ~130 kbps target remains to be seen.
The fact that the size of the test package increases does not concern me.
I'm not happy with the WMA 2-pass solution. If we can't achieve a similar bitrate for WMA files... So what? It's only Microsoft's fault that they don't offer more encoding options. Is it fair that we artificially try to boost a codecs performance for the test? I strongly believe the test should produce results applicable to the real world behaviour. The fact is, nobody encodes to 2-pass WMA. And if the only WMA VBR setting that comes close to 128kbps produces undersized results... Well, so be it.
For the sake of equity, we can't take the risk to handicap one competitor
Although I understand you desire for testing all contenders at equal conditions, sometimes practical issues forces you to make a compromise.
Do you agree that what Alex B would be a benefit to WMA or that it may even be to much of a benefit?
Then we would get 7 lossless files in each archive (6 contenders + reference). Some archives will reach 20 MB. The full test package will probably exceed 300 MB. DSL internet access are very common, but not systematical.
[a href="index.php?act=findpost&pid=344910"][{POST_SNAPBACK}][/a]
Why go totally lossless since only the WMA tracks would have to be lossless to make the scheme work?
Then we would get 7 lossless files in each archive (6 contenders + reference). Some archives will reach 20 MB. The full test package will probably exceed 300 MB. DSL internet access are very common, but not systematical.
[a href="index.php?act=findpost&pid=344910"][{POST_SNAPBACK}][/a]
Why go totally lossless since only the WMA tracks would have to be lossless to make the scheme work?
[a href="index.php?act=findpost&pid=344964"][{POST_SNAPBACK}][/a]
Yep, and for 18 samples, I guess the file size of the decompressed WMAs is going to be around 50 MB (losslessly encoded). AAC, Vorbis and MP3 can be decloded on the tester's PC so the files can be distributed as-is. So the whole package will take something like 150 MB or even less if I am not mistaken.
Is it fair that we artificially try to boost a codecs performance for the test?
[a href="index.php?act=findpost&pid=344961"][{POST_SNAPBACK}][/a]
We're trying to use each encoder with optimal setting. It's the most common attitude here on HA.org since four years
I strongly believe the test should produce results applicable to the real world behaviour.
Why not. Then MP3 CBR 128 with Franhofer fastenc vs WMA CBR 128 and maybe iTunes CBR 128 should be the close to the ideal testing configuration.
The fact is, nobody encodes to 2-pass WMA.
Probably not a majority.
The fact is, that WMA don't have to be handicaped in this test by using what people are using most commonly (which often correspond to less efficient solutions). If we don't handicap the other contenders, then WMA don't have to be treated unfairly.
Do you agree that what Alex B would be a benefit to WMA or that it may even be to much of a benefit?
[a href="index.php?act=findpost&pid=344964"][{POST_SNAPBACK}][/a]
If WMA 2-pass is working like common video encoders, then the bitrate allocated to the desired samples will entirely depend on the content and the complexity of the full track. There's no way to simulate it by creating an artificial musical content. The Debussy sample and NewYorkCity are very different. The bitrate distribution of these samples can't be the same for the 2nd pass if the first pass is computing the complexity from the real full track instead of an artificial sequence.
We can't extract a quiet opera sample or a strong metal passage and graft it into an external and artificial context.
The fact is, nobody encodes to 2-pass WMA.
Probably not a majority.
The fact is, that WMA don't have to be handicaped in this test by using what people are using most commonly (which often correspond to less efficient solutions). If we don't handicap the other contenders, then WMA don't have to be treated unfairly.
[a href="index.php?act=findpost&pid=344971"][{POST_SNAPBACK}][/a]
You constantly contradict yourself Guru.
So maybe we should just remove WMA from the test, since it can't be compared fairly?
You constantly contradict yourself Guru.
[a href="index.php?act=findpost&pid=344976"][{POST_SNAPBACK}][/a]
So maybe we should just remove WMA from the test, since it can't be compared fairly?
I'm totally in favour of this solution.
So maybe we should just remove WMA from the test, since it can't be compared fairly?
I'm totally in favour of this solution.
[a href="index.php?act=findpost&pid=344977"][{POST_SNAPBACK}][/a]
If the only option is to use WMA in a 2-pass mode, then I favour it too.
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
Did we scrap the idea of a completely new set of samples?
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
Did we scrap the idea of a completely new set of samples?
[a href="index.php?act=findpost&pid=344982"][{POST_SNAPBACK}][/a]
I'm sure some of the tracks can be gotten from the original submitters. There doesn't need to be an entirely new set of samples.
WMA should definitely be able to benefit from such a scheme, if WMA find the sample difficult.
Otherwise when the other encoders select something like 150-175 kbps the WMA encoder in 2-pass mode is forced to use 128 kbps. It would not be able to show it's potential. The test would be fair for WMA 2-pass only with samples extracted from complete tracks that are similarly difficult from the beginning to the end.
On the other hand with easy samples WMA 2-pass would possibly benefit from the situation. However, I suppose most of the samples should be somehow difficult to encode for getting meaningful results. As we already know, these new encoders are quite good.
Whether the resulting bitrate will match or even get closer to the ~130 kbps target remains to be seen.[a href="index.php?act=findpost&pid=344960"][{POST_SNAPBACK}][/a]
The procedure i suggested has at least a problem with this (besides the problem guruboolez explained). Without a way to analyze the dynamic WMA bitrates we will never know what average bitrate 2-pass WMA used for the sample passage. I hope the codec experts at HA can provide a solution to this dilemma.
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
This doesn't change the fact that encoding to 2-pass WMA is pretty much impossible with popular tools. You can compare it to WMA Pro. Not even WMP is supporting those two options.
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
Did we scrap the idea of a completely new set of samples?
[a href="index.php?act=findpost&pid=344982"][{POST_SNAPBACK}][/a]
I'm sure some of the tracks can be gotten from the original submitters. There doesn't need to be an entirely new set of samples.
[a href="index.php?act=findpost&pid=344983"][{POST_SNAPBACK}][/a]
In the worst case we could buy a couple of CDs if needed. So the sample selection would not be depended on the availability of the tracks. I can offer to buy one.
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
Also, why limit it to a single track? To be COMPLETELY fair, shouldn't we encode the whole album?
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
Also, why limit it to a single track? To be COMPLETELY fair, shouldn't we encode the whole album?
[a href="index.php?act=findpost&pid=344991"][{POST_SNAPBACK}][/a]
Depends on how the encoder is set up to encode albums. I assume it is set up track by track, but if not...
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
This doesn't change the fact that encoding to 2-pass WMA is pretty much impossible with popular tools. You can compare it to WMA Pro. Not even WMP is supporting those two options.
[a href="index.php?act=findpost&pid=344989"][{POST_SNAPBACK}][/a]
True. Sad but true. But these files are complient with WMA standard. As we said, popularity is not a criterion; portability and VBR are. There's no problem here.
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
Also, why limit it to a single track? To be COMPLETELY fair, shouldn't we encode the whole album?
[a href="index.php?act=findpost&pid=344991"][{POST_SNAPBACK}][/a]
Because the bitrate is distributed on second pass according to data computed during the first one (at least with video encoders). And the first pass is about one track, not the full album.
edit: ff123 was faster.
I suppose the possible WMA choices are these:
WMA standard VBR 50 = 100 kbps (less with classical)
WMA standard VBR 75 = 156 kbps (less with classical)
WMA standard VBR 2-pass = 128 kbps
WMA standard CBR = 128 kbps
WMA Pro VBR 50 = 134 kbps (probably less with classical)
No WMA
[span style='font-size:7pt;line-height:100%'](median average bit rates according to the my test tracks)[/span]
Actually, I am not personally interested about WMA. WMA Pro VBR would be the most interesting because it has not been tested and because of the claimed better quality. It would also have the correct bitrate.
When it comes to the availability of the tools, other tools besides WMP are available and HA can educate people. Remember Musepack? Somehow people who were interested found the new tools and learned to use them.
In the worst case we could buy a couple of CDs if needed. So the sample selection would not be depended on the availability of the tracks. I can offer to buy one.
[a href="index.php?act=findpost&pid=344990"][{POST_SNAPBACK}][/a]
I sent to Roberto:
Hongroise,
Mahler and
Debussy samples used during last listening test at 128 kbps. I could maybe be in possession of the CD corresponding to the Bartok one (The Julliard Stringquartet has recorded three times the full set of the hungarian composer, and I have two of them).
If WMA std is so difficult, why not just test WMA Pro instead? It would sure save a lot of time and clearly provide the most interesting results.. (no whining from the wma zealots is also a good thing)
Average bitrate for WMAPro with classical music: 118,72 kbps
The bitrate table (from A01 to V30):
142,1
124,4
178,8
159,9
91,3
56,6
125,6
144,5
84,7
130,6
123,5
86,9
105,5
123,3
118,3
144,5
90,1
130,4
122,6
118,5
142,9
148,3
118,7
89,8
45,9
104,8
152,9
108,7
89,9
142,3
126,3
112,3
152,6
103,8
145,8
124,7
124,8
126,0
116,1
112,5
130,8
99,3
121,1
117,7
108,1
104,5
101,5
125,4
105,1
125,4
132,6
138,4
123,3
149,0
150,4
143,0
130,4
142,4
145,2
138,1
137,9
138,6
145,1
119,4
145,3
110,6
113,2
100,7
125,5
129,2
84,7
121,0
122,8
96,6
120,5
99,2
100,2
117,7
129,2
115,7
90,4
149,2
113,4
98,9
140,9
134,9
100,9
129,4
104,4
121,6
69,0
66,7
65,0
84,7
56,1
56,1
110,9
114,6
121,5
120,9
135,9
149,5
95,9
170,2
120,8
109,7
150,3
95,1
63,9
101,0
114,3
93,1
107,4
143,5
124,9
60,5
54,8
60,7
133,5
156,3
33,3
143,4
142,0
162,2
164,1
77,4
134,6
131,2
122,1
133,2
136,4
160,3
125,1
130,5
84,0
135,9
150,4
128,2
124,1
123,2
111,5
132,0
143,7
114,9
143,5
116,9
151,1
129,1
_____
118,72
EDIT: the table is about 148 tracks instead of 150 (two complete files were missing when I made the encoding during this summer).
True. Sad but true. But these files are complient with WMA standard. As we said, popularity is not a criterion; portability and VBR are. There's no problem here.
[a href="index.php?act=findpost&pid=344995"][{POST_SNAPBACK}][/a]
I seem to remember Sebastian saying something like 'popular formats, but with the best settings'. (I.e. not necessarily the most common settings...)
I think the 2-pass mode would be a good and also interesting choice after all. The resulting file format is compatible with everything and it is a new untested option. The standard WMA 9.1 encoder can make 2-pass files, only proper UI software is needed. If the results happen to be good it would be a useful format when a predictable file size is needed.
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
Also, why limit it to a single track? To be COMPLETELY fair, shouldn't we encode the whole album?
[a href="index.php?act=findpost&pid=344991"][{POST_SNAPBACK}][/a]
Depends on how the encoder is set up to encode albums. I assume it is set up track by track, but if not...
[a href="index.php?act=findpost&pid=344994"][{POST_SNAPBACK}][/a]
Yes, but we are aiming at a target bitrate of 128kbps with general music. Not just one track. One single track can be hard to encode, or it can be an easy to encode track for the encoder. It does not represent the encoding difficulty of the whole CD, nor the music generally. What I mean is that, a VBR encode of that track can result in a bitrate of way above the 128kbps mark. Yet the whole CD encoded in the VBR mode can still result in 128kbps average.
wma 2 pass is a fair comparison as long as the entire track is encoded (real-world usage) and the sample is extracted from the decoded wav.
[a href="index.php?act=findpost&pid=344981"][{POST_SNAPBACK}][/a]
This doesn't change the fact that encoding to 2-pass WMA is pretty much impossible with popular tools. You can compare it to WMA Pro. Not even WMP is supporting those two options.
[a href="index.php?act=findpost&pid=344989"][{POST_SNAPBACK}][/a]
True. Sad but true. But these files are complient with WMA standard. As we said, popularity is not a criterion; portability and VBR are. There's no problem here.
[a href="index.php?act=findpost&pid=344995"][{POST_SNAPBACK}][/a]
Just a few posts before, you claimed to ErikS that VBR is not a criterion. Make up your mind.
Again this whole discussion is just yet another proof that we are going into this test completely unprepared.
I believe that tha basic question is: What is the advantage of 2-pass encoding for music? I see none. So why not use standard 1-pass WMA compression and end this debate?
2-pass encoding of movies makes sense, because you want to fit you movie on a CD/DVD. So you need an exact size. But you don't need it for audio tracks.
Yes, but we are aiming at a target bitrate of 128kbps with general music. Not just one track.
[a href="index.php?act=findpost&pid=345015"][{POST_SNAPBACK}][/a]
WMA VBR-2pass = 128 kbps for each musical genre, each tracks, each album. Whatever the complexity, the loudness or anything else. So where's the problem?
Just a few posts before, you claimed to ErikS that VBR is not a criterion. Make up your mind.
Could you give me a link?
I believe that tha basic question is: What is the advantage of 2-pass encoding for music? I see none. So why not use standard 1-pass WMA compression and end this debate?
[a href="index.php?act=findpost&pid=345018"][{POST_SNAPBACK}][/a]
A more efficient way to encode music at a given bitrate. Predictible size and a more efficient bitrate distribution. For a controlled bitrate, ABR is supposed to be better than CBR and VBR 2-pass is supposed to be better than ABR.
I don't think anyone is going to use WMA 2-pass at 128 kbps for compressing CDA disc image files when archiving.
Besides separate tracks it might be good for video soundtracks when predictable file size is needed.
After all it might not suffer or get advantage too much with the test samples. Previously LAME ABR has been quite good against VBR codecs. I don't know how many frames LAME ABR tests before it makes the bitrate decisions, but I suppose the number is small when compared with the sample durations.
Yes, but we are aiming at a target bitrate of 128kbps with general music. Not just one track.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345015")
WMA VBR-2pass = 128 kbps for each musical genre, each tracks, each album. Whatever the complexity, the loudness or anything else. So where's the problem?
[a href="index.php?act=findpost&pid=345019"][{POST_SNAPBACK}][/a]
You fail to understand how 2-pass works. Reread what others have said before and what the problem with 2-pass encoding is.
Just a few posts before, you claimed to ErikS that VBR is not a criterion. Make up your mind.
Could you give me a link?
[a href="index.php?act=findpost&pid=345019"][{POST_SNAPBACK}][/a]
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=344483]http://www.hydrogenaudio.org/forums/index....ndpost&p=344483[/url]
You fail to understand how 2-pass works. Reread what others have said before and what the problem with 2-pass encoding is.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345023")
I don't understand. Could someone explain me what Gambit is saying?
Just a few posts before, you claimed to ErikS that VBR is not a criterion. Make up your mind.
Could you give me a link?
[a href="index.php?act=findpost&pid=345019"][{POST_SNAPBACK}][/a]
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=344483]http://www.hydrogenaudio.org/forums/index....ndpost&p=344483[/url]
VBR is a criterion
for this test we (or rather I) proposed
to avoid further debate about other audio formats (especially about atrac3 vs WMA). But it's not an absolute one. We can discard this criterion if needed
(and only if needed). But once a criterion is selected, there's no need to come and discuss again about formats that don't fit this criterion. It's simple like that.
If we don't precise the criterions of choice, then the debate is going to be endless and WMA 12 four-pass will be released before the test start.
Actually, I guess no one here knows how WMA 2-pass works.
I would like to ask Microsoft this question:
"I have a 1 h 30 min movie soundtrack. It is mostly very quiet and includes only spoken dialog, but it has also a 15-minute loud battle scene, which is full of difficult to compress sound. Can WMA 2-pass analyze the complete soundtrack and allocate the maximum available bit rate for the whole 15-minute battle scene passage? What would be the average bitrate for the battle scene passage in case the used setting is WMA standard VBR 2-pass 128 kbps?"
Does anyone know if MS has a user forum for this kind of questions?
I also fail to see the problem with using wma 2-pass on a per-track basis, and then extracting the sample later. To me, it's like a form of ABR. Sure, some people might use it on an album basis, but others might not.
ff123
Actually, I guess no one here knows how WMA 2-pass works.
One person visiting this forum comes to my mind, but I am not sure if he is willing to reply
I also fail to see the problem with using wma 2-pass on a per-track basis, and then extracting the sample later. To me, it's like a form of ABR. Sure, some people might use it on an album basis, but others might not.
i agree. a song is the minimum unit encodeable (at least normally).
I suppose the possible WMA choices are these:
WMA standard VBR 50 = 100 kbps (less with classical)
WMA standard VBR 75 = 156 kbps (less with classical)
WMA standard VBR 2-pass = 128 kbps
WMA standard CBR = 128 kbps
WMA Pro VBR 50 = 134 kbps (probably less with classical)
No WMA
[a href="index.php?act=findpost&pid=345001"][{POST_SNAPBACK}][/a]
Even if unfair, maybe VBR 75 would be interesting. I say this because it is probably true that VBR 50 would be worst compared to the others, so giving it an advantage, we could see how it compares.
Problem with 2-pass is about getting the bitrate usage of the part used in the test, and the mentioned problem about full length samples.
Getting quality on portables, with a reasonable size is our aim. I suppose everyone would choose better quality than a few more files. ( ok, this is a "2 for 3" difference but still... )
OK then, let me explain. I use an example, as this will make it most obvious.
You have a CD. Let's say track 3 is the one from which we will use 30 seconds as a sample.
Now we have three possible scenarios:
1) A WMA VBR encoding of that album results in an average bitrate of 128kbps.
2)A 2-pass WMA encoding of the whole album in a single file, will of course, result in a 128kbps file.
3)A per track encoding will result in 128kbps too, of course.
Now the problem is, that while the quality of the 30 seconds sample from scenario 1 and 2 are comparable, 3 is not.
In a 2-pass mode the encoder allocates the bitrate for the 30 seconds sample after the first pass. This of course is dependant on the content of the rest of the data that makes up the file that includes the 30 seconds sample.
So if the 30 seconds sample is the most difficult part of the track and the rest of the track is easy to encode (like voice, for example), the encoder will allocate a high bitrate for the 30 difficult seconds that we will use as a sample. But if the whole track is difficult to encode, the 30 seconds sample will get a much lower bitrate. But again, if the whole track is hard to encode, and you encode the whole album in a 2 pass mode, the encoder can save more bits on other, easier to encode tracks.
Hmm, I'm not sure I explained it well (blame my non-native english)...
Even if unfair, maybe VBR 75 would be interesting. I say this because it is probably true that VBR 50 would be worst compared to the others, so giving it an advantage, we could see how it compares.[a href="index.php?act=findpost&pid=345041"][{POST_SNAPBACK}][/a]
It's an idea, but keep in mind that several people are reading the plots and the plots only. If WMA end at first place, a lot of people will interpret this as "WMA > AAC, Vorbis..." at the same bitrate.
It's not only unfair, it may have disastrous practical consequence.
The only solution would be to use VBR75 as high anchor (but we're not sure that this setting is good enough to play this role) and to let "high anchor" name in the bottom of the plot instead of "WMA" name.
156 kbps average is absolutely unacceptable IMHO - this is a huge difference, especially when taken into account that 128 kbps is a bit less than average Perceptual Entropy for average signals.
156 is usually in many times above the PE (energy / mask threshold) for average content.
This means that codec could be even imperfect (in terms of psymodel->noise allocation->coding chain) - but still give near-transparent results at such a high bit rate - while others would be crippled trying to shape obvious audible noise (that cannot go away) to reach the 128 kbps average level.
My vote is clerly against that.
Gambit, but what if the majority of the people encode individual tracks (instead of album images)? Wouldn't it make more sense to do what ff123 wants to do - encode on a per-track basis and then extract the sample?
Edit: BTW, I wouldn't want to use WMA at 156 kbps since that is way over the target bitrate of this test.
Gambit, but what if the majority of the people encode individual tracks (instead of album images)? Wouldn't it make more sense to do what ff123 wants to do - encode on a per-track basis and then extract the sample?
[a href="index.php?act=findpost&pid=345046"][{POST_SNAPBACK}][/a]
That's not the point.
I think I explained it bad.
Let me try like this:
Let's say an encoder has a problem encoding applause. If the applause is in a track that contains a vocal performance, the encoder will encode the applause without artifacts, because it can save bits on the voice. But if the applause is in a metal track, the encoder can't save bits on the rest of the track. So the applause will be encoded with artifacts.
So the problem with 2-pass is that the quality of the sample is highly dependand of the track that includes it.
So, really, 2-pass is only usable if you absolutely need to achieve a specific bitrate. Like encoding the audio for a 1CD movie. It is completely unsuitable for a VBR quality listening test. So we should either test WMA VBR50, even if it is often undersized, or completely omit WMA from the test. IMO, of course.
Edit: Or test all encoders in CBR mode, because that's what 2-pass WMA basically is. You don't target a quality level like with the other VBR encoders, but you target a bitrate.
Edit2: Sorry, I mean VBR50 not 75.
So the problem with 2-pass is that the quality of the sample is highly dependand of the track that includes it.
[a href="index.php?act=findpost&pid=345049"][{POST_SNAPBACK}][/a]
Exactly. But where's the problem? It has been requested in previous messages that the original contend of the track has to be tested to be representative of a real usage of the encoder with 2-pass encoding. As long as the obtained quality corresponds to the real usage of the encoder (i.e. cutted sample from a full encoding) there's no problem. The 30 sec sample would maybe sound differently if it was a part of a different content, but the fact is, that there's only two possible contents for one unique sample: the full track or the full album. Assuming that few peoples are using single files for listening to a full album without cue support or poor seeking abilities (there's rarely seekbar on portable players), it shouldn't be a problem to use single track as basis for the distribution.
So, really, 2-pass is only usable if you absolutely need to achieve a specific bitrate.
[a href="index.php?act=findpost&pid=345053"][{POST_SNAPBACK}][/a]
And this is also what we're asking to AAC VBR, MP3 VBR, Vorbis VBR in this test. The only difference is that these other encoders have all freedom to use the bitrate they need for each tracks. But on a full library (our bitrate estimation is based on that criterion and nothing else), all encoders must achieve a specific bitrate. For WMA it will be 128 kbps, for the other it will be 133 (iTunes) or 134 (LAME).
So we should either test WMA VBR75, even if it is often undersized, or completely omit WMA from the test. IMO, of course.
The first choice is irrelevant for a ~130 kbps listening test (exepted if it's based on classical music only...).
The second one is a working solution. But it won't satisfy most users interested to see how WMA perform.
Or test all encoders in CBR mode
I hope it's a joke...
because that's what 2-pass WMA basically is
Is it too hard to understand that the bitrate is varying within the track with VBR 2-pass but not (or just a bit) with CBR? That's the main difference between VBR (V for variable bitrate) and CBR (C for constant bitrate).
You still fail to see the basic difference between VBR and 2-pass encoding Guru.
With VBR, you target QUALITY. Size differes, quality remains.
With 2-pass you target a BITRATE. Size remains, quality differes.
That's why you don't get usable results with this test for WMA 2-pass. You know that encoders can have problems with certain types of sounds. With 2-pass, the encoder can be lucky and encode it well in one track, but encode it with with terrible artifacts in a different tracks. The VBR encoders will encode that sound always the same, regardless of the rest of the track.
I asked my question for Microsoft here:
http://www.microsoft.com/communities/newsg...8&lang=en&cr=US (http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.windowsmedia.encoder.optimization&cat=en_US_41d2b838-4978-4ff8-876a-1f8abe468de8&lang=en&cr=US)
"Subject: How capable the Windows Media Audio 2-pass VBR mode is?"
You still fail to see the basic difference between VBR and 2-pass encoding Guru.
[a href="index.php?act=findpost&pid=345057"][{POST_SNAPBACK}][/a]
You're talking about VBR 2-pass being "basically CBR" and when I answered to this you're saying that I'm clueless on this subject. Do you really think that I'm an ass about audio coding?
With 2-pass you target a BITRATE. Size remains, quality differes.
And because size is predictible you're calling it CBR! That's the point and this point only I contest. CBR doesn't mean predictible size, but constant bitrate (with small variations due to bitrate reservoir) among the complete encoding. Nothing else. Variable bitrate with predictible size is the purpose of 2 pass encodings (same for ABR) and can't certainly be named CBR.
You know that encoders can have problems with certain types of sounds. With 2-pass, the encoder can be lucky and encode it well in one track, but encode it with with terrible artifacts in a different tracks. The VBR encoders will encode that sound always the same, regardless of the rest of the track.
True
(if and still only if audio 2-pass is working like video 2-pass). But VBR 2-pass is the closest setting for WMA we have. VBR is unusable; there's no ABR; CBR is clearly worse than 2-pass. We're not using 2 pass encoding for our own pleasure, but to find the most acceptable solution. And comparing one-pass VBR encodings to two-pass VBR encodings is simply more acceptable and fair for WMA than testing VBR to CBR (i.e. what we do in most previous tests).
there's no ABR
[a href="index.php?act=findpost&pid=345064"][{POST_SNAPBACK}][/a]
Maybe I am mistaken, but I see the 2-pass VBR WMA is offering as some sort of ABR since the bitrate remains pretty constant with only small fluctuations.
You still fail to see the basic difference between VBR and 2-pass encoding Guru.
[a href="index.php?act=findpost&pid=345057"][{POST_SNAPBACK}][/a]
You're talking about VBR 2-pass being "basically CBR" and when I answered to this you're saying that I'm clueless on this subject. Do you really think that I'm an ass about audio coding?
With 2-pass you target a BITRATE. Size remains, quality differes.
And because size is predictible you're calling it CBR! That's the point and this point only I contest. CBR doesn't mean predictible size, but constant bitrate (with small variations due to bitrate reservoir) among the complete encoding. Nothing else. Variable bitrate with predictible size is the purpose of 2 pass encodings (same for ABR) and can't certainly be named CBR.
You know that encoders can have problems with certain types of sounds. With 2-pass, the encoder can be lucky and encode it well in one track, but encode it with with terrible artifacts in a different tracks. The VBR encoders will encode that sound always the same, regardless of the rest of the track.
True (if and still only if audio 2-pass is working like video 2-pass). But VBR 2-pass is the closest setting for WMA we have. VBR is unusable; there's no ABR; CBR is clearly worse than 2-pass. We're not using 2 pass encoding for our own pleasure, but to find the most acceptable solution. And comparing one-pass VBR encodings to two-pass VBR encodings is simply more acceptable and fair for WMA than testing VBR to CBR (i.e. what we do in most previous tests).
[a href="index.php?act=findpost&pid=345064"][{POST_SNAPBACK}][/a]
Well said. ;-)
Gambit - the rest of us seem to be on the train already. Are you getting on, too?
there's no ABR
[a href="index.php?act=findpost&pid=345064"][{POST_SNAPBACK}][/a]
Maybe I am mistaken, but I see the 2-pass VBR WMA is offering as some sort of ABR since the bitrate remains pretty constant with only small fluctuations.
[a href="index.php?act=findpost&pid=345066"][{POST_SNAPBACK}][/a]
ABR is technically different from 2-Pass VBR. Both are intended to reach a target bitrate with bitrate variations, but ABR don't have the same freedom than 2-pass. In video encoding, ABR is different from n-pass encoding (even if they share the same purpose for the end-user: predictible size and variable bitrate allocation according to the computed complexity).
We can't mix up all these concept. Same goes for AAC CBR: just because people could see some variation (bitrate reservoir) some are prompt to conclude that CBR correspond to ABR/VBR. From a technical point of vue, it isn't true (Ivan may recall the difference between CBR, ABR and VBR for AAC).
Therefore, WMA [Standard] offers CBR, VBR and VBR 2-pass.
No one has yet been able to answer how the wma 2-pass system works (possibly except one who cannot because of his work position). Is it really like a 2-pass video codec?
How about LAME ABR? I would very much like to know how long is the passage used for calculations. And a further question: has the passage a constant length or does it vary according to some factors. I searched for an answer a few months ago without results. Then I had a real-life situation with a video soundtrack. At first I was going to use LAME ABR, but I realized that finding a correct VBR setting would be better.
[span style='font-size:7pt;line-height:100%']Edit: typo[/span]
Gambit, I see your point. A track which is consistently complex throughout (which might be encoded at much higher than 128 average with a true VBR codec) would be penalized with wma 2-pass. In such a track its operation is closer to CBR than to VBR.
Similarly, it would have an advantage on tracks which are consistently easy to encode.
The situation would be neutral if the track is of average complexity.
However, on the plus side, it mimicks real-world usage, and of the wma options we have, it is probably the best compromise, assuming the true VBR settings do not give us good average bitrates.
ff123
156 kbps average is absolutely unacceptable IMHO - this is a huge difference, [a href="index.php?act=findpost&pid=345045"][{POST_SNAPBACK}][/a]
I wouldn't want to use WMA at 156 kbps since that is way over the target bitrate of this test.
[a href="index.php?act=findpost&pid=345046"][{POST_SNAPBACK}][/a]
This test is more about 130kbps, 156-130 = 26kbps , 130-100 = 30kbps , I see it is pretty much in the middle anyway.
Yet, I agree this can give an incorrect image of the codec efficiency. If we find a way to correctly test 2-pass, I'm ok.
You're talking about VBR 2-pass being "basically CBR" and when I answered to this you're saying that I'm clueless on this subject. Do you really think that I'm an ass about audio coding?
[a href="index.php?act=findpost&pid=345064"][{POST_SNAPBACK}][/a]
I'm sorry, of course not. We always get so heated in our debates. But with all respect, you are a bit dense and stubborn sometimes.
CBR doesn't mean predictible size,[a href="index.php?act=findpost&pid=345064"][{POST_SNAPBACK}][/a]
It means exactly that.
And comparing one-pass VBR encodings to two-pass VBR encodings is simply more acceptable and fair for WMA than testing VBR to CBR (i.e. what we do in most previous tests).
[a href="index.php?act=findpost&pid=345064"][{POST_SNAPBACK}][/a]
I'm not so sure. And I would rather favour testing CBR WMA than 2-pass WMA. It might not be fair, but 2-pass is not completely fair either. Plus, with 2-pass you have the usability factor against it too. 2-pass WMA is almost as useless in a real world scenario as WMA Pro.
Gambit - the rest of us seem to be on the train already. Are you getting on, too?
[a href="index.php?act=findpost&pid=345067"][{POST_SNAPBACK}][/a]
I will be so nice and not reply to that...
AFAIK VBR 2-pass helps to achieve constant and the best possible quality of the whole sound item at a given file size (or given ABR which is the same). Its efficiency increases with the increase of the sound item length. Evidently the results of encoding in both modes pure VBR and VBR 2-pass will be the same if resulting file sizes (ABR) will be equal. So VBR 2-pass could be considered in listening tests as pure VBR mode (it just helps to choose appropriate quality parameter for VBR mode, automaticaly). The only advantage of VBR 2-pass is that the final ABR is predictable but in case of listening tests it doesn’t matter.
I'm not so sure. And I would rather favour testing CBR WMA than 2-pass WMA. It might not be fair, but 2-pass is not completely fair either.
[a href="index.php?act=findpost&pid=345083"][{POST_SNAPBACK}][/a]
True. But you know the adage: always choose the lesser evil. 2-pass is surely not as ideal than a usual VBR mode outputing ~130 kbps, but it's supposed to perform better than CBR 128 kbps. Assuming that 1-pass VBR doesn't fit the targeted bitrate, the second logical choice is 2-pass. Then would come ABR, and finally CBR as worst possible one.
2-pass WMA is almost as useless in a real world scenario as WMA Pro.
Why? If I had to use WMA std at 128 kbps, I would be more interested by 2-pass encoding than one simple CBR encoding. I recall the purpose of the test: trying to get the maximum quality for each competitor. VBR_50 and CBR_128 don't correspond to this idea.
Gambit - the rest of us seem to be on the train already. Are you getting on, too?
[a href="index.php?act=findpost&pid=345067"][{POST_SNAPBACK}][/a]
I will be so nice and not reply to that...
[a href="index.php?act=findpost&pid=345092"][{POST_SNAPBACK}][/a]
It's just that you kept repeatedly explaining to people who had already understood what you didn't seem to understand yourself.
WMA Pro VBR 50 = 134 kbps (probably less with classical)
[a href="index.php?act=findpost&pid=345001"][{POST_SNAPBACK}][/a]
I will have to agree with Gambit on the WMA 2-pass issue, at least if we assume that it works on principles similar to those of video 2-pass coding.
For all other codecs, we use a large amount of samples to estimate the average bitrate of the codec. Naturally, if we used WMA 2-pass on all those samples specifying a bitrate, we would be spot on, and if we extracted each track from the entire collection afterwards, we would probably have similart behaviour as the other codecs.
A first approximation, and an acceptable one imo would be to use it on an entire album and extract the samples afterwards. The approximation on the track level could be far to crude (however, we don't have any evidence), and it is obviously to crude for 30 seconds of a track.
As we don't know if the approximation would be too crude on the track level, and as it obviously is not possible to encode entire albums for thereafter to extract the tracks, it is not a good idea to use WMA 2-pass in this test (imo).
As I stated before, the primary concern is filesize. After all, the only purpose for a lossy audio codec is to make the file as small as possible while retaining reasonable quality. How an encoder does that, is the problem of that encoder. File size is measurable, quality is not. Hence: a listening test. Keep one factor constant, (or reasonably so) and test the other.
A small file is not a problem, that is _good_. We should just on a maximum bitrate, not a target bitrate. Just imagine if wma Q50 @ 100 kb/s would prove to be as good as other codecs @135 kb/s
As to the wma settings issue, 2-pass mode really is a double-edged sword. 2-pass does not equate to higher quality. It can, but just as easily it can mean less quality than you would have gotten with a vbr q-based setting. So I see no unfair advantage there. (and, again, even if wma would have an advantage, it is not wma's fault that other codecs do not have a 2-pass mode).
Anyway, why not ask somebody at microsoft or in their wma forum: "what is the best setting to use for wma std provided the constraining parameters are filesize and a widely-available encoder?"
Besides, the general concensus here at HA seems to be that wma is inferior in quality, maybe we should have it encode at Q75, just to prove the point (just joking for the humour-impaired, of course going above the set maximum bitrate would be cheating...)
So the problem with 2-pass is that the quality of the sample is highly dependand of the track that includes it.
[a href="index.php?act=findpost&pid=345049"][{POST_SNAPBACK}][/a]
Can't the same thing be said for a poorly tuned VBR or ABR system?
A certain sample that contains hard to encode sequences may fail to trigger a sufficient increase in frame size, thus resulting in artifacts. This could be considered a weakness of the encoder.
Alternatively, a poor ABR or VBR (or even quality presets) system could fail to reduce frame sizes on certain 'easy' parts of the tracks that result in wasting of bits thus restricting the potential for larger frame sizes for hard parts of the whole file.
If a different sample is selected that correctly triggers the VBR switching, then this defficiency will be avoided, or rather will remain un-noticed.
Hence I see this as being more related to what samples the test uses, rather than an explanation of why a 2-pass encoder should be automatically disqualified from this test.
For all other codecs, we use a large amount of samples to estimate the average bitrate of the codec.
[a href="index.php?act=findpost&pid=345154"][{POST_SNAPBACK}][/a]
And for WMA? You can use 1, 10 or 10000 albums, the average bitrate will be 128 kbps. There's simply no need to build such table with CBR or 2-pass encoders.
Moreover, we're not using samples for measuring the bitrate of other formats, but full tracks or even album. This is not exactly the same.
A first approximation, and an acceptable one imo would be to use it on an entire album and extract the samples afterwards.
An approximation of what?
We're not looking for an approximation, we're looking for a way to get the
exact signal a listener would get by encoding his music with WMA 2-pass. To do this, we have to encode either the full track or the full album in this mode, then we must decode it, and finally extracting from it the short sample. Now assuming that rare are the people using one monolithic encoding for encoding one album into a lossy and gapless format; assuming that even rarer are the people doing this for their portable players (which don't support .cue index and have most often minimal abilities in seeking) then the best way to get representative and accurate 2-pass samples is to encode each track (and not album) and then extracting the desired sample.
The only drawback is that we can't publish a bitrate table for the
short samples (because there's no way to our knowledge to see if WMA used 105 or 158 kbps for the extracted part) but only for the full tracks, which are all encoded at 128 kbps ±1 kbps. What's more, I against the idea of publishing a bitrate table for each sample if you could get the value for the full track.
Here's a suggestion for the 2-pass WMA problem:
Encode the samples using all the other encoders first, and then use the average bitrate from each sample as the target bitrate for 2-pass WMA (i.e. use different bitrate for each sample based on the average produced by the other encoders).
The fairness of this method would depend on how much variation there was between the bitrates produced by the other encoders. The less variation there is, the fairer this method is.
Yes I know it's not ideal, but neither are the other suggestions put forward so far. This method has the added advantage of being easier to implement compared to some of the other methods.
Here's a suggestion for the 2-pass WMA problem:
Encode the samples using all the other encoders first, and then use the average bitrate from each sample as the target bitrate for 2-pass WMA (i.e. use different bitrate for each sample based on the average produced by the other encoders).
[a href="index.php?act=findpost&pid=345199"][{POST_SNAPBACK}][/a]
There are two problems with this:
1/ it supposes that we'll use different setting for each sample with WMA. This solution wasn't retained for any other formats, and therefore it'll be problematic to accept this method for WMA only.
2/ more pragmatic: it's apparently not possible. WMA 2-pass is working for a limited choice of bitrate: 128 - 160 - 192 etc...
There would be no problem if there were more choices for the standard 1-pass VBR, settings like q65 or q60.
However, I have found means two split wma files losslessly. The results I got prove that 2-pass is a real 2-pass and indeed full tracks should be encoded. The possibility to split files makes also possible to check the approximate average bitrate by checking the file size vs duration. It is not exact science, because the splitter tool is not very precise and the files contain a new wma header. I'll post a couple of samples and my report, but I have to prepare them first.
Alex B> Great! Could you tell us what tool you're using for splitting WMA files?
FYI, the codecs and settings discussion is now over. WMA Standard will be used at 128 kbps 2-pass VBR since it's the fairest option.
I would like to ask Ivan to send me the updated AAC encoder by Monday the latest. If I don't have it by then, I will have to use the current version.
Some words regarding samples - I cannot encode full tracks and then cut the samples out because I don't own most of the songs that are going to be tested. Posting the whole tracks losslessly on the Internet is illegal in most parts of the world and letting someone else do the encoding/decoding job is not an option.
Finally, anyone has some special sample proposals? I am going to spend some time today finilazing what I think should be tested and post my samples list tomorrow.
Alex B> Great! Could you tell us what tool you're using for splitting WMA files?[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345209")
I did a lot of research. I actually read trough the Windows Media Encoder user manual documents, which are available here: [a href="http://download.microsoft.com/download/6/1/a/61a9cd90-7d3c-48b7-9ffb-3c5f00d3c1e0/encoder.exe]WME help documents[/url]. After reading the 91-page "encoder_v9.DOC" document I started to read the document named "File_Editor.doc" and realized that I already have the needed tool. The tool named "Windows Media File Editor" is included in the WME package.
[span style='font-size:7pt;line-height:100%']Edit: 91, not 69-page[/span]
WMA Standard will be used at 128 kbps 2-pass VBR since it's the fairest option.
(...)
I cannot encode full tracks and then cut the samples out (...)
[a href="index.php?act=findpost&pid=345217"][{POST_SNAPBACK}][/a]
If you can't use 2-pass encoding in the right way (full track), it would be better to discard it. Result won't correspond to a
real (or representative) encoding with this mode. 2-pass encoding on a short sample correspond to a wrong usage (see video tests).
For full tracks, you have different possibilities. I see:
A/ trying to collect (you don't have to precise
how) the complete tracks corresponding to samples used in the past
B/ trying to collect the complete tracks corresponding to fresh samples submitted by members
C/ asking to submitters to encode themselves in 2-pass mode and to send you the extracted part
D/ using your own musical library as basis. You could submit a long list of possible samples, and let members to express their opinion. Then it will be possible for you to encode the tracks in 2 pass, to decode them and to cut them (precisely).
E/ I already offered to buy one CD. If needed it could work like this: Just order the CD from amazon.de (if available). PM your bank account number and I'll legally buy it from you. I'll pay the CD price and for the letter stamp. (Direct money transactions are not expensive inside EU). Of course, if you happen to like the CD you don't have to sell it to me.
In case others are willing to do the same buying a few CDs would not be a problem. Just let me select first .
If you can't use 2-pass encoding in the right way (full track), it would be better to discard it.
Only in that real world usage doesnt usually target a bitrate for small samples -except cbrs. If you set the average bitrate for the whole track and then cut out the samples your intrested in, the samples could have a tendency to be above the average full track bitrate.
As I see it, if its 2 pass method is the only way to achieve the target bitrate with wma it is proper to use it, its just a slight handicap for the encoder thats its bitrate choice for individual samples is less flexible than other encoders which can achieve the tests average bitrate ,more flexibly. A handicap brought on by wma not providing any way to achieve the specific average bitrate while remaining flexible. So its basically stuck using ABR because it provides no suitable vbr mode for the tests requirements.
I wouldnt focus much on the technical foibles of the 2 pass method, forget how many passes are involved and just regard it as an ABR mode
WMA Standard will be used at 128 kbps 2-pass VBR since it's the fairest option.
(...)
I cannot encode full tracks and then cut the samples out (...)
[a href="index.php?act=findpost&pid=345217"][{POST_SNAPBACK}][/a]
If you can't use 2-pass encoding in the right way (full track), it would be better to discard it. Result won't correspond to a real (or representative) encoding with this mode. 2-pass encoding on a short sample correspond to a wrong usage (see video tests).
[a href="index.php?act=findpost&pid=345223"][{POST_SNAPBACK}][/a]
Using CBR would be unfair. The 2-pass VBR WMA is offering at least gives WMA the possibility to variate the bit allocation a bit.
The usual ABR behavior does not differ much when full tracks are encoded instead of samples because the analyzed frame is quite short. At most the difference is small and can been seen only in the beginning and the end of the sample. (How small? Can anyone answer to my question about the LAME ABR behavior?)
WMA 2-pass VBR is a different beast. It really takes into account the complete track content and selects an appropriate VBR quality setting.
[span style='font-size:7pt;line-height:100%']Edit: Edited the ABR explanation a bit because I don't have real knowledge about the encoders' internals. My opinion is based only on what I have seen happening with usual track lengths.[/span]
FYI, the codecs and settings discussion is now over. WMA Standard will be used at 128 kbps 2-pass VBR since it's the fairest option.
I would like to ask Ivan to send me the updated AAC encoder by Monday the latest. If I don't have it by then, I will have to use the current version.
Some words regarding samples - I cannot encode full tracks and then cut the samples out because I don't own most of the songs that are going to be tested. Posting the whole tracks losslessly on the Internet is illegal in most parts of the world and letting someone else do the encoding/decoding job is not an option.
Finally, anyone has some special sample proposals? I am going to spend some time today finilazing what I think should be tested and post my samples list tomorrow.
[a href="index.php?act=findpost&pid=345217"][{POST_SNAPBACK}][/a]
If you can't encode entire tracks and then extract the sample, I agree with guru that wma should be discarded. Nobody says that the full tracks should be posted losslessly on the Internet. As far as I know, the call to gather the full tracks of previous samples has not been put out yet -- you might be able to gather a significant portion of them.
ff123
Nobody says that the full tracks should be posted losslessly on the Internet. As far as I know, the call to gather the full tracks of previous samples has not been put out yet -- you might be able to gather a significant portion of them.
[a href="index.php?act=findpost&pid=345241"][{POST_SNAPBACK}][/a]
I don't understand what you mean. I need the full tracks in order to prepare the samples, so the files have to be put on the Internet somewhere for me to download them, which is illegal in most countries. Also, even classical music is copyrighted.
Using CBR would be unfair. The 2-pass VBR WMA is offering at least gives WMA the possibility to variate the bit allocation a bit.
[a href="index.php?act=findpost&pid=345239"][{POST_SNAPBACK}][/a]
Yes, but only a bit.
Using 2-pass on short, splitted samples lead to different problems:
1/ we don't know if WMA 2-pass was really
optimized for short musical parts. The same goes for video 2-pass encoding: the bitrate can't be properly distributed if you're encoding a short 10 seconds clip. There's a minimal length required to see the bitrate allocation and distribution working efficiently with video clips. The same problem
may happen with audio 2-pass. This is not the biggest issue, but it's a real one IMO.
2/ it has been said several times: a short sample directly encoded in 2-pass would logically differ from the same sample extracted from a complete tracks encoded in 2-pass.
The sound and the sound quality won't be the same anymore. People are interested to test how an encoder sound in real life with optimal settings; using short samples as basis to feed the encoder won't represent the real potential of WMA 2-pass.
3/ by encoding the selected short samples directly in 2-pass,
WMA won't be allowed to use more than 128 kbps for these samples. Take a difficult sample: LAME, Vorbis, AAC could encode it at whatever the bitrate they want (usually: 150...160 kbps). WMA bitrate will be 128 kbps for this sample. But if 2-pass is used properly (i.e. by encoding a complete track which should allow the encoder to perform an efficient distribution of the bitrate), then our extracted part from the 2-pass encoding may correspond to a higher bitrate. Therefore, all our encoders will be allowed to use more or less than 128 kbps, according to the estimated complexity of the sample (and also according to the relative complexity of the remaining parts of the track for WMA 2-pass).
A short samples bitrate, with WMA 2-pass encodings of short samples would look like: AAC MP3 OGG WMA
sample_01 142 146 145 128
sample_02 137 134 138 128
sample_03 132 126 128 128
sample_04 128 96 101 128
sample_05 167 169 181 128
But
a short samples bitrate, with WMA 2-pass encodings performed on full tracks would theoretically look like: AAC MP3 OGG WMA
sample_01 142 146 145 137
sample_02 137 134 138 132
sample_03 132 126 128 145
sample_04 128 96 101 87
sample_05 167 169 181 152
In the first case, WMA is handicap by the target bitrate; in the second case, WMA could perform like other VBR encoders by using more or less than 128 kbps on the part we're going to listen, to grade and to rank. That's the main difference. In the first case: apples and oranges (
why LAME and Vorbis are allowed to use 160 kbps but not WMA? this is unfair, the test is biased, the test don't prove anything, it's usual WMA bashing, etc...); in the second cases: oranges and oranges (with two varieties of the same fruits).
I believe that the only fair way to encode WMA 2-pass is to join all samples (not whole tracks, jest the 30 sec samples) into one file and then encode this file. If you do this, it will give WMA the possibility to increase or decrease bitrate for various samples.
Using a 2-pass encoding for a 30 sec clip will in fact produce a CBR file IMO. Using it will produce unrealistic results for WMA. It will have higher quality on easy tracks, but worse quality on complex tracks. If somebody then looks only at the results, it would seem that WMA is better/worse for some kind of music then other encoders, but it wouldn't be truth. The difference will be there because of using 2-pass for WMA.
The 2-pass setting is not only unfair for WMA but on some samples it would handicap other codecs too. I really think WMA Std should be discarded because we probably won't be able to do a fair comparison. I would prefer to see WMA Pro or ATRAC3, we all know anyway that WMA will be the worst codec here. I never saw a credible comparison of WMA Pro and ATRAC against other codecs.
Nobody says that the full tracks should be posted losslessly on the Internet. As far as I know, the call to gather the full tracks of previous samples has not been put out yet -- you might be able to gather a significant portion of them.
[a href="index.php?act=findpost&pid=345241"][{POST_SNAPBACK}][/a]
I don't understand what you mean. I need the full tracks in order to prepare the samples, so the files have to be put on the Internet somewhere for me to download them, which is illegal in most countries. Also, even classical music is copyrighted.
[a href="index.php?act=findpost&pid=345247"][{POST_SNAPBACK}][/a]
I mean that you haven't tried to contact the original submitters of the samples yet.
ff123
Nobody says that the full tracks should be posted losslessly on the Internet. As far as I know, the call to gather the full tracks of previous samples has not been put out yet -- you might be able to gather a significant portion of them.
[a href="index.php?act=findpost&pid=345241"][{POST_SNAPBACK}][/a]
I don't understand what you mean. I need the full tracks in order to prepare the samples, so the files have to be put on the Internet somewhere for me to download them, which is illegal in most countries. Also, even classical music is copyrighted.
[a href="index.php?act=findpost&pid=345247"][{POST_SNAPBACK}][/a]
I mean that you haven't tried to contact the original submitters of the samples yet.
ff123
[a href="index.php?act=findpost&pid=345255"][{POST_SNAPBACK}][/a]
But of what use would that be? It would be illegal to receive the tracks if I didn't pay for them (what I don't want to do).
I believe that the only fair way to encode WMA 2-pass is to join all samples (not whole tracks, jest the 30 sec samples) into one file and then encode this file. If you do this, it will give WMA the possibility to increase or decrease bitrate for various samples.
[a href="index.php?act=findpost&pid=345250"][{POST_SNAPBACK}][/a]
Well, this is probably better than just using a single short sample every time, but loses the important advantage of mimicking what the user would actually do.
ff123
Edit: For this to work right, the collection of samples ought to be a mixture of easy and complex.
I believe that the only fair way to encode WMA 2-pass is to join all samples (not whole tracks, jest the 30 sec samples) into one file and then encode this file. If you do this, it will give WMA the possibility to increase or decrease bitrate for various samples.
Hmm... This has a problem because most of the samples must be quite difficult, otherwise we might get 5 winners.
Using a 2-pass encoding for a 30 sec clip will in fact produce a CBR file IMO. Using it will produce unrealistic results for WMA. It will have higher quality on easy tracks, but worse quality on complex tracks. If somebody then looks only at the results, it would seem that WMA is better/worse for some kind of music then other encoders, but it wouldn't be truth. The difference will be there because of using 2-pass for WMA.
The 2-pass setting is not only unfair for WMA but on some samples it would handicap other codecs too. I really think WMA Std should be discarded because we probably won't be able to do a fair comparison. I would prefer to see WMA Pro or ATRAC3, we all know anyway that WMA will be the worst codec here. I never saw a credible comparison of WMA Pro and ATRAC against other codecs.
[a href="index.php?act=findpost&pid=345250"][{POST_SNAPBACK}][/a]
WMA 2-pass VBR has not been tested. What if it happens to be a good alternative?
The reasoning can be explained in the context, and also the usefulness of the 2-pass method in some situations (like with video soundtracks).
Edit: in any case it would not be like CBR, even with short samples it would be like ABR (or better than the usual ABR, depenging on how big time frames ABR actually analyzes).
Edit: in any case it would not be like CBR, even with short samples it would be like ABR.
You're partlialy right, but I think that ABR encoding is not much like 2-pass and it is not a good option. I don't know exactly how does it work with audio, but I suppose it is similar to video ABR (I'm more experienced with video encoding then audio compression). The encoder is choosing the current birate according to complexity of the source and it is also based on target bitrate. Now imagine you have file with easy first part and difficult second part. The encoder will start encoding a because the part is easy, it will use a much higher bitrate then it should (or would in VBR or 2-pass). Now it comes to the difficult part and starts using higher bitrate. But after a while the encoder sees that the bitrate is too high a starts lowering it to achieve the desired size. The reslut is obviously even worse then CBR.
I made this picture for explaining better what I meant with the 2-pass VBR problem in case the short samples are encoded instead of the complete tracks:
(http://kotisivu.mtv3.fi/alexb/ha/2pass.png)[a href="index.php?act=findpost&pid=344438"][{POST_SNAPBACK}][/a]
Did you read through this thread?
I hope the LAME developers can explain how LAME ABR works. LAME ABR is a good example because it is widely used and established.
But of what use would that be? It would be illegal to receive the tracks if I didn't pay for them (what I don't want to do).
[a href="index.php?act=findpost&pid=345257"][{POST_SNAPBACK}][/a]
I could email you some. Nobody would know about it.
But of what use would that be? It would be illegal to receive the tracks if I didn't pay for them (what I don't want to do).
[a href="index.php?act=findpost&pid=345257"][{POST_SNAPBACK}][/a]
I could email you some. Nobody would know about it.
[a href="index.php?act=findpost&pid=345275"][{POST_SNAPBACK}][/a]
No, thanks, that's really not an option. It's enough for someone to read your post and the listening test announcement page where I am going to describe the used methods etc. to know that I have illegally obtained the tracks. Really, Germany is pretty anal about this stuff. If I am not mistaken, a HA moderator even received a C&D for posting
30 seconds samples of copyrighted material on his page.
But of what use would that be? It would be illegal to receive the tracks if I didn't pay for them (what I don't want to do).
[a href="index.php?act=findpost&pid=345257"][{POST_SNAPBACK}][/a]
I could email you some. Nobody would know about it.
[a href="index.php?act=findpost&pid=345275"][{POST_SNAPBACK}][/a]
No, thanks, that's really not an option. It's enough for someone to read your post and the listening test announcement page where I am going to describe the used methods etc. to know that I have illegally obtained the tracks. Really, Germany is pretty anal about this stuff. If I am not mistaken, a HA moderator even received a C&D for posting 30 seconds samples of copyrighted material on his page.
[a href="index.php?act=findpost&pid=345286"][{POST_SNAPBACK}][/a]
Heh. Ok BTW, what's a C&D?
I'm in a rush and I have to run, so I don't have time for my usual flaming of clueless people... j/k
But please, please, let's not rush the test. There are only 4 days left, and I don't think we are ready.
I also noriced that Shine seems to have been selected as a low anchor. When did we agree on this? Why not Blade? It might be a moot point, but I'd rather see there an encoder that actually had some real world significance.
Samples. Did we decide on the samples yet?
WMA. We didn't solve that one either.
Honestly, let's not rush things. If the test needs to be postponed, we shouldn't hesitate to do so.
However, I have found means two split wma files losslessly. The results I got prove that 2-pass is a real 2-pass and indeed full tracks should be encoded. The possibility to split files makes also possible to check the approximate average bitrate by checking the file size vs duration. It is not exact science, because the splitter tool is not very precise and the files contain a new wma header. I'll post a couple of samples and my report, but I have to prepare them first.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345206")
Here's the report
Directory of Z:\Test\WMA std 2-pass 128 kbps
Satie.wav 1.768.096 > duration =10 s
Yello.wav 1.768.104 > duration =10 s
Joined.wav 3.532.104 > duration =20 s
Satie.wma 166.485 > duration =10 s > av. bitrate ~133,2 kbps
Yello.wma 172.461 > duration =10 s > av. bitrate ~138,0 kbps
Joined.wma 327.877 > duration =20 s > av. bitrate ~131,2 kbps
Satie CUTTED.wma 94.599 > duration ~10 s > av. bitrate ~ 75,7 kbps
Yello CUTTED.wma 238.023 > duration ~10 s > av. bitrate ~190,4 kbps
8 file(s) found Total file size 8.067.749 bytes
With these short samples the average bitrate seems to be from 131 to 138 kbps. The cutted files clearly prove that 2-pass works well (at least with short files).
Here is the lossless 20 s joined sample if someone likes to verify this: [a href="http://kotisivu.mtv3.fi/alexb/ha/Joined.rar]Joined.rar[/url]. I can keep it available for a short while, but because the tools are included with the Windows Media Encoder package I would like to ask people to make their own test samples.
I also noriced that Shine seems to have been selected as a low anchor. When did we agree on this? Why not Blade? It might be a moot point, but I'd rather see there an encoder that actually had some real world significance.
[a href="index.php?act=findpost&pid=345290"][{POST_SNAPBACK}][/a]
It's time for you to understand that we don't request from an anchor to be interesting by itself. How many time must we explain this point?! It must be bad. Period. That's enough. Shine, Blade, lowpassed PCM, ISO encoders... everything is good as long as it sounds poorly to most if not all listeners.
However, I have found means two split wma files losslessly. The results I got prove that 2-pass is a real 2-pass and indeed full tracks should be encoded. The possibility to split files makes also possible to check the approximate average bitrate by checking the file size vs duration. It is not exact science, because the splitter tool is not very precise and the files contain a new wma header. I'll post a couple of samples and my report, but I have to prepare them first.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345206")
Here's the report
Directory of Z:\Test\WMA std 2-pass 128 kbps
Satie.wav 1.768.096 > duration =10 s
Yello.wav 1.768.104 > duration =10 s
Joined.wav 3.532.104 > duration =20 s
Satie.wma 166.485 > duration =10 s > av. bitrate ~133,2 kbps
Yello.wma 172.461 > duration =10 s > av. bitrate ~138,0 kbps
Joined.wma 327.877 > duration =20 s > av. bitrate ~131,2 kbps
Satie CUTTED.wma 94.599 > duration ~10 s > av. bitrate ~ 75,7 kbps
Yello CUTTED.wma 238.023 > duration ~10 s > av. bitrate ~190,4 kbps
8 file(s) found Total file size 8.067.749 bytes
With these short samples the average bitrate seems to be from 131 to 138 kbps. The cutted files clearly prove that 2-pass works well (at least with short files).
Here is the lossless 20 s joined sample if someone likes to verify this: [a href="http://kotisivu.mtv3.fi/alexb/ha/Joined.rar]Joined.rar[/url]. I can keep it available for a short while, but because the tools are included with the Windows Media Encoder package I would like to ask people to make their own test samples.
[a href="index.php?act=findpost&pid=345291"][{POST_SNAPBACK}][/a]
That looks promising. As pointed out, this method depends on making sure that the sample set has a reasonable bitrate distribution for the true vbr codecs.
ff123
I'm in a rush and I have to run, so I don't have time for my usual flaming of clueless people... j/k
But please, please, let's not rush the test. There are only 4 days left, and I don't think we are ready.
I also noriced that Shine seems to have been selected as a low anchor. When did we agree on this? Why not Blade? It might be a moot point, but I'd rather see there an encoder that actually had some real world significance.
Samples. Did we decide on the samples yet?
WMA. We didn't solve that one either.
Honestly, let's not rush things. If the test needs to be postponed, we shouldn't hesitate to do so.
[a href="index.php?act=findpost&pid=345290"][{POST_SNAPBACK}][/a]
I'm ok with either Shine or Blade. Blade does have an additional advantage of being used in a previous test.
ff123
Directory of Z:\Test\WMA std 2-pass 128 kbps
Satie.wav 1.768.096 > duration =10 s
Yello.wav 1.768.104 > duration =10 s
Joined.wav 3.532.104 > duration =20 s
Satie.wma 166.485 > duration =10 s > av. bitrate ~133,2 kbps
Yello.wma 172.461 > duration =10 s > av. bitrate ~138,0 kbps
Joined.wma 327.877 > duration =20 s > av. bitrate ~131,2 kbps
Satie CUTTED.wma 94.599 > duration ~10 s > av. bitrate ~ 75,7 kbps
Yello CUTTED.wma 238.023 > duration ~10 s > av. bitrate ~190,4 kbps
8 file(s) found Total file size 8.067.749 bytes
[a href="index.php?act=findpost&pid=345291"][{POST_SNAPBACK}][/a]
Excellent work!
It seems that 2-pass is really working like a two-pass encoder, with a mid/long term bitrate distribution.
I don't have WMEncoder to test this on my side, but what you did is very nice. Thanks a lot!
Heh. Ok BTW, what's a C&D?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345289")
[a href="http://en.wikipedia.org/wiki/C%26D]http://en.wikipedia.org/wiki/C%26D[/url]
I don't have WMEncoder to test this on my side[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345298")
Oh, I didn't know. Here are the wma files: [a href="http://kotisivu.mtv3.fi/alexb/ha/wma2pass.rar]wma2pass.rar[/url]. Be my guest!
I am sure you'll find them interesting. Take care when Yello starts the second passage...
[span style='font-size:7pt;line-height:100%']Edit: typo[/span]
I'm in a rush and I have to run, so I don't have time for my usual flaming of clueless people... j/k
[a href="index.php?act=findpost&pid=345290"][{POST_SNAPBACK}][/a]
Thanks God for that.
But please, please, let's not rush the test. There are only 4 days left, and I don't think we are ready.
[a href="index.php?act=findpost&pid=345290"][{POST_SNAPBACK}][/a]
I agree. I will do my best to find a solution ASAP, though.
I also noriced that Shine seems to have been selected as a low anchor. When did we agree on this? Why not Blade? It might be a moot point, but I'd rather see there an encoder that actually had some real world significance.
[a href="index.php?act=findpost&pid=345290"][{POST_SNAPBACK}][/a]
Come on, this has been discussed to death.
Samples. Did we decide on the samples yet?
[a href="index.php?act=findpost&pid=345290"][{POST_SNAPBACK}][/a]
We did not discuss them, but I am working in the background and I wanted to publish my proposal tomorrow or Monday the latest. I also asked people if they have any samples (do we need a call for samples? - I think I can use 8 from the ones posted for the 64 kbps listening test).
WMA. We didn't solve that one either.
[a href="index.php?act=findpost&pid=345290"][{POST_SNAPBACK}][/a]
Exactement, that's what we're doing now. Unfortunately, I didn't expect WMA to be such a PITA. But hey, you're the one to blame - you wanted WMA Standard (joking ).
Honestly, let's not rush things. If the test needs to be postponed, we shouldn't hesitate to do so.
[a href="index.php?act=findpost&pid=345290"][{POST_SNAPBACK}][/a]
Yep, I am already considering this. If I don't have a result by Monday evening, I will have to postpone.
Sebastian Mares> I'm not sure about Germany, but here in Czech Republic it absolutely legal to download any music from internet. It should be the same for whole Europe. You're only not allowed to shere copyrighted material.
Alex B> As I said, I'm not sure about audio encoding, but what I explained is the way it used to work in video codecs. This is the reason, why you won't find ABR in modern video codecs (XviD, x264). It was only presented in DivX 3.xx. BTW, of course I did read the whole thread. I don't think I said anything which would be in contradiction with the pictures you posted.
I think your samples are perfectly showing the issue we would get by simply compressing the samples with 2-pass WMA. The results will be very different from "real world usage". Talking about this, I would never recomend to anybody to use 2-pass encoding for music.
I don't have WMEncoder to test this on my side[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345298")
Oh, I didn't know. Here are the wma files: [a href="http://kotisivu.mtv3.fi/alexb/ha/wma2pass.rar]wma2pass.rar[/url]. Be my guest!
I am sure you'll find them interesting. Take care when Yello starts the second passage... [a href="index.php?act=findpost&pid=345311"][{POST_SNAPBACK}][/a]
Oops, I forgot to press send on my FTP client. It is available now.
[span style='font-size:7pt;line-height:100%']Edit: a typo again...[/span]
Sebastian Mares> I'm not sure about Germany, but here in Czech Republic it absolutely legal to download any music from internet. It should be the same for whole Europe. You're only not allowed to shere copyrighted material.
[a href="index.php?act=findpost&pid=345316"][{POST_SNAPBACK}][/a]
AFAIK, it's illegal here.
Sebastian Mares> I'm not sure about Germany, but here in Czech Republic it absolutely legal to download any music from internet. It should be the same for whole Europe. You're only not allowed to shere copyrighted material.
[a href="index.php?act=findpost&pid=345316"][{POST_SNAPBACK}][/a]
Then some HA.org members would act against the law by sharing their full samples with Sebastian.
There are similar debates in France. I could maybe download music and movies, but I'm sure that it would be illegal for me to sent to Sebastian full tracks corresponding to Debussy, Mahler and Hongroise
I don't have WMEncoder to test this on my side[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345298\")
Oh, I didn't know. Here are the wma files: [a href=\"http://kotisivu.mtv3.fi/alexb/ha/wma2pass.rar]wma2pass.rar[/url]. Be my guest!
I am sure you find them interesting. Take care when Yello starts the second passage...
[a href=\"index.php?act=findpost&pid=345311\"][{POST_SNAPBACK}][/a]
Indeed, they're interested. I ABXed without problems both samples encoded with the same setting but in a different environment (short sample alone - short sample extracted from a wider ensemble):
[span style=\'font-size:14pt;line-height:100%\']
Satie.wav[/span]
foo_abx v1.2 report
foobar2000 v0.8.3
2005/11/26 20:16:11
File A: file://C:\My Downloads\wma2pass\Satie 2-PASS.wma
File B: file://C:\My Downloads\wma2pass\Satie CUTTED.wma
20:16:13 : Test started.
20:16:26 : 01/01 50.0%
20:16:31 : 02/02 25.0%
20:16:36 : 03/03 12.5%
20:16:40 : 04/04 6.3%
20:16:45 : 05/05 3.1%
20:16:50 : 06/06 1.6%
20:16:54 : 07/07 0.8%
20:16:58 : 08/08 0.4%
20:17:01 : 09/09 0.2%
20:17:05 : 10/10 0.1%
20:17:08 : 11/11 0.0%
20:17:11 : 12/12 0.0%
20:17:18 : Test finished.
----------
Total: 12/12 (0.0%)
[span style=\'font-size:14pt;line-height:100%\']
Yello.wav[/span]
foo_abx v1.2 report
foobar2000 v0.8.3
2005/11/26 20:17:43
File A: file://C:\My Downloads\wma2pass\Yello 2-PASS.wma
File B: file://C:\My Downloads\wma2pass\Yello CUTTED.wma
20:17:47 : Test started.
20:18:10 : 01/01 50.0%
20:18:18 : 02/02 25.0%
20:18:28 : 03/03 12.5%
20:18:36 : 04/04 6.3%
20:18:52 : 05/05 3.1%
20:18:58 : 06/06 1.6%
20:19:10 : 07/07 0.8%
20:19:15 : 08/08 0.4%
20:19:20 : 09/09 0.2%
20:19:28 : 10/10 0.1%
20:19:33 : 11/11 0.0%
20:19:41 : 12/12 0.0%
20:19:42 : Test finished.
----------
Total: 12/12 (0.0%)
For
yello.wav, there's no doubt: the "CUTTED" sample is less distorted than the "2-PASS" version.
For the piano sample (
satie.wav), the appreciation is less distinct. On normal playback volume, the "CUTTED" version (75 kbps) sound less irritating than the "2-PASS" one (130 kbps). This latter suffers from unconstant noise and offers an obvious ringing. The "CUTTED" sample on the other side has no ringing, simply because all noise is cleaned (probably a strong lowpass). With higher playback volume/replaygain, the ringing of the first is highly increased whereas the second one is now wounded by clearly perceptibe metallic artefacts. But even here, I tend to prefer the 75 kbps encoding
Thank you again for these samples, Alex B
Regarding providing full length samples...
An alternative could be to ask people with full length sampling to prepare the WMA samples.
It sound easy enough to do and the tools for preparing the WMA samples are easy to download and install.
Following this idea Sebastion would delegate the preparation of WMA files volounteers.
Regarding providing full length samples...
An alternative could be to ask people with full length sampling to prepare the WMA samples.
It sound easy enough to do and the tools for preparing the WMA samples are easy to download and install.
Following this idea Sebastion would delegate the preparation of WMA files volounteers.
[a href="index.php?act=findpost&pid=345340"][{POST_SNAPBACK}][/a]
That is not an option because I prefer to encode the samples myself and have full control over the process. If something goes wrong with the samples, I should be the one to blame.
Edit: Grammar.
An alternative could be to ask people with full length sampling to prepare the WMA samples.
[a href="index.php?act=findpost&pid=345340"][{POST_SNAPBACK}][/a]
I thought about it. One thing bothers me (but I don't know if it also bothers Sebastian): the organizer is loosing the control of the testing material. He has to trust to other people's work, without any possibility to control it.
If I was to Sebastian's place, I wouldn't keep my piece of mind with such delegation. Not that I suspect people of malevolence, but mistakes happen so easily... Mistake may happen on: encoder's configuration and/or decoding process. 15 persons performing this task = risk of mistake 15 time higher!
Your idea is good, but the risk that something wrong happen is also much higher.
WMA 2-pass is seriously increasing the difficulty of the organisation!
EDIT: Sebastian was faster and is obviously bothered by the same concern
WMA 2-pass is seriously increasing the difficulty of the organisation!
The problem of using VBR 2-pass seems to me overestimated. Let me explain. In case of VBR encoding you have two choices:
1. to choose quality parameter and to get some final bit rate for a sound item (pure VBR);
2. to choose final bit rate and to get some quality parameter chosen by encoder during the first pass (VBR 2-pass).
Nothing else. It’s just another way of using VBR, convenient in some cases. There is no any problem with encoding short tracks with VBR 2-pass - the coder will do the best inside the track as Vorbis, mp3 or aac.
If only we had enough people having the full length samples...
Then each sample could come from two or more sources and Sebastian could compare the results. The samples would then only be accepted if the wave forms when decoded were bit identical.
This would reduce the risk of errors significantly.
I know this will be additional work and just the task of finding sufficient delegates could stall the process for quite some time.
However as it stands now, IMO it would not be fair to WMA not to use 2-pass encoding of the full track.
As I see it we either have to provide full tracks or drop WMA from the test, since I see no reason for including WMA in the test with less than optimal settings.
Just my two euro cents worth:
http://www.archive.org/audio/etree.php (http://www.archive.org/audio/etree.php)
- offers 28651 losslessly recorded legal and free live concerts. How about using the archive for making a couple of samples? The recordings are usually very dynamic and demanding for the encoders, but not clipped and compressed like many current commercial recordings. It would be relatively easy to find good modern Jazz/Alternative etc samples from there. I have downloaded a few interesting shows and I think I could recommend something.
- How about using the Yello sample I posted. I think it would be a good sample of the electronic music genre. The complete track is included in my bitrate table: http://www.hydrogenaudio.org/forums/index....ndpost&p=344516 (http://www.hydrogenaudio.org/forums/index.php?showtopic=38955&view=findpost&p=344516). I could make the sample a bit better (perhaps a little longer and make small fade in/outs if preferred. What is the usual way?). I could also do to the wma 2-pass encoding for this sample. (Since I would be the original sample submitter perhaps I could be trusted...).
Sebastian, in this way you could go through the samples one by one and you wouldn't have to worry too much about how difficult it is to gather the complete tracks. If some of the selected samples don't work out you could change the sample with a similar sample from your collection or from a trusted source. I suppose rare killer samples are not needed or preferred for this test anyway.
I am a newbie here, but I think you can trust several capable forum members if they promise to be careful and do the encoding exactly as defined. They would encode only this single format and single sample, not 6 x 18 samples.
WMA 2-pass is seriously increasing the difficulty of the organisation!
The problem of using VBR 2-pass seems to me overestimated. Let me explain. In case of VBR encoding you have two choices:
1. to choose quality parameter and to get some final bit rate for a sound item (pure VBR);
2. to choose final bit rate and to get some quality parameter chosen by encoder during the first pass (VBR 2-pass).
Nothing else. It’s just another way of using VBR, convenient in some cases. There is no any problem with encoding short tracks with VBR 2-pass - the coder will do the best inside the track as Vorbis, mp3 or aac.
[a href="index.php?act=findpost&pid=345361"][{POST_SNAPBACK}][/a]
The problem is that pure VBR mode is different than 2-pass VBR mode.
Example... Take the following song: lalalala BOOM CRUNCH BANG lalalala
When encoding the full track with pure VBR, let's say "lalalala" gets 80 kbps and "BOOM CRUNCH BANG" gets 192 kbps.
When encoding the "BOOM CRUNCH BANG" part alone, pure VBR will also allocate 170 kbps or even 192 kbps for example.
So in both cases, the complicated part gets a very similar bitrate (in our case, about 170 to 192 kbps).
However, with 2-pass VBR, when encoding the whole track, the result will be similar to pure VBR.
The story is different when encoding the "BOOM CRUNCH BANG" part, however. Instead of using 170 - 192 kbps, the encoder is forced to use ~128 kbps. Therefore, the quality will be inferior.
With all due respect ... wouldn't it be a better choice to use the WMA setting that is closest to 'default' ?
My point is ... imagine a given noob firing up his WMP, inserting a CD and ripping it to WMA VBR ... what would be the most likely setting to use ?
There is only one default VBR setting in WMP ... no 2-Pass mentioned.
See for yourself ...
(http://www.jeanlucpicard.org/pictures/Snap2.png)
BTW, could someone be kind enough to test if MP3Tag (http://www.mp3tag.de/en/) reports the correct bitrate for WMAs? The program has a very nice export feature that can be used to create bitrate tables.
With all due respect ... wouldn't it be a better choice to use the WMA setting that is closest to 'default' ?
My point is ... imagine a given noob firing up his WMP, inserting a CD and ripping it to WMA VBR ... what would be the most likely setting to use ?
There is only one default VBR setting in WMP ... no 2-Pass mentioned.
See for yourself ...
(http://www.jeanlucpicard.org/pictures/Snap2.png)
[a href="index.php?act=findpost&pid=345371"][{POST_SNAPBACK}][/a]
I don't know if WMP uses quality based VBR or bitrate based VBR (yes, call me clueless regarding this since I never used WMP for ripping).
If it's bitrate based, it's doing a two-pass encoding, if it's quality based, the bitrate is either too low (Q50) or too high (Q75).
From the VBS I am using to encode to WMA:
[-a_mode] <mode_number>
Audio encoding to be used.
0: 1-pass CBR (default).
1: 2-pass CBR.
2: Quality-based VBR.
3: Bit rate-based VBR (two-pass).
4: Bit rate-based peak VBR (two-pass).
As you can see, bitrate based VBR implies 2-pass. The only other VBR method left is quality based which has unreliable bitrate prediction.
The story is different when encoding the "BOOM CRUNCH BANG" part, however. Instead of using 170 - 192 kbps, the encoder is forced to use ~128 kbps. Therefore, the quality will be inferior.
[a href="index.php?act=findpost&pid=345369"][{POST_SNAPBACK}][/a]
Correct. But thу question is: will this quality be equal (worse, better) to the one produced by other codecs that encoded this "BOOM CRUNCH BANG" at the same ~128? (asuming that exactly this part is our test excerpt). If other codecs gives higher final bit rates for the part, then we should use appropriate final bit rate for VBR 2-pass. But AFAIK the listening test is 128 kbit/s which means that VBR 2-pass suits it the best.
The story is different when encoding the "BOOM CRUNCH BANG" part, however. Instead of using 170 - 192 kbps, the encoder is forced to use ~128 kbps. Therefore, the quality will be inferior.
[a href="index.php?act=findpost&pid=345369"][{POST_SNAPBACK}][/a]
Correct. But thу question is: will this quality be equal (worse, better) to the one produced by other codecs that encoded this "BOOM CRUNCH BANG" at the same ~128? (asuming that exactly this part is our test excerpt). If other codecs gives higher final bit rates for the part, then we should use appropriate final bit rate for VBR 2-pass. But AFAIK the listening test is 128 kbit/s which means that VBR 2-pass suits it the best.
[a href="index.php?act=findpost&pid=345375"][{POST_SNAPBACK}][/a]
I think, you think that the bitrate of 128 kbps has to be maintained on a per-track basis, which is not true. Therefore, LAME for example can use a higher bitrate for the specific sample and a lower bitrate for another sample. 2-pass VBR however forces the same bitrate to be used for both tracks.
Sure ... but your test should (IMO, at least) resemble some real-world-scenario which makes it possible to draw some big-picture conclusions. A real-world scenario does not contain geeks like us that read the WMA documentation to create a commandline ... Joe Average uses WMP with the given setting - and that's why I think you should use the very same setting for WMA coding.
I understand that you are afraid that WMA comes out with a bitrate too high (or too low) and thus gains some advantage (or loses some ground) regarding the other codecs' performance ... but who cares? As long as you document the bitrate distribution for each sample, you'll be fine.
Remember ... WMA is CD quality at 64 kbps so the other codecs will get blown to smithereens anyway
WMA 2-pass is seriously increasing the difficulty of the organisation!
The problem of using VBR 2-pass seems to me overestimated. Let me explain. In case of VBR encoding you have two choices:
1. to choose quality parameter and to get some final bit rate for a sound item (pure VBR);
2. to choose final bit rate and to get some quality parameter chosen by encoder during the first pass (VBR 2-pass).
Nothing else. It’s just another way of using VBR, convenient in some cases. There is no any problem with encoding short tracks with VBR 2-pass - the coder will do the best inside the track as Vorbis, mp3 or aac.
[a href="index.php?act=findpost&pid=345361"][{POST_SNAPBACK}][/a]
As said many times, this depends greatly on the sampled track.
As a test I just encoded the complete Yello track and cutted the sampled passage with the WMeditor. This new sample is 157 kbps. When the same sampled passage was encoded separately the bitrate was 138 kbps.
From the original 10s wave sample Lame -V5 --vbr-new produces 157 kbps (yes, it is correct, the same as above) and Vorbis b4.51 at q4.25 produces even a bigger value, 172 kbps.
Sure ... but your test should (IMO, at least) resemble some real-world-scenario which makes it possible to draw some big-picture conclusions. A real-world scenario does not contain geeks like us that read the WMA documentation to create a commandline ... Joe Average uses WMP with the given setting - and that's why I think you should use the very same setting for WMA coding.
I understand that you are afraid that WMA comes out with a bitrate too high (or too low) and thus gains some advantage (or loses some ground) regarding the other codecs' performance ... but who cares? As long as you document the bitrate distribution for each sample, you'll be fine.
Remember ... WMA is CD quality at 64 kbps so the other codecs will get blown to smithereens anyway
[a href="index.php?act=findpost&pid=345377"][{POST_SNAPBACK}][/a]
I cannot tolerate a deviation of more or less than 10%. Also, that fancy WMP dialog you see triggers the same settings as the command lines that were discussed here - it has nothing to do with geekness. Some prefer the dialog, some the command line - the result is the same.
As a test I just encoded the complete Yello track and cutted the sampled passage with the WMeditor. This new sample is 157 kbps. When the same sampled passage was encoded separately the bitrate was 138 kbps.
[a href="index.php?act=findpost&pid=345378"][{POST_SNAPBACK}][/a]
Yes, this contradiction seems to be the result of choosing the way of calculating final bit rates for codecs - using big quantaties of sound material, which I think is very reasonable. So within this approach the use of VBR 2-pass could be IMHO according to per-test sample basis: for each test sample to choose final bit rate of VBR 2-pass closest to the one of other codecs.
As a test I just encoded the complete Yello track and cutted the sampled passage with the WMeditor. This new sample is 157 kbps. When the same sampled passage was encoded separately the bitrate was 138 kbps.
[a href="index.php?act=findpost&pid=345378"][{POST_SNAPBACK}][/a]
Yes, this contradiction seems to be the result of choosing the way of calculating final bit rates for codecs - using big quantaties of sound material, which I think is very reasonable. So within this approach the use of VBR 2-pass could be IMHO according to per-test sample basis: for each test sample to choose final bit rate of VBR 2-pass closest to the one of other codecs.
[a href="index.php?act=findpost&pid=345380"][{POST_SNAPBACK}][/a]
So you suggest adapting the bitrate of each sample? That is nonsense.
I don't know if WMP uses quality based VBR or bitrate based VBR (yes, call me clueless regarding this since I never used WMP for ripping).
Well, if I understand you and the WMP settings dialog right, it is quality based. For each level on the quality scale it gives a broad range of possible bitrate outcomes:
1: 40-75 kbps
2: 50-95 kbps
3: 85-145 kbps
4: 135-215 kbps
5: 240-355 kbps
Sure ... but your test should (IMO, at least) resemble some real-world-scenario which makes it possible to draw some big-picture conclusions. A real-world scenario does not contain geeks like us that read the WMA documentation to create a commandline
[a href="index.php?act=findpost&pid=345377"][{POST_SNAPBACK}][/a]
The same scenario doesn't mention latest Aoyumi works either. And I'm not sure that LAME -V5 is a part of the casting.
I recall one major principle of this test and the previous ones: using each selected formats at their best possible settings.
Try to prove to someone claiming that WMA is as good or better than LAME/AAC... that he is wrong with for only proof a test which missed out VBR for WMA but which use it with all other competitors. How serious would it be? There is a possibility to use VBR with WMA at this bitrate: either we're using it or we give up with WMA. But we can't flaw the test just because we're encountering some difficulties with one contender. Such attitude could simply discredit the whole exercise. Personaly, if I didn't know the circumstances nor HA.org, I would be very suspicious with the results of a test neglecting one contender.
... Joe Average uses WMP with the given setting - and that's why I think you should use the very same setting for WMA coding.
Joe Average and many of his friends also use Music Match Jukebox, WMP or iTunes for MP3 encoding. It's not a reason to handicap MP3 for this test. The same policy should apply to WMA: if the major WMA encoding tool omit 2-pass encoding, it's not a reason for not using it.
I understand that you are afraid that WMA comes out with a bitrate too high (or too low) and thus gains some advantage (or loses some ground) regarding the other codecs' performance ... but who cares? As long as you document the bitrate distribution for each sample, you'll be fine.
Because a test featuring encoders with completely different bitrate is senseless. Try this experience. Make a listening test on your side, including Nero Digital at 80 kbps, iTunes AAC at 128, LAME at 140, Vorbis at 32 and WMA at 192, and publish the results here and on different boards. You'll quickly discover who cares and how much they care bout the equity of the exercise.
Remember ... WMA is CD quality at 64 kbps so the other codecs will get blown to smithereens anyway
One requirement of listening tests aspiring to scientificity is precisely to remove all elements of subjectivity like passions (zealotery, bashing, sarcasm, etc...) and our beliefs. The existence of eye-catching claims for WMA is not a reason for us to denigrate or neglect this format. Either we keep the same attitude of equity for each format or we cancel the test. But we can't be serious and careful with four formats and childish or careless with the remaining one.
BTW, could someone be kind enough to test if MP3Tag (http://www.mp3tag.de/en/) reports the correct bitrate for WMAs? The program has a very nice export feature that can be used to create bitrate tables.[a href="index.php?act=findpost&pid=345372"][{POST_SNAPBACK}][/a]
You made me to download the v.2.34 and upgrade my 2.33. Now, when I try to open the "Customize columns" dialog the program produces a runtime error ...
However, unfortunately it can't display wma bitrates correctly. For example, the about 75 kbps Satie.wma file shows up as 176 kbps.
You could try it with the wma samples I provided. I calculated the bitrates from the real file sizes.
BTW, could someone be kind enough to test if MP3Tag (http://www.mp3tag.de/en/) reports the correct bitrate for WMAs? The program has a very nice export feature that can be used to create bitrate tables.[a href="index.php?act=findpost&pid=345372"][{POST_SNAPBACK}][/a]
You made me to download the v.2.34 and upgrade my 2.33. Now, when I try to open the "Customize columns" dialog the program produces a runtime error ...
[a href="index.php?act=findpost&pid=345387"][{POST_SNAPBACK}][/a]
Weird, works fine here. Maybe you should post on the MP3Tag bug forum.
Edit: Seems that MP3Tag reports the same bitrate as Windows Explorer.
So you suggest adapting the bitrate of each sample? That is nonsense.
[a href="index.php?act=findpost&pid=345381"][{POST_SNAPBACK}][/a]
Only for one encoder - VBR 2-pass and only because it hasn't appropriate tools for managing targeted bit rates that we need in this particular listening test.
Another compromise solution is to calculate targeted bit rates of the contenders on the basis of selected test samples only (chained one after another)
So you suggest adapting the bitrate of each sample? That is nonsense.
[a href="index.php?act=findpost&pid=345381"][{POST_SNAPBACK}][/a]
Only for one encoder - VBR 2-pass and only because it hasn't appropriate tools for managing targeted bit rates that we need in this particular listening test.
[a href="index.php?act=findpost&pid=345393"][{POST_SNAPBACK}][/a]
Another compromise solution is to calculate targeted bit rates of the contenders on the basis of selected test samples only (chained one after another)
[a href="index.php?act=findpost&pid=345396"][{POST_SNAPBACK}][/a]
Sorry, but none of the two options are desirable because they don't represent real world usage at all. Also, option 1 makes me think that we are the VBR "engine" since it's suddenly our job to customize the bitrate according to the sample - that's the encoder's job.
Sorry, but none of the two options are desirable because they don't represent real world usage at all. Also, option 1 makes me think that we are the VBR "engine" since it's suddenly our job to customize the bitrate according to the sample - that's the encoder's job.
[a href="index.php?act=findpost&pid=345397"][{POST_SNAPBACK}][/a]
The second option really doesn't represent but the first one would be not far from real world. In the case VBR job isn't so difficult - the choice of bit rates is limited by 96, 128, 160, 192. I think we could cope with ... for the sake of science
You made me to download the v.2.34 and upgrade my 2.33. Now, when I try to open the "Customize columns" dialog the program produces a runtime error ...
This bug should be fixed with Mp3tag 2.34a (http://mp3tag.de/en/download.html). Thanks for reporting.
However, unfortunately it can't display wma bitrates correctly. For example, the about 75 kbps Satie.wma file shows up as 176 kbps.
I'm using the "Bitrate" attribute to get the bitrate and I have no idea what WMP10 uses to get the information.
Best regards,
~ Florian
Sorry, but none of the two options are desirable because they don't represent real world usage at all. Also, option 1 makes me think that we are the VBR "engine" since it's suddenly our job to customize the bitrate according to the sample - that's the encoder's job.[a href="index.php?act=findpost&pid=345397"][{POST_SNAPBACK}][/a]
The second option really doesn't represent but the first one would be not far from real world. In the case VBR job isn't so difficult - the choice of bit rates is limited by 96, 128, 160, 192. I think we could cope with ... for the sake of science [a href="index.php?act=findpost&pid=345404"][{POST_SNAPBACK}][/a]
It seems like too much human intervention to vary the bitrate for samples according to the others averages, the other codecs might benefit from having their bitrate tweaked too -it could get very messy.
I summarised the test modes for wma, hope its on track:
cbr-unfair handicap
vbr-cant achieve test target bitrate without manipulating the sample corpus
2pass vbr (abr with forsight) on individual samples to test bitrate
-unfair handicap
2pass vbr of full tracks, samples extracted
-involves a great deal of extra work
(legalities, clipping time, possible further tuning to reach average target bitrate)
2pass vbr on samples linked and then separated
-involves some extra work (the pasting and clipping needs to be rather exact)
2pass vbr (abr+) targeting average bitrate of other encoders for each sample
-the other codecs control their own bitrates, with no help or hinderance from rest
(would be a bad handicap if wma allocates bitrate in a different way from the pack average)
Another compromise solution is to calculate targeted bit rates of the contenders on the basis of selected test samples only (chained one after another)
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345396")
Such bitrate table is available on the bottom of [a href="http://www.rjamorim.com/test/multiformat128/results.html]Roberto's page[/url].
The average bitrate of the bunch of sample differs from the average bitrate computed by different members (and based on various full tracks/album). The difference is not that big (few kbps), but it would be much greater without two samples (Debussy and ItCouldBeSweet) which are significantly lowering the bitrate of the sample. By the way, ItCouldBeSweet was intentionaly introduced in this test to reduce the distortion between CBR encoders average bitrate (128 kbps) and VBR ones (142 kbps -> 135 kbps with two very low bitrate samples). See this reply (http://www.hydrogenaudio.org/forums/index.php?showtopic=21904&view=findpost&p=213811).
We shouldn't base our bitrate estimation on short samples. At least not if we could get a more accurate value.
For the same reason, it wouldn't be a good idea to chain all samples and to encode the resulting file in two-pass mode. The resulting file would be composed of 15 difficult samples but only three low-complex samples. The bitrate allocation mechanism can't work efficiently on such basis. Tracks are usually not composed by 83% (15/18) of difficult material; and when they are, the average bitrate of VBR encoders would be closer to 142 kbps than 135.
Again, to respect the equity and to respect the way VBR 2-pass is working, we need to extract our samples from their original environment. It's a real pain to test, but it's the only way to test it properly. All other solutions
(2-pass encoding for 1/ short samples 2/short samples merged into a big one 3/ short samples merged into an artificial creation supposed to represent an "average" track) are only lowering the efficiency and the power of the 2-pass VBR mode available with WMA. It's clearly an handicap. We don't know if it would have an audible impact on sound quality for most samples, but what really matters is that the equity of the testing conditions wouldn't be respected.
And the most important element is that our resulting test signal (for WMA) would simply differ from what a WMA user listening to New York City, DaFunkor Pelleas and Melisande encoded with the same setting would hear. I don't know if people realize what it means. The problem is not that WMA 2-pass is not representative of real-world usage, but that the files we're going to test wouldn't be representative of WMA 2-pass!
With 1-pass VBR encoding, the quality only depends on the selected setting (-V5, -q4, etc...) but with 2-pass encoding, the quality also depend from the content of the full track (on which is based the bitrate allocation). Hence the necessity of using WMA 2-pass on the original tracks and nothing else.
However, unfortunately it can't display wma bitrates correctly. For example, the about 75 kbps Satie.wma file shows up as 176 kbps.
I'm using the "Bitrate" attribute to get the bitrate and I have no idea what WMP10 uses to get the information.
Best regards,
~ Florian
[a href=\"index.php?act=findpost&pid=345405\"][{POST_SNAPBACK}][/a]
OK, no problem. I guess we can rely on the file size and duration in seconds, right?
Bitrate = Filesize in bytes * 0.008 / Duration in seconds
Example: Thin Lizzy - Whiskey In The Jar, 342 seconds, 6.840.511, 160 kbps CBR:
6840511 * 0.008 = 54724.088
54724.088 / 342 = 160,012 kbps
Edit: I think you can use the following export that generates a decent HTML file:
$puts(counter, 0)
<html>
<head>
<title>Bitrate Table</title>
<style type="text/css">
body { background-color: #fff; color: #000; }
th { border-bottom: 1px solid black; text-align: left; }
.even { background-color: #ddd; }
</style>
</head>
<body>
<table width="100%%">
<tr>
<th>Artist</th>
<th>Title</th>
<th>Album</th>
<th>File Size</th>
<th>Duration</th>
<th>Bitrate</th>
</tr>
$loop(%_folderpath_rel%)
$puts(counter, $add($get(counter), 1))
<tr$if($odd($get(counter)),, class="even")>
<td>%artist%</td>
<td>%title%</td>
<td>%album%</td>
<td>$fmtNum(%_file_size_bytes%)</td>
<td>$fmtNum(%_length_seconds%)</td>
<td>$mul($div($div(%_file_size_bytes%, %_length_seconds%), 1000), 8)</td>
</tr>
$loopend()
</table>
<hr>
<p>
Total Size: $fmtNum(%_total_size_raw%)<br>
Total Duration: $fmtNum(%_total_time_raw%)<br>
Bitrate: $mul($div($div(%_total_size_raw%, %_total_time_raw%), 1000), 8)
</p>
</body>
</html>
I considered adding WMA Professional instead of Standard since the reported bitrate for Q50 (which is pure VBR) was around 134 kbps. However, according to my tests, it's 154 kbps: http://www.maresweb.de/bitrate.htm (http://www.maresweb.de/bitrate.htm)
Damn, forget about that. I used Q75. I will re-encode the files. Sorry!
I'm ok with either Shine or Blade. Blade does have an additional advantage of being used in a previous test.
ff123
[a href="index.php?act=findpost&pid=345296"][{POST_SNAPBACK}][/a]
Thanks. If we have two options, and one is even slightly better, why not use that one? We do want to get the maximum out of this test, don't we?
I don't know if WMP uses quality based VBR or bitrate based VBR (yes, call me clueless regarding this since I never used WMP for ripping).
If it's bitrate based, it's doing a two-pass encoding, if it's quality based, the bitrate is either too low (Q50) or too high (Q75).
[a href="index.php?act=findpost&pid=345373"][{POST_SNAPBACK}][/a]
WMP doesn't offer the 2-pass mode. That's why I already said that it's similar to WMA Pro in that regard. It has very low real world usage value.
A few more things.
You should try encoding the samples with VBR50. If the bitrate is close to 128kbps or above, we could use those instead of the 2-pass encodes. I think this would be preferable to that damn 2-pass mode.
And for the bitrate tables you can use Mr QuestionMan. It should report correct bitrates for all formats and if you need a different export function, I can quickly add it.
Edit: Oh, and I think we agreed that it's useless to test WMA Pro. If we can't test WMA properly, we should just drop it. We know anyway that it certainly won't win the test...
Edit: Oh, and I think we agreed that it's useless to test WMA Pro. If we can't test WMA properly, we should just drop it. We know anyway that it certainly won't win the test...
[a href="index.php?act=findpost&pid=345495"][{POST_SNAPBACK}][/a]
Well, what can we do if WMA Standard is boyloving? I'd prefer testing WMA Pro over MusePack or ATRAC if we can't test Standard.
Edit: I am currently encoding my collection to Standard Q50 and Q75 and Professional to Q50. Whatever comes close to 128 kbps (+/- 10%; 12 kbps) is going to be an option.
Edit: Oh, and I think we agreed that it's useless to test WMA Pro. If we can't test WMA properly, we should just drop it. We know anyway that it certainly won't win the test...
[a href="index.php?act=findpost&pid=345495"][{POST_SNAPBACK}][/a]
Well, what can we do if WMA Standard is boyloving? I'd prefer testing WMA Pro over MusePack or ATRAC if we can't test Standard.
[a href="index.php?act=findpost&pid=345497"][{POST_SNAPBACK}][/a]
We just drop it. One less contender would just make the test easier. The test will be HARD enough anyway.
One thing to watch out for when comparing WMA bitrates: MS have a habit of considering 1024 bits = 1 Kb, and not 1000 bits = 1 Kb. This is why some non-MS tools show WMA 2-pass files as 131 Kb/s, but the MS tools show the same files as 128 Kb/s (128 * 1.024 = 131).
I'm ok with either Shine or Blade. Blade does have an additional advantage of being used in a previous test.
ff123
[a href="index.php?act=findpost&pid=345296"][{POST_SNAPBACK}][/a]
Thanks. If we have two options, and one is even slightly better, why not use that one? We do want to get the maximum out of this test, don't we?
[a href="index.php?act=findpost&pid=345495"][{POST_SNAPBACK}][/a]
Are you trolling, or do you have problems to understand what an anchor is?. We're not looking for the "maximum" for a low anchor, but for the competitors themselves. An anchor is not a competitor. There's not point to try to optimize the low anchor quality; otherwise it wouldn't be an anchor anymore. We simply looking for something bad, really bad.
WMP doesn't offer the 2-pass mode. That's why I already said that it's similar to WMA Pro in that regard. It has very low real world usage value.
Do you read what people are saying? THE PURPOSE OF THE TEST IS NOT TO TEST WHAT POPULAR TOOLS ARE OFFERING. Otherwise, bye bye aoTuV, bye bye LAME beta. How many time must we repeat this, especially to a moderator!
And for the bitrate tables you can use Mr QuestionMan. It should report correct bitrates for all formats and if you need a different export function, I can quickly add it.
WMA and WMAPro bitrate estimation are wrong with MQM in some cases.
WMA Pro at Q50 results in 134 kbps on my library. Now testing Standard at Q50 and Q75, although I am sure the bitrate will be too low or too high.
We're not looking for the "maximum" for a low anchor, but for the competitors themselves. An anchor is not a competitor. There's not point to try to optimize the low anchor quality; otherwise it wouldn't be an anchor anymore. We simply looking for something bad, really bad.[a href="index.php?act=findpost&pid=345509"][{POST_SNAPBACK}][/a]
Uhm, for me an anchor implies not only something that should be a reference in the current test, but also something that should anchor the test in respect to other tests. That way one could look at two different tests performed at different times, with different competitors and by different people and get an idea about how the codecs compare to each other over the two tests, since the anchor is the same and should recieve comparable score in both tests.
Do the ITU papers on listening tests mention anything about the use of anchors?
I'm ok with either Shine or Blade. Blade does have an additional advantage of being used in a previous test.
ff123
[a href="index.php?act=findpost&pid=345296"][{POST_SNAPBACK}][/a]
Thanks. If we have two options, and one is even slightly better, why not use that one? We do want to get the maximum out of this test, don't we?
[a href="index.php?act=findpost&pid=345495"][{POST_SNAPBACK}][/a]
Are you trolling, or do you have problems to understand what an anchor is?. We're not looking for the "maximum" for a low anchor, but for the competitors themselves. An anchor is not a competitor. There's not point to try to optimize the low anchor quality; otherwise it wouldn't be an anchor anymore. We simply looking for something bad, really bad.
[a href="index.php?act=findpost&pid=345509"][{POST_SNAPBACK}][/a]
Are you dense, or do you have a problem understanding that I want to make the test as efficient as possible? If we can use the same anchor as in a previous test and that will give us great comparison value, why not make use of it? This test will be VERY demanding, so we should try to get the most out of it. What's there hard to understand? I very well understand that it has to be bad, don't try to twist my words. It's a minor issue, I agree, but I'm a perfectionist and I try to do everything as good as possible.
WMP doesn't offer the 2-pass mode. That's why I already said that it's similar to WMA Pro in that regard. It has very low real world usage value.
Do you read what people are saying? THE PURPOSE OF THE TEST IS NOT TO TEST WHAT POPULAR TOOLS ARE OFFERING. Otherwise, bye bye aoTuV, bye bye LAME beta. How many time must we repeat this, especially to a moderator!
[a href="index.php?act=findpost&pid=345509"][{POST_SNAPBACK}][/a]
How is that the same? LAME beta is just a progression of LAME. It's what everybody uses and even more people will use in the future. The same with aoTuV.
And wtf has my moderator status to do with it? Please go read TOS #1 again.
And for the bitrate tables you can use Mr QuestionMan. It should report correct bitrates for all formats and if you need a different export function, I can quickly add it.
WMA and WMAPro bitrate estimation are wrong with MQM in some cases.
[a href="index.php?act=findpost&pid=345509"][{POST_SNAPBACK}][/a]
I fixed this, and will include it in the new version. Sidenote: WMA is very #@%^^*%&.
WMP doesn't offer the 2-pass mode. That's why I already said that it's similar to WMA Pro in that regard. It has very low real world usage value.
Do you read what people are saying? THE PURPOSE OF THE TEST IS NOT TO TEST WHAT POPULAR TOOLS ARE OFFERING. Otherwise, bye bye aoTuV, bye bye LAME beta. How many time must we repeat this, especially to a moderator!
[a href="index.php?act=findpost&pid=345509"][{POST_SNAPBACK}][/a]
How is that the same? LAME beta is just a progression of LAME. It's what everybody uses and even more people will use in the future. The same with aoTuV.
[a href="index.php?act=findpost&pid=345514"][{POST_SNAPBACK}][/a]
I think what Guru meant is that MMJB, iTunes, etc. are more popular rippers and encoders and we are not using them because of that.
WMA is very #@%^^*%&.
[a href="index.php?act=findpost&pid=345514"][{POST_SNAPBACK}][/a]
At least one thing we have in common.
Edit: BTW, what's wrong with using MP3Tag for the conversion if I rely on duration and file size?
fb2k just encoded 48/318 files and the average bitrate of WMA Standard at Q50 is 104 kbps. Should I let it finish or is it enough to see that Q50 is too low?
I think what Guru meant is that MMJB, iTunes, etc. are more popular rippers and encoders and we are not using them because of that.
[a href="index.php?act=findpost&pid=345515"][{POST_SNAPBACK}][/a]
No, Guru is specifically refering to the LAME beta and Aoyumi's version of Vorbis. Eg, non final versions.
Edit: BTW, what's wrong with using MP3Tag for the conversion if I rely on duration and file size?
[a href="index.php?act=findpost&pid=345515"][{POST_SNAPBACK}][/a]
I think that's the best solution with WMA. I don't think there is a single software out there apart from latest WMP that reports bitrates correctly all the time for WMA files.
I think what Guru meant is that MMJB, iTunes, etc. are more popular rippers and encoders and we are not using them because of that.
[a href="index.php?act=findpost&pid=345515"][{POST_SNAPBACK}][/a]
No, Guru is specifically refering to the LAME beta and Aoyumi's version of Vorbis. Eg, non final versions.
[a href="index.php?act=findpost&pid=345518"][{POST_SNAPBACK}][/a]
Really, I think he was referring to popular tools. You were saying that WMA 2-pass VBR is useless to test since nobody uses it in real world and that we should test what WMP offers. Therefore, since LAME beta and AoTuV are not really included in popular tools (again, MMJB, iTunes, etc.), we shouldn't test them because they are meaningless too.
Edit: BTW, what's wrong with using MP3Tag for the conversion if I rely on duration and file size?
[a href="index.php?act=findpost&pid=345515"][{POST_SNAPBACK}][/a]
I think that's the best solution with WMA. I don't think there is a single software out there apart from latest WMP that reports bitrates correctly all the time for WMA files.
[a href="index.php?act=findpost&pid=345518"][{POST_SNAPBACK}][/a]
Excellent.
At this time, I would like to thank Florian for this great tool.
fb2k just encoded 48/318 files and the average bitrate of WMA Standard at Q50 is 104 kbps. Should I let it finish or is it enough to see that Q50 is too low?
[a href="index.php?act=findpost&pid=345517"][{POST_SNAPBACK}][/a]
Try to encode the samples only. As I said, if they reach 128ish bitrates, I think they should be prefered instead of the 2-pass encodes.
fb2k just encoded 48/318 files and the average bitrate of WMA Standard at Q50 is 104 kbps. Should I let it finish or is it enough to see that Q50 is too low?
[a href="index.php?act=findpost&pid=345517"][{POST_SNAPBACK}][/a]
36 songs later and we're still averaging 104 kbps.
I will check Q75 when I get back from the graveyard.
Regarding the samples, that question is still open since we didn't decide on the samples, yet. Again, does anyone have some samples which he or she thinks that should be tested? If not, I will use 8 posted from the 64 kbps test and 12 from Roberto's test.
Q75 will be too high. It reaches ~160kbps bitrates on my files.
We just drop it. One less contender would just make the test easier. The test will be HARD enough anyway.
[a href="index.php?act=findpost&pid=345498"][{POST_SNAPBACK}][/a]
I propose a third option if 128K 2-pass WMA ends up being dropped because of an inability to legally attain full songs.
Instead of using Shine or Blade as an MP3 low anchor, use 64K CBR WMA standard for the anchor.
I would like to cast a vote to include wma@q50 anyway...with the understanding that:
1) it comes out on top in the test: wma truly is a superior audio codec since it performs as well as another codec while using a lower bitrate.
2) it comes out on bottom in the test: we can draw no conclusion since the bitrate was much lower.
After all, outcome (1) is the claim made by microsoft.
for wma q75 the inverse would be true, but it would be too difficult to test.
I'm ok with either Shine or Blade. Blade does have an additional advantage of being used in a previous test.
ff123
[a href="index.php?act=findpost&pid=345296"][{POST_SNAPBACK}][/a]
Thanks. If we have two options, and one is even slightly better, why not use that one? We do want to get the maximum out of this test, don't we?
[a href="index.php?act=findpost&pid=345495"][{POST_SNAPBACK}][/a]
Are you trolling, or do you have problems to understand what an anchor is?. We're not looking for the "maximum" for a low anchor, but for the competitors themselves. An anchor is not a competitor. There's not point to try to optimize the low anchor quality; otherwise it wouldn't be an anchor anymore. We simply looking for something bad, really bad.
[a href="index.php?act=findpost&pid=345509"][{POST_SNAPBACK}][/a]
Are you dense, or do you have a problem understanding that I want to make the test as efficient as possible? If we can use the same anchor as in a previous test and that will give us great comparison value, why not make use of it? (...) but I'm a perfectionist and I try to do everything as good as possible.
[a href="index.php?act=findpost&pid=345514"][{POST_SNAPBACK}][/a]
The debate about the anchor is one week old. You didn't proposed BLADE, but HE-AAC. There's nothing wrong about using Blade instead of Shine, but there's a problem with restarting a debate that is supposed to be over. Moreover, Blade wasn't used last year, but two years ago, with a completely different set of sample. For that simple reason, Blade can't be safely considered as "a great comparison value".
How is that the same? LAME beta is just a progression of LAME.
It's a matter of coherence. You can't say on one side "don't use 2-pass because it's not available with the dominant software" and use on the other side encoders such as aoTuV or LAME beta which are not available with dominant softwares. WMP doesn't offer 2-pass encoding, but WMP, iTunes, MMJB, Nero... don't offer 3.97 beta and aoTuV beta 4.51. If we allow LAME and Vorbis to compete with non-popular tools, there's no reason to refuse this right to WMA.
It's what everybody uses and even more people will use in the future. The same with aoTuV.
Everybody can use 2-pass encoding with adequate tools (DbPowerAmp, CLI encoders).
Actually, thinking about it WMA Q50 is really useless for this test. Not only are the bitrates too low, the quality is awful too, as my test (http://www.hydrogenaudio.org/forums/index.php?showtopic=38938) shows.
And damn, I just noticed that the bitrates are wrong there too. #@%$& piece of WMA crap...
fb2k just encoded 48/318 files and the average bitrate of WMA Standard at Q50 is 104 kbps. Should I let it finish or is it enough to see that Q50 is too low?
[a href="index.php?act=findpost&pid=345517"][{POST_SNAPBACK}][/a]
36 songs later and we're still averaging 104 kbps.
[a href="index.php?act=findpost&pid=345521"][{POST_SNAPBACK}][/a]
I fear that you're loosing your time. Two persons have already noted that WMA VBR50 produce too low bitrate, and that VBR 75 produce a too high one. VBR 50/75 are not corresponding to our current needs (~130 kbps). VBR 2-pass would be nice as replacement setting, but our problem is now logistic (collecting tracks, cutting them accurately, etc...). Then you have WMA CBR 128, with all possible reproachs of people which are going to blame your test (and yourself) for having deliberately handicap WMA just to see the other win the test.
If I was at your place, I'd simply forget WMA. I agree since the beginning with Gambit that 4 competitors may be enough. But you can also replace it by another VBR competitor, such as MPC or WMAPro if this latter offer a VBR mode suitable for our needs.
WMP doesn't offer 2-pass encoding, but WMP, iTunes, MMJB, Nero... don't offer 3.97 beta and aoTuV beta 4.51. If we allow LAME and Vorbis to compete with non-popular tools, there's no reason to refuse this right to WMA.
[a href="index.php?act=findpost&pid=345526"][{POST_SNAPBACK}][/a]
I still don't think it's the same. I don't know, can anybody else state their opinion?
Plus, most people that use WMP rip to WMA, iTunes to AAC, Nero offers LAME, dunno about MMJB.
Everybody can use 2-pass encoding with adequate tools (DbPowerAmp, CLI encoders).
[a href="index.php?act=findpost&pid=345526"][{POST_SNAPBACK}][/a]
That's true. But it's also true that nobody will.
Plus, most people that use WMP rip to WMA, iTunes to AAC, Nero offers LAME, dunno about MMJB.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345531")
iTunes is also suitable for MP3 encoding, and the popularity of iTunes encourage me to think that many many people are using it for MP3 ripping.
Nero Digital use LAME, but I don't think that LAME beta is currently provided.
Winamp also use LAME, but you can see in the changelog that they're explicitely waiting for the next stable version to leave the 3.96 engine ([a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38089&hl=winamp]source[/url]).
MMJB and WMP are using Fraunhofer engine.
For aoTuV beta 4.51 (and all other beta), there's only OggDropXPd (and maybe some others). The most popular Vorbis tools are sticking with official encoders.
My opinion:
Please include WMA; just use 2-pass separately on each sample (unless someone provides evidence that CBR will provide better performance). Yes this isn't entirely fair and there will be zealots who will complain, but smart people will be able to interpret the results in a sensible manner. Useful information about the performance of WMA can still be obtained from comparing 2-pass WMA to pure VBR from other encoders.
MP3, AAC, Vorbis, and WMA are the ideal formats for this test and not including one of them would be a poor decision. Just make sure your analysis explains why using 2-pass VBR was not ideal, and mention the samples where it might have hurt/helped WMA.
My opinion:
Please include WMA; just use 2-pass separately on each sample (unless someone provides evidence that CBR will provide better performance).
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345538")
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=345335]http://www.hydrogenaudio.org/forums/index....ndpost&p=345335[/url]
Two-pass on short samples doesn't correspond to the same setting on full track. Did you read the debate?
With 1-pass VBR encoding, the quality only depends on the selected setting (-V5, -q4, etc...) but with 2-pass encoding, the quality also depend from the content of the full track (on which is based the bitrate allocation). Hence the necessity of using WMA 2-pass on the original tracks and nothing else.
[a href="index.php?act=findpost&pid=345428"][{POST_SNAPBACK}][/a]
OK. Agree.
I am currently encoding my collection to Standard Q50 and Q75 and Professional to Q50. Whatever comes close to 128 kbps (+/- 10%; 12 kbps) is going to be an option.
[a href="index.php?act=findpost&pid=345497"][{POST_SNAPBACK}][/a]
Pro Q50 would be a real solution. Have you got the results already?
WMA standard VBR 2-pass
proper bitrate = possible, needs extra steps
tools available = yes, but not in WMP
portability = the end format is standard WMA
WMA Pro VBR
proper bitrate = q50, no extra steps are needed (I'm confident about this)
tools available = yes, but not in WMP
portability = usable on all computers that have a recent WM distro installed, but not with portables (AFAIK)
I actually fear that WMA standard VBR 2-pass might be clearly lower quality with many samples than the other contenders and the test results would partly just repeat older findings (technically it can't be any better than VBR 1-pass if the bitrates match).
WMA Pro might be better and show its potential. However, MS has obviously stated that WMA Pro is mainly intended for compressing hi-resolution formats starting from 44.1 khz/24-bit.
Would it be a good idea to quickly ABX a few samples for getting some idea what we are dealing with? (guruboolez, what samples have been problematic for WMA?). I could encode some samples right away and put them online. They could be listened against lossless and e.g. Vorbis.
[span style='font-size:7pt;line-height:100%']Edit: layout & typo[/span]
Pro Q50 would be a real solution. Have you got the results already?
[a href="index.php?act=findpost&pid=345552"][{POST_SNAPBACK}][/a]
My own preferences if WMA 2-pass can't be used for logistic reasons (sorted by priority):
1/- no replacement at all (4 competitors + 1 anchor)
2/- replacement with WMAPro VBR (if VBR 50 is confirmed to be 130...135 kbps on average)
3/- replacement with MPC -q4
9/- replacement with WMA CBR 128
I only feel uncomfortable with the last choice. The three others suits to me. A poll would be a nice solution to make the choice
Would it be a good idea to quickly ABX a few samples for getting some idea what we are dealing with? (guruboolez, what samples have been problematic for WMA?). I could encode some samples right away and put them online. They could be listened against lossless and e.g. Vorbis.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345554")
I don't have a real experience with WMA. My latest comparison between WMA and WMAPro is [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=10551&st=0]this one[/url]. Note that I used two-pass for both (I wasn't aware about what implied 2-pass encoding).
My opinion based on quick and very small comparison is that WMA isn't competitive (to my demanding ears) at 128 kbps but that WMAPro is very good. BTW, WMAPro ends at the first place on my first classical comparison (http://www.hydrogenaudio.org/forums/index.php?showtopic=14091&hl)I performed two years ago. I kept a good
souvenir from this format
EDIT: link corrected. I must leave now.
what about codingtechnologies aac? it was, afaik, till now never tested and as its available in winamp i guess its not so unimportant
in any case i would use nero aac and itunes vbr aac
of course vorbis and of couse lame
so for me the list is
- codingtechnologies
- nero
- itunes
- vorbis
- lame
discussion about codecs is over now...
I suggest following:
wma at 50 is probably a common used format/setting, portables.
So,
include wma at q50, ie. ca. 104 kbit/s average,
and adopt the other participants to vbr at averaging 104 with real world music.
So, the test target bitrate is lowered from 130 to 104, but would this be bad ?
I think, it is a primary goal of new vbr codecs to achieve good quality for portables at averaged smaller bitrates than 128/130k.
So, try wma at 104k, lame mp3 at V6? (or abr 104) etc, if it is possible to come close to 104 without much sacrifices like adjusting wma to 130.
To adjust wma to 130, has a lot of disadvantages, as we have seen, the 2-pass-wma solutions are either impracticable or useless in real world scenario or flawed in theoretical test concept.
Going down from 130 to 104 will make testing easier for participants, also.
So, the test target bitrate is lowered from 130 to 104, but would this be bad ?[a href="index.php?act=findpost&pid=345583"][{POST_SNAPBACK}][/a]
Yes
what about codingtechnologies aac? it was, afaik, till now never tested and as its available in winamp i guess its not so unimportant
in any case i would use nero aac and itunes vbr aac
of course vorbis and of couse lame
so for me the list is
- codingtechnologies
- nero
- itunes
- vorbis
- lame
[a href="index.php?act=findpost&pid=345573"][{POST_SNAPBACK}][/a]
It's a multi-format test, not an AAC test + Vorbis + LAME. Having two AAC encoders could already be a point of discussion, but Nero and iTunes are apparently not negotiable, so it'll just have to do...
With 1-pass VBR encoding, the quality only depends on the selected setting (-V5, -q4, etc...) but with 2-pass encoding, the quality also depend from the content of the full track (on which is based the bitrate allocation). Hence the necessity of using WMA 2-pass on the original tracks and nothing else.[a href="index.php?act=findpost&pid=345428"][{POST_SNAPBACK}][/a]
OK. Agree.
I think you guys are misplacing something there. The 2passing is only an automated method of achieving an average bitrate, no reason for it to be considered worse than the manual method used for the rest of the codecs vbr modes -observation and selection of each codecs modes to fit the sample corpus.
When 2pass encoding a full track the bitrate of the sample depends on the samples (complexity*length vs the rest of the tracks complexity*length) *the target bitrate for the whole track (which will differ from the samples target bitrate depending on those details) The length and varying complexity of the rest of the tracks informs the variability of the samples bitrate, but doing 2pass of the whole tracks requires a final check and possible tweak of settings to make sure the average target is achieved within the samples' subset of the encoded tracks.
2pass encoding of just the joined sample corpus, will use the same mechanism to achieve the target bitrate for the sample corpus only, no further checks would be required (because the 2pass target is the full check) and the variability of each samples bit allocation will be informed by each's size and complexity relative to the rest of the sample corpus. (this relationship is the same for all the codecs vbr encodeds) I believe the differences in approach would cancel out and the achieved bitrates would be very close to those achieved by the
suitably checked/tweaked full track version.
I just mention that for the forums technical interests, manualy selecting vbr is ok in my reckoning if uncertainties about the 2pass method are prescient.
Pro Q50 would be a real solution. Have you got the results already?
[a href="index.php?act=findpost&pid=345552"][{POST_SNAPBACK}][/a]
Yes, the bitrate is 134 kbps, so it could be an option. I am opening a poll now...
FYI, WMA Std. Q75 produced an average of 160 kbps as Gambit said. Therefore, it's not an option.
Here is a poll regarding the fifth competitor: http://www.hydrogenaudio.org/forums/index....showtopic=39187 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39187)
Could a moderator (Gambit?) please make it poll-only so that replying is not possible?
My opinion:
Please include WMA; just use 2-pass separately on each sample (unless someone provides evidence that CBR will provide better performance).
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345538")
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=345335]http://www.hydrogenaudio.org/forums/index....ndpost&p=345335[/url]
Two-pass on short samples doesn't correspond to the same setting on full track. Did you read the debate?
[a href="index.php?act=findpost&pid=345540"][{POST_SNAPBACK}][/a]
Yes I did. I disagree with your conclusion that this makes it unsuitable to be included in the test unless full tracks are encoded. In my opinion the benefits of including the one of the most popular audio formats outweighs the fact that there may have to be caveats in the analysis.
FYI, WMA Std. Q75 produced an average of 160 kbps as Gambit said. Therefore, it's not an option.
Here is a poll regarding the fifth competitor: http://www.hydrogenaudio.org/forums/index....showtopic=39187 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39187)
Could a moderator (Gambit?) please make it poll-only so that replying is not possible?
[a href="index.php?act=findpost&pid=345607"][{POST_SNAPBACK}][/a]
why not adding codingtechnologies aac to the poll?
why not adding codingtechnologies aac to the poll? [a href="index.php?act=findpost&pid=345610"][{POST_SNAPBACK}][/a]
We already have too many AAC codecs. Having half of the competitors AAC would look too unbalanced, IMO.
FYI, WMA Std. Q75 produced an average of 160 kbps as Gambit said. Therefore, it's not an option.
Here is a poll regarding the fifth competitor: http://www.hydrogenaudio.org/forums/index....showtopic=39187 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39187)
Could a moderator (Gambit?) please make it poll-only so that replying is not possible?
[a href="index.php?act=findpost&pid=345607"][{POST_SNAPBACK}][/a]
why not adding codingtechnologies aac to the poll?
[a href="index.php?act=findpost&pid=345610"][{POST_SNAPBACK}][/a]
Did you read the other replies from this page? It was said already that 2 AAC encoders are enough.
so wma9 is worth more than an aac codec, just because its not aac? that could be called unbalanced
so wma9 is worth more than an aac codec, just because its not aac? that could be called unbalanced
[a href="index.php?act=findpost&pid=345614"][{POST_SNAPBACK}][/a]
Pointless discussion. Please stop.
The decision is either some sort of WMA, nothing or MPC. That's it. AAC can be tested later.
When does the poll close? Please add the info.
When does the poll close? Please add the info.
[a href="index.php?act=findpost&pid=345618"][{POST_SNAPBACK}][/a]
It never will.
Really, depends on how it works out. Maybe Tuesday or something.
OK. A couple of days is better than a few hours for getting meaningful results.
I am postponing the test; updating the first page...
This topic has 15 pages so forgive me if it has been asked or explained somewhere already, but why in the poll (http://www.hydrogenaudio.org/forums/index.php?showtopic=39187&hl=) there's WMA Standard VBR at Q50 if it produces average bitrate of 104kbps, couldn't it be say Q60 to give something closer to 128kbps?
This topic has 15 pages so forgive me if it has been asked or explained somewhere already, but why in the poll (http://www.hydrogenaudio.org/forums/index.php?showtopic=39187&hl=) there's WMA Standard VBR at Q50 if it produces average bitrate of 104kbps, couldn't it be say Q60 to give something closer to 128kbps?
[a href="index.php?act=findpost&pid=345639"][{POST_SNAPBACK}][/a]
There is no Q60. If MS implemented Q60, world would be so much better. No wars, no poverty, etc.
Oh
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]
I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
[a href="index.php?act=findpost&pid=344597"][{POST_SNAPBACK}][/a]
Any news on this?
I noticed that WMP stores encoder quality settings in HKCU\SOFTWARE\MICROSOFT\MediaPlayer\Preferences\WMARecordQuality, values are:
Q0 = 0x19
Q25 = 0x32
Q50 = 0x48
Q75 = 0x5A
Q100 = 0x62
I'm not going to check on my system what would happen if it were set to something between 0x48 & 0x5A, but perhaps it would affect bitrate without problems...
I noticed that WMP stores encoder quality settings in HKCU\SOFTWARE\MICROSOFT\MediaPlayer\Preferences\WMARecordQuality, values are:
Q0 = 0x19
Q25 = 0x32
Q50 = 0x48
Q75 = 0x5A
Q100 = 0x62
I'm not going to check on my system what would happen if it were set to something between 0x48 & 0x5A, but perhaps it would affect bitrate without problems...
[a href="index.php?act=findpost&pid=345648"][{POST_SNAPBACK}][/a]
Q60 doesn't exist as WMA profile. I just tried to encode to Q60 using the VBS that comes with WME and I received the following file:
Windows Media Audio 9.1
VBR Quality 98, 44 kHz, stereo 1-pass VBR
The average bitrate is 449 kbps.
I guess that changing the settings from the registry and encoding with WMP will result in a similar file.
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]
I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
[a href="index.php?act=findpost&pid=344597"][{POST_SNAPBACK}][/a]
Any news on this?
[a href="index.php?act=findpost&pid=345646"][{POST_SNAPBACK}][/a]
Well, I could buy a Q10 ogg of it - but I guess that won't do really.
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]
I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
[a href="index.php?act=findpost&pid=344597"][{POST_SNAPBACK}][/a]
Any news on this?
[a href="index.php?act=findpost&pid=345646"][{POST_SNAPBACK}][/a]
Well, I could buy a Q10 ogg of it - but I guess that won't do really.
[a href="index.php?act=findpost&pid=345685"][{POST_SNAPBACK}][/a]
No, sorry. I have a ~160 kbps VBR version of it myself, but no lossless one. I was hoping that PoisonDan could upload his version.
Instead of the range I wanted first, could someone post 2:54-3:24 (again, assuming long version).
Does anyone have 1:16 - 1:42 of Eminem - White America?
This is exactly why I wanted a damn poll only. I have the feeling that this is going to turn out into a huge bullshit like the other thread did.
Can't you people simply vote and shut up?
[a href="index.php?act=findpost&pid=345676"][{POST_SNAPBACK}][/a]
That's the spirit, eh?
Edit: And 20 days is plenty. If you can't test all samples, you can test only half of them or whatever.
[a href="index.php?act=findpost&pid=345682"][{POST_SNAPBACK}][/a]
20 days is NOT enough. It will just cause less people to participate. And I don't know why the hurry, we waited long enough for this test. A few more days won't hurt and will give a better chance to all the interested people to participate.
My opinion is that we will get more meaningfull results if we don't try to test everything at the same time. There will be less results, less variation when the codec count goes up, and this is especially something to consider for 128kbps test. This wouldn't be so big problem for a low bitrate test.
Better would be to arrange another test relatively quickly after this and include some other codec/codecs, and keep the total codec count down.
[a href="index.php?act=findpost&pid=345650"][{POST_SNAPBACK}][/a]
IMO, that would be a waste of resources. When doing such a big public listening test, I think we should test as many encoders as we can and 5 seems reasonable to me.
[a href="index.php?act=findpost&pid=345653"][{POST_SNAPBACK}][/a]
Well, in my opinion it is indeed waste of resources to try to make people find differences from 5 different codecs at 128kbps most of which are probably better than last time, because I can imagine it's hard for many people and that reflects to results.
Listening fatigue plays so big part here, I can say this from personal experience.
/my opinion
[a href="index.php?act=findpost&pid=345662"][{POST_SNAPBACK}][/a]
I agree with JohnV.
Does anyone have 1:16 - 1:42 of Eminem - White America?
[a href="index.php?act=findpost&pid=345705"][{POST_SNAPBACK}][/a]
I think I can get that.
That's the spirit, eh?
Yes, that's the spirit since you and others threw this thread in the gutter doing some of the dirtiest tricks imaginable.
20 days is NOT enough. It will just cause less people to participate.
Bullshit. Every single one of my tests ran for less than 20 days. Everyone that were really going to participate produced results in that time frame. And I hardly ever had people asking for more time.
This is exactly why I wanted a damn poll only. I have the feeling that this is going to turn out into a huge bullshit like the other thread did.
Can't you people simply vote and shut up?
[a href="index.php?act=findpost&pid=345676"][{POST_SNAPBACK}][/a]
That's the spirit, eh?
[a href="index.php?act=findpost&pid=345713"][{POST_SNAPBACK}][/a]
Coming from the right person, what...
Seriously, I asked people politely not to reply. Few moments later, I can already see some replies. What's wrong with following what I said? Also, as you can see from the last two similies, it was not really meant the way I wrote it.
Edit: And 20 days is plenty. If you can't test all samples, you can test only half of them or whatever.
[a href="index.php?act=findpost&pid=345682"][{POST_SNAPBACK}][/a]
20 days is NOT enough. It will just cause less people to participate. And I don't know why the hurry, we waited long enough for this test. A few more days won't hurt and will give a better chance to all the interested people to participate.
[a href="index.php?act=findpost&pid=345713"][{POST_SNAPBACK}][/a]
Roberto's test ran for about 18 days and he managed to get reliable results. What makes you think 20 are no enough? Only because
you can't make it doesn't mean others can't either.
My opinion is that we will get more meaningfull results if we don't try to test everything at the same time. There will be less results, less variation when the codec count goes up, and this is especially something to consider for 128kbps test. This wouldn't be so big problem for a low bitrate test.
Better would be to arrange another test relatively quickly after this and include some other codec/codecs, and keep the total codec count down.
[a href="index.php?act=findpost&pid=345650"][{POST_SNAPBACK}][/a]
IMO, that would be a waste of resources. When doing such a big public listening test, I think we should test as many encoders as we can and 5 seems reasonable to me.
[a href="index.php?act=findpost&pid=345653"][{POST_SNAPBACK}][/a]
Well, in my opinion it is indeed waste of resources to try to make people find differences from 5 different codecs at 128kbps most of which are probably better than last time, because I can imagine it's hard for many people and that reflects to results.
Listening fatigue plays so big part here, I can say this from personal experience.
/my opinion
[a href="index.php?act=findpost&pid=345662"][{POST_SNAPBACK}][/a]
I agree with JohnV.
[a href="index.php?act=findpost&pid=345713"][{POST_SNAPBACK}][/a]
Why doesn't that surprise me?
Does anyone have 1:16 - 1:42 of Eminem - White America?
[a href="index.php?act=findpost&pid=345705"][{POST_SNAPBACK}][/a]
I think I can get that.
[a href="index.php?act=findpost&pid=345713"][{POST_SNAPBACK}][/a]
Very nice.
Edit: See you closed my other thread. Can people still vote?
That's the spirit, eh?
Yes, that's the spirit since you and others threw this thread in the gutter doing some of the dirtiest tricks imaginable.
[a href="index.php?act=findpost&pid=345714"][{POST_SNAPBACK}][/a]
Yadda, yadda...
20 days is NOT enough. It will just cause less people to participate.
Bullshit. Every single one of my tests ran for less than 20 days. Everyone that were really going to participate produced results in that time frame. And I hardly ever had people asking for more time.
[a href="index.php?act=findpost&pid=345714"][{POST_SNAPBACK}][/a]
I'd like to object that this is a bit different situation as we have December. Which will be very stressful for most people and they won't have much free time until the holidays. And the test will be over then. But do as you like...
I'd like to object that this is a bit different situation as we have December. Which will be very stressful for most people and they won't have much free time until the holidays. And the test will be over then. But do as you like...
[a href="index.php?act=findpost&pid=345717"][{POST_SNAPBACK}][/a]
I think I could extend it to last until December 18th. Anything more than that probably doesn't make sense because of Christmas / New Year and the related "problems".
[a href="index.php?act=findpost&pid=343245"][{POST_SNAPBACK}][/a]
What problems? To be honest, I think people will have more time to test during the holidays. They will have some days free, and you know we are nerds, many of us use the free days to work on our hobby coding projects and stuff like that. So a listening test is perfect for that time. (Yes, I need to get a life. )
[a href="index.php?act=findpost&pid=343270"][{POST_SNAPBACK}][/a]
Seriously, I asked people politely not to reply. Few moments later, I can already see some replies. What's wrong with following what I said? Also, as you can see from the last two similies, it was not really meant the way I wrote it.
[a href="index.php?act=findpost&pid=345716"][{POST_SNAPBACK}][/a]
You can't seriously expect people not wanna discuss that...
Why doesn't that surprise me?
[a href="index.php?act=findpost&pid=345716"][{POST_SNAPBACK}][/a]
Did you actually ever participate in such a test before? Do you realize the difficulty of this test?
Edit: See you closed my other thread. Can people still vote?
[a href="index.php?act=findpost&pid=345716"][{POST_SNAPBACK}][/a]
Meh, seems not. I have reopened it again.
I'd like to object that this is a bit different situation as we have December. Which will be very stressful for most people and they won't have much free time until the holidays. And the test will be over then. But do as you like...
[a href="index.php?act=findpost&pid=345717"][{POST_SNAPBACK}][/a]
I think I could extend it to last until December 18th. Anything more than that probably doesn't make sense because of Christmas / New Year and the related "problems".
[a href="index.php?act=findpost&pid=343245"][{POST_SNAPBACK}][/a]
What problems? To be honest, I think people will have more time to test during the holidays. They will have some days free, and you know we are nerds, many of us use the free days to work on our hobby coding projects and stuff like that. So a listening test is perfect for that time. (Yes, I need to get a life. )
[a href="index.php?act=findpost&pid=343270"][{POST_SNAPBACK}][/a]
[a href="index.php?act=findpost&pid=345718"][{POST_SNAPBACK}][/a]
Erm, yes and? Holidays for me = 24.12.-2.1. And the test ends on the 25th...
Do you realize the difficulty of this test?[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
Me! Me! I realize the difficulty of this test. I conducted 5 128kbps tests
Seriously, I asked people politely not to reply. Few moments later, I can already see some replies. What's wrong with following what I said? Also, as you can see from the last two similies, it was not really meant the way I wrote it.
[a href="index.php?act=findpost&pid=345716"][{POST_SNAPBACK}][/a]
You can't seriously expect people not wanna discuss that...
[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
But I am not interested in that at this time. Everything I asked for was that people simply give their vote for one of the four options. That't is.
Edit: I don't want replies because I KNOW that thread is going to end like this one if I allow replies.
Why doesn't that surprise me?
[a href="index.php?act=findpost&pid=345716"][{POST_SNAPBACK}][/a]
Did you actually ever participate in such a test before? Do you realize the difficulty of this test?
[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
Unfortunately, I didn't. Only in a 64 kbps listening test.
Do you realize the difficulty of this test?[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
Me! Me! I realize the difficulty of this test. I conducted 5 128kbps tests
[a href="index.php?act=findpost&pid=345721"][{POST_SNAPBACK}][/a]
You are probably thinking only from the arranger's point of view. I'm thinking from a tester's point of view. At least 4 of the codecs tested have had improvements.
It may be quite hard for many people to rank the codecs which are quite good, and no amount of extra time is gonna make it much easier. And this will no doubt reflect to the results.
Do you realize the difficulty of this test?[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
Me! Me! I realize the difficulty of this test. I conducted 5 128kbps tests
[a href="index.php?act=findpost&pid=345721"][{POST_SNAPBACK}][/a]
You are probably thinking only from the arranger's point of view. I'm thinking from a tester's point of view. At least 4 of the codecs tested have had improvements.
It may be quite hard for many people to rank the codecs which are quite good, and no amount of extra time is gonna make it much easier. And this will no doubt reflect to the results.
[a href="index.php?act=findpost&pid=345745"][{POST_SNAPBACK}][/a]
Yes, I agree. (After trying to ABX a handful of my own songs in Vorbis at Q4, I realize this is not going to be easy.)
Maybe a future test should be set at a slightly lower bitrate, so as to make it easier to separate the best codecs out there (?). I mean, WMA and MP3 shouldn't be too much of a problem even at ~128, at least not if there are any crashes or sharp noises in the samples.
That's the spirit, eh?
Yes, that's the spirit since you and others threw this thread in the gutter doing some of the dirtiest tricks imaginable.
[a href="index.php?act=findpost&pid=345714"][{POST_SNAPBACK}][/a]
Hmm.. speaking of "some of the dirtiest tricks imaginable," I seem to recall an old saying about a pot and kettle....
Back on topic, I think that Gambit actually has a point. More on that...
Seriously, I asked people politely not to reply. Few moments later, I can already see some replies. What's wrong with following what I said? Also, as you can see from the last two similies, it was not really meant the way I wrote it.
[a href="index.php?act=findpost&pid=345716"][{POST_SNAPBACK}][/a]
You can't seriously expect people not wanna discuss that...
[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
But I am not interested in that at this time. Everything I asked for was that people simply give their vote for one of the four options. That't is.
Edit: I don't want replies because I KNOW that thread is going to end like this one if I allow replies.
[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
As Gambit already pointed out, you can't honestly expect people to not respond on a
discussion board. It simply isn't going to happen that way.
And despite whether you (or me, or any of the other participants of the discussion) like it, this test has become controversial. Although apparently Roberto -- and probably a handful of other people in the thread -- seem to think it is the fault of people like Gambit and I that this thread has been that way; this simply isn't true. It's a reflection of the problematic way in which this test proposal was initially "prepared" and what kind of exceptations various parties seemed to have had, coupled with multiple apparent communication failures.
Regardless, since it
is controversial, you're likely to get continued debate up until the very end and simply requesting it probably won't do anything. No offense Sebastian, really, but I don't think you were prepared to run a test of this sort on this scale, or knew what to expect from the ensuing debate, and it shows. It's good to see you haven't given up, but getting upset at people wanting to debate the issues isn't going to help either.
Since there's been too much discussion to turn back and rethink some of the principles behind the whole thing and start fresh, what can be done to move forward at this point?
1. It seems the main problem still plauging the test is what format to use for the final choice, and how to use it. Since there's still disagreement, and since some don't want to have another discussion about it (i.e., not wanting a discussion but only poll), then assuming you want the appearance of consensus, I think I'd agree with JohnV that maybe it's better to just leave it out and come back later. This possibly brings a couple benefits: a) Disagreement over 2-pass issues can be avoided for now and possibly thought out in a less rushed manner for next time, which brings b) less worry about obtaining samples, and c) more likelyhood of participation and decent results since the overall test will be less tiring. I admit I am disappointed with this idea simply from the prospect of WMA not being tested, especially given the feeling I've had recently, prompted by certain discussions, that perhaps WMA isn't being treated objectively on the boards here. But there has to be a compromise somewhere.
2. If 1 is followed, then a more thorough subsequent test can be held with codecs like ATRAC, MPC, WMA, and WMA Pro, perhaps with some sort of high scoring reference format from the first test. That'd probably make for a more managable and interesting test set. One benefit is that the next time around, lessons learned from this discussion would be more readily applicable. Another is that a quick followup would likely retain some of the momentum of the first test, which could only be a good thing. Perhaps it could even garner more interest than the first. Finally, more people would likely be happy with the overall result of both tests since there would be no need to leave out some of the formats that are being excluded right now. This eliminates the voting problem.
Considering that, I disagree with Sebastian that following 2 would be a waste of resources. The result could be almost as valuable as the initial test, and the pacing could really end up to be beneficial on multiple accounts.
You are probably thinking only from the arranger's point of view.[a href="index.php?act=findpost&pid=345745"][{POST_SNAPBACK}][/a]
I'm thinking from the point of view of "the arranger that dealt with hundreds of testers while conducing his tests". People complained more when I tested 8 codecs on my low bitrate test than when I tested 6 at 128kbps.
I don't see a problem here. My tests had no anchor - so we had 6 real contenders - and, as far as I know, people didn't have trouble with them. Testing 5, no matter if they improved somewhat in the mean time, should pose no real big problem.
I really don't like to be the type of person who nitpicks about these sort of things, but enough is enough For quite some time people have been referring to the process of organizing and executing these tests as "conducing," but this is wrong. It should be "conducting."
You conduct a test or experiment, you do not conduce it. The person in charge is the conductor. Sorry for that interruption, but it is somewhat grating to continue to see such a mixup.
I really don't like to be the type of person who nitpicks about these sort of things, but enough is enough For quite some time people have been referring to the process of organizing and executing these tests as "conducing," but this is wrong. It should be "conducting."[a href="index.php?act=findpost&pid=345772"][{POST_SNAPBACK}][/a]
"people" no. Mostly me. I gotta get over that addiction of writing conducing.
I really don't like to be the type of person who nitpicks about these sort of things, but enough is enough For quite some time people have been referring to the process of organizing and executing these tests as "conducing," but this is wrong. It should be "conducting."[a href="index.php?act=findpost&pid=345772"][{POST_SNAPBACK}][/a]
"people" no. Mostly me. I gotta get over that addiction of writing conducing.
[a href="index.php?act=findpost&pid=345774"][{POST_SNAPBACK}][/a]
Naa, I've seen probably 10 other people say it the same way. It's a very slight difference in spelling so it's easy to mistake, but it has quite a different meaning.
You are probably thinking only from the arranger's point of view.[a href="index.php?act=findpost&pid=345745"][{POST_SNAPBACK}][/a]
I don't see a problem here. My tests had no anchor - so we had 6 real contenders - and, as far as I know, people didn't have trouble with them. Testing 5, no matter if they improved somewhat in the mean time, should pose no real big problem.
[a href="index.php?act=findpost&pid=345767"][{POST_SNAPBACK}][/a]
We will see. I hope you are right. But if people have trouble hearing any difference or ABXing these, the ranking may be quite random from sample to sample. Bringing more codecs in which are relatively near the transparency level, makes it almost exponentially more difficult to rate against other contenders, and not much statistical difference can be achieved in results. Lets hope this won't be the case though and we will get good results.
Even if we should get similar results, it shows us that all encoders grew up and really reached an excellent quality at 128 kbps.
That's the spirit, eh?
Yes, that's the spirit since you and others threw this thread in the gutter doing some of the dirtiest tricks imaginable.
[a href="index.php?act=findpost&pid=345714"][{POST_SNAPBACK}][/a]
Hmm.. speaking of "some of the dirtiest tricks imaginable," I seem to recall an old saying about a pot and kettle....
Back on topic, I think that Gambit actually has a point. More on that...
Seriously, I asked people politely not to reply. Few moments later, I can already see some replies. What's wrong with following what I said? Also, as you can see from the last two similies, it was not really meant the way I wrote it.
[a href="index.php?act=findpost&pid=345716"][{POST_SNAPBACK}][/a]
You can't seriously expect people not wanna discuss that...
[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
But I am not interested in that at this time. Everything I asked for was that people simply give their vote for one of the four options. That't is.
Edit: I don't want replies because I KNOW that thread is going to end like this one if I allow replies.
[a href="index.php?act=findpost&pid=345719"][{POST_SNAPBACK}][/a]
As Gambit already pointed out, you can't honestly expect people to not respond on a discussion board. It simply isn't going to happen that way.
And despite whether you (or me, or any of the other participants of the discussion) like it, this test has become controversial. Although apparently Roberto -- and probably a handful of other people in the thread -- seem to think it is the fault of people like Gambit and I that this thread has been that way; this simply isn't true. It's a reflection of the problematic way in which this test proposal was initially "prepared" and what kind of exceptations various parties seemed to have had, coupled with multiple apparent communication failures.
Regardless, since it is controversial, you're likely to get continued debate up until the very end and simply requesting it probably won't do anything. No offense Sebastian, really, but I don't think you were prepared to run a test of this sort on this scale, or knew what to expect from the ensuing debate, and it shows. It's good to see you haven't given up, but getting upset at people wanting to debate the issues isn't going to help either.
Since there's been too much discussion to turn back and rethink some of the principles behind the whole thing and start fresh, what can be done to move forward at this point?
1. It seems the main problem still plauging the test is what format to use for the final choice, and how to use it. Since there's still disagreement, and since some don't want to have another discussion about it (i.e., not wanting a discussion but only poll), then assuming you want the appearance of consensus, I think I'd agree with JohnV that maybe it's better to just leave it out and come back later. This possibly brings a couple benefits: a) Disagreement over 2-pass issues can be avoided for now and possibly thought out in a less rushed manner for next time, which brings b) less worry about obtaining samples, and c) more likelyhood of participation and decent results since the overall test will be less tiring. I admit I am disappointed with this idea simply from the prospect of WMA not being tested, especially given the feeling I've had recently, prompted by certain discussions, that perhaps WMA isn't being treated objectively on the boards here. But there has to be a compromise somewhere.
2. If 1 is followed, then a more thorough subsequent test can be held with codecs like ATRAC, MPC, WMA, and WMA Pro, perhaps with some sort of high scoring reference format from the first test. That'd probably make for a more managable and interesting test set. One benefit is that the next time around, lessons learned from this discussion would be more readily applicable. Another is that a quick followup would likely retain some of the momentum of the first test, which could only be a good thing. Perhaps it could even garner more interest than the first. Finally, more people would likely be happy with the overall result of both tests since there would be no need to leave out some of the formats that are being excluded right now. This eliminates the voting problem.
Considering that, I disagree with Sebastian that following 2 would be a waste of resources. The result could be almost as valuable as the initial test, and the pacing could really end up to be beneficial on multiple accounts.
[a href="index.php?act=findpost&pid=345764"][{POST_SNAPBACK}][/a]
I never said that my organization was perfect and I admit there were some errors on my side like not defining the exact purpose of the test from the beginning. I also try to learn so that next time, this can be avoided.
However, you can't blame me for everything either. It's not my fault that people started bitching about LAME developers or started insulting Guru. I also don't find it very friendly that Gambit continued to insult me when I decided something he didn't agree with. I doubt you would like to go to school and have the maths teacher insult you because you don't know how to determin the maxiumum and minumum points of a function.
Regarding the poll, I wanted to open the poll to obtain some results quickly without starting a debate like this one. We already ran out of time which is partially my fault. But I didn't imagine that WMA is so problematic and that's why it took so much time. Another reason why we ran out of time is that people consequently failed to understand some points I made clear, like that we don't want to test popular settings, but popular formats at their best settings. Then we spent several pages debatting about a low anchor which has absolutely no relevance to this test. The low anchor should sound bad, that's it.
Anyways, have to jet to school now.
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]
I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
[a href="index.php?act=findpost&pid=344597"][{POST_SNAPBACK}][/a]
Any news on this?
[a href="index.php?act=findpost&pid=345646"][{POST_SNAPBACK}][/a]
Sorry for the long delay. It turns out I only have the radio edit, not the long version. I could still provide this version if you want it...
I'm a bit confused: Is the poll still open? The first post of this thread mentions that WMA Std will be used.
(I'm sorry if I missed a post that clears this up)
The poll is open. Just keep the discussion here, please.
Poll: The Fifth Element (http://www.hydrogenaudio.org/forums/index.php?showtopic=39187)
[span style='font-size:8pt;line-height:100%']P.S./Edit:
- Sebastian hasn't changed the first post (yet) after opening the poll.
- In my opinion it's obvious why he wanted to keep the discussion in one place.
- I've found this discussion normal. It would have been odd if no debate had happened. Though, perhaps some of the replies were not focused enough and based on proper argumentation.[/span]
[span style='font-size:7pt;line-height:100%']Edit 2: typo[/span]
It's not my fault that people started bitching about LAME developers or started insulting Guru. I also don't find it very friendly that Gambit continued to insult me when I decided something he didn't agree with.
[a href="index.php?act=findpost&pid=345840"][{POST_SNAPBACK}][/a]
Well done young padawan, Roberto teached you well... If it takes for me to be the dick of this thread to make the test better, then so be it. But I would point out the flaws I see to anybody, it's nothing personal. And would you rather discuss those things before the test starts with us or get them thrown in your face afterwards by somebody else and hence jeopardize the validity of the results?
And regarding the poll... As of now, there are 97 votes. Judging from previous tests, not even half of those people will participate. So you have people voting that I like to call (yes, I love that word ) "clueless". Because they are interested in the results, which is fine, but they don't realize the difficulty of the test with another contestant. So the results of the poll from people who will actually participate in the test might be completely different.
And regarding the poll... As of now, there are 97 votes. Judging from previous tests, not even half of those people will participate.[a href="index.php?act=findpost&pid=345896"][{POST_SNAPBACK}][/a]
Another poll?
- Sure! I love testing.
- Not sure yet, but probably.
- Seriously doubt, but you never know.
- No. It sucks.
And regarding the poll... As of now, there are 97 votes. Judging from previous tests, not even half of those people will participate.[a href="index.php?act=findpost&pid=345896"][{POST_SNAPBACK}][/a]
Well, there is no way of limiting the "5th element poll" to people who will participate fully in the test, so there's not much to be done about that.
I can't see any harm in hearing what people on these forums in general would want to include in the test.
Edit: To clarify, I mean that "I can't see any harm in hearing which of those options people on these forums would want to include in the test."
I'm a bit confused: Is the poll still open? The first post of this thread mentions that WMA Std will be used.
(I'm sorry if I missed a post that clears this up)
[a href="index.php?act=findpost&pid=345889"][{POST_SNAPBACK}][/a]
The poll is still open, yes.
It's not my fault that people started bitching about LAME developers or started insulting Guru. I also don't find it very friendly that Gambit continued to insult me when I decided something he didn't agree with.
[a href="index.php?act=findpost&pid=345840"][{POST_SNAPBACK}][/a]
Well done young padawan, Roberto teached you well... If it takes for me to be the dick of this thread to make the test better, then so be it. But I would point out the flaws I see to anybody, it's nothing personal. And would you rather discuss those things before the test starts with us or get them thrown in your face afterwards by somebody else and hence jeopardize the validity of the results?
[a href="index.php?act=findpost&pid=345896"][{POST_SNAPBACK}][/a]
WTF are you talking about? Do you want to say that Roberto influenced me to think you're a dick? Sorry to disappoint you, but I found that myself.
And pointing out flaws looks different that starting to bitch and flame IMO. You simply came into this thread and started writing how clueless we all are and how God-like you are. I had to ask you and Dibrom multiple times to provide some constructive critism and not only "this is shit", "you are the most clueless guy in this thread", etc.
And regarding the poll... As of now, there are 97 votes. Judging from previous tests, not even half of those people will participate. So you have people voting that I like to call (yes, I love that word ) "clueless". Because they are interested in the results, which is fine, but they don't realize the difficulty of the test with another contestant. So the results of the poll from people who will actually participate in the test might be completely different.
[a href="index.php?act=findpost&pid=345896"][{POST_SNAPBACK}][/a]
You had no problem with 5 competitors as long as the format YOU wanted to be included (WMA Standard) was tested. Now that that's not possible, you had to find something to bitch about.
"Stay" I don't have - but I could get it (a FLAC) at allofmp3 if no-one else has it...
[a href="index.php?act=findpost&pid=344565"][{POST_SNAPBACK}][/a]
I have it, but I probably won't be able to upload it until Sunday. Hopefully that won't be a problem.
[a href="index.php?act=findpost&pid=344597"][{POST_SNAPBACK}][/a]
Any news on this?
[a href="index.php?act=findpost&pid=345646"][{POST_SNAPBACK}][/a]
Sorry for the long delay. It turns out I only have the radio edit, not the long version. I could still provide this version if you want it...
[a href="index.php?act=findpost&pid=345873"][{POST_SNAPBACK}][/a]
I have to check if the radio edit contains the portion I want.
Come on guys. Chill. There have been a million misunderstandnings so far. Just forget it all.
We just want to take part in a listening test here.
Because they are interested in the results, which is fine, but they don't realize the difficulty of the test with another contestant. So the results of the poll from people who will actually participate in the test might be completely different.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345896")
And did you realized the difficulty when you posted your first messages in the topic? You're now requesting four competitors, but you were first in favour of both wma and wmapro ([a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=341454]Nov 12 2005[/url]). Does it mean that you expected first to remove one of the three major competitors (LAME, Vorbis, AAC)?
Note that it's really difficult to follow your thoughts:
• « I definitely would like to see both WMA Standard and Pro included (...) can say whatever you want about WMA, but it at least had gapless support way before some of the other codecs had it (...) And I'm very much interested in the differences between WMA Standard and Pro. »
http://www.hydrogenaudio.org/forums/index....ndpost&p=341454 (http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=341454)
• « (...) and I think we agreed that it's useless to test WMA Pro »
http://www.hydrogenaudio.org/forums/index....ndpost&p=345495 (http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=345495)
• « #@%$& piece of WMA crap... »
http://www.hydrogenaudio.org/forums/index....ndpost&p=345527 (http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=345527)
Very interesting first, then useless and finally piece of crap And then you wonder about the lack of organisation in the debate?
EDIT: mainly typo
Anyways, regarding MPC, what's the point in testing a dead format? No HW support (maybe with the Rockbox firmware, but that's used by geeks only ), uncertain patent status, painfully slow development (if any), no multi-channel, no high sampling rates, slow seeking, cannot be used in movies...
By the way, you can't even say that the format is the best since Vorbis beat it last time and I am sure AAC slowly reaches its level, too (if it hasn't already).
[a href="index.php?act=findpost&pid=343203"][{POST_SNAPBACK}][/a]
well, now it is clear, why mpc wasn't chosen/considered for the test.
if the conductor is prejudiced...
HW support:
Rockbox -> iriver and more in future probably
but already way more important:
PocketPCs, PPC, PDA
^^ widespread hardware
But this hardware issues cannot be the guideline for HA, to test codecs !
example: Consider Vorbis:
hardware support ?
I cannot go to the local shops and buy a vorbis player. Vorbis is mainly iriver & PPC only also !
The considered WMA pro: hardware support ?!
We cannot and should not decide about hardware here,
here we can test the most modern codecs, even if they are not in use by Joe Average.
To the other "arguments" regarding mpc:
Klemm has opened source, shouldn't be patent issues anymore. The situation is comparable to vorbis somehow, if a patent lawyer wants to find something, he will, as the codecs deal all with music compression
slow development of mpc:
This cannot be an argument. This is a sign, that mpc reached as lossy encoder the limits of development. And for our tests, mpc gets another advantage, because you can use either 1.14 or 1.15(v) as high or medium-high anchor to connect old tests with new tests of the other codecs with newer versions.
The most annoying and even wrong assumptions are following 2:
"Vorbis beat mpc last time ?!"When happened that ?
IIRC, MPC & Vorbis were tied at top of those 2 multiformat 128k tests,
the pre-previous time MPC was a little bit higher rated, but Vorbis inside the statistical margin, and the last test vice versa,
ie. Vorbis a little bit higher, but both in the statistical overlap.
That means, MPC & Vorbis are both the cream.
And especially for these results, MPC should be considered again to compete against the other formats, as then you have a anchor to watch the improvements (or not!) of the other formats.
"I am sure AAC slowly reaches its level, too (if it hasn't already)."oh, we don't need listening tests anymore, instead we ask for the feeling of Sebastian, he will tell us, which is the ranking
Summary:
A conductor shouldn't be prejudiced against certain formats.
A test is about comparing formats, not about deselecting formats prior the test.
We shoudn't care about hardware support, the development goes towards more memory and more cpu capacity. By design of HA, we should deal with possible advanced formats, what the industry does later, we cannot influence much now.
The technics will reveal itself, and therefor, it is up to us, to test without prejudices.
As Dibrom mentioned, and wich occured always during the exorbitant long discussion,
the test purposes haven#t been very clear or convincing, formats for portable players, vbr formats at best possible settings.
portable player: here you introduce already grey shades, unclearness. PPC is portable, but excluded ?
irivers with rockbox are excluded ?
Both exlcusions are unfair, as both are the future and present !
This shows, we should stay open minded and simply concentarte on testing.
Of course,
too many formats in 1 test row, are bad.
But then there should be a roadmap to test some other formats.
Independent from "portable" formats at best settings, I mentioned earlier, that a ranking of common formats with common settings would be very very interesting, to show, how big are the differences to the best of the cream.
especially, to argue with Joe Average at other discussion baords eg., or in real life, eg. even with industry.
About testing:
We should carry out a test to get a ranking with the common formats with most used settings,
and that contains wma at q50, ca. 104k, doesn't it ?
because our new vbr codecs are all advanced and wma is vbr also, and certain codecs claim to have "CD-quality" already at 64k, and vbr can get good quality at lower qualities than old common 128k cbr encodings, it makes a lot sense and our testing probably easier to setup, and easier to carry out out for the testers, if the target quality of the test is not 128 anymore, but ca. 104.
And I don't tolerate an answer like from Busemann before to this idea, who said "no" without reasoning, because I gave a reason.
"A conductor shouldn't be prejudiced against certain formats."
True. But easier said than done.
"A test is about comparing formats, not about deselecting formats prior the test."
True. But you can only include so many encoders in one test. Some kind of selection has to be made. Since Sebastian is conducting the test, he has the final word. If you don't like it - conduct a parallell test of your own.
"We shoudn't care about hardware support, the development goes towards more memory and more cpu capacity."
Don't really understand what you mean. Anyway. This is an open forum. Sebastian can conduct whatever test he likes.
"Of course, too many formats in 1 test row, are bad.
But then there should be a roadmap to test some other formats."
Go ahead and make one.
"Independent from "portable" formats at best settings, I mentioned earlier, that a ranking of common formats with common settings would be very very interesting, to show, how big are the differences to the best of the cream. especially, to argue with Joe Average at other discussion baords eg., or in real life, eg. even with industry."
Sure, I'd like to see that too. Set it up why don't you.
"About testing:
We should carry out a test to get a ranking with the common formats with most used settings, and that contains wma at q50, ca. 104k, doesn't it ?"
Yes. But that's another test in that case. (At least judging from the current poll outcome.)
"because our new vbr codecs are all advanced and wma is vbr also, and certain codecs claim to have "CD-quality" already at 64k, and vbr can get good quality at lower qualities than old common 128k cbr encodings, it makes a lot sense and our testing probably easier to setup, and easier to carry out out for the testers, if the target quality of the test is not 128 anymore, but ca. 104."
I agree, a test at lower bitrate would probably be easier for the tester, and maybe more interesting to some. However, this test has been decided for the 128 kbps range.
"And I don't tolerate an answer like from Busemann before to this idea, who said "no" without reasoning, because I gave a reason."
True, his answer wasn't exactly helpful.
Yes, I knew the weak spots of my writing above, you need to make choices & decisions, i wrote it above.
But I am somehow disappointed, which reasons Sebastian finally wrote about his decision to exclude mpc. Those cannot stand longer without being corrected, eg. in his 1st post to the poll about the 5th format, he links to his flawed arguments contra mpc.
I appreciate his efforts to conduct a 128k test, but the even sometimes hateful discussions here, have shown, that something here is not optimal.
Any test preparation needs a lot of time and efforts, from the guys, who setup the test, and from participants.
At least, a test should give some meaningful results.
of course, everybody can setup any test like he wanna do, but in case, the test goes wrong, he doesn't need to wonder, if there is criticismn or even no participation.
And because these tests even at "low" 128k vbr require efforts (thanks to format developments, don't forget the devs!),
it would be very wise, to still change now eg. the targeted bitrate down to something, like 104k.
As the test should only be about the very popular and commercial formats MP3/Lame, AAC, Vorbis, then WMA standard belongs in adaquate manner to the test ?
(you might notice here, that I don't argue for mpc in this test setup anymore, which i did in past for mpc as known anchor to get comparison with the previous tests in past, but now i argue clearly to include wma standard to this next test, if it should be about popular formats)
So, as solution it wouldn't hurt anybody personally involved here, to adjust the target bitrate of all competitors down to something like 104k ?
or are technical reasons against, any unseen new difficulties, like previously with 128k and wma-problem ?
Another alternative solution comes to my mind, if the primary goal should still be popular formats at 128k, but with wma-standard ?
--> test at 128k abr/cbr
This would give an interesting ranking/result, if wma standard offers 128k abr or cbr. Could the other competitors aac, Vorbis, (lame no problem), be adjusted to 128k abr/cbr ?
iirc, Vorbis might give trouble with this alternative test, a pure vbr mode only ?
If a format has only a pure vbr mode, it gets advantage over the others which are forced to abr/cbr.
Another alternative:
exclusion of wma-standard and test at 128k vbr.
But then the goal of testing popular portable formats is missed clearly.
let#s discuss the other poll alternatives:
None [ 20 ] [19.23%]
WMA Professional, VBR Q50 [ 48 ] [46.15%]
WMA Standard, VBR Q50 [ 14 ] [13.46%]
MusePack [ 22 ] [21.15%]
WMA Professional, VBR Q50
would reach the 128k target, but makes testing wma pro sense in the goal of this test ?
I don't think so, which portables, or hardware can play wma pro besides PC ?
Near to none ?
WMA Standard, VBR Q50
average at 104k,
2 possibilities to test this:
1. adjust target bitrate of the complete test down to ca. 104k.
Any disadvantages ?
I don#t see important ones, I think, all vbr formats mentioned here for the test, should compete more or less well even at 104k vbr. Very interesting format test for portable hardware.
2. test wma-stzandard at q50 together with the others at 128k.
disadvantage: should be clearly unfair test for wma-50, the test results regarding wma-50 won't have a value, wma-50 would be kind of low anchor, maybe it is so good, that.... it is on par with the others, that would be positive surprise, but at bitrates from 100-130k you cannot expect wonders, as all encoders are tuned meanwhile well.
Musepack:
Musepack at q4 at ca. 128k vbr is a very interesting competitor always,
but it isn't very popular, though there is some more or less hardware support, portable.
Advantage to include musepack:
it would give an anchor to compare the results and developemnts of the new format versions to previous 128k tests.
Summary:
We see, that all mentioned solutions have advantages and disadvantages.
The only solution, which might work well with wma-standard, is lowering target bitrate.
But this hardware issues cannot be the guideline for HA, to test codecs !
example: Consider Vorbis:
hardware support ?
I cannot go to the local shops and buy a vorbis player. Vorbis is mainly iriver & PPC only also !
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345917")
[a href="http://wiki.xiph.org/index.php/PortablePlayers]http://wiki.xiph.org/index.php/PortablePlayers[/url]
The considered WMA pro: hardware support ?!
WMAPro is a possibility since it appears that WMA can't be easily take into consideration. There's a vacant place and it could only be taken by a non-hardware-compatible format.
To the other "arguments" regarding mpc:
Klemm has opened source, shouldn't be patent issues anymore. The situation is comparable to vorbis somehow, if a patent lawyer wants to find something, he will, as the codecs deal all with music compression
Open-source is not a criterion of this test. MPC is very different from Vorbis, simply because it sucks at low bitrate. Good performance at low bitrate is apparently a must to become popular. MPC is for a niche. Open-source or not.
slow development of mpc:
This cannot be an argument. This is a sign, that mpc reached as lossy encoder the limits of development.
It's wrong. According to Klemm, there were many point which needed to be developed:
- SV8
- intensity stereo
- tuning of PNS
- problems with low volume sample
If development has end, it's simply because Klemm don't work anymore on the codec and that no developer has skill enough to work deeply on the codec itself.
The most annoying and even wrong assumptions are following 2:
"Vorbis beat mpc last time ?!"
When happened that ?
IIRC, MPC & Vorbis were tied at top of those 2 multiformat 128k tests,
the pre-previous time MPC was a little bit higher rated, but Vorbis inside the statistical margin, and the last test vice versa,
ie. Vorbis a little bit higher, but both in the statistical overlap.
That means, MPC & Vorbis are both the cream.
WMAPro was tied with AAC, Vorbis and MPC on the first listening test. But it wasn't a reason to include it again for the second one.
MPC is good at such bitrate (not always). Nobody is against this idea
now.
Summary:
A conductor shouldn't be prejudiced against certain formats.
True. But as far as I can see it, Sebastian opened a poll to decide between MPC and WMAPro. His prejudice towards MPC doesn't affect the final decision.
A test is about comparing formats, not about deselecting formats prior the test.
Without selection, 15 or more encoders could pretend to be tested.
I did such tests this summer. 12 or 13 encoders were included. I did preliminary pools to make a choice. It's the only solution if you want to test all "valid" encoders and settings. But it's not possible in the current configuration (collective test). Therefore, we
must "deselect formats prior the test".
We shoudn't care about hardware support, the development goes towards more memory and more cpu capacity. By design of HA, we should deal with possible advanced formats, what the industry does later, we cannot influence much now.
The technics will reveal itself, and therefor, it is up to us, to test without prejudices.
The "advanced format" is one criterion among others. I proposed it here (http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=343061). There's no reason to favour it over one another. And I'm not sure that "the design of HA" is a valid argument. The same argument was used in the past by people blaming Roberto (and others) for testing 128 kbps instead of 200 kbps.
As Dibrom mentioned, and wich occured always during the exorbitant long discussion,
the test purposes haven#t been very clear or convincing, formats for portable players, vbr formats at best possible settings.
They were quickly decided. Not on the beginning. But once decided, there's no point to debate about it again -- excepted when a serious issue is revealed (the problem with WMA 2-pass is one of them explaining why the portable argument, which was the major one util yesterday, can't be followed anymore. Hence the poll to let people make their choice between MPC and WMAPro).
PPC is portable, but excluded ?
irivers with rockbox are excluded ?
A PPC is a small computer. Rockbox is just a hack. The industry is not supporting MPC.
So, as solution it wouldn't hurt anybody personally involved here, to adjust the target bitrate of all competitors down to something like 104k ?
or are technical reasons against, any unseen new difficulties, like previously with 128k and wma-problem ?[a href="index.php?act=findpost&pid=345935"][{POST_SNAPBACK}][/a]
Two answers:
- we must be sure than iTunes VBR and LAME VBR could also be set to reach 104 kbps. I doubt so...
- the purpose of the test was to evaluate encoders at 128 kbps. 135 kbps is close to this value; 104 is really far (closer to 96 kbps than 128).
I have nothing against the idea of testing encoders at ~100 kbps; but it would also be nice to see tests at ~130 kbps, with or without WMA.
Another alternative solution comes to my mind, if the primary goal should still be popular formats at 128k, but with wma-standard ?
--> test at 128k abr/cbr
This would give an interesting ranking/result, if wma standard offers 128k abr or cbr. Could the other competitors aac, Vorbis, (lame no problem), be adjusted to 128k abr/cbr ?
Forcing ABR/CBR is not very interesting. With format like Vorbis it has no sense. It's also going against latest development of encoders (LAME VBR, iTunes VBR).
Another alternative:
exclusion of wma-standard and test at 128k vbr.
But then the goal of testing popular portable formats is missed clearly.
Obviously! The portable criterion can't be invoked anymore if WMA is missing. We need to use another main criterion. High Quality VBR as example. Or why not
High Quality VBR encoders still in development? There's only four formats corresponding to this criterion: AAC, MP3, Vorbis, WMAPro. Then, no debate anymore
PPC is portable, but excluded ?
irivers with rockbox are excluded ?
A PPC is a small computer. Rockbox is just a hack. The industry is not supporting MPC.
[a href="index.php?act=findpost&pid=345936"][{POST_SNAPBACK}][/a]
I have taken most of your arguments above already as basis to my writing previously, there is nothing new.
But I comment again, i repeat me, on the hardware arguments.
A explained above, that hardware & industry reasons have a relative weak & questionable, debateable basis. wouldn't consider rockboy as hack, iirc, it is legally, freely programmed, no use of industries patents/firmware.
by coincidence, I know somebody from England well, who was participating at rockbox dev., who previously developed also very successful at DVD-player firmware (<.. that partially deserves more to be called hacking in some respects...).
by design, a dvd-player is quite close to PC, flashing firmwares is supported by industry for widespread players for updates eg.
So, hardware developemnt is a process in progress.
And the limits between hardware players and PC or Pocket-PCs are vanishing,
all in all, it is useless for us, to discuss this.
The consumer decides, if he buys proprietary products as ipod, or general usb stick, or HD-based player, or pocketPC.
Are we here supporters and testers in the name of the industry ?
I don't think so, no.
Please read my ideas above, what can be done, to get an interesting test of popular formats.
I think, testing wma-pro is at last priority, if you test wma-pro, you can include also mpc without any arguing.
Priority:
The question is still, and i have shown possible solutions, of howto include and test wma-standard finally.
edit-addon:
well! If the goal of the test is rewritten, to test only vbr formats at 128k in active development, then you can exclude wma and mpc, but as then the test competitors are down to 4, it is no rpob, to include mpc again as anchor to get comaprison with previous tests, exactly to value the progress of new lame, new vorbis, new aac.
But I am somehow disappointed, which reasons Sebastian finally wrote about his decision to exclude mpc. Those cannot stand longer without being corrected, eg. in his 1st post to the poll about the 5th format, he links to his flawed arguments contra mpc.
[a href="index.php?act=findpost&pid=345935"][{POST_SNAPBACK}][/a]
What is flawed about them?
Portable --> If you count PPC as argument, why not count Notebooks, too? They are portable. And then, Monkey's Audio, VQF, WavPack, LPAC... they're all portable.
Patent --> Being open source doesn't mean that it's patent-free. MPC is based on MP2 which uses patented technology.
What other arguments are flawed?
well! If the goal of the test is rewritten, to test only vbr formats at 128k in active development, then you can exclude wma and mpc, but as then the test competitors are down to 4, it is no rpob, to include mpc again as anchor to get comaprison with previous tests, exactly to value the progress of new lame, new vorbis, new aac.
[a href="index.php?act=findpost&pid=345940"][{POST_SNAPBACK}][/a]
Why should WMA be excluded then? Both WMA Standard and WMA Professional provide VBR encoding and are being developed. The only problem with WMA Standard is that the "pure" VBR mode produces bitrates which are too high or too low for this test. I don't see a problem with WMA Professional, though.
Sebastian, you are absolutely right, you have given answers yourself,
indeed, me and many other people probably use notebooks as portable hardware with mpc, wavpack etc.
Read my thoughts above, then you find, what is wrong about your mpc assumptions, but that won't probably help for the test, no usage.
But if you read, you will find, that I think, the general idea of your test, is it worth, otherwise, I woudn't write here so much.
And you will notice, hopefully, that I don#t go into the endless debates like others here did.
You will find, that I think, that testing popular formats, (important: popular!, instead writing or claiming of industrial hardware portable formats), is necessary.
So, I would try to make a fair comparison of Lame-mp3, Vorbis, AAC & wma-standard,
at fair bitrates,
ie., testing 128k of vbr is of course interesting, but as wma-standard cannot be compared here,
the logical conclusion is:
try to find another bitrate, lower than 128k, which is produceable by all competitors.
edit-addon:
Citing the Vorbis Portable Player page :
Portable Digital Assistants (PDAs)
PDAs are also cable of operating as portable music players using available software applications.
So, pda's (= PPC) are considered to be portable player even by Vorbis wiki, I see no problem here, and so mpc can be played on pda also.
But as reasoned above, it isn't worth to discuss the hardware/industry issues, the limits between hardware players and Pcs are vanishing.
I see no sense in testing MPC again. It has been tested at this bitrate and didn't change much in ages. Also the bitrate is not very likely to be used with MPC in my opinion.
Well, as you can all see, I included MPC to the poll. If MPC wins, I have no problem testing it. I excluded it from the test until now because I gave priority to WMA Standard since it was more popular. However, now that WMA Std. doesn't work the way I (and I guess others, too) thought, we can test MPC.
well! If the goal of the test is rewritten, to test only vbr formats at 128k in active development, then you can exclude wma and mpc, but as then the test competitors are down to 4, it is no rpob, to include mpc again as anchor to get comaprison with previous tests, exactly to value the progress of new lame, new vorbis, new aac.
[a href="index.php?act=findpost&pid=345940"][{POST_SNAPBACK}][/a]
Why should WMA be excluded then? Both WMA Standard and WMA Professional provide VBR encoding and are being developed. The only problem with WMA Standard is that the "pure" VBR mode produces bitrates which are too high or too low for this test. I don't see a problem with WMA Professional, though.
[a href="index.php?act=findpost&pid=345945"][{POST_SNAPBACK}][/a]
the 128k vbr is the point.
it is now fact, that wma standard cannot be tested well at 128k vbr.
And wma-pro offers 128k vbr, but is a weird, not popular (or hardware supported portable) format, as well as mpc and probably many other formats.
See my other suggestions,, solutions to include wma-standard.
Citing my ideas as single sentences will not reveal the complete picture.
ilikedirtthe2nd Posted Today, 03:23 PM
I see no sense in testing MPC again. It has been tested at this bitrate and didn't change much in ages. Also the bitrate is not very likely to be used with MPC in my opinion. exactly that is advatage of mpc, usage as comparable anchor to past tests, to reveal the developemnt of new lame, new aac, new vorbis.
If somebody points to another comparable anchor format to past tests, we can use that instead, the idea is not about mpc, but about anchor for cross-comparisons.
You can imagine, that mpc was developed a lot at q4, because Klemms personal transparency level with mpc at his equipment during developing was below q5, something with q4.xx , iirc.
So, the dev. himself will have carried out a lot of tests around q4, so it was not that surprising that mpc was at top together with vorbis in those previous 128k tests.
Sebastian Mares Posted Today, 03:23 PM
Well, as you can all see, I included MPC to the poll. If MPC wins, I have no problem testing it. I excluded it from the test until now because I gave priority to WMA Standard since it was more popular. However, now that WMA Std. doesn't work the way I (and I guess others, too) thought, we can test MPC. And again, we agree on testing wma-standard is of higher priority than including mpc as comparable anchor or testing it again.
So, i wrote a lot above, to convince you guys, to change the test setup a little bit, to test wma-standard together with lame, aac, vorbis.
The poll shows still, that mpc got ca. 25%, that mpc is still of interest at ha, why not, as mpc is still good quality.
However, you can't blame me for everything either. It's not my fault that people started bitching about LAME developers or started insulting Guru.
Heh. I can only speak for myself, but I didn't insult Guru. I don't remember Gambit insulting him either, but maybe I'm wrong. However, I certainly
do remember Guru resorting to peronsal attacks (and I recall that it was at this point that the split thread took the biggest turn for the worst -- this was shortly after I tried to end it peacefully by the way) on me in the argument multiple times. Seems you got it backwards there, bub.
Of course I don't expect certain folk to remember it so inconveniently...
I also don't find it very friendly that Gambit continued to insult me when I decided something he didn't agree with.
It was rude of Gambit to make the comment that he did. But on the other hand, he was making an observation based on actual behavior. Given all that has transpired in this thread and now this, I can't say that what he said was wrong (sorry), even if it wasn't very tactful to make it a public observation.
I doubt you would like to go to school and have the maths teacher insult you because you don't know how to determin the maxiumum and minumum points of a function.
[a href="index.php?act=findpost&pid=345840"][{POST_SNAPBACK}][/a]
I don't think it was even really about the math (for the most part), which just says another thing about communication...
WTF are you talking about? Do you want to say that Roberto influenced me to think you're a dick? Sorry to disappoint you, but I found that myself.
And pointing out flaws looks different that starting to bitch and flame IMO.
You simply came into this thread and started writing how clueless we all are and how God-like you are. I had to ask you and Dibrom multiple times to provide some constructive critism and not only "this is shit", "you are the most clueless guy in this thread", etc.
Ah, yes, of course. Another one of those "I like criticism, but only when it's not critical, and only when I don't take it personally."
For your information, the original criticisms
were constructive (I notice you didn't even respond to my "constructive criticisms" in my last post -- how's that for selective reading?). They became unconstructive once
someone decided to take them personally and use them as a launching point to rail on with his tirades about how damn evil HA and myself are, and how there are insidious conspiracies against LAME lurking in dark shadows and the hearts of HA admins alike. Or something like that, but probably worse, and
definitely much scarier.
I notice that in both of your complaints, you didn't once fault this person, yet place the blame on Gambit and I. Again, how convenient for you!
You say you aren't influenced? Hrmm.. could have fooled me. You're being influenced by someone's fantasy, whether it's simply yours or someone elses.
*sigh* I'm really surprised to see this sort of nonsense coming from you, after all that has been gone through to end earlier problems in this thread. As test coordinator, I'd have
thought you'd want to keep this kind of thing out of the discussion, and keep the discussion on topic.
But anyway, I guess it's illustrative. You and your buddies have made much of a point by now that you
only want debate on
your terms, within very specific constraints (mostly translating to: "don't question anything relating to the test" ), otherwise you get very upset. That's not the way that HA works (granting exception for forum rules), and certainly shouldn't be the way a
supposedly democractic process in test preparation should transpire. It's simply not worth it to me to continue to bother with this thread under those considerations, so I suppose you won't have to look forward to any more "unconstructive" criticisms from me at any rate. In hind sight, it was quite stupid of me to ever optimistically come back for seconds to begin with....
Sebastian, may I make a suggestion:
Why don't you start over again, in a new thread. We can leave all of this behind us. You can clearly state in the first post what the purpose/goal of the test is, and any discussion can go on from there.
another alternative:
1. test new vorbis, new aac, new lame, (eg. mpc or certain old lame version (3.90.x ?, formats/versions used already in previous 128k listening test) as cross-anchor) at 128k vbr,
2. select a format (of the above) as competitor (or some) at 104k vbr against wma-standard 50.
maybe this is meaningful.
For your information, the original criticisms were constructive (I notice you didn't even respond to my "constructive criticisms" in my last post -- how's that for selective reading?). They became unconstructive once someone decided to take them personally
[a href="index.php?act=findpost&pid=345961"][{POST_SNAPBACK}][/a]
Your original criticisms were maybe constructive (I doubt so), but were totally off-topic. The way LAME developers are working on their encoder has nothing to do or to be debate in this thread.
It was rude of Gambit to make the comment that he did. But on the other hand, he was making an observation based on actual behavior. Given all that has transpired in this thread and now this, I can't say that what he said was wrong (sorry), even if it wasn't very tactful to make it a public observation.
He made an observation?! He treated Sebastian as "clueless person" (before abusing of the moderating tools by removing the insult and cleaning my quote ). Clueless mean "someone who knows nothing about nothing". Is Sebastian a clueless guy? Does he really know nothing? How could you tell that Gambit's words were just an observation?! It's offending. Based on nothing.
It was rude of Gambit to make the comment that he did. But on the other hand, he was making an observation based on actual behavior. Given all that has transpired in this thread and now this, I can't say that what he said was wrong (sorry), even if it wasn't very tactful to make it a public observation.
[a href="index.php?act=findpost&pid=345961"][{POST_SNAPBACK}][/a]
Could you give me some precise and objective facts why you think I am the most clueless guy in this thread?
Edit: You can send me the reasons via PM so we don't have to continue the off-topicness in this thread.
I doubt you would like to go to school and have the maths teacher insult you because you don't know how to determin the maxiumum and minumum points of a function.
[a href="index.php?act=findpost&pid=345840"][{POST_SNAPBACK}][/a]
I don't think it was even really about the math (for the most part), which just says another thing about communication...
[a href="index.php?act=findpost&pid=345961"][{POST_SNAPBACK}][/a]
It was an example. Gambit replied that I am the most clueless guy in this thread after I wrote that WMP encodes to WMA Standard, CBR 128 kbps, with DRM by default. And I wrote this because some other user suggested testing all codecs at their default values.
WTF are you talking about? Do you want to say that Roberto influenced me to think you're a dick? Sorry to disappoint you, but I found that myself.
And pointing out flaws looks different that starting to bitch and flame IMO.
You simply came into this thread and started writing how clueless we all are and how God-like you are. I had to ask you and Dibrom multiple times to provide some constructive critism and not only "this is shit", "you are the most clueless guy in this thread", etc.
Ah, yes, of course. Another one of those "I like criticism, but only when it's not critical, and only when I don't take it personally."
For your information, the original criticisms were constructive (I notice you didn't even respond to my "constructive criticisms" in my last post -- how's that for selective reading?). They became unconstructive once someone decided to take them personally and use them as a launching point to rail on with his tirades about how damn evil HA and myself are, and how there are insidious conspiracies against LAME lurking in dark shadows and the hearts of HA admins alike. Or something like that, but probably worse, and definitely much scarier.
I notice that in both of your complaints, you didn't once fault this person, yet place the blame on Gambit and I. Again, how convenient for you!
You say you aren't influenced? Hrmm.. could have fooled me. You're being influenced by someone's fantasy, whether it's simply yours or someone elses.
*sigh* I'm really surprised to see this sort of nonsense coming from you, after all that has been gone through to end earlier problems in this thread. As test coordinator, I'd have thought you'd want to keep this kind of thing out of the discussion, and keep the discussion on topic.
But anyway, I guess it's illustrative. You and your buddies have made much of a point by now that you only want debate on your terms, within very specific constraints, otherwise you get very upset. That's not the way that HA works (granting exception for forum rules), and certainly shouldn't be the way a supposedly democractic process in test preparation should transpire. It's simply not worth it to me to continue to bother with this thread under those considerations, so I suppose you won't have to look forward to any more "unconstructive" criticisms from me at any rate. In hind sight, it was quite stupid of me to ever optimistically come back for seconds to begin with...
[a href="index.php?act=findpost&pid=345961"][{POST_SNAPBACK}][/a]
No, I don't want to debate on my terms. Whenever asked for, I gave my opinion why something should or shouldn't be done in one way or another. The opinions I had were based on arguments I thought are good enough. The only on-topic discussion we had in this thread was about codecs and settings.
For codecs, I
clearly stated in the first post that Nero AAC, iTunes AAC, LAME and AoTuV are going to be tested. The only redundant format is AAC which is included twice, but that's because Nero released their new encoder and because Ivan and Garf wanted some results from more people and samples. iTunes AAC was included because it performed very well and it would be nice to compare it against the other formats.
What was left was the discussion for either two other competitors or a competitor and a low anchor. I soon realized that a low anchor is needed because of the facts already mentioned. This made room for only one competitor. People suggested WMA Pro, WMA Std, MPC and ATRAC. I gave my opinion on ATRAC: the format is not very popular since only Sony SW/HW players support it (or MDs, but only LP2 encodes to a bitrate similar to 128 kbps and then again, hardware encoders don't produce the same bitstream as software encoders, especially since SonicStage was updated, while MDs units were not). Regarding MPC, please read the post that I linked to in the poll. What was left was WMA Pro and Std. I first wanted to test Pro, but then changed my mind and decided to test Std because it's more popular and therefore more meaningful. I also didn't exclude Pro from an extension test together with ATRAC if people really want it. So, the codec collection was pretty much done.
Oh, low anchor... I decided to use Shine because it simply produces low quality files. Although Blade is crap, too, it features a rudimentary psymodel that gives a slight advantage over Shine - an advantage which is not desired for a low anchor.
Later, we found out that 2-pass VBR with WMA Standard cannot be used and that VBR quality mode produces files with too high/low bitrate. So we had to open the "discussion" about the fifth competitor ("discussion" in quotation marks because I only wanted a poll so I don't have to lose time reading arguments and whatever between stuff like "John Doe: XYZ", "I'm with John Doe", "Yeah, XYZ" or "I think format ABC should be tested and not what you have in the poll"...
The settings discussion also derailed because some people had nothing better to do than start discussing what LAME should use as default and how disorganized LAME developers are and what a bad experience you had. That's why I wanted to test the formats at their best possible settings. If Gabriel (who should know what LAME does) suggested --vbr-new, I use --vbr-new. The same thing I do with Nero AAC. If Ivan recommends XYZ, I will use XYZ.
Anyways, I would really like to stop discussion these personal issues. If you think I am a jerk, fine, do so, I have no problem with that.
If you think this test is a huge bullshit, don't take it.
Sebastian, may I make a suggestion:
Why don't you start over again, in a new thread. We can leave all of this behind us. You can clearly state in the first post what the purpose/goal of the test is, and any discussion can go on from there.
[a href="index.php?act=findpost&pid=345962"][{POST_SNAPBACK}][/a]
And start another bash fest in such a short time? Nah.
The idea is not bad, but why not stick to the topic from now on?
another alternative:
1. test new vorbis, new aac, new lame, (eg. mpc or certain old lame version (3.90.x ?, formats/versions used already in previous 128k listening test) as cross-anchor) at 128k vbr,
2. select a format (of the above) as competitor (or some) at 104k vbr against wma-standard 50.
maybe this is meaningful.
[a href="index.php?act=findpost&pid=345967"][{POST_SNAPBACK}][/a]
I didn't understand that. Could you paraphrase the sentences, please?
Sebastian, may I make a suggestion:
Why don't you start over again, in a new thread. We can leave all of this behind us. You can clearly state in the first post what the purpose/goal of the test is, and any discussion can go on from there.
[a href="index.php?act=findpost&pid=345962"][{POST_SNAPBACK}][/a]
And start another bash fest in such a short time? Nah.
The idea is not bad, but why not stick to the topic from now on?
Because, as you can see, this is going nowhere...
Alternatively, you could just change the title for this topic/thread to "Multiformat 128 kbps Listening Test - pre-test flame war"
Nero AAC, iTunes AAC, LAME and AoTuV
wma-standard --> problems, alternative solutions possible.
You are right with this selection, those are the popular formats.
But to get facts about the popular wma (wma is written at nearly every hardware player today), the test should be configured such, that either wma-standardq50 gets a fair comparison (at 104k), or the test is splitted, the other formats at 128k vbr, then a 2nd test with wma-standard q50 at 104k vbr, with 1 or a few competitors.
about the anchor:
if the anchor is too low, you could add simply noise in a file. The anchor should be low, but not too low, well all is relative. Shine, Blade, any will do.
But for cross comparisons to future and past tests, we should consider a "cross-anchor" concept.
A quite stable format/encoder version, which has meaning, maybe of the past or present, but which isn't too bad performing,
ie. some old Blade version is too bad.
MPC 1.14 or 1.15(v) would suit to this concept,
the specific mpc version depending on,imo, which has been in use in past tests,
or a stable lame, like 3.90.x, which was also well tested and quite well performing overall.
Nero AAC, iTunes AAC, LAME and AoTuV
wma-standard --> problems, alternative solutions possible.
You are right with this selection, those are the popular formats.
But to get facts about the popular wma (wma is written at nearly every hardware player today), the test should be configured such, that either wma-standardq50 gets a fair comparison (at 104k), or the test is splitted, the other formats at 128k vbr, then a 2nd test with wma-standard q50 at 104k vbr, with 1 or a few competitors.
[a href="index.php?act=findpost&pid=345978"][{POST_SNAPBACK}][/a]
Ah, got the point now. Well, such a test would have to be conduced in January the earliest since a parallel test would be too difficult. We can talk about that later.
about the anchor:
if the anchor is too low, you could add simply noise in a file. The anchor should be low, but not too low, well all is relative. Shine, Blade, any will do.
But for cross comparisons to future and past tests, we should consider a "cross-anchor" concept.
A quite stable format/encoder version, which has meaning, maybe of the past or present, but which isn't too bad performing,
ie. some old Blade version is too bad.
MPC 1.14 or 1.15(v) would suit to this concept,
the specific mpc version depending on,imo, which has been in use in past tests,
or a stable lame, like 3.90.x, which was also well tested and quite well performing overall.
[a href="index.php?act=findpost&pid=345978"][{POST_SNAPBACK}][/a]
Something like that is only useful when the same samples and the same listeners are used (this sounds like we "use" listeners ). When changing samples and/or listeners, we can't really do a 1:1 comparison.
But for cross comparisons to future and past tests, we should consider a "cross-anchor" concept.
[a href="index.php?act=findpost&pid=345978"][{POST_SNAPBACK}][/a]
The idea sounds good, but with low anchor + historical reference we have two additional formats featuring in each test. Comparison between tests are also difficult to perform (Sebastian recalled them).
I really believe the 2pass is not a problem if used on the joined sample corpus with minimum gaps between samples and an accurate cue sheet - the same joined corpus and cue sheet could be used for all the codecs, then decoded, recut, and losslessly packed for distribution to testers. Distributing lossless decodes of the all the test samples could make the test 'blinder' and avoid duplication of work and possible errors in the encoding/decoding process.
Wma's 2pass vbr process is a bitrate targeting method only. It will distribute bitrate among the test samples in the same manner as it would if the samples where encoutered inside their full tracks (its all relative -there is no degradation of the 2pass calibrated vbr performance when the individual samples complexity is related to the sample collection - than related to the sample collection inside its source tracks)
Its only true that short duration samples would have slightly more vbr freedom than long samples, and samples relative shortness will differ relating to their parent track or the bulk sample corpus, but the bitrate of all the other manualy chosen vbr settings relate to the corpus only (they are related when they are totaled and checked against the test target bitrate) and* if 2pass vbr on full tracks was used the achieved bitrates of the samples would still need to be summed and checked against the tests target bitrate.
Wma's 2pass vbr process is a bitrate targeting method only. It will distribute bitrate among the test samples in the same manner as it would if the samples where encoutered inside their full tracks
[a href="index.php?act=findpost&pid=345984"][{POST_SNAPBACK}][/a]
Right. But a real track is not composed by 15 "difficult" short samples mixed into one big file. Therefore, the bitrate distribution of WMA 2-pass would differ between our test files and real encodings. It's the biggest concern. We can't test something that don't correspond to what users would get by using the same setting.
I really believe the 2pass is not a problem if used on the joined sample corpus with minimum gaps between samples and an accurate cue sheet - the same joined corpus and cue sheet could be used for all the codecs, then decoded, recut, and losslessly packed for distribution to testers. Distributing lossless decodes of the all the test samples could make the test 'blinder' and avoid duplication of work and possible errors in the encoding/decoding process.
Wma's 2pass vbr process is a bitrate targeting method only. It will distribute bitrate among the test samples in the same manner as it would if the samples where encoutered inside their full tracks (its all relative -there is no degradation of the 2pass calibrated vbr performance when the individual samples complexity is related to the sample collection - than related to the sample collection inside its source tracks)
Its only true that short duration samples would have slightly more vbr freedom than long samples, and samples relative shortness will differ relating to their parent track or the bulk sample corpus, but the bitrate of all the other manualy chosen vbr settings relate to the corpus only (they are related when they are totaled and checked against the test target bitrate) and* if 2pass vbr on full tracks was used the achieved bitrates of the samples would still need to be summed and checked against the tests target bitrate.
[a href="index.php?act=findpost&pid=345984"][{POST_SNAPBACK}][/a]
- Providing all samples losslessly => total package size is over 300 MB. While most people might have DSL, why not make things as easy as possible for the testers?
- It has been said already that using such a synthetic scenario would not reflect real-world usage at all. The problem is that combining all samples would result in one very complex file. Usually, audio files are no complex all the way, but also have parts that are easier to encode.
But a real track is not composed by 15 "difficult" short samples mixed into one big file. Therefore, the bitrate distribution of WMA 2-pass would differ between our test files and real encodings. It's the biggest concern. We can't test something that don't correspond to what users would get by using the same setting.[a href="index.php?act=findpost&pid=345986"][{POST_SNAPBACK}][/a]
But the other codecs achieve the target bitrate using a prescribed setting - their setting also only produces the target bitrate for 15 difficult files, in real world usage their setting is likely to give lower bitrate, in the same way that wma's 2pass chosen (inaccessible) setting would result in a lower bitrate. Its the vbr quality being tested that produces target bitrate for these samples. The 2 pass is just a rather accurate method of targeting bitrate, only it doesnt return the precise settings needed to target the bitrate, just the encoded file.
But the other codecs achieve the target bitrate using a prescribed setting - their setting also only produces the target bitrate for 15 difficult files, in real world usage their setting is likely to give lower bitrate, in the same way that wma's 2pass chosen (inaccessible) setting would result in a lower bitrate.
No, not if I've understood how normal (1-pass) VBR works:
The chosen samples will get the same bitrates, whether included in the original song or not. This is becaue one-pass VBR (vorbis, aac, mp3) sets the bitrate for each frame based only on the complexity of that very frame, i.e. regardless of how easy or difficult the rest of the track is. This is equal to quality targeted encoding, as opposed to bitrate targeted encoding.
Or maybe I just don't understand what you're trying to say.
..The chosen samples will get the same bitrates, whether included in the original song or not. ...Or maybe I just don't understand what you're trying to say.[a href="index.php?act=findpost&pid=345991"][{POST_SNAPBACK}][/a]
Thats not neccessarily your fault
Ill try a different angle.
The test is compare performance of different vbr codecs performance running at optimal settings to produce a target average bitrate across a challenging sample corpus. The settings for most codecs are manualy/expertly prescribed. WMA provides a 2pass method to achieve the bitrate automatically and no way to achieve it manually. I see no problem. The other codecs achieved bitrate *also relates only to the test corpus and does not reflect real world performance in that respect.
But the other codecs achieve the target bitrate using a prescribed setting - their setting also only produces the target bitrate for 15 difficult files, in real world usage their setting is likely to give lower bitrate[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345989")
No, the achieved bitrates for the VBR presets are determined by using a large range of tracks.
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38955]http://www.hydrogenaudio.org/forums/index....showtopic=38955[/url]
Why using VBR 2-pass on just the sample is wrong was explained by Alex B a while ago.
No, the achieved bitrates for the VBR presets are determined by using a large range of tracks.
http://www.hydrogenaudio.org/forums/index....showtopic=38955 (http://www.hydrogenaudio.org/forums/index.php?showtopic=38955)
Actualy, estimated using a large range of tracks, if the bitrate doesnt fit this sample corpus, the setting or corpus would be changed to fit.
edit: The vbr settings are determinded (finally) by the sample corpus used, no more no less.
Forgive me, I don't quite get your last comment...
Since I wasn't around when the last test was conducted - how exactly do you go about distributing a test like this? Are we given a package to download, which includes software and a load of samples and instructions?
You are given a package with all executable files: Java ABC/HR, command line decoders, and batch files to process the samples.
Then, you download each sample package at a time or the whole sample collection at once.
Download this, and read the readme:
http://pessoal.onda.com.br/rjamorim/abc-hr_bin.zip (http://pessoal.onda.com.br/rjamorim/abc-hr_bin.zip)
You'll get an idea on how a listening test works.
Thank you for that Roberto. I have taken a look and am trying to get myself up to speed. The readme is excellent, and I have looked at ff123's page (http://ff123.net/64test/practice.html).
However, Sebastian, can you please confirm that this is the method that you will use? If I am to familiarise myself with ABC/HR Java I just want to check that I'm facing in the right direction!
Will it be 0.5beta, the latest version available at Rarewares?
NB: I have a 95% chance of downloading the necessary files, but a 10% chance of actually returning any results. However I would very much like to participate, to both extend my knowledge of such things and contribute to the results of the test.
Forgive me, I don't quite get your last comment...
[a href="index.php?act=findpost&pid=346004"][{POST_SNAPBACK}][/a]
Its basically not true what you said, that the other codecs settings are determined from a larger range of samples, prelim tests may include more samples but the final achieved average bitrate is for the sample corpus used, it doesnt include parent tracks or other unused samples, and if the bitrate is off target, new setting is required (ideally) or a corpus change (much less ideal)
edit: well it seems this detail is wrong, drat, dont let it get in the way of the flow though - it shouldnt, it couldve gone either way
Since I wasn't around when the last test was conducted - how exactly do you go about distributing a test like this? Are we given a package to download, which includes software and a load of samples and instructions?
You are given a package with all executable files: Java ABC/HR, command line decoders, and batch files to process the samples.
Then, you download each sample package at a time or the whole sample collection at once.
Download this, and read the readme:
http://pessoal.onda.com.br/rjamorim/abc-hr_bin.zip (http://pessoal.onda.com.br/rjamorim/abc-hr_bin.zip)
You'll get an idea on how a listening test works.
Thank you for that Roberto. I have taken a look and am trying to get myself up to speed. The readme is excellent, and I have looked at ff123's page (http://ff123.net/64test/practice.html).
However, Sebastian, can you please confirm that this is the method that you will use? If I am to familiarise myself with ABC/HR Java I just want to check that I'm facing in the right direction!
Will it be 0.5beta, the latest version available at Rarewares?
NB: I have a 95% chance of downloading the necessary files, but a 10% chance of actually returning any results. However I would very much like to participate, to both extend my knowledge of such things and contribute to the results of the test.
[a href="index.php?act=findpost&pid=346013"][{POST_SNAPBACK}][/a]
Yes, I can confirm that.
I hope you can find time to test some samples.
Forgive me, I don't quite get your last comment...
[a href="index.php?act=findpost&pid=346004"][{POST_SNAPBACK}][/a]
Its basically not true what you said, that the other codecs settings are determined from a larger range of samples, prelim tests may include more samples but the final achieved average bitrate is for the sample corpus used, it doesnt include parent tracks or other unused samples, and if the bitrate is off target, new setting is required (ideally) or a corpus change (much less ideal)
[a href="index.php?act=findpost&pid=346015"][{POST_SNAPBACK}][/a]
No, this is not true.
It only makes sense to take the setting that achieves ~ 128 kbps on a large variety of samples and as far as i understand this is also done. If not, the test would be pointless.
If any codec&setting's actual average bitrate for the actual test samples used fell outside target bitrate - adjustments would be made to its setting or the test samples to allow it compete with the others complying with the average bitrate requirement of the test.
Its pure distraction and false to say that the other codecs are targeting the bitrate requirement any more fairly than wma would target directly with 2pass.
The other codecs bitrates are tuned for the sample corpus effectively with a 'manual multipass' method, that wma cant use because its finer settings arent available. WMA can tune its fine settings itself with an automatic 2 pass method. Thats the bare facts of this debate. A real world usage consideration has been applied towards wma's 2pass method which has not been recognised against the others.
Real world usage cant be tested without imitating real world usage, but using plenty of different samples allows differences in test focus and real world usage to average out (the more times you role a dice, the more likely the average score of the summed dice rolls is to be close to the middle)
If any codec&setting's actual average bitrate for the actual test samples used fell outside target bitrate - adjustments would be made to its setting or the test samples to allow it compete with the others complying with the average bitrate requirement of the test.
Its pure distraction and false to say that the other codecs are targeting the bitrate requirement any more fairly than wma would target directly with 2pass.
The other codecs bitrates are tuned for the sample corpus effectively with a 'manual multipass' method, that wma cant use because its finer settings arent available. WMA can tune its fine settings itself with an automatic 2 pass method. Thats the bare facts of this debate. A real world usage consideration has been applied towards wma's 2pass method which has not been recognised against the others.
Real world usage cant be tested without imitating real world usage, but using plenty of different samples allows differences in test focus and real world usage to average out (the more times you role a dice, the more likely the average score of the summed dice rolls is to be close to the middle)
[a href="index.php?act=findpost&pid=346040"][{POST_SNAPBACK}][/a]
Sebastian - can you confirm that this is the method used? I.e. that the bitrate of the other contenders is tuned based on the sample corpus rather than a large batch of full songs and tracks.
No, I cannot confirm that. The settings used are going to be settings that average 128 kbps on a large batch of full songs. If the codecs reach a bitrate that it too high with the given samples in this test, I will replace the one or the other sample.
If I could only find ff123's post regarding a similar decision in a test ran in the past...
No, I cannot confirm that. The settings used are going to be settings that average 128 kbps on a large batch of full songs. If the codecs reach a bitrate too high with the given samples in this thread, I will replace the one or the other sample with a different one.
That doesnt contradict what I said....
If any codec&setting's actual average bitrate for the actual test samples used fell outside target bitrate - adjustments would be made to its setting or the test samples to allow it compete with the others complying with the average bitrate requirement of the test.
Either adjusting the setting or the samples is a deviation from real world usage.
And the fairness of tweaking the test material to raise or lower a codecs utilised bitrate for the challenging corpus is obviously unideal -could benefit or be detrimental to that codecs performance.
But if you decide to do it that way, and find settings averaging 128 kbs for a wide range, then
just 2pass the linked test corpus attached to the extra 'normal material' to produce your average 128kbs wma encode, and do the normal manual multipass of same material, to find/check out the settings for the other codecs to get you their encodes.
(if distibuting all of the samples blindly and losslessly is too much, just distribute the wma decodes losslessly ((if they cant be cut well)) )
No, I cannot confirm that. The settings used are going to be settings that average 128 kbps on a large batch of full songs. If the codecs reach a bitrate too high with the given samples in this thread, I will replace the one or the other sample with a different one.
That doesnt contradict what I said....
If any codec&setting's actual average bitrate for the actual test samples used fell outside target bitrate - adjustments would be made to its setting or the test samples to allow it compete with the others complying with the average bitrate requirement of the test.
Either adjusting the setting or the samples is a deviation from real world usage.
And the fairness of tweaking the test material to raise or lower a codecs utilised bitrate for the challenging corpus is obviously unideal -could benefit or be detrimental to that codecs performance.
But if you decide to do it that way, and find settings averaging 128 kbs for a wide range, then just 2pass the linked test corpus attached to the extra 'normal material' to produce your average 128kbs wma encode, and do the normal manual multipass of same material, to find out the settings for the other codecs to get you their encodes.
[a href="index.php?act=findpost&pid=346054"][{POST_SNAPBACK}][/a]
Oh - sorry - I missed that bit. I think this is getting too advanced for my walnut brain.
Just let me know when I can download the test.
No, I cannot confirm that. The settings used are going to be settings that average 128 kbps on a large batch of full songs. If the codecs reach a bitrate too high with the given samples in this thread, I will replace the one or the other sample with a different one.
That doesnt contradict what I said....
[a href="index.php?act=findpost&pid=346054"][{POST_SNAPBACK}][/a]
It was a direct reply to naylor83's question.
If any codec&setting's actual average bitrate for the actual test samples used fell outside target bitrate - adjustments would be made to its setting or the test samples to allow it compete with the others complying with the average bitrate requirement of the test.
Either adjusting the setting or the samples is a deviation from real world usage.
And the fairness of tweaking the test material to raise or lower a codecs utilised bitrate for the challenging corpus is obviously unideal -could benefit or be detrimental to that codecs performance.
But if you decide to do it that way, and find settings averaging 128 kbs for a wide range, then just 2pass the linked test corpus attached to the extra 'normal material' to produce your average 128kbs wma encode, and do the normal manual multipass of same material, to find out the settings for the other codecs to get you their encodes.
(if distibuting all of the samples blindly and losslessly is too much, just distribute the wma decodes losslessly ((if they cant be cut well)) )
[a href="index.php?act=findpost&pid=346054"][{POST_SNAPBACK}][/a]
Well, when using some settings, the encoders proved already that they produce ~128 kbps with most audio tracks. If they reach an average bitrate that is higher than 140 kbps in this test, that is a special case and that's why I see no problem replacing one particular sample that boosts the bitrate with something less complex. Also, you have to keep in mind that you won't have files that are 100% complex at home (I mean, over the whole length) like you have with 20 or 30 seconds samples.
I think we discussed the 2-pass problem to death now. The poll is also due soon and it seems that WMA Pro won (it can change, though). I will include whatever was democratically elecected as mention already.
Oh - sorry - I missed that bit. I think this is getting too advanced for my walnut brain.
Just let me know when I can download the test.
[a href="index.php?act=findpost&pid=346056"][{POST_SNAPBACK}][/a]
No probs naylor, the subject matter is indeed unweildy and Ive probably contradicted myself here and there. Im just trying to fix some misapprehension of wmas 2pass vbr tuning method, which seems to me to have got stuck in the works and could actualy be a very handy tool here.
When you compare the processes of manualy finding settings to produce vbr encodes averaging a target bitrate for the test corpus+a bulk of normal tracks, and automaticaly producing an optimal vbr encode which (rather reliably) achieves the target bitrate for the same material - the idea that 2passing is innappropriate doesnt stand up for me. The example of how different the samples would sound inside their parent tracks doesnt make sense to me, because the 2 pass discerns and then uses a global vbr setting for its encode. Its just that different routes are being taken to get to the same desired result.
I think we discussed the 2-pass problem to death now. The poll is also due soon and it seems that WMA Pro won (it can change, though). I will include whatever was democratically elecected as mention already.[a href="index.php?act=findpost&pid=346059"][{POST_SNAPBACK}][/a]
Ok, I couldnt spell it clearer, maybe it will grow on you before the next test
At this stage selecting good samples is critical. I wouldn't worry about the bitrates until the samples are gathered. The bitrates will probably be just fine. For example, over 140 kbps is nothing to worry about if all encoders show more or less similar behavior with the same sample. My 25 varied full-length tracks produced this graph:
(http://kotisivu.mtv3.fi/alexb/ha/128bitrates2_gr.gif)[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=344516")
Only the WMA VBR q75 bitrates look a bit strange (besides the too high overall average bitrate for this test). The others have of course variation from track to track because the encoders use different technologies, but the average bitrate is not the only factor that affects the quality.
As we know the current encoders are quite good and there is indeed a danger that the samples will be too transparent for many of the testers. I posted a small test report, which explains a problem I encountered recently: [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=39233&hl=]A little VBR ~88 kbps ABX-test[/url]. It would be interesting to see some HA members' ABX results of the sample I provided.
Edit:
I forgot to mention that these values don't give us any idea what kind of bitrate fluctuation happens between the track start and end. That may also vary greatly between the encoders.
Ok. Now I understand why you would want to change the samples if they turn out to make some encoders produce too high a bitrate.
I guess it's so that the ratings/scores for the encoders represent more or less what they would get for music in general, or if tested with a very large set of songs.
WTF are you talking about? Do you want to say that Roberto influenced me to think you're a dick? Sorry to disappoint you, but I found that myself.
And pointing out flaws looks different that starting to bitch and flame IMO. You simply came into this thread and started writing how clueless we all are and how God-like you are. I had to ask you and Dibrom multiple times to provide some constructive critism and not only "this is shit", "you are the most clueless guy in this thread", etc.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=345905")
The thread speaks for itself. I went back and reread everything, and I stand behind everything I said. And I don't think I was ever inappropriate, with the exception of the "clueless" comment. And with that I quickly recognized my mistake and removed my post. So I always tried to keep this strictly in the discussion level and never said or took anything personally. Can you and your friends say the same? (No need to answer, as I said, the thread speaks for itself.)
It was rude of Gambit to make the comment that he did. But on the other hand, he was making an observation based on actual behavior. Given all that has transpired in this thread and now this, I can't say that what he said was wrong (sorry), even if it wasn't very tactful to make it a public observation.
[a href="index.php?act=findpost&pid=345961"][{POST_SNAPBACK}][/a]
Could you give me some precise and objective facts why you think I am the most clueless guy in this thread?
[a href="index.php?act=findpost&pid=345971"][{POST_SNAPBACK}][/a]
Just FYI. I didn't say you are the most clueless guy in this thread. I said that with your comments you prove that you are one of the most clueless people in this thread. I was mostly refering to the LAME discussion where you obviously didn't understand the problem. And objective facts? Just look at the 25 pages of this thread. You came completely unprepared into this, but that's not the problem. The problem is how you and others deal with criticism.
It was rude of Gambit to make the comment that he did. But on the other hand, he was making an observation based on actual behavior. Given all that has transpired in this thread and now this, I can't say that what he said was wrong (sorry), even if it wasn't very tactful to make it a public observation.
He made an observation?! He treated Sebastian as "clueless person" (before abusing of the moderating tools by removing the insult and cleaning my quote ).
[a href="index.php?act=findpost&pid=345968"][{POST_SNAPBACK}][/a]
I didn't abuse anything. Any user can go back and edit their posts. I decided to take it back, as I realized that some people might take that comment personally, and I wanted to avoid that. Sadly that is something that you fail at.
Note that it's really difficult to follow your thoughts:
• « I definitely would like to see both WMA Standard and Pro included (...) can say whatever you want about WMA, but it at least had gapless support way before some of the other codecs had it (...) And I'm very much interested in the differences between WMA Standard and Pro. »
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=341454]http://www.hydrogenaudio.org/forums/index....ndpost&p=341454[/url]
• « (...) and I think we agreed that it's useless to test WMA Pro »
http://www.hydrogenaudio.org/forums/index....ndpost&p=345495 (http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=345495)
• « #@%$& piece of WMA crap... »
http://www.hydrogenaudio.org/forums/index....ndpost&p=345527 (http://www.hydrogenaudio.org/forums/index.php?showtopic=38723&view=findpost&p=345527)
Very interesting first, then useless and finally piece of crap And then you wonder about the lack of organisation in the debate?
[a href="index.php?act=findpost&pid=345916"][{POST_SNAPBACK}][/a]
Guru, the trick with quoting stuff out of context is getting old. Everybody can go back and see how things really are...
Anyway, this is really getting off-topic, so there is no need to reply. I just wanted to set things straight. I will stay out of this test from now on.
I really wish you a successful test. Good luck.
Come on, please, make this stop...
Edit: I swear this is the last time I write about the things that went wrong with this thread... Warn me if I do it again! The real problem began on page 6 where Dibrom wrote that the test is going into the wrong direction although nothing was decided. Also, you Gambit began bashing Gabriel because you didn't want Shine to be included. You started nitpicking because Gabriel wrote that the anchor shouldn't really be seen as competitor and therefore doesn't really require intensive testing. He obviously didn't use the right words but that doesn't give you the right to call bullshit what he says and also indirectly call him an idiot because you never expected him to write such BS. I think some statements like that are enough to make the whole atmosphere tense.
---
What I wanted to know now is whether you want to decide something regarding the sample or should I create the sample set and you swallow it as-is?
Basically, I have 18 new samples which are all new. We could either make if open for discussion which ones of those 18 to replace with old samples and which ones to keep, or I go over the set myself and replace what I think is appropiate (although that's going to be hard).
This is getting absolutely ridiculous ... maybe it would be better to take WMA completely off the list and add the actual official OggEnc build instead to see how it performs against aotuv ...
This is getting absolutely ridiculous ... maybe it would be better to take WMA completely off the list and add the actual official OggEnc build instead to see how it performs against aotuv ...
[a href="index.php?act=findpost&pid=346105"][{POST_SNAPBACK}][/a]
Well, that would be rather unballanced - we'd have 2 AAC encoders and 2 Vorbis encoders. As mentioned already, I will include whatever wins in the poll about the 5th competitor.
Well, that would be rather unballanced - we'd have 2 AAC encoders and 2 Vorbis encoders. As mentioned already, I will test whatever wins in the poll about the 5th competitor.
[a href="index.php?act=findpost&pid=346106"][{POST_SNAPBACK}][/a]
That's why I already put in my vote for WMA Pro (although I've been for WMA Std in the first place for known reasons) ... it's just the simple fact that I really don't like the way this discussion gets 'personal'.
We claim to be an unbiased community in search for the truth in all matters related to audio but the way some people behave seems to prove the exact opposite.
Sorry ... I just needed to discharge some steam ...
What I wanted to know now is whether you want to decide something regarding the sample or should I create the sample set and you swallow it as-is?
Basically, I have 18 new samples which are all new. We could either make if open for discussion which ones of those 18 to replace with old samples and which ones to keep, or I go over the set myself and replace what I think is appropiate (although that's going to be hard).
[a href="index.php?act=findpost&pid=346100"][{POST_SNAPBACK}][/a]
It wouldn't be useful to publish the raw samples for various reasons. However, what would be very useful is if you could publish a table of the bitrates for the samples x encoders. Ideally the average bitrate should be the same for all encoders (with the exception of the anchor of course).
Perhaps some people are still co-operative and willing to help (without trying to start a new debate about the samples).
I too would like to see a table of the samples. You could include the bitrates produced by the already selected encoders.
You can use my table as a templete in case you don't have one already.
bitrates_public.xls (http://kotisivu.mtv3.fi/alexb/ha/bitrates_public.xls)
The graph part is included in the xls file.
If you can make the samples available perhaps some opinions about the suitability from experienced persons would be good to have, but that is up to you.
It wouldn't be useful to publish the raw samples for various reasons. However, what would be very useful is if you could publish a bitrates for the samples x encoders. Ideally the average bitrate should be the same for all encoders (with the exception of the anchor of course).[a href="index.php?act=findpost&pid=346108"][{POST_SNAPBACK}][/a]
I dont think it matters much if the setting is shown to reach a target for a larger 'targeting corpus' or 'sample corpus' One or the other is preferable to targeting both because manipulations either way could distort the tests difficulty for affected codecs.
I brought up the subject of bitrate targeting to show how wma's 2pass can be used > to target a bitrate, whether thats for sample corpus or target corpus (specified or casualy unspecified /'generaly establishing known behaviour') the found settings are to achieve some targeted bitrate [while allowing normal variation between samples] 2pass vbr can be used in a way which is fairly analogus to the manual 'unspecific' method to hit the same target (bitrate@material type).
I thought it was decided soberly to include wma standard in the test, i can see compelling reasons to do so. You really want to drop it now because of a small technical inconviences and what will be recognised as a misunderstanding once the penny drops?
I dont think it matters much if the setting is shown to reach a target for a larger 'targeting corpus' or 'sample corpus' One or the other is preferable to targeting both because manipulations either way could distort the tests difficulty for affected codecs.
I don't see any reasons to why the average bitrate shouldn't be same over both the small corpus of actual test samples and the nearly infinite corpus which include all music. It's just the same as to say that the sample corpus should be selected so that it is representative to the larger corpus as far as bitrate for the chosen codecs is concerned. It's not unfair in any way - quite the opposite I think...
I brought up the subject of bitrate targeting to show how wma's 2pass can be used > to target a bitrate, whether thats for sample corpus or target corpus (specified or casualy unspecified /'generaly establishing known behaviour') the found settings are to achieve some targeted bitrate [while allowing normal variation between samples] 2pass vbr can be used in a way which is fairly analogus to the manual 'unspecific' method to hit the same target (bitrate@material type).
Well, in my opinion 2-pass vbr can be used if and only if the sample correspond to the "infinite corus" when it comes to average bitrate.
I thought it was decided soberly to include wma standard in the test, i can see compelling reasons to do so. You really want to drop it now because of a small technical inconviences and what will be recognised as a misunderstanding once the penny drops?
[a href="index.php?act=findpost&pid=346115"][{POST_SNAPBACK}][/a]
What are you talking about? Is this directed to me or someone else? I always preferred to test WMA Pro over plain WMA, so I most definitely don't have any problems with this switch. The technical inconveniences you mention have nothing to do with that.
Ideally the average bitrate should be the same for all encoders (with the exception of the anchor of course).
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=346108")
How come? Ideally the VBR encoders are used with the fair predefined quality settings. It doesn't matter what bitrates the individual samples are. This test will test encoders in a quality-based mode (VBR). The bitrate findings are gathered here: [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38955]Bitrate estimates for the upcoming 128 kbps test[/url]
OR
If you mean that the bitrates should be the same for all encoders when a big amount of various music files are encoded, then the suitable quality settings are already found, except for Nero if a new version is going to be tested.
Also, WMA Pro VBR q50 and Musepack q4 fit fine in the target range. Here are the Musepack bitrates for my test file set: Musepack 1.15v q4 = average 133 kbps, median 129 kbps.
[span style='font-size:7pt;line-height:100%'](This set, which I originally gathered for a LAME VBR bitrate evaluation has proven to be quite average for the "various" genre. I observed the bitrate behavior of my LAME aps file archive when I selected the test set. I started with 200 selected tracks. Then I compared the bitrates with my lossless archive and made also several test encodings at lower bitrates. Eventually I ended up with this set of 25 tracks.)[/span]
[span style='font-size:7pt;line-height:100%']Edit: added "if a new Nero..."[/span]
I don't see any reasons to why the average bitrate shouldn't be same over both the small corpus of actual test samples and the nearly infinite corpus which include all music.
because the test corpus is expected to be more challenging, more complex.
It's just the same as to say that the sample corpus should be selected so that it is representative to the larger corpus as far as bitrate for the chosen codecs is concerned. It's not unfair in any way - quite the opposite I think...
I dont think its unfair either, if manipulations where required to make both sample and 'infinite' fit targets those manipulations
could be unfair. Not a big issue for me as i said...
I brought up the subject of bitrate targeting to show how wma's 2pass can be used > to target a bitrate,
Well, in my opinion 2-pass vbr can be used if and only if the sample correspond to the "infinite corus" when it comes to average bitrate.
We are in agreeance then. 2pass can target that bitrate just as well as the rest of the codecs 'manual multipass' can target it - by using a normalised subset of the ideal 'infite corpus'
I thought it was decided soberly to include wma standard in the test, i can see compelling reasons to do so. You really want to drop it now because of a small technical inconviences and what will be recognised as a misunderstanding once the penny drops?
What are you talking about? Is this directed to me or someone else?
Its directed to readers of the thread.
I always preferred to test WMA Pro over plain WMA, so I most definitely don't have any problems with this switch. The technical inconveniences you mention have nothing to do with that.
Your preference is noted
I imagine how through the course of this thread Sebastian Mares is slowly but surely beginning to look more and more like his Avatar.
It wasnt my intention to contribute to that but I fear I may have, sorry mr Mares.
I think its gonna be a hell of a test in the end'
Stick it out
'gnight
Well, looks like WMA Professional is gonna win the poll for the 5th contender.
Good thing for Microsoft and WMA when the test results are out, since most people outside HA probably don't realize there's actually a significant difference between these codecs. The other one, higher quality - almost nowhere used, other one lower quality and used very widely.
Interesting to see the results though.
because the test corpus is expected to be more challenging, more complex. [a href="index.php?act=findpost&pid=346126"][{POST_SNAPBACK}][/a]
Why?
I think you make a couple of mistakes if you do that.
Firstly, who defines complex? Is it defined by looking at how high bitrate the encoders allocate for a particular part? Then you only pick samples where they succeed to allocate an appropriate amount of bits, and exclude those where there might be a flaw in the psymodel and it assignes a low bitrate to a complex sound.
Secondly, if you have some other criteria for complex, then you only sample a small subset of the big music corpus, and will miss inportant aspects of how the encoders perform on other parts - for example if you define complex as having a high PE, then you exclude almost all classical music. And I know some people will complain then.. (Besides favouring ecoders which use PE for their bitrate allocation, but then I'm back to my first argument.)
[partially my fault. But I didn't imagine that WMA is so problematic and that's why it took so much time. [a href="index.php?act=findpost&pid=345840"][{POST_SNAPBACK}][/a]
Already I feel we can draw a very valuable conclusion from all this discussion: that it is one thing to say that wma has good quality compared to other codecs, quite another to prove it. I wonder then how microsoft does it?
Well, looks like WMA Professional is gonna win the poll for the 5th contender.
Good thing for Microsoft and WMA when the test results are out, since most people outside HA probably don't realize there's actually a significant difference between these codecs. The other one, higher quality - almost nowhere used, other one lower quality and used very widely.
Interesting to see the results though.
[a href="index.php?act=findpost&pid=346135"][{POST_SNAPBACK}][/a]
True, but if you think about it...we also say that mp3 is still competitive despite promises of aac replacing it...but this is only valid if the encoder is lame. if we were to do a comparison of (just as example) lame and itunes' mp3 encoder, it would both be mp3 but quality would be quite different...Nevertheless people can also say: mp3 won, so I'll just put mp3 as the default in my itunes.
because the test corpus is expected to be more challenging, more complex. [a href="index.php?act=findpost&pid=346126"][{POST_SNAPBACK}][/a]
Firstly, who defines complex?
Secondly, if you have some other criteria for complex, then you only sample a small subset of the big music corpus, and will miss inportant aspects of how the encoders perform on other parts - for example if you define complex as having a high PE, then you exclude almost all classical music. [a href="index.php?act=findpost&pid=346138"][{POST_SNAPBACK}][/a]
Thirdly, what's a corpus ?
Well, looks like WMA Professional is gonna win the poll for the 5th contender.
Good thing for Microsoft and WMA when the test results are out, since most people outside HA probably don't realize there's actually a significant difference between these codecs. The other one, higher quality - almost nowhere used, other one lower quality and used very widely.
Interesting to see the results though.
[a href="index.php?act=findpost&pid=346135"][{POST_SNAPBACK}][/a]
On a sidenote: Is there a command line encoding utility for WMA Pro? I'd like to try that beast out.
Thirdly, what's a corpus ?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=346194")
[a href="http://en.wikipedia.org/wiki/Corpus]http://en.wikipedia.org/wiki/Corpus[/url]
http://en.wikipedia.org/wiki/Text_corpus (http://en.wikipedia.org/wiki/Text_corpus)
I think we have the 1st portable with WMA Pro support - http://www.dapreview.net/news.php (http://www.dapreview.net/news.php). Btw. I gave my vote to WMA Pro already yesterday.
I think we have the 1st portable with WMA Pro support - http://www.dapreview.net/news.php (http://www.dapreview.net/news.php). Btw. I gave my vote to WMA Pro already yesterday.
I think not. Quote from news:
WMA9 Professional, WMA9 Lossless and copyright-protected WMA files are
not supported.
WMAPro is supported by JVC on a DVD Player:
http://www.fullcompass.com/Products/pages/SKU--83465/ (http://www.fullcompass.com/Products/pages/SKU--83465/)
There's also this one:
http://dv411.com/srdvd100.html (http://dv411.com/srdvd100.html)
But no portable players yet.
because the test corpus is expected to be more challenging, more complex. [a href="index.php?act=findpost&pid=346126"][{POST_SNAPBACK}][/a]
Why?
I think you make a couple of mistakes if you do that.
Firstly, who defines complex? Is it defined by looking at how high bitrate the encoders allocate for a particular part? <snip>
I meant complex casualy as tending to encode with more bits, not for a deliberate policy 'saying for test purposes that the samples
should be more complex, its just that on balance, samples choosen for human listening interests will probably tend to average out higher in bitrate than those choosen entirely randomly.
Theres sure merit for choosing entirely randomly, or making sure the samples alone achieve overall the average bitrate, I could go with that, but then to be honest I dont see my listening abilities being able to perform this test myself. The decisions are up to test participants and conductors. Im interested in the result, much preferably with wma standard included. Im interested in getting people to understand the 'wma 2pass solution' which negates the previously percieved 'wma 2pass problem' So that if you want to test it you can.
JohnV said that Nero will probably use ABR for the test, that makes the unfairness of rejecting 2pass wma standard options even more stark.
And the vote is unfair, i didnt vote in it, because obviously wma standard would be too badly handicapped by using q50 (20% reduction in target bitrate) It wouldnt be handicaped or unfairly advantaged by using 2pass on the joined samples, *especialy if the samples
are to target average bitrate themselves, or a larger targeting corpus is included in the encode.
But during the course of this loopy thread, wma standard has been discouraged for ultimately invalid objections to its only suitable encoding mode - thats gonna reflect poorly on the test and forums expertise for those with the insight to see the 2pass method for what it really is (an ideal automated analogue of manual bitrate targeting processes)
It wouldnt be handicaped or unfairly advantaged by using 2pass on the joined samples, *especialy if the samples are to target average bitrate themselves, or a larger targeting corpus is included in the encode.
[a href="index.php?act=findpost&pid=346238"][{POST_SNAPBACK}][/a]
No, but the distribution of WMA bits in each sample wouldn't be identical to a real world encoding of the same, full tracks at the same setting (128, 2-pass). Therefore the results of the test would be very hard to interpret, or even meaningless.
It wouldnt be handicaped or unfairly advantaged by using 2pass on the joined samples, *especialy if the samples are to target average bitrate themselves, or a larger targeting corpus is included in the encode.
[a href="index.php?act=findpost&pid=346238"][{POST_SNAPBACK}][/a]
No, but the distribution of WMA bits in each sample wouldn't be identical to a real world encoding of the same, full tracks at the same setting (128, 2-pass). Therefore the results of the test would be very hard to interpret, or even meaningless.
[a href="index.php?act=findpost&pid=346241"][{POST_SNAPBACK}][/a]
Neros ABR wouldnt be identical either, and other vbr encodes, depending on their encoders precise implementation wouldnt be precisely the same for encoded sample only, than in-track encodes. Theres also encoding start up differences to fuss over, each vbr individual vbr encode can have 'run-in' differences and 'run-out' differences as the small sample starts and finnishes.
And as for how different the 2pass encodes would sound from their specific settings, firstly if no identifiable advantage or disadvantages can be proposed (very difficult to propose because of the relativistic* nature of the global vbr setting within the final 2pass) particular differences will balance themselves out, helping one sample while hindering another (there are many many such variances in individual performances in a test like this) Secondly the differences, should in fact be very small when the sample corpus bitrate is normalised, or the 2pass bitrate is achieved using an added larger 'targeting corpus'
Its very easy to recognise that there will be differences, you need to look deeper, compare overall processes to rate the importance of the possible differences. As soon as you examine and contrast the (normaly unerecogised) manual process of targeting a bitrate that is used in one way or another to specify or check all the codecs settings - with wma st. automated process, you will realise what these differences amount to in practice - nada
[span style='font-size:7pt;line-height:100%']edit: *, sorry - jargon abused. I wasnt refering to Einstiens theory there, just how the bitrate distribution tends to remain relative across time with a global vbr method even as different mean bitrates are targeted[/span]
Neros ABR wouldnt be identical either, and other vbr encodes, depending on their encoders precise implementation wouldnt be precisely the same for encoded sample only, than in-track encodes. Theres also encoding start up differences to fuss over, each vbr individual vbr encode can have 'run-in' differences and 'run-out' differences as the small sample starts and finnishes.
Didn't think about that.
And as for how different the 2pass encodes would sound from their specific settings, firstly if no identifiable advantage or disadvantages can be proposed (very difficult to propose because of the relativistic nature of the global vbr setting within the final 2pass) particular differences will balance themselves out, helping one sample while hindering another (there are many many such variances in individual performances in a test like this) Secondly the differences, should in fact be very small when the sample corpus bitrate is normalised, or the 2pass bitrate is achieved using an added larger 'targeting corpus'
Its very easy to recognise that there will be differences, you need to look deeper, compare overall processes to rate the importance of the possible differences. As soon as you examine and contrast the (normaly unerecogised) manual process of targeting a bitrate that is used in one way or another to specify or check all the codecs settings - with wma st. automated process, you will realise what these differences amount to in practice - *nada
[a href="index.php?act=findpost&pid=346246"][{POST_SNAPBACK}][/a]
Ok. Can't quite get my mind around that, at least not right now. I'll just take your word for it.
Now - will Sebastian also take your word for it?
Now - will Sebastian also take your word for it?
If they cant visualise what Im saying (the equivalence with manual multipass targeting method that goes on explicitly or in kind) I wouldnt suggest anyone takes
my word for it - a concensus from Codec Dev's would be persuasive.
run-in differences are mitigated by setting abchr-java to start playing 1 second into the sample.
If nero's abr is like lame's abr, it doesn't work the same way as wma vbr 2-pass. That is, it would be a 1-pass abr, so there wouldn't be the same problem with encoding just short samples.
ff123
Well, looks like WMA Professional is gonna win the poll for the 5th contender.
Good thing for Microsoft and WMA when the test results are out, since most people outside HA probably don't realize there's actually a significant difference between these codecs. The other one, higher quality - almost nowhere used, other one lower quality and used very widely.
Interesting to see the results though.
[a href="index.php?act=findpost&pid=346135"][{POST_SNAPBACK}][/a]
On a sidenote: Is there a command line encoding utility for WMA Pro? I'd like to try that beast out.
[a href="index.php?act=findpost&pid=346200"][{POST_SNAPBACK}][/a]
There is WMCmd.vbs which you run with cscript.exe from a console.
The fifth competitor poll can be closed now - the winner is WMA Professional Q50 9.1 which received a total of 60 votes out of 122 (49.18%).
MusePack came second with 25 votes, followed by None with 23 votes and WMA Standard Q50 with 14 votes.
Congrats for the winner!
And now back to the samples...
My foobar (0.8.3) settings:
Encoder: CSCRIPT.EXE
Extension: wma
Parameters: "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" -input %s -output %d -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24
Format is: lossy
Highest BPS mode supported: 24
Tag: none
[ ] Pass floating point data
[ ] Encoder requires accurate length
Display name: WMA PRO Q50
[ ] Hide console window
[ ] Never use source file BPS
Can someone please confirm?
Edit: Original batch file command line:
CSCRIPT.EXE "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" -input %1 -output "%~dpn1" -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24
Edit 2: foobar 0.9 Converter profile for WMA PRO Q50:
Encoder: Custom
Encoder: CSCRIPT.EXE
Extension: wma
Parameters: "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" -input %s -output %d -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24
Format is: lossy
Highest BPS mode supported: 24
Display name: WMA PRO Q50
My foobar (0.8.3) settings:
Encoder: CSCRIPT.EXE
Extension: wma
Parameters: "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" -input %s -output %d -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24
Format is: lossy
Highest BPS mode supported: 24
Tag: none
[ ] Pass floating point data
[ ] Encoder requires accurate length
Display name: WMA PRO Q50
[ ] Hide console window
[ ] Never use source file BPS
Can someone please confirm?
Edit: Original batch file command line:
CSCRIPT.EXE "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" -input %1 -output "%~dpn1" -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24[a href="index.php?act=findpost&pid=346283"][{POST_SNAPBACK}][/a]
Didn't work:
INFO (foo_clienc) : CLI encoder: C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs
INFO (foo_clienc) : Destination file: file://E:\test\sweep.wma
INFO (foo_clienc) : Source file: file://E:\test\sweep.wav
INFO (foo_clienc) : 44100Hz 24bps 2ch
ERROR (foo_clienc) : Encoding failed
The command prompt window that opened shows this:
Microsoft (R) Windows Script Host Version 5.6
Copyright (C) Microsoft Corporation 1996-2001. All rights reserved.
_
The last line is a blinking cursor, but the window is halted.
dbPowerAMP works without problems.
run-in differences are mitigated by setting abchr-java to start playing 1 second into the sample.
If nero's abr is like lame's abr, it doesn't work the same way as wma vbr 2-pass. That is, it would be a 1-pass abr, so there wouldn't be the same problem with encoding just short samples.
Neros 1pass ABR will have
relatively massive run-in discrepancies with the samples encoded insitu. If wmas 2pass targeting is unnacceptable Neros ABR is definitely out of the question.
Congrats for the winner!
And now back to the samples...
The vote was really flawed -you all know it.
Now my last attempt at waving the truth in front you guys:
Suppose 3 phrases joined: A, B and C
It would be wonky for the prepass calculated vbr setting not to be global,
assuming it is, this equation would be true:
phraseA_Bitrate*phraseA_Duration
+phraseB_Bitrate*phraseB_Duration
+phraseC_Bitrate*phraseC_Duration
=target_Bitrate*Total_Duration (=total bit allocation)
Next define Demandrate, a kind of passage complexity estimate from the encoders preferences, high for passages which would demand more bits, low for passages which would demand less.
phrase_Demandrate=phrase_Bitrate/target_Bitrate
phrase_Bitrate=phrase_Demandrate*target_Bitrate
Substituting phrase_Bitrate for its Demandrate expression in previous equation...
(phraseA_Demandrate*target_Bitrate)*phraseA_Duration
+(phraseB_Demandrate*target_Bitrate)*phraseB_Duration
+(phraseC_Demandrate*target_Bitrate)*phraseC_Duration
=target_Bitrate*Total_Duration
devide both sides of equation by target_Bitrate to leave:
phraseA_Demandrate*phraseA_Duration
+phraseB_Demandrate*phraseB_Duration
+phraseC_Demandrate*phraseC_Duration
=Total_Duration
That equation illustrates (by being linear) that the bitrate demands of A and B (Phrase_Demandrate*phraseDuration) remain in proportion with each other, nomatter what the demand of C. To be fair C's demand only needs to be normalised because if it is greater than (A+B)/2, A+B is deprived, if it is less A+B is boosted.
C can be normalised most easily by removing it altogether, alternatively it could be a sample of the same DemandRate as (A+B)/2
The longer C's duration, the greater its DemandRates effect on A and B's bit allocation, at 0 it ceases to have an effect.
The difference between doing the 2pass bitrate targeting on only A+B, and doing it on A+B+C where C is the clipped portion of the parent tracks is
at least as transient to the test results, as the difference between the test samples used, and the tracks people will actualy encode guided by the tests results(!!)
That's me done.
Good luck
[span style='font-size:7pt;line-height:100%']edit: speling[/span]
What I wanted to know now is whether you want to decide something regarding the sample or should I create the sample set and you swallow it as-is?
Repeating myself:
What do you want to test?
Is your sample representative of the population that relates to your test question?
Are you ready to defend that sample set? Can it be validly defended?
Personally, I can live with either decision (your set or a shouting match about which samples "absolutely need" to be included)
I know that opening the sample discussion will cause another chaotic discussion and many will learn in the process.
This will take time, but the sample set might be "better" in the end (in some ways), or perhaps not better at all. At least it'll be different, that's for sure.
Now, is it worth opening this discussion?
Your call, imho. I'd be slightly more inclined to open the discussion on the sample set than take a ready-made set, but you must weigh the pros/cons as the test organizer.
At least the start of the test will be pushed further into the future, if discussion is started. I have no qualms with that, you have to think of your own schedules/time use too.
Best regards,
Halcyon
Didn't work:
INFO (foo_clienc) : CLI encoder: C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs
INFO (foo_clienc) : Destination file: file://E:\test\sweep.wma
INFO (foo_clienc) : Source file: file://E:\test\sweep.wav
INFO (foo_clienc) : 44100Hz 24bps 2ch
ERROR (foo_clienc) : Encoding failed
Thanks for testing. It works fine for me - it was more the settings that I was querying.
Maybe your "WMCmd.vbs" is not in "C:\Program Files\Windows Media Components\Encoder"?
Curious.
INFO (foo_clienc) : CLI encoder: C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs
I think you have incorrectly set "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" as the encoder, not "CSCRIPT.EXE".
If I edit the path to WMCmd.vbs to create an error I get:
INFO (foo_clienc) : CLI encoder: CSCRIPT.EXE
INFO (foo_clienc) : Destination file: file://C:\noname.wma
INFO (foo_clienc) : Source file: file://C:\noname.wav
INFO (foo_clienc) : 44100Hz 24bps 2ch
ERROR (foo_clienc) : Encoding failed
Note the "CLI encoder: CSCRIPT.EXE".
[span style='font-size:8pt;line-height:100%']Edit: spelling[/span]
Neros 1pass ABR will have relatively massive run-in discrepancies with the samples encoded insitu. [span style='font-size:7pt;line-height:100%']edit: speling[/span]
[a href="index.php?act=findpost&pid=346292"][{POST_SNAPBACK}][/a]
In short 10-20 sec hard samples our ABR may reach about 140kbps average because of the initial ABR buffer size. Full length tracks are equal or below 133kbps in almost all cases.
INFO (foo_clienc) : CLI encoder: C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs
I think you have incorrectly set "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" as the encoder, not "CSCRIPT.EXE".[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=346309")
Thanks.
I had a wrong line there and I fixed it. Now it seems to start encoding, but I get this prompt when foobar sends the temp file to the encoder:
"Microsoft ® Console Based Script Host has encountered a problem and needs to close. We are sorry for the inconvenience..."
I have installed about everything related with WM encoding, but nothing separately related with scripting. (Should I install something from e.g. here: [a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnanchor/html/scriptinga.asp]MSDN / WIndows Scripting[/url]?)
Also, Windows Media Encoder itself (the GUI program) doesn't work anymore on this PC. It uses all available memory and crashes when trying to select a source file. I have tried all usual tricks e.g. uninstall/reinstall, search MS knowledgebase etc. It used to work about a year ago...
Only dbPowerAMP works.
Sorry about this being a bit OT, but it is related to a test contender and how people can use it.[span style='font-size:7pt;line-height:100%']Edit: typo[/span]
Now my last attempt at waving the truth in front you guys:
Suppose 3 phrases joined: A, B and C
[ .... ]
[a href="index.php?act=findpost&pid=346292"][{POST_SNAPBACK}][/a]
Ahem... it is SUPPOSED that 128+128+128/3 = 128 . That's not what we talked about. The target_bitrate for A,B,C, and Total are not necessarily equal, so you cannot simplify them like you did.
2-pass Encoding is doing an encode (generally with several tools disabled to speed it up) to get an estimated demandrate for each section of the sample. Depending on the deviation from the targetbitrate, the encoder increases or decreases the target quality, and starts the second encoding (this time, a real encode), probably an ABR one, but taking the previous results in consideration.
1-pass (usual ABR) does not have this "take information" step, It just starts encoding in fragments of a delimited time (like, saying, 2 seconds), and tries to constraint the demandrate to the targetbitrate.
One cannot do a mixture of both, it is either 1-pass or 2-pass.
In short 10-20 sec hard samples our ABR may reach about 140kbps average because of the initial ABR buffer size. Full length tracks are equal or below 133kbps in almost all cases.
[a href="index.php?act=findpost&pid=346317"][{POST_SNAPBACK}][/a]
And an average Kbps of 140, would mean the first section was kicking in even higher than that (and towards the end, the reverse as the ABR algorithm tries balance it out). - Certainly quite a different bitrate distribution than the sample could recieve encoded insitu.
@Jaz - leave the condescension to those who know how to read a freekin' equation
[span style='font-size:7pt;line-height:100%']edit: more seplling and grammur[/span]
@Jaz - leave the condescension to those who know how to read a freekin' equation
Could we please have proper tone...
I too fail to see the point in the formulas you present.
Adding engineering units to the formulas might help you to see what I mean:
phrase_Demandrate
kbps = phrase_Bitrate
kbps / target_Bitrate
kbpsphrase_Bitrate
kbps = phrase_Demandrate
kbps * target_Bitrate
kbpsNow according to my old math book:
kbps / kbps = factor without unit
kbps * kbps = kbps²
It's like apples and pears, you are not supposed to compare them.
Now on the other hand...
If what you meant to say was that phrase_Demandrate is not a rate but a factor, illustrating if the bitrate is above or below the target_Bitrate the formula could be rewritten:
phrase_Demandfactor = phrase_Bitrate
kbps / target_Bitrate
kbpsphrase_Bitrate
kbps = phrase_Demandfactor * target_Bitrate
kbpsFollowing your substitutions the resulting formula would be:
phraseA_Demandfactor * phraseA_Duration
s+ phraseB_Demandfactor * phraseB_Duration
s+ phraseC_Demandfactor * phraseC_Duration
s= Total_Duration
sSince
phraseA_Duration
s+ phraseB_Duration
s+ phraseC_Duration
s= Total_Duration
sthe Demandfactor is just a representation of how much above/below the average bitrate the phrase bitrate is.
Nothing new here.
@Jaz - leave the condescension to those who know how to read a freekin' equation
[span style='font-size:7pt;line-height:100%']edit: more seplling and grammur[/span]
[a href="index.php?act=findpost&pid=346420"][{POST_SNAPBACK}][/a]
You put me in need to open my Oxford Dictionary... Yet, the the verb "condescend" means "to do something that one's rank, abilites etc do not require one to do". That, of course, in the context of your sentence is clueless.
sehested said the rest.
Condescend (http://dictionary.reference.com/search?q=condescend)- To descend to the level of one considered inferior; lower oneself.
- To deal with people in a patronizingly superior manner.
Come on - We (hopefully) just got over a load of flaming let's please not start another war.
Edit: That's to both BTW. And me. And you.
[span style='font-size:14pt;line-height:100%']Interpretation of the poll:[/span]
I am a little surprised, that wma-pro is such a clear winner with close to 50%.
This means, HA members seem to be interested in wma-pro.
None-votes:
Or did they chose wma-pro, as it wouldn't cause probs in the 128k test setup ?
That the test setup is of concern to HA members, shows the high result, that none additional format should be added to test, ie., votes for make a simple test, so that there are no flaws.
HA members don't show much interest in wma-standard, or are concerned about the test setup again, difficulty with 128k target bitrate.
No surprise (for me) at HA: Interest of HA members watching, how MPC performs with new encoders. Maybe those voters considered also the test setup, mpc as comparable anchor.
Each vote could be driven by several parameters, which can be in no particular order:
- personal capabilities regarding of existing portable hardware
- personal interest regarding of future hardware
- scientific (not influenced by above personal interests) curiousity of a ranking of the various formats, so that even unpopular encoders were selected
- thoughts about a good test setup
So, as we are here with a more general scientific and curiousity interest, and we aren#t prejudiced towards encoders in generally, otherwise we would stand on same low level as other groups ("Hifi-Wigwam" ), we will be open to testing all formats, of course considering quality & practical usability.
So, I suggest, testing ogg, acc 2x, lame, maybe with the winner wma-pro,
but considering the whole picture,
I suggest instead something different, as we have aac in 2 modern variants in the test, there is no reason against, to add a second lame, the old but famous 3.90.3.
A 2nd consecutive test should contain then a comparable anchor format encoder, and wma , wma-pro(or instead lame 3.90.3) , mpc, if possible at lower bitrate as 128k, not only because of the wma-standard-128k-problem.
of course, it is not a big problem, if wma-pro and the lame 3.90.3 are swapped in those 2 tests.
@Jaz - leave the condescension to those who know how to read a freekin' equation
Could we please have proper tone...
I got frustrasted because Jaz obviously just skimmed over my post and decided to confront me with the first thing that confused him, after several pages of me patiently trying to get you guys to grapple with the reality of what you have previously rejected for indestinct (lack of) reasoning, Im supposed to take Jaz's 'ahem'ing SUPPOSED nonsensical objections without bother. Maybe I should, but Im weak
I too fail to see the point in the formulas you present.
The point is to model the result of the bitrate targeting process mathematically, to make it easier for those with the means to do so, to conceptualise its nature. Ive been doing alot of mathematical modelling recently and am confident the model is analogous to the 2passes actual performance, it could be tweaked for example with further scaling to Demandrate by a function of target bitrate, but based on the axiom that the prepass discerns a global vbr adjustment its linearity and thus the conclusion I drew from it are secure.
Its true the mathematical rendering of the result shouldnt add anything, but it might help someone with a chance to understand the performance of the process to focus mathematically on the Object in contention.
Adding engineering units to the formulas might help you to see what I mean:
phrase_Demandrate kbps = phrase_Bitrate kbps / target_Bitrate kbps
phrase_Bitrate kbps = phrase_Demandrate kbps * target_Bitrate kbps
Now according to my old math book:
kbps / kbps = factor without unit
kbps * kbps = kbps²
It's like apples and pears, you are not supposed to compare them.
I noticed the possible confusion with units too, but decided to leave the names as they where, so I could waste peoples time who were only reading my posts to find fault rather than reason, to dismiss rather than correct, to fuzz the subject rather than clarify... with a little wild goose chase
DemandRate
never claimed to be same units as Bitrate, it
is bitrate/bitrate,
(phrase_Demandrate=phrase_Bitrate/target_Bitrate)
Its just a variable name, what the name refers to is defined mathematically and in planest possible english I could muster, for your comprehension of my post, if that is your goal.
Nothing new here.
Agreed just another social critique, skillful but still completely missing the point.
People, this is supposed to be an Objective forum no? You've collectively rejected the 2pass method for wma without valid reason, and applied no scrutiny at all to Neros choice of ABR encoding and every attempt ive made to help you come to terms with these things has been met with....windbush....silence.... irrelevant nit-picking or insubstantial redirections. I dont know what else I can say now.
HA / HiFiWigWam whats the difference here ?
I can guess the mob response 'oh hes getting all uppity now, the devs are saying nothing he must be wrong, I can get him with this point he slipped up on even though I havent a clue about most of what he wrote, guru said this, ff123 said that, JohnV's a Nero Engineer he cant be mistaken.'
Want to knock me down a peg or two? Lets have more posts criticising my manner then.
Want to fatigue the arguement and carry on regardless when its knackered? More posts finding faults -real or half imagined.
I suggest stop concentrating on demoting the sole detractor, and put your talents to figuring the subject out yourselves and then explaining your understanding. How much counter explaination has there been??
If Nero go on ahead with submitting their ABR without admitting its acute differences from in-situ encoding (it could hardly be
more different - that should not be hard to see, if you cant see it, this is not your field of expertese and you can forget about understanding how wma's 2pass targeting can
resolve its own differences*)...
....Ill be laughing like a man condemned.
*at least it is VBR! If my model is wrong, its individual bitrate choices can deviate slightly from in-situ encode. Neros ABR bitrate distribution bears very little relation to the in-situ other than the
Average -that is an easily confirmable fact dudes.
freekin' peace,
Why do you keep comparing 2-pass VBR with ABR when they are not the same thing as Guru and others pointed out? And using a synthetical sample that is made of almost 100% of complex material has absolutely no real world meaning.
Why do you keep comparing 2-pass VBR with ABR when they are not the same thing as Guru and others pointed out? And using a synthetical sample that is made of almost 100% of complex material has absolutely no real world meaning.
[a href="index.php?act=findpost&pid=346544"][{POST_SNAPBACK}][/a]
I think he's on to something that we can't grasp. (I can speak for myself anyway.)
Edit: ChiGung - why don't you encode a few tracks and samples and show us the results. Maybe we'd get your point
ChiGung's point is that Nero ABR has a similar (although not identical) problem to wma 2-pass when using short samples to encode. That is, the bitrate if encode the sample is not the same as if you were to encode the entire track.
His suggestion (I think) is to use all of the samples pasted together for wma 2-pass to work on, which is not a bad idea as long as the samples are not biased towards being too complex (which I think previous sample sets have been).
ff123
Why do you keep comparing 2-pass VBR with ABR when they are not the same thing as Guru and others pointed out? And using a synthetical sample that is made of almost 100% of complex material has absolutely no real world meaning.
I think he's on to something that we can't grasp. (I can speak for myself anyway.)
Edit: ChiGung - why don't you encode a few tracks and samples and show us the results. Maybe we'd get your point
[a href="index.php?act=findpost&pid=346548"][{POST_SNAPBACK}][/a]
Because my machines are a mess with other stuff, and i dont have either codec at hand, but Ive been writing my own codec for the last few months and am very familiar with the technologies Im describing. You only need to visualise the basic nature of the different processes to see 'how they compare'
Im comparing them because Wma std was deselected on criteria that the proposed Nero ABR method (of running only on samples) would not pass, and if examined and conducted properly Wma std could pass.
(its not even a criteria I would have thought essential to the tests veracity, but whatever~)
Neros ABR reacts to bit demand as it encounters it and changes its vbr setting (raises or lowers its factor of allocation of bits for demand) to stay within an bitrate allocation limits (the ABR target) over a time limit specified in its design relating to expected minimum playback buffer size. If the audio preceeding a sample is higher than average demand, the ABR process will start by allocating (significantly) less bits at the beginning of the sample than if the preceeding audio was lower than average demand, and through the length of the sample the 'generosity of allocation' will reverse to counter act it. This effect will have a 'half-life' of several seconds at least. To be similar to in-situ encoding ABR must have a section of the preceeding audio as long as the span of its 'playback buffer' design to run in, because of how its bitrate distribution can be very different if it starts straight into the sample.
edit: (this starting of encoding with ABR of sample without its preceeding section, would tend to be a hinderance to the codecs performance rather than a boost - but still a not insignificant difference from in-situ performance)
The Nero developers and others know this but are keeping quiet about it, because theyre either not reading the thread or it suits their agenda to let me go on sounding like a lone madman in the faces of rightly bewildered and innocent testers.
edit:( more cranky paranoid bs no doubt )
It is a little disturbing to me that there has been no substantial feedback from experts about all this. But ill stop taking it seriously -until I forget myself again Im sorry for getting prickly with people, I shouldnt have but Im tired re- explaining the situation. The thread is basically being let down by the usual experts it relies on to put these things straight.
I would like to duck out now.
Users' last post was very well considered and I apologise for leapfrogging it.
ChiGung's point is that Nero ABR has a similar (although not identical) problem to wma 2-pass when using short samples to encode. That is, the bitrate if encode the sample is not the same as if you were to encode the entire track.
His suggestion (I think) is to use all of the samples pasted together for wma 2-pass to work on, which is not a bad idea as long as the samples are not biased towards being too complex (which I think previous sample sets have been).
ff123
[a href="index.php?act=findpost&pid=346570"][{POST_SNAPBACK}][/a]
Ah thankyou ff123 - sentient feedback. Sorry for all the hot air, ill leave you guys to it now.
Regards,
andy
Neros ABR reacts to bit demand as it encounters it and changes its vbr setting (raises or lowers its factor of allocation of bits for demand) to stay within an bitrate allocation limits (the ABR target) over a time limit specified in its design relating to expected minimum playback buffer size. If the audio preceeding a sample is higher than average demand, the ABR process will start by allocating (significantly) less bits at the beginning of the sample than if the preceeding audio was lower than average demand, and through the length of the sample. This effect will have a 'half-life' of several seconds at least. To be similar to in-situ encoding ABR must have a section of the preceeding audio as long as the span of its 'playback buffer' design to run in, because of how its bitrate distribution can be very different if it starts straight into the sample.
First of all, "ABR" is just a CBR with relatively larger bit reservoir. In the new encoder, this bit reservoir is proportional to the final size (of 2.5%), with additional "startup" bitres size, that (by the way) every CBR encoder without a requirement for low startup decoding delay has.
Little bit of theory, in general case of an perceptual encoder with bit reservoir, typical to most CBR and ABR encoders in the market - bit rate control works by trying to maintain the bit reservoir
half-full by adding and donating bit reservoir bits depending on the frame complexity and bitres statistics.
Basically - I don't know what is the big problem with this, this way is the way commercial encoders with bit reservoir worked since the invention of audio coding. Of course, if you drain the bit reservoir, the next "hard to encode" sample would have less bits - but I cannot recall that anyone complained about this in previous tests, including the MPEG ones - only difference is "run in" time (time to fill and empty the bit reservoir, which depends on the bitres size, but also on the encoder strategy, could be pretty quick - or pretty long).
By the way - your "run in / run out" situation could happen on the set of, say, 10-100 silent and pathological samples in theory - so, what do you propose? To eliminate all CBR encoders, or force them to use bit-reservoir of 0 bits in order to make sure all the frames are encoded with the same quality and NOT depending on their order? That is just pure nonsense.
Basically forcing encoders to behave as you wish is not really reflecting typical consumer use - people encode music tracks and expect some constant quality in those tracks. Period - goal of the ABR encoder is to maintain the average bit rate and constant quality
on the music track - which is exactly what we do (or at least I think we do).
If you start putting additional demands, we might end up in a very strange situation that nothing could be tested:
- VBR encoders sometimes do not generate files within 128 kbps limits (hey, let's do Vorbis -q 4.25 with fatboy, velvet and castanets for starters) - also, they might undercode sample set for quality vs. bit rate check - and thanks to this we would increase their quality level and pronounce it "average 128 kbps" -
exactly depending on the material you used for checking the bit rate distribution- Some CBR encoders with bad bit buffer management would drain all their bit reservoir asap, and end up with poor quality (but that is not problem of the test
- Some ABR encoders might encode material with slight difference depending on order of the material - but, as I explained, that can happen even on the frame-level (few zero frames, vice versa)
- 2-pass VBR encoders might do the same (they would, actually)
- CBR encoders would be handicaped if we feed them with material which has PE distribution clearly higher than the test requirements...
Etc... so, at the end - we won't be testing anything as somebody would anyway complain about something.
The Nero developers and others know this but are keeping quiet about it, because theyre either not reading the thread or it suits their agenda to let me go on sounding like a lone madman in the faces of rightly bewildered and innocent testers.
Oh, please...
As far as I am concerned, WMA 2-pass should be tested. But, if I recall correctly - it was rejected because it is:
a) Not widely used
b) 2-pass encoding is not widely available in encoding apps
As far as I am concerned, WMA 2-pass should be tested. But, if I recall correctly - it was rejected because it is:
a) Not widely used
b) 2-pass encoding is not widely available in encoding apps
No not really. WMA 2-pass was selected for this test, but were replaced by WMA Pro due to the following problems with WMA std:
- obtaining full length samples within time frame of this test for proper 2-pass encoding proved impracticable
- alternative settings did not result in bit rates within comparable range for this test
If I have understood ChiGung correctly (I'm not a mathematician) I proposed a quite similar method for VBR 2-pass about 20 pages ago (150s average part - 30 s sample - 150 s average part), but later I realized that it would be
1) too complicated to explain for the general audience
2) not practical for the test conductor.
3) can be claimed to not represent a real life situation, thus unfair
VBR 2-pass has a limited usability in real life. I would rather encode e.g. a video soundtrack using unlimited VBR unless the average bitrate must be exact. Often an exact average bitrate is not needed with 2-channel 128 kbps audio soundtracks, since the major part of the used bandwidth usage consists of the video file anyway and the overall bitrate can be slightly adjusted with the 2-pass video codec settings. The situation may be different with multi-channel audio formats that generally need higher bitrates to sound good.
In my opinion WMA Pro VBR (1-pass) is a bit more interesting contender for this test.
WMA standard VBR 50 (1-pass) should be tested in a separate ~100 kbps test.
Ivan explained many things, but I hope LAME developers can answer to my question about the LAME ABR behavior even the ABR mode is not going to be tested this time. In the past it has been used instead of VBR at lower bitrates. It would be nice to know how it takes into account the preceding and following parts when it determines the bitrate allocation.
As far as I am concerned, WMA 2-pass should be tested. But, if I recall correctly - it was rejected because it is:
a) Not widely used
b) 2-pass encoding is not widely available in encoding apps
Which is precisely the point why some of the issues (not this particular though) are being debated.
If 2-pass is not ok for WMA, because it's not "widely used", then why use Lame developmental versions (betas/alphas), when they are not "widely used".
Why use Aotuv tunings, when they are not "widely used" or "widely available in encoding apps".
To Sebastian:
All this silliness about codec selection can/could've be(en) solved by stating a solid research question.
Is it for example "widely used codecs in their most widely used versions"
or
"Best of breed codecs in their best tuned versions"
If the rules are not the same for all the contestants, it's not really very fair test (and could be criticized in scientific terms).
I know this is beating a dead horse and I don't want to advocate any changes at this point. Organizing these things is hard enough as it is
Maybe this should work as a reminder to future test organizers, about how test methodologies are set up. Proper research question first (formulate a null hypothesis, if you will), everything else after that and based on that.
It is a little disturbing to me that there has been no substantial feedback from experts about all this
I already suggested something that should solve this potential problem: let encoders adapt themselves to the content by not testing the first few seconds of the sample.
If you encode a few seconds at the beginning that are not tested by the listener, you solve the problems of psymodel adaptation and bitrate management adaptation.
Otherwise, of course if in an encoder you define a big enough bit reservoir, some very short samples might totally fit into the remaining reservoir space, leading a very big local bitrate, although the encoder would still respect its bitrate/reservoir constraints.
bit rate control works by trying to maintain the bit reservoir half-full
hint: Targetting an half-full state is perhaps not the best choice It seems to me that usually most people are targetting a low value like 10 or 20% fullness...
I hope LAME developers can answer to my question about the LAME ABR behavior
Our current ABR is quite crude: We are doing bit allocation by lowering the target bitrate by 10% for long blocks, and on short blocks we are allocating based on PE, without considering bitrate.
Overall it works, but this is quite basic.
As far as I am concerned, WMA 2-pass should be tested. But, if I recall correctly - it was rejected because it is:
a) Not widely used
b) 2-pass encoding is not widely available in encoding apps
Which is precisely the point why some of the issues (not this particular though) are being debated.
If 2-pass is not ok for WMA, because it's not "widely used", then why use Lame developmental versions (betas/alphas), when they are not "widely used".
Why use Aotuv tunings, when they are not "widely used" or "widely available in encoding apps".
To Sebastian:
All this silliness about codec selection can/could've be(en) solved by stating a solid research question.
Is it for example "widely used codecs in their most widely used versions"
or
"Best of breed codecs in their best tuned versions"
If the rules are not the same for all the contestants, it's not really very fair test (and could be criticized in scientific terms).
I know this is beating a dead horse and I don't want to advocate any changes at this point. Organizing these things is hard enough as it is
Maybe this should work as a reminder to future test organizers, about how test methodologies are set up. Proper research question first (formulate a null hypothesis, if you will), everything else after that and based on that.
[a href="index.php?act=findpost&pid=346596"][{POST_SNAPBACK}][/a]
For God's sakes guys. If I were Sebastian I would abandon this whole listening test because you guys just cannot move on.
Guru set out the goals quite clearly. We want to test the Latest encoders (after all we want to test progress), but only the ones that have HW support. This would mean that WMA Std in 2 pass mode. Sadly we cannot do 2-pass because that would required to be executed on full tracks and Sebastian rightly do not want to get embroiled on copyright issues. So there was a poll what to do with WMA. People chose WMA Pro.
Yes the goals are not fully met, but we have to move on. Why do we need to do this navel-gazing all the time.
While I am at it @Gambit and @Dibrom for their recent criticisms
Sage advices while sitting on the fence regardless however constructive they seem to be are unnecessary. There is a point in any teamwork, when people have to move on and when a constructive-looking critisim is a hindrance and can easily jepordize the whole project. Especially when the whole thing is an voluntary work by Sebastian Guru and a little minority who drives the whole thing. Very intelligent and rightly-respected people like you should realize that and for a greater benefit should keep quiet.
We should be talking what samples we should use etc, not what codec to use.
Triza
I will post the samples I have in a few minutes as HE-AAC or MP3 so you have an idea. What I am looking for are some orchestral samples. Hope that Guru can post some or I am going to use from Roberto's test. Also, I still have hopes that PoisonDan is going to post the Sash! sample I requested.
Sadly we cannot do 2-pass because that would required to be executed on full tracks and Sebastian rightly do not want to get embroiled on copyright issues.
Sorry if already asked, but i don't think anybody would get into trouble for doing this, i mean, he won't keep the whole song, right? He would just encode the file, decode, cut, and delete. In fact, a 30 segs clip is as legal as this. (i have seen no laws about the 30 secs clips being allowed). If we wanted to be strict to the law, we whould be asking for permissions even on 30 secs tracks. Cripping the test because of this is sad.
Sorry, it is not an option. I know what happened to an HA administrator because of the great German laws. Discussion is over.
then the test won't be serving its purpose. as stated, an econder would adapt to a whole track diferently than to a 30 secs clip.
edit: maybe we can buy CDs, with a donation.
then the test won't be serving its purpose. as stated, an econder would adapt to a whole track diferently than to a 30 secs clip.
edit: maybe we can buy CDs, with a donation.
[a href="index.php?act=findpost&pid=346637"][{POST_SNAPBACK}][/a]
Huh? Don't we run listening tests with samples for at least three years? There was nothing wrong with them until now.
And buying CDs now is too late. I don't want to postpone the test again.
Samples: http://www.hydrogenaudio.org/forums/index....showtopic=39288 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39288)
Huh? Don't we run listening tests with samples for at least three years? There was nothing wrong with them until now.
And buying CDs now is too late. I don't want to postpone the test again.
its just my opinion. one tends to try make it better each time. i understand you desire to do it. but i think a postponed and correct test is better than a questioned but sooner one. anyway, don't take ofence, is just a thought.
Huh? Don't we run listening tests with samples for at least three years? There was nothing wrong with them until now.
And buying CDs now is too late. I don't want to postpone the test again.
its just my opinion. one tends to try make it better each time. i understand you desire to do it. but i think a postponed and correct test is better than a questioned but sooner one. anyway, don't take ofence, is just a thought.
[a href="index.php?act=findpost&pid=346642"][{POST_SNAPBACK}][/a]
I still fail to see why you call it "questioned". Were Roberto's or Guru's tests questioned because samples were encoded instead of full tracks?
ChiGung's point is that Nero ABR has a similar (although not identical) problem to wma 2-pass when using short samples to encode. That is, the bitrate if encode the sample is not the same as if you were to encode the entire track.
His suggestion (I think) is to use all of the samples pasted together for wma 2-pass to work on, which is not a bad idea as long as the samples are not biased towards being too complex (which I think previous sample sets have been).
ff123
[a href="index.php?act=findpost&pid=346570"][{POST_SNAPBACK}][/a]
Ah thankyou ff123 - sentient feedback. Sorry for all the hot air, ill leave you guys to it now.
Regards,
andy
[a href="index.php?act=findpost&pid=346579"][{POST_SNAPBACK}][/a]
Thanks for the clean short version, ff123. I with you now, ChiGung.
ChiGung's point is that Nero ABR has a similar (although not identical) problem to wma 2-pass when using short samples to encode. That is, the bitrate if encode the sample is not the same as if you were to encode the entire track.
His suggestion (I think) is to use all of the samples pasted together for wma 2-pass to work on, which is not a bad idea as long as the samples are not biased towards being too complex (which I think previous sample sets have been).
ff123
[a href="index.php?act=findpost&pid=346570"][{POST_SNAPBACK}][/a]
Ah thankyou ff123 - sentient feedback. Sorry for all the hot air, ill leave you guys to it now.
Regards,
andy
[a href="index.php?act=findpost&pid=346579"][{POST_SNAPBACK}][/a]
Thanks for the clean short version, ff123. I with you now, ChiGung.
[a href="index.php?act=findpost&pid=346645"][{POST_SNAPBACK}][/a]
If sample sets are less complex, we might have the problem that all encoders end up tied.
And again, problem with such a synthetic track is that it has almost no real world meaning. Also, I'd have to distribute more samples losslessly (or are there any lossless splitters for MP4 and Vorbis?) which is bad for testers.
Sebastian,
The samples are looking great!
Average bitrate between codec is excellent.
I look so much forward to performing this test
Keep up the good work!
Sebastian,
The samples are looking great!
Average bitrate between codec is excellent.
I look so much forward to performing this test
Keep up the good work!
[a href="index.php?act=findpost&pid=346649"][{POST_SNAPBACK}][/a]
Thanks. Well, the problem is that with the samples I have ATM, LAME and WMA Pro exceed the 10% tolerance margin (max is 140.8 kbps). Nero AAC is only 0.1 kbps away from it. Maybe the average bitrate can be lowered with a movie dialog sample for example.
Some comments about the bitrate table:
Did you try this Nero encoder with full tracks? What is the used setting? Does it produce a comparable average bitrate with the other encoders when a big amount of varied complete tracks is encoded?
BTW, did you use -q 4.25 for Vorbis? (I suppose the other encoders have only one possible setting.)
For Vorbis, I used -q 4.
For Nero AAC, I used the recommended streaming profile. On my music collection, it produced an average of 132 kbps.
In my opinion Vorbis -q 4.25 would be closer with LAME V5n and WMA Pro 50 (I have no large scale experience of WMA Pro 50). The AAC encoders work differently.
guruboolez chose -q 4 for various tracks and 4.25 for classical in his test, but according to the results I have seen now I think 4.25 would have been a correct setting for all genres. Perhaps 4.2 could be used too. I could try if it makes any difference on my test tracks.
[span style='font-size:7pt;line-height:100%']Edit: typo[/span]
I'm new to this (still) and excuse this question if it silly or I'm jumping ahead of things, but, when I try saving the clips, they all come up as "index.php"? What am I doing wrong?
I'm new to this (still) and excuse this question if it silly or I'm jumping ahead of things, but, when I try saving the clips, they all come up as "index.php"? What am I doing wrong?
[a href="index.php?act=findpost&pid=346665"][{POST_SNAPBACK}][/a]
Which browser are you using?
I'm new to this (still) and excuse this question if it silly or I'm jumping ahead of things, but, when I try saving the clips, they all come up as "index.php"? What am I doing wrong?
[a href="index.php?act=findpost&pid=346665"][{POST_SNAPBACK}][/a]
Which browser are you using?
[a href="index.php?act=findpost&pid=346666"][{POST_SNAPBACK}][/a]
IE 6.0...never ran into troubles like this with any other sample?
Some general thoughts about listening tests:
- the test samples of various format contenders themselves don't need to average to a certain bitrate. Any higher or lower bitrate on samples will work, because that is the goal to be tested: "Does the encoder a good job on distributing bitrate to frames ?"
That means, if we have a broad overview, of the average bitrate a format with specified quality setting averages at,
then it is fine. The samples can vary a lot from that targeted bitrate. That is vbr.
- Of course, it is another question, which samples you select. Only killer samples, or normal music, or some mix.
- I suggest selecting samples, which result to ca. the target bitrate, but also samples, which go way above the target bitrate with certain encoders,
but also samples, which result clearly below target bitrate with certain encoders.
So, you test "easy to encode" samples, "average to encode" samples, and "difficult to encode" samples.
It will then be interesting, if various encoders behave differently regarding bitrate sample distribution, and more important: "do they have success, eg. when they give certain samples only a low bitrate, or lower bitrate than other contenders ?"
- Maybe somebody here with a widefold music CD collection, covering several genres, from pop, jazz, classic, rock, techno, audio books, metal etc., should mix some albums, and keep them as reference in a Lossless format, for test encodes with current and upcoming formats/encoders, settings.
So, that we have some HA-reference, which setting yields to which average bitrate.
Eg. vorbis q4 , or q4.25, a q-step of 0.25 might have already some bigger influence in the 100-128k bitrate area, and it should be decided & tested with the vbr formats, that for each format a setting is chosen, which does yield on a broad average of music to same bitrate (with only few percent difference).
Guruboolez did something like this with his classic collection, don't know, if he has some other CDs/genres, which could make it complete.
I could help here, have various classic, jazz, pop, rock etc., but I don#t have installed every format yet. And don#t have the internet/program knowledge, to present the results such good, how guru does it. Or does foobar give everything automatically, or probably a connection/script necessary, to combine foobars values with excel ?
Eg., for 3.97 Lame -V5 --vbr-new, I can confirm, that 130 kbit/s is an average, but it depends a lot on albums/music. Eg., if you would ask somebody, who owns mainly modern clip-compressed pop music, he would say, that lame 3.97 -V5 -vbr-new will yield to 140, not 130k.
Thanks. Well, the problem is that with the samples I have ATM, LAME and WMA Pro exceed the 10% tolerance margin (max is 140.8 kbps). Nero AAC is only 0.1 kbps away from it. Maybe the average bitrate can be lowered with a movie dialog sample for example.
All the codec is on the high side of the target bitrate of 128 kbps and indeed - as you point out - some are outside the 10% tolerance we have defined.
However the difference between lowest average bitrate (137.0 iTunes) and highest average bitrate (143.1 WMA Pro) is only 4%, which is excellent.
I see no need to adjust the average bitrate for any of the codec.
Although some could argue that the average bitrate is closer to that of a 160 kbps test, I would prefer to think of it as a 128+ kbps test, since 128 kbps VBR results in an average bitrate slightly above 128 kbps.
I'd also keep the settings as-is and check the final result when I have all samples. If some codecs continue to fall outside 140.8 kbps, I might replace a sample or two.
What's with the sample Cockney Rebel ("Sebastian")?
The file sample I downloaded is WavePack 855 kbps but the sound has severe artifacts.
I'd also keep the settings as-is and check the final result when I have all samples. If some codecs continue to fall outside 140.8 kbps, I might replace a sample or two.
I don't understand this method. The vbr codecs are free in their behaviour.
Only importance of the to be tested quality settings is, that the target bitrate is nearly the same for all contenders, ie. 128k+- over various music genres.
What's with the sample Cockney Rebel ("Sebastian")?
The file sample I downloaded is WavePack 855 kbps but the sound has severe artifacts.
[a href="index.php?act=findpost&pid=346678"][{POST_SNAPBACK}][/a]
Hmm... It's from "Rock Legends - All-Time Greatest Rock Ballads Volume 1" (track 9). I copied it with EAC, extracted the 35 seconds and encoded them with WavPack.
What kind of artifacts are you talking about? Some that are typical to lossy encoders or something else?
What kind of artifacts are you talking about? Some that are typical to lossy encoders or something else?
Distortion on singers voice most noticably from 0:09 to 0:20
What kind of artifacts are you talking about? Some that are typical to lossy encoders or something else?
Distortion on singers voice most noticably from 0:09 to 0:20
[a href="index.php?act=findpost&pid=346682"][{POST_SNAPBACK}][/a]
The tracks were copied from vinyl - maybe that's the reason.
I'd also keep the settings as-is and check the final result when I have all samples. If some codecs continue to fall outside 140.8 kbps, I might replace a sample or two.
I don't understand this method. The vbr codecs are free in their behaviour.
Only importance of the to be tested quality settings is, that the target bitrate is nearly the same for all contenders, ie. 128k+- over various music genres.
[a href="index.php?act=findpost&pid=346679"][{POST_SNAPBACK}][/a]
Yes, but if there are a lot of difficult samples and the average bitrate exceeds 128 kbps + 12.8 kbps, it would be better to replace one of the complex samples so the average bitrate remains near what's tested.
Nero for example produces an average of 131-132 kbps with my music collection and that's what Ivan said, too. With these particular samples however, the bitrate is boosted to more than 140 kbps. That's why I think that replacing one sample is appropriate.
then the test won't be serving its purpose. as stated, an econder would adapt to a whole track diferently than to a 30 secs clip.[a href="index.php?act=findpost&pid=346637"][{POST_SNAPBACK}][/a]
You could conduce a test yourself. I believe Argentinian law is much nicer than german law, in that aspect.
I have the Steve Harley & Cockney Rebel - The Cream Of compilation which includes Sebastian. I could quickly access my LAME 3.90.3 --alt-preset extreme encoding. Steve's voice is "distorded" on my version too. Perhaps the track is not a good choice for an encoder test.
I would like see a strong female voice included. A jazz or opera singer who can sing load and high would be good. Someone who can break glasses with her voice... this kind of sample might produce low bitrates and be difficult for the encoders at the same time.
As ff123, user and I among others suggested earlier, it would be good to try to achieve a bitrate distribution of the sample set which resembles the distribution from a large random set. So perhaps add a few samples which lower the average a bit...?
As said before, the used encoding options should be determined by checking the bitrates of a large amount of varied complete tracks and after that kept.
The samples should represent many musical genres and be difficult enough to produce audible artifacts. This is a listening test and no one should be interested what bitrates the individual encoded samples are. I don't think the bitrates should even be checked. The encoders should be left on their own to do the best they can with the sample material.
Naturally, the results of the preceding large-scale bitrate testing should be published.
I updated my bitrate table with Vorbis -q 4.20 and gathered all previous results in the same table:
bitrates_public2.xls (http://kotisivu.mtv3.fi/alexb/ha/bitrates_public2.xls)
Sadly we cannot do 2-pass because that would required to be executed on full tracks and Sebastian rightly do not want to get embroiled on copyright issues.
Sorry if already asked, but i don't think anybody would get into trouble for doing this, i mean, he won't keep the whole song, right? He would just encode the file, decode, cut, and delete...
I think it was resolved that the differences concerned are similar in a kind and in severity to Neros ABR encoding choice. iirc all devs who expressed an opinion see little problem with 2passing a joined sample corpus for Wma stds encode
if its wished to include wma std in the test. ff123 agreed this would be fine if the samples where of average complexity, id suggest if they're mean achieved bitrate of the 'normal' vbrs doesnt match the target bitrate the 2pass could target the mean achieved bitrates of the other encoders to fairly compensate.
All in all, its not some to worry about at this stage, whatever the samples used a suitible Wma std targeting process is possible if required.
and be difficult enough to produce audible artifacts.
[a href="index.php?act=findpost&pid=346720"][{POST_SNAPBACK}][/a]
I'm repeating myself, but I still wonder how you decide that a track is difficult...?
and be difficult enough to produce audible artifacts.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=346720")
I'm repeating myself, but I still wonder how you decide that a track is difficult...?
[a href="index.php?act=findpost&pid=346736"][{POST_SNAPBACK}][/a]
I guess I'm the wrong person to answer to this question. Check my "failure" here:
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=39233]A little VBR ~88 kbps ABX-test[/url]
Vorbis should not be tested with -q 4.00, but in around 4.25. See the bitrate estimate thread for reasons why.
I guess I'm the wrong person to answer to this question. Check my "failure" here:
A little VBR ~88 kbps ABX-test (http://www.hydrogenaudio.org/forums/index.php?showtopic=39233)
[a href="index.php?act=findpost&pid=346737"][{POST_SNAPBACK}][/a]
You should be able to say
how to select "difficult" samples anyhow. I guess that you by linking to this topic want to say that it should be done by listening tests and by someone else than you. Am I right?
Then, sure, I can understand your point - you want the test to be a bit easier so that not all codecs will end up tied. But first a group of people have to sit down and listen to a lot of material in order to select some that produce artefacting with some encoders. I just don't thing that's very practical. The group should as you hint at be people who have above average hearing, and they should go through a sizeable amount of test clips in order to whisk out some interesting ones. Then these people could just as well give the score directly and there would be no need for the public listening test.
Or, you could try to reuse samples from previous tests which separated the codecs well. Then you run into the risk that you bias this test towards the winner of the last. That could be acceptable I guess, but again I would prefer to not do like that, and it does indeed look like Sebastian is selecting a whole new sample set for this test.
I don't see any reason to why it shouldn't be selected in a way that it's bitrate mean and distribution will resemble "average music". What average music is, we could discuss, but I'm fine if it happens to be those libraries from which the current bitrate estimation for the encoders is taken.
I suppose it would be good if a couple of experienced testers could try the samples and tell their opinion in case the test conductor is not sure about the selection. They don't have to test all samples and with every encoder.
BTW, did you try the sample I provided e.g. with Vorbis? No one has posted test results yet. Is that sample too easy for Vorbis -q 1.5?
BTW, did you try the sample I provided e.g. with Vorbis? No one has posted test results yet. Is that sample too easy for Vorbis -q 1.5?
[a href="index.php?act=findpost&pid=346765"][{POST_SNAPBACK}][/a]
Not yet, no. If you could provide a flac version of that file I could give it a try. (I'm using a mac)
Not yet, no. If you could provide a flac version of that file I could give it a try. (I'm using a mac)[a href="index.php?act=findpost&pid=346768"][{POST_SNAPBACK}][/a]
FLAC may be a more universal format. I replaced it with a FLAC version.
[span style='font-size:7pt;line-height:100%']Edit: I added also the lossy test files. The files are not big and Vorbis b4.5 may not be available on all platforms.[/span]
I would like see a strong female voice included. A jazz or opera singer who can sing load and high would be good. Someone who can break glasses with her voice... this kind of sample might produce low bitrates and be difficult for the encoders at the same time.
Here's sample of a strong female voice
Cæcilie Norby - Life on Mars (http://www.hydrogenaudio.org/forums/index.php?amp;showtopic=39288&view=findpost&p=346779)
[span style='font-size:14pt;line-height:100%']
2-pass encoding: why it is necessary to extract samples from an entire encoding and the original track.[/span]
wmeditor.exe is a tool provided with WMEncoder.exe. The purpose of this small application is to cut WMA files without recoding them. The tool is not extremely precise (accuracy seems to be around ½ sec, at least with VBR) but it’s enough to extract a part from a VBR encoding in order to measure the bitrate of this small part.I encoded four samples from the latest multiformat listening test at 128 kbps with
WMA VBR 2-pass, with three different method or environment. As you know, the allocated bitrate of one sample depends a lot from the content of the whole file.
• Method#1 = the sample was extracted from a full track encoded in 2-pass mode. The full track is the original one. The bitrate of the full track should correspond to the targeted one, but our extracted sample could differ from it, and will depend from the complexity of the sample as well from the complexity of the track.
• Method#2 = the sample is directly encoded in 2-pass mode. The final bitrate necessary correspond to the targeted one (here: 128 kbps), regardless of the complexity of the sample.
• Method#3 = the sample was extracted from a full track encoded in 2-pass mode. The full track is a virtual track composed with 18 samples (used in latest 128 multiformat test) merged together. The bitrate of the full track should correspond to the targeted one, but our extracted sample could differ from it, and will depend from the complexity of the sample as well from the complexity of the track.
In summary:
method#1 = 1st step = encoding (full & original track) – 2nd step: extraction
method#2 = 1st step = extraction – 2nd step: encoding
method#3 = 1st step = encoding (full but fake track) – 2nd step: extraction
BARTOK.WAV#1 = 0:22.895 356 kb 124,39 kbps
#2 = 0:23.024 373 kb 129,60 kbps
#3 = 0:23.231 338 kb 116,40 kbps
DEBUSSY.WAV#1 = 0:29.210 385 kb 105,44 kbps
#2 = 0:29.999 478 kb 127,47 kbps
#3 = 0:29.674 291 kb 78,45 kbps
HONGROISE.WAV#1 = 0:29.393 490 kb 130,93 kbps
#2 = 0:30.000 478 kb 127,47 kbps
#3 = 0:29.396 326 kb 88,72 kbps
MAHLER.WAV#1 = 0:29.767 717 kb 192,70 kbps
#2 = 0:29.999 484 kb 129,07 kbps
#3 = 0:30.058 536 kb 142,66 kbps
#1 #2 #3
bartok 124,39 129,60 116,40
debussy 105,44 127,47 78,45
hongroise 130,93 127,47 88,72
mahler 192,70 129,07 142,66
__________________________
138,37 128,40 106,56
=> method#2 and method#3 are both inaccurate. None correspond to what the listener would get by encoding his own CD with VBR 2-pass mode. The third method is in this case the worse, and
would handicap WMA by 30 kbps per (classical) sample!
The corresponding samples (each sample encoded three time with the same seting but in three different environment) are uploaded here (http://www.hydrogenaudio.org/forums/index.php?showtopic=39309&view=findpost&p=346795).
I'm new to this (still) and excuse this question if it silly or I'm jumping ahead of things, but, when I try saving the clips, they all come up as "index.php"? What am I doing wrong?
[a href="index.php?act=findpost&pid=346665"][{POST_SNAPBACK}][/a]
Which browser are you using?
[a href="index.php?act=findpost&pid=346666"][{POST_SNAPBACK}][/a]
IE 6.0...never ran into troubles like this with any other sample?
[a href="index.php?act=findpost&pid=346667"][{POST_SNAPBACK}][/a]
Did you try just left-clicking, instead of right-click -> save as?
=> method#2 and method#3 are both inaccurate. None correspond to what the listener would get by encoding his own CD with VBR 2-pass mode. The third method is in this case the worse, and would handicap WMA by 30 kbps per (classical) sample!
[a href="index.php?act=findpost&pid=346785"][{POST_SNAPBACK}][/a]
That's very interesting. Really puts it, black on white, that none other than method one is acceptable if 2-pass WMA std is to be tested.
Could you do a similar test for Nero's AAC, to test the relevancy of ChiGung's worries?
I could, but I need a tool¹ to cut losslessly an AAC or MP4 file. Does that exist? mp4box maybe?
¹ not too hard to use
Updated the bitrate table with Vorbis Q 4.25: http://maresweb.de/bitrates.htm (http://maresweb.de/bitrates.htm)
I could, but I need a tool¹ to cut losslessly an AAC or MP4 file. Does that exist? mp4box maybe?
¹ not too hard to use
[a href="index.php?act=findpost&pid=346830"][{POST_SNAPBACK}][/a]
Of course, didn't think about that...
As said before, the used encoding options should be determined by checking the bitrates of a large amount of varied complete tracks and after that kept.
The samples should represent many musical genres and be difficult enough to produce audible artifacts. This is a listening test and no one should be interested what bitrates the individual encoded samples are. I don't think the bitrates should even be checked. The encoders should be left on their own to do the best they can with the sample material.
Naturally, the results of the preceding large-scale bitrate testing should be published.
[a href="index.php?act=findpost&pid=346720"][{POST_SNAPBACK}][/a]
Completely agree. The only problem I see is that final bit rate for a codec depends highly on nature of sound material, so if large “bit rate estimation” corpus consists of mostly classical music there will be one estimate, in case of death-metal – another estimate. There is no any final figure regardless of sound material used. I see several approaches to solve this “problem of definitions”:
1. To fill “bit rate estimation” corpus with various genres in equal proportions. Say, classical + pop + instrumental + speech.
2. To calculate final bit rates for each genre separately (the question of comparison will arise);
3. To select for each genre a representative CD and use them in all future listening tests for defining bit rates. A kind of “standard meter”.
Also a few thoughts concerning choice of test samples.
1. It seems to me that they have to be as different as possible, no matter what bit rates are achieved. If we decided to measure final bit rates on a large corpus, then there is no need to do this the second time on selected test samples. Actually I don’t see any idea behind doing this twice.
2. In this particular listening test they have to be short (15-20 sec.) and representative in genre sense. The first is for minimizing listening fatigue of mostly non trained listeners and the latter is for making test samples to be closer to “real world” usage. Some “difficult for encoding” samples are ok as well, I suppose.
Also a few thoughts concerning choice of test samples.
1. It seems to me that they have to be as different as possible, no matter what bit rates are achieved. If we decided to measure final bit rates on a large corpus, then there is no need to do this the second time on selected test samples. Actually I don’t see any idea behind doing this twice.
2. In this particular listening test they have to be short (15-20 sec.) and representative in genre sense. The first is for minimizing listening fatigue of mostly non trained listeners and the latter is for making test samples to be closer to “real world” usage. Some “difficult for encoding” samples are ok as well, I suppose.
[a href="index.php?act=findpost&pid=346875"][{POST_SNAPBACK}][/a]
I'd like to say that 30 secs is a good length. Often when testing at this kind of bitrate it is very difficult to find parts of a track which are distinguishable from the original, even with full length songs.
Also a few thoughts concerning choice of test samples.
1. It seems to me that they have to be as different as possible, no matter what bit rates are achieved. If we decided to measure final bit rates on a large corpus, then there is no need to do this the second time on selected test samples. Actually I don’t see any idea behind doing this twice.
[a href="index.php?act=findpost&pid=346875"][{POST_SNAPBACK}][/a]
There were complains in the past. CBR/ABR encoders were limited to 128 kbps whereas VBR ones reached 140...145 kbps on average with the selected samples (see the first 128 kbps listening test). Some persons considered the setting as flawed. They assumed that VBR would sound poorer with low bitrate samples. To limit the complaints, Roberto tried to lower the bitrate of the selected samples, and introduced one or two low bitrate samples which were very useful in that regard. They also revealed that some VBR encoders such LAME or MPC have serious troubles with this kind of material, considered as "easy" to encode.
In my opinion, a listening test — especially based on VBR encoders — shouldn't discard any samples for the simple reason that the final bitrate varies too much from the target. High variations are a part of VBR encoding, and such samples, including extreme ones, should be tested as a part of VBR.
Moreover, if all contenders are VBR encodings, I don't think that we should put too much attention on the final bitrate of the tested samples. As long as the average bitrate of ALL CONTENDERS are very close, it doesn't matter that the average bitrate is 100 kbps or 160 kbps. Of course, it would be nicer to see something closer to the target bitrate. Here, with a target corresponding to 135 kbps rather to 128 kbps, it wouldn't be a problem if the average bitrate of the selected samples would reach 142...145 kbps.
2. In this particular listening test they have to be short (15-20 sec.) and representative in genre sense. The first is for minimizing listening fatigue of mostly non trained listeners and the latter is for making test samples to be closer to “real world” usage. Some “difficult for encoding” samples are ok as well, I suppose.
[a href="index.php?act=findpost&pid=346875"][{POST_SNAPBACK}][/a]
I agree. I prefer 5...6 sec samples. Therefore, everybody would rank the same musical part. It's not necessary the case with 30 sec samples, especially if several accoustical phenomenons occur during these 30 seconds.
But I know that few people shares my point of view. I'm myself unable to cut 5 or 6 sec from a sample: samples don't sound musical anymore
Also a few thoughts concerning choice of test samples.
1. It seems to me that they have to be as different as possible, no matter what bit rates are achieved. If we decided to measure final bit rates on a large corpus, then there is no need to do this the second time on selected test samples. Actually I don’t see any idea behind doing this twice.
[a href="index.php?act=findpost&pid=346875"][{POST_SNAPBACK}][/a]
There were complains in the past. CBR/ABR encoders were limited to 128 kbps whereas VBR ones reached 140...145 kbps on average with the selected samples (see the first 128 kbps listening test). Some persons considered the setting as flawed. They assumed that VBR would sound poorer with low bitrate samples. To limit the complaints, Roberto tried to lower the bitrate of the selected samples, and introduced one or two low bitrate samples which were very useful in that regard. They also revealed that some VBR encoders such LAME or MPC have serious troubles with this kind of material, considered as "easy" to encode.
In my opinion, a listening test — especially based on VBR encoders — shouldn't discard any samples for the simple reason that the final bitrate varies too much from the target. High variations are a part of VBR encoding, and such samples, including extreme ones, should be tested as a part of VBR.
Moreover, if all contenders are VBR encodings, I don't think that we should put too much attention on the final bitrate of the tested samples. As long as the average bitrate of ALL CONTENDERS are very close, it doesn't matter that the average bitrate is 100 kbps or 160 kbps. Of course, it would be nicer to see something closer to the target bitrate. Here, with a target corresponding to 135 kbps rather to 128 kbps, it wouldn't be a problem if the average bitrate of the selected samples would reach 142...145 kbps.
[a href="index.php?act=findpost&pid=346880"][{POST_SNAPBACK}][/a]
I would like to go one step further, and claim that it wouldn't matter if the encoders produced very different avg bitrates for the sample corpus either, as long as they have been set to produce a similar avg bitrate for some large, varied music collection.
If that were the case, and the test were to reveal that one of the encoders with a low average was far behind the others in terms of sound quality, it would simply show that the encoder in question hadn't got it's priorities right.
Edit: Maybe my reasoning is flawed. Only if you have a sample set where each VBR encoder has "highs" and "lows" (avg bitrates) can you determine which one distributes the bits in the best way...
2. In this particular listening test they have to be short (15-20 sec.) and representative in genre sense. The first is for minimizing listening fatigue of mostly non trained listeners and the latter is for making test samples to be closer to “real world” usage. Some “difficult for encoding” samples are ok as well, I suppose.
[a href="index.php?act=findpost&pid=346875"][{POST_SNAPBACK}][/a]
I agree. I prefer 5...6 sec samples. Therefore, everybody would rank the same musical part. It's not necessary the case with 30 sec samples, especially if there several accoustical phenomenons occur during these 30 seconds.
But I know that few people shares my point of view. I'm myself unable to cut 5 or 6 sec from a sample: it's not musical anymore
[a href="index.php?act=findpost&pid=346884"][{POST_SNAPBACK}][/a]
I didn't think of that aspect. Good point.
... Some persons considered the setting as flawed. They assumed that VBR would sound poorer with low bitrate samples. To limit the complaints, Roberto tried to lower the bitrate of the selected samples
In case of VBR encoding this could be translated as "They assumed that VBR would sound poorer with some other sound samples." cause actual bit rate completely depends on nature of samples. That's why most different sound samples are appreciated in listening tests.
In my opinion, a listening test — especially based on VBR encoders — shouldn't discard any samples for the simple reason that the final bitrate varies too much from the target. High variations are a part of VBR encoding, and such samples, including extreme ones, should be tested as a part of VBR.
Moreover, if all contenders are VBR encodings, I don't think that we should put too much attention on the final bitrate of the tested samples. As long as the average bitrate of ALL CONTENDERS are very close, it doesn't matter that the average bitrate is 100 kbps or 160 kbps. Of course, it would be nicer to see something closer to the target bitrate. Here, with a target corresponding to 135 kbps rather to 128 kbps, it wouldn't be a problem if the average bitrate of the selected samples would reach 142...145 kbps.
Agree. So may be instead of achieving targeted bit rate for testing samples it's better to spend efforts for choosing 3-5 realy "hard to encode" samples. Other ones could be just excerpts from different genres on Sebastian's choice.
I agree. I prefer 5...6 sec samples. Therefore, everybody would rank the same musical part. It's not necessary the case with 30 sec samples, especially if several accoustical phenomenons occur during these 30 seconds.
But I know that few people shares my point of view. I'm myself unable to cut 5 or 6 sec from a sample: samples don't sound musical anymore
[a href="index.php?act=findpost&pid=346884"][{POST_SNAPBACK}][/a]
Yes, this is a good point. In fact this is a part of big big question: will the results be identical for both tests with whole music works and short samples from them? Please don't try to discuss this here Here we just give a simple answer - YES. Serious flaws in some parts of work could ruin the whole impression.
EDIT. And I prefer around 10 sec. samples containing one difficult place surrounded by something in order the sample to be a piece of music.
Added macabre, don't let me be misunderstood, kraftwerk and bigyellow, so there are 18 samples now.
http://maresweb.de/bitrates2.htm (http://maresweb.de/bitrates2.htm)
What do you think? I would replace the one or the other sample with the recently posted orchestral samples - maybe even macabre since the other two are more dynamic IMO.
Two more things... Seing that the bitrate is around 140 kbps, I think I can tell Vorbis to encode to 4.2 or 4.25 (what do you guys think - which one to go with?).
Also, I will cut the voice from Elizabeth since it's a bit "confusing".
I still fail to see why you call it "questioned". Were Roberto's or Guru's tests questioned because samples were encoded instead of full tracks?
maybe at the time no one realiced about this problems. it doesn't mean they where wrong, just that they can be better.
=> method#2 and method#3 are both inaccurate. None correspond to what the listener would get by encoding his own CD with VBR 2-pass mode. The third method is in this case the worse, and would handicap WMA by 30 kbps per (classical) sample!
you have guru's analisis here to validate my thinking.
You could conduce a test yourself. I believe Argentinian law is much nicer than german law, in that aspect.
if somebody want to send me the FLACs, i would do it myself. anyway, are you sure the 30 secs samples are legal?
You could conduce a test yourself. I believe Argentinian law is much nicer than german law, in that aspect.
if somebody want to send me the FLACs, i would do it myself. anyway, are you sure the 30 secs samples are legal?
[a href="index.php?act=findpost&pid=346914"][{POST_SNAPBACK}][/a]
No idea. They're better than full songs, though.
Anyways, encoding full tracks might be interesting for 2-pass or ABR encoders only since AFAIK, VBR encoders will not encode differently (maybe only minor deviations, but nothing to care about). And trying to obtain full songs is pretty much a pain only because WMA doesn't offer a "reliable" VBR option.
Nero might also be handicapped a bit, but that was the decision of the developers - I could've used VBR.
Two more things... Seing that the bitrate is around 140 kbps, I think I can tell Vorbis to encode to 4.2 or 4.25 (what do you guys think - which one to go with?).
Also, I will cut the voice from Elizabeth since it's a bit "confusing".
[a href="index.php?act=findpost&pid=346913"][{POST_SNAPBACK}][/a]
...But, shouldn't the settings be tuned for a target bitrate on a large music corpus, and *not* for the sample corpus?
Two more things... Seing that the bitrate is around 140 kbps, I think I can tell Vorbis to encode to 4.2 or 4.25 (what do you guys think - which one to go with?).
Also, I will cut the voice from Elizabeth since it's a bit "confusing".
[a href="index.php?act=findpost&pid=346913"][{POST_SNAPBACK}][/a]
...But, shouldn't the settings be tuned for a target bitrate on a large music corpus, and *not* for the sample corpus?
[a href="index.php?act=findpost&pid=346927"][{POST_SNAPBACK}][/a]
Yes, but the difference between the encoders shouldn't be more than 10% so that's why I think 4.25 can still be used. I mixed that up yesterday evening when I thought that the bitrate must not deviate more than 10% from the target bitrate of the listening test. My bad.
You can safely skip this question - unless you are interested in statistics (and conducting your own test in the future), so that I won't anger Triza again
...But, shouldn't the settings be tuned for a target bitrate on a large music corpus, and *not* for the sample corpus?
Yes, unless the sample is known to represent the population under study, in which case both should result in the same settings. However, being that the population is not clearly defined...
This of course, combined with what Guruboolez has already posted about bit-rate estimation methods and their related accuracies.
However, as it is a relatively big undertaking (esp. proving it), I think people should be relatively happy with a non-random sampling. Esp. when manually picked to include various tracks of different bit-rate averages. Good enough.
[span style='font-size:14pt;line-height:100%']2-pass encoding: why it is necessary to extract samples from an entire encoding and the original track.[/span]....
For those still interested, a reply discussing these results and suggestions of how 2pass method can be more fairly employed here (http://www.hydrogenaudio.org/forums/index.php?showtopic=39309&view=findpost&p=346908) (in case its missed).
Updated bitrate table with Vorbis -q 4.25: http://maresweb.de/bitrates2.htm (http://maresweb.de/bitrates2.htm)
Two more things... Seing that the bitrate is around 140 kbps, I think I can tell Vorbis to encode to 4.2 or 4.25 (what do you guys think - which one to go with?).
Also, I will cut the voice from Elizabeth since it's a bit "confusing".
[a href="index.php?act=findpost&pid=346913"][{POST_SNAPBACK}][/a]
...But, shouldn't the settings be tuned for a target bitrate on a large music corpus, and *not* for the sample corpus?
[a href="index.php?act=findpost&pid=346927"][{POST_SNAPBACK}][/a]
Yes, but the difference between the encoders shouldn't be more than 10% so that's why I think 4.25 can still be used. I mixed that up yesterday evening when I thought that the bitrate must not deviate more than 10% from the target bitrate of the listening test. My bad.
[a href="index.php?act=findpost&pid=346938"][{POST_SNAPBACK}][/a]
Ok. Well, as long as you're tweaking it and looking at the effects on a more "general" avg bitrate. (And you are, if I understand you correctly.)
I didn't understand what you mean. What I wanted to say is that if -q 4.25 produces a bitrate that is more than 10% away from the other competitors, I have to replace a sample. So far, everything looks good, I think.
If its fair to say that encodes which result in an unusualy high or low bitrate compared to the others are examples of samples which the encoder concernsed finds more unusual ~in a way than the others do..... then there is a possibility that the encoder would be selectively advantaged by discarding them
Yes, but if the difference is too high, you cannot compare the two encoders.
Yes, but if the difference is too high, you cannot compare the two encoders.
[a href="index.php?act=findpost&pid=346960"][{POST_SNAPBACK}][/a]
Ah I see what you mean -for the mean bitrate over the whole corpus rather than individual samples.
Whats odd is if the mean corpus bitrate is far off for one encoder, then there is something about the corpus that that encoder finds unusual - assuming the corpus is large enough to average out the random differences. hmmm...
Yes, but if the difference is too high, you cannot compare the two encoders.
[a href="index.php?act=findpost&pid=346960"][{POST_SNAPBACK}][/a]
Ah I see what you mean -for the mean bitrate over the whole corpus rather than individual samples.
[a href="index.php?act=findpost&pid=346961"][{POST_SNAPBACK}][/a]
Exactly.
Yes, but if the difference is too high, you cannot compare the two encoders.[a href="index.php?act=findpost&pid=346960"][{POST_SNAPBACK}][/a]
Ah I see what you mean -for the mean bitrate over the whole corpus rather than individual samples.[a href="index.php?act=findpost&pid=346961"][{POST_SNAPBACK}][/a]
Exactly. [a href="index.php?act=findpost&pid=346965"][{POST_SNAPBACK}][/a]
How about this, if all the encoders are close to 128 kbs for a large target corpus, and for the listening corpus are all somewhat above it, (resulting from a correlation between perceptual listening interest and codec-wide demand factor) Some codecs average behaviour might be to stick tighter to the target bitrate than others, so those codecs would produce encodes closer to the target bitrate than others. One codec that 'sticks out' of line with the listening corpus, might be just displaying its normal behaviour in that while it achieves the target over normal average material, its choosen discreet allocation varys more widely than the others, it might look like its unusualy high compared to the others, but if the others are all higher than the target over average, it might just be displaying more 'reactivity' to demand factor, and manipulating the samples to deliberately lower bit allocation for it alone, could be distortive to its performance.
Im not sure yet, but the way to avoid such uncertainty of fairness, could be to try to make *all codecs meet a target bit allocation mean, accross the targeting material AND the sample material.
An axiom that Id hold onto, is the more samples used the less danger of random variations surviving in the result,
and the less reactive manipulation to the samples, the less danger of affecting a codec unfairly.
(???)
A couple of comments about the electronic genre samples:
- If I happen to have the same 29 s version of the "Kraftwerk" sample (The Robots) it has an obvious background hiss. (I downloaded my sample a long time ago from a HA link.) Is it a vinyl recording?
- The "Electronic" sample I provided is a bit short. I would give the testers a bit longer time to adopt. The sample is quite extraordinary.
I have a representative collection of electronic music and I would like to propose a couple of new samples, but I don't have time to do it immediately. (Would tomorrow be OK?)
For example: A bit longer "Yello" sample and a sample from the new live version of "Die Roboter" from the latest Kraftwerk album. You may also want to consider selecting only one electronic sample. I may be able to find the strong and shrill female voice I requested earlier, which could be used instead one of the electronic samples.
An axiom that Id hold onto, [...]
[a href="index.php?act=findpost&pid=346970"][{POST_SNAPBACK}][/a]
I thought I would never see this word out of an algebraic book...
- If I happen to have the same 29 s version of the "Kraftwerk" sample (The Robots) it has an obvious background hiss. (I downloaded my sample a long time ago from a HA link.) Is it a vinyl recording?
[a href="index.php?act=findpost&pid=346971"][{POST_SNAPBACK}][/a]
I have no idea where the sample comes from - I have it from Roberto's collection.
I have a representative collection of electronic music and I would like to propose a couple of new samples, but I don't have time to do it immediately. (Would tomorrow be OK?)
[a href="index.php?act=findpost&pid=346971"][{POST_SNAPBACK}][/a]
Well, I would like to complete the sample set tomorrow morning, clear up some final stuff regarding settings and then prepare the listening test so that everything is ready on 3rd.
For example: A bit longer "Yello" sample and a sample from the new live version of "Die Roboter" from the latest Kraftwerk album. You may also want to consider selecting only one electronic sample. I may be able to find the strong and shrill female voice I requested earlier, which could be used instead one of the electronic samples.
[a href="index.php?act=findpost&pid=346971"][{POST_SNAPBACK}][/a]
Well, I agree that 3 electronic samples are a bit much, but they are different. The Sash sample is a mixture of Pop and Trance and it's also quite difficult (1012 kbps WavPack). Kraftwerk is pure techno and it's not very "wild". The sample you posted is pretty aggressive and IMO, the 10 seconds I have are enough.
Edit: You could post the female voice sample and I'll have a look at it. Eventually, I
might replace kraftwerk and Yello and use ravel and your female sample.
I'm new to this (still) and excuse this question if it silly or I'm jumping ahead of things, but, when I try saving the clips, they all come up as "index.php"? What am I doing wrong?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=346665")
Which browser are you using?
[a href="index.php?act=findpost&pid=346666"][{POST_SNAPBACK}][/a]
IE 6.0...never ran into troubles like this with any other sample?
[a href="index.php?act=findpost&pid=346667"][{POST_SNAPBACK}][/a]
Did you try just left-clicking, instead of right-click -> save as?
[a href="index.php?act=findpost&pid=346813"][{POST_SNAPBACK}][/a]
I've tried both: 1) Left-click gets me the "save file" window. The file that results is labeled "index.php"(or whatever I decide to name it) and after saving it, nothing will open the file
2) "Right-click" gets me the "Smart-Window" which has a few options:
"Open"
"Open in new window"
"Save Target as..."
"Print Target"
"Copy Shortcut"
"Add To Favorites" &
"Properties"
I've tried all these that could logically be tried, in every way I can imagine? I've even downloaded and installed "Wavpack" to see if the file is a type my machine doesn't recognize, but still nothing?
I tried looking at the "Properties" and it gives me a little information about the link itself, they are:
"Address URL:"
[a href="http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=1833]http://www.hydrogenaudio.org/forums/index....pe=post&id=1833[/url]
AND
"Type:"
PHP?ACT=ATTACH&TYPE=POST&ID=1833 File
I'm hoping that one of you sharper peeps can tell me something else to try?
Any info you could give would be greatly appreciated...Thanks!
Well, I agree that 3 electronic samples are a bit much, but they are different. The Sash sample is a mixture of Pop and Trance and it's also quite difficult (1012 kbps WavPack). Kraftwerk is pure techno and it's not very "wild". The sample you posted is pretty aggressive and IMO, the 10 seconds I have are enough.
OK, I am just proposing things.
The hiss I mentioned is present in the both old Kraftwerk samples (4 s and 29 s at http://ff123.net/samples.html (http://ff123.net/samples.html)). I suppose the samples are OK if the hiss has not bothered anyone before. Perhaps it is just a part of the old recording.
Edit: You could post the female voice sample and I'll have a look at it. Eventually, I might replace kraftwerk and Yello and use ravel and your female sample.[a href="index.php?act=findpost&pid=346980"][{POST_SNAPBACK}][/a]
I'll browse through my archives and hopefully I'll find her.
I have a couple of artists in my mind.
Any info you could give would be greatly appreciated...Thanks!
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=346997")
Well, for a start - have you tried Firefox? (In general I mean, not specifically for this task...) If you haven't I seriously would recommend getting it. They've just launched version 1.5 at [a href="http://mozilla.com]http://mozilla.com[/url] yesterday. (The links to the samples worked in my Firefox anyway. No reason for them not to work in IE either, really...)
Edit: mistakes
Edit: You could post the female voice sample and I'll have a look at it. Eventually, I might replace kraftwerk and Yello and use ravel and your female sample.
I'll browse through my archives and hopefully I'll find her.
I have a couple of artists in my mind.
@ Alex B: What did you think about this one:
Cæcilie Norby - Life on Mars (http://www.hydrogenaudio.org/forums/index.php?amp;showtopic=39288&view=findpost&p=346779)
Updated bitrate table with Vorbis -q 4.25: http://maresweb.de/bitrates2.htm (http://maresweb.de/bitrates2.htm)
[a href="index.php?act=findpost&pid=346951"][{POST_SNAPBACK}][/a]
I think that it would be more fair to use a lower quality setting for Vorbis.
Since the bit rate control of Vorbis is very flexible it seems wrong to select a setting that will give it the highest average bitrate of all encoders. People could challenge such a decision as favoritism towards Vorbis.
At -q 4 Vorbis has an average bitrate similar to that of iTunes and Nero and that would in my opinion be the right setting.
Any info you could give would be greatly appreciated...Thanks!
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=346997")
Well, for a start - have you tried Firefox? (In general I mean, not specifically for this task...) If you haven't I seriously would recommend getting it. They've just launched version 1.5 at [a href="http://mozilla.com]http://mozilla.com[/url] yesterday. (The links to the samples worked in my Firefox anyway. No reason for them not to work in IE either, really...)
Edit: mistakes
[a href="index.php?act=findpost&pid=347000"][{POST_SNAPBACK}][/a]
Yes Naylor, I've used Firefox before. The only thing I don't like about it is no sound, so I stick with IE, no big deal though.
I think somethings just "not in the cards", when it comes to me participating in these public listening tests....I seem to get these strange technical dificulties just when I'd like to try?
Updated bitrate table with Vorbis -q 4.25: http://maresweb.de/bitrates2.htm (http://maresweb.de/bitrates2.htm)
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=346951")
I think that it would be more fair to use a lower quality setting for Vorbis.
Since the bit rate control of Vorbis is very flexible it seems wrong to select a setting that will give it the highest average bitrate of all encoders. People could challenge such a decision as favoritism towards Vorbis.
At -q 4 Vorbis has an average bitrate similar to that of iTunes and Nero and that would in my opinion be the right setting.
[a href="index.php?act=findpost&pid=347002"][{POST_SNAPBACK}][/a]
According to this post:
I updated my bitrate table with Vorbis -q 4.20 and gathered all previous results in the same table:
[a href="http://kotisivu.mtv3.fi/alexb/ha/bitrates_public2.xls]bitrates_public2.xls[/url]
[a href="index.php?act=findpost&pid=346724"][{POST_SNAPBACK}][/a]
Vorbis -q 4 reached an average of exactly 128 kbps on Alex B's side. On the other hand, -q 4.2 and -q 4.25 didn't produce too high bitrates (around 134 kbps which is OK). So, what do the experts thing - should -q 4 be used or -q 4.2(5)?
I am going to replace Kraftwerk with Ravel since 3 electronica are really too much.
Vorbis -q 4 reached an average of exactly 128 kbps on Alex B's side. On the other hand, -q 4.2 and -q 4.25 didn't produce too high bitrates (around 134 kbps which is OK). So, what do the experts thing - should -q 4 be used or -q 4.2(5)?
[a href="index.php?act=findpost&pid=347010"][{POST_SNAPBACK}][/a]
Well, my spontaneous reaction is that you should keep it at q4, since 128 kbps was the target of this test - at least to begin with
Yes Naylor, I've used Firefox before. The only thing I don't like about it is no sound, so I stick with IE, no big deal though.
(OT) You could try this extension:
https://addons.mozilla.org/extensions/morei...ication=firefox (https://addons.mozilla.org/extensions/moreinfo.php?id=1030&application=firefox)
It'll import the default windows sounds into Firefox. (Most noticably, the 'click'.)
I think somethings just "not in the cards", when it comes to me participating in these public listening tests....I seem to get these strange technical dificulties just when I'd like to try?
[a href="index.php?act=findpost&pid=347005"][{POST_SNAPBACK}][/a]
Of course you should participate. It's my first test too
Regarding the DL problem - you could use a DL manager, I guess.
@ Alex B: What did you think about this one: Cæcilie Norby - Life on Mars (http://www.hydrogenaudio.org/forums/index.php?amp;showtopic=39288&view=findpost&p=346779)[a href="index.php?act=findpost&pid=347001"][{POST_SNAPBACK}][/a]
She's great, but I'm afraid that the encoders might like her too and be too gentle.
(Seriously, it's too difficult to say anything without a lossless sample and ABX testing.)
Not bad, but as Alex said, it might be too simple for 128 kbps. Thanks for providing, though.
@ Alex B: What did you think about this one: Cæcilie Norby - Life on Mars (http://www.hydrogenaudio.org/forums/index.php?amp;showtopic=39288&view=findpost&p=346779)[a href="index.php?act=findpost&pid=347001"][{POST_SNAPBACK}][/a]
She's great, but I'm afraid that the encoders might like her too and be too gentle.
(Seriously, it's too difficult to say anything without a lossless sample and ABX testing.)
[a href="index.php?act=findpost&pid=347020"][{POST_SNAPBACK}][/a]
Well I thought you were looking for a simple track with a strong female voice...
I would like see a strong female voice included. A jazz or opera singer who can sing load and high would be good. Someone who can break glasses with her voice... this kind of sample might produce low bitrates and be difficult for the encoders at the same time.
She's a jazz singer and her mother is an opera singer.
Anyway, I brought this one from iTunes so I don't have the lossless version. However I 'm willing to go out and buy the CD, cause I like it a lot.
Well, my spontaneous reaction is that you should keep it at q4, since 128 kbps was the target of this test - at least to begin with [{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347014")
Unfortunately there are no LAME -V4.7, iTunes VBR 123 kbps and WMA Pro45 options available. The new Nero encoder has been a secret so far and I don't know what options are available.
Vorbis Q4 makes lower bitrates than the other encoders with my various test track set.
Noxer found it to make even lower bitrates: [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=38955&view=findpost&p=343486]http://www.hydrogenaudio.org/forums/index....ndpost&p=343486[/url].
guruboolez got similar results with 150 classical tracks: http://foobar2000.net/divers/temp/128_bitrate_classical.xls (http://foobar2000.net/divers/temp/128_bitrate_classical.xls).
[span style='font-size:7pt;line-height:100%']Edit: a dot, a couple of enters and a link[/span]
Regarding the DL problem - you could use a DL manager, I guess.
[a href="index.php?act=findpost&pid=347019"][{POST_SNAPBACK}][/a]
Like Gozzila?...I suppose I could try this, but my problem is not getting the file, it's that what I get is trying to label itself: .php
Even the bit-size of the file I get matches the size your post says it should be...only it has this wierd extension and for some reason wants to name itself "index"?
This is really frustrating....does anyone even know what type files use extension: .php?
Regarding the DL problem - you could use a DL manager, I guess.
[a href="index.php?act=findpost&pid=347019"][{POST_SNAPBACK}][/a]
Like Gozzila?...I suppose I could try this, but my problem is not getting the file, it's that what I get is trying to label itself: .php
Even the bit-size of the file I get matches the size your post says it should be...only it has this wierd extension and for some reason wants to name itself "index"?
This is really frustrating....does anyone even know what type files use extension: .php?
[a href="index.php?act=findpost&pid=347027"][{POST_SNAPBACK}][/a]
Try Opera.
I could, but I need a tool¹ to cut losslessly an AAC or MP4 file. Does that exist? mp4box maybe?
¹ not too hard to use
MP4Box can do it:
-splitx start:end
extracts a new file from specified start to end times (in sec-
onds). This will remove all MPEG-4 Systems media. When input
file is an ISO-Media file (QT, MP4, 3GP), if no output is speci-
fied THE INPUT FILE IS OVERWRITTEN.
ex. "MP4Box -splitx 5:35 test.m4a". Although that produces test_5_35.m4a for me, so the manual seems to be out of date.
I haven't been able to listen to all the proposed test samples yet, but I ask instead: Is there any audiobook/dialogue from movie included? At least one classical and one jazz-like piece I hope...? And not only male singers but also female singers...? Then fine!
On to my main question: have you decided which version of abc/hr to use yet? I can inform you that if you choose the latest 0.5, it requires JRE 1.5 which is not available for Mac OS X up until 10.3 (which is what I use). I'm hoping that it will be ok to use either 0.5 or some older version which I can run, but I'm not so sure if the encryption etc are the same then... Schnoffler? Are you reading this?
I haven't been able to listen to all the proposed test samples yet, but I ask instead: Is there any audiobook/dialogue from movie included? At least one classical and one jazz-like piece I hope...? And not only male singers but also female singers...? Then fine!
On to my main question: have you decided which version of abc/hr to use yet? I can inform you that if you choose the latest 0.5, it requires JRE 1.5 which is not available for Mac OS X up until 10.3 (which is what I use). I'm hoping that it will be ok to use either 0.5 or some older version which I can run, but I'm not so sure if the encryption etc are the same then... Schnoffler? Are you reading this?
[a href="index.php?act=findpost&pid=347040"][{POST_SNAPBACK}][/a]
prior to the version which used JRE 1.5, you could get clicking sounds during playback.
ff123
prior to the version which used JRE 1.5, you could get clicking sounds during playback.
ff123
[a href="index.php?act=findpost&pid=347048"][{POST_SNAPBACK}][/a]
OK, but clicking sound is better than no sound at all. At least if it's not too frequent, and I never had a problem with it before... Upgrading my entire OS just to participate in the listening test is not an option for me. There may be a few others as well... Are we allowed to use older clicking versions?
I haven't been able to listen to all the proposed test samples yet, but I ask instead: Is there any audiobook/dialogue from movie included? At least one classical and one jazz-like piece I hope...? And not only male singers but also female singers...? Then fine!
[a href="index.php?act=findpost&pid=347040"][{POST_SNAPBACK}][/a]
No audiobook / movie dialogue since I think each encoder should cope with that at 128 kbps. I proposed that only for lowering the bitrate in case an encoder is off the 10% tolerance margin.
Classical: Ravel and Macabre, as well as Carbonelli. (Edit: First two are orchestral.)
Jazz-Like: Not really. I have this Bodyheat sample, but that's not really Jazz. Anyone wants to post one? You still have five hours left for doing so.
Female: Paris Combo - Senor
Edit: Jazz: I can replace the Gang Of Four sample with ATrain.
Jazz-Like: Not really. I have this Bodyheat sample, but that's not really Jazz. Anyone wants to post one? You still have five hours left for doing so.
Female: Paris Combo - Senor
Edit: Jazz: I can replace the Gang Of Four sample with ATrain.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347088")
Me! Me! Me!
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=39288&view=findpost&p=347102]Jazz sample[/url]
Jazz-Like: Not really. I have this Bodyheat sample, but that's not really Jazz. Anyone wants to post one? You still have five hours left for doing so.
Female: Paris Combo - Senor
Edit: Jazz: I can replace the Gang Of Four sample with ATrain.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347088")
Me! Me! Me!
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=39288&view=findpost&p=347102]Jazz sample[/url]
[a href="index.php?act=findpost&pid=347103"][{POST_SNAPBACK}][/a]
Thanks for providing. I am going to replace Natural's Not In It with the Jazz sample you provided.
prior to the version which used JRE 1.5, you could get clicking sounds during playback.
ff123
[a href="index.php?act=findpost&pid=347048"][{POST_SNAPBACK}][/a]
OK, but clicking sound is better than no sound at all. At least if it's not too frequent, and I never had a problem with it before... Upgrading my entire OS just to participate in the listening test is not an option for me. There may be a few others as well... Are we allowed to use older clicking versions?
[a href="index.php?act=findpost&pid=347052"][{POST_SNAPBACK}][/a]
Is a VM an option?
I uploaded three opera voice samples.
http://www.hydrogenaudio.org/forums/index....ndpost&p=347150 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39288&view=findpost&p=347150)
I uploaded three opera voice samples.
http://www.hydrogenaudio.org/forums/index....ndpost&p=347150 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39288&view=findpost&p=347150)
[a href="index.php?act=findpost&pid=347151"][{POST_SNAPBACK}][/a]
Thanks!
I have enough samples now.
Check this one out too if you didn't already: http://www.hydrogenaudio.org/forums/index....ndpost&p=347155 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39288&view=findpost&p=347155)
I'll make test encodings of the four voice samples I uploaded with e.g. LAME and try to quickly ABX them. I didn't have time to do that before. I'll tell what I found if anything. I think I'll try a step lower quality setting for making the possible artifacts more audible.
Is a VM an option?
[a href="index.php?act=findpost&pid=347145"][{POST_SNAPBACK}][/a]
VM as in virtual machine? I'd be happy to download a Java VM that I can run abc/hr under, but it's not available as far as I know.. Or did you mean the kind that emulates a PC? Well, then I still have to install an operating system on top of the one I have now - and I have neither enough hd space nor enough money (for the software) for that.
Easiest would clearly be to just use an old version of abc/hr if it is compatible with the cryptation etc...
Is a VM an option?
[a href="index.php?act=findpost&pid=347145"][{POST_SNAPBACK}][/a]
VM as in virtual machine? I'd be happy to download a Java VM that I can run abc/hr under, but it's not available as far as I know.. Or did you mean the kind that emulates a PC? Well, then I still have to install an operating system on top of the one I have now - and I have neither enough hd space nor enough money (for the software) for that.
Easiest would clearly be to just use an old version of abc/hr if it is compatible with the cryptation etc...
[a href="index.php?act=findpost&pid=347162"][{POST_SNAPBACK}][/a]
I meant a virtual PC running Linux for example. There are trial version of VMware and MS Virtual PC available. Additionally, I could set up a minimal Linux system on VMware and make it available to the public. I think it
could work with the free VMware Player.
Edit: Aren't there live Linux distributions that can run on Mac?
I meant a virtual PC running Linux for example. There are trial version of VMware and MS Virtual PC available. Additionally, I could set up a minimal Linux system on VMware and make it available to the public. I think it could work with the free VMware Player.
[a href="index.php?act=findpost&pid=347166"][{POST_SNAPBACK}][/a]
It could work. But is it worth it? What is the problem with using an older version of abc/hr from your point of view?
Edit: Aren't there live Linux distributions that can run on Mac?
[a href="index.php?act=findpost&pid=347166"][{POST_SNAPBACK}][/a]
Yes, but as I said, I'm not going to change OS just for a listening test...
I meant a virtual PC running Linux for example. There are trial version of VMware and MS Virtual PC available. Additionally, I could set up a minimal Linux system on VMware and make it available to the public. I think it could work with the free VMware Player.
[a href="index.php?act=findpost&pid=347166"][{POST_SNAPBACK}][/a]
It could work. But is it worth it? What is the problem with using an older version of abc/hr from your point of view?
[a href="index.php?act=findpost&pid=347168"][{POST_SNAPBACK}][/a]
I don't know how serious the clicking problem is.
It seems to be difficult to quickly ABX the voice samples I provided. I tried LAME -V5 --vbr-new and QT AAC vbr128. I guess the famous golden ears would be needed. I didn't try lower settings after all, because that would have not been the same.
One note though, the Tosca samples sung by Maria Callas are remastered mono recordings. LAME seems to make lower bitrates than the other encoders because it can use 100% ms frames in joint stereo mode. I don't know if that gives any advantage to the other encoders. I guess the others just waste space.
It seems to be difficult to quickly ABX the voice samples I provided. I tried LAME -V5 --vbr-new and QT AAC vbr128. I guess the famous golden ears would be needed. I didn't try lower settings after all, because that would have not been the same.
One note though, the Tosca samples sung by Maria Callas are remastered mono recordings. LAME seems to make lower bitrates than the other encoders because it can use 100% ms frames in joint stereo mode. I don't know if that gives any advantage to the other encoders. I guess the others just waste space.
[a href="index.php?act=findpost&pid=347175"][{POST_SNAPBACK}][/a]
Well, I think the samples are pretty easy. LAME and Vorbis produced bitrates varying between ~90 kbps and ~130 kbps, I assume the two ABR encoders iTunes and Nero will give similar results, the only difference being WMA Pro which might allocate a bit more. That's why I think none of the samples should be included. Hope you don't mind.
Well, I think the samples are pretty easy. LAME and Vorbis produced bitrates varying between ~90 kbps and ~130 kbps, I assume the two ABR encoders iTunes and Nero will give similar results, the only difference being WMA Pro which might allocate a bit more. That's why I think none of the samples should be included. Hope you don't mind.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347177")
I don't mind. This is not a matter of my minding.
As I said before I tried to find voice samples that would have produced quite low bitrates and be difficult at the same time, samples that could make the pshycoacoustic models more or less fail. It seems that only vigorous testing or previously heard artifacts (in real life) can help to find that kind of samples.
This "herding_calls" sample would be difficult at least for LAME:
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=37003&hl=]http://www.hydrogenaudio.org/forums/index....topic=37003&hl=[/url]
http://www.hydrogenaudio.org/forums/index....ndpost&p=326235 (http://www.hydrogenaudio.org/forums/index.php?showtopic=37002&view=findpost&p=326235)
... but it is more like a killer sample.
I posted a couple of samples by Björk: http://www.hydrogenaudio.org/forums/index....ndpost&p=347195 (http://www.hydrogenaudio.org/forums/index.php?showtopic=39288&view=findpost&p=347195). I think these samples are more likely to produce artifacts, not just because of her strong voice, but also because of the other sounds included. Especially the effect sounds in the "Storm" sample make it ABXable. It seems that plain human voice is too easy too encode.
I hope this has at least minor scientific value in case Sebastian has made his decisions.
Assuming I use Storm, which sample would you replace? It's pretty hard to sort out samples now.
Assuming I use Storm, which sample would you replace? It's pretty hard to sort out samples now. [a href="index.php?act=findpost&pid=347234"][{POST_SNAPBACK}][/a]
What is your current list? I am a bit uncertain about that.
I have downloaded all available samples. I could put them into a playlist and listen to through them. Hopefully that would bring up some idea about which samples are more redundant. Did you try that yourself? You have them all in a lossless format, so you should have better chances to evaluate them.
I have downloaded all available samples. I could put them into a playlist and listen to through them. Hopefully that would bring up some idea about which samples are more redundant. Did you try that yourself? You have them all in a lossless format, so you should have better chances to evaluate them.
[a href=\"index.php?act=findpost&pid=347261\"][{POST_SNAPBACK}][/a]
Yep, and that's why it is so hard.
These are the ones I have:
02_-_White_America
BigYellow
bodyheat
Carbonelli
Coladito
electronic
Elizabeth
Elton_John___Song_For_Guy__edit_
eric_clapton
Jazz
macabre
Opeth___The_Drapery_Falls_sample
Paris_Combo___Senor
ravel
Santa Esmeralda - Don't Let Me Be Misunderstood
Sash_-_Mysterious_Times_30sec
School
Yann Tiersen - Les Jours Heureux
I think I could take out macabre since I have the Ravel sample. But the question is - is that kind of sample needed?
I think I could take out macabre since I have the Ravel sample. But the question is - is that kind of sample needed?
I'm not perhaps the right person to answer on that particular sample.
I do wish however, that you'd consider adding a sample (if you haven't already) that includes very low volume parts (with high dynamic variation). The miyake sample I posted is just an example, but something similar would be nice.
I've noticed that many of the codecs can still fail easily on this type of track at c. 128kbps.
That is, if you think that kind of low volume test is worthwhile.
Well, the problem is that it is really hard to decide which samples to throw out now. The only sample I can think of would be macabre since we have Ravel as orchestral track. But then, which sample to choose: Björk or yours?
Edit: Or maybe throw out Yello and replace it by Björk. We have Sash! for electronica.
I listened through the samples and I am quite clueless too. The choice is difficult.
In regard to the choice between the two orchestral samples I noticed again the dynamics problem with the Ravel sample I provided. The Ravel recording is not compressed in any way. The sample has internally already high dynamic variation, but the complete album has more. For example, here are some replay gain values of the album:
The Ravel sample: -7.54
The complete track: -4.40
The previous track: +10.29
The complete album: +0.27 (album gain)
The sample passage is about the loudest part of the symphony and it is intended to produce something like 18 db higher average volume level than the previous quiet track when listening to the original album. When I listened to it together with the other samples I had to increase the volume setting a lot before I was able to hear the subtle nuances included with this sample.
[span style='font-size:7pt;line-height:100%']Edit: typo & removed my OT comment.[/span]
Well, the problem is that it is really hard to decide which samples to throw out now. The only sample I can think of would be macabre since we have Ravel as orchestral track. But then, which sample to choose: Björk or yours?
Edit: Or maybe throw out Yello and replace it by Björk. We have Sash! for electronica.
[a href="index.php?act=findpost&pid=347341"][{POST_SNAPBACK}][/a]
Just a though: The Yello sample is (I presume) pretty tough for the encoders. It has wide stereo separation and some very sharp, distinct sounds.
Now, whether that means it should be included instead of Sash! or not, I don't know.
Edit: Yello, not Yellow (duh!)
I think the Björk sample "storm" is great! So are electronic.mp4 (Yello track) and to some extent the Sash track (Mysterious times).
I'd guess that they also represent more of the style of music that is really encoded in real-life, than the one I was proposing.
Let me make this easier for you. Forget the low volume request I made. There are enough low volume parts in other acoustic orchestral samples
OK, discussion is now over. I am just preparing the samples.
Thanks to everyone for suggestions, help and comments. Also, thanks to all who provided samples!
OK, discussion is now over. I am just preparing the samples.
Thanks to everyone for suggestions, help and comments. Also, thanks to all who provided samples!
[a href="index.php?act=findpost&pid=347520"][{POST_SNAPBACK}][/a]
Finally the test has been prepared and I want to take part in it but in a bit different way. I will add the same codecs with the same settings to SoundExpert (SE) testing service. They will be placed to the group, say, “Special 128 Kbit/s test” with short explanation of the test purpose (and reference to the HA test of course). First (starting) ratings based on my own listening tests and grades will be published simultaneously with the HA ones in order the both tests to be independent. Then if God send some testers to SE the ratings will start to change and finally come to some stable figures.
I think it would be interesting to compare results of both tests. SE ones are based on different methodology and use different sound samples. Will there be any correlation? I use the word “correlation” because both results can’t be compared directly as SE ratings will occupy the range above 5 points (5 to 7 range, I think) of Infinite Grade Impairment Scale and relative positions will matter only.
For me as the developer of SoundExpert two following questions would be interesting as well:
How ratings will move as participants will add their grades to my starting ones? Will the difference be substantial?
So my question is to Sebastian: could we agree now about the exact date ant time of publishing results? And could you, please, show the final codecs set with their settings to be used. Thank you in advance.
OK, discussion is now over. I am just preparing the samples.
Thanks to everyone for suggestions, help and comments. Also, thanks to all who provided samples!
[a href="index.php?act=findpost&pid=347520"][{POST_SNAPBACK}][/a]
Finally the test has been prepared and I want to take part in it but in a bit different way. I will add the same codecs with the same settings to SoundExpert (SE) testing service. They will be placed to the group, say, “Special 128 Kbit/s test” with short explanation of the test purpose (and reference to the HA test of course). First (starting) ratings based on my own listening tests and grades will be published simultaneously with the HA ones in order the both tests to be independent. Then if God send some testers to SE the ratings will start to change and finally come to some stable figures.
I think it would be interesting to compare results of both tests. SE ones are based on different methodology and use different sound samples. Will there be any correlation? I use the word “correlation” because both results can’t be compared directly as SE ratings will occupy the range above 5 points (5 to 7 range, I think) of Infinite Grade Impairment Scale and relative positions will matter only.
For me as the developer of SoundExpert two following questions would be interesting as well:
How ratings will move as participants will add their grades to my starting ones? Will the difference be substantial?
So my question is to Sebastian: could we agree now about the exact date ant time of publishing results? And could you, please, show the final codecs set with their settings to be used. Thank you in advance.
[a href="index.php?act=findpost&pid=347523"][{POST_SNAPBACK}][/a]
Still preparing - slow down. Unfortunately, I was stuck in a traffic jam yesterday, so I am slightly off with my schedule. Nothing to worry about, though, I hope.
- Settings are:
iTunes 6.0.1.3: 128 kbps, VBR
LAME 3.97b2: -V5 --vbr-new --noreplaygain
Nero 3.1.0.2 (only I have it ATM): Streaming Profile (make sure LC is selected!)
Shine 0.1.4: -b 128
AoTuV 4.51: -q 4.25 (or 4,25 depending on system settings)
WMA Pro 9.1 (using VBS): -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24 - Date for results depends if the test has to be extended or not - it's too soon to say.
Settings are:
iTunes 6.0.1.3: 128 kbps, VBR
LAME 3.97b2: -V5 --vbr-new --noreplaygain
Nero 3.1.0.2 (only I have it ATM): Streaming Profile (make sure LC is selected!)
Shine 0.1.4: -b 128
AoTuV 4.51: -q 4.25 (or 4,25 depending on system settings)
WMA Pro 9.1 (using VBS): -a_codec WMA9PRO -a_mode 2 -a_setting Q50_44_2_24
Thank you. Everything is clear to me exept "Nero" - what application or component (dll) do you mean specifying the version?
Date for results depends if the test has to be extended or not - it's too soon to say.[a href="index.php?act=findpost&pid=347529"][{POST_SNAPBACK}][/a]
OK. We'll return to this later.
Thank you. Everything is clear to me exept "Nero" - what application or component (dll) do you mean specifying the version?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347544")
aacenc32.dll
[a href="http://img231.imageshack.us/my.php?image=properties2ew.png](http://img231.imageshack.us/img231/6412/properties2ew.th.png)[/url]
BTW, I am going to use ABC/HR 0.5a since JRE is available for Windows and Linux. Mac users can take the test on a live Linux system like Ubuntu or install a trial version of a VM software.
aacenc32.dll
It seems to be technical pre build Ivan provided to you for the test. I've got 3.0.0.2 in my system. If it is so, may I ask you to encode my single test file (http://www.soundexpert.info/refsamples/se_ref.rar) (8.71Mb), zip it and send to me in some way?
EDIT: file size
BTW, I am going to use ABC/HR 0.5a since JRE is available for Windows and Linux. Mac users can take the test on a live Linux system like Ubuntu
Not possible.
or install a trial version of a VM software.
[a href="index.php?act=findpost&pid=347554"][{POST_SNAPBACK}][/a]
Are you sure sound drivers in linux running on emulated PC on an ibook will sound better than simply using an old abc/hr which uses JRE 1.4?
I'm disappointed if you enforce the use of abc/hr v0.5 since I can't take the test then.
BTW, I am going to use ABC/HR 0.5a since JRE is available for Windows and Linux. Mac users can take the test on a live Linux system like Ubuntu
Not possible.
or install a trial version of a VM software.
[a href="index.php?act=findpost&pid=347554"][{POST_SNAPBACK}][/a]
Are you sure sound drivers in linux running on emulated PC on an ibook will sound better than simply using an old abc/hr which uses JRE 1.4?
I'm disappointed if you enforce the use of abc/hr v0.5 since I can't take the test then.
[a href="index.php?act=findpost&pid=347557"][{POST_SNAPBACK}][/a]
OK, I'll check if configs made with 0.5 work with 0.4 and vice versa.
OK, I'll check if configs made with 0.5 work with 0.4 and vice versa.
[a href="index.php?act=findpost&pid=347559"][{POST_SNAPBACK}][/a]
Thanks. Have you had any contact with schnoffler? He should know about the differences...
aacenc32.dll
It seems to be technical pre build Ivan provided to you for the test. I've got 3.0.0.2 in my system. If it is so, may I ask you to encode my single test file (http://www.soundexpert.info/refsamples/se_ref.rar) (8.71Mb), zip it and send to me in some way?
EDIT: file size
[{POST_SNAPBACK}][/a]
(http://index.php?act=findpost&pid=347555")
You can test yourself
[a href="http://www.rarewares.org/rja/nero_aac_encoder_3.1.rar]http://www.rarewares.org/rja/nero_aac_encoder_3.1.rar[/url]
http://www.rarewares.org/rja/nero_aac_encoder_v3_1_0_1.rar (http://www.rarewares.org/rja/nero_aac_encoder_v3_1_0_1.rar)
http://www.rarewares.org/rja/nero_aac_encoder_v3_1_0_2.rar (http://www.rarewares.org/rja/nero_aac_encoder_v3_1_0_2.rar)
Extract
Aac.dll and
aacenc32.dll to
%CommonProgramFiles%\Ahead\AudioPlugins\Extract
NeroIPP.dll to
%CommonProgramFiles%\Ahead\Lib\Extract
libmmd.dll to
%SystemRoot%\system32\.
OK, I'll check if configs made with 0.5 work with 0.4 and vice versa.
[a href="index.php?act=findpost&pid=347559"][{POST_SNAPBACK}][/a]
Thanks. Have you had any contact with schnoffler? He should know about the differences...
[a href="index.php?act=findpost&pid=347562"][{POST_SNAPBACK}][/a]
I sent him a PM since I also have a problem that no sound comes out of my speakers.
...
Mac users can take the test on a live Linux system like Ubuntu
Not possible.[a href="index.php?act=findpost&pid=347557"][{POST_SNAPBACK}][/a]
Why not? Live Linux system boots off a CD-ROM disc to RAM and nothing is written to a HDD. No need to remap disk partitions or install software, but internet connection or CD/DVD writer are required (to save results).
...
Mac users can take the test on a live Linux system like Ubuntu
Not possible.[a href="index.php?act=findpost&pid=347557"][{POST_SNAPBACK}][/a]
Why not? Live Linux system boots off a CD-ROM disc to RAM and nothing is written to a HDD. No need to remap disk partitions or install software, but internet connection or CD/DVD writer are required (to save results).
[a href="index.php?act=findpost&pid=347571"][{POST_SNAPBACK}][/a]
Yep, that's what I would say, too. Schnoefler was kind enough to offer a 0.5 version of ABC/HR for Java that operates with JRE 1.4, however, the clicking noise is still there because of how JRE 1.4 deals with the mixer.
He also recommends using 0.5 configs with 0.5 and 0.4 configs with 0.4 because of the encryption which might cause trouble.
You can test yourself
Thanks a lot. (http://www.rarewares.org/rja/nero_...............[/quote)
Slowly getting there... Sorting out final problems with ABC/HR not calculating the right offset here - works fine on schnofler's side, though.
Oh, and it seems that the version he told me that he can provide can use JRE 1.5 if it's available and when not, it falls back to 1.4 (he and I hope ).
Oh, and it seems that the version he told me that he can provide can use JRE 1.5 if it's available and when not, it falls back to 1.4 (he and I hope ).
[a href="index.php?act=findpost&pid=347592"][{POST_SNAPBACK}][/a]
Sounds good. You've put in a great effort, so far, Sebastian.
Oh, and it seems that the version he told me that he can provide can use JRE 1.5 if it's available and when not, it falls back to 1.4 (he and I hope ).
[a href="index.php?act=findpost&pid=347592"][{POST_SNAPBACK}][/a]
Sounds good. You've put in a great effort, so far, Sebastian.
[a href="index.php?act=findpost&pid=347593"][{POST_SNAPBACK}][/a]
Schnofler is the one to worship for ABC/HR for Java.
You can test yourself
Thanks a lot.
[a href="index.php?act=findpost&pid=347575"][{POST_SNAPBACK}][/a]
(http://www.rarewares.org/rja/nero_...............[/quote)
Notice that in this build for the test *only* the "Streaming :: Medium" with LC-AAC which produces about 128-133kbps average with ABR coding is tweaked and ready for the test using our new generation codec core.
You can test yourself
Thanks a lot.
[a href="index.php?act=findpost&pid=347575"][{POST_SNAPBACK}][/a]
(http://www.rarewares.org/rja/nero_...............[/quote)
Notice that in this build for the test *only* the "Streaming :: Medium" with LC-AAC which produces about 128-133kbps average with ABR coding is tweaked and ready for the test using our new generation codec core.
[a href="index.php?act=findpost&pid=347609"][{POST_SNAPBACK}][/a]
Wait a second, now I am a bit confused. Didn't you guys say that exactly this version of the DLL is going to be included in the Nero package and that you are sending me this so I don't have to wait for the whole Nero package to be released?
Edit:
Sebastian Mares> For the experimental encoder of Nero Digital, I'm a bit puzzled. The test would include on one hand four encoders released for general purpose and widely available to the public and on the other hand one encoder released for the only purpose of the test.
This is a non-issue, Nero AAC encoder that will be used in the test will be in the next regular web-release of Nero7. We had similar cases earleir and all changes ended up in the next public build of N7.
[a href="index.php?act=findpost&pid=344136"][{POST_SNAPBACK}][/a]
Will the next update of Nero Digital include the tested encoder and produce exactly the same output? If the answer is positive, I agree with you about the non-issue
[a href="index.php?act=findpost&pid=344140"][{POST_SNAPBACK}][/a]
It will be identical copy
[a href="index.php?act=findpost&pid=344141"][{POST_SNAPBACK}][/a]
I think it is enough if the used setting produces identical files with the release version. That can be checked after the release. They said they had not enough time to tweak the other settings.
I see no reason why they should not include the tweaks in case the test result is good. If the result is bad, hmm... then perhaps they have a good reason to do more work on it, it will not harm the validity of the other results.
I don't think they could have made a "test only" encoder that performs only with the tested setting. It is not a racecar engine. If that would be possible they should include it in the Nero package anyway and use it only with this specific setting.
Actually, LAME 3.97b2 was released a couple of days ago and it is already included in the test without any kind of experience. The same can be said about Vorbis 4.51. Though, the Vorbis changes were said to be at below q3.
In that sense we have three racecar engines here.
[span style='font-size:7pt;line-height:100%']Edit: a typo again[/span]
Ah, so the output is an identical copy, not the DLL itself. Sorry, I misunderstood that.
I am pretty much ready. Waiting for Roberto to show up in ICQ so he can grab the samples and host them. FYI, I did not include any of the new samples posted (silent with much dynamics, Björk, etc.). Maybe in the next test.
How much run-in delay are you going to put in the abchr-java config files? I think Gabriel recommended several seconds.
ff123
2 seconds (Gabriel recommended this via PM).
...
Mac users can take the test on a live Linux system like Ubuntu
Not possible.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=347557")
Why not? Live Linux system boots off a CD-ROM disc to RAM and nothing is written to a HDD. No need to remap disk partitions or install software, but internet connection or CD/DVD writer are required (to save results).
[a href="index.php?act=findpost&pid=347571"][{POST_SNAPBACK}][/a]
That may be very possible, but it doesn't help unless there is also a JRE v1.5 for it. And I can't see PowerPC in this list: [a href="http://java.sun.com/j2se/1.5.0/system-configurations.html]http://java.sun.com/j2se/1.5.0/system-configurations.html[/url]
Oh, and it seems that the version he told me that he can provide can use JRE 1.5 if it's available and when not, it falls back to 1.4 (he and I hope ).
[a href="index.php?act=findpost&pid=347592"][{POST_SNAPBACK}][/a]
Sounds good. You've put in a great effort, so far, Sebastian.
[a href="index.php?act=findpost&pid=347593"][{POST_SNAPBACK}][/a]
Benno is the one to worship for ABC/HR for Java.
[a href="index.php?act=findpost&pid=347595"][{POST_SNAPBACK}][/a]
Excellent. Should I test it on my ancient iBook before the test starts?
That may be very possible, but it doesn't help unless there is also a JRE v1.5 for it. And I can't see PowerPC in this list: http://java.sun.com/j2se/1.5.0/system-configurations.html (http://java.sun.com/j2se/1.5.0/system-configurations.html)[a href="index.php?act=findpost&pid=347699"][{POST_SNAPBACK}][/a]
Well, for all that it's worth, MacOS isn't mentioned there either :B
That may be very possible, but it doesn't help unless there is also a JRE v1.5 for it. And I can't see PowerPC in this list: http://java.sun.com/j2se/1.5.0/system-configurations.html (http://java.sun.com/j2se/1.5.0/system-configurations.html)[a href="index.php?act=findpost&pid=347699"][{POST_SNAPBACK}][/a]
Well, for all that it's worth, MacOS isn't mentioned there either :B
[a href="index.php?act=findpost&pid=347705"][{POST_SNAPBACK}][/a]
If it was, I wouldn't have asked for an older version in the first place. The JRE 1.5 for OS X 10.4 is only available from apple for some reason. Seems silly both to only announce it there, and to only release it for 10.4, but that's apple...