Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Public Listening Test [2010] (Read 176092 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Public Listening Test [2010]

Reply #75
I would like see to DivX's AAC encoder on the test.

320kbps or V0 Mp3 would make a good high anchor if samples such as: eig, kraftwerk, Show Me Your Spine and castanets are used.
"I never thought I'd see this much candy in one mission!"

Public Listening Test [2010]

Reply #76
I'd welcome to have winamp's CT AAC encoder included.
As a high anchor I guess Lame -V2 can be worse for some problem samples than a good AAC encoder @128 kbps. From previous discussion I thought lossless was to be the high anchor.

Right. Taking into account that test will have a high amount of difficult samples even AAC at 128 kbps won't be rated too high as to be compared to high anchor.

The following rules will be enough to avoid the use of high anchor:
Code: [Select]
Remove all listeners from analysis who
a) graded the reference lower than 4.5,
b) graded the low anchor higher than all competitors.
c) didn't grade the low anchor.


Then the proposal list (without high anchor) is:
Nero, Apple, CT, Divx. Low anchor: LC-AAC 64 kbps.


Public Listening Test [2010]

Reply #77
I thought I'd just pipe in and say I'm opposed to using Apple TVBR for the reasons already cited by others.  No disrespect to nao, but I have a hard time understanding how easily his encoder slipped right into the mix.

EDIT: Does TVBR require QuickTime Pro?  This would be another reason why I'd be opposed to TVBR.

Public Listening Test [2010]

Reply #78
I thought I'd just pipe in and say I'm opposed to using Apple TVBR for the reasons already cited by others.  No disrespect to nao, but I have a hard time understanding how easily his encoder slipped right into the mix.

Poll results
Poll was made before nao's post.
Many people want to see TVBR even they aren't MAC users.


Public Listening Test [2010]

Reply #80
Yeah, I don't like it either.  It takes 5 minutes (by the time track tag information is added) to encode one TVBR QuickTime AAC file under Windows using QuickTime Pro.  Aside from nao's utility, QuickTime Pro is required for TVBR AAC encoding on Windows.  I would have preferred CVBR simply because it is integrated into iTunes and is (in my opinion) easy to use.  The masses want something else though so I can live with that.  I still wish that it were included alongside QuickTime TVBR.

Public Listening Test [2010]

Reply #81
I agree. Since I have yet to see a blind test comparing CVBR and TVBR for multiple items, done my multiple people (if there is one, please point me to it!), why not include both in this test? Moreover, right now the poll is only slightly in favor of TVBR.

Chris
If I don't reply to your reply, it means I agree with you.

Public Listening Test [2010]

Reply #82
One thing I'm afraid that the difference between TVBR and CVBR are far less than between different encoders. People will have hard time to try to find any difference on T/CVBR  instead of actually compare different encoders.

We can do poll and see but at least now I didn't find any sample where the difference between TVBR and CVBR was any pronounceable. If you find one let's know.




Public Listening Test [2010]

Reply #83
The effect of TVBR is mainly space saving at higher bitrates. You can use overkill quality settings comparable to iTunes Plus without wasting as much disk space. I don't think that there would be much difference at 128kbit/s. Still it would be interesting to see this verified. As C.R. Helmrich noted, there isn't much good data, yet.

Public Listening Test [2010]

Reply #84
This Poll will define T&CVBR's comparison.

Public Listening Test [2010]

Reply #85
As now Apple TVBR is available for Windows users shall we drop CVBR?

Encoder settings for 128 kbps.
Nero -q 0.xx
Divx -v 4
CT(media coder) LC, maybe shifted to ~130 kbps

There isn't much to choose.


Apple(qtaacenc) -tvbr 65/--cvbr 128
--high (default)  or --highest?

Public Listening Test [2010]

Reply #86
IMO we should use --highest.

Though not many people will be interested:
Now that we got qtaacenc I just did a small listening test with harp40_1 and trumpet_myPrince which were the most problematic samples of 'mine' when used with Nero.
--tvbr 65 and --cvbr 128 were both not transparent, and they also sounded pretty identical to me.
--abr 128 wasn't transparent as well of course, but the result sounded better to me than the VBR results.
I know many people think that only VBR is the way to go, true VBR if possible, but the variable bitrate method ABR can have its merits on its own when it's up to quality, not efficiency. It also uses high bitrate if necessary but obeys to another quality decision making process than does VBR.
More interesting: AFAIK there was never a listening test ABR vs. VBR, at least not in recent years with up-to-date encoders. There's always just the widespread beleive that VBR is the best way to go qualitywise. I don't want to say that's wrong, but it would be fine to have this beleive verified in a test.

My suggestion: let's do a pre-test QT ABR vs. CVBR and use the best.
(Of course that would bring the problem to also give the ABR way a chance for the other encoders who have this option in case ABR will be the winner, but a priori the situation that ABR should win against CVBR isn't what most people expect to happen. And if it should occur I think it's worth while thinking then about how to deal with the situation in the real test).
lame3995o -Q1.7 --lowpass 17

Public Listening Test [2010]

Reply #87
As now Apple TVBR is available for Windows users shall we drop CVBR?

I say no because the vast majority of Windows users (and likely HA users as well) are going to still use iTunes instead of QuickTime.

Public Listening Test [2010]

Reply #88
halb,
I think we came back to specific options or versions of one competitor.
Then we should give an opportunity to Nero's ABR or another version as well.
We've reached to conclusion that only one version of Nero will be tested and untill now only VBR. (however I also find samples where Nero ABR perfoms better than VBR).

Then we should pre-test: Divx vs CT, Nero ABR(maybe even ABR+2pass), Nero VBR, Apple ABR, Apple CVBR and I don't want to mention Apple quality settings --normal vs --high. It's too much unless we can arrive to conclusion to discard some of those pre-tests.

I tend to propose to rely on developer suggestions as they know their codec better than any other one:

Nero VBR (the same as in previous public test)
I've sent an e-mail to Apple developer and he said that VBR should be the best option.

However personally now after your mentions  I'm interested to see ABR and VBR at least in my tests (and maybe in pre-test).

Public Listening Test [2010]

Reply #89
I understand your considerations, and part of them came to my mind as well when considering ABR.
Anyway glad to hear that you encountered experience like mine and want to do a personal test or even the pre-test also with ABR.
lame3995o -Q1.7 --lowpass 17

Public Listening Test [2010]

Reply #90
iTunes AAC encodes at fixed quality option which is identical to qtaacenc's --normall.
If --highest will be enabled in QTAacEnc (QTAE) then it won't pure CVBR/ABR vs TVBR comparison but TVBR+highest vs CVBR/ABR+normal. But it  shouldn't be necesary  pure CVBR vs TVBR. I’m fine with TVBR+highest vs CVBR/ABR+normal comparison.

Public Listening Test [2010]

Reply #91
I'm not quite sure I understand what you're suggesting.

Is it this?:
iTunes uses CVBR and ABR, and for this reason you want to use iTunes' fixed quality setting which corresponds to qtaacens's --normal.
As iTunes doesn't support TVBR you'd like to use --highest for TVBR.
lame3995o -Q1.7 --lowpass 17

Public Listening Test [2010]

Reply #92
Yes, that's correct.

Maybe there are people who are interested in  pure T/CVBR comparison  with --normal for both QTAE and iTunes.  As there is no any quality setting except default/fixed --normal in iTunes.

But I'm more interested in TVBR+highest vs CVBR/ABR+(fixed)normal.

Public Listening Test [2010]

Reply #93
This open discussion will have enough time but can't be endless. All items (codec, settings, bitrate, samples, etc) will have a deadline. The plans are to begin the test in early March.

Untill this moment the list of AAC encoders is almost established and its deadline is 1 week (untill 1st of February). 
Please, speak now.

List of AAC encoders to test is:
1.Nero 1.5.1
2.Apple TVBR
3.Apple CVBR or ABR
4. The winner of pretest (CT vs Divx)


Public Listening Test [2010]

Reply #94
As for 3. I'd like to see ABR in the test because CVBR results are expected to be more or less identical to those of TVBR. I guess optimum strategy however to keep everybody happy is to have a pretest ABR vs. CVBR.
I'm not so happy comparing TVBR --highest against CVBR/ABR --normal, but I hope this doesn't change things a lot.
lame3995o -Q1.7 --lowpass 17

Public Listening Test [2010]

Reply #95
I've tried a few samples for ABX test on ABR vs TVBR vs CVBR and have found  that ABR felt badly on Fatboy-like samples. The bitrate is extremely high for such killer samples and ABR is bitrate restricted on them.

I think we attribute too detailed attention just only to Apple encoder. It will be very hard to test all settings and modes for all encoders.  We can't just test settings for Apple encoder without give same possibility to Nero.

For some good reasons Nero and Apple devs both suggest VBR. I don't think it's coincidence.


Opinions?

Deadlines:
1. List of codecs (February 1)
2. Settings (February 8)
3. Samples (February 15)
...

Public Listening Test [2010]

Reply #96
OK, I see the arguments (and your findings), so let's forget about ABR.
Thanks anyway for having given it a chance.
lame3995o -Q1.7 --lowpass 17

Public Listening Test [2010]

Reply #97
The rules:
Code: [Select]
Remove all listeners from analysis who
1. graded the reference lower than 4.5
2. graded the low anchor higher than all competitors.
3. didn't grade the low anchor.
4. didn't grade any of competitors.

Chris has suggested to change 1st rule to "graded the reference lower than 5.0" here

I tend to disagree.
Imagine situation that there is codec A which has very noticeble artifacts on sample and ranked at pretty low score while codec B did a good job on lowpassing an old noisy record. Codec B could be ranked higher than reference lossless -> reference can be ranked lower than 5.0 as described in http://www.rarewares.org/rja/ListeningTest.pdf

The result could be:
Code: [Select]
Codec A - 3.0
Reference A -5.0

Codec B - 5.0
Reference B - 4.9


It's valid result for me.

Public Listening Test [2010]

Reply #98
Nero Bitrate settings:

-q 0.42 is comparable with Apple T&CVBR 128 kbps on classic music
while -q 0.41 is on rock/alternative music

-q 0.415 can be reasonable setting for this test.

It will be good if someone else can confirm that too.

Public Listening Test [2010]

Reply #99
Regarding the bitrates, I have run some tests using my standard "Various" and "Classical" file sets.

Sow far I have encoded the sets with:

- iTunes VBR @ 96 kbps and @ 128 kbps
- QT tvbr @ -64 (i.e. about 128 kbps), "best" quality  -- or whatever the actual string is, I don't have the settings in front of me now.

Am I understanding correctly, that the constrained VBR setting in QT is more likely to be tested than the "constrained" iTunes VBR? I'd prefer iTunes because then the test would tell if using QT is worthwhile. (edit: ... i.e. if using QT tvbr instead of iTunes is worthwhile)

I could add the latest Nero and try to find a matching setting. Apparently the test is going to be a "128" test, or is "96" still a possibilty?

In general, does anyone have an opinion of which method should be used for checking the AAC bitrates? Should we blindly trust the applications that just read the header data? For instance, foobar and Mr QuestionMan don't always agree. Actually, apparently Mr Q. can't read QT tvbr files at all, but that can be fixed by retagging the files.