Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Autumn 2006 Listening Test (Read 144572 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Autumn 2006 Listening Test

Reply #250 –
If I see it correctly that's fastenc cbr 128 from WMP.
As the WMP proposal came from jmartis it would be best if he told us which version he has in mind.
lame3995o -Q1.7 --lowpass 17

Autumn 2006 Listening Test

Reply #251 –
Quote from: halb27 on
If I see it correctly that's fastenc cbr 128 from WMP.
As the WMP proposal came from jmartis it would be best if he told us which version he has in mind.

I believe both WMP 10 and 11 use FHG Fastenc v3.3.x.44 (as ACM codec)

Autumn 2006 Listening Test

Reply #252 –
Quote from: jmartis on
I believe both WMP 10 and 11 use FHG Fastenc v3.3.x.44 (as ACM codec)

WMP 11 beta installer includes version 3.4.0.0.

Edit. The latest version of l3codecp.acm from WMP11 (beta 2) can be found here.
 

Autumn 2006 Listening Test

Reply #254 –
Guys, I see no point discussing this further because we won't get any clear result. I will stick to 128 kbps CBR using Windows Media Encoder. Now let's discuss samples - do you think only new samples should be used? If not, which old samples do you recommend?

Autumn 2006 Listening Test

Reply #255 –
Quote from: Sebastian Mares on
Guys, I see no point discussing this further because we won't get any clear result. I will stick to 128 kbps CBR using Windows Media Encoder. Now let's discuss samples - do you think only new samples should be used? If not, which old samples do you recommend?

What Windows Media Encoder do you mean? I know WME9 but this encodes only to wma (or have I overlooked something?) Guess you mean WMP10 or WMP11?

ADDED:
As for the samples for simplicity (and comparision) can't we just take those from the last 128 kbps listening test or take them as a basis for discussion. And add a series of problem samples?
lame3995o -Q1.7 --lowpass 17

Autumn 2006 Listening Test

Reply #258 –
I almost hate to interject at this point... as you seem to all know a whole lot more about this than I do... but doesn't iTunes use a FhG MP3 encoder?

Autumn 2006 Listening Test

Reply #259 –
@Sebasian:

I don't see any objection using the samples from the last 128 kbps test you did.
"We cannot win against obsession. They care, we don't. They win."

Autumn 2006 Listening Test

Reply #261 –
Quote from: Sebastian Mares on
Yeah, but it would be wise to use some killer samples too. I would like to know which ones you think should be used and which of the old samples you think should be excluded this time.

I don't see any reason to exclude some samples.


My suggestion.

One hypothesis: using the same samples to feed two similar tests is necessary (but not sufficient) for cross-comparison purpose. So why not using the same 12 samples used in the past during the last MP3@128 listening test? We could use this 12 samples, and add 6 other ones. That would make 18 samples. And as appendix to the final result, the conducer could publish the plots corresponding to the 12 first samples and put in regard the results obtained in 2004. It's maybe a way (good or bad?) to show the progress done in two years and that would be the second teaching of this test (the first one is of course the relative quality of all competitors).

What do you think? Am I clear?

Autumn 2006 Listening Test

Reply #263 –
This thread is like very old opera, almost 2 months discussions and no conclusion, congratulations for author!

Autumn 2006 Listening Test

Reply #264 –
Quote from: Sebastian Mares on
Don't you think 18 samples are too much?

In my opinion the optimal number of samples depends on the overall motivation. Because I really can't evaluate this level I absolutely can't answer to your question. If people are motivated, 18 samples should be fine ; if they aren't, even 12 would be too much.

Autumn 2006 Listening Test

Reply #265 –
Considering that it was hard to have enough contributions to conclude the low bitrate AAC test, I think that 18 samples on mid bitrate might be too much.

While we have some people that will be motivated to provide results, unfortunately too many people are interested by the results but think that they don't need to contribute, and are simply waiting for other people to take the test.

Autumn 2006 Listening Test

Reply #266 –
With so many samples, I think that I will have a much much higher number of results for samples 1 to 5 compared to 15 to 18.

Quote from: adamjk on
This thread is like very old opera, almost 2 months discussions and no conclusion, congratulations for author!


Discussion about encoders is over and we now moved to discussing samples. I am sorry that I had to leave to Romania and clarify some things, but "real life" is more important to me. I would really like to accelerate the whole pre-discussion thing so I can take care of the preparations and get the test started.

Autumn 2006 Listening Test

Reply #267 –
I think all of the potential samples should included in the running.  Then randomly select what samples will be used.  It's the only way to be fair.

Picking and choosing samples carefully is not fair, you would be influencing the results.
Vorbis-q0-lowpass99
lame3.93.1-q5-V9-k-nspsytune

Autumn 2006 Listening Test

Reply #268 –
Quote from: Gabriel on
Considering that it was hard to have enough contributions to conclude the low bitrate AAC test, I think that 18 samples on mid bitrate might be too much.

While we have some people that will be motivated to provide results, unfortunately too many people are interested by the results but think that they don't need to contribute, and are simply waiting for other people to take the test.

Low bitrate help tester to discern lossy from reference. But it was pretty difficult to tell each contender in AAC 48kbps test. I'm not surprised if this can result in smaller number of tester.

Last 128kbps multiformat test had more tester. 18 samples is good for the test which people are interested in.

Autumn 2006 Listening Test

Reply #269 –
Is there any news on when these tests are likely to proceed?

Autumn 2006 Listening Test

Reply #270 –
As for the problem samples I suggest we choose from each of the problem categories (pre-echo, ringing, warbling, ... , as well as distortions that don't match well with one of the preceding categories) two or three samples and choose one sample out of each category by chance as gameplaya suggested.

As for a pre-echo sample I suggest castanets.
As for the not-clearly classifiable distortions I suggest harp40_1 (in case somebody can classify it: put it in that category).
lame3995o -Q1.7 --lowpass 17

Autumn 2006 Listening Test

Reply #272 –
Quote from: Sebastian Mares on
Thanks. "seemed to work" - well, they do if they have the right syntax, but do people agree that they sound well?

Why don't you verify this by yourself?

Quote from: Sebastian Mares on
And do you guys think I should use some cryptic command line arguments or should I stick to the simple -V setting (and -X2 to write a Xing header with TOC)?

How could you decide or taking a decision without previous testing?
the "some cryptic command line arguments" (as you called them) were the
result of a very SERIOUS, DEDICATED, HARD and LONG test which I performed many months ago, but that the people here didn't check by LAZINESS.

Obviously, this is a very irresponsible position deprecate the results from this hard test without testing
nor verification; right Sebastian?

Going to the topic again:
How do you know that [-V60 -X2] is better or not than [-V60 -X2
-HF2 -SBT450 -TX0] for example? Obviously, you cannot know it; much less, recommend it or suggest it for a public test.

At least, for high bitrate test; they IF do a lot better difference there, and more specifically, TX" switch, whose default value in Helix is not the adequate.

BTW, I don't have idea if in low bitrate the "some cryptic command line arguments" do a good result there; and now I don't have the patience nor the motivation for performing ABX tests for obvious reasons, but
if you think to use Helix in this public listening test AT LEAST would have to take more seriously my test as an initial reference; because it was performed according to the rules from HA.

Autumn 2006 Listening Test

Reply #273 –
level, I didn't mean to offend anyone. Problem is that I cannot test myself because the result would have no meaning (one person's opinion) especially since I have a pretty bad hearing (left ear is totally deaf). The only solution would be running a pre-test, but then we have the problem that actually all encoders should go through a pre-test to decide the best settings. Such a pre-test requires a lot of work and I have no idea if enough people are willing to spend the time - probably not.

And with cryptic I really mean cryptic since you cannot tell anything by reading "-HF2 -SBT450 -TX0" - it's not like "--vbr-new" or "--preset standard" which is more or less English. Anyways, this isn't supposed to have any meaning since we are not comparing how good the names of the switches sound),

Autumn 2006 Listening Test

Reply #274 –
The problem with HELIX is that no reference/trusted commandline exists. A bit like LAME before the --alt-preset era. You can either rely on default settings (which are maybe well-tuned, or maybe not) or rely on people's own setting. For the latter there's one risk: to find (and therefore use for test) extravagant tunings, inferior to the default one.

You can also think about suggesting a pre-test for HELIX but I already exposed the risk of being caught in a trap. A pre-test for HELIX? Then one not pre-tests for all other competitors like Fraunhofer or iTunes! And indded there's no reason to put a huge effort on finding the best available setting for one competitor and to refuse it at the same time for all other ones. Just think about Fhg possible VBR benefits...

It's up to you, but I wouldn't take the risk to use a personal tuning of HELIX - unless a developer gives you the green light for this. It's not that I don't trust level's commandline but there's no kind of validation for this.