192 kbit/s listening test at SoundExpert

Topic: 192 kbit/s listening test at SoundExpert (Read 96420 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

192 kbit/s listening test at SoundExpert

Reply #75 – 2006-07-17 08:16:06

Quote from: jimmy69 on 2006-07-17 07:20:03

In the case, I know that these results are not reliable but wow I wasn't expecting wma to be on top. I was expecting it to be around last or second to last.

In case of present testing activity first more or less reliable results are expected in a month or so. Now moving ratings are more like entertainment

192 kbit/s listening test at SoundExpert

Reply #76 – 2006-07-17 14:16:25

Let's see how long AAC+ stays in the top sector.

192 kbit/s listening test at SoundExpert

Reply #77 – 2006-07-17 14:25:48

Quote from: SebastianG on 2006-07-17 14:16:25

Let's see how long AAC+ stays in the top sector.

According to the current results, AAC+ is ~30% better than Nero LC-AAC and Vorbis aoTuV and -more important- with a high reliability of these results. It implies that AAC+ should stay not on top (as WMA and MPC current positions are not reliable thus fixed) but at least as clearly superior to Nero LC-AAC, Vorbis aoTuV and LAME MP3. If positions will change in the future then the reliability indicator would appear as... not reliable

Am I wrong?

192 kbit/s listening test at SoundExpert

Reply #78 – 2006-07-17 16:03:22

Quote from: guruboolez on 2006-07-17 14:25:48

According to the current results, AAC+ is ~30% better than Nero LC-AAC and Vorbis aoTuV and -more important- with a high reliability of these results. It implies that AAC+ should stay not on top (as WMA and MPC current positions are not reliable thus fixed) but at least as clearly superior to Nero LC-AAC, Vorbis aoTuV and LAME MP3. If positions will change in the future then the reliability indicator would appear as... not reliable

Am I wrong?

You right, you right, guru, - reliability indicator is not reliable indeed. Now it shows error tube for last 10 values (new rating value is calculated each time a grade for this codec is returned). To be true, the figure “10” was spun out of thin air just to test this indicator. Now statistics show that the error tube has to be three times longer at least (and to be dependent in some way on number of testing points). New indicator is almost ready and will be implemented in a week or two (with other new features). But now it is informative only for me

192 kbit/s listening test at SoundExpert

Reply #79 – 2006-08-17 20:55:35

Sorry but I don't get the point to those tests...

Have a look at guruboolez's and have a look at SoundExpert's listening tests. I'm not fluent (I'm french speaking) so maybe I missed something ? All those tests seem to be completly subjective...

Final judge: MY EARS

192 kbit/s listening test at SoundExpert

Reply #80 – 2006-08-18 04:04:49

http://www.soundexpert.info/coders192.jsp;...id=CMOCFENMCBDA

Looks to me like the results are solidifying and lame is really making a very nice showing. The concept of degrees of transparency is an interesting one. I sort of imagine it as leaving a little room for some singal processing / EQ without making otherwise masked artifacts audible.

192 kbit/s listening test at SoundExpert

Reply #81 – 2006-08-18 05:42:30

Quote

I see no reason to perform an ABX comparison test at a bitrate above 130-150kbps. Cause the 128kbps test show that 90-something % of all test samples are already transparent. So I can't imagine how hard it would be to differ lossy and source at a bitrate 50% above this. Good luck!

I am looking at this test and I am thinking the exact same thing myself. Countless tests have been performed I find those results hard to believe

192 kbit/s listening test at SoundExpert

Reply #82 – 2006-08-18 06:47:59

Here:

Quote from: Serge Smirnoff on 2006-07-02 09:11:19

Sorry, this is my fault – there is no popular explanation of SoundExpert (SE) testing methodology on the site still (except two boring ones in pro zone). So I have to explain over and over again: SE testing uses sound artifacts amplification technique (SARTAMP) which helps to evaluate perceived audio quality of devices and compression algorithms with extremely low artifacts and distortions, not audible in “normal” cases. Thus, all SE ratings above 5.00 show amount of perception margin of human auditory system for those tested devices (only codecs now). The absence of reference sample in SE test setup is not essential as the artifacts are clearly audible to more or less extent. The actual aim of SE listening session is to grade the annoyance of those artifacts.

Since 2001 a fixed set of reference (critical) sound samples (from SQAM disc mostly) is used for SE listening tests. All average bit rates are calculated on basis of these samples and called “file based bit rate” (FBR). As the sample set consist of different sound pieces including solo instruments, voices and speech the resulting bit rates (in VBR mode) are slightly lower than in case of pop/rock music (quite close to the bit rates for classical music though).

Edit: smilie and unnecessary comment removed.

192 kbit/s listening test at SoundExpert

Reply #83 – 2006-08-18 23:44:08

Why use -b320 with stereo in lame 3.90.3 instead the --alt--preset insane ?
EDIT:included the version of lame.

192 kbit/s listening test at SoundExpert

Reply #84 – 2006-08-20 01:17:09

Quote from: Diow on 2006-08-18 23:44:08

Why use -b320 with stereo in lame 3.90.3 instead the --alt--preset insane ?
EDIT:included the version of lame.

Because it's just another: "What's the difference between a camel ?"

Another nonsense: Am I gonna hear the difference between transparency and transparency ?

Or: am I God ? (God, please forgive me).

192 kbit/s listening test at SoundExpert

Reply #85 – 2006-08-23 21:22:51

Quote from: Diow on 2006-08-18 23:44:08

Why use -b320 with stereo in lame 3.90.3 instead the --alt--preset insane ?
EDIT:included the version of lame.

This was my suggestion. Exactly speaking I suggested to use 3.90.3 -b320 as well as api.
But this was a bit of too much Lame participants to Serge which I can understand. At that time I prefered -b320 GPSYCHO over api NSPSYTUNE, so the -b320 usage made it into the test.

You are quite right as to prefer the joint stereo using api, and I have changed my mind meanwhile and I prefer 3.90.3 joint stereo nspsytune as well.

Sorry that we don't have the api results because of this history.

192 kbit/s listening test at SoundExpert

Reply #86 – 2006-09-14 14:20:10

While current 192 kbit/s listening test is going smoothly (but slowly, to be true), there were several improvements of ratings’ visualization made:

•   In addition to actual rating a bar now displays min and max values. As each device is tested under nine different sound samples, there are nine different ratings for a device. Now min and max ones are indicated.
•   Calculation of ratings’ reliability was toughened to make it more reliable itself (specially for guruboolez ). This became possible after analysis of ratings' behavior during previous listening tests.
•   Short help section appeared on every ratings page under bars.
•   Tiny ruler was added to each bar to increase diagram usability.

All these improvements are fully visible only on 192 kbit/s page because only those ratings are updated now.

192 kbit/s listening test at SoundExpert

Reply #87 – 2006-11-07 09:47:32

Finally 192 kbit/s listening test is coming to the end. Longer testing period was due to summer inactivity of visitors and toughening criteria of ratings reliability. Now each rating needs more returned grades to become reliable. Nevertheless it’s time to finish the test because new WMA pro codec is waiting for testing. Also HA people are going to start new test and I wouldn’t like the tests to be overlapped. So I encourage everybody to test a few files during this week in order to finish this test. After that we could start discussion on new one which I suppose will include people’s most popular codecs and settings and new competitors of course.

Here is a link to a test file for your convenience. Thank you in advance for participation.

EDIT: test link added

192 kbit/s listening test at SoundExpert

Reply #88 – 2007-01-09 10:22:08

Finally 192 kbit/s listening test has finished with the following results:

Ogg Vorbis, AAC+, AAC LC and WMA 9.1 showed the best audio quality
MP3 and MPC are second
AAC LC from iTunes is last due to the bug

See 192 kbit/s page for details. SoundExpert says “thank you” for participation and “sorry” for such long period of testing which was necessary for experiments with reliability parameter of ratings. Next tests will be substantially shorter. Once again - Thank you!

192 kbit/s listening test at SoundExpert

Reply #89 – 2007-01-15 06:32:55

I would like to somment on the usability:

AAC-LC - I agree, there are many iPods, but how many other players can play AAC? Some mobile phones, maybe.

Ogg Vorbis - Almost every good HW player supports it (Cowon - iAudio, iRiver even iPod with Rockbox firmware). It's a sad true, that there is almost no Ogg Vorbis support in car players and cellphones. There are probably no SW players (or only very few) without Ogg Vorbis support. And don't forget it's the only free fromat from the contenders.

AAC+ - Very poor HW support.

WMA 9.1 - Almost no HW support, very poor SW support.

MPC - Supported in HW only via RockBox. SW support isn't very good neither.

MP3 - Supported everywhere.

192 kbit/s listening test at SoundExpert

Reply #90 – 2007-02-07 23:37:58

Quote from: vlada on 2007-01-15 06:32:55

I would like to somment on the usability:

AAC-LC - I agree, there are many iPods, but how many other players can play AAC? Some mobile phones, maybe.

Ogg Vorbis - Almost every good HW player supports it (Cowon - iAudio, iRiver even iPod with Rockbox firmware). It's a sad true, that there is almost no Ogg Vorbis support in car players and cellphones. There are probably no SW players (or only very few) without Ogg Vorbis support. And don't forget it's the only free fromat from the contenders.

AAC+ - Very poor HW support.

WMA 9.1 - Almost no HW support, very poor SW support.

MPC - Supported in HW only via RockBox. SW support isn't very good neither.

MP3 - Supported everywhere.

Actually the post dipped me into serious thinking. Finally I realized that SoundExpert has no resources and (what is more important) no desire to monitor coders popularity and usability. This is just not our business (at least at the moment). The project is entirely about sound quality and nothing else. So it is just irresponsible to make recommendations according to something we are not sure of.

From now SE recommendations on coders choice will concern sound quality only, revealed during SE listening tests. This policy will be clearly stated in short disclaimer. Corresponding changes have been already made on the site.

192 kbit/s listening test at SoundExpert

Reply #91 – 2007-05-28 11:40:11

Quote from: vlada on 2007-01-15 06:32:55

I would like to somment on the usability:

AAC-LC - I agree, there are many iPods, but how many other players can play AAC? Some mobile phones, maybe.

Ogg Vorbis - Almost every good HW player supports it (Cowon - iAudio, iRiver even iPod with Rockbox firmware). It's a sad true, that there is almost no Ogg Vorbis support in car players and cellphones. There are probably no SW players (or only very few) without Ogg Vorbis support. And don't forget it's the only free fromat from the contenders.

AAC+ - Very poor HW support.

WMA 9.1 - Almost no HW support, very poor SW support.

MPC - Supported in HW only via RockBox. SW support isn't very good neither.

MP3 - Supported everywhere.

Hi Serge! What is about new Lame 3.98 beta 3???

192 kbit/s listening test at SoundExpert

Reply #92 – 2007-05-28 15:13:56

Quote from: Sl@vik on 2007-05-28 11:40:11

Hi Serge! What is about new Lame 3.98 beta 3???

If the version 3.98 beta 3 is close to its final release (like in case of 3.97 beta 2), I'm ready to add it.

Notice