Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat@128kbps listening test - FINISHED (Read 181385 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multiformat@128kbps listening test - FINISHED

Reply #50 –
I'm very sorry I couldn't participate in this test.

However, many thanks to you Roberto - and isn't it nice to have such interesting results? I don't think many people expected this.

To me, these bitrates are very interesting. It's amazing both how good, and how bad, some things can sound at 128kbps.

Also, I bet almost every sample in this test was light years better than most people's typical experience of 128kbps lossy coding!

Cheers,
David.

Multiformat@128kbps listening test - FINISHED

Reply #51 –
Quote
Roberto> what software did you used to obtain wma9 files? Is it VBR-2 pass 128 kbps? What decoder? I've tried to reproduce the same wavform with different settings, and I wasn't able to do it.

I already asked him about this.
http://www.hydrogenaudio.org/forums/index....ndpost&p=210584

EDIT:It's certainly Bitrate VBR 128kbps, 44kHz, stereo VBR 1pass.

BTW, a thread is made at Slashdot about the test results.
Vorbis And Musepack Win 128kbps Multiformat Test

Multiformat@128kbps listening test - FINISHED

Reply #52 –
harashin> I need to try again. I probably did a mistake. Thanks


Other question: did someone post these results on minidisc ddicated boards? It would be interesting, because a lot of MD users often said in these boards that atrac3@132 = mp3@192...

Multiformat@128kbps listening test - FINISHED

Reply #53 –
Thanks rjamorim for this test, and all testers for their time and results!

And imagine, the first Hungarian internet music store uses WMA instead of MP3., because it has "better wuality"  (And DRM, but they could have used AAC for DRM'd music)

Yeah, and thanks for the Hungarian references in the samples!  (Bartók, Dances Hongroises - or whatever, they're "Magyar táncok" in Hungarian  )
Life is Real...
(But not in audio :) )

Multiformat@128kbps listening test - FINISHED

Reply #54 –
Quote
Other question: did someone post these results on minidisc ddicated boards? It would be interesting, because a lot of MD users often said in these boards that atrac3@132 = mp3@192...

Actually, I used to hear things to the effect of "LP2 (132kbps) sounds better than a 320kbps MP3!!!" As much as I would love to send an email to the webmaster of the MD Community page or create a thread on one of their forums, I don't think posting the results will have much of an effect. If my past experiences are any indication of the MD community's openess to such data, it will be ignored or disputed via the usual subjective, empirically-dubious arguments one might expect to be prevelant on such boards. For example, when I ABX'ed ATRAC-R with ease a few years ago and posted the results in the most unbiased manner possible, I was either flamed for being "anti-MD" or just outright ignored. The truth hurts, eh? 

Roberto, kudos on another great test!

Multiformat@128kbps listening test - FINISHED

Reply #55 –
Cygnus X1> I've also read that MD > CD, because MD is 292 kbps and CD 176 

This current test is nevertheless different: there's not only one “biased” tester who posted false results, but a whole community, with 18 samples, on ABX conditions. Some opinions might change

Multiformat@128kbps listening test - FINISHED

Reply #56 –
Cool, these results clearly show that

WMA 1ST DEATH :[

/me wonders what the Extremetech editors will think when - if - they see these results...
Over thinking, over analyzing separates the body from the mind.

Multiformat@128kbps listening test - FINISHED

Reply #58 –
Quote
This just in:

http://slashdot.org/comments.pl?sid=108647&cid=9238730

and

http://slashdot.org/comments.pl?sid=108647&cid=9238686

"actual ranking is Vorbis, iTunes, MPC, Lame, WMA, Atrac3" after "a quick-n-dirty compensation, [using] the average scores times 128 over the average bitrate."

ff123

Not very hard to guess that these "compensators" appear who don't want to understand the concept of vbr and how the test was conducted. Though last time I remember something like this on HA also. 
Juha Laaksonheimo

Multiformat@128kbps listening test - FINISHED

Reply #59 –
Quote
rjamorim, can you plz make a zoomed "music store codecs only" chart too (aac, wma9, atrac3), i think it would be very interesting and important to have such a chart handy for showing people that when they have to choose where they should buy songs from, that not only the prices, but also the quality is very important and varries a lot

Here it is. I probably won't add this graph to the results page. There are already plenty of graphs there.
http://www.rjamorim.com/test/multiformat128/plot18b.png

Quote
Quote
maybe it would make sence to rename "iTunes" to "iTunes AAC" in the summary chart, so that people do not mistake the iTunes result with its lousy mp3-encoder?

yepa and maybe add "mp3" to lame too, (and maybe ogg to vorbis) at least in the final chart to exclude all possible misunderstandings


God, no. If people are that uninformed, they shouldn't be even reading those results.

Quote
A big thank you to Roberto for his efforts in conducting this test. Let's hope that it is not the last too


Penultimate

Quote
That's why I suggested to put the bitrates into the score graphs for each sample ... so everyone can see at which average bitrate the codec's result has been obtained.


That will lead peopel to linking bitrates with scores, just like happened at /. - and that is wrong.

Quote
I think a lot of people thought that the test was going to be very easy (me included), "Come on, it's 128kbit! That sounds like crap, everybody knows that.".


No worries, next test will be at 48kbps. Even people with crappy $5 speakers (like me  ) and tone deaf will be able to participate.

Quote
it's a pitty that wma9 Pro was included in the test  ...last test it was included it performed quite well


It hasn't changed a bit since last test. And I personally believe including WMA Pro in that test was a mistake (my second biggest mistake in test conduction, perhaps). When I included it, I expected microsoft would soon start pushing it with all the might of their marketing department to make it replace WMA Std. Alas, that didn't happen. Microsoft seems to have settled on focusing WMA Pro on DVD players and industry usage, and keeping WMA Std. for consumer usage (portables, online stores, ripping at home...)

Moving on to next post...

Multiformat@128kbps listening test - FINISHED

Reply #60 –
Quote
About the test results, I noticed that for some samples there are no confidence intervals on the graphs (bartok_strings, leahy, mahler, ordinary world). Did everybody score exactly the same on these samples, or maybe you just forgot to put the intervals on the graphs?

Those samples had too few listeners and/or results were too close to each other. When that happens, friedman.exe doesn't output the LSD (which is essential to build the confidence intervals) and says that results are "not significant" (what in practice means they are tied)

Quote
Other question: did someone post these results on minidisc ddicated boards? It would be interesting, because a lot of MD users often said in these boards that atrac3@132 = mp3@192...


MWAHAHA!

Multiformat@128kbps listening test - FINISHED

Reply #61 –
Quote
No worries, next test will be at 48kbps. Even people with crappy $5 speakers (like me  ) and tone deaf will be able to participate.

Time for me to give it a try 
Even the "Waiting" sample was un-ABX-able for me, when I tried at higher bitrates, I might be deaf 
Life is Real...
(But not in audio :) )
 

Multiformat@128kbps listening test - FINISHED

Reply #62 –
Quote
Quote
Other question: did someone post these results on minidisc ddicated boards? It would be interesting, because a lot of MD users often said in these boards that atrac3@132 = mp3@192...


MWAHAHA!

Some body has posted the results at http://forums.minidisc.org/viewtopic.php?p=22300 Nobody has dered to answer yet. Maybe all the minidisc guys have got heart attack after reading the results.

Multiformat@128kbps listening test - FINISHED

Reply #63 –
Quote
Here it is. I probably won't add this graph to the results page. There are already plenty of graphs there.
http://www.rjamorim.com/test/multiformat128/plot18b.png

why not? i would vote for that everyone who thinks about buying songs from a music store, which offers wma@128, should be forced to stare at this graph for two hours 

maybe adding lame to this graph would be also good, to proove that probably even the "kazaa music store" will offer better quality at more reasonable prices than wma-based ones
I know, that I know nothing (Socrates)

Multiformat@128kbps listening test - FINISHED

Reply #64 –
Quote
Some body has posted the results at http://forums.minidisc.org/viewtopic.php?p=22300 Nobody has dered to answer yet. Maybe all the minidisc guys have got heart attack after reading the results.

Let's all go over there and flame them.  Mwahahaha!

Multiformat@128kbps listening test - FINISHED

Reply #66 –
Quote
Not very hard to guess that these "compensators" appear who don't want to understand the concept of vbr and how the test was conducted. Though last time I remember something like this on HA also.
I have actually been waiting for the complaining about unfair bitrates to start. I personally find it hard to understand the complaining, after the reasoning behind it has been explained over and over.

Btw: Slashdot seems like a nice place to waste time. Why bother with the facts when you can assume things instead and base the discussion on these assumptions (in this case why bother reading how the test was performed. Lets just assume how it was done and base the discussion on that). I only visit Slashdot when someone post a link on this forum, and I don't plan spending more time there either.

Reading discussions at Slashdot, makes me want to thank the Hydrogenaudio staff for keeping Hydrogenaudio a source of information, instead of a source of speculation, assumption and incorrect information. A big thank you to you all!

Also thanks to Roberto for yet another interesting test. Too bad they are always conducted when I have to prepare for my exams, but I don't really have a suitable listening environment for performing such tests anyway.

Multiformat@128kbps listening test - FINISHED

Reply #67 –
Quote
maybe adding lame to this graph would be also good, to proove that probably even the "kazaa music store" will offer better quality at more reasonable prices than wma-based ones

You won't find many lame -V5 --athaa-sensitivity 1 at Kazaa, I reckon 

Multiformat@128kbps listening test - FINISHED

Reply #68 –
IMHO the test is meaningless.  If codecs can go over the 128 kbps average then essentially these codecs have cheated. 

Don't get me wrong I've encoded most of my tunes into Vorbis as I feel it's a better codec than others, but I would like to see a fair comparison that fits the title of this listening test!!

Multiformat@128kbps listening test - FINISHED

Reply #69 –
Quote
IMHO the test is meaningless.  If codecs can go over the 128 kbps average then essentially these codecs have cheated.

oh, come on! not again...

Multiformat@128kbps listening test - FINISHED

Reply #70 –
Good Test
Just the average bitrate is a little bit to high imo (itunes is ok though)...

*edit* havn't seen ep0ch's post before...

Multiformat@128kbps listening test - FINISHED

Reply #71 –
Quote
oh, come on! not again...


Heh, it was asked for  Just my opinion!

I don't see how you can possibly disagree that it would have been a worse test if each sample had been encoded to give an average 128kbps per sample... oh I'll shut up to keep you all happy!!

Multiformat@128kbps listening test - FINISHED

Reply #72 –
*yawns* Well the failure of CBR codecs in this test are due to their developers: they did not write a code smart enough to alter bitrate allocation dynamically. VBR is the key issue that requires proper tuning -> the test is fair.

Well ATRAC has a cool name and that's all about it I guess. It looks like this test will be referred many times to whoever claiming ATRAC@132 sounds better than MP3@320 in the upcoming years.  Maybe it's a good idea to open a Codec Comparison Forum to post these tests for future easy access.
The object of mankind lies in its highest individuals.
One must have chaos in oneself to be able to give birth to a dancing star.

Multiformat@128kbps listening test - FINISHED

Reply #73 –
I really wish people wouldn't post links to slashdot.  It's an incomparable juggernaut of stupidity.  I can't resist the urge to go over and yell at them.  It usually takes me about half an hour to realize that I'm just sticking my finger in a dike.

Quote
IMHO the test is meaningless. If codecs can go over the 128 kbps average then essentially these codecs have cheated.

AHHHHHHH!!  You've got slashdotitus!  Do they make a vaccine for this yet?

*runs off to wash hands*
I am *expanding!*  It is so much *squishy* to *smell* you!  *Campers* are the best!  I have *anticipation* and then what?  Better parties in *the middle* for sure.
http://www.phong.org/

Multiformat@128kbps listening test - FINISHED

Reply #74 –