Skip to main content

Topic: 64kbps public listening test (Read 50067 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • rjamorim
  • [*][*][*][*][*]
64kbps public listening test
Reply #50
Quote
Is it possible to re-assignate a different place for the different plot ? I mean : from left to right, the winner (HE-AAC) to loser (Real?). This may be useful, to see on which sample winner(s) fail(ed). Random position aren't useful in my opinion.

Yes, but that will take longer, since I'll have to redo all the excel plots.

I'll do it later tonight, since I must now leave to Uni.
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • smok3
  • [*][*][*][*][*]
  • Moderator
64kbps public listening test
Reply #51
@rjamorim:

tnx for organizing and conducting the test.

flame verloren:
can't open http://audio.ciara.us/test/64test/comments...s/comments.html

B)
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung

64kbps public listening test
Reply #52
Thank you rjamorim for making my results 'anon01'. 
ruxvilti'a

  • rjamorim
  • [*][*][*][*][*]
64kbps public listening test
Reply #53
Quote
flame verloren:
can't open http://audio.ciara.us/test/64test/comments...s/comments.html

OK, I changed comments.html to index.html.

So please use this link now:
http://audio.ciara.us/test/64test/comments/

Hopefully it'll work :B
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • rjamorim
  • [*][*][*][*][*]
64kbps public listening test
Reply #54
Quote
Thank you rjamorim for making my results 'anon01'. 

Well, dude, you didn't specify if you wanted me to associate your results with your name...

Now you're "just another number"
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • bond
  • [*][*][*][*][*]
64kbps public listening test
Reply #55
good idea calling it low/high anchor (perhaps you should explain than on top of the page what these are, which encoder, settings and of course bitrate

Quote
from left to right, the winner (HE-AAC) to loser (Real?). This may be useful, to see on which sample winner(s) fail(ed). Random position aren't useful in my opinion.

also nice
but i think it would be good if the order is the same in every plot, so perhaps the codecs should be ordered as they were judged in the overall results (so it is also very easy to see when a codec performed worse and when better)...

and one last optical thing:
if you do the plots again, plz write the codec names horizontally, perhaps use a smaller type size too because now the names are really too big imho (like half the size of the spreadsheet itself)
I know, that I know nothing (Socrates)

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
64kbps public listening test
Reply #56
Glad to know anon1's name. It's one of the rare listener that spoke about reverberation of vorbis encodings.

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
64kbps public listening test
Reply #57
I was sad to be anon05 - I'll have to demand credit for my hard work next time - though I didn't work half as hard as you guys who did them all!

What about processed by rank rather than grade? Is that too much hard work?

Cheers,
David.

  • ff123
  • [*][*][*][*][*]
  • Developer (Donating)
64kbps public listening test
Reply #58
Quote
I was sad to be anon05 - I'll have to demand credit for my hard work next time - though I didn't work half as hard as you guys who did them all!

What about processed by rank rather than grade? Is that too much hard work?

Cheers,
David.

It should be easy to process by rank -- just run the raw data through the statistical tool again, this time using the friedman analysis.

ff123

Edit:
Both versions of abchr need to have a field for nickname (default nick is anonymous) and a checkbox to indicate whether the listener wants his comments associated with his nickname (just to be doubly sure about privacy); this would be written to a line in the results file.
  • Last Edit: 22 September, 2003, 11:35:05 AM by ff123

  • c_haese
  • [*][*]
  • Developer
64kbps public listening test
Reply #59
Quote
OK, I changed comments.html to index.html.

So please use this link now:
http://audio.ciara.us/test/64test/comments/

Hopefully it'll work :B

It's not working for me. The page comes up, but none of the links on that page seem to work. Oh, and the title of that page says "AAC@128kbps test comments."

-Carsten

  • Niknak
  • [*]
64kbps public listening test
Reply #60
Can you give us the actual bitrates or filesizes with each codec and for each of the samples? Are there any significant differences?

  • de Mon
  • [*][*][*][*]
64kbps public listening test
Reply #61
I can't understand why didn't the best version of Vorbis used - GT3??? Why did  realy hissy and not tuned version compared with quiet old and tuned products when there exists GT3?
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech

  • Continuum
  • [*][*][*][*]
64kbps public listening test
Reply #62
GT3 is only real tuned for high bitrates (>160 kbit). It would be no better at 64 kbit than the standard version.

  • sony666
  • [*][*][*][*][*]
64kbps public listening test
Reply #63
the mp3PRO result is interesting.. too bad the format never took off due to the licencing  nightmare  I see the same fate for AAC coming.. unless there will be a free and superior "LAME aac" encoder, mp3 is gonna stay (yes, except the divx ripping ppl)

thanks roberto e.a. for organizing this
  • Last Edit: 22 September, 2003, 02:00:42 PM by sony666

  • JohnV
  • [*][*][*][*][*]
  • Developer
64kbps public listening test
Reply #64
Quote
I can't understand why didn't the best version of Vorbis used - GT3??? Why did  realy hissy and not tuned version compared with quiet old and tuned products when there exists GT3?

The post 1.0 cvs version used includes actually some new tweakings for low bitrates, and as continuum said, it's gt3 which has nothing new for -q0.
  • Last Edit: 22 September, 2003, 02:23:35 PM by JohnV
Juha Laaksonheimo

  • JohnV
  • [*][*][*][*][*]
  • Developer
64kbps public listening test
Reply #65
Quote
the mp3PRO result is interesting.. too bad the format never took off due to the licencing¬† nightmare  I see the same fate for AAC coming.. unless there will be a free and superior "LAME aac" encoder, mp3 is gonna stay (yes, except the divx ripping ppl)

1. mp3Pro is no standard -> AAC/AAC-HE is
2. mp3Pro is completely proprietary ->  there are several different AAC implementations
3. FAAC is getting better and it's free. It might not be state of the art, but it's not that bad it used to be. I think it's very soon and partly already ready to challenge at least Lame MP3.

There's still no doubt that mp3 will be around for a looong time..
  • Last Edit: 22 September, 2003, 02:37:22 PM by JohnV
Juha Laaksonheimo

64kbps public listening test
Reply #66
I can't read the comments from the site.

If I click any link on that page
after trying 20 times to connect and then getting always redirected
Mozilla says the site exceeds redirection limit.
  • Last Edit: 22 September, 2003, 03:16:39 PM by AstralStorm
ruxvilti'a

  • de Mon
  • [*][*][*][*]
64kbps public listening test
Reply #67
Thanks you for your replies, Continuum and JohnV. I had not known about that info.
Ogg Vorbis for music and speech [q-2.0 - q6.0]
FLAC for recordings to be edited
Speex for speech

  • Dologan
  • [*][*][*][*]
  • Members (Donating)
64kbps public listening test
Reply #68
Heh, I find it weird that I usually scored MP3Pro higher than HE-AAC. Those two (apart from Lame) were generally the only ones that needed careful listening to single out, but once I got hold of the artifacts I usually found HE-AAC's to be a little bit more annoying than those of MP3Pro, although there were times where HE-AAC was amazingly difficult to ABX for me, whereas that didn't happen with MP3Pro
Another deviation of mine from the average is that often WMA was usually almost at the level of FhG 64 in terms of annoyance, separated from QT and Real...

My overall: LAME > MP3Pro, HE-AAC > Ogg > Real = QT > WMA > FhG

~Dologan

  • ff123
  • [*][*][*][*][*]
  • Developer (Donating)
64kbps public listening test
Reply #69
Results have been slashdotted:

http://slashdot.org/article.pl?sid=03/09/2...tid=141&tid=188

Get your comments and rebuttals in while the thread's still hot.

ff123

64kbps public listening test
Reply #70
My results:
Code: [Select]
LAME     MP3Pro   HE-AAC   Vorbis   LC-AAC   Real     WMA      FHG
 4.07     3.39     3.13     2.67     2.22     2.05     1.77     1.55

LAME is better than MP3Pro, HE-AAC, Vorbis, LC-AAC, Real, WMA, FHG
MP3Pro is better than Vorbis, LC-AAC, Real, WMA, FHG
HE-AAC is better than LC-AAC, Real, WMA, FHG
Vorbis is better than WMA, FHG
LC-AAC is better than FHG
ruxvilti'a

64kbps public listening test
Reply #71
Great test Roberto. However, I also cannot access the comments page. Would it be too much to ask you to email them to me (the same email I sent you the results with)? Thanks.

  • rjamorim
  • [*][*][*][*][*]
64kbps public listening test
Reply #72
Quote
Results have been slashdotted:

http://slashdot.org/article.pl?sid=03/09/2...tid=141&tid=188

Get your comments and rebuttals in while the thread's still hot.

ff123

Quote
"Hydrogenaudio has just wrapped up a listening test of various audio codecs at 64kbps."


That pisses me off. A lot.

Sure, it was "Hydrogenaudio" that stayed up till 5:30 AM sorting results, it was "Hydrogenaudio" that took shit on every issue that surfaced, it was "Hydrogenaudio" that screened 500+ results ditching the ones ranking the reference...

I'm not an asshole, but I think credit should be given where credit is due. Credits weren't given to ff123's test either, but at least it wasn't misrepresented as being conduced by HA.

sigh...

OK, rant off.


@pseudoacoustic: Check the 4th post in this same thread
http://rarewares.hydrogenaudio.org/rja/comments.zip
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • ff123
  • [*][*][*][*][*]
  • Developer (Donating)
64kbps public listening test
Reply #73
I always find stuff to reply to in the /. threads way too late.  In the current thread there were a couple that got my goat (having to do with the legitimacy of subjective testing and the drawing of the confidence interval bars).

ff123

  • Dologan
  • [*][*][*][*]
  • Members (Donating)
64kbps public listening test
Reply #74
Quote
That pisses me off. A lot.

Sure, it was "Hydrogenaudio" that stayed up till 5:30 AM sorting results, it was "Hydrogenaudio" that took shit on every issue that surfaced, it was "Hydrogenaudio" that screened 500+ results ditching the ones ranking the reference...

I'm not an asshole, but I think credit should be given where credit is due. Credits weren't given to ff123's test either, but at least it wasn't misrepresented as being conduced by HA.

sigh...

OK, rant off.

Don't be sad, Roberto. Who needs Slashdot appreciation, anyway? We all love you here at HA.org, we all know you're da man. 
  • Last Edit: 22 September, 2003, 11:21:06 PM by dologan