Skip to main content

Topic: Public Multiformat Listening Test @ ~64 kbps [March/April 2011] (Read 49774 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Alex B
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #50
The test is extended until the 10th of April.
We still need more results for some particular samples.

Thanks, I was just about to request for an extension.

I noticed this thread only 5 days ago (didn't expect a new test) and did 5 samples on last Sunday. This week has been busy for me and I may not have any spare time and energy left for testing more samples before the next weekend.

I will post the numbers of those samples in a few minutes.

I was also going ask for a list of the samples that do not have enough results yet.


Since the test has now been extended, you could copy your first post in this thread, add a link that points to this thread, add a note about the new deadline, and post it to the News Submissions forum: http://www.hydrogenaudio.org/forums/index.php?showforum=22

For some reason no admin or mod has directly announced the news, even though we requested that earlier. The "news submission" route is quite unnecessary for a HA listening test. Who ever organized the hosting of the samples (was it Garf?) could have posted the news in the very beginning.

EDIT

If (hopefully when) the news post is validated and published on the front page this thread could be linked as the comments thread and moved to the Validated News forum as I requested earlier: http://www.hydrogenaudio.org/forums/index....st&p=749581
  • Last Edit: 30 March, 2011, 03:29:55 PM by Alex B

  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #51
Thanks, I was just about to request for an extension.

Don't be angry, but from Day 0 it was clear that test will be extended.    Previously such test had to be extended twice http://listening-tests.hydrogenaudio.org/sebastian/mf-64-1/
There are a lot of people who still have not finished to submit their results and they are important.


I was also going ask for a list of the samples that do not have enough results yet.

We need more results for samples 11 and 14-30 (14,15....29,30) and results for other samples are welcome as well.

Since the test has now been extended, you could copy your first post in this thread, add a link that points to this thread, add a note about the new deadline, and post it to the News Submissions forum: http://www.hydrogenaudio.org/forums/index.php?showforum=22

Done
  • Last Edit: 30 March, 2011, 03:44:23 PM by IgorC

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #52
Since the test has now been extended, you could copy your first post in this thread, add a link that points to this thread, add a note about the new deadline, and post it to the News Submissions forum: http://www.hydrogenaudio.org/forums/index.php?showforum=22

Done


This was completely useless for the news page so I just cannibalized it.

Quote
For some reason no admin or mod has directly announced the news, even though we requested that earlier. The "news submission" route is quite unnecessary for a HA listening test.


No admin or mod approved anything because nothing suitable was posted to the news submissions forum.
  • Last Edit: 30 March, 2011, 03:55:43 PM by Garf

  • Alex B
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #53
Thanks.

I think IgorC did a small mistake when he did not post the first post of this thread to the "News Submissions" like Sebastian used to do. (For example: http://www.hydrogenaudio.org/forums/index....howtopic=56397).
  • Last Edit: 30 March, 2011, 04:02:43 PM by Alex B

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #54
Thanks.

I think IgorC did a small mistake when he did not post the first post of this thread to the "News Submissions" like Sebastian used to do. (For example:
http://www.hydrogenaudio.org/forums/index....howtopic=56397).


I would have been rejected anyway (or just sat there) because the format of the post was totally unsuitable.

Edit: I've edited it into submission, but I had to move part of IgorC's comment into his next post. Merged both threads. Should be fine now.
  • Last Edit: 30 March, 2011, 04:15:00 PM by Garf

  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #55
Thank you, Garf.
Sorry for the mess.

  • Alex B
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #56
Of course you could have followed the rules at http://www.hydrogenaudio.org/forums/index....showtopic=20857.

I.e. posted only the bare minimum, essential "news" info in the first post and...
... All other additional information like screenshots, full changelogs, personal opinions/comments may be added in a 2nd post, provided that they are really important.

However, this a collaborative effort. IgorC is not alone. Others who are more experienced can and should help (as Garf did now).

I just did a quick calculation of the work effort that is requested from the testers. Assuming that we would get 20 results for each sample and each codec would need 2-3 minutes of testing time, making 10-15 minutes for each sample, 30 samples would need 30 x 20 x 12.5 minutes = 125 hours of intensive testing work. On average each tester would spend 6 hours and 15 minutes for testing all 30 samples. Actually, I don't know if 2-3 minutes per codec is even a realistic estimate. In my experience at least some samples/codecs that are difficult to test need more time.
  • Last Edit: 30 March, 2011, 05:55:14 PM by Alex B

  • Dakeryas
  • [*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #57
I've done the following "Once ABC/HR is open, click "Open ABC/HR Config..." and load the file "SampleXX.ecf"" after having unzipped all samples according to the .png and having launched the DecodeAll.bat.

I can change the playback range, add comments, but no sound would play when clicking on either "Play sample (q)" or "Play sample (w)" or "Play reference (e)". Any ideas?

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #58
I've done the following "Once ABC/HR is open, click "Open ABC/HR Config..." and load the file "SampleXX.ecf"" after having unzipped all samples according to the .png and having launched the DecodeAll.bat.

I can change the playback range, add comments, but no sound would play when clicking on either "Play sample (q)" or "Play sample (w)" or "Play reference (e)". Any ideas?


Check the settings to see which output audio device is selected.

  • ExUser
  • [*][*][*][*][*]
  • Read-only
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #59
The test is extended until the 10th of April.
We still need more results for some particular samples.
I will post the numbers of those samples in a few minutes.
Awesome. This should give me a window of a day or two to tear through some samples. Thanks for the extension.

  • mixminus1
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #60
I'm in the process of plowing through all 30, and should be done by tomorrow afternoon.

HTH!
"Not sure what the question is, but the answer is probably no."

  • Dakeryas
  • [*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #61
I've done the following "Once ABC/HR is open, click "Open ABC/HR Config..." and load the file "SampleXX.ecf"" after having unzipped all samples according to the .png and having launched the DecodeAll.bat.

I can change the playback range, add comments, but no sound would play when clicking on either "Play sample (q)" or "Play sample (w)" or "Play reference (e)". Any ideas?


Check the settings to see which output audio device is selected.



Solved because the microphone was indeed the default output, many thanks !

I'll test a little as from now on!

  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #62
Thank you to everyone for their results.
All participants have received the answer.
If someone haven't receive it then post here and also can send me  PM or email.

Some participants initially had problems with ABC-HR  Java because previously their ABX tool was other (like foobar ABX, ff123's)
I will ask if someone  already has a few results send them as soon as possible to avoid the loss of their time. This way there is time to analyse them and report to listener if everything is good.

More results are needed for samples 9,11,13,14 and 16-30 (16,17,...,29,30). Especially for 9,11,27,29.
Though results for other samples are welcome as well if the listener has enough time.



  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #63
The test will end on 10th of April  (at 11:00 of GMT -3:00). Definitely.
Finally there will be the results. 




  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #64
Reminder.
The test will end in less than 12 hours.

  • enry2k
  • [*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #65
One of the samples is the Susan Vega's Tom's diner song, the track originally used by the mpeg researchers to tune up the MP3 format:-)

  • jmvalin
  • [*][*][*][*][*]
  • Developer
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #66
One of the samples is the Susan Vega's Tom's diner song, the track originally used by the mpeg researchers to tune up the MP3 format:-)


I think that applies to *all* audio codecs, really.

  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #67
The test is finished.

  • sauvage78
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #68
IgorC:
Whatever the result, I am grateful that you helped jmvalin with CELT when so far I miserably failed to find the motivation to help him. 
  • Last Edit: 10 April, 2011, 12:59:46 PM by sauvage78
Rip & Check: EAC Secure [Low/C2]+CUETools [AR Confidence 2+]
NAS (Backup): Flac -4 (for Speed) | CDImage+CUE with F2K
DAP (Playback): Opus 128Kbps | Tracks with F2KM on Android (LG G5)
Video: VP10 (2160p30@24Mbps/2160p60@48Mbps) Asap !!!

  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #69
Thank you
Sure, no problem 

The main purpose of this test was directed not only for us (users) who are interested to know what codec to use in some particular scenarios but also to help to developers of AAC, CELT and Vorbis to improve their encoders.


All results have been already processed.
Though the approval of other organizator is still required.
  • Last Edit: 10 April, 2011, 01:33:55 PM by IgorC

  • mixminus1
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #70
IgorC:
Whatever the result, I am grateful that you helped jmvalin with CELT

So am I!

All I'll say about CELT right now is:  "Impressive.  Most impressive."
"Not sure what the question is, but the answer is probably no."

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #71
I just confirmed IgorC the results look OK and sent some additional analysis.

Thanks to everyone that submitted results. There were over 500 valid ones! Thanks to this, the results of the test are highly significant, and will allow for some clear conclusions. Listening tests like this are not possible without volunteers that put in some of their free time, so really, this is your accomplishment. Also thanks to IgorC for conducting the test and setting up everything.

Now that I've seen the results I must say, they aren't even remotely close to what I expected! A lot has changed in 4 years...

So IgorC, hurry up on those webpages to satisfy everybodies curiosity
  • Last Edit: 11 April, 2011, 06:25:29 PM by Garf

  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #72
thank you, Garf

and thank to all participants
http://www.hydrogenaudio.org/forums/index....showtopic=88023

  • IgorC
  • [*][*][*][*][*]
Public Multiformat Listening Test @ ~64 kbps [March/April 2011]
Reply #73
I would ask to Admins to close this topic.
The test is end.

Thank you.